VGATHERQPS

VGATHERQPS loads singles from scattered addresses, each a base plus a signed 64-bit index times a scale of 1, 2, 4 or 8. VGATHERDPS explains what all gathers share: the mask that selects the elements and is cleared as they load, the register rules that raise #UD, the faults it can resume after, and why compiled code rebuilds the mask before each gather.

A destination half as wide#

A 64-bit index fetching a 32-bit element means the destination needs half the width of the index register: four indices in a YMM register fill an XMM destination, and eight in a ZMM register a YMM one, so there is no 512-bit destination. The 128-bit form has two indices and fills only the low half of its XMM destination, zeroing bits 127:64. The cases recorded on Zen 5 show each width:

• vgatherqps xmm4, dword ptr [base + ymm9*4], xmm11 loaded elements 0 and 3, from indices -10 and 6, whose mask elements had their sign bits set; elements 1 and 2 kept their old values, and XMM11 was cleared, as was everything above bit 127 of both registers;
• vgatherqps xmm1, dword ptr [base + xmm14*4], xmm8 had a sign bit only in mask element 1: it loaded that element, from index -5, kept element 0 and zeroed the upper half of XMM1;
• vgatherqps ymm17{k5}, dword ptr [base + zmm30*1] read seven of its eight elements, those K5 selected, from the byte offsets 21 and 32, several times each, and 54, two of them not multiples of 4; element 3, whose mask bit was clear, kept its value, and K5 ended as zero.

In compiled code#

In our builds at -O2, and at -O3 with -march=znver5, neither GCC 14 nor Clang 23 vectorized d[i] = t[idx[i]] over float data with int64_t indices for -march=x86-64-v3, x86-64-v4, skylake or znver5: only the intrinsics give the instruction. For _mm256_i64gather_ps, Clang makes the all-ones mask with VPCMPEQD and zeroes the destination with VXORPS first; GCC makes the mask with VCMPEQPS. For _mm512_i64gather_ps with -march=x86-64-v4, both make an all-ones K1 with KXNORB, eight bits for eight elements; for znver5, GCC loads it from a constant with KMOVB instead.

AMD’s latency spreadsheet for Zen 5 lists VGATHERQPS as microcoded, as it does every gather. VGATHERQPD loads doubles with 64-bit indices and fills its whole destination.

In the optimization guide

What the optimization guide says about VGATHERQPS, a line per claim. Each links to the claim in the guide, with its sources and where they differ.

Vectors

• Gathers and scatters: Where the layout of the data is known, vector loads and shuffles usually beat a gather.
• Gathers and scatters: Issue gathers early, gather many elements at once, use doubleword indices for 32-bit data and zero the destination first.
• Gathers and scatters: Gathers beat scalar loads on Skylake, while AMD lists them as microcode on Zen 3, Zen 4 and Zen 5.

Crestmont and Gracemont

• Vectors and AVX2: Gathers are microcoded on these cores and take about 50 cycles.

Golden Cove and Raptor Cove

• Vectors, AVX-512 and AMX: A gather is a load micro-op for each element plus a few more: LLVM’s models give an eight-element VPGATHERDD 12 micro-ops and a latency of 30 cycles, and the 16-element one of Sapphire Rapids 19 micro-ops.

Haswell and Broadwell

• Gathers: Haswell is the first core with gathers, and each one runs as many micro-ops; Broadwell runs them much faster.

Skylake to Cooper Lake

• Vectors on all parts: Gathers are faster than on Broadwell and beat a sequence of scalar loads unless they fetch only two elements or the data comes from memory rather than the caches.

Zen 2

• Vectors and floating point: Gathers such as VPGATHERDD are microcoded, in AMD’s spreadsheet and LLVM’s model, so vector loads and shuffles beat them when the indices are known in advance.

Zen 3

• Vectors and floating point: Gathers such as VPGATHERDD are microcoded, and a register loaded in pieces is slower than one full-width load, so vector loads and shuffles beat both when the indices are known in advance.

Zen 4

• Vectors and floating point: A gather is slower than vector loads and shuffles when the indices are known in advance, and a register loaded in pieces is slower than one full-width load.

Zen 5

• Vectors and floating point: A gather is slower than vector loads and shuffles when the indices are known in advance.

Forms

Latency and throughput

Data flow

Where the elements of the sources go, element 0 on the right; results that depend on a control value are drawn for an example. How to read.

Operation

What the forms do, in pseudocode that our tests run against results recorded on real processors. How to read.

x = mem.index.number
if x in (xmm1.number, xmm3.number):
    fault(UD)
if xmm1.number == xmm3.number:
    fault(UD)
for i in range(VL // 64):
    if xmm3.sd[i] < 0:
        a = mem.index.sq[i] * mem.scale
        xmm1.d[i] = load(mem.base + a, 32)
for i in range(VL // 64, 4):
    xmm1.d[i] = 0
xmm3.write(0)
# xmm1 is zeroed from bit 128 up

if mem.index.number == ymm1.number:
    fault(UD)
for i in range(VL // 64):
    if k1[i]:
        a = mem.index.sq[i] * mem.scale
        ymm1.d[i] = load(mem.base + a, 32)
for i in range(VL // 64, 4):
    ymm1.d[i] = 0
k1.write(0)  # all of it
# ymm1 is zeroed from bit 128 (xmm) or 256 (ymm) up

Operands

Flags

None: XED records no flag that the forms test or change.

Exceptions

The exception class is XED's name for the set of exception conditions that a form follows. How to read.

VEX forms

avx-type-12

EVEX forms

AVX512-E12

Attributes

What else XED records about the forms. How to read.

All forms

gather

EVEX forms

write masking; memory fault suppression

Intrinsics

The C intrinsics of Clang 23.1.2 that compile to VGATHERQPS, each with the form it compiles to and the Clang options it needs. The forms come from compiling every intrinsic and decoding the result with Intel XED; all intrinsics A–Z explains the method.

CPUID and processors

Each form belongs to an XED ISA set. A form can be used when the processor reports every CPUID bit of one of its ISA set's alternatives.

More AVX2 instructions

• VBROADCASTI128
• VBROADCASTSD
• VBROADCASTSS
• VEXTRACTI128
• VGATHERDPD
• VGATHERDPS
• VGATHERQPD
• VINSERTI128
• VMOVNTDQA
• VMPSADBW
• VPABSB
• VPABSD
• VPABSW
• VPACKSSDW
• VPACKSSWB
• VPACKUSDW
• VPACKUSWB
• VPADDB
• VPADDD
• VPADDQ
• VPADDSB
• VPADDSW
• VPADDUSB
• VPADDUSW
• VPADDW
• VPALIGNR
• VPAND
• VPANDN
• VPAVGB
• VPAVGW
• VPBLENDD
• VPBLENDVB
• VPBLENDW
• VPBROADCASTB
• VPBROADCASTD
• VPBROADCASTQ
• VPBROADCASTW
• VPCMPEQB
• VPCMPEQD
• VPCMPEQQ

All 138 AVX2 instructions

More AVX10.1 instructions

• VFNMSUB213PH
• VFNMSUB213PS
• VFNMSUB213SD
• VFNMSUB213SH
• VFNMSUB213SS
• VFNMSUB231PD
• VFNMSUB231PH
• VFNMSUB231PS
• VFNMSUB231SD
• VFNMSUB231SH
• VFNMSUB231SS
• VFPCLASSPD
• VFPCLASSPH
• VFPCLASSPS
• VFPCLASSSD
• VFPCLASSSH
• VFPCLASSSS
• VGATHERDPD
• VGATHERDPS
• VGATHERQPD
• VGETEXPPD
• VGETEXPPH
• VGETEXPPS
• VGETEXPSD
• VGETEXPSH
• VGETEXPSS
• VGETMANTPD
• VGETMANTPH
• VGETMANTPS
• VGETMANTSD
• VGETMANTSH
• VGETMANTSS
• VGF2P8AFFINEINVQB
• VGF2P8AFFINEQB
• VGF2P8MULB
• VINSERTF32X4
• VINSERTF32X8
• VINSERTF64X2
• VINSERTF64X4
• VINSERTI32X4

All 686 AVX10.1 instructions