VZEROUPPER

A legacy SSE instruction writes the low 128 bits of its destination register and must keep the bits above unchanged. While those upper bits hold anything but zeros, every SSE instruction carries that duty, and Intel’s and AMD’s optimization manuals describe what it costs:

• On Intel’s Haswell, Broadwell and Ice Lake client, each switch between AVX code that left upper bits set and SSE code costs about as much as saving the registers with XSAVE.
• On Skylake, each SSE instruction instead depends on the old value of its destination and merges its upper bits into the result.
• On Golden Cove cores, as in Sapphire Rapids, microcode handles such mixing, and Intel’s example of it runs hundreds of times slower than on Ice Lake server.
• On AMD’s Zen 5, SSE instructions become 256- or 512-bit operations that merge the upper data, with the same added dependency.

VZEROUPPER clears bits 128 and up of vector registers 0 to 15 (0 to 7 outside 64-bit mode), after which SSE code runs without any of this. It leaves their low 128 bits, the mask registers and ZMM16 to ZMM31 alone, and Intel’s manual gives it zero latency.

GCC and Clang emit it before a function that has used the upper halves returns or calls another function (see vector extensions in the ABIs); Intel also advises it before creating a thread. It is not needed between 128-bit and 256-bit AVX code, because VEX-encoded 128-bit instructions clear the upper bits of their destination themselves, and Intel counts instructions that write only ZMM16 to ZMM31 among those that keep the upper state clean.

The trap is that it clears all sixteen registers at once, whatever they hold: hand-written code must not execute it while a YMM or ZMM register still holds a value needed later, such as a __m256 argument or result. VZEROALL clears the whole registers. After either, the AVX state is in its initial configuration, so XSAVEOPT and its kin may skip saving it.

In the optimization guide

What the optimization guide says about VZEROUPPER, a line per claim. Each links to the claim in the guide, with its sources and where they differ.

Vectors

• Mixing SSE and AVX, and VZEROUPPER: Legacy SSE code that runs while the upper bits of the vector registers hold data pays on every core, from a merge to a fault.
• Mixing SSE and AVX, and VZEROUPPER: VZEROUPPER costs no latency; Intel places it before calls into possible SSE code, at the end of functions and before starting a thread.
• Mixing SSE and AVX, and VZEROUPPER: Only vector registers 0 to 15 make the upper state dirty, so AVX-512 code kept in registers 16 to 31 needs no VZEROUPPER.
• Mixing SSE and AVX, and VZEROUPPER: GCC 14.2 and LLVM add VZEROUPPER before calls and returns of functions that used the upper bits, for nearly every target they tune for.

Bulldozer to Excavator

• Vectors and floating point: AMD advises against mixing legacy SSE and AVX code: an SSE instruction keeps bits 255:128 of its destination and so waits for their last write, which a VZEROUPPER before the SSE code avoids. sources differ

Golden Cove and Raptor Cove

• Vectors, AVX-512 and AMX: A legacy SSE instruction run while the upper halves of the vector registers hold data needs a microcode assist that merges them: Intel measured a three-instruction sequence at 388 cycles where Ice Lake took 1.

Haswell and Broadwell

• AVX2 and FMA: Mixing SSE with 256-bit AVX code works as on Sandy Bridge: the switches save and restore the upper halves of all the YMM registers, each costing on the order of an XSAVE in Intel’s table and about 70 cycles in Agner Fog’s tests, and VZEROUPPER at the end of AVX code avoids them.

Ice Lake to Rocket Lake

• Vectors and AVX-512: Agner Fog found the same cost of mixing SSE with AVX as on Skylake: no switch of state, but a legacy SSE instruction that writes an XMM register while upper halves hold data depends on the register’s old value. sources differ

Jaguar and Puma

• Vectors and floating point: With a YMM register’s halves renamed apart, Agner Fog found no penalty for mixing SSE and AVX code, and LLVM’s tuning for Jaguar, unlike that for the Bulldozer family, inserts no VZEROUPPER.

Sandy Bridge and Ivy Bridge

• Front end: The renamer removes NOP, VZEROUPPER, FXCHG and the zeroing idioms (XOR, SUB and PSUBx of a register with itself, PXOR and XORPS/XORPD likewise) from the pipeline at no cost, since their result needs no input; a so-called ones idiom such as cmpeqps xmm1, xmm1 still has to run, but waits for no source since its result does not depend on the data either.
• AVX, SSE and denormals: Once any YMM register’s upper half holds AVX data, the first legacy SSE instruction after it splits the whole YMM register file into a scratchpad, and the first AVX instruction after that joins it back together, not the per-register scheme Skylake later uses; Agner Fog measured about 70 cycles for each of those two switches.

Skylake to Cooper Lake

• Vectors on all parts: Mixing legacy SSE with AVX no longer costs a switch of the whole register state.

Zen and Zen+

• Vectors and floating point: Mixing legacy SSE and AVX instructions costs nothing, since each YMM register is two 128-bit registers; Zen 2 brings the penalty back.

Zen 2

• Vectors and floating point: Running a legacy SSE instruction while any YMM register holds a nonzero upper half, or AVX after it, traps to save or restore those bits of all 16 registers at a cost of about 100 cycles; vzeroupper after AVX code avoids it. sources differ

Zen 3

• Vectors and floating point: Running a legacy SSE instruction while any YMM register holds a nonzero upper half, or AVX after it, traps to save or restore those bits of all 16 registers at a cost of about 100 cycles; vzeroupper after AVX code avoids it. sources differ

Zen 4

• Vectors and floating point: Legacy SSE instructions are slower while bits 511:128 of the vector registers are not zero, since each then has to merge them; vzeroupper after AVX code avoids it.

Zen 5

• Vectors and floating point: Legacy SSE instructions are slower while the upper bits of the vector registers are not zero, since each then has to merge them; vzeroupper after AVX code avoids it.

Forms

Operation

What the forms do, in pseudocode that our tests run against results recorded on real processors. How to read.

for i in range(16):
    zmm[i] = xmm[i]                  # bits 127:0 kept, the rest zeros

Operands

None: XED lists no operands for this form.

Flags

None: XED records no flag that the forms test or change.

Exceptions

The exception class is XED's name for the set of exception conditions that a form follows. How to read.

All forms

avx-type-8

Attributes

What else XED records about the forms. How to read.

All forms

may abort a TSX transaction

Intrinsics

The C intrinsics of Clang 23.1.2 that compile to VZEROUPPER, each with the form it compiles to and the Clang options it needs. The forms come from compiling every intrinsic and decoding the result with Intel XED; all intrinsics A–Z explains the method.

CPUID and processors

Each form belongs to an XED ISA set. A form can be used when the processor reports every CPUID bit of one of its ISA set's alternatives.

More AVX instructions

• VPTEST
• VPUNPCKHBW
• VPUNPCKHDQ
• VPUNPCKHQDQ
• VPUNPCKHWD
• VPUNPCKLBW
• VPUNPCKLDQ
• VPUNPCKLQDQ
• VPUNPCKLWD
• VPXOR
• VRCPPS
• VRCPSS
• VROUNDPD
• VROUNDPS
• VROUNDSD
• VROUNDSS
• VRSQRTPS
• VRSQRTSS
• VSHUFPD
• VSHUFPS
• VSQRTPD
• VSQRTPS
• VSQRTSD
• VSQRTSS
• VSTMXCSR
• VSUBPD
• VSUBPS
• VSUBSD
• VSUBSS
• VTESTPD
• VTESTPS
• VUCOMISD
• VUCOMISS
• VUNPCKHPD
• VUNPCKHPS
• VUNPCKLPD
• VUNPCKLPS
• VXORPD
• VXORPS
• VZEROALL

All 253 AVX instructions