VFMADD231PS

VFMADD231PS computes b × c + a for each single of a vector and rounds once, where VMULPS followed by VADDPS rounds twice. The fused result is the exact value rounded once, so it can differ slightly from the two-step one, a difference that Intel’s optimization manual asks numerical code with convergence tests to allow for. Compilers fuse a multiplication and an addition into this instruction only when their floating-point contraction settings allow it.

The three digits say which operands play which role, numbered as in Intel’s syntax; the result always goes to operand 1, and only operand 3 may come from memory:

The 231 form keeps the running sum in its destination, which is why dot products and matrix kernels are built on it.

That same running sum is its main performance trap. Each step of a reduction waits for the previous one through the addend, so the loop runs at the latency of the instruction rather than its throughput. AMD’s Zen 5 guide notes that an addition has the shorter latency when the dependency runs through the addend, and that unrolling the loop to break the dependency performs best; Intel’s guide likewise asks to weigh FMA latency against an addition’s and shows the sum unrolled. The remedy is several independent accumulators, added together after the loop: Clang 23 at -O2 with -ffast-math compiles a dot product to VFMADD231PS on four of them.

The EVEX forms add masking, a broadcast single from memory, and, in the 512-bit register form, a rounding mode given in the instruction ({er}) in place of MXCSR’s, which then records no exceptions. VFMADD231PD and VFMADD231SS are the double and scalar versions; AMD’s earlier four-operand FMA4 form is VFMADDPS.

In the optimization guide

What the optimization guide says about VFMADD231PS, a line per claim. Each links to the claim in the guide, with its sources and where they differ.

Vectors

• Horizontal operations and reductions: GCC 14.2 avoids tight FMA chains in vectorized loops for Zen 1 to Zen 4, Alder Lake and Sapphire Rapids.

Bulldozer to Excavator

• Vectors and floating point: Both FMA pipes add, multiply and fuse multiply-add with a latency of 6 cycles, one less when an FMA feeds the next one. sources differ

Haswell and Broadwell

• AVX2 and FMA: Two FMA units, on ports 0 and 1, take a 256-bit FMA each a cycle with a latency of 5 cycles, so one core finishes 16 single-precision or 8 double-precision multiply-adds a cycle, 32 or 16 floating-point operations: twice Sandy Bridge’s peak.

Zen and Zen+

• Vectors and floating point: A float addition takes three cycles, a multiplication three in single precision and four in double, and an FMA five, each on two pipes; AMD’s spreadsheet, LLVM’s model and Agner Fog agree.

Zen 2

• Vectors and floating point: A float addition or multiplication takes three cycles and an FMA five, each on two pipes, and VPADDD one cycle on any of three; AMD’s spreadsheet, LLVM’s model and Agner Fog agree.

Zen 4

• Vectors and floating point: LLVM’s model gives VPADDD a latency of one cycle on any of the four pipes, and VFMADD231PS four cycles on pipe 0 or 1; the 512-bit forms keep their pipe for two cycles, so they finish half as many per cycle.

Forms

Latency and throughput

Data flow

Where the elements of the sources go, element 0 on the right; results that depend on a control value are drawn for an example. How to read.

Operation

What the forms do, in pseudocode that our tests run against results recorded on real processors. How to read.

for i in range(VL // 32):
    if k1[i]:  # 1 without a write mask
        a = zmm1.f32[i]
        b = zmm2.f32[i]
        c = src3.f32[i]
        zmm1.f32[i] = fmadd(b, c, a)
    elif zeroing:
        zmm1.f32[i] = 0
    # else: zmm1.f32[i] keeps its value
# m32bcst: src3.f32[i] is the m32 for every i
# {er}: the form's rounding, not MXCSR.RC, and no flags
# zmm1 is zeroed from bit VL up

Operands

Flags

None: XED records no flag that the forms test or change.

Exceptions

The exception class is XED's name for the set of exception conditions that a form follows. How to read.

VEX forms

avx-type-2

EVEX forms

AVX512-E2

Attributes

What else XED records about the forms. How to read.

All forms

follows MXCSR

EVEX forms

write masking

EVEX memory forms

embedded broadcast (tuple Full); memory fault suppression

Intrinsics

The C intrinsics of Clang 23.1.2 that compile to VFMADD231PS, each with the form it compiles to and the Clang options it needs. The forms come from compiling every intrinsic and decoding the result with Intel XED; all intrinsics A–Z explains the method.

CPUID and processors

Each form belongs to an XED ISA set. A form can be used when the processor reports every CPUID bit of one of its ISA set's alternatives.

More FMA instructions

• VFMADD132PD
• VFMADD132PS
• VFMADD132SD
• VFMADD132SS
• VFMADD213PD
• VFMADD213PS
• VFMADD213SD
• VFMADD213SS
• VFMADD231PD
• VFMADD231SD
• VFMADD231SS
• VFMADDSUB132PD
• VFMADDSUB132PS
• VFMADDSUB213PD
• VFMADDSUB213PS
• VFMADDSUB231PD
• VFMADDSUB231PS
• VFMSUB132PD
• VFMSUB132PS
• VFMSUB132SD
• VFMSUB132SS
• VFMSUB213PD
• VFMSUB213PS
• VFMSUB213SD
• VFMSUB213SS
• VFMSUB231PD
• VFMSUB231PS
• VFMSUB231SD
• VFMSUB231SS
• VFMSUBADD132PD
• VFMSUBADD132PS
• VFMSUBADD213PD
• VFMSUBADD213PS
• VFMSUBADD231PD
• VFMSUBADD231PS
• VFNMADD132PD
• VFNMADD132PS
• VFNMADD132SD
• VFNMADD132SS
• VFNMADD213PD

All 60 FMA instructions

More AVX10.1 instructions

• VFCMULCPH
• VFCMULCSH
• VFIXUPIMMPD
• VFIXUPIMMPS
• VFIXUPIMMSD
• VFIXUPIMMSS
• VFMADD132PD
• VFMADD132PH
• VFMADD132PS
• VFMADD132SD
• VFMADD132SH
• VFMADD132SS
• VFMADD213PD
• VFMADD213PH
• VFMADD213PS
• VFMADD213SD
• VFMADD213SH
• VFMADD213SS
• VFMADD231PD
• VFMADD231PH
• VFMADD231SD
• VFMADD231SH
• VFMADD231SS
• VFMADDCPH
• VFMADDCSH
• VFMADDSUB132PD
• VFMADDSUB132PH
• VFMADDSUB132PS
• VFMADDSUB213PD
• VFMADDSUB213PH
• VFMADDSUB213PS
• VFMADDSUB231PD
• VFMADDSUB231PH
• VFMADDSUB231PS
• VFMSUB132PD
• VFMSUB132PH
• VFMSUB132PS
• VFMSUB132SD
• VFMSUB132SH
• VFMSUB132SS

All 686 AVX10.1 instructions