TDPBF16PS

TDPBF16PS is the bfloat16 matrix multiply of AMX: C += A × B, where each dword of A and B holds a pair of bfloat16 values and C holds singles. It has the shape rules and faults of TDPBSSD, with pairs in place of groups of four bytes: A is M rows of K pairs, B is K rows of N pairs, so B holds two consecutive rows of its matrix interleaved in each dword, and C is M rows of N singles. With full tiles, one instruction makes 16 × 16 sums of 32 products each. Its vector counterpart is VDPBF16PS, which shares its number formats but not its order of additions.

The order of the sums#

For each element of C, the instruction keeps two partial sums, starting from zero: one of the products of the low halves of the pairs, one of the high halves. Each product goes into its sum with a single rounding, as a fused multiply-add does, k from 0 upwards. At the end the two sums are added together, and that total is added to the old value of C, each with its own rounding. The accumulator thus comes in last, where VDPBF16PS starts from it, and a loop in C that adds the same products in another order may round differently.

Fixed arithmetic#

The rounding is always to nearest even. Bfloat16 inputs and single values that are denormal count as zero, and denormal results become zero. MXCSR is neither read nor written and no exception is raised or flagged, so overflows to infinity and invalid operations pass silently into C.

Feeding it#

Intel’s optimization manual turns the single-precision results of one network layer into the bfloat16 inputs of the next with VCVTNE2PS2BF16, which rounds two vectors of singles and packs them into one register of pairs (Section 20.11.4, Example 20-23). With -mamx-bf16, GCC 14 and Clang 23 at -O2 compile _tile_dpbf16ps(0, 1, 2) to tdpbf16ps tmm0, tmm1, tmm2, and Clang 23’s tile type __tile1024i has __tile_dpbf16ps. AMX-FP16’s TDPFP16PS is the same instruction for IEEE half precision.

In the optimization guide

What the optimization guide says about TDPBF16PS, a line per claim. Each links to the claim in the guide, with its sources and where they differ.

Golden Cove and Raptor Cove

• Vectors, AVX-512 and AMX: On Sapphire and Emerald Rapids, a tile multiplication (tdp*) can start every 16 cycles and takes 52; a tile load every 8, taking 45; and loading the tile configuration takes about 204 cycles, worth doing once rather than in every call.

Forms

Latency and throughput

Operation

What the forms do, in pseudocode that our tests run against results recorded on real processors. How to read.

on = xcr0 if cr4 >> 18 & 1 else 0    # XCR0, where CR4.OSXSAVE is set
if on >> 17 & 3 != 3:                # TILECFG and TILEDATA
    fault(UD)
if palette != 1:                     # 0: no tiles configured; ACE's
    fault(UD)                        # palette 2 multiplies no tiles
t = tmm1.number                      # three different tiles
if t == tmm2.number or t == tmm3.number:
    fault(UD)
if tmm2.number == tmm3.number:
    fault(UD)
if 0 in (tmm1.rows, tmm2.rows, tmm3.rows):
    fault(UD)                        # one TILECFG leaves out
if tmm1.colsb % 4 or tmm2.colsb % 4:
    fault(UD)                        # rows of whole dwords
if tmm2.rows != tmm1.rows:           # A: M rows, as C
    fault(UD)
if tmm2.colsb // 4 != tmm3.rows:     # A: K dwords, B: K rows
    fault(UD)
if tmm3.colsb != tmm1.colsb:         # B: N dwords, as C
    fault(UD)
# K and N within the TMUL unit's 16 and 64 bytes (CPUID leaf
# 1EH): true of every tile of palette 1
if xfd >> 18 & 1:                    # the OS keeps the tiles from
    fault(NM)                        # this thread
for m in range(tmm1.rows):
    x = tmm2.row[m]                  # row m of A
    for n in range(tmm1.colsb // 4):
        e = o = cvt_f32_ne(0)        # two sums, from +0
        for k in range(tmm2.colsb // 4):
            y = tmm3.row[k]          # row k of B
            a = x.bf16[2 * k]
            b = y.bf16[2 * n]
            e = fmadd_ne(a, b, e)
            a = x.bf16[2 * k + 1]
            b = y.bf16[2 * n + 1]
            o = fmadd_ne(a, b, o)
        c = tmm1.row[m].f32[n]
        r = add_ne(e, o)
        tmm1.row[m].f32[n] = add_ne(c, r)
start_row = 0
# tmm1 is zeroed past its rows and colsb

Operands

Flags

None: XED records no flag that the forms test or change.

Exceptions

The exception class is XED's name for the set of exception conditions that a form follows. How to read.

All forms

AMX-E4

Attributes

What else XED records about the forms. How to read.

All forms

may abort a TSX transaction

Intrinsics

The C intrinsics of Clang 23.1.2 that compile to TDPBF16PS, each with the form it compiles to and the Clang options it needs. The forms come from compiling every intrinsic and decoding the result with Intel XED; all intrinsics A–Z explains the method.

CPUID and processors

Each form belongs to an XED ISA set. A form can be used when the processor reports every CPUID bit of one of its ISA set's alternatives.