Optimizing code for AMD Zen 5

Two decode pipes each deliver as many as four instructions a cycle from a 32-byte window. One thread alone decodes at most four instructions per cycle; with two threads, each has a pipe of its own. [AMD 58455, Section 2.9, page 33]

Of the four decode slots, only the first can take an instruction longer than 10 bytes, so code that runs from the decoders should not have more than one such instruction in any four; long NOPs used as padding are the usual case. [AMD 58455, Sections 2.8.3.1 and 2.9, pages 32 and 33]

Branch targets aligned to 16 bytes give the decoders their full width. [AMD 58455, Section 2.9, page 33]

An op cache entry holds up to six instructions, from at most two adjacent 64-byte lines, with six slots for 32-bit immediates and displacements between them. A 64-bit immediate takes two slots, so instructions with large constants fill entries sooner. [AMD 58455, Section 2.9.1, page 34]

The front end switches from the decoders to the op cache only at a taken branch, and back when the op cache misses. Hot code that fits in the op cache avoids the switches: with two threads on the core, each should keep to half of it, which limits how far loops are worth unrolling. [AMD 58455, Section 2.9.1, page 34]

For padding, fewer and longer NOPs are better than many short ones. Zen 5 decodes a NOP of up to 15 bytes, made by adding 66h prefixes to the 8-byte form, without a penalty; earlier AMD processors such as Family 15h slow down with more than three 66h prefixes, so code for all of them stops at 11 bytes. [AMD 58455, Section 2.8.3.1, Tables 6 and 7, pages 31 and 32]

With its 16K entries, the L1 BTB predicts direct jumps, direct calls and indirect branches with a single target with no bubble; returns and indirect branches with several targets cost two cycles. A prediction from the 8K-entry L2 BTB costs eight. [AMD 58455, Section 2.8.1.2, page 27]

A BTB entry holds two branches, so code entering a 64-byte line best meets at most two branches in it before leaving: a third takes another entry and more cycles to predict, unless an earlier branch is unconditional or nearly always taken. [AMD 58455, Section 2.8.3, page 30]

Loop alignment rarely matters. For a hot loop, AMD suggests starting it at a 64-byte boundary or, slightly better, ending it on the last byte of a 64-byte line. [AMD 58455, Section 2.8.3, page 30]

A conditional branch that has never been taken is not in the BTB and is predicted not taken, and branches that never go the other way stay out of the global history. Branches that are mostly not taken are the cheapest. [AMD 58455, Section 2.8.1.5, page 29]

The return address stack has 52 entries per thread. A call to the next instruction is treated specially and does not unbalance it, but in 64-bit code lea rax, [rip] is the simpler way to read RIP. The longer return that AMD once advised for Family 10h is not needed: a one-byte ret costs nothing more. [AMD 58455, Section 2.8.1.3, page 28]

Indirect branches with one target are predicted from the BTB; those with several use a 3,072-entry table indexed with global history, so fewer targets per branch predict better. [AMD 58455, Section 2.8.1.4, pages 28 and 29]

A cmp, test, add, sub, inc, dec, and, or or xor followed by a conditional branch becomes one macro-op, also with a memory operand, within limits on the sizes of its displacement and immediate. [AMD 58455, Section 2.9.3, page 36]

New in Zen 5, a register-to-register mov fuses with a following ALU instruction that writes the same register and reads only registers or an immediate, as in mov eax, edi then add eax, esi. [AMD 58455, Section 2.9.4, pages 36 and 37]

Register-to-register moves of 32-bit and 64-bit general registers and of whole vector registers take no execution cycle, nor do the zeroing idioms such as xor eax, eax and vpxor, which also break the dependency on the old value. [AMD 58455, Sections 2.9.2 and 2.9.5, pages 35 and 37]

PUSH, POP, CALL and RET update RSP without a dependency, and loads, stores, lea and add or sub of a small immediate through RSP can use the tracked value. Other instructions that read or write RSP cost an extra macro-op and reset the tracking. [AMD 58455, Section 2.9.6, page 38]

PDEP and PEXT run natively, three per cycle with a latency of three cycles; code with a separate path for processors where they are slow should take the fast path. [AMD 58455, Section 2.10.2, page 41]

Three ALUs multiply, 64 by 64 bits in three cycles; a multiplication that writes two registers delivers the second one a cycle later. [AMD 58455, Section 2.10.2, page 41]

Division takes about 8 cycles plus 1 for every 9 bits of the quotient, and a typical 64-bit division can start every 8 cycles. [AMD 58455, Section 2.10.2, page 41]

An lea with a base, an index and a displacement is two macro-ops, where one with two of them is one. [AMD 58455, Section 2.10.2, page 41]

Some versions of Zen 5 make TZCNT, BLSI, BLSR and BLSMSK wait for the old value of their destination. Clearing the destination with xor first removes the wait. [AMD 58455, Section 2.9.9, page 39]

The vector units are 512 bits wide, so most 512-bit operations are one macro-op like their 256-bit forms, and AMD recommends 512-bit vectors where the data allows. Zen 4 issues them over two cycles on 256-bit paths. [AMD 58455, Sections 2.11 and 2.11.2, pages 43 and 46; AMD 58455, Section 2.3, Table 5, page 17; AMD 57647, Section 2.11, page 34]

Of the four vector pipes, 0 and 1 multiply and run FMA, 2 and 3 add and convert, 1 divides and takes square roots, and all four run moves and logic. Shuffles and shifts run on 1 and 2. [AMD 58455, Section 2.11.1, page 45]

An FMA, or an AVX-512 operation with merge masking, needs a third source and competes for the source bus of pipe 4 or 5, which may stall it. [AMD 58455, Section 2.11, page 44]

AMD’s latency spreadsheet gives VPADDD a latency of one cycle and a throughput of four per cycle, on any of the four vector pipes. [AMD 58455, latency spreadsheet, row 179 (VPADDD)]

Legacy SSE instructions are slower while the upper bits of the vector registers are not zero, since each then has to merge them; vzeroupper after AVX code avoids it. [AMD 58455, Section 2.11.6, page 49; Agner Fog, Section 25.11, page 257]

Multiplication, division and square root may run again with a longer latency when an input or result is denormal; setting FTZ and DAZ in MXCSR avoids it, at the cost of IEEE 754 results. x87 code has no such switch. [AMD 58455, Sections 2.11.2 and 2.11.4, pages 46 and 47]

An FMA shortens a chain that runs through a multiplication into an addition, but lengthens one that runs through the addend: there a separate addition, or unrolling with several accumulators, is faster. [AMD 58455, Section 2.11.7, page 49]

A gather is slower than vector loads and shuffles when the indices are known in advance. [AMD 58455, Section 2.11.2, page 46]

The load-store unit runs four memory operations each cycle. Loads may fill all four slots, vector loads two of them; stores may fill two, and a 512-bit store one. [AMD 58455, Section 2.12, page 50]

A load with a base and a displacement, or a base and an index without scale, reaches an integer register in 4 cycles. A scaled index, a base with an index and a displacement, or a segment base that is not zero adds one cycle. [AMD 58455, Section 2.12, pages 51 and 52]

An access that crosses a 64-byte line takes two cycles of the cache instead of one, and one that crosses a 4 KB page can cost much more. Aligned data is as fast with the unaligned instructions, such as movups, as with the aligned ones. [AMD 58455, Section 2.6.2, page 20]

A load gets its value from an earlier store still in the store queue when that one store holds every byte of the load, wherever the load starts within it. [AMD 58455, Section 2.12, page 50]

Store forwarding matches on bits 11:0 of the address, so several stores in flight to addresses that differ by multiples of 4 KB slow down a load that needs one of them. [AMD 58455, Section 2.12, page 50]

The load queue tracks 64 loads that have not completed, the store queue holds 104 stores, and 124 cache misses can be outstanding at once. [AMD 58455, Section 2.12, pages 50 and 51]

PREFETCHT0 fills every data cache level, PREFETCHT1 all but the L1, and PREFETCHT2 all but the L1 and L2. PREFETCHNTA marks the line to leave the L2 early and never enter the L3. [AMD 58455, Section 2.13.1, Table 8, page 54; AMD 58455, Section 2.6.4, page 23]

rep movs and rep stos of at least the size of the L3 store with streaming stores that bypass the caches, and on some systems smaller ones skip reading the lines they overwrite whole. Either way, other cores may see the stores of one such instruction in a different order than they were made. [AMD 58455, Section 2.13.6, page 58]

Locked instructions are fastest when their memory operand stays within one 64-byte line, where Zen 4 wants it within 16 bytes. [AMD 58455, Section 2.15, page 60; AMD 57647, Section 2.15, page 46]

With two threads, the store queue, the retire queue and the op queue are split in half; the schedulers, register files and load queue are shared up to a limit per thread; the caches, TLBs and the op cache are shared freely. [AMD 58455, Section 2.14, Table 10, page 60]

Two threads that run the same code share BTB and op cache entries only if that code sits at identical linear and physical addresses; threads that run different code collide less in the BTB when it is in different linear pages. [AMD 58455, Section 2.14, page 60]

PAUSE stops the thread’s dispatch for about 64 cycles, which saves power and leaves resources to the other thread while a spin loop waits. [AMD 58455, Section 2.9.8, page 39]