Optimizing code for Intel Redwood Cove

The micro-op queue in front of allocation holds 192 micro-ops for one thread or 96 for each of two, up from 144 and 72, so the loop stream detector can replay larger loops without the decoders or the micro-op cache. The front-end chapter compares the sizes of other cores. [Intel 248966, Section 2.1.1, page 2-1]

Granite Rapids, not Meteor Lake, adds PREFETCHIT0 and PREFETCHIT1, which fetch code ahead of a jump to it: the first into every cache level, the second into all but the L1. They take only a RIP-relative address, best one where an instruction starts, and are no-ops in any other form and on processors without them (CPUID leaf 7, subleaf 1, EDX bit 14). [Intel 248966, Section 2.1.1, page 2-1; Intel 325383, Chapter 4, PREFETCHh, pages 4-414 and 4-415; LLVM 23.1.2, X86.td, line 1270]

A register-to-register mov followed by an operation that overwrites the moved-to register, as mov rax, rbx then sub rax, rcx, becomes one micro-op computing rbx - rcx into RAX: the three-operand form the instruction set lacks, with one slot from the queue to retirement. It works only for code delivered by the micro-op cache, never for the decoders. Zen 5 fuses such pairs too. [Intel 248966, Section 2.1.1.3, Table 2-1, page 2-3]

A load into a register followed by an operation on that register and another one fuses likewise into a single load-and-operate micro-op, even where no instruction encodes the pair, as with mov eax, [mem] then sub eax, ebx. Again only code from the micro-op cache fuses. [Intel 248966, Section 2.1.1.3, Table 2-2, page 2-3]

The fused load must be a plain mov of 32 or 64 bits (opcode 8Bh, no sign extension) or a legacy SSE movaps, movapd, movups, movupd, movdqa or movdqu; its address may have no index register and may not be RIP-relative. The operation must be an integer one of that width or a 128-bit packed one, and its other source must be a register other than the destination, not an immediate. Nothing with a VEX or EVEX prefix fuses, so the pair fuses in SSE code and not in the same code compiled for AVX. [Intel 248966, Section 2.1.1.3, page 2-4]

A conditional jump with a 3Eh prefix is a taken hint: when the predictor holds nothing on the branch, the decoder turns its default not-taken guess into taken and fetches from the target. A taken branch then costs a redirection at decode instead of a misprediction found at execution; a branch that is not taken pays both. Golden Cove ignores the prefix. [Intel 248966, Section 2.1.1.1, page 2-2]

The hint applies to Jcc only, not to jrcxz or loop. A 2Eh not-taken prefix is decoded and ignored, and an instruction that has a 3Eh prefix is hinted taken whatever other prefixes it carries, 2Eh included. [Intel 248966, Section 2.1.1.1, page 2-2]

Hints pay on branches that run rarely and are mostly taken, where the predictor has likely forgotten them, as in programs with more branches than the predictor holds; on a hot branch it only makes the code longer. Intel leaves their placement to profile-guided compilers, and LLVM emits them only on request and for Granite Rapids. [Intel 248966, Section 2.1.1.1, page 2-2; LLVM 23.1.2, X86.td, line 1273]

New events find candidates: FRONTEND_RETIRED.ANY_ANT counts retired conditional branches that were predicted not taken for want of history, FRONTEND_RETIRED.MISP_ANT those of them that were mispredicted, against BR_INST_RETIRED.COND for all of them; branch records sampled with the first two tell which way each branch usually goes. [Intel 248966, Section 2.1.1.2, page 2-3]

Floating-point multiplication takes three cycles, one fewer than on Golden Cove, where an addition takes three but can feed another in two. Intel says nothing of FMA. [Intel 248966, Section 2.1.1, page 2-2]

Meteor Lake, in LLVM’s -march=meteorlake, has the instruction set of Alder Lake: AVX2 and AVX-VNNI, without AVX-512 or AMX. Granite Rapids has Sapphire Rapids’, AVX-512 and AMX included, plus AMX-FP16 and the code prefetches; its D parts add AMX-COMPLEX. [LLVM 23.1.2, X86.td, line 2123; LLVM 23.1.2, X86.td, line 1269; LLVM 23.1.2, X86.td, line 1277]

On Granite Rapids, AMX also multiplies tiles of half-precision values, with TDPFP16PS; Intel’s manual gives its rates only for the older tile operations. [Intel 248966, Section 2.1.1, page 2-2]

Intel says locked instructions got faster, and two threads on one core run better together, without numbers for either. [Intel 248966, Section 2.1.1, page 2-2]

A core can draw more memory bandwidth than Golden Cove, from the deeper L2 miss queue and new prefetchers: one for arrays of pointers, and one that fills the L3 with whole pages ahead of a stream. [Intel 248966, Section 2.1.1.4, page 2-4]

The array-of-pointers prefetcher watches a load that steps through memory with a constant stride, takes each value it loads as an address, and prefetches the data there, as for p[i]->field in a loop over i. [Intel 248966, Section 2.1.1.5, Figure 2-1, page 2-4]

It follows only a 64-bit integer load aligned to 8 bytes, whose result is the base register of the second load, with no index register. It prefetches the line the pointer points to, ignoring the offset of the second load, and does nothing for an offset over 32 bytes, so a small offset or none works best. Bit 7 of MSR 1A4h turns it off. [Intel 248966, Section 2.1.1.5, page 2-5]

The L3 page prefetcher sees a stream near the end of a 4 KB page, in either direction, and brings the page two pages ahead into the L3, all of it, without using L1 or L2 buffers. It helps streams over more than 8 KB and may waste bandwidth on shorter ones; bit 6 of MSR 1A4h turns it off. [Intel 248966, Section 2.1.1.6, page 2-5]

Like Golden Cove, each core has eight general-purpose counters, but it counts 11 architectural events, against 8 on Golden Cove: the measuring chapter says how to use them. [Intel 325384, Section 22.3.11.1, Table 22-57, pages 22-81 and 22-82]