Optimizing code for Intel Crestmont

Two decode clusters of three decoders each take turns: at each taken branch, the next block of code goes to the other cluster, so that both decode at once, out of order, and the micro-ops are put back in order at allocation. [Intel 248966, Sections 4.1.2 and 4.1.4, pages 4-3 and 4-4]

Long blocks without a taken branch, such as unrolled loops, no longer leave one cluster idle: when 32 micro-ops pass without a switch, the hardware notes a switch point near the 24th, and once it has seen the same point twice, the predictor keeps it. Unconditional jumps to the next instruction, which hand-written code for Tremont used for this, are not needed. [Intel 248966-049, Section 4.1.3, page 4-4; Intel 248966, Section 4.1.3, page 4-4]

The decoders find instruction boundaries from bits stored with the instruction cache. Code just fetched into it has none: two length decoders mark 16 bytes a cycle each, adding two cycles, which programs with a large code footprint pay most often. Tremont kept a second-level cache of these bits instead. [Intel 248966-049, Section 4.1.4, pages 4-4 and 4-5; Intel 248966, Section 4.1.4, page 4-4]

Each cluster decodes as Tremont did, so an instruction should carry at most 4 bytes of prefixes and escape bytes. Agner Fog measured long delays on Tremont beyond that, and for some combinations of three, such as F3 with 66 or a REX prefix. [Intel 248966-049, Section 4.1.4, page 4-5; Intel 248966, Section 4.1.4, page 4-4; Agner Fog, Section 16.2, page 188]

Each decoder emits one micro-op per instruction for most code, including an instruction that loads, computes and stores, and the CET instructions; instructions that need more come from microcode, and the two clusters can each run a short sequence of it, in either order. A 256-bit instruction, an integer multiply or divide with two results, and push of a memory operand become two or more micro-ops only at allocation. [Intel 248966, Sections 4.1.4 and 4.1.5, pages 4-4 and 4-5]

Crestmont predicts 128 bytes a cycle, against 32 on Gracemont, when the block starts in the first, even 64-byte half of a 128-byte aligned block; a taken branch into the odd half gets only its 64 bytes that cycle. [Intel 248966, Sections 4.1 and 4.1.2, pages 4-1 and 4-3]

Intel’s manual does not say whether these cores fuse a compare with the branch after it, and Agner Fog found no such fusion on Tremont. [Intel 248966, Sections 4.1.4 and 4.1.5, pages 4-4 and 4-5; Agner Fog, Section 16.5, page 189]

LLVM pads code for these cores with NOPs of up to 15 bytes, which it expects them to decode quickly; its Tremont tuning does not. [LLVM 23.1.2, X86.td, line 1432; LLVM 23.1.2, X86.td, line 755; LLVM 23.1.2, X86.td, line 1379]

A first predictor follows a taken branch every cycle without a bubble; a second one, with 5,000 targets on Gracemont and 6,000 on Crestmont and path history, checks it and corrects it three cycles later. A branch neither predictor knows is found by the decoders, later still. [Intel 248966-049, Section 4.1.2, page 4-3; Intel 248966, Sections 4.1 and 4.1.2, pages 4-1 and 4-3]

Two branches can execute a cycle, on ports 30 and 31. [Intel 248966, Section 4.1.6, page 4-6]

The top-down events tell the causes of a starved front end apart: TOPDOWN_FE_BOUND.BRANCH_RESTEER for the three-cycle corrections, BRANCH_DETECT for branches found by the decoders, ICACHE and ITLB for misses, PREDECODE for missing boundary bits or too many prefixes, CISC for microcode. The matching FRONTEND_RETIRED events mark the instruction after the bubble. [Intel 248966, Sections 4.1.2 and 4.1.4, pages 4-3 and 4-4]

Four ALU ports run simple integer operations, four a cycle; two of them, 01 and 02, also multiply and divide. Crestmont made division faster than on Gracemont; Intel gives no numbers. [Intel 248966, Sections 4.1 and 4.1.6, pages 4-1, 4-5 and 4-6]

An instruction that reads memory, computes and writes the result back allocates as one micro-op but executes as four: a load, the operation, the store address and the store data, on four ports. [Intel 248966, Section 4.1.6, page 4-6]

An lea with two of base, index and displacement and no scale runs on any ALU port in one cycle; one with all three is two micro-ops and a cycle longer; a scaled index with no displacement keeps it one micro-op, but on port 02 only. LLVM avoids three-operand lea for these cores. [Intel 248966, Section 4.1.6, page 4-6; LLVM 23.1.2, X86.td, line 1429; LLVM 23.1.2, X86.td, line 581]

Register moves, NOPs and zeroing idioms such as xor eax, eax are handled at allocation, and so are some loads, by memory renaming; they may never execute. [Intel 248966, Section 4.1.5, page 4-5]

On Gracemont, an instruction that needs two allocation lanes and falls on the fifth is split across two cycles. [Intel 248966-049, Section 4.1.5, page 4-5]

The E-cores keep the arithmetic flags as one register, so ADCX and ADOX depend on each other unless an instruction that writes all flags comes between them. Code that interleaves the two carry chains for the P-cores gains less of that parallelism here. [Intel 248966, Section 4.1.9.8, page 4-10]

LLVM’s tuning for these cores says that popcnt has a false dependency on its destination register, which its Alder Lake P-core tuning removes; Intel’s manual does not mention one. [LLVM 23.1.2, X86.td, line 1434; LLVM 23.1.2, X86.td, line 590; LLVM 23.1.2, X86.td, line 1421]

Vector units are 128 bits wide. Three vector ports take any mix of two floating-point additions, multiplications or FMAs a cycle, 16 single-precision or 8 double-precision operations, and three integer or shuffle operations, besides the AES and SHA units. [Intel 248966, Sections 4.1.6 and 4.1.9, pages 4-6 and 4-8]

A 256-bit instruction is one micro-op until allocation, then two 128-bit halves on two ports, so it takes twice the slots of a 128-bit one to allocate, execute and retire. [Intel 248966, Sections 4.1.5 and 4.1.9, pages 4-5 and 4-8]

Shuffles that move data across the 128-bit halves are slower: VPERM2I128, VPERM2F128, VPERMD, VPERMQ, VPERMPS and VPERMPD on 256 bits are two micro-ops with a latency of 7 cycles, the second holding a port for two cycles. Variable shuffles within each half are fast in LLVM’s tuning for these cores. [Intel 248966, Section 4.1.9.1, pages 4-8 and 4-9; LLVM 23.1.2, X86.td, line 1433]

A 256-bit broadcast from memory is one operation on top of the load, a cycle more; from a register it is two. [Intel 248966, Section 4.1.9.2, page 4-9]

Inserting 128 bits from memory into a 256-bit register, and widening loaded elements with vpmovzx, vpmovsx or the conversions VCVTPS2PD, VCVTDQ2PD and VCVTPH2PS, take two micro-ops, the second waiting for the first: 2 cycles after the load for the inserts and integer widenings, 6 for the conversions. From a register, the two halves run side by side. [Intel 248966, Section 4.1.9.3, page 4-9]

VBLENDVPS and VBLENDVPD on 256 bits are microcoded: 3 cycles of latency, one every 4 cycles. [Intel 248966, Section 4.1.9.4, page 4-9]

VPTEST, VTESTPS and VTESTPD on 256 bits are two dependent micro-ops, 7 cycles from the vectors to the flags, one a cycle. [Intel 248966, Section 4.1.9.5, page 4-10]

Gathers are microcoded and take about 50 cycles. [Intel 248966, Section 4.1.9.6, page 4-10]

vmaskmov and vpmaskmov loads and stores run one a cycle on 128 bits and one every two cycles on 256 bits, and can slow down when a masked-off element would fault if accessed. [Intel 248966, Section 4.1.9.7, page 4-10]

Crestmont has twice Gracemont’s SIMD integer multipliers, which doubles the throughput of the VNNI dot products, and adds VPDPBSSD and its kin for every pairing of signed and unsigned bytes. [Intel 248966, Sections 4.1, 4.1.6 and 4.1.8.3, pages 4-1, 4-5 and 4-8]

Crestmont adds the VEX forms of VPMADD52LUQ and VPMADD52HUQ, for big-integer arithmetic, and runs them in the floating-point multipliers. [Intel 248966, Sections 4.1.6 and 4.1.8.2, pages 4-5 and 4-8]

Crestmont converts BF16 and FP16 values to single precision in the load pipelines, with the AVX-NE-CONVERT instructions, so the conversion uses no vector port and overlaps arithmetic. [Intel 248966, Section 4.1.8.1, page 4-8]

Two loads and two stores a cycle, each of up to 16 bytes, so 32 bytes read and 32 written. The L1 data cache has two ports, without bank conflicts. [Intel 248966, Section 4.1.7, page 4-7]

A load whose address comes from an earlier load plus a displacement of 0 to 1023, as in following the links of a list, arrives in 3 cycles instead of 4. [Intel 248966, Section 4.1.7, page 4-7]

A load may run before older stores whose addresses are not known yet, and gets data from a store as fast as from the cache when that store’s address and data are ready. Crestmont forwards from a store that covers the load only in part in more cases than Gracemont. [Intel 248966, Section 4.1.7, pages 4-6 and 4-7]

Crestmont handles locked loads faster than Gracemont, which helps atomic operations; Intel gives no numbers. [Intel 248966, Section 4.1.7, page 4-7]

Three L1 prefetchers: the next line after a miss, strides per instruction, which cross pages and start their translations, and the next page when a stream nears its end. The L2 adds a stream prefetcher and one for more complex patterns, which may fill only the L3. [Intel 248966, Section 4.1.7, pages 4-7 and 4-8]

The four cores of a module share the L2’s bandwidth and its requests to the rest of the chip: 64 misses in flight, and 48 evictions on Crestmont against 16 on Gracemont, with a share kept for each core. [Intel 248966, Section 4.1.7, pages 4-7 and 4-8; Intel 248966-049, Section 4.1.7, page 4-9]

Crestmont adds linear address masking (LAM), but not on Meteor Lake. [Intel 248966, Section 4.1, page 4-1]

On a hybrid processor both kinds of core run the same instructions, so the P-cores lose what the E-cores lack: AVX-512, AVX-512 FP16 and TSX. Code for these processors stops at AVX2 and AVX-VNNI. [Intel 248966, Section 2.3.1, page 2-6; Agner Fog, Chapter 13, page 173]

LLVM enables AVX-IFMA, AVX-VNNI-INT8 and AVX-NE-CONVERT for -march=sierraforest, an E-core-only part, but not for -march=meteorlake, which keeps the Alder Lake set, so code it builds for Meteor Lake leaves out the instructions Crestmont added. [LLVM 23.1.2, X86.td, line 1439; LLVM 23.1.2, X86.td, line 1440; LLVM 23.1.2, X86.td, line 1443; LLVM 23.1.2, X86.td, line 1449; LLVM 23.1.2, X86.td, line 2123]

CPUID leaf 7 sets bit 15 of EDX on a hybrid part, and leaf 1Ah gives the type of the core the thread runs on in bits 31:24 of EAX: 20h for an E-core, 40h for a P-core. On Alder Lake both kinds report the same family and model, so the model number cannot tell them apart; early microcode gave each its own. [Intel 253665, Section 21.3, Table 21-21, page 21-29; Intel 253665, Section 21.3, Table 21-63, page 21-66; Linux 7.2 x86, include/asm/cpufeatures.h, line 432; Linux 7.2 x86, include/asm/intel-family.h, line 213; Linux 7.2 x86, include/asm/intel-family.h, line 220; Agner Fog, Chapter 13, page 173]

Intel’s Thread Director tells the operating system how each thread would run on each kind of core, from four classes of code it watches: scalar, vector, VNNI dot products, and spinning on pause. Vector floating-point code gains more from a P-core than scalar integer code does. [Intel 248966, Section 2.3.2.1, pages 2-6 and 2-7]

An E-core runs a thread faster than a P-core whose two hardware threads are both busy, and faster when the other cores of its module are idle; background work suits the E-cores. [Intel 248966, Sections 2.3.2.2 to 2.3.2.4, pages 2-9 and 2-10]

Intel asks programs not to pin threads or processes to cores, to wait with UMWAIT, TPAUSE or PAUSE rather than spin, so that the scheduler sees the waiting, and on Windows to state each thread’s quality of service through the power throttling API. Agner Fog finds it unrealistic to tune a program for each hybrid model, and relies on the operating system instead. [Intel 248966, Section 2.3.3, page 2-10; Agner Fog, Section 13.1, page 174]