Optimizing code for Intel Ice Lake

Fetch still brings 16 bytes of code a cycle into four decoders, as on Skylake, while everything behind them got wider, so code that misses the micro-op cache is limited by decoding more than before. Agner Fog advises keeping the hottest code under about 2,000 instructions, which the micro-op cache holds, and not unrolling loops further than needed. Golden Cove fetches 32 bytes a cycle into six decoders. [Agner Fog, Sections 12.2 and 12.15, pages 163 and 171]

The micro-op cache and the loop buffer each deliver up to 5 micro-ops a cycle, the loop buffer whatever the length of the instructions. [Agner Fog, Sections 12.3 and 12.4, page 164]

An arithmetic or logic instruction with a 16-bit immediate, such as add ax, 1234, costs 3 cycles in the decoders, and unlike on Skylake a 16-bit mov with an immediate pays too; the front-end chapter compares other cores. [Agner Fog, Section 12.2, page 163]

The micro-op queue holds 70 micro-ops for each thread, against 64 on Skylake, and its loop stream detector replays loops of up to 70 micro-ops whether one thread runs or two. It sends 5 micro-ops a cycle, so it unrolls a short loop in the queue to keep the last group of each pass from going out partly empty. [Intel 248966, Section 2.5.1.1, page 2-18; Intel 248966, Section 3.4.2.4, page 3-18]

A cmp, add, sub, inc or dec fuses with a following jump on zero, carry or an unsigned or signed comparison, and test or and with any conditional jump; or, xor, adc, sbb, shifts and js, jo and jp never fuse. Unlike on Skylake, the first instruction may not have a memory operand, though an immediate is fine. Fusion now happens after decoding, and the pair runs as one micro-op on the branch unit. [Agner Fog, Section 12.6, Table 12.1, page 165]

An instruction that reads memory and computes, one fused micro-op until allocation splits it, now takes two decoder slots where earlier cores gave it one, while a compare fused with its jump counts as one; Agner Fog found that no fusion lifts the rate above 5 instructions a cycle. Behind allocation, up to 9 micro-ops a cycle have been seen. [Agner Fog, Sections 12.2, 12.4 and 12.5, pages 163 and 164]

Intel says the predictor is better, without details, and Agner Fog could not find how many targets its buffer holds. The return stack holds 22 entries, against 16 on Skylake. [Intel 248966, Section 2.5.1.1, page 2-18; Agner Fog, Section 12.15, page 172; Agner Fog, Section 3.8, page 31]

Taken branches run at one every cycle or every two, not-taken ones at two a cycle, and more than two branches in 16 bytes of code slow them down, so hot code should fall through on its common path, as on Skylake. [Agner Fog, Sections 12.9 and 12.15, pages 166 and 172]

Intel’s old rule against two conditional branches whose last bytes share an aligned 8-byte block while their targets agree in the low 6 bits no longer applies from Ice Lake on; the branches chapter gives the rest of the density rules. [Intel 248966, Section 3.4.1, page 3-4]

Each of the four integer ports has an LEA unit, against two on Skylake, where three-operand forms ran on port 1 only; LLVM’s tuning for these cores no longer counts any lea form as slow. [Intel 248966, Sections 2.5.1 and 2.5.1.2, Table 2-6, pages 2-17 to 2-19; LLVM 23.1.2, X86.td, line 1204; LLVM 23.1.2, X86.td, line 581]

A new integer divider on port 1 makes DIV and IDIV much faster than on Skylake, in latency and in throughput; neither Intel nor Agner Fog gives numbers. Golden Cove keeps the divider on port 1. [Intel 248966, Sections 2.5.1 and 2.5.1.2, Figure 2-4 and Table 2-6, pages 2-17 to 2-19; Agner Fog, Section 12.15, page 172]

The rest of integer execution is laid out as on Skylake: one port, port 1, for multiplication, bsr, the double shifts and PDEP, with the high half of a 64-bit multiplication on port 5, and two ports each for shifts and for the bit manipulation instructions; LLVM’s model gives them Skylake’s latencies. [Intel 248966, Section 2.5.1.2, Tables 2-6 and 2-7, pages 2-19 and 2-20; LLVM 23.1.2, X86SchedIceLake.td, line 136; LLVM 23.1.2, X86SchedIceLake.td, line 823; LLVM 23.1.2, X86SchedIceLake.td, line 828]

LLVM’s tuning for these cores, unlike its tuning for Skylake, no longer zeroes the destination of popcnt to break a dependency on its old value. [LLVM 23.1.2, X86.td, line 1204; LLVM 23.1.2, X86.td, line 590]

Zeroing idioms such as xor eax, eax and pxor of a register with itself still break the dependency on the old value, but now take an execution unit. Moves between vector registers of 128 bits or more are removed when renamed, chains of them included, while moves between general registers, unlike on Skylake, are not. [Agner Fog, Section 12.8, pages 165 and 166]

Operations on AH, BH, CH and DH still run on ports 1 and 5 with a latency of one cycle and one more for what depends on them, as on Skylake; Golden Cove made them slower. [Intel 248966, Section 3.5.2.3, page 3-32]

Ice Lake brought AVX-512 to client processors, with the extensions of the Skylake servers and VBMI, VBMI2, IFMA, VNNI, BITALG and VPOPCNTDQ, plus VAES, VPCLMULQDQ, GFNI and SHA. LLVM gives Tiger Lake VP2INTERSECT, MOVDIRI, MOVDIR64B and the shadow stack on top, and Rocket Lake nothing beyond the client Ice Lake. [Intel 248966, Sections 2.5.1 and 2.5.1.5, pages 2-17 and 2-24; LLVM 23.1.2, X86.td, line 1195; LLVM 23.1.2, X86.td, line 1233; LLVM 23.1.2, X86.td, line 2102; LLVM 23.1.2, X86.td, line 1164]

Ports 0 and 1 each have a 256-bit FMA unit and join, as on Skylake’s servers, to run a 512-bit operation on port 0; port 5 has 512-bit integer and shuffle units but no FMA. The client parts thus run one 512-bit FMA, multiplication or addition a cycle, the same work as two 256-bit ones, and 512-bit integer additions at two a cycle against three for narrower ones. [Intel 248966, Section 2.5.1.2, Table 2-6, page 2-19; Intel 248966, Section 2.5.1.6, page 2-25; Agner Fog, Section 12.9, page 166; LLVM 23.1.2, X86SchedIceLake.td, line 319]

Floating-point addition, multiplication and FMA all take 4 cycles, as on Skylake; Golden Cove gave addition a faster unit. Most 512-bit operations take as long as narrower ones, division and square root excepted. [Agner Fog, Sections 12.9 and 12.15, pages 167 and 172; LLVM 23.1.2, X86SchedIceLake.td, line 257; LLVM 23.1.2, X86SchedIceLake.td, line 316]

A second shuffle unit on port 1 takes shuffles that stay within 128-bit lanes, up to 256 bits wide, so such shuffles no longer all wait for port 5; those crossing lanes, such as VPERMD, and all 512-bit ones still run there, taking 3 cycles if they cross. Rewriting a shuffle to work within lanes lets it use the new unit, and LLVM’s tuning stops turning shuffles into shifts as it does for Skylake’s servers. [Intel 248966, Section 2.5.1, Figure 2-4, page 2-17; Intel 248966, Section 15.11, page 15-32; Agner Fog, Section 12.9, Table 12.2, page 167; LLVM 23.1.2, X86SchedIceLake.td, line 392; LLVM 23.1.2, X86SchedIceLake.td, line 602; LLVM 23.1.2, X86.td, line 1204; LLVM 23.1.2, X86.td, line 693]

A result passed between the one-cycle vector integer operations of ports 0 and 1 and the FMA or vector multiply units, either way, arrives a cycle late, as on Golden Cove; moves between general and vector registers take 2 cycles. [Intel 248966, Section 2.5.1.2, Table 2-8, page 2-20; Agner Fog, Section 12.9, page 168]

Agner Fog found the same cost of mixing SSE with AVX as on Skylake: no switch of state, but a legacy SSE instruction that writes an XMM register while upper halves hold data depends on the register’s old value. vzeroupper after AVX code, or keeping AVX code in registers 16 to 31, avoids it. Intel measured a mixed sequence at one cycle on the Sunny Cove servers, where Golden Cove takes 388. [Agner Fog, Section 12.10, pages 168 and 169; Intel 248966, Section 3.11.6.1, Table 3-12, pages 3-80 and 3-81]

The upper parts of the 512-bit units sleep while unused, and 512-bit code runs about 4.5 times slower for some 50,000 cycles after it starts, as on Skylake. [Agner Fog, Section 12.9, page 168]

Masking is free in most instructions, but a masked load adds a micro-op on port 0 or 5 and a cycle, a masked store about 13 cycles of latency, against about 10 on Skylake, and a masked move between registers is never removed by renaming. Zero masking, unlike merge masking, does not wait for the old destination. [Agner Fog, Section 12.9, page 168]

A subnormal result from normal inputs, or a multiplication or division with a subnormal input, costs about 120 cycles; additions with one do not, and FTZ with DAZ avoids it, as on Skylake. [Agner Fog, Section 12.9, page 168]

Ice Lake’s client cores are the first whose frequencies may differ from core to core, each chosen by what that core runs. The top frequency under AVX-512 sits much closer to the one under AVX2 than on Skylake’s servers, and when power limits the clock under AVX2 as well, AVX-512 may come out ahead, needing fewer instructions for the same work. [Intel 248966, Section 2.5.1.6, page 2-25]

Intel still advises building with AVX-512 and comparing against AVX2 before porting a program, as some code runs slower with the longer vectors; Agner Fog finds 512-bit vectors worth using in most cases, the lower clock costing less than they gain. [Intel 248966, Section 2.5.1.6, page 2-25; Agner Fog, Section 12.15, pages 171 and 172]

LLVM’s tuning for these cores counts VPMULLQ as slow: its model makes it three micro-ops with a latency of 15 cycles. [LLVM 23.1.2, X86.td, line 1220; LLVM 23.1.2, X86SchedIceLake.td, line 1946]

Agner Fog finds gathers efficient, more so than on older cores. LLVM’s model gives an eight-element VPGATHERDD five micro-ops and 22 cycles, as for Skylake, and a 16-element one 25. [Agner Fog, Section 12.15, page 172; LLVM 23.1.2, X86SchedIceLake.td, line 2203; LLVM 23.1.2, X86SchedIceLake.td, line 2156]

VPDPBUSD, which replaces a three-instruction sequence of byte multiplications and additions, gains only two times on the client parts against three times on the servers, because on the clients the older sequence’s addition can also use port 5. VPDPWSSD, its 16-bit sibling, gains nothing on the client parts. [Intel 248966, Section 8.2.1, Figure 8-1, page 8-2; Intel 248966, Sections 8.2.1 and 8.2.2, page 8-4]

AES rounds run on ports 0 and 1, so existing AES-NI code can reach twice its old peak rate. VAES then works on up to four blocks in one instruction, for up to 3.3 times the speed of AES-NI code in Intel’s figures. [Intel 248966, Section 2.5.1, page 2-17; Agner Fog, Section 12.9, Table 12.2, page 167; Intel 248966, Section 19.1, page 19-1]

VPCLMULQDQ does four carry-less multiplications at once in a 512-bit register, up to four times the rate of PCLMULQDQ, for the GHASH of AES-GCM. [Intel 248966, Section 19.2, page 19-2]

VPMADD52LUQ and VPMADD52HUQ multiply eight pairs of 52-bit numbers and add the low or the high 52 bits of each product to a 64-bit accumulator, for the big-number arithmetic of RSA. [Intel 248966, Section 19.4, page 19-4]

Two loads a cycle of up to 512 bits each and, new, two stores a cycle of up to 256 bits each, against one on Skylake; a 512-bit store takes both store data ports. Golden Cove loads three a cycle. [Intel 248966, Sections 2.5.1 and 2.5.1.3, pages 2-17 to 2-22; Agner Fog, Section 12.9, page 167]

Two stores reach the L1 cache in one cycle only as a pair: both within one cache line, of one memory type, write-back or write-combining, neither crossing a line or a page. Aligned stores, placed so that consecutive ones fall in one line, pair best; a loop that alternates between two arrays pairs nothing unless unrolled to store twice to each in turn, and a call, storing its return address, breaks the pairs around it. [Intel 248966, Section 2.5.1.3, Examples 2-6 and 2-7, pages 2-22 and 2-23; Agner Fog, Section 12.9, page 167]

The L1 data cache takes two 64-byte loads and a 64-byte store, or two 32-byte ones, each cycle; the L2 delivers 64 bytes a cycle at best and 48 sustained, against about 29 on the client Skylake, the L3 32 and 21. [Intel 248966, Section 2.5.1.3, Table 2-9, page 2-21]

Ice Lake predicts which store a load will read from and gives the load that store’s data without reading memory, for integer loads only. Neither may use an index register or a RIP-relative address, and 16-bit and high-byte registers miss out: an lea computing the address ahead, or a movzx instead of a narrow load, keeps such code eligible. [Intel 248966, Section 2.5.1.4, Example 2-8, pages 2-23 and 2-24; Agner Fog, Section 12.13, pages 170 and 171]

Ordinary forwarding takes 5 cycles for up to 128 bits and 7 or more for wider data, 2 more when a store of 128 bits or more is misaligned or one crosses a cache line. When it fails, as for a load wider than the store, the load waits 19 or 20 cycles, against about 11 on Skylake. A load still waits for a pending store to an address with the same cache set and offset as if they overlapped, like Skylake. [Agner Fog, Sections 12.12 and 12.13, page 170]

rep movsb is fast for 1 to 128 bytes, reported by CPUID leaf 7, EDX bit 4, while rep stosb is unchanged. Inlined, it copies as fast as 256-bit AVX code when the length is known only at run time, nearly as fast for short known lengths (9 cycles against 3 to 7) and faster from 2 KB up; a zero length is still best handled by a branch. Golden Cove made short rep stosb, rep cmpsb and rep scasb fast as well. [Intel 248966, Section 3.7.6.1, page 3-63; Intel 248966, Section 15.16.3, page 15-61]

12 write-combining buffers, against 10 on the cores from Nehalem to Skylake. [Intel 248966, Section 3.6.9, page 3-55]

Two threads on a core share the caches, the decoders and the execution units; code limited by fetch or cache size, or that already runs over two instructions a cycle, gains nothing from a second thread, as on Skylake. [Agner Fog, Sections 12.14 and 12.15, pages 171 and 172]

Each thread has 8 general-purpose counters, twice as many as before, and a fourth fixed counter for issue slots, from which the core gives the four top-level shares of the top-down method without using the eight; the AnyThread bit is gone. [Intel 248966, Section 2.5.1.5, pages 2-24 and 2-25]