ENQCMDS instruction (ENQCMD, APX)

ENQCMDS is the kernel’s form of ENQCMD: it reads a 64-byte command from memory and writes it in one piece to a device’s enqueue register, a shared work queue’s address in MMIO space, and ZF tells whether the device took it. ENQCMD’s page describes the write, its ordering and the retry status. The difference is where the identity of the sender comes from. ENQCMD takes the process address space ID (PASID) from the IA32_PASID MSR and always marks the command as sent by a user; ENQCMDS takes both from the command itself, the PASID from bits 19:0 and the privilege bit, 1 for supervisor, from bit 31. Bits 30:20 of the source must be zero.

Where it runs

Since it lets software claim any address space, ENQCMDS needs CPL 0: at CPL 1 to 3 it raises #GP(0), as it does for a destination that is not aligned to 64 bytes and for nonzero bits 30:20. It also runs in real mode, where the CPL is 0, and raises #GP in virtual-8086 mode. The destination register is an offset in ES, which no prefix can change; #UD comes only from a missing CPUID bit, the same as ENQCMD’s (leaf 7, ECX bit 29), or a LOCK prefix. ZF is set for retry and cleared for success, and the other arithmetic flags are cleared. APX adds an EVEX form for its new registers.

In Linux

Linux 7.3’s arch/x86/include/asm/special_insns.h wraps ENQCMDS as bytes, returning -EAGAIN when ZF is set. The driver for Intel’s data streaming and in-memory analytics accelerators (drivers/dma/idxd) uses it for shared work queues, and MOVDIR64B for dedicated ones: for its own commands it uses a PASID set up for the kernel’s DMA mappings, and when a program submits a command through the driver’s device file, the driver clears the privilege bit and puts in the program’s PASID. A busy queue is retried 32 times by default, a number the administrator can change.

Not run here

Zen 5 reports no ENQCMD in CPUID, so the probe left ENQCMDS unrun. GCC 14 and Clang 23 provide _enqcmds(dst, src) with -menqcmd, which returns ZF; for a loop that retries 32 times, GCC 14 at -O2 keeps ENQCMDS, SETE and a test in a loop, while Clang 23 unrolls all 33 tries into ENQCMDS and JNE pairs.

Forms

Every encoding of ENQCMDS that Intel XED knows, one row per XED instruction form (iform). How to read the notation.

Instruction	Encoding	Requires
`ENQCMDS r16/32/64, m512` ENQCMDS_GPRa_MEMu32	`F3 0F 38 F8 /r`	ENQCMD
`ENQCMDS r16/32/64, m512` ENQCMDS_GPRav_MEMu32_APX	`EVEX.LLZ.F3.MAP4.SCALABLE F8 /r`	APX_F + ENQCMD

Latency and throughput

What the scheduling models of LLVM 23.1.2 give for each form, not measurements: latency / reciprocal throughput in cycles, the ports under them. A row is a model, or models that agree; a dash, a form that its processors lack. Memory forms: latency from the address, ports without the load and store units. Which processors each model covers.

Model	`r32, m512`
Sapphire Rapids	1 / 1p49

Operation

What the forms do, in pseudocode that our tests run against results recorded on real processors. How to read.

if CPL > 0:
    fault(GP, 0)
a = dest.read()                  # 64 bits, 32 with a 67 prefix; ES,
                                 # which no prefix overrides
if a % 64:
    fault(GP, 0)                 # 64-byte aligned
v = src.read()                   # 64 bytes, not read atomically
if v >> 20 & 0x7FF:
    fault(GP, 0)                 # bits 30:20 must be 0
zf = 1 - enqueue(a, v)           # the PASID (19:0) and the privilege
                                 # (31) as the source gives them
of = sf = af = pf = cf = 0

Not run on a processor: no processor at hand has ENQCMD.

Operands

What each form reads and writes, as Intel XED records it: the operands of its syntax, in order, and the implicit ones that the syntax does not name. How to read.

Operand	Kind	Action
`ENQCMDS r16/32/64, m512`
`r16/32/64`	explicit	read
`m512`	explicit	read
`RFLAGS`	implicit	write

Flags

M modified, 0 cleared; blank: not affected. How to read.

OF	SF	ZF	AF	PF	CF
All forms
0	0	M	0	0	0

Exceptions

The exception class is XED's name for the set of exception conditions that a form follows. How to read.

Legacy forms: none in XED
EVEX forms: APX-EVEX-ENQCMD

Attributes

What else XED records about the forms. How to read.

All forms: privileged: CPL 0 only

CPUID and processors

Each form belongs to an XED ISA set. A form can be used when the processor reports every CPUID bit of one of its ISA set's alternatives.

CPUID requirements and processors of the XED ISA sets above. Alternatives are separated by “or”; processors follow XED’s chip model, whose AMD processors end with Zen 2, and the CPU data of this site for Zen 3 to Zen 6.

ISA set	CPUID	Processors
`ENQCMD` ENQCMD	ENQCMD `CPUID.(EAX=07H,ECX=0):ECX[29]` ENQCMD	Sapphire Rapids, Emerald Rapids, Granite Rapids, Diamond Rapids, Sierra Forest, Clearwater Forest
`APX_F_ENQCMD` APX	APX_F + ENQCMD `CPUID.(EAX=07H,ECX=1):EDX[21]` APX_F `CPUID.(EAX=07H,ECX=0):ECX[29]` ENQCMD	Nova Lake, Diamond Rapids

More ENQCMD instructions

ENQCMD

More APX instructions

All 88 APX instructions

Where it runs#

In Linux#

Not run here#