SYSCALL instruction (SYSCALL/SYSRET)

SYSCALL, without FRED, does this in one step:

saves the address of the next instruction in RCX and RFLAGS in R11;
clears the flags that are set in the IA32_FMASK MSR, such as IF, so that the kernel starts with interrupts off;
loads CS and SS with selectors from bits 47:32 of IA32_STAR, with fixed descriptor contents for a flat 64-bit code segment at privilege level 0;
jumps to the address in IA32_LSTAR.

It does not change RSP. The kernel’s entry code usually executes SWAPGS to reach per-CPU data, switches to a kernel stack and saves the user registers. RCX and R11 are lost to the caller, which is why the Linux convention passes the fourth argument in R10 instead of RCX (see the system call conventions).

SYSRET reverses it: RIP from RCX, RFLAGS from R11, and CS and SS from bits 63:48 of IA32_STAR, to 64-bit code with REX.W or to 32-bit compatibility mode without. It does not restore RSP either. In Intel’s manual it raises a general-protection fault, still at privilege level 0, if RCX holds a non-canonical address, so a kernel that has already loaded the user’s stack pointer must prevent that or handle the fault on a stack of its own; AMD’s manual has no such check (see SYSRET).

SYSCALL works only when IA32_EFER.SCE is set; otherwise it raises #UD. With FRED enabled it delivers an event of its own type instead, and the kernel returns with ERETU.

Forms

Every encoding of SYSCALL that Intel XED knows, one row per XED instruction form (iform). How to read the notation.

Instruction	Encoding	Requires
`SYSCALL` 64-bit mode only SYSCALL	`0F 05`	LM
`SYSCALL` not in 64-bit mode SYSCALL_AMD	`0F 05`	Base instruction set

Operation

What the forms do, in pseudocode that our tests run against results recorded on real processors. How to read.

Forms only in 64-bit mode

if not efer & 1:
    fault(UD)                    # EFER.SCE clear
if not canonical(lstar):
    fault(GP, 0)                 # Intel's, where 4-level paging
                                 # runs on a processor of 5 levels,
                                 # whose LSTAR takes 57 bits
rcx = rip                        # the next instruction
r11 = rflags & ~0x10000          # RF clear in AMD's manual; Intel's
                                 # copies RFLAGS as it is
f = rflags & ~fmask              # the flags SFMASK names cleared
cf = f & 1
pf = f >> 2 & 1
af = f >> 4 & 1
zf = f >> 6 & 1
sf = f >> 7 & 1
tf = f >> 8 & 1
if_ = f >> 9 & 1
df = f >> 10 & 1
of = f >> 11 & 1
iopl = f >> 12 & 3
nt = f >> 14 & 1
ac = f >> 18 & 1
vif = f >> 19 & 1
vip = f >> 20 & 1
id = f >> 21 & 1
rf = 0
vm = 0
s = star >> 32 & 0xFFFF          # STAR[47:32]
cs = s & 0xFFFC                  # 64-bit code of DPL 0: CPL 0
ss = s + 8                       # data of DPL 0
rip = lstar
# RSP stays: the kernel switches to its own stack

Not run on a processor: SYSCALL is a system call of the program that runs it, which the probe never makes.

Forms not in 64-bit mode

if not efer & 1:
    fault(UD)                    # EFER.SCE clear
s = star >> 32 & 0xFFFF          # STAR[47:32]
cs = s & 0xFFFC
ss = s + 8
if efer >> 10 & 1:               # compatibility mode (EFER.LMA)
    rcx = rip                    # into 64-bit mode, as above, at
    r11 = rflags & ~0x10000      # CSTAR's entry point
    f = rflags & ~fmask
    cf = f & 1
    pf = f >> 2 & 1
    af = f >> 4 & 1
    zf = f >> 6 & 1
    sf = f >> 7 & 1
    tf = f >> 8 & 1
    if_ = f >> 9 & 1
    df = f >> 10 & 1
    of = f >> 11 & 1
    iopl = f >> 12 & 3
    nt = f >> 14 & 1
    ac = f >> 18 & 1
    vif = f >> 19 & 1
    vip = f >> 20 & 1
    id = f >> 21 & 1
    rf = 0
    vm = 0
    rip = cstar
else:                            # legacy mode: 32-bit code
    ecx = rip
    if_ = 0
    rf = 0
    vm = 0
    rip = star & 0xFFFFFFFF      # STAR[31:0]
# Intel's processors raise #UD outside 64-bit mode

Not run on a processor: this SYSCALL exists outside 64-bit mode only, where the probe does not run, and is a system call of the program that runs it.

Operands

What each form reads and writes, as Intel XED records it: the operands of its syntax, in order, and the implicit ones that the syntax does not name. How to read.

Operand	Kind	Action
`SYSCALL` 64-bit mode only
`RIP`	implicit	write
`RCX`	implicit	write
`R11`	implicit	write
`RFLAGS`	implicit	write
`SYSCALL` not in 64-bit mode
`rIP`	implicit	write
`RFLAGS`	implicit	write

Flags

M modified, 0 cleared; blank: not affected. How to read.

OF	SF	ZF	AF	PF	CF	DF	IF	TF	AC	IOPL	NT	RF	VM	VIF	VIP	ID
All forms
M	M	M	M	M	M	M	M	M	M	M	M	0	0	M	M	M

Attributes

What else XED records about the forms. How to read.

Forms only in 64-bit mode: may abort a TSX transaction

CPUID and processors

Each form belongs to an XED ISA set. A form can be used when the processor reports every CPUID bit of one of its ISA set's alternatives.

CPUID requirements and processors of the XED ISA sets above. Alternatives are separated by “or”; processors follow XED’s chip model, whose AMD processors end with Zen 2, and the CPU data of this site for Zen 3 to Zen 6.

ISA set	CPUID	Processors
`LONGMODE` General-purpose System SYSCALL/SYSRET	LM `CPUID.80000001H:EDX[29]` LM	Every x86-64 processor in XED’s model and Zen 3 to Zen 6
`AMD` SYSCALL/SYSRET	No CPUID bit: part of the base instruction set.	K10 (Family 10h), Bulldozer, Piledriver, Zen, Zen+, Zen 2, Zen 3, Zen 4, Zen 5, Zen 6

More SYSCALL/SYSRET instructions

SYSRET