AVX-512 Instruction Groups and Programming Intrinsics

While AVX-512 is most visibly an extension of AVX and AVX2 to a 512 bit width, AVX-512VL instructions are 128 or 256 bits wide. The VL subset comprises 27% of AVX-512 intrinsics and is often of greater interest than 512 bit operation. AMD Zen 4 processors implement AVX-512 at 256 bit width and Intel processors may not be faster at 512 bits than they are at 256 bits. AVX-512 and AVX-512VL's primary advantage over previous instruction sets (AVX, AVX2, and FMA) is arguably reduction in register spilling due to expansion from 16 ymm to 32 zmm registers and the addition of eight mask registers. The number of intrinsics triples to provide mask and maskz versions which make use of the mask registers. Of the 14 current instruction groups four are general (F, VL, DQ, BW) and 10 accelerate more specific workloads (BITALG, BF16, CD, FP16, IFMA52, VBMI, VBMI2, VNNI, VPINTERSECT, and VPOPCNT).

group	what	instructions	intrinsics	Zen	Sunny Cove	Skylake	Knights	VL intrinsics
F	foundation	389	1435	4	Rocket, Tiger, Ice Lake	X, Xeon 2017+	all
VL	128 and 256 bit widths	223	1208	4	Rocket, Tiger, Ice Lake	X, Xeon 2017+		1208
DQ	doubleword and quadword	87	399	4	Rocket, Tiger, Ice Lake	X, Xeon 2017+		176
BW	byte and word	150	764	4	Rocket, Tiger, Ice Lake	X, Xeon 2017+		446
CD	conflict detection	8	42	4	Rocket, Tiger, Ice Lake	X, Xeon 2017+	all	28
BITALG	population count expansion	5	24	4	Rocket, Tiger, Ice Lake			16
IFMA52	big integer FMA	3	18	4	Rocket, Tiger, Ice Lake	(Cannon Lake)		12
VBMI	vector byte manipulation	8	30	4	Rocket, Tiger, Ice Lake	(Cannon Lake)		20
VBMI2	vector byte manipulation	21	150	4	Rocket, Tiger, Ice Lake			100
VNNI	vector neural network	5	36	4	Rocket, Tiger, Ice Lake	Cascade Lake		24
VPOPCNT	population count	3	18	4	Rocket, Tiger, Ice Lake		Mill	12
BF16	half precision	5	27	4				18
FP16	half precision	96	938					600
VP2INTERSECT	vector pair to mask pair	2	6					4
ER	exponential and reciprocal	12	60				all
PF	prefetch	9	20				all
4FMAPS	single precision 4x1 FMA	4	12				Mill
4NNIW	vector neural network	2	6				Mill
total		1031	5193					2060

AVX-512 was introduced by Intel in 2016 on Xeon Phi processors (Knights Landing and, later, Knights Corner). Beginning in Q3 2017, Intel Skylake X-series parts (i7 and i9) and Xeon processors enabled support 3959 of the 5139 AVX-512 intrinsics now defined by Intel. In Q3 2019, Ice Lake (Sunny Cove microarchitecture) expanded the set to 4130 intrinsics and the Golden Cove microarchitecture to 5095 intrinsics (announced Q4 2021 but supporting parts appear unlikely to ship until 2023). Xeon Phi's 4FMAPS, 4NNIW, and PF instruction groups have been superceded by more recent groups and architectural changes, thus appearing to be obsolete. ER instructions are valuable to some floating point but have not been reimplemented.

For AMD parts, the table above is based on Phoronix' performance analysis as AMD hasn't yet updated the AMD64 Architecture Programmer's Manual or Revision Guide for AMD Family 19h Models.

For Intel parts, the table above derives from the Intel Intrinsics Guide, Intel ARK, and Intel 64 and IA-32 Architectures Software Developer's Manuals. It will therefore be inaccurate if Intel's information is inaccurate or if transcription errors were made. In particular, sections 15.2-4 of the architecture manual, volume 1, require software check for F before using other groups. However, the Intrinsics Guide does not indicate corresponding dependencies for many groups. The spreadsheet in this repo lists each group's instructions and intrinsics.

AVX-512 Availablity

release dates	processor	laptop, desktop	workstation, server
Q1 2023?	Sapphire Rapids		repeatedly delayed
Q3 2022	Zen 4	7000
Q1 2021 to Q3 2021	Rocket Lake	i5, i7, i9	E, W
Q3 2020 to Q3 2021	Tiger Lake	i3, i5, i7, i9	W
Q2 2020	Cooper Lake		Gold, Platinum
Q3 2019 to Q2 2021	Ice Lake	i3, i5, i7	W, Silver, Gold, Platinum
Q2 2019 to Q1 2020	Cascade Lake	i9	W, Bronze, Silver, Gold, Platinum
Q3 2018	Cannon Lake	i3-8121U
Q3 2017 to Q2 2018	Skylake	X-Series i7, i9	W, Bronze, Silver, Gold, Platinum
Q4 2017	Knights Mill		Phi
Q2 2016 to Q4 2016	Knights Landing		Phi

Prior to AMD's Zen 4 release in September 2022, AVX-512 was most readily—albeit somewhat briefly—available on Intel 11^th generation i5, i7, and i9 parts (Rocket Lake) before being disabled on 12^th generation (Alder and, likely, Raptor Lake). Cascade Lake and Skylake Xeons provided AVX-512, along with more limited availabiliy from Cooper, Tiger, and Ice Lake. Cannon Lake i3s (Palm Cove microarchitecture) are rare and the Kaby and Coffee Lake Skylake iterations lack AVX-512. Alder Lake P-cores implement 14 instruction groups (Sunny Cove instructions plus BF16, FP16, and VP2INTERSECT) but typically have AVX-512 fused off. AVX-512 can be enabled on early Alder Lakes but Intel has suppressed this ability through microcode.

Specfic processors are listed in this repo's spreadsheet and the table above uses Intel ARK release dates for Intel parts. No Pentium or Celeron processor supports AVX-512. AMD did not support AVX-512 prior to Zen 4.

Instruction Group Dependencies and CPUID Flags

The 18 AVX-512 instruction groups have individual CPUID flags and, in principle, an instruction could require an arbitrary number of groups be present. In practice, this is rarely a concern as the only dependencies which exist are on groups F and VL. Since VL is not present independent of F on any of Intel's processors, its dependency is always satisfied. Similarly, all processors with instruction groups containing VL dependent intrinsics implement VL. There are also 324 128 or 256 bit intrinsics for which the Intrinsics Guide does not indicate a VL requirement. These are primarily ss and sd floating point intrinsics which modify only the low 32 or 64 bits of a register.

It appears reasonable to assume all Skylake and later processors with AVX-512 support will implement at least the F, CD, VL, DQ, and BW groups. While Intel could choose otherwise, doing would complicate hardware implementation, compiler support, and software compatibility. It is also plausible the BITALG, IFMA52, VBMI, VBMI2, VNNI, and VPOPCNT groups will be consistently implemented together from Ice Lake forward for the same reasons. However, Intel does not seem to have made any official statement regarding future compatibility. Similarly, AMD does not appear to have indicated what groups a Zen 4 implementation might support. However, an AMD implementation seems likely to be broadly compatible with Intel's.

Since AVX-512 is restricted to 512 bit widths on the now discontinued Xeon Phis, these processors do not implement the vector length extensions of the VL group. They therefore lack dependencies between groups and share only the F and CD groups with Skylake and later implementations.

Performance Considerations

The Skylake-Cascade Lake and Sunny Cove-Cypress Cove microarchitectures provide SIMD computation on ports 0, 1, and 5. It appears AVX-512 operation is obtained by combining ports 0 and 1 and, when two AVX-512 instructions per clock are supported, possibly by combining ports 5 and 6. In addition to downclocking when the instructions and threads used trigger downclocking by crossing thermal license boundaries, use of AVX-512 may sometimes be slower than implementing the same workload with AVX and AVX2. This occurs because the instruction rate decrease from 3x256 per clock to 2x512 may not be offset by wider loads and stores (Fog 2015, Stackoverflow), zmm register availability, or use of more efficient instructions. AVX-512 may be similarly disadvantageous on processors restricted to 1x512, which includes bronze, silver, some 5000 series gold, and D Skylake Xeons as well as Knights processors. As of September 2022, how the addition of port 10 and other microarchitectural changes in Golden Cove may alter these considerations is uncertain.

In general, compute kernel throughput is sensitive to instruction level parallelism available within the kernel's inner loop and a processor's FMA, ALU, shift, floating point divide, shuffle, and load and store capabilities. For Ice Lake, Intel indicates one AVX-512 FMA and shuffle and two AVX-512 ALUs (e.g. Cutress 2019). Some kernels may therefore execute more quickly at 256 bit width due to accessing two FMA and shuffle units rather than one. Additionally, using the 1400 128 and 256 bit AVX-512VL intrinsics to reduce register spilling from 32 zmm instead of 16 ymm registers may be more efficient than the width of the 3700 512 bit intrinsics. In some cases 128 bit kernels can also be faster than 256 or 512 bit ones due to computational details of the kernel such as dependencies between AVX lanes.

It's therefore useful to profile SIMD implementations at 128, 256, and 512 bit widths across processors and amounts of computation to be performed. In performance code segments, this can result in width dispatching being controlled by loop content or iteration count rather than which instructions are supported by the processor.