build: whisper-server for processors without AVX2 (ivybridge, sandybridge)

[greatcoat]

In short: every x64 CPU whisper-server we publish needs AVX2, so it crashes on startup on older PCs (Intel before 2013's Haswell, AMD FX). This adds two builds for those processors, a startup check so each refuses to run on a processor below its level, and a CI check that runs each one on an emulated processor of exactly that generation. OpenWhispr then falls back to them (OpenWhispr/openwhispr#2396). Refs OpenWhispr/openwhispr#2356, also reported in OpenWhispr/openwhispr#1418, OpenWhispr/openwhispr#1613 and OpenWhispr/openwhispr#793.

Problem

-DGGML_NATIVE=OFF does not mean "baseline". When not cross-compiling, ggml turns every x86 option up to AVX2 on (ggml/CMakeLists.txt:141-158). The 0.0.10 CI logs print exactly that:

windows-x64-cpu (MSVC 19.44): Adding CPU backend variant ggml-cpu: /arch:AVX2 GGML_AVX2;GGML_FMA;GGML_F16C;__BMI2__;GGML_BMI2
linux-x64-cpu (GCC 11.4):     Adding CPU backend variant ggml-cpu: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2 ...

The server is statically linked, so nothing picks a CPU variant at run time. On a processor with AVX but no AVX2, FMA or BMI2, it dies in ggml_cpu_init() before it serves anything:

• Windows: STATUS_ILLEGAL_INSTRUCTION, exit code 3221225501.
• Linux: SIGILL.

OpenWhispr/openwhispr#1613's Event Viewer fault offset 0xc072d disassembles to a BMI2 shlx eax,edx,eax inside ggml_cpu_init in the shipped exe. The same instruction sits at the same offset in 0.0.8 and 0.0.10.

The ggml-cpu-ivybridge.dll that reporters found in OpenWhispr's resources/bin belongs to llama-server's llama.cpp bundle. whisper-server imports no ggml DLL.

Change

• Two more static, CPU-only whisper-server builds per Windows and Linux x64, at upstream ggml's own levels:
  • ivybridge: SSE4.2 + AVX + F16C. Covers Intel Ivy Bridge and AMD Piledriver/Steamroller/Jaguar, which includes every processor named in the reports.
  • sandybridge: SSE4.2 + AVX. Covers Intel Sandy Bridge and AMD Bulldozer.
• Four new release assets: whisper-server-{win32,linux}-x64-cpu-{ivybridge,sandybridge}.zip. The Windows zips carry the same MSVC runtime DLLs as the existing CPU zip.
• MSVC has no switch for F16C without AVX2, which is why upstream builds no MSVC ivybridge variant. The Windows ivybridge build defines __F16C__, and MSVC compiles ggml's F16C intrinsics under /arch:AVX. Only ggml/src/ggml-cpu reads that macro.
• A startup level check for the two new builds (examples/server/cpu-level-check.cpp, opt-in with -DOPENWHISPR_CPU_LEVEL_CHECK=ON):
  • Why: the first emulated run showed the ivybridge build starting and serving on a Sandy Bridge. ggml builds its fp16 tables with scalar code and first runs F16C inside the transcription kernels, so the build would only have crashed mid-transcription, after the app's startup fallback could see it.
  • What it does: before main() it compares what ggml's CPU code was built for (ggml_cpu_has_avx/f16c/fma/avx2/bmi2) with CPUID and XGETBV. On a mismatch it prints e.g. whisper-server: built for F16C, which this processor does not support and raises ud2, the same illegal-instruction fault the processor would.
  • Why only the new builds: Rosetta 2 hides AVX from CPUID yet runs AVX2 code (checked locally), and some hypervisors do the same. The existing builds must behave exactly as before.
• Each CPU job asserts the exact Adding CPU backend variant line of all three builds. The primary lines must stay exactly as today. The Windows job also checks the ivybridge cache for /D__F16C__ in both C and C++ flags.
• Nothing existing changes: assets, names and flags are untouched, and so are the GPU and macOS builds.

Why two levels, not one: without F16C, ggml converts every fp16 weight one element at a time through a lookup table, and Whisper's models are fp16. In local x86_64 runs of jfk.wav (Rosetta, loaded host, so ratios only), the sandybridge build was about 7× slower than ivybridge. ivybridge was close to the AVX2 build.

CI

• verify-cpu-levels (new, matrix win32/linux) runs .github/scripts/verify-cpu-levels.py. It uses Intel SDE -ivb/-snb on Windows and QEMU user mode -cpu IvyBridge/SandyBridge on Ubuntu 24.04; 22.04's QEMU 6.2 cannot emulate AVX. Per platform it checks:
  1. and 2. Each build starts the way OpenWhispr starts it (real ggml-tiny.bin, Silero VAD, the app's arguments), transcribes samples/jfk.wav via /inference, and its system_info reports exactly its level.
  2. The primary is stopped at startup by the emulator itself, on Ivy Bridge (SDE's chip check names the instruction) and, under QEMU, on a Piledriver-class AMD FX (AVX + F16C + FMA, no AVX2/BMI2). This proves the emulator enforces the level, and that the primary dies before serving, which the app's fallback relies on.
  3. Each level build is refused at startup by its level check, naming exactly what's missing: ivybridge on Sandy Bridge (F16C), and sandybridge on Westmere (AVX).
  4. Each build starts with llama.cpp's libraries beside it, as in OpenWhispr's resources/bin, because ggml_backend_load_all() loads them.
• Every download is pinned by sha256. Intel's own SDE mirror refuses non-browser clients, so SDE comes from the petarpetrovt/setup-sde GitHub mirror. No third-party action code runs, and its sha256 independently matches the one oven-sh/bun pins for the same Intel file.
• Every pull request to master now builds the two CPU jobs and runs verify-cpu-levels. Nothing proved PR build: bundle MSVC runtime DLLs into win32 zips #6 before it merged, for example.
  • A concurrency group cancels a PR's superseded run. Tag and dispatch runs never share a group.
  • Each result prints as its check finishes, and an error inside a check is a FAIL line, not a traceback. On PRs the macOS and GPU jobs and create-release are skipped.
• The workflow token is contents: read by default. Only create-release gets contents: write, and it now also waits for verify-cpu-levels.

Evidence

Configure lines on real MSVC 19.44 and GCC 11.4. The primary lines are identical to 0.0.10:

windows: build:             /arch:AVX2 GGML_AVX2;GGML_FMA;GGML_F16C;__BMI2__;GGML_BMI2
windows: build-ivybridge:   /arch:AVX GGML_AVX        (+ /D__F16C__ in C and C++ flags)
windows: build-sandybridge: /arch:AVX GGML_AVX
linux:   build:             -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2 GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2
linux:   build-ivybridge:   -msse4.2;-mf16c;-mavx GGML_SSE42;GGML_F16C;GGML_AVX
linux:   build-sandybridge: -msse4.2;-mavx GGML_SSE42;GGML_AVX

First run (36502940851, before the level check). 5 of 6 checks passed per platform. Check 4 failed on both: whisper-server-*-cpu-ivybridge on sandybridge started serving. That failure is why the level check exists.

Run 36504535683 (level check): all 12 checks pass.

Run 36509459145 (hardened script, with the AMD FX-class and no-AVX checks): all 15 checks pass.

== verify-cpu-levels (linux, ubuntu-24.04, qemu)
PASS: whisper-server-linux-x64-cpu-ivybridge on ivybridge: transcribed 'And so my fellow Americans ask not what your country can do for you, ask what you can do for your country.', reports ['AVX', 'F16C']
PASS: whisper-server-linux-x64-cpu-sandybridge on sandybridge: transcribed 'And so my fellow Americans ask not what your country can do for you, ask what you can do for your country.', reports ['AVX']
PASS: whisper-server-linux-x64-cpu on ivybridge: stopped at startup as expected (killed by an illegal instruction)
PASS: whisper-server-linux-x64-cpu-ivybridge on sandybridge: stopped at startup as expected (whisper-server: built for F16C, which this processor does not support)
PASS: whisper-server-linux-x64-cpu-sandybridge on westmere: stopped at startup as expected (whisper-server: built for AVX, which this processor does not support)
PASS: whisper-server-linux-x64-cpu on piledriver: stopped at startup as expected (killed by an illegal instruction)
PASS: whisper-server-linux-x64-cpu-ivybridge with llama.cpp libraries on ivybridge: started
PASS: whisper-server-linux-x64-cpu-sandybridge with llama.cpp libraries on sandybridge: started
== verify-cpu-levels (win32, windows-2022, sde)
PASS: whisper-server-win32-x64-cpu-ivybridge.exe on ivybridge: transcribed 'And so my fellow Americans ask not what your country can do for you, ask what you can do for your country.', reports ['AVX', 'F16C']
PASS: whisper-server-win32-x64-cpu-sandybridge.exe on sandybridge: transcribed 'And so my fellow Americans ask not what your country can do for you, ask what you can do for your country.', reports ['AVX']
PASS: whisper-server-win32-x64-cpu.exe on ivybridge: stopped at startup as expected (SDE-ERROR: Executed instruction not valid for specified chip (IVYBRIDGE): 0x7ff7d8f5072d: shlx eax, edx, eax)
PASS: whisper-server-win32-x64-cpu-ivybridge.exe on sandybridge: stopped at startup as expected (whisper-server: built for F16C, which this processor does not support)
PASS: whisper-server-win32-x64-cpu-sandybridge.exe on westmere: stopped at startup as expected (whisper-server: built for AVX, which this processor does not support)
PASS: whisper-server-win32-x64-cpu-ivybridge.exe with llama.cpp libraries on ivybridge: started
PASS: whisper-server-win32-x64-cpu-sandybridge.exe with llama.cpp libraries on sandybridge: started

Local (Rosetta) check of the level-check build:

• With AVX hidden, it exits by SIGILL, printing built for AVX / built for F16C.
• With ROSETTA_ADVERTISE_AVX=1, it runs normally.

Not verified here

• Real Sandy Bridge, Ivy Bridge or AMD FX hardware: every run-time proof is emulated.
• Speed on those processors.
• MSVC's handling of the ivybridge tree, SDE on GitHub's Windows runners, and QEMU 8.2's CPU models. All three are first seen in this PR's run.

Release

Merging this does not publish anything. 0.0.11 is a manual workflow_dispatch on master. verify-cpu-levels must pass before create-release runs.

🤖 Generated with Claude Code