same-source-three-drivers

One GPU kernel, written once in Slang, run three ways on the same Apple M4 Pro:

                 ┌─ slangc → MSL ───────────────────────────────→ native Metal
kernels/*.slang ─┤
                 └─ slangc → SPIR-V → Vulkan ─┬─ MoltenVK   (SPIRV-Cross → MSL) → Metal
                                              └─ KosmicKrisp (Mesa NIR → Metal 4) → Metal

The question: what does it cost to reach the GPU through each translation layer, and where exactly does that cost come from?

Headline finding. On an M4 Pro, a chain of small dependent compute dispatches costs ~1.2 µs per dispatch on native Metal and ~12 µs on MoltenVK's default configuration, about 10× more.

• The cause is how MoltenVK lowers vkCmdPipelineBarrier when Metal argument buffers are on, which is the default: it ends the Metal encoder and synchronises through an MTLFence (source).
• Turning argument buffers off (MVK_CONFIG_USE_METAL_ARGUMENT_BUFFERS=0) brings MoltenVK back to native speed.
• Rebuilding MoltenVK's exact fence pattern by hand in native Metal reproduces the cost: ~13 µs.
• KosmicKrisp, the new Mesa driver (Vulkan 1.4 conformant since Sept 2026), uses a Metal 4 barrier inside the encoder instead, and pays ~2.5 µs.

The penalty is a roughly fixed ~8–11 µs per barrier, so it matters for small kernels and fades for big ones: 5.3× slower at 16 KB per dispatch, 1.04× at 16 MB.

In a real app: running llama.cpp's Vulkan backend on MoltenVK with argument buffers off makes LLM token generation 11.9% faster (213 → 239 tok/s, Qwen2.5-0.5B). The direction holds, but a naive "barriers × 11 µs" model overpredicts the saving by ~5×. See Real-world check.

Quick start

Needs Apple Silicon, macOS 26+, the Xcode command-line tools, and the LunarG Vulkan SDK 1.4.363.0 or newer. The SDK ships KosmicKrisp, MoltenVK and slangc. No sudo is needed:

./vulkansdk-macOS-1.4.363.0.app/Contents/MacOS/vulkansdk-macOS-1.4.363.0 \
    --root ~/VulkanSDK/1.4.363.0 --accept-licenses --default-answer --confirm-command install copy_only=1

make spike                                 # one correctness pass on all three paths
python3 bench/run.py --rounds 15           # dispatch-overhead ladder (~2 min)
python3 bench/run.py --sweep-n --rounds 8  # work-size sweep (~20 s)

bench/run.py picks the Vulkan driver per process with VK_DRIVER_FILES, so one vk_bench binary runs on either driver.

What is measured

The kernel (kernels/axpy.slang) computes y += a·x. It is deliberately tiny, so launch and synchronisation cost dominate. Every dispatch reads the y the previous one wrote, so each one depends on the last and needs a barrier.

rung	what it times	paths
submit_each	record + submit + wait for every dispatch	all
batched	all D dispatches in one command buffer / encoder, with a barrier between each	all
reused	record once, then only submit + wait (Vulkan: re-submitted command buffer; Metal: MTLIndirectCommandBuffer). The analogue of CUDA Graphs	all
batched_nobarrier	ablation: barriers removed. Dispatches race, so results are not checked; timing only	Vulkan
batched_untracked	control: untracked buffers + memoryBarrierWithScope inside one encoder	Metal
encoder_per_dispatch	control: a fresh (tracked) encoder per dispatch	Metal
fence_per_dispatch	control: MoltenVK's lowering by hand (end encoder, updateFence, new encoder waitForFence)	Metal

Paths: metal, kosmickrisp, moltenvk, and moltenvk_no_argbuf (MoltenVK with MVK_CONFIG_USE_METAL_ARGUMENT_BUFFERS=0).

Method

• Correctness every rep. With x = 1, a = 1, after D dependent dispatches every y[i] must be exactly D. A dropped, reordered or racing dispatch breaks the count. 4,200 / 4,200 checked reps passed (the 900 ablation reps are unchecked by design).
• Noise control. Each (path, rung, D) cell runs in 15 separate rounds. Every round shuffles the order of all cells, so GPU clock ramps and background load hit every path alike. Each cell takes 1 untimed warmup and 5 timed reps. The machine was on AC power for the whole run (recorded per rep).
• Fit, don't divide. Each rung is timed at D = 10, 100, 1000 and 4000 and fitted as time = fixed + per_dispatch × D. This separates the one-off submit/wait cost from the cost each extra dispatch adds.

Results (M4 Pro, macOS 26.6.2, Vulkan SDK 1.4.363.0)

Full tables: results/2026-10-16-0105-dispatch-n4096.md and results/2026-10-16-0107-sweep-n.md.

Marginal cost per dependent dispatch (batched, n = 4096 floats, fitted over 300 reps per row):

path	per-dispatch µs
native Metal (tracked, one encoder)	1.19
native Metal, untracked + memoryBarrierWithScope	1.18
native Metal, new encoder per dispatch	1.32
MoltenVK, USE_METAL_ARGUMENT_BUFFERS=0	1.21
KosmicKrisp	2.45
MoltenVK, default	12.15
native Metal, MoltenVK's fence pattern by hand	13.08

reused shows the same ordering: Metal 1.19, MoltenVK-no-argbuf 1.28, KosmicKrisp 2.56, MoltenVK default 12.42.

Barriers removed (batched_nobarrier): MoltenVK default 1.28, MoltenVK-no-argbuf 1.22, KosmicKrisp 0.81 µs. Without barriers, MoltenVK's extra cost disappears.

Work-size sweep (batched, 200 dependent dispatches, median µs per dispatch):

floats per dispatch	native Metal	KosmicKrisp	MoltenVK default	MoltenVK no-argbuf	MoltenVK ÷ Metal
4,096 (16 KB)	1.77	8.45	9.44	1.74	5.34×
65,536 (256 KB)	4.80	12.59	10.14	4.77	2.11×
262,144 (1 MB)	15.73	20.67	20.70	16.23	1.32×
1,048,576 (4 MB)	52.48	61.08	59.30	52.59	1.13×
4,194,304 (16 MB)	211.01	220.72	219.07	210.96	1.04×

At D = 200 the one-off submit cost is spread over fewer dispatches than in the fitted table, which is why the small-n numbers are higher there.

Real-world check: llama.cpp decode

Does the barrier cost show up outside a microbenchmark? I ran llama.cpp's own llama-bench text-generation test (-p 0 -n 128 -r 3 -ngl 99, 6 shuffled rounds) with Qwen2.5-0.5B-Instruct Q4_K_M. This small model means short kernels, the regime where the sweep says barriers matter. Script: bench/llama_check.py. Full table: results/2026-10-16-0113-llama-decode.md.

path	median tok/s	range	ms / token
llama.cpp Metal backend	274.7	272–288	3.64
Vulkan backend on MoltenVK, argument buffers off	238.7	233–240	4.19
Vulkan backend on MoltenVK, default	213.3	212–215	4.69
Vulkan backend on KosmicKrisp	202.8	201–204	4.93

Barriers per token, measured. I counted Vulkan calls with the SDK's VK_LAYER_LUNARG_api_dump layer at 8 and 24 generated tokens and took the difference. llama.cpp issues about 246 vkCmdPipelineBarrier, 365 vkCmdDispatch and 10 vkQueueSubmit per token.

Prediction vs. measurement.

• Predicted: if every barrier cost the microbenchmark's extra 10.94 µs, argument buffers off should save 246 × 10.94 µs ≈ 2.69 ms per token.
• Measured: 0.50 ms per token.
• Verdict: the mechanism and direction are confirmed, but the simple per-barrier model overpredicts by ~5×. I haven't isolated why. Candidates:
  • real kernels overlap or hide part of the fence wait
  • many of llama.cpp's barriers come back to back, so they don't each start a fresh encoder
  • the argument-buffers-off path has costs of its own with llama.cpp's many bindings

A second surprise. KosmicKrisp has the cheapest barriers of the two Vulkan drivers, yet it's the slowest path in real decode. So in this workload something other than barriers dominates its cost. Shader code quality is the obvious suspect, but I haven't tested it.

Where the time goes

1. Ablation. Removing the barriers makes MoltenVK as fast as native Metal, so the cost lives in the barrier.
2. Configuration. Of MoltenVK's documented settings, only MVK_CONFIG_USE_METAL_ARGUMENT_BUFFERS=0 removes it (12.15 → 1.21 µs). The docs describe argument buffers as on by default and "in most cases" faster (Configuration Parameters). For dependent compute chains they are the opposite.
3. MoltenVK source, at the commit this SDK ships (db66022). In MVKCmdPipelineBarrier::encode, outside a render pass, when argument buffers and a residency set are in use, a barrier calls endCurrentMetalEncoding() and then setBarrier(). The next encoder then waits on an MTLFence (MVKCmdPipeline.mm#L160, MVKCommandBuffer.mm#L708).
4. Reproduction in native Metal. Ending the encoder costs little: a fresh tracked encoder per dispatch is 1.32 µs. The fence wait costs a lot: the same chain synchronised with updateFence/waitForFence is 13.08 µs, matching MoltenVK.
5. The contrast. KosmicKrisp's kk_CmdPipelineBarrier2 keeps the compute encoder open and issues one Metal 4 barrierAfterEncoderStages:beforeEncoderStages: (Mesa kk_cmd_buffer.c, comment: "We chain encoders, so an intra-encoder barrier is enough here").

Implied fix, untested in MoltenVK itself. For compute→compute barriers, keep the encoder open and use an in-encoder barrier: memoryBarrierWithScope: on Metal 3, or the Metal 4 barrier KosmicKrisp uses. The batched_untracked control shows this ordering is correct and costs 1.18 µs.

What this does not show

• One machine, one OS, one SDK. M4 Pro, macOS 26.6.2, MoltenVK 1.4.2, KosmicKrisp from SDK 1.4.363.0. Other Apple GPUs and versions may differ.
• No GPU-side timeline. All times are CPU wall-clock from submit to completion. I have not captured an Instruments Metal System Trace, so I can't say how much of the fence cost is GPU idle time and how much is driver work.
• I haven't patched MoltenVK. The fix above is inferred from the native-Metal controls, not built and measured inside MoltenVK.
• KosmicKrisp source. I read Mesa main (fetched 2026-10-16). The SDK's KosmicKrisp is built from 269a955c89, a commit not on upstream main, so its code may differ slightly.
• Unexplained. On the fence-based paths, per-dispatch cost rises from ~10 µs at D = 1000 to ~12–13 µs at D = 4000. It does so in native Metal with a single fence too, so it isn't MoltenVK's fence pool. I don't know the cause yet.
• The real-world model is incomplete. The llama.cpp check confirms the direction (+11.9% with argument buffers off), but the per-barrier model overpredicts the saving ~5×, and I haven't isolated why. I also haven't checked whether argument buffers off changes llama.cpp's output. Only speed was measured.
• Only one kernel shape. Real workloads mix barrier counts, kernel sizes and resource counts. The sweep shows that the penalty only matters when kernels are short.

Layout

kernels/axpy.slang     the one kernel source
host/vk_bench.cpp      Vulkan host (MoltenVK or KosmicKrisp via VK_DRIVER_FILES)
host/mtl_bench.mm      native Metal host + the Metal-only controls
host/bench_common.h    args, timing, exact-count check, JSON-lines output
bench/run.py           shuffled-round harness, linear fit, work-size sweep
results/               dated summaries (raw JSONL is gitignored)
docs/EXPLAINER.md      plain-language walkthrough of every concept and file