Add the llama.cpp case study

[donald-heddesheimer]

cadence pointed at code it did not grow up with: llama.cpp's CUDA backend (4df29be), Qwen2.5-1.5B Q4_K_M, on the A4000. Four inserted scopes, three run configurations, llama-bench -n 128 -r 3 as the referee.

The finding that matters most

The published overhead number holds up outside the benchmark that produced it. docs/overhead.md says 3390 ns per CADENCE_KERNEL. With llama.cpp's CUDA graphs disabled so per-node scopes actually run, counting scopes against llama-bench's own throughput gives:

configuration	tok/s	ms/token	scopes/token	ns/scope
uninstrumented	228.20	4.382	0	—
per-op, first placement	212.94	4.696	97	3227
per-op, fusion covered	182.43	5.482	361	3044

Two measurements at scope counts differing by 3.7x, on somebody else's kernels, within 10% of the published figure and 6% of each other. This is the first evidence in the repo that the number means anything off its home turf.

It also puts a price on the README's own advice: wrapping every graph node costs 20.1% of throughput; wrapping the graph launch instead is free (248.27 vs 248.64 tok/s).

The capture guard fires on real third-party code

llama.cpp captures its decode graph. The report opens with WARNING 535 scope(s) skipped -- their stream was capturing into a CUDA graph, and the advice in that message — wrap the graph launch instead — is literally what the working configuration does. There's a quieter consequence too: after capture, ggml_cuda_compute_forward is never called again, so per-node rows report n between 88 and 176 across 367 decodes. The n column is the only thing that says so.

Also: cadence's GPU measurement independently reproduces llama.cpp's own throughput (3.93 ms cuda-graph device mean vs llama-bench's 4.03 ms/token), and decode turns out to be 99.8% GPU — 8.30 µs of host per 3.93 ms of graph.

I got the instrumentation wrong and the report said so

My first patch scoped ggml_cuda_compute_forward, the obvious line. It missed 97.7% of the matrix multiplies and every single RMS_NORM, because a fused group launches inside ggml_cuda_try_fuse and then continues past compute_forward. 35,909 scopes became 133,257 once the scope covered both. The tell was that the per-op device rows couldn't account for the graph-compute host span. That's in the write-up, not quietly fixed.

Three defects in cadence, reported not fixed

None show up on a loop with a handful of stages, which is the only shape the tests have ever covered.

1. WriteSummary states the key conclusion backwards. On the per-op run it prints 2.61ms is launch and synchronization for a workload that is 99.8% GPU-bound. It adds one mean per label, valid only if each label occurs once per iteration — MUL_MAT occurs 178 times per token. Weighting by occurrence gives 5.4 ms of device work, not the 205 µs it claims. Fix: weight by count / iterations, or withhold the line when a count exceeds the iteration count.
2. WriteWorstIterations is unbounded. ~250 spans print on one line — several thousand unreadable characters, three times over.
3. IterationSpanMs ranks by the wrong span when the host scope doesn't enclose the GPU work. It picked an iteration for its 18.9 µs host jitter while the genuinely slowest GPU iteration never appeared.

I'd like to fix all three as a follow-up PR rather than widen this one — say the word. (1) is the one I'd not want to leave: it is a confident, wrong, quotable sentence.

Scope

docs/case-study.md plus a README paragraph and a .gitignore negation. The llama.cpp patch stays out of this repo — llama.cpp is not a dependency of cadence and shouldn't become one — but all four insertions are quoted in the write-up.

Clocks aren't pinned, so the doc tells the reader to read ratios; every configuration was run against its own baseline in the same session.