Skip to content

Keep scaled queries in F32 for causal GQA - #4

Open
morluto wants to merge 1 commit into
antirez:mainfrom
morluto:agent/keep-scaled-query-f32
Open

Keep scaled queries in F32 for causal GQA#4
morluto wants to merge 1 commit into
antirez:mainfrom
morluto:agent/keep-scaled-query-f32

Conversation

@morluto

@morluto morluto commented Aug 11, 2026

Copy link
Copy Markdown

Fixes #3.

Why

The causal GQA Metal kernel expanded a BF16 query to F32, applied the attention scale, then rounded that product back to BF16 before performing the QK contraction in F32:

Before: BF16 Q -> F32 scale -> BF16 round -> F32 QK contraction
After:  BF16 Q -> F32 scale --------------> F32 QK contraction

Because shared_query is already a threadgroup float array, the intermediate round does not save storage. It only discards precision before the scores and softmax are computed.

At head_dim=128, an A/B test of the real kernel over 64 deterministic trials and 2,097,152 outputs produced:

Metric versus explicit F32 attention Before After Change
Maximum absolute error 0.0924071 0.00390625 95.77% lower
RMSE 0.00194536 0.000868307 55.37% lower
BF16 output mismatch rate 21.6468% 0.007486% 99.97% lower

The variants differed in 21.65% of output elements. Both were bit-for-bit deterministic.

What changed

  • Keep bf16_to_f32(query) * scale in F32 through the QK contraction.
  • Add a fixture-free regression test that exercises the production Metal kernel at head_dim=128 and compares it with an explicit F32 causal-GQA reference.
  • Run the new test from the standard make test target.

The test uses numerical tolerances rather than device-specific hashes. A mutation check confirmed that it passes the fixed code and fails when the old BF16 re-round is restored:

Eight-trial regression Maximum error RMSE BF16 mismatch rate
F32-scaled query 0.00390613 0.000864323 0.00839%
Old BF16 re-round 0.0924071 0.00197766 21.5992%

Performance

The real kernel was timed with 200 dispatches per sample, two series per variant, GPU command-buffer timestamps, and the first ramp-up sample in each series excluded.

Variant Median dispatch time
BF16 re-round 21.274 us
F32 retained 20.491 us

The timing distributions overlap; the relevant conclusion is that no performance regression was observed.

Validation

Validated on Apple M5, macOS 26.3, arm64, Metal 4:

make -j"$(sysctl -n hw.logicalcpu)" h3_gqa_tests h3_audio_gpu_tests h3_tests
./h3_gqa_tests
./h3_audio_gpu_tests
./h3_tests
git diff --check
  • h3_gqa_tests: passed
  • h3_audio_gpu_tests: passed
  • h3_tests: passed, 1,768 checks
  • No Metal compilation or runtime errors

Scope

This kernel is used by the Qwen3-VL prompt encoder. The DiT attention path uses MPSGraph SDPA and is unchanged.

This PR is scoped to the directly reproduced kernel-level precision loss and its regression coverage.

@morluto
morluto marked this pull request as ready for review August 11, 2026 09:39
AlexanderIstomin added a commit to AlexanderIstomin/h3.c that referenced this pull request Aug 15, 2026
All four come from pull requests open against antirez/h3.c that have not
been merged yet. Each was applied to this fork, built, and exercised
here rather than taken on trust.

Keep scaled queries in F32 for causal GQA (morluto, PR antirez#4). The kernel
expanded a BF16 query to F32, applied the attention scale, then rounded
the product back to BF16 before the QK contraction — into a threadgroup
float array, so the rounding saved no storage and only discarded
precision. Their regression test, included here, measures max absolute
error 0.0039 and a 0.0076% BF16 mismatch rate on this machine, against
0.092 and 21.6% for the old rounding. This changes the numerics of every
generation, which is why the test ships with it.

Resolve a relative shader path against the executable's directory
(guyz, PR antirez#14). Adapted rather than cherry-picked: upstream patches the
inline path in h3_gpu_create, while this fork resolves paths in
h3_gpu_shader_path, so the fallback lands there and covers the
precompiled metallib too. Without it the binary only runs from the
directory holding h3_shaders.metal.

Clamp H3_VAE_TILE_PIXELS to 320 (guyz, PR antirez#14). Tiles above 320 produce
a grid artifact across the frame. This fork's automatic search already
stopped at 320; only the environment override could exceed it.

Keep video VAE RGB output finite (morluto, PR antirez#9) and reject
unrepresentable aligned frame counts (morluto, PR antirez#11). Both close paths
where a bad value propagates instead of failing: NaN passes two ordered
clamp comparisons untouched, and a large frame count overflows while
being aligned.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Qwen causal GQA rounds scaled queries to BF16 before the QK contraction

1 participant