Skip to content

[EXL3] preserve mixed-prefill numerics with paired M8 FC2 - #2

Draft
yatesdr wants to merge 4 commits into
stack/pr110-base-20260801from
feature/pr110-paired-m8-prefill-20260801
Draft

[EXL3] preserve mixed-prefill numerics with paired M8 FC2#2
yatesdr wants to merge 4 commits into
stack/pr110-base-20260801from
feature/pr110-paired-m8-prefill-20260801

Conversation

@yatesdr

@yatesdr yatesdr commented Aug 1, 2026

Copy link
Copy Markdown
Owner

What

Preserves block-64 mixed-K3/K4 prefill route packing and bounded workspace while restoring deterministic block-8 FC2 arithmetic inside the same cooperative grid.

The kernel maps each FC2 compute block to its block-64 metadata parent, computes exact M8 subtiles, and reuses each decoded weight tile across a pair of independent M8 outputs. The mixed-Trellis ABI is bumped so stale cubins cannot be reused across the record/launch contract change.

Why

The block-64 mixed-Trellis prefill path changed FC2 reduction order. Captured-input checks showed FC1 and activation outputs were bit exact while FC2 alone diverged and was nondeterministic at the production tile geometry. That raised prescribed KLD from the 0.07776700789368204 reference to 0.07888687230031491.

The paired-M8 implementation restores the reference arithmetic without returning to serial homogeneous K3/K4 launches or unbounded workspace.

Validation

  • production-shape operator proof: final output bit exact, 0 changed elements;
  • paired-M8 operator time: 5,162.58 us versus 8,873.07 us for the naive exact M8 implementation (1.7187x);
  • focused mixed-kernel capture/mapping tests passed;
  • prescribed MTP0/DCP1 KLD: 0.07776700789368204 over 2,047 positions, exact reference parity;
  • cold 64k prefill: 1,737.11 server tok/s, cached_tokens=0;
  • MTP0/C1: 38.96 tok/s;
  • MTP3/C16: 234.05 aggregate tok/s, 16/16 active, zero errors;
  • MTP0/480k KV pool: 609,024 tokens;
  • cold 350k retrieval: exact 738216;
  • no stale-cubin, capture, collective, fallback, or restart failure.

Dependency

This draft is stacked on local-inference-lab#110. Its base branch is an immutable copy of that PR head so the review diff contains only the exact-FC2 implementation. Retarget to master after local-inference-lab#110 merges.

Companion vLLM planner: yatesdr/vllm-opt#1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant