moe: add isolated mixed K3/K4 Trellis comparison path

[MadeBy561]

Draft comparison PR ahead of r14. This adds an opt-in mixed K3/K4 Trellis path for willfalco/GLM-5.2-EXL3-TR3-3.25bpw.

The existing uniform K3 path is untouched: no existing kernel function or class changed, and W4A16FusedMoeKernel is AST-identical to current master.

We have struggled to close the mixed-model gap. Our current TP4/DCP4 results are:

• MTP0 C1/0: 42.0 tok/s
• MTP3 C1/0: 101.1 tok/s
• MTP3 C1 at 16K/32K/64K/128K: 99.4/98.0/96.0/98.2 tok/s
• Normal interactive use is generally 80–90 tok/s and trails NF3

This is for pre-r14 comparison, not merge yet. We would appreciate a diff against the reported 48 MTP0 / 106 MTP3 mixed path.

Validation:

• fused-vs-two-pass CUDA oracle passed on K3-only, K4-only, and mixed routes
• deterministic replay and coherent generation passed
• py_compile, Ruff, and source-isolation checks passed

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 02800653-7fd8-4dad-bddf-20d0f5ef6a62

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[voipmonitor]

Superseded by #104 and the matching vLLM integration in local-inference-lab/vllm#190.

Thank you for publishing the comparison path and measurements. We reproduced the useful architectural ideas here: one global route pack, global expert descriptors, a single cooperative mixed grid, and exact-M specialization.

The final implementation differs in three material ways:

1. The default (64, 256, 64, 256) tile is not numerically safe for the production GLM shape at large M. At M=3072 it measured relative L2 error 8.32e-3; the replacement enforces fc1_tile_k >= 128 and measures approximately 4.1e-8.
2. Raw kernel A/B showed no speed advantage for this implementation after both paths used the safe tile: M1 was 58.22 vs 58.19 us, and M3072 was 6661.04 vs 6649.15 us in favor of the replacement, both effectively parity.
3. Exact-M specialization improved CC1 E2E by only 0.22%, below run-to-run noise, so the replacement keeps one bounded capacity plan instead of adding an M-indexed launch table.

#104 also adds production-shape GPU regression coverage, checkpoint validation tools, and the measured TP4/DCP4 results (48.37 tok/s MTP0 CC1 and 105.23 tok/s MTP3 CC1). Closing this draft so reviewers have one canonical implementation.