You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Successor to #1333, which is closed as a graveyard: of its five sections, §1/§2/§4.5 were already struck RESOLVED, §3's premise has since become false (the calib data shipped), and §5's first item was done in code but never checked off. This issue carries forward only what is still real, re-verified against main on 2026-08-23 and reframed — #1333's wording assumed a Python reference engine, which the #1357 ladder deleted.
Framing change that applies throughout: there is no second engine. "Both engines agree" is now "the compiled engine agrees with its frozen goldens" (parity_tests/goldens/, refreshed via pin_goldens.py). See .claude/rules/rust-core/parity.md Rule 1.
A. Parity-suite coverage gaps
Against aic-core/rust/aiconfigurator-core/parity_tests/test_engine_step_parity.py. Each item states what I verified.
Dense CP coverage is still zero. Verified: 4 CP cases exist (up from Follow-ups from the perf_interp v2 redesign (PR #1303): rust CP model, rust shared-layer, DSV4 CP calib data, minor gaps #1333's 3), all b200_sxm + sglang, all MoE — Qwen3-235B-A22B (cp8, cp4), Kimi-K2.5 (cp8), DeepSeek-V4-Flash (cp8, isl=8192). Add a dense long-context case (e.g. Llama-3.1-70B or Qwen3-32B, tp1/cp4, isl 16k–32k). Note cli_estimate has no cp_size param, so CP is mixed-step-only today; consider threading cp through the static/agg surfaces.
Sparse-CP numeric parity — now unblocked and triggered.Follow-ups from the perf_interp v2 redesign (PR #1303): rust CP model, rust shared-layer, DSV4 CP calib data, minor gaps #1333 asked for a symmetric-failure assertion that would "flip to a numeric-parity assertion the day the calib tables ship". They shipped: dsv4_csa_topk_calib_perf.parquet is present on h100_sxm, h200_sxm, b200_sxm, b300_sxm and gb200 under sparse_attention/sglang/0.5.14/. So this should go straight in as a numeric case, not a failure-symmetry one.
pp_size > 1 coverage is zero. Verified: no pp_size=2+ anywhere in the suite. Add one dense pp=2 case.
attention_dp is backend-skewed. Verified: sglang ×6, vllm ×1, trtllm ×0. Add a trtllm adp8 case.
Tolerance — partially addressed, needs a decision not a change. Verified: PARITY_RTOL = 0.01 is still the general tolerance, but HYBRID_PARITY_RTOL = 1e-4 already exists for the shared-table class with a comment explaining why. Decide whether the static surfaces should join the tighter tier rather than blanket-tightening.
Dropped from #1333 §5: the anti-vacuous guard, which is done — test_engine_step_parity.py:2025 ("Anti-vacuous guard: prove the golden comparison actually bites") plus a sibling in test_compile_engine_parity.py. It was never checked off.
B. DSV4 CP validation track
Different in kind from section A — this is data/calibration quality, not test coverage. Carried from #1333 §3; I did not re-verify these two against current data, so triage before starting.
All-gather calibration. The CP comm terms are formula-derived. Instrument a real sglang DSV4 CP timeline (cp_utils-style, as GLM-5 did) to pin the full-isl latent gather (kv_lora+rope per token, pre-cache-write), the kv_score gather, and the replicated per-rank compressor compute (currently unmodeled — consider an analytic term first, standalone bench later). Version-annotate the comm model and test with the sglang commit.
Unsaturated-anchor guard. step=0 boundary utils are still climbing (+25.6% median over-prediction if held). Add a util-slope threshold before holding, or borrow plateau util from larger-step slices. Land with its own LOO validation.
§3's "csa_topk_calib is collected nowhere" premise — false as of the sglang 0.5.14 drop.
§4's "caller-less wideep compute variants hold on the proxy until a caller appears" — dormant by design; it re-opens when a caller appears, not on a schedule.
§4's "pre-existing cargo test failures on main (parse_b200_sxm, 4× py::tests)" — not verified here (would need a full Rust build). If still failing, that deserves its own issue rather than a bullet in a coverage list.
Successor to #1333, which is closed as a graveyard: of its five sections, §1/§2/§4.5 were already struck RESOLVED, §3's premise has since become false (the calib data shipped), and §5's first item was done in code but never checked off. This issue carries forward only what is still real, re-verified against
mainon 2026-08-23 and reframed — #1333's wording assumed a Python reference engine, which the #1357 ladder deleted.Framing change that applies throughout: there is no second engine. "Both engines agree" is now "the compiled engine agrees with its frozen goldens" (
parity_tests/goldens/, refreshed viapin_goldens.py). See.claude/rules/rust-core/parity.mdRule 1.A. Parity-suite coverage gaps
Against
aic-core/rust/aiconfigurator-core/parity_tests/test_engine_step_parity.py. Each item states what I verified.b200_sxm+sglang, all MoE —Qwen3-235B-A22B(cp8, cp4),Kimi-K2.5(cp8),DeepSeek-V4-Flash(cp8, isl=8192). Add a dense long-context case (e.g. Llama-3.1-70B or Qwen3-32B, tp1/cp4, isl 16k–32k). Notecli_estimatehas nocp_sizeparam, so CP is mixed-step-only today; consider threading cp through the static/agg surfaces.dsv4_csa_topk_calib_perf.parquetis present on h100_sxm, h200_sxm, b200_sxm, b300_sxm and gb200 undersparse_attention/sglang/0.5.14/. So this should go straight in as a numeric case, not a failure-symmetry one.pp_size > 1coverage is zero. Verified: nopp_size=2+anywhere in the suite. Add one dense pp=2 case.attention_dpis backend-skewed. Verified: sglang ×6, vllm ×1, trtllm ×0. Add a trtllm adp8 case.PARITY_RTOL = 0.01is still the general tolerance, butHYBRID_PARITY_RTOL = 1e-4already exists for the shared-table class with a comment explaining why. Decide whether the static surfaces should join the tighter tier rather than blanket-tightening.Dropped from #1333 §5: the anti-vacuous guard, which is done —
test_engine_step_parity.py:2025("Anti-vacuous guard: prove the golden comparison actually bites") plus a sibling intest_compile_engine_parity.py. It was never checked off.B. DSV4 CP validation track
Different in kind from section A — this is data/calibration quality, not test coverage. Carried from #1333 §3; I did not re-verify these two against current data, so triage before starting.
Not carried forward
cargo testfailures on main (parse_b200_sxm, 4×py::tests)" — not verified here (would need a full Rust build). If still failing, that deserves its own issue rather than a bullet in a coverage list.