test: Python fixture parity for Linux-canonical equiv suite (#215) - #219
Merged
Conversation
Spec for backfilling missing python/<mod>/tests/fixtures/<case>_params.py so the Linux-canonical equivalence suite covers the full committed baseline corpus, not just a hand-picked subset. Scope locked across 3 Codex review passes: - 4 new fixtures: fp_jt60, ti_w, tr_m0904, wrx_jt60 - Rename ti_iter01_params.py -> ti_min_params.py (5 importers) - Register wrx_jt60 in test_run/test_definitions.conf - eq_jt60 deferred to follow-up issue (eqjt60.gs never committed) Builds on the Linux-canonical equiv policy (PR #216, 2026-05-26 spec). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors test_run/inputs/fp_jt60.in (&fp namelist) into Python so the Linux-canonical equiv suite at test_run/baselines/fp_jt60/metrics.json gains a fp_jt60 wired test alongside fp_dt1 and fp_iter01. KNAMFP=' ' is no-op in the wrapper path and listed in UNREGISTERED_KEYS for documentation only. Spec: docs/superpowers/specs/2026-06-01-fixture-parity-backfill-design.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Code-quality review on commit 91cf233 found PROFN2/PROFT2/PMAX/MODELC were placed in SCALARS but are not bare-scalar-registered in fp/fp_param_registry.f90: - PROFN2 / PROFT2: not in fp_param_registry at all (in plcomm) - move to UNREGISTERED_KEYS. Precedent: fp_iter01_params.py:73. - PMAX: array-only (registry L114-116, idx >= 1). Move to ARRAYS as {1: 20.0} per fp_iter01_params.py:57 - bare `PMAX=20.D0` is Fortran namelist scalar that sets PMAX(1)=20 only, NOT broadcast. - MODELC: array-only (registry L149-151). Move to ARRAYS as {1: 4} per fp_iter01_params.py:61 - same scalar-broadcast Fortran rule. Without this fix Layer 1 equivalence at 1e-10 would fail silently (fp.set_param returning ierr but apply not checking) and physics would drift on PMAX[2..NSMAX] / MODELC[2..NSMAX] defaults. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors test_run/inputs/ti_w.in into Python so the Linux-canonical equiv suite at test_run/baselines/ti_w/metrics.json gains a wired test for the W-impurity (NSMAX=3, NRMAX=20) transport case. Uses the MATRIX_ARRAYS 2D-key pattern established by ti_ar_params.py for MODEL_BND(1,3) and BND_VALUE(1,3). Spec: docs/superpowers/specs/2026-06-01-fixture-parity-backfill-design.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors test_run/inputs/tr_m0904.in (MODELG=2 analytic geometry, NSMAX=4, NTMAX=50) into Python so the Linux-canonical equiv suite at test_run/baselines/tr_m0904/metrics.json gains a wired test. MDNCLS and PNBCD appear in the namelist but have no CASE entry in tr/tr_param_registry.f90 (pre-audit confirmed). They are listed in UNREGISTERED_KEYS for documentation only. KNOWN_ISSUE.md documents env-dependent drift on non-canonical builds; canonical Linux CI is the anchor environment. If drift recurs on the canonical runner a new issue will be filed before any xfail. Spec: docs/superpowers/specs/2026-06-01-fixture-parity-backfill-design.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors test_run/inputs/wrx_jt60.in (33-key &wr namelist, MODELG=2, NRAYMAX=2) into Python so the Linux-canonical equiv suite at test_run/baselines/wrx_jt60/metrics.json gains a wired test alongside wrx_iter01 and wrx_demo. PROFN1/PROFN2/PROFT1/PROFT2 are array-only in wrx_param_registry.f90 (L113-125, IF idx > SIZE pattern). They are placed in ARRAYS as [v, v] NSMAX=2 broadcast lists per wrx_iter01_params.py:45-48 precedent -- NOT in SCALARS where bare-name set_param would return ierr=1. Fortran replicated-initializer (RFIN(1)=2*110.D3) is expanded inline as list values. All ARRAYS entries are lists per wrxlib fixture convention; a defensive isinstance check in apply() catches future dict-form mistakes early (wrxlib's enumerate path iterates list values, not dict items). KNAMWR is an output filename used only by the Fortran driver and is documented in UNREGISTERED_KEYS but not applied via the wrapper. Spec: docs/superpowers/specs/2026-06-01-fixture-parity-backfill-design.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The fixture file was named ti_iter01_params.py but its docstring (L8) admits it actually mirrors ti_min.in, with BASELINE_NAME="ti_min" and NTMAX=2. The misleading filename propagated to five test importers: - test_equivalence.py (test_ti_min) - test_property_boundary.py - test_sweep.py (two imports) - test_property_fanout.py Rename closes the discrepancy with no behavioral change; the file's SCALARS / ARRAYS / apply contents are unchanged. The docstring is cleaned to drop the historical "named iter01 but mirrors ti_min" notice. Spec: docs/superpowers/specs/2026-06-01-fixture-parity-backfill-design.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The wrx_jt60 baseline at test_run/baselines/wrx_jt60/metrics.json existed without a corresponding entry in test_definitions.conf, so run_tests.sh never exercised it on the Fortran-driver side. Adding the one-line registration enables `./run_tests.sh wrx_jt60` for baseline regeneration workflows on Linux. Spec: docs/superpowers/specs/2026-06-01-fixture-parity-backfill-design.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Codex pre-push review (MED) flagged that the `assert isinstance(arr, list)` guard in wrx_jt60_params.apply() is stripped when Python runs under `-O`, which would silently restore the dict-form misapply bug the guard was added to prevent. Replace the assert with an explicit `raise TypeError(...)` so the check survives optimization flags. Behavior identical for list input; dict input now reliably raises TypeError instead of conditionally passing through. Spec: docs/superpowers/specs/2026-06-01-fixture-parity-backfill-design.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…215) Codex pre-push review (2026-06-02) flagged three spec-vs-code divergences in the fixture-parity design doc. The §5 tables described the original fixture layout; the as-shipped fixtures were corrected during implementation after auditing each module's *_param_registry.f90. Add a §0 Erratum summarizing the as-shipped content, with authoritative commit references (5270916 for fp_jt60, 92da504 for tr_m0904, a3fa6ad for wrx_jt60 confirmation). Original §5 tables preserved as historical record. The unifying pattern: any namelist key registered as array-only (IF idx > SIZE; X(idx)=value) in the registry MUST live in ARRAYS not SCALARS — bare-name set_param against an array-only registration parses to idx=0 and silently returns ierr. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
3 tasks
Owner
Author
|
Closing to re-trigger CI workflow (base was switched from master to develop after initial push, which did not re-fire pull_request workflow event). Reopening immediately. |
PR #219's canonical Linux CI run (27113686677, 2026-06-08) showed 3 failures at the 1e-10 equivalence layer. Per spec §6.3 drift policy (\"do NOT revert; file new issue; mark @xfail(strict=True)\"), apply xfail decorators with strict=True so: 1. CI is unblocked (3 known XFAILs do not fail the build) 2. When the root cause is fixed (registry extension, baseline regen, wrapper-path fix), the marker XPASSes → CI fails red → engineer removes the marker Failure summary + linked follow-up issues: - test_jt60 (fp): profile fields drift 0.5-51% — PROFN2/PROFT2 are plcomm scalars with no fp_param_registry CASE entry; wrapper can't reproduce the baseline. Tracked as #222. - test_ti_w (ti): TilibRunError ierr=3 (TI_ERR_CALC_FAILED) on get_state. Likely KID_NS/ADAS-table coverage gap in wrapper path for W-impurity (Z=74). Tracked as #223. - test_m0904 (tr): AJRFT missing + 309 mismatches. Pre-AJRFT baseline (#190 pattern) + env-dependent drift documented in KNOWN_ISSUE.md. Spec §6.3 explicitly anticipated this case. Tracked as #224. CI env: gfortran 13.3.0 (Ubuntu 13.3.0-6ubuntu2~24.04.1). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
#215) Canonical Linux CI on PR #219 (run 27113686677) revealed 3 of 4 new test methods fail at 1e-10. Per spec §6.3 explicit drift policy, the 3 are now @pytest.mark.xfail(strict=True) pending root-cause fixes tracked in #222 (fp_jt60 PROFN2/PROFT2 plcomm gap), #223 (ti_w W-impurity wrapper path), #224 (tr_m0904 pre-AJRFT baseline + env drift). Add §0.1 to record the as-shipped CI outcome (1 PASSED + 3 XFAIL'd + 1 rename PASSED) so the spec doc and the merged code agree. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Owner
Author
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 4862d91. Configure here.
This was referenced Jun 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Backfills four missing Python fixtures and renames a mislabelled fixture so the Linux-canonical equivalence suite covers every committed baseline that the wrapper can replay.
fp_jt60,ti_w,tr_m0904,wrx_jt60(one each inpython/{fp,ti,tr,wrx}lib/tests/fixtures/)test_jt60,test_ti_w,test_m0904,test_jt60in each module'stest_equivalence.pyti_iter01_params.py→ti_min_params.py(filename was misleading;BASELINE_NAMEwas alreadyti_min) — 5 importers + 1 docstring xref updatedwrx_jt60intest_run/test_definitions.conf(orphan baseline → driver-side wired)Builds on the Linux-canonical equiv policy from PR #216.
Canonical Linux CI outcome (2026-06-08, run 27113686677)
Per spec §6.3 drift policy, 3 of the 4 new test methods failed at 1e-10 and are marked
@pytest.mark.xfail(strict=True)with linked follow-up issues. The 4th new test PASSED. The rename PASSED.python/fplib/.../test_equivalence.py::test_jt60python/tilib/.../test_equivalence.py::test_ti_wpython/trlib/.../test_equivalence.py::test_m0904python/wrxlib/.../test_equivalence.py::test_jt60python/tilib/.../test_equivalence.py::test_ti_min(post-rename)strict=Trueensures the markers auto-fail (XPASS → CI red) once the root cause is fixed, forcing marker removal. Removal conditions are documented inline in each xfail'sreasonstring.How to verify on Linux CI
In the
python-tests.ymlpytest summary:test_jt60(wrx) andtest_ti_min(ti, post-rename) → PASSEDtest_jt60(fp),test_ti_w,test_m0904(tr) → XFAIL (not FAILED, not PASSED)Registry-correction note (§0 Erratum in spec)
Three fixtures required SCALARS-vs-ARRAYS corrections during implementation after auditing each module's
*_param_registry.f90. Keys registered as array-only MUST live inARRAYS, notSCALARS. Details indocs/superpowers/specs/2026-06-01-fixture-parity-backfill-design.md§0 + §0.1.Out of scope (follow-up)
eq_jt60fixture is blocked on regeneratingeqjt60.gs/eqdata.jt60(never committed). Filed as separate issue.except Exception:in ti_apply_arrayhelpers — PARTIAL via refactor(tilib): narrow ti_ar fixture except to TilibParamError (#218) #220 (ti_ar done); ti_w portion deferred to a follow-up commit after test: Python fixture parity for Linux-canonical equiv suite (#215) #219 merges.Spec
docs/superpowers/specs/2026-06-01-fixture-parity-backfill-design.md(committed at00e18956; erratum ate91d03f7; post-CI xfail §0.1 at4862d912).Test plan
git grep ti_iter01_params python/returns zero — verifiedwrx_jt60appears in./test_run/run_tests.sh --list— verifiedwrx_jt60) + 1 PASSED (ti_minpost-rename) + 3 XFAIL'd (fp_jt60,ti_w,tr_m0904)Closes #215
🤖 Generated with Claude Code
Note
Low Risk
Changes are limited to test fixtures, equivalence wiring, and docs; no Fortran or runtime API changes. CI behavior shifts only by adding tracked xfails and one new passing equiv case.
Overview
Backfills Linux-canonical Layer 1 equivalence coverage for four baselines that had no Python replay fixtures (
fp_jt60,ti_w,tr_m0904,wrx_jt60), plus a design spec with registry errata and post-CI notes.Each new case adds a
*_params.pynamelist mirror and atest_equivalencemethod.ti_iter01_params.pyis renamed toti_min_params.pywith imports updated in equivalence, sweep, and property tests.wrx_jt60is added totest_run/test_definitions.conffor the Fortran driver list.On canonical CI,
wrx_jt60and renamedtest_ti_minpass;fp_jt60,ti_w, andtr_m0904are@pytest.mark.xfail(strict=True)(#222–#224) for wrapper/registry gaps, W-impurity calc failure, and stale/drifting TR baseline. Fixtures follow registry rules (e.g. array-only keys inARRAYS,ti_w2DMODEL_BNDkeys,wrxlist-only arrays withTypeErrorguard).Reviewed by Cursor Bugbot for commit 4862d91. Bugbot is set up for automated code reviews on this repo. Configure here.