Skip to content

Test - TEST - Add deck/API parity tests with the known drifts marked broken - #546

Merged
matt-pharr merged 3 commits into
developfrom
test/deck-api-parity
Oct 26, 2026
Merged

matt-pharr merged 3 commits into
developfrom
test/deck-api-parity

Conversation

@matt-pharr

@matt-pharr matt-pharr commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

Release note

  • Audience: developers
  • Numerical impact: none (harness @ 0cd8239)
  • Migration: none

The deck (main on a gpec.toml) and the scripting API are now checked against each other on the same inputs for Galerkin, Riccati Δ′, ascii/hdf5 file forcing, an analytic equilibrium on the two-pass auto grid, deck matching under gal_match_flag, and the fixed kinetic source. Where the two paths are known to disagree today, the test is marked @test_broken, so the fix for each drift shows up as an unexpected pass to flip.

Regression report

SDCC fan-out at 0cd823948 (vs develop @ 2842dfc2d) is running after the rebase onto develop 2842dfc2d;
the full report replaces this block when it lands.

Notes for reviewers

Tests only (test/runtests_solve_api.jl, +83/−2); no src changes. Rebased onto develop 2842dfc. Every new run uses the coarse Solovev fixture deck; the file's testset time goes from 39.5 s to 48.2 s (+8.7 s).

Parity that holds today (rtol 1e-12, exact in practice):

  • Galerkin (gal_nx=32): Δ′ matrix and raw from main equal solve(..., Galerkin(nx=32)).
  • Riccati: with the deck's own automatic chunking, Δ′ matrix, raw and coil columns equal the deck run. The existing nchunks=40 check at 1e-6 is kept.
  • Matching: a deck with gal_match_flag (galerkin inner model) equals solve(MatchProblem(gal), GGJModel(solver=:galerkin)) on a Galerkin(cut_solution=true) result, for bpen, the matched ξ and the composite inner profiles. Since develop's :ray model returns no inner profiles by default, the galerkin model is the one that shows the cut-solution drift.
  • File forcing: since the file-forcing normalization fix, the deck and a fresh RMPField agree on permeability and forcing_b for ascii and hdf5.

Known drifts marked @test_broken (2), and the change expected to flip each:

Drift Flipped by
Analytic equilibrium on the two-pass auto grid: solve cannot re-form it and stays on the 1025-knot pass-1 grid, while the deck re-forms on a refined grid storing the analytic config on PlasmaEquilibrium
Cut solution implied under gal_match_flag: the API without cut_solution=true has no composite inner profiles, where the deck has them the change that routes deck matching through a MatchProblem clause (#554)

Two drifts this PR first marked broken were fixed on develop before the rebase. The file-forcing normalization now holds, so the hdf5 check here is a plain @test and the ascii case uses develop's own test. The fixed kinetic source is now accepted by solve, and develop's deck/API parity test for it replaces the broken one this PR had.

The two-pass case runs with psi_accuracy=3e-2 and no local-stability scan. At the defaults the local-stability scan on the 1024-interval pass-1 grid made it about 34 s.

🤖 Generated with Claude Code

@logan-nc logan-nc left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read in full. Ready to approve from my read; nothing pushed.

One cross-PR hazard to settle at merge time: the two-pass test sets "local_stability_flag" => false as a [ForceFreeStates] override. #568 removes that key without a deprecation entry, so once both land the override is an unknown keyword. On #568's semantics the fixture deck carries a [LocalStability] section, so the cheap form is delete!(inputs, "LocalStability") in _stage_deck rather than the flag. Whichever of the two merges second takes that line. The fixed-kinetic @test_broken flips with #550, as the body says.

@logan-nc logan-nc left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving at 0ad81a2 on the comment review above. The one open item (the local_stability_flag override in the two-pass test once #568 lands) is a merge-order note, not a blocker.

matt-pharr and others added 3 commits October 26, 2026 09:46
…broken

Run main and the scripting API on the same Solovev deck for Galerkin, Riccati Δ′,
hdf5 file forcing, the analytic two-pass grid and gal_match_flag matching. Mark the
known drifts @test_broken: file-forcing normalization, the analytic two-pass grid,
the cut solution implied under gal_match_flag, and the fixed kinetic source.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ount

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The :ray model returns no inner profiles by default since #536, so the
cut-solution drift only shows with real-ψ profiles.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@matt-pharr
matt-pharr force-pushed the test/deck-api-parity branch from 0ad81a2 to 0cd8239 Compare October 26, 2026 08:48
@matt-pharr
matt-pharr merged commit 578376d into develop Oct 26, 2026
10 checks passed
@matt-pharr
matt-pharr deleted the test/deck-api-parity branch October 26, 2026 09:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test Tests only

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants