Skip to content

Repo - BUGFIX! - Make reruns self-contained and flag vanished harness outputs - #639

Merged
matt-pharr merged 3 commits into
developfrom
bugfix/review-robustness
Nov 2, 2026
Merged

matt-pharr merged 3 commits into
developfrom
bugfix/review-robustness

Conversation

@matt-pharr

@matt-pharr matt-pharr commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Release note

  • Audience: users
  • Numerical impact: none (harness @ 3ac5be2)
  • Migration: a gpec.h5 written before this change whose deck names a kinetic_file or a file wall can no longer be replayed or loaded with equilibrium_from_h5; re-run it from its deck.

A rerun from gpec.h5 is now self-contained: the run records its kinetic profiles and its wall file, and the replay uses those copies instead of re-reading the files the original deck named. Previously, an edited or deleted kinetic file silently changed a "frozen" replay. The regression harness now reports a tracked output that disappears, or changes storage type, as a change, so --fail-on-change catches it.

Changes

  • Rerun snapshot (ForceFreeStates output, Rerun.jl):
    • save_input_files! writes the equilibrium's kinetic profiles to Input/RawInputs/Kinetic, in the kinetic HDF5 schema, and a file wall's text to Input/RawInputs/Wall/wall_file_raw. The kinetic profiles are recorded even when they were attached through the API rather than by a deck kinetic_file.
    • A rerun writes them back out as <stem>_kinetic.h5 and <stem>_wall.dat and points the deck at them, as it already did for <stem>_forcing.h5 and <stem>_tolerances.toml.
    • equilibrium_from_h5 builds the kinetic profiles from the snapshot and applies the stored deck's composition.
    • Following the v2 clean-break rule, a stored deck that names either file without its snapshot stops with "re-run from the original deck" instead of falling back to the external file.
    • write_kinetic_h5 and _read_kinetic_h5 are split into group-level write_kinetic_group! and _read_kinetic_group, so the same schema code reads and writes the snapshot.
  • Rerun overwrite guard: the output and every side file are also refused when they are the source under another name (a symlink, a hardlink, or a case-only difference on a case-insensitive file system), using samefile. Before, only an identical path was caught.
  • Regression harness:
    • A quantity present on the earlier ref and missing on the later one is CHANGED (missing).
    • A change of storage type is CHANGED (type).
    • Both count toward --fail-on-change.
    • A quantity only the later ref has is NEW, which is counted in the summary but does not fail, so PRs that add outputs still pass.
    • Missing on both refs stays N/A.
    • docs/development/regression-harness.md now defines every status.
  • Vacuum: the 3D singular-quadrature cache is checked and rebuilt under a lock, and each caller gets the data built for its own parameters. Before, two concurrent 3D solves on different grids could be handed each other's quadrature. Nothing in GPEC runs two solves concurrently today.
  • Repo: the committed ci/conventions/__pycache__/check_subject.cpython-310.pyc is deleted, and __pycache__/ is added to .gitignore.

Tests

  • New tests cover each change: snapshot replay of the kinetic profiles and a file wall (and the missing-snapshot errors), the symlinked output and side-file guard, equilibrium_from_h5 reading the snapshot, a kinetic deck run recording Input/RawInputs/Kinetic, and the harness statuses (regression-harness/test/runtests_compare.jl).
  • The affected files all pass on SDCC: runtests_rerun_from_h5, runtests_frozen_equilibrium, runtests_error_fields, runtests_kinetic, runtests_vacuum, runtests_h5_schema, runtests_deck_roundtrip, and the harness test.

Regression report

The baseline is the branch's base, 9a25490c1, so develop commits merged after the branch was cut do not appear in the diff. Develop has since been merged in (8dcd34f); its only conflict was a docstring both sides edited, and this PR's own src/ changes are unchanged from the stamped commit. This run also uses the branch's own harness code, which applies the new statuses.

Run:  regress-fanout robust_3ac5be2e2 (SDCC sirius), julia 1.12.6, pinned manifest
Ref 1: 9a25490c1
Ref 2: bugfix/review-robustness @ 3ac5be2e2

diiid_error_field               10 unchanged
diiid_kinetic_multi_n            8 unchanged
diiid_multi_n                   19 unchanged
diiid_n1                        53 unchanged
diiid_n1_riccati                17 unchanged
diiid_n1_riccati_precompiled    17 unchanged
diiid_slayer_n1                 22 unchanged
diiid_slayer_n1_chi_perp_e      22 unchanged
diiid_slayer_n1_coupled          7 unchanged
diiid_slayer_n1_toroidal         5 unchanged
diiid_slayer_n1_winding         22 unchanged
diiid_truncation_n1             12 unchanged
efit_fixedbdy_separatrix         5 unchanged
gal_resistive_diiid             11 unchanged
gal_resistive_diiid_galinner    11 unchanged
gal_resistive_pe                 8 missing/N/A (N/A on both refs, as on develop)
ggj_ray_q500i                    4 unchanged
ggj_reference                    4 unchanged
lar_ggj_tearing_n1               3 unchanged
lar_reversed_shear_n1           22 unchanged
solovev_kinetic_nuzero          14 unchanged
solovev_multi_n                 15 unchanged
solovev_n1                      22 unchanged
tj_kinetic_calculated           20 unchanged
tj_kinetic_multiion              6 unchanged
tj_kinetic_ntv                   7 unchanged
tj_n1                           25 unchanged

Notes for reviewers

🤖 Generated with Claude Code

https://claude.ai/code/session_014wKUyAaxKdTJ5ch1boMFE1

…tputs, and remove the placeholder RLAR and CLAR torque

- Rerun: record the kinetic profiles (Input/RawInputs/Kinetic) and a file wall's text
  (Input/RawInputs/Wall/wall_file_raw) in gpec.h5; a rerun writes them back out as
  <stem>_kinetic.h5 and <stem>_wall.dat, and equilibrium_from_h5 reads the profiles from the
  snapshot, so a replay never re-reads the files the source run named. A stored deck naming either
  file without its snapshot stops with a re-run-from-the-deck error.
- Rerun: refuse an output or side file that is the source under another name (symlink, hardlink,
  case-only alias), not only the same path.
- Regression harness: a quantity that disappears is CHANGED (missing) and a storage-type change is
  CHANGED (type), and both fail --fail-on-change; a quantity only the newer ref has is NEW.
- KineticForces: remove the RLAR method (a linear-amplitude placeholder for the kappa integral)
  and CLAR (which returned zero); rlar_flag and clar_flag are no longer keys.
- Vacuum: build the 3D singular quadrature under a lock and return the data built for the caller's
  parameters.
- Repo: drop the committed __pycache__ bytecode and ignore __pycache__/.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@matt-pharr
matt-pharr requested a review from logan-nc November 2, 2026 11:46
@github-actions github-actions Bot added bugfix Something was wrong and now is not changed-results Results move or an interface breaks - read before upgrading labels Nov 2, 2026
@matt-pharr matt-pharr added this to the GPEC v2.0.0 milestone Nov 2, 2026
@logan-nc logan-nc changed the title Repo - BUGFIX! - Make reruns self-contained, flag vanished harness outputs, and remove the placeholder RLAR and CLAR torque Repo - BUGFIX! - Make reruns self-contained and flag vanished harness outputs Nov 2, 2026
@logan-nc

logan-nc commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator

Merged develop in (8dcd34f) so this merges cleanly. The only conflict was a KineticForces/Utils.jl docstring that both sides edited; I took develop's wording.

#625 had already removed RLAR and CLAR, so that part of this PR's diff is gone. I dropped it from the title and the release note so the changelog doesn't list it twice.

Harness stamp: the "Title and release note" check will now flag 3ac5be2e2 as stale, because the merge brings in develop's own src/ changes from #625 to #627, which were harnessed in their own PRs. This PR's own src/ patch against develop is unchanged from the stamped commit. That check is not a required merge check, and we're leaving it red on purpose instead of running the harness again before the release.

Also checked on the merged tree: the kinetic snapshot round trip now includes the lnLambda, nu_i and nu_e fields that #627 added, and every field comes back identical.

@logan-nc
logan-nc enabled auto-merge November 2, 2026 13:04
@logan-nc
logan-nc added this pull request to the merge queue Nov 2, 2026
@matt-pharr
matt-pharr removed this pull request from the merge queue due to a manual request Nov 2, 2026
@matt-pharr
matt-pharr merged commit 917aa40 into develop Nov 2, 2026
4 of 5 checks passed
@matt-pharr
matt-pharr deleted the bugfix/review-robustness branch November 2, 2026 13:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Something was wrong and now is not changed-results Results move or an interface breaks - read before upgrading

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants