v0.14.0: results ownership boundary: slot-backed results, explicit persistence, typed fit output - #40
Open
InfinityMonkeyAtWork wants to merge 29 commits into
Open
v0.14.0: results ownership boundary: slot-backed results, explicit persistence, typed fit output#40InfinityMonkeyAtWork wants to merge 29 commits into
InfinityMonkeyAtWork wants to merge 29 commits into
Conversation
…TODO - llms.txt: fits are governed by Project.show_output / auto_export, not a save_img kwarg (passing save_img to a fit_* method raises TypeError); keep show_plot / save_img as the plot- and setup-helper convention. - TODO.md: correct the road-to-v1 rationale to "contract surfaces (1-4) before the website (5)" after the stability-policy item was dropped, and note that llms.txt / AGENTS.md must be reconciled against the API tiers.
Under `-n auto --dist worksteal`, tests writing fit-completion CSV/PNG side effects into the shared `tests_fits/` tree could race on colliding `File.name`/`model_name` output paths. Default the shared helper to auto_export=False so a test writes to disk only when export is its actual subject; those tests opt in with auto_export=True and redirect path_results to tmp_path. test_default_is_true now asserts the production default on a bare Project. Full default (parallel) suite green; tests_fits/ no longer created.
The workers>1 MCMC pool started spawn workers directly, so a `%run notebook.ipynb` session — where __main__.__file__ is the notebook JSON — would crash every worker as spawn tries to runpy the JSON. The SbS executor already guards this via sanitized_spawn_main; the MCMC pool did not. Relocate sanitized_spawn_main from utils/sbs.py to a neutral utils/spawn.py (fitlib importing sbs would be circular, and the helper is generic, not SbS-specific) and wrap the MCMC pool with it, matching SbS. Bump version to 0.13.1.
Populate PLAN.md with the six-phase design (decisions settled 2026-07-14: full auto-export layout break, FitResults-homed plot API with File sugar, slot-backed accessors). Tag both TODO items [ACTIVE] and record the Simulator.sigma_data staleness sibling case in the mutation-guard item.
Add SavedFitSlot.correl (varying-parameter correlation matrix, captured only when the optimizer produced covariance — None for covariance-less fits and project joint fits, slice 0 for SbS) and acceptance_fraction in the mcmc payload. Bump the archive schema to 3; the reader accepts 2 and 3 (older archives load the new fields as None), appends stay same-version only. Extract the matrix builder into ulmfit.correl_to_df and delegate File.get_correlations to it. Round-trip, v2-tolerance, and unknown-version tests; fit_archive_schema.md updated.
FitResults gains get_fit_results / get_correlations / get_conf_intervals / get_mcmc, reading the latest matching SavedFitSlot; File.get_* become thin delegates (with a new optional model= filter) and File._result_model is deleted. The accessors now serve sbs fits (slice-0 payloads) and loaded archives, and hand out copies. Behavior changes: covariance-less fits raise from get_correlations instead of returning an identity-with- zeros matrix, and fits on Files without data record no slot so accessors raise (data_base-only test fixtures now set file.data). MCMCResult.acceptance_fraction is None for schema-2 archives.
results_to_df and results_to_fit_2d are now pure conversions: the fit_pars.csv / fit_2d.csv writes, the per-parameter plotting, and the save_df/save_2d flag logic move into their sole caller, _save_sbs_fit_legacy (byte-identical output, pinned by the export-parity tests). That legacy block is slated for removal when auto-export routes through the slot exporter.
fit_slice_by_slice / File.fit_2d / Project.fit_2d now auto-export via export_fit(path_results, overwrite=True) — the grouped slot layout — and display inline from the captured SavedFitSlot (_display_fit_2d_maps / _display_sbs_fit), so the shown figure equals the exported one. Project.fit_2d displays per-file maps instead of reading saved PNGs back. Fit-time diagnostics keep writing under model_path(). Deletes save_sbs_fit / save_2d_fit and the _save_*_fit_legacy impls; the _append_*_slot helpers return the slot. Export-parity tests now pin auto-export == explicit export; changelog and repo-architecture doc updated for the breaking layout change.
Every slot gains fit_settings — stages, per-stage methods, try_ci, the SbS seeding recipe, and MC sampling settings when MCMC ran (worker counts deliberately excluded: serial == parallel dispatch is a tested invariant). SbS slots additionally gain params_meta (slice-invariant [name, vary, min, max, expr], captured from slice 0) and params_stderr (per-slice standard errors, previously discarded; feeds the planned 1D trace-fitting weights). _display_sbs_fit reads vary flags from the slot now. All additive within the unreleased schema 3; v2 archives still load with the new fields as None. Schema doc, changelog, round-trip and capture tests updated.
FitResults now accepts fingerprint-matched axes providers (live Files from Project.results, SavedFiles from FitResults.load) and gains plot_fit (observed/fit/residual, all fit types) and plot_param_evolution (SbS, varied-only default via params_meta), with File.plot_* sugar and config resolution config= > live plot_config > defaults. plot_residuals uses real axes now (index fallback kept). The fit methods' inline display routes through this API — the _display_* helpers are gone, so the figure shown at fit time is the one the API reproduces from a slot or a loaded archive. fitlib is imported lazily to avoid a package-init cycle.
Rewrite the fit_results.py section of repo_architecture.md around the ownership contract; add docs/api/fit_results.rst (FitResults + MCMCResult) and replace the deleted save_sbs_fit / save_2d_fit autodoc entries with a results/plotting/persistence section, making the docs build warning-free again. llms.txt gains plot_fit and a note that the get_*/plot_* accessors read persisted records (live sessions and loaded archives alike). TODO.md: both [ACTIVE] items are done — replaced by a slim typed-result-object follow-up; the v1.0.0 legacy-shim item reflects the removed savers. Version 0.13.1 -> 0.14.0 (breaking auto-export layout + archive schema 3).
Move the durable decisions (ownership contract, auto-export layout break, slot-backed accessor semantics, schema-3 additions, and the rejected alternatives) into docs/design/archive/results-ownership-and-plotting.md; reset PLAN.md to its empty template per the completion protocol.
np.asarray is a no-copy passthrough for ndarray input, so the returned acceptance_fraction aliased the slot's stored array, violating the accessors-return-copies contract (review finding). Regression test pins copy semantics for acceptance_fraction, flatchain, and the ci table.
Populate PLAN.md with Phases 7-8, continuing the results-ownership branch (decisions settled 2026-07-17 in review discussion): remove Project.auto_export entirely — fits compute, display, and capture slots but never write; persistence stays the explicit save_fits/export_fits pair fed from the slot history. Everything auto-export wrote maps to persisted slot data or new on-demand plot_mcmc / plot_sbs_slices methods. Unify on the ./fit_results/<name>/ output root (path_results and model_path removed as dead), default Project.name becomes my_project, and plot save_path=None means display-only. Phase 8 replaces the internal raw result 5-list with a typed class. TODO: tag the unified-results item [ACTIVE]; add a new item for persisting project-level joint fits as a first-class record (shared-parameter map, joint covariance, sibling identity), noting the joint covariance currently evaporates.
- delete Project.auto_export / Project.path_results / File.model_path; project.yaml files still setting the removed keys fail loudly - fit methods and fitlib.fit_wrapper drop all disk IO (save_output / save_path / num_fmt / delim, the CSV/TXT dump block, SbS per-slice writes, post-fit export calls); show_output display gating unchanged - add FitResults.plot_mcmc (walker acceptance + corner from the persisted payload, live sessions and archives) and File.plot_sbs_slices (per-slice panels from live results_sbs) - Project.name default "test" -> "my_project" - fix explicit dict SbS seeds crashing slot capture: fit_settings now records the normalized seed template - reshape test_auto_export.py into test_fit_side_effects.py (fits write nothing, verified via cwd isolation); export parity re-anchored on two explicit exports; docs, examples, and changelog updated
…er, Model.result docstring
Component decomposition for 1D fits (baseline/spectrum/SbS) was live-only — describe_model(detail=1) and plot_sbs_slices could show it, but FitResults.plot_fit never could, live or archived. No researcher trusts a 1D fit whose components they can't see, so persist them directly instead of accepting that gap permanently. - add components/component_names to SavedFitSlot; bump schema 3 -> 4, extend SUPPORTED_READ_VERSIONS; write/read follow the existing additive-optional-field pattern (correl/mcmc). 2d untouched — no per-component concept there. - baseline/spectrum/sbs slot construction evaluates components on the full grid then crops to e_lim, mirroring the fit curve's existing evaluate-then-crop correctness (grid-dependent components don't evaluate the same on a pre-cropped grid). component_names comes directly from model.components, not parsed from params - a static par_profile attachment splices a nested model's params into its host component's name block without adding a components entry, which would break prefix-based re-derivation. - FitResults._plot_fit_1d renders the decomposition when present, falls back to the lean sum-only view for pre-schema-4 archives. - extend the round-trip assertion (F1/F6/F8 families, including the F6 static-profile edge case) with a components round-trip check and a sum-of-components-equals-fit invariant; add schema-3 backward-compat coverage and two focused _plot_fit_1d rendering tests. - update docs/design/fit_archive_schema.md (the stated wire-format contract) with the schema-4 version history entry, the components / component_names dataset section, and the reader-mapping / cheat-sheet table updates; verified with a sphinx -W build.
The fit archive already stores the full, uncropped data/energy/time once per file (SavedFile), independent of any slot's cropped observed/fit. aux_axis (File.aux_axis, the auxiliary physical axis used by par_profile-attached models, e.g. depth) was the one array in that same family never persisted - reloading an archive with no live File loses it entirely. Add it now, before the archive's full-data story is otherwise considered complete. - add optional aux_axis field to SavedFile; bump schema 4 -> 5, extend SUPPORTED_READ_VERSIONS; write/read follow the conditional-write / .get()-fallback pattern already used for correl/mcmc/components (not data/energy/time's unconditional write - most files have no aux axis at all). - thread live.aux_axis through the SavedFile construction used by save_fits. - extend the F1/F6/F8 baseline round-trip test with an aux_axis check (present + array-equal for F6/F8, None for F1, via the loaded FitResults axes-provider); add a schema-4 downgrade helper and a compat test confirming pre-5 archives still load with aux_axis=None. - update docs/design/fit_archive_schema.md (the stated wire-format contract) with the schema-5 version history entry and the aux_axis file-group layout entry; verified with a sphinx -W build.
describe_model showed the full data range with fit-limit marker lines;
post-fit displays (plot_fit) showed only the fit-limits-cropped window.
The archive already persists the full, uncropped data/energy/time once
per file (SavedFile) independent of any slot's cropped observed/fit, so
give plot_fit a full_range mode instead of forcing one view down to
match the other: real full data, with fit/residual/components drawn
only inside the fit window (NaN outside it, never a fabricated padding
value) plus dashed boundary lines - the same visual language
describe_model already uses.
- FitResults._axes_for/_full_observed_for/_pad_axis reconstruct the real
full-range view per fit type: baseline (mean over persisted
base_t_ind), spectrum (resolves the raw time_point/time_range/
time_type selection against the persisted time axis), sbs/2d
(provider.data directly - already full). Falls back to today's cropped
view when reconstruction isn't possible; never raises.
- Extracted File._resolve_time_selection into a standalone
utils.arrays.resolve_time_selection so spectrum's reconstruction can
resolve indices with no live File.
- fitlib.plt_fit_res_2d's title/scale calc now uses nanmin/nanmax for
the NaN-padded fit array; identical result when no NaN is present.
- fit_baseline/fit_spectrum now route their post-fit display through
self.plot_fit(...) instead of a bespoke plt_fit_res_1d call, dropping
the show_init overlay (not a completed-fit artifact, can't be
reconstructed from the archive) along with the dead initial_guess
extraction and bespoke title construction.
- full_range is a real PlotConfig member (Project.full_range-backed,
True out of the box), not a bool hardcoded at each call site -
data_slice/x_lim/y_lim already establish "what to show" as a
legitimate PlotConfig concern, not just style. plot_fit's full_range
parameter is bool | None = None ("use config"), resolved once via
cfg.full_range. All 6 live post-fit display call sites
(fit_baseline/fit_spectrum/fit_slice_by_slice/File.fit_2d/
Project.fit_baselines/Project.fit_2d) call plot_fit with no full_range
kwarg at all, so a project.yaml override or a per-call override
actually takes effect everywhere instead of being silently ignored
(project.yaml keys are only applied if a matching Project attribute
already exists) or shadowed by a hardcoded True.
- Add TestFullRangeConfigResolution: config default is True, a live
fit's post-fit display honors config.full_range in both directions,
and a per-call override wins over config - verified each failure mode
actually fails without its fix via temporary-revert round-trips.
- No schema/wire-format change - reuses already-persisted fields.
While discussing whether the initial parameter guess should count as fit-result provenance (a different seed can land the same algorithm on a different local minimum), found that SavedFitSlot.params' init_value column (baseline/spectrum/2d) was silently wrong for stages=2 fits. lmfit's Minimizer.prepare_fit() unconditionally resets Parameter.init_value = Parameter.value at the start of every stage. fit_wrapper's stage 2 starts from stage 1's *output* Parameters object, so a two-stage result's init_value ends up reflecting stage 1's output, not the true original seed - verified directly against the installed lmfit source. stages=1 fits were already correct. - Add utils.lmfit.restore_true_init_values(result_params, par_ini). Fixed at the source, in fitlib.fit_wrapper itself, right after stage 2's mini.minimize() and before the result is printed or returned: _result_params() returns result.params by reference (no copy), so this is the same object that becomes FitOutput.par_fin.params, which means the fix reaches lmfit.report_fit's own printed "(init = ...)" annotation and any direct consumer of FitOutput.par_fin - not just code that happens to pass through slot construction. An earlier pass applied the correction downstream in _append_baseline_slot/ _append_spectrum_slot/_append_2d_slot instead; moved here after review because the live printed report would otherwise still show stage 1's output, contradicting the archive. - Regression tests confirm a stages=2 fit's persisted init_value matches the true seed rather than stage 1's output, and - more directly - that both FitOutput.par_fin.params[name].init_value and report_fit's printed "(init = ...)" (captured via capsys, matching lmfit's own .7g format) show the true seed for the local-optimization stage. Verified each fails without its fix via temporary-revert round-trips. - docs/design/fit_archive_schema.md: clarified init_value's semantics and pointed at fitlib.fit_wrapper as the fix location. No schema/ wire-format change - same column, same shape, corrected values, no version bump. - Out of scope (separately noted in PLAN.md): SbS initial-guess persistence and the seed-as-provenance schema extension for baseline/spectrum/2d.
The initial guess is fit-result provenance, not incidental — a different seed can land the same optimizer on a different local minimum. SavedFitSlot now persists fit_ini (model evaluated at the true pre-fit seed, all 4 fit types) and, for SbS, per-slice params_init (every slice, not just slice 0 — the seed was already available at no extra cost). FitResults.plot_fit renders the archive-side "initial guess" overlay via new PlotConfig.show_init (Project-backed, default True, resolved the same way as full_range).
The energy-resolved (1D) branch already said "...: initial guess" in its title; the Dynamics and 2D branches didn't, so a live model preview could be mistaken for a saved fit result. Hoist the title string once and thread it through all three; plt_fit_res_2d gains an optional figure-level title= (suptitle), additive and unused by existing callers.
model_info is None on the (most common) default active-model path and an int on index-based selection, so the hoisted title_mod literally rendered `Model: "None"` or `Model: "0"`. mod is already guaranteed non-None at that point; mod.name is the real resolved name regardless of how the caller identified the model.
- schema-5 bullet wrongly described aux_axis as "the full energy or time array" and claimed it enables full_range; aux_axis is a separate auxiliary physical axis (e.g. depth) for par_profile models, and full_range doesn't reference it at all -- the full data/energy/time were already unconditionally persisted before this schema bump (confirmed against commit 5c97fb6 and a grep for aux_axis in fit_results.py). - the "fits never write to disk" bullet's accepted-losses note predates schema 6 and called per-slice par_ini "re-derivable from fit_settings"; it's now directly persisted (params_init/fit_ini), just not exported to the legacy CSV tree (confirmed: export_fits writes no such CSV today).
… does _plot_fit_1d has used _slot_title (file, model, fit type, yaml stem) since the explicit plotting API landed; the 2D/SbS branch returns early into plt_fit_res_2d with no title at all, so multi-file loops (e.g. Project.fit_2d()) show visually identical, unlabeled panels. plt_fit_res_2d already gained an optional title= kwarg for the describe_model fix; reuse it here with the same _slot_title used by the 1D path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Completes the results-data ownership contract and plotting disentanglement
(execution record:
docs/design/archive/results-ownership-and-plotting.md):SavedFitSlotis the authoritative record of every completed fit,captured eagerly at fit time;
FitResultsis the single read/query/plotsurface over slots — live sessions (
Project.results) and loaded archives(
FitResults.load) behave identically.File.get_*/File.plot_*are thinsugar.
show_output), and record slots; persistence is only ever the explicitsave_fits(HDF5) /export_fits(CSV/PNG tree) pair, both slot-fed, withone default output root
./fit_results/<name>/.MCMC acceptance fraction, SbS per-slice stderr + parameter metadata, and
fit_settingsoptimizer/seeding provenance on every slot.FitOutput(frozen dataclass:par_ini/par_fin/conf_ci/emcee_fin/emcee_ci) replaces the raw 5-element result list internally.Breaking changes
Project.auto_export,Project.path_results,File.model_pathremoved;a
project.yamlstill setting removed keys fails loudly with migration hints.Everything auto-export wrote is reproducible:
plot_mcmc(walker/corner fromthe persisted payload),
plot_sbs_slices(per-slice panels, live sessions),save_baseline_fit/save_spectrum_fit(kept, explicit-only).Project.namedefaults to"my_project"(was"test").File.get_correlationsraises for covariance-less fits instead of returningan identity matrix.
Model.resultisFitOutput | None(no list indexing);fitlib.plt_fit_res_1d(par_init=)→par_ini;save_sbs_fit/save_2d_fitremoved (useexport_fit(s)).Added
plot_fit,plot_param_evolution,plot_residuals,plot_mcmc,plot_sbs_slices. Inline fit-time display routes through the same API, sothe figure shown at fit time is the one reproduced later.
get_fit_results/get_correlations/get_conf_intervals/get_mcmc) onFitResults, working on SbS fits andloaded archives.