results_stale scoped to the base frontiers; scenario set flags on its own marker - #142
Merged
Conversation
… its own marker A scenario-only model update (only scenario_config changed) echoed status.results_stale: true while both base frontiers still matched the model — read as "re-solve the base run" when the only stale side was the scenario set. The evidence pointed at the flag's scope, not the echo mechanics: _results_stale folded the scenario set's composite stamp into one blanket boolean, and its consumers treat that boolean as "a BASE frontier predates the edit" — so a scenario-only edit ALSO made the next base solve drop a still-valid exact overlay (dropped_frontier_note) and made fill_gaps refuse a legitimate fill. models.py's design comment (#130) already stated the intent: "a scenarios-only edit must not read as staleness on the base run". Fix: _results_stale reads the base runs only (exploratory + exact overlay, each against the inputs it reads). The scenario set keeps its own #130 marker — models.scenario_results_stale — now echoed beside the base flag in model update's status (same present-only-when-it-fires convention as model get and the scenario reads) and in model load's status. The frontier-drop and fill_gaps guards become correct with no change of their own, since the flag now means what they assume. Two tests asserting the blanket semantics were re-pointed to the split (scenario staleness on its marker, base flag clear); the run_scenarios does-not-vouch test passes unchanged — a base frontier that predates an edit still flags. Tests: 2 new in tests/test_scenario_staleness.py (status echo split; a scenario-only edit costs no frontier on the next solve), each failing on the prior behavior — the drop repro failed with the exact overlay discarded. Full suite: 1049 passed, 2 skipped; ui 37 passed.
With NO base runs, the scoped flag hit the nothing-to-vouch-for branch and stayed true forever — a fresh, successful run_scenarios on a problem that deliberately has no base run echoed results_stale: true (with no scenario marker), misdirecting to "re-solve the base", and flipped model load's has_results gate to solve guidance on a solved scenario-only bundle. With no base run, the scenario-only workflow decides: a populated scenario set means solved (its freshness stays the scenario marker's question); a problem with no stored results anywhere still flags on a structural edit. Test: scenario-only run_scenarios lands at results_stale false / no scenario marker, never-solved stays flagged, and a later scenario edit flags the scenario marker with the base flag still clear — fails pre-fix. Full suite: 1050 passed, 2 skipped; ui 37 passed.
cafzal
force-pushed
the
claude/scenario-stale-echo
branch
from
August 4, 2026 02:21
e550a79 to
08a6aa0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Defect
A scenario-only
model update(onlyscenario_configchanged; objectives/options/scores/constraints untouched) echoedstatus.results_stale: true— read as "re-solve the base run" although both base frontiers still matched the model, and apparently contradicting the staleness comment near the update handler.Which way the evidence pointed
Both ways — and the flag's scope is the real defect. The echo was "correct" under the code's blanket semantics (
_results_stalefolded the scenario set's composite stamp — #130 — into one boolean spanning every stored frontier), and two tests asserted that blanket behavior. But:engine/models.py's Scenario-side staleness: the fingerprint treatment base runs already had #130 design comment already states the intent: "a scenarios-only edit must not read as staleness on the base run".dropped_frontier_note, certification lost), andfill_gapsrefused a legitimate fill.So the fix scopes the flag rather than re-wording the comment.
Fix
_results_stalereads the base frontiers only (exploratory run + exact overlay, each against the inputs it reads). A scenario-only edit leaves it false.models.scenario_results_stale) — now echoed beside the base flag inmodel update's status andmodel load's status, same present-only-when-it-fires convention asmodel getand the scenario reads. A scenario-only edit reportsresults_stale: false+scenario_results_stale: true: the echo says exactly which re-solve is owed.fill_gapsguards become correct with no change of their own — the flag now means what they assume.test_run_scenarios_does_not_vouch_for_a_stale_base_frontierpasses unchanged (a base frontier predating an edit still flags). Docs: update-handler comments, model docstring, architecture.mdupdaterow.Tests
2 new in
tests/test_scenario_staleness.py, written first and failing on the prior behavior:test_scenario_only_edit_echo_stales_the_scenario_set_not_the_base(the reported echo; also asserts a base-input edit still flips both)test_scenario_only_edit_does_not_cost_a_base_frontier_on_the_next_solve(failed on main with the exact overlay discarded)Full suite: 1049 passed, 2 skipped; ui 37 passed.