Leakance identifiability campaign: NO-GO — riverbed leakance is not identifiable from gauged discharge (selective equifinality)#25
Merged
Conversation
…+ detectability bound) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ates, eval-days lever, deterministic noise floor Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…etry cap Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ay0 attr on round files Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ing instead of round count Sort the combined (both-delta) probe list by gauge-footprint size descending before the greedy first-fit loop so large footprints are placed first (FFD). Deterministic tie-breaking via secondary key (comid, delta). Remove the stop-at-40 guard that was aborting before writing the CSV; the wall-clock gate now lives in the stage-2 timing check. Print round count prominently with probes-per-round distribution (min/median/max). Result: 64 rounds, 572 probes, 0 duplicate (round,comid) pairs, 0 gauge-sharing violations. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ge — analysis measures each probe only at staid_nearest Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… deltas is a hard round floor Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…detectability) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…GO (dilution, not transmission) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…s / experiment / pass-fail / next steps) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…/plant/3-student design) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…parallel students) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…erride in forward_eval Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ew follow-up) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…, NaN-padded from epoch) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…obe-binary pattern) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…view follow-up) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… zeta answer key Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…review follow-up) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…m leakance_hourly_on) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…s; document student_b's intentional head/routing split (review follow-up) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…udents (23/96 expressibility checkpoint, user decision) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…nale, summary warning Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…+ decision rule)
Produces the floor table (mean/median L1 by rho, warmup, and basin-size
stratum), the decay-shape plot (mean solid + median dashed per rho), and
the large-uparea day-probe {0,5,30,60,87} that shows big rivers don't
decay within 88 days. Pre-registered rule (spec §4): any (warmup ≤ 60,
rho ≤ 180) all-stratum mean ≤ 0.25 → OPTION A; else OPTION B REQUIRED.
Result: OPTION B REQUIRED — best config-only floor is 1.4204 at rho=178
warmup=150 (5.7× the bar). Median is ≤ 0.10 everywhere — the mean is
driven entirely by large-river gauges (uparea > 1578 km²) whose residuals
stay ≈ 3.7–4.8 m³/s at day 60 with no config-level cure.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…forward injection, floor validation) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ge cache + forward_eval_reaches split Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tateCache store Adds the optional `experiment.state_cache` config key (serde default None) and the focused `src/data/store/state_cache.rs` store module that lazily reads one day-boundary discharge row per `collate` call from the netCDF produced by `--mode state-cache`. When unset, every code path is byte-identical to prior behavior (no reads, no field populated, existing guard suites + sandbox ABSOLUTE MATCH verified). When set, `RoutingBatch::initial_state` and `RoutingTensors::initial_state` carry the window-start Q vector (N reaches) for Task 3 to consume. Key decisions: - Lazy row reads (~256 KB per collate call, one file-open) vs ~1.3 GB eager load — lazy preferred per plan - StateCache is a focused module per repo convention (no Box<dyn Store>) - Both collate and collate_window attach the row (same date-index logic) - Row-0 approximation documented in StateCache struct doc Guards: cargo test (all lib+integration), leakance_gradcheck, leakance_off_parity, zeta_accum, compare_ddr_sandbox ABSOLUTE MATCH. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ring) — replaced by spatial join Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ck unchanged) Add `initial_state: Option<Tensor<Autodiff<I>, 1>>` to `MuskingumCunge::setup_inputs`. When `Some(q0)`, seeds the router from the provided discharge instead of running the hotstart heuristic; when `None`, byte-identical to the pre-Task-3 code path (carry_state / hotstart logic unchanged). Threaded through `forward`, `forward_eval_core`, `forward_with_frozen_params`, and `probe_forward` via `tensors.initial_state`. All non-cache callers pass `None`. Guards: leakance_gradcheck / leakance_off_parity / zeta_accum all pass; compare_ddr_sandbox ABSOLUTE MATCH (max abs 1.5e-5 m³/s). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Path-gated integration test opens merit_channel_attributes_v1.nc via AttributesStore::open (the exact code path MeritGagesDataset::open uses) and asserts all seven Phase-A variables are present and resolve to finite values for real CONUS COMIDs. Guard trio + DDR sandbox all pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…uity run_state_cache created a fresh MuskingumCunge engine per chunk via forward_eval_reaches. With a fresh engine discharge_t = None, so carry_state = chunk_idx > 0 was a no-op (the condition !carry_state || self.discharge_t.is_none() was always true and the hotstart always ran). Each 15-day chunk restarted from an independent hotstart instead of continuing from the previous chunk's final state. This made the cache store hotstart-derived (not continuous) values for days 16+ — which are actually worse initial conditions than the natural per-window hotstart because they were equilibrated to a different forcing context. Result: cached floor (1.6598) > uncached (1.5446), both far from the continuous residual (0.00759). Fix: save the final discharge vector (last column of the runoff tensor) after each chunk and inject it as tensors.initial_state for the next chunk. The Some(q0_ext) branch in setup_inputs takes priority over carry_state/hotstart, producing truly continuous day-boundary states. carry_state is now always false (redundant with the injection). Guards: leakance_off_parity, zeta_accum, state_cache_forward, and compare_ddr_sandbox ABSOLUTE MATCH all pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…kance phase + joint fine-tune + equal-budget control Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
carry_state is a no-op through forward_eval: forward_eval_core constructs a fresh MuskingumCunge per call, so discharge_t is always None and the hotstart branch always runs regardless of the flag. run_teacher therefore generated the synthetic obs as a PIECEWISE trajectory restarting from hotstart every 15-day chunk. Empirical confirmation: on the three largest gauges (DA 38-41k km2), mean |dQ| at obs indices congruent to the chunk boundary (phase 7 mod 15 from the 1980-01-01 epoch) is 35-56 m3/s vs a 2-5 baseline; 19 of the top-20 day-to-day jumps land exactly on restart days. These discontinuities are the reason the state-cache floor validation could not collapse toward the continuous residual: a continuous 90-day window can never track an obs trajectory that restarts 5-6 times inside it. Fix: run_teacher now uses forward_eval_reaches (extended to accept zeta/overrides) so it can extract the per-reach final column and inject it as tensors.initial_state for the next chunk — same mechanism as the run_state_cache fix (f073d7d). Gauge aggregation moves to the call site via scatter_add_by_group. Cross-chunk boundary keeps the documented one-hourly-step approximation (the last q' row of each chunk is unconsumed), negligible at daily-mean resolution. Known remaining users of the dead carry_state flag (NOT fixed here, behavior unchanged): run_perturb and training/eval.rs::evaluate — both have always chunk-restarted every 15 days. Flagged for follow-up. Guards: leakance_gradcheck, leakance_off_parity, zeta_accum, state_cache_forward pass; compare_ddr_sandbox ABSOLUTE MATCH. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…-restarts every 15 days The evaluate() chunk loop passed carry_state = chunk_idx > 0 to forward_eval, but forward_eval_core constructs a fresh MuskingumCunge per call. With a fresh engine discharge_t = None, the !carry_state || self.discharge_t.is_none() branch always ran the hotstart regardless of the flag. Each 15-day chunk restarted from an independent hotstart derived from its own q'[0] — producing 35-56 m³/s jumps at every chunk boundary on large gauges (empirically measured in probe_zeta_gradient). Fix (mirrors the run_teacher pattern from b4ceeb3): - Switch the KanHead arm from forward_eval to forward_eval_reaches to obtain the per-reach (n_reaches, chunk_hours) output before scatter-aggregation. - After each chunk, extract the final per-reach discharge column as Vec<f32> and store it in prev_final_state. - At the start of the next chunk, inject it as tensors.initial_state before the forward call. The Some(q0_ext) branch in setup_inputs takes priority over carry_state/hotstart, producing truly continuous chunk-boundary states. - carry_state is now always false (redundant with the injection). - Fix the module doc-comment (it claimed carry_state=i>0 propagated state — that was always false and is now removed). FrozenParams decision: forward_with_frozen_params returns gauge-aggregated (G, T) with no per-reach output to extract. A forward_with_frozen_params_reaches variant would be invasive for a legacy verification-only path. Decision: leave FrozenParams cold-restarting; emit an eprintln! WARN on every non-first chunk. Note on eval output change: eval OUTPUTS CHANGE BY DESIGN after this fix — the continuity correction removes cold-restart seams. The sandbox example is single-window and is unaffected (ABSOLUTE MATCH confirmed). Tests: - tests/eval_chunk_continuity.rs::state_injection_gives_continuous_boundary: verifies col 0 of injected chunk2 == chunk1's final col (no jump); cold-restart col 0 differs (proves the test detects the defect). - tests/eval_chunk_continuity.rs::asymmetric_split_is_byte_identical_to_single_shot: demonstrates byte-identical concatenation with asymmetric overlap split (chunk1 has T_1+1 rows, chunk2 starts 1 row back); guards the initial_state injection mechanism produces exact routing continuity on the NdArray CPU backend. Guards: zeta_accum (6/6), leakance_gradcheck (8/8), leakance_off_parity (3/3), compare_ddr_sandbox ABSOLUTE MATCH (1.5e-5 m³/s < 1e-3 threshold). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…iority chain, 3km cap — as built) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…hes continuous routing Root cause of floor validation failing the 0.25 bar (quick10 scored 3.37): the prior agent's collate() lookbehind read one extra q_prime_daily row BEFORE window_start (lookbehind day as row 0), causing DisaggHead to use lookbehind day's mass as routing day 0's forcing — a one-day shift on every training and floor window. The detection condition in forward_eval_core was also off-by-one (d_daily > n_hourly/24+1 is never true when d_daily = rho_days+1), so the intended trim never fired. Fix: - Revert collate() q_prime_daily, precip, and state_cache to the pre-lookbehind reads (no lookbehind needed for the continuous floor path) - Remove dead read_precip_window_lookbehind helper - Revert forward_eval_core to simple disagg call with explanation comment - Keep teacher/state-cache changes in probe_zeta_gradient.rs: BATCH_SIZE_DAYS=365 and collate_window_with_daily_lookahead for non-final chunks (appends one extra row at the END — DisaggHead's d_use<d branch uses it as the true `next` for the last chunk day, eliminating right-clamp without needing any special handling in forward_eval_core) Add two lookahead unit tests to disagg_head.rs verifying: (a) last-day output changes when the lookahead differs from the clamp; (b) earlier days are byte-identical; (c) a clamp-equivalent lookahead is byte-identical to no-lookahead. Guards: 178 lib tests pass, all integration tests pass, compare_ddr_sandbox reports ABSOLUTE MATCH (max abs 1.5e-5 m³/s << 1e-3 threshold). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…merge; v2 stays global) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ID join, NaN-fill, single-path byte-identical) `data_sources.attributes` now accepts either a bare path (existing configs unchanged) or a YAML list of paths. A single-element list or bare string routes through the existing `AttributesStore::open` unchanged — byte-identical to pre-C0. Multiple paths use the new `open_multi` path which: - Probes each file to assign each `input_var_names` variable to exactly one store (hard error on ambiguity or missing variable) - Calls `open_aligned` per store (NaN-fill absent COMIDs, keeping all requested COMIDs in the index so `finalize_attrs` column alignment is exact) - Hstacks feature blocks in `attr_names` order - Loads normalization stats from each store's adjacent statistics JSON The common COMID basis is `conus.order` — the full CONUS topological order passed by `MeritGagesDataset::open`. With `open_aligned`, every batch COMID found by `finalize_attrs.index.position` hits the index; absent-store COMIDs have NaN attrs which are then filled via `row_means` (existing `fill_nans` path, unchanged). Also updates: `dump_parameters.rs` (two dump functions), `cli/plan.rs` and `cli/sources.rs` (fingerprinting: single-path keeps "attributes" key for lock-file compat; multi-path uses "attributes_0", "attributes_1", …), `baseline/cache.rs` (two direct DataSources constructions in tests), `tests/data_dataset.rs` and `tests/training_verification.rs` (attributes path-existence checks). Guards: all 185 lib tests pass; all integration tests pass; sandbox ABSOLUTE MATCH (max abs diff 1.5e-5 m³/s << 1e-3 threshold). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…hree-leg gate, expected outcomes
…quifinality by smearing (78% reaches active, 0.1% at plants), PROMOTE ruled out
…gated, default on] Change the leakance head term from `(depth - d_gw)` to `max(0, depth - d_gw)` so gaining reaches (depth ≤ d_gw) produce zeta ≡ 0 and receive zero gradients on all leakance params (K_D, d_gw, leakance_factor). Gated behind `params.leakance_losing_only` (default true for Phase C; false recovers the prior unclamped path byte-identically for the recovery answer key). Changes: - leakance.rs: add `losing_only: bool` to `zeta_forward` (relu gate on m) and `zeta_backward` (mask_fill of gzeta on depth > d_gw; subgradient at kink = 0) - config.rs: add `leakance_losing_only: bool` to `Params` (default true) and `ParamsRaw`, with parsing and 3 config tests - mmc_op.rs: add `losing_only: bool` to `LeakanceSaved`; thread flag from `cfg.params.leakance_losing_only` through `forward_chain_inner` and the eval-time diagnostic; pass to `zeta_backward` in `TimestepLeakanceOp::backward` - tests/leakance_gradcheck.rs: extend with 5 new test functions covering gaining chain (K_D/d_gw/factor grads = 0 analytical and FD), mixed chain full gradcheck (all 8 parents), and per-reach gate validation on K_D All guards green: leakance_gradcheck (14 tests), leakance_off_parity (3), zeta_accum (6); compare_ddr_sandbox ABSOLUTE MATCH (max rel 2.1e-7). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements Phase C Task 2: a static per-reach multiplier zeta ← zeta · mask where mask[i] = (corridor_impervious[i] ≤ threshold) ? 1.0 : 0.0. Changes: - config.rs: add `params.leakance_impervious_threshold: f32` (default 0.7) + `ParamsRaw` Option + parsing; 2 config unit tests - routing/leakance.rs: `zeta_forward` / `zeta_backward` each accept `mask: Option<Tensor<I, 1>>`; mask applied after losing_only gate in backward so gzeta is zero for impervious reaches - routing/mmc_op.rs: `LeakanceTensors.mask`, `LeakanceSaved.mask`, propagated through `forward_chain_inner` → `zeta_forward` and `TimestepLeakanceOp:: backward` → `zeta_backward`; `timestep_forward_leakance` gains `impervious_mask` param; eval-time zeta diagnostic multiplied by mask - routing/mmc.rs: `SpatialParameters.impervious_mask` (inner backend, not autodiff-tracked); `MuskingumCunge.impervious_mask`; stored at setup_inputs, forwarded to LeakanceTensors at route_timestep - All 17 SpatialParameters literal sites updated with `impervious_mask: None` - leakance_gradcheck.rs: +2 tests: `gradcheck_impervious_mask_reach2_zero` (mask[2]=0 → leakance grads at reach 2 are 0, others finite), `gradcheck_all_ones_mask_equals_no_mask` (byte-identical to None) - zeta_accum.rs: +2 tests: `zeta_accum_masked_reach_zero` (eval diagnostic zeroed at masked reach), `zeta_accum_all_ones_mask_same_as_no_mask` Back-compat: absent mask (None) → no tensor allocation, no multiplication, byte-identical to PC1. Guards: all 16 leakance_gradcheck + 8 zeta_accum + 3 leakance_off_parity pass; sandbox ABSOLUTE MATCH (max abs 1.5e-5 m³/s). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ask [absent=no-op] Populates SpatialParameters.impervious_mask from the corridor_impervious attribute column at dataset collation time, completing the PC2 wire-up. Changes: - data/dataset.rs: add leakance_impervious_threshold: Option<f32> to MeritGagesDataset (Some only when use_leakance=true AND corridor_impervious in attr_names; None = all-ones no-op back-compat); add impervious_mask to RoutingBatch, StaticNetworkCache, and RoutingTensors<B>; add build_mask_from_raw_col(raw, threshold) -> Vec<f32> (pub(crate), NaN→1.0) and build_impervious_mask(&self, divide_comids) -> Option<Vec<f32>>; wire into collate, build_static_network, collate_window_inner, and to_tensors; 5 new unit tests (4 pure-fn + 1 back-compat guard) - training/forward.rs: forward() extracts Option<Tensor<I,1>> via .inner() from the Autodiff tensors; forward_eval_core() passes the inner-backend mask directly; forward_with_frozen_params() stays None (no-leakance path) - tests/leakance_off_parity.rs: add impervious_mask: None to RoutingTensors struct literal (field required by new struct layout) NaN design choice: no StreamCat coverage (reach absent from attrs) → 1.0 (leakance allowed). Absence of imperviousness data ≠ impervious concrete. Back-compat: any config where corridor_impervious is not among attr_names (all existing configs) leaves impervious_mask = None throughout — byte- identical to the PC2 state. Guards: 16 leakance_gradcheck + 8 zeta_accum + 3 leakance_off_parity pass; 5 new dataset mask tests pass; sandbox ABSOLUTE MATCH (max abs 1.5e-5 m³/s). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… clamp, impervious hard-zero, K_D widened)
…ut field anti-correlates with groundwater (rho=-0.36); selective equifinality
…ed discharge (evidence chain, mechanism, paper framing)
…cibility artifacts)
taddyb
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Resolves the leakance identifiability question with a definitive NO-GO: riverbed GW–SW exchange (zeta) is not identifiable from gauged discharge and not promotable, established by removing every alternative explanation one experiment at a time. The learned field is active and marginally skill-improving in aggregate (+0.006 KGE on affected gauges) yet anti-correlated with independent groundwater data (field–WTD ρ = −0.36). This is the extreme, fully-controlled case of the paper's selective equifinality thesis.
Scientific summary:
docs/2026-07-06-leakance-nogo-scientific-summary.mdWhat's in this PR (the campaign)
docs/2026-07-03-...): gradient reaches every reach (starvation REFUTED); real losses undetectable at gauges (53× below the 5% band).docs/2026-07-04-...): planted leakance unrecoverable (R1 = 0.008) even on a clean objective — equifinality by smearing (78% of reaches active, 0.1% of flux at plants).merit_channel_attributes_v1.nc): bed-relative channel WTD, losing fraction, corridor imperviousness, alluvium, BFI, bankfull geometry. Caught + fixed the MERIT-SWORD bugfix1 COMID-remap bug.--mode state-cache), continuity fixes to teacher/cache/eval chunk loops (deadcarry_state), disagg lookahead. IC-noise floor 1.5 → 0.11 m³/s on losing gauges.data_sources.attributesas a COMID-joined list (single-path byte-identical).Invariants held throughout
compare_ddr_sandboxABSOLUTE MATCH (1.5e-5 m³/s) after every routing-core change.leakance_gradcheck16/16),leakance_off_parity3/3,zeta_accum8/8.Test Plan
cargo test --lib(185 passing; one C0 tmp-file test race under parallel runs — passes in isolation, tracked as follow-up)Note
Merge as a merge commit (not squash) — the findings docs cite specific commit SHAs.
🤖 Generated with Claude Code