[PROBE: mass] Detect multi-decadal storage drift under repeated forcing

CONTRIBUTORS.md

@@ -21,6 +21,7 @@ long.
 | `mass/causality` | Zhi Li (CU Boulder) |
 | `mass/dry-down` | Zhi Li (CU Boulder) |
 | `mass/steady-state` | Zhi Li (CU Boulder) |
+| `mass/multi-decadal-drift` | Bing Li ([@hiter-joe](https://github.com/hiter-joe)) |
 | `mass/extreme-rain` | Zhi Li (CU Boulder) |
 | `mass/runoff-bounds` | Zhi Li (CU Boulder) |
 | `mass/area-invariance` | Zhi Li (CU Boulder) |

README.md

@@ -68,16 +68,15 @@ not.
 
 ## The probes
 
-Fourteen: twelve under mass, one each under energy and momentum. Each was
-merged only after the acceptance gate saw it pass four physical models, a
-bucket that conserves water exactly, two hand-written FLEX models and the
-NWS's SAC-SMA with Snow-17, and fail a purpose-built broken one on the
-named criterion. A probe that fails a
-physical model is examined before the model is; that is the first thing done
-with any probe pull request. Eleven of the fourteen can be scored on a model
+Sixteen: thirteen under mass, two under energy and one under momentum. The
+acceptance gate separates each probe's applicable physical baselines from
+purpose-built broken models on named criteria. The mass probes must pass an
+exact bucket, two hand-written FLEX models and the NWS's SAC-SMA with Snow-17.
+A probe that fails a physical model is examined before the model is; that is
+the first thing done with any probe pull request. Eleven of the sixteen can be scored on a model
 that reports runoff and nothing else. `ht list` prints them;
-[ROADMAP.md](ROADMAP.md#probes-we-want) has the seventeen more we want, all
-unclaimed.
+[ROADMAP.md](ROADMAP.md#probes-we-want) lists further probes; check the issues
+for current claims before proposing one.
 
 | Probe | Law | What it asks | The broken model it catches |
 | --- | --- | --- | --- |
@@ -87,6 +86,7 @@ unclaimed.
 | [`mass/causality`](probes/mass/causality) | mass | One storm added mid-record: nothing may change before it, and runoff must answer after it. | `reference_anticipating` |
 | [`mass/dry-down`](probes/mass/dry-down) | mass | Two years without rain: runoff can only fall, and no more may drain than the catchment held. | `reference_climatology` |
 | [`mass/steady-state`](probes/mass/steady-state) | mass | Three years of the same day: does everything settle, and does the budget balance once it has? | `reference_restless` |
+| [`mass/multi-decadal-drift`](probes/mass/multi-decadal-drift) | mass | Fifty years of repeated warm weather: does budget-compensating storage drift exceed capacity? | `reference_slow_drift` |
 | [`mass/extreme-rain`](probes/mass/extreme-rain) | mass | The largest storm scaled up to ten times: runoff may not fall, nor exceed the rain that was added. | `reference_saturating` |
 | [`mass/runoff-bounds`](probes/mass/runoff-bounds) | mass | Over ten years, is the runoff possible at all: at least rain minus demand minus storage, at most rain plus storage? The mass question a runoff-only model has to answer. | `reference_degenerate`, `reference_overflowing` |
 | [`mass/area-invariance`](probes/mass/area-invariance) | mass | The same weather on the same catchment told as ten times larger: every depth must be identical. | `reference_area_leak` |
@@ -115,12 +115,12 @@ goes untested.
 
 | Model | Kind | What it does | Standing |
 | --- | --- | --- | --- |
-| [`google_flood_forecast`](models/google_flood_forecast) | submitted | The mean-embedding forecast LSTM behind Google Flood Hub, at the published weights. Predicts discharge and nothing else. | **FAIL (INCOMPLETE)**, 5 of 14 probes passed. Runoff-only, so three budget probes cannot ask it anything; of the seven that ask on runoff alone it passes the runoff bounds, memory and area, and fails step, extreme rain, phase, and a 0.18 mm/day dip after an added storm |
-| [`dhbv2`](models/dhbv2) | submitted | δHBV 2.0, the MHPI group's differentiable HBV: neural networks write the parameters of a bucket model that reports its stores and its evaporation. | **FAIL (VIOLATION)**, 10 of 14 probes passed. Its learned regional-groundwater term, declared as `gwex`, closes the budget to 1e-8; what remains is a learned field capacity twice the catchment's, a response to doubled rain above the rain added, a runoff depth that changes with the area it is told, and a third more runoff when snow falls as rain |
+| [`google_flood_forecast`](models/google_flood_forecast) | submitted | The mean-embedding forecast LSTM behind Google Flood Hub, at the published weights. Predicts discharge and nothing else. | **FAIL (INCOMPLETE)**, 5 of 16 probes passed. Runoff-only, so the budget probes cannot ask it anything; of the seven that ask on runoff alone it passes the runoff bounds, memory and area, and fails step, extreme rain, phase, and a 0.18 mm/day dip after an added storm |
+| [`dhbv2`](models/dhbv2) | submitted | δHBV 2.0, the MHPI group's differentiable HBV: neural networks write the parameters of a bucket model that reports its stores and its evaporation. | **FAIL (ERROR)**, 10 of 16 probes passed; the new drift evaluation could not run because Docker was unavailable. Earlier evaluated probes show VIOLATION. Its learned regional-groundwater term, declared as `gwex`, closes the budget to 1e-8; what remains is a learned field capacity twice the catchment's, a response to doubled rain above the rain added, a runoff depth that changes with the area it is told, and a third more runoff when snow falls as rain |
 | `reference_bucket` | exact | conserves water exactly by construction | must pass every probe that can ask it anything; INCOMPLETE on the energy probe, which needs fluxes it does not report |
-| [`flex_lumped`](models/flex_lumped) | physical | lumped FLEX/HBV: interception, beta-partitioned unsaturated store, fast and slow reservoirs, triangular lag | must pass every probe that can ask it anything; **PASS**, 14 of 15, INCOMPLETE on the energy probe |
-| [`flex_topo`](models/flex_topo) | physical | FLEX-Topo: plateau, hillslope and wetland units on real Wark fractions sharing one groundwater store | must pass every probe that can ask it anything; **PASS**, 14 of 15, INCOMPLETE on the energy probe |
-| [`sacsma_snow17`](models/sacsma_snow17) | physical | the NWS's SAC-SMA with Snow-17 and a gamma unit hydrograph, ported from the legacy Fortran and checked against it | must pass every probe that can ask it anything; **PASS**, 14 of 15, INCOMPLETE on the energy probe |
+| [`flex_lumped`](models/flex_lumped) | physical | lumped FLEX/HBV: interception, beta-partitioned unsaturated store, fast and slow reservoirs, triangular lag | must pass every probe that can ask it anything; **PASS**, 15 of 16, INCOMPLETE on the energy probe |
+| [`flex_topo`](models/flex_topo) | physical | FLEX-Topo: plateau, hillslope and wetland units on real Wark fractions sharing one groundwater store | must pass every probe that can ask it anything; **PASS**, 15 of 16, INCOMPLETE on the energy probe |
+| [`sacsma_snow17`](models/sacsma_snow17) | physical | the NWS's SAC-SMA with Snow-17 and a gamma unit hydrograph, ported from the legacy Fortran and checked against it | must pass every probe that can ask it anything; **PASS**, 15 of 16, INCOMPLETE on the energy probe |
 | `reference_coupled` | exact | the bucket with snow sublimation and a surface energy budget: every kilogram converted at the latent heat of the phase it actually underwent | must pass every criterion of `energy/latent-heat-et-consistency` |
 | `reference_two_head` | broken | a water head and an energy head that never meet: both budgets close to 1e-15 and the latent heat implies an evaporation it never reported | caught by `flux_identity` |
 | `reference_constant_lambda` | broken | coherent, but converts every kilogram at the same latent heat of vaporisation | caught by `flux_identity` |
@@ -134,6 +134,7 @@ goes untested.
 | `reference_climatology` | broken | emits the seasonal mean whatever falls, and keeps flowing without rain | caught by `dry_down` |
 | `reference_saturating` | broken | caps its daily runoff, so an extreme storm adds rain and no runoff | caught by `monotone_response` |
 | `reference_restless` | broken | a recession on an internal thirty-day clock, so it never settles | caught by `steady_state` |
+| `reference_slow_drift` | broken | a fixed runoff reporting deficit hidden in accumulating soil storage | caught by `state_bounds` over fifty years |
 | `reference_overflowing` | broken | reports its runoff plus 80% of the rain again, from nowhere | caught by `runoff_bounds` |
 | `reference_area_leak` | broken | loses a share of runoff that grows with the area it is told | caught by `invariance` (area) |
 | `reference_overshooting` | broken | a derivative term sharpens its hydrograph, so an added storm lowers later flow | caught by `response_nonnegativity` |

ROADMAP.md

@@ -95,12 +95,11 @@ Snowfall minus melt minus sublimation minus the change in SWE.
 *Discriminates:* models that quietly lose water at the rain-snow transition,
 a very common bug that a whole-catchment budget can absorb.
 
-### `mass/multi-decadal-drift` · starter · **unclaimed**
-Fifty years with no trend in the forcing. Total storage must not drift
-secularly.
-*Discriminates:* a leak too small to trip a ten-year 5 percent threshold but
-large enough to be unphysical over a climate-relevant record. Good first
-probe: the criterion already exists, the case is the contribution.
+### `mass/multi-decadal-drift` · **merged**
+Fifty years of repeated warm weather expose a runoff reporting deficit hidden
+in accumulating soil storage: the budget closes, but storage exceeds capacity.
+Checks physical bounds rather than rejecting every bounded trend.
+Contributed by Bing Li (@hiter-joe).
 
 ### `mass/routing-network-closure` · standard · **unclaimed**
 A branching network. Mass must close reach by reach, not only basin-wide.

models/reference_slow_drift/ht_adapter.py

@@ -0,0 +1,86 @@
+#!/usr/bin/env python3
+"""Exact bucket physics with a deliberately drifting storage reporter.
+
+The bucket equations match reference_bucket. Keeping this adapter self-contained
+lets the subprocess sandbox run it without importing files outside its model.
+"""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import json
+from pathlib import Path
+
+BIAS_MM_PER_DAY = 0.012
+COLUMNS = ["time", "pr", "evspsbl", "mrro", "mrso", "snw", "canopy", "channel"]
+MODEL = {"name": "reference_slow_drift", "version": "1.0.0"}
+
+
+def simulate(forcing, static):
+    soil_cap = static["soil_capacity_mm"]
+    canopy_cap = static["canopy_capacity_mm"]
+    soil, swe, canopy = 0.5 * soil_cap, 0.0, 0.0
+    rows = []
+    for index, step in enumerate(forcing):
+        pr, tas, pet = step["pr"], step["tas"], step["pet"]
+        snowfall = pr if tas < static["snow_threshold_degC"] else 0.0
+        rain = pr - snowfall
+        swe += snowfall
+        melt = min(swe, static["degree_day_factor_mm_per_C_day"]
+                   * max(tas - static["snow_threshold_degC"], 0.0))
+        swe -= melt
+        intercepted = min(canopy_cap - canopy, rain + melt)
+        canopy += intercepted
+        throughfall = rain + melt - intercepted
+        canopy_evap = min(canopy, pet)
+        canopy -= canopy_evap
+        pet_left = pet - canopy_evap
+        soil += throughfall
+        surface = max(0.0, soil - soil_cap)
+        soil -= surface
+        baseflow = static["baseflow_coefficient"] * soil
+        soil -= baseflow
+        soil_evap = min(soil, pet_left * min(1.0, soil / (0.5 * soil_cap)))
+        soil -= soil_evap
+        # The error begins at input day one, including spinup, without feedback.
+        rows.append({
+            "time": step["time"], "pr": pr,
+            "evspsbl": canopy_evap + soil_evap,
+            "mrro": surface + baseflow - BIAS_MM_PER_DAY,
+            "mrso": soil + BIAS_MM_PER_DAY * (index + 1),
+            "snw": swe, "canopy": canopy, "channel": 0.0,
+        })
+    return rows
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--request", required=True)
+    args = parser.parse_args()
+    request_path = Path(args.request).resolve()
+    request = json.loads(request_path.read_text())
+    if request.get("timestep", "PT1D") != "PT1D":
+        raise SystemExit("reference_slow_drift supports PT1D only")
+    io_dir = request_path.parent
+    with open(io_dir / request["input"]["forcing"], newline="") as fh:
+        forcing = list(csv.DictReader(fh))
+    for row in forcing:
+        for key in ("pr", "tas", "pet"):
+            row[key] = float(row[key])
+    static = json.loads((io_dir / request["input"]["static"]).read_text())
+    rows = simulate(forcing, static)
+    output = io_dir / request["output"]["table"]
+    output.parent.mkdir(parents=True, exist_ok=True)
+    with open(output, "w", newline="") as fh:
+        writer = csv.DictWriter(fh, fieldnames=COLUMNS)
+        writer.writeheader()
+        writer.writerows(rows)
+    (io_dir / request["output"]["run"]).write_text(
+        json.dumps({"status": "ok", "model": MODEL, "n_steps": len(rows)})
+    )
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

models/reference_slow_drift/model.yaml

@@ -0,0 +1,18 @@
+name: reference_slow_drift
+version: "1.0.0"
+description: >
+  An exact bucket whose reporter subtracts 0.012 mm/day from runoff and
+  accumulates the deficit in soil storage. Closure survives; long-run
+  storage bounds do not. A deliberately unphysical negative control.
+authors: ["Bing Li"]
+license: PolyForm-Noncommercial-1.0.0
+runner: subprocess
+entrypoint: ["python3", "ht_adapter.py"]
+timestep: PT1D
+emits:
+  fluxes: [pr, evspsbl, mrro]
+  states: [mrso, snw, canopy, channel]
+needs_forcing: [pr, tas, pet]
+supports:
+  perturbation: true
+resources: {cpu: 1, memory_gb: 1, gpu: false}

models/result.csv

@@ -181,3 +181,8 @@ run_date,model,version,suite_version,runner,probe,verdict,reason,window,seeds,de
 2026-09-09,sacsma_snow17,1.0.0,0.1.0,subprocess,energy/latent-heat-et-consistency,FAIL,INCOMPLETE,not run,,"does not report hfls, hfss, hfg"
 2026-09-09,dhbv2,0.5.4-hbv2ep100.3,0.1.0,docker,energy/latent-heat-et-consistency,FAIL,INCOMPLETE,not run,,"does not report hfls, hfss, hfg"
 2026-09-09,google_flood_forecast,0.1.0-828dfc5,0.1.0,docker,energy/latent-heat-et-consistency,FAIL,INCOMPLETE,not run,,"does not report pr, evspsbl, hfls, hfss, hfg, mrso, snw, canopy"
+2026-09-09,flex_lumped,1.0.0,0.1.0,subprocess,mass/multi-decadal-drift,PASS,OK,full record,1349990263 1856942377 216410844 723362958 1230315072,residual 0.000% of driver
+2026-09-09,flex_topo,1.0.0,0.1.0,subprocess,mass/multi-decadal-drift,PASS,OK,full record,1349990263 1856942377 216410844 723362958 1230315072,residual 0.000% of driver
+2026-09-09,dhbv2,0.5.4-hbv2ep100.3,0.1.0,docker,mass/multi-decadal-drift,FAIL,ERROR,full record,1349990263 1856942377 216410844 723362958 1230315072,"the docker CLI is installed but the daemon is not reachable. Start Docker, or use --runner subprocess for an in-repo model."
+2026-09-09,sacsma_snow17,1.0.0,0.1.0,subprocess,mass/multi-decadal-drift,PASS,OK,full record,1349990263 1856942377 216410844 723362958 1230315072,residual 0.000% of driver
+2026-09-09,google_flood_forecast,0.1.0-828dfc5,0.1.0,docker,mass/multi-decadal-drift,FAIL,INCOMPLETE,not run,,"does not report pr, evspsbl, mrso, snw, canopy"

probes/mass/multi-decadal-drift/README.md

@@ -0,0 +1,99 @@
+# Multi-decadal storage drift
+
+A budget can close exactly while a model hides a persistent runoff reporting
+error in invented storage. This probe asks whether that storage stays physical
+over fifty years, not whether a fitted storage trend is exactly zero.
+
+## Case
+
+A seed generates a five-year block of warm daily weather. Rain arrives every
+third day with a seasonal envelope and seeded amplitude; temperature remains
+between 10 and 20 degrees C, and PET between 0.7 and 1.3 mm/day. Repeat that
+block for five years of spinup and fifty scored years (20,075 input rows).
+There is no imposed secular trend in the block totals. The case is periodic,
+not realistic stochastic weather or a statistically stationary process.
+
+Model years contain 365 days. Gregorian timestamps advance continuously;
+seasonality follows the forcing index, not Gregorian day-of-year. The warm
+case avoids snow accumulation and isolates soil/canopy capacity. Each seed
+generates different rain, reproducibly, without a committed forcing dataset.
+
+The minimum evaluation window is 18,250 days, preventing the default submitted
+model flood-event window from discarding the long horizon. Model states run
+continuously through spinup; the reporter's bias is not reset at scoring time.
+
+## Criteria and units
+
+At the daily step, the budget residual is
+
+`r[t] = P[t] + gwex[t] - ET[t] - Q[t] - (S[t] - S[t-1])` in mm.
+
+`S` includes every reported water store, including groundwater and routing
+storage where present. `gwex` includes declared external exchange; the supplied
+precipitation remains the denominator. Closure requires
+`abs(sum(r)) / sum(P) <= 0.05`. Rain is positive in aggregate, so no denominator
+floor is needed. The initial storage is the last spinup state, not the first
+scored state.
+
+The discriminating criterion is `state_bounds`: soil stays in [0, 320] mm,
+canopy in [0, 2] mm, and the remaining reported stores stay nonnegative.
+The existing numerical slack is 1e-6 mm. Extra groundwater is not folded into
+the soil capacity. FLEX uses case capacities (area-weighted for FLEX-Topo);
+SAC-SMA scales its upper/tension-zone capacities and reports lower free water
+as groundwater. No universal finite groundwater capacity is imposed here.
+
+The existing ET plausibility, non-degeneracy, and forcing-fidelity criteria
+guard against trivial compensating partitions and altered drivers.
+
+## Discrimination
+
+`reference_slow_drift` runs the exact bucket internally, but reports
+`Q' = Q - epsilon` and `soil' = soil + epsilon * elapsed_days`, with
+`epsilon = 0.012 mm/day`. The deficit and storage increment cancel in closure.
+The bias is independent of seed, probe identity, and requested run length.
+It does not feed back into the bucket's internal state.
+
+Over fifty scored years, the added storage grows by 219 mm, in addition to
+21.9 mm accumulated during spinup. Over a ten-year scored prefix it grows by
+only 43.8 mm. Tests require the short prefix to stay within bounds and the
+long run to fail specifically on `state_bounds`, while both close their
+budgets and the negative control's runoff remains nonnegative in this case.
+
+The gate requires reference_bucket, flex_lumped, flex_topo, and sacsma_snow17
+to pass. Merely extending the precipitation-normalized closure check does
+not amplify a constant fractional leak; eventual storage-bound violations
+are the added scientific target.
+
+The final forcing strengthens the seasonal rain and PET cycles relative to
+the issue's preliminary experiment, so correct models satisfy the existing
+anti-degeneracy checks without lowering their thresholds. Five- and ten-year
+warmups are both tested. FLEX-Topo still adjusts slightly after five years;
+the probe permits this bounded adjustment, and the tests compare late-cycle
+means rather than requiring the first scored cycle to be fully equilibrated.
+
+## Limits
+
+This probe cannot detect every slow leak or a trend that remains within valid
+bounds. A model can fabricate bounded states and compensating fluxes and still
+pass; passing is not proof of physical understanding. Legitimate equilibration,
+seasonal storage, and declared groundwater exchange are not themselves failures.
+The negative reporter can produce negative runoff in drier, unrelated cases;
+the tests explicitly exclude that alternative failure on this forcing.
+
+## Reproduce
+
+The local gate passes all five fixed seeds for all four physical baselines,
+and the slow reporter fails only `state_bounds`. The focused tests also cover
+three independent seeds, reproducibility, prefix consistency and longer spinup.
+The archive records the three evaluated physical models as PASS and Google as
+INCOMPLETE (required budget variables absent). The local dhbv2 attempt is
+recorded as ERROR because Docker was unavailable, not as a physical violation;
+its container evaluation remains to be rerun before final acceptance.
+
+```sh
+ht validate
+ht gate --probe mass/multi-decadal-drift
+pytest -q tests/test_multi_decadal_drift.py
+```
+
+Author: Bing Li (@hiter-joe). Proposal: https://github.com/Flood-Lab/HydroTuring/issues/13.