RL recentering, part 2: causalrl.ope, the real-data suite, and v3.0.0

[raphaelrrcoelho]

The last five conversions from docs/rl_recentering/PLAN.md, and the breaking release they were aimed at. Part 1 (#39) fixed the agents; this fixes everything that pointed at them.

What changed

T4 — dropped the statistics tooling (07cec3c). Every symbol was tested against one question: can anything under agents/ or envs/ reach it? These could not. Gone: certify_mean, certify_quantile, moment_diagnostic, tail_index_hill, weighted_quantile, bootstrap_quantile_ci, MomentDiagnostic, stream_quantile_certificate, backends/quantile_sketch.py, experimental/ope.py, and the whole meanfield/ package — ~900 lines, 262 → 258 exports. certify_mean was emitting EstimandSpec(query="do", …) for a computation containing no intervention. The MSM kernels in the same modules stay; estimate/nuisance.py is kept but demoted to internal, since four modules depend on it.

T2 — the real-data suite goes through the front door (cc182d9). .act() was called nowhere in it, causal_mbrl_coat.py imported no causalrl function at all, and causal_mbrl_obd.py binarised 80 bandit arms to fit a signature. Now: certify_policy over all 80 arms natively, and .act() per unit on Twins/LaLonde/NHEFS feeding the policy that gets valued and certified.

This also found a bug in published output. Coat's hand-rolled sensitivity loop rescaled every propensity by the same Γ without self-normalising — not Tan's MSM, which lets each unit's odds move independently within [1/Γ, Γ] and takes the sharp extremum. The correct bands are tighter: Γ=1.3 was reported as [1.86, 3.15], is [2.07, 2.61]. REAL_DATA.md is corrected with the reason recorded, not silently overwritten.

T5 — the decision front door names actions and rewards (8ed0400). certify_decision(outcomes=, treated=) → (rewards=, actions=); an 80-arm bandit no longer comes back with a "prefer treated" verdict. Plus estimate_sequential_value(actions=, reward=), policy_contrast_from_dowhy, policy_from_econml_cate, certify_transported_policy_value. No numerics changed.

T3 — causalrl.ope (2c863e1). The OPE surface was scattered across five packages. Now one home: ipw.py, sequential.py, bounds.py, certify.py. identification/bounds.py drops 587 → 305 lines and is finally what its name says. All 258 top-level exports resolve unchanged.

Two import cycles had to be designed around, both of which only fail on some import orders. identification/decision.py imports msm_contribution_bounds, so importing any ope submodule ran ope/__init__ → ope.certify → identification.decision → back into a half-initialised ope; a second ran through estimate/__init__'s re-export of stream_policy_value. Fixed with the lazy __getattr__ pattern the top-level __init__ already uses. All fourteen entry-point modules verified importing first from a cold cache.

T8 — the docs say what the library is (2f74294). Deliberately last; the plan's own note is "worthless before Tasks 1-2 make it true." One description sentence replaces three different ones across pyproject.toml, README.md, mkdocs.yml, docs/index.md, CITATION.cff and the package docstring. docs/api.md leads with agents and environments instead of burying them at slots 14-16 of 28. A sixth guide — the first that trains an agent — runs OnlineCausalMBRL from a 3,000-row confounded log through its own randomized experiments: the I-MEC collapses 6 DAGs → 1 and regret goes 0.220 → 0.000 in ~0.3s.

Release 3.0.0 (9a72071) and a test fix (b653177).

The results cut against us, and the README now says so

Scored as an RL practitioner would — .act() per unit, then regret against ground truth — the causal point estimates lose:

dataset	our policy	verdict
Twins (11,984 pairs, both potential outcomes known)	0.8316 vs oracle 0.8747 — regret 0.0431, worst of the learned policies, behind the trivial "always the heavier twin"	abstain, Γ≈1.07
LaLonde (priced by the NSW experiment)	enrols 73.5% for $476/person, where its own marginal rule leaves $0; off-policy value −$453 has the wrong sign	abstain, Γ≈1.10, conformal gate returns no finite bound
NHEFS	contextual policy is degenerate at 100%	abstain, Γ≈1.42

Both abstentions are right, and on LaLonde the randomized experiment vindicates the refusal. This is the recorded finding and it is now the README's positioning: the defensible edge is the decision and certificate layer, not the number.

What I could not verify

The claim that these layers run on real data at production scale was cut. The datasets here are real but modest — LaLonde n=614, Twins n=11,984, a 10k slice of OBD — and the only scale evidence anywhere is a synthetic 600k-row log. The README says "run on real data" and stops there.

mkdocs build --strict is not runnable in this venv, so I ran it in an ephemeral overlay: builds clean in 20.6s.

Gate

ruff clean, pyright 0/0/0, coverage 97.3%, mkdocs --strict clean, all six guides and the columnar-sim example run, and causalrl-3.0.0-py3-none-any.whl verified from a throwaway install — 3.0.0, 258 exports, causalrl.ope intact. The only ruff hits are 14 E501s in the untracked examples/causal_corr2cause_prompted.py, which belongs to another project and does not exist on CI.