diff --git a/docs/topstep_100k_verdict.html b/docs/topstep_100k_verdict.html new file mode 100644 index 0000000..e6fbd59 --- /dev/null +++ b/docs/topstep_100k_verdict.html @@ -0,0 +1,493 @@ + + + + + +Topstep 100K combine — go/no-go verdict + + + + +

Topstep 100K combine — go/no-go verdict

+

PPO combine-manager on fractal-zigzag triggers — 3-seed confirmation of the +Optuna sweep winner on a holdout the sweep never touched. Sweep: study +topstep-100k-v1, 2023-01-02 → 2025-06-01. Holdout evaluation: test months +2025-06 → 2026-06. Six symbols (NQ, ES, RTY, YM, GC, SI), one policy, one +account. Report date 2026-07-12; issue #5, parent PRD #1.

+ +
+
VERDICT: ITERATE
+

+ The sweep winner fails every acceptance gate on the held-out year. Pooled + across three seeds, 3 of 852 combine attempts passed (0.35%) against the 60% + bar; 813 attempts (95.4%) busted the EOD-trailing max-loss; and the trained + policy underperforms both no-skill baselines — random-take passed + 11.5% of its attempts and take-every-signal 5.2%, both with the same zero + skill and far less churn. Do not pay for a real combine on this + configuration. One materially different round (longer training, stronger + survival shaping, entry-quality filtering — see Next levers) is justified + before writing the approach off; but be clear that this is + DISCARD-leaning evidence: round one showed no positive + skill anywhere, and if round two cannot at least beat the no-skill + baselines, the honest verdict becomes DISCARD.

+
+ +
+
0.35%
+
pooled OOS pass rate (bar: 60%)
+
852
+
combine attempts (3 seeds)
+
813
+
account blowups (bar: 0)
+
3
+
median days to target (3 passes)
+
+ +

The question and the gates

+

Would a policy trading the fractal-zigzag triggers pass a Topstep 100K combine +($6,000 profit target, $3,000 EOD-trailing max loss, $2,000 daily loss limit — +soft breach, halts the day — 10-contract cap, 1 tick slippage per side + +$1.40/side/contract commission)? The parent PRD sets three acceptance gates, +all of which must pass before paying for a real combine attempt:

+ + + + + + + + + + + +
GateBarObservedResult
1 — OOS combine pass rate≥ 60%0.35% pooled over 852 attemptsFAIL
2 — both baselines beatenpolicy strictly > each baselinerandom-take: not beaten (11.5% vs 0.35%); take-every-signal: not beaten (5.2% vs 0.35%)FAIL
3 — seed stabilityevery seed ≥ 50% with zero blowupsseed 0: 0.7% / 290 busts; seed 1: 0.0% / 266 busts; seed 2: 0.4% / 257 bustsFAIL
+ +

Protocol

+

The sweep (40 Optuna trials, TPE + median pruning, composite objective of pass +rate, days-to-pass, and a zero-blowup floor) scored each trial over 26 +walk-forward windows (3 months train / 1 month test) on 2023-01-02 → 2025-06-01, +training a fresh PPO policy (stable-baselines3 defaults, 10,000 timesteps) per +window. The confirmation reruns the logged winner configuration — the trial-19 +knobs from the study storage (passed at 3-decimal precision; see Reproduction), +nothing re-tuned — with only the seed varying (seeds 0, 1, 2), over the same +3m/1m walk-forward protocol, on data whose test months never appeared anywhere +in the sweep. Both no-skill +baselines (random-take and take-every-signal, 1 contract, never flattening +early) run through the identical evaluation seam. Terminal states come from the +one pure combine simulator (topstep.simulate_combine) in every case.

+ +

The winning knobs (Optuna trial 19, study topstep-100k-v1)

+ + + + + + + + +
KnobMeaningValue
activate_rstop-move activation threshold (R multiples)0.554
trail_atr_ktrailing band (ATR multiples)2.518
dll_penaltydaily-loss proximity penalty (reward shaping)1.769
mll_penaltytrailing-drawdown proximity penalty (reward shaping)0.650
+

Full precision, as logged: activate_r=0.5538683666181621, +trail_atr_k=2.5175435982558527, dll_penalty=1.7692588740326491, +mll_penalty=0.64957433407365; sweep seed 19. PPO hyperparameters stayed at +stable-baselines3 defaults and trigger parameters were frozen, per the PRD.

+ +

What the sweep already said

+

The confirmation was run knowing the sweep verdict was already negative — the +point of the 3-seed run is to make the negative result solid, not to rescue it. +Across the 38 completed trials: 19,841 pooled combine attempts, 18,813 busts +(94.8%), 41 passes (0.2%). Every completed trial scored below the blow floor +(−100): every configuration blew accounts, repeatedly. The best composite score +was −544 (trial 19); 39 of 40 trials had a 0% pass rate, one had 1% (trial 35). +The three highest-ranked rows in the study (scores −117 to −120) are a pruned +or interrupted trial's partial averages over a handful of windows, not better +configurations.

+ + + + + + + + + + + +
RankTrialStateScorePassBustsAttempts
116pruned−117.00%5154
20interrupted−118.80%150158
331pruned−120.50%4143
419 (winner)complete−544.00%444470
521complete−544.00%444470
35 more completed trials−551 … −6590–1%451–559479–586
+

Study bookkeeping: 41 logged trials — 38 complete, 2 pruned, 1 interrupted +(RUNNING leftover from a restart; the study is resumable and trial numbering +continued past it).

+ +

3-seed confirmation on the holdout

+

13 walk-forward windows covered test months 2025-06 through 2026-06 (the last +window has only four June sessions). Each policy run saw 89,814 candidate +signals and took 27,205–28,369 trades; each baseline took ~8,000 (the policy's +early flattens shorten holds, so fewer signals arrive while a position is +open). The three seeds agree — consistently negative: 0.7%, 0.0% and 0.4% pass +rates with 257–290 trailing-max-loss busts each. The attempt counts are +themselves diagnostic: the policy churns through 270–304 fresh $100K accounts +where the baselines consume 87–116, because it busts roughly three times as +fast. The sweep's in-sample verdict replicates out of sample with only the +seed varying.

+ +
+
+ passed + busted (trailing MLL) + busted (DLL) + timeout +
+
+ +policy seed 0 +policy seed 0 — passed: 2 of 304 (1%) +policy seed 0 — busted-MLL: 290 of 304 (95%) +policy seed 0 — timeout: 12 of 304 (4%) +busted-MLL 95% +304 +policy seed 1 +policy seed 1 — busted-MLL: 266 of 278 (96%) +policy seed 1 — timeout: 12 of 278 (4%) +busted-MLL 96% +278 +policy seed 2 +policy seed 2 — passed: 1 of 270 (0%) +policy seed 2 — busted-MLL: 257 of 270 (95%) +policy seed 2 — timeout: 12 of 270 (4%) +busted-MLL 95% +270 +baseline random-take +baseline random-take — passed: 10 of 87 (11%) +baseline random-take — busted-MLL: 64 of 87 (74%) +baseline random-take — timeout: 13 of 87 (15%) +busted-MLL 74% +87 +baseline take-every-signal +baseline take-every-signal — passed: 6 of 116 (5%) +baseline take-every-signal — busted-MLL: 97 of 116 (84%) +baseline take-every-signal — timeout: 13 of 116 (11%) +busted-MLL 84% +116 +att. + +
+
Where combine attempts end, per run. Every bar is the full set of + rolling OOS attempts for that run; segment width is the share of attempts. + DLL busts are structurally zero: under published Topstep mechanics the daily + loss limit is a soft breach (the day halts, the account survives), so the + simulator never emits busted-DLL.
+
+ + + + + + + + + + +
RunAttemptsPassedPass rateBusted (MLL)TimeoutMedian days to targetSignalsTaken
policy seed 030420.7%290123.089,81428,369
policy seed 127800.0%26612n/a89,81427,872
policy seed 227010.4%257126.089,81427,205
baseline random-take871011.5%64136.089,8147,665
baseline take-every-signal11665.2%97136.089,8148,382
+ +

Bust breakdown

+

Every one of the 813 policy busts — and all 161 baseline busts — was +busted-MLL, the $3,000 EOD-trailing max loss. DLL busts are zero by +construction: the $2,000 daily loss limit is a soft breach (the session halts, +the account survives, matching published Topstep mechanics), so its damage +shows up a day or two later when the end-of-day trailing anchor catches the +drawdown. The remaining 36 policy attempts (and 26 baseline attempts) timed out +at the 30-session-day cap without reaching the $6,000 target. The three passes +that did occur took 3–6 sessions and overshot the target wildly (final +equities $109K–$188K against a $106K finish line) — lottery-ticket outcomes of +the same oversized behavior that busts the other 95% of attempts, not evidence +of skill.

+ +

Per-symbol attribution (policy, pooled across the 3 seeds)

+
+
+ + +NQ +NQ: $-135,301 net over 25,296 trades +$-135,301 +ES +ES: $-560,586 net over 10,034 trades +$-560,586 +RTY +RTY: $-168,282 net over 12,306 trades +$-168,282 +YM +YM: $-305,762 net over 9,672 trades +$-305,762 +GC +GC: $-331,792 net over 15,529 trades +$-331,792 +SI +SI: $3,633 net over 10,609 trades +$3,633 + +
+
Net P&L per symbol summed over all policy attempts across the three + seeds, derived from the simulator's own equity path (friction included). One + series — bar length carries the value; labels at the bar ends.
+
+ + + + + + + + + + + +
SymbolTradesNet P&LBusting trades
NQ25,296-$135,301215
ES10,034-$560,58663
RTY12,306-$168,28221
YM9,672-$305,76229
GC15,529-$331,792233
SI10,609+$3,633252
+

The PRD's metals suspicion is confirmed: GC and SI carry the largest +per-contract dollar swings and account for 485 of the 813 busts (SI 252, +GC 233) from only 31% of the trades — the sizing head and DLL-proximity +observations did not tame them. ES is the largest net loser (−$560,586 +pooled); SI's small positive net (+$3,633, driven by one +$241K seed-2 window) +does not offset its 252 account kills. No symbol is net-positive across all +three seeds, so the failure is not one bad market — it is the sizing/exit +behavior everywhere. (Dollar figures are sums over the simulator's own equity +paths across all attempts, friction included; they measure behavior quality, +not a tradable P&L.)

+ +

Why ITERATE and not DISCARD

+

Two facts keep this at ITERATE — barely. First, the policy losing to its own +no-skill baselines is a classic under-training symptom, not yet proof the +approach is dead: at 10,000 PPO timesteps per window the policy learned to +take 3.4x more trades than take-every-signal at learned sizes up to the +10-contract cap, without learning the survival behavior the reward shaping was +supposed to teach — the training budget, the shaping ranges, or both are +inadequate, and neither has been given a fair test. Second, the baselines +themselves put a measured ceiling on the raw signal: random-take at 1 contract +passes 11.5% of attempts. That is 5x short of the 60% bar, but it is not zero +— there is some exploitable edge in the triggers for a policy that sizes +sanely. Against that: nothing in round one showed the RL layer adding value +anywhere — every seed, every symbol, every window. This is DISCARD-leaning +evidence. The concrete tripwire: if round two (levers below) still cannot beat +both no-skill baselines, discard the PPO-combine-manager approach and keep +only the simulator + verdict machinery.

+ +

Next levers (round two)

+ + +

Reproduction

+
# the sweep that produced the winner (already run; study is resumable)
+uv run python -m futures_foundation.rl.optuna_sweep \
+    --data-dir data --start 2023-01-02 --end 2025-06-01 \
+    --n-trials 40 --timesteps 10000 \
+    --storage sqlite:///optuna_topstep_100k.db --study-name topstep-100k-v1
+
+# the 3-seed confirmation behind this report (as actually run: winner knobs
+# at 3-decimal precision via --params; --storage/--study-name re-derives the
+# full-precision equivalent from the study)
+uv run python -m futures_foundation.rl.seed_confirm \
+    --data-dir data \
+    --params '{"activate_r": 0.554, "trail_atr_k": 2.518,
+               "dll_penalty": 1.769, "mll_penalty": 0.65}' \
+    --start 2025-03-01 --end 2026-06-04 \
+    --timesteps 10000 --seeds 0 1 2 \
+    --out seed_confirm_results.json
+

Data note: the 3-minute Parquet files under data/ are gitignored run +inputs — a fresh clone (or a git worktree) must restore them (or point +--data-dir at a checkout that has them) before either command runs.

+

Honesty notes: attempts within a window share one walk-forward policy, so they +are not fully independent games; the per-contract $/R scale is set from the +median stop distance over the loaded slice (the sweep did the same); training +months 2025-03 → 2025-05 of the first confirmation window overlap the sweep's +data range (training data only — every test month is post-2025-06 and was +never seen by the sweep or the winner selection); and 10,000 PPO timesteps per +window is a deliberate laptop-scale budget, which bounds how much skill this +round could have demonstrated (see Next levers).

+ +
+

Generated 2026-07-12 — +issue #5 (3-seed confirmation + go/no-go verdict report), parent PRD #1. +Machinery: futures_foundation/rl/seed_confirm.py, +futures_foundation/rl/optuna_sweep.py, futures_foundation/rl/multi_combine.py, +futures_foundation/topstep.py.

+ + diff --git a/futures_foundation/rl/seed_confirm.py b/futures_foundation/rl/seed_confirm.py new file mode 100644 index 0000000..84cd0fc --- /dev/null +++ b/futures_foundation/rl/seed_confirm.py @@ -0,0 +1,314 @@ +"""3-seed confirmation of the sweep winner + go/no-go gates (issue #5). + +The final gate before paying for a real combine. The Optuna sweep +(optuna_sweep) logs the winner in its study storage; this module +re-derives that exact configuration (winning_config -> build_strategy) +and retrains it on N seeds — ONLY the seed varies — over the same +walk-forward protocol the sweep used (fresh PPO per window, rolling OOS +combine attempts on the held-out test months), on a HOLDOUT slice the +sweep never touched. Both no-skill baselines run through the identical +evaluate/summarize seam. The result is scored against the three PRD +acceptance gates: + + 1. pooled OOS combine pass rate >= 60% + 2. the policy beats BOTH baselines (random-take, take-every-signal) + 3. every seed >= 50% pass rate with ZERO blowups + +THE documented command (the real confirmation; hours on CPU/MPS): + + uv run python -m futures_foundation.rl.seed_confirm \ + --data-dir data \ + --storage sqlite:///optuna_topstep_100k.db \ + --study-name topstep-100k-v1 \ + --start 2025-03-01 --end 2026-06-04 \ + --timesteps 10000 --seeds 0 1 2 \ + --out seed_confirm_results.json +""" +import argparse +import json +from pathlib import Path + +import numpy as np +import pandas as pd + +from futures_foundation.topstep import SYMBOL_SPECS, TOPSTEP_100K + +from .fractal_zigzag import SYMBOLS, compute_obs_features, load_3min_parquet +from .multi_combine import (SymbolEpisode, evaluate_account_attempts, + per_symbol_attribution, random_take_policy, + summarize_attempts, take_every_signal_policy) +from .optuna_sweep import _month_masks, _window_months, build_strategy + +#: PRD acceptance bars — pooled pass rate and the per-seed stability floor. +PASS_BAR = 0.60 +SEED_BAR = 0.50 + + +def evaluate_gates(seed_summaries: dict, baseline_summaries: dict, + pass_bar: float = PASS_BAR, + seed_bar: float = SEED_BAR) -> dict: + """Score the PRD acceptance gates over per-seed and baseline summaries + (the summarize_attempts shape, pooled per run). + + Gate 1 pools attempts across ALL seeds (attempt-weighted, not a mean + of rates). Gate 2 requires a STRICT beat of every baseline — a tie + with no-skill is not evidence of skill. Gate 3 requires every seed to + clear `seed_bar` with zero blowups. `ship` is True only when all + three gates pass; Iterate-vs-Discard on a no-ship is a judgment call + that belongs to the verdict report, not this function.""" + attempts = sum(s["attempts"] for s in seed_summaries.values()) + passed = sum(s["passed"] for s in seed_summaries.values()) + pooled = (passed / attempts) if attempts else 0.0 + busted = sum(s["busted"] for s in seed_summaries.values()) + beats = {name: pooled > b["pass_rate"] + for name, b in baseline_summaries.items()} + per_seed = {seed: s["pass_rate"] >= seed_bar and s["busted"] == 0 + for seed, s in seed_summaries.items()} + gates = { + "pass_rate": {"value": pooled, "bar": pass_bar, + "passed": attempts > 0 and pooled >= pass_bar}, + "baselines_beaten": {"value": beats, + "passed": bool(beats) and all(beats.values())}, + "seed_stability": {"value": per_seed, "busted": busted, + "passed": bool(per_seed) + and all(per_seed.values())}, + } + return {"gates": gates, "attempts": attempts, "pooled_pass_rate": pooled, + "ship": all(g["passed"] for g in gates.values())} + + +def make_runner(datas: dict, params: dict, train_months: int = 3, + test_months: int = 1, timesteps: int = 10_000, + max_days: int = 30, trades_per_day: int = 6, + train_fn=None): + """runner(kind, seed) -> pooled holdout result for one policy arm. + + kind: "ppo" (fresh PPO per window from the winner params, the seed + under test), "random" (coin-flip take at size 1), "take-every" + (always take at size 1). Mirrors optuna_sweep.make_data_evaluator + window-for-window — same strategy construction, same $/R scaling, + same fresh-account run_state per window — so the confirmation is + scored by the exact protocol that scored the sweep, but keeps + attempt-level detail (fills) for bust breakdown and per-symbol + attribution. `train_fn(train_eps, seed) -> policy` injects a stub + trainer in tests; None lazy-loads the SB3 default.""" + windows = list(_window_months(datas, train_months, test_months)) + if not windows: + raise ValueError("data slice too short for even one " + f"{train_months}+{test_months}-month window") + + def fit(train_eps, seed): + if train_fn is not None: + return train_fn(train_eps, seed) + from .ppo import make_ppo_trainer # lazy: needs SB3 + gymnasium + return make_ppo_trainer(total_timesteps=timesteps).train( + train_eps, seed) + + def runner(kind: str, seed: int) -> dict: + from .pipeline import _episodes + attempts, per_window = [], [] + signals = taken = skipped = 0 + for tr_months, te_months in windows: + rs_train, rs_test = {"cum_r": []}, {"cum_r": []} + train_eps, test_eps = [], [] + for sym, (df, ctx) in datas.items(): + strat = build_strategy(sym, params, + trades_per_day=trades_per_day) + entries = strat.detect_entries(df, df, sym) + if not len(entries): + continue + tick_size, tick_value = SYMBOL_SPECS[sym] + strat.dollars_per_r = ( + float(np.median(entries["sl_distance"])) + * tick_value / tick_size) + if kind == "ppo": + train_eps += _episodes( + strat, df, ctx, _month_masks(df.index, tr_months), + rs_train) + test_eps += [SymbolEpisode(dt, env, strat, df.index) + for dt, env in _episodes( + strat, df, ctx, + _month_masks(df.index, te_months), rs_test)] + if not test_eps or (kind == "ppo" and not train_eps): + continue # window without entries + ctx_dim = test_eps[0].env.ctx_dim + if kind == "ppo": + policy = fit(train_eps, seed) + elif kind == "random": + policy = random_take_policy(ctx_dim, seed) + elif kind == "take-every": + policy = take_every_signal_policy(ctx_dim) + else: + raise ValueError(f"unknown policy kind: {kind!r}") + result = evaluate_account_attempts( + policy, test_eps, rules=TOPSTEP_100K, max_days=max_days, + run_state=rs_test) + attempts += result["attempts"] + signals += result["signals"] + taken += result["taken"] + skipped += result["skipped_while_open"] + per_window.append(summarize_attempts(result["attempts"])) + return {"summary": summarize_attempts(attempts), + "attribution": per_symbol_attribution(attempts, + TOPSTEP_100K), + "signals": signals, "taken": taken, + "skipped_while_open": skipped, "windows": len(per_window), + "per_window": per_window, + "attempts": [{"state": a["state"], "days": a["days"], + "trades": a["trades"], "equity": a["equity"], + "note": a["note"]} for a in attempts]} + + return runner + + +def confirm(runner, seeds, baseline_seed: int = 0) -> dict: + """The issue #5 protocol: N policy retrains (only the seed varies — + same winner params, same windows) plus both no-skill baselines + through the identical runner, gated by evaluate_gates.""" + seed_results = {int(s): runner("ppo", int(s)) for s in seeds} + baselines = {"random-take": runner("random", baseline_seed), + "take-every-signal": runner("take-every", 0)} + verdict = evaluate_gates( + {s: r["summary"] for s, r in seed_results.items()}, + {n: r["summary"] for n, r in baselines.items()}) + return {"seeds": seed_results, "baselines": baselines, + "verdict": verdict} + + +# ── CLI ────────────────────────────────────────────────────────────────────── +def _print_run(name: str, r: dict) -> None: + s = r["summary"] + busts = ", ".join(f"{k}={v}" for k, v in s["bust_breakdown"].items()) + med = (f"{s['median_days_to_pass']:.1f}" + if s["median_days_to_pass"] is not None else "n/a") + print(f"-- {name} --") + print(f"windows={r['windows']} signals={r['signals']} " + f"taken={r['taken']} skipped-while-open={r['skipped_while_open']}") + print(f"attempts={s['attempts']} passed={s['passed']} " + f"({s['pass_rate']:.0%}) busted={s['busted']}" + f"{f' [{busts}]' if busts else ''} timeout={s['timeout']} " + f"median-days-to-pass={med}") + for sym in SYMBOLS: + if sym in r["attribution"]: + a = r["attribution"][sym] + print(f" {sym:<4} trades={a['trades']:<5} " + f"net=${a['net_pnl']:>12,.2f} busts={a['busts']}") + + +def _print_gates(verdict: dict) -> None: + g = verdict["gates"] + print("== PRD acceptance gates ==") + print(f"gate 1 pooled pass rate {g['pass_rate']['value']:.1%} " + f"(bar {g['pass_rate']['bar']:.0%}, " + f"{verdict['attempts']} attempts) " + f"{'PASS' if g['pass_rate']['passed'] else 'FAIL'}") + beats = ", ".join(f"{k}: {'yes' if v else 'no'}" + for k, v in g["baselines_beaten"]["value"].items()) + print(f"gate 2 baselines beaten ({beats}) " + f"{'PASS' if g['baselines_beaten']['passed'] else 'FAIL'}") + per_seed = ", ".join(f"seed {k}: {'ok' if v else 'FAIL'}" + for k, v in g["seed_stability"]["value"].items()) + print(f"gate 3 seed stability ({per_seed}; " + f"{g['seed_stability']['busted']} busts total) " + f"{'PASS' if g['seed_stability']['passed'] else 'FAIL'}") + print("VERDICT: SHIP" if verdict["ship"] else + "VERDICT: NO-SHIP — Iterate vs Discard is the report's call") + + +def main(argv=None): + p = argparse.ArgumentParser( + description="3-seed confirmation of the Optuna sweep winner on a " + "holdout slice, with no-skill baselines and the PRD " + "acceptance gates.") + p.add_argument("--data-dir", default="data", + help="directory holding _3min.parquet files") + p.add_argument("--symbols", nargs="+", default=list(SYMBOLS), + choices=sorted(SYMBOL_SPECS)) + p.add_argument("--storage", default=None, + help="Optuna storage URL holding the finished sweep; " + "the winner is re-derived from it") + p.add_argument("--study-name", default="topstep-combine-sweep") + p.add_argument("--params", default=None, + help="JSON dict of winner params (activate_r, " + "trail_atr_k, dll_penalty, mll_penalty) — " + "alternative to --storage") + p.add_argument("--start", default="2025-03-01", + help="slice start; the first train_months months are " + "training lead-in, so the first TEST month starts " + "train_months later — the holdout boundary") + p.add_argument("--end", default="2026-06-04") + p.add_argument("--train-months", type=int, default=3) + p.add_argument("--test-months", type=int, default=1) + p.add_argument("--timesteps", type=int, default=10_000, + help="PPO timesteps per (seed, window) train") + p.add_argument("--seeds", type=int, nargs="+", default=[0, 1, 2]) + p.add_argument("--baseline-seed", type=int, default=0, + help="rng seed for the random-take baseline") + p.add_argument("--max-days", type=int, default=30, + help="session days per combine attempt before timeout") + p.add_argument("--trades-per-day", type=int, default=6, + help="synthetic session-day length during training") + p.add_argument("--out", default=None, + help="write the full result dict as JSON here") + args = p.parse_args(argv) + + if args.params: + params = json.loads(args.params) + winner = {"params": params, "seed": None} + elif args.storage: + import optuna + from .optuna_sweep import winning_config + study = optuna.load_study(study_name=args.study_name, + storage=args.storage) + winner = winning_config(study) + params = winner["params"] + else: + p.error("one of --storage or --params is required") + + print(f"== 3-seed confirmation: {' '.join(args.symbols)} ==") + print(f"winner params: {json.dumps(params, sort_keys=True)}") + print(f"seeds: {args.seeds} (sweep winner's own seed: " + f"{winner['seed']})") + datas = {} + for sym in args.symbols: + df = load_3min_parquet(Path(args.data_dir) / f"{sym}_3min.parquet") + df = df.loc[pd.Timestamp(args.start, tz="UTC"): + pd.Timestamp(args.end, tz="UTC")] + datas[sym] = (df, compute_obs_features(df)) + print(f"{sym}: bars={len(df):,} ({df.index[0]} .. {df.index[-1]})") + windows = list(_window_months(datas, args.train_months, + args.test_months)) + print(f"{len(windows)} walk-forward windows ({args.train_months}m train" + f" / {args.test_months}m test); test months " + f"{windows[0][1][0]} .. {windows[-1][1][-1]}") + + runner = make_runner(datas, params, train_months=args.train_months, + test_months=args.test_months, + timesteps=args.timesteps, max_days=args.max_days, + trades_per_day=args.trades_per_day) + result = confirm(runner, args.seeds, baseline_seed=args.baseline_seed) + + for seed, r in result["seeds"].items(): + _print_run(f"policy seed {seed}", r) + for name, r in result["baselines"].items(): + _print_run(f"baseline {name}", r) + _print_gates(result["verdict"]) + + if args.out: + result["config"] = { + "params": params, "winner_seed": winner["seed"], + "seeds": list(args.seeds), "symbols": list(args.symbols), + "start": args.start, "end": args.end, + "train_months": args.train_months, + "test_months": args.test_months, + "timesteps": args.timesteps, "max_days": args.max_days, + "trades_per_day": args.trades_per_day, + "test_month_range": [str(windows[0][1][0]), + str(windows[-1][1][-1])]} + Path(args.out).write_text(json.dumps(result, indent=1, + default=str)) + print(f"results written: {args.out}") + + +if __name__ == "__main__": + main() diff --git a/tests/test_seed_confirm.py b/tests/test_seed_confirm.py new file mode 100644 index 0000000..01f24eb --- /dev/null +++ b/tests/test_seed_confirm.py @@ -0,0 +1,135 @@ +"""3-seed confirmation + go/no-go gates (issue #5). + +Stub runners only — no SB3, no PPO, no data (the real retrain is driven +from the CLI). One seam per acceptance criterion: + AC1 — confirm() retrains once per seed with ONLY the seed varying, and + runs both no-skill baselines through the same runner + AC2 — evaluate_gates scores every PRD gate: pooled pass rate >= 60%, + both baselines strictly beaten, every seed >= 50% with zero + blowups; ship only when all three pass +""" +import pytest + +from futures_foundation.rl.seed_confirm import (PASS_BAR, SEED_BAR, confirm, + evaluate_gates) + + +def _summary(pass_rate=0.0, days=None, busted=0, attempts=10): + """A summarize_attempts-shaped dict (the gates' input contract).""" + passed = round(pass_rate * attempts) + return {"attempts": attempts, "passed": passed, "pass_rate": pass_rate, + "busted": busted, + "bust_breakdown": {"busted-MLL": busted} if busted else {}, + "timeout": attempts - passed - busted, + "median_days_to_pass": days} + + +def _gates(seeds, baselines): + return evaluate_gates(seeds, baselines) + + +NO_SKILL = {"random-take": _summary(pass_rate=0.0), + "take-every-signal": _summary(pass_rate=0.0)} + + +# ───────────────────────────────────────── AC2: the three PRD gates +def test_all_gates_pass_ships(): + v = _gates({0: _summary(0.7, days=10.0), 1: _summary(0.6, days=12.0), + 2: _summary(0.8, days=9.0)}, NO_SKILL) + assert all(g["passed"] for g in v["gates"].values()) + assert v["ship"] is True + + +def test_pooled_pass_rate_below_bar_fails_gate_1(): + v = _gates({0: _summary(0.5, days=10.0), 1: _summary(0.5, days=10.0), + 2: _summary(0.5, days=10.0)}, NO_SKILL) + assert v["gates"]["pass_rate"]["passed"] is False + assert v["ship"] is False + + +def test_pooled_rate_is_attempt_weighted_not_a_mean_of_rates(): + # 90% of 100 attempts + 0% of 2 attempts pools to ~88%, not 45% + v = _gates({0: _summary(0.9, days=5.0, attempts=100), + 1: _summary(0.0, attempts=2)}, NO_SKILL) + assert v["pooled_pass_rate"] == pytest.approx(90 / 102) + + +def test_tie_with_a_baseline_is_not_beaten(): + # 0% policy vs 0% baselines: no skill demonstrated, gate 2 fails + v = _gates({0: _summary(0.0), 1: _summary(0.0), 2: _summary(0.0)}, + NO_SKILL) + assert v["gates"]["baselines_beaten"]["passed"] is False + assert v["ship"] is False + + +def test_one_baseline_ahead_fails_gate_2(): + v = _gates({0: _summary(0.7, days=10.0), 1: _summary(0.7, days=10.0), + 2: _summary(0.7, days=10.0)}, + {"random-take": _summary(0.1), + "take-every-signal": _summary(0.9, days=3.0)}) + assert v["gates"]["baselines_beaten"]["value"]["random-take"] is True + assert (v["gates"]["baselines_beaten"]["value"]["take-every-signal"] + is False) + assert v["gates"]["baselines_beaten"]["passed"] is False + + +def test_one_seed_below_50_fails_stability(): + v = _gates({0: _summary(0.9, days=5.0), 1: _summary(0.9, days=5.0), + 2: _summary(0.4, days=5.0)}, NO_SKILL) + assert v["gates"]["seed_stability"]["value"][2] is False + assert v["gates"]["seed_stability"]["passed"] is False + assert v["ship"] is False + + +def test_a_single_blowup_fails_stability_even_at_high_pass_rate(): + v = _gates({0: _summary(0.9, days=5.0), 1: _summary(0.9, days=5.0), + 2: _summary(0.9, days=5.0, busted=1)}, NO_SKILL) + assert v["gates"]["seed_stability"]["value"][2] is False + assert v["gates"]["seed_stability"]["busted"] == 1 + assert v["ship"] is False + + +def test_zero_attempts_never_ships(): + v = _gates({0: _summary(attempts=0), 1: _summary(attempts=0)}, + NO_SKILL) + assert v["attempts"] == 0 + assert v["gates"]["pass_rate"]["passed"] is False + assert v["ship"] is False + + +def test_bars_are_the_prd_numbers(): + assert PASS_BAR == 0.60 + assert SEED_BAR == 0.50 + + +# ────────────────────── AC1: only the seed varies; baselines share the seam +def test_confirm_runs_each_seed_once_plus_both_baselines(): + calls = [] + + def runner(kind, seed): + calls.append((kind, seed)) + return {"summary": _summary(0.7, days=10.0), "attribution": {}, + "signals": 5, "taken": 5, "skipped_while_open": 0, + "windows": 1, "per_window": [], "attempts": []} + + result = confirm(runner, seeds=[7, 8, 9], baseline_seed=42) + assert calls == [("ppo", 7), ("ppo", 8), ("ppo", 9), + ("random", 42), ("take-every", 0)] + assert sorted(result["seeds"]) == [7, 8, 9] + assert sorted(result["baselines"]) == ["random-take", + "take-every-signal"] + # the verdict is wired from the runs' own summaries + assert result["verdict"]["ship"] is False # baselines tie at 70% + + +def test_confirm_verdict_reflects_per_run_summaries(): + def runner(kind, seed): + s = (_summary(0.8, days=8.0) if kind == "ppo" + else _summary(0.0)) + return {"summary": s, "attribution": {}, "signals": 5, "taken": 5, + "skipped_while_open": 0, "windows": 1, "per_window": [], + "attempts": []} + + result = confirm(runner, seeds=[0, 1, 2]) + assert result["verdict"]["ship"] is True + assert result["verdict"]["pooled_pass_rate"] == pytest.approx(0.8)