PPO combine-manager on fractal-zigzag triggers — 3-seed confirmation of the
+Optuna sweep winner on a holdout the sweep never touched. Sweep: study
+topstep-100k-v1, 2023-01-02 → 2025-06-01. Holdout evaluation: test months
+2025-06 → 2026-06. Six symbols (NQ, ES, RTY, YM, GC, SI), one policy, one
+account. Report date 2026-07-12; issue #5, parent PRD #1.
+
+
+
VERDICT: ITERATE
+
+ The sweep winner fails every acceptance gate on the held-out year. Pooled
+ across three seeds, 3 of 852 combine attempts passed (0.35%) against the 60%
+ bar; 813 attempts (95.4%) busted the EOD-trailing max-loss; and the trained
+ policy underperforms both no-skill baselines — random-take passed
+ 11.5% of its attempts and take-every-signal 5.2%, both with the same zero
+ skill and far less churn. Do not pay for a real combine on this
+ configuration. One materially different round (longer training, stronger
+ survival shaping, entry-quality filtering — see Next levers) is justified
+ before writing the approach off; but be clear that this is
+ DISCARD-leaning evidence: round one showed no positive
+ skill anywhere, and if round two cannot at least beat the no-skill
+ baselines, the honest verdict becomes DISCARD.
+
+
+
+
0.35%
+
pooled OOS pass rate (bar: 60%)
+
852
+
combine attempts (3 seeds)
+
813
+
account blowups (bar: 0)
+
3
+
median days to target (3 passes)
+
+
+
The question and the gates
+
Would a policy trading the fractal-zigzag triggers pass a Topstep 100K combine
+($6,000 profit target, $3,000 EOD-trailing max loss, $2,000 daily loss limit —
+soft breach, halts the day — 10-contract cap, 1 tick slippage per side +
+$1.40/side/contract commission)? The parent PRD sets three acceptance gates,
+all of which must pass before paying for a real combine attempt:
+
+
+
Gate
Bar
Observed
Result
+
+
1 — OOS combine pass rate
≥ 60%
+
0.35% pooled over 852 attempts
FAIL
+
2 — both baselines beaten
policy strictly > each baseline
+
random-take: not beaten (11.5% vs 0.35%); take-every-signal: not beaten (5.2% vs 0.35%)
The sweep (40 Optuna trials, TPE + median pruning, composite objective of pass
+rate, days-to-pass, and a zero-blowup floor) scored each trial over 26
+walk-forward windows (3 months train / 1 month test) on 2023-01-02 → 2025-06-01,
+training a fresh PPO policy (stable-baselines3 defaults, 10,000 timesteps) per
+window. The confirmation reruns the logged winner configuration — the trial-19
+knobs from the study storage (passed at 3-decimal precision; see Reproduction),
+nothing re-tuned — with only the seed varying (seeds 0, 1, 2), over the same
+3m/1m walk-forward protocol, on data whose test months never appeared anywhere
+in the sweep. Both no-skill
+baselines (random-take and take-every-signal, 1 contract, never flattening
+early) run through the identical evaluation seam. Terminal states come from the
+one pure combine simulator (topstep.simulate_combine) in every case.
+
+
The winning knobs (Optuna trial 19, study topstep-100k-v1)
Full precision, as logged: activate_r=0.5538683666181621,
+trail_atr_k=2.5175435982558527, dll_penalty=1.7692588740326491,
+mll_penalty=0.64957433407365; sweep seed 19. PPO hyperparameters stayed at
+stable-baselines3 defaults and trigger parameters were frozen, per the PRD.
+
+
What the sweep already said
+
The confirmation was run knowing the sweep verdict was already negative — the
+point of the 3-seed run is to make the negative result solid, not to rescue it.
+Across the 38 completed trials: 19,841 pooled combine attempts, 18,813 busts
+(94.8%), 41 passes (0.2%). Every completed trial scored below the blow floor
+(−100): every configuration blew accounts, repeatedly. The best composite score
+was −544 (trial 19); 39 of 40 trials had a 0% pass rate, one had 1% (trial 35).
+The three highest-ranked rows in the study (scores −117 to −120) are a pruned
+or interrupted trial's partial averages over a handful of windows, not better
+configurations.
+
+
+
Rank
Trial
State
Score
Pass
Busts
Attempts
+
+
1
16
pruned
−117.0
0%
51
54
+
2
0
interrupted
−118.8
0%
150
158
+
3
31
pruned
−120.5
0%
41
43
+
4
19 (winner)
complete
−544.0
0%
444
470
+
5
21
complete
−544.0
0%
444
470
+
…
—
35 more completed trials
−551 … −659
0–1%
451–559
479–586
+
+
+
Study bookkeeping: 41 logged trials — 38 complete, 2 pruned, 1 interrupted
+(RUNNING leftover from a restart; the study is resumable and trial numbering
+continued past it).
+
+
3-seed confirmation on the holdout
+
13 walk-forward windows covered test months 2025-06 through 2026-06 (the last
+window has only four June sessions). Each policy run saw 89,814 candidate
+signals and took 27,205–28,369 trades; each baseline took ~8,000 (the policy's
+early flattens shorten holds, so fewer signals arrive while a position is
+open). The three seeds agree — consistently negative: 0.7%, 0.0% and 0.4% pass
+rates with 257–290 trailing-max-loss busts each. The attempt counts are
+themselves diagnostic: the policy churns through 270–304 fresh $100K accounts
+where the baselines consume 87–116, because it busts roughly three times as
+fast. The sweep's in-sample verdict replicates out of sample with only the
+seed varying.
+ Where combine attempts end, per run. Every bar is the full set of
+ rolling OOS attempts for that run; segment width is the share of attempts.
+ DLL busts are structurally zero: under published Topstep mechanics the daily
+ loss limit is a soft breach (the day halts, the account survives), so the
+ simulator never emits busted-DLL.
+
+
+
+
Run
Attempts
Passed
Pass rate
Busted (MLL)
Timeout
Median days to target
Signals
Taken
+
+
policy seed 0
304
2
0.7%
290
12
3.0
89,814
28,369
+
policy seed 1
278
0
0.0%
266
12
n/a
89,814
27,872
+
policy seed 2
270
1
0.4%
257
12
6.0
89,814
27,205
+
baseline random-take
87
10
11.5%
64
13
6.0
89,814
7,665
+
baseline take-every-signal
116
6
5.2%
97
13
6.0
89,814
8,382
+
+
+
+
Bust breakdown
+
Every one of the 813 policy busts — and all 161 baseline busts — was
+busted-MLL, the $3,000 EOD-trailing max loss. DLL busts are zero by
+construction: the $2,000 daily loss limit is a soft breach (the session halts,
+the account survives, matching published Topstep mechanics), so its damage
+shows up a day or two later when the end-of-day trailing anchor catches the
+drawdown. The remaining 36 policy attempts (and 26 baseline attempts) timed out
+at the 30-session-day cap without reaching the $6,000 target. The three passes
+that did occur took 3–6 sessions and overshot the target wildly (final
+equities $109K–$188K against a $106K finish line) — lottery-ticket outcomes of
+the same oversized behavior that busts the other 95% of attempts, not evidence
+of skill.
+
+
Per-symbol attribution (policy, pooled across the 3 seeds)
+
+
+
+
+ Net P&L per symbol summed over all policy attempts across the three
+ seeds, derived from the simulator's own equity path (friction included). One
+ series — bar length carries the value; labels at the bar ends.
+
+
+
+
Symbol
Trades
Net P&L
Busting trades
+
+
NQ
25,296
-$135,301
215
+
ES
10,034
-$560,586
63
+
RTY
12,306
-$168,282
21
+
YM
9,672
-$305,762
29
+
GC
15,529
-$331,792
233
+
SI
10,609
+$3,633
252
+
+
+
The PRD's metals suspicion is confirmed: GC and SI carry the largest
+per-contract dollar swings and account for 485 of the 813 busts (SI 252,
+GC 233) from only 31% of the trades — the sizing head and DLL-proximity
+observations did not tame them. ES is the largest net loser (−$560,586
+pooled); SI's small positive net (+$3,633, driven by one +$241K seed-2 window)
+does not offset its 252 account kills. No symbol is net-positive across all
+three seeds, so the failure is not one bad market — it is the sizing/exit
+behavior everywhere. (Dollar figures are sums over the simulator's own equity
+paths across all attempts, friction included; they measure behavior quality,
+not a tradable P&L.)
+
+
Why ITERATE and not DISCARD
+
Two facts keep this at ITERATE — barely. First, the policy losing to its own
+no-skill baselines is a classic under-training symptom, not yet proof the
+approach is dead: at 10,000 PPO timesteps per window the policy learned to
+take 3.4x more trades than take-every-signal at learned sizes up to the
+10-contract cap, without learning the survival behavior the reward shaping was
+supposed to teach — the training budget, the shaping ranges, or both are
+inadequate, and neither has been given a fair test. Second, the baselines
+themselves put a measured ceiling on the raw signal: random-take at 1 contract
+passes 11.5% of attempts. That is 5x short of the 60% bar, but it is not zero
+— there is some exploitable edge in the triggers for a policy that sizes
+sanely. Against that: nothing in round one showed the RL layer adding value
+anywhere — every seed, every symbol, every window. This is DISCARD-leaning
+evidence. The concrete tripwire: if round two (levers below) still cannot beat
+both no-skill baselines, discard the PPO-combine-manager approach and keep
+only the simulator + verdict machinery.
+
+
Next levers (round two)
+
+
More PPO timesteps per window. 10,000 timesteps per (trial, window) is
+ laptop-scale; the policy barely leaves its initialization. The upstream
+ existence proof for this parameterization trained far longer. First lever:
+ raise timesteps 10–20× on the top handful of configurations and re-score.
+
Reward-shaping ranges. Every trial busted, which says the searched
+ penalty ranges (dll_penalty 0–2, mll_penalty 0–4) never made
+ account-survival dominate the reward. Widen the MLL-proximity penalty range,
+ and consider a hard veto observation threshold near the trailing floor.
+
Entry-quality filter. The take-every-signal baseline matching the policy
+ says the policy learned no useful take/skip discrimination at this budget.
+ Pre-filtering triggers (e.g. by the SSL backbone's win-rate head or a simple
+ R-quality gate) shrinks the action problem PPO has to solve.
+
The study is resumable.optuna_topstep_100k.db persists all 41 trials;
+ round two extends the same study (load_if_exists) — trial numbering and
+ per-trial seeds continue without collision, so nothing from round one is lost.
+
+
+
Reproduction
+
# the sweep that produced the winner (already run; study is resumable)
+uv run python -m futures_foundation.rl.optuna_sweep \
+ --data-dir data --start 2023-01-02 --end 2025-06-01 \
+ --n-trials 40 --timesteps 10000 \
+ --storage sqlite:///optuna_topstep_100k.db --study-name topstep-100k-v1
+
+# the 3-seed confirmation behind this report (as actually run: winner knobs
+# at 3-decimal precision via --params; --storage/--study-name re-derives the
+# full-precision equivalent from the study)
+uv run python -m futures_foundation.rl.seed_confirm \
+ --data-dir data \
+ --params '{"activate_r": 0.554, "trail_atr_k": 2.518,
+ "dll_penalty": 1.769, "mll_penalty": 0.65}' \
+ --start 2025-03-01 --end 2026-06-04 \
+ --timesteps 10000 --seeds 0 1 2 \
+ --out seed_confirm_results.json
+
Data note: the 3-minute Parquet files under data/ are gitignored run
+inputs — a fresh clone (or a git worktree) must restore them (or point
+--data-dir at a checkout that has them) before either command runs.
+
Honesty notes: attempts within a window share one walk-forward policy, so they
+are not fully independent games; the per-contract $/R scale is set from the
+median stop distance over the loaded slice (the sweep did the same); training
+months 2025-03 → 2025-05 of the first confirmation window overlap the sweep's
+data range (training data only — every test month is post-2025-06 and was
+never seen by the sweep or the winner selection); and 10,000 PPO timesteps per
+window is a deliberate laptop-scale budget, which bounds how much skill this
+round could have demonstrated (see Next levers).