Skip to content

Implement offline backtest simulation - #4

Open
lee101 wants to merge 105 commits into
mainfrom
codex/implement-fake-backtest-simulation
Open

Implement offline backtest simulation#4
lee101 wants to merge 105 commits into
mainfrom
codex/implement-fake-backtest-simulation

Conversation

@lee101

@lee101 lee101 commented May 22, 2025

Copy link
Copy Markdown
Owner

Summary

  • add a FakeAlpaca shim for in-memory trading
  • create offline_backtest.py to run a two-week offline simulation
  • document how to run the offline backtest in the README

Testing

  • python -m pytest -q (fails: No module named pytest)

lee101 added 30 commits June 21, 2024 14:13
@lee101
Merge pull request #1 from lee101/chronos
5ad719f 
Chronos
@lee101
looping canceling in new process
547ac51
@lee101
fixup dates
6dd928f
@lee101
fix limiting orders kept clsoing at market
f623d22
@lee101
fix import
0beb518
@lee101
f rounding
b34c39c
@lee101
more aggressive sell ramp to exit positions - todo market order as a …
9654b3a 
…last resort
@lee101
fix async
a0f5f08
@lee101
fixlog
171e204
@lee101
fix debouncing
7b52228
@lee101
fix debouncing
cea9514
@lee101
clean context
0130076
@lee101
another layer of cooldown
3f18e22
@lee101
fix not backing out when its not trading time for normal assets - cou…
7243de3 
…ld trade under the value
@lee101
filetypeonlychangenewforecasts
1f4b08d
@lee101
wip
e27563e
@lee101
wip
07990e7
@lee101
wip
f9a2721
@lee101
fix
61cf6ae
@lee101
fix final day
1a02247
@lee101
diff handling
a51260a
@lee101
fix/todo
f0badc1
@lee101
wip
1b1ad53
@lee101
ramp from bid to ask
da94105
@lee101
print more details of when we should trade
d5beb68
@lee101
test other strat
a84a9a7
@lee101
dont change cache in test
3cdbb9c
@lee101
wip todo fix math
5fdff88
@lee101
fix tests
60f345f
@lee101
fix fees calc
bcef95f
@lee101
lee101 force-pushed the main branch 2 times, most recently from 198c06f to 4b5c43f Compare March 24, 2026 07:45
lee101 added a commit that referenced this pull request Mar 25, 2026
@lee101 @claude
feat: crypto70 sweep s401-700 progress — s507 NEW #2 (+8290%), s404 NEW
1ec5633 
#4 (+7295%)

Key discoveries:
- s507 (s501-600 range): +8290% ann at 5bps, Sortino=6.43, 5bps>8bps
- s404 (s401-500 range): +7295% ann at 5bps, Sortino=6.28, 5bps≈8bps≈10bps
- s602 (s601-700 range): +2594% ann at 5bps, Sortino=5.46
- s704 (s701-800 range): +1927% ann at 5bps, Sortino=5.36
- s507 and s404 saved to crypto70_champions/
- 46 seeds now ranked at 5bps (up from 34)
- Sweeps running: s121-200 (60/80), s201-300 (23/100), s301-400 (16/100),
  s401-500 (6/100), s501-600 (8/100), s601-700 (8/100), s701-800 (5/100)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
lee101 added a commit that referenced this pull request Mar 25, 2026
@lee101 @claude
sweep: crypto70 s401-900 new champions — s410 NEW #21 (+2945%@5bps), …
8dfffe0 
…s802 NEW #40 (+981%)

Additional champions beyond s404/s507/s609/s704 already committed:
- s410: +2945%@5bps, Sortino=5.76, 5bps>8bps very robust (second spike in s401-500)
- s802: +981%@5bps, Sortino=N/A (first s801-900 range find, very early)
- s506: +723%@5bps (solid but not exceptional)
- s801-900 sweep launched, monitor updated with champion save logic
- s701-800 monitor updated with champion save logic

Pattern confirmed: each 100-seed range produces 1-2 exceptional isolated spikes.
s401-500 is unusual with TWO champions (s404 #4 and s410 #21).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
lee101 added a commit that referenced this pull request Mar 25, 2026
@lee101 @claude
feat: crypto70 seed sweep s301-900 — s416 NEW TOP-10 (+4091%), s321 (…
3f4b545 
…+3063%), s513/s804 ROBUST

New 5bps production champions discovered:
- s416 (s401-500): +4091% ann, Sortino=6.18, 5bps>8bps ROBUST — NEW #11!
- s321 (s301-400): +3063% ann, Sortino=5.07
- s513 (s501-600): +2525% ann, Sortino=5.50, 5bps>8bps ROBUST
- s804 (s801-900): +1943% ann, Sortino=5.19, 5bps>8bps ROBUST
- s192 (s121-200): +1886% ann, Sortino=4.67, 5bps>8bps ROBUST
- s415 (s401-500): +1757% ann, Sortino=4.79, 5bps>8bps ROBUST
- s807 (s801-900): +934% ann, Sortino=6.37, 5bps>8bps ROBUST (high Sortino!)
- s191 (s121-200): +1457% ann, Sortino=3.90, ROBUST

s401-500 is the HOTTEST range: s404(#4)+s416(#11)+s410(#21)+s415(#31) = 4 champions
Added crypto70_5bps_leaderboard.csv — automated 5bps production leaderboard

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
lee101 added a commit that referenced this pull request Mar 25, 2026
@lee101 @claude
crypto70: new champions s456(+8802%), s452(+8002%) enter top-6
e67f11b 
Updated top-10 5bps leaderboard (145 evaluated):
#4 s456: +8,802% (ultra-robust: 5bps > 8bps, Sortino=6.71)
#6 s452: +8,002% (ultra-robust: 5bps > 8bps, Sortino=6.65)
#7 s734: +7,160% (ultra-robust: 5bps > 8bps)
#10 s446: +6,536% (ultra-robust: 5bps > 8bps)
#15 s827: +4,801% (ultra-robust: 5bps > 8bps)

7 sweeps ongoing: s201-900 at 55-62% complete

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
lee101 added a commit that referenced this pull request Mar 25, 2026
@lee101 @claude
crypto70 NEW ALL-TIME CHAMPION: s275 +23,595% ann at 5bps
d7d6f36 
Leaderboard updated (169 evaluated at 5bps):
#1 s275: +23,595% (Sortino=9.0, ultra-robust: 5bps > 8bps)
#2 s240: +17,642%
#3 s434: +10,359%
#4 s71:  +9,381%
#5 s456: +8,802% (new)
#6 s507: +8,273%
#7 s452: +8,002% (new)

Top-10 mean: +10,796% ann | 7/10 ultra-robust (5bps >= 8bps)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
lee101 added a commit that referenced this pull request Apr 13, 2026
@lee101 @claude
feat: crypto70 sweep s401-700 progress — s507 NEW #2 (+8290%), s404 NEW
79d3707 
#4 (+7295%)

Key discoveries:
- s507 (s501-600 range): +8290% ann at 5bps, Sortino=6.43, 5bps>8bps
- s404 (s401-500 range): +7295% ann at 5bps, Sortino=6.28, 5bps≈8bps≈10bps
- s602 (s601-700 range): +2594% ann at 5bps, Sortino=5.46
- s704 (s701-800 range): +1927% ann at 5bps, Sortino=5.36
- s507 and s404 saved to crypto70_champions/
- 46 seeds now ranked at 5bps (up from 34)
- Sweeps running: s121-200 (60/80), s201-300 (23/100), s301-400 (16/100),
  s401-500 (6/100), s501-600 (8/100), s601-700 (8/100), s701-800 (5/100)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
lee101 added a commit that referenced this pull request Apr 13, 2026
@lee101 @claude
sweep: crypto70 s401-900 new champions — s410 NEW #21 (+2945%@5bps), …
3b57876 
…s802 NEW #40 (+981%)

Additional champions beyond s404/s507/s609/s704 already committed:
- s410: +2945%@5bps, Sortino=5.76, 5bps>8bps very robust (second spike in s401-500)
- s802: +981%@5bps, Sortino=N/A (first s801-900 range find, very early)
- s506: +723%@5bps (solid but not exceptional)
- s801-900 sweep launched, monitor updated with champion save logic
- s701-800 monitor updated with champion save logic

Pattern confirmed: each 100-seed range produces 1-2 exceptional isolated spikes.
s401-500 is unusual with TWO champions (s404 #4 and s410 #21).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
lee101 added a commit that referenced this pull request Apr 13, 2026
@lee101 @claude
feat: crypto70 seed sweep s301-900 — s416 NEW TOP-10 (+4091%), s321 (…
0ff8b97 
…+3063%), s513/s804 ROBUST

New 5bps production champions discovered:
- s416 (s401-500): +4091% ann, Sortino=6.18, 5bps>8bps ROBUST — NEW #11!
- s321 (s301-400): +3063% ann, Sortino=5.07
- s513 (s501-600): +2525% ann, Sortino=5.50, 5bps>8bps ROBUST
- s804 (s801-900): +1943% ann, Sortino=5.19, 5bps>8bps ROBUST
- s192 (s121-200): +1886% ann, Sortino=4.67, 5bps>8bps ROBUST
- s415 (s401-500): +1757% ann, Sortino=4.79, 5bps>8bps ROBUST
- s807 (s801-900): +934% ann, Sortino=6.37, 5bps>8bps ROBUST (high Sortino!)
- s191 (s121-200): +1457% ann, Sortino=3.90, ROBUST

s401-500 is the HOTTEST range: s404(#4)+s416(#11)+s410(#21)+s415(#31) = 4 champions
Added crypto70_5bps_leaderboard.csv — automated 5bps production leaderboard

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
lee101 added a commit that referenced this pull request Apr 13, 2026
@lee101 @claude
crypto70: new champions s456(+8802%), s452(+8002%) enter top-6
c56d853 
Updated top-10 5bps leaderboard (145 evaluated):
#4 s456: +8,802% (ultra-robust: 5bps > 8bps, Sortino=6.71)
#6 s452: +8,002% (ultra-robust: 5bps > 8bps, Sortino=6.65)
#7 s734: +7,160% (ultra-robust: 5bps > 8bps)
#10 s446: +6,536% (ultra-robust: 5bps > 8bps)
#15 s827: +4,801% (ultra-robust: 5bps > 8bps)

7 sweeps ongoing: s201-900 at 55-62% complete

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
lee101 added a commit that referenced this pull request Apr 13, 2026
@lee101 @claude
crypto70 NEW ALL-TIME CHAMPION: s275 +23,595% ann at 5bps
84d4143 
Leaderboard updated (169 evaluated at 5bps):
#1 s275: +23,595% (Sortino=9.0, ultra-robust: 5bps > 8bps)
#2 s240: +17,642%
#3 s434: +10,359%
#4 s71:  +9,381%
#5 s456: +8,802% (new)
#6 s507: +8,273%
#7 s452: +8,002% (new)

Top-10 mean: +10,796% ann | 7/10 ultra-robust (5bps >= 8bps)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
lee101 added a commit that referenced this pull request Apr 18, 2026
@lee101 @claude
docs(xgb): packing-grid verdict — keep top_n=1, packing fails ship rule
620edc8 
Task #23 sweep (seed=2, 400/5/0.03, 34-window OOS) shows every top_n>1
cell is strictly worse than top_n=1 equal on BOTH sortino (Δ≤−0.57)
AND worst-DD (Δ≥+2.22). Mode axis (equal/softmax/score_norm) is flat
within each top_n (<0.2% median spread). Concentration on the 846-wide
argmax IS the edge; picks #2-#4 dilute without diversifying.

DD-reduction campaign (regime gate + vol-target + packing) conclusively
closed. Structural worst-DD floor for top_n=1 seed=2 is ~27.4%.
Leverage≤1.25 remains the only safe PnL lever until task #21 (hourly
intrabar sim) enables per-position stop-loss, or a different signal
family (shorts/pairs/crypto) adds genuine diversification.

Artifacts: analysis/xgbnew_packing_grid/top{1..4}_{equal,softmax,score_norm}/
Log: logs/xgb_packing_grid_20260418_112712.log

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
lee101 added a commit that referenced this pull request Apr 19, 2026
@lee101
fix(xgbnew/live_trader): enforce death-spiral guard on SELL, record b…
83b9a62 
…uy price (HARD RULE #3)

Prior to this fix xgbnew/live_trader.py submitted raw Alpaca MarketOrderRequests
without feeding either side of the death-spiral guard:

  - BUY path never called record_buy_price() → guard had no cost basis.
  - SELL path never called guard_sell_against_death_spiral() → runaway
    panic-sells at >50 bps below the most recent buy would NOT crash the loop.

This violated HARD RULE #3 in CLAUDE.md for any future live deploy.

Changes
-------
- BUY: poll the submitted order's filled_avg_price (new
  _poll_filled_avg_price helper, 30s timeout) and feed that to
  record_buy_price; fall back to last_close if the fill poll times out.
- SELL: new _get_position_details helper reads current_price +
  avg_entry_price from Alpaca's Position objects; guard fires with
  current_price before each submit. Guard RuntimeError propagates (the
  documented "crash the loop" behaviour), supervisor autorestart handles
  the rest.
- Both paths now import from src.alpaca_singleton so the state lives in
  the same alpaca_live_writer_buys.json that alpaca_wrapper uses.

Tests
-----
- tests/test_xgbnew_live_trader_guard.py (new, 6 tests): filled-avg-price
  poll happy path + cancel path, Position detail extraction, guard
  raise/allow round-trip with 50 bps tolerance, buy-price round-trip
  to disk.
- tests/test_alpaca_singleton.py (16 tests) still green — HARD RULE #4.
lee101 added a commit that referenced this pull request Apr 26, 2026
@lee101 @claude
alpaca_singleton: add force_live; xgb launch.sh hardens env + EOD flags
4fba0ec 
enforce_live_singleton now accepts force_live=True so entry points whose
own CLI has already selected live trading do not silently fall through
to a paper no-op when env_real.PAPER is unset or stale.  This pairs with
xgbnew.live_trader._enforce_live_startup_guards, which calls
require_explicit_live_trading_enable and then enforce_live_singleton
with force_live=True.  A new test covers the ALP_PAPER=1 + force_live=True
case so HARD RULE #4 stays satisfied.

deployments/xgb-daily-trader-live/launch.sh now unsets the
ALPACA_SINGLETON_OVERRIDE / ALPACA_DEATH_SPIRAL_OVERRIDE break-glass env
vars after sourcing ~/.secretbashrc so a leaked terminal override never
follows the supervisor unit, and enables the EOD deleveraging flags
(--eod-deleverage --eod-max-gross-leverage 2.0 ...) that are now
required by the preflight gate.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

codex

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@lee101