A realistic order-book simulator, classical optimal-execution benchmarks, and a from-scratch PPO agent — built to find out whether reinforcement learning actually beats classical execution once the backtest stops lying.
It doesn't. And the interesting part of this project is how the honest answer was reached — the simulator realism, the baselines, and the ablations that turned an apparent 2.4 bps RL win into a 0.6 bps loss against a one-line strategy.
Transaction-cost analysis is where trading strategies quietly live or die, and it is exactly where backtests are easiest to fool yourself with. This repo is a study in backtest hygiene: slippage, queue position, market impact, fees, and latency modeled explicitly, and every headline result stress-tested against the simplest baseline that could explain it away.
On 16 days of real Binance BTCUSDT data, evaluated out-of-sample with purged walk-forward splits:
| Comparison | Result | Verdict |
|---|---|---|
| PPO vs taker TWAP | −2.42 bps [−2.71, −2.11], wins 8/9 buckets | Looks like a clean RL win ✅ |
| PPO vs passive TWAP (one line of code) | +0.60 bps [+0.39, +0.81], wins 0/9 buckets | RL loses ❌ |
The agent's entire "edge" over the classical benchmarks was fee savings from posting instead of crossing (~2.3 bps) — and it captured them slightly worse than a trivial passive scheduler. Every benchmark in the naïve comparison was a pure taker, so PPO got credit for discovering nothing more than "post, don't cross."
The defensible claim: PPO does not beat classical execution on this simulator — it approximates a passive TWAP, marginally worse. That is a more useful result than a flattering one, and it is the kind of finding that survives an interviewer asking "what was your baseline?"
Most student trading projects chase alpha. Almost none touch execution — the other half of trading, and the half that separates people who have run a real backtest from people who have not. This lab demonstrates:
- Backtest realism — an event-driven L2 replay simulator with market impact, order-book resilience, queue-position fills, maker/taker fees, and latency, each independently toggleable.
- Classical foundations — TWAP, VWAP, POV, and closed-form Almgren–Chriss with a risk-aversion sweep that traces a genuine efficient frontier.
- RL, implemented and interrogated — PPO written from scratch (~300 lines of PyTorch, no stable-baselines) so every design choice can be defended, then evaluated with purged walk-forward splits and bootstrap confidence intervals.
- Intellectual honesty — an ablation study quantifying exactly how much a naïve simulator overstates performance, plus a documented trail of the modeling bugs that would have inverted the conclusions if left unfixed.
All figures are out-of-sample on real BTCUSDT (ADV ≈ 17,590 BTC/day ≈ $1.1B; daily vol ≈ 1.78%).
| Strategy | Mean | Std | CVaR₉₅ | Notes |
|---|---|---|---|---|
| TWAP | 7.40 | 13.2 | 39.1 | reference |
| Almgren–Chriss (λ=1e-5) | 8.48 | 8.7 | 30.1 | +1.1 bps mean buys a 34% cut in tail risk (p=0.016) |
| POV(10%) | 19.77 | 26.1 | 84.1 |
Almgren–Chriss with λ=0 reproduces TWAP to ~1e-14, validating the closed form. The λ sweep traces a clean cost/variance frontier — the textbook result, on real data.
Fitting ΔP/P = Y·σ·(Q/ADV)^δ:
| Estimator | δ (impact exponent) | R² |
|---|---|---|
| Naïve per-observation log-log fit | 0.14 | 0.03 |
| Binned fit (literature-standard) | 0.57 | 0.93 |
The naïve fit "disproves" the square-root law; the correct estimator recovers δ consistent with 0.5 in both volatility regimes. The methodology is the finding — see the notebook.
Train an agent in a simulator missing one realism feature, then score it in the full simulator:
| Ablated feature | Agent believes it earns | Agent actually costs | Overstatement |
|---|---|---|---|
| No queue position | −0.32 bps (a profit!) | 4.85 bps | 5.2 bps |
| Naïve (all realism off) | −1.35 bps (a profit!) | 3.65 bps | 5.0 bps |
A careless backtest reports execution as profitable. It costs ~4 bps. And the distortion is largest for queue position — precisely the feature the passive strategy exploits.
src/execlab/
├── sim/ # the centerpiece: event-driven L2 replay simulator
│ ├── simulator.py # order matching, book-walking, impact, latency
│ ├── impact.py # square-root law (metaorder-level, slicing-invariant)
│ ├── fills.py # passive queue-position model
│ ├── fees.py # maker/taker fees
│ └── latency.py # stale market data + delayed orders
├── strategies/ # TWAP, VWAP, POV, Almgren–Chriss (closed form + calibration)
├── rl/ # PPO from scratch, execution env, training loop
│ ├── ppo.py # clipped surrogate, GAE, Beta policy — ~300 lines
│ ├── env.py # scale-free state, per-step shortfall reward
│ └── agent.py # training + strategy adapter
├── evaluation/ # purged walk-forward splits (no lookahead)
├── calibration/ # empirical square-root-law fitting
├── data/ # Binance capture + historical reconstruction + synthetic
└── experiments/ # Hydra-driven runners for every phase
Every modeling assumption — and every way the simulator is deliberately optimistic — is documented in DESIGN.md.
- No lookahead. VWAP profiles and Almgren–Chriss parameters are calibrated only on days strictly before the evaluation window. RL uses purged, embargoed walk-forward splits (features look backward, so leakage is blocked in both directions).
- Paired, drift-controlled comparisons. Every strategy runs the same parent orders on the same seeds; shortfall is dominated by shared price drift, so a paired bootstrap is the only honest test.
- Impact is calibrated from the data, not assumed — and applied at the metaorder level so it is invariant to how an order is sliced.
- Reproducible. Seeded throughout; config-driven with Hydra; each run records its resolved config.
- 137 tests,
ruffclean, GitHub Actions CI — including tests that pin the invariants which caught real bugs (impact must not scale with slice count; the benchmark set must not be taker-only).
Each of these produced a confident, plausible, wrong result before it was caught. They are documented as "recorded failures" in DESIGN.md because catching them is the actual skill:
- Per-child square-root impact made shortfall grow with slice count (4.6 → 90 bps on an identical order) — the simulator rewarded dumping the whole order at once and punished every execution algorithm. Fixed by modeling impact at the metaorder level.
- Taker-only benchmarks made the RL agent look like it beat classical execution. Adding one passive baseline inverted the result (this is the headline above).
- A timestamp-unit bug silently compressed each day of market data by 1000×. Now caught by a span assertion.
- A level-index depth profile captured ~1% of BTCUSDT's real liquidity; the fallback default overstated near-touch depth ~10×. Depth is now measured as a cumulative curve in basis points.
pip install -e ".[dev,rl]"
# Offline: everything runs on a seeded synthetic market (no network, no keys)
python -m execlab.experiments.run_benchmarks
pytest
# Real data: download Binance history, calibrate depth from a live capture, run all phases
python -m execlab.data.downloader --symbol BTCUSDT --dates 2026-07-10 2026-07-11 ...
execlab-capture --symbol BTCUSDT --minutes 30 --levels 5000 --profile-out data/profiles/BTCUSDT.json
bash scripts/run_real_data.sh BTCUSDT runs/real| Command | What it does |
|---|---|
run_benchmarks |
TWAP/VWAP/POV/Almgren–Chriss over 540 parent orders |
run_rl |
Train PPO, evaluate with purged walk-forward + bootstrap CIs |
run_passive_check |
The decisive test: is the RL edge more than passivity? |
run_impact_study |
Fit the square-root impact law; compare to literature |
run_ablations |
Quantify how much each realism feature flatters the agent |
Results (CSVs + parquet) land under runs/<name>/, each with its resolved config.
Python 3.11 · NumPy · pandas · SciPy · PyTorch (PPO from scratch) · Hydra · pytest · ruff · GitHub Actions
Deep dive: DESIGN.md documents every simulator assumption and every recorded failure. Findings: FINDINGS_TEMPLATE.md.