Skip to content

Hellblazer704/execlab

Repository files navigation

execlab — an execution research lab

A realistic order-book simulator, classical optimal-execution benchmarks, and a from-scratch PPO agent — built to find out whether reinforcement learning actually beats classical execution once the backtest stops lying.

It doesn't. And the interesting part of this project is how the honest answer was reached — the simulator realism, the baselines, and the ablations that turned an apparent 2.4 bps RL win into a 0.6 bps loss against a one-line strategy.

Transaction-cost analysis is where trading strategies quietly live or die, and it is exactly where backtests are easiest to fool yourself with. This repo is a study in backtest hygiene: slippage, queue position, market impact, fees, and latency modeled explicitly, and every headline result stress-tested against the simplest baseline that could explain it away.


TL;DR — the headline finding

On 16 days of real Binance BTCUSDT data, evaluated out-of-sample with purged walk-forward splits:

Comparison Result Verdict
PPO vs taker TWAP −2.42 bps [−2.71, −2.11], wins 8/9 buckets Looks like a clean RL win ✅
PPO vs passive TWAP (one line of code) +0.60 bps [+0.39, +0.81], wins 0/9 buckets RL loses

The agent's entire "edge" over the classical benchmarks was fee savings from posting instead of crossing (~2.3 bps) — and it captured them slightly worse than a trivial passive scheduler. Every benchmark in the naïve comparison was a pure taker, so PPO got credit for discovering nothing more than "post, don't cross."

The defensible claim: PPO does not beat classical execution on this simulator — it approximates a passive TWAP, marginally worse. That is a more useful result than a flattering one, and it is the kind of finding that survives an interviewer asking "what was your baseline?"


Why this project

Most student trading projects chase alpha. Almost none touch execution — the other half of trading, and the half that separates people who have run a real backtest from people who have not. This lab demonstrates:

  • Backtest realism — an event-driven L2 replay simulator with market impact, order-book resilience, queue-position fills, maker/taker fees, and latency, each independently toggleable.
  • Classical foundations — TWAP, VWAP, POV, and closed-form Almgren–Chriss with a risk-aversion sweep that traces a genuine efficient frontier.
  • RL, implemented and interrogatedPPO written from scratch (~300 lines of PyTorch, no stable-baselines) so every design choice can be defended, then evaluated with purged walk-forward splits and bootstrap confidence intervals.
  • Intellectual honesty — an ablation study quantifying exactly how much a naïve simulator overstates performance, plus a documented trail of the modeling bugs that would have inverted the conclusions if left unfixed.

Results at a glance

All figures are out-of-sample on real BTCUSDT (ADV ≈ 17,590 BTC/day ≈ $1.1B; daily vol ≈ 1.78%).

Phase 2 — classical benchmarks (implementation shortfall vs arrival, bps)

Strategy Mean Std CVaR₉₅ Notes
TWAP 7.40 13.2 39.1 reference
Almgren–Chriss (λ=1e-5) 8.48 8.7 30.1 +1.1 bps mean buys a 34% cut in tail risk (p=0.016)
POV(10%) 19.77 26.1 84.1 ⚠️ 11% of fills hit the exhausted-book model — number not trusted

Almgren–Chriss with λ=0 reproduces TWAP to ~1e-14, validating the closed form. The λ sweep traces a clean cost/variance frontier — the textbook result, on real data.

Phase 4 — empirical market-impact law

Fitting ΔP/P = Y·σ·(Q/ADV)^δ:

Estimator δ (impact exponent)
Naïve per-observation log-log fit 0.14 0.03
Binned fit (literature-standard) 0.57 0.93

The naïve fit "disproves" the square-root law; the correct estimator recovers δ consistent with 0.5 in both volatility regimes. The methodology is the finding — see the notebook.

Phase 5 — how much does a naïve simulator lie?

Train an agent in a simulator missing one realism feature, then score it in the full simulator:

Ablated feature Agent believes it earns Agent actually costs Overstatement
No queue position −0.32 bps (a profit!) 4.85 bps 5.2 bps
Naïve (all realism off) −1.35 bps (a profit!) 3.65 bps 5.0 bps

A careless backtest reports execution as profitable. It costs ~4 bps. And the distortion is largest for queue position — precisely the feature the passive strategy exploits.


Architecture

src/execlab/
├── sim/            # the centerpiece: event-driven L2 replay simulator
│   ├── simulator.py    # order matching, book-walking, impact, latency
│   ├── impact.py       # square-root law (metaorder-level, slicing-invariant)
│   ├── fills.py        # passive queue-position model
│   ├── fees.py         # maker/taker fees
│   └── latency.py      # stale market data + delayed orders
├── strategies/     # TWAP, VWAP, POV, Almgren–Chriss (closed form + calibration)
├── rl/             # PPO from scratch, execution env, training loop
│   ├── ppo.py          # clipped surrogate, GAE, Beta policy — ~300 lines
│   ├── env.py          # scale-free state, per-step shortfall reward
│   └── agent.py        # training + strategy adapter
├── evaluation/     # purged walk-forward splits (no lookahead)
├── calibration/    # empirical square-root-law fitting
├── data/           # Binance capture + historical reconstruction + synthetic
└── experiments/    # Hydra-driven runners for every phase

Every modeling assumption — and every way the simulator is deliberately optimistic — is documented in DESIGN.md.


What makes it rigorous

  • No lookahead. VWAP profiles and Almgren–Chriss parameters are calibrated only on days strictly before the evaluation window. RL uses purged, embargoed walk-forward splits (features look backward, so leakage is blocked in both directions).
  • Paired, drift-controlled comparisons. Every strategy runs the same parent orders on the same seeds; shortfall is dominated by shared price drift, so a paired bootstrap is the only honest test.
  • Impact is calibrated from the data, not assumed — and applied at the metaorder level so it is invariant to how an order is sliced.
  • Reproducible. Seeded throughout; config-driven with Hydra; each run records its resolved config.
  • 137 tests, ruff clean, GitHub Actions CI — including tests that pin the invariants which caught real bugs (impact must not scale with slice count; the benchmark set must not be taker-only).

Bugs found and fixed — the honest engineering trail

Each of these produced a confident, plausible, wrong result before it was caught. They are documented as "recorded failures" in DESIGN.md because catching them is the actual skill:

  • Per-child square-root impact made shortfall grow with slice count (4.6 → 90 bps on an identical order) — the simulator rewarded dumping the whole order at once and punished every execution algorithm. Fixed by modeling impact at the metaorder level.
  • Taker-only benchmarks made the RL agent look like it beat classical execution. Adding one passive baseline inverted the result (this is the headline above).
  • A timestamp-unit bug silently compressed each day of market data by 1000×. Now caught by a span assertion.
  • A level-index depth profile captured ~1% of BTCUSDT's real liquidity; the fallback default overstated near-touch depth ~10×. Depth is now measured as a cumulative curve in basis points.

Quickstart

pip install -e ".[dev,rl]"

# Offline: everything runs on a seeded synthetic market (no network, no keys)
python -m execlab.experiments.run_benchmarks
pytest

# Real data: download Binance history, calibrate depth from a live capture, run all phases
python -m execlab.data.downloader --symbol BTCUSDT --dates 2026-07-10 2026-07-11 ...
execlab-capture --symbol BTCUSDT --minutes 30 --levels 5000 --profile-out data/profiles/BTCUSDT.json
bash scripts/run_real_data.sh BTCUSDT runs/real
Command What it does
run_benchmarks TWAP/VWAP/POV/Almgren–Chriss over 540 parent orders
run_rl Train PPO, evaluate with purged walk-forward + bootstrap CIs
run_passive_check The decisive test: is the RL edge more than passivity?
run_impact_study Fit the square-root impact law; compare to literature
run_ablations Quantify how much each realism feature flatters the agent

Results (CSVs + parquet) land under runs/<name>/, each with its resolved config.


Tech

Python 3.11 · NumPy · pandas · SciPy · PyTorch (PPO from scratch) · Hydra · pytest · ruff · GitHub Actions

Deep dive: DESIGN.md documents every simulator assumption and every recorded failure. Findings: FINDINGS_TEMPLATE.md.

About

Execution research lab: realistic L2 replay simulator, classical optimal-execution benchmarks (TWAP/VWAP/POV/Almgren-Chriss), and a from-scratch PPO agent — with honest, ablation-tested findings.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors