Shadow Dealer — a byte-exact C++ port of a paper market-making engine, and what latency actually buys

What. A byte-exact C++20 twin of a Python paper market-making measurement engine (python/, the read-only reference) — its replay journals are byte-identical to the original over a pinned tape (2 582 846 records, 5 symbols, 0 diffs) — deployed live on a cloud VPS. Why. To prove a faithful port and answer, on a real venue, what reacting faster actually buys a market maker. Finding. Almost nothing: the dealer already quotes at the venue's feed-granularity floor, and its adverse selection is a structural feed/volatility tax that speed cannot shave.

① The dealer sits at the feed floor. Reaction floor ~2 ms ≪ effective feed-update granularity ~20–140 ms (10–70×). No architecture on this venue can quote fresher than the feed updates; the residual adverse selection is a feed/volatility tax, not a latency tax.

② Speed buys volume, not per-fill quality. On a common, undelayed yardstick, adverse selection concentrates in the shared flow both speeds take (−3.30 ht, latency-immune by construction); neither arm's marginal flow is materially toxic. A faster config wins +13 % more fills, not cleaner ones. (Two measurement artifacts were caught before publication — see CPP-REPORT7.md §6.)

Live paper dealer (read-only dashboard, IP-redacted). The P&L is AS-dominated — that is the finding. Reproduce the counterfactual on any settled window with one command: README-analysis.md. Findings: FINDINGS-CPP.md · full method + adversarial review: CPP-REPORT7.md.

Paper-only, forever. No API keys, no orders, no order-capable code path — same invariant as the reference (ARCHITECTURE.md §4). This repo only reads a recorded tape and computes.

Status — live C++ dealer + replay-equivalence gate + counterfactual latency study (S7)

The Stage-1 exit gate is green: the C++ replay shell reproduces the committed canonical journals byte-for-byte over the full pinned hour, all 5 symbols — 2 582 846 tape records in, 104 053 journal records compared, 0 diffs (beliefs, decisions, and serialization alike). See Stage-1 exit gate below.

Ported and conformance-verified against the actual reference (golden vectors driven through the reference classes, not hand transcriptions):

Module	What	Plane	Reference
cpp/engine/types	int64 aliases, to_half_ticks (banker's round-half-even), SymbolConfig	decision (int, exact)	dealer/accounting.py, params.yaml
cpp/engine/json	python_repr (byte-exact CPython float repr) + compact JsonWriter	serialization (byte-exact)	capture/writer.py
cpp/engine/records	quote / fill / markout / pnl / identity_breach structs — exact field names + order	serialization	dealer/instance.py
cpp/engine/accounting	Ledger: SC=ε(m−p)>0, AS, IC, identity SC+AS+IC==PnL in integer half-ticks	decision (int, exact)	dealer/accounting.py
cpp/engine/fills	QuoteBook + FillEngine: strict-through P/C, cooldown, position cap, quote pairing	decision (int, exact)	dealer/fills.py
cpp/engine/toxicity	Toxicity α̂ EWMA estimator (markout / imbalance / liq-proximity)	belief (float)	dealer/toxicity.py
cpp/engine/filter (new)	GridFilter grid-posterior fair-value filter + microprice (predict / micro / trade / regrid)	belief (float)	dealer/filter.py
cpp/engine/gm	Glosten-Milgrom fixed-point quoting + inventory skew + half-tick emission	belief + decision	dealer/gm.py
cpp/engine/instance (new)	DealerInstance: data-time clock, cycles, quoting, fill-detect, markouts, snapshots, warm restore — emits records into an abstract JournalSink (still ZERO I/O)	belief + decision	dealer/instance.py
cpp/engine/resume (new)	resume_state — the full nested engine snapshot, byte-exact	serialization	dealer/instance.py::_snapshot
cpp/shells/ replay shell (new)	gz tape reader, (rx, priority, seq) merge, gz-member journal writer (hourly rotation, 500-record flush), warm-resume reader, config loader, run_replay	I/O (all of it)	dealer/app.py, tailer.py, journal.py, config.py, capture/writer.py

The engine's zero-I/O property is enforced at configure time: CMake fails the build if any cpp/engine/ source includes a stream header, names an iostream symbol, or calls a C file-I/O function. Every file operation lives in cpp/shells/.

Conformance model — canonical scalar-ordered engine (Option A, harness-side)

The D6 experiment (session 2) proved the original BLAS-vs-ordered two-plane gate impossible: a C++ ordered-loop port cannot be decision-for-decision byte-exact against the BLAS reference (≥528 root flips — tiny ≤9-ULP per-op seeds amplify through the filter/gm discrete branch points). The owner ruled Option A, harness-side (CPP-REPORT2 §3.4): the conformance reference is the canonical scalar-ordered engine — every reduction an ascending-index float64 loop, every transcendental scalar libm — installed over the frozen reference at replay time by vectors/canonical_harness.py (monkeypatch; reference untouched). The C++ engine mirrors it op-for-op, so the gate is now bit-exact end-to-end: belief floats (v/sd/c/d/p[], bid_c/ask_c) at 0 ULP and decision integers exact. The BLAS delta is measured and documented (canonical vs BLAS = 758 root flips), not absorbed; the archived BLAS journals live under journals_blas_d6/ as the D6 artifact.

Validity controls (all byte-identical, decoded records, full slice): stock BLAS replay == archived BLAS; identity-backend replay == stock (transcription faithful); two canonical replays == each other (determinism); a fresh canonical replay == the committed journals/ (ground-truth binding). See vectors/CONFORMANCE.md and CPP-REPORT3.

Stage-1 exit gate (S4)

Full pinned hour, all 5 symbols, C++ replay vs the committed canonical journals. Parity unit is the decoded JSON record (gzip container bytes are not a target). 0 diffs.

symbol	tape records	quotes	fills+markouts	pnl+resume_state	diffs
BTCUSDT	852 916	15 179	6 224	1 080	0
ETHUSDT	1 043 808	15 917	12 108	1 080	0
SOLUSDT	335 749	14 591	1 532	1 080	0
XRPUSDT	190 265	14 681	2 244	1 080	0
DOGEUSDT	160 108	14 597	1 580	1 080	0
total	2 582 846	74 965	23 688	5 400	0

Controls: (a) two independent runs byte-identical; (b) kill-mid-replay → restart → warm-resume is byte-identical to the reference's warm-resume on all 5 symbols (see the note below); (c) the full test suite and a 5-symbol replay are clean and byte-identical under ASan/UBSan as well as RelWithDebInfo.

Reference defect found by the resume control. The reference's own warm-resume is not bit-identical to its uninterrupted run: resume_from = last_rx skips the record at last_rx, whose payload was never in the anchor (the snapshot fires in the cycle phase, before dispatch), so one update_micro is dropped per restart. Beliefs drift on every symbol and, on knife-edge symbols, decisions flip — SOL loses 6 fills (192→186), DOGE 2 (198→196). The reference's own crash-resume test misses it (synthetic tape, integer-only asserts). Per owner ruling the port reproduces this byte-for-byte for the gate, and a corrected anchor then landed as a separate post-gate commit — so this port does guarantee warm-resume == uninterrupted (test_resume_invariant, 104 053 records, all 5 symbols, beliefs + fills), while --reference-anchor keeps the reference-faithful behaviour reproducible. Details in CONFORMANCE.md and CPP-REPORT4 §3.3 / §6.

Performance — per decision (owner ruling, CPP-REPORT5 §6)

A decision is a process() call that crosses a quote-cadence boundary: the calls that run the full path predict_to → maybe_regrid → gm solve → quote emission and record serialization. Both sides measured identically over the same pinned hour, into a null journal sink (compute + serialization, no gz framing, no file I/O).

symbol	decisions	C++ p50	C++ p99	C++ max	Python p50	Python p99	speedup
BTCUSDT	13 802	6.21 µs	30.5 µs	65.5 µs	141.5 µs	318.3 µs	22.8×
ETHUSDT	14 280	6.17 µs	35.5 µs	77.1 µs	152.1 µs	382.3 µs	24.7×
SOLUSDT	13 774	6.75 µs	24.3 µs	90.6 µs	230.2 µs	576.8 µs	34.1×
XRPUSDT	13 121	6.83 µs	25.9 µs	72.5 µs	231.0 µs	689.8 µs	33.8×
DOGEUSDT	12 513	6.79 µs	25.7 µs	99.8 µs	229.9 µs	747.6 µs	33.9×

Per decision the engine is 23–34× the Python reference (p50 6.2–6.8 µs vs 141–231 µs).

Per-event, for continuity with the Stage-1 appendix: C++ p50 0.46–0.50 µs vs Python 17.5–17.8 µs; all 5 symbols end-to-end ~3.3 s for 2 582 846 records including gz decode, JSON parse, engine, serialization and gz write. The flat sub-µs p50 is the bookTicker→update_micro path over the 101-point grid; the tail is the quote-cycle path (O(K²) Toeplitz convolution + GM fixed-point).

Live operating envelope, measured on a 2 h 5-symbol soak against real Binance: ~2 000–2 200 msg/s at ~2.7 % of one core, ~15 MB RSS, 4.41 GB/day of tape + journals (CPP-REPORT5 §5).

Live analysis — S7 counterfactual (internal; not published)

A paired counterfactual over the live tape (settled 3.62 h window, Asia+EU) asked what an incumbent-speed dealer (cadence 100 ms + modelled arrival delay) would have suffered vs the deployed C++ dealer (cadence 250 ms, ~2 ms arrival). Adverse selection is scored against a common undelayed book mid — the engine's internal as_ht grades each arm in its own belief frame and inverts the truth (a named finding, measurement-frame contamination, CPP-REPORT7 §4).

Measured values only, with provenance. Python-era analysis numbers are marked n/a — raw tape decommissioned (R3); the feed floor is a property of the venue feed, so it is era-invariant.

quantity	C++ (this deploy)	Python era	method / provenance
Reaction floor	~2.0 ms	n/a (tape R3)	rx−E p50 2.0 ms + ~40 µs compute; box glibc, 5-sym
Effective staleness (feed) floor	~20–140 ms (p95)	same (era-invariant)	bookTicker inter-update p95 by symbol = 10–70× reaction
True forward AS / fill	~−2.5 ht	n/a (tape R3)	common-reference markout @5 s, matched flow
— shared flow (both speeds)	−3.30 ht [−4.01,−2.56]	latency-immune by constr.	matched on trade id, 5-min block-boot CI
— C++-marginal flow	−1.36 ht [−2.65,−0.23]	—	not materially toxic (milder than shared)
Fills captured	+13 % vs incumbent	n/a (tape R3)	frame-independent volume
Per-decision compute	35–46 µs (box)	141–231 µs (measured)	null-sink, same pinned hour (CPP-REPORT5 §6)

Headline: the dealer already sits at the feed floor — no architecture on this venue quotes fresher than the feed updates. Adverse selection concentrates in the shared flow both speeds take (−3.30 ht), which is latency-immune by construction — a structural feed/volatility tax, not a latency tax. Speed buys volume (+13 % fills), not per-fill quality (neither arm's marginal flow is materially toxic). Two artifacts were caught before publication (measurement-frame contamination; difference-of-aggregates over non-nested sets — CPP-REPORT7 §6). See FINDINGS-CPP.md and CPP-REPORT7.md; re-run path in README-analysis.md.

Operations (deployed box)

• Dashboard is locked down: statusd binds 127.0.0.1:8787 only (no public listener); reach it via an SSH tunnel (ssh -f -N -L 8787:127.0.0.1:8787 shadow). It is read-only — the control endpoints are retired (POST /api/control → 410, no control.json write); it reads engine heartbeat + journals, never writes them. Details in STATUSD-NOTE.md.
• Re-run the S7 analysis on any settled window with one parameterized command — see README-analysis.md; the counterfactual controls live in analysis/.

Layout (assembled monorepo)

python/            the read-only Python reference (the spec): dealer/ + capture/ engine
                   source, params/config, ARCHITECTURE/DESIGN/FINDINGS/PROGRESS docs
cpp/               the byte-exact C++ port (build root — configure/build here)
  engine/          pure library, ZERO I/O (include/sd/*.hpp, src/*.cpp)
  shells/          the replay + live drivers: ALL file I/O + replay_main / live_main CLIs
  conformance/     GTest suite (byte-exactness gates)
  cmake/           dependency wiring (FetchContent: GTest, nlohmann, simdjson; find ZLIB)
  docs/reference/  frozen copies of ARCHITECTURE / DESIGN / FINDINGS / PROGRESS (spec)
vectors/           the CONFORMANCE KIT (committed ground truth): tape slice, fresh replay
                   journals, golden vectors, float corpus, MANIFEST.md — binds python↔cpp
analysis/          the S7 live-analysis pipeline (counterfactual + controls) + gate_check
deploy/            the solo box deploy kit (systemd unit, statusd, RUNBOOK, verify.sh)
archive/           Python-era essentials tarball — gitignored (never committed/published)

Build & test

Requires CMake ≥ 3.25, a C++20 compiler, and network access at configure time (FetchContent pulls pinned GoogleTest / nlohmann-json / simdjson; zlib comes from the platform SDK). Configure from cpp/ (the conformance kit is found at ../vectors):

cd cpp
cmake --preset default        # configure (RelWithDebInfo, strict -Wall -Wextra -Werror)
cmake --build build/default
ctest --preset default        # or: ./build/default/conformance/conformance_tests

# AddressSanitizer + UndefinedBehaviorSanitizer
cmake --preset asan && cmake --build build/asan && ctest --preset asan

The engine is built with -Wall -Wextra -Werror -Wshadow; third-party headers are SYSTEM so they never trip -Werror.

Byte-exactness gates (all green)

• test_float_repr — python_repr vs json.dumps(x) for 213,328 distinct doubles (every real journal float + edge cases + random bit patterns), reconstructed from IEEE-754 bits (parser-independent).

• test_json_parity — re-emitting real journal lines (quote/fill/markout/pnl) from the fresh replay reproduces the original bytes exactly (~17.5k lines) + synthetic null/edge cases.

• test_accounting — the C++ Ledger reproduces the actual Python Ledger over the golden vectors (hand-verified examples, a 600-fill random property path, snapshot/restore, spread-violation and identity-breach cases).

• test_types — to_half_ticks banker's-rounding parity + the 5 symbol tick sizes.

• test_fills — QuoteBook + FillEngine vs the actual dealer/fills.py: a 109-case fill-rule truth table (price-relation × side × P/C × cooldown × cap × suppressed-side × off-grid EPS boundary), strictly-earlier active_at, and 1600+ integrated active_at→check→commit sequence events.

• test_toxicity — Toxicity vs the actual dealer/toxicity.py over 11 scenarios / 1679 captured states, asserting the belief floats reproduce bit-exactly (max 0 ULP).

• test_filter — GridFilter vs the actual dealer/filter.py (canonical backend) over 10 scenarios / 15 104 steps: belief floats (mean/sd/c/d/p) reproduce max 0 ULP, counters exact.

• test_gm — dealer/gm.py (canonical) over 556 cases: bid_c/ask_c max 0 ULP; all decision integers + fixed-point (iters,cycled) exact.

• test_chain — chained integration: a fresh GridFilter driven through the recorded canonical per-symbol event stream (5 symbols, 100 000 filter-ops, 2314 warm gm decisions, 56 snapshots) reproduces beliefs 0 ULP and gm decisions bit-exact over the pinned slice.

• test_resume_invariant — a full kill-mid-replay → restart → warm-resume run reproduces an uninterrupted run exactly: 104 053 decoded records, all 5 symbols, beliefs and fills alike (the guarantee this port adds over the reference — see the resume note above).

39/39 tests green in both build configs (RelWithDebInfo + ASan/UBSan). See vectors/MANIFEST.md for provenance/SHA256s, vectors/CONFORMANCE.md for the canonical gate, and CPP-REPORT1/2/3.md for the session reports.