KuCoin order-book stream from a private trading project: KuCoin's 10 ms increment feed, 500-level shadow book, decision-relevant publish gating. One dependency (aiohttp). The feed is public market data.
From the trading box this file was pulled out of (Tokyo VPS, ~170 pairs subscribed, weeks of continuous runtime):
| Metric | Value |
|---|---|
| Processing latency | around 25 µs each call on my PC + ~80% less load as 80% of order book updates don't pass the requirements |
| CPU | ~0.05 lightsail cores |
| RAM | ~150 MB |
| Pairs per socket | 90 (shard limit) |
| Feed | increment@10ms, 500-level snapshot |
- Shards. Pairs split into 90-pair sockets. Each shard reconnects on its own exponential backoff (0.25 s doubling to a 30 s cap), and the feed reports healthy only when every shard of the current generation is healthy.
- One send budget per socket. Subscribes, unsubscribes, recovery resubscribes, and keepalive pings all share a single lock and a sliding window of 90 sends per 10 s, kept under the official 100 per 10 s limit. A recovery storm can never starve the pings, and vice versa.
- Shadow book, sorted once. The 500-level snapshot is sorted a single time. Deltas remove and insert only the touched prices with bisect.
- Publish gating. A delta publishes downstream only if something a trading decision could depend on changed: the two best prices, or the depth-walk outcome for the largest deployable size. Churn deeper than that walk can never alter a trade, so it's dropped. A refill inside the walk (a shallow-depth refusal becoming viable) changes the signature and republishes.
- Sequence integrity. A gap (start greater than previous + 1) invalidates the cached book, drops it, and forces an unsubscribe plus resubscribe.
pip install -e . # or just: pip install aiohttp
python example.py # BTC-USDT + ETH-USDT, 20 prints or 60 s
python example.py SOL-USDT --max-updates 5 --seconds 30Optional: pip install orjson for faster parsing.
Offline, recorded frames under tests/fixtures:
python -m unittest discover -s tests -t .Covers the three decision-heavy components: bisect book inserts (a delta touches only the changed levels), the sequence-gap to invalidation to resubscribe path, and the publish-gating predicate.
fast_market_stream.py: the stream itselflogger.py,core/profiler.py: stdlib stand-ins for the private project's logger and profiler (same call surface, no dependencies)reference_cache.py: minimal in-memory book cache with the private cache's call contractexample.py: live smoke run against the public feedtests/: offline tests on recorded frames
MIT, see LICENSE.