-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathslides.html
More file actions
367 lines (342 loc) · 18.4 KB
/
Copy pathslides.html
File metadata and controls
367 lines (342 loc) · 18.4 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Trading-With-Alpaca — Slides</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/reveal.js@5.1.0/dist/reset.css">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/reveal.js@5.1.0/dist/reveal.css">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/reveal.js@5.1.0/dist/theme/black.css">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/reveal.js@5.1.0/plugin/highlight/monokai.css">
<style>
.reveal h1, .reveal h2, .reveal h3 { text-transform: none; letter-spacing: 0; }
.reveal h1 { font-size: 2.0em; }
.reveal h2 { font-size: 1.5em; color: #4ec9b0; }
.reveal h3 { font-size: 1.1em; color: #c8c8c8; }
.reveal section { text-align: left; }
.reveal section.center { text-align: center; }
.reveal pre { width: 100%; box-shadow: none; font-size: 0.55em; }
.reveal pre code { padding: 12px; border-radius: 6px; max-height: 600px; }
.reveal table { font-size: 0.55em; margin: 0 auto; }
.reveal table th { color: #4ec9b0; border-bottom: 1px solid #4ec9b0; }
.reveal table td, .reveal table th { padding: 4px 10px; }
.reveal img { border: none; box-shadow: 0 4px 24px rgba(0,0,0,0.6); border-radius: 6px; max-height: 520px; }
.reveal .small { font-size: 0.65em; }
.reveal .tiny { font-size: 0.5em; }
.reveal code { color: #ce9178; }
.reveal a { color: #4ec9b0; }
.two-col { display: grid; grid-template-columns: 1fr 1fr; gap: 30px; align-items: start; }
.pill { display: inline-block; padding: 2px 10px; border-radius: 12px; background: #264f78; color: #9cdcfe; font-size: 0.7em; margin-right: 6px; }
.accent { color: #4ec9b0; }
.warn { color: #f48771; }
ul.tight li { margin: 4px 0; }
.footer { position: fixed; bottom: 12px; right: 18px; font-size: 0.5em; color: #666; }
.mermaid-fallback { font-family: ui-monospace, SFMono-Regular, Menlo, monospace; font-size: 0.6em; line-height: 1.4; color: #d4d4d4; }
</style>
</head>
<body>
<div class="reveal">
<div class="slides">
<!-- ============== TITLE ============== -->
<section class="center">
<h1>Trading-With-Alpaca</h1>
<h3>An Alpaca crypto pipeline whose code is <span class="accent">generated by an AG2 multi-agent system</span></h3>
<p class="small" style="margin-top: 60px;">
<span class="pill">AG2 / autogen</span>
<span class="pill">Gemini via OpenRouter</span>
<span class="pill">Alpaca crypto</span>
<span class="pill">Redis</span>
<span class="pill">SQLite</span>
<span class="pill">Flask</span>
</p>
</section>
<!-- ============== TWO LAYERS ============== -->
<section>
<h2>Two layers, one repo</h2>
<div class="two-col">
<div>
<h3>1. Codegen layer</h3>
<p class="small"><code>alpaca_codegen.py</code></p>
<ul class="tight small">
<li>AG2 multi-agent system</li>
<li>Researcher → Coder → Verifier</li>
<li>Emits the runtime pipeline</li>
<li>Bug fix = edit prompt + regen</li>
</ul>
</div>
<div>
<h3>2. Runtime pipeline</h3>
<p class="small"><code>generated/</code></p>
<ul class="tight small">
<li>Crypto market data ingest</li>
<li>Paper-trading client + CLI</li>
<li>Trade-event audit → SQLite</li>
<li>Flask web UI (Escalator)</li>
</ul>
</div>
</div>
<p class="small" style="margin-top: 30px;">
The runtime is itself a <span class="accent">reproducible LLM artifact</span>.
Bugs can be fixed as direct edits (fast) or as prompt edits + regen (durable).
</p>
</section>
<!-- ============== HIGH-LEVEL DIAGRAM ============== -->
<section>
<h2>High-level component diagram</h2>
<pre class="mermaid-fallback"><code>
[Codegen layer] [Runtime / generated/]
┌─────────────────────┐ ┌─────────────────────────────┐
│ Researcher ──┐ │ │ subscriber.py trader.py │
│ ▼ │ ── emits ──▶ │ cli.py webgui.py │
│ Coder ◀── Verifier│ │ trade_stream.py persister │
└─────────────────────┘ └─────────────────────────────┘
Alpaca crypto WS ──▶ subscriber ──▶ Redis ──▶ cli / webgui (read)
Alpaca trading WS ──▶ trade_stream ─▶ Redis Stream ──▶ persister ─▶ SQLite
cli / webgui ─── REST ───▶ trader ───▶ Alpaca trading REST (write)
</code></pre>
<p class="small">
Each long-running worker is independent — any can crash and restart.
Redis is the shared substrate; the Stream + consumer-group is what makes
persister safe to resume.
</p>
</section>
<!-- ============== AGENTS ============== -->
<section>
<h2>Codegen — the three agents</h2>
<table>
<thead>
<tr><th>Agent</th><th>AG2 class</th><th>Role</th></tr>
</thead>
<tbody>
<tr>
<td><span class="accent">Researcher</span></td>
<td><code>AssistantAgent</code></td>
<td>Writes <code>RESEARCH.md</code> — Alpaca crypto WS + REST brief. Run once, cached; <code>--force-research</code> to refresh.</td>
</tr>
<tr>
<td><span class="accent">Coder</span></td>
<td><code>AssistantAgent</code></td>
<td>Per stage, emits the full source of one target file in one fenced block. Sees the brief + earlier-stage files.</td>
</tr>
<tr>
<td><span class="accent">Verifier</span></td>
<td><code>UserProxyAgent</code> + <code>LocalCommandLineCodeExecutor</code></td>
<td>Python: <code>py_compile</code> + <code>import</code> in temp dir. HTML: sanity check. Errors loop back to Coder, up to 5 attempts.</td>
</tr>
</tbody>
</table>
<p class="small" style="margin-top: 24px;">
<strong>No GroupChat.</strong> Orchestration is manual in <code>run_stage()</code> —
each Coder turn is a single-turn <code>initiate_chat(... max_turns=1)</code>; the
Verifier runs synchronously and feeds errors back as the next message.
</p>
</section>
<!-- ============== STAGES ============== -->
<section>
<h2>Stages — ordered, contextual, retried</h2>
<table>
<thead>
<tr><th>#</th><th>Stage</th><th>Target</th><th>Lang</th><th>Priors (read-only context)</th></tr>
</thead>
<tbody>
<tr><td>0</td><td><code>subscriber</code></td><td><code>subscriber.py</code></td><td>python</td><td>—</td></tr>
<tr><td>1</td><td><code>trader</code></td><td><code>trader.py</code></td><td>python</td><td>—</td></tr>
<tr><td>2</td><td><code>cli</code></td><td><code>cli.py</code></td><td>python</td><td>subscriber, trader</td></tr>
<tr><td>3</td><td><code>trade_stream</code></td><td><code>trade_stream.py</code></td><td>python</td><td>trader</td></tr>
<tr><td>4</td><td><code>persister</code></td><td><code>persister.py</code></td><td>python</td><td>trade_stream</td></tr>
<tr><td>5</td><td><code>webgui_html</code></td><td><code>web/index.html</code></td><td>html</td><td>—</td></tr>
<tr><td>6</td><td><code>webgui</code></td><td><code>webgui.py</code></td><td>python</td><td>subscriber, trader, persister, web/index.html</td></tr>
</tbody>
</table>
<p class="small" style="margin-top: 20px;">
HTML was split out — emitting Flask backend + ~500-line HTML/CSS/JS in
one response was hitting the <span class="warn">32k output-token boundary</span>.
Splitting halved the per-stage burden.
</p>
</section>
<!-- ============== INVOCATION ============== -->
<section>
<h2>Driving the codegen</h2>
<pre><code class="language-bash">python alpaca_codegen.py # generate any missing files
python alpaca_codegen.py --stage all # regenerate everything
python alpaca_codegen.py --stage subscriber # one stage
python alpaca_codegen.py --stage trader BTC/USD ETH/USD # custom symbols
python alpaca_codegen.py --force-research # also redo RESEARCH.md
python alpaca_codegen.py --fix err.txt --stage cli # feed traceback back</code></pre>
<p class="small" style="margin-top: 24px;">
Default mode is <code>missing</code> — only stages whose target is absent.
On final retry failure, the last attempt is still written to disk for inspection.
</p>
<p class="small">
Model: Gemini via OpenRouter (default <code>google/gemini-3.1-pro-preview</code>,
override with <code>GEMINI_MODEL</code>).
</p>
</section>
<!-- ============== DURABLE RULES ============== -->
<section>
<h2>Durable rules — institutional memory for the LLM</h2>
<p class="small">
When a runtime bug recurs after a regen, the right fix is to add a rule to the
<code>CODER_PREAMBLE</code> "Durable rules" block — not to keep patching the generated file.
</p>
<ul class="tight small">
<li><code>CryptoDataStream.run()</code> is <span class="accent">sync</span> — never <code>await</code> it; use <code>await stream._run_forever()</code> from inside <code>asyncio.run(main())</code>.</li>
<li>There is <span class="warn">no <code>CryptoExchange</code> enum</span> — exchange fields are plain strings.</li>
<li>Redis <code>HSET</code> rejects <code>None</code> — filter via a <code>_clean()</code> helper.</li>
<li>Alpaca crypto orderbook WS sends <span class="accent">deltas, not snapshots</span> — merge into per-side hashes (<code>size>0</code> upsert, <code>size==0</code> delete).</li>
<li>For crypto symbols use <code>TimeInForce.GTC</code> or <code>IOC</code> — <code>DAY</code> is rejected.</li>
</ul>
</section>
<!-- ============== RUNTIME — TWO HALVES ============== -->
<section>
<h2>Runtime — two halves</h2>
<div class="two-col">
<div>
<h3>Half 1 · Market data (read)</h3>
<p class="small">
<code>subscriber.py</code> → <code>CryptoDataStream</code> → trades, quotes, L2 orderbook.
Writes Redis: <code>alpaca:latest:<SYM></code>, <code>alpaca:recent:<SYM>:trade|quote</code>,
<code>alpaca:ob:bids|asks:<SYM></code>, <code>alpaca:symbols</code>.
</p>
<p class="small">
<code>cli.py</code> and <code>webgui.py</code> read Redis only — no Alpaca call on the read path.
</p>
</div>
<div>
<h3>Half 2 · Trading (write) + audit</h3>
<p class="small">
<code>trader.py</code> — sync wrapper over <code>TradingClient</code> (REST), shared by CLI + webgui.
</p>
<p class="small">
<code>trade_stream.py</code> → <code>TradingStream</code> WS → <code>XADD alpaca:trade_updates</code> →
<code>persister.py</code> drains via consumer group → SQLite.
</p>
<p class="small accent">
The split is the point: persister can crash and resume from the last acked id without losing events.
</p>
</div>
</div>
</section>
<!-- ============== REDIS SCHEMA ============== -->
<section>
<h2>Redis schema</h2>
<table>
<thead>
<tr><th>Key / Stream</th><th>Type</th><th>Writer</th><th>Reader</th></tr>
</thead>
<tbody>
<tr><td><code>alpaca:latest:<SYM></code></td><td>Hash</td><td>subscriber</td><td>cli <code>latest</code>, webgui fallback</td></tr>
<tr><td><code>alpaca:recent:<SYM>:trade</code></td><td>List</td><td>subscriber</td><td>cli <code>recent</code></td></tr>
<tr><td><code>alpaca:recent:<SYM>:quote</code></td><td>List</td><td>subscriber</td><td>cli <code>recent</code></td></tr>
<tr><td><code>alpaca:orderbook:<SYM></code></td><td>Hash</td><td>subscriber</td><td>webgui (metadata only)</td></tr>
<tr><td><code>alpaca:ob:bids:<SYM></code></td><td>Hash</td><td>subscriber</td><td>webgui <code>/api/orderbook</code></td></tr>
<tr><td><code>alpaca:ob:asks:<SYM></code></td><td>Hash</td><td>subscriber</td><td>webgui <code>/api/orderbook</code></td></tr>
<tr><td><code>alpaca:symbols</code></td><td>Set</td><td>subscriber</td><td>cli <code>symbols</code>, webgui</td></tr>
<tr><td><code>alpaca:trade_updates</code></td><td>Stream</td><td>trade_stream</td><td>persister</td></tr>
</tbody>
</table>
<p class="small" style="margin-top: 20px;">
Orderbook is <span class="accent">delta-merged</span>: <code>size>0</code> upserts, <code>size==0</code> deletes.
Webgui sorts and slices to top-10 per side at read time.
</p>
</section>
<!-- ============== UI: ESCALATOR ============== -->
<section class="center">
<h2>Web UI · Escalator</h2>
<p class="small">10 ask rows above 10 bid rows · click a row → pre-filled order modal</p>
<img src="images/escalator.png" alt="Escalator depth ladder">
</section>
<!-- ============== UI: ACTIVITY ============== -->
<section class="center">
<h2>Web UI · Activity</h2>
<p class="small">Recent orders + fills from SQLite (with $ notional)</p>
<img src="images/activity.png" alt="Activity tab">
</section>
<!-- ============== UI: SUMMARY ============== -->
<section class="center">
<h2>Web UI · Summary</h2>
<p class="small">Account snapshot, positions, realized PnL by symbol</p>
<img src="images/summary.png" alt="Summary tab">
</section>
<!-- ============== ORDER PLACEMENT FLOW ============== -->
<section>
<h2>Order placement + audit (one click → SQLite)</h2>
<pre class="mermaid-fallback"><code>
User clicks ASK row at price P (Escalator)
│
▼
Modal: side=BUY, type=limit, limit_price=P
│ POST /api/order { sym, buy, limit, P, qty, live:false }
▼
webgui.py ──▶ trader.buy_limit(...) ──▶ Alpaca REST (paper=true)
│ │
│ │ (independent path)
│ ▼
│ Alpaca WS ──▶ trade_stream.py
│ │ XADD
│ ▼
│ alpaca:trade_updates
│ │ XREADGROUP
│ ▼
│ persister.py
│ │
│ ▼
│ SQLite: trade_events,
│ orders (UPSERT),
│ fills (INSERT OR IGNORE)
▼
Activity tab later ◀── /api/orders, /api/fills ◀── webgui ◀── SQLite
</code></pre>
</section>
<!-- ============== INVARIANTS ============== -->
<section>
<h2>Invariants worth preserving across regens</h2>
<ol class="tight small">
<li><span class="accent">Paper is the default</span>. <code>live=True</code> / <code>--live</code> is the only opt-in. CLI prints a stderr warning.</li>
<li><span class="accent">No network at import time</span>. Env reads + connections happen inside <code>main()</code> / <code>get_client()</code>. This is what lets the verifier <code>import</code> each module without creds.</li>
<li><span class="accent">Independent restartability</span>. Each worker can crash and resume. Redis Stream + consumer-group acks make persister safe.</li>
<li><span class="accent">Defensive web rendering</span>. No webgui endpoint may 500 on missing data. Empty Redis → <code>[]</code>/<code>{}</code>; missing SQLite → <code>[]</code>; missing creds → null/empty.</li>
<li><span class="accent">Crypto TIF</span>. <code>GTC</code> or <code>IOC</code> only; never <code>DAY</code>. Use <code>qty=</code>, not <code>notional=</code>.</li>
<li><span class="accent">Orderbook merge</span>. Per-side hashes; delta semantics; never overwrite the whole book.</li>
</ol>
</section>
<!-- ============== RUN ============== -->
<section>
<h2>Run it</h2>
<pre><code class="language-bash">pip install -r requirements.txt
export ALPACA_API_KEY_ID=... ALPACA_API_SECRET_KEY=...
docker run -p 6379:6379 redis
python generated/subscriber.py # market data -> Redis
python generated/trade_stream.py # trade events -> Redis Stream
python generated/persister.py # Redis Stream -> SQLite
python generated/webgui.py # http://127.0.0.1:5000
# CLI alternative
cd generated
python cli.py symbols
python cli.py latest BTC/USD
python cli.py account
python cli.py buy BTC/USD 0.001 # paper by default; --live to opt in</code></pre>
</section>
<!-- ============== END ============== -->
<section class="center">
<h1>Thanks</h1>
<p class="small" style="margin-top: 40px;">
<code>README.md</code> · <code>DESIGN.md</code> · <code>CLAUDE.md</code>
</p>
<p class="small">github.com — <code>TradingWithAlpaca</code></p>
</section>
</div>
</div>
<script src="https://cdn.jsdelivr.net/npm/reveal.js@5.1.0/dist/reveal.js"></script>
<script src="https://cdn.jsdelivr.net/npm/reveal.js@5.1.0/plugin/highlight/highlight.js"></script>
<script src="https://cdn.jsdelivr.net/npm/reveal.js@5.1.0/plugin/notes/notes.js"></script>
<script>
Reveal.initialize({
hash: true,
slideNumber: 'c/t',
transition: 'fade',
plugins: [ RevealHighlight, RevealNotes ]
});
</script>
</body>
</html>