Skip to content

Commit d0889f3

Browse files
PigbibiPigbibi
committed
Add a US equity live switch runbook
1 parent 4ff8cf1 commit d0889f3

6 files changed

Lines changed: 553 additions & 0 deletions

‎README.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,8 @@ For the stricter end-state rules for new US equity strategies, see:
4848
- [`docs/us_equity_execution_translation_spec.zh-CN.md`](./docs/us_equity_execution_translation_spec.zh-CN.md)
4949
- [`docs/us_equity_release_cutover_plan.md`](./docs/us_equity_release_cutover_plan.md)
5050
- [`docs/us_equity_release_cutover_plan.zh-CN.md`](./docs/us_equity_release_cutover_plan.zh-CN.md)
51+
- [`docs/us_equity_live_switch_runbook.md`](./docs/us_equity_live_switch_runbook.md)
52+
- [`docs/us_equity_live_switch_runbook.zh-CN.md`](./docs/us_equity_live_switch_runbook.zh-CN.md)
5153

5254
## Package layout
5355

@@ -148,6 +150,8 @@ Cloud Run and self-hosted runner deployments should continue to deploy the strat
148150

149151
- [`docs/us_equity_cross_platform_strategy_spec.md`](./docs/us_equity_cross_platform_strategy_spec.md)
150152
- [`docs/us_equity_cross_platform_strategy_spec.zh-CN.md`](./docs/us_equity_cross_platform_strategy_spec.zh-CN.md)
153+
- [`docs/us_equity_live_switch_runbook.md`](./docs/us_equity_live_switch_runbook.md)
154+
- [`docs/us_equity_live_switch_runbook.zh-CN.md`](./docs/us_equity_live_switch_runbook.zh-CN.md)
151155

152156
### 目录结构
153157

‎README.zh-CN.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,8 @@
3737
- [`docs/us_equity_execution_translation_spec.zh-CN.md`](./docs/us_equity_execution_translation_spec.zh-CN.md)
3838
- [`docs/us_equity_release_cutover_plan.md`](./docs/us_equity_release_cutover_plan.md)
3939
- [`docs/us_equity_release_cutover_plan.zh-CN.md`](./docs/us_equity_release_cutover_plan.zh-CN.md)
40+
- [`docs/us_equity_live_switch_runbook.md`](./docs/us_equity_live_switch_runbook.md)
41+
- [`docs/us_equity_live_switch_runbook.zh-CN.md`](./docs/us_equity_live_switch_runbook.zh-CN.md)
4042

4143
[English README](./README.md)
4244

@@ -96,3 +98,5 @@ quant-platform-kit @ git+https://github.com/QuantStrategyLab/QuantPlatformKit.gi
9698
- [美股跨平台策略规范（中文）](./docs/us_equity_cross_platform_strategy_spec.zh-CN.md)
9799
- [美股执行翻译规范（英文）](./docs/us_equity_execution_translation_spec.md)
98100
- [美股执行翻译规范（中文）](./docs/us_equity_execution_translation_spec.zh-CN.md)
101+
- [美股线上切换与回滚手册（英文）](./docs/us_equity_live_switch_runbook.md)
102+
- [美股线上切换与回滚手册（中文）](./docs/us_equity_live_switch_runbook.zh-CN.md)

‎docs/us_equity_live_switch_runbook.md‎

Lines changed: 269 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,269 @@
1+
# US equity live switch and rollback runbook
2+
3+
This document is the operational runbook for switching live US equity strategy profiles on the current broker platforms.
4+
5+
Use this after the shared package and platform code is already deployed.
6+
7+
Do **not** use it to justify switching a profile that is not already supported by the platform status matrix.
8+
9+
## Scope
10+
11+
Current live US equity profiles:
12+
13+
- `global_etf_rotation`
14+
- `tqqq_growth_income`
15+
- `soxl_soxx_trend_income`
16+
- `russell_1000_multi_factor_defensive`
17+
- `qqq_tech_enhancement`
18+
19+
Current runtime platforms:
20+
21+
- `ibkr`
22+
- `schwab`
23+
- `longbridge`
24+
25+
For the current five-profile scope, all three platforms now report the full matrix as `eligible=true` and `enabled=true`. That means live switching is now an operational change, not a strategy-contract migration.
26+
27+
## Standard switch path
28+
29+
Use the same path every time:
30+
31+
1. verify the target profile is `eligible=true` and `enabled=true`
32+
2. update the GitHub-managed runtime variables for the target service
33+
3. rerun or wait for `Sync Cloud Run Env`
34+
4. verify the Cloud Run env on the service
35+
5. verify the first heartbeat or execution notification
36+
37+
Do not change service names as part of a strategy switch.
38+
39+
## Service inventory
40+
41+
| Platform | Service | Identity split |
42+
| --- | --- | --- |
43+
| IBKR | `interactive-brokers-quant-service` | `ACCOUNT_GROUP` |
44+
| Schwab | `charles-schwab-quant-service` | single service |
45+
| LongBridge HK | `longbridge-quant-hk-service` | `ACCOUNT_REGION=HK` |
46+
| LongBridge SG | `longbridge-quant-sg-service` | `ACCOUNT_REGION=SG` |
47+
48+
## Step 1: verify the target profile before touching env
49+
50+
Run the platform status script inside the platform repo.
51+
52+
### IBKR
53+
54+
```bash
55+
cd /Users/lisiyi/Projects/InteractiveBrokersPlatform
56+
PYTHONPATH=/Users/lisiyi/Projects/QuantPlatformKit/src:/Users/lisiyi/Projects/UsEquityStrategies/src:. \
57+
.venv/bin/python scripts/print_strategy_profile_status.py --json
58+
```
59+
60+
### Schwab
61+
62+
```bash
63+
cd /Users/lisiyi/Projects/CharlesSchwabPlatform
64+
PYTHONPATH=/Users/lisiyi/Projects/QuantPlatformKit/src:/Users/lisiyi/Projects/UsEquityStrategies/src:. \
65+
/Users/lisiyi/Projects/LongBridgePlatform/.venv/bin/python scripts/print_strategy_profile_status.py --json
66+
```
67+
68+
### LongBridge
69+
70+
```bash
71+
cd /Users/lisiyi/Projects/LongBridgePlatform
72+
PYTHONPATH=/Users/lisiyi/Projects/QuantPlatformKit/src:/Users/lisiyi/Projects/UsEquityStrategies/src:. \
73+
.venv/bin/python scripts/print_strategy_profile_status.py --json
74+
```
75+
76+
Required result:
77+
78+
- the target `canonical_profile` exists
79+
- `eligible` is `true`
80+
- `enabled` is `true`
81+
82+
If any of those checks fail, stop. That is a code or rollout problem, not a live switch problem.
83+
84+
## Step 2: know which extra envs the profile needs
85+
86+
| Profile | Extra runtime inputs beyond `STRATEGY_PROFILE` |
87+
| --- | --- |
88+
| `global_etf_rotation` | none |
89+
| `tqqq_growth_income` | none |
90+
| `soxl_soxx_trend_income` | none |
91+
| `russell_1000_multi_factor_defensive` | feature snapshot path + snapshot manifest path |
92+
| `qqq_tech_enhancement` | feature snapshot path + snapshot manifest path + strategy config path |
93+
94+
Notes:
95+
96+
- `qqq_tech_enhancement` on IBKR may also keep a reconciliation output path when the deployment wants that artifact.
97+
- When switching away from a feature-snapshot profile, remove stale snapshot/config envs from the service instead of leaving them behind.
98+
99+
## Step 3: update GitHub-managed runtime variables
100+
101+
Preferred operational path:
102+
103+
- update GitHub repository variables or environment variables
104+
- let `.github/workflows/sync-cloud-run-env.yml` apply the change
105+
106+
### IBKR
107+
108+
Required:
109+
110+
- `STRATEGY_PROFILE`
111+
- `ACCOUNT_GROUP`
112+
- `IB_ACCOUNT_GROUP_CONFIG_SECRET_NAME`
113+
114+
Optional:
115+
116+
- `IBKR_DRY_RUN_ONLY`
117+
118+
Feature-snapshot profiles additionally need:
119+
120+
- `IBKR_FEATURE_SNAPSHOT_PATH`
121+
- `IBKR_FEATURE_SNAPSHOT_MANIFEST_PATH`
122+
- `IBKR_STRATEGY_CONFIG_PATH` for `qqq_tech_enhancement`
123+
124+
Remove when not needed:
125+
126+
- `IBKR_FEATURE_SNAPSHOT_PATH`
127+
- `IBKR_FEATURE_SNAPSHOT_MANIFEST_PATH`
128+
- `IBKR_STRATEGY_CONFIG_PATH`
129+
- `IBKR_RECONCILIATION_OUTPUT_PATH`
130+
131+
### Schwab
132+
133+
Required:
134+
135+
- `STRATEGY_PROFILE`
136+
137+
Optional:
138+
139+
- `SCHWAB_DRY_RUN_ONLY`
140+
141+
Feature-snapshot profiles additionally need:
142+
143+
- `SCHWAB_FEATURE_SNAPSHOT_PATH`
144+
- `SCHWAB_FEATURE_SNAPSHOT_MANIFEST_PATH`
145+
- `SCHWAB_STRATEGY_CONFIG_PATH` when the chosen profile uses an external config file
146+
147+
Remove when not needed:
148+
149+
- `SCHWAB_FEATURE_SNAPSHOT_PATH`
150+
- `SCHWAB_FEATURE_SNAPSHOT_MANIFEST_PATH`
151+
- `SCHWAB_STRATEGY_CONFIG_PATH`
152+
153+
### LongBridge
154+
155+
Required:
156+
157+
- `STRATEGY_PROFILE`
158+
- `ACCOUNT_PREFIX`
159+
- `ACCOUNT_REGION`
160+
- `LONGPORT_SECRET_NAME`
161+
- `LONGPORT_APP_KEY_SECRET_NAME`
162+
- `LONGPORT_APP_SECRET_SECRET_NAME`
163+
164+
Optional:
165+
166+
- `LONGBRIDGE_DRY_RUN_ONLY`
167+
168+
Feature-snapshot profiles additionally need:
169+
170+
- `LONGBRIDGE_FEATURE_SNAPSHOT_PATH`
171+
- `LONGBRIDGE_FEATURE_SNAPSHOT_MANIFEST_PATH`
172+
- `LONGBRIDGE_STRATEGY_CONFIG_PATH`
173+
174+
Remove when not needed:
175+
176+
- `LONGBRIDGE_FEATURE_SNAPSHOT_PATH`
177+
- `LONGBRIDGE_FEATURE_SNAPSHOT_MANIFEST_PATH`
178+
- `LONGBRIDGE_STRATEGY_CONFIG_PATH`
179+
180+
## Step 4: rerun env sync and verify Cloud Run
181+
182+
Wait for the platform workflow to finish:
183+
184+
- `Sync Cloud Run Env`
185+
186+
Then verify the service directly.
187+
188+
### Example checks
189+
190+
```bash
191+
gcloud run services describe interactive-brokers-quant-service \
192+
--project interactivebrokersquant \
193+
--region us-central1 \
194+
--format='flattened(spec.template.spec.containers[0].env[])'
195+
```
196+
197+
```bash
198+
gcloud run services describe charles-schwab-quant-service \
199+
--project charlesschwabquant \
200+
--region us-central1 \
201+
--format='flattened(spec.template.spec.containers[0].env[])'
202+
```
203+
204+
```bash
205+
gcloud run services describe longbridge-quant-hk-service \
206+
--project longbridgequant \
207+
--region asia-east2 \
208+
--format='flattened(spec.template.spec.containers[0].env[])'
209+
```
210+
211+
```bash
212+
gcloud run services describe longbridge-quant-sg-service \
213+
--project longbridgequant \
214+
--region asia-southeast1 \
215+
--format='flattened(spec.template.spec.containers[0].env[])'
216+
```
217+
218+
Verify:
219+
220+
- `STRATEGY_PROFILE` matches the intended target
221+
- feature-snapshot envs exist only for feature-snapshot profiles
222+
- stale dry-run or artifact envs were removed when the new profile does not need them
223+
224+
## Step 5: verify the first runtime output
225+
226+
Do not stop at Cloud Run env.
227+
228+
Verify the first heartbeat or execution notification shows:
229+
230+
- the expected display name
231+
- the expected account prefix (`[HK]` / `[SG]` for LongBridge)
232+
- no stale strategy-specific service-name suffix in the LongBridge notification prefix
233+
234+
If the profile uses feature snapshots, also verify:
235+
236+
- the snapshot file exists at the configured path
237+
- the manifest matches the expected contract version
238+
- the managed symbols in the first notification match the intended strategy
239+
240+
## Rollback rules
241+
242+
Rollback is simple if you keep it operational:
243+
244+
1. restore the last known good `STRATEGY_PROFILE`
245+
2. restore or remove the companion snapshot/config envs so they match that profile
246+
3. rerun `Sync Cloud Run Env`
247+
4. verify Cloud Run env again
248+
5. verify the next heartbeat or execution notification
249+
250+
Do not use any of these as rollback mechanisms:
251+
252+
- old service names
253+
- ad hoc local edits on Cloud Run
254+
- partially reverting only one feature-snapshot env
255+
256+
If the service cannot start after a switch:
257+
258+
1. revert the service env first
259+
2. only then investigate code or dependency issues
260+
261+
## Recommended operator checklist
262+
263+
For every live switch, record these five items in the change note or operator log:
264+
265+
1. service name
266+
2. old profile
267+
3. new profile
268+
4. extra envs added or removed
269+
5. first successful heartbeat or execution timestamp

0 commit comments

Comments
 (0)