Skip to content

Commit e25621c

Browse files
committed
Add execution report heartbeat alerts
1 parent 8927190 commit e25621c

3 files changed

Lines changed: 497 additions & 0 deletions

File tree

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
name: Execution Report Heartbeat
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
lookback_hours:
7+
description: "Report lookback window in hours."
8+
required: false
9+
type: string
10+
default: "36"
11+
fail_workflow_on_alert:
12+
description: "Fail this workflow when an alert is emitted."
13+
required: false
14+
type: choice
15+
default: "true"
16+
options:
17+
- "true"
18+
- "false"
19+
schedule:
20+
- cron: "10 23 * * 1-5"
21+
22+
env:
23+
GCP_PROJECT_ID: firstradequant
24+
GCP_WORKLOAD_IDENTITY_PROVIDER: projects/1088907247379/locations/global/workloadIdentityPools/github-actions/providers/github-main
25+
GCP_WORKLOAD_IDENTITY_SERVICE_ACCOUNT: firstrade-platform-deploy@firstradequant.iam.gserviceaccount.com
26+
27+
jobs:
28+
heartbeat:
29+
name: Check execution report heartbeat
30+
runs-on: ubuntu-latest
31+
permissions:
32+
contents: read
33+
id-token: write
34+
env:
35+
RUNTIME_HEARTBEAT_NAME: FirstradePlatform
36+
RUNTIME_HEARTBEAT_LOOKBACK_HOURS: ${{ inputs.lookback_hours || vars.RUNTIME_HEARTBEAT_LOOKBACK_HOURS || '36' }}
37+
RUNTIME_HEARTBEAT_FAIL_WORKFLOW_ON_ALERT: ${{ inputs.fail_workflow_on_alert || vars.RUNTIME_HEARTBEAT_FAIL_WORKFLOW_ON_ALERT || 'true' }}
38+
RUNTIME_HEARTBEAT_ACCEPT_STAGES: ${{ vars.RUNTIME_HEARTBEAT_ACCEPT_STAGES }}
39+
RUNTIME_HEARTBEAT_REJECT_STAGES: ${{ vars.RUNTIME_HEARTBEAT_REJECT_STAGES }}
40+
FIRSTRADE_GCS_STATE_BUCKET: ${{ vars.FIRSTRADE_GCS_STATE_BUCKET }}
41+
FIRSTRADE_STATE_PREFIX: ${{ vars.FIRSTRADE_STATE_PREFIX }}
42+
GLOBAL_TELEGRAM_CHAT_ID: ${{ vars.GLOBAL_TELEGRAM_CHAT_ID }}
43+
CRISIS_ALERT_TELEGRAM_CHAT_IDS: ${{ vars.CRISIS_ALERT_TELEGRAM_CHAT_IDS }}
44+
CRISIS_ALERT_TELEGRAM_API_BASE_URL: ${{ vars.CRISIS_ALERT_TELEGRAM_API_BASE_URL }}
45+
TELEGRAM_TOKEN: ${{ secrets.TELEGRAM_TOKEN }}
46+
CRISIS_ALERT_TELEGRAM_BOT_TOKEN: ${{ secrets.CRISIS_ALERT_TELEGRAM_BOT_TOKEN }}
47+
steps:
48+
- name: Checkout repository
49+
uses: actions/checkout@v6
50+
51+
- name: Authenticate to Google Cloud
52+
uses: google-github-actions/auth@v3
53+
with:
54+
workload_identity_provider: ${{ env.GCP_WORKLOAD_IDENTITY_PROVIDER }}
55+
service_account: ${{ env.GCP_WORKLOAD_IDENTITY_SERVICE_ACCOUNT }}
56+
57+
- name: Set up gcloud
58+
uses: google-github-actions/setup-gcloud@v3
59+
60+
- name: Check recent execution report
61+
run: python scripts/execution_report_heartbeat.py

README.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -292,6 +292,14 @@ set `RUNTIME_GUARD_REQUIRE_SUCCESS=true` and choose
292292
Scheduler run. The default leaves that heartbeat check off to avoid false alerts
293293
outside trading windows.
294294

295+
`Execution Report Heartbeat` (`.github/workflows/execution-report-heartbeat.yml`)
296+
is the stricter completion check. It runs on weekdays after the expected US
297+
window and verifies that a recent strategy-run JSON exists under
298+
`FIRSTRADE_GCS_STATE_BUCKET` / `FIRSTRADE_STATE_PREFIX`. It reads the latest
299+
report status/stage and alerts if no recent report exists or the latest reports
300+
are error-like. The deploy service account needs object read/list access on the
301+
state bucket.
302+
295303
## Cloud Run Shape
296304

297305
`main.py` exposes:
@@ -438,6 +446,12 @@ Run、OIDC/IAM/audience 配错、Cloud Run 返回 4xx/5xx,或容器启动/导
438446
`RUNTIME_GUARD_REQUIRE_SUCCESS=true`,并把 `RUNTIME_GUARD_LOOKBACK_MINUTES` 设成覆盖
439447
Firstrade 预期 Scheduler 运行时间的窗口。默认不强制心跳,避免非交易窗口误报。
440448

449+
更严格的完成检查是 `Execution Report Heartbeat`
450+
`.github/workflows/execution-report-heartbeat.yml`)。它会在工作日美股预期窗口后检查
451+
`FIRSTRADE_GCS_STATE_BUCKET` / `FIRSTRADE_STATE_PREFIX` 下最近的 strategy-run JSON,
452+
读取 `status/stage/errors`,如果没有近期 report 或 report 呈错误状态就发 Telegram。
453+
GitHub deploy service account 需要对 state bucket 有对象读取/列举权限。
454+
441455
请不要把 Firstrade 登录凭据、MFA secret、cookie 文件提交到 Git。`.env`
442456
`.runtime/``ft_cookies*.json` 已经在 `.gitignore` 中。
443457

0 commit comments

Comments
 (0)