Skip to content

Commit f3fd140

Browse files
authored
Add crypto artifact manifest contract
1 parent 29cf621 commit f3fd140

15 files changed

Lines changed: 507 additions & 19 deletions

README.md

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -12,8 +12,9 @@ Core upstream artifacts:
1212
2. `data/output/latest_ranking.csv`
1313
3. `data/output/live_pool.json`
1414
4. `data/output/live_pool_legacy.json`
15-
5. `data/output/release_manifest.json`
16-
6. `data/output/release_status_summary.json`
15+
5. `data/output/artifact_manifest.json`
16+
6. `data/output/release_manifest.json`
17+
7. `data/output/release_status_summary.json`
1718

1819
## Upstream Boundary
1920

@@ -28,7 +29,7 @@ Core upstream artifacts:
2829

2930
In practice, that means:
3031

31-
- upstream publishes and explains `latest_universe`, `latest_ranking`, `live_pool`, `release_manifest`, and release-status summaries
32+
- upstream publishes and explains `latest_universe`, `latest_ranking`, `live_pool`, `artifact_manifest`, `release_manifest`, and release-status summaries
3233
- downstream consumes the official live-pool contract plus publish metadata and emits only runtime/execution status
3334
- research CSVs, shadow-track diagnostics, and monthly review outputs stay upstream and are not part of the minimum downstream execution contract
3435

@@ -40,7 +41,7 @@ The repository is now intentionally split into two tracks:
4041
- data source: `Binance Spot only`
4142
- universe mode: `core_major`
4243
- publish cadence: `monthly`
43-
- default outputs: `latest_universe.json`, `latest_ranking.csv`, `live_pool.json`, `live_pool_legacy.json`
44+
- default outputs: `latest_universe.json`, `latest_ranking.csv`, `live_pool.json`, `live_pool_legacy.json`, `artifact_manifest.json`
4445
- `Experimental external-data track`
4546
- used for research, comparison, and validation only
4647
- not enabled by default
@@ -245,10 +246,10 @@ Validate the local production artifacts before publish or rollback:
245246
.venv/bin/python scripts/validate_release_contract.py --mode core_major --expected-pool-size 5
246247
```
247248

248-
Require a generated manifest as part of the check:
249+
Require generated release and artifact manifests as part of the production check:
249250

250251
```bash
251-
.venv/bin/python scripts/validate_release_contract.py --mode core_major --expected-pool-size 5 --require-manifest
252+
.venv/bin/python scripts/validate_release_contract.py --mode core_major --expected-pool-size 5 --require-manifest --require-artifact-manifest
252253
```
253254

254255
Operator workflow details, rollback steps, and research-vs-production boundaries are documented in `docs/operator_runbook.md`.
@@ -370,7 +371,7 @@ Downstream consumers should rely on these core fields in `data/output/live_pool.
370371
- `symbol_map`
371372
- `source_project`
372373

373-
Publish-time pointer fields such as `storage_prefix`, `current_prefix`, `live_pool_uri`, `live_pool_legacy_uri`, `latest_universe_uri`, and `latest_ranking_uri` are stable when present in the published Firestore payload, but they are release/distribution metadata rather than research features.
374+
Publish-time pointer fields such as `storage_prefix`, `current_prefix`, `live_pool_uri`, `live_pool_legacy_uri`, `artifact_manifest_uri`, `latest_universe_uri`, and `latest_ranking_uri` are stable when present in the published Firestore payload, but they are release/distribution metadata rather than research features.
374375

375376
Optional additive research extensions:
376377

README.zh-CN.md

Lines changed: 24 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,10 @@
99
1. `data/output/latest_universe.json`
1010
2. `data/output/latest_ranking.csv`
1111
3. `data/output/live_pool.json`
12+
4. `data/output/live_pool_legacy.json`
13+
5. `data/output/artifact_manifest.json`
14+
6. `data/output/release_manifest.json`
15+
7. `data/output/release_status_summary.json`
1216

1317
## 当前状态
1418

@@ -18,7 +22,7 @@
1822
- 数据源:仅 `Binance Spot`
1923
- universe mode:`core_major`
2024
- 发布频率:`monthly`
21-
- 默认输出:`latest_universe.json``latest_ranking.csv``live_pool.json``live_pool_legacy.json`
25+
- 默认输出:`latest_universe.json``latest_ranking.csv``live_pool.json``live_pool_legacy.json``artifact_manifest.json`
2226
- `Experimental external-data track`
2327
- 仅用于研究、比较和验证
2428
- 默认不启用
@@ -147,6 +151,20 @@ pip install -r requirements.txt
147151
- 输出设置
148152
- GCS / Firestore 发布设置
149153

154+
## 发布契约检查
155+
156+
发布或回滚前,先校验本地生产产物:
157+
158+
```bash
159+
.venv/bin/python scripts/validate_release_contract.py --mode core_major --expected-pool-size 5
160+
```
161+
162+
生产发布链应同时要求 release manifest 和 profile-aware artifact manifest:
163+
164+
```bash
165+
.venv/bin/python scripts/validate_release_contract.py --mode core_major --expected-pool-size 5 --require-manifest --require-artifact-manifest
166+
```
167+
150168
## 最小可运行流程
151169

152170
1. 下载历史数据
@@ -215,12 +233,15 @@ pip install -r requirements.txt
215233
- `data/output/live_pool_legacy.json`
216234
- Firestore summary document
217235

236+
`data/output/artifact_manifest.json` 是 profile-aware wrapper,负责声明 artifact contract version、主 artifact、相关文件路径和校验和;它不是 `live_pool.json` 的字段复制。
237+
218238
一些发布期辅助字段,例如:
219239

220240
- `storage_prefix`
221241
- `current_prefix`
222242
- `live_pool_uri`
223243
- `live_pool_legacy_uri`
244+
- `artifact_manifest_uri`
224245
- `latest_universe_uri`
225246
- `latest_ranking_uri`
226247

@@ -290,6 +311,7 @@ make monthly-shadow-build
290311
- official baseline
291312
- `data/output/live_pool.json`
292313
- `data/output/live_pool_legacy.json`
314+
- `data/output/artifact_manifest.json`
293315
- `data/output/release_manifest.json`
294316
- shadow candidate tracks
295317
- `data/output/shadow_candidate_tracks/track_summary.csv`
@@ -322,6 +344,7 @@ make monthly-build-telegram
322344

323345
- 只发送简短的 monthly build/publish health summary
324346
- 使用已有的 `monthly_shadow_build_summary.json``live_pool.json``release_manifest.json``track_summary.csv`
347+
- 生产发布链还会检查 `artifact_manifest.json`,但 Telegram 文本只展示摘要状态
325348
- 如果 Telegram 凭证缺失,会跳过而不是报错中断
326349
- 不改变 monthly build 行为,也不是 review 包生成器
327350

docs/integration_contract.md

Lines changed: 42 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,41 @@ The upstream project publishes a monthly `core_major` live pool and exposes it t
1010

1111
## Canonical Downstream Files
1212

13+
### `artifact_manifest.json`
14+
15+
This profile-aware manifest is the canonical contract wrapper for downstream
16+
runtimes. It lets the execution platform validate the artifact family without
17+
binding to `CryptoLeaderRotation` internals.
18+
19+
Required stable fields:
20+
21+
- `manifest_type = strategy_artifact`
22+
- `contract_version = crypto_leader_rotation.live_pool.v1`
23+
- `strategy_profile = crypto_leader_rotation`
24+
- `artifact_type = live_pool`
25+
- `artifact_name = crypto_leader_rotation_live_pool`
26+
- `as_of_date`
27+
- `snapshot_as_of`
28+
- `version`
29+
- `mode`
30+
- `symbol_count`
31+
- `symbols`
32+
- `source_project`
33+
- `generated_at`
34+
- `primary_artifact = live_pool`
35+
- `artifacts`
36+
37+
The `artifacts` mapping includes relative file paths and SHA-256 checksums for:
38+
39+
- `latest_universe`
40+
- `latest_ranking`
41+
- `live_pool`
42+
- `live_pool_legacy`
43+
44+
Downstream platforms should treat this manifest as the strategy artifact
45+
contract and keep legacy `live_pool_legacy.json` parsing as a compatibility
46+
path, not as the only contract shape.
47+
1348
### `live_pool_legacy.json`
1449

1550
This is the most convenient file for older downstream scripts that expect a direct symbol mapping.
@@ -123,9 +158,12 @@ Payload example:
123158
"current_prefix": "gs://example-bucket/crypto-leader-rotation/current",
124159
"live_pool_legacy_uri": "gs://example-bucket/crypto-leader-rotation/current/live_pool_legacy.json",
125160
"live_pool_uri": "gs://example-bucket/crypto-leader-rotation/current/live_pool.json",
161+
"artifact_manifest_uri": "gs://example-bucket/crypto-leader-rotation/current/artifact_manifest.json",
126162
"latest_universe_uri": "gs://example-bucket/crypto-leader-rotation/current/latest_universe.json",
127163
"latest_ranking_uri": "gs://example-bucket/crypto-leader-rotation/current/latest_ranking.csv",
128164
"versioned_live_pool_legacy_uri": "gs://example-bucket/crypto-leader-rotation/releases/2026-03-13-core_major/live_pool_legacy.json",
165+
"versioned_artifact_manifest_uri": "gs://example-bucket/crypto-leader-rotation/releases/2026-03-13-core_major/artifact_manifest.json",
166+
"artifact_contract_version": "crypto_leader_rotation.live_pool.v1",
129167
"generated_at": "2026-03-13T13:00:00+00:00",
130168
"source_project": "crypto-leader-rotation"
131169
}
@@ -136,8 +174,9 @@ The Firestore document intentionally excludes the full ranking CSV. Downstream r
136174
Stable vs additive fields:
137175

138176
- stable core fields: `as_of_date`, `version`, `mode`, `pool_size`, `symbols`, `symbol_map`, `source_project`
139-
- publish-only pointer fields: `storage_prefix`, `current_prefix`, `live_pool_uri`, `live_pool_legacy_uri`, `latest_universe_uri`, `latest_ranking_uri`, `versioned_live_pool_legacy_uri`
177+
- publish-only pointer fields: `storage_prefix`, `current_prefix`, `live_pool_uri`, `live_pool_legacy_uri`, `artifact_manifest_uri`, `latest_universe_uri`, `latest_ranking_uri`, `versioned_live_pool_legacy_uri`, `versioned_artifact_manifest_uri`
140178
- additive observability field: `generated_at`
179+
- artifact wrapper field: `artifact_contract_version`
141180

142181
## GCS Path Layout
143182

@@ -148,6 +187,7 @@ gs://<bucket>/crypto-leader-rotation/releases/<YYYY-MM-DD-mode>/latest_universe.
148187
gs://<bucket>/crypto-leader-rotation/releases/<YYYY-MM-DD-mode>/latest_ranking.csv
149188
gs://<bucket>/crypto-leader-rotation/releases/<YYYY-MM-DD-mode>/live_pool.json
150189
gs://<bucket>/crypto-leader-rotation/releases/<YYYY-MM-DD-mode>/live_pool_legacy.json
190+
gs://<bucket>/crypto-leader-rotation/releases/<YYYY-MM-DD-mode>/artifact_manifest.json
151191
```
152192

153193
Current pointers:
@@ -157,6 +197,7 @@ gs://<bucket>/crypto-leader-rotation/current/latest_universe.json
157197
gs://<bucket>/crypto-leader-rotation/current/latest_ranking.csv
158198
gs://<bucket>/crypto-leader-rotation/current/live_pool.json
159199
gs://<bucket>/crypto-leader-rotation/current/live_pool_legacy.json
200+
gs://<bucket>/crypto-leader-rotation/current/artifact_manifest.json
160201
```
161202

162203
## Local Shadow Release History

docs/operator_runbook.md

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ Primary production outputs:
1010
- `data/output/latest_ranking.csv`
1111
- `data/output/live_pool.json`
1212
- `data/output/live_pool_legacy.json`
13+
- `data/output/artifact_manifest.json`
1314
- `data/output/release_manifest.json`
1415
- `data/output/release_status_summary.json`
1516
- `data/output/release_status_summary.md`
@@ -86,7 +87,7 @@ Boundary rules:
8687
3. Run explicit contract validation:
8788

8889
```bash
89-
.venv/bin/python scripts/validate_release_contract.py --mode core_major --expected-pool-size 5
90+
.venv/bin/python scripts/validate_release_contract.py --mode core_major --expected-pool-size 5 --require-manifest --require-artifact-manifest
9091
```
9192

9293
4. Run publish preflight without external writes:
@@ -125,7 +126,7 @@ Boundary rules:
125126

126127
- `requirements-lock.txt` is present and matches the intended release dependency set.
127128
- Local artifacts are non-empty and pass `scripts/validate_release_contract.py`.
128-
- `live_pool.json`, `live_pool_legacy.json`, and `release_manifest.json` agree on `as_of_date`, `version`, `mode`, `pool_size`, and `source_project`.
129+
- `live_pool.json`, `live_pool_legacy.json`, and `release_manifest.json` agree on `as_of_date`, `version`, `mode`, `pool_size`, and `source_project`; `artifact_manifest.json` agrees on `as_of_date`, `version`, `mode`, `source_project`, and `symbol_count`.
129130
- `GCP_PROJECT_ID`, `GCS_BUCKET`, `FIRESTORE_COLLECTION`, and `FIRESTORE_DOCUMENT` are set correctly for real publish.
130131
- Historical backfills use `--allow-stale` explicitly; do not silently publish stale artifacts.
131132

@@ -141,7 +142,7 @@ Symptoms:
141142
Actions:
142143

143144
- Re-run `scripts/build_live_pool.py`
144-
- Inspect `data/output/latest_universe.json`, `live_pool.json`, `live_pool_legacy.json`, and `latest_ranking.csv`
145+
- Inspect `data/output/latest_universe.json`, `latest_ranking.csv`, `live_pool.json`, `live_pool_legacy.json`, and `artifact_manifest.json`
145146
- Confirm `pool_size`, `symbols`, `symbol_map`, `version`, and `source_project` are present and aligned
146147

147148
### Stale artifacts
@@ -192,17 +193,18 @@ Use rollback only when the newest publish is clearly bad or malformed.
192193
- GCS `crypto-leader-rotation/releases/<version>/`
193194
- the last good `data/output/release_manifest.json`
194195

195-
2. Restore the four canonical artifacts from that version into `data/output/`:
196+
2. Restore the five canonical artifacts from that version into `data/output/`:
196197

197198
- `latest_universe.json`
198199
- `latest_ranking.csv`
199200
- `live_pool.json`
200201
- `live_pool_legacy.json`
202+
- `artifact_manifest.json`
201203

202204
3. Validate the restored payload locally:
203205

204206
```bash
205-
.venv/bin/python scripts/validate_release_contract.py --mode core_major --expected-pool-size 5
207+
.venv/bin/python scripts/validate_release_contract.py --mode core_major --expected-pool-size 5 --require-artifact-manifest
206208
```
207209

208210
4. Regenerate the manifest and verify publish preflight:

docs/validation_status.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ The repository is now frozen around this production default:
1515
- `latest_ranking.csv`
1616
- `live_pool.json`
1717
- `live_pool_legacy.json`
18+
- `artifact_manifest.json`
1819

1920
This is the only path that should be treated as the formal production baseline.
2021

@@ -125,6 +126,7 @@ Validated in-repo:
125126

126127
- `scripts/build_live_pool.py` produces the default `Production v1` live output
127128
- `scripts/publish_release.py --dry-run` builds a correct production release manifest
129+
- `scripts/validate_release_contract.py --require-artifact-manifest` validates the profile-aware artifact contract
128130
- `scripts/write_release_heartbeat.py` writes a small logs-branch heartbeat file
129131
- GitHub Actions workflow YAML parses correctly
130132
- release versioning, GCS object keys, and Firestore payload layout are consistent
@@ -136,6 +138,7 @@ Validated artifacts:
136138
- `data/output/latest_ranking.csv`
137139
- `data/output/live_pool.json`
138140
- `data/output/live_pool_legacy.json`
141+
- `data/output/artifact_manifest.json`
139142
- `data/output/release_manifest.json`
140143
- `data/output/heartbeat/monthly/<version>.json`
141144

scripts/build_live_pool.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,13 +62,15 @@ def main() -> None:
6262
),
6363
expected_pool_size=int(config["export"]["live_pool_size"]),
6464
max_age_days=args.contract_max_age_days,
65+
require_artifact_manifest=True,
6566
require_freshness=not bool(args.as_of_date or args.allow_stale),
6667
)
6768
logger.info(
68-
"Release contract validated | version=%s | pool_size=%s | manifest_present=%s",
69+
"Release contract validated | version=%s | pool_size=%s | release_manifest_present=%s | artifact_manifest_present=%s",
6970
validation["version"],
7071
validation["pool_size"],
7172
validation["manifest_present"],
73+
validation["artifact_manifest_present"],
7274
)
7375
logger.info("Export payload:\n%s", result["live_payload"])
7476

scripts/run_release_status_summary.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,7 @@ def build_release_status_payload(
8585
universe = load_json(root / "latest_universe.json")
8686
live_pool = load_json(root / "live_pool.json")
8787
manifest = load_optional_json(root / "release_manifest.json") or {}
88+
artifact_manifest = load_optional_json(root / "artifact_manifest.json") or {}
8889
ranking = pd.read_csv(root / "latest_ranking.csv")
8990

9091
validation = validate_release_outputs(
@@ -94,6 +95,7 @@ def build_release_status_payload(
9495
expected_pool_size=live_pool.get("pool_size"),
9596
max_age_days=max_age_days,
9697
require_manifest=True,
98+
require_artifact_manifest=True,
9799
require_freshness=require_freshness,
98100
)
99101

@@ -134,6 +136,7 @@ def build_release_status_payload(
134136
"latest_universe_symbol_count": len(list(universe.get("symbols", []))),
135137
"latest_ranking_row_count": int(len(ranking)),
136138
"latest_ranking_selected_count": int(selected_mask.sum()) if not selected_mask.empty else 0,
139+
"artifact_contract_version": str(artifact_manifest.get("contract_version", "")).strip(),
137140
"ranking_preview": ranking_preview_rows,
138141
},
139142
"publish_summary": {
@@ -147,6 +150,7 @@ def build_release_status_payload(
147150
"validation": {
148151
"ok": bool(validation.get("ok")),
149152
"manifest_present": bool(validation.get("manifest_present")),
153+
"artifact_manifest_present": bool(validation.get("artifact_manifest_present")),
150154
"age_days": validation.get("age_days"),
151155
"errors": list(validation.get("errors", [])),
152156
"warnings": list(validation.get("warnings", [])),
@@ -156,6 +160,7 @@ def build_release_status_payload(
156160
"latest_ranking": str(root / "latest_ranking.csv"),
157161
"live_pool": str(root / "live_pool.json"),
158162
"release_manifest": str(root / "release_manifest.json"),
163+
"artifact_manifest": str(root / "artifact_manifest.json"),
159164
},
160165
}
161166

@@ -190,6 +195,7 @@ def render_markdown(payload: dict[str, Any]) -> str:
190195
- latest_universe symbol count: {artifact['latest_universe_symbol_count']}
191196
- latest_ranking row count: {artifact['latest_ranking_row_count']}
192197
- latest_ranking selected count: {artifact['latest_ranking_selected_count']}
198+
- artifact contract version: {artifact['artifact_contract_version'] or 'n/a'}
193199
194200
### Ranking preview
195201
@@ -207,6 +213,7 @@ def render_markdown(payload: dict[str, Any]) -> str:
207213
208214
- ok: {validation['ok']}
209215
- manifest_present: {validation['manifest_present']}
216+
- artifact_manifest_present: {validation['artifact_manifest_present']}
210217
- age_days: {validation['age_days']}
211218
212219
### Errors

scripts/validate_release_contract.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,11 @@ def parse_args() -> argparse.Namespace:
3737
action="store_true",
3838
help="Require release_manifest.json and validate it against the live pool contract.",
3939
)
40+
parser.add_argument(
41+
"--require-artifact-manifest",
42+
action="store_true",
43+
help="Require artifact_manifest.json and validate the profile-aware artifact contract.",
44+
)
4045
parser.add_argument(
4146
"--reference-date",
4247
default=None,
@@ -55,6 +60,7 @@ def main() -> None:
5560
reference_date=args.reference_date,
5661
max_age_days=args.max_age_days,
5762
require_manifest=args.require_manifest,
63+
require_artifact_manifest=args.require_artifact_manifest,
5864
require_freshness=not args.allow_stale,
5965
)
6066
print(json.dumps(validation, ensure_ascii=False, indent=2))

0 commit comments

Comments
 (0)