Skip to content

Commit e531cd5

Browse files
committed
docs: publish performance diagnostics
1 parent ac7bb90 commit e531cd5

15 files changed

Lines changed: 241 additions & 56 deletions

AGENTS.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,8 @@ Use conventional commit prefixes (`feat:`, `fix:`, `docs:`, `ci:`); keep message
4646
## Security & Configuration Tips
4747

4848
Never commit `.env`, credentials, broker account data, local state, market-data
49-
snapshots, executed notebook outputs, or generated reports. Use `.env.example`
50-
as the template. Synthetic data is limited to tests and clearly labeled dry
51-
runs. Preserve pre-trade risk checks, paper-mode defaults, and point-in-time
52-
data discipline.
49+
snapshots, or executed notebook outputs. Generated performance charts require
50+
explicit owner approval, adjacent provenance, an input digest, and artifact
51+
hashes; ordinary reports remain ignored. Use `.env.example` as the template.
52+
Synthetic data is limited to tests and clearly labeled dry runs. Preserve
53+
pre-trade risk checks, paper-mode defaults, and point-in-time data discipline.

CLAUDE.md

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -100,12 +100,13 @@ or vol-scaled book is NOT required to sum to 1. Violations raise
100100
(threadpoolctl) around the HMM fit so backtests are bit-for-bit reproducible; a
101101
non-converged EM near a regime boundary otherwise flips under multithreaded
102102
float ordering. Keep model fits seeded and deterministic.
103-
- **Explicit research data.** Do not commit market-data snapshots, generated
104-
performance charts, or executed notebook outputs. Reports and notebooks must
105-
use either an owner-supplied permitted CSV or an explicit live-provider opt-in;
106-
they must never silently substitute generated prices after a fetch failure.
107-
Synthetic fixtures remain appropriate for tests and the clearly labeled
108-
`paper_trade_cycle.py --offline` dry run.
103+
- **Explicit research data.** Do not commit market-data snapshots or executed
104+
notebook outputs. Reports and notebooks must use either an owner-supplied
105+
permitted CSV or an explicit live-provider opt-in; they must never silently
106+
substitute generated prices after a fetch failure. Derived performance charts
107+
may be published only with explicit owner approval, adjacent provenance, an
108+
input digest, and artifact hashes. Synthetic fixtures remain appropriate for
109+
tests and the clearly labeled `paper_trade_cycle.py --offline` dry run.
109110

110111
## Layer ABCs to subclass
111112

PERFORMANCE.md

Lines changed: 32 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,35 @@
11
# Performance Evaluation
22

3-
This repository does not publish a fixed performance result. Market data,
4-
executed notebook outputs, and generated charts are intentionally excluded.
5-
Evaluate the reference strategies on data you are permitted to use and retain
6-
the source metadata needed to reproduce the run.
3+
This repository publishes one owner-authorized reference run as derived chart
4+
artifacts. The raw market data remains uncommitted. Ordinary generated reports
5+
and executed notebook outputs remain excluded unless publication is explicitly
6+
approved and accompanied by complete provenance and artifact hashes.
7+
8+
## Published Reference Run
9+
10+
The README charts were generated on June 15, 2026 with yfinance 1.4.1 adjusted
11+
closes for QQQ, VGT, GLD, TLT, SPY, and VIG. The source spans January 4, 2016 to
12+
December 31, 2025; evaluation spans January 2, 2018 to December 31, 2025 after
13+
503 warm-up sessions. The owner confirms permission to publish the derived
14+
charts; that permission is not independently verified by the software.
15+
16+
| Metric | Value |
17+
|---|---:|
18+
| CAGR | -0.07% |
19+
| Annualized volatility | 6.53% |
20+
| Sharpe | 0.02 |
21+
| Maximum drawdown | -14.22% |
22+
| Annualized one-way turnover | 10.84x |
23+
| Sum of modeled cost fractions | 15.06% |
24+
| SPY Sharpe, gross | 0.78 |
25+
| Equal-weight six-ETF Sharpe, gross | 0.96 |
26+
27+
Strategy returns are net of 3 bps commission and 10 bps flat slippage per
28+
trade; benchmarks are gross and the ADV cap is inactive. The DSR of 0.065 uses
29+
an assumed 10 trials and a single-series variance estimate. Because the true
30+
historical trial count is unknown, it is not a validated multiple-testing
31+
correction. Exact source and artifact hashes are in
32+
`docs/img/performance_manifest.json`.
733

834
## Generate a Report
935

@@ -65,8 +91,8 @@ The command writes `reports/report.md` plus these plots under `reports/img/`:
6591

6692
The Markdown report links every plot and includes performance metrics,
6793
evaluation settings, data provenance, and the monthly-return table. Outputs
68-
remain ignored; do not copy them into the README without a redistributable input
69-
dataset and complete provenance.
94+
remain ignored by default. Publish them only with explicit owner approval,
95+
complete adjacent provenance, an input digest, and hashes for every artifact.
7096

7197
The current command must not fabricate diagnostics it cannot support. Add
7298
walk-forward or live-start boundaries only when the run records those regimes;

README.md

Lines changed: 86 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -27,17 +27,84 @@ w_t = R_t( T_t( A_t( S_t( X<=t ) ) ) )
2727

2828
**Design targets are not performance claims.** The reference strategies retain
2929
aspirational Sharpe targets of 1.10 (multi-asset rotation) and 0.9 (momentum
30-
ML), but this repository publishes no fixed backtest result. Evaluate them on
31-
data you are permitted to use, record the true number of trials, and compare
32-
against appropriate benchmarks. See [PERFORMANCE.md](PERFORMANCE.md).
30+
ML). The published reference run below materially misses its target and both
31+
gross benchmarks; it demonstrates the audit path rather than strategy quality.
32+
See [PERFORMANCE.md](PERFORMANCE.md).
3333

3434
---
3535

3636
## Project Demo
3737

38-
The best demonstration is a locally generated, provenance-bearing report rather
39-
than a permanent equity curve detached from its input data. With an authorized
40-
price file, run:
38+
The repository owner confirms permission to publish these derived charts. The
39+
raw market-data file remains uncommitted; its digest and the complete chart
40+
hashes are recorded in
41+
[`performance_manifest.json`](docs/img/performance_manifest.json).
42+
43+
![Multi-Asset Rotation diagnostic overview](docs/img/report_overview.png)
44+
45+
| Reference-run result | Value |
46+
|---|---:|
47+
| Evaluation window | 2018-01-02 to 2025-12-31 |
48+
| CAGR | -0.07% |
49+
| Annualized volatility | 6.53% |
50+
| Sharpe | 0.02 |
51+
| Maximum drawdown | -14.22% |
52+
| Annualized one-way turnover | 10.84x |
53+
| Sum of modeled cost fractions | 15.06% |
54+
| SPY Sharpe, gross | 0.78 |
55+
| Equal-weight six-ETF Sharpe, gross | 0.96 |
56+
57+
Strategy returns are net of 3 bps commission and 10 bps flat slippage per
58+
trade; benchmark returns are gross. The ADV cap is inactive because this run
59+
has no volume input. The reported DSR is 0.065 using an assumed 10 trials and a
60+
single-series variance estimate. The true historical trial count is unknown,
61+
so that DSR is not a validated multiple-testing correction.
62+
63+
<details>
64+
<summary><strong>Open the full diagnostic gallery</strong></summary>
65+
66+
### Equity Versus Benchmarks
67+
68+
![Net strategy equity versus gross benchmarks](docs/img/equity_vs_benchmarks.png)
69+
70+
### Drawdown
71+
72+
![Strategy underwater drawdown](docs/img/drawdown.png)
73+
74+
### Rolling Sharpe
75+
76+
![Rolling 126-session Sharpe](docs/img/rolling_sharpe.png)
77+
78+
### Rolling Risk
79+
80+
![Rolling annualized volatility and beta to SPY](docs/img/rolling_risk.png)
81+
82+
### Allocation And Exposure
83+
84+
![Post-trade allocation, exposure, and cash](docs/img/allocation_and_exposure.png)
85+
86+
### Turnover And Costs
87+
88+
![Executed turnover and cumulative modeled cost fractions](docs/img/turnover_and_costs.png)
89+
90+
### Monthly Returns
91+
92+
![Monthly net-return heatmap](docs/img/monthly_returns.png)
93+
94+
### Return Distribution
95+
96+
![Daily net-return distribution and normal Q-Q diagnostic](docs/img/return_distribution.png)
97+
98+
</details>
99+
100+
The published run uses Yahoo Finance data retrieved through yfinance 1.4.1 on
101+
June 15, 2026. It covers QQQ, VGT, GLD, TLT, SPY, and VIG from January 4, 2016
102+
through December 31, 2025; the first 503 sessions are signal warm-up. The local
103+
adjusted-close CSV has SHA-256
104+
`3b256ef083794a67f173d635a3cd297237b7a3d01489ccf53cc807c99602607c`.
105+
Permission is owner-asserted and not independently verified by the software.
106+
107+
To generate the same report from an authorized file matching that digest:
41108

42109
```bash
43110
PYTHONPATH=. python scripts/generate_report.py \
@@ -49,18 +116,9 @@ PYTHONPATH=. python scripts/generate_report.py \
49116
--adjustment-method "$DATA_ADJUSTMENT_METHOD"
50117
```
51118

52-
Open `reports/report.md`. It links a compact diagnostic overview and detailed
53-
performance, drawdown, rolling-risk, allocation, exposure, turnover, cost,
54-
monthly-return, and tail-distribution plots. The report also records the input
55-
digest, evaluation window, warm-up, cost assumptions, DSR settings, and whether
56-
all publication metadata was supplied. Metadata records the owner's assertions;
57-
the tool does not determine whether a license permits redistribution.
58-
59-
The architecture diagram below is safe to publish because it is independent of
60-
market data. A generated `report_overview.png` should be added to this README
61-
only when the input license permits publication of derived images and the
62-
adjacent text identifies the source, date window, file digest, costs, benchmark
63-
treatment, and true research trial count. Never publish an equity curve alone.
119+
The command writes `reports/report.md` and the same nine-plot diagnostic set.
120+
Metadata records owner-supplied assertions; the tool does not determine whether
121+
a license permits publication or redistribution.
64122

65123
---
66124

@@ -179,9 +237,11 @@ credential variables.
179237

180238
## Performance Reporting
181239

182-
No market-data snapshot, executed notebook output, generated chart, or fixed
183-
performance number is published in this repository. This avoids redistributing
184-
provider data and prevents a changing data vintage from looking immutable.
240+
The README publishes one owner-authorized reference run as derived images with
241+
adjacent provenance and an artifact manifest. Raw market data, executed
242+
notebook output, and ordinary local reports remain excluded. Published results
243+
must identify their data vintage and must not be silently regenerated from a
244+
different input file.
185245

186246
For a licensed local dataset, run:
187247

@@ -195,12 +255,10 @@ The report records the file path, SHA-256 digest, observed date window, cost
195255
assumptions, signal warm-up, DSR trial count/variance assumption, and whether
196256
liquidity constraints are active.
197257
Each run writes `reports/report.md`, a compact `report_overview.png`, and eight
198-
detailed diagnostics under ignored `reports/img/`. These are local research
199-
evidence, not repository fixtures. The README intentionally contains no fixed
200-
performance plot because the repository does not ship a redistributable input
201-
dataset. See
202-
[PERFORMANCE.md](PERFORMANCE.md) for interpretation requirements and known
203-
limitations.
258+
detailed diagnostics under ignored `reports/img/`. These remain local research
259+
evidence unless the owner explicitly approves publication and adds complete
260+
provenance plus artifact hashes. See [PERFORMANCE.md](PERFORMANCE.md) for the
261+
reference run, interpretation requirements, and known limitations.
204262

205263
---
206264

@@ -369,6 +427,7 @@ quantcortex/ # repo root
369427
├── local_data/README.md # ignored local-data schemas and provenance rules
370428
├── reports/ # ignored generated charts and report output
371429
├── docs/
430+
│ ├── img/ # approved reference-run plots + hash manifest
372431
│ ├── production-readiness.md # blockers before production capital
373432
│ └── history-rewrite-plan.md # optional purge procedure; not executed
374433
├── docker-compose.yml
193 KB
Loading

docs/img/drawdown.png

79.4 KB
Loading

docs/img/equity_vs_benchmarks.png

110 KB
Loading

docs/img/monthly_returns.png

75 KB
Loading

docs/img/performance_manifest.json

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
{
2+
"schema_version": 1,
3+
"generated_at": "2026-06-15",
4+
"generator_commit": "ac7bb90754b4b1b488af3b7bdc561b046c7c4aa3",
5+
"source": {
6+
"provider": "Yahoo Finance via yfinance 1.4.1",
7+
"permission_basis": "Repository owner confirms permission to publish derived charts",
8+
"permission_independently_verified": false,
9+
"retrieved_at": "2026-06-15",
10+
"symbols": [
11+
"QQQ",
12+
"VGT",
13+
"GLD",
14+
"TLT",
15+
"SPY",
16+
"VIG"
17+
],
18+
"adjustment_method": "yfinance adjusted close with auto_adjust=False",
19+
"price_window": "2016-01-04 to 2025-12-31",
20+
"input_sha256": "3b256ef083794a67f173d635a3cd297237b7a3d01489ccf53cc807c99602607c",
21+
"raw_input_committed": false
22+
},
23+
"evaluation": {
24+
"strategy": "multi_asset_rotation",
25+
"window": "2018-01-02 to 2025-12-31",
26+
"sessions": 2011,
27+
"warmup_sessions": 503,
28+
"required_warmup_sessions": 274,
29+
"execution_timing": "close signal executes at the next bar close",
30+
"rebalance": "first available session each week",
31+
"commission_bps": 3.0,
32+
"slippage_bps": 10.0,
33+
"transfer_tax_bps": 0.0,
34+
"adv_cap_applied": false,
35+
"strategy_returns": "net",
36+
"benchmark_returns": "gross",
37+
"dsr_trials_assumed": 10,
38+
"true_historical_trial_count": "unknown",
39+
"dsr_cross_trial_variance": "single-series estimate"
40+
},
41+
"metrics": {
42+
"total_return": -0.005230439506423212,
43+
"cagr": -0.000656934729176939,
44+
"annualized_volatility": 0.06528824792584632,
45+
"sharpe": 0.022628754062896092,
46+
"sortino": 0.030394685432320295,
47+
"calmar": -0.004619337898382643,
48+
"max_drawdown": -0.1422140453087335,
49+
"annualized_one_way_turnover": 10.84416722565801,
50+
"sum_of_modeled_cost_fractions": 0.15056228172947278,
51+
"var_95_daily": 0.007188401820487274,
52+
"cvar_95_daily": 0.011103253411059867,
53+
"deflated_sharpe": 0.06542961955250098,
54+
"spy_sharpe_gross": 0.7824200403668903,
55+
"equal_weight_sharpe_gross": 0.9648098190098434
56+
},
57+
"artifacts": {
58+
"allocation_and_exposure.png": "780bf3900b07e0b45c6208f0fefa1d1e1ef7cdf7327e74738046f81b5db5eb13",
59+
"drawdown.png": "75af12f0b1d6727d33d5c03dcaca48a9dc3b5bcd93130328229eaa5483bdcec9",
60+
"equity_vs_benchmarks.png": "5d6da4b67f0139910408f88075b6897528406e8a1ff354ea6002b3e556499bde",
61+
"monthly_returns.png": "5f8682589b1e9ce845f23403c23dbc50c0aedbf9a279bdf1dd302d444ad86126",
62+
"report_overview.png": "487bad1468f44247b9f2533e8b8043e200107c7039fc27cc8413e4530e845a4c",
63+
"return_distribution.png": "00ffd9316b0974e5a5e33f3d2ffc0da630c3cf4329108cb8d7133830a804eac4",
64+
"rolling_risk.png": "f745988f5f91f276e912201e0ff7052dadf9150236f2b2464c1ad64bbe783d60",
65+
"rolling_sharpe.png": "74f989c9b405473f9eeb9208ac2ef360b222a44ded2956d0b63482cf7bdc54a0",
66+
"turnover_and_costs.png": "105243d3fbb110a7ce55e71c2b3a543e82203a6b3b9de2e7f2265d02b74154f5"
67+
}
68+
}

docs/img/report_overview.png

289 KB
Loading

0 commit comments

Comments
 (0)