Skip to content

Commit 2529867

Browse files
Pigbibicodex
andauthored
fix: gate recommendation review by horizon maturity (#20)
* fix: gate recommendation review by horizon maturity Co-Authored-By: Codex <noreply@openai.com> * fix: require report-date price coverage Co-Authored-By: Codex <noreply@openai.com> * fix: bound review start price delay Co-Authored-By: Codex <noreply@openai.com> * fix: tighten review date and maturity boundaries Co-Authored-By: Codex <noreply@openai.com> --------- Co-authored-by: Codex <noreply@openai.com>
1 parent a6c28ab commit 2529867

5 files changed

Lines changed: 235 additions & 30 deletions

File tree

docs/advisory_contract.md

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -226,7 +226,7 @@ Recommendation follow-up reviews are separate audit artifacts generated from
226226
past final recommendations and cached point-in-time prices:
227227

228228
```text
229-
schema_version = 1
229+
schema_version = 2
230230
mode = recommendation_review
231231
as_of
232232
generated_at
@@ -240,12 +240,15 @@ policy
240240

241241
Each `review_items[]` row carries the original report date, review date, symbol,
242242
primary horizon, price interval, absolute return, benchmark return,
243-
benchmark-relative return, outcome label, market data source, and original
244-
selection scores when present.
243+
benchmark-relative return, trading-observation count, maturity status, outcome
244+
label, market data source, and original selection scores when present. The
245+
minimum maturity is 10 trading days for short/medium and 252 trading days for
246+
long. Summary performance metrics are reported separately for each horizon.
245247

246248
Allowed outcome labels:
247249

248250
- `pending`
251+
- `in_progress`
249252
- `insufficient_price_data`
250253
- `outperforming`
251254
- `inline`

docs/system_design.md

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -116,8 +116,13 @@ price source without turning the repository into a paid market-data store.
116116

117117
Recommendation follow-up review is a separate artifact. It reads past final
118118
recommendations, cached prices, and a benchmark, then reports absolute and
119-
relative returns by horizon. It is used for research accountability and data
120-
quality checks; it does not create new recommendations or execution targets.
119+
relative returns by horizon. Maturity is measured in trading observations:
120+
short and medium horizons require at least 10 trading days, while long requires
121+
252. Before maturity, an item remains `pending` or `in_progress` and cannot be
122+
labeled `outperforming` or `lagging`. Summary metrics (sample size, mean,
123+
median, and hit rate) stay within each horizon, and top symbols are
124+
de-duplicated. It is used for research accountability and data quality checks;
125+
it does not create new recommendations or execution targets.
121126

122127
A separate no-network smoke command validates the three-repository contract:
123128

docs/system_design.zh-CN.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -105,7 +105,7 @@ scripts/build_advisory_artifacts.py
105105

106106
市场确认现在在 Yahoo chart 免费入口外面加了一层轻量价格缓存。线上 workflow 会用 GitHub Actions cache 恢复和保存 `.cache/market-data`。这样 Yahoo 临时不可用时,公开报告仍可以尽量用近期缓存继续生成;推荐跟踪复盘也可以使用 point-in-time 价格来源,而不把本仓库变成付费行情存储仓库。
107107

108-
推荐跟踪复盘是单独 artifact。它读取历史最终推荐、缓存价格和基准指数,按周期计算绝对收益、相对收益和结果状态。它只用于研究问责和数据质量检查,不生成新的推荐,也不输出执行目标。
108+
推荐跟踪复盘是单独 artifact。它读取历史最终推荐、缓存价格和基准指数,按周期计算绝对收益、相对收益和结果状态。复盘按交易日判断成熟度:短线和中线至少 10 个交易日、长线至少 252 个交易日;未达到门槛只能是 `pending``in_progress`,不得提前标记为 `outperforming`/`lagging`。汇总只在各周期内部计算样本量、平均值、中位数和命中率,领先标的去重。它只用于研究问责和数据质量检查,不生成新的推荐,也不输出执行目标。
109109

110110
跨仓库契约用 no-network smoke 命令验证:
111111

src/quant_advisor_research/recommendation_review.py

Lines changed: 102 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
import argparse
44
import datetime as dt
55
import json
6+
import statistics
67
from pathlib import Path
78
from typing import Any
89

@@ -20,6 +21,8 @@
2021
"medium": "中线",
2122
"long": "长线",
2223
}
24+
MIN_MATURITY_TRADING_DAYS = {"short": 10, "medium": 10, "long": 252}
25+
MAX_START_BAR_DELAY_DAYS = 7
2326

2427

2528
def utc_now_iso() -> str:
@@ -46,10 +49,23 @@ def final_recommendations(report: dict[str, Any]) -> list[dict[str, Any]]:
4649
def first_bar_on_or_after(bars: list[PriceBar], target: dt.date) -> PriceBar | None:
4750
for bar in sorted(bars, key=lambda item: item.date):
4851
if bar.date >= target:
49-
return bar
52+
if (bar.date - target).days <= MAX_START_BAR_DELAY_DAYS:
53+
return bar
54+
return None
5055
return None
5156

5257

58+
def start_bar_for_report(bars: list[PriceBar], target: dt.date) -> PriceBar | None:
59+
ordered = sorted(bars, key=lambda item: item.date)
60+
exact = next((bar for bar in ordered if bar.date == target), None)
61+
if exact:
62+
return exact
63+
previous = last_bar_on_or_before(ordered, target)
64+
if previous and (target - previous.date).days <= MAX_START_BAR_DELAY_DAYS:
65+
return previous
66+
return first_bar_on_or_after(ordered, target)
67+
68+
5369
def last_bar_on_or_before(bars: list[PriceBar], target: dt.date) -> PriceBar | None:
5470
candidates = [bar for bar in bars if bar.date <= target]
5571
return max(candidates, key=lambda item: item.date) if candidates else None
@@ -61,11 +77,20 @@ def return_between(start: PriceBar, end: PriceBar) -> float:
6177
return round(end.close / start.close - 1, 6)
6278

6379

64-
def outcome_label(relative_return: float | None, *, elapsed_days: int, has_price_data: bool) -> str:
80+
def outcome_label(
81+
relative_return: float | None,
82+
*,
83+
elapsed_days: int,
84+
has_price_data: bool,
85+
horizon: str,
86+
trading_intervals: int,
87+
) -> str:
6588
if elapsed_days <= 0:
6689
return "pending"
6790
if not has_price_data:
6891
return "insufficient_price_data"
92+
if trading_intervals < MIN_MATURITY_TRADING_DAYS.get(horizon, MIN_MATURITY_TRADING_DAYS["medium"]):
93+
return "in_progress"
6994
if relative_return is None:
7095
return "insufficient_price_data"
7196
if relative_return >= 0.02:
@@ -109,9 +134,9 @@ def build_review_item(
109134
data_source: str,
110135
) -> dict[str, Any]:
111136
symbol = str(pick.get("symbol", "")).upper()
112-
start_bar = first_bar_on_or_after(symbol_bars, report_as_of)
137+
start_bar = start_bar_for_report(symbol_bars, report_as_of)
113138
end_bar = last_bar_on_or_before(symbol_bars, review_as_of)
114-
benchmark_start = first_bar_on_or_after(benchmark_bars, report_as_of)
139+
benchmark_start = start_bar_for_report(benchmark_bars, report_as_of)
115140
benchmark_end = last_bar_on_or_before(benchmark_bars, review_as_of)
116141
has_price_data = bool(start_bar and end_bar and start_bar.date <= end_bar.date)
117142

@@ -131,21 +156,41 @@ def build_review_item(
131156
relative_return = round(absolute_return - benchmark_return, 6)
132157

133158
elapsed_days = (review_as_of - report_as_of).days
159+
horizon = str(pick.get("primary_horizon", ""))
160+
maturity_days = MIN_MATURITY_TRADING_DAYS.get(horizon, MIN_MATURITY_TRADING_DAYS["medium"])
161+
trading_intervals = max(trading_observations - 1, 0)
162+
if elapsed_days <= 0:
163+
maturity_status = "pending"
164+
elif not has_price_data:
165+
maturity_status = "insufficient_price_data"
166+
elif trading_intervals < maturity_days:
167+
maturity_status = "in_progress"
168+
else:
169+
maturity_status = "matured"
134170
return {
135171
"report_as_of": report_as_of.isoformat(),
136172
"review_as_of": review_as_of.isoformat(),
137173
"symbol": symbol,
138174
"name": str(pick.get("name", "")),
139-
"primary_horizon": str(pick.get("primary_horizon", "")),
175+
"primary_horizon": horizon,
140176
"primary_horizon_label": str(pick.get("primary_horizon_label", "")),
141177
"start_price_date": start_date,
142178
"end_price_date": end_date,
143179
"elapsed_calendar_days": elapsed_days,
144180
"trading_observations": trading_observations,
181+
"trading_intervals": trading_intervals,
182+
"maturity_required_trading_days": maturity_days,
183+
"maturity_status": maturity_status,
145184
"absolute_return": absolute_return,
146185
"benchmark_return": benchmark_return,
147186
"relative_return": relative_return,
148-
"outcome": outcome_label(relative_return, elapsed_days=elapsed_days, has_price_data=has_price_data),
187+
"outcome": outcome_label(
188+
relative_return,
189+
elapsed_days=elapsed_days,
190+
has_price_data=has_price_data,
191+
horizon=horizon,
192+
trading_intervals=trading_intervals,
193+
),
149194
"market_data_source": data_source if has_price_data else "",
150195
"combined_score": pick.get("combined_score"),
151196
"source_score": pick.get("source_score"),
@@ -157,24 +202,49 @@ def average(values: list[float]) -> float | None:
157202
return round(sum(values) / len(values), 6) if values else None
158203

159204

205+
def median(values: list[float]) -> float | None:
206+
return round(float(statistics.median(values)), 6) if values else None
207+
208+
160209
def summarize_items(items: list[dict[str, Any]]) -> dict[str, Any]:
161-
evaluated = [item for item in items if isinstance(item.get("relative_return"), (int, float))]
210+
evaluated = [
211+
item
212+
for item in items
213+
if item.get("maturity_status") == "matured" and isinstance(item.get("relative_return"), (int, float))
214+
]
162215
by_horizon: dict[str, dict[str, Any]] = {}
163216
for horizon in ("short", "medium", "long"):
164217
horizon_items = [item for item in items if item.get("primary_horizon") == horizon]
165-
horizon_evaluated = [item for item in horizon_items if isinstance(item.get("relative_return"), (int, float))]
218+
horizon_evaluated = [item for item in evaluated if item.get("primary_horizon") == horizon]
219+
horizon_returns = [float(item["relative_return"]) for item in horizon_evaluated]
220+
horizon_ranked = sorted(
221+
[item for item in horizon_evaluated if item.get("outcome") == "outperforming"],
222+
key=lambda item: (float(item.get("relative_return", 0)), str(item.get("symbol", ""))),
223+
reverse=True,
224+
)
166225
by_horizon[horizon] = {
167226
"label": HORIZON_LABELS_ZH[horizon],
168227
"item_count": len(horizon_items),
169228
"evaluated_count": len(horizon_evaluated),
229+
"sample_size": len(horizon_evaluated),
170230
"pending_count": sum(1 for item in horizon_items if item.get("outcome") == "pending"),
231+
"in_progress_count": sum(1 for item in horizon_items if item.get("outcome") == "in_progress"),
232+
"matured_count": sum(1 for item in horizon_items if item.get("maturity_status") == "matured"),
171233
"insufficient_price_data_count": sum(
172234
1 for item in horizon_items if item.get("outcome") == "insufficient_price_data"
173235
),
174-
"average_relative_return": average([float(item["relative_return"]) for item in horizon_evaluated]),
236+
"average_relative_return": average(horizon_returns),
237+
"median_relative_return": median(horizon_returns),
238+
"hit_rate": round(
239+
sum(1 for item in horizon_evaluated if item.get("outcome") == "outperforming") / len(horizon_evaluated),
240+
6,
241+
)
242+
if horizon_evaluated
243+
else None,
244+
"top_outperformers": unique_symbols(horizon_ranked),
175245
}
176246
ranked = sorted(
177-
evaluated,
247+
[item for item in evaluated if item.get("outcome") == "outperforming"],
178248
key=lambda item: (float(item.get("relative_return", 0)), str(item.get("symbol", ""))),
179249
reverse=True,
180250
)
@@ -183,12 +253,24 @@ def summarize_items(items: list[dict[str, Any]]) -> dict[str, Any]:
183253
"evaluated_count": len(evaluated),
184254
"pending_count": sum(1 for item in items if item.get("outcome") == "pending"),
185255
"insufficient_price_data_count": sum(1 for item in items if item.get("outcome") == "insufficient_price_data"),
186-
"average_relative_return": average([float(item["relative_return"]) for item in evaluated]),
256+
# Do not pool short-, medium-, and long-horizon performance into one statistic.
257+
"average_relative_return": None,
258+
"median_relative_return": None,
259+
"hit_rate": None,
187260
"by_horizon": by_horizon,
188-
"top_outperformers": [item["symbol"] for item in ranked[:5]],
261+
"top_outperformers": unique_symbols(ranked)[:5],
189262
}
190263

191264

265+
def unique_symbols(items: list[dict[str, Any]]) -> list[str]:
266+
symbols: list[str] = []
267+
for item in items:
268+
symbol = str(item.get("symbol", ""))
269+
if symbol and symbol not in symbols:
270+
symbols.append(symbol)
271+
return symbols[:5]
272+
273+
192274
def build_recommendation_review(
193275
*,
194276
report_paths: list[str | Path],
@@ -245,7 +327,7 @@ def build_recommendation_review(
245327
)
246328

247329
return {
248-
"schema_version": "1",
330+
"schema_version": "2",
249331
"mode": "recommendation_review",
250332
"as_of": as_of.isoformat(),
251333
"generated_at": utc_now_iso(),
@@ -274,9 +356,9 @@ def render_recommendation_review_markdown(review: dict[str, Any]) -> str:
274356
f"- 复盘条目:{summary.get('item_count', 0)}",
275357
f"- 已可评估:{summary.get('evaluated_count', 0)}",
276358
f"- 待观察:{summary.get('pending_count', 0)}",
359+
f"- 进行中:{sum(item.get('in_progress_count', 0) for item in summary.get('by_horizon', {}).values())}",
277360
f"- 缺少价格数据:{summary.get('insufficient_price_data_count', 0)}",
278-
f"- 平均相对收益:{display_percent(summary.get('average_relative_return'))}",
279-
f"- 领先标的:{', '.join(summary.get('top_outperformers', [])) or '暂无'}",
361+
"- 不同持有期不合并计算平均收益;以下按周期分别统计。",
280362
"",
281363
"## 周期分布",
282364
"",
@@ -286,7 +368,9 @@ def render_recommendation_review_markdown(review: dict[str, Any]) -> str:
286368
item = summary.get("by_horizon", {}).get(horizon, {})
287369
lines.append(
288370
f"- {item.get('label', horizon)}{item.get('evaluated_count', 0)}/{item.get('item_count', 0)} 已评估,"
289-
f"平均相对收益 {display_percent(item.get('average_relative_return'))}"
371+
f"样本量 {item.get('sample_size', 0)},平均 {display_percent(item.get('average_relative_return'))},"
372+
f"中位数 {display_percent(item.get('median_relative_return'))},命中率 {display_percent(item.get('hit_rate'))},"
373+
f"领先标的 {', '.join(item.get('top_outperformers', [])) or '暂无'}"
290374
)
291375
warnings = review.get("data_quality_warnings", [])
292376
if warnings:
@@ -302,6 +386,8 @@ def render_recommendation_review_markdown(review: dict[str, Any]) -> str:
302386
f"- 价格区间:{item.get('start_price_date') or '无'}{item.get('end_price_date') or '无'}",
303387
f"- 绝对收益:{display_percent(item.get('absolute_return'))}",
304388
f"- 相对 {review.get('benchmark')}{display_percent(item.get('relative_return'))}",
389+
f"- 成熟度:{item.get('maturity_status')}(需要 {item.get('maturity_required_trading_days', 0)} 个交易日,"
390+
f"当前 {item.get('trading_observations', 0)} 个)",
305391
f"- 状态:{item.get('outcome')}",
306392
"",
307393
]

0 commit comments

Comments
 (0)