28 lines (21 loc) · 1.2 KB

Retrieval Dev Multi-Seed Summary

Comparison: single_interest -> trace

Seed	Baseline NDCG@20	TRACE NDCG@20	Rel. gain	Baseline Recall@50	TRACE Recall@50	Rel. gain
20260714	0.011037	0.012203	10.57%	0.023709	0.033158	39.86%
20260715	0.009499	0.012161	28.02%	0.020107	0.031787	58.08%
20260716	0.007908	0.008387	6.05%	0.017675	0.022068	24.86%

Three-Seed Aggregate

Metric	Baseline mean	TRACE mean	Relative change	TRACE std
ndcg@20	0.009481	0.010917	15.14%	0.002191
recall@50	0.020497	0.029004	41.50%	0.006046
coverage@50	0.797834	0.712376	-10.71%	0.004803
strict_cold_recall@50	0.016546	0.027253	64.72%	0.009092
warm_tail_recall@50	0.031983	0.043243	35.21%	0.005663

Evidence Gates

Positive direction in all seeds: True
Mean-user paired bootstrap NDCG@20 CI excludes zero: True
Mean-user paired bootstrap Recall@50 CI excludes zero: True
LogQ negative result replicated: True

This report contains Dev evidence only. Consult reports/final_results_summary.json for the separate audited outcome.