epic: characterization results reporting — report / verdict / AI over one archived record

[jonathaneoliver]

Spec

Design doc: docs/characterization-results-design.md (PR #878).

Model

The three consumers of a characterization run — a written report (quantitative "how well"), a good/bad verdict, and AI input (the sweep) — are three projections over one archived per-play record {config (IV), continuous metrics (DV), labels (thresholded), hypothesis?}. Labels are a lossy, thresholded view (s1=1.2s and s6=5s both get no qoe_vst_breach), so the continuous report is first-class — the only view that ranks the "good" configs. The report's unit is the study (N reps × M variations), not the physical ≤4-concurrent group.

Scope (gaps from the design doc)

• Gap 0 — Study join key. study_id (stamp at launch) and/or a saved scenario facet filter, plus a control_events correlation to strip environmental (non-config) failures at small-N, and interleave configs within reps.
• Gap 1 — Study report renderer. Join per-arm config × plays-summary × labels into a comparison + IV→DV response curve; always show the metric value even when "good"; reps → distribution. UI-first. (First cut, with Gap 0.)
• Gap 2 — Wire the verdict. Populate the dead char-matrix verdict column from worstSeverity() + qoe_tier_* (absolute, class-aware).
• Gap 3 — Baseline / relative regression. A golden-run/prior-run diff — the thing labels can't catch (TTFF drift with no stall).
• Gap 4 — Hypothesis field. Add why/why_text to char-matrix/mode runs to complete the experiment-record model + feed the AI.
• Gap 5 — Mode-test convergence. Decide: mode tests drop their in-process Report.Summary for the archive plays-summary, or keep only their sample-rich per-step detail.

Recommended first cut

Gap 0 + Gap 1 together — deliver the study report over the archive (subsumes the current hand-query). Verdict-wiring (Gap 2) and baselines (Gap 3) layer on after.

[jonathaneoliver]

Agile sizing (epic)

• Points: 21 — epic-scale: spans 6 gaps across the forwarder (study join, report query, verdict), the dashboard (UI study report), and char-matrix (verdict wiring, hypothesis) + a design decision. Should be broken into per-gap stories; Gap 0 + Gap 1 is the natural first story.
• Effort: ~2–3 weeks for the full epic — Gap 0+1 first cut ≈ 1 week (the join + the study report over the existing plays-summary/scenario/labels), the remaining gaps layer on.
• Priority: P2 (next sprint) — the design is landed (docs(characterization): results architecture design doc #878) and the substrate already exists; this is post-design foundational build, no active fire.
• Value: high — delivers the quantitative performance report (valuable in its own right, per the design) and unifies report / verdict / AI over one archived record. Labels alone are a lossy thresholded view that can't rank the "good" configs, so the continuous report is a genuinely new capability.

Rationale for the non-obvious calls: P2 over P1 because it's foundational-not-urgent (design-doc-first, substrate in place); 21 not lower because it's a multi-component, multi-PR epic rather than one story — split before scheduling.