Component selection: remaining follow-ups (post cross-validation work)

[kgdunn]

Tracking issue for the remaining, non-blocking follow-ups to the component-selection / cross-validation work. The core feature is shipped and merged; this captures what's left.

Already shipped (context)

• Adopt process-improve 1.38 cross-validation with selectable rules #84 — adopt process-improve 1.38 cross-validation + full rule selector (PCA ekf/row_wise, rules 1se/min/q2_increment/randomization).
• Component selection Phase 2: Q² 1-SE band, stability badges, and docs #85 — Phase 2: ±1 SE band on the R²/Q² plot, PLS stability / vote-share badges, docs/COMPONENT_SELECTION.md.
• Consume process-improve 1.39 q2_se; remove local Q²-SE adapter #86 — consume process-improve 1.39's q2_se directly; deleted the local _q2_standard_error adapter (upstreamed in process-improve#410).

These items are also listed in docs/COMPONENT_SELECTION.md (Open items / TODO).

To do

• Surface the randomization p-values. When the PLS randomization rule is selected, process_improve returns per-component randomization_pvalues; the ScorePilot adapter (core/cross_validation.py) currently drops them. Thread them through CrossValidation → the API schema → the R²/Q² card as a per-component significance annotation. Only meaningful for the PLS randomization rule (null otherwise).

• Show the full vote distribution, not just the modal share. We surface recommended_vote_share (modal) and the unstable flag, but process_improve also returns the full selection_distribution (vote share per component count across CV repeats). Expose it (e.g. a small bar under the recommendation) so an unstable choice shows where the alternatives landed. PLS only.

• Persist the chosen rule / scheme per variant. Today ComponentExplorer's rule and cv_scheme reset to the per-kind defaults on every model load; they are not stored on Model. To let a variant remember "this model was chosen with q2_increment / row_wise", add a field (in preprocessing JSON or a real column) and round-trip it through fit + the cross-validation endpoint.

• Cross-validation performance. PLS defaults to n_repeats=10, so the /cross-validation endpoint refits 10 · n_splits models on every rule/scheme change. Fine at current dataset sizes; if it gets sluggish on large blocks, cache by (model_id, rule, scheme, ceiling) or lower the default n_repeats above some size threshold.

• Multi-Y PLS Q² breakdown. Only the aggregate ("total") Q²Y curve is shown. process_improve's r2y_validated already has per-response columns; offer a per-Y Q²Y breakdown for multi-target PLS models.

Notes

• The reusable LineSeries.band capability (added in Component selection Phase 2: Q² 1-SE band, stability badges, and docs #85) is generic; any future band work should reuse it rather than adding domain logic to lib/plots.
• Keep the core/ boundary clean: if any of these needs a statistic process_improve doesn't expose, prefer upstreaming it (as we did for q2_se) over a local reimplementation.

Filed at the end of the cross-validation work session; non-urgent.