Add component-selection methods: Horn's Parallel Analysis, ckf, and double cross-validation

[kgdunn]

Summary

Broaden how the library chooses the number of components. Today PCA uses Wold's PRESS-ratio criterion (PCA.select_n_components) and PLS uses minimum RMSECV. Established chemometrics tooling additionally offers, and practitioners frequently prefer:

• Horn's Parallel Analysis (a widely respected eigenvalue-vs-random criterion we lack entirely),
• column-wise k-fold cross-validation for PCA (ckf) (and element-wise, ekf), which behave better than naive PCA CV, and
• double cross-validation (2CV) for PLS, the rigorous way to avoid optimistic component selection.

Priority 3 of 4 in the exploratory-methods parity set (see also: ASCA/VASCA, MEDA/oMEDA, PLS-DA). Small, self-contained, immediately useful.

Motivation

Component selection drives every downstream interpretation. Relying on a single criterion is fragile:

• Wold's R / PRESS-ratio can be unstable and is sensitive to the threshold.
• Naive PCA cross-validation (leaving out whole rows and reconstructing) is known to be biased because the left-out rows still influence centering/scaling and the subspace; ckf (column-wise k-fold) and ekf (element-wise k-fold) correct this and are the recommended PCA CV schemes.
• For PLS, selecting A on the same CV used to report performance is optimistic; double CV separates selection (inner loop) from performance estimation (outer loop).
• Parallel Analysis gives an entirely different, distribution-based answer (keep components whose eigenvalue beats what random data of the same shape produces) and is a strong cross-check that costs little to add.

Offering several criteria, and letting them agree or disagree, is more trustworthy than one number.

What to add

1. Horn's Parallel Analysis

Generate B surrogate datasets of the same shape (N x K). Two surrogate strategies, both worth supporting:

• random normal data (classic Horn), and
• column-wise permutation of the real data (Buja and Eyuboglu) which preserves marginals.

For each surrogate compute the sorted eigenvalues; retain real component a while its eigenvalue exceeds a chosen percentile (default 95th) of the surrogate eigenvalue distribution at position a.

parallel_analysis(X, n_surrogates=1000, method="permutation"|"normal", percentile=95, random_state=...) -> a result Bunch with the suggested component count, the real eigenvalues, and the surrogate percentile curve (so it can be plotted: real scree vs random scree).

2. ckf / ekf cross-validation for PCA

Implement the column-wise k-fold (ckf) algorithm (and element-wise, ekf) for PCA reconstruction error, following Camacho and Ferrer (2012). Expose as new method= options on the existing PCA.select_n_components (keep "wold" as default for back-compatibility; add "ckf", "ekf"). Return the cumulative press/Q2-style curve so callers can plot it.

3. Double cross-validation for PLS

Add PLS.double_cross_validate(...) (or a mode="double" on the existing CV entry point): an outer CV loop for unbiased performance, with an inner CV loop selecting A on each outer-training partition. Return per-outer-fold selected A, the distribution of selected A, and the unbiased outer Q2 / RMSE. Reuse the existing CV plumbing in _pls.py (cross_validate, select_n_components) rather than writing a new splitter; honor random_state.

Proposed implementation

• Math in process_improve/multivariate/ (a new _model_selection.py, or extend _pca.py / _pls.py where the existing selection logic lives). Re-export public entry points via methods.py.
• Follow repo conventions: results returned as sklearn.utils.Bunch with named fields (matches the existing select_n_components return style); accept custom sklearn CV splitters where applicable; random_state for reproducibility.
• Add a plot helper (scree vs random-eigenvalue overlay for parallel analysis; CV-curve plot for ckf/double-CV) consistent with the existing explained_variance_plot styling.
• Do not reimplement generic CV splitting - reuse sklearn splitters and the existing helpers.

ScorePilot surfacing (downstream)

ScorePilot's component explorer currently auto-recommends via Wold (PCA) / RMSECV (PLS). Once these exist, it can show a small "how many components?" panel: the parallel-analysis scree overlay and the ckf/double-CV curve, letting the user see where the criteria agree. Math stays upstream; ScorePilot adapts the Bunch output.

Acceptance criteria

• parallel_analysis on data with a known low-rank signal plus noise recovers the correct number of components; deterministic with a fixed random_state; returns the surrogate percentile curve for plotting.
• PCA.select_n_components(method="ckf") runs and returns a reconstruction-error curve; documented difference vs "wold".
• PLS.double_cross_validate returns outer-fold-selected A, the selection distribution, and an unbiased outer performance estimate; reproducible with random_state.
• Back-compatible: existing default behavior of select_n_components unchanged.
• Tests on synthetic low-rank data and at least one real dataset; NumPy-style docstrings with Examples.

References

• Horn, J.L. (1965). "A rationale and test for the number of factors in factor analysis." Psychometrika 30:179-185.
• Buja, A., Eyuboglu, N. (1992). "Remarks on parallel analysis." Multivariate Behavioral Research 27(4):509-540.
• Camacho, J., Ferrer, A. (2012). "Cross-validation in PCA: a comparative study." J. Chemometrics 26:361-373. (ckf / ekf.)
• Westerhuis, J.A. et al. (2008). "Assessment of PLSDA cross validation." Metabolomics 4:81-89. (double CV rationale.)

Notes for the implementing session

• Read the existing select_n_components in _pca.py and _pls.py and the cross_validate method before adding new modes - extend, don't fork.
• Keep return types consistent with the existing Bunch conventions.
• Bump the version (MINOR) and update CHANGELOG.md per CLAUDE.md; confirm changelog wording with the maintainer.