Add PLS-DA (PLS discriminant analysis) for classification

[kgdunn]

Summary

Add PLS-DA (Partial Least Squares Discriminant Analysis), the standard chemometrics classification method, built on the existing PLS estimator. Established tooling treats classification as a first-class workflow; we currently offer PLS regression only, despite ScorePilot already carrying a CLASS identifier role on columns (the metadata exists, the method does not).

Priority 4 of 4 in the exploratory-methods parity set (see also: ASCA/VASCA, MEDA/oMEDA, parallel-analysis + double-CV).

Motivation

A huge fraction of multivariate analysis is "which class does this sample belong to, and why?": good vs bad batch, supplier A vs B vs C, product grade, defect mode. PLS-DA is the workhorse for this because it inherits everything that makes PLS attractive - it handles many correlated variables, more variables than samples, and yields interpretable loadings/VIP/contributions - while producing a classifier. We already have a solid PLS implementation to build on, so the incremental cost is modest.

What is PLS-DA?

PLS-DA is ordinary PLS regression where the response Y is a class-indicator matrix:

• For G classes, encode Y as an N x G dummy matrix (one-hot; the binary case can use a single +1/-1 column).
• Fit PLS of X on this dummy Y.
• Predict the dummy response for new samples and assign the class by a decision rule: argmax of the predicted columns, or a calibrated per-class threshold (Bayesian threshold from the predicted-value distributions) for better handling of unequal priors / "none of the above".

Beyond hard labels, PLS-DA should expose the usual classification diagnostics: confusion matrix, per-class sensitivity / specificity, overall accuracy, and (binary) ROC/AUC, plus a permutation test of model validity (permute the labels, refit, compare classification performance to the null) because PLS-DA is notoriously prone to overfitting / chance separation on wide data.

Proposed implementation

• New PLSDA class in process_improve/multivariate/ (likely _plsda.py), re-exported from methods.py.
• Compose on / subclass the existing PLS (reuse its NIPALS fit, scores, loadings, weights, VIP, contributions, T2/SPE) - do not reimplement the PLS math. Add the classification layer on top.
• sklearn-compatible: inherit BaseEstimator + ClassifierMixin (and keep TransformerMixin for transform to scores), following the repo's "lightweight mixins, no coupling to a concrete sklearn estimator" rule in CLAUDE.md.
  • fit(X, y) accepts a 1-D label vector (strings or ints); build the dummy Y internally; store classes_.
  • predict(X) -> predicted class labels.
  • predict_proba(X) (or decision_function) -> per-class scores; document the chosen calibration (softmax of predicted dummies, or Bayesian-threshold posterior).
  • score(X, y) -> accuracy (sklearn convention, higher is better).
• Decision rule configurable: "max" (argmax) default, plus "bayes" threshold option.
• Diagnostics methods/attributes: confusion_matrix_, per-class sensitivity_ / specificity_, accuracy_; a permutation_test(n_permutations=..., random_state=...) returning the null distribution and a p-value; binary roc_auc.
• Carry over the standard PLS plots (score plot colored by class, loadings, VIP) and add a confusion-matrix view in plots.py.
• Cross-validated classification: reuse the CV plumbing from _pls.py; ideally integrate with the double cross-validation added in the component-selection issue, since PLS-DA is exactly where naive CV is most misleading.

ScorePilot surfacing (downstream)

ScorePilot already lets a column be tagged with the CLASS identifier role. With PLS-DA upstream, ScorePilot can offer a "build a classifier" path: pick the class column, fit PLS-DA, show the class-colored score plot, confusion matrix, per-class metrics, VIP, and the permutation-test p-value. All math stays in process_improve per the ScorePilot core/ boundary rule.

Acceptance criteria

• PLSDA fits on a labeled dataset (2-class and multi-class), predicts labels, and exposes predict_proba.
• Confusion matrix, per-class sensitivity/specificity, and accuracy are correct on a held-out split.
• Permutation test distinguishes a genuinely separable dataset from a label-shuffled one (p-value behaves correctly), reproducible with random_state.
• sklearn compatibility verified: works in a Pipeline, supports clone, get_params/set_params.
• Tests cover binary and multi-class, real datasets plus synthetic separable/non-separable data; NumPy-style docstrings with Examples.

References

• Barker, M., Rayens, W. (2003). "Partial least squares for discrimination." J. Chemometrics 17:166-173.
• Brereton, R.G., Lloyd, G.R. (2014). "Partial least squares discriminant analysis: taking the magic away." J. Chemometrics 28:213-225.
• Westerhuis, J.A. et al. (2008). "Assessment of PLSDA cross validation." Metabolomics 4:81-89. (validation pitfalls / permutation testing.)

Notes for the implementing session

• Read _pls.py thoroughly first; the goal is a thin classification layer over it, reusing scores/loadings/VIP/contributions.
• The legacy course material already covers PLS-DA conceptually (classification.tex referenced in case-study issue Case study: FMC batch process — multiblock batch PLS #154), useful for narrative/docs.
• Bump the version (MINOR - new model type) and update CHANGELOG.md per CLAUDE.md; confirm changelog wording with the maintainer.