Summary
Add two structure-discovery diagnostics that are standard in established chemometrics toolboxes but missing here:
- MEDA (Missing-data methods for Exploratory Data Analysis): a K x K variable-relationship map computed through a fitted PCA/PLS model, cleaner than a raw correlation matrix because it uses the model subspace.
- oMEDA (observation-based MEDA): a per-variable bar diagnostic that quantifies which variables drive the difference between two groups of observations in the model subspace.
Priority 2 of 4 in the exploratory-methods parity set (see also: ASCA/VASCA, parallel-analysis + double-CV, PLS-DA). Low implementation effort, high fit with our existing diagnostics and (downstream) with ScorePilot's brushing-and-linking UI.
Motivation
We already compute per-observation, per-variable contributions (_diagnostics.py) and a correlation-loadings plot (plots.py), but we have nothing that:
- Shows the variable-to-variable relationship structure as captured by the model (MEDA). A raw correlation heatmap mixes in directions the model discards; MEDA shows only the relationships the latent subspace actually represents, which is what you want when interpreting loadings.
- Answers the very common question "what separates these two groups of points?" directly (oMEDA). Contribution plots explain a single observation against the model center; oMEDA explains group A vs group B.
Both slot naturally into the existing convenience-method pattern (methods on PCA/PLS forwarding to standalone functions in _diagnostics.py).
What is MEDA?
MEDA produces a K x K matrix where entry (i, j) measures how strongly variable i and variable j are related within the A-component model subspace. The method is rooted in missing-data imputation theory: intuitively, for each variable it asks how well that variable is reconstructed from the others through the model, and decomposes that reconstruction by source variable. The result is a symmetric (or near-symmetric) map in [0, 1] (or [-1, 1] for the signed variant) that is then plotted as a heatmap, often seriated so that blocks of co-varying variables are visually adjacent.
Crucially MEDA is computed from the model (loadings / residuals), so increasing the number of components changes the map: it reveals which variable groupings are explained at A components.
Implementation note: implement the index faithfully from the reference below rather than from this paraphrase. The canonical algorithm builds on the model's reconstruction of the data covariance and the leave-one-variable-out imputation used in the missing-data PCA literature.
What is oMEDA?
oMEDA takes the fitted model plus a contrast (dummy) vector d over the observations, with entries +1 for group A, -1 for group B, and 0 for observations not in the comparison. It returns a length-K vector of per-variable weights showing which variables most explain the A-vs-B separation in the model subspace, rendered as a signed bar plot. This is the multivariate analogue of "diff the two groups, but only along the directions the model trusts."
Proposed implementation
- Add standalone functions to
process_improve/multivariate/_diagnostics.py:
meda(model, ...) -> K x K DataFrame (variable x variable), with an option for the seriation/ordering and for signed vs squared index.
omeda(model, group_a, group_b, ...) (or accept an explicit dummy vector d) -> length-K Series of per-variable weights.
- Bind both as convenience methods on
PCA and PLS (and, where meaningful, MBPLS/MBPCA) using the existing _model_method forwarder pattern documented in CLAUDE.md, so help/inspect.signature stay accurate and the methods are overridable.
- Plotting in
plots.py:
meda_plot(model) -> Plotly heatmap (with optional seriation), matching the existing plot styling/Plot wrapper.
omeda_plot(model, ...) -> signed bar plot, consistent with the SPE/T2 contribution bar plots already present.
- Reuse existing model attributes (
loadings_, residuals, scores_) and the missing-data machinery in _nipals.py. Do not duplicate imputation code that already exists.
ScorePilot surfacing (downstream)
This is a strong match for ScorePilot's LinkGroup brushing: brush a set of points into group A, brush another into group B, and call oMEDA to get the "what separates them" bar chart - then click a bar to drill into that variable's raw data (the linked-plots and variable-inspector machinery already exist). MEDA becomes a variable-relationship heatmap view alongside the loadings plot. All math stays upstream in process_improve; ScorePilot only adapts the output.
Acceptance criteria
meda(model) returns a K x K matrix; on a dataset with two clearly correlated variable blocks, the map shows those blocks; the map changes sensibly as n_components varies.
omeda(...) on a dataset with two groups that differ in a known subset of variables ranks those variables highest, with correct signs.
- Convenience methods present on PCA and PLS;
inspect.signature reports the underlying function minus model.
- Plots render and are covered by the existing Plotly-assertion test style.
- Tests use real datasets plus synthetic data with a known structure; NumPy-style docstrings with Examples.
References
- Camacho, J. (2010). "Missing-data theory in the context of exploratory data analysis." Chemometrics and Intelligent Laboratory Systems 103(1):8-18.
- Camacho, J. (2011). "Observation-based missing data methods for exploratory data analysis to unveil the connection between observations and variables in latent subspace models." J. Chemometrics 25:592-600. (MEDA and oMEDA.)
Notes for the implementing session
- Read
_diagnostics.py (contributions, VIP), plots.py (bar + heatmap styling), and _nipals.py (imputation) first.
- Keep functions pure over the fitted model; no state added to
__init__.
- Bump the version (MINOR) and update
CHANGELOG.md per CLAUDE.md; confirm changelog wording with the maintainer.
Summary
Add two structure-discovery diagnostics that are standard in established chemometrics toolboxes but missing here:
Priority 2 of 4 in the exploratory-methods parity set (see also: ASCA/VASCA, parallel-analysis + double-CV, PLS-DA). Low implementation effort, high fit with our existing diagnostics and (downstream) with ScorePilot's brushing-and-linking UI.
Motivation
We already compute per-observation, per-variable contributions (
_diagnostics.py) and a correlation-loadings plot (plots.py), but we have nothing that:Both slot naturally into the existing convenience-method pattern (methods on PCA/PLS forwarding to standalone functions in
_diagnostics.py).What is MEDA?
MEDA produces a K x K matrix where entry (i, j) measures how strongly variable i and variable j are related within the A-component model subspace. The method is rooted in missing-data imputation theory: intuitively, for each variable it asks how well that variable is reconstructed from the others through the model, and decomposes that reconstruction by source variable. The result is a symmetric (or near-symmetric) map in [0, 1] (or [-1, 1] for the signed variant) that is then plotted as a heatmap, often seriated so that blocks of co-varying variables are visually adjacent.
Crucially MEDA is computed from the model (loadings / residuals), so increasing the number of components changes the map: it reveals which variable groupings are explained at A components.
Implementation note: implement the index faithfully from the reference below rather than from this paraphrase. The canonical algorithm builds on the model's reconstruction of the data covariance and the leave-one-variable-out imputation used in the missing-data PCA literature.
What is oMEDA?
oMEDA takes the fitted model plus a contrast (dummy) vector
dover the observations, with entries +1 for group A, -1 for group B, and 0 for observations not in the comparison. It returns a length-K vector of per-variable weights showing which variables most explain the A-vs-B separation in the model subspace, rendered as a signed bar plot. This is the multivariate analogue of "diff the two groups, but only along the directions the model trusts."Proposed implementation
process_improve/multivariate/_diagnostics.py:meda(model, ...)-> K x K DataFrame (variable x variable), with an option for the seriation/ordering and for signed vs squared index.omeda(model, group_a, group_b, ...)(or accept an explicit dummy vectord) -> length-K Series of per-variable weights.PCAandPLS(and, where meaningful, MBPLS/MBPCA) using the existing_model_methodforwarder pattern documented inCLAUDE.md, sohelp/inspect.signaturestay accurate and the methods are overridable.plots.py:meda_plot(model)-> Plotly heatmap (with optional seriation), matching the existing plot styling/Plotwrapper.omeda_plot(model, ...)-> signed bar plot, consistent with the SPE/T2 contribution bar plots already present.loadings_, residuals,scores_) and the missing-data machinery in_nipals.py. Do not duplicate imputation code that already exists.ScorePilot surfacing (downstream)
This is a strong match for ScorePilot's
LinkGroupbrushing: brush a set of points into group A, brush another into group B, and call oMEDA to get the "what separates them" bar chart - then click a bar to drill into that variable's raw data (the linked-plots and variable-inspector machinery already exist). MEDA becomes a variable-relationship heatmap view alongside the loadings plot. All math stays upstream inprocess_improve; ScorePilot only adapts the output.Acceptance criteria
meda(model)returns a K x K matrix; on a dataset with two clearly correlated variable blocks, the map shows those blocks; the map changes sensibly asn_componentsvaries.omeda(...)on a dataset with two groups that differ in a known subset of variables ranks those variables highest, with correct signs.inspect.signaturereports the underlying function minusmodel.References
Notes for the implementing session
_diagnostics.py(contributions, VIP),plots.py(bar + heatmap styling), and_nipals.py(imputation) first.__init__.CHANGELOG.mdperCLAUDE.md; confirm changelog wording with the maintainer.