Skip to content

Attribute the two PCA cross-validation schemes correctly - #289

Merged
kgdunn merged 1 commit into
mainfrom
claude/pca-cross-validation-attribution
Sep 11, 2026
Merged

Attribute the two PCA cross-validation schemes correctly#289
kgdunn merged 1 commit into
mainfrom
claude/pca-cross-validation-attribution

Conversation

@kgdunn

@kgdunn kgdunn commented Sep 10, 2026

Copy link
Copy Markdown
Owner

Two claims in latent-variable-modelling/principal-component-analysis/determining-the-number-of-components-to-use-in-the-model-with-cross-validation.rst are wrong. Both are live on the site at <https://learnche.org/pid/latent-variable-modelling/principal-component-analysis/determining-the-number-of-components-to-use-in-the-model-with-cross-validation>. They surfaced while researching a question about why a PCA is cross-validated cell by cell rather than row by row, and they are unrelated to the batch case studies in #273, which is why this is a separate pull request.

The Wold attribution

The section describes building the model on the retained rows, projecting the held-out rows onto the loadings to get their scores, reconstructing and taking residuals, and calls it "Wold's cross-validation procedure". Wold's 1978 paper, the one that introduced cross-validation for component models, did not hold out rows. It held out a scattered pattern of individual elements along pseudo-diagonals and estimated them as missing values with NIPALS.

The consequence sat further down, where the text had Bro and co-authors recommending element deletion "instead". That stages a correction of Wold that never happened, because Wold was already deleting elements thirty years earlier.

Changed:

  • The procedure is named for what it is, a row-wise procedure, in both the description and the caution that follows it.
  • The review's contribution is stated as the independence requirement it actually is: a prediction must not use the value it is predicting.
  • The element-wise remedy is credited to Wold, with Eastment and Krzanowski (1982), Technometrics, 24, 73-77 named for the two-model route to the same property, and the row-wise procedure described as the later simplification.

I could not read Wold (1978) directly, since it is paywalled with no open copy. Four mutually independent secondary sources agree on what it did: Owen and Perry in the Annals of Applied Statistics, Perry's Stanford thesis, Camacho and Ferrer, and the Bioconductor pcaMethods documentation.

The Simca-P label

The section says the bar plot is "Simca-P's output, from the row-wise scheme", and builds a "same data, both schemes" comparison on that. Simca-P cross-validates a PCA element-wise, by the Eastment and Krzanowski route: rows are held out to estimate the loadings, columns to estimate the scores. Its row-wise cross-validation is reserved for PLS and OPLS, which is the likely source of the mix-up. The Simca 15 User Guide states this in section 16.8.1.

The recorded curve agrees. In figures/pca/barplot-for-R2-and-Q2.R the Simca-P series falls at components 3, 4, 6 and 7. The row-projection scheme cannot produce that: its loadings are nested, so the reconstruction error can only shrink as components are added, and a Q² of one minus PRESS over the total sum of squares cannot dip. The ProSensus series recorded beside it never falls and tracks R² to within a few points, which is what the row-projection scheme does produce.

Changed:

  • The heading and framing become "the same data, two element-wise predictors", naming what each of the two actually does.
  • The closing sentence had drawn a contrast that no longer exists, since it credited the element-wise scheme with not letting a held-out value contribute to its own prediction as though the other curve did. It now draws the conclusion that follows: two schemes forming their predictions differently read these data the same way.

Verification

  • make check-code-file on this file, which runs the chapter in order against the released library: 4 blocks, all passed. The element-wise curve is unchanged at 0.292, 0.404, 0.374, 0.343, 0.341, 0.240, 0.100, 0.193, 0.696, 0.850, 0.807.
  • make text: zero warnings.
  • No number in the section moved. No code block was touched.
  • No em-dashes in the added prose.
  • CITATION.cff: version 2026.09.10, date-released 2026-09-10.

Paired change

The same mislabel is in the figure's legend. kgdunn/figures#96 corrects it and regenerates pca/q2-across-packages.png. That one should merge first, since the PDF build resolves figures from the figures repository's default branch.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE


Generated by Claude Code

Two claims in this section were wrong, and both are live on the site.

The row-projection procedure is credited to Wold. Wold's 1978 paper,
which introduced cross-validation for component models, held out a
scattered pattern of individual elements and estimated them as missing
values; it did not hold out rows. The section then had Bro and
co-authors recommending element deletion "instead", which stages a
correction of Wold that never happened, since Wold was already deleting
elements. The procedure is now named for what it is, the review's point
is stated as the independence requirement it is, and the element-wise
remedy is credited to Wold, with Eastment and Krzanowski (1982) named
for the two-model route to the same property.

The bar plot is labelled Simca-P's output "from the row-wise scheme".
Simca-P cross-validates a PCA element-wise, by the Eastment and
Krzanowski route; its row-wise scheme is for PLS. The recorded curve
agrees: it falls at four component counts, which the row-projection
scheme cannot do, because nested loadings make its reconstruction error
monotone. So the comparison that followed was element-wise against
element-wise, not the contrast it claimed. It now says so, and its
closing sentence draws the conclusion that actually follows: two schemes
forming their predictions differently read these data the same way.

No number moved. The four code blocks of the chapter pass, and the
element-wise curve is unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
@kgdunn
kgdunn merged commit 3deab5f into main Sep 11, 2026
4 checks passed
@kgdunn
kgdunn deleted the claude/pca-cross-validation-attribution branch September 11, 2026 05:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants