Skip to content

Three batch case studies for the Applications chapter (DuPont batch PCA, SBR batch PLS, FMC multiblock batch PLS) - #273

Open
kgdunn wants to merge 109 commits into
mainfrom
claude/issues-154-156-process-3cu4iq
Open

Three batch case studies for the Applications chapter (DuPont batch PCA, SBR batch PLS, FMC multiblock batch PLS)#273
kgdunn wants to merge 109 commits into
mainfrom
claude/issues-154-156-process-3cu4iq

Conversation

@kgdunn

@kgdunn kgdunn commented Sep 3, 2026

Copy link
Copy Markdown
Owner

Three worked case studies on batch data for the Applications chapter, placed after the batch process monitoring page. They are the book-side counterpart of the three case studies added to process-improve in kgdunn/process-improve#537 (issues #154, #155 and #156 there, merged), and every figure is generated by a matplotlib script in kgdunn/figures#84, #91 and #92.

The three pages are read as three separate HTML pages and as consecutive pages of the PDF, so each one says where it sits in the set: every introduction names what the earlier pages contributed, the SBR page points forward to the FMC blocks where the attribute it predicts last is set, the DuPont and FMC pages point at the SBR on-line section for the sample-by-sample check, and each closing paragraph hands over to the next page.

This PR is paired with the 1.83.2 release, which is now on PyPI. The three batch pages import loaders and on-line methods that no earlier release carried, and the FMC page's cell-wise cross-validated value needs the 1.83.1 fix that lets the element-wise scheme cross-validate a block which already has missing cells.

What changed

  • product-development-product-improvement/batch-case-study-dupont.rst (label APPS_batch_case_dupont): batch PCA on the 55 batches of the DuPont polymerization reactor. Model A on all batches, the SPE outlier (batch 49) diagnosed with SPE contributions per tag and per sample (the event Nomikos, 1996, attributes to the heating system), the score outliers (batches 50 to 55), the second group found after rebuilding (batches 37, 39, 43 to 48) read as a cluster against the model centre (a contribution runs between two points, and either can be an actual observation or a synthetic one: the model centre, or a group average the reader supplies; the model B score plot draws that direction as an arrow from a square at the group's average point to the centre, and the contribution figure pairs the per-tag bars, each member a dot, with the raw trajectories of the three largest-contributing tags and Flow-2 over the first 30 samples), and the final model C on the remaining 40 batches, verified by projecting the 15 left-out batches onto it: all 15 lie above its SPE limit, seven above its T2 limit as well, while the four poor-quality batches kept in the training set (38, 40, 41, 42) stay inside both limits, the observability lesson. The page says that the original study also left those four out and built its monitoring model on 36 batches with three components, and it compares the two unfolding layouts as Westerhuis, Kourti and MacGregor (1999) measured them on this same data set. The quality list from the laboratory records plays no part in building any model.
  • One meaning per colour on that page, after review: the second group is purple triangles in every figure it appears in (it had been orange in two figures and purple in a third, where orange already meant batch 49), and the four poor-quality batches are magenta diamonds in the model C figure, whose two panels sit side by side. Shape carries the identity alongside colour, so each set is recognised across figures and in print. The choice was checked with a colour-vision and contrast validator against the four colours each shares its figure with. The code blocks now draw those highlights themselves, rather than leaving them to the committed figure while the caption described them.
  • product-development-product-improvement/batch-case-study-sbr.rst (label APPS_batch_case_sbr): batch PLS from six trajectories of the simulated SBR reactor to five latex quality attributes. The faults are stated as Nomikos and MacGregor (1994) simulated them: 30% more impurity from the start of batch 37, 50% more from midway through batch 34. The two batches land in different places of the score plot, whose marker area carries each batch's SPE, so the plot answers both questions asked of a batch; contributions, time-varying weights, per-cell R2, a sustained-departure diagnostic on the raw trajectories (a signed distance from the other batches in their standard deviations, and a robust version from the median and 1.4826 MAD smoothed with an EWMA, drawn together as dashed and solid lines), and the fitted quality of the faulty batches against their measured values, with RMSEE and the leave-one-batch-out RMSEP printed on the panels and a band of two RMSEP around the parity line. Two further sections answer the page's third question, whether the model could have said so while the batches were running: "Predicting quality before the batch ends" (label APPS_batch_case_sbr_online_prediction) traces the final-quality prediction from a batch-so-far with the library's missing-data score estimate, as the leave-one-batch-out RMSEP per attribute (with the sample at which each attribute's error falls below its standard deviation, the error averaged over the five attributes, and the reason the particle size is predicted last, from the 1995 paper) and as batch 4's evolving prediction against its final prediction and measured value; "Would the model have caught it on-line?" (label APPS_batch_case_sbr_online) fits a reference model on the 51 normal batches, explains why the T2 limit is the same at every sample while the spread of the score estimates is not (with a figure of that spread falling over the batch, and the per-sample covariance credited as the 1995 paper states it and the 2004 paper computes it), runs batches 37 and 34 through per-sample T2 and SPE limits (compared with the samples at which the original papers flagged the same two batches), reads the residual shares at the alarm sample, lays out as a chain of bullets the false-alarm rate of the three-sample rule on the reference batches (the autocorrelated single-sample SPE flags one normal batch in four, a smoother limit through spe_window does not change that, the cumulative SPE does at the price of a later detection, and the 99.9% residual limit of Nomikos, 1996, does without one, with the wider point made by Ramaker et al., 2006), and shows the trajectory forecast of Wold et al. (2009) for both faults.
  • product-development-product-improvement/batch-case-study-fmc.rst (label APPS_batch_case_fmc): the ladder of two-component models on the FMC batch dryer, introduced as a bulleted ladder (PCA on quality, PLS from each initial-condition block, multiblock PLS on both, batch PCA and batch PLS on the trajectories, batch multiblock PLS on all three blocks), with the clock time at each aligned sample as the eleventh trajectory of every batch model, as in the original study, and the operating-condition block described for what it is: the cake weight and eight landmarks read off the trajectories at the alignment. The data section is half its earlier length, in short paragraphs. Every score plot on the page is coded by the plant's classification, a colour and a marker shape per class (blue circles good, purple triangles abnormal, gold squares high solvent), through a group_scatter helper in Plotly, and the bar charts shade alternate positions behind the bars. The multiblock PLS on the two initial-condition blocks is drawn as a 2 x 3 figure: the super scores in the first column, and for each block its scores above its weights, so that batch 20, inside the chemistry cloud and far outside the operating-condition one, is read against the weights that place it there. The loading panels of the batch PCA carry the cumulative R2 of every cell after two components on a second axis, and the two phase ends are marked on the raw overlays, the loading panels, the per-sample panel of batch 20's contribution figure and the raw panels of the four-batch figure. Batch 20's SPE contributions, computed by the library from its observed cells, replace the batch-41 figure and the paragraph that said contribution plots need complete trajectories. A table near the end gives the R2 of each block, per component, for every model of the ladder, and the on-line paragraph says that every method on the page handles a batch observed so far and points at the SBR section that shows it. A section (label APPS_batch_case_fmc_block_scores) reads the block scores of the final model: four batches classed good (2, 3, 6, 7) sit among the abnormal batches in the trajectory block and among the good ones in both initial-condition blocks, and what separates them from their nearest abnormal neighbours is the length of the later phases (a longer cool-down, a shorter and steeper ramp), not a set point that was moved; its closing bullets say what the record can and cannot tell about why, with the caution of Nomikos and MacGregor (1995) that the model describes correlation, not cause and effect.
  • Later rounds on the FMC page, from review: a three-panel figure of what the alignment did (the dryer temperature against clock time, the same batches against aligned sample, and the clock time at each aligned sample, with the shortest, a middling and the longest batch drawn in colour, and the echoed numbers behind the sentence: 59 batches, 93 to 200 time units, every one aligned to 325 samples); a figure for the operating-condition PLS, so the passage on batch 20's recipe timings has something to look at rather than three numbers quoted in the prose; batches 61 and 14 named in every later score plot, so the two quality groups can be followed from figure to figure; the three phases named on the per-sample panel of batch 20's contribution figure, each at whichever end of its phase the bars leave headroom; and "disposition" replaced by "classification" on this page and the one sentence that used it on the DuPont page.
  • A later round on the quality ladder, from review: the multiblock PLS on the two initial-condition blocks now has its own row in the ladder tables, so the reader meets its 29.2 and 36.4% in a table rather than only in prose. The tables accumulate one rung per section: three rows after the two single-block PLS models, four after the multiblock section, five once the trajectory block is added. Neither the multiblock nor the batch PLS row carries a cross-validated value, and the caption now gives both reasons. Three prose fixes went with it: the dryer temperature tag is named in monospace where its loading changes sign, the missing-cell paragraph drops a walk-through of batch 20's scores and keeps the point, and the explanation of the empty Q2 cells is corrected. It had said a PCA has no second block to predict a held-out batch from, which contradicts the cross-validation section of the latent-variable chapter; a PCA can be cross-validated by rows, and the objection is that the held-out row supplies the scores that rebuild it, so the value reaches 100% once the components equal the variables.
  • The first row of the quality ladder now carries a cross-validated number. It had none, because the element-wise scheme could not cross-validate a block that already has missing cells, and the dryer's quality block has 19 of them; that is fixed in 1.83.1. The page holds out a seventh of the measured cells at a time and averages over fifty groupings, giving 33.4% after one component and 25.1% after two. One grouping is not steady enough to print: the second-component value swings 13.9 to 33.4 across ten seeds, where fifty leave a seed-to-seed spread of about a point. The value sits in the shared Q2 column, beside a new held out column that says what a held-out group is in each row: single cells for that PCA, whole batches for the two PLS models. That column is what makes one shared column legitimate, because the PCA row answers a different question from the two PLS rows: how well one attribute is predicted from the other seven of the same batch, not how well all eight are predicted for an unseen batch. The paragraph that used to explain the empty cells shrank accordingly, and the two PLS rows are still comparable with each other and with every later rung.
  • A new FMC section, "How many components does the quality block support?", tables the fit and the cell-wise cross-validated value over four components, since the chapter fitted two and printed the value in passing without ever saying what it was for. The two part company at once: R2 climbs 50.0, 70.3, 78.8, 86.6% while Q2 goes 33.4, 25.1, -44.6, -84.2%, so a three-component model predicts a held-out cell worse than the block's own average. The two-component model is kept and the section says why: the second component is what separates the classes in the score plot, and every model in the ladder is read at two components, which is what makes the rungs comparable. The cross-validation moved up to where the model is fitted, so the chapter computes it once. No number already in the tables changed; the two that were there are the first two of the four now shown.
  • The prose was shortened and rebuilt to the style guide, after review. The pages read as walls of text: paragraphs of 100 to 190 words, and one 536-word bullet block on the SBR page. Two passes fixed it. The first split, cut or bulleted every paragraph over 90 words. The second applied "keep the claim, its reason and its consequence, and stop" sentence by sentence, and removed the colons and semicolons splicing two ideas into one sentence, which CLAUDE.md forbids and which the first pass had introduced. Lists that were running prose became bullets: the dryer's three phases, the two readings of a PLS weight, the two ways of summing a contribution, the two statistics that catch batches 34 and 37, the three responses to the SPE false-alarm rate. Prose across the three pages fell from 10,228 words to 8,424; the DuPont page, the one reviewed, is down 29% with a median paragraph of 42 words and none over 70. No detail was dropped: every number, citation, cross-reference and defined symbol is kept, and no code block was touched.
  • Labels on the score plots are placed by measurement, not by hand. In the chapter's Plotly code a label_corner helper puts each batch's name in the emptiest quarter around its marker, in units of each axis's own range so the choice matches what the reader sees, and counts each name it places as a neighbour for the next, so two batches side by side take different corners. Checked for every labelled batch on the four score plots of the page: each lands in a corner with no neighbour within a tenth of the plot. The committed PNGs get the equivalent treatment in DuPont figures: the eight-batch cluster keeps one colour and one shape figures#92, where a checker reports any label that runs off the figure, intrudes into a marker, lands on another label or sits under a legend.
  • Every score plot on the three pages carries the percent of variance each component explains on its axes (R2 of X per component, labelled R2X on the PLS-type models and their block scores; R2Y on the multiblock super scores), and the prose no longer repeats numbers that can be read off a plot. Hotelling's statistic is set as maths throughout the prose, including in the three places where it sat inside :ref: link text and a maths role cannot nest.
  • The references on the three pages link to literature.learnche.org where an entry exists (thirteen papers and chapters), and four names on the pages link to their source in the process-improve repository (batch_dtw, BatchPCA, BatchPLS, MBPLS).
  • product-development-product-improvement/batch-process-monitoring.rst: a short paragraph pointing at the three case studies, three further readings added to the list (Westerhuis, Kourti and MacGregor, 1999; Garcia-Munoz, Kourti and MacGregor, 2004; Arteaga and Ferrer, 2002), a subsection on the scores of a batch observed so far, which names the three missing-data estimators the library offers and why a projection is preferred to filling the future with an average trajectory, and the reading list renamed to "References" with a label (APPS_batch_readings) so the case studies can send the reader to it.
  • .github/workflows/check-code.yml: the pyoptex chapter job installs process-improve[expt,batch] rather than [expt]. That job runs the whole product-development chapter so the mixed-level page sees the namespace its chapter builds, and the chapter now also holds two pages that read an openmv.net workbook, whose Excel reader lives in the [batch] extra. Neither extra pins plotly, so the pair still resolves against pyoptex's plotly 5.
  • Makefile: check-code-chapter and check-code-file pass --strict-output, as check-code already did. Without it a per-chapter run reported a number that had stopped reproducing but still passed, so a quick local check proved less than it appeared to.
  • CLAUDE.md: the writing and figure rules distilled from the review of these pages (length and focus, paragraph construction, figures and captions, process), added under the author's voice section so the same corrections are not needed again.
  • product-development-product-improvement/index.rst: the three pages in the toctree after batch-process-monitoring.
  • latent-variable-modelling/principal-component-analysis/latent-variable-contribution-plots.rst: a label (LVM_contribution_plots) so the case studies can cross-reference the contribution-plot section.
  • CITATION.cff: version 2026.09.10, date-released 2026-09-10.

The LDPE cross-validation page is no longer part of this PR. This branch had refreshed its Q2 numbers after process-improve #546 changed how the element-fold scheme judges convergence; #287 has since landed the better version of the same fix on main, passing the raw block so that no warning is raised at all, and this branch now carries main's file unchanged.

Every figure has a Plotly code block before it, following the chapter rework playbook; each page reads top to bottom as one script. The data come from openmv.net (polymerization.csv, batch-dryer.xlsx, sbr-batch-reactor.xlsx, all three live) through the load_dupont, load_fmc and load_sbr loaders of process-improve 1.83.2, whose BatchPLS.predict_online, predict_online_trace, online_rmse and BatchMonitor the two on-line sections use. The FMC page also depends on two changes in #537: contributions for rows with missing cells (batch 20's figure) and the fix to PCA.r2_per_variable_ on the NIPALS path (the R2 curves of the loading panels; the collector-tank cells read 0.07 before the fix and 0.78 after).

Verification

  • tools/check_code_blocks.py (the gate from Execute every Python case in the book in CI, and fix the drift against process-improve 1.78 #274) on each of the three pages with --strict-output, which compares the results echoed in the code, against the released library, every data set read from openmv.net: DuPont 9 blocks, SBR 15 blocks, FMC 14 blocks, all passed. Run as a chapter up to the FMC page: 71 blocks, all passed. Every number the FMC prose quotes is echoed after the print that produces it.
  • The whole product-development chapter under the pyoptex job's environment (process-improve[expt,batch] plus pyoptex, which resolves to plotly 5.24.1): 101 blocks, all passed, 0 skipped, 11 minutes, well inside the job's 45-minute timeout. That rehearsal is what the workflow change above is based on, and the job itself is green on the current head against the release.
  • Every number quoted in the prose reproduces: DuPont R2 55.9% cumulative for model A, batch 49 SPE 39.3 against 29.1 with 80% of its SPE in samples 55 to 65, the group's mean t2 and t3 of 15.0 and 14.8 against -3.0 with 66% and 90% of the contribution in samples 0 to 25, the 15 left-out batches all above model C's SPE limit and seven above its T2 limit, batches 38, 40, 41, 42 inside both; SBR R2Y 65.3 / 6.9%, the per-fifth contribution shares, the sustained-departure samples (37: conversion 9, latex density 13; 34: temperatures and energy released 103 to 105, conversion 123, latex density 129; the robust version within a few samples of these), the SPE ranks of the two faulty batches (37 the smallest of 53, 34 the twelfth), and the RMSEE and leave-one-batch-out RMSEP of the composition (0.00106 and 0.00122) and the particle size (1.87 and 2.42); SBR on-line: leave-one-batch-out RMSEP / sd averaged over the five attributes 2.99, 1.39, 1.06, 0.62, 0.58 after 10, 25, 50, 150, 200 samples, first sample with the error below the sd 48 (composition), 129 (particle size), 50 (branching, cross-linking), 21 (polydispersity), batch 4 predicted at 1251 after 10 samples against 1257.1 final and 1256.9 measured, T2 limit 10.5, batch 37 T2 alarm after 23 samples, batch 34 SPE alarm after 105 samples (five after the impurity enters), reference false-alarm rates 0.2% and 1.2% with three-sample SPE alarms in 13 of 51 reference batches (one for T2), pooled limits 1.34% and 15 of 51 with batch 34 still at 105, cumulative SPE 3 of 51 with batch 34 at 112, the 99.9% limit 0 of 51 with batch 34 at 106, residual shares 40 / 30 / 16 / 11% at the alarm sample; FMC quality block R2 50.0, 70.3, 78.8, 86.6% and cell-wise Q2 33.4, 25.1, -44.6, -84.2% over four components, the R2 ladder 70.3% (PCA on Y), 22.2 / 26.2% (PLS per Z block), 36.4% (MBPLS on Z), 37.6% (batch PCA), 41.0% (batch PLS), 47.0% (batch MBPLS) with super VIP 0.86 / 1.07 / 1.06, the per-block, per-component R2 of every model in the comparison table, the phase ends at samples 175 and 249, the 59 batches running 93 to 200 time units before alignment and 325 samples after it, the R2 per cell averaged per tag (collector tank level and clock time 0.71, agitator speed 0.08), batch 20's SPE contributions (205 missing cells between samples 34 and 109; dryer pressure 49%; 58% of the total in the first phase, where its dryer pressure averaged 85 units against 37 for the average batch), batch 13's leading contributions from the clock time and the collector level, batches 2, 3, 6, 7 placed with the abnormal batches by the trajectory block and with the good ones by both initial-condition blocks, their nearest abnormal neighbours 42, 43, 44, 47, 50, the four's ramp 24 against 32 clock samples and cool-down 50 against 38, peak set points 86.9 against 87.2.
  • make text: zero warnings, after the prose rewrite, after the merge of main, and after the component-count section.
  • make html: builds, with no goatcounter string anywhere in the output.
  • No em-dashes in the added prose; no prose line longer than 100 characters in the three new pages.

Related pull requests

🤖 Generated with Claude Code

https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE

Batch data has had a placeholder page in the Applications chapter with a
reading list and no worked example. These three pages supply the examples,
each on a dataset hosted on openmv.net and reproduced by the code shown on
the page, so a reader can regenerate every figure with process-improve 1.79:

- the DuPont polymerization reactor: batch PCA on 55 batches, SPE and score
  contributions that name the variables and the time of an event, the
  exclude-and-rebuild loop, and the four poor-quality batches the ten
  trajectories cannot distinguish from good ones (observability);
- the simulated SBR reactor: batch PLS to five latex quality attributes, the
  same injected fault from the start of one batch and from the middle of
  another landing in different places of the score plot, a sustained-departure
  diagnostic on the raw trajectories, and the fitted quality of both batches;
- the FMC batch dryer: the ladder from a PCA on quality through PLS on each
  initial-condition block, multiblock PLS, batch PCA and batch PLS on the
  trajectories, to a batch multiblock PLS joining all three blocks.

The pages follow the chapter rework playbook: a Plotly code block before each
figure, the committed PNGs generated by matplotlib scripts in the figures
repository (batch/batch-case-*-figures.py), and every quoted number checked
by running the extracted code against the real data. The contribution-plot
section of the PCA chapter gains a label so the case studies can link to it,
the batch process monitoring page points at the three studies, and the
citation metadata is bumped to today's date.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
The SPE figures had the batch number on the horizontal axis, which is the
order the batches appear in the file rather than anything about the batch,
and a stem from every dot down to zero. Each case study now draws Hotelling's
T2 against the SPE: how far a batch sits along the model's components against
how far it sits away from them, with both 95% limits as reference lines. The
quadrants say which of the two questions a batch answers unusually, and the
pages explain the trade-off against the run-order layout rather than
presenting one as correct.

Two errors in the prose came out of the change:

- The DuPont score plot caption said batches 50 to 55 lie outside the 95%
  confidence ellipse. Batch 51 is inside it, with a T2 of 3.5 against a limit
  of 6.6; it is above the SPE limit instead. Colour in both DuPont figures
  now marks which limit a batch exceeds.
- The SBR page said two batches are above the SPE limit. There are three: 8
  at 39.3, 16 at 35.3 and 15 on the limit at 34.6.

Each page defines the plotting helper itself, as it already does for the raw
trajectory overlay, so every page still runs top to bottom on its own. Every
code block was extracted and run against the real data, and make text builds
with no warnings.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
Dark blue marks every unremarkable batch in the score and influence plots, so
using it to highlight batch 54 in the raw trajectory overlay gave one colour
two meanings. Batch 54 is now aqua in the figure, the code block and the
caption, matching the colour it carries in the two scatter plots.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
…w fixes

The second cluster was carried by batch 39 as a representative member. Batch
39 has its own quirks: on t2 its TempH-1 and Press-2 contributions run against
the rest of the group, and across the eight members TempH-1 ranges from -3.8
to +2.4 and averages to almost nothing. Reading one member therefore points at
a tag the group does not share.

The section now takes the group as the object. The columns are centred, so the
model centre is the origin, and contributions are linear in the row: the mean
of the members' contribution vectors is the contribution of the group mean and
adds up to the group's mean score, 15.0 on t2 and 14.8 on t3 against -3.0 for
the other 40 batches. Two figures replace the batch 39 one: the per-tag and
per-sample contribution with every member drawn as a dot, and the raw
trajectories of the three tags it names. The raw panels stop at sample 30
because over the whole batch the gap between the group and the rest is under
2% of the panel height for two of those three tags, while samples 0 to 25
carry 66% of the t2 and 90% of the t3 contribution.

Applied across all three case studies in the same pass:

- Fewer numbers in the paragraphs after each influence plot, and the DuPont
  aside about drawing a statistic against the batch number cut to its
  conclusion.
- i, k and j defined where the contribution vector is introduced, with K tags
  and J time samples as in the course notes, restated at the point of use in
  the other two cases. The score formula now carries the same double index
  instead of reusing k as a running index over all the unfolded cells.
- Round brackets no longer do double duty in the contribution captions: the
  panel words move to the front, as they already are in the other captions,
  leaving the brackets to the (tag, time) pair.
- The raw-trajectory overlays carry labels, and the six places that point back
  at one now say it is an earlier figure and link to it.

Every code block was extracted and run against the real data; make text and
make html both build with no warnings.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
Kevin's own chapter in Comprehensive Chemometrics (2.10, 2009) covers two of
the three datasets used here, so the pages should agree with it and say where
they came from.

DuPont page:
- Name the two unfolding layouts. The page only ever said "unfolds the
  batches batchwise" without saying what the alternative was, which leaves a
  reader who goes to the literature next with no way to place it.
- Record the independent confirmation of batch 49. The chapter monitors batch
  49 on-line against a three-component model on batches 1 to 36 and names the
  heating- and cooling-medium temperatures and Press-2 and Press-3 as too low
  from sample 57 to 65, with Press-2 the largest contributor. That is four of
  the five tags and the same window this page reaches from a two-component
  model on all 55 batches, after they finished. Checked against the raw data:
  over samples 55 to 65 those four tags sit 5 to 7 batch-to-batch standard
  deviations below the other batches, further than any other tag, and Press-2
  is furthest.
- Say that a contribution is a weighted difference between a point and a
  reference, either of which may be a group average, so the group-to-centre
  figure is a standard use rather than an invention of this page.
- Add the chapter to the readings.

FMC page:
- Give the three phases their landmark boundaries (agitator to high speed,
  then the dryer temperature maximum) instead of describing only the activity
  in each.
- Say how the alignment was done: a maturity variable in the first two phases
  and linear stretching in the third, and why one was needed at all. This is
  also what ClockTime is a record of.
- Name the feature-block model in "Where to go next" as the landmark feature
  approach, with what it costs and where it suits, and note that all three
  kinds of model separate the on-specification batches on this dryer.
- Say that the block ordering (operation ahead of chemistry) is what the
  original study set out to establish.

SBR page: "unfolds each batch batchwise", for one vocabulary across the three.

Citation bumped to today. make text and make html both clean, zero
goatcounter hits.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
The Wold-consistency pass put a paragraph two sentences later that names the
same tags and says "from sample 57 until sample 65", so the window was stated
three times in eight lines (57 to 63, then 55 to 65, then 57 to 65) and the tag
list twice. State each once: the tag ranking without its five percentages, now
that the figure's middle panel carries the number on every bar, and the window
only as the 55 to 65 span that carries 80% of the residual.

Drops the print of the seven largest shares with it, since no prose quotes it.

make text and make html both clean, zero goatcounter hits.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
The reading of w2 said a batch with a high t2 "ran warmer, released less energy
and converted less". Warmer and less energy released do not sit together: a
slower reaction gives off less heat, so on its own it does not run warmer.

The reactor temperature is in fact the tag this component involves least, 8.0%
of its squared weight against 17% to 21% for each of the other five, with a mean
weight over the second half of -0.003 against +0.035 for the cooling-water and
jacket temperatures. What moves is the cooling and jacket side, not the reactor
measurement. The page now says that, and says the component reports the two
groups moving together without settling which drives the other.

The causal reading stays where the evidence for it is, after the order of the
departures, and now carries the caveat it needs: the dataset names its tags and
no more, so nothing says whether a temperature is measured at a service inlet
or a return, and the reading fits the order without being established by it.

make text and make html both clean, zero goatcounter hits.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
Two changes to the SBR departure block.

The np.convolve line was doing real work with no explanation: convolving a 0/1
column with `run` ones is a moving sum over every window of `run` consecutive
samples, "valid" keeps only the windows that fit inside the batch so entry i
starts at sample i, a window summing to `run` is an unbroken stretch, and
argmax finds the first one. The argmax-returns-0 case is why `any()` is tested
first. All of that is now in the comment.

The figure plotted the absolute value of z, which threw away the direction of
each departure. It now plots z itself, with the band between plus and minus two
standard deviations shaded, so the reader can see that batch 34's service
temperatures rise while its energy released and extent of reaction fall. The
onset rule still tests the magnitude, so the reported samples are unchanged: 9
and 13 for batch 37, and 103, 104, 105, 123 and 129 for batch 34, all re-run
against the data.

make text and make html both clean, zero goatcounter hits.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
Two sections on the SBR page. The first estimates the scores of a batch
that has run for k samples from the observed cells alone and traces the
final-quality prediction as the batch progresses: RMSEE and
leave-one-batch-out RMSEP per attribute, and batch 4's particle size and
composition against the final prediction and the measured value. The
second fits a reference model on the 51 normal batches and runs batches
37 and 34 through per-sample T2 and SPE limits: batch 37 trips T2 after
23 samples, batch 34 the SPE of the newest sample after 105, five samples
after the impurity enters, and the trajectory forecast of Wold et al.
(2009) shows why one fault can be forecast and the other only flagged.
The FMC page points at this section for the on-line step it defers.

The DuPont page's TempH-1 range now reports the per-member contributions
on each component (-3.8 to +6.3 on t2, -3.1 to +2.1 on t3) rather than a
range that reproduced on neither.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
The SBR departure chart now carries both signed distances of each faulty
batch from the other batches per tag: the robust one (median and 1.4826
MAD of the others, EWMA-smoothed with lambda 0.3) solid and the
mean-and-standard-deviation one dashed, in one code block and one figure,
and the four paragraphs that walked through every onset are replaced by
one. The on-line section no longer attributes the reference batches'
short SPE alarm runs to a rough limit: pooling neighbouring samples into
each limit (spe_window) leaves the alarms where they are, and the limit
changes by under 2% from sample to sample, so the runs come from the
autocorrelation of each batch's SPE.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
The DuPont page states that the quality list plays no part in building
the models, reads the eight-batch group's contribution in half the
length, and checks the final model by projecting the 15 left-out batches
onto it (all above the SPE limit, seven above the T2 limit as well)
instead of tabulating the four poor-quality batches; a forward reference
points at the SBR page for the sample-by-sample version. Every score
plot on the three pages carries the percent of variance explained on its
axes, and the prose no longer repeats numbers readable off the plots.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
…timators

Readers have the book and nothing else, so the three pages no longer
refer to the course notes the case studies grew out of. The SBR page's
aside on the projection to the model plane and the two simpler fills is
gone: the trimmed score regression it uses is introduced, and that is
enough here.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
Keeps this branch's citation date, the later of the two.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
The section states the three steps as bullets, computes only the
leave-one-batch-out RMSEP, and drops the numbers that can be read off
the two figures. The batch-4 band is two prediction errors at each sample.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
An arrow from the second group's average point to the model centre,
labelled "contribution direction", ties the score plot to the group
contribution that follows. The group is defined before the plot, and the
page imports numpy for the label angle.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
The figure keeps the per-tag bars with the member dots and shows the raw
trajectories of TempC-1, Press-3, Press-2 and Flow-2 over the first 30
samples in place of the per-sample panel; the separate raw figure is gone.
The code prints the share of the group contribution in the first 25
samples, which the prose used to read off the dropped panel. The score
plot's arrow starts at a square marker with a smaller label.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
The mid-batch prediction section now states the first sample at which
each attribute's leave-one-batch-out RMSEP falls below its standard
deviation, and the error averaged over the five attributes after 10, 25,
50, 150 and 200 samples. Both are printed and echoed in the code block,
so the checker compares them.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
…r wide

The score-plot caption points at batch 4, the batch used later for the
mid-batch prediction, and the batch-4 figure's band is one prediction
error rather than two, in the code, the caption and the prose.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
… score spread

The limit depends only on the number of components and of reference
batches, so it is the same at every sample; what changes is the spread of
the score estimates across the reference batches, which the per-sample T2
divides by. A new figure shows that spread falling to the spread of the
final scores over the batch. The case-specific comparison for one batch
is dropped.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
…in CLAUDE.md

Both pages are cut back to the author's voice: passages that restated
what a figure, a caption or a printed result already shows are removed,
numbers readable off a plot are dropped from the prose, method asides the
point does not need are shortened, and each paragraph carries one idea.
The rules distilled from this review (length and focus, paragraph
construction, figures and captions, process) are added to CLAUDE.md so
the same corrections are not needed again.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
…e models

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
The reference batches define normal operation, so their alarms are false
alarms; the 99% limit gives the expected crossings, but the SPE crossings
come in runs because the statistic is autocorrelated, so the three-sample
rule flags one normal batch in four; a smoother limit does not change that;
the cumulative SPE does, at the price of a later detection; the rule must
be tuned on the reference batches before an alarm can be trusted. The
cumulative-SPE numbers are now printed and echoed for the checker.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
A forward reference named as "below"; the T2 limit no longer described as
computed at every sample; the SPE introduced as the other question a model
answers; the residual-share reading in its own paragraph.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
…t in purple

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
Six passages described how to call the library rather than what the
analysis does: which keyword to pass, what a loader returns, which
argument switches a behaviour. The code block sits directly beside each
one and shows it.

Function and class names stay, since the reader needs to know what to
reach for. What goes is the usage detail: scale=False against MCUVScaler,
by_tag=True, spe_window=2, the return signatures of load_dupont and
load_sbr, and the aside on unfolding with dict_to_wide.

Facts about the data that happened to sit inside those sentences are kept:
100 samples by 10 tags, the six reactor trajectories, the five-sample
smoothing window and its citation.

make text succeeds with zero warnings.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
The paragraph gave two of the six numbers in prose and left the reader to
compare a PCA against two PLS models without saying they answer different
questions. A reader who reads 70.3% beside 22.2% concludes the PCA is the
better model.

The table gives the quality explained by each of the first three rungs,
cumulative after one and after two components, with a column naming what
each model explains the quality from. The paragraph after it says plainly
that the first row is a description of the quality block by itself and the
other two are predictions from measurements made before the batch ran.

All six values are printed in the code block and echoed in comments, so the
checker compares every cell of the table.

make check-code-file on this page: 71 blocks, 71 passed, 0 failed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
claude and others added 9 commits September 9, 2026 21:32
The paragraph described the rule in one dense sentence and then reported
its outcome in another, so a reader met the counts before understanding
what produced them.

It now separates the two: a batch sitting among a group is a claim about a
picture, so the rule takes the average score point of each group and places
every batch with the nearer of the two. Then the outcome, with what it
means spelled out: four batches with ordinary chemistry, ordinary operating
conditions and trajectories that look abnormal, and batch 5 set aside
because its operating conditions are abnormal too.

The caption says the figure now draws each batch joined to the average
point of the group it is placed with.

make text succeeds with zero warnings.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
The two tables that compare the single-block models now carry a
cross-validated column beside the fit, so a reader who asks why the
quality PCA explains 70% while the PLS models explain a quarter sees
the two questions side by side. Q2Y is printed for the two
initial-condition blocks, whose held-out estimate on 46 batches holds
its sign across fold splits; the PCA row has nothing to predict a
held-out batch from, and the trajectory block's held-out estimate on
3575 columns moves too much between splits to quote as one number, so
those cells stay empty and the caption says why.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
A comment that echoes a print is compared with what the block printed,
but only if it does not read as prose, and the test counted every word
including the quoted keys of a dict. A printed dict of tags therefore
looked like a sentence and was dropped, and so was any line of results
carrying several words. The DuPont page had no echo the checker kept at
all, and all five on the SBR page were discarded, so numbers those pages
quote in print rested on nothing the gate could re-check.

Words inside quotes no longer count, and three numbers or more mark a
line as a result rather than an explanation. Sixteen comments across the
book become checked, every one an echo of real output.

The DuPont page gains an echo after each of its prints, and the FMC page
after the four that had none, so the class counts, the quality PCA, the
multiblock model on the two initial-condition blocks and the chemistry
row of the comparison table are all compared on every run.

Batch 20's dryer temperature over the solvent-collection phase averaged
23.7 against the other batches, not 23.6, which no echo would have
caught; it is printed now. The two sentences after that figure repeated
its caption word for word and are gone. The first table's caption said
the batches are held out seven at a time, where seven is the number of
folds, each of six or seven batches, averaged over ten splits.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
…apers

A close read of the three pages against the data sets, the library and
the papers they cite turned up claims that do not hold. Each is fixed
where it stands, and the numbers behind them are printed and echoed so
the checker compares them on every run.

Wrong about the data:

- Batch 13 of the dryer did not collect the least solvent of any batch
  or reach that level in the fewest clock samples: it is eleventh from
  the bottom on the collector level and fourth on the clock time at the
  end of the first phase, and its cooling phase is the fifth longest of
  the 46, so it did not cool fastest either. The two ranks are printed.
- Five batches classed good sit among the abnormal in the trajectory
  block, not four; the four are those that also sit with the good
  batches in both initial-condition blocks, which the next paragraph
  already set out.
- Batch 13 is not furthest along the first component: batch 10 is.
- ``Y11`` is not mirrored between batches 61 and 14. It is batch 61's
  largest contribution and batch 14's smallest.
- Batch 20 ran hot in the solvent-collection phase, not over the ramp,
  where it averages 57.8 against 56.0 and sits inside the band.
- The cake weight is not one of the four overlaid trajectories, so the
  overlay cannot show a heavy charge.
- Batch 55 does not have the highest t2 in the DuPont model A; batch 53
  does. Batch 51 is not far out along the components, which the page
  says itself two pages earlier: it is above the SPE limit only.
- Batch 49's cooling-medium temperature dips below the others from
  sample 56 and rejoins them by 65, rather than falling away after 60.
- On the second component of the DuPont group, TempC-1 and Press-3
  carry 46% of the displacement, and Press-2 is as large as Press-3.
- The robust distance moves batch 34's onsets by a few samples, but for
  batch 37 it starts the conversion at the first sample and flags two
  service temperatures the standard-deviation distance never flags.
- Batch 34 is caught with a little under half its batch still to run.

Wrong about a method:

- A squared residual over the sum of squares is a share of the squared
  SPE, not of the SPE.
- The reference batches' alarm rate is measured on the batches the
  limits were fitted to, so it is a lower bound for a new batch.
- The band on the parity plot is two leave-one-out errors, wider than
  the scatter of the fitted values drawn inside it.
- Only one of the two initial-condition blocks is measured before the
  batch runs; the other is read off its own trajectories.
- The chemistry block's 52% is cumulative over two components and its
  6.5% is one component of another model, so the two are now stated on
  the same footing.
- The reactor temperature's weights change sign from sample to sample
  rather than barely moving.
- The sample reported for an alarm is the first of the three
  consecutive samples, so an operator sees it two samples later.

Wrong about a paper:

- Wold and co-workers build thirteen landmark features on this dryer,
  each read from one phase, not blocks grouped by what they describe.
- The second DuPont group is the group Nomikos and MacGregor single out
  in the same plane of a three-component model of the same 48 batches,
  and is now credited to them.
- The variable-wise summary of the residual is not due to the 2003
  paper, so the attribution is dropped rather than moved.
- The original dryer study is a troubleshooting analysis of historical
  data and builds no monitoring tools.
- Comprehensive Chemometrics 2.10 is the chapter; the volume is 2.
- The reading list carries two Nomikos and MacGregor papers of 1995, so
  the one behind the SPE result is named.

Also: the contribution arrow in the model B score plot is redrawn from
the centre out to the group, the direction the contributions run
(kgdunn/figures#96), a duplicated bullet is removed from the DuPont
reading list, prose lines are wrapped at 100 characters, and the
citation date moves to today.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
A pass over every figure directive on the three pages, each PNG read
against its caption, its alt text, the paragraphs around it and both the
figure script and the chapter's own Plotly code.

The chapter code did not draw what the caption described:

- the block score plots gained the spokes from each batch to its group's
  average point and the cross at each average, which the caption names;
- the DuPont model A score plot drew every batch in one colour while its
  caption names orange and aqua;
- the alignment figure colours the shortest, a middling and the longest
  batch, as its caption and the committed figure show, where the code
  had picked out only two;
- the influence plot labels batch 47, which the prose names as just
  below the SPE limit.

Claims a reader would check against the picture:

- Temp1 and Temp2 are the dryer temperature at the end of the first
  phase and the peak, not initial temperatures;
- the dryer temperature loading changes sign at about sample 130, not at
  the end of the first phase at 175;
- batch 20 has two gaps in its record, samples 95 to 109 in every tag
  but the collector level and 34 to 44 in five of them, not one;
- the impurity line on the SBR monitoring panel is dotted, where the
  dashed line in that panel is the limit;
- batch 54's contributions are positive almost everywhere rather than
  everywhere, and batch 51 sits to the right of the DuPont cloud.

The robust departure onsets of batch 37 are not explained by the scale
alone: with the median and MAD but no EWMA those two tags never leave
the band, so the smoothing is what carries them out, and the sentence
now says both.

Figures follow in kgdunn/figures#95.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
The Plotly block that draws the DuPont model B score plot put the arrow
head at the origin, so a reader running the chapter's code got the arrow
pointing from the group back to the model centre, against the
contribution it labels. Plotly takes the tail at (ax, ay), so the two
are now the other way round, matching the committed figure.

The standard-deviation distance does not keep batch 37's service
temperatures inside the band: it crosses on 34 of the 200 samples, with
a longest run of 14, short of the 20 the rule needs. The sentence now
says what is true, that it never holds them outside long enough to
count.

Removed: two literal output blocks on the DuPont page that reprinted
what the code above already echoes as comments, and the duplicated
"full list of readings" bullet on the SBR and FMC pages, which repeated
the sentence four lines above it through a second label.

The batch 13 alt text carried the two superlatives the caption and prose
were corrected away from, and the SBR alt text called the impurity line
dashed where the figure draws it dotted.

The comparison table rounds each cell on its own, so a row can add up to
a tenth away from the cumulative value quoted for the same model; the
caption says so rather than leaving a reader to find 23.1 + 14.6 = 37.7
against 37.6.

Paragraphs an earlier edit left ragged are rewrapped. make text is
clean, and make check-code-file on the FMC page passes with the new
spoke-drawing code.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
From the verified findings of the figure, code, consistency, style and
RST passes over the three pages.

The DuPont section is headed with five score outliers and then reads
four of them. The one left out, batch 53, is the highest of all in the
second component, at 38.1 against 35.8 for batch 55, which the prose
did read. Both are now read together: their contributions have the same
three tags in the same order.

Three code blocks did not draw the figure beside them. The middle panel
of the batch multiblock figure pairs each block's R2X with its super
VIP, where the code drew the super weights; batch 13's contribution
panel is the per-tag summary, which needs by_tag; and batch 20's
contribution figure shows the per-cell panel first, which the block
skipped. A fourth block titled its bar chart "after 105 samples" while
plotting the shares left in the variable by the last turn of the loop,
which are the shares after 109.

Corrected claims:

- over the ramp it is batch 20's dryer pressure that stays above the
  other batches, on 89% of its observed samples there; its dryer
  temperature averages 57.8 against 56.0 and sits inside the band;
- the super VIP puts the operating conditions and the trajectories
  level, which the earlier models did not: they agree on the chemistry
  coming last, and the sentence now says that;
- batch 20's blank cells are two gaps that differ by tag;
- batch 37 is fitted further from its measured composition than batch
  34, so "not fitted equally well" is replaced by what the numbers say;
- the 2009 chapter sets out the projection to the model plane and sends
  the reader elsewhere for the other estimators, so trimmed score
  regression is no longer attributed to its equations;
- the reading list carries two Nomikos and MacGregor papers of 1995, so
  the second citation of them names the paper as well.

The R2 figure gains a label, so the sentence that pointed at it by
position now links to it.

make text is clean and the chapter checker passes: 102 blocks, 77
passed, 25 skipped for pyoptex, 0 failed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
The operating-condition block does not hold the length of each phase.
Its four duration columns satisfy Time1 = Time2 + Time3 exactly for
every batch, Time2 tracks the ramp and Time3 the cool-down, and nothing
in the block follows the solvent-collection phase, which itself runs 24
to 112 time units. The page says which two lengths it has.

The four anomalous batches "ramped in 24 clock samples against 32": in
the aligned record the ramp is 74 samples for every batch, and 24 and 32
are clock time, the units the page already uses for the 93 to 200 a
batch runs. "Clock samples" appeared nowhere else and collided with the
aligned sample the axes are labelled in.

The departure analysis reports no onset at all for the reactor
temperature of batch 34: it leaves the band but its longest run outside
is 11 samples against the 20 the rule needs. So it did not find the
reactor-then-service order the residual shares show; the sentence now
says what each of the two reports.

Also: the third citation of Nomikos and MacGregor, 1995, names its
journal, since the reading list carries two of their papers from that
year; and the alt text of the prediction-error figure said the
particle-size curve falls below one after about 120 samples where the
block echoes 129.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE

kgdunn commented Sep 10, 2026

Copy link
Copy Markdown
Owner Author

A final check before review

A last pass over the three pages against the data sets, the library, and the papers they cite, with every quoted number recomputed against the released 1.83.0 on the live data. Nine lenses (statistics, papers, numbers, expert reader, style, cross-page consistency, figures, code, RST), each finding then put to an adversarial verifier that read the source before ruling. Twelve findings were refuted, one of which would have replaced a correct sentence about the SPE with a false one.

The gate was letting numbers through

tools/check_code_blocks.py compares an echo comment with what the block printed, but only when the comment does not "read as prose", and that test counted every word including the quoted keys of a dict. A printed dict of tag names therefore looked like a sentence and was dropped. The DuPont page had no echo the checker kept at all, and all five on the SBR page were discarded, so the numbers those pages quote in print rested on nothing the gate could re-check.

Words inside quotes no longer count, and three numbers or more mark a line as a result. Sixteen comments across the book become checked, every one an echo of real output; no genuine explanation was caught up in it. Echoes were then added after every print on the DuPont page and the four on the FMC page that had none.

That change caught a wrong number immediately, and so did CI on run 90: the FMC page said seven batches classed high in residual solvent were fitted low by 0.58 in a span of 1.41. There are six, fitted low by 0.47 in a span of 1.26, and the page said "six" ten lines earlier.

Corrected against the data

  • Batch 20's other-batch dryer-temperature average is 23.7, not 23.6.
  • Batch 13 did not collect the least solvent, reach that level in the fewest clock samples, or cool fastest: it is 11th from the bottom on the collector level, 4th on the clock time at the end of the first phase, and its cooling phase is the fifth longest of the 46.
  • Five batches classed good sit among the abnormal in the trajectory block, not four.
  • Batch 13 is not furthest along the first component; batch 10 is.
  • Y11 is not mirrored between batches 61 and 14: it is batch 61's largest contribution and batch 14's smallest.
  • Batch 20 ran hot in solvent collection, not over the ramp; over the ramp it is the dryer pressure that stays high, on 89% of its observed samples there.
  • Temp1 and Temp2 are the dryer temperature at the end of the first phase and the peak, not initial temperatures.
  • The operating-condition block holds no length of the first phase: Time1 equals Time2 plus Time3 exactly for every batch, so it records the ramp and the cool-down only.
  • "24 clock samples" is clock time, not aligned samples: in the aligned record the ramp is 74 samples for every batch.
  • The dryer-temperature loading changes sign at about sample 130, not at the phase end at 175.
  • Batch 53, not 55, has the highest t2 in DuPont model A, and was never read although the section names five score outliers. Batch 51 is not far out along the components, which the page says itself two pages earlier.
  • Batch 49's cooling-medium temperature dips from sample 56 and rejoins by 65, rather than falling away after 60.
  • On the second component of the DuPont group, TempC-1 and Press-3 carry 46% of the displacement, and Press-2 is as large as Press-3.
  • The departure analysis records no onset for batch 34's reactor temperature: it leaves the band but its longest run outside is 11 samples against the 20 the rule needs, so it did not find the reactor-then-service order the residual shares show.

Corrected about a method

A squared residual over the sum of squares is a share of the squared SPE. The reference batches' alarm rate is a lower bound, since the limits were fitted to those batches. The parity band is two leave-one-out errors, wider than the scatter of the fitted values inside it. Only one of the two initial-condition blocks is measured before the batch runs. The chemistry block's 52% is cumulative over two components while its 6.5% is one component of another model. The reactor temperature's weights change sign from sample to sample rather than barely moving. The sample reported for an alarm is the first of the three consecutive ones, so an operator sees it two samples later.

Corrected about a paper

Wold and co-workers build thirteen landmark features on this dryer, each read from one phase, not blocks grouped by what they describe; their chapter sets out the projection to the model plane and points elsewhere for the other estimators, so trimmed score regression is no longer attributed to its equations. The second DuPont group is the group Nomikos and MacGregor single out in the same plane of a three-component model of the same 48 batches, and is now credited to them. The variable-wise summary of the residual is not due to the 2003 paper. The original dryer study is a troubleshooting analysis of historical data and builds no monitoring tools. Comprehensive Chemometrics 2.10 is the chapter and 2 the volume. Three citations of "Nomikos and MacGregor (1995)" now name the paper, since the reading list carries two from that year.

Figures made to show what the captions send the reader to look for

The chapter's own Plotly code did not draw the spokes and group-average crosses the block-scores caption describes, drew the DuPont model A plot in one colour where its caption names two, coloured two batches in the alignment figure where the caption and the PNG show three, paired the wrong quantities in the multiblock middle panel, drew the per-cell panel where the caption promised two, left batch 47 unlabelled though the prose names it, and titled a bar chart "after 105 samples" while plotting the shares left in the variable by the last turn of a loop. In the committed figures the group-average crosses sat under the markers that hid them, the contribution arrow pointed against the contribution it labels, and the bar-chart legend sat on the two tallest super-VIP bars (kgdunn/figures#95).

Verification

  • make check-code over the whole book with the stricter echo rule: 7 chapters, all passed.
  • make check-code-chapter CHAPTER=product-development-product-improvement: 102 blocks, 77 passed, 25 skipped for pyoptex, 0 failed.
  • make text: zero warnings. No prose line over 100 characters on the three pages.
  • check_labels.py on all three figure scripts: clean.
  • Regenerating all 36 case-study PNGs reproduces the committed files pixel for pixel apart from the three this pass changed.
  • The branch was conflicting with main on CITATION.cff, which is why no workflow had dispatched since 22:48; the merge is in and the date kept at today's.

Companion changes: kgdunn/figures#95 (three figures) and kgdunn/process-improve#553 (the SBR page's batch 4 is nearest the average quality, not trajectory, and two stale printed results on the FMC page).

🤖 Generated with Claude Code

https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE


Generated by Claude Code

kgdunn and others added 8 commits September 10, 2026 00:57
Read as an author of the original study would read it, the quality PCA
section says the wrong thing twice.

The four attributes named as low in the abnormal group are not all low
there. Scaled, the abnormal batches average Y1 -0.60 and Y6 -0.75 but
Y4 +0.75 and Y10 +1.05, and the good batches mirror that. The first
loading takes both signs to match, Y1 +0.38 and Y6 +0.44 against Y4
-0.43 and Y10 -0.42, and Y4 and Y6 correlate at -0.87. So the component
contrasts two sets of attributes; it is not a general level of quality,
which is what the page concluded from it.

The ladder's step from 26.2% to 41.0% compares two fits from blocks of
nine and 3575 columns on the same 46 batches. Width lifts a fit on its
own, and held out in folds the ordering does not survive, which the
table caption below already concedes for that block. The sentence now
gives both numbers and says what the gap does not establish.

The original study reads the shorter ramp and the longer cool-down as
compensating for the adverse features of those batches' trajectories,
which is the causal reading the bullet above it declines. Saying it
found "the same reading" put two adjacent bullets in contradiction and
attributed the page's own caution to the study.

On the SBR page: both faulty batches are in the fit, so each is extreme
along a component its own deviation helped to define, which is why the
fault lands in the scores and little is left in the residual. And of the
three trajectories left out, only the feed temperature is quiet enough
to call noise, so the sentence now reports what each varies by rather
than naming the source.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
Two sentences beside the quality table, both raised in review.

"A cross-validated number moves with the split into folds, so it is read
for its sign and its size, not its second decimal" warned against a
precision the table never prints, and steered the reader past the one
thing the column is good for. The held-out value of the operating
conditions falls from the first component to the second while its fit
rises, and that reproduces: over eight splits into folds it averages
17.9 then 15.0, against -4.5 and -4.4 for the chemistry. So the second
component adds fit to both blocks and nothing to prediction, which is
what the sentence now says, read off the table.

"The PCA has no held-out batches to predict, since there is nothing to
predict them from" is true of the experiment this column reports and
false as it will be read. A PCA is cross-validated by holding out single
cells rather than whole rows, and on this quality block that gives about
30% after one component. The sentence now names the experiment: the
column holds whole batches out, and for that a PCA has no second block
to predict from.

make text is clean; no code block changed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
The quality PCA can be cross-validated, by holding out cells rather than
whole batches, and the paragraph now says so: the empty cells mark an
experiment a PCA cannot do, not an absence of cross-validation.

The number itself waits, and the reasons are recorded in a comment
beside the table so the next editor does not rediscover them.

The released library raises LinAlgError on this block, since the
element-wise scheme reached the caller's own missing cells before the
fix that ships from 1.83.1, so a block computing it today would fail the
checker against the release. Dropping the incomplete rows is not a way
round it: only 27 of the 46 batches have a full quality row, and their
values swing from +11.8 to -16.2 on the second component.

Even with the fix the second component is not stable enough to print
from one split: over ten it runs 13.9 to 33.4, mean 26.3, against 30.6
to 38.9, mean 33.8, on the first. An average over splits is the quantity
to quote, which is what the two PLS rows already report.

And a cell-wise value answers a different question from those two rows,
which hold whole batches out, so it needs its own column or its own
sentence rather than the one that column carries.

make text is clean; no code block changed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
The ladder of models was tabulated after the two single-block PLS models
and again after the batch PLS, but the multiblock PLS on the two
initial-condition blocks was left out of both, so the reader met its
36.4% only in prose. Add a table straight after the multiblock section
carrying that row, and repeat the row in the trajectory table, so the
tables accumulate one rung per section.

Neither the multiblock nor the batch PLS row can carry a cross-validated
value: the package cross-validates a single-block PLS only, and on 46
batches the held-out estimate for a block of 3575 columns moves too much
between fold splits to quote. The caption now says both reasons.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
The sentence about a loading changing sign named the tag in prose only,
so a reader looking for it in the figure or in their own data had no
label to match. Give it the tag name in monospace, as the page does at
first mention.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
The paragraph made a simple point at length: a missing cell has no
contribution, and the rest of the batch is unaffected. It then walked
through batch 20's scores and the NIPALS estimate, which the reader does
not need to accept the point. Keep the point and the shape of the
contribution vector; drop the walk-through.

Also correct the argument name in the deferred-work comment above the
first quality table. The element-wise scheme is selected with
cv_scheme="ekf", not method="ekf".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
The page said a PCA has no second block to predict a held-out batch
from, so its cells are empty. That contradicts the cross-validation
section of the latent-variable chapter, which does hold whole rows out
of a PCA and reports a Q2. A PCA can be cross-validated by rows; the
objection is that the held-out row supplies the scores that rebuild it,
so the value reaches 100% once the components equal the variables. Say
that, and link to the section that sets it out.

Also record in the deferred-work comment that the element-wise value
depends on the number of element-folds more than on the seed: 8.8% at
cv=2, 26.3% at cv=7 and 35.9% at leave-one-cell-out on this block.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
The quality ladder's first row had no held-out value, because the
element-wise scheme could not cross-validate a block that already has
missing cells, and the dryer's quality block has 19 of them. That is
fixed in 1.83.1, now published, so the page computes the value.

It holds out a seventh of the measured cells at a time and averages over
fifty partitions. One partition is not enough to print: the
second-component value swings 13.9 to 33.4 across ten seeds, where fifty
partitions leave a seed-to-seed spread of about a point. The block costs
six seconds more in the checker.

The number sits in a sentence rather than in the held-out column, since
it answers a different question from the two PLS rows beside it: how
well one attribute is predicted from the other seven of the same batch,
not how well all eight are predicted for an unseen batch.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
The chapter fitted a two-component PCA on the quality block and printed its
cross-validated value in passing, leaving the reader without the one thing
the value is for: how many components the block actually carries.

A short section now answers that. R2 and the cell-wise Q2 are tabled over
four components, where they part company at once: Q2 peaks at one component
and is negative from three, so a three-component model predicts a held-out
cell worse than the block's own average. The two-component model is kept, and
the section says plainly why, so the choice is visible rather than implied.

The three quality-ladder tables gain a "held out" column and the PCA row's
Q2 cells are filled with the cell-wise value that was previously buried in
the prose. The column is what makes that legitimate: the PLS rows hold out
whole batches and are comparable with each other and with every later rung,
the PCA row holds out single cells and is not. The paragraph that used to
explain the empty cells shrinks accordingly.

The cross-validation moves up to where the model is fitted, so the chapter
computes it once. No published number changes; the two values already in the
tables are the first two of the four now shown.

make check-code-file passes all 71 blocks of the chapter, make text has no
warnings, make html builds with no goatcounter string in the output.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
The page described its six models as a ladder and kept reaching for the
figure: climbing it, its rungs, the rungs being comparable, the ladder
paying off. The models are a sequence that adds one block at a time, and
saying that plainly costs nothing and asks less of the reader.

Seven places, all prose. One of them made a claim the table does not
support, and it is corrected rather than reworded: the quality column
reads 70.3, 22.2, 26.2, 36.4, 41.0, 47.0, so "each adds to the one before"
only holds once the PCA is set aside, since that model describes the
quality block rather than predicting it. The sentence now says so.

make text and make html both build with no warnings, and the built HTML
carries no goatcounter string.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
The section said a PLS model from each initial-condition block answers the
question one block at a time, without saying which two blocks or in which
order. The code below it fits the chemistry first, so the sentence now
names them in that order.

make text builds with no warnings.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
The paragraph explained in passing that a negative Q2 means the batches are
predicted worse than by the average batch. The section above it already
makes that point where the reader first meets a negative value, so the
gloss is cut rather than repeated.

"The operating conditions keep about half of their fitted value" went with
it. Half is the ratio at two components, 11.1 against 26.2; at one
component the same ratio is 14.8 against 20.7, near three quarters. The
sentence did not say which column it meant, so it described one and
misdescribed the other. It now says what the table shows: the chemistry is
negative at both components and the operating conditions positive at both.

make text builds with no warnings.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
The tables named the blocks as bare words, "PLS from Zchem" and "PLS from
Zop", while the prose introducing the same four blocks set them as maths.
The tables now match: Z with a chem or op subscript, and X where a row
names the trajectory block.

Two labels were asked for; five carry a block symbol, so all five change
rather than half. "Multiblock PLS on Z", "Batch PCA on X" and "Batch PLS
on X" would otherwise have sat in the same column as the converted ones.
The column headers of the per-block table change with them, since they
name the same objects as the row labels beside them.

The sentence opening the initial-conditions section goes with them. It
named the two blocks in monospace, which read as the Python variables
rather than the blocks themselves, and it is parallel in structure to the
maths sentence later on the page.

Labels with no symbol are untouched: "PCA on quality", "Batch multiblock
PLS", and the plain-words column describing each block. Code blocks keep
the identifiers Zchem and Zop, which are variable names.

make text builds with no warnings.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
Both captions said the package cross-validates a single-block PLS and not a
multiblock one, which reads as a property of the method. It is not. Holding
out whole batches is sound for a multiblock model for the same reason it is
sound for the two single-block rows: the super score of a held-out batch
comes from its Z blocks alone, so its quality never reaches its own
prediction, and having two Z blocks rather than one does not touch that
argument.

The captions now say the package does not yet implement it. The second one
also separates its two reasons, since the batch PLS row is empty for an
unrelated one: on 46 batches the held-out estimate of a 3575-column block
moves too much between groupings to quote as a single number.

make text builds with no warnings.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LXthGpHLQFfGubBiKFtGAE
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants