Skip to content

docs: assessment of the 2009 design notebooks (June, July, September)#89

Open
kgdunn wants to merge 2 commits into
mainfrom
claude/scorepilotsimilar-tool-ideas-lnb3nl
Open

docs: assessment of the 2009 design notebooks (June, July, September)#89
kgdunn wants to merge 2 commits into
mainfrom
claude/scorepilotsimilar-tool-ideas-lnb3nl

Conversation

@kgdunn

@kgdunn kgdunn commented Jun 10, 2026

Copy link
Copy Markdown
Owner

Records what in the June, July, and September 2009 design notebooks (the earlier workflow-DAG chemometrics tool) is superseded by ScorePilot's current architecture, and what is worth building on.

Superseded: lazy/copy-on-change workflow copies, traits/weakrefs, custom digraph, coroutine pipelines, masked-array exclusion bookkeeping — all dissolved by immutable datasets + per-variant PreprocessingSpec + parent_id lineage.

Worth building on (ranked in the doc):

  1. Score excluded samples against the fitted model ("move to validation rather than exclude") and record it in the Logbook
  2. Named validation/test sets + an apply-model-to-new-data endpoint (currently no out-of-sample mechanism exists)
  3. Model inversion for product/process design (Jaeckle & MacGregor): invert PLS/PCR from a desired Y spec to candidate X conditions, with Y-space feasibility pre-check and null-space exploration
  4. "Transformations never alter shape; tools may" invariant for future lags/batch-unfolding support
  5. Preprocessing breadth: MSC/SNV, smoothing, trimming, winsorising; robust PCA
  6. Monitoring triggers/notifiers over T²/SPE limits (future module)
  7. Adaptive model update ("refresh this model on dataset D" as a child variant)
  8. Multiblock / SIMCA / PLS-DA / missing-data-tolerant fitting, upstream in process_improve
  9. UX: variable-range name patterns, odd/even row splits, scatter-plot matrix / correlation map / sparklines

Docs-only change; version bumped 0.22.0 → 0.22.1 with uv.lock refreshed. (The second July upload was byte-identical to the notebook already covered.)

https://claude.ai/code/session_01CErviAK6p4MuZiVWsQT3vR

claude added 2 commits June 10, 2026 20:44
Summarize the July and September 2009 notebooks (workflow-DAG
chemometrics tool design): what is superseded by the immutable
dataset + PreprocessingSpec + parent_id architecture, and what is
worth building on — scoring excluded samples against the fitted
model, named validation sets / predict endpoint, the
tools-vs-transformations shape invariant, monitoring triggers,
and the multiblock/classification model roadmap.

https://claude.ai/code/session_01CErviAK6p4MuZiVWsQT3vR
The June notebook adds a worked treatment of latent-variable model
inversion for product design (Jaeckle & MacGregor), a Y-space
feasibility pre-check, null-space exploration, plus preprocessing
breadth (MSC/SNV, winsorising), robust PCA, adaptive model update,
and EDA plot ideas. The July upload was byte-identical to the
notebook already covered.

https://claude.ai/code/session_01CErviAK6p4MuZiVWsQT3vR
@kgdunn kgdunn changed the title docs: assessment of the 2009 design notebooks docs: assessment of the 2009 design notebooks (June, July, September) Jun 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants