Refactor algorithm dispatch into registries and add a scientific validation harness

[richford]

Summary

This PR makes jsCAT's algorithm families (ability estimators, item selectors) pluggable through registries, adds an automated golden-test harness that validates every estimator against catR on each CI run, and establishes the contribution infrastructure (CONTRIBUTING.md, PR template, .ai/rules/) needed to review algorithmic PRs, including AI-generated ones, on machine-checkable evidence rather than trust.

Motivation

A recent community PR (WLE estimation, #53) was psychometrically sound and also completely AI-generated. I suspect these kinds of contributions will become more and more common. I haven't merged #53 yet because I want to make sure we have a better architecture for these kinds of contributions. To add his method, @ben-domingue had to edit Cat in three places (the validMethods array, the if/else dispatch, and duplicated optimizer scaffolding), and its validation evidence was a one-off plot. This PR restructures the library so that the same contribution becomes: one new file, one registry line, tests, and regenerated fixtures that CI re-validates forever. After this lands, I'll ask @ben-domingue to re-submit the WLE PR against this architecture as its first test case.

Architecture

graph LR
    Cat[Cat] -->|"getEstimator(method)"| ER[estimators/registry.ts]
    Cat -->|"getSelector(method)"| SR[selectors/registry.ts]
    ER --> MLE[mle.ts]
    ER --> EAP[eap.ts]
    ER -.->|"future: one file + one line"| WLE[wle.ts]
    SR --> MFI[mfi.ts]
    SR --> CL[closest.ts]
    SR --> RND[random.ts]
    SR --> FX[fixed.ts]
    SR --> MID[middle.ts]
    MLE & EAP --> LL[log-likelihood.ts<br/>optimize.ts]

Loading

• Estimators implement AbilityEstimator in src/estimators/; selectors implement ItemSelector in src/selectors/. The registry objects are the single source of truth — the method type unions, runtime validation, and dispatch all derive from them.
• Shared numerics are factored out once: logLikelihood and maximizeOverTheta (Powell scaffolding).
• No public API changes. Method names remain case-insensitive strings at the boundary (typed as autocompleting loose unions), error messages are unchanged, and all 224 pre-existing tests pass without modification.

Bug fixes

• Cat's private likelihood seeded its reduce with 1, returning ln L + 1. Benign today (the offset cancelled in the MLE argmax and EAP posterior ratio) but a landmine for any future consumer; the shared helper seeds at 0.
• Clowder._selectNextItem used Math.random() for the validated/unvalidated coin flip while everything else used the seeded RNG, breaking the reproducibility that randomSeed promises. Now uses this._rng().
• Jest's v8 coverage provider reported nondeterministic numbers (identical runs oscillated 94–100%); switched to the Istanbul provider. Coverage is now stable at 100% statements/functions/lines, 99.1% branches.

Golden validation harness

npm run fixtures:generate deterministically builds a 50-item × 100-examinee fixture set (fixed seed; 2PL plus c = 0.15 items to exercise the 3PL path). src/__tests__/golden.test.ts asserts on every run:

1. Characterization: estimates exactly reproduce the committed baseline — catches unintended numerical drift from refactors.
2. External reference: estimates agree with catR (validation/generate-r-reference.R) within tolerance — catches algorithmic error. No R needed at test time.

Observed agreement on the committed fixtures:

Estimator	max abs diff vs catR	mean	Source of difference
EAP	2.4e-6	3.5e-8	Same rectangular quadrature
MLE	3.6e-3	3.5e-4	Powell vs. catR optimizer, same objective

Tolerances are set at max 0.02 / mean 0.002 (~5× headroom) with observed values documented in the test.

Contribution infrastructure

• CONTRIBUTING.md: dev setup, architecture overview, and an end-to-end tutorial for adding an estimator (WLE used as the worked example), plus the requirements for algorithmic PRs: literature reference, documented model scope, analytic tests, golden fixtures.
• PR template with the algorithmic checklist and an AI-disclosure section. Policy: AI-assisted PRs are welcome; validation evidence must be machine-checkable.
• .ai/rules/ (7 rules) surfaced via AGENTS.md + CLAUDE.md symlink, so AI coding agents pick up the architecture and validation requirements automatically.
• Repo hygiene: untracked .idea/ and validation/.DS_Store; validation workflow documented in validation/README.md.

Test plan

• npm test: 233/233 passing, including the live catR comparison
• npm run lint: clean (one pre-existing warning in corpus.ts)
• npx tsc --noEmit: clean

Follow-ups (separate PRs)

• Re-submission of WLE estimation against the new extension points (the test case for this architecture). Note for that review: catR's method = "WL" implements Warm's exact correction, which differs from the penalized-ML form under 3PL — the golden test may legitimately surface a scoping discussion on the c > 0 fixture items.
• Pin or vendor the optimization-js GitHub-fork dependency; move @types/node to devDependencies.
• ESM/CJS dual build and toolchain updates (TS 4.7 → 5.x, Jest 28 → 29+).

AI assistance disclosure

Architecture, implementation, tests, and documentation drafted with Claude (Cowork); design direction, R reference generation, and review by @richford. All numerical changes are covered by the characterization baseline and the catR reference comparison described above.

[coveralls]

Coverage Report for CI Build 27384496497

Coverage decreased (-0.3%) to 99.713%

Details

• Coverage decreased (-0.3%) from the base build.
• Patch coverage: 100 of 100 lines across 13 files are fully covered (100%).
• No coverage regressions found.

Uncovered Changes

No uncovered changes found.

Coverage Regressions

No coverage regressions found.

---

Coverage Stats

Relevant Lines:	466
Covered Lines:	466
Line Coverage:	100.0%
Relevant Branches:	231
Covered Branches:	229
Branch Coverage:	99.13%
Branches in Coverage %:	Yes
Coverage Strength:	231812.3 hits per line

---

💛 - Coveralls