invariants + Bench 2 final arm: the convenient artifact is the misleading one (#4523)

[JDerekLomas]

Three instrument failures in one afternoon, all the same shape: the artifact a reasonable person reaches for first was the wrong one — because it was complete, always-present, or already open, which are exactly the properties a stale or partial record has.

1. A failures-only log reads as a complete log. worker-0.jsonl records only failures and skips. Reading it produced "2,016 failures, zero successes — the job is broken", which I published as an operational alert on OCR fabricates Sanskrit Hindu scripture on Tibetan manuscript folios — 529 books, 355 of them badged first translations #4523. The job was healthy: its output directory grew 7,430 → 7,451 files in 15 minutes with valid Tibetan in them. Retracted. Verify a worker by its output growing, not by its error log being non-empty.
2. An err field capturing a stderr tail reported the same benign trailing warning (kenlm bindings are not installed, printed on every invocation including successful ones) for every failure, hiding the real causes.
3. pages.image_width is on 100% of rows; image_metadata.width on 27.3% — and where both exist they disagree 4,000/4,000 sampled (image_width is the display derivative; one page reads 1200×800 against a real 3888×2592 master). A corpus-wide "42% of pages are too low-res to re-OCR" was built on the populated field and was wrong.

Credit to the peer session that caught (1) and (2) and then self-corrected on (3); I verified all three against the database before writing this.

Docs only — no code paths touched.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
sourcelibrary-v2	Ignored	Preview	Sep 4, 2026 10:10pm UTC

[JDerekLomas]

Scope grew: this branch now also carries the final Bench 2 arm, which is the first engine comparison reported honestly (coverage + conditional + unconditional accuracy, one estimator across arms).

At matched coverage, on the 100-page Wikisource tier:

	Kraken cond / uncond @cov	Gemini cond / uncond @cov
Greek (n=25)	99.3% / 91.4% @92%	97.8% / 89.7% @92%
Latin (n=45)	97.7% / 82.5% @84%	96.3% / 81.7% @85%
German (n=30)	94.7% / 44.2% @47%	99.4% / 96.0% @97%

Three things worth noting:

1. Greek is the first specialist result with both a real sample and matched coverage. The earlier "Kraken beats Gemini on Greek by +0.74pp" was n=4 with no confidence interval and I should not have quoted it; this supersedes it with n=25 at equal coverage.
2. German is a clear specialist loss, and only the new reporting shows it. Conditional accuracy alone reads 94.7% — respectable. The truth is 47% coverage and 44.2% unconditional: it simply cannot read half the pages. That is exactly the failure the old single-number reporting hid.
3. Latin is a tie.

Docs + eval only; no production code paths.