feat(extract): Phase 18 C5 — LLM extraction assist under the substrate rule

[bryanmatthewsimonson]

The last built-able Phase 18 slice (COMPLEX_CONTENT_DESIGN.md §6). Built rebuild-free like #188 — dist/ untouched, nothing live until you rebuild; CI validates the bundle.

What you get

Scans stop being a dead end. A no-text-layer PDF now offers consent-gated LLM transcription instead of the old refusal. The transcription IS the capture — honestly labeled: extraction.method = 'llm:<model>', a persistent 🤖 banner, the original bytes archived one click away. Consent states plainly the document leaves the device (size + pages shown), Escape declines, and every refusable condition — gates, size cap, page cap, archive space — is checked before the consent modal, because a consent that ends in a foreseeable refusal is a wasted consent.

Scrambled layouts get "Reconstruct with LLM…" — offered only where deterministic extraction itself says it struggled (the warnings banner). The substrate rule, mechanized: every model span is a search key, re-grounded via quote-grounding and either re-canonicalized to the document's own bytes or dropped and counted. The stored capture never contains model-authored body text.

Wire addition (additive, per CLAUDE.md's call-out rule): extraction-method + source-hash tags on 30023 (§6.3/§7) — so consumers can distinguish "deterministic text layer" from "model-transcribed" forever. Read back as a thin record, merged with the session's rich one by source_hash. No existing tag moves.

Documented deferral: page-range chunking. The API has no page-range parameter and splitting PDFs needs a PDF library in the SW — over-cap documents (100 pages / 22MB) refuse with a clear pre-consent error instead.

The reversed-table attack

The adversarial review of the implemented diff (4 dimensions → 18 findings → two skeptics each → 17 confirmed unanimous, all fixed) found something that goes to the heart of the tool: quote-grounding's exact/normalized tiers are substring searches, so '9' grounded inside 'COVID-19', '201' inside '2017', '12' inside '12%' — and the repro assembled a reversed mortality table (2020→7, 2021→41 against a document saying the opposite) that reported "all spans verified against the document text." A fabricated data table would have entered your corpus stamped verified.

Three changes, and the third is the load-bearing one:

1. Token-bounded short keys — a match under 15 chars counts only when its substrate neighbors are non-alphanumeric. Kills the sub-token shards.
2. Substantive-cell requirement — a table survives only if the substrate vouches for at least one grounded worded cell. A table of number-shaped strings that would match anywhere is the model's table, not the document's.
3. Disclosure over fake verification. Row/column association is never machine-checkable against a scrambled substrate — that's the feature's own premise (a correct reconstruction legitimately has non-local cells; a locality check would break it). So the toast and a persistent structure-mode banner (+llm: in the method, which round-trips the wire and therefore banners relay copies too) say plainly: text re-grounds to document bytes; table structure is the model's reading — verify against the archived original.

The other 14 confirmed findings, all fixed

• Base64 inflation: the "30MB" raw cap produced ~40MB requests against the 32MB API limit — guaranteed post-consent rejection. Now 22MB raw, with the math pinned in a test.
• Stale pageMap: a reconstruction without model page hints kept the OLD map against the NEW markdown — wrong page numbers stamped into published claim anchors. The map is deleted when unmapped; absent beats wrong.
• Silent edit clobbering: pending edits refuse the flow up front, and the multi-minute flight re-checks on landing — your state wins, the API cost is sunk.
• Method chaining: re-runs composed …+llm:a+llm:b (a malformed wire value); extractionMethod is now idempotent over the deterministic base, and the post-success banner has no second button anyway (the body is no longer the substrate).
• Stale audit keys (auditableHash), unbannered transcribed copies after Load archive, thin wire read-back replacing rich local records, ignored stored:false from the archive, modal a11y (Escape/focus/dialog).

Accepted residue, documented in JOURNAL: Cancel on the scans path still lands on the ?pdf= error screen (scans dead-ended there pre-C5 too), and a plausible table whose labels genuinely appear in the text can still carry wrong associations — which is exactly what the persistent banner is for.

Verification

• npm test — 1764 pass, 0 fail (45 new: 33 re-grounding engine incl. the reversed-table repros, 12 wire/prompts/client)
• lint 0 errors · node --check on every touched file · npm run build NOT run (your live extension is untouched)
• SMOKE §18 gains 18.32–18.37 (the C5 walk — needs you, a browser, and an API key)
• Independent of fix(archive): "Load archive" served the wrong article — gate on identity #185–feat(scholar): Phase 18 C2 tail — PMC references, ar5iv full text, Crossref #188 (based on main). Note for merge order: this touches reader/index.js regions distinct from the archive stack's, but merging fix(archive): "Load archive" served the wrong article — gate on identity #185–fix(archive): reload must not drift the x tag — carry the preimage, prove it #187 first is cleaner.

🤖 Generated with Claude Code