D79: deterministic structure skeleton + bottom-up, consumed summaries (design)

[fazpu]

Design-only PR: new decision D79 in decisions.md plus §4.1 in plan/designs/e0_files_design.md.

What it decides

The one-shot E0 structurer (one frontier-model call over ≤200K chars returning the whole tree — spans, roles, summaries — as one strict-JSON response with char-offset proposals) is replaced, shape not contract:

1. Skeleton parsed deterministically from the conversion-controlled markdown's explicit headings; LLM only as a fallback for structureless documents, returning searchable heading strings, never raw offsets (the snap can make offsets well-formed but never correct — no anchor recovery exists).
2. Summaries bottom-up: one bounded flash-class call per leaf section, parents composed from child one-liners; parallel, content-hash cached per section, so re-ingests only re-summarize changed sections. Bounded context is exactly where cheap models are reliable (§2.4 documents where they aren't: long-context strict-JSON).
3. Summaries consumed, not archived: into E1 context prefixes (as §4 already claimed but reality never did) and into the E2 bundle as the D31 "section path + summary" element (target + ancestors) — closes the summary half of E2 context bundle diverges from the D31 design: section summary and entity hints are missing #163. added_context gains a summary tag, grounding gate verifies it, extractor version bumps.

Pipeline order already runs structure before extraction; this closes the consumption gap the owner asked for ("page index first, then claim extraction with summaries as context").

Sequencing

Implementation behind #161 (loss ledger) so the bundle change lands with its measurement in place. Entity hints stay out — separate decision gated on #148 lint (see D79 rejected alternatives).

Owner-directed 2026-07-27: the system must scale.

• I have read and agree to the RememberStack Contributor License Agreement v1.0.

[fazpu]

Revision 68ac4c0 applies both reviews (Grok-4.5 and Codex gpt-5.6-sol xhigh, both REQUEST-CHANGES; findings overlapped heavily). Load-bearing changes: placement moves to the root reduction call; role assignment specified (deterministic title rules → bounded title-only classifier → explicit body, own generation); fallback triggers on heading density / oversized leaves with block-contained string anchors; summary calls bounded by construction (block-grain sharding, parent-direct blocks, balanced fan-in, full cache key). The two deepest findings — summary-as-grounding-source is a fact-injection channel past D32 layer 2 (both reviews), and ancestor summaries break D56 edit-local reuse (Codex) — are resolved by one principle now in the decision: summaries are orientation-only — never an added_context source (no summary kind exists) and excluded from extraction_input_hash, so re-summarization never fans out into re-extraction. Also: restored the §4A crossref heading the first cut swallowed, added the Refines-D71 clause and the generation/migration story, and de-overclaimed the wording (cliffs are observed code risks; luna is extraction-tier; ~50K is the near-cap ceiling; cost win is expected-to-be-measured).

[github-actions]

Coverage report

This PR does not seem to contain any modification to coverable code.