feat(fidelity): measurable word-fidelity program; corpus round trips reach zero findings

[m0ustaflex]

TL;DR:

Repro file:
wordprocessingml-comprehensive.docx — the corpus fixture whose round trip surfaced every engine defect fixed here (46 findings when it joined the corpus, 0 at head).

Summary:

• Adds the Word-fidelity program as an openspec change: the criteria catalogue (159 criteria × 6 axes), the design decisions, and the phased plan.
• New crates/ooxml-fidelity: two independent oracles — a canonical structural fingerprint and a WML semantic digest — plus an element census, a declared-normalizations registry, and the byte rules. The oracles parse package bytes with their own reader, so a model gap cannot hide a save loss; diffs are paths, never booleans.
• New corpus harness: sha256-pinned manifest, per-fixture findings goldens regenerable only under GOLDEN_UPDATE=1, a fixed-point gate (save → reopen → save byte-identical), builder-drift gates, and meta-oracles (teeth, blind-spot pairs, floor).
• Nine save-fidelity fixes driving the corpus findings to zero on both fixtures, including a byte-stable fixed point and generic preservation of foreign markup, unknown attributes, and custom root bindings across the document, headers, and footers.
• A ledger (scorecard/ledger.json) enforced by tests: the catalogue and the ledger cannot diverge, no status outranks its target, every claim names a test that exists, every known defect is reproduced by a ceiling probe, and every measured number recomputes exactly. Regeneration refuses a number that moved the wrong way.

Reading guide (one pass, in commit order):

1. 7e4168d docs(docx): specify the word fidelity program and its criteria catalogue — start here; everything after implements it. The scorecard spec's tables are normative: the ledger test parses them.

2. 98c3edd feat(fidelity): package oracles, round-trip gates, and the ledger seed — the instruments (xml reader, fingerprint, census, digest, registry) and the first gates. The ledger is seeded honestly: statuses as they were, no gate names invented.

3. 348a710 feat(fidelity): comprehensive corpus with manifest, findings, and builder gates — the deterministic fixture and its builder-drift gate, the manifest gate, and the corpus runner pinning 46 findings as the measured gap.

4. 99c1ea7 → 8a6ed7d (seven commits) — the ratchet at work: each save fix regenerates the findings goldens downward, and the golden diff in each commit is the proof. 12e29db (fixed point; field and comment identity reuse) is the deepest.

5. 8105002 feat(docx): preserve foreign markup, attributes, and root bindings through save — the generic unknown-XML carry, wired through parse → model → wasm → TS → save.

6. 4ff9f31 — the collaboration seed regenerated once for the save fixes above.

7. 2c9e855 feat(fidelity): enforce the ledger with a scorecard gate and defect ceilings — the ledger stops being a document and becomes a gate; defects.rs probes each known defect so headroom cannot be invented.

8. c2a480f refactor(docx): share the complex-field builder between the two parsers — the two paragraph parsers held byte-identical copies of the complex-field machinery; one shared builder now serves both.

9. 48776b4 fix(docx): first review round — review findings: the story roots never declare the chartex family, so authored cx*/aink/am3d/oel root declarations now survive as custom bindings (the filter is anchored to the serializer's own prefix set, drift-tested); raw foreign blocks inside table cells are replayed instead of silently dropped; the thin editing envelope carries header bindings; test-module docs tightened.

Every commit builds and passes its own tests, so the series bisects.

Not covered here (tracked in tasks.md and the ledger): the edit-fidelity lane (phase 6), layout and visual gates (phase 8), the acceptance fixture (3.1–3.3), and the Word evidence protocol (phase 9). Three known defects remain, each pinned by a probe in defects.rs.

Word evidence is a record, not an image. The comprehensive fixture's manifest entry names the Word build, the date, and what was observed. No screenshot is committed: an image no test compares is prose that costs bytes.

Pre-existing on main: scripts/python-bindings.test.ts fails two release-wiring assertions on origin/main as well; this branch neither causes nor fixes them.

Test plan:

• cargo test workspace green (1764 tests, includes the new fidelity, corpus, defect, and scorecard gates)
• cargo clippy --all-targets -- -D warnings and cargo fmt --check clean
• bun test suites green locally after bun run build:docx-wasm, except the two pre-existing python-bindings failures noted above
• Collaboration seeds byte-identical (cd apps/demo && bun run check:seeds)
• Negative verification: strip an element from a saved package → census and digest both fire (fidelity::the_guard_has_teeth)
• Negative verification: reorder two attributes in an unmodelled part → byte rule 2 fires while the same noise in a modelled part does not
• Negative verification: edit a fixture builder without regenerating → scripts/corpus-fixtures.test.ts fails
• Counterfactual: disable the foreign-markup carry (raw_foreign_inline) → both survival gates fail
• Counterfactual: fix a pinned defect without lowering its ceiling → defects.rs fails
• CI typecheck:packages green (fails locally on an unrelated build:fonts gap)

[openooxml-bot]

All contributors have signed the CLA — thank you! ✍️ ✅

_{Posted by the CLA bot.}

[greptile-apps]

Greptile Summary

The PR introduces independent DOCX fidelity oracles, a pinned corpus and scorecard program, and broad parse-to-save preservation for foreign OOXML markup while fixing several round-trip losses.

• Adds canonical fingerprints, WML semantic digests, element censuses, normalization rules, and path-based reports.
• Adds corpus, fixed-point, defect-ceiling, builder-drift, and scorecard gates.
• Extends document, header, footer, paragraph, table, section, field, and client-boundary models to preserve more authored data.
• One namespace-classification mismatch still emits invalid foreign markup for several root-declared prefixes.

Confidence Score: 4/5

The PR should not merge until root-declared foreign namespaces are emitted consistently; the remaining comment-density issue is non-blocking.

Foreign cx-family markup can survive in RawXml while its root namespace declaration is filtered out, causing the serializer to emit unresolved prefixes in document, header, or footer XML.

Files Needing Attention: crates/docx-parse/src/document.rs, crates/docx-parse/src/inline.rs, crates/docx-parse/src/serializer/parts.rs

Important Files Changed

Filename	Overview
crates/docx-parse/src/document.rs	Adds document-root namespace preservation, but filters prefixes that the corresponding serializer roots do not actually declare.
crates/docx-parse/src/inline.rs	Adds broad raw foreign-element capture and shared complex-field handling; its serializer-owned prefix classification contributes to the namespace gap.
crates/docx-parse/src/serializer/parts.rs	Re-emits custom root bindings across document, header, and footer parts, but its fixed namespace sets omit prefixes filtered from custom bindings.
crates/ooxml-fidelity/src/wml.rs	Introduces the independent WML semantic digest used to detect path-level round-trip differences.
crates/ooxml-fidelity/src/fingerprint.rs	Adds canonical package-part fingerprinting with declared normalization behavior.
crates/betteroffice-docx/tests/corpus.rs	Adds manifest-pinned corpus round trips, findings goldens, and a save-reopen-save fixed-point gate.
crates/betteroffice-docx/tests/scorecard.rs	Enforces catalogue, ledger, gate, ceiling, and measured-number consistency, with overly verbose comments under repository guidance.
crates/docx-edit/src/wasm.rs	Carries document-level custom root bindings through the thin editing envelope.
packages/docx/src/docx/rustParseFacade.ts	Decodes the new document-root binding field across the Rust-to-TypeScript boundary.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[DOCX package bytes] --> B[Independent fidelity reader]
  B --> C[Fingerprint]
  B --> D[Element census]
  B --> E[WML semantic digest]
  A --> F[DOCX parser]
  F --> G[Document model and RawXml]
  G --> H[Serializer]
  H --> I[Saved DOCX]
  I --> J[Reopen and resave]
  C --> K[Round-trip findings]
  D --> K
  E --> K
  I --> K
  J --> L[Fixed-point gate]
  K --> M[Corpus goldens and scorecard]

Loading

_{Reviews (1): Last reviewed commit: "refactor(docx): share the complex-field ..." | Re-trigger Greptile}

[greptile-apps]

Root namespace bindings are dropped

When a document, header, or footer declares a cx, cx1–cx8, aink, am3d, or oel namespace on its story root, custom_root_bindings filters out the declaration even though the corresponding serializer root does not emit it. Replayed foreign markup then contains an unresolved prefix, producing namespace-invalid OOXML that consumers can reject, repair, or discard.

Knowledge Base Used: DOCX parsing and editing

[greptile-apps]

Test documentation is overly verbose

The new fidelity tests repeatedly add multi-line narratives and implementation-explaining comments where the helpers and assertions are already clear. This conflicts with the repository's concise-documentation convention and adds prose that can drift from the executable gates; the same pattern appears in the scorecard and defect test modules.

Context Used: AGENTS.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[m0ustaflex]

Review round landed as 48776b4.

• Root namespace bindings (P1): confirmed, and wider than reported — DOCUMENT_NAMESPACES and HEADER_FOOTER_NAMESPACES share the same 20-prefix set without the chartex family, so the document root was affected too. The binding filter is now anchored to the serializer's own prefix set (is_story_root_prefix, drift-tested against both boilerplates), and chartex_declarations_on_story_roots_survive_the_round_trip gates it. Reproducing this also surfaced that raw foreign blocks inside table cells were silently dropped by the cell serializer, and that the thin editing envelope dropped header bindings under the wasm feature; both fixed and gated.
• Comment density (P2): test-module headers and probe docs tightened.

Workspace: 1764 tests green, clippy -D warnings clean, wasm builds, seeds byte-identical.

[eliahilse]

Thanks a lot for the effort! Will look into tmrw :)