Skip to content

fix: preserve DOCX leading content and support explicit repair revisions - #493

Merged
suguanYang merged 2 commits into
mainfrom
fix/wangbinqi/docx-leading-content
Nov 2, 2026
Merged

suguanYang merged 2 commits into
mainfrom
fix/wangbinqi/docx-leading-content

Conversation

@suguanYang

@suguanYang suguanYang commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Summary

 DOCX postprocessing
-  discard body rows without a section heading
+  retain the title and introduction under the document root
 job(document_id, reparse=true)
+  allow identical bytes to replace that existing document revision

Preserve the four introductory paragraphs before the first level-one heading in the large DOCX. Add an explicit reparse option that requires an existing document ID, allowing the same stored original to repair its revision while preserving the canonical document. Ordinary duplicate uploads, cross-document duplicates and active content claims remain rejected.

Evidence

  • Before: The real DOCX parser regression omitted all four leading paragraphs; explicit same-document repair raised ALREADY_EXISTS; schema accepted reparse without a target.
    After: 50 focused tests pass, including database-backed duplicate-content/admission contracts, unchanged ordinary-update rejection, explicit same-document revision repair, long root-body splitting, and progressive DOCX/MD/TXT contracts.
  • Ruff checks pass; Pyright with Python 3.12 reports zero errors. Manual review found no actionable issues. Complete PR CI passed; production v1.2.43 API/Worker rollout and acceptance of the original 25 MB DOCX are complete.
  • Coordinate with Hipo's explicit same-Source repair admission and SDK 2.8.1 delimiter/checkpoint fixes. Ship Knowhere API and Worker before initiating the repair.

Merge Danger

Door: two-way

Blast Radius: parsing

Existing published snapshots remain immutable. Repair creates a new parse revision under the existing document ID. The explicit option defaults to false; normal content deduplication remains in place.

Production acceptance: the repaired final contains all 768 full original paragraphs, all 16 complete image paths, and the end sentinel. The same Hipo Source and Knowhere canonical document are preserved, with one explicit repair parse revision. Live deployment and deadline/SIGKILL/resume evidence are recorded in Ontos-AI/hipo#452 (follow-up #453).

@suguanYang suguanYang changed the title fix(parser): preserve DOCX introductory paragraphs fix: preserve DOCX leading content and support explicit repair revisions Nov 2, 2026
@suguanYang
suguanYang merged commit 394faf3 into main Nov 2, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant