feat(pdf): port docling's LayoutPostprocessor for #61 classification#64
Merged
artiz merged 24 commits intoJul 13, 2026
Merged
Conversation
…classification The RT-DETR layout detections were used almost raw (a flat 0.3 threshold plus a single greedy overlap pass), which over-detected tables and let a high-score picture suppress a lower-score structured region. Port the cleanup docling's LayoutPostprocessor applies before assembly: - Per-label confidence thresholds (CONFIDENCE_THRESHOLDS): a picture / table / list-item now needs >= 0.5 (0.45 for headings/code/index/etc), not the 0.3 base. Drops the low-confidence detections that double-emit. - Bucketed overlap resolution: regular, picture and wrapper clusters are de-overlapped independently, so a picture no longer suppresses a table or table-of-contents index that scores lower. - Cross-type picture-vs-table rule (_handle_cross_type_overlaps): a picture coinciding with a table (IoU > 0.8) is dropped, keeping the structured table. - Containment drop: a regular region absorbed by a table/index/picture is that special's child, not a separate block, so it is not emitted twice (form/key-value wrappers excluded — this pipeline renders their content from the contained regions). - Render a document_index (table of contents) as a table via TableFormer, instead of skipping it. vs docling groundtruth: table_mislabeled 108 -> 88 (survey no longer rendered as tables), 2203 183 -> 177, right_to_left_03 66 -> 60, normal_4pages 54 -> 52; every previously-exact PDF stays exact. redp5110's table-of-contents is now correctly a table (was a picture); its remaining byte diff is a TableFormer dot-leader column-matching gap tracked in docling-project#60. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017fKFbatf55BK5j8vr3dZjj Signed-off-by: Artem Kustikov <artem.kustikov@gmail.com>
docling renders `mAP @0.5` (and emails) with no space after the `@`; the groundtruth never contains "@ " anywhere. Our TableFormer cell text joined the PDF's word cells (`mAP` `@` `0.5`) with spaces, producing `mAP @ 0.5`. That single extra space widened the header cell and, through the column padding, shifted every row of the table. Gluing `@` to its follower in the assembled cell text drops 2206.01062 from 164 to 134 diff lines vs docling groundtruth (one header cell re-aligns the whole column); 2305-pg9 stays cell-for-cell exact and no other snapshot moves. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017fKFbatf55BK5j8vr3dZjj Signed-off-by: Artem Kustikov <artem.kustikov@gmail.com>
Apply `cargo fmt` (pinned 1.96.0) to the bucketed-resolve and per-label threshold code so the fmt · clippy gate passes. No behavior change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017fKFbatf55BK5j8vr3dZjj Signed-off-by: Artem Kustikov <artem.kustikov@gmail.com>
6554892 to
42ffd7b
Compare
Replace the geometric two-column reading-order heuristic with a faithful port of docling-ibm-models' ReadingOrderPredictor (reading_order_rb.py). Per page it builds an up/down adjacency graph from element geometry (an element is "below" another that is strictly above it and horizontally overlapping, unless a third element interrupts the run), horizontally dilates narrow boxes toward their column neighbours, then depth-first traverses from the top-most elements. page_header/page_footer are ordered as their own groups (first/last), matching docling's per-page split. Everything runs in bottom-left origin with docling's exact BoundingBox predicates; the disabled l2r/r2l maps are omitted. This fixes multi-column reading order (author blocks, two-column body) that the geometric sort got wrong: 2206.01062 134 -> 106 2203.01017v2 177 -> 169 redp5110 230 -> 210 vs docling groundtruth, with every previously-exact PDF still exact (2305-pg9, multi_page, picture_classification, right_to_left_01/02) — the port reproduces docling's order on single-column pages too. normal_4pages moves +4 (a Korean cover page entangled with the separate furniture / heading-number issues). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017fKFbatf55BK5j8vr3dZjj Signed-off-by: Artem Kustikov <artem.kustikov@gmail.com>
Add the intra-page half of docling's `ReadingOrderPredictor.predict_merges`: after ordering, join a text fragment with the following text fragment that lies strictly to its right when the running tail ends with a lowercase letter / comma / hyphen and the candidate starts with a letter. This merges the author columns of an academic title page (`… bpf@zurich.ibm.com Christoph Auer …`) and a paragraph that wraps into the next column into a single block, matching docling. page_header/footer/table/picture/caption/ footnote (and already-consumed regions) are skipped when scanning for the next fragment; cross-page/vertical continuations stay with `merge_continuations`. vs docling groundtruth: 2206.01062 106 -> 94, 2203.01017v2 169 -> 161, redp5110 210 -> 204, with every previously-exact PDF still exact and no other snapshot moved. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017fKFbatf55BK5j8vr3dZjj Signed-off-by: Artem Kustikov <artem.kustikov@gmail.com>
docling never emits U+2022 (•) in its Markdown; an inline CCS-concept separator that our text layer reads as `•` is rendered `·` (U+00B7) by docling. Map it in `clean_text`. List bullets are unaffected — `·` is already in the list-marker strip set — so only inline bullets change (2206.01062 92 vs 94). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017fKFbatf55BK5j8vr3dZjj Signed-off-by: Artem Kustikov <artem.kustikov@gmail.com>
The raw document.xml line-diff byte-compares the four <location value> provenance tokens each laid-out block emits. On the PDF path those come from docling's layout clusters in the reference and ours in the output; the same region is boxed a few grid units apart, so geometry lines diff even when text and structure match. - dclx_diff.py: line-diff that counts a <location> pair as equal when the two values are within --tol grid units; text/structure/unmatched lines stay byte-exact (tol=0 reproduces diff | grep -c '^[<>]'). - dclx_conformance.sh: apply DCLX_TOL (default 2) to PDF fixtures only; OOXML/other formats whose geometry is read from the same source file stay exact. Symlink models/.pdfium into the run cwd so the binary finds its assets when invoked from the scratch dir. - dclx_pdf_tol_sweep.sh: convert each fixture once, re-diff at a ladder of tolerances (exact -> +/-2/5/10 -> geometry-ignored). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017fKFbatf55BK5j8vr3dZjj Signed-off-by: Artem Kustikov <artem.kustikov@gmail.com>
docling's PDF .dclx puts four <location> tokens (l,t,r,b on a 0-511 normalized page grid) on every laid-out block; our output emitted none and collapsed each block to one inline line, so ~half the reference (the provenance lines) had no counterpart. This wraps each text/heading/table/ picture region in Node::Located with the region bbox normalized exactly as docling_core's _create_location_tokens_for_bbox does (round(512*coord/page_dim), clamped 0-511, top-left origin). Our heron cluster boxes match docling's to within ~1 grid unit. merge_continuations / hold_start now look through the Located wrapper (as_paragraph / is_picture_node helpers) so cross-column/page paragraph stitching still works; a merged paragraph keeps the first fragment's provenance. Markdown/JSON already render Located transparently, so their output is unchanged (multi_page markdown stays byte-exact). multi_page .dclx similarity 45% -> ~72% at +/-5 grid-unit tolerance. List-item, code, and furniture provenance are follow-ups. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017fKFbatf55BK5j8vr3dZjj Signed-off-by: Artem Kustikov <artem.kustikov@gmail.com>
16 .dclx archives generated from published docling 2.112.0 (document.xml only, no page images), the reference dclx_conformance.sh scores the PDF --to dclx output against. Matches the tracked groundtruth for the other formats. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017fKFbatf55BK5j8vr3dZjj Signed-off-by: Artem Kustikov <artem.kustikov@gmail.com>
Record the .dclx metric (51% exact / 60% at +/-2 grid units, issue docling-project#32 target >=50%), the 0-511 location normalization we match, why the +/-2 tolerance is honest (only PDF geometry from docling's own layout run; text/structure stay byte-exact; DCLX_TOL=0 reproduces raw diff), the per-fixture split, and the remaining serialization gaps (list/code/ furniture provenance) vs the model-level table/reading-order blockers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017fKFbatf55BK5j8vr3dZjj Signed-off-by: Artem Kustikov <artem.kustikov@gmail.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017fKFbatf55BK5j8vr3dZjj Signed-off-by: Artem Kustikov <artem.kustikov@gmail.com>
Re-port of the PDF list/code/furniture provenance onto issue-61's DocLang model (which already carries ListItem.location + list_item_overlay, so the list path reuses the existing field rather than a Located wrapper). - List items set ListItem.location (emitted after the <ldiv>) and the bullet as <marker>·</marker>, matching docling's block. - Code blocks emit <location> after <code> (emit_code/emit_located gain an optional location). - Page headers/footers become Node::PageFurniture -> <page_header>/ <page_footer> with a furniture layer + location + text at their reading-order position; Markdown/JSON omit them like other furniture. merge_continuations/hold_start look past furniture (is_merge_trailer) so a paragraph continuing across a page break still stitches. PDF .dclx similarity 45% -> 63% at +/-2 grid units (exact 52%, ceiling 65%) on the issue-61 base. Markdown byte-exact (multi_page 0, 2305 32, normal_4pages 54). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017fKFbatf55BK5j8vr3dZjj Signed-off-by: Artem Kustikov <artem.kustikov@gmail.com>
525d8ee to
62c1fd1
Compare
The PDF Markdown metric moved with the docling-project#61 layout/reading-order work and the fixture set shrank to 14 (the two Korean image-only pages lost their text groundtruth): now 5/14 exact, 6/14 whitespace-normalized. Several heavy docs improved (2206.01062 164->92, 2203.01017v2 183->161, table_mislabeled 108->88, right_to_left_03 66->60, 2305.03393v1 32->30); code_and_formula now differs by 5 (code block reflowed + trailing newline). Non-PDF .dclx xlsx 87%->85%; mean unchanged at 91%. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017fKFbatf55BK5j8vr3dZjj Signed-off-by: Artem Kustikov <artem.kustikov@gmail.com>
Our single space before the 1/4 numerator is the faithful rendering; docling's extra space comes from embedded-font OS/2 metrics and matching it would degrade output + risk RTL geometry (a prior trial broke right_to_left_01). Keep the faithful output; close docling-project#63 without a byte-match. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017fKFbatf55BK5j8vr3dZjj Signed-off-by: Artem Kustikov <artem.kustikov@gmail.com>
…project#62) docling drops the line-wrap hyphen between two digit runs (a reference page range `pp. 545-` + `561` -> `545561`); a same-line range (`1162-1167`) never reaches the continuation path and keeps its hyphen. Extends region_text's wrap-dehyphenation, previously alphabetic-only, to the digit-digit case. 2305.03393v1 30->28, 2203.01017v2 161->157; all exact fixtures stay exact. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017fKFbatf55BK5j8vr3dZjj Signed-off-by: Artem Kustikov <artem.kustikov@gmail.com>
9eb4cae to
6713a1e
Compare
…g-project#60) Word-to-cell matching now runs docling's full pipeline (docling-ibm-models tf_cell_matcher.py + matching_post_processor.py, ported bug-for-bug in tf_match.rs): intersection-over-word matching, cell-class-aware good/bad split per column, column-median snapping of unmatched cells, adjacent-column de-duplication, best-intersection final assignment, and row/column-band orphan pickup, followed by docling's per-grid-position response merge and row/col index compression. The bbox decoder's cell classes (previously ignored) feed the post-processor, and the table crop reproduces docling's exact rounding chain (round(bbox) -> x2 -> x1024/h -> round, banker's rounding both times). 2203.01017v2 157->150, redp5110_sampled 204->202; every other groundtruth fixture unchanged (2305-pg9 stays cell-for-cell exact). DOCLING_RS_TF_SIMPLE_MATCH=1 reverts to the old best-overlap matcher. DOCLING_RS_TF_MATCH_DUMP=dir dumps matcher inputs for scripts/test/tf_match_reference.py, which replays them through docling's Python post-processor; that parity harness shows the port reproduces the reference on identical inputs, isolating redp5110's remaining TOC gap to the OTSL predictions themselves (model-level). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EY5KAiquN4YpVf2PXEQkVT Signed-off-by: artiz <artem.kustikov@gmail.com>
…rator docling sorts reading-order heads/children with functools.cmp_to_key(PageElement.__lt__), a fuzzy geometric relation that is not a strict total order on mixed overlap groups. Python's Timsort quietly produces an order; Rust's slice::sort_by detects the Ord violation and panics (odf's LibreOffice PDF exports trip it on the first file, which also left the committed odf snapshots unregenerable and stale). Replace those three sorts with CPython's binarysort (natural-run detection + stable binary insertion) - exactly Timsort for n < 64, a plain stable sort under a consistent comparator, and never panicking. Refresh the odf snapshots, stale since the reading-order port: they pick up the Python-parity ordering, the earlier bullet normalization, and false-picture suppression. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EY5KAiquN4YpVf2PXEQkVT Signed-off-by: artiz <artem.kustikov@gmail.com>
…d table numbers Conformance table picks up the dehyphenation (docling-project#62) and matcher (docling-project#60) deltas: 2305.03393v1 28, 2203.01017v2 150, redp5110_sampled 202. The docling-project#60 blocker entry now records what is ported (matching + crop rounding, with the parity harness) and what remains model-level (the OTSL tag stream on the hard crops). DocLang avg re-measured unchanged at 63% (+/-2). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EY5KAiquN4YpVf2PXEQkVT Signed-off-by: artiz <artem.kustikov@gmail.com>
…zer (docling-project#62) The docling-project#62 quote gap was never a font-decoding problem: docling-parse decodes the subsetted TeX fonts' /Differences + /ToUnicode correctly (quotedblleft -> U+201C), then its cell sanitizer (pdf_sanitators/constants.h) maps every typographic quote - single and double alike - to the ASCII apostrophe, and U+201A to a comma. A double quote in docling output only ever comes from a literal quotedbl glyph (2206's 'text in the wild" pairs a curly open with a literal close). Our clean_text mapped curly doubles to '"' with a Hangul-only exception that patched one symptom; replace it with docling's table (also covering the full hyphen family U+2010..U+2015). 2206.01062 92->86, 2203.01017v2 150->136; all other fixtures unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EY5KAiquN4YpVf2PXEQkVT Signed-off-by: artiz <artem.kustikov@gmail.com>
…der (docling-project#62) docling orders a cluster's cells by their docling-parse cell index (LayoutPostprocessor._sort_cells: sorted(cells, key=c.index)) - the sanitizer's output order, which our cells slice already is. The geometric band-sort loses that on off-baseline glyphs: 2206's inline math '>' sits ~2pt above its line's band and drifted into the following line ('( > 10 pages)' became '( 10 pages) ... complex > tables'). Keep the band-sort only for the legacy pdfium cell path, whose cells carry no docling-parse order. 2203 136->130, 2206 86->80, 2305 28->26, normal_4pages 56->44, redp5110 202->194, table_mislabeled 88->86; the five exact fixtures (both RTL ones included) stay exact. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EY5KAiquN4YpVf2PXEQkVT Signed-off-by: artiz <artem.kustikov@gmail.com>
…order fixes The pdf table/text fixtures pick up the quote normalization and the docling-parse index-order cell join; several of them (2206, 2305, normal_4pages, right_to_left_03, llncsdoc, mets_gbs, scanned/*) had been stale since PR docling-project#29 - the odf panic (fixed two commits ago) aborted every full snapshot regeneration before it reached them. The scanned/OCR fixtures were re-pinned from a source-rebuilt model set (models-v1 assets are unreachable from this environment); if the release-model environment disagrees on OCR text, re-pin them there once. Signed-off-by: artiz <artem.kustikov@gmail.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EY5KAiquN4YpVf2PXEQkVT Signed-off-by: artiz <artem.kustikov@gmail.com>
84f1836 to
84c1e3a
Compare
…reshed numbers Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EY5KAiquN4YpVf2PXEQkVT Signed-off-by: artiz <artem.kustikov@gmail.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EY5KAiquN4YpVf2PXEQkVT Signed-off-by: artiz <artem.kustikov@gmail.com>
dump_bbox reverse-engineered the DocLang location normalization (docling-project#32, done - every laid-out block now carries provenance); probe_page's header already called itself a throwaway. The remaining examples are the live debugging toolkit for conformance work. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EY5KAiquN4YpVf2PXEQkVT Signed-off-by: artiz <artem.kustikov@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The RT-DETR layout detections were used almost raw (a flat 0.3 threshold plus a single greedy overlap pass), which over-detected tables and let a high-score picture suppress a lower-score structured region. Port the cleanup docling's LayoutPostprocessor applies before assembly:
vs docling groundtruth: table_mislabeled 108 -> 88 (survey no longer rendered as tables), 2203 183 -> 177, right_to_left_03 66 -> 60, normal_4pages 54 -> 52; every previously-exact PDF stays exact. redp5110's table-of-contents is now correctly a table (was a picture); its remaining byte diff is a TableFormer dot-leader column-matching gap tracked in #60.
Claude-Session: https://claude.ai/code/session_017fKFbatf55BK5j8vr3dZjj