Skip to content

fix: consolidated rows carry per-cell provenance, not the first row's (26.6.9) - #33

Merged
ancongui merged 1 commit into
mainfrom
fix/consolidation-per-cell-provenance
Jun 15, 2026
Merged

fix: consolidated rows carry per-cell provenance, not the first row's (26.6.9)#33
ancongui merged 1 commit into
mainfrom
fix/consolidation-per-cell-provenance

Conversation

@ancongui

Copy link
Copy Markdown
Contributor

Summary

Fixes a provenance-loss bug in the post-extraction consolidation transform: a cross-document consolidated group (e.g. a "current cap table" merged across several deeds) gave every output row the first input row's bounding box, page, confidence and judge verdict.

Root cause

_rebuild_rows (llm_transformer.py) already computed each output row's row-level pages/confidence from its cited contributors (26.6.8), but every cell still borrowed its bbox/pages/confidence from rows[0] through a single template_by_name map, and dropped each cell's judge/notes to the default uncertain/0.

Observable symptoms in GET /workflows/{id}context.consolidated_field_groups:

  • All rows of cap_table_vigente shared one identical bbox/pages/confidence fingerprint — byte-for-byte equal to the first source document's first row.
  • Row-level pages (from real contributors) no longer matched the cell-level pages (from rows[0]).
  • Every cell came back judge.status = uncertain, confidence = 0.

UI overlays drawn from these bboxes pointed every member at the same rectangle on the same (wrong) page.

Fix

Resolve each output row's provenance source in priority order:

  1. input rows cited via _source_rows;
  2. for a pure 1:1 rewrite (no row in the batch cites anything and the counts match) — the input row at the same index;
  3. input rows matched by a distinctive identity token (an uncited consolidation row);
  4. rows[0] as a last resort.

Within a row, each field takes its best-grounded contributor cell (one with a bbox wins over one without, then higher confidence), and judge/notes now survive the transform. Value-rewriting transforms are unchanged when no contributor is matched.

Test plan

  • 5 new tests in test_llm_transformer_provenance.py (best-per-cell across contributors, single cited contributor, judge propagation, positional rewrite, identity fallback) — written failing first, then green.
  • Full unit suite: 292 passed.
  • ruff check / ruff format --check: clean. pyright: 0 errors.

🤖 Generated with Claude Code

… (26.6.9)

_rebuild_rows threaded row-level provenance from cited contributors but
still stamped every output cell's bbox/pages/confidence from rows[0] via a
single template_by_name map, and dropped each cell's judge/notes to the
default uncertain/0. A cross-document consolidation (e.g. a current cap
table merged across several deeds) therefore gave every member the first
member's bounding box, page and confidence, so UI overlays pointed every
row at the same rectangle and row-level pages stopped matching cell pages.

Resolve each output row's provenance source in priority order: cited
_source_rows -> positional 1:1 for a pure rewrite -> distinctive-identity
match for an uncited row -> rows[0] as a last resort. Within a row each
cell takes its best-grounded contributor cell (bbox present first, then
higher confidence), and judge/notes now survive the transform.
@ancongui
ancongui merged commit c6ebeda into main Jun 15, 2026
6 checks passed
@ancongui
ancongui deleted the fix/consolidation-per-cell-provenance branch June 15, 2026 12:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant