Skip to content

feat: page-aware mAP scoring (thread page_aware through the pipeline)#158

Open
Divya-Bhargavi wants to merge 3 commits into
awslabs:devfrom
Divya-Bhargavi:dbharga_bbox_page_aware_map
Open

feat: page-aware mAP scoring (thread page_aware through the pipeline)#158
Divya-Bhargavi wants to merge 3 commits into
awslabs:devfrom
Divya-Bhargavi:dbharga_bbox_page_aware_map

Conversation

@Divya-Bhargavi

Copy link
Copy Markdown
Contributor

Summary

Threads page-awareness through the mAP scoring pipeline, building on the
page_aware BBoxIoUComparator from this base branch (sr/bbox-page-aware).

That branch made the comparator page-aware, but the end-to-end mAP path still
constructed a plain BBoxIoUComparator(), so a prediction with correct
coordinates on the wrong page would still score as a hit in mean_ap. This
PR closes that gap.

Changes

  • MAPCalculator(page_aware=...) — passes the flag to its
    BBoxIoUComparator.
  • BBoxMAPAccumulator(page_aware=...) — forwards to MAPCalculator for the
    bulk path.
  • compare_with(add_bbox_metrics=True, bbox_page_aware=...) (and the
    StructuredModel wrapper) — single-document path.

Default stays page_aware=False, so existing behavior is unchanged. No
data-model change was needed: page-suffixed boxes
([[x1, y1], [x2, y2], page] / [x1, y1, x2, y2, page]) already flow from
_bbox extras straight to the comparator.

Behavior

With page_aware=True, a field is a localization hit only when the page matches
and IoU ≥ threshold:

GT page Pred page Coords page-aware mAP
1 1 identical 1.0
1 2 identical 0.0 (wrong page)
1 (none) identical 0.0 (page-less is a miss)

Page-unaware (default) ignores the page and scores the geometry only.

Usage

# Bulk
evaluator = BulkStructuredModelEvaluator(
    accumulators=[BBoxMAPAccumulator(page_aware=True)],
)

# Single document
result = gt.compare_with(pred, add_bbox_metrics=True, bbox_page_aware=True)

with page-suffixed rich values, e.g.
{"_value": "Acme", "_bbox": [[10, 20], [200, 50], 1]}.

Testing

  • 11 new tests across the calculator, compare_with, and bulk accumulator
    paths (same-page hit, wrong-page miss, page-less miss, default-unaware).
  • Full suite: 1452 passed, 2 skipped.
  • ruff check and ruff format --check clean on changed files.

Stacked on sr/bbox-page-aware; once that merges, this can retarget to dev.

By submitting this pull request, I confirm that you can use, modify, copy, and
redistribute this contribution, under the terms of your choice.

sromoam and others added 2 commits June 26, 2026 16:43
* Page awareness in scoring
* Evaluation configuratin, page_aware flag on the bboxIOUComparator.
* Tests
* Documentation
Builds on the page_aware BBoxIoUComparator: adds a page_aware flag to
MAPCalculator and BBoxMAPAccumulator, and a bbox_page_aware flag to
compare_with(add_bbox_metrics=True), so the end-to-end mAP path enforces
page matching (a correct box on the wrong page scores as a miss). Default
stays page-unaware for backward compatibility.

- MAPCalculator/BBoxMAPAccumulator: page_aware param -> BBoxIoUComparator
- compare_with / StructuredModel.compare_with: bbox_page_aware param
- 11 tests across calculator, compare_with, and bulk accumulator paths
- docs: page-aware mAP scoring section in bbox-map-metrics.md
@adiadd
adiadd changed the base branch from sr/bbox-page-aware to dev June 30, 2026 20:37
Add TestPageAwareMultiPageAggregate to cover documents with several fields
spread across pages. Existing page-aware tests are all single-field/single-box;
these pin the aggregate mean_ap over a mix of correct- and wrong-page fields
(0.5 mixed, 1.0 all-correct, 0.0 all-wrong) plus a page-unaware control that
regresses if page-awareness stops being applied.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants