Skip to content

Broaden source-evidenced math limitation detection - #182

Merged
silverstein merged 1 commit into
masterfrom
codex/math-gap-e1b-20260903
Sep 4, 2026
Merged

Broaden source-evidenced math limitation detection#182
silverstein merged 1 commit into
masterfrom
codex/math-gap-e1b-20260903

Conversation

@silverstein

Copy link
Copy Markdown
Member

Summary

  • broaden MATH_NOT_RECONSTRUCTED to source-evidenced lowercase operators and merged-item equations
  • retain fail-closed exclusions for prose, ambiguous values, footnotes, table content, and table checkmarks
  • keep source/share modules, schemas, tool-contract hashes, and the Phase 1 source-bound oracle aligned

Measured evidence

  • full paired olmOCR-bench run: 1,403 PDFs / 7,019 tests / zero conversion failures
  • arxiv_math declaration coverage: 146/522 (28.0%) to 220/522 (42.1%)
  • 420 fewer silent math-proxy failures; 76 documents gained the declaration and none lost it
  • source-content Markdown body byte-identical for 1,403/1,403 documents
  • combined headers/footers plus long-tiny-text control rate: 2/328 (0.61%)

Verification

  • affected/adjoining Mac bank: 170 passed, 6 intentional skips
  • Node-native: 62 passed, 9 intentional skips
  • reproducible share contract: 51 tools, 14 prompts, 122 licensed SBOM components
  • source/share parity and git diff --check pass

Claim boundary

This improves detection and declaration of unreconstructed mathematical content. It does not reconstruct equations, improve the directional math pass score, establish competitor superiority, or authorize a public benchmark claim. The first full Mac aggregate encountered unrelated shared-host timing contention; every observed failure was closed by oracle regeneration or isolated passing runs, and aggregate stability remains separately tracked.

@silverstein
silverstein merged commit 4f45387 into master Sep 4, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant