Merge PR #1: identity reset, a gate that can actually fail, and the c… #43
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # The fidelity gate, run on a pinned Linux oracle. | |
| # | |
| # Why Linux is canonical: the fidelity numbers depend on the renderer's fonts. | |
| # Local Windows runs render with real Arial/Times New Roman; this container | |
| # renders with Liberation metrics-compatible substitutes. Same code, different | |
| # wraps. One environment has to be the reference, and it must be the one | |
| # everyone can reproduce -- so CI is the number of record and local runs are | |
| # indicative. | |
| # | |
| # Two lanes (testkit/runall.py lanes): refine OFF is the uncontaminated | |
| # converter number; refine ON is the product default. refine() tunes against | |
| # the same renderer the gate measures with, so only the pair is meaningful. | |
| # | |
| # Provisioning is scripts/bootstrap.sh, the same command a contributor runs, so | |
| # CI cannot drift away from the documented setup without going red. --strict | |
| # makes a missing oracle a failure here: an incomplete corpus silently produces | |
| # numbers for a different corpus than the one on record. | |
| # | |
| # The dependency versions come from uv.lock (--frozen). The goldens are pinned | |
| # to the PyMuPDF version -- measured: 1.26 and 1.24 both put 02_research_paper | |
| # p2 at 4 blocks where 1.28 puts 7 -- so an unpinned resolve would fail the | |
| # golden step for a reason that has nothing to do with this repository's code. | |
| name: gate | |
| on: | |
| push: | |
| branches: [main] | |
| pull_request: | |
| jobs: | |
| gate: | |
| runs-on: ubuntu-24.04 | |
| timeout-minutes: 60 | |
| steps: | |
| - uses: actions/checkout@v4 | |
| - uses: actions/setup-python@v5 | |
| with: | |
| python-version: "3.12" | |
| - name: Install uv (uv.lock is the pinned truth) | |
| uses: astral-sh/setup-uv@v5 | |
| with: | |
| enable-cache: true | |
| - name: Provision the oracles (LibreOffice, Chromium, fonts) + deps | |
| run: bash scripts/bootstrap.sh --strict | |
| - name: Generate the 16-document corpus | |
| run: | | |
| uv run python testkit/gen_corpus.py testkit/adv | |
| uv run python corpus/make_corpus.py | |
| - name: Unit tests (write purity, corpus degradation) | |
| run: | | |
| uv run python tests/test_purity.py | |
| uv run python tests/test_corpus_degradation.py | |
| - name: Golden IR - the parser's output must not drift | |
| run: uv run python testkit/golden_ir.py verify | |
| - name: Fidelity gate, both lanes | |
| env: | |
| REFINE: lanes | |
| run: uv run python testkit/runall.py testkit/adv corpus/pdfs | |
| - name: Backend parity - the licence-swap verdict | |
| # Reporting-only by design until the swap lands: this is the number M2 | |
| # is driving to zero, so a non-zero count is the expected state, not a | |
| # broken build. It runs in CI because it once lost the ability to run | |
| # at all (an undeclared pypdfium2) and nobody noticed for weeks -- a | |
| # gate that cannot run looks exactly like a gate that passes. | |
| continue-on-error: true | |
| run: uv run python testkit/backend_parity.py --refine 3 | |
| - name: Upload lane results | |
| if: always() | |
| uses: actions/upload-artifact@v4 | |
| with: | |
| name: gate-results | |
| path: | | |
| testkit/batch/lane_norefine/results.json | |
| testkit/batch/lane_refine/results.json | |
| if-no-files-found: warn |