The latest-state copy of a record is slimmed too (#1129) #2312
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: Python suite ratchet | |
| # The python test suite is not otherwise run here. That is how nine test files which could not | |
| # even import went unnoticed: every test in them errored, and nothing reported it. | |
| # | |
| # Making the suite a blocking gate today would mean fixing 131 pre-existing failures before | |
| # anything else could land. This job takes the middle path -- it records what fails today in | |
| # tools/python_suite_baseline.json and fails only when a change ADDS a failure. Existing | |
| # failures stay visible, and the baseline shrinks as they are fixed (rerun with --record). | |
| on: | |
| push: | |
| branches: | |
| - main | |
| pull_request: | |
| # No `branches:` filter here, deliberately. This key filters on the BASE branch, and a | |
| # stacked pull request is based on the branch below it, not on main -- so this job did not | |
| # run on one at all. Worse, when the branch below it merged, GitHub retargeted the pull | |
| # request to main and did NOT re-run anything: the stale checks from the feature-branch | |
| # days stayed green and the pull request merged reporting success from a job that had | |
| # never seen it. Running on every pull request costs a few Actions minutes and is the only | |
| # way the gate covers the changes that actually reach main. | |
| workflow_dispatch: | |
| permissions: | |
| contents: read | |
| jobs: | |
| ratchet: | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 45 | |
| steps: | |
| - uses: actions/checkout@v4 | |
| - uses: actions/setup-python@v5 | |
| with: | |
| python-version: "3.10" | |
| # Nothing was installed here, so `import yaml` raised and the two ShippedStackTest cases | |
| # that read the shipped prometheus.yml errored on every run -- the monitoring config went | |
| # unchecked while the job that checks it reported the failure as pre-existing. It is the | |
| # only third-party import in the suite; everything else runs on the standard library. | |
| - name: Install what the suite imports | |
| run: python3 -m pip install --disable-pip-version-check pyyaml==6.0.2 | |
| - name: Run the suite against the recorded baseline | |
| run: python3 tools/run_python_suite_ratchet.py |