Fail CI on a migration graph that forks WHEN MERGED, not just on the branch - #611
Merged
Conversation
…branch Two branches can each add `0098_<something>.py` depending on `0097`. The filenames differ, so there is no textual conflict, GitHub reports the second PR MERGEABLE/CLEAN, and both branches are individually valid. The moment the second merges, `cardpicker` has two leaf nodes - and pytest-django builds its test database by running `migrate`, so the fork fails at test-database SETUP on EVERY branch in the repo, not just the one that introduced it. This has now happened twice: at 0096 (#568 vs #570) and at 0098 (#573 vs #601). #576 repaired the first fork but prevented nothing, which is why the second arrived within days. Nothing in CI failed either time. WHY THE MERGE RESULT IS THE WHOLE POINT #601's checks were 10/10 green with the collision already live on master - they had run against master BEFORE #573 landed, and GitHub does not re-run a PR's checks when its base moves. A check reading only the PR branch's files sees one leaf and passes; the fork exists only in the merge. So `check_migration_leaves.py --base origin/<base_ref>` unions the worktree's migrations with the base branch's CURRENT tip, resolved at run time, honouring anything the PR deletes (`--no-renames` is load-bearing: a renumber is a delete+add of near-identical content and git otherwise reports it as a rename, which would resurrect the old number and fail a PR that had already fixed itself). HOW IT DECIDES Static `ast` read of every `migrations/` package: filenames are nodes, each file's `dependencies` gives same-app edges, `run_before` gives reversed ones, and a squash's `replaces` removes the nodes it stands in for. Migration modules are never imported or executed, so this needs no settings module, no installed apps, no postgres and no `requirements.txt` - it runs on a bare `actions/setup-python` in about a second. Non-literal dependency entries (`migrations.swappable_dependency(settings.AUTH_USER_MODEL)`, in seven of this repo's migrations) are cross-app by construction and are skipped, not guessed at. Exit code is the finding count, matching docs_lint.py's and check_protected_core_license.py's convention. Findings: more than one leaf per app (the failure), a duplicate NNNN number prefix within an app (the same defect one step earlier, and the actionable instruction), and a dependency naming a migration that does not exist. WHAT IT CANNOT DO, STATED PLAINLY A check run that PASSED before the base moved stays green in GitHub's UI. No CI job can fix that from the inside; branch protection's "Require branches to be up to date before merging" is the setting that closes it, and this makes the forced re-run meaningful. `merge_group` is wired up so a merge queue would close it too. The workflow is its own file rather than another entry in docs-lint.yml, which four open PRs are already editing. Every path glob uses `**`: a single `*` does not match a slash, so `MPCAutofill/cardpicker/*.py` would miss `migrations/` entirely (#588 hit exactly that). Demonstrated red-then-green against the real collision, and kept as permanent regression coverage in `.github/scripts/tests/test_check_migration_leaves.py` (15 tests) rather than as a one-off local run: a scratch repo where master has `0098_card_illustration_consensus_fields` and a feature branch has `0098_rename_printings_count_catalogued`, both on 0097, asserts clean on the branch alone, two leaves against the merge, and clean again once renumbered to 0099 - plus no-finding cases for a normal single-migration PR, a PR touching no migrations, cross-app dependencies, swappable dependencies, squashes and this repo's own tree. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013NhYmT1PxCcyemA16dFDxN
This was referenced Jul 29, 2026
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two branches can each add
0098_<something>.pydepending on0097. The filenames differ, so there is no textual conflict, GitHub reports the second PR MERGEABLE/CLEAN, and both branches are individually valid. The moment the second merges,cardpickerhas two leaf nodes — andpytest-djangobuilds its test database by runningmigrate, so the fork fails at test-database SETUP on every branch in the repo, not just the one that introduced it.This has happened twice: at 0096 (#568 vs #570) and at 0098 (#573 vs #601). #576 repaired the first fork but prevented nothing, which is why the second arrived within days. Nothing in CI failed either time.
Why checking the branch is not enough
#601's checks were 10/10 green with the collision already live on master — they had run against master before #573 landed, and GitHub does not re-run a PR's checks when its base moves. A check that reads only the PR branch's files sees one leaf and passes; the fork exists only in the merge.
So the job resolves the base branch's current tip at run time and checks the union:
--no-renameson the deletion diff is load-bearing, not stylistic: renumbering a migration is a delete+add of near-identical content, which git otherwise reports asR. Without it the old number is resurrected from the base and the PR fails for a collision it had already fixed. A unit test pins this.Demonstrated red-then-green against the real collision
Run against #601's pre-rebase commit
6c0af23dwith master atd0442239:origin/masterorigin/master0099_on top of master's0098_)Case B verbatim:
Cross-checked against Django's own
MigrationLoader, which reports the same two leaves on the same tree — the static reader and the loader agree.That reconstruction is kept as permanent regression coverage in
.github/scripts/tests/test_check_migration_leaves.py(15 tests), not as a one-off local run: a scratch git repo where master has0098_card_illustration_consensus_fieldsand a feature branch has0098_rename_printings_count_catalogued, both on0097, asserting clean on the branch alone, two leaves against the merge, and clean again once renumbered.Proven end to end in real CI, not just locally
Throwaway draft #612 (now closed, branch deleted) added a second
0098depending on0097on top of this branch. Run 30483031963:That proves both halves of the flagged trap: the
**/migrations/**path filter actually fires on a PR whose only relevant change is undermigrations/, and the job goes red with inline annotations. On this PR itself both jobs pass.How it decides
Static
astread of everymigrations/package. Filenames are nodes; each file'sdependenciesgives same-app edges (mirroringMigrationLoader.graph.leaf_nodes(app));run_beforegives reversed ones; a squash'sreplacesremoves the nodes it stands in for. Migration modules are never imported or executed — no settings module, no installed apps, no postgres, norequirements.txt. It runs on a bareactions/setup-pythonin about a second. Non-literal dependency entries (migrations.swappable_dependency(settings.AUTH_USER_MODEL), in seven of this repo's migrations) are cross-app by construction and are skipped, not guessed at.Findings: more than one leaf per app (the failure); a duplicate
NNNNprefix within an app (the same defect one step earlier, and the actionable instruction); a dependency naming a migration that does not exist. Exit code is the finding count, matchingdocs_lint.py's andcheck_protected_core_license.py's convention.What it cannot do, stated plainly
A check run that passed before the base moved stays green in GitHub's UI. No CI job can fix that from the inside. Branch protection's "Require branches to be up to date before merging" is the setting that closes it; this makes the forced re-run meaningful.
merge_groupis wired up so a merge queue would close it too.Because the workflow is path-filtered it does not run on a PR touching no migrations — correct (such a PR cannot fork the graph), but it means marking it required would leave those PRs waiting on a check that never reports. Either leave it unrequired, or require it and drop the
pathsfilter; the script is cheap enough to always run, which is the routecoverage-delta.ymlalready takes.Coordination notes
.github/workflows/migration-graph.yml), not an entry indocs-lint.yml— Derive the PROTECTED CORE roster from the policy doc; gate the decrypt tool #587/Roster tethers: scan recursively; document the identity they could not see #588/Tether the extractor manifest: image_evidence.py -> cohort constants (code-to-code) #590/docs_lint rule 5: MANIFEST.md coverage, both directions #591 are already editing that file'spathslist.**. A single*in a GitHub path filter does not match a slash, soMPCAutofill/cardpicker/*.pywould missmigrations/entirely (Roster tethers: scan recursively; document the identity they could not see #588 hit exactly that)..github/scripts/check_protected_core_license.pyand Tether the extractor manifest: image_evidence.py -> cohort constants (code-to-code) #590'scheck_extractor_manifest_sync.py: stdlib only, no Django, exit code = finding count.Verification
python3 .github/scripts/tests/test_check_migration_leaves.py— 15 tests, OKcheck_migration_leaves.py --base origin/masteron this branch — clean