fix(ci): pin the lint toolchain — unpinned ruff has red-failed main for a week - #492
Merged
Merged
Conversation
…or a week CI has failed on main every day since at least 2026-07-29. Both lint gates installed ruff UNPINNED while the repo pins ruff==0.15.13: ci.yml pip install "ruff>=0.5" -> floated to 0.16.1 custodian-audit.yml pip install ruff vulture ty -> same drift `ruff check .` went from clean to 1996 errors; the Custodian audit reported 1222 findings (the ruff group alone — vulture was clean in CI). None of them real. [tool.ruff.lint] selects a deliberate rule set, and its own comment records BLE001 and S110 as DROPPED — "too noisy across codebase, real legitimate uses". A newer ruff re-enables exactly those: of the 1222, BLE001 accounted for 316 and UP045 for 290. Verified on the same tree: ruff 0.16.1 -> 1222 findings, ruff 0.15.13 -> "All checks passed!" across the full `ruff check .`, root files included. The tree was never dirty. Both jobs now install `-e ".[dev]"`, so the version comes from [project.optional-dependencies].dev. One source of truth, and no version literal left in the workflows to drift again. custodian-audit.yml already carried a paragraph explaining that Custodian itself must be SHA-pinned, because tracking @main once let an upstream change emit "a phantom finding fleet-wide". The next line then installed that pinned auditor's TOOLS unpinned and reproduced the same failure one level down. Pinning the auditor while floating what the auditor runs pins nothing. Also drops `|| true` from the repo install. Best-effort was actively harmful here: on failure the adapters find no ruff, Custodian reports it "not installed" and skips it, and the gate passes vacuously — a green check that audited nothing. Same root cause one layer up, fixed separately in ProtocolWarden/Custodian#72: find_tool() preferred Custodian's own venv over the audited repo's, so a globally-installed custodian-multi reproduced this identically off-CI. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ProtocolWarden
added a commit
that referenced
this pull request
Aug 4, 2026
…roject The custodian-audit workflow hardcodes its own Custodian SHA, separate from pyproject's, and its comment requires the two be bumped together. This PR moved pyproject d6ba8ab -> 7a780b7 without it, so CI would have kept installing the old adapter and the vulture fail-open would have survived in the one place it matters most — the required gate. d6ba8ab predates Custodian 261bbb5, which fixed the adapter building `vulture <src> --min-confidence=N <tests>`, an argument order vulture's argparse rejects (exit 2, empty stdout) that was then read as "no dead code". That is why #492 observed "vulture was clean in CI" while vulture was installed and this repo in fact had 621 findings at the default confidence: every run failed and every failure was swallowed. It is the same vacuous-green mode the adjacent step already warns about for a missing ruff. Also drops the unpinned `pip install vulture`. vulture is a dev dependency now, so `.[dev]` pins it (2.16) beside ruff and ty — removing the moving part rather than relocating it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Aug 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI has been red on
mainevery day since at least 2026-07-29Both lint gates install ruff unpinned, while the repo pins
ruff==0.15.13:ci.ymlpip install "ruff>=0.5"custodian-audit.ymlpip install ruff vulture tyruff check .went from clean to 1996 errors. The Custodian audit reported 1222 findings (the ruff group alone — vulture was clean in CI).None of them are real.
[tool.ruff.lint]selects a deliberate rule set, and its own comment recordsBLE001andS110as dropped — "too noisy across codebase, real legitimate uses". A newer ruff re-enables exactly those: of the 1222,BLE001was 316 andUP045290.Verified on the identical tree:
ruff check .(whole repo, root files included)The tree was never dirty.
Fix
Both jobs now
pip install -e ".[dev]", so the version comes from[project.optional-dependencies].dev. One source of truth, and no version literal left in the workflows to drift again.custodian-audit.ymlalready carried a paragraph explaining that Custodian itself must be SHA-pinned, because tracking@mainonce let an upstream change emit "a phantom finding fleet-wide". The very next line then installed that pinned auditor's tools unpinned and reproduced the same failure one level down. Pinning the auditor while floating what the auditor runs pins nothing.This also drops
|| truefrom the repo install. Best-effort was actively harmful here: on failure the adapters find no ruff, Custodian reports it "not installed" and skips it, and the gate passes vacuously — a green check that audited nothing is worse than a red one.Related
Same root cause one layer up, fixed separately in ProtocolWarden/Custodian#72:
find_tool()preferred Custodian's own venv over the audited repo's, so a globally-installedcustodian-multireproduced this identically off-CI.This PR is a prerequisite for #491, which is blocked on the same red checks.
🤖 Generated with Claude Code