From 64afdb7a909d2d8e7a3fccf667fa3479248fd790 Mon Sep 17 00:00:00 2001 From: ProtocolWarden <32967198+ProtocolWarden@users.noreply.github.com> Date: Mon, 3 Aug 2026 17:09:22 -0400 Subject: [PATCH] =?UTF-8?q?fix(ci):=20pin=20the=20lint=20toolchain=20?= =?UTF-8?q?=E2=80=94=20unpinned=20ruff=20has=20red-failed=20main=20for=20a?= =?UTF-8?q?=20week?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CI has failed on main every day since at least 2026-07-29. Both lint gates installed ruff UNPINNED while the repo pins ruff==0.15.13: ci.yml pip install "ruff>=0.5" -> floated to 0.16.1 custodian-audit.yml pip install ruff vulture ty -> same drift `ruff check .` went from clean to 1996 errors; the Custodian audit reported 1222 findings (the ruff group alone — vulture was clean in CI). None of them real. [tool.ruff.lint] selects a deliberate rule set, and its own comment records BLE001 and S110 as DROPPED — "too noisy across codebase, real legitimate uses". A newer ruff re-enables exactly those: of the 1222, BLE001 accounted for 316 and UP045 for 290. Verified on the same tree: ruff 0.16.1 -> 1222 findings, ruff 0.15.13 -> "All checks passed!" across the full `ruff check .`, root files included. The tree was never dirty. Both jobs now install `-e ".[dev]"`, so the version comes from [project.optional-dependencies].dev. One source of truth, and no version literal left in the workflows to drift again. custodian-audit.yml already carried a paragraph explaining that Custodian itself must be SHA-pinned, because tracking @main once let an upstream change emit "a phantom finding fleet-wide". The next line then installed that pinned auditor's TOOLS unpinned and reproduced the same failure one level down. Pinning the auditor while floating what the auditor runs pins nothing. Also drops `|| true` from the repo install. Best-effort was actively harmful here: on failure the adapters find no ruff, Custodian reports it "not installed" and skips it, and the gate passes vacuously — a green check that audited nothing. Same root cause one layer up, fixed separately in ProtocolWarden/Custodian#72: find_tool() preferred Custodian's own venv over the audited repo's, so a globally-installed custodian-multi reproduced this identically off-CI. Co-Authored-By: Claude Opus 5 --- .console/log.md | 35 +++++++++++++++++++++++++++ .github/workflows/ci.yml | 11 ++++++++- .github/workflows/custodian-audit.yml | 19 ++++++++++----- 3 files changed, 58 insertions(+), 7 deletions(-) diff --git a/.console/log.md b/.console/log.md index 05306f9b0..39d17061f 100644 --- a/.console/log.md +++ b/.console/log.md @@ -1,3 +1,38 @@ +## 2026-08-03 — fix(ci): pin the lint toolchain, ending a week of red CI on main + +CI has failed on `main` every day since at least 2026-07-29. Cause: both lint gates +installed ruff **unpinned** while the repo pins `ruff==0.15.13`. + +- `ci.yml` — `pip install "ruff>=0.5"` floated to 0.16.1. `ruff check .` went from + clean to **1996 errors**. +- `custodian-audit.yml` — `pip install ruff vulture ty`, same drift. The audit + reported **1222 findings** (the ruff group alone; vulture was clean in CI). + +None of them were real. `[tool.ruff.lint]` selects a deliberate rule set and its own +comment records BLE001 and S110 as DROPPED — "too noisy across codebase, real +legitimate uses". A newer ruff re-enables exactly those: of the 1222, BLE001 was 316 +and UP045 290. Verified locally on the same tree: ruff 0.16.1 → 1222, ruff 0.15.13 → +`All checks passed!` on the full `ruff check .`, root files included. + +Both now install `-e ".[dev]"`, taking the version from +`[project.optional-dependencies].dev` so there is one source of truth and no version +literal in the workflows to drift again. + +The irony worth recording: `custodian-audit.yml` already carried a paragraph +explaining that Custodian itself must be SHA-pinned because tracking `@main` once let +an upstream change emit "a phantom finding fleet-wide". The very next line then +installed that pinned auditor's *tools* unpinned, reproducing the same failure one +level down. Pinning the auditor while floating what the auditor runs pins nothing. + +Also made the repo install non-best-effort. It was `pip install -e . || true`; on +failure the adapters find no ruff, Custodian reports "not installed" and SKIPS it, +and the gate passes vacuously — a green check that audited nothing, which is worse +than a red one. + +Related, same root cause one layer up: Custodian's `find_tool()` preferred its own +venv over the audited repo's, so a globally-installed `custodian-multi` reproduced +this identically off-CI. Fixed in ProtocolWarden/Custodian#72. + ## 2026-07-15 — feat(reviewer): ACTIVATE the council — populate guardrail_paths (§G1) The council's go-live. C1/C2/C3 all merged; `reviewer.council.guardrail_paths` diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 6de94eb1e..ab2b3b9aa 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -20,7 +20,16 @@ jobs: with: python-version: "3.11" - name: Install ruff - run: pip install "ruff>=0.5" + # Install the repo's own pinned toolchain rather than `ruff>=0.5`. A lint + # gate has to run the version the config was written against: `[tool.ruff]` + # here selects a deliberate rule set (BLE001 and S110 are explicitly + # DROPPED as too noisy, per the comment in pyproject), and a newer ruff + # re-enables rules that config never opted into. `ruff>=0.5` floated up to + # 0.16.1 and this job went from clean to 1996 errors — every one of them + # phantom — red-failing CI on main daily from ~2026-07-29. Taking the pin + # from [project.optional-dependencies].dev keeps one source of truth; do + # not reintroduce a version literal here. + run: pip install -e ".[dev]" - name: Run ruff run: ruff check . diff --git a/.github/workflows/custodian-audit.yml b/.github/workflows/custodian-audit.yml index f6b742ef8..04ec48429 100644 --- a/.github/workflows/custodian-audit.yml +++ b/.github/workflows/custodian-audit.yml @@ -26,13 +26,20 @@ jobs: run: | python -m pip install --upgrade pip pip install "custodian[tools] @ git+https://github.com/ProtocolWarden/Custodian.git@d6ba8ab245c6f4e79e9f8fffd4e4221bfaf266e8" - pip install ruff vulture ty + pip install vulture - - name: Install repo (best-effort, for adapter passes) - run: | - if [ -f pyproject.toml ]; then - pip install -e . || true - fi + - name: Install repo and its pinned lint toolchain + # `.[dev]` (not plain `.`) so the adapters run OC's OWN pinned ruff/ty. + # The reproducibility argument in the step above applies one level down: + # pinning Custodian while installing `ruff` unpinned just moves the moving + # part. It floated to 0.16.1 and this gate reported 1222 findings against a + # tree the pinned ruff (0.15.13) calls clean — the same phantom-finding + # failure the Custodian pin was added to prevent. + # + # NOT best-effort (`|| true`) any more: a failed install left the adapters + # with no ruff at all, which Custodian reports as "not installed" and skips. + # The gate then passes vacuously — worse than failing, because it looks green. + run: pip install -e ".[dev]" - name: Materialize boundary artifact file # Decode the boundary disclosure artifact from the base64 CONTENT secret