Skip to content

Lane B: the recall tier, the candidate contract and the pipeline - #20

Merged
tom-snyder merged 10 commits into
mainfrom
feat/lane-b-recall
Oct 4, 2026
Merged

tom-snyder merged 10 commits into
mainfrom
feat/lane-b-recall

Conversation

@tom-snyder

Copy link
Copy Markdown
Member

Phase 6 of the plan: Lane B, first-party source, built as a recall tier with no model, as the gate decided.

What changed

  • The owner's rule selection (data/rules/selection.json, 2026-10-03) narrows the recall federation to budget: 0xdea's noisy bad-words rule, the assert rules (bandit B101 and GitLab's), ten 0xdea code-review heuristics, and test, doc and example trees are out. GitLab's c/ rules are excluded on licence grounds: each is headed "License: GPL 2.0" (generated from flawfinder), under a repository LICENSE that says MIT.
  • Vendored rule pack (data/rules): 177 rule files (GitLab sast-rules at 53bf5cf6, six directories: 59 MIT, 27 Apache-2.0 from gosec, 52 Apache-2.0 from bandit; 0xdea at dae50da6, 39 MIT), their licence bodies, and a per-rule provenance manifest generated by python -m anvil_eval.vendor_rules. A GitLab rule's licence is the stricter of its own header and its companion test file's; 131 rules ported from LGPL-3.0 find-sec-bugs and security-code-scan are excluded (the owner's decision after the review), and any other licence outside MIT and Apache-2.0 refuses the run.
  • internal/recall: opengrep 1.26.0, gosec 2.29.0 and bandit 1.9.4 as subprocesses, version-checked against their pins; absent, failed and clean are three different answers; the scanned repository's suppressions (nosemgrep, #nosec, # nosec, .semgrepignore, .bandit, bandit's and opengrep's silent default excludes) are all switched off; gosec's file coverage is counted (build tags, cgo, Go outside a module are incomplete coverage) and cgo is off while it loads packages; candidates carry a parser-named enclosing function, the CWE, the untrusted snippet and full provenance.
  • internal/laneb: every candidate becomes an unconfirmed SAST finding (confidence 1, no model, no rank), cites its rule in tool.extensions with anvil/ruleProvenance, and checked-in API specs are harvested onto the SAST run for the dynamic tier.
  • Record 1.1.0 (additive): run.tool.extensions[], result.rule, anvil/ruleProvenance; wire schema updated. Store migration 0002: scan_run.recall_candidates, the per-scan count.
  • The scan: anvil scan --repo runs each lane its configuration enables and its policy allows; Lane B is on by default (recall:); a lane left off is stated, and a scan with nothing found and a lane off exits 3, never 0. anvil recall PATH prints candidates without recording.
  • Evaluation: candidates-per-scan re-measured through anvil recall on the 177-rule pack: curl 225 (was 10,714), Anvil 79, flask 9, express 2, spring-petclinic 0; largest push 27 (was 88). Under both bars. The row's decision is left as it was.
  • Known limit, kept by the owner: the path exclusions match by name, so a repository can hide code under spec/ or docs/; stated in the README and CONTRACT.md.
  • No ranker: the owner's decision; nothing is labelled, and on PrimeVul both rankers were at chance.
  • CI: a new job, "Lane B end to end (real tools)", installs the pinned tools by digest, runs the real-tools fixture test with a fatal opt-in and test/e2e/laneb.sh, and proves the opt-in is fatal by hiding opengrep.
  • Docs: CONTRACT.md amendment, NOTICE §1a, THIRD-PARTY-LICENSES.md, README per-language coverage, data/LICENSES/opengrep-binary-distribution.md (the packaging choice, left to the owner), docs/gates/lane-b.json (the exit gate, row by row) and docs/reviews/lane-b-gate.md (the same-family adversarial review).

How it was checked

gofmt, go vet ./..., go test -count=1 ./... and -race, with and without Trivy; eval ruff and 165 tests; test/e2e/fixture.sh and test/e2e/laneb.sh with the real tools. Guards seen failing before trusted: removing --disable-nosem, --ignore-nosec or bandit's exclude sentinel, disabling the GitLab directory allowlist or the symlink handling, planting a third-party import, and mapping a candidate to true_positive each fail a test. A same-family adversarial review found two blocking and six should-fix issues; every one is fixed or decided by the owner (docs/reviews/lane-b-gate.md).

The owner's selection of 2026-10-03 (data/rules/selection.json) narrows the
recall federation to budget: 0xdea's noisy bad-words rule, the assert rules,
ten code-review heuristics and test, doc and example trees are out, and
GitLab's c/ rules are excluded because each is headed GPL 2.0 (generated
from flawfinder). 308 rule files are vendored with their licence bodies and
a per-rule provenance manifest; internal/recall verifies the pack on every
load and refuses anything off its allowlists.

internal/recall runs opengrep, gosec and bandit as subprocesses with the
scanned repository's own suppressions switched off, distinguishes an absent
tool, a failed tool and a clean run, and returns candidates with a
function-authoritative identity, a line hint, the CWE, the untrusted
snippet and full provenance. internal/laneb places every candidate on the
record as unconfirmed, carries rule provenance in tool.extensions (record
1.1.0), harvests checked-in API specs for the dynamic tier, and the scan
records the candidate count on scan_run (migration 0002).

The candidates-per-scan instrument now measures through anvil recall.
… CI job

CONTRACT.md records the additive record change (rule provenance on tool
extensions, result.rule) and how Lane B fills the frozen slots. NOTICE and
THIRD-PARTY-LICENSES.md record the vendored corpora; the opengrep packaging
choice is set out in data/LICENSES/opengrep-binary-distribution.md and left
to the owner. The README states per-language coverage. CI gains a job that
runs Lane B with the real pinned tools, with a negative control.
225 candidates on curl's full scan (was 10,714), 27 on the largest push
(was 88): under both bars. Measured through anvil recall at 861f8c1; the
row's decision is left to the owner.
Licence: a GitLab rule's licence is now read from the stricter of its own
header and its companion test file's, which names the analyser GitLab
ported it from. By the owner's decision, the 131 rules derived from
find-sec-bugs and security-code-scan (LGPL-3.0) are left out and listed in
the manifest; the 52 bandit-derived rules are Apache-2.0 under bandit's
archived licence. The pack is 177 rule files.

Security: bandit's default excludes no longer skip named files; gosec's file
coverage is counted against the Go files it should load, so another
platform's build tags, cgo and Go outside any module are incomplete
coverage; cgo is off when loading packages; a match in a symbolic link is
recorded instead of failing the scan; anvil recall on nothing is not clean.
The path exclusions stay, by the owner's decision, with their limit stated.

The opengrep distribution note now cites LGPL-2.1 section 4 and lists what
the release bundle carries. The review and its dispositions are in
docs/reviews/lane-b-gate.md; the exit gate is docs/gates/lane-b.json.
225 on curl's full scan, 27 on the largest push, under both bars; the
row's decision is left to the owner.
…self-contained

CI checks out what git records: the stand-in binaries were recorded as
100644 (this checkout has core.fileMode off), and two vendoring tests read
the local gosec archive and bandit install, which CI does not have.
candidates-per-scan is PASS (225 a full scan, 27 a push, under both bars);
the one Java rule whose companion states no licence is dropped (176 rule
files); opengrep is an operator-installed prerequisite, so no Anvil
artifact conveys its binary; the path exclusions are final for v1 with
their limit stated.
@tom-snyder
tom-snyder merged commit eca1d68 into main Oct 4, 2026
8 checks passed
@tom-snyder
tom-snyder deleted the feat/lane-b-recall branch October 4, 2026 18:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant