Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -434,6 +434,65 @@ jobs:
echo "$out" | grep -q 'PROVEN repo SCA scan' || {
echo "::error::the repo SCA link is not PROVEN in a job that installs Trivy and its database"; exit 1; }

# Lane B (Phase 6) with the real, pinned recall tools. The Go job runs the
# same chain with the tools' recorded reports replayed
# (internal/recall/recalltest); this job is what proves the recording is
# faithful, and that the built binary scans the planted fixture end to end.
laneb-e2e:
name: Lane B end to end (real tools)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version-file: go.mod
cache: true
- uses: actions/setup-python@v5
with:
python-version: "3.13"

- name: Install the pinned recall tools
run: |
set -euo pipefail
# Pinned by version and digest, the values in
# eval/tools/opengrep/MANIFEST.toml and data/rules/selection.json.
OPENGREP_SHA256=40c21299eeddabf743b856daa843d24f9d4a027130671cd45b3b21776fd9ab26
curl -sSfL -o opengrep \
"https://github.com/opengrep/opengrep/releases/download/v1.26.0/opengrep_manylinux_x86"
echo "${OPENGREP_SHA256} opengrep" | sha256sum -c -
sudo install -m 0755 opengrep /usr/local/bin/opengrep
GOSEC_SHA256=6431b119741c1f4a50fdfcf94e782e16b9e642afc8c7fa9b5d39d48bf3003095
curl -sSfL -o gosec.tar.gz \
"https://github.com/securego/gosec/releases/download/v2.29.0/gosec_2.29.0_linux_amd64.tar.gz"
echo "${GOSEC_SHA256} gosec.tar.gz" | sha256sum -c -
tar -xzf gosec.tar.gz gosec
sudo install -m 0755 gosec /usr/local/bin/gosec
python -m pip install bandit==1.9.4
python -m pip install -r eval/requirements.txt
opengrep --version && gosec -version && bandit --version

- name: The opt-in is set, so a missing tool must FAIL rather than refuse quietly
env:
ANVIL_LANEB_E2E: "1"
run: go test -count=1 -v -run TestTheFixtureWithTheRealTools ./internal/recall/

- name: Lane B end to end on the fixture, through the built binary
run: sh test/e2e/laneb.sh

- name: Negative control -- the job must fail if opengrep disappears
env:
ANVIL_LANEB_E2E: "1"
run: |
set -uo pipefail
sudo mv /usr/local/bin/opengrep /usr/local/bin/opengrep.hidden
if go test -count=1 -run TestTheFixtureWithTheRealTools ./internal/recall/; then
sudo mv /usr/local/bin/opengrep.hidden /usr/local/bin/opengrep
echo "::error::The Lane B E2E test PASSED with no opengrep present. The opt-in is not fatal, so this job proves nothing."
exit 1
fi
sudo mv /usr/local/bin/opengrep.hidden /usr/local/bin/opengrep
echo "confirmed: with ANVIL_LANEB_E2E=1 and no opengrep, the test fails"

# The spine's Go control-plane decision permits Python in exactly three places, and the
# evaluation harness is one of them. eval/ is a pure-Python tree.
#
Expand Down
60 changes: 59 additions & 1 deletion NOTICE
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,53 @@ in THIRD-PARTY-LICENSES.md with its full attribution text.
github.com/mattn/go-isatty v0.0.24 MIT
github.com/ncruces/go-strftime v1.0.0 MIT

================================================================================
1a. REDISTRIBUTED IN SOURCE FORM TODAY (Lane B's vendored rules, data/rules)
================================================================================

Lane B's rule pack is committed under data/rules, the owner's selection of
2026-10-03. Every rule file is copied byte for byte, keeping its own header,
and its licence was read from that header or, where a corpus has no per-file
headers, from the corpus LICENSE body. Each licence body is archived beside the
rules and hashed in data/rules/MANIFEST.json; internal/recall refuses a pack
whose bodies or rules do not match.

GitLab sast-rules (gitlab-org/security-products/sast-rules) at
53bf5cf6df3c51b6c02110f5a638b5e6213666cd
A rule's licence is read from the stricter of its own "License:" header
and the header of its GitLab companion test file, which names the
analyser GitLab ported it from. Rules whose companion names an LGPL-3.0
analyser (find-sec-bugs, security-code-scan: 131 rules) are not vendored;
MANIFEST.json lists each.
52 rule files -- MIT, Copyright (c) 2011-present GitLab Inc. The MIT
notice is in data/rules/gitlab-sast-rules/LICENSE.
6 rule files (javascript/) whose companions read "MIT (c) JS Foundation
and other contributors, https://js.foundation" -- MIT, Copyright JS
Foundation and other contributors; the MIT permission notice is the one
in data/rules/gitlab-sast-rules/LICENSE.
27 rule files (go/), derived from gosec -- Apache-2.0, (c) gosec.
52 rule files (python/), whose companions read "Apache 2.0 (c) PyCQA",
derived from bandit -- Apache-2.0. bandit's plugin sources carry
"Copyright 2014 Hewlett-Packard Development Company, L.P." (15 files)
and other contributors' notices (read from bandit 1.9.4, 2026-10-03);
GitLab's companions credit PyCQA.
For both Apache-2.0 groups: Section 4(a), a copy of the licence is at
data/rules/gitlab-sast-rules/LICENSE.gosec-Apache-2.0.txt (gosec's
LICENSE.txt from its 2.29.0 release archive) and
LICENSE.bandit-Apache-2.0.txt (bandit 1.9.4's licence body from its
wheel). Section 4(c): the files are copied unmodified with their own
headers. Section 4(d) applies only if the Work carries a NOTICE file;
the gosec 2.29.0 release archive and the bandit 1.9.4 wheel carry none
(checked 2026-10-03). GitLab's LICENSE puts doc/ under CC BY-SA 4.0 and
keeps third-party components under their own licences, so only the six
directories csharp, go, java, javascript, python and scala are read, and
a test refuses any other.

0xdea semgrep-rules (0xdea/semgrep-rules) at
dae50da6e6e629750f1daec73a01ef9206a372ab
39 rule files, rules/c, no per-file headers -- MIT, Copyright (c) 2022
raptor. The MIT notice is in data/rules/0xdea-semgrep-rules/LICENSE.

================================================================================
2. APACHE-2.0 ARTIFACTS WITH SECTION 4(d) NOTICE DUTIES
================================================================================
Expand Down Expand Up @@ -85,6 +132,11 @@ duty is recorded now so that the packaging steps cannot ship without it.
AFL++ is excluded by the spine's exclusion list because its LICENSE is AGPL-3.0
regardless of what its README says.

gosec (securego/gosec) 2.29.0 -- Apache-2.0, read from LICENSE.txt in the
pinned release archive. bandit (PyCQA/bandit) 1.9.4 -- Apache-2.0, read
from its LICENSE body at the tag. Lane B runs both as subprocesses.
Duty: Section 4 on redistribution of either. Status: PENDING (packaging).

Docker Engine, Docker Compose -- Apache-2.0
Duty: Section 4(d) notice IF redistributed as binaries. Anvil invokes
them; it does not currently redistribute them. Status: NOT TRIGGERED.
Expand Down Expand Up @@ -198,7 +250,13 @@ documented reason, and must never appear in this file or in any Anvil artifact:
irrelevant to it.
Semgrep-maintained rules internal business use only.
opengrep/opengrep-rules archived, NOASSERTION, LGPL-2.1 + Commons Clause.
Use AikidoSec/opengrep-rules instead (MIT).
Lane B uses GitLab's sast-rules and 0xdea's
rules instead (data/rules).
GitLab sast-rules c/ every rule headed "License: GPL 2.0", generated
from flawfinder. Excluded by the owner on
2026-10-03.
lambdasec/autogrep Apache-2.0 on the repository, with GPL, AGPL
and unlicensed rules inside.
CIS Benchmark content in any form, including rules written by reading
one.
Gemma 1-3, Llama, StarCoder2,
Expand Down
78 changes: 63 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,16 @@ An open-source, **profit-free**, self-hostable system that finds vulnerabilities
code repositories and proposes fixes — using locally-served open-weight models, and never browsing the
live web at inference time.

**Status (2026-10-03): Phases 0–5 of 10 are done; Phase 6, Lane B, is next.** Phase 5's evaluation decided the
model tier: the small detection model did not earn its place (`eval/register.yaml`), so the owner shrank it to
recall rules plus an optional ranker, and the record contract is frozen at v1. `anvil scan` runs Lane A end to end: it matches a repository (through Trivy) or a host inventory
(through Anvil's own comparator) against the advisory cache, seals the SAST half, writes the record and its
findings to the store, marks each finding new, persisting, regressed or fixed against earlier scans, and writes
the record as SARIF 2.1.0. What is not built yet: Lane B's recall tier, remediation,
the dynamic tier running live (it is built but has never touched a kernel or a target), and the release packages.
**Status (2026-10-03): Phases 0–6 of 10 are built; Phase 7, remediation, is next.** Phase 5's evaluation
decided the model tier: the small detection model did not earn its place (`eval/register.yaml`), so the owner
shrank it to recall rules, and Phase 6 built Lane B that way, with no ranker (none was measured beating chance).
`anvil scan --repo` runs both lanes into one SAST half: Lane A matches dependencies through Trivy when the
operator enables its database, and Lane B runs its rule pack over the first-party source and records every
match as an `unconfirmed` finding for the coding agent's triage gate. `anvil scan --host` matches a host
inventory through Anvil's own comparator. Every scan seals the SAST half, writes the record and its findings to
the store, marks each finding new, persisting, regressed or fixed against earlier scans, and writes the record
as SARIF 2.1.0. What is not built yet: remediation and its triage gate, the dynamic tier running live (it is
built but has never touched a kernel or a target), and the release packages.
A fresh clone admits no real advisory feed until an operator acquires and certifies the publishers' licence
texts (`mirror/README.md`); `testdata/lanea-fixture` is an offline snapshot that proves the chain without one.

Expand All @@ -26,8 +29,18 @@ export ANVIL_CONFIG=$PWD/anvil.yml
./anvil findings
```

`sh test/e2e/fixture.sh` runs the same sequence, plus a repository scan, and asserts every exit status. Exit
statuses: 0 clean, 1 findings, 2 usage, 3 refused, 4 missing tool, 5 error; a refusal or a missing tool is never 0.
With opengrep 1.26.0, gosec 2.29.0 and bandit 1.9.4 on `PATH`, Lane B scans its planted fixture (12 unconfirmed
findings):

```bash
printf 'version: 1\nstateDir: state\nrecall:\n rules: data/rules\n' > anvil.yml
./anvil scan --repo testdata/laneb-fixture --target laneb-fixture --out repo.sarif
./anvil recall testdata/laneb-fixture # the same candidates as JSON, nothing recorded
```

`sh test/e2e/fixture.sh` runs the Lane A sequence, plus a repository scan, and `sh test/e2e/laneb.sh` the Lane B
one; both assert every exit status. Exit statuses: 0 clean, 1 findings, 2 usage, 3 refused, 4 missing tool,
5 error; a refusal or a missing tool is never 0.

## What it does

Expand All @@ -36,15 +49,46 @@ Two detection lanes, one audit record, and a remediation tier that proposes and
- **Lane A — deterministic, zero inference.** SBOM and host-package matching by version comparator.
Owns dependency and host findings. CVE/OSV/GHSA describe vulnerable *package versions*, and a version
comparator answers that exactly, for free.
- **Lane B — first-party source.** A deterministic recall tier (opengrep over GitLab's sast-rules and C/C++
rules, gosec, bandit) produces candidates, each placed on the record as `unconfirmed` for the coding agent's
triage gate to decide. The evaluation of 2026-10-03 deleted the planned small-model adjudicator: the primary
candidate flipped its verdict on a wrong advisory only about half the time and ranked vulnerable against patched
code no better than chance. A ranker ships only if it measurably beats chance on Anvil's own candidates.
- **Lane B — first-party source.** A deterministic recall tier (opengrep over GitLab's sast-rules and 0xdea's
C/C++ rules, plus gosec and bandit) produces candidates, each placed on the record as `unconfirmed` for the
coding agent's triage gate to decide. The evaluation of 2026-10-03 deleted the planned small-model
adjudicator: the primary candidate flipped its verdict on a wrong advisory only about half the time and ranked
vulnerable against patched code no better than chance. No ranker ships: none has been measured beating chance
on Anvil's own candidates, and none of them are labelled.
- **Dynamic tier.** Ships as a **separate artifact** (`anvil-dast`), separately installed, requiring
explicit attestation before it probes anything.
- **Remediation.** Proposes patches. It does not merge them.

## What Lane B covers

Lane B runs only the rules in `data/rules`, the owner's selection of 2026-10-03, pinned by commit and hashed
file by file, plus gosec and bandit. **No permissive rule corpus gives broad multi-language taint (dataflow)
recall**: 16 of the 176 rule files are taint rules, and Java, Scala and C# are thin:

| Language | Rule files | Of them taint rules | Native analyser |
|---|---|---|---|
| C and C++ | 39 (0xdea, MIT) | 0 | none |
| Go | 27 (GitLab, Apache-2.0, derived from gosec) | 5 | gosec 2.29.0 (SSA, type-checked) |
| Python | 67 (GitLab: 52 Apache-2.0, derived from bandit; 15 MIT) | 1 | bandit 1.9.4 (AST) |
| Java | 12 (GitLab, MIT) | 4 | none |
| Scala | 19 (GitLab, MIT) | 6 | none |
| C# | 1 (GitLab, MIT) | 0 | none |
| JavaScript and TypeScript | 11 (GitLab, MIT) | 0 | none |

Anything else (Ruby, PHP, Kotlin, Rust, Swift, shell, …) is not covered at all. Two kinds of GitLab rules are
excluded for their licences: the C rules, each headed "License: GPL 2.0" because it is generated from flawfinder,
and 131 Java, Scala and C# rules whose GitLab companion test files name find-sec-bugs or security-code-scan,
both LGPL-3.0, as their source (a rule's licence is read from the stricter of its own header and its
companion's), plus one Java rule whose companion states no licence at all. Test, test-data,
documentation and example trees are not reported on. **Those exclusions match by name, so a repository can place
first-party code under `spec/`, `docs/` or `fixtures/` and Lane B will not see it**; every scan reports how many
source files they kept out. Otherwise the scanned repository cannot switch a rule off: its `nosemgrep`, `#nosec`
and `# nosec` comments, `.semgrepignore` and `.bandit` files, and bandit's and opengrep's own default excludes are
all ignored, and anything a tool was given and did not analyse (another platform's Go build tags, cgo, a rule
timeout) is recorded as incomplete coverage. Measured on
2026-10-03 on five sample repositories, the selection produces 0 to 225 candidates a full scan (curl is the most;
`eval/results/candidates-per-scan.json`), under the budget of 500. Every scan records its count.

## Three rules that are enforced in code, not documentation

1. **Never auto-merge a patch.** The best measured security-patch rate on real CVEs is 34.0%.
Expand All @@ -56,7 +100,7 @@ Two detection lanes, one audit record, and a remediation tier that proposes and

| Artifact | Contains |
|---|---|
| `anvil` | Lane A, Lane B, record, store, remediation. **No network-probing capability compiled in.** |
| `anvil` | Lane A, Lane B, record, store, remediation. **No network-probing capability compiled in.** Lane B's tools run as separate processes; the operator installs them (opengrep 1.26.0, gosec 2.29.0, bandit 1.9.4), and Anvil checks their versions. |
| `anvil-dast` | The dynamic tier. Separate release, separate install, explicit attestation. |

This is a split in the build, not a configuration flag, because a boolean inside a single shipped
Expand Down Expand Up @@ -90,6 +134,10 @@ Advisory data reaches the cache only through the licence gate: a feed is admitte
text has been acquired, read, and pinned by digest in `mirror/LICENSE-MANIFEST.toml`, and share-alike sources are
quarantined in tier 2.

**Lane B is on by default** (`recall:` in the configuration names the rule pack and the tools; `enabled: false`
turns it off). Its rules are MIT and Apache-2.0, and a repository scan whose rule pack or tools are missing exits 4,
never 0. A scan with a lane turned off and nothing found exits 3.

**The Trivy database is off by default** (the owner's decision of 2026-10-03). Trivy decides repository
dependencies against its own database, which aggregates share-alike sources (Ubuntu, Alpine) that tier 2
quarantines, and whose publishers state no redistribution terms. Repository SCA therefore runs only when the
Expand Down
Loading
Loading