Skip to content

Remediation: the consumption controller, the exit gate and the triage measurement - #21

Merged
tom-snyder merged 4 commits into
mainfrom
feat/remediation
Oct 4, 2026
Merged

tom-snyder merged 4 commits into
mainfrom
feat/remediation

Conversation

@tom-snyder

Copy link
Copy Markdown
Member

Phase 7: the remediation tier. anvil remediate turns a sealed audit's findings into draft pull requests whose body is the evidence, and never merges.

What it does

  • Consumption controller (internal/remediation): enqueues every patchable finding of a sealed audit (every Lane B unconfirmed one included), then per fix group leases, triages, generates, anchors SEARCH/REPLACE edits in the exact scanned blob, applies with git apply --3way, climbs the validation ladder, runs the exploit oracle, commits with three trailers and opens a draft pull request from a fork. The lease is renewed at every step boundary; a crash is recovered from the trailers.
  • Triage gate: its verdict is its own statement in triage_verdict, never written over Lane B's evidence. It is off by default: its precision is unmeasured.
  • Generation runs in its own process (anvil generate-isolated) whose whole environment is one marker variable; only the model endpoint's socket is allowed. Endpoints have a trust tier; a public hosted one is refused unless allowed, and warns.
  • The target is untrusted: the build and tests run in bubblewrap on an export of the patched tree (no .git, no network, nothing else visible); git runs with hooks, fsmonitor and helpers off; only source files may be patched.
  • Exploit oracle: the only road to "verified fixed"; it counts only mutated payloads that are live exploits on the scanned code. Nothing implements its replayer until Phase 8, so every proposal is labelled unverified-security for now.
  • Pull requests: drafts only, at most three open per repository, supersession with a linking comment, stale closing, acceptance rate with its Wilson interval. The forge client can reach six routes and none merges; a push token that can write the upstream is refused.
  • Store migration 0003 (additive): audit_record.audit_id, triage_verdict, remediation_log, fix_pr. A repository scan now records its base commit.
  • anvil triage PATH runs the triage gate over a tree and records nothing; the triage-precision measurement drives it.

Evidence

  • The remediation exit gate: docs/gates/remediation.json, fourteen structural rows, each citing a test that was seen failing under a deliberate mutation. TestNoGateTestCanSkip refuses a skip in any cited test; two stale conditional skips in internal/handoff became failures.
  • Same-family adversarial review: docs/reviews/remediation-gate.md. Four blocking findings, all fixed and closed on the critic's re-check; eleven others fixed.
  • Register: the owner's decisions of 2026-10-04 (patch-quality bars signed, CWE-Bench-Java skipped for v1, every GPU run deferred to after Phase 9); a new pre-registered row, triage-precision, whose CPU-only sample is drawn (1,034 labelled Lane B candidates on Juliet, base rate 0.415).

Not proven here

No model has run the tier, and no real repository has been touched (an in-process forge only). Patch quality, task cards and triage precision wait for the GPU runs after Phase 9; batch size waits for real pull requests. Where a host lacks bubblewrap or unprivileged user namespaces, the build sandbox refuses to run, and its test proves the refusal rather than the confinement (docs/controls.md, U12).

…nd the coder

The triage-precision row is pre-registered: the triage gate's precision on
Lane B candidates labelled by NIST's Juliet test suites (public domain in
the US, CC0 elsewhere), measured through anvil triage. Its sample step runs
on the CPU; the model run waits for the GPU runs the owner set for after
Phase 9, with patch quality and task cards.

The register records the owner's decisions of 2026-10-04: the
patch-quality bars signed as proposed, CWE-Bench-Java skipped for v1, and
every GPU run deferred. Patch quality's corpus is referred to as a local
reproducible-vulnerability corpus; its exact pins stay outside the
repository. The coder's manifests and licence are added.
…w's fixes

The coding agent's side of Anvil, in internal/remediation: enqueue a
sealed audit, then lease, triage, rank and group, generate in an isolated
process, anchor and apply with git apply --3way, climb the validation
ladder in bubblewrap on an export of the patched tree, replay through the
exploit oracle, commit the judged tree with trailers, re-anchor, and open a
draft pull request from a fork. Nothing merges; the forge client has no
merge route. The triage gate is off by default: its precision is
unmeasured.

Store migration 0003 adds audit_record.audit_id, triage_verdict,
remediation_log and fix_pr; a repository scan records its base commit.
anvil remediate runs one cycle, anvil triage runs the triage gate over a
tree and records nothing, and anvil generate-isolated is the generation
process. Gate 3's egress allowlist gains internal/remediation (the model
endpoint and the forge only).

The remediation exit gate (docs/gates/remediation.json) cites a test for
each of its fourteen rows, each seen failing under a deliberate mutation;
no cited test may skip, and two stale conditional skips in
internal/handoff became failures. The same-family review
(docs/reviews/remediation-gate.md) found four blocking problems, all fixed
and closed on its re-check, and eleven others, all fixed.
The Go job installs bubblewrap, lifts Ubuntu's AppArmor restriction on
unprivileged user namespaces for the runner, and sets
ANVIL_SANDBOX_REQUIRED, under which TestTheBuildHasNoNetwork fails if the
sandbox is missing instead of proving only the fail-closed refusal. A
following step asserts the test passed and did not take that branch.
1,068 Juliet test-case files through anvil recall: 1,693 candidates, 1,034
labelled (429 in bad functions, 605 in good ones). The base rate a triage
gate must beat is 0.415 by the function label, 0.048 by function and CWE.
The model run waits for after Phase 9, by the owner's decision.
@tom-snyder
tom-snyder merged commit d8f9f5b into main Oct 4, 2026
8 checks passed
@tom-snyder
tom-snyder deleted the feat/remediation branch October 4, 2026 18:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant