Contract gaps: trust validation step, licence note, zstd payload, handoff fixes

THIRD-PARTY-LICENSES.md

@@ -22,7 +22,7 @@ go list -m -f '{{.Dir}}' modernc.org/sqlite | xargs -I{} sha256sum {}/LICENSE
 
 ## Go module dependencies — compiled into `anvil` and `anvil-dast`
 
-All ten are BSD-3-Clause or MIT. **No copyleft, no unclear licence, no `NOASSERTION` in the set.** All
+All eleven are BSD-3-Clause or MIT. **No copyleft, no unclear licence, no `NOASSERTION` in the set.** All
 are Apache-2.0 compatible in the inbound direction.
 
 | Module | Version | SPDX | Copyright holder | LICENSE file SHA-256 |
@@ -37,9 +37,12 @@ are Apache-2.0 compatible in the inbound direction.
 | `github.com/dustin/go-humanize` | v1.0.1 | MIT | Dustin Sallings (2005–2008) | `a973b4498c13eb74baa2a8e5c351426a6826f2fcdd909916dbe53ee2e755fd71` |
 | `github.com/mattn/go-isatty` | v0.0.24 | MIT | Yasuhiro Matsumoto | `08eab1118c80885fa1fa6a6dd7303f65a379fcb3733e063d20d1bbc2c76e6fa1` |
 | `github.com/ncruces/go-strftime` | v1.0.0 | MIT | Nuno Cruces (2022) | `38ae43959daf953a393a585b2988672cb65a5a541aca0d0be5e72595a0a16883` |
+| `github.com/klauspost/compress` | v1.20.1 | BSD-3-Clause; MIT for `zstd/internal/xxhash` | The Go Authors (2012), Klaus Post (2019); Caleb Spare (2016) for xxhash; The Snappy-Go Authors (2011) for `internal/snapref` | `0d9e582ee4bff57bf1189c9e514e6da7ce277f9cd3bc2d488b22fbb39a6d87cf` (all three bodies in `third_party/klauspost-compress/`) |
 
-Only `modernc.org/sqlite` is a direct requirement; the other nine arrive through it. All nine are
-therefore pinned by `go.sum` and move only when the sqlite pin moves.
+`modernc.org/sqlite` and `github.com/klauspost/compress` are the direct requirements; the other nine
+arrive through sqlite, are pinned by `go.sum`, and move only when the sqlite pin moves. The compress module
+is the audit payload's zstd codec, added 2026-10-03; `third_party/klauspost-compress/PIN.md` records the
+three licence bodies that apply to the packages it links, and a test re-hashes them.
 
 ### BSD-3-Clause — required notice
 

go.mod

@@ -2,7 +2,10 @@ module github.com/Susquehanna-Syntax/Anvil
 
 go 1.26.5
 
-require modernc.org/sqlite v1.56.0
+require (
+	github.com/klauspost/compress v1.20.1
+	modernc.org/sqlite v1.56.0
+)
 
 require (
 	github.com/dustin/go-humanize v1.0.1 // indirect

go.sum

@@ -6,6 +6,8 @@ github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
 github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
 github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
 github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
+github.com/klauspost/compress v1.20.1 h1:T7kKElXUMXrUJ2E9QhQhxFtcK5rPyLdsGZvdbLMPdiQ=
+github.com/klauspost/compress v1.20.1/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
 github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsReI=
 github.com/mattn/go-isatty v0.0.24/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A=
 github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=

internal/dast/authz/egress_chokepoint_test.go

@@ -199,6 +199,10 @@ var inertImports = map[string]string{
 	"unicode/utf8":         "rune encoding",
 	"modernc.org/sqlite":   "the pure-Go, file-backed SQLite driver the spine's Go control-plane decision mandates; no network protocol",
 	"golang.org/x/sys/cpu": "CPU feature detection",
+	// Checked 2026-10-03: `go list -deps` of this package names no net,
+	// net/*, crypto/tls or os/exec package; it compresses byte slices it is
+	// handed (internal/store's audit payload codec).
+	"github.com/klauspost/compress/zstd": "the pinned pure-Go zstd codec for audit_record.payload; byte slices in, byte slices out",
 }
 
 // inertSymbols is the single, narrow escape hatch: a package that is NOT inert

internal/handoff/claim.go

@@ -425,7 +425,7 @@ func (q *Queue) handleFor(ctx context.Context, handoffID int64, leaseToken strin
 		IdempotencyKey:   idemKey.String,
 		leaseToken:       leaseToken,
 	}
-	if path, err := q.PacketPath(r.Fingerprint); err == nil {
+	if path, err := q.packetPath(r.Fingerprint); err == nil {
 		h.PacketPath = path
 	}
 	return h, nil
@@ -530,6 +530,11 @@ func (q *Queue) RenewLeaseContext(ctx context.Context, h Handle) (Handle, error)
 // work is untouched. That is the mechanism by which reclaiming and
 // re-processing is idempotent: not a lock, a compare-and-swap on the exact
 // lease.
+//
+// Releasing to 'ready' here KEEPS the attempt spent: it is how an attempt that
+// ran and failed goes back to the ready set (scanctl's ConsumeOne does exactly
+// that), and `attempts` counts attempts started. A holder that is giving the
+// finding back WITHOUT having attempted it calls HandBack instead.
 func (q *Queue) ReleaseLease(h Handle, to record.HandoffState) error {
 	return q.ReleaseLeaseContext(context.Background(), h, to)
 }
@@ -574,6 +579,50 @@ func (q *Queue) ReleaseLeaseContext(ctx context.Context, h Handle, to record.Han
 	return nil
 }
 
+// HandBack ends a lease WITHOUT an attempt: the holder gives the finding back to
+// the ready set and the attempt its claim counted is returned.
+//
+// `attempts` counts attempts STARTED, so the claim increments it, and a crash
+// or a failed attempt keeps it spent. A holder that declines the work before
+// attempting it (it is shutting down, it is the wrong worker for this finding)
+// has not started an attempt. Before HandBack existed the only way out was
+// ReleaseLease(ready), which kept the attempt, so two hand-backs with
+// max_attempts=2 stranded a row that had never been attempted as
+// ready-but-exhausted, forever (the sealing, claims and masking review's
+// finding F10).
+//
+// The cost is stated rather than hidden: a holder that hands back on every
+// lease never exhausts the row. A consumer that cannot make progress on a
+// finding must release it with a disposition, not hand it back.
+func (q *Queue) HandBack(h Handle) error {
+	return q.HandBackContext(context.Background(), h)
+}
+
+// HandBackContext is HandBack with a caller-supplied context.
+func (q *Queue) HandBackContext(ctx context.Context, h Handle) error {
+	if err := q.checkRecordVersion(ctx, h); err != nil {
+		return err
+	}
+	res, err := q.db.ExecContext(ctx,
+		`UPDATE handoff
+		    SET state = ?, claimed_by = NULL, lease_expires_at = NULL, updated_at = ?,
+		        attempts = CASE WHEN attempts > 0 THEN attempts - 1 ELSE 0 END
+		  WHERE handoff_id = ? AND state = ? AND claimed_by = ? AND lease_expires_at = ?`,
+		string(record.HandoffStateReady), formatTime(q.Now()),
+		h.HandoffID, string(record.HandoffStateLeased), h.WorkerID, h.leaseToken)
+	if err != nil {
+		return fmt.Errorf("handoff: handing back row %d: %w", h.HandoffID, err)
+	}
+	n, err := res.RowsAffected()
+	if err != nil {
+		return fmt.Errorf("handoff: handing back row %d: %w", h.HandoffID, err)
+	}
+	if n == 0 {
+		return q.explainLeaseLost(ctx, h)
+	}
+	return nil
+}
+
 // HasDynamicEvidence reports whether a DAST half that ended in this state can
 // have produced a reproduction of a finding.
 //
@@ -695,10 +744,15 @@ func (q *Queue) explainLeaseLost(ctx context.Context, h Handle) error {
 // directory at all.
 var ErrNoPacketDir = errors.New("handoff: no PacketDir is configured")
 
-// PacketPath returns where a fingerprint's packet lives. The fingerprint is
+// packetPath returns where a fingerprint's packet lives. The fingerprint is
 // validated as 64 hex first, which is also what stops a crafted value from
 // escaping PacketDir.
-func (q *Queue) PacketPath(fingerprint string) (string, error) {
+//
+// It is unexported on purpose. Exported, it handed any caller the file for any
+// fingerprint, and the file is the payload: a path is a read around
+// packetGate. The lease holder still gets its own packet's path on
+// Handle.PacketPath, which is the one caller entitled to it.
+func (q *Queue) packetPath(fingerprint string) (string, error) {
 	if err := ValidateFingerprint(fingerprint); err != nil {
 		return "", err
 	}
@@ -813,7 +867,7 @@ func (q *Queue) WritePacketContext(ctx context.Context, h Handle, data []byte) (
 		return "", err
 	}
 	fingerprint := h.Fingerprint
-	final, err := q.PacketPath(fingerprint)
+	final, err := q.packetPath(fingerprint)
 	if err != nil {
 		return "", err
 	}
@@ -874,7 +928,7 @@ func (q *Queue) ReadPacketContext(ctx context.Context, h Handle) ([]byte, error)
 	if err := q.packetGate(ctx, h); err != nil {
 		return nil, err
 	}
-	path, err := q.PacketPath(h.Fingerprint)
+	path, err := q.packetPath(h.Fingerprint)
 	if err != nil {
 		return nil, err
 	}
@@ -900,7 +954,7 @@ func (q *Queue) ReadPacketContext(ctx context.Context, h Handle) ([]byte, error)
 // Dropping a packet that is not there is success, because the packet is a
 // cache and its absence is the desired state.
 func (q *Queue) DropPacket(fingerprint string) error {
-	path, err := q.PacketPath(fingerprint)
+	path, err := q.packetPath(fingerprint)
 	if errors.Is(err, ErrNoPacketDir) {
 		return nil
 	}

internal/handoff/handoff_test.go

@@ -10,6 +10,7 @@ import (
 	"go/token"
 	"os"
 	"path/filepath"
+	"reflect"
 	"strings"
 	"sync"
 	"testing"
@@ -247,9 +248,9 @@ func (f *fixture) state(handoffID int64) record.HandoffState {
 // would test the gate instead of the sweep.
 func (f *fixture) packetExists(fingerprint string) bool {
 	f.t.Helper()
-	path, err := f.q.PacketPath(fingerprint)
+	path, err := f.q.packetPath(fingerprint)
 	if err != nil {
-		f.t.Fatalf("PacketPath: %v", err)
+		f.t.Fatalf("packetPath: %v", err)
 	}
 	_, err = os.Stat(path)
 	if err != nil && !errors.Is(err, os.ErrNotExist) {
@@ -1264,8 +1265,8 @@ func TestPacketPathRejectsTraversal(t *testing.T) {
 		"",
 		strings.Repeat("z", 64),
 	} {
-		if _, err := f.q.PacketPath(bad); err == nil {
-			t.Errorf("PacketPath(%q) was accepted", bad)
+		if _, err := f.q.packetPath(bad); err == nil {
+			t.Errorf("packetPath(%q) was accepted", bad)
 		}
 	}
 }
@@ -1278,8 +1279,8 @@ func TestPacketDirIsOptional(t *testing.T) {
 	if err != nil {
 		t.Fatalf("New: %v", err)
 	}
-	if _, err := q.PacketPath(fp(16)); !errors.Is(err, ErrNoPacketDir) {
-		t.Errorf("PacketPath without a dir: err = %v, want ErrNoPacketDir", err)
+	if _, err := q.packetPath(fp(16)); !errors.Is(err, ErrNoPacketDir) {
+		t.Errorf("packetPath without a dir: err = %v, want ErrNoPacketDir", err)
 	}
 	if err := q.DropPacket(fp(16)); err != nil {
 		t.Errorf("DropPacket without a dir: %v", err)
@@ -1995,3 +1996,82 @@ func TestHasDynamicEvidenceClassifiesEveryDastStatus(t *testing.T) {
 			len(want), len(record.DastStatusValues()))
 	}
 }
+
+// TestAVoluntaryHandBackReturnsItsAttempt is the sealing, claims and masking
+// review's finding F10. With max_attempts=2, two hand-backs used to leave the
+// row ready with attempts=2/2, so the third claim was ErrExhausted although
+// nothing had been attempted. HandBack returns the attempt; ReleaseLease(ready),
+// which is how a FAILED attempt goes back, keeps it spent.
+func TestAVoluntaryHandBackReturnsItsAttempt(t *testing.T) {
+	f := newFixture(t, Options{MaxAttempts: 2})
+	audit := f.sealedAudit()
+	fingerprint, row := f.enqueue(40, record.ConsumptionClassStaticOnly, audit)
+
+	for i := 1; i <= 3; i++ {
+		h, err := f.q.Claim(fingerprint, "worker-a")
+		if err != nil {
+			t.Fatalf("claim %d after %d hand-backs: %v", i, i-1, err)
+		}
+		if h.Attempt != 1 {
+			t.Errorf("claim %d: Attempt = %d, want 1 (hand-backs must not count)", i, h.Attempt)
+		}
+		if err := f.q.HandBack(h); err != nil {
+			t.Fatalf("hand-back %d: %v", i, err)
+		}
+	}
+	got, err := f.q.Get(row.HandoffID)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if got.Attempts != 0 || got.State != record.HandoffStateReady || got.ClaimedBy != "" {
+		t.Fatalf("after three hand-backs: state=%q attempts=%d claimed_by=%q, want ready, 0, none",
+			got.State, got.Attempts, got.ClaimedBy)
+	}
+
+	// A failed attempt released to 'ready' keeps the attempt it used.
+	h, err := f.q.Claim(fingerprint, "worker-b")
+	if err != nil {
+		t.Fatal(err)
+	}
+	if err := f.q.ReleaseLease(h, record.HandoffStateReady); err != nil {
+		t.Fatal(err)
+	}
+	if got, _ := f.q.Get(row.HandoffID); got.Attempts != 1 {
+		t.Errorf("ReleaseLease(ready) returned the attempt: attempts=%d, want 1", got.Attempts)
+	}
+
+	// A hand-back from a lease that was reclaimed is refused, like any stale holder.
+	if err := f.q.HandBack(h); !errors.Is(err, ErrLeaseLost) {
+		t.Errorf("HandBack on a released lease: err = %v, want ErrLeaseLost", err)
+	}
+}
+
+// TestDisposeRefusesExpired is the sealing, claims and masking review's
+// finding F13: 'expired' belongs to the claim-timeout reaper, and Dispose used
+// to let any caller set it on a row whose deadline had not passed.
+func TestDisposeRefusesExpired(t *testing.T) {
+	f := newFixture(t, Options{})
+	audit := f.sealedAudit()
+	_, row := f.enqueue(41, record.ConsumptionClassStaticOnly, audit)
+	if err := f.q.Dispose(row.HandoffID, record.HandoffStateExpired); !errors.Is(err, ErrIllegalTransition) {
+		t.Fatalf("Dispose(expired): err = %v, want ErrIllegalTransition", err)
+	}
+	if got := f.state(row.HandoffID); got != record.HandoffStateReady {
+		t.Errorf("state = %q after a refused Dispose, want ready", got)
+	}
+}
+
+// TestNoExportedPathToAnArbitraryPacket: an exported function that maps any
+// fingerprint to its packet file is a read around packetGate, because the file
+// is the payload. Only the lease holder's Handle carries a path.
+func TestNoExportedPathToAnArbitraryPacket(t *testing.T) {
+	typ := reflect.TypeOf(&Queue{})
+	for i := 0; i < typ.NumMethod(); i++ {
+		m := typ.Method(i)
+		if m.Type.NumIn() == 2 && m.Type.In(1).Kind() == reflect.String &&
+			m.Type.NumOut() == 2 && m.Type.Out(0).Kind() == reflect.String {
+			t.Errorf("(*Queue).%s maps a string to a string; if that string is a packet path, it "+
+				"bypasses packetGate", m.Name)
+		}
+	}
+}

internal/handoff/reaper.go

@@ -345,7 +345,7 @@ func (q *Queue) ExpireClaimTimeoutsContext(ctx context.Context) (ReapReport, err
 // dropPacketIfPresent unlinks a packet and reports whether one was there. A
 // Queue with no PacketDir has nothing to drop, which is not an error.
 func (q *Queue) dropPacketIfPresent(fingerprint string) (bool, error) {
-	path, err := q.PacketPath(fingerprint)
+	path, err := q.packetPath(fingerprint)
 	if errors.Is(err, ErrNoPacketDir) {
 		return false, nil
 	}

internal/handoff/state_machine.go

@@ -629,6 +629,12 @@ func (q *Queue) EnqueueContext(ctx context.Context, req EnqueueRequest) (Row, er
 // A leased row is not disposable this way — ReleaseLease is the only exit from
 // 'leased', because only the lease holder may decide the outcome of its own
 // attempt.
+//
+// 'expired' is not disposable this way either. It is the claim-timeout
+// terminal state, and the reaper that owns that clock (ExpireClaimTimeouts) is
+// its only writer. Through Dispose any caller could expire a finding whose
+// deadline had not passed (the sealing, claims and masking review's finding
+// F13).
 func (q *Queue) Dispose(handoffID int64, to record.HandoffState) error {
 	return q.DisposeContext(context.Background(), handoffID, to)
 }
@@ -638,7 +644,7 @@ func (q *Queue) DisposeContext(ctx context.Context, handoffID int64, to record.H
 	if err := CheckTransition(record.HandoffStateReady, to); err != nil {
 		return err
 	}
-	if to == record.HandoffStateLeased {
+	if to == record.HandoffStateLeased || to == record.HandoffStateExpired {
 		return &TransitionError{From: record.HandoffStateReady, To: to}
 	}
 	res, err := q.db.ExecContext(ctx,

internal/record/CONTRACT.md

@@ -658,3 +658,21 @@ skipped: `SpecHarvest{Outcome: SpecHarvestSkipped}` marshals `files` as `null`,
 the schema refused while the Go validator accepted. The `harvest_skipped` branch now admits `null` and
 `[]` alike (they mean the same thing when there is no list to have been written down), and the
 `harvest_ran` branch narrows back to an array, which is where the distinction is load-bearing.
+
+---
+
+## Amendment 2026-10-03 — the contract gaps closed before the freeze (plan node contractgaps)
+
+Six open items were carried into Phase 4. Each now either lands with a test or is stated here as a
+deliberate limit.
+
+| Item | Outcome | Where it is held |
+|---|---|---|
+| No slot for the licence manual-override note | **Closed.** The slot (`anvil/advisory.licenseManualNote`) was added on 2026-08-23, but record emission still carried the note out of band on `Emission`. Emission now writes it into the record, beside the excerpt it licenses, and `Emission` no longer has the field. | `internal/record/lanea/emit_test.go` (the seven Lane A fields, read from the serialised record) |
+| No trust-validation step | **Closed.** `anvil/trust.validationStep` names the validation behind every `verified` label in a result — `default`, any `fields` entry, and the inline trust of an advisory excerpt, a licence note or a reproduction span. It is required, non-blank, exactly when some label is `verified`, and refused when none is. Emission fills it only when a `verified` string travels in the result. | `ValidateResultTrust`; `TestVerifiedNamesItsValidationStep`; the wire schema's `trustAssertion` (`if default is verified then validationStep is required`, checked with `jsonschema` 4.26.0 on 2026-10-03) |
+| An unpinned payload codec | **Closed.** `audit_record.payload` is zstd from `github.com/klauspost/compress` v1.20.1 (pure Go, BSD-3-Clause), pinned in `go.sum`; its three applicable licence bodies are archived and re-hashed (`third_party/klauspost-compress/PIN.md`). `payload_sha256` is the SHA-256 of the **canonical JSON**, not of the compressed bytes, so it identifies the record whatever the encoder. Decoding refuses a payload over 256 MiB decompressed, a corrupt frame, and bytes whose hash is not the row's. | `internal/store/payload.go`, `payload_test.go` |
+| A voluntary lease hand-back burned an attempt | **Closed.** `Queue.HandBack` ends a lease without an attempt and returns the one the claim counted. `ReleaseLease(ready)` still keeps it, because that is how a failed attempt goes back (scanctl's `ConsumeOne`). Stated limit: a holder that hands back on every lease never exhausts the row. | `TestAVoluntaryHandBackReturnsItsAttempt` |
+| `expired` settable outside the reaper | **Closed.** `Dispose` refuses `expired`; `ExpireClaimTimeouts` is its only writer. | `TestDisposeRefusesExpired` |
+| An exported, ungated packet path | **Closed.** `ReadPacket` and `WritePacket` were already gated; `Queue.PacketPath(fingerprint)` still mapped any fingerprint to its file, which is a read around the gate. It is unexported. A lease holder still receives its own packet's path on the `Handle`. `DropPacket` stays exported: it only removes a regenerable cache. | `TestNoExportedPathToAnArbitraryPacket` |
+
+Every new check was seen failing with its fix reverted before it was trusted.