feat: add signed run receipts with crabbox run --attest and crabbox verify#996
Conversation
|
Codex review: needs real behavior proof before merge. Reviewed July 9, 2026, 2:12 AM ET / 06:12 UTC. Summary Reproducibility: not applicable. as a bug reproduction; this is a feature PR. The capability can be exercised with Review metrics: 3 noteworthy metrics.
Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Rank-up moves:
Proof guidance:
Risk before merge
Maintainer options:
Next step before merge
Maintainer decision needed
Security Review detailsBest possible solution: Land only after maintainers accept the documented self-signed integrity-only v1 and current-head terminal proof shows Do we have a high-confidence way to reproduce the issue? Not applicable as a bug reproduction; this is a feature PR. The capability can be exercised with Is this the best way to solve the issue? Unclear until maintainer security intent is confirmed. The implementation is provider-neutral and well covered by tests, but the self-signed verifier semantics are a product/security boundary. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against 5d9080e21b15. Label changesLabel changes:
Label justifications:
Evidence reviewedWhat I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
Review history (9 earlier review cycles; latest 8 shown)
|
d7eea6b to
83070b1
Compare
|
All four review items are addressed in 83070b1d and the PR body proof is refreshed against that head.
The v1 trust boundary question (self-signed, no trust roots, no transparency log) is stated in the body as a deliberate v2 boundary and stays with maintainers. |
|
@clawsweeper re-run |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
The digest race is fixed in b7ada56f and the body proof is refreshed against that head.
|
|
@clawsweeper re-run |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
@clawsweeper re-review Both remaining findings are addressed and the body proof is regenerated against head 7ec4e38.
Also folded in since the last review: strict receipt schema validation before any signature decision (unknown fields, missing required fields, wrong types, non-integer spellings, unsupported schema versions, invalid timestamps and digests all exit 2), with the unknown-field rejection shown live in the proof block. |
|
🦞🧹 I asked ClawSweeper to review this item again. |
…erify crabbox run --attest <path> writes an Ed25519 signed receipt of a successful run: provider, lease, command, exit code, timing, and the SHA-256 of the combined live output stream, in canonical JSON. The new crabbox verify command recomputes the canonical bytes, checks the signature, and prints PASS or FAIL with the signer key fingerprint. The signing key is minted per user on first use under the user config dir; --attest-key signs with an existing PKCS8 PEM ed25519 key and fails closed. Receipts stay provider neutral in core: both the direct SSH run path and the delegated run path write them, with output digests only where core observes the full stream.
Reject receipts whose JSON contains duplicate object keys before signature verification, since parsers disagree on which duplicate wins and an edited receipt could show one value while the signature checks another. Preflight the --attest path against lease output, emit proof, capture stdout and stderr, and download destinations. Forbid --attest and --attest-key in watch and shard forwarding, matching --emit-proof. Drop the CHANGELOG edit; release notes are release owned.
The receipt digest hash was shared by the stdout and stderr writers, which write from separate goroutines during a run, so concurrent writes could race and corrupt log_sha256 before signing. Wrap the hash in a mutex-guarded writer, matching the synchronized run log buffer those streams already share, and cover mixed concurrent stream writes plus arrival-order hashing with regressions that run under the race detector.
Writing a receipt to a nested path failed because the private output writer expects the parent directory to exist. Create the receipt directory with the same private mode used by proof output before writing, matching writeRunProof, and cover a nested --attest path with a regression that also checks the created directory mode.
Delegated providers can return successful run metadata without a LeaseID, but receipts wrote lease_id unconditionally and verification required it as a non-empty string, so such a receipt could never verify. Treat lease_id like the other run-dependent fields: omit it when empty, validate it only when present, and build delegated receipts through a dedicated helper mirroring the delegated proof writer. Add a regression covering a delegated result with no lease id that round trips to a passing verification with the empty fields omitted.
7ec4e38 to
c28da3c
Compare
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
@clawsweeper re-review Exact current-head behavior proof is now in the PR body for |
|
Maintainer decision: accept the documented opt-in, self-signed v1 boundary as integrity evidence only. |
Maintainer review update
Exact reviewed head:
c28da3ce2718b03c6cdbceb0e698e1e12250e154, rebased onto currentmain.--attest-keywithout--attestand validated override keys before any remote execution.trust=self-signed; aPASSis meaningful only after matching the fingerprint through a trusted channel.Unreleasedchangelog entry with contributor credit.Exact-head verification:
go test -race ./...,go vet ./...,go build -trimpath -o /tmp/crabbox-pr996-current ./cmd/crabbox, andnode scripts/check-command-docs.mjsall pass. Four autoreview rounds addressed every accepted finding; the final review is clean with no actionable findings.The maintainer exact-head proof below exercises the final trust marker and path hardening over real loopback SSH. The longer proof later in the body was captured by the contributor at head
7ec4e381and covers tamper/schema rejection.Summary
Crabbox is built around run evidence: proof blocks (
--emit-proof), content-addressed capsule manifests, timing JSON, JUnit summaries, artifact manifests with SHA-256 digests. None of that evidence is cryptographically signed, and there is no command to check any of it after the fact. A reviewer or an agent overseer has to trust pasted output.This change adds the minimal provider-neutral primitive on top of what runs already collect:
crabbox run --attest <path>writesreceipt.jsonafter a successful run: a flat JSON record of provider, lease, command, exit code, timing, and the SHA-256 of the combined stdout and stderr stream as observed by the client, signed with a per-user Ed25519 key.crabbox verify <receipt.json>recomputes the canonical bytes, checks the signature, and prints PASS or FAIL with the signer public key fingerprint andtrust=self-signed. Exit 0 on PASS, 1 on signature mismatch, 2 for receipts that cannot be checked at all.<user config dir>/crabbox/attest/id_ed25519.pem(PKCS8 PEM, 0600, same mint-once pattern as the per-lease SSH keys).--attest-key <path>signs with an existing PKCS8 PEM Ed25519 key and fails closed; it never creates a key at an override path.Everything uses the Go standard library (
crypto/ed25519,crypto/x509,encoding/pem); no new dependencies.Design notes
signatureitself: the object minussignatureis re-marshaled as compact JSON, which sorts map keys lexicographically. Becausepublic_keyis inside the signed payload, swapping in a different key also fails verification. Empty optional fields are omitted, never emitted as empty strings, so signer and verifier agree without per-field logic.--capture-stdout. It deliberately does not hash the internal log ring buffer, which truncates at 8 MiB. Because the stdout and stderr streams write from separate goroutines, the digest is wrapped in a mutex-guarded writer, matching the synchronized run log buffer those streams already share, solog_sha256is always the hash of the bytes in arrival order and can never be corrupted by concurrent writes.run_idandlog_sha256; the delegated path preserves available session identity fields but omitslog_sha256because a delegatedRunResultcarries only a bounded log excerpt, and hashing an excerpt would be misleading evidence. Dispatch stays by feature, not provider name, perdocs/vision.md.--attestmirrors--emit-proof: path-valued, success-only, rejected with--sync-only, and preflighted with the same local output path checks.verifyrejects receipts whose JSON contains duplicate object keys anywhere (exit 2), because JSON parsers disagree on which duplicate wins and an edited receipt could show one value to a reader while the signature still verifies against another; beyond that, receipts are strictly schema validated before any signature decision: unknown fields, missing required fields, wrong types, non-integer number spellings, unsupported schema versions, invalid timestamps, and malformed digests are all rejected as malformed (exit 2), soPASScan only ever be printed for a well-formed, integrity-checked record. The receipt and effective signing-key paths are preflighted against every writable local output with filesystem-aware alias detection;--attest/--attest-keyare forbidden inwatchandshardforwarding, matching--emit-proof; missing parent directories use private modes; concurrent first use publishes exactly one stable key; and delegated receipts preserve every available session identity while omitting unavailable fields. The contributor changelog edit was dropped and a maintainer-owned entry was added during landing review.Config and secret implications
<user config dir>/crabbox/attest/id_ed25519.pem, created 0600 in a 0700 directory through atomic no-replace publication. It is a signing identity, not a provider credential, and never leaves the machine.--attest-keytakes a file path, never key material.Verification
gofmt -l $(git ls-files '*.go')cleango vet ./...cleango run golang.org/x/tools/cmd/deadcode@v0.45.0 -test ./...emptygo test -race ./...all packages passgo build -trimpath -o bin/crabbox ./cmd/crabboxbuildsnode scripts/check-command-docs.mjspasses (54 command docs, includes the newverifypage andcrabbox verify --helpexit 0)scripts/check-go-coverage.sh 90.0passesinternal/cli/attest_test.go: receipt round trip through the realverifycommand dispatch, empty-optional-field omission, a tamper table (mutated exit code, mutated command, foreign public key, corrupt signature encoding, deleted signature, duplicateexit_codekey with the signed value last, duplicatecommandkey with the signed value last, truncated JSON, missing file) asserting exit codes 1 vs 2, receipt path collision preflights against capture and download destinations, key mint idempotence with 0600/0700 permission checks, and--attest-keyoverride semantics including fail-closed on a missing path and rejection of non-Ed25519 keys.watchandshardforbidden-flag suites cover--attestand--attest-keyrejection. Digest-writer regressions run under the race detector: concurrent mixed stdout/stderr writes through the shared digest produce the exact expected hash with no data race (an unsynchronized hash in the same shape fails-raceimmediately), and a sequential mixed-stream case pins arrival-order hashing. Schema-validation regressions cover unsupported schema versions, unknown fields, decimal integer spellings, and invalid digests; a nested--attestpath regression checks created directory modes; and a delegated-receipt regression covers a provider returning successful run metadata with no LeaseID, asserting the empty fields are omitted and the receipt still verifies.Exact-head real behavior proof
Built and exercised
c28da3ce2718b03c6cdbceb0e698e1e12250e154with an isolated home directory, the staticsshprovider explicitly pinned, a real local sshd, and temporary work/output paths. Temporary paths and the local username are redacted below; no cloud lease was created.The collision attempt exits during local preflight: there is no target acquisition or remote command output. The independently computed hash exactly matches the receipt, and the verifier makes the self-signed trust boundary explicit.
Contributor real behavior proof
Behavior addressed: run evidence in Crabbox cannot be verified after the fact; there is no signed receipt and no verify command, so any pasted run result is trust-me.
Real environment tested: the real
crabboxbinary built from pristine main d2ad413 and from this branch at head 7ec4e38, driving a realcrabbox runover loopback SSH with the static ssh provider (real sshd, real sync, real remote command execution) on macOS; nothing simulated.Exact steps or command run after this patch:
crabbox run --static-host 127.0.0.1 --static-user yuvald --attest receipts/nested/receipt.json -- /bin/sh -c 'echo attest demo; uname -s'(a nested receipt path whose directories do not exist yet), thencrabbox verifyon the untouched receipt, after a one-field tamper, after a duplicate-key tamper that keeps the signed value last, and after injecting an unknown field.Evidence after fix at contributor head
7ec4e381(current output additionally includestrust=self-signed):Observed result after fix: a real run over loopback SSH created the missing receipt directories with private modes and produced a signed receipt whose output digest independently matches the observed live output; verifying the untouched receipt prints PASS with the signer fingerprint and exit 0, flipping a single field flips the same command to FAIL signature mismatch with exit 1, and both a duplicate-key edit that keeps the signed value last and an injected unknown field are rejected outright as malformed with exit 2 instead of printing PASS, where pristine main rejects the verify command entirely and has no attest flags.
What was not tested: no delegated-provider live run (the delegated receipt path is exercised in the Go suite only); no Windows host; key rotation and multi-user trust decisions are out of scope for v1.
Not in this change (deliberate v2 boundary)
A v1 receipt is client-side, self-signed evidence: it proves the record is unmodified since signing and was signed by the embedded key, whose fingerprint
verifyprints for out-of-band comparison. It does not prove who holds that key (no trust roots or identity binding), that the run occurred as described (the signer is the party that ran it), or that a receipt was not withheld and regenerated (no transparency log). Coordinator key custody, DSSE or in-toto envelope formats, trust roots, and a remote receipt registry are deliberately out of scope for this slice.