chore(audit): weekly cloud audit — completion-gate false-greens, security-KB race, projection & retention gaps#92
Draft
tzone85 wants to merge 3 commits into
Draft
chore(audit): weekly cloud audit — completion-gate false-greens, security-KB race, projection & retention gaps#92tzone85 wants to merge 3 commits into
tzone85 wants to merge 3 commits into
Conversation
…ity-KB race, projection wiring, orphan-DB retention, investigator filter) Automated weekly audit of the default branch. Fanned out parallel sub-agents across concurrency / silent-failure / event-sourcing-wiring / security, and adversarially verified each candidate before fixing. Every fix has a regression test. - engine/verification_loop: the completion gate reported green on code whose tests do not compile. parseGoTestJSON skipped go-test-json events with an empty Test field, so package-level compile failures (build-fail / FailedBuild, no Test field) counted as 0 failing while `go build ./...` (which skips _test.go) passed — REQ_COMPLETED on a non-compiling suite. Count package-level build failures; treat a non-zero `go test` exit with 0 parsed failures as a structural failure. Also thread ctx + bounded timeouts through checkTests / ensureDependencies so a hung LLM-generated test can't wedge the gate forever. - security/knowledge + engine/security_gate: concurrent story pipelines share one SecurityGate with no lock and a truncating os.WriteFile, so two stories that both learn a rule race their KB writes — a torn write yields invalid JSON, and since LoadKnowledgeBase only falls back to baseline on a MISSING file, that permanently and silently disables the security gate. Make Save atomic (temp + rename) and serialize load→merge→save under a mutex, reloading the persisted KB so concurrent stories don't clobber each other's learned rules. - state/sqlite: AGENT_TERMINATED was emitted by the dashboard kill handler and the controller (both call Project expecting a mutation) but had no case in the projection switch, so it silently no-op'd — killed agents stayed status='idle' with a live session→story mapping that crash recovery kept consuming. Add the case; set status='terminated' and clear current_story_id. - devdb/recovery: ReleaseOrphans' retention window (RetainHours/minAge) was dead on the docker path — the provider never populates CreatedAt, and a zero time is always Before(cutoff), so every orphan was reaped regardless of retention, including another concurrent requirement's in-use databases. Fail safe: keep orphans whose CreatedAt is zero (unknown age) for human review. - engine/investigator: the run_command shell filter blocked chaining but left redirection (> <), background (&), and bare variable expansion ($) open, so an allowlisted prefix could write/exfiltrate outside the repo before sh -c. Reject the same canonical metacharacter set the native runtime already enforces. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UPCbBB875JzQVMetfqqido
…se green) internal/criteria/evaluator.go — evalCoverageAbove defaults its target to "./..." and passed `go test -cover ./...` output to parseCoverage, whose FindStringSubmatch returns only the FIRST "coverage: X%" line. With one line per package, the gate read only the alphabetically-first package's number: a repo where package aaa is 100% and package zzz is 5% passed a `coverage_above: 80` criterion. Compute the statement-weighted aggregate from a merged -coverprofile via `go tool cover -func` (total: line) instead, falling back to the old parse only when the profile can't be reduced. Regression test: TestCoverageAbove_MultiPackage_AggregatesNotFirstPackage. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UPCbBB875JzQVMetfqqido
…026-5856 CI govulncheck flagged GO-2026-5856 (Encrypted Client Hello privacy leak in crypto/tls), present in the go1.26.4 standard library and CALLED via web.Server.Start, docker.Client.Ping, metrics.Recorder.Record, and update.Checker. Fixed in go1.26.5. Bump the go.mod toolchain directive and the CI setup-go pins to 1.26.5 — the same remediation pattern the repo already uses for the earlier 1.26.x stdlib CVEs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UPCbBB875JzQVMetfqqido
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Weekly automated security/quality audit of the default branch. Fanned out parallel
sub-agents across concurrency, silent-failure, event-sourcing-wiring, and security
categories, adversarially verified every candidate, and fixed the ones that survived.
Seven real code defects fixed (each with a regression test), plus one CALLED
standard-library vulnerability surfaced by CI govulncheck.
Findings fixed
1. Completion gate reports green on code whose tests do not compile
internal/engine/verification_loop.go·parseGoTestJSON/checkTestsparseGoTestJSONskipped everygo test -jsonevent with an emptyTestfield,and
checkTestsdiscarded the command's exit code. A test-package compilefailure is reported by
go test -jsononly as package-scoped events(
"build-fail", and a package"fail"carryingFailedBuild) that have noTestfield — so they were dropped andTestsFailingcame back0. Becausego build ./...skips_test.gofiles,BuildPasseswastrue. Net effect:ShouldRunFixCyclereturnedfalseand the completion gate emittedREQ_COMPLETEDfor a composed mainline whose test suite does not even compile —exactly the cross-story drift (story A changes a signature, story B's test still
calls the old one) the gate exists to catch. Verified empirically with a real
go test -jsonrun.Fix:
parseGoTestJSONnow counts package-levelbuild-fail/FailedBuildevents as failures (without double-counting ordinary assertion failures), and
checkTeststreats a non-zerogo testexit with zero parsed failures as astructural failure. Regression tests:
TestParseGoTestJSON_CountsTestCompileFailure,TestParseGoTestJSON_NormalResultsUnaffected,TestRunVerificationLoop_NonCompilingTests_NotGreen.2. Completion-gate verification runs unbounded subprocesses and drops its context
internal/engine/verification_loop.go·RunVerificationLoop/checkTests/ensureDependenciesRunVerificationLoop(ctx, …)received a context but never threaded it into itssubprocess calls, and
checkTests/ensureDependenciesran with no timeout.The gate runs these over an LLM-generated composed mainline; a single hung test
(infinite loop, blocked on stdin/network) or a stalled
go mod download/npm installwould block beforeREQ_COMPLETED/REQ_BLOCKEDcould ever beemitted, and the dropped
ctxmeant even Ctrl-C onnxd resumecouldn't cancelit. The sibling paths (
validateBuild, the Gemma runtime) already bound theirsubprocesses; this one was left open.
Fix: thread
ctxthrough and run every verification subprocess viaexec.CommandContext, with aboundedContexthelper that applies a backstoptimeout when the caller's ctx has no deadline (deps 5m, tests 15m) while
preserving caller cancellation.
3. Security knowledge base: non-atomic write + missing lock can permanently disable the gate
internal/security/knowledge.go·KnowledgeBase.Save;internal/engine/security_gate.go·SecurityGate.upskillpostExecutionPipelineruns one goroutine per completed story, all sharing oneSecurityGate, which had no mutex. Whenauto_learnis on (default), two storiesin the same wave that both learn a rule race two truncating
os.WriteFilecallsagainst the same
kbPath: best case a lost update, worst case interleaved bytesproducing invalid JSON. And
LoadKnowledgeBaseonly falls back to the baselinewhen the file is missing — a corrupt file errors forever, and
ReviewStoryerrors are swallowed ("continuing to merge"), so one torn write silently and
permanently disables the security gate for all future stories. (The sibling
routing.Savealready solved this with temp-file + rename; the KB hadn't.)Fix:
Savenow writes to a temp file andos.Renames it into place (atomic onPOSIX).
upskilltakes anupskillMuand, under the lock, reloads the persistedKB before merging so concurrent stories build on each other's learned rules
instead of clobbering them; a cheap pre-check skips the lock entirely when there
is nothing learnable. Regression test:
TestKnowledgeBase_Save_AtomicUnderConcurrency(runs under-race).4. Event-sourcing wiring gap: AGENT_TERMINATED silently no-ops in the projection
internal/state/sqlite.go·SQLiteStore.ProjectThe dashboard "kill agent" handler (
web/handlers.go) and the controller'sauto-cancel (
engine/controller.go) both emitAGENT_TERMINATEDand callProject(evt)expecting a mutation — but the switch had no case for it, so itfell through to
default: return nil. The agent row stayedstatus='idle'withits
session_name/current_story_idstill populated. Consequences:nxd agentsand the dashboard show a killed agent as a live idle one, and crashrecovery's
buildSessionStoryMapkeeps mapping the dead session to its story,so recovery can try to reconcile an already-killed tmux session. The dashboard's
agentStatusStylealready renders a"terminated"status — this is theprojection wiring it was waiting on.
Fix: add a
case EventAgentTerminated→projectAgentTerminated, which setsstatus='terminated'and clearscurrent_story_id(dropping the dead sessionfrom the recovery map). Regression test:
TestProject_AgentTerminated_TransitionsRow.5. Investigator shell-command filter weaker than the runtime's — allows redirection / expansion
internal/engine/investigator.go·Investigator.isCommandAllowedThe investigator's
run_commandfilter blocked only chaining(
; && || | $(`\n) before routing the command throughshellexec→sh -c. It left redirection (>,<), background (&), bare variableexpansion (
$HOME,${IFS}), and\r\t\x00\open — so an allowlistedprefix (
cat,grep,ls) combined with redirection could write or exfiltrateoutside the repo without any chaining operator, e.g.
cat internal/secret > /home/user/.bashrc. The native runtime'srun_commandguard already rejects the full metacharacter set; the two enforcement points had
drifted.
Fix: reject the same canonical set the runtime uses
(
;&|$`<>\n\r\t\x00\\). Regression test:TestInvestigator_CommandAllowlist_RejectsRedirectionAndExpansion(and theexisting chaining tests still pass).
6. Orphan-DB retention window is silently defeated on the Docker path — can delete in-use databases
internal/devdb/recovery.go·ReleaseOrphansReleaseOrphansprotects freshly-created orphan DBs from reaping withif db.CreatedAt.After(cutoff). But the Docker provider (the production path)never populates
CreatedAt— Postgres exposes no reliable per-database creationtime, and
dbFromNameleaves it zero. A zerotime.Timeis alwaysBefore(cutoff), so every Docker orphan is deleted regardless of theconfigured
RetainHours/minAge: the retention window is dead. NXD shares onehost Postgres across requirements, so
nxd resumefor requirement A classifiesrequirement B's live DBs (different story IDs) as orphans — and the window meant
to spare them never fires, so A's resume can drop B's in-use databases. The unit
test and the
nullprovider both setCreatedAt, which is why the first roundmissed it; only the real Docker
Listpath leaves it zero. (lifecycle.goalready guards
CreatedAt.IsZero();ReleaseOrphansdidn't.)Fix: treat an unknown (zero)
CreatedAtas "too young to reap; keep for humanreview" —
if db.CreatedAt.IsZero() || db.CreatedAt.After(cutoff). Fail safe,since the timestamp can't be backfilled from Postgres. Regression test:
TestReleaseOrphans_ZeroCreatedAtKeptNotDeleted.7.
coverage_abovegate reads only the first package's coverage — false green on multi-package targetsinternal/criteria/evaluator.go·evalCoverageAbove/parseCoverageevalCoverageAbovedefaults its target to./...when no target is set, thenran
go test -cover ./...and passed the output toparseCoverage, whoseFindStringSubmatchreturns only the firstcoverage: X%line.go test -cover ./...prints one such line per package, so the gate read only thealphabetically-first package's number: a repo where package
aaais 100% andpackage
zzzis 5% passes acoverage_above: 80criterion. A gating qualitycheck silently reporting green is exactly the false-negative this audit targets.
Fix: write a merged
-coverprofileand compute the statement-weighted aggregatevia
go tool cover -func(total:line); fall back to the old per-run parseonly when the profile can't be reduced. Regression test:
TestCoverageAbove_MultiPackage_AggregatesNotFirstPackage(aaa=100%, zzz≈0% →aggregate far below 80% → correctly fails).
8. CALLED stdlib vulnerability GO-2026-5856 (crypto/tls) — surfaced by CI govulncheck
go.mod·toolchain;.github/workflows/ci.yml·go-versionCI govulncheck flagged
GO-2026-5856(Encrypted Client Hello privacy leak incrypto/tls), present in the go1.26.4 standard library and CALLED viaweb.Server.Start,docker.Client.Ping,metrics.Recorder.Record, andupdate.Checker. Fixed in go1.26.5.Fix: bump the
go.modtoolchain directive and the CI setup-go pins from1.26.4to1.26.5— the same remediation pattern the repo already uses forthe earlier 1.26.x stdlib CVEs. Build/vet/tests re-verified against go1.26.5.
(govulncheck could not be run in the audit sandbox —
vuln.go.devis network-blocked there — so this one came from CI, which can reach the vuln DB.)
Flagged but NOT actioned (needs a maintainer decision)
A first-round agent flagged
vxd/vortexreferences (and the private repo URL…/vortex-dispatch) indocs/history/vxd.yaml,docs/history/*.md, andCHANGELOG.md. I deliberately did not scrub these: the repo's own codifiedleak policy,
scripts/check-leaks.sh, guards specific client names, a corporateemail domain, and the private
tzone85/project-xrepo, but does not listvxd/vortexand explicitly allowlistsdocs/history/(the CI Leak checkpasses on this branch). Rewriting archived history against a checked-in,
considered policy is a maintainer call, not an automated one — flagging it here
instead. If the sibling name should be guarded, add
vxd|vortextocheck-leaks.shFORBIDDEN and scrub accordingly.Verification
go build ./...— clean (go1.26.5)go vet ./...— cleango test ./... -count=1— passgo test ./internal/engine/ ./internal/security/ ./internal/state/ -race— pass (covers the new concurrency fix)govulncheck ./...— the CALLED vuln GO-2026-5856 (finding docs: clarify API key usage in configuration guide #8) was found by CI govulncheck and fixed by the toolchain bump; the audit sandbox itself cannot reachvuln.go.dev(403 on CONNECT), so CI is the authoritative govulncheck run here.golangci-lint— could not run in the audit sandbox (v2.5.0 refuses ago 1.26module because its own build targetsgo 1.25, and no 1.26-ready release resolves); the CILintjob is authoritative.go vetis clean locally.🤖 Generated with Claude Code