audit: version-aware verdicts + install gate (fixes the CVE-remediation catch-22) - #29
Open
superbiche wants to merge 14 commits into
Open
audit: version-aware verdicts + install gate (fixes the CVE-remediation catch-22)#29superbiche wants to merge 14 commits into
superbiche wants to merge 14 commits into
Conversation
…t, stop blocking remediations The check pipeline previously posted version:"latest" to OSV, which returns every advisory ever filed, and verdicted on a version-blind count — blocking the very bumps that fix CVEs (third occurrence, inbox 2026-07-31) while the fabricated @latest spec made pinned host-allow entries unmatchable. - Resolve the actual install target before verdicting: exact specs as-is; npm dist-tag for installs; in-range max from package.json + lockfile ranges for updates (nested brace-expansion -> 2.1.4, not 5.x); registry latest for pip/uv, cargo, composer, go. Failure degrades to package-level audit with a WARN floor (version_unresolved) — never silent GO. - Classify advisories per resolved version (affecting/remediated/unfixed/ ambiguous) with a jq semver library; only affecting ones drive the verdict; install.block_severities (default critical) escalates to BLOCK. OSV outage now fails closed (previously counted as zero CVEs). - New --gate install mode: GO proceeds and records the pinned version in machine-written install-known.json (evidence -> check receipt; offline fallback within auto_allow_ttl_days). WARN proceeds only on a pinned host-allow entry matching the RESOLVED version or opt-in tolerated causes. Hints always pinned; nothing suggests @latest. - Socket failures refuse with an explicit infrastructure-failure message + recovery path (socket login / safe doctor, which now reports Socket wiring) — never conflatable with a package finding. - Wrappers and safe install stop fabricating @latest, pass --op/--gate, and log every gate decision to the safe-run audit log. - New offline suite tests/audit/check_version_aware.sh (29 cases: in-range update resolution, remediated pass, critical block, resolved host-allow match, socket-outage refusal + tolerate knob, || degradation, registry/ OSV outage fail-closed, no-@latest guard); install suite extended (72). Claude-Session: https://claude.ai/code/session_01Rk1wZEXbFB23TamMZYuRi6
…osed) Review: GPT-5.6-sol xhigh, tmp/pr29-verdict-core-FINDINGS.md (DO-NOT-SHIP, 5 blockers + 2 majors). All CORRECTIVE, all closed: - F1 server-authoritative exact queries: a version-scoped OSV hit can never be downgraded to GO by the local comparator (defect class: local re-derivation overriding authoritative upstream data; swept the classification path — the degraded package-only path keeps local matching but its floor is WARN, never GO). - F2 OSV pagination: follow next_page_token (bounded 10), validate vulns type, fail closed on repeated token/malformed pages (defect class: partial-response-as-complete; the packument/registry fetches are single documents, not paginated — checked). - F3 target selectors threaded: npm --tag/--registry/--prefix and composer --working-dir reach resolution (--dist-tag/--registry/--project-dir check flags); custom pip/uv/cargo/composer indexes degrade to the unresolved refusal (defect class: audited context != install context; swept all four wrapper families via safe_install_scan_target_flags, per-family tables). - F4 npm selection semantics: latest-tag preference (npm-pick-manifest), read -a instead of globbing word-split, root-overrides mention degrades (residual: engines/peer/dedupe constraints — architectural, proposed below). Defect class: reimplementing a resolver the ecosystem owns. - F5 install-known lifecycle: adverse WARN/BLOCK revokes the entry for that version; both timeout readers (zsh + bash mirror — sibling surfaces swept) require verdict == "GO"; WARN_TOLERATED records stay distinguishable. - F6 severity ladder: qualitative labels + computed CVSS v3 base score (most severe wins), CVSS v4-only floors at high; standard-severity critical now blocks without database_specific. - F7 ecosystem normalization: cargo->rust, composer->php, pip->python at check entry; cargo install --version threads the pin into the spec. Suites: tests/audit/check_version_aware.sh rewritten with a version-aware OSV mock + pagination state (50 cases), tests/install/run.sh 72 cases, both green; dispatcher integration 15/15. ARCHITECTURAL residual for operator ruling: full npm parity (engines, peerDeps, workspaces, lockfileVersion interplay) needs npm's own resolver as the oracle or a post-install lockfile verification sweep (proposed for the bulk-audit PR). Claude-Session: https://claude.ai/code/session_01Rk1wZEXbFB23TamMZYuRi6
…er precedence, evidence lifecycle Delta re-check verdict on 69ccac8: DO-NOT-SHIP 3 (F1/F7 fixed; residuals in F2-F6 + new F8). All CORRECTIVE residuals closed; F6's CVSS-v4 scoring is ARCHITECTURAL and goes to the operator as a proposal (inbox note), per R-12. - F3 source identity (defect class: audited identity != installed artifact): any custom source now floors the verdict at WARN for EVERY ecosystem and exactness — computed after resolution, so the exact-version short circuit can no longer skip it. pip/uv --find-links and --no-index are scanned as custom sources; the uv wrapper scans target flags at all; `safe install` (bin/safe) threads --tag/--registry/--prefix/--repository/--working-dir into the audit. Escape hatches: install.trusted_registries (URL-prefix list, npm/PyPI defaults always trusted) or a pinned host-allow. Leading- dash project paths are anchored (./-x) before reaching jq/grep. - F2 pagination: a page must be an object, vulns an array when present, and next_page_token a non-empty string when present ({"next_page_token":false} no longer reads as complete-and-clean); all seen tokens tracked, not just the previous one. - F4 npm selection: vparse now parses prerelease identifier lists (numeric-before-alphanumeric, element-wise, prefix-shorter-first — jq array ordering matches all three SemVer rules), and version-qualified overrides keys ("pkg@^2.0.0") degrade resolution like bare keys. - F5 evidence lifecycle: an adverse advisory (affecting > 0 or blocklist) revokes matching install-known evidence BEFORE the host-allow override decides the current invocation; infra-only WARNs (socket/OSV outage) do not revoke — the stale-evidence fallback exists for exactly those windows. - F8 canonical keys: both stale readers (zsh + bash mirror) canonicalize cargo->rust/composer->php/pip|uv->python before the install-known lookup. Suites: audit 58/58 (7 new regression cases incl. a vparse unit check and the FIRST 9.8/changed-scope CVSS vectors), install 72/72, dispatcher 15/15. Claude-Session: https://claude.ai/code/session_01Rk1wZEXbFB23TamMZYuRi6
…ust, cumulative sources, scoped evidence Delta-2 verdict on 195d080: DO-NOT-SHIP 4. All four closed: - F2: pagination tokens must be purely printable ([[:graph:]]+) — a "\n" token survived validation but stripped to empty in command substitution and read as end-of-pagination (defect class: lossy shell round-trip of opaque protocol state). - F3: trust matching is origin-bounded (pypi.org.evil.example and userinfo tricks no longer match the PyPI default), and source selectors ACCUMULATE — pip/uv consider every given source, so one untrusted --find-links floors the verdict even when followed by a trusted index (swept all three scanners: zsh, bash mirror; gate-lib ported on the PR2 branch). - F4: prerelease identifiers convert to numbers ONLY on the SemVer numeric grammar (0|[1-9][0-9]*) — jq's broader number syntax typed "1e-2" numeric and let the resolver accept a version npm rejects. - F5: install-known receipts record the source they were gathered against and both stale readers require the current source to match (default- registry evidence can never vouch for a custom index); a low Socket score now revokes stale clean evidence like advisory/blocklist hits (only enumerated pure-infra outages retain it). Suites: audit 68/68 (10 new: whitespace token, 1e-2 grammar, evil-hostname origin, real-PyPI trust, cumulative sources, low-score refuse+revoke, source-scoped receipts), install 74/74 (cumulative pip sources reach the audit; stale evidence is source-scoped both ways). Claude-Session: https://claude.ai/code/session_01Rk1wZEXbFB23TamMZYuRi6
…ources, strict core grammar, discriminated identity - F3.1: npm_registry_url resolves the LAST accumulated --registry (npm options are last-wins); the trust floor still checks every selector. - F3.2: the trust floor runs on the EFFECTIVE source set — for npm this includes NPM_CONFIG_REGISTRY and project/user .npmrc, computed on every check including exact pins (no network). Receipt identity carries it; both stale-fallback readers mirror the derivation. Trusted mirrors stay frictionless via install.trusted_registries. - F4: packument dist-tags and candidate versions are validated against the strict SemVer grammar before selection (matching npm's selector); resolve_target_versions grammar-guards every resolved target; vparse maps non-numeric core segments to -1 (never outranks a valid version) instead of jq-numeric coercion, staying lenient for annotation-only PEP440 OSV event versions. - F5: install-known source identity is discriminated (implicit-default vs explicit:<sources>) so a literal selector spelled "default" can never collide with the sentinel; legacy source-less receipts map to implicit-default only. - F2 test nit: the ws-token fixture now writes valid JSON whose decoded pagination token is a real newline (printf '%s'). Tests: check_version_aware 83/83 (15 new assertions: last-wins fetch log, env/.npmrc floors + trusted-mirror UX, malformed packument selection and dist-tag degrade, sentinel discipline incl. literal-default collision); install suite 74/74 (reader-side sentinel, env-registry stale scoping, legacy mapping). Claude-Session: https://claude.ai/code/session_01Rk1wZEXbFB23TamMZYuRi6
…ces, full SemVer grammar - F3.1: wrapper source accumulation moves a repeated selector to the END (npm last-wins) instead of first-occurrence dedup; the resolver's last-word read now matches the registry npm actually fetches from. - F3.2: effective-source derivation extended and CENTRALIZED: lowercase npm_config_registry, scoped @scope:registry keys (env + project/user .npmrc, beating argv as npm's own key precedence does), PIP_INDEX_URL / PIP_EXTRA_INDEX_URL / UV_* env, pip.conf index-url/extra-index-url (PIP_CONFIG_FILE > venv > user), and GOPROXY chains. New plumbing subcommand `safe audit effective-sources` prints the discriminated identity; both stale readers call it instead of mirroring — one derivation for gate, writer, and readers (the reviewer's suggested fix). Un-swept surfaces (uv.toml/pyproject index tables, system pip.conf, cargo config.toml, project-declared composer repositories, yarn/bun native config) are documented in-code with the injectable-vs-operator- owned rationale. - F4: full SemVer 2.0.0 grammar everywhere a registry-writable version can reach selection: SEMVER_STRICT_RE in version_is_exact (npm/rust/php/go) and the jq semver_ok used for dist-tag preference and candidate filtering — leading-zero numeric prerelease identifiers ("1.0.0-01"), empty identifiers, and leading-zero cores are rejected, matching npm-pick-manifest. - N1: source sets canonicalize (per-element trailing-slash strip + last-wins dedup) identically on the writer and reader paths, so a trailing-slash selector no longer breaks same-source stale evidence. Tests: check_version_aware 101/101 (18 new: lowercase env, scoped-key floors incl. beside-argv, prerelease-grammar selection npm-oracle + dist-tag degrade, version_is_exact unit matrix, effective-sources plumbing matrix, slash-canonical receipts, PIP_INDEX_URL/GOPROXY floors, trusted pip mirror UX); install suite 75/75 (repeated-registry true-last case; stub forwards effective-sources to the real derivation). Claude-Session: https://claude.ai/code/session_01Rk1wZEXbFB23TamMZYuRi6
… argv, node-semver limits, credential redaction - F3.2: lowercase npm_config_registry now WINS over the uppercase form (npm's own precedence, oracle-verified); the wrapper scanners recognize raw scoped --@scope:registry[=] argv flags and --userconfig/--globalconfig (registry + scoped keys inside the pointed file become sources); python_env_sources adds PIP_FIND_LINKS / UV_FIND_LINKS / UV_EXTRA_INDEX_URL and the no-index switches (env + swept pip.conf keys); argv --find-links now records its actual endpoint instead of a blanket sentinel so operators can trust the specific location. - F4: npm_semver_within_limits enforces node-semver's version-length (256) and Number.MAX_SAFE_INTEGER core-component ceilings in version_is_exact (npm only — other ecosystems' parser limits differ) and the jq semver_ok mirrors them as string comparisons; an over-limit numeric spec now resolves as the dist-tag npm actually sees, and over-limit candidates are excluded from range selection. - N2: canonicalize_source drops URL userinfo everywhere identities, receipts, trust matching, or output are built (normalize_source_set, derive append, RESOLUTION_SOURCE, wrapper add_source before the audit log); operational fetch URLs keep credentials. Trust matching runs on the credential-free canonical endpoint. Tests: check_version_aware 116/116 (both-case env precedence, huge-core tag + range npm-oracles, node-semver limit unit, PIP_FIND_LINKS / UV_EXTRA_INDEX_URL / PIP_NO_INDEX floors, credential redaction on refusal output + receipts); install suite 77/77 (scoped argv + --userconfig sources reach the audit, credentials stripped from AUDIT log lines, find-links endpoint recording). Claude-Session: https://claude.ai/code/session_01Rk1wZEXbFB23TamMZYuRi6
…erational/display split, pip config parser - F3.2b: scoped CLI selectors are threaded as KEYED tokens (@scope:registry=URL) instead of flattened URLs, so npm_registry_url resolves the MATCHING scope's registry with npm's key precedence (CLI scoped > env scoped > project rc > user rc); generic last-wins selection skips scoped tokens; the trust floor checks each token's endpoint. npm_config_userconfig/NPM_CONFIG_USERCONFIG replace the user rc in both ladders and their registry keys sweep into the effective set. Also fixes a pre-existing parse_pkg_spec bug that split a bare scoped name at its leading @. - F3.2c: pip config files are read with pip/configparser syntax (both '=' and ':' delimiters, comments, indented continuation lines) via a small awk parser — line-grep missed forms pip itself accepts; later duplicate keys win per file. - F4: the 256-char node-semver ceiling is measured on the RAW input before the v prefix is stripped. - N2: redaction moved fully to the sinks: canonicalize_source now also strips query strings and fragments (token= credentials), while the wrappers no longer pre-redact — the accumulated set stays operational so authenticated packument fetches work again; identities, receipts, refusal output, and trust matching all remain credential-free. - N3: no-index booleans are parsed per-variable, case-insensitively, with pip's vocabulary — PIP_NO_INDEX=False or a 0/0 pair no longer floors a clean default-source install. Tests: check_version_aware 132/132 (scoped-key resolution npm-oracles incl. CLI-over-env, env userconfig floor, query-token redaction, authed-registry operational fetch + credential-free receipt, colon/continuation pip config, no-index boolean matrix, raw-length boundary, scoped plumbing identity); install suite 77/77 (keyed scoped token threading, operational credential preservation). Claude-Session: https://claude.ai/code/session_01Rk1wZEXbFB23TamMZYuRi6
… section semantics, boolean vocabulary - F3.2b: alternate npm config files thread to safe-audit AS PATHS (--npm-userconfig/--npm-globalconfig; wrapper value-flattening deleted): npm_user_rc/npm_global_rc slot them into both rc ladders with npm's tier precedence, so scoped keys resolve per package and generic keys take their true tier — no more false CLI precedence or lost scope. The effective-for-this-package endpoint replaces the all-keys sweep (which was also over-flooring). The prev-form --@scope:registry now consumes its value in both npm package parsers (the URL was audited as a package). - F3.2c: pip_config_values gains configparser semantics — option-name normalization (lowercase, underscores→hyphens), [install]-overrides- [global] section precedence independent of file order, and config paths iterate as quoted arrays (spaced PIP_CONFIG_FILE was silently missed). - N2: config-file credentials never leave the file (path threading), and the zsh wrappers clear the scanned target state on every gate exit so an argv credential cannot linger in the long-lived shell. - N3: python_bool_truthy accepts pip's full vocabulary (adds y/t). Tests: check_version_aware 140/140 (userconfig scoped-key npm-oracle, generic userconfig floor, section precedence, key normalization, spaced path, y/t booleans); install suite 78/78 (prev-form scoped value consumption + no URL-as-package, userconfig path threading with no value leakage, post-gate state cleared). Claude-Session: https://claude.ai/code/session_01Rk1wZEXbFB23TamMZYuRi6
…figParser fidelity, always-block cleanup
- F3.2b: option-like config paths (a file literally named "-q") anchor to
./ at every entry — flag parse in check/effective-sources, the env forms
via anchor_path in npm_user_rc/npm_global_rc, and the python config path
array — so grep/awk can never read them as their own options and fail
the sweep open.
- F3.2c: pip_config_values matches ConfigParser: continuation lines are
only lines indented DEEPER than their option (indented base options
parse as options), key normalization also strips one leading "--"
("--index-url =" is a valid pip config key), and full-line comments drop
at any indent without ending a value block.
- N2: scanned target state clears via a zsh always-block around every
scanning route (npm/pip families, uv, cargo, go, composer) — project
scans, parser fallbacks, unsupported subcommands, refusals, and
successful gates alike; cleanup is no longer tied to check_many alone.
Tests: check_version_aware 146/146 (dash-named argv/env userconfig and
PIP_CONFIG_FILE floor instead of failing open; indented option; --key
spelling; continuation regression guard); install suite 78/78
(project-install route clears the scanned state).
Claude-Session: https://claude.ai/code/session_01Rk1wZEXbFB23TamMZYuRi6
… refactor (round 10) Delta-9 N4: the *_impl split moved the inlined degraded guards behind strippable helper names and made the always blocks call a strippable cleanup helper — a public-only snapshot died 127 with command-not-found noise instead of the documented `safe: BLOCKED` exit 100. - The degraded guards move back INTO the public uv/cargo/go/composer wrappers (per the standing inline-guard contract), with the impl name added to the checked function list so a missing impl refuses legibly. - Every always block inlines the five state assignments instead of calling a helper a partial snapshot may strip. - The degraded test harness now strips *_impl alongside safe_install_*, and every degraded case rejects `command not found` on stderr. Tests: install suite 78/78 (degraded matrix now models the public-only snapshot); check_version_aware 146/146; other suites exit 0. Claude-Session: https://claude.ai/code/session_01Rk1wZEXbFB23TamMZYuRi6
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Third occurrence (inbox
2026-07-31-reveille-safe-blocks-security-bumps-cve-catch22.md): safe WARN-blocks the dependency bump that fixes the very CVE it warns about, and suggests the forbiddenhost-allow add <pkg>@latestshape. Root cause: wrappers fabricate@latestfor unversioned specs;safe audit checkpostsversion:"latest"to OSV, which cannot parse it and returns every advisory ever filed; the verdict is a version-blind count. The fixed-version ranges were fetched, persisted, and never consulted. The host-allow matcher hard-rejectslatest, so the operator ruling "never host-allow@latest" made pinned allows unmatchable onnpm updatepaths.Change
safe-audit, one implementation for the zsh wrappers and thebin/safemirror): exact specs as-is; npm dist-tag for installs; in-range max frompackage.json+ lockfile ranges for--op update(nestedbrace-expansionresolves to 2.1.4, not 5.x); registry latest for pip/uv, cargo, composer, go. Resolution failure degrades to a package-level audit with a WARN floor (version_unresolved) — never a silent GO.affecting/remediated/unfixed/ambiguous) via a jq semver library; only affecting advisories drive the verdict;install.block_severities(default["critical"]) escalates to BLOCK. An OSV outage now fails closed (previously it counted as zero CVEs — a fail-open hole). Exit codes and plainchecksemantics unchanged.--gate installmode used by wrappers andsafe install: GO proceeds with no operator terminal and records the pinned resolved version in machine-written~/.config/safe/run/install-known.json(evidence → check receipt; offline fallback when the audit times out, withininstall.auto_allow_ttl_days). WARN proceeds only on a pinned host-allow entry matching the resolved version (fixing the unmatchable-pin dead end) or on opt-ininstall.auto_allow_toleratecauses. Refusal hints are always pinned; nothing ever renders@latest.safe doctornow reports Socket CLI/token wiring.safe installappendinstall:<ecosystem> | … | GATE | … | PROCEED/AUTO-…/REFUSED_*lines to the safe-run audit log (they were stderr-only).Verification
tests/audit/check_version_aware.sh— 29 cases, all green: in-range update resolution, remediated-pass, critical-block, resolved host-allow match, socket-outage refusal + tolerate knob,||-range degradation, registry/OSV outage fail-closed, and a no-@latest-in-hints guard.tests/install/run.shextended and green (72 cases);tests/audit/smoke.sh,tests/audit/external_binary.sh,tests/integration/dispatcher.sh,tests/run/safe_audit_integration.shall green.brace-expansion@^2.0.0,--op update --gate install): resolves 2.1.4 viaregistry.npmjs.org, and the hint issafe run host-allow add brace-expansion@2.1.4 --reason "...". Note: current upstream OSV data says GHSA-mh99-v99m-4gvg affects[0, 5.0.8)(advisory revised after an incomplete fix), so 2.1.4 correctly still WARNs today — with an honest, pinned, actionable message instead of a version-blind count.Part 1 of the safe overhaul (verdict core). Next: PATH-executable wrappers + mise coverage, bulk audit, agent contract surface.
@superbiche · maintainer · drafted with Claude Fable 5.
https://claude.ai/code/session_01Rk1wZEXbFB23TamMZYuRi6