diff --git a/docs/src/enrichers/epss.md b/docs/src/enrichers/epss.md index 6f8a97c..7e210c4 100644 --- a/docs/src/enrichers/epss.md +++ b/docs/src/enrichers/epss.md @@ -4,36 +4,51 @@ bomdrift queries the [Exploit Prediction Scoring System (EPSS)] from FIRST.org for every CVE-aliased advisory and surfaces the per-CVE score (0.0 – 1.0) in markdown / terminal / SARIF output. +## Why this signal + EPSS estimates the probability that a given CVE will be exploited in the next 30 days. Combined with severity it gives reviewers a sharper signal -than CVSS alone — a Critical CVE with EPSS 0.01 is far less urgent than a +than CVSS alone: a Critical CVE with EPSS 0.01 is far less urgent than a Medium CVE with EPSS 0.85. -## Output +## Algorithm -- **Markdown**: per-advisory badge `EPSS 0.87` after the severity label. -- **Terminal**: same badge, no markup. -- **JSON**: `enrichment.vulns[purl][i].epss_score` numeric field. -- **SARIF**: `properties.epssScore` on `bomdrift.cve` results. +For every CVE-aliased advisory surfaced by the [OSV.dev CVE lookup](./osv-cve.md), +bomdrift fetches the EPSS score from FIRST.org and attaches it to the +finding. When an advisory is keyed by GHSA but has CVE aliases, the score +is the **max across all CVE aliases**, so a GHSA covering two CVEs +surfaces the worse of the two. -When an advisory is keyed by GHSA but has CVE aliases, the score is the -**max across all CVE aliases** so a GHSA covering two CVEs surfaces the -worse of the two. +## Threshold -## Threshold gating +EPSS has no surfacing threshold: every scored advisory shows its score. +The optional gate is `--fail-on-epss `, which exits 2 when any +advisory has a score at or above the value: ```bash bomdrift diff before.json after.json --fail-on-epss 0.5 ``` -Exits 2 when any advisory has score ≥ 0.5. 0.5 is roughly the top decile -of actively-exploited CVEs; tune for your team's risk appetite. +0.5 is roughly the top decile of actively-exploited CVEs; tune for your +team's risk appetite. -## Calibration +## Output -- `--cache-ttl-hours ` (v0.9.6+) — overrides the default 24h disk - cache TTL for the EPSS scores cache. -- `--fail-on-epss ` — threshold gate; see [Threshold gating](#threshold-gating). +- **Markdown**: per-advisory badge `EPSS 0.87` after the severity label. +- **Terminal**: same badge, no markup. +- **JSON**: `enrichment.vulns[purl][i].epss_score` numeric field. +- **SARIF**: `properties.epssScore` on `bomdrift.cve` results. + +## Network + +- **Source**: FIRST.org `/api/v1/epss`. +- **Caching**: 24h TTL at `/bomdrift/epss/.json`. Negative + results (CVEs FIRST.org returned no score for) are cached to avoid + re-querying recently-published CVEs that haven't been scored yet. +- **Best-effort**: like every bomdrift enricher, EPSS is best-effort. A + network failure or a malformed response surfaces a `BOMDRIFT_DEBUG=1` + stderr note and the diff renders with empty `epss_score` fields. EPSS + being unreachable is never a reason to block a PR review. ## Disabling @@ -50,17 +65,16 @@ no_epss = true Both forms skip the FIRST.org HTTP call AND the disk cache lookup. -## Caching +## Calibration -24h TTL at `/bomdrift/epss/.json`. Negative results -(CVEs FIRST.org returned no score for) are cached to avoid re-querying -recently-published CVEs that haven't been scored yet. +- `--cache-ttl-hours ` (v0.9.6+) overrides the default 24h disk cache + TTL for the EPSS scores cache. +- `--fail-on-epss ` is the threshold gate; see [Threshold](#threshold). -## Best-effort +## See also -Like every bomdrift enricher, EPSS is best-effort: a network failure or -a malformed response surfaces a `BOMDRIFT_DEBUG=1` stderr note and the -diff renders with empty `epss_score` fields. EPSS being unreachable is -never a reason to block a PR review. +- [OSV.dev CVE lookup](./osv-cve.md) +- [CISA KEV](./kev.md) +- [Enrichers overview](./overview.md) [Exploit Prediction Scoring System (EPSS)]: https://www.first.org/epss/ diff --git a/docs/src/enrichers/kev.md b/docs/src/enrichers/kev.md index f628151..28b03b9 100644 --- a/docs/src/enrichers/kev.md +++ b/docs/src/enrichers/kev.md @@ -4,35 +4,47 @@ bomdrift downloads the [CISA Known Exploited Vulnerabilities catalog] and flips a `KEV` flag on every advisory whose primary id or aliases include a CVE listed in the catalog. +## Why this signal + CISA KEV is the highest-confidence "actively exploited in the wild" signal -available — CISA only adds CVEs to the catalog after observing real-world +available: CISA only adds CVEs to the catalog after observing real-world exploitation. It's a tighter filter than `--fail-on critical-cve` (which fires on CVSS High or above regardless of exploitation evidence). -## Output +## Algorithm -- **Markdown**: bold `**KEV**` badge after the severity / EPSS label. -- **Terminal**: plain `KEV` token. -- **JSON**: `enrichment.vulns[purl][i].kev` boolean field. -- **SARIF**: `properties.kev: true` on `bomdrift.cve` results when set. +bomdrift fetches the bulk catalog once, then flips a boolean `KEV` flag on +every advisory (from the [OSV.dev CVE lookup](./osv-cve.md)) whose primary +id or CVE aliases appear in the catalog. -## Threshold gating +## Threshold + +KEV is a boolean flag, not a scored threshold: an advisory either is in the +catalog or it is not. The optional gate is `--fail-on kev`, which exits 2 +when any advisory has its KEV flag set: ```bash bomdrift diff before.json after.json --fail-on kev ``` -Exits 2 when any advisory has its KEV flag set. `--fail-on any` also -includes KEV. +`--fail-on any` also includes KEV. -## Calibration +## Output -### `--cache-ttl-hours ` (v0.9.6+) +- **Markdown**: bold `**KEV**` badge after the severity / EPSS label. +- **Terminal**: plain `KEV` token. +- **JSON**: `enrichment.vulns[purl][i].kev` boolean field. +- **SARIF**: `properties.kev: true` on `bomdrift.cve` results when set. + +## Network -The 24h TTL for the catalog file is now configurable via the unified -cache-TTL knob. Lower for faster CISA-update propagation in -long-running self-hosted runners; raise when running offline or against -archived SBOMs. +- **Source**: CISA known-exploited catalog (one bulk JSON download). +- **Caching**: 24h TTL on the bulk catalog JSON at + `/bomdrift/kev/catalog.json`. Once-daily refresh matches + CISA's publication cadence. +- **Best-effort**: a network failure logs at `BOMDRIFT_DEBUG=1` and the + diff renders with KEV flags absent. A stale catalog (within the 24h + window) is preferred over re-fetching on every run. ## Disabling @@ -47,16 +59,17 @@ or in `.bomdrift.toml`: no_kev = true ``` -## Caching +## Calibration -24h TTL on the bulk catalog JSON at -`/bomdrift/kev/catalog.json`. Once-daily refresh matches CISA's -publication cadence. +- `--cache-ttl-hours ` (v0.9.6+) overrides the default 24h TTL for the + catalog file via the unified cache-TTL knob. Lower it for faster + CISA-update propagation in long-running self-hosted runners; raise it + when running offline or against archived SBOMs. -## Best-effort +## See also -Network failure logs at `BOMDRIFT_DEBUG=1` and the diff renders with KEV -flags absent. A stale catalog (within the 24h window) is preferred over -re-fetching on every run. +- [OSV.dev CVE lookup](./osv-cve.md) +- [EPSS](./epss.md) +- [Enrichers overview](./overview.md) [CISA Known Exploited Vulnerabilities catalog]: https://www.cisa.gov/known-exploited-vulnerabilities-catalog diff --git a/docs/src/enrichers/maintainer-age.md b/docs/src/enrichers/maintainer-age.md index d823dda..a822d79 100644 --- a/docs/src/enrichers/maintainer-age.md +++ b/docs/src/enrichers/maintainer-age.md @@ -3,52 +3,39 @@ Flag newly added dependencies (GitHub, GitLab, Codeberg) whose top contributor's first commit is suspiciously recent. **The xz/Jia Tan pattern.** -## Why it matters +## Why this signal The xz-utils backdoor (CVE-2024-3094, Mar 2024) was the work of "Jia Tan", a GitHub identity that started contributing roughly two years before -landing the malicious payload. The pattern — a brand-new account -becoming the de facto sole maintainer of a low-traffic but -widely-depended-upon package — is a leading indicator of long-game -supply-chain takeovers. +landing the malicious payload. The pattern, a brand-new account becoming +the de facto sole maintainer of a low-traffic but widely-depended-upon +package, is a leading indicator of long-game supply-chain takeovers. We can't catch Jia Tan in retrospect, but we can flag the *next* one earlier in their arc by surfacing "this package's top contributor opened their first PR less than 90 days ago" at the moment a new dep is added. -## Threshold - -**90 days** by default. Intentionally aggressive: most legitimate new -packages will trip this on initial introduction. That's fine — a human -reviewer can dismiss "the package is brand-new and the author is its -only maintainer" trivially. - -The expensive miss is the **silent takeover** of an existing package by -a recently-arrived contributor, which is what the 90-day window -captures. Tune for your environment via `--young-maintainer-days ` -or `[diff] young_maintainer_days = ` (v0.9.6+); see -[Calibration](#calibration) below. - -## How it works +## Algorithm For each `cs.added` component with a `source_url` on a supported host: ### GitHub -1. **GET `/repos/{owner}/{repo}/contributors?per_page=1`** — top +1. **GET `/repos/{owner}/{repo}/contributors?per_page=1`** for the top contributor login. -2. **GET `/repos/{owner}/{repo}/contributors?per_page=1&anon=true`** — - contributor count from Link `rel="last"` page number. **Skip if > 50.** -3. **GET `/repos/{owner}/{repo}/commits?author=&per_page=1`** — - paginate to last page for the author's oldest commit (`commit.author.date`). +2. **GET `/repos/{owner}/{repo}/contributors?per_page=1&anon=true`** for + the contributor count from Link `rel="last"` page number. **Skip if > 50.** +3. **GET `/repos/{owner}/{repo}/commits?author=&per_page=1`** and + paginate to the last page for the author's oldest commit + (`commit.author.date`). ### GitLab -1. **GET `/api/v4/projects/{url-encoded}/repository/contributors?order_by=commits&sort=desc&per_page=1`** — - top contributor name (GitLab identifies contributors by author name, not - login) and total count via `X-Total` header. **Skip if > 50.** -2. **GET `.../commits?author=&per_page=1`** — paginate to last page - via Link header for the author's oldest commit (`authored_date`). +1. **GET `/api/v4/projects/{url-encoded}/repository/contributors?order_by=commits&sort=desc&per_page=1`** + for the top contributor name (GitLab identifies contributors by author + name, not login) and total count via `X-Total` header. **Skip if > 50.** +2. **GET `.../commits?author=&per_page=1`** and paginate to the last + page via Link header for the author's oldest commit (`authored_date`). ### Codeberg @@ -56,33 +43,73 @@ URL parsing and dispatch are implemented. The per-author first-commit lookup is stubbed pending verification of the Forgejo v1.20+ API shape; Codeberg components produce no finding in this release. -## Skipped cases +### Skipped cases - Components without a `source_url` (CycloneDX `externalReferences` - with no `vcs` entry, etc.) -- silently skipped. -- Source URLs not from `github.com`, `gitlab.com`, or `codeberg.org` -- + with no `vcs` entry, etc.) are silently skipped. +- Source URLs not from `github.com`, `gitlab.com`, or `codeberg.org` are silently skipped. -- Repositories with **> 50 contributors** -- skipped because the "top +- Repositories with **> 50 contributors** are skipped because the "top contributor's first commit" loses meaning on monorepos and multi-vendor projects. -- Repositories returning 404, 401, or 403 -- skipped silently (private +- Repositories returning 404, 401, or 403 are skipped silently (private repo or missing token). Per-repo results are cached within a single bomdrift run so repeated `cs.added` entries from the same project don't re-issue the same three requests. -## Network behavior +## Threshold + +**90 days** by default. Intentionally aggressive: most legitimate new +packages will trip this on initial introduction. That's fine, a human +reviewer can dismiss "the package is brand-new and the author is its +only maintainer" trivially. The expensive miss is the **silent takeover** +of an existing package by a recently-arrived contributor, which is what +the 90-day window captures. + +The signal is always informational. It **never** trips `--fail-on +critical-cve`; it surfaces only under `--fail-on any`. The intent is human +review, not gating: many legitimate packages have brand-new authors, and +the threshold is calibrated to surface the xz-style pattern, not to fail +the build automatically. +## Output + +The finding surfaces in the rendered diff alongside the other enrichers, +naming the package, its top contributor, and the days-since-first-commit. +A `--debug-calibration` row is also emitted (see [Calibration](#calibration)). + +## Network + +- **Source**: GitHub REST `/repos/.../contributors` + `/commits` (GitLab + and Codeberg equivalents per [Algorithm](#algorithm)). - **Per-request timeout**: 15 seconds. - **Token env vars**: `GITHUB_TOKEN` (Bearer), `GITLAB_TOKEN` - (PRIVATE-TOKEN), `CODEBERG_TOKEN` (Authorization: token). All optional; - missing token means unauthenticated requests (fine for low volume). + (PRIVATE-TOKEN), `CODEBERG_TOKEN` (Authorization: Bearer). All optional; + a missing token means unauthenticated requests (fine for low volume). `GITHUB_TOKEN` bumps the unauthenticated 60/hr cap to 5000/hr. - **No `octocrab`**: the `octocrab` crate would pull in tokio + ~70 - transitive crates. Hand-rolled `ureq` GETs + a 25-line ISO-8601 - parser keep the bomdrift binary under our 5 MB target. Same - constraint applies to the GitLab and Codeberg paths. + transitive crates. Hand-rolled `ureq` GETs + a 25-line ISO-8601 parser + keep the bomdrift binary under our 5 MB target. The same constraint + applies to the GitLab and Codeberg paths. +- **Per-component caching** within a single run (see [Algorithm](#algorithm)). + +## Disabling + +`--no-maintainer-age` skips the entire enricher (no GitHub, GitLab, or +Codeberg API calls): + +```bash +bomdrift diff before.json after.json --no-maintainer-age +``` + +Required for: + +- Offline runs and tests. +- CI environments where tokens are unset and unauthenticated rate + limits are too low for the diff being analyzed. +- Smoke tests of the deterministic offline signals. ## Calibration @@ -105,43 +132,25 @@ samples can guide tuning: maintainer-age|||90 ``` -## Disabling - -`--no-maintainer-age` skips the entire enricher (no GitHub, GitLab, or -Codeberg API calls). Required for: - -- Offline runs and tests. -- CI environments where tokens are unset and unauthenticated rate - limits are too low for the diff being analyzed. -- Smoke tests of the deterministic offline signals. - -```bash -bomdrift diff before.json after.json --no-maintainer-age -``` - -## Severity - -Always informational. The maintainer-age signal **never** trips -`--fail-on critical-cve`; it surfaces only under `--fail-on any`. The -intent is for human review, not gating: many legitimate packages have -brand-new authors, and the threshold is calibrated to surface the -xz-style pattern, not to fail the build automatically. - -## Calibration roadmap (v0.9.6+ status) +### Calibration roadmap (v0.9.6+ status) Past calibration backlog and how each item resolved: -- **Tunable threshold flag** — *shipped in v0.9.6* as - `--young-maintainer-days `. See [Calibration](#calibration) above. -- **Multi-signal fusion** — combine maintainer-age with the registry +- **Tunable threshold flag** is *shipped in v0.9.6* as + `--young-maintainer-days ` (above). +- **Multi-signal fusion**: combine maintainer-age with the registry enricher's "recently-published" or "maintainer-set-changed" findings to narrow the false-positive rate. The signals all surface in the same diff today; explicit fusion in a single composite finding is a v1.0 follow-up. -- **GraphQL pagination** — *decided not to pursue.* Adds a token - requirement (the GraphQL endpoint always wants auth) for one - saved round-trip per component. The `last-page` REST trick is - documented as the canonical approach; see the module doc-comment in - `src/enrich/maintainer.rs` for the rationale. +- **GraphQL pagination** is *decided not to pursue*. It adds a token + requirement (the GraphQL endpoint always wants auth) for one saved + round-trip per component. The `last-page` REST trick is documented as + the canonical approach; see the module doc-comment in + `src/enrich/maintainer/mod.rs` for the rationale. + +## See also -See [Roadmap](../roadmap.md) for the current backlog. +- [Registry metadata (npm/PyPI/crates.io)](./registry.md) +- [Roadmap](../roadmap.md) +- [Enrichers overview](./overview.md) diff --git a/docs/src/enrichers/osv-cve.md b/docs/src/enrichers/osv-cve.md index b903fbc..7ab8a0e 100644 --- a/docs/src/enrichers/osv-cve.md +++ b/docs/src/enrichers/osv-cve.md @@ -5,7 +5,30 @@ for added and version-bumped components, populating the **Vulnerabilities** section in the rendered output with advisory IDs (CVE, GHSA, MAL, etc.) and per-advisory severity. -## Two-stage lookup +## Why this signal + +Published advisories are the broadest supply-chain signal: a newly added +or version-bumped dependency may pull in a known-vulnerable release. OSV +unifies advisories across ecosystems, so a single lookup covers npm, PyPI, +Cargo, Maven, and more. + +### Why OSV.dev specifically? + +- **Cross-ecosystem unification.** OSV merges npm advisories from GHSA, + PyPI advisories from PyPA, Cargo advisories from RustSec, Maven + advisories from GHSA, etc. into a single API, so bomdrift doesn't + need ecosystem-specific clients. +- **Open API, no key required.** Every consumer of the `/v1/querybatch` + endpoint gets the same data without registration overhead. +- **Public schema.** The response shape is documented at + [ossf.github.io/osv-schema/](https://ossf.github.io/osv-schema/), so + bomdrift can reason about the shape without depending on an API + client crate that drags in tokio. + +## Algorithm + +A two-stage lookup: a batched query for advisory IDs, then a per-id fetch +for severity. ### Stage 1: `/v1/querybatch` @@ -27,15 +50,53 @@ follow-up GET to populate severity. Severity is sourced (in order): across the OSV corpus. 2. **Highest CVSS_V3 vector score** from the `severity[]` array, mapped to a label by the standard CVSS-v3 severity rating - (Critical ≥ 9.0, High ≥ 7.0, Medium ≥ 4.0, Low ≥ 0.1). + (Critical >= 9.0, High >= 7.0, Medium >= 4.0, Low >= 0.1). 3. **`Severity::None`** when neither shape is present. These advisories render with a `none` severity label and don't trip `--fail-on critical-cve`. -## On-disk severity cache +## Threshold + +OSV findings surface regardless of severity; the gate is `--fail-on`: + +| Threshold | Trips when... | +|---|---| +| `none` | Never. | +| `cve` | Any vuln finding present (regardless of severity). | +| `critical-cve` | Any finding with `severity >= High` (covers HIGH and CRITICAL). | +| `typosquat` | Any typosquat finding; OSV findings do not trip it. | +| `license-change` | Any same-version license change; OSV findings do not trip it. | +| `any` | Any finding of any kind, plus license-changed-without-version-bump. | + +The `critical-cve` name covers HIGH-or-CRITICAL because CRITICAL alone +is rare in the GHSA tagging and many actively-exploited advisories ship +as HIGH. The threshold name stays stable; the threshold value covers +the actionable bucket. + +## Output + +The enricher populates the **Vulnerabilities** section with advisory IDs +(CVE, GHSA, MAL, etc.) and per-advisory severity across the markdown, +terminal, JSON, and SARIF renderers. The [EPSS](./epss.md) and +[CISA KEV](./kev.md) enrichers layer probability and exploitation badges +onto the same advisories. + +## Network + +- **Source**: OSV.dev `/v1/querybatch` + `/v1/vulns/{id}`. +- **Per-request timeout**: 15 seconds. +- **No authentication**: both endpoints are unauthenticated public APIs. +- **User-Agent**: `bomdrift/` so the OSV team can attribute + traffic if needed. +- **Failures warn and continue**: a network mishap (DNS, timeout, 5xx) + emits a single stderr warning and the diff renders without the + Vulnerabilities section. The exit code remains 0 unless `--fail-on` + was set and a previously-cached vuln tripped it. + +### On-disk severity cache -Stage-2 lookups are N+1 in the worst case — one query per unique -advisory ID. bomdrift caches stage-2 responses on disk at +Stage-2 lookups are N+1 in the worst case, one query per unique advisory +ID. bomdrift caches stage-2 responses on disk at `/bomdrift/osv/.json` with a 24h TTL. ``` @@ -55,7 +116,7 @@ Each cache file looks like: } ``` -### Cache behavior +Cache behavior: - **Cache hits log nothing.** A successful 24h-fresh hit is silent. - **Cache misses are silent too.** Each miss issues a network fetch @@ -71,21 +132,18 @@ Each cache file looks like: and stale-severity risk (a published severity correction after 24h is rare and the renderer's contract is "best effort"). -### `--no-osv-cache` +Pass `--no-osv-cache` to force fresh fetches even within the 24h window +(it costs N+1 fetches per run; use sparingly). + +## Disabling -For paranoid reruns where you want fresh fetches even within the 24h -window: +`--no-osv` skips the entire OSV pipeline (both stages, no cache writes): ```bash -bomdrift diff before.json after.json --no-osv-cache +bomdrift diff before.json after.json --no-osv ``` -The cache itself is purely an optimization — the bypass flag always -works, it just costs N+1 fetches per run. Use sparingly. - -## `--no-osv` (offline mode) - -Skip the entire OSV pipeline (both stages, no cache writes). Use for: +Use for: - Tests and example scenarios where determinism matters more than freshness. @@ -93,47 +151,15 @@ Skip the entire OSV pipeline (both stages, no cache writes). Use for: - Quick smoke tests of the change-shape signals without the network latency. -```bash -bomdrift diff before.json after.json --no-osv -``` +## Calibration -## Severity → `--fail-on` mapping +- `--cache-ttl-hours ` (v0.9.6+) overrides the default 24h severity + cache TTL. +- `--no-osv-cache` bypasses the on-disk severity cache for a run without + disabling the lookup itself. -| Threshold | Trips when... | -|---|---| -| `none` | Never. | -| `cve` | Any vuln finding present (regardless of severity). | -| `critical-cve` | Any finding with `severity >= High` (covers HIGH and CRITICAL). | -| `typosquat` | Any typosquat finding; OSV findings do not trip it. | -| `license-change` | Any same-version license change; OSV findings do not trip it. | -| `any` | Any finding of any kind, plus license-changed-without-version-bump. | - -The `critical-cve` name covers HIGH-or-CRITICAL because CRITICAL alone -is rare in the GHSA tagging and many actively-exploited advisories ship -as HIGH. The threshold name stays stable; the threshold value covers -the actionable bucket. +## See also -## Network behavior - -- **Per-request timeout**: 15 seconds. -- **No authentication**: OSV.dev's `/v1/querybatch` and `/v1/vulns/{id}` - endpoints are both unauthenticated public APIs. -- **User-Agent**: `bomdrift/` so the OSV team can attribute - traffic if needed. -- **Failures warn and continue**: a network mishap (DNS, timeout, 5xx) - emits a single stderr warning and the diff renders without the - Vulnerabilities section. The exit code remains 0 unless `--fail-on` - was set and a previously-cached vuln tripped it. - -## Why OSV.dev specifically? - -- **Cross-ecosystem unification.** OSV merges npm advisories from GHSA, - PyPI advisories from PyPA, Cargo advisories from RustSec, Maven - advisories from GHSA, etc. into a single API, so bomdrift doesn't - need ecosystem-specific clients. -- **Open API, no key required.** Every consumer of the `/v1/querybatch` - endpoint gets the same data without registration overhead. -- **Public schema.** The response shape is documented at - [ossf.github.io/osv-schema/](https://ossf.github.io/osv-schema/), so - bomdrift can reason about the shape without depending on an API - client crate that drags in tokio. +- [EPSS](./epss.md) +- [CISA KEV](./kev.md) +- [Enrichers overview](./overview.md) diff --git a/docs/src/enrichers/overview.md b/docs/src/enrichers/overview.md index ddde47b..a1ae4be 100644 --- a/docs/src/enrichers/overview.md +++ b/docs/src/enrichers/overview.md @@ -15,7 +15,7 @@ without that enricher's findings. | [EPSS](./epss.md) | FIRST.org `/api/v1/epss` | yes | on | `--no-epss` | `--cache-ttl-hours`; `--fail-on-epss <0.0–1.0>` | | [CISA KEV](./kev.md) | CISA known-exploited catalog | yes | on | `--no-kev` | `--cache-ttl-hours`; `--fail-on kev` | | [Typosquat](./typosquat.md) | Embedded top-N lists, optional XDG cache | no | on | (none — pure compute) | `--typosquat-similarity-threshold` (v0.9.6) | -| [Multi-major version jump](./version-jump.md) | The diff itself | no | on | (none — pure compute) | (hard-coded `MIN_MAJOR_DELTA = 2` — see chapter for rationale) | +| [Multi-major version jump](./version-jump.md) | The diff itself | no | on | (none — pure compute) | `--multi-major-delta ` (v0.9.7; default 2) | | [Maintainer age](./maintainer-age.md) | GitHub REST `/repos/.../contributors` + `/commits` | yes | on | `--no-maintainer-age` | `--young-maintainer-days` (v0.9.6) | | [Registry metadata](./registry.md) | npm / PyPI / crates.io public APIs | yes | on (v0.9+) | `--no-registry` | `--recently-published-days`; `--cache-ttl-hours` | | [License policy](../license-policy.md) | SBOM `licenses` field + SPDX expression eval | no | on | (configured by allow/deny lists) | `--allow-licenses`, `--deny-licenses`, `--allow-exception`, `--deny-exception` | diff --git a/docs/src/enrichers/registry.md b/docs/src/enrichers/registry.md index 2ed082a..854cef8 100644 --- a/docs/src/enrichers/registry.md +++ b/docs/src/enrichers/registry.md @@ -14,7 +14,34 @@ and surfaces three kinds of finding: for the new version differs from the maintainer set listed for the old version. Classic xz / Jia Tan precursor. -## Sources +## Why this signal + +Recently-published, deprecated, and maintainer-set-changed are the npm +Shai-Hulud-style worm precursors: an attacker republishes a package (often +under a fresh maintainer set) within days of a takeover. Catching the +metadata shift at diff time surfaces the swap before the new version is +installed. + +### Why npm-only for maintainer-set-changed? + +PyPI and crates.io don't expose a clean "maintainers per version" +view in their public REST API: + +- **PyPI**: the `info.maintainer` and `info.author` fields are + free-text and inconsistent across releases. There's no historical + record per release. +- **crates.io**: `owners` is package-level, not version-level, so we + can't tell which owners had publish rights at the time of an + individual version. + +When the upstream APIs gain a per-version maintainer view we'll +extend the enricher; a future-version follow-up. + +## Algorithm + +For each newly-added component (and npm version-changed components for the +maintainer-set check), bomdrift fetches the package's registry metadata and +evaluates the three checks above. | Ecosystem | URL | Headers | |---|---|---| @@ -22,25 +49,37 @@ and surfaces three kinds of finding: | PyPI | `https://pypi.org/pypi//json` | — | | crates.io | `https://crates.io/api/v1/crates/` | `User-Agent: bomdrift/0.9.0 (https://github.com/Metbcy/bomdrift)` (required by crates.io) | -## Disk cache +## Threshold -Per ecosystem under `/bomdrift/registry//.json`, -24-hour TTL, atomic temp-file + rename writes. Mirrors the OSV / EPSS -/ KEV cache shape. +The recently-published check uses a day threshold (`--recently-published-days`, +default 14). The deprecated and maintainer-set-changed checks are boolean: +a package is flagged or it is not. The exit-2 gates are `--fail-on +recently-published` and `--fail-on deprecated`. -## Best-effort +## Output -A registry timeout, parse error, or unsupported ecosystem returns -`Ok` with no findings. Diff rendering NEVER blocks on registry -responses. +- **Markdown**: three new sections, "Recently published", "Deprecated + upstream", "Maintainer set changed (npm)", in the per-category area. +- **JSON**: `enrichment.recently_published`, `enrichment.deprecated`, + `enrichment.maintainer_set_changed`. +- **SARIF**: rules `bomdrift.recently-published`, `bomdrift.deprecated`, + `bomdrift.maintainer-set-changed` with stable + `partialFingerprints.primaryHash/v1`. +- **Calibration** rows (`--debug-calibration`): + `recently-published|||14`, + `deprecated|||any`, + `maintainer-set-changed|||1`. -## Calibration +## Network -- `--recently-published-days ` — override the default 14-day - threshold. Set `--recently-published-days 0` to disable that check - while keeping deprecation / maintainer-set-changed. -- `--cache-ttl-hours ` (v0.9.6+) — overrides the default 24h disk - cache TTL for the per-ecosystem registry caches. +- **Sources**: npm / PyPI / crates.io public APIs (see the table under + [Algorithm](#algorithm)). +- **Caching**: per ecosystem under + `/bomdrift/registry//.json`, 24-hour TTL, atomic + temp-file + rename writes. Mirrors the OSV / EPSS / KEV cache shape. +- **Best-effort**: a registry timeout, parse error, or unsupported + ecosystem returns `Ok` with no findings. Diff rendering NEVER blocks + on registry responses. ## Disabling @@ -51,39 +90,23 @@ bomdrift diff before.json after.json --no-registry Disables all three checks at once. Equivalent to `[diff] no_registry = true` in `.bomdrift.toml`. -## Flags +The full flag set: -- `--no-registry` — skip all three checks. -- `--recently-published-days ` — see [Calibration](#calibration). -- `--fail-on recently-published`, `--fail-on deprecated` — exit-2 +- `--no-registry` skips all three checks. +- `--recently-published-days ` (see [Calibration](#calibration)). +- `--fail-on recently-published`, `--fail-on deprecated` are exit-2 thresholds. -## Output - -- **Markdown**: three new sections — "Recently published", - "Deprecated upstream", "Maintainer set changed (npm)" — in the - per-category area. -- **JSON**: `enrichment.recently_published`, - `enrichment.deprecated`, `enrichment.maintainer_set_changed`. -- **SARIF**: rules `bomdrift.recently-published`, - `bomdrift.deprecated`, `bomdrift.maintainer-set-changed` with - stable `partialFingerprints.primaryHash/v1`. -- **Calibration** rows (`--debug-calibration`): - `recently-published|||14`, - `deprecated|||any`, - `maintainer-set-changed|||1`. - -## Why npm-only for maintainer-set-changed? +## Calibration -PyPI and crates.io don't expose a clean "maintainers per version" -view in their public REST API: +- `--recently-published-days ` overrides the default 14-day threshold. + Set `--recently-published-days 0` to disable that check while keeping + deprecation / maintainer-set-changed. +- `--cache-ttl-hours ` (v0.9.6+) overrides the default 24h disk cache + TTL for the per-ecosystem registry caches. -- **PyPI**: the `info.maintainer` and `info.author` fields are - free-text and inconsistent across releases. There's no historical - record per release. -- **crates.io**: `owners` is package-level, not version-level, so we - can't tell which owners had publish rights at the time of an - individual version. +## See also -When the upstream APIs gain a per-version maintainer view we'll -extend the enricher; a future-version follow-up. +- [Maintainer age signal](./maintainer-age.md) +- [Typosquat detection](./typosquat.md) +- [Enrichers overview](./overview.md) diff --git a/docs/src/enrichers/typosquat.md b/docs/src/enrichers/typosquat.md index 66ca673..0c05671 100644 --- a/docs/src/enrichers/typosquat.md +++ b/docs/src/enrichers/typosquat.md @@ -5,14 +5,14 @@ suspiciously close to a popular package in the same ecosystem. v0.4 covers **npm**, **PyPI**, **Cargo**, **Maven**, **Go**, **RubyGems**, **NuGet**, and **Composer** with rules tuned per ecosystem. -## The signal +## Why this signal Typosquatting is a real and recurring supply-chain attack pattern: -- The 2024 PyPI campaign that registered `colorama-0.4.7` — note the - trailing zero — to drop a credential stealer. -- The Mar 2026 axios incident's `plain-crypto-js@4.2.1` — a typo of the - legitimate `crypto-js` — used to exfiltrate via WAVESHAPER.V2. +- The 2024 PyPI campaign that registered `colorama-0.4.7` (note the + trailing zero) to drop a credential stealer. +- The Mar 2026 axios incident's `plain-crypto-js@4.2.1` (a typo of the + legitimate `crypto-js`) used to exfiltrate via WAVESHAPER.V2. - Sustained npm `lodash` lookalikes (`loadash`, `loadsh`, `loadshes`) through 2024–2026. @@ -23,6 +23,18 @@ prefix or suffix. The defender's task is to flag the candidate at PR review time, before `npm install` or `pip install` runs the malicious code. +### Reputational care + +The renderer wording is intentional: + +> *X is similar to Y* + +never *X is a typosquat of Y*. Flagging a legitimate package as a +malicious squat in a public PR comment is real reputational harm to +the package author. The structural similarity is observable; intent +is not. The human reviewing the PR is the analyst making the +determination. + ## Algorithm The core scoring is **Jaro-Winkler similarity** with a **suffix-containment @@ -37,7 +49,7 @@ below). | npm | lowercase | `-`, `_`, `.`, `/` | Jaro-Winkler + suffix boost | | PyPI | PEP 503 (lowercase, `-`/`_`/`.` collapse) | `-`, `_`, `.` | Jaro-Winkler + suffix boost | | Cargo | lowercase | `-` | Jaro-Winkler + suffix boost | -| Maven | lowercase | (n/a) | Levenshtein ≤ 2 on `artifactId` only | +| Maven | lowercase | (n/a) | Levenshtein <= 2 on `artifactId` only | | Go | lowercase | `-`, `/` | Jaro-Winkler on **last path segment** | | Gem | lowercase | `-`, `_` | Jaro-Winkler + suffix boost | | NuGet | lowercase (case-insensitive per spec) | `.` | Jaro-Winkler + suffix boost | @@ -55,7 +67,7 @@ below). specific separator sets so PyPI's `-`/`_`/`.` interchange doesn't leak into npm's wider set. 3. **Suffix containment with a substantial added prefix → boost.** When - the candidate ends with the legit name (length ≥ 5) AND the added + the candidate ends with the legit name (length >= 5) AND the added prefix is longer than 3 characters, the score is boosted to at least 0.95. This catches the deceptive `plain-crypto-js` pattern that pure JW alone misses (the long prefix kills base similarity). @@ -70,7 +82,7 @@ Go and Composer share an additional structural rule: the user-visible coordinate has a stable, long prefix (Go's `host/owner/`, Composer's `vendor/`) that's duplicated across many legitimate packages. Including the prefix in Jaro-Winkler scoring would inflate similarity past -anything useful — every Spring artifact would score 0.95+ against every +anything useful: every Spring artifact would score 0.95+ against every other Spring artifact, every Symfony package against every other Symfony package. @@ -93,34 +105,17 @@ JW similarity threshold. Maven coordinates are `groupId:artifactId`. The shared `groupId` prefix is often very long (`org.springframework.boot:`, `com.fasterxml.jackson.core:`) and would inflate Jaro-Winkler past -anything useful — every Spring artifact would score 0.95+ against +anything useful: every Spring artifact would score 0.95+ against every other Spring artifact. The Maven path skips JW + suffix- -containment entirely and uses **Levenshtein distance ≤ 2 on the +containment entirely and uses **Levenshtein distance <= 2 on the `artifactId` portion** only. `commons-lng3` differs from `commons-lang3` by Levenshtein 1 (insert `a`), so it fires regardless of whether the `groupId` matches. A different-`groupId` republish of an exact `commons-lang3` artifact -does **not** fire — that's a legitimate fork / republish, not a typo. - -## Reputational care - -The renderer wording is intentional: - -> *X is similar to Y* - -— never *X is a typosquat of Y*. Flagging a legitimate package as a -malicious squat in a public PR comment is real reputational harm to -the package author. The structural similarity is observable; intent -is not. The human reviewing the PR is the analyst making the -determination. +does **not** fire, that's a legitimate fork / republish, not a typo. -The CLI / Action exit code reflects this: typosquat findings are -always informational. `--fail-on typosquat` exists for projects that -want to gate on the structural signal explicitly, but it's never the -default. - -## Reference lists +### Reference lists Embedded snapshots ship in the binary: @@ -135,7 +130,7 @@ Embedded snapshots ship in the binary: | `data/nuget-top200.txt` | nuget.org v3 search API `?orderby=totalDownloads` | 200 | | `data/composer-top200.txt` | packagist.org popular categories (curated) | ~190 | -v0.7 expanded the curated Go, Composer, and Gem lists — the +v0.7 expanded the curated Go, Composer, and Gem lists: the ship-with-binary snapshots now cover the CNCF / HashiCorp / gRPC- ecosystem corners of Go, the Symfony / Laravel / Doctrine / testing / Packagist-popular tail of Composer, and the Rails / @@ -146,10 +141,47 @@ header in the data file so future curators can see provenance. Lists are intentionally smaller than `npm-top1k.txt` for the multi- ecosystem ships (v0.2 + v0.4): the algorithm is identical across ecosystems, so a smaller seed still proves the signal end-to-end. Lists -grow in subsequent releases without code changes — only the embedded +grow in subsequent releases without code changes, only the embedded snapshot does. -### Refreshing +## Threshold + +A finding surfaces when its Jaro-Winkler + suffix-boost score is at or +above **0.92** (the Maven Levenshtein path is the exception, see +[Algorithm](#algorithm)). The threshold is tunable via +`--typosquat-similarity-threshold`; see [Calibration](#calibration). + +Typosquat findings are always informational. `--fail-on typosquat` exists +for projects that want to gate on the structural signal explicitly, but +it's never the default. + +## Output + +The renderer surfaces each finding as `X is similar to Y` (see +[Reputational care](#reputational-care) for why the wording is +deliberate). The threshold also appears in `--debug-calibration` rows so +collected samples can guide tuning: + +``` +typosquat|||0.92 +``` + +### False-positive management + +The structural rules + thresholds aim for "no false positives on the +top 1000 of each ecosystem." If you discover a false positive in the +wild: + +1. Add a regression test in `src/enrich/typosquat/tests.rs` showing + the false positive doesn't fire. +2. Open a PR. Tightening the rule (rather than special-casing the + package name) is preferred, it drives a cleaner heuristic. + +## Network + +This signal is pure local computation and never touches the network for +scoring. Reference lists are embedded in the binary, with an optional XDG +cache for refreshed lists: ```bash bomdrift refresh-typosquat # all eight ecosystems @@ -172,6 +204,15 @@ of truth; refreshing those means editing `data/-top*.txt` and rebuilding bomdrift. PRs adding names to the curated lists are welcome. +## Disabling + +Pure compute, no network. There is no `--no-typosquat` flag, disabling +the typosquat enricher would defeat its primary purpose. To suppress +*specific* false-positive findings, hand-curate a per-component baseline +entry; see [Baseline & suppression — Worked example](../baseline.md#worked-example-triaging-a-false-positive). + +To gate exit code on typosquat findings, use `--fail-on typosquat`. + ## Calibration ### `--typosquat-similarity-threshold ` (v0.9.6+) @@ -180,49 +221,22 @@ Default `0.92`, range `[0.0, 1.0]`. Configurable via CLI flag or `[diff] typosquat_similarity_threshold = ` in `.bomdrift.toml`. The threshold applies to the JW + suffix-boost path (npm, PyPI, Cargo, -RubyGems, NuGet, Go, Composer). The Maven Levenshtein-≤-2 path is -hardcoded — Levenshtein distance and JW similarity aren't directly +RubyGems, NuGet, Go, Composer). The Maven Levenshtein-<=-2 path is +hardcoded: Levenshtein distance and JW similarity aren't directly comparable, so a single threshold flag would either over- or under-suppress on Maven. Recommended ranges: -- `0.95` — very strict; only catches near-perfect matches. Good for +- `0.95` is very strict; only catches near-perfect matches. Good for tightening down false positives in monorepos with many internally forked dependencies. -- `0.92` (default) — calibrated against the top-1000-of-each-ecosystem +- `0.92` (default) is calibrated against the top-1000-of-each-ecosystem test corpus to produce zero false positives there. -- `0.85` — lenient; catches softer near-misses at the cost of more +- `0.85` is lenient; catches softer near-misses at the cost of more false positives. Useful for paranoid security review of brand-new PyPI / npm packages. -The threshold also appears in `--debug-calibration` rows so collected -samples can guide tuning: - -``` -typosquat|||0.92 -``` - -## False-positive management - -The structural rules + thresholds aim for "no false positives on the -top 1000 of each ecosystem." If you discover a false positive in the -wild: - -1. Add a regression test in `src/enrich/typosquat.rs::tests` showing - the false positive doesn't fire. -2. Open a PR. Tightening the rule (rather than special-casing the - package name) is preferred — drives a cleaner heuristic. - -## Disabling - -Pure compute, no network. There is no `--no-typosquat` flag — disabling -the typosquat enricher would defeat its primary purpose. To suppress -*specific* false-positive findings, hand-curate a per-component baseline -entry; see [Baseline & suppression — Worked example](../baseline.md#worked-example-triaging-a-false-positive). - -To gate exit code on typosquat findings, use `--fail-on typosquat`. - ## See also - [CLI reference — `--typosquat-similarity-threshold`](../cli-reference.md#--typosquat-similarity-threshold-float) diff --git a/docs/src/enrichers/version-jump.md b/docs/src/enrichers/version-jump.md index 7e46c71..a75d754 100644 --- a/docs/src/enrichers/version-jump.md +++ b/docs/src/enrichers/version-jump.md @@ -4,10 +4,10 @@ Pure-compute, no network, no new dependencies. The version-jump heuristic flags dependency upgrades that cross **two or more** major versions in a single diff (e.g. `1.x → 4.x`). -## Why it's a useful signal +## Why this signal A single major bump (`1 → 2`) is the standard SemVer signal reviewers -already pay attention to — bomdrift does not flag it. **Two or more +already pay attention to, and bomdrift does not flag it. **Two or more majors at once** is the unusual case worth a closer look: - **Takeover swaps**: a maintainer transition followed by a major-version @@ -19,14 +19,13 @@ majors at once** is the unusual case worth a closer look: that silently jump several majors at once and bypass the usual SemVer guard-rails. -Always informational severity — never trips `--fail-on` thresholds -narrower than `any`. - -## Major-version extraction +## Algorithm -Hand-rolled, ~5 lines. We deliberately avoid the `semver` crate: full -SemVer parsing is unnecessary when only the major number is consulted, -and pulling the dep would add transitive weight for no functional gain. +Major-version extraction is hand-rolled, ~5 lines. We deliberately avoid +the `semver` crate: full SemVer parsing is unnecessary when only the major +number is consulted, and pulling the dep would add transitive weight for +no functional gain. A pair is flagged when both versions parse to a major +and the delta is at or above the threshold (default 2). ### Accepted forms (each yields a `Some(major)`) @@ -36,13 +35,53 @@ and pulling the dep would add transitive weight for no functional gain. - `3.0.0+build.123` → 3 (build metadata ignored) - `4` / `4-rc.1` → 4 (no minor required) -### Rejected forms (yield `None`, the pair is skipped — never flagged) +### Rejected forms (yield `None`, the pair is skipped, never flagged) - empty string - non-numeric (`latest`, `nightly`, `main`) -- leading-zero numbers (`01.2.3`) — ambiguous and almost always a sign +- leading-zero numbers (`01.2.3`), ambiguous and almost always a sign of a non-SemVer scheme; safer to skip than misinterpret. +### Examples + +| Before | After | Flagged? | +|---|---|---| +| `1.0.0` | `4.17.21` | yes (1 → 4) | +| `2.34.0` | `4.5.0` | yes (2 → 4) | +| `1.0.0` | `2.0.0` | no (single major bump) | +| `1.0.0` | `1.99.0` | no (no major bump) | +| `latest` | `nightly` | no (skipped, non-numeric) | +| `01.2.3` | `04.0.0` | no (skipped, leading-zero ambiguity) | + +See [`examples/version-jumps/`](https://github.com/Metbcy/bomdrift/tree/main/examples/version-jumps) +for a runnable scenario. + +## Threshold + +The multi-major delta threshold is `2` by default, exposed via +[`--multi-major-delta `](../cli-reference.md#--multi-major-delta-n) +(introduced in v0.9.7); see [Calibration](#calibration). Findings are +always informational severity and never trip `--fail-on` thresholds +narrower than `any`. + +## Output + +A flagged pair surfaces in the rendered diff naming the component and the +before/after majors (e.g. `1 → 4`). The finding is informational and +carries no severity gate of its own. + +## Network + +This signal is pure local computation and never touches the network. + +## Disabling + +There is no `--no-version-jump` flag, the check is pure compute and zero +cost. If you need to gate exit code only on version-jump findings, use +`--fail-on any`. To suppress a specific bump as a known-acceptable, write +a per-component baseline entry; see +[Baseline — When the bump is the false positive](../baseline.md#when-the-bump-is-the-false-positive). + ## Calibration The multi-major delta threshold is exposed as @@ -65,24 +104,8 @@ For per-component carve-outs use a baseline entry instead of dropping the global threshold; see [Baseline — When the bump is the false positive](../baseline.md#when-the-bump-is-the-false-positive). -## Disabling - -There is no `--no-version-jump` flag — pure compute, zero cost. If you -need to gate exit code only on version-jump findings, use `--fail-on -any`. To suppress a specific bump as a known-acceptable, write a -per-component baseline entry — see -[Baseline — When the bump is the false positive](../baseline.md#when-the-bump-is-the-false-positive). - -## Examples +## See also -| Before | After | Flagged? | -|---|---|---| -| `1.0.0` | `4.17.21` | yes (1 → 4) | -| `2.34.0` | `4.5.0` | yes (2 → 4) | -| `1.0.0` | `2.0.0` | no (single major bump) | -| `1.0.0` | `1.99.0` | no (no major bump) | -| `latest` | `nightly` | no (skipped — non-numeric) | -| `01.2.3` | `04.0.0` | no (skipped — leading-zero ambiguity) | - -See [`examples/version-jumps/`](https://github.com/Metbcy/bomdrift/tree/main/examples/version-jumps) -for a runnable scenario. +- [Baseline & suppression](../baseline.md#when-the-bump-is-the-false-positive) +- [CLI reference: `--multi-major-delta`](../cli-reference.md#--multi-major-delta-n) +- [Enrichers overview](./overview.md)