Skip to content

Commit 0f4b09b

Browse files
authored
Expand dependency source classification (#352)
* Expand dependency source classification Classify registry, Git, URL, file, project, and workspace origins from explicit Cargo, Bundler, pub, SwiftPM, and Python detector evidence. Preserve available source revisions, keep unknown evidence unset, and document the supported detector scope and safe exclusions. * Address dependency source review feedback Keep Swift source-control packages eligible for canonical SwiftURL matching, align Cargo lock-only workspace classification with metadata resolution, parse uv revisions from real Git URLs, and avoid empty Bundler metadata allocations. Add focused regression coverage and clarify matcher eligibility documentation.
1 parent 8f272fc commit 0f4b09b

41 files changed

Lines changed: 800 additions & 49 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

dev-docs/ARCHITECTURE.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -385,6 +385,21 @@ Workspace/reactor detectors (npm and pnpm lockfile, cargo, maven) emit one `Grap
385385

386386
**First-party packages are inventory, not enrichment targets** (`sdk.NodeIsEnrichable`). Application-typed nodes — workspace members, reactor modules, the project's own package — are absent from public advisory/registry sources, so querying OSV / deps.dev / scorecard / grype for them wastes lookups and risks coincidental name matches (a workspace member named like a real npm package would adopt its advisories). The predicate mirrors `NodeIsDiffable` and gates the two selection chokepoints (`matchers.RegistryPackagesForGraph`, the OSV matcher's graph iteration) plus external grype's result mapping. External grype's SBOM *input* is deliberately not filtered: `sbom.FromDepGraph` is shared with user-facing SBOM generation, where first-party components must remain visible — so first-party matches are dropped when grype results map back into the registry. First-party entries stay in the `packages` collection and SBOMs, just unenriched; external plugin matchers (ClearlyDefined, EOL) are expected to adopt the same predicate.
387387

388+
**Dependency source classification belongs to detectors.** Source is an
389+
occurrence fact, so the detector that reads the manifest, lockfile, or build
390+
tool output owns it. The engine and package auditor consume the canonical
391+
`sdk.Dependency.Source` value but never infer one from an ecosystem, package
392+
name, PURL, or repository metadata. Detectors classify only explicit evidence:
393+
for example Cargo `registry+` and `git+` sources, Bundler `GEM`/`GIT`/`PATH`
394+
sections, pub lock sources, SwiftPM pin kinds, Python direct-URL metadata, and
395+
Python lock source tables. When a format does not retain the selected origin,
396+
the source stays unknown. This trades some source-change coverage for avoiding
397+
false provenance claims and keeps external protocol-v1 detectors compatible.
398+
Source and matcher eligibility are related but not identical: SwiftPM remote
399+
source control is classified as Git, while remaining eligible because the
400+
repository URL is the canonical SwiftURL identity used for vulnerability
401+
matching.
402+
388403
### Decision: Bun text lockfiles are native; binary lockfiles degrade explicitly
389404

390405
`bun-detector` parses JSONC `bun.lock` versions 0 and 1 directly in Go. The parser removes comments and trailing commas with a string-aware state machine, inventories package tuples before constructing edges, models workspace roots as application nodes, and runs the shared Node relationship finalizer before per-workspace graph partitioning. Bun workspace entries therefore use the same multi-entry `sdk.GraphContainer` contract described above. The lockfile path never invokes or installs Bun, so committed text lockfiles remain deterministic and offline.

dev-docs/MODELS.md

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -128,6 +128,14 @@ Key helpers:
128128
- `sdk.RelationshipForPath(path)` — preserve an explicit relationship or derive direct/transitive from a root-to-target path.
129129
- `dep.RegistryMatchEligible()` — classify whether this occurrence may be sent to external registry enrichment.
130130

131+
`Dependency.Source` is occurrence evidence, not a guess based on package name
132+
or ecosystem. A detector sets it only when the manifest, lockfile, or build
133+
tool output proves the origin. Cargo, Bundler, the JavaScript package managers,
134+
pub, SwiftPM, and the pip, Pipenv, Poetry, and uv Python paths currently expose
135+
that evidence. Formats that do not retain the selected feed or source leave the
136+
field empty. An empty source remains eligible for matching for protocol-v1
137+
compatibility, but it cannot create a source-change finding in a diff.
138+
131139
An `unknown` relationship means that the package was present in the owning
132140
manifest but its parent could not be recovered. The component root is attached
133141
beneath the manifest/application root so it continues through matching,
@@ -137,7 +145,7 @@ for protocol-v1 plugins and is derived from graph structure by consumers.
137145

138146
Dependencies **do not** carry `Licenses`, `Vulnerabilities`, or `Scorecard` fields. Detection-time licenses ride along in metadata; matching-stage data lives on the registry package.
139147

140-
Registry matching eligibility is occurrence-based. Ordinary registry releases are eligible even when their `ResolvedURL` points at a custom registry or mirror. First-party/manifest nodes and occurrences sourced from project, workspace, link/file, Git, or arbitrary URL references are ineligible but remain in the complete graph and package registry for analysis, auditing, diff, SBOM, and output. Application type alone is not an ownership signal: an application artifact imported from an SBOM remains eligible unless it is marked first-party or has a non-registry source. An omitted source remains eligible for protocol-v1 and legacy detector compatibility. Before any built-in or external matcher runs, the engine passes it a cloned graph containing only eligible occurrences and eligible-to-eligible edges; the original graph and full registry continue to later stages unchanged.
148+
Registry matching eligibility is occurrence-based. Ordinary registry releases are eligible even when their `ResolvedURL` points at a custom registry or mirror. First-party/manifest nodes and occurrences sourced from project, workspace, link/file, Git, or arbitrary URL references are normally ineligible but remain in the complete graph and package registry for analysis, auditing, diff, SBOM, and output. Swift source-control packages are the exception: their repository URL is the canonical SwiftURL package identity, so Git-sourced Swift packages remain eligible for vulnerability matching. Application type alone is not an ownership signal: an application artifact imported from an SBOM remains eligible unless it is marked first-party or has a non-registry source. An omitted source remains eligible for protocol-v1 and legacy detector compatibility. Before any built-in or external matcher runs, the engine passes it a cloned graph containing only eligible occurrences and eligible-to-eligible edges; the original graph and full registry continue to later stages unchanged.
141149

142150
## `sdk.Package` — registry artifact (matching)
143151

docs/auditors/package.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -84,7 +84,7 @@ still downgrades a configured failure.
8484
## Limitations
8585

8686
- **Names, not behavior.** This auditor cannot tell whether a package is malicious — only whether its name is denied or resembles a protected one. Pair it with the vulnerability auditor for content risk.
87-
- **Source-change checks currently cover JavaScript package managers.** npm, pnpm, Yarn, and Bun detectors report the source details needed by this check. Other detectors do not yet provide enough source data, so this policy cannot report their source changes.
87+
- **Source-change checks need detector evidence.** Cargo, Bundler, npm, pnpm, Yarn, Bun, pub, SwiftPM, and the pip, Pipenv, Poetry, and uv Python paths report source details when their lockfile or tool output proves the origin. Other detectors leave the source unknown when their input cannot distinguish a registry package from another origin.
8888
- **Only Git and URL moves are review signals.** Moves to project, workspace, or file sources remain informational, even when registry matching eligibility changes.
8989
- **Source-change policy is diff-only.** `--fail-on source-change` is accepted by other audited commands but cannot match because they do not produce dependency detail transitions.
9090
- **Source changes need context.** Git and URL sources can be intentional. The warning asks a reviewer to confirm the new origin and pinned reference.

docs/detectors/ecosystems/dart/pub.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,3 +16,8 @@ Bomly uses this chain when it finds `pub` evidence.
1616
| Install-first support | No |
1717
| Remediation hints | None |
1818
| Native command hints | `dart`, `syft for bomly-lite` |
19+
20+
## Source details
21+
22+
Bomly reads hosted, Git, and path sources from `pubspec.lock` or native pub
23+
output. An unrecognized source type stays unknown.

docs/detectors/ecosystems/python/pip.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -102,6 +102,12 @@ pip-compile --constraint constraints.txt requirements.in
102102

103103
Re-scan.
104104

105+
## Source details
106+
107+
Pinned packages in `requirements.lock` are registry packages. On the inspection
108+
path, Bomly uses pip's `direct_url` record to identify Git, URL, and local-file
109+
installs. Missing or malformed source data stays unknown.
110+
105111
## Reachability (experimental)
106112

107113
> **Experimental.** Reachability is opt-in via `--analyze`. The feature is stable in shape but may evolve; ecosystem coverage is expanding.

docs/detectors/ecosystems/python/pipenv.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,11 @@ bomly scan --install-first --detectors pipenv-detector \
6666
--install-arg --dev --install-arg --deploy
6767
```
6868

69+
## Source details
70+
71+
Bomly reads registry, Git, URL, and local-file sources from `Pipfile.lock`.
72+
When Pipenv uses pip inspection, Bomly uses pip's `direct_url` record instead.
73+
6974
## Reachability (experimental)
7075

7176
> **Experimental.** Reachability is opt-in via `--analyze`. The feature is stable in shape but may evolve; ecosystem coverage is expanding.

docs/detectors/ecosystems/python/poetry.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,11 @@ urllib3 = ">=2.2.2"
8787

8888
Re-lock and re-scan.
8989

90+
## Source details
91+
92+
Bomly reads registry, Git, URL, and local-directory sources from each
93+
`poetry.lock` package source table. An unrecognized source type stays unknown.
94+
9095
## Reachability (experimental)
9196

9297
> **Experimental.** Reachability is opt-in via `--analyze`. The feature is stable in shape but may evolve; ecosystem coverage is expanding.

docs/detectors/ecosystems/python/uv.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,6 +92,11 @@ constraint-dependencies = ["urllib3>=2.2.2"]
9292

9393
Re-lock and re-scan.
9494

95+
## Source details
96+
97+
Bomly reads registry, Git, URL, path, and editable sources from each `uv.lock`
98+
package source table. An entry without source evidence stays unknown.
99+
95100
## Reachability (experimental)
96101

97102
> **Experimental.** Reachability is opt-in via `--analyze`. The feature is stable in shape but may evolve; ecosystem coverage is expanding.

docs/detectors/ecosystems/ruby/bundler.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,12 @@ gem "nokogiri", ">= 1.16.5"
7979

8080
`bundle lock --update nokogiri`. Re-scan.
8181

82+
## Source details
83+
84+
Bomly maps the `GEM`, `GIT`, and `PATH` sections in `Gemfile.lock` to registry,
85+
Git, and local-file sources. These details let `bomly diff` review a gem that
86+
moves to Git.
87+
8288
## Reachability
8389

8490
> **Not yet supported.** Bomly has no Ruby reachability analyzer today. `--analyze` produces `not_applicable` for Ruby packages.

docs/detectors/ecosystems/rust/cargo.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,11 @@ openssl = { git = "https://github.com/sfackler/rust-openssl", tag = "openssl-v0.
8585

8686
`cargo update`. Re-scan.
8787

88+
## Source details
89+
90+
Bomly reads registry, Git, path, and workspace sources from Cargo metadata or
91+
`Cargo.lock`. These details let `bomly diff` review a package that moves to Git.
92+
8893
## Reachability
8994

9095
> **Not yet supported.** Bomly has no Rust reachability analyzer today. `--analyze` produces `not_applicable` for crates.

0 commit comments

Comments
 (0)