feat(sbom): unrecognized licenses export as LicenseRef, not free text (phase 2.4 + SDK v0.9.2)

dev-docs/adr/0035-license-emission-is-validated-not-assumed.md

@@ -53,9 +53,11 @@ than the source said. This is the one deliberate cross-format difference, and
 it is recorded in `docs/SBOM.md`. CycloneDX composes only when a member is
 itself compound, where listing would degrade a real expression to free text.
 
-A set that mixes valid expressions with free text cannot compose without
-producing something that does not parse, so CycloneDX falls back to one entry
-per license and SPDX to the first value.
+A set that mixes valid expressions with free text cannot compose while the
+free text stays free text, so CycloneDX falls back to one entry per license.
+SPDX no longer falls back at all: the unrecognized member becomes a
+`LicenseRef-*`, which is a valid expression element, so the set composes
+whole. See the note below.
 
 SPDX `licenseConcluded` is always `NOASSERTION`. Concluded is the document
 creator's own determination, and Bomly has none to offer: every license it
@@ -92,9 +94,39 @@ what found the parser panic. The auditor is covered by the wrapper's own tests
 rather than by that target, which does not call it: routing the auditor through
 `licenseexpr` is what fixes its crash.
 
-One limitation is knowingly left in place. SPDX has no free-text license field,
-so a single unrecognized value is still written verbatim into
-`licenseDeclared`, which is not a valid SPDX expression. Representing it as a
-`LicenseRef` with a matching `hasExtractedLicensingInfos` entry is the correct
-fix and is tracked separately; it is a different decision from this one, and
-folding it in would change what an unknown license means in the document.
+One limitation was knowingly left in place, and has since been resolved --
+see the note below.
+
+
+## Note (2026-09-05): the SPDX free-text limitation is resolved
+
+This ADR recorded that SPDX has no free-text license field, so an unrecognized
+value was written verbatim into `licenseDeclared` where it is not a valid
+expression, and that representing it as a `LicenseRef` was the correct fix but
+a separate decision. That decision is taken: bomly-cli#410.
+
+An unrecognized value now mints a `LicenseRef-*` and the original text travels
+with the document in `hasExtractedLicensingInfos`. Unlike `NOASSERTION` this
+loses nothing -- ingest reads the text back -- and unlike the old behavior the
+field holds something SPDX defines.
+
+It also retires the mixed-validity fallback this ADR described. A `LicenseRef`
+is a valid expression element, so a set mixing recognized and unrecognized
+values composes whole instead of keeping the first value and dropping the
+rest. That fallback was the quieter half of the defect: it dropped licenses a
+source had actually declared, and said so only in a comment.
+
+Minting is `bomly-sdk/spdxkit`'s, not this repository's. The identifier has to
+be deterministic, collision-free across components assembled without
+coordination, and confined to the characters the SPDX idstring grammar allows;
+`MintLicenseRef` answers all three by hashing the whitespace-normalized text.
+What stays here is the policy this ADR is about -- which values are classified
+how, and that classification is by validation rather than by the field a value
+arrived in.
+
+The body above refers to `internal/licenseexpr`, the CLI-local wrapper that
+contained the parser's panics when this was decided. That package is gone
+(bomly-cli#428): `bomly-sdk/spdxkit` carries the same six functions and the
+same panic guard, and no package under `internal/` may import the parser
+directly. Read those references as naming the kit. The decision this ADR
+records is unchanged -- only where the code that implements it lives.

docs/SBOM.md

@@ -117,7 +117,7 @@ where it came:
 |---|---|---|
 | One SPDX identifier (`MIT`) | `license.id`, spelled canonically | the identifier |
 | A compound expression (`MIT OR Apache-2.0`) | `expression` | the expression |
-| Anything else (`see LICENSE file`) | `license.name`, as free text | the text as-is |
+| Anything else (`see LICENSE file`) | `license.name`, as free text | a `LicenseRef-*` identifier, with the original text in `hasExtractedLicensingInfos` |
 | Nothing | no `licenses` key | `NOASSERTION` |
 
 When a source records several licenses for one package, it is saying which
@@ -343,14 +343,9 @@ Some information necessarily becomes less specific during conversion:
   the PURL.
 - SPDX 2.3 holds one license expression per package, so several licenses are
   composed with `AND` there while CycloneDX lists them (see "How licenses are
-  written" above). A set that mixes real expressions with free text cannot be
-  composed without producing an expression that does not parse; SPDX then
-  keeps the first value, while CycloneDX keeps every license as its own entry.
-- SPDX has no free-text license field. CycloneDX carries an unrecognized
-  license as `license.name`, but SPDX writes it into `licenseDeclared`, where
-  it is not a valid SPDX expression. A strict SPDX consumer may reject such a
-  package. This only affects packages whose declared license is not an SPDX
-  identifier or expression.
+  written" above). Every license is kept either way: a value SPDX cannot hold
+  verbatim becomes a `LicenseRef-*`, which is a valid expression element, so a
+  mixed set composes rather than losing its members.
 
 Before treating a generated file as a release artifact, validate it with the
 standard validator required by the receiving system. Bomly's tests parse every

go.mod

@@ -15,7 +15,7 @@
 	github.com/bomly-dev/bomly-plugin-pyreach-analyzer v0.2.0
 	github.com/bomly-dev/bomly-plugin-scorecard-matcher v0.2.0
 	github.com/bomly-dev/bomly-plugin-syft-detector v0.2.0
-	github.com/bomly-dev/bomly-sdk v0.9.1
+	github.com/bomly-dev/bomly-sdk v0.9.2
 	github.com/charmbracelet/bubbles v1.0.0
 	github.com/charmbracelet/bubbletea v1.3.10
 	github.com/charmbracelet/lipgloss v1.1.0

go.sum

@@ -257,8 +257,8 @@ github.com/bomly-dev/bomly-plugin-scorecard-matcher v0.2.0 h1:VOK4+GfVGukbccabjD
 github.com/bomly-dev/bomly-plugin-scorecard-matcher v0.2.0/go.mod h1:3ux1Su5UCCKeF+wtttQrlw7r+B7sc6UXrK98Ybuq5gw=
 github.com/bomly-dev/bomly-plugin-syft-detector v0.2.0 h1:UlKwTqp+ZWu25leky9J6NITCKDdePpdgJQl/8dInApg=
 github.com/bomly-dev/bomly-plugin-syft-detector v0.2.0/go.mod h1:NVVrSMHkjC3VEDPtCI7+1c4tWBzwI1eve8tynO1pRoM=
-github.com/bomly-dev/bomly-sdk v0.9.1 h1:CqZiJamcaRU//7aq2zF65CxLrGST9wLc1OZDtIePtyw=
-github.com/bomly-dev/bomly-sdk v0.9.1/go.mod h1:7RJLUANK8xHMZ5/r45zYxGyKUHC8yQiVem22yM0MyZw=
+github.com/bomly-dev/bomly-sdk v0.9.2 h1:nVOdoEV5pKzv/4Pze1WXyfdS6o/wnLTBA1Y+VRxvcVw=
+github.com/bomly-dev/bomly-sdk v0.9.2/go.mod h1:Ulgp/NHlOLmbUvbKpuXylPf74LdCA54gpg+syK90Do8=
 github.com/bradleyjkemp/cupaloy/v2 v2.8.0 h1:any4BmKE+jGIaMpnU8YgH/I2LPiLBufr6oMMlVBbn9M=
 github.com/bradleyjkemp/cupaloy/v2 v2.8.0/go.mod h1:bm7JXdkRd4BHJk9HpwqAI8BoAY1lps46Enkdqw6aRX0=
 github.com/bufbuild/protocompile v0.14.1 h1:iA73zAf/fyljNjQKwYzUHD6AD4R8KMasmwa/FBatYVw=

internal/engine/consolidation/consolidation_test.go

@@ -7,17 +7,24 @@ import (
 	"github.com/bomly-dev/bomly-sdk"
 )
 
-// Two spellings of one Python package name -- case and separators -- mint the
-// same identity, so the second insertion folds into the first and the
-// consumer's edges point at one node.
+// Two spellings of one Python package -- name case and separators, and a
+// release-candidate version written in a different case -- mint the same
+// identity, so the second insertion folds into the first and the consumer's
+// edges point at one node.
+//
+// The version half came back with bomly-sdk v0.9.2, which canonicalizes a
+// PyPI version per PEP 440 before minting. It was briefly untrue: v0.9.0
+// stopped lowercasing every version containing a letter, which was right --
+// that rule corrupted Maven's "1.0-SNAPSHOT" -- and cost this fold until the
+// ecosystem-correct rule replaced the blanket one.
 //
 // Normalization moved into the constructor with ADR-0041, which is why this
 // case no longer builds two nodes and then collapses them: the second node
 // never exists.
 func TestEquivalentPythonSpellingsFoldOnInsertion(t *testing.T) {
 	g := sdk.New()
 	root := testnodes.Ref("app", "1.0.0")
-	pyA := testnodes.Dep(sdk.Coordinates{Ecosystem: "python", Name: "Requests_Toolbelt", Version: "1.0.0rc1"})
+	pyA := testnodes.Dep(sdk.Coordinates{Ecosystem: "python", Name: "Requests_Toolbelt", Version: "1.0.0RC1"})
 	pyB := testnodes.Dep(sdk.Coordinates{Ecosystem: "python", Name: "requests-toolbelt", Version: "1.0.0rc1"})
 
 	const want = "pkg:pypi/requests-toolbelt@1.0.0rc1"
@@ -52,38 +59,6 @@ func TestEquivalentPythonSpellingsFoldOnInsertion(t *testing.T) {
 	}
 }
 
-// Version case is NOT folded, and this pins that rather than leaving it
-// unstated.
-//
-// Identity spelling is packageurl-go's to decide (it is the library that owns
-// canonical package URLs), and it case-folds a version for exactly one type,
-// huggingface. The SDK used to lowercase every version containing a letter,
-// which folded this pair by accident while corrupting Maven's "1.0-SNAPSHOT"
-// into "1.0-snapshot" -- a different version, since Maven versions are case
-// sensitive. Dropping the blanket rule fixed Maven and cost this fold.
-//
-// PyPI itself does treat the two as one release: PEP 440 normalizes version
-// case, so "1.0.0RC1" and "1.0.0rc1" name the same distribution and cannot
-// both exist. Folding them therefore wants a PEP 440 normalizer applied to
-// pypi coordinates before the identity is minted -- in the SDK, where every
-// producer reaches it, and with a real dependency rather than a hand-written
-// version grammar. That is bomly-dev/bomly-sdk#39. Until it lands, the two
-// spellings are two identities, and this test fails when that changes, which
-// is the point.
-func TestPythonVersionCaseIsNotFoldedYet(t *testing.T) {
-	upper := testnodes.Dep(sdk.Coordinates{Ecosystem: "python", Name: "requests-toolbelt", Version: "1.0.0RC1"})
-	lower := testnodes.Dep(sdk.Coordinates{Ecosystem: "python", Name: "requests-toolbelt", Version: "1.0.0rc1"})
-	if upper.NodeID() == lower.NodeID() {
-		t.Fatalf("identities folded to %q; if a PEP 440 normalizer landed, fold this case back into "+
-			"TestEquivalentPythonSpellingsFoldOnInsertion and delete this test", upper.NodeID())
-	}
-	// The version each carries is the one its coordinates stated: the
-	// identity is authoritative for the field, and it preserved the spelling.
-	if upper.Version != "1.0.0RC1" || lower.Version != "1.0.0rc1" {
-		t.Fatalf("versions = %q and %q; want each preserved as written", upper.Version, lower.Version)
-	}
-}
-
 // A scoped npm name is canonicalized into namespace and name by the same
 // constructor gate, so the identity is percent-encoded exactly as the purl
 // spec requires.

internal/graphview/graphview.go

@@ -18,35 +18,25 @@
 // This is a leaf: it imports the SDK and nothing else from this repository, so
 // the codec, the renderers, and the TUI can all reach it without any of them
 // depending on each other.
+//
+// PurlFor is now a one-line delegation to sdk.NodePURL: the SDK took the
+// projection in v0.9.2 (bomly-dev/bomly-sdk#43), which is where it belongs.
+// ChildrenAmong and TopLevelParentIDs stay here -- what a document may name
+// and what counts as a top-level parent are the CLI's decisions, not the
+// model's.
 package graphview
 
 import sdk "github.com/bomly-dev/bomly-sdk"
 
 // PurlFor returns the package URL a node publishes, or "" when it has none.
 //
-// The kinds answer differently and the difference matters: a dependency
-// node's ID *is* its canonical package URL, a module's ID is the structural
-// "module:<path>#<purl>" grammar with its package URL in a separate field,
-// and a manifest has no package URL at all -- it is a file. Publishing NodeID
-// in a field consumers parse as a purl hands them a value that is not one.
-//
-// The deepest home for this is the SDK's GraphNode, which owns what a node
-// means (ADR-0040); it is here until bomly-dev/bomly-sdk#43 ships.
+// It delegates to sdk.NodePURL, which is where this belongs (ADR-0040): the
+// SDK owns what a node means, and the three kinds answer this differently.
+// The CLI carried the projection while bomly-dev/bomly-sdk#43 was open, and
+// v0.9.2 closed it -- the wrapper stays only so callers here read one name
+// alongside ChildrenAmong and TopLevelParentIDs, which remain CLI policy.
 func PurlFor(node sdk.GraphNode) string {
-	switch typed := node.(type) {
-	case *sdk.DependencyNode:
-		if typed == nil {
-			return ""
-		}
-		return typed.NodeID()
-	case *sdk.ModuleNode:
-		if typed == nil {
-			return ""
-		}
-		return typed.PURL()
-	default:
-		return ""
-	}
+	return sdk.NodePURL(node)
 }
 
 // ChildrenAmong names a node's children among the IDs a document actually

internal/sbom/codec_fuzz_test.go

@@ -175,8 +175,24 @@ func FuzzSPDXLicenseValue(f *testing.F) {
 		// may panic on any value a source can produce.
 		licenses := []License{{Value: value}, {SPDXExpression: value}}
 		_ = cycloneDXLicenses(licenses)
-		if got := spdxLicenseValue(licenses); got == "" {
+		got, extracted := spdxLicenseValue(licenses)
+		if got == "" {
 			t.Fatalf("spdx license value must never be empty, got %q for %q", got, value)
 		}
+		// Whatever the value was, the field SPDX will hold has to be
+		// something SPDX can hold: an expression, NOASSERTION, or references
+		// the document also carries the text for. A free-text value that
+		// reached the field verbatim is the defect #410 fixed.
+		if got != "NOASSERTION" && !spdxkit.Valid(got) {
+			t.Fatalf("license field %q does not parse as an SPDX expression, from %q", got, value)
+		}
+		for _, entry := range extracted {
+			if !entry.Valid() {
+				t.Fatalf("minted reference %q does not match its text %q", entry.RefID, entry.Text)
+			}
+			if !strings.Contains(got, entry.RefID) {
+				t.Fatalf("extracted %q is not named by the field %q", entry.RefID, got)
+			}
+		}
 	})
 }

internal/sbom/license_emission_test.go

@@ -8,6 +8,7 @@ import (
 	cdx "github.com/CycloneDX/cyclonedx-go"
 	"github.com/bomly-dev/bomly-cli/internal/testnodes"
 	"github.com/bomly-dev/bomly-sdk"
+	"github.com/bomly-dev/bomly-sdk/spdxkit"
 	v23 "github.com/spdx/tools-golang/spdx/v2/v2_3"
 )
 
@@ -249,14 +250,25 @@ func TestSPDXLicenseComposition(t *testing.T) {
 			want: "MIT AND (MIT OR GPL-2.0-only)",
 		},
 		{
-			// Joining free text would produce an expression that does not
-			// parse, so a mixed set keeps the previous first-value behavior.
-			name: "unparseable member falls back to the first value",
+			// A mixed set composes fully now (#410). The unrecognized member
+			// becomes a reference, which is a valid expression element, so
+			// nothing is dropped -- this used to keep "MIT" alone and lose
+			// the fact that a second license was declared at all.
+			name: "an unrecognized member composes as a reference",
 			licenses: []sdk.PackageLicense{
 				{Value: "MIT"},
 				{Value: "non-standard"},
 			},
-			want: "MIT",
+			want: "MIT AND " + spdxkit.MintLicenseRef("non-standard").RefID,
+		},
+		{
+			// A lone unrecognized value is a reference rather than free text
+			// in a field SPDX says must hold an expression.
+			name: "a single unrecognized value becomes a reference",
+			licenses: []sdk.PackageLicense{
+				{Value: "see LICENSE file"},
+			},
+			want: spdxkit.MintLicenseRef("see LICENSE file").RefID,
 		},
 	}