Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
598338c
feat: add libtea dependency for TEA integration
aurangzaib048 Mar 5, 2026
e23f243
feat: register tea subcommand group in CLI
aurangzaib048 Mar 5, 2026
3f10b93
feat: add 'tea fetch' convenience command
aurangzaib048 Mar 5, 2026
de7d6bc
fix: move tea_group import to top of file to satisfy E402 lint rule
aurangzaib048 Mar 5, 2026
19b04f6
fix: address code review findings for TEA CLI
aurangzaib048 Mar 5, 2026
b54fe62
fix: restore Sentry DSN default
aurangzaib048 Mar 5, 2026
6adbf74
feat: add TEA enrichment source with auto-discovery
aurangzaib048 Mar 6, 2026
564296b
fix: address review findings for TEA integration
aurangzaib048 Mar 6, 2026
e635472
chore: switch libtea from TestPyPI to PyPI
aurangzaib048 Mar 6, 2026
41253fb
fix: include CLE data in early exit condition for enrichment registry
aurangzaib048 Mar 6, 2026
8ec8e9c
style: fix ruff formatting in registry.py
aurangzaib048 Mar 6, 2026
17b9ddb
fix: address comprehensive review findings for TEA integration
aurangzaib048 Mar 9, 2026
28b698b
fix: address PR review findings for TEA integration
aurangzaib048 Mar 9, 2026
77d11d3
fix: address second round of PR review findings
aurangzaib048 Mar 9, 2026
6d3a697
fix: add registry CLE skip tests and mock TEA in non-TEA tests
aurangzaib048 Mar 9, 2026
f016d45
fix: harden TEA mock fixture and clarify SSRF comment
aurangzaib048 Mar 9, 2026
cefbb13
fix: make collection assignment type-safe in tea fetch
aurangzaib048 Mar 9, 2026
787ca82
fix: address review findings — strengthen token hash, modernize types…
aurangzaib048 Mar 9, 2026
cc57f4d
fix: scope metadata cache key by env config and add product-release-u…
aurangzaib048 Mar 9, 2026
e9e0ccd
fix: replace hardcoded _CLE_PROVIDERS with provides_cle protocol prop…
aurangzaib048 Mar 9, 2026
4def1d9
fix: clarify provides_cle docstring, fix token cache test, add --auth…
aurangzaib048 Mar 9, 2026
89f803e
style: fix ruff formatting in tea CLI
aurangzaib048 Mar 9, 2026
ed084b2
fix: clear TEA client cache in conftest fixture for test isolation
aurangzaib048 Mar 9, 2026
31d14fb
fix: remove dead fmt.url guard after _select_best_format
aurangzaib048 Mar 9, 2026
4b83c03
fix: add clear=True to test_fetch_passes_token env patch
aurangzaib048 Mar 9, 2026
673cfc9
fix: cache from_well_known failures to avoid repeated connection atte…
aurangzaib048 Mar 9, 2026
127918d
fix: retry-limited discovery failures and reject unsafe URLs in suppo…
aurangzaib048 Mar 9, 2026
db07f23
fix: guard None media_type, fix client_cache type, remove provides_cl…
aurangzaib048 Mar 9, 2026
0fdda5c
fix: rename _MAX_DISCOVERY_RETRIES to _MAX_DISCOVERY_ATTEMPTS, clarif…
aurangzaib048 Mar 9, 2026
6e25efb
test: add test for supports() rejecting unsafe TEA_BASE_URL
aurangzaib048 Mar 9, 2026
4313bc1
fix: resolve mypy strict mode errors in TEA files
aurangzaib048 Mar 17, 2026
31644ac
fix: validate URL scheme in SSRF check and redact URLs in logs
aurangzaib048 Mar 17, 2026
14f6fc9
fix: resolve DNS in SSRF check and update early-exit comment
aurangzaib048 Mar 17, 2026
832bba2
fix: use ip.is_global to reject multicast/broadcast in SSRF check
aurangzaib048 Mar 17, 2026
f43a8f9
fix: add clear=True to base_url env patches, update fetch_metadata do…
aurangzaib048 Mar 23, 2026
c020091
fix: cache SSRF check, resilient libtea imports, supports() checks ov…
aurangzaib048 Mar 23, 2026
d8be4d1
fix: cache _is_safe_url results to avoid repeated DNS resolution
aurangzaib048 Mar 23, 2026
9536703
fix: update create_default_registry docstring for two-phase early exit
aurangzaib048 Mar 23, 2026
902deb6
fix: replace assert with explicit _error check for fmt.url
aurangzaib048 Mar 23, 2026
504f568
fix: hash base URL in client cache key to avoid retaining credentials
aurangzaib048 Mar 23, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ dependencies = [
"conan>=2.0,<3",
"questionary>=2.0.1,<3",
"pipdeptree>=2.0.0",
"libtea>=0.4.0,<1",
]

[project.urls]
Expand Down Expand Up @@ -106,7 +107,6 @@ exclude = [
"docs/",
]


[tool.ruff]
line-length = 120
target-version = "py310"
Expand Down
14 changes: 11 additions & 3 deletions sbomify_action/_enrichment/enricher.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
PURLSource,
PyPISource,
RepologySource,
TeaSource,
)


Expand All @@ -57,16 +58,20 @@ def create_default_registry() -> SourceRegistry:

Tier 2 - Primary Aggregators (40-49):
- DepsDevSource (40) - Google Open Source Insights
- TeaSource (43) - TEA server (auto-discovers from PURL type, TEA_BASE_URL overrides)
- EcosystemsSource (45) - ecosyste.ms multi-ecosystem aggregator
Comment thread
aurangzaib048 marked this conversation as resolved.

Tier 3 - Fallback Sources (70-99):
- PURLSource (70) - local PURL extraction for OS packages (no API)
- ClearlyDefinedSource (75) - license and attribution data
- RepologySource (90) - cross-distro metadata (rate-limited)

Sources are queried sequentially in priority order. If a source returns
all required NTIA fields (description, licenses, supplier), subsequent
sources are skipped.
Sources are queried sequentially in priority order with two-phase early exit:
1. If NTIA fields (description, licenses, supplier) AND all CLE fields
(release_date, eos, eol) are filled, remaining sources are skipped.
2. If only NTIA fields are filled but CLE is missing, non-CLE sources
are skipped while lifecycle-capable sources (provides_cle=True)
continue to run.

Returns:
Configured SourceRegistry
Expand All @@ -80,6 +85,7 @@ def create_default_registry() -> SourceRegistry:
registry.register(ConanSource())
registry.register(DebianSource())
registry.register(DepsDevSource())
registry.register(TeaSource())
Comment thread
aurangzaib048 marked this conversation as resolved.
registry.register(EcosystemsSource())
registry.register(PURLSource())
registry.register(ClearlyDefinedSource())
Expand Down Expand Up @@ -271,6 +277,7 @@ def clear_all_caches() -> None:
from .sources.pubdev import clear_cache as clear_pubdev
from .sources.pypi import clear_cache as clear_pypi
from .sources.repology import clear_cache as clear_repology
from .sources.tea import clear_cache as clear_tea

clear_license_db()
clear_lifecycle()
Expand All @@ -280,6 +287,7 @@ def clear_all_caches() -> None:
clear_conan()
clear_debian()
clear_depsdev()
clear_tea()
clear_ecosystems()
clear_clearlydefined()
clear_repology()
Expand Down
1 change: 1 addition & 0 deletions sbomify_action/_enrichment/metadata.py
Original file line number Diff line number Diff line change
Expand Up @@ -129,4 +129,5 @@ def has_data(self) -> bool:
or self.maintainer_name
or self.cle_eos
or self.cle_eol
or self.cle_release_date
)
36 changes: 24 additions & 12 deletions sbomify_action/_enrichment/registry.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
"""Source registry for managing data source plugins."""

from typing import Any, Dict, List, Optional
from typing import Any

import requests
from packageurl import PackageURL
Expand Down Expand Up @@ -33,7 +33,7 @@ class SourceRegistry:

def __init__(self) -> None:
"""Initialize an empty registry."""
self._sources: List[DataSource] = []
self._sources: list[DataSource] = []

def register(self, source: DataSource) -> None:
"""
Expand All @@ -47,7 +47,7 @@ def register(self, source: DataSource) -> None:
self._sources.append(source)
logger.debug(f"Registered data source: {source.name} (priority={source.priority})")

def get_sources_for(self, purl: PackageURL) -> List[DataSource]:
def get_sources_for(self, purl: PackageURL) -> list[DataSource]:
"""
Get all applicable sources for a PURL, sorted by priority.

Expand All @@ -66,13 +66,16 @@ def fetch_metadata(
purl: PackageURL,
session: requests.Session,
merge_results: bool = True,
) -> Optional[NormalizedMetadata]:
) -> NormalizedMetadata | None:
"""
Fetch metadata using the priority chain of sources.

Tries sources in priority order, stopping early when we have
sufficient data (description, licenses, supplier). Only continues
to lower-priority sources if critical fields are missing.
Tries sources in priority order with two-phase early exit:
1. If NTIA fields (description, licenses, supplier) AND all CLE
fields (release_date, eos, eol) are filled, stop entirely.
2. If only NTIA fields are filled but CLE is missing, skip
non-CLE sources and continue only with lifecycle-capable
sources (those declaring ``provides_cle = True``).

Args:
purl: Parsed PackageURL object
Expand All @@ -87,13 +90,22 @@ def fetch_metadata(
logger.debug(f"No sources available for PURL type: {purl.type}")
return None

result: Optional[NormalizedMetadata] = None
result: NormalizedMetadata | None = None

for source in sources:
# Stop early if we already have all core NTIA fields
# Two-phase early exit: (1) if NTIA + all CLE fields are filled, stop entirely;
# (2) if only NTIA is filled but CLE is missing, skip non-CLE sources and
# continue only with lifecycle-capable sources (those declaring provides_cle=True).
if result and result.description and result.licenses and result.supplier:
logger.debug(f"Skipping {source.name} - already have sufficient data for {purl.name}")
break
if result.cle_release_date and result.cle_eos and result.cle_eol:
logger.debug(f"Skipping {source.name} - already have sufficient data for {purl.name}")
break
Comment thread
aurangzaib048 marked this conversation as resolved.
# NTIA complete but CLE missing — only continue with lifecycle-capable sources.
# Sources opt in by declaring `provides_cle = True` (not in the Protocol;
# checked via getattr so non-CLE sources need no changes).
if not getattr(source, "provides_cle", False):
Comment thread
aurangzaib048 marked this conversation as resolved.
logger.debug(f"Skipping {source.name} - NTIA complete, not a CLE provider for {purl.name}")
continue
Comment thread
aurangzaib048 marked this conversation as resolved.
Comment thread
aurangzaib048 marked this conversation as resolved.

try:
metadata = source.fetch(purl, session)
Expand All @@ -114,7 +126,7 @@ def fetch_metadata(

return result

def list_sources(self) -> List[Dict[str, Any]]:
def list_sources(self) -> list[dict[str, Any]]:
"""
List all registered sources with their priorities.

Expand Down
2 changes: 2 additions & 0 deletions sbomify_action/_enrichment/sources/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
from .purl import PURLSource
from .pypi import PyPISource
from .repology import RepologySource
from .tea import TeaSource

__all__ = [
"ClearlyDefinedSource",
Expand All @@ -26,4 +27,5 @@
"PURLSource",
"PyPISource",
"RepologySource",
"TeaSource",
]
4 changes: 4 additions & 0 deletions sbomify_action/_enrichment/sources/lifecycle.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,10 @@ class LifecycleSource:
def name(self) -> str:
return "sbomify-lifecycle-db"

@property
def provides_cle(self) -> bool:
return True

@property
def priority(self) -> int:
# Priority 5: Very high - local data with no API calls
Expand Down
Loading
Loading