Skip to content

feat: search rules for data-driven distro matching customization and lookup - #3751

Open
kzantow wants to merge 22 commits into
anchore:mainfrom
kzantow-anchore:feat/search-rules-rapidfort
Open

kzantow wants to merge 22 commits into
anchore:mainfrom
kzantow-anchore:feat/search-rules-rapidfort

Conversation

@kzantow

@kzantow kzantow commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

This PR adds DB-driven search rules to allow queries to include additional data sets based on per-package matched values (e.g. version containing .rf#, echo#, ubuntu#), results are assigned priority to assess vuln applicability across multiple conflicting results, and the PR uses these changes to support RapidFort curated-image advisories.

Search rules

  • New v6 SearchRule model and store (DB schema 6.1.10). Each Match*/Exclude* predicate (distro name/version, ecosystem, package name/version, etc.) is an exact value or a regex. Every rule needs at least one indexed literal predicate (e.g. at least package type deb) to limit the impact to matching performance
  • A matching rule replaces a search with altered criteria: distro and/or package name, tagged with the rule's priority (search.RuleCriteria); multiple rules can match a single query to include the original query or different, lower priority data sets
  • Rewrites are applied centrally to most matching processes (apk, dpkg, rpm, language, CPE) use them without per-matcher code

Match behavior with conflicting evidence

An increasing problem we haven't properly handled is a single package having multiple applicable records describing the same vulnerability: an APK package might have NVD, Alpine, and a derivative distro back-porting fixes at a faster cadence, where Alpine and the distro only report fixes rather than disclose vulnerabilities. Deciding across these disparate data sources is increasingly important in the current landscape and this PR attempts to solve this with a few exceptions noted below. The general approach which also accounts for search rule matched records is:

  • New result.Rank: when results for one vulnerability disagree, a result from a rule's search beats the package's own result. After that, the higher rule priority wins, then the stronger match type
  • SplitVulnerable sorts records into vulnerable and not vulnerable based on heuristics and rankings:
    • The highest-ranked search with a range that covers the installed version decides
    • Unaffected (NAK) records deny regardless of rank
    • An advisory whose fix is exactly the installed version suppresses only candidates whose CVEs are all covered by that fix

RapidFort

  • New rapidfort-* distros (alpine, ubuntu, redhat, including Ubuntu variants) and a DB transformer for RapidFort vuln data
  • A distro identifier remaps the detected base distro to its RapidFort equivalent when image labels or marker files show a RapidFort-curated image. It drops base channels such as esm/eus. Config: distro-identifiers.rapidfort.apply:auto|never (default auto)
  • Search rules layer RapidFort more specific release streams over base streams, so a RapidFort fix or NAK overrides the upstream record

Misc. output fixes

  • relatedVulnerabilities is sorted, so --by-cve picks a stable canonical CVE
  • Fix versions that are equal but spelled differently are deduped (e.g. 0:1.2-3 and 1.2-3)
  • Reported fix versions at or below the installed version are dropped

Notes

vaibhav-rf and others added 15 commits July 6, 2026 16:36
- Add RapidFort-specific matcher that identifies containers via image
  labels and routes vulnerability lookups through the RapidFort DB
- Refactor matcher selection into the matcher package
- Add Red Hat advisory support: parse OS FixedIn identifiers and emit
  release-identifier refs with vendor advisory URLs in v6 transform
- Fix binary lookup match type (ExactDirectMatch) and expand test
  coverage for label detection and distro version parsing

Signed-off-by: Vaibhav Thatai <vaibhav@rapidfort.com>
Signed-off-by: Vaibhav Thatai <vaibhav@rapidfort.com>
Signed-off-by: Janit Lodha <janit.lodha@rapidfort.com>
Signed-off-by: Janit Lodha <janit.lodha@rapidfort.com>
Signed-off-by: Janit Lodha <janit.lodha@rapidfort.com>
… and document its semantics

Signed-off-by: Janit Lodha <janit.lodha@rapidfort.com>
Signed-off-by: Janit Lodha <janit.lodha@rapidfort.com>
…alignment

Signed-off-by: Janit Lodha <janit.lodha@rapidfort.com>
…r clarity

Signed-off-by: Janit Lodha <janit.lodha@rapidfort.com>
…fallback rule

Signed-off-by: Janit Lodha <janit.lodha@rapidfort.com>
…atcher-selection accessor

Signed-off-by: Janit Lodha <janit.lodha@rapidfort.com>
Updating Rapidfort Images Detection Mechanism
…d Ubuntu variant additions

Signed-off-by: Janit Lodha <janit.lodha@rapidfort.com>
Signed-off-by: Keith Zantow <kzantow@gmail.com>

# Conflicts:
#	grype/db/v6/build/transformers/os/transform_test.go
Signed-off-by: Keith Zantow <kzantow@gmail.com>
@kzantow
kzantow force-pushed the feat/search-rules-rapidfort branch from 11a26a5 to 5cdd53a Compare September 29, 2026 15:39
Comment thread grype/matcher/internal/distro.go Outdated
Signed-off-by: Keith Zantow <kzantow@gmail.com>
@willmurphyscode willmurphyscode self-assigned this Oct 2, 2026
Comment thread grype/matcher/internal/distro.go Outdated
Comment thread grype/presenter/models/match.go Outdated
Comment thread grype/db/v6/vulnerability_provider.go Outdated
log.WithFields("error", err).Warn("unable to load search rules from DB; falling back to built-in defaults")
rows = nil
}
if rows == nil {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is subtle that it's explicitly nil, vs just len(rows) == 0. Do we know whether at https://github.com/anchore/grype/pull/3751/changes#diff-20f991f6ce703181473acf0176595a408530d7b0678faa25905642a248557e71R28 gorm will init the row slice? I think maybe it will: https://github.com/go-gorm/gorm/blob/v1.31.2/scan.go#L295-L303

@kzantow kzantow Oct 5, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is explicitly set to nil if there is an error to fall back; added a comment so this is more clear

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the intended behavior if the table is present but the query works, but the table is empty?

@kzantow kzantow Oct 5, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the table is present but empty, it's intended to mean no search rules -- the hardcoded ones could be wrong. I don't know that's the right behavior, but that's the reason for the current implementation

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think migrating the db will create the empty table? So if a new client imports an old DB, the table gets created as empty instead of with defaults. If the table is present at DB build time but empty, it will correctly get zero rows, but I'm not sure it's correct for a table that's absent at DB build time (because grype-db is behind, for example) to get zero rows.

Comment thread grype/db/v6/build/transformers/os/transform.go
Comment thread grype/db/v6/search_rewrite.go

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants