Auto-discover a shipped baseline so distributed skills carry their own suppressions

[airadier]

Summary

Please consider auto-discovering a baseline that ships alongside a skill, so a distributed skill can carry its own vetted false-positive/by-design suppressions and downstream scans surface only genuinely new findings — without every consumer needing to know and pass --baseline.

Context

We publish a set of Agent Skills and run SkillSpector both in our CI and as a recommended check for the people who install the skills. A large share of the findings are inherent to what the skills legitimately do and are documented, e.g.:

• Threat-detection / incident-response docs that reference attacker techniques as material under investigation (IMDS 169.254.169.254, /etc/shadow, curl … | bash sample data, MITRE ATT&CK tactic tables).
• An intentionally privileged host agent (--privileged, docker.sock) that host monitoring requires.
• Credential auto-discovery from documented SYSDIG_* environment variables.

We maintain a reviewed .skillspector-baseline.yaml (rules: with a reason: on each) that suppresses these with explanations, so only new/actionable findings remain.

Problem

skillspector scan <skill> does not load a baseline unless --baseline <path> is passed explicitly (confirmed: a .skillspector-baseline.yaml placed next to a skill has no effect without the flag). When skills are distributed, the person scanning them generally won't know a curated baseline exists — and for a single-skill install there's no obvious repo-root file to point at. So consumers see all the known false positives raw, which erodes trust in the tool's output.

Proposal

Auto-discover a shipped baseline, e.g. one or more of:

1. If <scanned-dir>/.skillspector-baseline.yaml (and/or a repo-root one) exists, load it by default — with a flag to disable (--no-baseline) and a clear report of which suppressions were applied and their reason: (so suppression stays transparent, not silent).
2. A SKILL.md frontmatter field that points at or inlines the skill's suppressions.

This mirrors how most linters auto-load a config from the target directory, and it lets skill authors ship explained suppressions rather than asking each user to supply a flag.

Related (possible bug)

skillspector scan <dir> --recursive --baseline <file> appears to not apply the baseline (suppressed count stays 0 across the multi-skill run), whereas a single-skill skillspector scan <skill> --baseline <file> applies it correctly. If intended, worth documenting; if not, honoring --baseline in --recursive mode would help multi-skill repos.

Why it helps the ecosystem

Skill authors can ship vetted, human-explained suppressions with the skill, so any downstream SkillSpector run — not just the author's CI — surfaces only new or genuinely actionable findings. That improves signal for everyone and keeps the "suppressed with a documented reason" audit trail intact.

Happy to help test or contribute a PR if this direction is welcome.

[Spectorian]

I’d separate the two asks. Recursive scans should honor an explicitly supplied baseline; that bug is already tracked in #201/#205. Auto-discovery raises a different trust question: a baseline shipped by the skill author should not silently suppress findings in a consumer’s scan. A safer direction may be to detect and surface the baseline without applying it, require explicit opt-in before using it, and clearly report that an author-supplied baseline was applied, including what it suppressed and why. Would that explicit-trust model work for your downstream users and non-interactive CI flows?

[airadier]

I’d separate the two asks. Recursive scans should honor an explicitly supplied baseline; that bug is already tracked in #201/#205. Auto-discovery raises a different trust question: a baseline shipped by the skill author should not silently suppress findings in a consumer’s scan. A safer direction may be to detect and surface the baseline without applying it, require explicit opt-in before using it, and clearly report that an author-supplied baseline was applied, including what it suppressed and why. Would that explicit-trust model work for your downstream users and non-interactive CI flows?

Definitely that would work. So either either detect the baseline and apply it, but be verbose about an existing baseline has been applied, then add an option to opt-out, or clearly indicate that a author-provided baseline exists, and offer the option to opt-in.

My main use case is that we are publishing a set of skills, and the scan is usually quite verbose and detects several FPs. The number of FPs could refrain a user from adopting the skills based on a scan result until the user can clearly find out understand why some of those findings should be acceptable per the skill purpose.

Thanks!