Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 22 additions & 60 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# skillspector-action

GitHub Action for scanning AI agent skills with [NVIDIA SkillSpector](https://github.com/NVIDIA/SkillSpector).
Scan AI agent skills with [NVIDIA SkillSpector](https://github.com/NVIDIA/SkillSpector) from GitHub Actions.

This action is unofficial and experimental. It is designed for skill authors, skill registries, and internal AI agent platform teams that want a normal GitHub Actions experience without installing SkillSpector in every workflow.
This action is for skill authors and skill registry maintainers who want a simple CI check for `SKILL.md` repositories. It runs static analysis by default, uploads reports, and only fails CI when you ask it to.

## Quick start

Expand All @@ -16,7 +16,7 @@ on:

permissions:
contents: read
security-events: write # required only when upload-sarif is true
security-events: write # only needed when upload-sarif is true

jobs:
scan:
Expand All @@ -30,48 +30,45 @@ jobs:
upload-sarif: true
```

Default behavior is report-only:
By default the action reports findings without failing CI:

```yaml
fail-on: none
```

Use `fail-on` or `min-score` when you want CI to fail:
Fail CI only for the severities you care about:

```yaml
- uses: NPJigaK/skillspector-action@v1
with:
path: .
changed-only: true
fail-on: critical
min-score: "90"
upload-sarif: true
```

## What it does
## What you get

- Finds `SKILL.md` files under `path`.
- Runs SkillSpector static analysis by default with `--no-llm`.
- On pull requests, `changed-only: true` scans only changed skill directories.
- Writes JSON, SARIF, and Markdown reports to `skillspector-results`.
- Uploads reports as a workflow artifact.
- Optionally uploads SARIF to GitHub Code Scanning.
- Fails only when `fail-on` or `min-score` says to fail.
- Automatic `SKILL.md` discovery under `path`.
- Static SkillSpector scanning by default, without LLM API keys.
- Pull request mode with `changed-only: true`.
- JSON, SARIF, Markdown summary, and artifact reports.
- Optional GitHub Code Scanning upload.
- Opt-in CI failure with `fail-on` or `min-score`.

## Inputs

| Input | Default | Description |
| --- | --- | --- |
| `path` | `.` | Repository path or `SKILL.md` file to scan. |
| `changed-only` | `false` | On pull requests, scan only changed skill directories. If the PR diff cannot be resolved, the action falls back to full discovery. |
| `changed-only` | `false` | On pull requests, scan only changed skill directories. |
| `fail-on` | `none` | One of `none`, `high`, or `critical`. |
| `min-score` | empty | Optional numeric risk score threshold that fails the action. |
| `upload-sarif` | `false` | Upload `results.sarif` with `github/codeql-action/upload-sarif@v4`. |
| `upload-sarif` | `false` | Upload the SARIF report to GitHub Code Scanning. |
| `exclude` | empty | Newline or comma separated glob patterns to exclude. |
| `baseline` | empty | Optional baseline JSON path for suppressing accepted findings. |
| `baseline` | empty | Optional baseline file for suppressing accepted findings. |
| `llm` | `false` | Enable SkillSpector LLM analysis. Static mode is used by default. |
| `artifact-name` | `skillspector-results` | Artifact name for generated reports. |
| `image` | empty | Override the scanner image for development. Normal users should not set this. |

## Outputs

Expand All @@ -87,7 +84,7 @@ Use `fail-on` or `min-score` when you want CI to fail:

## Permissions

For local reports only:
For artifact and summary reports only:

```yaml
permissions:
Expand All @@ -102,50 +99,15 @@ permissions:
security-events: write
```

Code Scanning is available for public repositories on GitHub.com and for organization-owned repositories with GitHub Code Security enabled.

## Security notes

Use `pull_request`, not `pull_request_target`, when scanning untrusted pull request code. A privileged `pull_request_target` workflow that checks out fork code can expose write tokens and secrets to attacker-controlled content.

`llm` defaults to `false`. Do not pass LLM API keys to workflows that run on untrusted fork pull requests. When `llm: false`, this action does not pass common LLM provider environment variables into the scanner container.

For high-security environments, pin third-party actions to full commit SHAs. When this action is called by a full commit SHA, it resolves the default runtime image to `ghcr.io/npjigak/skillspector-action:sha-<commit>`. Tag users such as `@v1` receive the matching `v1` image tag.

## GHCR and cost

The scanner runtime is published as a public GHCR image. Users do not need to pull it manually or authenticate to GHCR. GitHub public packages are free to use, and public container images can be pulled anonymously.

The workflow still consumes the caller repository's normal GitHub Actions usage. Public repositories using standard GitHub-hosted runners are free; private repositories use the account or organization Actions allowance and billing settings.

## Baseline

Use `baseline` to suppress findings that were already reviewed and accepted.

```json
{
"version": 1,
"findings": [
{
"rule_id": "SS001",
"path": "skills/example/SKILL.md",
"message_hash": "0123456789abcdef"
}
]
}
```

`message_hash` is a SHA-256 prefix of the normalized finding message. Suppressed findings are excluded from failure decisions and counted separately in `results.json` and the job summary.

## Release process
Use `pull_request`, not `pull_request_target`, when scanning untrusted pull request code.

This repository publishes `ghcr.io/npjigak/skillspector-action` from `.github/workflows/publish-image.yml`.
`llm` defaults to `false`. Do not pass LLM API keys to workflows that run on untrusted fork pull requests.

Published tags include:
For high-security environments, pin third-party actions to full commit SHAs.

- branch tags such as `main`;
- semantic versions such as `1.0.0`;
- major versions such as `1`;
- commit tags such as `sha-<commit>`.
## More documentation

Create a `v1.0.0` Git tag and move the `v1` action tag when publishing a compatible release.
- [Advanced usage](docs/advanced-usage.md): baseline, exclude patterns, LLM scans, and pinning.
- [Maintainer guide](docs/maintainer.md): releases, runtime image publishing, and implementation notes.
79 changes: 79 additions & 0 deletions docs/advanced-usage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# Advanced Usage

This page covers options most users do not need on their first setup.

## Changed-only scanning

Use `changed-only: true` on pull requests to scan only skill directories touched by the PR.

```yaml
- uses: NPJigaK/skillspector-action@v1
with:
path: .
changed-only: true
```

If the pull request diff cannot be resolved, the action falls back to full skill discovery instead of skipping scans.

## Exclude patterns

Use `exclude` to skip generated or vendored skill directories.

```yaml
- uses: NPJigaK/skillspector-action@v1
with:
exclude: |
vendor/**
archived/**
```

Patterns are matched against repository-relative paths using shell-style globs.

## Baseline

Use `baseline` when a finding has already been reviewed and accepted.

```yaml
- uses: NPJigaK/skillspector-action@v1
with:
baseline: .skillspector-baseline.json
```

Baseline file format:

```json
{
"version": 1,
"findings": [
{
"rule_id": "SS001",
"path": "skills/example/SKILL.md",
"message_hash": "0123456789abcdef"
}
]
}
```

`message_hash` is a SHA-256 prefix of the normalized finding message. Suppressed findings are removed from JSON/SARIF failure decisions and counted separately in the summary.

## LLM scans

Static mode is the default. Enable LLM analysis only when the workflow can safely access the required provider credentials.

```yaml
- uses: NPJigaK/skillspector-action@v1
with:
llm: true
```

Do not enable LLM scans for untrusted fork pull requests if the job has access to API keys.

## Pinning

For high-security workflows, pin actions to full commit SHAs:

```yaml
- uses: NPJigaK/skillspector-action@<full-commit-sha>
```

When called by a full commit SHA, the action resolves the matching runtime image tag for that commit.
72 changes: 72 additions & 0 deletions docs/maintainer.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Maintainer Guide

This document is for maintainers of `NPJigaK/skillspector-action`.

## Runtime image

The action is a composite action that runs a containerized scanner wrapper. The default runtime image is published to GHCR:

```text
ghcr.io/npjigak/skillspector-action
```

Normal users do not need to know or configure this image. The `image` input exists for development and debugging only.

## Release process

1. Make sure `main` is green.
2. Create and push a semantic version tag, for example:

```bash
git tag v1.0.0
git push origin v1.0.0
```

3. Move or create the matching major action tag:

```bash
git tag -f v1 v1.0.0
git push -f origin v1
```

4. Confirm `.github/workflows/publish-image.yml` succeeds.

## Published image tags

The publish workflow emits:

- branch tags such as `main`;
- semantic version tags such as `1.0.0` and `v1.0.0`;
- major tags such as `1` and `v1`;
- commit tags such as `sha-<commit>`.

The `v*` image tags matter because users normally call the action as:

```yaml
uses: NPJigaK/skillspector-action@v1
```

That action ref resolves to the matching `v1` runtime image tag.

## SkillSpector pin

The Dockerfile pins upstream SkillSpector with `SKILLSPECTOR_REF`.

When updating the pin:

1. Update `Dockerfile`.
2. Update `tests/test_workflows.py`.
3. Run `python -m pytest -q`.
4. Confirm the PR CI builds the Docker image.

## Development image override

The hidden `image` input can be used to test an unpublished runtime image:

```yaml
- uses: NPJigaK/skillspector-action@v1
with:
image: ghcr.io/npjigak/skillspector-action:main
```

Do not recommend this input in user-facing setup docs.
27 changes: 25 additions & 2 deletions tests/test_readme.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,9 @@ def test_readme_documents_core_usage_and_security_notes() -> None:
assert "changed-only" in readme
assert "pull_request_target" in readme
assert "LLM" in readme
assert "GHCR" in readme
assert "public packages" in readme
assert "baseline" in readme
assert "docs/advanced-usage.md" in readme
assert "docs/maintainer.md" in readme


def test_readme_documents_inputs_and_outputs() -> None:
Expand All @@ -24,3 +24,26 @@ def test_readme_documents_inputs_and_outputs() -> None:

for output_name in ["sarif", "json", "markdown", "risk-score", "risk-severity", "findings-count", "scanned-count"]:
assert f"`{output_name}`" in readme


def test_readme_excludes_maintainer_and_internal_runtime_details() -> None:
readme = Path("README.md").read_text(encoding="utf-8")

assert "Release process" not in readme
assert "ghcr.io/npjigak/skillspector-action" not in readme
assert "`image`" not in readme
assert "public packages" not in readme


def test_advanced_usage_doc_contains_power_user_details() -> None:
doc = Path("docs/advanced-usage.md").read_text(encoding="utf-8")

for text in ["baseline", "exclude", "LLM", "pin", "message_hash"]:
assert text in doc


def test_maintainer_doc_contains_release_and_runtime_details() -> None:
doc = Path("docs/maintainer.md").read_text(encoding="utf-8")

for text in ["Release process", "GHCR", "ghcr.io/npjigak/skillspector-action", "sha-<commit>", "`image`"]:
assert text in doc