From 54ef26acc0dd808bfaca8c367962fcc146c49708 Mon Sep 17 00:00:00 2001 From: KC Berg Date: Tue, 7 Jul 2026 17:01:03 -0600 Subject: [PATCH 01/29] feat(hawkscan): add scan-planning discovery reference Adds references/scan-planning.md, the code-first discovery procedure that configures HawkScan for the highest-value DAST coverage: find every API surface (docs-first, then per-framework route detection), recommend code changes for spec/auth gaps, ask the user rather than guess on ambiguous URL space, configure one stackhawk.yml per surface ordered by DAST value, and cross-check config shapes against docs.stackhawk.com / hawk config show. Absorbs and supersedes the docs-first source table from app-discovery.md; the route-inventory derivation guidance here is the contract the future quality-gate reference reuses to diff actual scan coverage against expectation. --- .../hawkscan/references/scan-planning.md | 178 ++++++++++++++++++ 1 file changed, 178 insertions(+) create mode 100644 plugins/hawkscan/skills/hawkscan/references/scan-planning.md diff --git a/plugins/hawkscan/skills/hawkscan/references/scan-planning.md b/plugins/hawkscan/skills/hawkscan/references/scan-planning.md new file mode 100644 index 00000000..7e27e07b --- /dev/null +++ b/plugins/hawkscan/skills/hawkscan/references/scan-planning.md @@ -0,0 +1,178 @@ +# Scan Planning Reference (Discovery) + +Discovery is how the hawkscan skill decides *what* to scan and *how* before it writes or +edits a single `stackhawk.yml`. Run it in three situations: + +- **First scan of a repo.** No `stackhawk.yml` exists yet (SKILL.md Step 1a / Step 2a) — + discovery produces the surface inventory that Step 2a's config generation consumes. +- **The quality gate looped back with a structural gap.** A post-scan coverage check found + routes the scan never reached, or a surface it didn't know existed. Re-run discovery + against that specific gap rather than starting over. +- **The user asks to re-plan the scan.** New API surface added, a monorepo grew a service, + an old assumption no longer holds. + +Discovery is stateless: there is no plan file. `stackhawk.yml` is the only durable artifact, +and anything the user tells you that isn't a config field belongs as a comment in +`stackhawk.yml` or in the repo's `CLAUDE.md`/`AGENTS.md` — see "Ask, don't guess" below. + +Work code-first, interactively: explore before asking, ask the user before guessing, and +never stall waiting for information you could get by reading three more files or asking one +direct question. + +## Contents +- [Discover the app's API surfaces](#discover-the-apps-api-surfaces) +- [Recommend code changes for gaps](#recommend-code-changes-for-gaps) +- [Ask, don't guess](#ask-dont-guess) +- [Configure per surface](#configure-per-surface) +- [Cross-checking](#cross-checking) + +--- + +## Discover the app's API surfaces + +### Pass 1 — read what the repo already says + +Most repos document themselves. Read these, in priority order, before exploring code — +treat what they say as authoritative and don't rediscover by grepping what a doc already +states: + +| Source | Typically documents | +|--------|----------------------| +| `AGENTS.md` | run/build/test commands, layout, conventions | +| `CLAUDE.md` | same, written for agents — often the richest source | +| `GEMINI.md`, `.github/copilot-instructions.md` | agent run/build guidance | +| `.cursor/rules/*` | project conventions and setup steps | +| `README*` | quickstart, run command, default host/port | +| `CONTRIBUTING*` | local dev setup, how to run services and tests | +| `docs/` setup / quickstart / architecture pages | deeper API and service-boundary detail | + +Harvest: run command + host/port, which API surfaces exist and where their source lives +(useful in a monorepo), and any documented spec location or dev/test credential. + +### Pass 2 — explore code per surface type + +For anything the docs didn't answer, find the surfaces the way a developer would: read +manifests, entry points, and route definitions. Identify **every** distinct API surface in +the repo — a service can have more than one (e.g. a REST API plus an internal gRPC service). +For each surface found, record its type and derive a route inventory using the matching +technique: + +| Surface | Detection | Route inventory / derivation | +|---------|-----------|-------------------------------| +| REST — Spring | `@RestController`, `@Controller` classes | `grep -rEo '@(Get\|Post\|Put\|Delete\|Patch\|Request)Mapping' src/` — count matches | +| REST — Express/Node | `app.get/post/put/delete`, `router.get/post/...` | `grep -rEo "\.(get\|post\|put\|delete\|patch)\(" src/` (or route files under `routes/`) | +| REST — Rails | `config/routes.rb` | count route-verb lines: `grep -cE '^\s*(get\|post\|put\|patch\|delete)\s' config/routes.rb` | +| REST — Django | `urls.py` | count `path(`/`re_path(` entries: `grep -rc 'path(' **/urls.py` | +| REST — Go (gin/mux/chi) | `router.GET(`, `r.HandleFunc(`, `mux.NewRouter()` | `grep -rEo '\.(GET\|POST\|PUT\|DELETE\|PATCH)\(' .` | +| REST — .NET | `[HttpGet]`, `[HttpPost]`, `[Route(...)]` attributes | `grep -rEo '\[Http(Get\|Post\|Put\|Delete\|Patch)\]' .` | +| REST — OpenAPI spec | checked-in `openapi.{json,yaml,yml}`, `swagger*.{json,yaml}`; or served at `/v3/api-docs`, `/swagger.json`, `/openapi.json` once the app is running | count `paths:` entries: `yq '.paths | keys | length' openapi.yaml` (or the JSON equivalent) | +| GraphQL | schema files (`*.graphql`, `*.gql`); server libs in manifests (`graphql`, `apollo-server`, `graphql-yoga`, `graphene`, etc.); a `/graphql` endpoint | count root fields: `grep -A50 'type Query' schema.graphql \| grep -cE '^\s+\w+'` (repeat for `type Mutation`) | +| gRPC | `.proto` files; `grpc`/`protobuf` deps in the manifest | count `rpc` methods per service: `grep -c '^\s*rpc ' *.proto` | +| JSON-RPC / SOAP | fixed-path JSON-RPC dispatcher; WSDL files (`*.wsdl`), SOAP endpoint patterns | count ` --text` for the exact field shape — don't hand-write + the block from memory. +2. Configure auth per Phase 1c in SKILL.md — one auth block per surface if surfaces have + different login mechanisms (common when a gateway and an upstream service use different + credential types). +3. Validate every surface's config before treating discovery as complete: + ```bash + hawk validate config + hawk validate api # when a spec is wired + hawk validate auth # when authentication: is present + ``` + +**Before the first scan of a fresh repo, present a one-screen summary and get the user's +confirmation.** The summary should state, per surface: the surface type, which config file +covers it, the scan order and why, the auth approach, and the expected route count with how +it was derived (the same grep/spec-count evidence gathered above). Confirming this up front +is cheap; discovering the scan order or auth assumption was wrong after a scan has already +run is not. + +## Cross-checking + +Don't configure `openApiConf`, `graphqlConf`, `grpcConf`, `soapConf`, or any other +spec-wiring block from memory alone — these shapes change across hawk releases. Two sources, +in order of preference: + +1. **`hawk config show
--text`** — the canonical, version-matched recipe for the + installed CLI. This is always correct for the hawk version actually in use and should be + the default source for field syntax. +2. **docs.stackhawk.com**, via WebFetch or WebSearch when available — useful for narrative + guidance `hawk config show` doesn't carry (why a shape works the way it does, worked + examples for less common combinations). Prefer it over recalling the shape from training + data whenever there's any doubt. + +If the two disagree, trust `hawk config show` — it reflects the CLI version actually +installed, not a documentation snapshot that may be ahead of or behind it. From 0c633be7c2df1f9c2fc2b46644289e2fa683d932 Mon Sep 17 00:00:00 2001 From: KC Berg Date: Tue, 7 Jul 2026 17:08:47 -0600 Subject: [PATCH 02/29] feat(hawkscan): add scan-quality gate reference Adds the post-scan quality gate reference that runs after every scan, before findings are parsed into fix tasks. Encodes the four settled gate parameters: structural-only gap threshold (route-diff is always computed/reported as evidence but never fails the gate alone), a 2-interactive/1-autonomous iteration cap counted in rescans, config-vs- environment signal classing, and sequential value-ordered multi-config execution with per-config gate/caps. Co-Authored-By: Claude Fable 5 --- .../hawkscan/references/scan-quality.md | 174 ++++++++++++++++++ 1 file changed, 174 insertions(+) create mode 100644 plugins/hawkscan/skills/hawkscan/references/scan-quality.md diff --git a/plugins/hawkscan/skills/hawkscan/references/scan-quality.md b/plugins/hawkscan/skills/hawkscan/references/scan-quality.md new file mode 100644 index 00000000..f9752e74 --- /dev/null +++ b/plugins/hawkscan/skills/hawkscan/references/scan-quality.md @@ -0,0 +1,174 @@ +# Scan Quality Reference + +Run this gate after **every** scan — full scans and rescans alike — before findings are +parsed into fix tasks (SKILL.md calls it Step 4.5, sitting between Step 3's scan run and +Step 4's fix-task generation). It is a feedback loop into config tuning, **not a governor**: +gate state never blocks a finding from being reported and fixed. A thin scan that only +reached a fraction of the surface can still have found something real, and that finding +gets fixed now, regardless of what the gate says. The gate's job is narrower: notice when a +scan almost certainly missed API surface, and drive a bounded, additive config fix so the +*next* scan reaches more of it. + +The gate derives its expectation fresh every time it runs. There is no persisted plan or +prior-run state to compare against — recompute the expected surface from the repo and the +effective config each time, the same way discovery would if run again right now. + +**Multi-config execution.** When a repo has more than one `stackhawk.yml` surface (discovery +writes one config per surface, ordered by DAST value — primary or backing API first, a SPA +frontend last), scan and gate **sequentially**, one config at a time, in that same +value order. Gate immediately after each config's scan rather than batching every scan +first and gating afterward — an auth-wall caught and fixed on config 1 saves the identical +wasted scan on config 2. The iteration caps described below (2 interactive / 1 autonomous) +apply **per config**, not to the run as a whole. The autonomous post-code-change loop scans +only the config(s) whose surface the change touched — that scan must still cover that +surface's entire scope (the fix-ALL-findings rule from Step 4 is unchanged) — defaulting to +the primary surface's config when it's unclear which surface a change touched. A full sweep +across every config is an interactive operation only; never trigger one autonomously. + +## Contents +- [Derive the expectation (fresh, every scan)](#derive-the-expectation-fresh-every-scan) +- [The four checks](#the-four-checks) +- [On gaps: iterate the config](#on-gaps-iterate-the-config) +- [Reporting rules (replace the old autonomy lock)](#reporting-rules-replace-the-old-autonomy-lock) +- [Degradation](#degradation) + +--- + +## Derive the expectation (fresh, every scan) + +Before you can tell whether a scan under-covered its target, you need something to compare +it against. Build that expectation the same way every time, right after the scan finishes: + +- **Spec wired** (`openApiConf`, `graphqlConf`, `grpcConf`, `soapConf`): the expectation is + the paths or operations the spec declares. `hawk op scan uris --format json` is + config-aware — it reads the effective scan config and automatically fetches + graphql-operations or jsonrpc-methods when `graphqlConf` or `jsonRpcConf` is present in + that config (force either with `--graphql` / `--jsonrpc` if auto-detection misses it). + That gives you the real scanned-URI set to diff against the spec's declared set. +- **No spec wired**: re-derive the expected route inventory using the same per-framework + route greps as the discovery reference (`scan-planning.md`) — the same detection + heuristics, the same counting technique, run fresh against the current code, not carried + over from whatever discovery counted last time. Code changes between scans; the + expectation should too. +- **Spec-less apps that keep coming up short**: if the grepped expectation and the scanned + URIs repeatedly diverge because there's nothing authoritative to wire, that's a signal to + recommend generating a spec — the same code-change recommendations discovery makes + (`scan-planning.md`'s "Recommend code changes for gaps" section: `springdoc-openapi` for + Spring, `swagger-jsdoc`/`@nestjs/swagger` for Node, `drf-yasg`/`drf-spectacular` for + Django REST Framework, `swashbuckle` for .NET). Surface the recommendation; don't apply it + unrequested. +- `hawk op scan uris` and `hawk op scan config` require hawk op ≥ 0.11.7. Older CLIs fall + back per the Degradation section below — detect the gap by the subcommand erroring or not + existing, don't ask the user which version they have. + +## The four checks + +Run all four after every scan. Each has its own command, pass condition, and a stable +reason identifier used when reporting a gap — SKILL.md's fix-task step consumes these +identifiers directly, so keep them exactly as written here: `coverage-gap`, +`spec-not-wired`, `surface-unscanned`, `auth-validate-failed`, `auth-wall`, `all-4xx`, +`env-unreachable`. + +| # | Check | What it measures | Command(s) | Pass condition | Reason identifier(s) | +|---|-------|-------------------|------------|-----------------|------------------------| +| 1 | Coverage | Scanned URIs vs. the expectation derived above | `hawk op scan uris --format json` diffed against the expectation list | **Never fails the gate.** Always computed and reported as evidence, e.g. "14 of 41 planned routes untouched," with the untouched routes listed | `coverage-gap` (evidence only) | +| 2 | Auth | Live auth validation before the scan; auth-wall signals during it | `hawk validate auth stackhawk.yml` pre-scan; `hawk op scan metrics ` for auth-wall flags | `validate auth` exited 0, and no auth-wall flag on any surface that has `authentication:` configured | `auth-validate-failed`, `auth-wall` | +| 3 | Surface-completeness | Every surface discovery found has a config, that config actually ran, and any spec found for it is wired into that config | `hawk op scan config ` (effective config the scan ran with); fall back to reading the local `.yml` files | Every discovered surface maps to a config that exists, ran this scan, and has its spec wired if one exists | `surface-unscanned`, `spec-not-wired` | +| 4 | Health | Connection-failure spikes, timeout streaks, app unreachable mid-scan; all-4xx response patterns on a configured surface | `hawk op scan metrics ` for flags; `hawk op scan get ` for `url_count` and status summary | No connection-failure spike, no timeout streak, no mid-scan unreachability, and no configured surface returning all 4xx | `env-unreachable`, `all-4xx` | + +**Signal classes.** Every non-evidence gap above is either config-class or environment-class +— the class decides what you're allowed to do about it: + +- **Config-class** — `spec-not-wired`, `surface-unscanned`, `auth-wall` while the app is + demonstrably up, `all-4xx` on a configured surface (this is usually auth-shaped: the + surface is reachable but every request is being rejected). These route into the normal + bounded config iteration described below. +- **Environment-class** — `env-unreachable`, covering connection-failure spikes, timeout + streaks, and the app going unreachable mid-scan. **Never touch the config for these.** + Verify the app is up using the same playbook as SKILL.md's Step 1c / Step 6 exit-1 + diagnosis, restart or wait if needed, then retry the scan once — that retry does not + consume a config-iteration. If the same environment-class signal recurs on the retry, + report it as an environment problem and stop; do not keep retrying and do not reach for a + config edit to work around it. + +Performance signals — slow paths, rate-limited endpoints, a scan that ran long — are not a +gate concern at all. They route to the `optimize` skill's existing suggestion path (SKILL.md +Step 4, surfaced once per session for full scans ≥ 20 minutes), never to this gate. + +## On gaps: iterate the config + +When one or more config-class gaps are open, loop back to SKILL.md's Step 2b (tune existing +`stackhawk.yml`) with this shape: + +- **Batch, then rescan once.** Collect every gate-named fix from this scan's gap set — wire + the missing spec, enable `hawk.spider.ajax`, fix the `authentication:` block, add a config + for an unscanned surface — into a single edit pass. Rescan once after applying all of + them together. Don't rescan after each individual fix; that wastes scans the gate exists + to save. +- **Only iterate when the config actually changed.** If a reported gap has no config-level + fix — a monorepo slice with no visibility into the missing service, dynamic routing that + needs a code change, a spec that genuinely doesn't exist yet — report it and move on. + Don't burn an iteration re-scanning a config you didn't touch. +- **Iteration cap: 2 interactive, 1 autonomous — per config, and counted in rescans, not + edits.** A batch of five config edits followed by one rescan is one iteration. This + matches the Autonomous Security Loop's existing guard rail of at most one fix-rescan cycle + when running unattended. +- **At the cap, stop.** Report the remaining structural gaps by reason identifier, the + coverage route-diff evidence (the untouched-route list from check 1), and concrete next + steps — including any code-change recommendations discovery would make for surfaces that + can't be closed by config alone. Never claim the scan is done, or the app is secure, while + gaps remain open at the cap. +- **A severe coverage gap with all four structural checks green** is not a config-iteration + target — it means the scan reached everything it was told to reach and still found little. + Route it to SKILL.md's Step 6 empty-data handling instead: suggest the + `stackhawk-data-seed` skill rather than continuing to edit the scan config. + +**Additive-only.** Every fix applied during iteration must add coverage: wire a spec, turn +on the Ajax Spider, correct an auth recipe, add a missing surface's config. Iteration must +**never** widen `excludePaths`, raise `failureThreshold`, add `excludePlugins`, or otherwise +weaken the scan so it happens to pass the gate. If the only way to close a gap is to reduce +what gets scanned, that is not a pass — report the honest gap instead of manufacturing a +clean one. + +Environment-class gaps never enter this loop at all — see the Signal classes note above: +verify the app, retry once for free, and stop if it recurs. + +## Reporting rules (replace the old autonomy lock) + +- **Findings are always reported and always fixable**, independent of gate state. The gate + never withholds, delays, or downgrades a finding — a scan that under-covered the surface + can still have found a real, exploitable bug, and that bug gets fixed on this pass. +- **Never say the scan is "done and secure"** (or any equivalent framing) while a + config-class or environment-class gap is still open. Say plainly what was scanned, what + wasn't, and why — the reason identifiers exist so this statement is specific, not vague. +- **The autonomous post-code-change loop no longer treats exit 0 as silent success.** It + runs this gate after its scan, and if gaps remain once the autonomous iteration cap (1) is + reached, it reports them explicitly — reason identifiers, coverage route-diff evidence, + and next steps — instead of accepting a clean exit code at face value. +- **Performance-class signals are never gate gaps.** If health metrics point at slowness + rather than unreachability or all-4xx, that's an `optimize` suggestion, not a quality-gate + reason identifier. +- Keep every reported gap tied to one of the seven reason identifiers above so downstream + reporting — and the Step 4.5 handoff into fix-task generation — can consume it + programmatically rather than parsing prose. + +## Degradation + +`hawk op scan uris` and `hawk op scan config` need hawk op ≥ 0.11.7. Detect the absence by +the subcommand erroring or not existing at all — don't ask the user to check their version, +just degrade silently and keep going: + +- **Coverage (check 1)** degrades from a full scanned-vs-expected route diff to a single + before/after number: `url_count` from `hawk op scan get `. You lose the + untouched-route list; still report the number as evidence, just note it's coarser. +- **Surface-completeness (check 3)** degrades from reading the effective scan config via + `hawk op scan config` to reading the local `.yml` files directly to confirm a surface's + config exists and its spec block is present. +- **Auth and health (checks 2 and 4)** are unaffected — `hawk op scan metrics` and + `hawk validate auth` work on all hawk op versions and remain the primary signal either way. + +`hawk op scan stats` does not exist yet (it's a follow-up ticket) but will eventually add +measured, per-path engine statistics rather than the grep- and `url_count`-based estimates +this reference relies on today. When it ships, prefer its measured signal over the inferred +approach described here. Until then, its absence is expected — degrade silently and don't +block or warn on it being missing. From deb111d92c71a0b8745d6c384bc6073818ec09f5 Mon Sep 17 00:00:00 2001 From: KC Berg Date: Tue, 7 Jul 2026 17:15:12 -0600 Subject: [PATCH 03/29] fix(hawkscan): address review findings in scan-quality gate reference - Add auth-validate-failed to the config-class bullet so the signal-class list is actually closed over all non-evidence identifiers - Use a per-surface config placeholder in the auth check command instead of a hardcoded stackhawk.yml, matching the file's multi-config framing - Reconcile the surface-completeness degradation fallback to name both the local per-surface yml files and hawk validate config output Co-Authored-By: Claude Fable 5 --- .../skills/hawkscan/references/scan-quality.md | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/plugins/hawkscan/skills/hawkscan/references/scan-quality.md b/plugins/hawkscan/skills/hawkscan/references/scan-quality.md index f9752e74..2288e829 100644 --- a/plugins/hawkscan/skills/hawkscan/references/scan-quality.md +++ b/plugins/hawkscan/skills/hawkscan/references/scan-quality.md @@ -72,17 +72,18 @@ identifiers directly, so keep them exactly as written here: `coverage-gap`, | # | Check | What it measures | Command(s) | Pass condition | Reason identifier(s) | |---|-------|-------------------|------------|-----------------|------------------------| | 1 | Coverage | Scanned URIs vs. the expectation derived above | `hawk op scan uris --format json` diffed against the expectation list | **Never fails the gate.** Always computed and reported as evidence, e.g. "14 of 41 planned routes untouched," with the untouched routes listed | `coverage-gap` (evidence only) | -| 2 | Auth | Live auth validation before the scan; auth-wall signals during it | `hawk validate auth stackhawk.yml` pre-scan; `hawk op scan metrics ` for auth-wall flags | `validate auth` exited 0, and no auth-wall flag on any surface that has `authentication:` configured | `auth-validate-failed`, `auth-wall` | +| 2 | Auth | Live auth validation before the scan; auth-wall signals during it | `hawk validate auth .yml` pre-scan (against the config being gated — repos with multiple per-surface configs validate each one); `hawk op scan metrics ` for auth-wall flags | `validate auth` exited 0, and no auth-wall flag on any surface that has `authentication:` configured | `auth-validate-failed`, `auth-wall` | | 3 | Surface-completeness | Every surface discovery found has a config, that config actually ran, and any spec found for it is wired into that config | `hawk op scan config ` (effective config the scan ran with); fall back to reading the local `.yml` files | Every discovered surface maps to a config that exists, ran this scan, and has its spec wired if one exists | `surface-unscanned`, `spec-not-wired` | | 4 | Health | Connection-failure spikes, timeout streaks, app unreachable mid-scan; all-4xx response patterns on a configured surface | `hawk op scan metrics ` for flags; `hawk op scan get ` for `url_count` and status summary | No connection-failure spike, no timeout streak, no mid-scan unreachability, and no configured surface returning all 4xx | `env-unreachable`, `all-4xx` | **Signal classes.** Every non-evidence gap above is either config-class or environment-class — the class decides what you're allowed to do about it: -- **Config-class** — `spec-not-wired`, `surface-unscanned`, `auth-wall` while the app is - demonstrably up, `all-4xx` on a configured surface (this is usually auth-shaped: the - surface is reachable but every request is being rejected). These route into the normal - bounded config iteration described below. +- **Config-class** — `spec-not-wired`, `surface-unscanned`, `auth-validate-failed` (the fix + lives in the config's `authentication:` block), `auth-wall` while the app is demonstrably + up, `all-4xx` on a configured surface (this is usually auth-shaped: the surface is + reachable but every request is being rejected). These route into the normal bounded + config iteration described below. - **Environment-class** — `env-unreachable`, covering connection-failure spikes, timeout streaks, and the app going unreachable mid-scan. **Never touch the config for these.** Verify the app is up using the same playbook as SKILL.md's Step 1c / Step 6 exit-1 @@ -162,8 +163,9 @@ just degrade silently and keep going: before/after number: `url_count` from `hawk op scan get `. You lose the untouched-route list; still report the number as evidence, just note it's coarser. - **Surface-completeness (check 3)** degrades from reading the effective scan config via - `hawk op scan config` to reading the local `.yml` files directly to confirm a surface's - config exists and its spec block is present. + `hawk op scan config` to local signals: read the per-surface `.yml` files directly to + confirm a surface's config exists and its spec block is present, and use the + `hawk validate config` output for each file to confirm the config is structurally sound. - **Auth and health (checks 2 and 4)** are unaffected — `hawk op scan metrics` and `hawk validate auth` work on all hawk op versions and remain the primary signal either way. From 4a99b844c20060250508d843705cbf7d659c0cad Mon Sep 17 00:00:00 2001 From: KC Berg Date: Tue, 7 Jul 2026 17:20:03 -0600 Subject: [PATCH 04/29] feat(hawkscan): wire discovery + quality gate into SKILL.md Rewires SKILL.md around the new scan-planning.md (code-first discovery) and scan-quality.md (post-scan quality gate) references: shrinks Step 1a to a pointer, adds a gate-gap bullet to Step 2b, inserts Step 4.5 Quality Gate between the scan run and findings parsing, adds gate guard rails to the Autonomous Security Loop and Common Mistakes, adds hawk op scan uris/config rows to Companion Skills, and updates the frontmatter description. No mode-dispatch section added; app-discovery.md stays linked. Co-Authored-By: Claude Fable 5 --- plugins/hawkscan/skills/hawkscan/SKILL.md | 92 ++++++++++++++--------- 1 file changed, 57 insertions(+), 35 deletions(-) diff --git a/plugins/hawkscan/skills/hawkscan/SKILL.md b/plugins/hawkscan/skills/hawkscan/SKILL.md index ae949a23..a05323c4 100644 --- a/plugins/hawkscan/skills/hawkscan/SKILL.md +++ b/plugins/hawkscan/skills/hawkscan/SKILL.md @@ -3,17 +3,19 @@ name: hawkscan version: 2.2.0 description: > Runs the HawkScan DAST security loop — configure, scan, fix all reported - vulnerabilities (not just your changes), rescan to verify. Use when the user - asks to run or perform a security/DAST scan, to test an app or API for - vulnerabilities, or to verify a vulnerability is fixed; and AUTONOMOUSLY right - after you complete a code change (feature, bugfix, refactor) — "done" means - "done and secure," so run the loop without asking permission. Do NOT trigger - for: informational questions about what HawkScan is, detects, or how it works - (e.g. "what vulnerabilities does HawkScan find?"); editing stackhawk.yml or - other config without running a scan; querying existing findings, security - posture, untriaged counts, or scan history (use the stackhawk-api skill); - documentation-only changes; installing or setting up the CLI; or when the - user explicitly says to skip scanning. + vulnerabilities (not just your changes), rescan to verify. Performs + code-first discovery to configure high-value scans, and quality-gates every + scan against code-derived expectations before findings are parsed. Use when + the user asks to run or perform a security/DAST scan, to test an app or API + for vulnerabilities, or to verify a vulnerability is fixed; and AUTONOMOUSLY + right after you complete a code change (feature, bugfix, refactor) — "done" + means "done and secure," so run the loop without asking permission. Do NOT + trigger for: informational questions about what HawkScan is, detects, or how + it works (e.g. "what vulnerabilities does HawkScan find?"); editing + stackhawk.yml or other config without running a scan; querying existing + findings, security posture, untriaged counts, or scan history (use the + stackhawk-api skill); documentation-only changes; installing or setting up + the CLI; or when the user explicitly says to skip scanning. --- # HawkScan Skill @@ -59,6 +61,8 @@ The `api` skill wraps read-only StackHawk platform lookups via the `hawk` CLI (` | Triage a finding | `hawk op scan triage --scan --hash --status false-positive --note ""` | | Bulk triage from file | `hawk op scan triage --scan --from-file triage.yaml` | | Annotate w/o triage perm | `hawk op finding note --scan --hash --note "..."` | +| Get scanned URIs | `hawk op scan uris --format json` (hawk op ≥ 0.11.7; skip if absent) | +| Get effective scan config | `hawk op scan config ` (hawk op ≥ 0.11.7; skip if absent) | These reads require the combined `hawk` CLI; the `api` skill covers setup (`hawk init --browser` or the `HAWK_API_KEY` env var). @@ -125,32 +129,21 @@ enable only detected flags via `hawk op app tech-flags`; if none, skip. > 2. `CI=true` or `GITHUB_ACTIONS=true` → `CI` > 3. Git branch `main`/`master`/`production` → `Production`; `staging` → `Staging`; otherwise → `Development` -### Step 1a: Understand the Application +### Step 1a: Discover What to Scan -Before generating config, build a working understanding of the target: **how it runs -locally, how to reach its API, whether it's a SPA, and whether exercising it requires -auth.** Work in three passes; stop as soon as you can answer all four: +Before generating config, run code-first discovery: find every API surface in the repo +(REST, GraphQL, gRPC, SPA, ...), derive a route inventory per surface, and get the run +command, host/port, and auth shape needed to reach each one. Prefer the repo's own docs +over guessing, explore code to fill gaps, and ask the user directly for whatever remains +unresolved — never stall or invent. -1. **Read what the repo already documents.** Most repos describe how to run themselves — - prefer their own words over guessing, and treat them as authoritative. - → Source files and what to harvest: [`references/app-discovery.md`](references/app-discovery.md) +→ Full discovery workflow — per-surface detection, route-inventory derivation, gap + recommendations, and the user-confirmed summary required before the first scan: + [`references/scan-planning.md`](references/scan-planning.md) +→ Docs-first source table (which files to read and what to harvest): + [`references/app-discovery.md`](references/app-discovery.md) -2. **Explore to fill the gaps.** For anything the docs don't answer, investigate the repo - the way a developer would when handed an unfamiliar codebase — read the run/build - manifests, entry points, and route definitions — until you can state, concretely: - - **Run command + host/port** — how to start it locally and where it listens. - - **API style + base path** — REST (note an OpenAPI/Swagger spec if one is served or - checked in), GraphQL, gRPC, or a plain HTML web app. - - **SPA?** — a client-rendered JS front end changes what we scan. - - **Auth?** — does reaching the real endpoints require a login, and what shape does it - appear to use? This is an *understanding* signal only; picking, obtaining, and - configuring the credential is Step 1c / Phase 1c. - -3. **Confirm the unknowns with the user.** Never stall or invent. Ask directly for - anything still unresolved — most often the start command, host/port, or the login. When - file access is limited, skip straight to asking for all four. - -Record the four answers — Step 1c and Step 2 consume them. +Record the answers — Step 1c and Step 2 consume them. **SPA rule:** if the app is a client-rendered JS front end, never scan it without the Ajax Spider, and note that a separate backend API is usually the higher-value target. Full @@ -269,6 +262,8 @@ After seeding, re-run `hawk validate auth stackhawk.yml` and continue. Review the config against the current app state: +- **Gate reported gaps?** Tune what the gate names (wire spec, enable ajax spider, fix + auth) — additive-only; see [`references/scan-quality.md`](references/scan-quality.md). - **Low path count?** Check in order: SPA/JS app → enable `hawk.spider.ajax: true`; API spec available → wire `openApiConf`/`graphqlConf`; specific known-deep paths → add `seedPaths`. Omit `seedPaths` unless there is a specific reason — they're rarely needed when Ajax Spider or a spec is configured. - **Auth failing?** Verify `authentication` block; re-fetch the relevant recipe via `hawk config show
--text` (Phase 1c). - **Too noisy / too slow?** Add `app.excludePaths` or `app.includePaths`; tune `hawk.spider.maxDurationMinutes`. @@ -342,6 +337,31 @@ hawk rescan --scan-id --json-output # fast fix verification — r --- +## Step 4.5: Quality Gate + +Run this gate after **every** scan — full scans and rescans alike, any exit code — before +parsing findings into fix tasks. It is a feedback loop into config tuning, **not a +governor**: gate state never blocks a finding from being reported and fixed; a thin scan +that missed surface can still have found something real. + +Run the four structural checks and derive the expectation fresh each time, exactly as +`references/scan-quality.md` describes — coverage, auth, surface-completeness, and health. +On config-class gaps (`spec-not-wired`, `surface-unscanned`, `auth-validate-failed`, +`auth-wall`, `all-4xx`), loop back to Step 2b with an additive-only fix, batch every gap +into one edit pass, and rescan once. Iteration cap: 2 interactive / 1 autonomous, per +config, counted in rescans. On environment-class gaps (`env-unreachable`), never edit +config — verify the app is up, retry once for free, and stop if it recurs. Multi-config +repos gate sequentially, one config at a time, immediately after that config's scan. + +At the cap, or once checks are clean, proceed to Step 4 regardless — findings are always +reported and fixable. Never say a scan is "done and secure" while a gate gap is open; state +plainly what was scanned, what wasn't, and why, using the reason identifiers. + +→ Full check definitions, commands, degradation without hawk op ≥ 0.11.7, and the + iteration/reporting rules: [`references/scan-quality.md`](references/scan-quality.md) + +--- + ## Step 4: Parse Findings and Generate Fix Tasks Use `--json-output` for structured results (requires Dev Release v5.3.41+). Suppresses all @@ -429,6 +449,8 @@ initialized; skill is active. - Max one fix-rescan cycle per task — if findings remain after fixing, report them - Always announce: "Running security scan...", "Found N vulnerabilities, fixing...", "Rescanning..." - Interruptible — stop immediately if the user says to +- Report gate gaps — never silently accept exit 0 from a thin scan +- Never claim "done and secure" while gate gaps are open --- @@ -437,7 +459,7 @@ initialized; skill is active. - **Don't scan before the app is running.** HawkScan will exit 1 with a connection error. - **Try `https://` first — HawkScan accepts self-signed certificates.** Only fall back to `http://` if the scan actually fails to connect with a TLS error. - **Don't hardcode API keys or credentials in `stackhawk.yml`.** Use env vars. -- **Low path count ≠ clean app.** Feed an API spec or auth config before concluding the app is secure. If auth succeeds but endpoints return empty data, seed the backend (Phase 1c.6). +- **Never accept an exit-0 scan that fails the quality gate.** Untouched planned routes, auth-walls, or an unscanned surface mean config iteration, not done — see [`references/scan-quality.md`](references/scan-quality.md). - **Don't ignore exit code 42.** It's a deliberate signal that findings crossed the threshold — treat it as a build failure. - **String interpolation mid-value doesn't work.** `host: "https://${HOST}/api"` will NOT interpolate. Use `host: ${FULL_HOST_URL}`. - **Never refer to the scanner as ZAP.** The product is HawkScan / StackHawk. The underlying engine is HSTE, not ZAP. Neither "ZAP" nor "OWASP ZAP" should appear in output or instructions. From 54ab75a5526d939989b0070c1c6fc2060d68184d Mon Sep 17 00:00:00 2001 From: KC Berg Date: Tue, 7 Jul 2026 17:25:19 -0600 Subject: [PATCH 05/29] refactor(hawkscan): slim app-discovery.md to docs-first stub app-discovery's goal-driven framing and discovery procedure were absorbed into references/scan-planning.md. Slim the file to a pointer paragraph plus the docs-first source table, which SKILL.md's Step 1a still links to directly. --- .../hawkscan/references/app-discovery.md | 34 +++---------------- 1 file changed, 5 insertions(+), 29 deletions(-) diff --git a/plugins/hawkscan/skills/hawkscan/references/app-discovery.md b/plugins/hawkscan/skills/hawkscan/references/app-discovery.md index fb47a00f..f35828ea 100644 --- a/plugins/hawkscan/skills/hawkscan/references/app-discovery.md +++ b/plugins/hawkscan/skills/hawkscan/references/app-discovery.md @@ -1,9 +1,10 @@ # App Discovery Reference -How to understand a target application before configuring a scan — what it is, how to run -it locally, how to reach its API, and whether it needs auth. Used by Step 1a in SKILL.md. -Prefer the repo's own documentation; explore only to fill gaps; ask the user for whatever -remains. Do not treat any of this as a fixed checklist of commands. +This file's discovery procedure has been superseded by `references/scan-planning.md` — see +Step 1a in SKILL.md for the full workflow (per-surface detection, route-inventory +derivation, gap recommendations, and the user-confirmed summary required before the first +scan). What remains here is the docs-first source table below, which SKILL.md links to +directly and which stands on its own. ## Read the repo's own docs first @@ -23,28 +24,3 @@ order, and treat what they say as authoritative: Harvest the **run/start command**, the **local host + port**, the **API style** (REST/OpenAPI, GraphQL, gRPC, or a plain web app), and any **documented dev/test login or seed data**. If a file answers a question, don't rediscover it by exploring. - -## The four answers discovery must produce - -From docs, exploration, or the user — be able to state all four before generating config: - -1. **Run command + host/port** — the exact way to start the app locally and the URL it - listens on. Feeds Step 1c ("App running?") and the `host:` in Step 2. -2. **API style + base path** — REST (an OpenAPI/Swagger spec, served or checked in, - enables `openApiConf`), GraphQL (`graphqlConf`), gRPC (`grpcConf`), or an HTML web app. - Determines the config shape in Step 2. -3. **SPA or not** — a client-rendered JS front end (React, Vue, Angular, Svelte, Next, - Nuxt, and the like) requires the Ajax Spider and changes target selection. See the SPA - rule in Step 1a of SKILL.md. -4. **Auth needed?** — does reaching the real endpoints require a login, and what shape does - it appear to use (form/session, bearer token, OAuth, custom)? Understand this so you - scan authenticated when required and don't waste a run against a login wall. **Stop - there** — picking, configuring, obtaining, or seeding the credential is owned by Step 1c - / Phase 1c in SKILL.md and the `stackhawk-data-seed` skill. Do not duplicate that here. - -## When you can't determine something - -Don't stall and don't invent. After reading the docs and a reasonable look through the -repo, ask the user directly for whatever is unresolved — most often the exact start -command, the host/port, or how to log in locally. When file access is limited, skip -exploration and ask up front for all four answers. From 0d6f136e93a4ffdc056a1bed8ea6418d716f5ecf Mon Sep 17 00:00:00 2001 From: KC Berg Date: Tue, 7 Jul 2026 17:29:18 -0600 Subject: [PATCH 06/29] feat(hawkscan): wire scan-planning + scan-quality into cursor rules; bump minor Adds MAPPINGS rows for the new scan-planning.md (code-first discovery) and scan-quality.md (post-scan quality gate) references so Cursor users get the same guidance as Claude/Codex. Bumps 2.1.1 -> 2.2.0 across all platform manifests and SKILL.md frontmatter. --- .../stackhawk-hawkscan-scan-planning.mdc | 184 ++++++++++++++++++ .../rules/stackhawk-hawkscan-scan-quality.mdc | 182 +++++++++++++++++ cursor/.cursor/rules/stackhawk-hawkscan.mdc | 68 ++++--- scripts/generate-cursor-rules.sh | 2 + 4 files changed, 412 insertions(+), 24 deletions(-) create mode 100644 cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc create mode 100644 cursor/.cursor/rules/stackhawk-hawkscan-scan-quality.mdc diff --git a/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc b/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc new file mode 100644 index 00000000..ca9baa5e --- /dev/null +++ b/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc @@ -0,0 +1,184 @@ +--- +description: > + HawkScan scan-planning (discovery) reference: code-first discovery of an app's API surfaces before writing or editing stackhawk.yml — reading repo docs first, per-framework route inventory, recommending code changes for structural gaps, asking the user instead of guessing, and producing a per-surface stackhawk.yml config. Use for the first scan of a repo, a quality-gate structural gap, or a user-requested re-plan. +globs: +alwaysApply: false +--- +# Scan Planning Reference (Discovery) + +Discovery is how the hawkscan skill decides *what* to scan and *how* before it writes or +edits a single `stackhawk.yml`. Run it in three situations: + +- **First scan of a repo.** No `stackhawk.yml` exists yet (SKILL.md Step 1a / Step 2a) — + discovery produces the surface inventory that Step 2a's config generation consumes. +- **The quality gate looped back with a structural gap.** A post-scan coverage check found + routes the scan never reached, or a surface it didn't know existed. Re-run discovery + against that specific gap rather than starting over. +- **The user asks to re-plan the scan.** New API surface added, a monorepo grew a service, + an old assumption no longer holds. + +Discovery is stateless: there is no plan file. `stackhawk.yml` is the only durable artifact, +and anything the user tells you that isn't a config field belongs as a comment in +`stackhawk.yml` or in the repo's `CLAUDE.md`/`AGENTS.md` — see "Ask, don't guess" below. + +Work code-first, interactively: explore before asking, ask the user before guessing, and +never stall waiting for information you could get by reading three more files or asking one +direct question. + +## Contents +- [Discover the app's API surfaces](#discover-the-apps-api-surfaces) +- [Recommend code changes for gaps](#recommend-code-changes-for-gaps) +- [Ask, don't guess](#ask-dont-guess) +- [Configure per surface](#configure-per-surface) +- [Cross-checking](#cross-checking) + +--- + +## Discover the app's API surfaces + +### Pass 1 — read what the repo already says + +Most repos document themselves. Read these, in priority order, before exploring code — +treat what they say as authoritative and don't rediscover by grepping what a doc already +states: + +| Source | Typically documents | +|--------|----------------------| +| `AGENTS.md` | run/build/test commands, layout, conventions | +| `CLAUDE.md` | same, written for agents — often the richest source | +| `GEMINI.md`, `.github/copilot-instructions.md` | agent run/build guidance | +| `.cursor/rules/*` | project conventions and setup steps | +| `README*` | quickstart, run command, default host/port | +| `CONTRIBUTING*` | local dev setup, how to run services and tests | +| `docs/` setup / quickstart / architecture pages | deeper API and service-boundary detail | + +Harvest: run command + host/port, which API surfaces exist and where their source lives +(useful in a monorepo), and any documented spec location or dev/test credential. + +### Pass 2 — explore code per surface type + +For anything the docs didn't answer, find the surfaces the way a developer would: read +manifests, entry points, and route definitions. Identify **every** distinct API surface in +the repo — a service can have more than one (e.g. a REST API plus an internal gRPC service). +For each surface found, record its type and derive a route inventory using the matching +technique: + +| Surface | Detection | Route inventory / derivation | +|---------|-----------|-------------------------------| +| REST — Spring | `@RestController`, `@Controller` classes | `grep -rEo '@(Get\|Post\|Put\|Delete\|Patch\|Request)Mapping' src/` — count matches | +| REST — Express/Node | `app.get/post/put/delete`, `router.get/post/...` | `grep -rEo "\.(get\|post\|put\|delete\|patch)\(" src/` (or route files under `routes/`) | +| REST — Rails | `config/routes.rb` | count route-verb lines: `grep -cE '^\s*(get\|post\|put\|patch\|delete)\s' config/routes.rb` | +| REST — Django | `urls.py` | count `path(`/`re_path(` entries: `grep -rc 'path(' **/urls.py` | +| REST — Go (gin/mux/chi) | `router.GET(`, `r.HandleFunc(`, `mux.NewRouter()` | `grep -rEo '\.(GET\|POST\|PUT\|DELETE\|PATCH)\(' .` | +| REST — .NET | `[HttpGet]`, `[HttpPost]`, `[Route(...)]` attributes | `grep -rEo '\[Http(Get\|Post\|Put\|Delete\|Patch)\]' .` | +| REST — OpenAPI spec | checked-in `openapi.{json,yaml,yml}`, `swagger*.{json,yaml}`; or served at `/v3/api-docs`, `/swagger.json`, `/openapi.json` once the app is running | count `paths:` entries: `yq '.paths | keys | length' openapi.yaml` (or the JSON equivalent) | +| GraphQL | schema files (`*.graphql`, `*.gql`); server libs in manifests (`graphql`, `apollo-server`, `graphql-yoga`, `graphene`, etc.); a `/graphql` endpoint | count root fields: `grep -A50 'type Query' schema.graphql \| grep -cE '^\s+\w+'` (repeat for `type Mutation`) | +| gRPC | `.proto` files; `grpc`/`protobuf` deps in the manifest | count `rpc` methods per service: `grep -c '^\s*rpc ' *.proto` | +| JSON-RPC / SOAP | fixed-path JSON-RPC dispatcher; WSDL files (`*.wsdl`), SOAP endpoint patterns | count ` --text` for the exact field shape — don't hand-write + the block from memory. +2. Configure auth per Phase 1c in SKILL.md — one auth block per surface if surfaces have + different login mechanisms (common when a gateway and an upstream service use different + credential types). +3. Validate every surface's config before treating discovery as complete: + ```bash + hawk validate config + hawk validate api # when a spec is wired + hawk validate auth # when authentication: is present + ``` + +**Before the first scan of a fresh repo, present a one-screen summary and get the user's +confirmation.** The summary should state, per surface: the surface type, which config file +covers it, the scan order and why, the auth approach, and the expected route count with how +it was derived (the same grep/spec-count evidence gathered above). Confirming this up front +is cheap; discovering the scan order or auth assumption was wrong after a scan has already +run is not. + +## Cross-checking + +Don't configure `openApiConf`, `graphqlConf`, `grpcConf`, `soapConf`, or any other +spec-wiring block from memory alone — these shapes change across hawk releases. Two sources, +in order of preference: + +1. **`hawk config show
--text`** — the canonical, version-matched recipe for the + installed CLI. This is always correct for the hawk version actually in use and should be + the default source for field syntax. +2. **docs.stackhawk.com**, via WebFetch or WebSearch when available — useful for narrative + guidance `hawk config show` doesn't carry (why a shape works the way it does, worked + examples for less common combinations). Prefer it over recalling the shape from training + data whenever there's any doubt. + +If the two disagree, trust `hawk config show` — it reflects the CLI version actually +installed, not a documentation snapshot that may be ahead of or behind it. diff --git a/cursor/.cursor/rules/stackhawk-hawkscan-scan-quality.mdc b/cursor/.cursor/rules/stackhawk-hawkscan-scan-quality.mdc new file mode 100644 index 00000000..5535d336 --- /dev/null +++ b/cursor/.cursor/rules/stackhawk-hawkscan-scan-quality.mdc @@ -0,0 +1,182 @@ +--- +description: > + HawkScan scan-quality (post-scan quality gate) reference: run after every scan and rescan, before findings become fix tasks. Derives a fresh coverage expectation each run (spec-wired or route-inventory), the four coverage checks, and a bounded, additive-only stackhawk.yml config iteration to close gaps — never blocks a real finding from being reported and fixed. +globs: +alwaysApply: false +--- +# Scan Quality Reference + +Run this gate after **every** scan — full scans and rescans alike — before findings are +parsed into fix tasks (SKILL.md calls it Step 4.5, sitting between Step 3's scan run and +Step 4's fix-task generation). It is a feedback loop into config tuning, **not a governor**: +gate state never blocks a finding from being reported and fixed. A thin scan that only +reached a fraction of the surface can still have found something real, and that finding +gets fixed now, regardless of what the gate says. The gate's job is narrower: notice when a +scan almost certainly missed API surface, and drive a bounded, additive config fix so the +*next* scan reaches more of it. + +The gate derives its expectation fresh every time it runs. There is no persisted plan or +prior-run state to compare against — recompute the expected surface from the repo and the +effective config each time, the same way discovery would if run again right now. + +**Multi-config execution.** When a repo has more than one `stackhawk.yml` surface (discovery +writes one config per surface, ordered by DAST value — primary or backing API first, a SPA +frontend last), scan and gate **sequentially**, one config at a time, in that same +value order. Gate immediately after each config's scan rather than batching every scan +first and gating afterward — an auth-wall caught and fixed on config 1 saves the identical +wasted scan on config 2. The iteration caps described below (2 interactive / 1 autonomous) +apply **per config**, not to the run as a whole. The autonomous post-code-change loop scans +only the config(s) whose surface the change touched — that scan must still cover that +surface's entire scope (the fix-ALL-findings rule from Step 4 is unchanged) — defaulting to +the primary surface's config when it's unclear which surface a change touched. A full sweep +across every config is an interactive operation only; never trigger one autonomously. + +## Contents +- [Derive the expectation (fresh, every scan)](#derive-the-expectation-fresh-every-scan) +- [The four checks](#the-four-checks) +- [On gaps: iterate the config](#on-gaps-iterate-the-config) +- [Reporting rules (replace the old autonomy lock)](#reporting-rules-replace-the-old-autonomy-lock) +- [Degradation](#degradation) + +--- + +## Derive the expectation (fresh, every scan) + +Before you can tell whether a scan under-covered its target, you need something to compare +it against. Build that expectation the same way every time, right after the scan finishes: + +- **Spec wired** (`openApiConf`, `graphqlConf`, `grpcConf`, `soapConf`): the expectation is + the paths or operations the spec declares. `hawk op scan uris --format json` is + config-aware — it reads the effective scan config and automatically fetches + graphql-operations or jsonrpc-methods when `graphqlConf` or `jsonRpcConf` is present in + that config (force either with `--graphql` / `--jsonrpc` if auto-detection misses it). + That gives you the real scanned-URI set to diff against the spec's declared set. +- **No spec wired**: re-derive the expected route inventory using the same per-framework + route greps as the discovery reference (`scan-planning.md`) — the same detection + heuristics, the same counting technique, run fresh against the current code, not carried + over from whatever discovery counted last time. Code changes between scans; the + expectation should too. +- **Spec-less apps that keep coming up short**: if the grepped expectation and the scanned + URIs repeatedly diverge because there's nothing authoritative to wire, that's a signal to + recommend generating a spec — the same code-change recommendations discovery makes + (`scan-planning.md`'s "Recommend code changes for gaps" section: `springdoc-openapi` for + Spring, `swagger-jsdoc`/`@nestjs/swagger` for Node, `drf-yasg`/`drf-spectacular` for + Django REST Framework, `swashbuckle` for .NET). Surface the recommendation; don't apply it + unrequested. +- `hawk op scan uris` and `hawk op scan config` require hawk op ≥ 0.11.7. Older CLIs fall + back per the Degradation section below — detect the gap by the subcommand erroring or not + existing, don't ask the user which version they have. + +## The four checks + +Run all four after every scan. Each has its own command, pass condition, and a stable +reason identifier used when reporting a gap — SKILL.md's fix-task step consumes these +identifiers directly, so keep them exactly as written here: `coverage-gap`, +`spec-not-wired`, `surface-unscanned`, `auth-validate-failed`, `auth-wall`, `all-4xx`, +`env-unreachable`. + +| # | Check | What it measures | Command(s) | Pass condition | Reason identifier(s) | +|---|-------|-------------------|------------|-----------------|------------------------| +| 1 | Coverage | Scanned URIs vs. the expectation derived above | `hawk op scan uris --format json` diffed against the expectation list | **Never fails the gate.** Always computed and reported as evidence, e.g. "14 of 41 planned routes untouched," with the untouched routes listed | `coverage-gap` (evidence only) | +| 2 | Auth | Live auth validation before the scan; auth-wall signals during it | `hawk validate auth .yml` pre-scan (against the config being gated — repos with multiple per-surface configs validate each one); `hawk op scan metrics ` for auth-wall flags | `validate auth` exited 0, and no auth-wall flag on any surface that has `authentication:` configured | `auth-validate-failed`, `auth-wall` | +| 3 | Surface-completeness | Every surface discovery found has a config, that config actually ran, and any spec found for it is wired into that config | `hawk op scan config ` (effective config the scan ran with); fall back to reading the local `.yml` files | Every discovered surface maps to a config that exists, ran this scan, and has its spec wired if one exists | `surface-unscanned`, `spec-not-wired` | +| 4 | Health | Connection-failure spikes, timeout streaks, app unreachable mid-scan; all-4xx response patterns on a configured surface | `hawk op scan metrics ` for flags; `hawk op scan get ` for `url_count` and status summary | No connection-failure spike, no timeout streak, no mid-scan unreachability, and no configured surface returning all 4xx | `env-unreachable`, `all-4xx` | + +**Signal classes.** Every non-evidence gap above is either config-class or environment-class +— the class decides what you're allowed to do about it: + +- **Config-class** — `spec-not-wired`, `surface-unscanned`, `auth-validate-failed` (the fix + lives in the config's `authentication:` block), `auth-wall` while the app is demonstrably + up, `all-4xx` on a configured surface (this is usually auth-shaped: the surface is + reachable but every request is being rejected). These route into the normal bounded + config iteration described below. +- **Environment-class** — `env-unreachable`, covering connection-failure spikes, timeout + streaks, and the app going unreachable mid-scan. **Never touch the config for these.** + Verify the app is up using the same playbook as SKILL.md's Step 1c / Step 6 exit-1 + diagnosis, restart or wait if needed, then retry the scan once — that retry does not + consume a config-iteration. If the same environment-class signal recurs on the retry, + report it as an environment problem and stop; do not keep retrying and do not reach for a + config edit to work around it. + +Performance signals — slow paths, rate-limited endpoints, a scan that ran long — are not a +gate concern at all. They route to the `optimize` skill's existing suggestion path (SKILL.md +Step 4, surfaced once per session for full scans ≥ 20 minutes), never to this gate. + +## On gaps: iterate the config + +When one or more config-class gaps are open, loop back to SKILL.md's Step 2b (tune existing +`stackhawk.yml`) with this shape: + +- **Batch, then rescan once.** Collect every gate-named fix from this scan's gap set — wire + the missing spec, enable `hawk.spider.ajax`, fix the `authentication:` block, add a config + for an unscanned surface — into a single edit pass. Rescan once after applying all of + them together. Don't rescan after each individual fix; that wastes scans the gate exists + to save. +- **Only iterate when the config actually changed.** If a reported gap has no config-level + fix — a monorepo slice with no visibility into the missing service, dynamic routing that + needs a code change, a spec that genuinely doesn't exist yet — report it and move on. + Don't burn an iteration re-scanning a config you didn't touch. +- **Iteration cap: 2 interactive, 1 autonomous — per config, and counted in rescans, not + edits.** A batch of five config edits followed by one rescan is one iteration. This + matches the Autonomous Security Loop's existing guard rail of at most one fix-rescan cycle + when running unattended. +- **At the cap, stop.** Report the remaining structural gaps by reason identifier, the + coverage route-diff evidence (the untouched-route list from check 1), and concrete next + steps — including any code-change recommendations discovery would make for surfaces that + can't be closed by config alone. Never claim the scan is done, or the app is secure, while + gaps remain open at the cap. +- **A severe coverage gap with all four structural checks green** is not a config-iteration + target — it means the scan reached everything it was told to reach and still found little. + Route it to SKILL.md's Step 6 empty-data handling instead: suggest the + `stackhawk-data-seed` skill rather than continuing to edit the scan config. + +**Additive-only.** Every fix applied during iteration must add coverage: wire a spec, turn +on the Ajax Spider, correct an auth recipe, add a missing surface's config. Iteration must +**never** widen `excludePaths`, raise `failureThreshold`, add `excludePlugins`, or otherwise +weaken the scan so it happens to pass the gate. If the only way to close a gap is to reduce +what gets scanned, that is not a pass — report the honest gap instead of manufacturing a +clean one. + +Environment-class gaps never enter this loop at all — see the Signal classes note above: +verify the app, retry once for free, and stop if it recurs. + +## Reporting rules (replace the old autonomy lock) + +- **Findings are always reported and always fixable**, independent of gate state. The gate + never withholds, delays, or downgrades a finding — a scan that under-covered the surface + can still have found a real, exploitable bug, and that bug gets fixed on this pass. +- **Never say the scan is "done and secure"** (or any equivalent framing) while a + config-class or environment-class gap is still open. Say plainly what was scanned, what + wasn't, and why — the reason identifiers exist so this statement is specific, not vague. +- **The autonomous post-code-change loop no longer treats exit 0 as silent success.** It + runs this gate after its scan, and if gaps remain once the autonomous iteration cap (1) is + reached, it reports them explicitly — reason identifiers, coverage route-diff evidence, + and next steps — instead of accepting a clean exit code at face value. +- **Performance-class signals are never gate gaps.** If health metrics point at slowness + rather than unreachability or all-4xx, that's an `optimize` suggestion, not a quality-gate + reason identifier. +- Keep every reported gap tied to one of the seven reason identifiers above so downstream + reporting — and the Step 4.5 handoff into fix-task generation — can consume it + programmatically rather than parsing prose. + +## Degradation + +`hawk op scan uris` and `hawk op scan config` need hawk op ≥ 0.11.7. Detect the absence by +the subcommand erroring or not existing at all — don't ask the user to check their version, +just degrade silently and keep going: + +- **Coverage (check 1)** degrades from a full scanned-vs-expected route diff to a single + before/after number: `url_count` from `hawk op scan get `. You lose the + untouched-route list; still report the number as evidence, just note it's coarser. +- **Surface-completeness (check 3)** degrades from reading the effective scan config via + `hawk op scan config` to local signals: read the per-surface `.yml` files directly to + confirm a surface's config exists and its spec block is present, and use the + `hawk validate config` output for each file to confirm the config is structurally sound. +- **Auth and health (checks 2 and 4)** are unaffected — `hawk op scan metrics` and + `hawk validate auth` work on all hawk op versions and remain the primary signal either way. + +`hawk op scan stats` does not exist yet (it's a follow-up ticket) but will eventually add +measured, per-path engine statistics rather than the grep- and `url_count`-based estimates +this reference relies on today. When it ships, prefer its measured signal over the inferred +approach described here. Until then, its absence is expected — degrade silently and don't +block or warn on it being missing. diff --git a/cursor/.cursor/rules/stackhawk-hawkscan.mdc b/cursor/.cursor/rules/stackhawk-hawkscan.mdc index 5f7df8f5..550b588f 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan.mdc @@ -50,6 +50,8 @@ The `api` skill wraps read-only StackHawk platform lookups via the `hawk` CLI (` | Triage a finding | `hawk op scan triage --scan --hash --status false-positive --note ""` | | Bulk triage from file | `hawk op scan triage --scan --from-file triage.yaml` | | Annotate w/o triage perm | `hawk op finding note --scan --hash --note "..."` | +| Get scanned URIs | `hawk op scan uris --format json` (hawk op ≥ 0.11.7; skip if absent) | +| Get effective scan config | `hawk op scan config ` (hawk op ≥ 0.11.7; skip if absent) | These reads require the combined `hawk` CLI; the `api` skill covers setup (`hawk init --browser` or the `HAWK_API_KEY` env var). @@ -116,32 +118,21 @@ enable only detected flags via `hawk op app tech-flags`; if none, skip. > 2. `CI=true` or `GITHUB_ACTIONS=true` → `CI` > 3. Git branch `main`/`master`/`production` → `Production`; `staging` → `Staging`; otherwise → `Development` -### Step 1a: Understand the Application +### Step 1a: Discover What to Scan -Before generating config, build a working understanding of the target: **how it runs -locally, how to reach its API, whether it's a SPA, and whether exercising it requires -auth.** Work in three passes; stop as soon as you can answer all four: +Before generating config, run code-first discovery: find every API surface in the repo +(REST, GraphQL, gRPC, SPA, ...), derive a route inventory per surface, and get the run +command, host/port, and auth shape needed to reach each one. Prefer the repo's own docs +over guessing, explore code to fill gaps, and ask the user directly for whatever remains +unresolved — never stall or invent. -1. **Read what the repo already documents.** Most repos describe how to run themselves — - prefer their own words over guessing, and treat them as authoritative. - → Source files and what to harvest: [`references/app-discovery.md`](references/app-discovery.md) +→ Full discovery workflow — per-surface detection, route-inventory derivation, gap + recommendations, and the user-confirmed summary required before the first scan: + [`references/scan-planning.md`](references/scan-planning.md) +→ Docs-first source table (which files to read and what to harvest): + [`references/app-discovery.md`](references/app-discovery.md) -2. **Explore to fill the gaps.** For anything the docs don't answer, investigate the repo - the way a developer would when handed an unfamiliar codebase — read the run/build - manifests, entry points, and route definitions — until you can state, concretely: - - **Run command + host/port** — how to start it locally and where it listens. - - **API style + base path** — REST (note an OpenAPI/Swagger spec if one is served or - checked in), GraphQL, gRPC, or a plain HTML web app. - - **SPA?** — a client-rendered JS front end changes what we scan. - - **Auth?** — does reaching the real endpoints require a login, and what shape does it - appear to use? This is an *understanding* signal only; picking, obtaining, and - configuring the credential is Step 1c / Phase 1c. - -3. **Confirm the unknowns with the user.** Never stall or invent. Ask directly for - anything still unresolved — most often the start command, host/port, or the login. When - file access is limited, skip straight to asking for all four. - -Record the four answers — Step 1c and Step 2 consume them. +Record the answers — Step 1c and Step 2 consume them. **SPA rule:** if the app is a client-rendered JS front end, never scan it without the Ajax Spider, and note that a separate backend API is usually the higher-value target. Full @@ -260,6 +251,8 @@ After seeding, re-run `hawk validate auth stackhawk.yml` and continue. Review the config against the current app state: +- **Gate reported gaps?** Tune what the gate names (wire spec, enable ajax spider, fix + auth) — additive-only; see [`references/scan-quality.md`](references/scan-quality.md). - **Low path count?** Check in order: SPA/JS app → enable `hawk.spider.ajax: true`; API spec available → wire `openApiConf`/`graphqlConf`; specific known-deep paths → add `seedPaths`. Omit `seedPaths` unless there is a specific reason — they're rarely needed when Ajax Spider or a spec is configured. - **Auth failing?** Verify `authentication` block; re-fetch the relevant recipe via `hawk config show
--text` (Phase 1c). - **Too noisy / too slow?** Add `app.excludePaths` or `app.includePaths`; tune `hawk.spider.maxDurationMinutes`. @@ -333,6 +326,31 @@ hawk rescan --scan-id --json-output # fast fix verification — r --- +## Step 4.5: Quality Gate + +Run this gate after **every** scan — full scans and rescans alike, any exit code — before +parsing findings into fix tasks. It is a feedback loop into config tuning, **not a +governor**: gate state never blocks a finding from being reported and fixed; a thin scan +that missed surface can still have found something real. + +Run the four structural checks and derive the expectation fresh each time, exactly as +`references/scan-quality.md` describes — coverage, auth, surface-completeness, and health. +On config-class gaps (`spec-not-wired`, `surface-unscanned`, `auth-validate-failed`, +`auth-wall`, `all-4xx`), loop back to Step 2b with an additive-only fix, batch every gap +into one edit pass, and rescan once. Iteration cap: 2 interactive / 1 autonomous, per +config, counted in rescans. On environment-class gaps (`env-unreachable`), never edit +config — verify the app is up, retry once for free, and stop if it recurs. Multi-config +repos gate sequentially, one config at a time, immediately after that config's scan. + +At the cap, or once checks are clean, proceed to Step 4 regardless — findings are always +reported and fixable. Never say a scan is "done and secure" while a gate gap is open; state +plainly what was scanned, what wasn't, and why, using the reason identifiers. + +→ Full check definitions, commands, degradation without hawk op ≥ 0.11.7, and the + iteration/reporting rules: [`references/scan-quality.md`](references/scan-quality.md) + +--- + ## Step 4: Parse Findings and Generate Fix Tasks Use `--json-output` for structured results (requires Dev Release v5.3.41+). Suppresses all @@ -420,6 +438,8 @@ initialized; skill is active. - Max one fix-rescan cycle per task — if findings remain after fixing, report them - Always announce: "Running security scan...", "Found N vulnerabilities, fixing...", "Rescanning..." - Interruptible — stop immediately if the user says to +- Report gate gaps — never silently accept exit 0 from a thin scan +- Never claim "done and secure" while gate gaps are open --- @@ -428,7 +448,7 @@ initialized; skill is active. - **Don't scan before the app is running.** HawkScan will exit 1 with a connection error. - **Try `https://` first — HawkScan accepts self-signed certificates.** Only fall back to `http://` if the scan actually fails to connect with a TLS error. - **Don't hardcode API keys or credentials in `stackhawk.yml`.** Use env vars. -- **Low path count ≠ clean app.** Feed an API spec or auth config before concluding the app is secure. If auth succeeds but endpoints return empty data, seed the backend (Phase 1c.6). +- **Never accept an exit-0 scan that fails the quality gate.** Untouched planned routes, auth-walls, or an unscanned surface mean config iteration, not done — see [`references/scan-quality.md`](references/scan-quality.md). - **Don't ignore exit code 42.** It's a deliberate signal that findings crossed the threshold — treat it as a build failure. - **String interpolation mid-value doesn't work.** `host: "https://${HOST}/api"` will NOT interpolate. Use `host: ${FULL_HOST_URL}`. - **Never refer to the scanner as ZAP.** The product is HawkScan / StackHawk. The underlying engine is HSTE, not ZAP. Neither "ZAP" nor "OWASP ZAP" should appear in output or instructions. diff --git a/scripts/generate-cursor-rules.sh b/scripts/generate-cursor-rules.sh index 60afcc67..139f0bb9 100755 --- a/scripts/generate-cursor-rules.sh +++ b/scripts/generate-cursor-rules.sh @@ -25,6 +25,8 @@ MAPPINGS=( "hawkscan/skills/hawkscan/references/installation.md|stackhawk-hawkscan-install|HawkScan installation: Homebrew (macOS), manifest-driven binary download, package installers (.pkg/.msi), verification (hawk version). Self-contained binary — no separate Java install needed.||false" "hawkscan/skills/hawkscan/references/false-positives.md|stackhawk-hawkscan-false-positives|HawkScan false positives guide: identifying false positives, deciding fix vs suppress, excludePaths and failureThreshold config, excluding scan plugins, reporting accepted risk.||false" "hawkscan/skills/hawkscan/references/platform-model.md|stackhawk-hawkscan-model|HawkScan StackHawk platform model: Org → App → Env → Scan hierarchy, what goes in stackhawk.yml vs what doesn't, finding triage state machine (New/Accepted/False Positive/Reopened) and agent behavior per state, tags for commit traceability (_STACKHAWK_GIT_COMMIT_SHA, _STACKHAWK_GIT_BRANCH), technology flags (platform UI only — no API yet), hawk rescan --scan-id for fast fix verification in the agentic loop, create vs reuse decision trees for Apps and Envs.||false" + "hawkscan/skills/hawkscan/references/scan-planning.md|stackhawk-hawkscan-scan-planning|HawkScan scan-planning (discovery) reference: code-first discovery of an app's API surfaces before writing or editing stackhawk.yml — reading repo docs first, per-framework route inventory, recommending code changes for structural gaps, asking the user instead of guessing, and producing a per-surface stackhawk.yml config. Use for the first scan of a repo, a quality-gate structural gap, or a user-requested re-plan.||false" + "hawkscan/skills/hawkscan/references/scan-quality.md|stackhawk-hawkscan-scan-quality|HawkScan scan-quality (post-scan quality gate) reference: run after every scan and rescan, before findings become fix tasks. Derives a fresh coverage expectation each run (spec-wired or route-inventory), the four coverage checks, and a bounded, additive-only stackhawk.yml config iteration to close gaps — never blocks a real finding from being reported and fixed.||false" "api/skills/api/SKILL.md|stackhawk-api|Use when querying the StackHawk platform for security reporting, findings analysis, or app management. Triggers include \"stackhawk api\", \"security posture\", \"findings report\", \"show me findings\", \"untriaged findings\", \"which apps\", \"scan history\", \"security dashboard\", \"triage\", \"what needs attention\". Uses the combined hawk CLI (hawk op …) for all platform queries. Do NOT use for running scans (use the stackhawk-hawkscan rule for \"scan my app\", \"hawkscan\", \"stackhawk.yml\", \"DAST\") or for fixing/remediating code or vulnerabilities — this skill only reads and reports platform data.||false" "api/skills/api/references/hawk-op-shortcuts.md|stackhawk-api-hawk-op|hawk op shortcuts for the StackHawk API: user intent → single hawk op command mapping for scan drill-down, app/user/team/policy/repo/oas/config/secret/audit listing, hosted scan control, env management. All platform queries go through hawk op — no raw REST fallback.||false" "api/skills/api/references/reporting-recipes.md|stackhawk-api-recipes|StackHawk API reporting recipes: org security posture summary, app deep dive (scan -> alerts -> findings), stale apps detection, scan diff (what changed since last scan). Pre-built compositions using hawk op commands. Prefer hawk op shortcuts (stackhawk-api-hawk-op) for the deep-dive chain; use these recipes for the per-env untriaged posture view.||false" From 02021743a4d7d9ee6d6a5ca3f709cc61034943c0 Mon Sep 17 00:00:00 2001 From: KC Berg Date: Tue, 7 Jul 2026 17:42:53 -0600 Subject: [PATCH 07/29] fix(hawkscan): wire quality gate into autonomous loop reference + review polish Insert an explicit Step 4.5 quality-gate invocation into the autonomous loop's 7-step detail (before findings triage and again after the rescan), cap autonomous config-fix iterations at 1, reconcile the exit-1 retry guard rail with the gate's environment-class rule, and qualify the clean-report wording so a zero-findings rescan with open gate gaps is never reported as "done and secure". Also: add missing soapConf to scan-planning's spec-wiring list, remove an overclaimed SKILL.md-consumes-identifiers contract from scan-quality.md, qualify SKILL.md's Step 4.5 "any exit code" for the no-scan-started case, restore precision to the Step 1a discovery hand-off line, and retitle app-discovery.md so its H1 matches its actual (docs-first table) content. Regenerated cursor/ .mdc mirrors. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01WWyT8CN77z9xrJhRLjrtJC --- .../stackhawk-hawkscan-scan-planning.mdc | 6 ++-- .../rules/stackhawk-hawkscan-scan-quality.mdc | 14 ++++---- cursor/.cursor/rules/stackhawk-hawkscan.mdc | 11 +++--- plugins/hawkscan/skills/hawkscan/SKILL.md | 11 +++--- .../hawkscan/references/app-discovery.md | 2 +- .../hawkscan/references/autonomous-loop.md | 36 +++++++++++++++---- .../hawkscan/references/scan-planning.md | 6 ++-- .../hawkscan/references/scan-quality.md | 14 ++++---- 8 files changed, 65 insertions(+), 35 deletions(-) diff --git a/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc b/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc index ca9baa5e..d2ed8fb1 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc @@ -146,9 +146,9 @@ so the rationale for what scans first is visible, not implicit. For each surface: 1. Wire the spec you found or confirmed in discovery: `openApiConf` for REST, - `graphqlConf` for GraphQL, `grpcConf` + the `.proto` for gRPC. Use - `hawk config show --text` for the exact field shape — don't hand-write - the block from memory. + `graphqlConf` for GraphQL, `grpcConf` + the `.proto` for gRPC, `soapConf` + the WSDL for + SOAP. Use `hawk config show --text` for the exact field shape — don't + hand-write the block from memory. 2. Configure auth per Phase 1c in SKILL.md — one auth block per surface if surfaces have different login mechanisms (common when a gateway and an upstream service use different credential types). diff --git a/cursor/.cursor/rules/stackhawk-hawkscan-scan-quality.mdc b/cursor/.cursor/rules/stackhawk-hawkscan-scan-quality.mdc index 5535d336..c78f48c3 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan-scan-quality.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan-scan-quality.mdc @@ -70,10 +70,10 @@ it against. Build that expectation the same way every time, right after the scan ## The four checks Run all four after every scan. Each has its own command, pass condition, and a stable -reason identifier used when reporting a gap — SKILL.md's fix-task step consumes these -identifiers directly, so keep them exactly as written here: `coverage-gap`, -`spec-not-wired`, `surface-unscanned`, `auth-validate-failed`, `auth-wall`, `all-4xx`, -`env-unreachable`. +reason identifier used when reporting a gap — this gate's own reporting (Step 4.5 in +SKILL.md) uses these identifiers directly, so keep them exactly as written here across any +edits to this file: `coverage-gap`, `spec-not-wired`, `surface-unscanned`, +`auth-validate-failed`, `auth-wall`, `all-4xx`, `env-unreachable`. | # | Check | What it measures | Command(s) | Pass condition | Reason identifier(s) | |---|-------|-------------------|------------|-----------------|------------------------| @@ -155,9 +155,9 @@ verify the app, retry once for free, and stop if it recurs. - **Performance-class signals are never gate gaps.** If health metrics point at slowness rather than unreachability or all-4xx, that's an `optimize` suggestion, not a quality-gate reason identifier. -- Keep every reported gap tied to one of the seven reason identifiers above so downstream - reporting — and the Step 4.5 handoff into fix-task generation — can consume it - programmatically rather than parsing prose. +- Keep every reported gap tied to one of the seven reason identifiers above so this gate's + own reporting stays specific rather than parsing prose — the identifiers must stay stable + across edits to this file. ## Degradation diff --git a/cursor/.cursor/rules/stackhawk-hawkscan.mdc b/cursor/.cursor/rules/stackhawk-hawkscan.mdc index 550b588f..2f632d56 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan.mdc @@ -132,7 +132,8 @@ unresolved — never stall or invent. → Docs-first source table (which files to read and what to harvest): [`references/app-discovery.md`](references/app-discovery.md) -Record the answers — Step 1c and Step 2 consume them. +Record what discovery produced — surfaces, run command, host/port, auth shape — Step 1c and +Step 2 consume them. **SPA rule:** if the app is a client-rendered JS front end, never scan it without the Ajax Spider, and note that a separate backend API is usually the higher-value target. Full @@ -329,9 +330,11 @@ hawk rescan --scan-id --json-output # fast fix verification — r ## Step 4.5: Quality Gate Run this gate after **every** scan — full scans and rescans alike, any exit code — before -parsing findings into fix tasks. It is a feedback loop into config tuning, **not a -governor**: gate state never blocks a finding from being reported and fixed; a thin scan -that missed surface can still have found something real. +parsing findings into fix tasks. Exception: on exit 1 where no scan was actually started +(config parse failure, app unreachable pre-scan), there is no `scanId` to gate against — skip +straight to Step 6's diagnosis / the environment-class path instead. It is a feedback loop +into config tuning, **not a governor**: gate state never blocks a finding from being reported +and fixed; a thin scan that missed surface can still have found something real. Run the four structural checks and derive the expectation fresh each time, exactly as `references/scan-quality.md` describes — coverage, auth, surface-completeness, and health. diff --git a/plugins/hawkscan/skills/hawkscan/SKILL.md b/plugins/hawkscan/skills/hawkscan/SKILL.md index a05323c4..9b6e54e9 100644 --- a/plugins/hawkscan/skills/hawkscan/SKILL.md +++ b/plugins/hawkscan/skills/hawkscan/SKILL.md @@ -143,7 +143,8 @@ unresolved — never stall or invent. → Docs-first source table (which files to read and what to harvest): [`references/app-discovery.md`](references/app-discovery.md) -Record the answers — Step 1c and Step 2 consume them. +Record what discovery produced — surfaces, run command, host/port, auth shape — Step 1c and +Step 2 consume them. **SPA rule:** if the app is a client-rendered JS front end, never scan it without the Ajax Spider, and note that a separate backend API is usually the higher-value target. Full @@ -340,9 +341,11 @@ hawk rescan --scan-id --json-output # fast fix verification — r ## Step 4.5: Quality Gate Run this gate after **every** scan — full scans and rescans alike, any exit code — before -parsing findings into fix tasks. It is a feedback loop into config tuning, **not a -governor**: gate state never blocks a finding from being reported and fixed; a thin scan -that missed surface can still have found something real. +parsing findings into fix tasks. Exception: on exit 1 where no scan was actually started +(config parse failure, app unreachable pre-scan), there is no `scanId` to gate against — skip +straight to Step 6's diagnosis / the environment-class path instead. It is a feedback loop +into config tuning, **not a governor**: gate state never blocks a finding from being reported +and fixed; a thin scan that missed surface can still have found something real. Run the four structural checks and derive the expectation fresh each time, exactly as `references/scan-quality.md` describes — coverage, auth, surface-completeness, and health. diff --git a/plugins/hawkscan/skills/hawkscan/references/app-discovery.md b/plugins/hawkscan/skills/hawkscan/references/app-discovery.md index f35828ea..d184f0d5 100644 --- a/plugins/hawkscan/skills/hawkscan/references/app-discovery.md +++ b/plugins/hawkscan/skills/hawkscan/references/app-discovery.md @@ -1,4 +1,4 @@ -# App Discovery Reference +# App Discovery — Docs-First Source Table This file's discovery procedure has been superseded by `references/scan-planning.md` — see Step 1a in SKILL.md for the full workflow (per-surface detection, route-inventory diff --git a/plugins/hawkscan/skills/hawkscan/references/autonomous-loop.md b/plugins/hawkscan/skills/hawkscan/references/autonomous-loop.md index bff60692..29af898f 100644 --- a/plugins/hawkscan/skills/hawkscan/references/autonomous-loop.md +++ b/plugins/hawkscan/skills/hawkscan/references/autonomous-loop.md @@ -48,7 +48,21 @@ hawk scan --json-output Capture `scan.id` from the JSON output — you'll need it for rescan. -**5. If findings exist** +**5. Quality gate** + +Before triaging findings, run the quality gate (`scan-quality.md`'s four checks — coverage, +auth, surface-completeness, health) against this scan. On config-class gaps +(`spec-not-wired`, `surface-unscanned`, `auth-validate-failed`, `auth-wall`, `all-4xx`), loop +back to config tuning with a single additive-only batched fix and rescan once — the +autonomous cap here is **1** config-fix iteration (interactive workflows get 2; this +unattended loop gets 1). On environment-class gaps (`env-unreachable`: connection failures, +timeouts, app unreachable mid-scan), never edit the config: verify the app is up, retry the +scan once for free, and if the same environment-class signal recurs on the retry, report it +as an environment problem and stop — do not retry again or reach for a config edit. At the +cap, or once the gate is clean, proceed to the next step regardless — findings are always +reported and fixable independent of gate state. + +**6. If findings exist** Run the Step 5 triage filter first (per-path `status` field): - Skip `FALSE_POSITIVE` / `RISK_ACCEPTED` paths entirely @@ -63,7 +77,7 @@ Fix in severity order: High → Medium → Low; within same severity: injection Commit format: `fix: resolve [CWE-XXX] [vulnerability type] found by HawkScan` -**6. Rescan** +**7. Rescan** Rescan is the default for all fix-verify cycles: @@ -87,12 +101,20 @@ Decision table for when you're tempted to skip rescan: | "Rescan might miss something" | Rescan re-runs exactly the plugins that fired on the parent scan. It's more targeted, not less. | | "The fix added new endpoints that need scanning" | This IS a full-scan condition — use full scan. | -**7. Report** +**8. Quality gate (post-rescan)** -- If clean: "Rescan complete. Zero new findings. All security issues have been resolved." +Run the same gate again against the rescan, per the rule that it runs after every scan and +rescan alike. The autonomous cap still applies per config across this whole task (scan + +rescan together), not per gate invocation — don't treat the rescan as a fresh budget for a +second config-fix iteration. + +**9. Report** + +- If clean **and no gate gaps remain open**: "Rescan complete. Zero new findings. All security issues have been resolved." +- If clean **but gate gaps remain open** (cap reached with config-class gaps still unresolved, or an environment-class gap that recurred): report the gaps by reason identifier, the coverage evidence, and the remaining next steps. Do not say "all security issues have been resolved" or otherwise imply the app is done and secure while a gap is open. - If findings remain: "Rescan found [N] remaining issues that require manual review:" and list them. - If findings were filtered by triage state: "Skipped [X] findings already triaged as RISK_ACCEPTED / FALSE_POSITIVE." -- If findings were marked FALSE_POSITIVE in Step 5: "Marked [N] findings as false positive — review at https://app.stackhawk.com/scans/\". +- If findings were marked FALSE_POSITIVE in step 6: "Marked [N] findings as false positive — review at https://app.stackhawk.com/scans/\". --- @@ -102,4 +124,6 @@ Decision table for when you're tempted to skip rescan: - **Max one fix-rescan cycle per task.** If the rescan still has findings after fixing, report the remaining issues rather than looping indefinitely. The user can ask for a follow-up. - **Always announce what you're doing.** The developer should see "Running security scan...", "Found N vulnerabilities, fixing...", "Rescanning to verify..." in the output. - **Interruptible.** If the user interrupts or says to stop, stop immediately. -- **Don't block on scan failures.** If `hawk scan` exits with code 1 (config error, app unreachable), report the error and suggest fixes rather than retrying in a loop. +- **Don't block on scan failures.** If `hawk scan` exits with code 1 (config error, app unreachable), report the error rather than retrying in a loop — with one exception: an environment-class failure (connection refused, timeout, unreachable) gets exactly one free retry after verifying the app is up. If the same environment-class signal recurs on that retry, report it and stop; don't retry a second time. +- **Report gate gaps — never silently accept exit 0 from a thin scan.** +- **Never claim "done and secure" while gate gaps are open.** diff --git a/plugins/hawkscan/skills/hawkscan/references/scan-planning.md b/plugins/hawkscan/skills/hawkscan/references/scan-planning.md index 7e27e07b..957fd61b 100644 --- a/plugins/hawkscan/skills/hawkscan/references/scan-planning.md +++ b/plugins/hawkscan/skills/hawkscan/references/scan-planning.md @@ -140,9 +140,9 @@ so the rationale for what scans first is visible, not implicit. For each surface: 1. Wire the spec you found or confirmed in discovery: `openApiConf` for REST, - `graphqlConf` for GraphQL, `grpcConf` + the `.proto` for gRPC. Use - `hawk config show --text` for the exact field shape — don't hand-write - the block from memory. + `graphqlConf` for GraphQL, `grpcConf` + the `.proto` for gRPC, `soapConf` + the WSDL for + SOAP. Use `hawk config show --text` for the exact field shape — don't + hand-write the block from memory. 2. Configure auth per Phase 1c in SKILL.md — one auth block per surface if surfaces have different login mechanisms (common when a gateway and an upstream service use different credential types). diff --git a/plugins/hawkscan/skills/hawkscan/references/scan-quality.md b/plugins/hawkscan/skills/hawkscan/references/scan-quality.md index 2288e829..8f8a0ff3 100644 --- a/plugins/hawkscan/skills/hawkscan/references/scan-quality.md +++ b/plugins/hawkscan/skills/hawkscan/references/scan-quality.md @@ -64,10 +64,10 @@ it against. Build that expectation the same way every time, right after the scan ## The four checks Run all four after every scan. Each has its own command, pass condition, and a stable -reason identifier used when reporting a gap — SKILL.md's fix-task step consumes these -identifiers directly, so keep them exactly as written here: `coverage-gap`, -`spec-not-wired`, `surface-unscanned`, `auth-validate-failed`, `auth-wall`, `all-4xx`, -`env-unreachable`. +reason identifier used when reporting a gap — this gate's own reporting (Step 4.5 in +SKILL.md) uses these identifiers directly, so keep them exactly as written here across any +edits to this file: `coverage-gap`, `spec-not-wired`, `surface-unscanned`, +`auth-validate-failed`, `auth-wall`, `all-4xx`, `env-unreachable`. | # | Check | What it measures | Command(s) | Pass condition | Reason identifier(s) | |---|-------|-------------------|------------|-----------------|------------------------| @@ -149,9 +149,9 @@ verify the app, retry once for free, and stop if it recurs. - **Performance-class signals are never gate gaps.** If health metrics point at slowness rather than unreachability or all-4xx, that's an `optimize` suggestion, not a quality-gate reason identifier. -- Keep every reported gap tied to one of the seven reason identifiers above so downstream - reporting — and the Step 4.5 handoff into fix-task generation — can consume it - programmatically rather than parsing prose. +- Keep every reported gap tied to one of the seven reason identifiers above so this gate's + own reporting stays specific rather than parsing prose — the identifiers must stay stable + across edits to this file. ## Degradation From 0c6c7b752559704ca4134f519aaff1e694326b88 Mon Sep 17 00:00:00 2001 From: KC Berg Date: Tue, 7 Jul 2026 17:46:10 -0600 Subject: [PATCH 08/29] fix(hawkscan): count-proof autonomous loop label Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01WWyT8CN77z9xrJhRLjrtJC --- cursor/.cursor/rules/stackhawk-hawkscan.mdc | 2 +- plugins/hawkscan/skills/hawkscan/SKILL.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/cursor/.cursor/rules/stackhawk-hawkscan.mdc b/cursor/.cursor/rules/stackhawk-hawkscan.mdc index 2f632d56..ef910c00 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan.mdc @@ -433,7 +433,7 @@ initialized; skill is active. **Skip for:** documentation-only changes; config edits without code changes; exploratory tasks; when the user explicitly says to skip. -**The loop** (full 7-step detail with guard rails): +**The loop** (full step-by-step detail with guard rails): → [`references/autonomous-loop.md`](references/autonomous-loop.md) **Guard rails summary:** diff --git a/plugins/hawkscan/skills/hawkscan/SKILL.md b/plugins/hawkscan/skills/hawkscan/SKILL.md index 9b6e54e9..9341b5c0 100644 --- a/plugins/hawkscan/skills/hawkscan/SKILL.md +++ b/plugins/hawkscan/skills/hawkscan/SKILL.md @@ -444,7 +444,7 @@ initialized; skill is active. **Skip for:** documentation-only changes; config edits without code changes; exploratory tasks; when the user explicitly says to skip. -**The loop** (full 7-step detail with guard rails): +**The loop** (full step-by-step detail with guard rails): → [`references/autonomous-loop.md`](references/autonomous-loop.md) **Guard rails summary:** From 789fcad2da9a6441f935fe17daab0e60c616a36c Mon Sep 17 00:00:00 2001 From: KC Berg Date: Tue, 7 Jul 2026 18:16:25 -0600 Subject: [PATCH 09/29] feat(hawkscan): settle multi-surface config naming + app/env mapping Same applicationId, one environment per surface (dev-rest / dev-graphql), configs named stackhawk-.yml. Decided with KC at the dogfood checkpoint 2026-07-07. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01WWyT8CN77z9xrJhRLjrtJC --- cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc | 8 ++++++++ .../hawkscan/skills/hawkscan/references/scan-planning.md | 8 ++++++++ 2 files changed, 16 insertions(+) diff --git a/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc b/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc index d2ed8fb1..544b6192 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc @@ -143,6 +143,14 @@ SPA frontend goes last (per the SPA guidance above, it typically finds only head issues). This order is not cosmetic — state it explicitly in the summary you show the user, so the rationale for what scans first is visible, not implicit. +**Naming and platform mapping for multi-surface repos:** a single-surface repo uses plain +`stackhawk.yml`. With multiple surfaces, name each config `stackhawk-.yml` (e.g. +`stackhawk-rest.yml`, `stackhawk-graphql.yml`) and run each scan as +`hawk scan stackhawk-.yml`. All surfaces share one `applicationId`; give each +surface its own environment (e.g. `dev-rest`, `dev-graphql`). Findings compare scan-to-scan +within an environment, so per-surface envs keep each surface's history coherent instead of +interleaving a REST scan's baseline with a GraphQL scan's. + For each surface: 1. Wire the spec you found or confirmed in discovery: `openApiConf` for REST, diff --git a/plugins/hawkscan/skills/hawkscan/references/scan-planning.md b/plugins/hawkscan/skills/hawkscan/references/scan-planning.md index 957fd61b..f3b18212 100644 --- a/plugins/hawkscan/skills/hawkscan/references/scan-planning.md +++ b/plugins/hawkscan/skills/hawkscan/references/scan-planning.md @@ -137,6 +137,14 @@ SPA frontend goes last (per the SPA guidance above, it typically finds only head issues). This order is not cosmetic — state it explicitly in the summary you show the user, so the rationale for what scans first is visible, not implicit. +**Naming and platform mapping for multi-surface repos:** a single-surface repo uses plain +`stackhawk.yml`. With multiple surfaces, name each config `stackhawk-.yml` (e.g. +`stackhawk-rest.yml`, `stackhawk-graphql.yml`) and run each scan as +`hawk scan stackhawk-.yml`. All surfaces share one `applicationId`; give each +surface its own environment (e.g. `dev-rest`, `dev-graphql`). Findings compare scan-to-scan +within an environment, so per-surface envs keep each surface's history coherent instead of +interleaving a REST scan's baseline with a GraphQL scan's. + For each surface: 1. Wire the spec you found or confirmed in discovery: `openApiConf` for REST, From c3a9f6742dbfc0d054b356a627588e67510ba006 Mon Sep 17 00:00:00 2001 From: KC Berg Date: Wed, 8 Jul 2026 15:24:25 -0600 Subject: [PATCH 10/29] feat(hawkscan): openapi-specs accuracy reference + base-path-mismatch gate signal MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Dogfood against ecordel-restapi showed a scan can 404 on 71% of its URIs when app.host and the spec disagree on the servlet context path, while login still works — and the old guidance only fired when NO spec was found, trusting any checked-in file. - New references/openapi-specs.md: framework-generic procedure for an ACCURATE spec — prefer a spec the running app serves; suggest the concrete code/build change that makes the framework generate one (12-stack matrix + find-out fallback); base/context-path alignment rules; mandatory resolve-check (curl host+spec-path) before scanning; hand-derived spec as last resort. - scan-planning.md: spec verdict is now verified-accurate | stale/unverified | missing (a stale spec is functionally missing); gap recommendation covers inaccurate specs, not just absent ones. - scan-quality.md + SKILL.md Step 4.5: new base-path-mismatch config-class gate signal for the mixed 404/200 pattern all-4xx misses. - Cursor MAPPINGS + regenerated rules (idempotent); bump 2.2.0 -> 2.3.0. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01WWyT8CN77z9xrJhRLjrtJC --- .claude-plugin/marketplace.json | 12 +- .claude/skills/skill-authoring/SKILL.md | 2 +- .codex-plugin/plugin.json | 2 +- VERSION | 2 +- .../stackhawk-hawkscan-openapi-specs.mdc | 166 ++++++++++++++++++ .../stackhawk-hawkscan-scan-planning.mdc | 35 ++-- .../rules/stackhawk-hawkscan-scan-quality.mdc | 8 +- cursor/.cursor/rules/stackhawk-hawkscan.mdc | 6 +- gemini-extension.json | 2 +- plugin.json | 2 +- plugins/api/.claude-plugin/plugin.json | 2 +- plugins/api/.codex-plugin/plugin.json | 2 +- plugins/api/skills/api/SKILL.md | 2 +- .../hawkscan-ci/.claude-plugin/plugin.json | 2 +- plugins/hawkscan-ci/.codex-plugin/plugin.json | 2 +- .../hawkscan-ci/skills/hawkscan-ci/SKILL.md | 2 +- plugins/hawkscan/.claude-plugin/plugin.json | 2 +- plugins/hawkscan/.codex-plugin/plugin.json | 2 +- plugins/hawkscan/skills/hawkscan/SKILL.md | 8 +- .../hawkscan/references/openapi-specs.md | 160 +++++++++++++++++ .../hawkscan/references/scan-planning.md | 35 ++-- .../hawkscan/references/scan-quality.md | 8 +- plugins/optimize/.claude-plugin/plugin.json | 2 +- plugins/optimize/.codex-plugin/plugin.json | 2 +- plugins/optimize/skills/optimize/SKILL.md | 2 +- .../.claude-plugin/plugin.json | 2 +- .../.codex-plugin/plugin.json | 2 +- .../skills/stackhawk-data-seed/SKILL.md | 2 +- plugins/wingman/.claude-plugin/plugin.json | 2 +- plugins/wingman/.codex-plugin/plugin.json | 2 +- scripts/generate-cursor-rules.sh | 1 + 31 files changed, 423 insertions(+), 58 deletions(-) create mode 100644 cursor/.cursor/rules/stackhawk-hawkscan-openapi-specs.mdc create mode 100644 plugins/hawkscan/skills/hawkscan/references/openapi-specs.md diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index 622d7f9b..1000577d 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -14,7 +14,7 @@ "category": "security", "source": "./plugins/hawkscan", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/hawkscan", - "version": "2.2.0" + "version": "2.3.0" }, { "name": "stackhawk-api", @@ -22,7 +22,7 @@ "category": "security", "source": "./plugins/api", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/api", - "version": "2.2.0" + "version": "2.3.0" }, { "name": "hawkscan-ci", @@ -30,7 +30,7 @@ "category": "security", "source": "./plugins/hawkscan-ci", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/hawkscan-ci", - "version": "2.2.0" + "version": "2.3.0" }, { "name": "stackhawk-data-seed", @@ -38,7 +38,7 @@ "category": "security", "source": "./plugins/stackhawk-data-seed", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/stackhawk-data-seed", - "version": "2.2.0" + "version": "2.3.0" }, { "name": "wingman", @@ -46,7 +46,7 @@ "category": "security", "source": "./plugins/wingman", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/wingman", - "version": "2.2.0" + "version": "2.3.0" }, { "name": "stackhawk-optimize", @@ -54,7 +54,7 @@ "category": "security", "source": "./plugins/optimize", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/optimize", - "version": "2.2.0" + "version": "2.3.0" } ] } diff --git a/.claude/skills/skill-authoring/SKILL.md b/.claude/skills/skill-authoring/SKILL.md index 582cd20a..376eecff 100644 --- a/.claude/skills/skill-authoring/SKILL.md +++ b/.claude/skills/skill-authoring/SKILL.md @@ -1,6 +1,6 @@ --- name: skill-authoring -version: 2.2.0 +version: 2.3.0 description: > Guides authoring and maintaining agent skills in this repository — enforcing Anthropic best practices, bumping versions, regenerating Cursor rules, and diff --git a/.codex-plugin/plugin.json b/.codex-plugin/plugin.json index 8e582044..71e5a48c 100644 --- a/.codex-plugin/plugin.json +++ b/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk", - "version": "2.2.0", + "version": "2.3.0", "description": "Dynamic application and API security testing (DAST) powered by HawkScan. Configure scans, interpret findings, and query the StackHawk platform API.", "author": { "name": "StackHawk", diff --git a/VERSION b/VERSION index e3a4f193..cc6612c3 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -2.2.0 \ No newline at end of file +2.3.0 \ No newline at end of file diff --git a/cursor/.cursor/rules/stackhawk-hawkscan-openapi-specs.mdc b/cursor/.cursor/rules/stackhawk-hawkscan-openapi-specs.mdc new file mode 100644 index 00000000..7f620d0a --- /dev/null +++ b/cursor/.cursor/rules/stackhawk-hawkscan-openapi-specs.mdc @@ -0,0 +1,166 @@ +--- +description: > + HawkScan OpenAPI spec accuracy reference: framework-generic procedure for getting an accurate OpenAPI spec into openApiConf for a REST surface — prefer a spec the running app serves, suggest the code/build change that makes the framework generate one, verify it resolves against the app (base/context path), and derive one by hand only as a last resort. Use whenever wiring or fixing openApiConf, or when a scan's spec paths return 404s. +globs: +alwaysApply: false +--- +# OpenAPI Spec Accuracy Reference + +A REST scan is only as good as the spec behind it. An **accurate** spec is one whose +operations match the app's real routes *and* resolve against the scanned host — same base +path, same context path. An inaccurate spec is worse than no spec: HawkScan faithfully +requests every path it declares, so wrong paths become a wall of 404s that bury real +coverage and generate recurring false-positive noise on every rescan. + +This reference is framework-generic. Use it whenever a REST surface needs a spec wired into +`openApiConf` — discovery (SKILL.md Step 1a / `scan-planning.md`) points here, and the +quality gate points here when it reports a `spec-not-wired` or coverage gap. + +## Contents +- [The one rule: an unverified spec is a hypothesis](#the-one-rule-an-unverified-spec-is-a-hypothesis) +- [Get an accurate spec (preference order)](#get-an-accurate-spec-preference-order) +- [Framework spec-generation matrix](#framework-spec-generation-matrix) +- [Suggesting the code/build change](#suggesting-the-codebuild-change) +- [Base path and context path](#base-path-and-context-path) +- [Mandatory: verify the spec resolves](#mandatory-verify-the-spec-resolves) +- [Deriving a spec by hand (no tooling)](#deriving-a-spec-by-hand-no-tooling) + +--- + +## The one rule: an unverified spec is a hypothesis + +A checked-in `openapi.yaml`, a spec you generated, and a spec the app serves are all +*claims* about the API until you prove they resolve against the running app. Never wire a +spec into `openApiConf` and scan without the verification in "Mandatory: verify the spec +resolves" below. A checked-in file is the least trustworthy source — it drifts from the code, +can be stale, hand-maintained, or (as seen in the wild) emptied by a commit — so treat its +presence as a starting hypothesis, not a done task. + +## Get an accurate spec (preference order) + +Prefer a spec the framework produces from the *running app* over any static artifact. Work +down this list; stop at the first that yields a spec that passes verification. + +1. **The running app already serves one.** Probe the common served endpoints once the app is + up (see the matrix for framework-specific paths): + ```bash + for p in /openapi.json /openapi.yaml /v3/api-docs /v3/api-docs.yaml \ + /swagger.json /swagger/v1/swagger.json /api-docs /q/openapi; do + curl -sf -o /dev/null -w "%{http_code} $p\n" "$APP_BASE_URL$p" + done + ``` + A 200 here is the best source — it reflects real routing, including base/context path. + Wire it live with `openApiConf.path: ` so the scan always reads the current + spec. +2. **The framework can generate/serve one with a small change, but it isn't enabled.** This is + the ideal outcome — see "Suggesting the code/build change". Add the tooling, rebuild, + then fetch as in step 1. +3. **Only a static spec is checked in.** Verify it's current (path count matches the route + grep from discovery; paths resolve — see verification). If it's stale, prefer regenerating + via step 1/2. If you must use it, wire `openApiConf.filePath: ` and verify. +4. **No spec and no tooling for this stack.** Derive one by hand from the route inventory — + see "Deriving a spec by hand". + +## Framework spec-generation matrix + +Examples, not an exhaustive list. The method generalizes: identify the framework from the +manifest, then determine its idiomatic OAS support. For anything not listed, **find out** — +`hawk config show`/docs won't help here, so search "`` openapi generation" or +check the framework's own docs before concluding none exists. + +| Stack | Idiomatic OAS tooling | Serves at | Notes | +|-------|----------------------|-----------|-------| +| FastAPI (Python) | built in | `/openapi.json` | Already served — no change needed; just probe. | +| NestJS (Node) | `@nestjs/swagger` | `/api-json` (configurable) | First-party. | +| Express / Fastify (Node) | `swagger-jsdoc` + `swagger-ui-express`; `@fastify/swagger` | configurable | JSDoc-annotation or plugin based. | +| Spring Boot (Java/Kotlin) | `springdoc-openapi-starter-webmvc-ui` | `/v3/api-docs` | Reflects `server.servlet.context-path` in its server URL. | +| Quarkus (Java) | `quarkus-smallrye-openapi` | `/q/openapi` | Often on by default in dev. | +| Django REST Framework | `drf-spectacular` | `/api/schema/` | `drf-yasg` is the older alternative. | +| Flask (Python) | `apiflask`, `flasgger`, `flask-smorest` | configurable | — | +| ASP.NET Core | `Swashbuckle.AspNetCore` or `NSwag` | `/swagger/v1/swagger.json` | Often scaffolded already. | +| Go | `swaggo/swag`, `danielgtaylor/huma` | configurable | swag generates from annotations. | +| Rails | `rswag` | configurable | Spec-from-request-specs. | +| Laravel (PHP) | `l5-swagger`, `scramble` | configurable | Scramble infers without annotations. | +| Rust | `utoipa`, `aide` | configurable | — | + +## Suggesting the code/build change + +When the framework can emit an accurate spec but doesn't yet (preference step 2), this is the +highest-value move: a served, framework-generated spec is accurate by construction and stays +current as the code changes. Explore and suggest a **concrete** change — not "consider adding +OpenAPI support": + +- Name the exact dependency and where it goes (the build file + coordinate/version). +- Show the minimal config/annotation to expose the endpoint, and state the served path. +- State the DAST value it unlocks ("wires N routes HawkScan otherwise can't see; replaces a + stale/hand-derived spec that would 404"). +- Scope it dev/test-only where the framework allows, so it needn't ship to production. + +Present this as a recommendation with the diff and let the user decide — the skill does not +apply unrequested code changes (see "Recommend code changes for gaps" in `scan-planning.md`). +Make the suggestion good enough that saying yes is a one-step change, then rebuild and fetch +the served spec. + +## Base path and context path + +The most common accuracy failure is a base-path mismatch. HawkScan builds each request as the +configured host **plus the spec's operation path**; a spec whose paths omit the app's base or +context path 404s on every operation. Detect the app's base path during discovery and make +the effective `host + spec-path` land on real routes: + +| Framework | Where the base/context path lives | +|-----------|-----------------------------------| +| Spring MVC | `server.servlet.context-path` (application.yml/properties) | +| Spring WebFlux | `spring.webflux.base-path` | +| Express | the mount prefix in `app.use('/api/v1', router)` | +| Django | the URL prefix in the root `urls.py` `include()` | +| ASP.NET | `UsePathBase(...)` / route prefix | +| Rails | `scope`/`namespace` in `config/routes.rb` | + +Two ways to make it resolve — pick one and be consistent: +- Put the base path in `app.host` (e.g. `http://localhost:5000/api/v1`) and keep spec paths, + `loginPath`, and `testPath` **relative**; or +- Keep `app.host` at the origin and ensure the spec's operation paths already include the base + path (a framework-served spec usually does). + +Do **not** split the difference — e.g. stripping the context path from `app.host` and +hardcoding it only into `loginPath`/`testPath`. That fixes auth but leaves every +`openApiConf` path unprefixed, which is exactly how a scan ends up 71% 404s while login still +works. If `hawk validate auth` misbehaves with a path component in `host`, resolve it without +desyncing the spec's base path — verify the outcome with the resolve check below. + +## Mandatory: verify the spec resolves + +Before the first real scan, prove the wired spec produces real requests — regardless of +source. This is the check that catches base-path mismatch, stale specs, and wrong hosts. + +1. **Count sanity.** Spec operation count should be in the same ballpark as the route grep + from discovery. A spec with far fewer/more paths than the code has is suspect. +2. **Resolve check.** Take 2–3 spec paths and request them **as the scanner will build them** + — effective `host` + operation path — against the running app: + ```bash + # effective base the scan uses (app.host, incl. any path component) + curl -s -o /dev/null -w "%{http_code} %{url_effective}\n" "$SCAN_HOST/authors" + curl -s -o /dev/null -w "%{http_code} %{url_effective}\n" "$SCAN_HOST/api/v1/authors" + ``` + If the spec-derived URL 404s while a base/context-path-prefixed variant returns a real + status (200/401/403), the base path is wrong — fix `app.host` or the spec before scanning. + A raw container 404 (e.g. Tomcat's default page) rather than an application response is a + strong tell that the request never reached the app's routing. +3. After the scan, confirm with `hawk op scan uris `: the scanned URIs should be the + spec's real routes, not un-prefixed shadows of them. + +## Deriving a spec by hand (no tooling) + +When the stack has no OAS generator (or adding one is out of scope), build a minimal spec from +the route inventory discovery already produced. Keep it small and correct rather than +complete: + +- One `paths` entry per real route; include path parameters as `{id}` style templates. +- Set `servers[].url` to the real base (including context path) — but remember HawkScan keys + off `host + operation path`, so still run the resolve check; don't rely on `servers` alone. +- Include request bodies/params only where they matter for reaching the endpoint. +- Wire via `openApiConf.filePath` and verify as above. + +A hand-derived spec is a maintenance liability — always prefer suggesting the framework +tooling (step 2) so the spec regenerates from code instead of drifting. diff --git a/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc b/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc index 544b6192..28b30f8b 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc @@ -71,17 +71,24 @@ technique: | REST — Django | `urls.py` | count `path(`/`re_path(` entries: `grep -rc 'path(' **/urls.py` | | REST — Go (gin/mux/chi) | `router.GET(`, `r.HandleFunc(`, `mux.NewRouter()` | `grep -rEo '\.(GET\|POST\|PUT\|DELETE\|PATCH)\(' .` | | REST — .NET | `[HttpGet]`, `[HttpPost]`, `[Route(...)]` attributes | `grep -rEo '\[Http(Get\|Post\|Put\|Delete\|Patch)\]' .` | -| REST — OpenAPI spec | checked-in `openapi.{json,yaml,yml}`, `swagger*.{json,yaml}`; or served at `/v3/api-docs`, `/swagger.json`, `/openapi.json` once the app is running | count `paths:` entries: `yq '.paths | keys | length' openapi.yaml` (or the JSON equivalent) | +| REST — OpenAPI spec | checked-in `openapi.{json,yaml,yml}`, `swagger*.{json,yaml}`; or served at `/v3/api-docs`, `/swagger.json`, `/openapi.json` once the app is running | count `paths:` entries: `yq '.paths | keys | length' openapi.yaml` (or the JSON equivalent). **A found spec is a hypothesis, not a win — see `openapi-specs.md` for getting an *accurate* one.** | | GraphQL | schema files (`*.graphql`, `*.gql`); server libs in manifests (`graphql`, `apollo-server`, `graphql-yoga`, `graphene`, etc.); a `/graphql` endpoint | count root fields: `grep -A50 'type Query' schema.graphql \| grep -cE '^\s+\w+'` (repeat for `type Mutation`) | | gRPC | `.proto` files; `grpc`/`protobuf` deps in the manifest | count `rpc` methods per service: `grep -c '^\s*rpc ' *.proto` | | JSON-RPC / SOAP | fixed-path JSON-RPC dispatcher; WSDL files (`*.wsdl`), SOAP endpoint patterns | count ` --text` for the exact field shape — don't - hand-write the block from memory. + hand-write the block from memory. For REST, the spec must be *verified-accurate* before + it's wired: get it and prove it resolves per `openapi-specs.md` (a wrong spec silently + 404s every operation), and make sure `app.host` and the spec agree on the base/context + path. 2. Configure auth per Phase 1c in SKILL.md — one auth block per surface if surfaces have different login mechanisms (common when a gateway and an upstream service use different credential types). diff --git a/cursor/.cursor/rules/stackhawk-hawkscan-scan-quality.mdc b/cursor/.cursor/rules/stackhawk-hawkscan-scan-quality.mdc index c78f48c3..3563ceab 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan-scan-quality.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan-scan-quality.mdc @@ -73,11 +73,12 @@ Run all four after every scan. Each has its own command, pass condition, and a s reason identifier used when reporting a gap — this gate's own reporting (Step 4.5 in SKILL.md) uses these identifiers directly, so keep them exactly as written here across any edits to this file: `coverage-gap`, `spec-not-wired`, `surface-unscanned`, -`auth-validate-failed`, `auth-wall`, `all-4xx`, `env-unreachable`. +`auth-validate-failed`, `auth-wall`, `all-4xx`, `base-path-mismatch`, `env-unreachable`. | # | Check | What it measures | Command(s) | Pass condition | Reason identifier(s) | |---|-------|-------------------|------------|-----------------|------------------------| | 1 | Coverage | Scanned URIs vs. the expectation derived above | `hawk op scan uris --format json` diffed against the expectation list | **Never fails the gate.** Always computed and reported as evidence, e.g. "14 of 41 planned routes untouched," with the untouched routes listed | `coverage-gap` (evidence only) | +| 1b | Base-path resolve | Whether spec-derived paths actually resolve against the app, or systematically 404 because the config and spec disagree on the base/context path | In the scanned-URI list, look for the same route appearing both prefixed and un-prefixed (e.g. `/api/v1/authors` **and** bare `/authors`), or a cluster of spec paths all 404 while a prefixed variant succeeds; confirm with a curl of `host + spec-path` vs the prefixed variant | No systematic base-path 404 pattern — spec paths resolve to real routes. This **does** fail the gate (structural). Fix per `openapi-specs.md` "Base path and context path" | `base-path-mismatch` | | 2 | Auth | Live auth validation before the scan; auth-wall signals during it | `hawk validate auth .yml` pre-scan (against the config being gated — repos with multiple per-surface configs validate each one); `hawk op scan metrics ` for auth-wall flags | `validate auth` exited 0, and no auth-wall flag on any surface that has `authentication:` configured | `auth-validate-failed`, `auth-wall` | | 3 | Surface-completeness | Every surface discovery found has a config, that config actually ran, and any spec found for it is wired into that config | `hawk op scan config ` (effective config the scan ran with); fall back to reading the local `.yml` files | Every discovered surface maps to a config that exists, ran this scan, and has its spec wired if one exists | `surface-unscanned`, `spec-not-wired` | | 4 | Health | Connection-failure spikes, timeout streaks, app unreachable mid-scan; all-4xx response patterns on a configured surface | `hawk op scan metrics ` for flags; `hawk op scan get ` for `url_count` and status summary | No connection-failure spike, no timeout streak, no mid-scan unreachability, and no configured surface returning all 4xx | `env-unreachable`, `all-4xx` | @@ -88,8 +89,9 @@ edits to this file: `coverage-gap`, `spec-not-wired`, `surface-unscanned`, - **Config-class** — `spec-not-wired`, `surface-unscanned`, `auth-validate-failed` (the fix lives in the config's `authentication:` block), `auth-wall` while the app is demonstrably up, `all-4xx` on a configured surface (this is usually auth-shaped: the surface is - reachable but every request is being rejected). These route into the normal bounded - config iteration described below. + reachable but every request is being rejected), and `base-path-mismatch` (the fix is + aligning `app.host` and the spec's base/context path — `openapi-specs.md`). These route + into the normal bounded config iteration described below. - **Environment-class** — `env-unreachable`, covering connection-failure spikes, timeout streaks, and the app going unreachable mid-scan. **Never touch the config for these.** Verify the app is up using the same playbook as SKILL.md's Step 1c / Step 6 exit-1 diff --git a/cursor/.cursor/rules/stackhawk-hawkscan.mdc b/cursor/.cursor/rules/stackhawk-hawkscan.mdc index ef910c00..c1ece0a3 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan.mdc @@ -191,6 +191,10 @@ Do not proceed to Step 3 until validation passes. → API-type-specific config (OpenAPI, GraphQL, gRPC, seed paths, spider tuning): [`references/config-patterns.md`](references/config-patterns.md) +→ Getting an **accurate** OpenAPI spec (prefer a framework-served spec, suggest the + code/build change that generates one, verify it resolves before scanning — a wrong spec + silently 404s every path): [`references/openapi-specs.md`](references/openapi-specs.md) + ### Phase 1c: Authentication Configuration Use `hawk config show` to fetch the canonical recipe for the app's auth pattern. @@ -339,7 +343,7 @@ and fixed; a thin scan that missed surface can still have found something real. Run the four structural checks and derive the expectation fresh each time, exactly as `references/scan-quality.md` describes — coverage, auth, surface-completeness, and health. On config-class gaps (`spec-not-wired`, `surface-unscanned`, `auth-validate-failed`, -`auth-wall`, `all-4xx`), loop back to Step 2b with an additive-only fix, batch every gap +`auth-wall`, `all-4xx`, `base-path-mismatch`), loop back to Step 2b with an additive-only fix, batch every gap into one edit pass, and rescan once. Iteration cap: 2 interactive / 1 autonomous, per config, counted in rescans. On environment-class gaps (`env-unreachable`), never edit config — verify the app is up, retry once for free, and stop if it recurs. Multi-config diff --git a/gemini-extension.json b/gemini-extension.json index 0054094a..5a751e48 100644 --- a/gemini-extension.json +++ b/gemini-extension.json @@ -1,5 +1,5 @@ { "name": "stackhawk", - "version": "2.2.0", + "version": "2.3.0", "description": "StackHawk security scanning and API reporting skills. AUTONOMOUS: After completing any code change, invoke the hawkscan skill to scan, fix all reported vulnerabilities (not just your changes), and rescan to verify. Also invoke for any mention of hawkscan, stackhawk, DAST, security scan." } diff --git a/plugin.json b/plugin.json index c8aeee57..f2907847 100644 --- a/plugin.json +++ b/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk", - "version": "2.2.0", + "version": "2.3.0", "description": "StackHawk security scanning and API reporting skills. Includes hawkscan (DAST scanning) and api (security posture reporting).", "author": { "name": "StackHawk", diff --git a/plugins/api/.claude-plugin/plugin.json b/plugins/api/.claude-plugin/plugin.json index dc390d94..0ba8cf40 100644 --- a/plugins/api/.claude-plugin/plugin.json +++ b/plugins/api/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-api", - "version": "2.2.0", + "version": "2.3.0", "description": "Query the StackHawk platform API for security posture reporting, findings analysis, and app management. Uses the combined hawk CLI (hawk op \u2026) for all platform queries.", "author": { "name": "StackHawk", diff --git a/plugins/api/.codex-plugin/plugin.json b/plugins/api/.codex-plugin/plugin.json index 65575443..585cc1df 100644 --- a/plugins/api/.codex-plugin/plugin.json +++ b/plugins/api/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-api", - "version": "2.2.0", + "version": "2.3.0", "description": "Query the StackHawk platform API for security posture reporting, findings analysis, and app management. Uses the combined hawk CLI (hawk op \u2026) for all platform queries.", "author": { "name": "StackHawk", diff --git a/plugins/api/skills/api/SKILL.md b/plugins/api/skills/api/SKILL.md index 1a38465c..e748ad2b 100644 --- a/plugins/api/skills/api/SKILL.md +++ b/plugins/api/skills/api/SKILL.md @@ -1,6 +1,6 @@ --- name: api -version: 2.2.0 +version: 2.3.0 description: > Use this skill when a user or agent needs to query the StackHawk platform for security reporting, findings analysis, or app management. Triggers include: diff --git a/plugins/hawkscan-ci/.claude-plugin/plugin.json b/plugins/hawkscan-ci/.claude-plugin/plugin.json index 72719a3b..745ce5d0 100644 --- a/plugins/hawkscan-ci/.claude-plugin/plugin.json +++ b/plugins/hawkscan-ci/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "hawkscan-ci", - "version": "2.2.0", + "version": "2.3.0", "description": "Configure HawkScan in your CI/CD pipeline. Provider-agnostic: detects GitHub Actions, GitLab CI, Jenkins, CircleCI, Azure Pipelines, Bitbucket, Buildkite, and other CI systems from repo files, edits the pipeline file in place to add a HawkScan job, and prompts for HAWK_API_KEY storage (CI-native secrets store or external secrets manager).", "author": { "name": "StackHawk", diff --git a/plugins/hawkscan-ci/.codex-plugin/plugin.json b/plugins/hawkscan-ci/.codex-plugin/plugin.json index ba893c21..6e87a2e4 100644 --- a/plugins/hawkscan-ci/.codex-plugin/plugin.json +++ b/plugins/hawkscan-ci/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "hawkscan-ci", - "version": "2.2.0", + "version": "2.3.0", "description": "Configure HawkScan in your CI/CD pipeline. Provider-agnostic: detects GitHub Actions, GitLab CI, Jenkins, CircleCI, Azure Pipelines, Bitbucket, Buildkite, and other CI systems from repo files, edits the pipeline file in place to add a HawkScan job, and prompts for HAWK_API_KEY storage (CI-native secrets store or external secrets manager).", "author": { "name": "StackHawk", diff --git a/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md b/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md index 8c44d497..330906e1 100644 --- a/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md +++ b/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md @@ -1,6 +1,6 @@ --- name: hawkscan-ci -version: 2.2.0 +version: 2.3.0 description: > Use when the user wants to configure HawkScan in their CI/CD pipeline — triggers on "set up hawkscan in CI", "add stackhawk to my pipeline", diff --git a/plugins/hawkscan/.claude-plugin/plugin.json b/plugins/hawkscan/.claude-plugin/plugin.json index f59dfed1..fa656ce1 100644 --- a/plugins/hawkscan/.claude-plugin/plugin.json +++ b/plugins/hawkscan/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "hawkscan", - "version": "2.2.0", + "version": "2.3.0", "description": "Configure, run, and interpret HawkScan DAST results inside Claude Code. Generates stackhawk.yml configs, runs scans via CLI or Docker, and transforms security findings into prioritized fix tasks for your coding agent.", "author": { "name": "StackHawk", diff --git a/plugins/hawkscan/.codex-plugin/plugin.json b/plugins/hawkscan/.codex-plugin/plugin.json index 44ee20a7..feadc4b1 100644 --- a/plugins/hawkscan/.codex-plugin/plugin.json +++ b/plugins/hawkscan/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "hawkscan", - "version": "2.2.0", + "version": "2.3.0", "description": "Configure, run, and interpret HawkScan DAST results. Generates stackhawk.yml configs, runs scans via CLI or Docker, and transforms security findings into prioritized fix tasks for your coding agent.", "author": { "name": "StackHawk", diff --git a/plugins/hawkscan/skills/hawkscan/SKILL.md b/plugins/hawkscan/skills/hawkscan/SKILL.md index 9341b5c0..f29040ba 100644 --- a/plugins/hawkscan/skills/hawkscan/SKILL.md +++ b/plugins/hawkscan/skills/hawkscan/SKILL.md @@ -1,6 +1,6 @@ --- name: hawkscan -version: 2.2.0 +version: 2.3.0 description: > Runs the HawkScan DAST security loop — configure, scan, fix all reported vulnerabilities (not just your changes), rescan to verify. Performs @@ -202,6 +202,10 @@ Do not proceed to Step 3 until validation passes. → API-type-specific config (OpenAPI, GraphQL, gRPC, seed paths, spider tuning): [`references/config-patterns.md`](references/config-patterns.md) +→ Getting an **accurate** OpenAPI spec (prefer a framework-served spec, suggest the + code/build change that generates one, verify it resolves before scanning — a wrong spec + silently 404s every path): [`references/openapi-specs.md`](references/openapi-specs.md) + ### Phase 1c: Authentication Configuration Use `hawk config show` to fetch the canonical recipe for the app's auth pattern. @@ -350,7 +354,7 @@ and fixed; a thin scan that missed surface can still have found something real. Run the four structural checks and derive the expectation fresh each time, exactly as `references/scan-quality.md` describes — coverage, auth, surface-completeness, and health. On config-class gaps (`spec-not-wired`, `surface-unscanned`, `auth-validate-failed`, -`auth-wall`, `all-4xx`), loop back to Step 2b with an additive-only fix, batch every gap +`auth-wall`, `all-4xx`, `base-path-mismatch`), loop back to Step 2b with an additive-only fix, batch every gap into one edit pass, and rescan once. Iteration cap: 2 interactive / 1 autonomous, per config, counted in rescans. On environment-class gaps (`env-unreachable`), never edit config — verify the app is up, retry once for free, and stop if it recurs. Multi-config diff --git a/plugins/hawkscan/skills/hawkscan/references/openapi-specs.md b/plugins/hawkscan/skills/hawkscan/references/openapi-specs.md new file mode 100644 index 00000000..17c389ae --- /dev/null +++ b/plugins/hawkscan/skills/hawkscan/references/openapi-specs.md @@ -0,0 +1,160 @@ +# OpenAPI Spec Accuracy Reference + +A REST scan is only as good as the spec behind it. An **accurate** spec is one whose +operations match the app's real routes *and* resolve against the scanned host — same base +path, same context path. An inaccurate spec is worse than no spec: HawkScan faithfully +requests every path it declares, so wrong paths become a wall of 404s that bury real +coverage and generate recurring false-positive noise on every rescan. + +This reference is framework-generic. Use it whenever a REST surface needs a spec wired into +`openApiConf` — discovery (SKILL.md Step 1a / `scan-planning.md`) points here, and the +quality gate points here when it reports a `spec-not-wired` or coverage gap. + +## Contents +- [The one rule: an unverified spec is a hypothesis](#the-one-rule-an-unverified-spec-is-a-hypothesis) +- [Get an accurate spec (preference order)](#get-an-accurate-spec-preference-order) +- [Framework spec-generation matrix](#framework-spec-generation-matrix) +- [Suggesting the code/build change](#suggesting-the-codebuild-change) +- [Base path and context path](#base-path-and-context-path) +- [Mandatory: verify the spec resolves](#mandatory-verify-the-spec-resolves) +- [Deriving a spec by hand (no tooling)](#deriving-a-spec-by-hand-no-tooling) + +--- + +## The one rule: an unverified spec is a hypothesis + +A checked-in `openapi.yaml`, a spec you generated, and a spec the app serves are all +*claims* about the API until you prove they resolve against the running app. Never wire a +spec into `openApiConf` and scan without the verification in "Mandatory: verify the spec +resolves" below. A checked-in file is the least trustworthy source — it drifts from the code, +can be stale, hand-maintained, or (as seen in the wild) emptied by a commit — so treat its +presence as a starting hypothesis, not a done task. + +## Get an accurate spec (preference order) + +Prefer a spec the framework produces from the *running app* over any static artifact. Work +down this list; stop at the first that yields a spec that passes verification. + +1. **The running app already serves one.** Probe the common served endpoints once the app is + up (see the matrix for framework-specific paths): + ```bash + for p in /openapi.json /openapi.yaml /v3/api-docs /v3/api-docs.yaml \ + /swagger.json /swagger/v1/swagger.json /api-docs /q/openapi; do + curl -sf -o /dev/null -w "%{http_code} $p\n" "$APP_BASE_URL$p" + done + ``` + A 200 here is the best source — it reflects real routing, including base/context path. + Wire it live with `openApiConf.path: ` so the scan always reads the current + spec. +2. **The framework can generate/serve one with a small change, but it isn't enabled.** This is + the ideal outcome — see "Suggesting the code/build change". Add the tooling, rebuild, + then fetch as in step 1. +3. **Only a static spec is checked in.** Verify it's current (path count matches the route + grep from discovery; paths resolve — see verification). If it's stale, prefer regenerating + via step 1/2. If you must use it, wire `openApiConf.filePath: ` and verify. +4. **No spec and no tooling for this stack.** Derive one by hand from the route inventory — + see "Deriving a spec by hand". + +## Framework spec-generation matrix + +Examples, not an exhaustive list. The method generalizes: identify the framework from the +manifest, then determine its idiomatic OAS support. For anything not listed, **find out** — +`hawk config show`/docs won't help here, so search "`` openapi generation" or +check the framework's own docs before concluding none exists. + +| Stack | Idiomatic OAS tooling | Serves at | Notes | +|-------|----------------------|-----------|-------| +| FastAPI (Python) | built in | `/openapi.json` | Already served — no change needed; just probe. | +| NestJS (Node) | `@nestjs/swagger` | `/api-json` (configurable) | First-party. | +| Express / Fastify (Node) | `swagger-jsdoc` + `swagger-ui-express`; `@fastify/swagger` | configurable | JSDoc-annotation or plugin based. | +| Spring Boot (Java/Kotlin) | `springdoc-openapi-starter-webmvc-ui` | `/v3/api-docs` | Reflects `server.servlet.context-path` in its server URL. | +| Quarkus (Java) | `quarkus-smallrye-openapi` | `/q/openapi` | Often on by default in dev. | +| Django REST Framework | `drf-spectacular` | `/api/schema/` | `drf-yasg` is the older alternative. | +| Flask (Python) | `apiflask`, `flasgger`, `flask-smorest` | configurable | — | +| ASP.NET Core | `Swashbuckle.AspNetCore` or `NSwag` | `/swagger/v1/swagger.json` | Often scaffolded already. | +| Go | `swaggo/swag`, `danielgtaylor/huma` | configurable | swag generates from annotations. | +| Rails | `rswag` | configurable | Spec-from-request-specs. | +| Laravel (PHP) | `l5-swagger`, `scramble` | configurable | Scramble infers without annotations. | +| Rust | `utoipa`, `aide` | configurable | — | + +## Suggesting the code/build change + +When the framework can emit an accurate spec but doesn't yet (preference step 2), this is the +highest-value move: a served, framework-generated spec is accurate by construction and stays +current as the code changes. Explore and suggest a **concrete** change — not "consider adding +OpenAPI support": + +- Name the exact dependency and where it goes (the build file + coordinate/version). +- Show the minimal config/annotation to expose the endpoint, and state the served path. +- State the DAST value it unlocks ("wires N routes HawkScan otherwise can't see; replaces a + stale/hand-derived spec that would 404"). +- Scope it dev/test-only where the framework allows, so it needn't ship to production. + +Present this as a recommendation with the diff and let the user decide — the skill does not +apply unrequested code changes (see "Recommend code changes for gaps" in `scan-planning.md`). +Make the suggestion good enough that saying yes is a one-step change, then rebuild and fetch +the served spec. + +## Base path and context path + +The most common accuracy failure is a base-path mismatch. HawkScan builds each request as the +configured host **plus the spec's operation path**; a spec whose paths omit the app's base or +context path 404s on every operation. Detect the app's base path during discovery and make +the effective `host + spec-path` land on real routes: + +| Framework | Where the base/context path lives | +|-----------|-----------------------------------| +| Spring MVC | `server.servlet.context-path` (application.yml/properties) | +| Spring WebFlux | `spring.webflux.base-path` | +| Express | the mount prefix in `app.use('/api/v1', router)` | +| Django | the URL prefix in the root `urls.py` `include()` | +| ASP.NET | `UsePathBase(...)` / route prefix | +| Rails | `scope`/`namespace` in `config/routes.rb` | + +Two ways to make it resolve — pick one and be consistent: +- Put the base path in `app.host` (e.g. `http://localhost:5000/api/v1`) and keep spec paths, + `loginPath`, and `testPath` **relative**; or +- Keep `app.host` at the origin and ensure the spec's operation paths already include the base + path (a framework-served spec usually does). + +Do **not** split the difference — e.g. stripping the context path from `app.host` and +hardcoding it only into `loginPath`/`testPath`. That fixes auth but leaves every +`openApiConf` path unprefixed, which is exactly how a scan ends up 71% 404s while login still +works. If `hawk validate auth` misbehaves with a path component in `host`, resolve it without +desyncing the spec's base path — verify the outcome with the resolve check below. + +## Mandatory: verify the spec resolves + +Before the first real scan, prove the wired spec produces real requests — regardless of +source. This is the check that catches base-path mismatch, stale specs, and wrong hosts. + +1. **Count sanity.** Spec operation count should be in the same ballpark as the route grep + from discovery. A spec with far fewer/more paths than the code has is suspect. +2. **Resolve check.** Take 2–3 spec paths and request them **as the scanner will build them** + — effective `host` + operation path — against the running app: + ```bash + # effective base the scan uses (app.host, incl. any path component) + curl -s -o /dev/null -w "%{http_code} %{url_effective}\n" "$SCAN_HOST/authors" + curl -s -o /dev/null -w "%{http_code} %{url_effective}\n" "$SCAN_HOST/api/v1/authors" + ``` + If the spec-derived URL 404s while a base/context-path-prefixed variant returns a real + status (200/401/403), the base path is wrong — fix `app.host` or the spec before scanning. + A raw container 404 (e.g. Tomcat's default page) rather than an application response is a + strong tell that the request never reached the app's routing. +3. After the scan, confirm with `hawk op scan uris `: the scanned URIs should be the + spec's real routes, not un-prefixed shadows of them. + +## Deriving a spec by hand (no tooling) + +When the stack has no OAS generator (or adding one is out of scope), build a minimal spec from +the route inventory discovery already produced. Keep it small and correct rather than +complete: + +- One `paths` entry per real route; include path parameters as `{id}` style templates. +- Set `servers[].url` to the real base (including context path) — but remember HawkScan keys + off `host + operation path`, so still run the resolve check; don't rely on `servers` alone. +- Include request bodies/params only where they matter for reaching the endpoint. +- Wire via `openApiConf.filePath` and verify as above. + +A hand-derived spec is a maintenance liability — always prefer suggesting the framework +tooling (step 2) so the spec regenerates from code instead of drifting. diff --git a/plugins/hawkscan/skills/hawkscan/references/scan-planning.md b/plugins/hawkscan/skills/hawkscan/references/scan-planning.md index f3b18212..9d0f577d 100644 --- a/plugins/hawkscan/skills/hawkscan/references/scan-planning.md +++ b/plugins/hawkscan/skills/hawkscan/references/scan-planning.md @@ -65,17 +65,24 @@ technique: | REST — Django | `urls.py` | count `path(`/`re_path(` entries: `grep -rc 'path(' **/urls.py` | | REST — Go (gin/mux/chi) | `router.GET(`, `r.HandleFunc(`, `mux.NewRouter()` | `grep -rEo '\.(GET\|POST\|PUT\|DELETE\|PATCH)\(' .` | | REST — .NET | `[HttpGet]`, `[HttpPost]`, `[Route(...)]` attributes | `grep -rEo '\[Http(Get\|Post\|Put\|Delete\|Patch)\]' .` | -| REST — OpenAPI spec | checked-in `openapi.{json,yaml,yml}`, `swagger*.{json,yaml}`; or served at `/v3/api-docs`, `/swagger.json`, `/openapi.json` once the app is running | count `paths:` entries: `yq '.paths | keys | length' openapi.yaml` (or the JSON equivalent) | +| REST — OpenAPI spec | checked-in `openapi.{json,yaml,yml}`, `swagger*.{json,yaml}`; or served at `/v3/api-docs`, `/swagger.json`, `/openapi.json` once the app is running | count `paths:` entries: `yq '.paths | keys | length' openapi.yaml` (or the JSON equivalent). **A found spec is a hypothesis, not a win — see `openapi-specs.md` for getting an *accurate* one.** | | GraphQL | schema files (`*.graphql`, `*.gql`); server libs in manifests (`graphql`, `apollo-server`, `graphql-yoga`, `graphene`, etc.); a `/graphql` endpoint | count root fields: `grep -A50 'type Query' schema.graphql \| grep -cE '^\s+\w+'` (repeat for `type Mutation`) | | gRPC | `.proto` files; `grpc`/`protobuf` deps in the manifest | count `rpc` methods per service: `grep -c '^\s*rpc ' *.proto` | | JSON-RPC / SOAP | fixed-path JSON-RPC dispatcher; WSDL files (`*.wsdl`), SOAP endpoint patterns | count ` --text` for the exact field shape — don't - hand-write the block from memory. + hand-write the block from memory. For REST, the spec must be *verified-accurate* before + it's wired: get it and prove it resolves per `openapi-specs.md` (a wrong spec silently + 404s every operation), and make sure `app.host` and the spec agree on the base/context + path. 2. Configure auth per Phase 1c in SKILL.md — one auth block per surface if surfaces have different login mechanisms (common when a gateway and an upstream service use different credential types). diff --git a/plugins/hawkscan/skills/hawkscan/references/scan-quality.md b/plugins/hawkscan/skills/hawkscan/references/scan-quality.md index 8f8a0ff3..13eacd07 100644 --- a/plugins/hawkscan/skills/hawkscan/references/scan-quality.md +++ b/plugins/hawkscan/skills/hawkscan/references/scan-quality.md @@ -67,11 +67,12 @@ Run all four after every scan. Each has its own command, pass condition, and a s reason identifier used when reporting a gap — this gate's own reporting (Step 4.5 in SKILL.md) uses these identifiers directly, so keep them exactly as written here across any edits to this file: `coverage-gap`, `spec-not-wired`, `surface-unscanned`, -`auth-validate-failed`, `auth-wall`, `all-4xx`, `env-unreachable`. +`auth-validate-failed`, `auth-wall`, `all-4xx`, `base-path-mismatch`, `env-unreachable`. | # | Check | What it measures | Command(s) | Pass condition | Reason identifier(s) | |---|-------|-------------------|------------|-----------------|------------------------| | 1 | Coverage | Scanned URIs vs. the expectation derived above | `hawk op scan uris --format json` diffed against the expectation list | **Never fails the gate.** Always computed and reported as evidence, e.g. "14 of 41 planned routes untouched," with the untouched routes listed | `coverage-gap` (evidence only) | +| 1b | Base-path resolve | Whether spec-derived paths actually resolve against the app, or systematically 404 because the config and spec disagree on the base/context path | In the scanned-URI list, look for the same route appearing both prefixed and un-prefixed (e.g. `/api/v1/authors` **and** bare `/authors`), or a cluster of spec paths all 404 while a prefixed variant succeeds; confirm with a curl of `host + spec-path` vs the prefixed variant | No systematic base-path 404 pattern — spec paths resolve to real routes. This **does** fail the gate (structural). Fix per `openapi-specs.md` "Base path and context path" | `base-path-mismatch` | | 2 | Auth | Live auth validation before the scan; auth-wall signals during it | `hawk validate auth .yml` pre-scan (against the config being gated — repos with multiple per-surface configs validate each one); `hawk op scan metrics ` for auth-wall flags | `validate auth` exited 0, and no auth-wall flag on any surface that has `authentication:` configured | `auth-validate-failed`, `auth-wall` | | 3 | Surface-completeness | Every surface discovery found has a config, that config actually ran, and any spec found for it is wired into that config | `hawk op scan config ` (effective config the scan ran with); fall back to reading the local `.yml` files | Every discovered surface maps to a config that exists, ran this scan, and has its spec wired if one exists | `surface-unscanned`, `spec-not-wired` | | 4 | Health | Connection-failure spikes, timeout streaks, app unreachable mid-scan; all-4xx response patterns on a configured surface | `hawk op scan metrics ` for flags; `hawk op scan get ` for `url_count` and status summary | No connection-failure spike, no timeout streak, no mid-scan unreachability, and no configured surface returning all 4xx | `env-unreachable`, `all-4xx` | @@ -82,8 +83,9 @@ edits to this file: `coverage-gap`, `spec-not-wired`, `surface-unscanned`, - **Config-class** — `spec-not-wired`, `surface-unscanned`, `auth-validate-failed` (the fix lives in the config's `authentication:` block), `auth-wall` while the app is demonstrably up, `all-4xx` on a configured surface (this is usually auth-shaped: the surface is - reachable but every request is being rejected). These route into the normal bounded - config iteration described below. + reachable but every request is being rejected), and `base-path-mismatch` (the fix is + aligning `app.host` and the spec's base/context path — `openapi-specs.md`). These route + into the normal bounded config iteration described below. - **Environment-class** — `env-unreachable`, covering connection-failure spikes, timeout streaks, and the app going unreachable mid-scan. **Never touch the config for these.** Verify the app is up using the same playbook as SKILL.md's Step 1c / Step 6 exit-1 diff --git a/plugins/optimize/.claude-plugin/plugin.json b/plugins/optimize/.claude-plugin/plugin.json index 0836d856..a63e533e 100644 --- a/plugins/optimize/.claude-plugin/plugin.json +++ b/plugins/optimize/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-optimize", - "version": "2.2.0", + "version": "2.3.0", "description": "Analyze a codebase and produce an optimal HawkScan setup (tech flags, scan-policy plugin selection, stackhawk.yml corrections), apply it as a non-destructive trial, run one trial scan, and promote or discard.", "author": { "name": "StackHawk", diff --git a/plugins/optimize/.codex-plugin/plugin.json b/plugins/optimize/.codex-plugin/plugin.json index 75bbf996..6d01700c 100644 --- a/plugins/optimize/.codex-plugin/plugin.json +++ b/plugins/optimize/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-optimize", - "version": "2.2.0", + "version": "2.3.0", "description": "Analyze a codebase and produce an optimal HawkScan setup (tech flags, scan-policy plugin selection, stackhawk.yml corrections), apply it as a non-destructive trial, run one trial scan, and promote or discard.", "author": { "name": "StackHawk", diff --git a/plugins/optimize/skills/optimize/SKILL.md b/plugins/optimize/skills/optimize/SKILL.md index 175cf8c7..b4469621 100644 --- a/plugins/optimize/skills/optimize/SKILL.md +++ b/plugins/optimize/skills/optimize/SKILL.md @@ -1,6 +1,6 @@ --- name: optimize -version: 2.2.0 +version: 2.3.0 description: > Analyze a codebase and produce an optimal HawkScan setup — tech flags, scan-policy plugin selection, and stackhawk.yml corrections — then apply it as a non-destructive diff --git a/plugins/stackhawk-data-seed/.claude-plugin/plugin.json b/plugins/stackhawk-data-seed/.claude-plugin/plugin.json index cd8afda4..2a3e89da 100644 --- a/plugins/stackhawk-data-seed/.claude-plugin/plugin.json +++ b/plugins/stackhawk-data-seed/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-data-seed", - "version": "2.2.0", + "version": "2.3.0", "description": "Read a target repo and emit checked-in seed artifacts so HawkScan has data to scan. Generates SQL / HTTP / gRPC / Mongo / shell steps across the target repo and any upstream services it depends on, plus a manifest and a credentials handoff file that hawkscan consumes.", "author": { "name": "StackHawk", diff --git a/plugins/stackhawk-data-seed/.codex-plugin/plugin.json b/plugins/stackhawk-data-seed/.codex-plugin/plugin.json index 5c761014..5fbbb465 100644 --- a/plugins/stackhawk-data-seed/.codex-plugin/plugin.json +++ b/plugins/stackhawk-data-seed/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-data-seed", - "version": "2.2.0", + "version": "2.3.0", "description": "Read a target repo and emit checked-in seed artifacts so HawkScan has data to scan. Generates SQL / HTTP / gRPC / Mongo / shell steps across the target repo and any upstream services it depends on, plus a manifest and a credentials handoff file that hawkscan consumes.", "author": { "name": "StackHawk", diff --git a/plugins/stackhawk-data-seed/skills/stackhawk-data-seed/SKILL.md b/plugins/stackhawk-data-seed/skills/stackhawk-data-seed/SKILL.md index 0ae24501..bb8de9f2 100644 --- a/plugins/stackhawk-data-seed/skills/stackhawk-data-seed/SKILL.md +++ b/plugins/stackhawk-data-seed/skills/stackhawk-data-seed/SKILL.md @@ -1,6 +1,6 @@ --- name: stackhawk-data-seed -version: 2.2.0 +version: 2.3.0 description: > Set up checked-in seed data so authenticated HawkScan can reach non-trivial paths. Drives the `hawk perch seed` preflight, designs the minimum seed diff --git a/plugins/wingman/.claude-plugin/plugin.json b/plugins/wingman/.claude-plugin/plugin.json index 9d0eb79e..2521d4e0 100644 --- a/plugins/wingman/.claude-plugin/plugin.json +++ b/plugins/wingman/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "wingman", - "version": "2.2.0", + "version": "2.3.0", "description": "StackHawk umbrella plugin \u2014 installs the default StackHawk skill set (hawkscan, api, data-seed, optimize) in one command for the combined hawk CLI.", "author": { "name": "StackHawk", diff --git a/plugins/wingman/.codex-plugin/plugin.json b/plugins/wingman/.codex-plugin/plugin.json index 9d0eb79e..2521d4e0 100644 --- a/plugins/wingman/.codex-plugin/plugin.json +++ b/plugins/wingman/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "wingman", - "version": "2.2.0", + "version": "2.3.0", "description": "StackHawk umbrella plugin \u2014 installs the default StackHawk skill set (hawkscan, api, data-seed, optimize) in one command for the combined hawk CLI.", "author": { "name": "StackHawk", diff --git a/scripts/generate-cursor-rules.sh b/scripts/generate-cursor-rules.sh index 139f0bb9..255c6662 100755 --- a/scripts/generate-cursor-rules.sh +++ b/scripts/generate-cursor-rules.sh @@ -26,6 +26,7 @@ MAPPINGS=( "hawkscan/skills/hawkscan/references/false-positives.md|stackhawk-hawkscan-false-positives|HawkScan false positives guide: identifying false positives, deciding fix vs suppress, excludePaths and failureThreshold config, excluding scan plugins, reporting accepted risk.||false" "hawkscan/skills/hawkscan/references/platform-model.md|stackhawk-hawkscan-model|HawkScan StackHawk platform model: Org → App → Env → Scan hierarchy, what goes in stackhawk.yml vs what doesn't, finding triage state machine (New/Accepted/False Positive/Reopened) and agent behavior per state, tags for commit traceability (_STACKHAWK_GIT_COMMIT_SHA, _STACKHAWK_GIT_BRANCH), technology flags (platform UI only — no API yet), hawk rescan --scan-id for fast fix verification in the agentic loop, create vs reuse decision trees for Apps and Envs.||false" "hawkscan/skills/hawkscan/references/scan-planning.md|stackhawk-hawkscan-scan-planning|HawkScan scan-planning (discovery) reference: code-first discovery of an app's API surfaces before writing or editing stackhawk.yml — reading repo docs first, per-framework route inventory, recommending code changes for structural gaps, asking the user instead of guessing, and producing a per-surface stackhawk.yml config. Use for the first scan of a repo, a quality-gate structural gap, or a user-requested re-plan.||false" + "hawkscan/skills/hawkscan/references/openapi-specs.md|stackhawk-hawkscan-openapi-specs|HawkScan OpenAPI spec accuracy reference: framework-generic procedure for getting an accurate OpenAPI spec into openApiConf for a REST surface — prefer a spec the running app serves, suggest the code/build change that makes the framework generate one, verify it resolves against the app (base/context path), and derive one by hand only as a last resort. Use whenever wiring or fixing openApiConf, or when a scan's spec paths return 404s.||false" "hawkscan/skills/hawkscan/references/scan-quality.md|stackhawk-hawkscan-scan-quality|HawkScan scan-quality (post-scan quality gate) reference: run after every scan and rescan, before findings become fix tasks. Derives a fresh coverage expectation each run (spec-wired or route-inventory), the four coverage checks, and a bounded, additive-only stackhawk.yml config iteration to close gaps — never blocks a real finding from being reported and fixed.||false" "api/skills/api/SKILL.md|stackhawk-api|Use when querying the StackHawk platform for security reporting, findings analysis, or app management. Triggers include \"stackhawk api\", \"security posture\", \"findings report\", \"show me findings\", \"untriaged findings\", \"which apps\", \"scan history\", \"security dashboard\", \"triage\", \"what needs attention\". Uses the combined hawk CLI (hawk op …) for all platform queries. Do NOT use for running scans (use the stackhawk-hawkscan rule for \"scan my app\", \"hawkscan\", \"stackhawk.yml\", \"DAST\") or for fixing/remediating code or vulnerabilities — this skill only reads and reports platform data.||false" "api/skills/api/references/hawk-op-shortcuts.md|stackhawk-api-hawk-op|hawk op shortcuts for the StackHawk API: user intent → single hawk op command mapping for scan drill-down, app/user/team/policy/repo/oas/config/secret/audit listing, hosted scan control, env management. All platform queries go through hawk op — no raw REST fallback.||false" From f5afd26a59cbcda1f33aed86c4132a7d3f00c1ed Mon Sep 17 00:00:00 2001 From: KC Berg Date: Thu, 9 Jul 2026 14:12:47 -0600 Subject: [PATCH 11/29] feat(hawkscan): proactive spec-decision, external-spec discovery, seedPaths demotion MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Dogfood round 2 (firefly-iii + gitea) showed: references only loaded on friction (gitea hit a spec wall and opened openapi-specs.md -> 618/688 coverage; firefly's seedPaths 'worked' so nothing forced consultation -> 22/254). Five tuning changes: 1. SKILL.md Step 2a: a proactive, imperative REST-spec directive — get an accurate spec BEFORE the first scan, open+follow openapi-specs.md, run the resolve-check; seedPaths is a last resort not a shortcut. Fires without waiting for a failure. 2. openapi-specs.md: new preference-order step for a spec published OUTSIDE the repo (docs site, /api-docs repo, release asset) — Firefly III's exact case — ahead of derive-by-hand. 3. seedPaths demoted below a hand-derived spec (URLs-only: no methods/ bodies/params) in openapi-specs.md + SKILL.md low-path bullet. 4. resolve-check reinforced as a pre-first-scan step via the SKILL directive (gitea ran it reactively, burning a scan). 5. Companion note: select an app by UUID with --app-id on released hawk (--app is name-only there; newer hawk accepts UUID on --app); parse --format json defensively past the skills-version banner. scan-planning.md gap section points at the external-spec + resolve steps. Cursor rules regenerated (idempotent); 2.3.0 -> 2.3.1. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01WWyT8CN77z9xrJhRLjrtJC --- .claude-plugin/marketplace.json | 12 ++++++------ .claude/skills/skill-authoring/SKILL.md | 2 +- .codex-plugin/plugin.json | 2 +- VERSION | 2 +- .../rules/stackhawk-hawkscan-openapi-specs.mdc | 18 ++++++++++++++++-- .../rules/stackhawk-hawkscan-scan-planning.mdc | 10 ++++++---- cursor/.cursor/rules/stackhawk-hawkscan.mdc | 16 ++++++++++++---- gemini-extension.json | 2 +- plugin.json | 2 +- plugins/api/.claude-plugin/plugin.json | 2 +- plugins/api/.codex-plugin/plugin.json | 2 +- plugins/api/skills/api/SKILL.md | 2 +- plugins/hawkscan-ci/.claude-plugin/plugin.json | 2 +- plugins/hawkscan-ci/.codex-plugin/plugin.json | 2 +- .../hawkscan-ci/skills/hawkscan-ci/SKILL.md | 2 +- plugins/hawkscan/.claude-plugin/plugin.json | 2 +- plugins/hawkscan/.codex-plugin/plugin.json | 2 +- plugins/hawkscan/skills/hawkscan/SKILL.md | 18 +++++++++++++----- .../hawkscan/references/openapi-specs.md | 18 ++++++++++++++++-- .../hawkscan/references/scan-planning.md | 10 ++++++---- plugins/optimize/.claude-plugin/plugin.json | 2 +- plugins/optimize/.codex-plugin/plugin.json | 2 +- plugins/optimize/skills/optimize/SKILL.md | 2 +- .../.claude-plugin/plugin.json | 2 +- .../.codex-plugin/plugin.json | 2 +- .../skills/stackhawk-data-seed/SKILL.md | 2 +- plugins/wingman/.claude-plugin/plugin.json | 2 +- plugins/wingman/.codex-plugin/plugin.json | 2 +- 28 files changed, 96 insertions(+), 48 deletions(-) diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index 1000577d..cb2b2e92 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -14,7 +14,7 @@ "category": "security", "source": "./plugins/hawkscan", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/hawkscan", - "version": "2.3.0" + "version": "2.3.1" }, { "name": "stackhawk-api", @@ -22,7 +22,7 @@ "category": "security", "source": "./plugins/api", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/api", - "version": "2.3.0" + "version": "2.3.1" }, { "name": "hawkscan-ci", @@ -30,7 +30,7 @@ "category": "security", "source": "./plugins/hawkscan-ci", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/hawkscan-ci", - "version": "2.3.0" + "version": "2.3.1" }, { "name": "stackhawk-data-seed", @@ -38,7 +38,7 @@ "category": "security", "source": "./plugins/stackhawk-data-seed", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/stackhawk-data-seed", - "version": "2.3.0" + "version": "2.3.1" }, { "name": "wingman", @@ -46,7 +46,7 @@ "category": "security", "source": "./plugins/wingman", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/wingman", - "version": "2.3.0" + "version": "2.3.1" }, { "name": "stackhawk-optimize", @@ -54,7 +54,7 @@ "category": "security", "source": "./plugins/optimize", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/optimize", - "version": "2.3.0" + "version": "2.3.1" } ] } diff --git a/.claude/skills/skill-authoring/SKILL.md b/.claude/skills/skill-authoring/SKILL.md index 376eecff..107c8762 100644 --- a/.claude/skills/skill-authoring/SKILL.md +++ b/.claude/skills/skill-authoring/SKILL.md @@ -1,6 +1,6 @@ --- name: skill-authoring -version: 2.3.0 +version: 2.3.1 description: > Guides authoring and maintaining agent skills in this repository — enforcing Anthropic best practices, bumping versions, regenerating Cursor rules, and diff --git a/.codex-plugin/plugin.json b/.codex-plugin/plugin.json index 71e5a48c..db10c162 100644 --- a/.codex-plugin/plugin.json +++ b/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk", - "version": "2.3.0", + "version": "2.3.1", "description": "Dynamic application and API security testing (DAST) powered by HawkScan. Configure scans, interpret findings, and query the StackHawk platform API.", "author": { "name": "StackHawk", diff --git a/VERSION b/VERSION index cc6612c3..a6254504 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -2.3.0 \ No newline at end of file +2.3.1 \ No newline at end of file diff --git a/cursor/.cursor/rules/stackhawk-hawkscan-openapi-specs.mdc b/cursor/.cursor/rules/stackhawk-hawkscan-openapi-specs.mdc index 7f620d0a..86c64464 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan-openapi-specs.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan-openapi-specs.mdc @@ -58,8 +58,15 @@ down this list; stop at the first that yields a spec that passes verification. 3. **Only a static spec is checked in.** Verify it's current (path count matches the route grep from discovery; paths resolve — see verification). If it's stale, prefer regenerating via step 1/2. If you must use it, wire `openApiConf.filePath: ` and verify. -4. **No spec and no tooling for this stack.** Derive one by hand from the route inventory — - see "Deriving a spec by hand". +4. **A spec is published *outside* the repo.** Mature projects often maintain an accurate spec + somewhere other than the app repo — check before giving up: a docs site (e.g. + `api-docs..org`, a ReDoc/Swagger-UI page), a dedicated `/api-docs` or + `-openapi` repo, a spec published as a release asset or package, or a link in the + README/docs. This is exactly Firefly III's situation — no spec in-repo, but a complete one + at `firefly-iii/api-docs`. Fetch it, pin the version to match the app, wire `filePath`, and + verify it resolves (an external spec can lag the running app). +5. **No spec anywhere and no tooling for this stack.** Derive one by hand from the route + inventory — see "Deriving a spec by hand". ## Framework spec-generation matrix @@ -164,3 +171,10 @@ complete: A hand-derived spec is a maintenance liability — always prefer suggesting the framework tooling (step 2) so the spec regenerates from code instead of drifting. + +**A hand-derived spec still beats `hawk.spider.seedPaths`.** seedPaths only lists URLs to +visit — no methods, no request bodies, no path/query parameters — so a seedPaths-only scan +can GET a handful of collection endpoints but never exercises POST/PUT/PATCH, parameterized +routes, or request-body validation. That is why a spec-less scan collapses to a fraction of +the real surface (in dogfood: 22 scanned URIs against 254 routes). Only use seedPaths as a +supplement to a spec, or as a genuine last resort when even a hand-derived spec is impossible. diff --git a/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc b/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc index 28b30f8b..b370bafa 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc @@ -107,10 +107,12 @@ won't actually reach the API surface. Typical recommendations: - **No accurate OpenAPI spec.** This covers both "no spec at all" and "a spec exists but its paths don't resolve against the app" — treat them the same. The highest-value recommendation is a small code/build change that makes the framework *generate and serve* an accurate spec - (idiomatic tooling exists for most stacks). The full framework-generic procedure — probe for - a served spec, the per-stack tooling matrix, how to write a concrete suggestion, and the - base-path/resolve verification — is in `openapi-specs.md`. Do that rather than wiring a - stale checked-in file and hoping. + (idiomatic tooling exists for most stacks). Before deriving one by hand, also check whether + the project publishes a spec *outside* the repo (a docs site, a `/api-docs` repo, a + release asset) — mature OSS projects often do. The full framework-generic procedure — probe + for a served spec, the per-stack tooling matrix, how to write a concrete suggestion, the + external-spec check, and the base-path/resolve verification — is in `openapi-specs.md`. Do + that rather than wiring a stale checked-in file, or falling back to `seedPaths`, and hoping. - **No test credential exists.** Recommend a test-only token issuer or seed script (a dev-only endpoint or fixture that mints a valid session/JWT) gated behind an environment check so it never ships to production. diff --git a/cursor/.cursor/rules/stackhawk-hawkscan.mdc b/cursor/.cursor/rules/stackhawk-hawkscan.mdc index c1ece0a3..8994de03 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan.mdc @@ -55,6 +55,8 @@ The `api` skill wraps read-only StackHawk platform lookups via the `hawk` CLI (` These reads require the combined `hawk` CLI; the `api` skill covers setup (`hawk init --browser` or the `HAWK_API_KEY` env var). +**Selecting an app by UUID:** `--app` takes an application *name*. When you hold a UUID (e.g. from `stackhawk.yml`'s `applicationId`), pass it as `--app-id ` — passing a UUID to `--app` errors with "Application not found" on current released hawk. (Newer hawk also accepts a UUID on `--app`; `--app-id` works on both, so prefer it when you have an ID.) Also parse `--format json` output defensively — a "skills out of date" banner may precede the JSON. + The `stackhawk-data-seed` skill sets up checked-in backend seed data via `hawk perch seed`. Hand off to it when authentication fails because the backend has no valid credential (**Phase 1c.6**), or after a scan when auth succeeded but endpoints returned empty data @@ -191,9 +193,15 @@ Do not proceed to Step 3 until validation passes. → API-type-specific config (OpenAPI, GraphQL, gRPC, seed paths, spider tuning): [`references/config-patterns.md`](references/config-patterns.md) -→ Getting an **accurate** OpenAPI spec (prefer a framework-served spec, suggest the - code/build change that generates one, verify it resolves before scanning — a wrong spec - silently 404s every path): [`references/openapi-specs.md`](references/openapi-specs.md) +**REST surface? Get an accurate OpenAPI spec BEFORE the first scan — this is not optional.** +A REST scan with no spec (spider/`seedPaths` only) reaches a small fraction of the API; a +scan with a *wrong* spec 404s every path. Do not skip to a scan on the assumption a spec is +missing or "good enough" — **open and follow [`references/openapi-specs.md`](references/openapi-specs.md)** to +work its preference order (a spec the running app serves → a code/build change that generates +one → a spec published outside the repo → hand-derived), then run its resolve-check +(`host + spec-path` returns real routes, not 404s) before scanning. Reaching for +`hawk.spider.seedPaths` instead of a spec is a last resort, not a shortcut — read that file +first. Same idea for GraphQL/gRPC: wire the schema/proto, don't scan blind. ### Phase 1c: Authentication Configuration @@ -258,7 +266,7 @@ Review the config against the current app state: - **Gate reported gaps?** Tune what the gate names (wire spec, enable ajax spider, fix auth) — additive-only; see [`references/scan-quality.md`](references/scan-quality.md). -- **Low path count?** Check in order: SPA/JS app → enable `hawk.spider.ajax: true`; API spec available → wire `openApiConf`/`graphqlConf`; specific known-deep paths → add `seedPaths`. Omit `seedPaths` unless there is a specific reason — they're rarely needed when Ajax Spider or a spec is configured. +- **Low path count?** For a REST surface the fix is almost always an accurate spec — get one per `references/openapi-specs.md` (a wired, resolving `openApiConf` is worth far more than any spider tuning). Then: SPA/JS app → `hawk.spider.ajax: true`; GraphQL → wire `graphqlConf`. `hawk.spider.seedPaths` is a last resort (URLs only — no methods/bodies/params, so it can't reach POST/PUT or parameterized routes); prefer even a hand-derived spec over it, and omit it entirely once a spec is wired. - **Auth failing?** Verify `authentication` block; re-fetch the relevant recipe via `hawk config show
--text` (Phase 1c). - **Too noisy / too slow?** Add `app.excludePaths` or `app.includePaths`; tune `hawk.spider.maxDurationMinutes`. - **New API type added?** Add corresponding `graphqlConf`, `openApiConf`, etc. diff --git a/gemini-extension.json b/gemini-extension.json index 5a751e48..1d24ab35 100644 --- a/gemini-extension.json +++ b/gemini-extension.json @@ -1,5 +1,5 @@ { "name": "stackhawk", - "version": "2.3.0", + "version": "2.3.1", "description": "StackHawk security scanning and API reporting skills. AUTONOMOUS: After completing any code change, invoke the hawkscan skill to scan, fix all reported vulnerabilities (not just your changes), and rescan to verify. Also invoke for any mention of hawkscan, stackhawk, DAST, security scan." } diff --git a/plugin.json b/plugin.json index f2907847..0e54bc60 100644 --- a/plugin.json +++ b/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk", - "version": "2.3.0", + "version": "2.3.1", "description": "StackHawk security scanning and API reporting skills. Includes hawkscan (DAST scanning) and api (security posture reporting).", "author": { "name": "StackHawk", diff --git a/plugins/api/.claude-plugin/plugin.json b/plugins/api/.claude-plugin/plugin.json index 0ba8cf40..de4274e0 100644 --- a/plugins/api/.claude-plugin/plugin.json +++ b/plugins/api/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-api", - "version": "2.3.0", + "version": "2.3.1", "description": "Query the StackHawk platform API for security posture reporting, findings analysis, and app management. Uses the combined hawk CLI (hawk op \u2026) for all platform queries.", "author": { "name": "StackHawk", diff --git a/plugins/api/.codex-plugin/plugin.json b/plugins/api/.codex-plugin/plugin.json index 585cc1df..0f6c07dd 100644 --- a/plugins/api/.codex-plugin/plugin.json +++ b/plugins/api/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-api", - "version": "2.3.0", + "version": "2.3.1", "description": "Query the StackHawk platform API for security posture reporting, findings analysis, and app management. Uses the combined hawk CLI (hawk op \u2026) for all platform queries.", "author": { "name": "StackHawk", diff --git a/plugins/api/skills/api/SKILL.md b/plugins/api/skills/api/SKILL.md index e748ad2b..097f6fdf 100644 --- a/plugins/api/skills/api/SKILL.md +++ b/plugins/api/skills/api/SKILL.md @@ -1,6 +1,6 @@ --- name: api -version: 2.3.0 +version: 2.3.1 description: > Use this skill when a user or agent needs to query the StackHawk platform for security reporting, findings analysis, or app management. Triggers include: diff --git a/plugins/hawkscan-ci/.claude-plugin/plugin.json b/plugins/hawkscan-ci/.claude-plugin/plugin.json index 745ce5d0..1e29726c 100644 --- a/plugins/hawkscan-ci/.claude-plugin/plugin.json +++ b/plugins/hawkscan-ci/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "hawkscan-ci", - "version": "2.3.0", + "version": "2.3.1", "description": "Configure HawkScan in your CI/CD pipeline. Provider-agnostic: detects GitHub Actions, GitLab CI, Jenkins, CircleCI, Azure Pipelines, Bitbucket, Buildkite, and other CI systems from repo files, edits the pipeline file in place to add a HawkScan job, and prompts for HAWK_API_KEY storage (CI-native secrets store or external secrets manager).", "author": { "name": "StackHawk", diff --git a/plugins/hawkscan-ci/.codex-plugin/plugin.json b/plugins/hawkscan-ci/.codex-plugin/plugin.json index 6e87a2e4..4fbd878a 100644 --- a/plugins/hawkscan-ci/.codex-plugin/plugin.json +++ b/plugins/hawkscan-ci/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "hawkscan-ci", - "version": "2.3.0", + "version": "2.3.1", "description": "Configure HawkScan in your CI/CD pipeline. Provider-agnostic: detects GitHub Actions, GitLab CI, Jenkins, CircleCI, Azure Pipelines, Bitbucket, Buildkite, and other CI systems from repo files, edits the pipeline file in place to add a HawkScan job, and prompts for HAWK_API_KEY storage (CI-native secrets store or external secrets manager).", "author": { "name": "StackHawk", diff --git a/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md b/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md index 330906e1..ffc136ed 100644 --- a/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md +++ b/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md @@ -1,6 +1,6 @@ --- name: hawkscan-ci -version: 2.3.0 +version: 2.3.1 description: > Use when the user wants to configure HawkScan in their CI/CD pipeline — triggers on "set up hawkscan in CI", "add stackhawk to my pipeline", diff --git a/plugins/hawkscan/.claude-plugin/plugin.json b/plugins/hawkscan/.claude-plugin/plugin.json index fa656ce1..5f2d0242 100644 --- a/plugins/hawkscan/.claude-plugin/plugin.json +++ b/plugins/hawkscan/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "hawkscan", - "version": "2.3.0", + "version": "2.3.1", "description": "Configure, run, and interpret HawkScan DAST results inside Claude Code. Generates stackhawk.yml configs, runs scans via CLI or Docker, and transforms security findings into prioritized fix tasks for your coding agent.", "author": { "name": "StackHawk", diff --git a/plugins/hawkscan/.codex-plugin/plugin.json b/plugins/hawkscan/.codex-plugin/plugin.json index feadc4b1..dc09ba93 100644 --- a/plugins/hawkscan/.codex-plugin/plugin.json +++ b/plugins/hawkscan/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "hawkscan", - "version": "2.3.0", + "version": "2.3.1", "description": "Configure, run, and interpret HawkScan DAST results. Generates stackhawk.yml configs, runs scans via CLI or Docker, and transforms security findings into prioritized fix tasks for your coding agent.", "author": { "name": "StackHawk", diff --git a/plugins/hawkscan/skills/hawkscan/SKILL.md b/plugins/hawkscan/skills/hawkscan/SKILL.md index f29040ba..ee0e8579 100644 --- a/plugins/hawkscan/skills/hawkscan/SKILL.md +++ b/plugins/hawkscan/skills/hawkscan/SKILL.md @@ -1,6 +1,6 @@ --- name: hawkscan -version: 2.3.0 +version: 2.3.1 description: > Runs the HawkScan DAST security loop — configure, scan, fix all reported vulnerabilities (not just your changes), rescan to verify. Performs @@ -66,6 +66,8 @@ The `api` skill wraps read-only StackHawk platform lookups via the `hawk` CLI (` These reads require the combined `hawk` CLI; the `api` skill covers setup (`hawk init --browser` or the `HAWK_API_KEY` env var). +**Selecting an app by UUID:** `--app` takes an application *name*. When you hold a UUID (e.g. from `stackhawk.yml`'s `applicationId`), pass it as `--app-id ` — passing a UUID to `--app` errors with "Application not found" on current released hawk. (Newer hawk also accepts a UUID on `--app`; `--app-id` works on both, so prefer it when you have an ID.) Also parse `--format json` output defensively — a "skills out of date" banner may precede the JSON. + The `stackhawk-data-seed` skill sets up checked-in backend seed data via `hawk perch seed`. Hand off to it when authentication fails because the backend has no valid credential (**Phase 1c.6**), or after a scan when auth succeeded but endpoints returned empty data @@ -202,9 +204,15 @@ Do not proceed to Step 3 until validation passes. → API-type-specific config (OpenAPI, GraphQL, gRPC, seed paths, spider tuning): [`references/config-patterns.md`](references/config-patterns.md) -→ Getting an **accurate** OpenAPI spec (prefer a framework-served spec, suggest the - code/build change that generates one, verify it resolves before scanning — a wrong spec - silently 404s every path): [`references/openapi-specs.md`](references/openapi-specs.md) +**REST surface? Get an accurate OpenAPI spec BEFORE the first scan — this is not optional.** +A REST scan with no spec (spider/`seedPaths` only) reaches a small fraction of the API; a +scan with a *wrong* spec 404s every path. Do not skip to a scan on the assumption a spec is +missing or "good enough" — **open and follow [`references/openapi-specs.md`](references/openapi-specs.md)** to +work its preference order (a spec the running app serves → a code/build change that generates +one → a spec published outside the repo → hand-derived), then run its resolve-check +(`host + spec-path` returns real routes, not 404s) before scanning. Reaching for +`hawk.spider.seedPaths` instead of a spec is a last resort, not a shortcut — read that file +first. Same idea for GraphQL/gRPC: wire the schema/proto, don't scan blind. ### Phase 1c: Authentication Configuration @@ -269,7 +277,7 @@ Review the config against the current app state: - **Gate reported gaps?** Tune what the gate names (wire spec, enable ajax spider, fix auth) — additive-only; see [`references/scan-quality.md`](references/scan-quality.md). -- **Low path count?** Check in order: SPA/JS app → enable `hawk.spider.ajax: true`; API spec available → wire `openApiConf`/`graphqlConf`; specific known-deep paths → add `seedPaths`. Omit `seedPaths` unless there is a specific reason — they're rarely needed when Ajax Spider or a spec is configured. +- **Low path count?** For a REST surface the fix is almost always an accurate spec — get one per `references/openapi-specs.md` (a wired, resolving `openApiConf` is worth far more than any spider tuning). Then: SPA/JS app → `hawk.spider.ajax: true`; GraphQL → wire `graphqlConf`. `hawk.spider.seedPaths` is a last resort (URLs only — no methods/bodies/params, so it can't reach POST/PUT or parameterized routes); prefer even a hand-derived spec over it, and omit it entirely once a spec is wired. - **Auth failing?** Verify `authentication` block; re-fetch the relevant recipe via `hawk config show
--text` (Phase 1c). - **Too noisy / too slow?** Add `app.excludePaths` or `app.includePaths`; tune `hawk.spider.maxDurationMinutes`. - **New API type added?** Add corresponding `graphqlConf`, `openApiConf`, etc. diff --git a/plugins/hawkscan/skills/hawkscan/references/openapi-specs.md b/plugins/hawkscan/skills/hawkscan/references/openapi-specs.md index 17c389ae..8a2fa070 100644 --- a/plugins/hawkscan/skills/hawkscan/references/openapi-specs.md +++ b/plugins/hawkscan/skills/hawkscan/references/openapi-specs.md @@ -52,8 +52,15 @@ down this list; stop at the first that yields a spec that passes verification. 3. **Only a static spec is checked in.** Verify it's current (path count matches the route grep from discovery; paths resolve — see verification). If it's stale, prefer regenerating via step 1/2. If you must use it, wire `openApiConf.filePath: ` and verify. -4. **No spec and no tooling for this stack.** Derive one by hand from the route inventory — - see "Deriving a spec by hand". +4. **A spec is published *outside* the repo.** Mature projects often maintain an accurate spec + somewhere other than the app repo — check before giving up: a docs site (e.g. + `api-docs..org`, a ReDoc/Swagger-UI page), a dedicated `/api-docs` or + `-openapi` repo, a spec published as a release asset or package, or a link in the + README/docs. This is exactly Firefly III's situation — no spec in-repo, but a complete one + at `firefly-iii/api-docs`. Fetch it, pin the version to match the app, wire `filePath`, and + verify it resolves (an external spec can lag the running app). +5. **No spec anywhere and no tooling for this stack.** Derive one by hand from the route + inventory — see "Deriving a spec by hand". ## Framework spec-generation matrix @@ -158,3 +165,10 @@ complete: A hand-derived spec is a maintenance liability — always prefer suggesting the framework tooling (step 2) so the spec regenerates from code instead of drifting. + +**A hand-derived spec still beats `hawk.spider.seedPaths`.** seedPaths only lists URLs to +visit — no methods, no request bodies, no path/query parameters — so a seedPaths-only scan +can GET a handful of collection endpoints but never exercises POST/PUT/PATCH, parameterized +routes, or request-body validation. That is why a spec-less scan collapses to a fraction of +the real surface (in dogfood: 22 scanned URIs against 254 routes). Only use seedPaths as a +supplement to a spec, or as a genuine last resort when even a hand-derived spec is impossible. diff --git a/plugins/hawkscan/skills/hawkscan/references/scan-planning.md b/plugins/hawkscan/skills/hawkscan/references/scan-planning.md index 9d0f577d..06a25cca 100644 --- a/plugins/hawkscan/skills/hawkscan/references/scan-planning.md +++ b/plugins/hawkscan/skills/hawkscan/references/scan-planning.md @@ -101,10 +101,12 @@ won't actually reach the API surface. Typical recommendations: - **No accurate OpenAPI spec.** This covers both "no spec at all" and "a spec exists but its paths don't resolve against the app" — treat them the same. The highest-value recommendation is a small code/build change that makes the framework *generate and serve* an accurate spec - (idiomatic tooling exists for most stacks). The full framework-generic procedure — probe for - a served spec, the per-stack tooling matrix, how to write a concrete suggestion, and the - base-path/resolve verification — is in `openapi-specs.md`. Do that rather than wiring a - stale checked-in file and hoping. + (idiomatic tooling exists for most stacks). Before deriving one by hand, also check whether + the project publishes a spec *outside* the repo (a docs site, a `/api-docs` repo, a + release asset) — mature OSS projects often do. The full framework-generic procedure — probe + for a served spec, the per-stack tooling matrix, how to write a concrete suggestion, the + external-spec check, and the base-path/resolve verification — is in `openapi-specs.md`. Do + that rather than wiring a stale checked-in file, or falling back to `seedPaths`, and hoping. - **No test credential exists.** Recommend a test-only token issuer or seed script (a dev-only endpoint or fixture that mints a valid session/JWT) gated behind an environment check so it never ships to production. diff --git a/plugins/optimize/.claude-plugin/plugin.json b/plugins/optimize/.claude-plugin/plugin.json index a63e533e..77d0915d 100644 --- a/plugins/optimize/.claude-plugin/plugin.json +++ b/plugins/optimize/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-optimize", - "version": "2.3.0", + "version": "2.3.1", "description": "Analyze a codebase and produce an optimal HawkScan setup (tech flags, scan-policy plugin selection, stackhawk.yml corrections), apply it as a non-destructive trial, run one trial scan, and promote or discard.", "author": { "name": "StackHawk", diff --git a/plugins/optimize/.codex-plugin/plugin.json b/plugins/optimize/.codex-plugin/plugin.json index 6d01700c..878b2dd1 100644 --- a/plugins/optimize/.codex-plugin/plugin.json +++ b/plugins/optimize/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-optimize", - "version": "2.3.0", + "version": "2.3.1", "description": "Analyze a codebase and produce an optimal HawkScan setup (tech flags, scan-policy plugin selection, stackhawk.yml corrections), apply it as a non-destructive trial, run one trial scan, and promote or discard.", "author": { "name": "StackHawk", diff --git a/plugins/optimize/skills/optimize/SKILL.md b/plugins/optimize/skills/optimize/SKILL.md index b4469621..1f6de3f9 100644 --- a/plugins/optimize/skills/optimize/SKILL.md +++ b/plugins/optimize/skills/optimize/SKILL.md @@ -1,6 +1,6 @@ --- name: optimize -version: 2.3.0 +version: 2.3.1 description: > Analyze a codebase and produce an optimal HawkScan setup — tech flags, scan-policy plugin selection, and stackhawk.yml corrections — then apply it as a non-destructive diff --git a/plugins/stackhawk-data-seed/.claude-plugin/plugin.json b/plugins/stackhawk-data-seed/.claude-plugin/plugin.json index 2a3e89da..39fdd754 100644 --- a/plugins/stackhawk-data-seed/.claude-plugin/plugin.json +++ b/plugins/stackhawk-data-seed/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-data-seed", - "version": "2.3.0", + "version": "2.3.1", "description": "Read a target repo and emit checked-in seed artifacts so HawkScan has data to scan. Generates SQL / HTTP / gRPC / Mongo / shell steps across the target repo and any upstream services it depends on, plus a manifest and a credentials handoff file that hawkscan consumes.", "author": { "name": "StackHawk", diff --git a/plugins/stackhawk-data-seed/.codex-plugin/plugin.json b/plugins/stackhawk-data-seed/.codex-plugin/plugin.json index 5fbbb465..107a2688 100644 --- a/plugins/stackhawk-data-seed/.codex-plugin/plugin.json +++ b/plugins/stackhawk-data-seed/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-data-seed", - "version": "2.3.0", + "version": "2.3.1", "description": "Read a target repo and emit checked-in seed artifacts so HawkScan has data to scan. Generates SQL / HTTP / gRPC / Mongo / shell steps across the target repo and any upstream services it depends on, plus a manifest and a credentials handoff file that hawkscan consumes.", "author": { "name": "StackHawk", diff --git a/plugins/stackhawk-data-seed/skills/stackhawk-data-seed/SKILL.md b/plugins/stackhawk-data-seed/skills/stackhawk-data-seed/SKILL.md index bb8de9f2..e1ed0d62 100644 --- a/plugins/stackhawk-data-seed/skills/stackhawk-data-seed/SKILL.md +++ b/plugins/stackhawk-data-seed/skills/stackhawk-data-seed/SKILL.md @@ -1,6 +1,6 @@ --- name: stackhawk-data-seed -version: 2.3.0 +version: 2.3.1 description: > Set up checked-in seed data so authenticated HawkScan can reach non-trivial paths. Drives the `hawk perch seed` preflight, designs the minimum seed diff --git a/plugins/wingman/.claude-plugin/plugin.json b/plugins/wingman/.claude-plugin/plugin.json index 2521d4e0..4edd1c51 100644 --- a/plugins/wingman/.claude-plugin/plugin.json +++ b/plugins/wingman/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "wingman", - "version": "2.3.0", + "version": "2.3.1", "description": "StackHawk umbrella plugin \u2014 installs the default StackHawk skill set (hawkscan, api, data-seed, optimize) in one command for the combined hawk CLI.", "author": { "name": "StackHawk", diff --git a/plugins/wingman/.codex-plugin/plugin.json b/plugins/wingman/.codex-plugin/plugin.json index 2521d4e0..4edd1c51 100644 --- a/plugins/wingman/.codex-plugin/plugin.json +++ b/plugins/wingman/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "wingman", - "version": "2.3.0", + "version": "2.3.1", "description": "StackHawk umbrella plugin \u2014 installs the default StackHawk skill set (hawkscan, api, data-seed, optimize) in one command for the combined hawk CLI.", "author": { "name": "StackHawk", From 1c3038daa10cbf64898d375b59829e3590711b6c Mon Sep 17 00:00:00 2001 From: KC Berg Date: Thu, 9 Jul 2026 14:48:10 -0600 Subject: [PATCH 12/29] docs(skills): --app takes NAME|UUID across hawkscan/api/optimize Release is coordinated so hawkscan (carrying the hawkop --app name|uuid change) ships before the skills. Skills now present --app as accepting a name or a UUID and drop the --app-id fallback guidance (--app-id remains a working hidden alias). repo link keeps --app-id (that command wasn't unified). Rewrote the authoritative --app/--app-id explanation in api hawk-op-shortcuts.md, the hawkscan companion table + selecting-an-app note, tech-flags.md, platform-model.md, and optimize cli-contract.md. Cursor rules regenerated; 2.3.1 -> 2.3.2. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01WWyT8CN77z9xrJhRLjrtJC --- .claude-plugin/marketplace.json | 12 ++++++------ .claude/skills/skill-authoring/SKILL.md | 2 +- .codex-plugin/plugin.json | 2 +- VERSION | 2 +- cursor/.cursor/rules/stackhawk-api-hawk-op.mdc | 11 +++++------ cursor/.cursor/rules/stackhawk-hawkscan-model.mdc | 2 +- cursor/.cursor/rules/stackhawk-hawkscan.mdc | 12 ++++++------ cursor/.cursor/rules/stackhawk-optimize-cli.mdc | 2 +- gemini-extension.json | 2 +- plugin.json | 2 +- plugins/api/.claude-plugin/plugin.json | 2 +- plugins/api/.codex-plugin/plugin.json | 2 +- plugins/api/skills/api/SKILL.md | 2 +- .../api/skills/api/references/hawk-op-shortcuts.md | 11 +++++------ plugins/hawkscan-ci/.claude-plugin/plugin.json | 2 +- plugins/hawkscan-ci/.codex-plugin/plugin.json | 2 +- plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md | 2 +- plugins/hawkscan/.claude-plugin/plugin.json | 2 +- plugins/hawkscan/.codex-plugin/plugin.json | 2 +- plugins/hawkscan/skills/hawkscan/SKILL.md | 14 +++++++------- .../skills/hawkscan/references/platform-model.md | 2 +- .../skills/hawkscan/references/tech-flags.md | 2 +- plugins/optimize/.claude-plugin/plugin.json | 2 +- plugins/optimize/.codex-plugin/plugin.json | 2 +- plugins/optimize/skills/optimize/SKILL.md | 2 +- .../skills/optimize/references/cli-contract.md | 2 +- .../stackhawk-data-seed/.claude-plugin/plugin.json | 2 +- .../stackhawk-data-seed/.codex-plugin/plugin.json | 2 +- .../skills/stackhawk-data-seed/SKILL.md | 2 +- plugins/wingman/.claude-plugin/plugin.json | 2 +- plugins/wingman/.codex-plugin/plugin.json | 2 +- 31 files changed, 55 insertions(+), 57 deletions(-) diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index cb2b2e92..80e510f1 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -14,7 +14,7 @@ "category": "security", "source": "./plugins/hawkscan", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/hawkscan", - "version": "2.3.1" + "version": "2.3.2" }, { "name": "stackhawk-api", @@ -22,7 +22,7 @@ "category": "security", "source": "./plugins/api", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/api", - "version": "2.3.1" + "version": "2.3.2" }, { "name": "hawkscan-ci", @@ -30,7 +30,7 @@ "category": "security", "source": "./plugins/hawkscan-ci", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/hawkscan-ci", - "version": "2.3.1" + "version": "2.3.2" }, { "name": "stackhawk-data-seed", @@ -38,7 +38,7 @@ "category": "security", "source": "./plugins/stackhawk-data-seed", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/stackhawk-data-seed", - "version": "2.3.1" + "version": "2.3.2" }, { "name": "wingman", @@ -46,7 +46,7 @@ "category": "security", "source": "./plugins/wingman", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/wingman", - "version": "2.3.1" + "version": "2.3.2" }, { "name": "stackhawk-optimize", @@ -54,7 +54,7 @@ "category": "security", "source": "./plugins/optimize", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/optimize", - "version": "2.3.1" + "version": "2.3.2" } ] } diff --git a/.claude/skills/skill-authoring/SKILL.md b/.claude/skills/skill-authoring/SKILL.md index 107c8762..1611481f 100644 --- a/.claude/skills/skill-authoring/SKILL.md +++ b/.claude/skills/skill-authoring/SKILL.md @@ -1,6 +1,6 @@ --- name: skill-authoring -version: 2.3.1 +version: 2.3.2 description: > Guides authoring and maintaining agent skills in this repository — enforcing Anthropic best practices, bumping versions, regenerating Cursor rules, and diff --git a/.codex-plugin/plugin.json b/.codex-plugin/plugin.json index db10c162..88ab9fd2 100644 --- a/.codex-plugin/plugin.json +++ b/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk", - "version": "2.3.1", + "version": "2.3.2", "description": "Dynamic application and API security testing (DAST) powered by HawkScan. Configure scans, interpret findings, and query the StackHawk platform API.", "author": { "name": "StackHawk", diff --git a/VERSION b/VERSION index a6254504..e7034819 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -2.3.1 \ No newline at end of file +2.3.2 \ No newline at end of file diff --git a/cursor/.cursor/rules/stackhawk-api-hawk-op.mdc b/cursor/.cursor/rules/stackhawk-api-hawk-op.mdc index 725d76b4..5fb69fc3 100644 --- a/cursor/.cursor/rules/stackhawk-api-hawk-op.mdc +++ b/cursor/.cursor/rules/stackhawk-api-hawk-op.mdc @@ -109,8 +109,7 @@ extraction, no token handling. ### Latest scan for an app (overview + alerts) ```bash -hawk op scan get --app "" -hawk op scan get --app-id +hawk op scan get --app "" # --app takes a name or a UUID ``` Wraps: `/scan/{orgId}` + `/scan/{scanId}/alerts`. Returns a scan header, alert @@ -301,7 +300,7 @@ call — useful when building a reporting pipeline. - **Don't forget `--detail full` when you need the remediation/HTTP message payload.** The default `hawk op scan get` output is the overview — it won't include per-URI evidence. -- **Don't combine `--app` (name) and `--app-id` (UUID) in the same command.** - Pick one. `--app` resolves by name (case-insensitive exact match); `--app-id` is exact by UUID. - If a name matches **more than one** app, `hawk op` errors and tells you to use `--app-id` — - fall back to `--app-id ` (grab the UUID from `hawk op app list`). +- **`--app` takes a name or a UUID.** A UUID resolves exactly; a name resolves by + case-insensitive exact match. If a name matches **more than one** app, `hawk op` errors and + lists the matches — disambiguate by passing that app's UUID to `--app` (grab it from + `hawk op app list`). (`--app-id` still works as a deprecated hidden alias but isn't needed.) diff --git a/cursor/.cursor/rules/stackhawk-hawkscan-model.mdc b/cursor/.cursor/rules/stackhawk-hawkscan-model.mdc index 08ffbb09..0e1d5736 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan-model.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan-model.mdc @@ -407,6 +407,6 @@ lookups. Relevant commands and their documentation: |-------------------------------|------------------------------------------------------------| | List apps | `hawk op app list --format json` | | List envs for an app | `hawk op env list --app --format json` | -| Get scan findings with triage | `hawk op scan get --app --detail full --format json` | +| Get scan findings with triage | `hawk op scan get --app --detail full --format json` | These reads require the combined `hawk` CLI; the `api` skill covers setup (`hawk init --browser` or the `HAWK_API_KEY` env var). diff --git a/cursor/.cursor/rules/stackhawk-hawkscan.mdc b/cursor/.cursor/rules/stackhawk-hawkscan.mdc index 8994de03..92f2a215 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan.mdc @@ -40,13 +40,13 @@ The `api` skill wraps read-only StackHawk platform lookups via the `hawk` CLI (` | Purpose | Command | |-------------------------------|----------------------------------------------------------------------------------| | Check if App exists | `hawk op app list --format json` | -| Check if Env exists | `hawk op env list --app --format json` | -| Get findings with triage | `hawk op scan get --app --detail full --format json` | +| Check if Env exists | `hawk op env list --app --format json` | +| Get findings with triage | `hawk op scan get --app --detail full --format json` | | List ASM repos | `hawk op repo list --format json` | | Link app to ASM repo | `hawk op repo link --repo-id --app-id ` | -| Get tech flags | `hawk op app tech-flags get --app --format json` | -| Disable all tech flags | `hawk op app tech-flags disable-all --app --yes` | -| Set specific tech flags | `hawk op app tech-flags set --app Key=true` | +| Get tech flags | `hawk op app tech-flags get --app --format json` | +| Disable all tech flags | `hawk op app tech-flags disable-all --app --yes` | +| Set specific tech flags | `hawk op app tech-flags set --app Key=true` | | Triage a finding | `hawk op scan triage --scan --hash --status false-positive --note ""` | | Bulk triage from file | `hawk op scan triage --scan --from-file triage.yaml` | | Annotate w/o triage perm | `hawk op finding note --scan --hash --note "..."` | @@ -55,7 +55,7 @@ The `api` skill wraps read-only StackHawk platform lookups via the `hawk` CLI (` These reads require the combined `hawk` CLI; the `api` skill covers setup (`hawk init --browser` or the `HAWK_API_KEY` env var). -**Selecting an app by UUID:** `--app` takes an application *name*. When you hold a UUID (e.g. from `stackhawk.yml`'s `applicationId`), pass it as `--app-id ` — passing a UUID to `--app` errors with "Application not found" on current released hawk. (Newer hawk also accepts a UUID on `--app`; `--app-id` works on both, so prefer it when you have an ID.) Also parse `--format json` output defensively — a "skills out of date" banner may precede the JSON. +**Selecting an app:** `--app` takes an application **name or UUID** — pass whichever you have (e.g. the `applicationId` from `stackhawk.yml`, or the app's name). No separate flag is needed. Also parse `--format json` output defensively — a "skills out of date" banner may precede the JSON. The `stackhawk-data-seed` skill sets up checked-in backend seed data via `hawk perch seed`. Hand off to it when authentication fails because the backend has no valid credential diff --git a/cursor/.cursor/rules/stackhawk-optimize-cli.mdc b/cursor/.cursor/rules/stackhawk-optimize-cli.mdc index 6fe07fd0..68b5f714 100644 --- a/cursor/.cursor/rules/stackhawk-optimize-cli.mdc +++ b/cursor/.cursor/rules/stackhawk-optimize-cli.mdc @@ -12,7 +12,7 @@ alwaysApply: false | Fetch a base policy as JSON | `hawk op policy get --name ` | | Create/upsert a trial or permanent policy | `hawk op policy create --file --name [--display-name ] [--dry-run]` | | Delete a policy | `hawk op policy delete --name --yes [--dry-run]` | -| (Optional) set app default | `hawk op policy assign --app |--app-id --name [--dry-run]` | +| (Optional) set app default | `hawk op policy assign --app --name [--dry-run]` | | Read canonical tech flags | `hawk op app tech-flags get --app --format json` | | Get per-path scan metrics + signal flags | `hawk op scan metrics [--sort heaviest\|slowest\|erroring\|most-requested] [--top N] [--method ] [--operations] --format json` | diff --git a/gemini-extension.json b/gemini-extension.json index 1d24ab35..71cb35e2 100644 --- a/gemini-extension.json +++ b/gemini-extension.json @@ -1,5 +1,5 @@ { "name": "stackhawk", - "version": "2.3.1", + "version": "2.3.2", "description": "StackHawk security scanning and API reporting skills. AUTONOMOUS: After completing any code change, invoke the hawkscan skill to scan, fix all reported vulnerabilities (not just your changes), and rescan to verify. Also invoke for any mention of hawkscan, stackhawk, DAST, security scan." } diff --git a/plugin.json b/plugin.json index 0e54bc60..6164dd44 100644 --- a/plugin.json +++ b/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk", - "version": "2.3.1", + "version": "2.3.2", "description": "StackHawk security scanning and API reporting skills. Includes hawkscan (DAST scanning) and api (security posture reporting).", "author": { "name": "StackHawk", diff --git a/plugins/api/.claude-plugin/plugin.json b/plugins/api/.claude-plugin/plugin.json index de4274e0..f33d0605 100644 --- a/plugins/api/.claude-plugin/plugin.json +++ b/plugins/api/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-api", - "version": "2.3.1", + "version": "2.3.2", "description": "Query the StackHawk platform API for security posture reporting, findings analysis, and app management. Uses the combined hawk CLI (hawk op \u2026) for all platform queries.", "author": { "name": "StackHawk", diff --git a/plugins/api/.codex-plugin/plugin.json b/plugins/api/.codex-plugin/plugin.json index 0f6c07dd..80950888 100644 --- a/plugins/api/.codex-plugin/plugin.json +++ b/plugins/api/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-api", - "version": "2.3.1", + "version": "2.3.2", "description": "Query the StackHawk platform API for security posture reporting, findings analysis, and app management. Uses the combined hawk CLI (hawk op \u2026) for all platform queries.", "author": { "name": "StackHawk", diff --git a/plugins/api/skills/api/SKILL.md b/plugins/api/skills/api/SKILL.md index 097f6fdf..5e0239a6 100644 --- a/plugins/api/skills/api/SKILL.md +++ b/plugins/api/skills/api/SKILL.md @@ -1,6 +1,6 @@ --- name: api -version: 2.3.1 +version: 2.3.2 description: > Use this skill when a user or agent needs to query the StackHawk platform for security reporting, findings analysis, or app management. Triggers include: diff --git a/plugins/api/skills/api/references/hawk-op-shortcuts.md b/plugins/api/skills/api/references/hawk-op-shortcuts.md index 80ab438d..43a379e4 100644 --- a/plugins/api/skills/api/references/hawk-op-shortcuts.md +++ b/plugins/api/skills/api/references/hawk-op-shortcuts.md @@ -103,8 +103,7 @@ extraction, no token handling. ### Latest scan for an app (overview + alerts) ```bash -hawk op scan get --app "" -hawk op scan get --app-id +hawk op scan get --app "" # --app takes a name or a UUID ``` Wraps: `/scan/{orgId}` + `/scan/{scanId}/alerts`. Returns a scan header, alert @@ -295,7 +294,7 @@ call — useful when building a reporting pipeline. - **Don't forget `--detail full` when you need the remediation/HTTP message payload.** The default `hawk op scan get` output is the overview — it won't include per-URI evidence. -- **Don't combine `--app` (name) and `--app-id` (UUID) in the same command.** - Pick one. `--app` resolves by name (case-insensitive exact match); `--app-id` is exact by UUID. - If a name matches **more than one** app, `hawk op` errors and tells you to use `--app-id` — - fall back to `--app-id ` (grab the UUID from `hawk op app list`). +- **`--app` takes a name or a UUID.** A UUID resolves exactly; a name resolves by + case-insensitive exact match. If a name matches **more than one** app, `hawk op` errors and + lists the matches — disambiguate by passing that app's UUID to `--app` (grab it from + `hawk op app list`). (`--app-id` still works as a deprecated hidden alias but isn't needed.) diff --git a/plugins/hawkscan-ci/.claude-plugin/plugin.json b/plugins/hawkscan-ci/.claude-plugin/plugin.json index 1e29726c..3e6571e1 100644 --- a/plugins/hawkscan-ci/.claude-plugin/plugin.json +++ b/plugins/hawkscan-ci/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "hawkscan-ci", - "version": "2.3.1", + "version": "2.3.2", "description": "Configure HawkScan in your CI/CD pipeline. Provider-agnostic: detects GitHub Actions, GitLab CI, Jenkins, CircleCI, Azure Pipelines, Bitbucket, Buildkite, and other CI systems from repo files, edits the pipeline file in place to add a HawkScan job, and prompts for HAWK_API_KEY storage (CI-native secrets store or external secrets manager).", "author": { "name": "StackHawk", diff --git a/plugins/hawkscan-ci/.codex-plugin/plugin.json b/plugins/hawkscan-ci/.codex-plugin/plugin.json index 4fbd878a..abfb6ade 100644 --- a/plugins/hawkscan-ci/.codex-plugin/plugin.json +++ b/plugins/hawkscan-ci/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "hawkscan-ci", - "version": "2.3.1", + "version": "2.3.2", "description": "Configure HawkScan in your CI/CD pipeline. Provider-agnostic: detects GitHub Actions, GitLab CI, Jenkins, CircleCI, Azure Pipelines, Bitbucket, Buildkite, and other CI systems from repo files, edits the pipeline file in place to add a HawkScan job, and prompts for HAWK_API_KEY storage (CI-native secrets store or external secrets manager).", "author": { "name": "StackHawk", diff --git a/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md b/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md index ffc136ed..4b18ce2e 100644 --- a/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md +++ b/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md @@ -1,6 +1,6 @@ --- name: hawkscan-ci -version: 2.3.1 +version: 2.3.2 description: > Use when the user wants to configure HawkScan in their CI/CD pipeline — triggers on "set up hawkscan in CI", "add stackhawk to my pipeline", diff --git a/plugins/hawkscan/.claude-plugin/plugin.json b/plugins/hawkscan/.claude-plugin/plugin.json index 5f2d0242..6179fdb2 100644 --- a/plugins/hawkscan/.claude-plugin/plugin.json +++ b/plugins/hawkscan/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "hawkscan", - "version": "2.3.1", + "version": "2.3.2", "description": "Configure, run, and interpret HawkScan DAST results inside Claude Code. Generates stackhawk.yml configs, runs scans via CLI or Docker, and transforms security findings into prioritized fix tasks for your coding agent.", "author": { "name": "StackHawk", diff --git a/plugins/hawkscan/.codex-plugin/plugin.json b/plugins/hawkscan/.codex-plugin/plugin.json index dc09ba93..5796150f 100644 --- a/plugins/hawkscan/.codex-plugin/plugin.json +++ b/plugins/hawkscan/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "hawkscan", - "version": "2.3.1", + "version": "2.3.2", "description": "Configure, run, and interpret HawkScan DAST results. Generates stackhawk.yml configs, runs scans via CLI or Docker, and transforms security findings into prioritized fix tasks for your coding agent.", "author": { "name": "StackHawk", diff --git a/plugins/hawkscan/skills/hawkscan/SKILL.md b/plugins/hawkscan/skills/hawkscan/SKILL.md index ee0e8579..25aac2c3 100644 --- a/plugins/hawkscan/skills/hawkscan/SKILL.md +++ b/plugins/hawkscan/skills/hawkscan/SKILL.md @@ -1,6 +1,6 @@ --- name: hawkscan -version: 2.3.1 +version: 2.3.2 description: > Runs the HawkScan DAST security loop — configure, scan, fix all reported vulnerabilities (not just your changes), rescan to verify. Performs @@ -51,13 +51,13 @@ The `api` skill wraps read-only StackHawk platform lookups via the `hawk` CLI (` | Purpose | Command | |-------------------------------|----------------------------------------------------------------------------------| | Check if App exists | `hawk op app list --format json` | -| Check if Env exists | `hawk op env list --app --format json` | -| Get findings with triage | `hawk op scan get --app --detail full --format json` | +| Check if Env exists | `hawk op env list --app --format json` | +| Get findings with triage | `hawk op scan get --app --detail full --format json` | | List ASM repos | `hawk op repo list --format json` | | Link app to ASM repo | `hawk op repo link --repo-id --app-id ` | -| Get tech flags | `hawk op app tech-flags get --app --format json` | -| Disable all tech flags | `hawk op app tech-flags disable-all --app --yes` | -| Set specific tech flags | `hawk op app tech-flags set --app Key=true` | +| Get tech flags | `hawk op app tech-flags get --app --format json` | +| Disable all tech flags | `hawk op app tech-flags disable-all --app --yes` | +| Set specific tech flags | `hawk op app tech-flags set --app Key=true` | | Triage a finding | `hawk op scan triage --scan --hash --status false-positive --note ""` | | Bulk triage from file | `hawk op scan triage --scan --from-file triage.yaml` | | Annotate w/o triage perm | `hawk op finding note --scan --hash --note "..."` | @@ -66,7 +66,7 @@ The `api` skill wraps read-only StackHawk platform lookups via the `hawk` CLI (` These reads require the combined `hawk` CLI; the `api` skill covers setup (`hawk init --browser` or the `HAWK_API_KEY` env var). -**Selecting an app by UUID:** `--app` takes an application *name*. When you hold a UUID (e.g. from `stackhawk.yml`'s `applicationId`), pass it as `--app-id ` — passing a UUID to `--app` errors with "Application not found" on current released hawk. (Newer hawk also accepts a UUID on `--app`; `--app-id` works on both, so prefer it when you have an ID.) Also parse `--format json` output defensively — a "skills out of date" banner may precede the JSON. +**Selecting an app:** `--app` takes an application **name or UUID** — pass whichever you have (e.g. the `applicationId` from `stackhawk.yml`, or the app's name). No separate flag is needed. Also parse `--format json` output defensively — a "skills out of date" banner may precede the JSON. The `stackhawk-data-seed` skill sets up checked-in backend seed data via `hawk perch seed`. Hand off to it when authentication fails because the backend has no valid credential diff --git a/plugins/hawkscan/skills/hawkscan/references/platform-model.md b/plugins/hawkscan/skills/hawkscan/references/platform-model.md index 95697597..23bc81f3 100644 --- a/plugins/hawkscan/skills/hawkscan/references/platform-model.md +++ b/plugins/hawkscan/skills/hawkscan/references/platform-model.md @@ -401,6 +401,6 @@ lookups. Relevant commands and their documentation: |-------------------------------|------------------------------------------------------------| | List apps | `hawk op app list --format json` | | List envs for an app | `hawk op env list --app --format json` | -| Get scan findings with triage | `hawk op scan get --app --detail full --format json` | +| Get scan findings with triage | `hawk op scan get --app --detail full --format json` | These reads require the combined `hawk` CLI; the `api` skill covers setup (`hawk init --browser` or the `HAWK_API_KEY` env var). diff --git a/plugins/hawkscan/skills/hawkscan/references/tech-flags.md b/plugins/hawkscan/skills/hawkscan/references/tech-flags.md index 8fd28e63..5a7db52c 100644 --- a/plugins/hawkscan/skills/hawkscan/references/tech-flags.md +++ b/plugins/hawkscan/skills/hawkscan/references/tech-flags.md @@ -43,7 +43,7 @@ Flag names are **dot-namespaced** (e.g., `Language.Java.Spring`), **case-sensiti hawk op app tech-flags get --app --format json ``` -`--app ` accepts the application name. The `hawk op` commands also accept `--app-id ` if you prefer to use the application ID directly. +`--app ` accepts either the application name or its UUID — pass whichever you have. Returns a JSON object with all available flags and their current true/false state: diff --git a/plugins/optimize/.claude-plugin/plugin.json b/plugins/optimize/.claude-plugin/plugin.json index 77d0915d..fc306442 100644 --- a/plugins/optimize/.claude-plugin/plugin.json +++ b/plugins/optimize/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-optimize", - "version": "2.3.1", + "version": "2.3.2", "description": "Analyze a codebase and produce an optimal HawkScan setup (tech flags, scan-policy plugin selection, stackhawk.yml corrections), apply it as a non-destructive trial, run one trial scan, and promote or discard.", "author": { "name": "StackHawk", diff --git a/plugins/optimize/.codex-plugin/plugin.json b/plugins/optimize/.codex-plugin/plugin.json index 878b2dd1..58ce6412 100644 --- a/plugins/optimize/.codex-plugin/plugin.json +++ b/plugins/optimize/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-optimize", - "version": "2.3.1", + "version": "2.3.2", "description": "Analyze a codebase and produce an optimal HawkScan setup (tech flags, scan-policy plugin selection, stackhawk.yml corrections), apply it as a non-destructive trial, run one trial scan, and promote or discard.", "author": { "name": "StackHawk", diff --git a/plugins/optimize/skills/optimize/SKILL.md b/plugins/optimize/skills/optimize/SKILL.md index 1f6de3f9..90bbe1ed 100644 --- a/plugins/optimize/skills/optimize/SKILL.md +++ b/plugins/optimize/skills/optimize/SKILL.md @@ -1,6 +1,6 @@ --- name: optimize -version: 2.3.1 +version: 2.3.2 description: > Analyze a codebase and produce an optimal HawkScan setup — tech flags, scan-policy plugin selection, and stackhawk.yml corrections — then apply it as a non-destructive diff --git a/plugins/optimize/skills/optimize/references/cli-contract.md b/plugins/optimize/skills/optimize/references/cli-contract.md index 17751924..9f92115e 100644 --- a/plugins/optimize/skills/optimize/references/cli-contract.md +++ b/plugins/optimize/skills/optimize/references/cli-contract.md @@ -6,7 +6,7 @@ | Fetch a base policy as JSON | `hawk op policy get --name ` | | Create/upsert a trial or permanent policy | `hawk op policy create --file --name [--display-name ] [--dry-run]` | | Delete a policy | `hawk op policy delete --name --yes [--dry-run]` | -| (Optional) set app default | `hawk op policy assign --app |--app-id --name [--dry-run]` | +| (Optional) set app default | `hawk op policy assign --app --name [--dry-run]` | | Read canonical tech flags | `hawk op app tech-flags get --app --format json` | | Get per-path scan metrics + signal flags | `hawk op scan metrics [--sort heaviest\|slowest\|erroring\|most-requested] [--top N] [--method ] [--operations] --format json` | diff --git a/plugins/stackhawk-data-seed/.claude-plugin/plugin.json b/plugins/stackhawk-data-seed/.claude-plugin/plugin.json index 39fdd754..9ce16413 100644 --- a/plugins/stackhawk-data-seed/.claude-plugin/plugin.json +++ b/plugins/stackhawk-data-seed/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-data-seed", - "version": "2.3.1", + "version": "2.3.2", "description": "Read a target repo and emit checked-in seed artifacts so HawkScan has data to scan. Generates SQL / HTTP / gRPC / Mongo / shell steps across the target repo and any upstream services it depends on, plus a manifest and a credentials handoff file that hawkscan consumes.", "author": { "name": "StackHawk", diff --git a/plugins/stackhawk-data-seed/.codex-plugin/plugin.json b/plugins/stackhawk-data-seed/.codex-plugin/plugin.json index 107a2688..866576de 100644 --- a/plugins/stackhawk-data-seed/.codex-plugin/plugin.json +++ b/plugins/stackhawk-data-seed/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-data-seed", - "version": "2.3.1", + "version": "2.3.2", "description": "Read a target repo and emit checked-in seed artifacts so HawkScan has data to scan. Generates SQL / HTTP / gRPC / Mongo / shell steps across the target repo and any upstream services it depends on, plus a manifest and a credentials handoff file that hawkscan consumes.", "author": { "name": "StackHawk", diff --git a/plugins/stackhawk-data-seed/skills/stackhawk-data-seed/SKILL.md b/plugins/stackhawk-data-seed/skills/stackhawk-data-seed/SKILL.md index e1ed0d62..403feb56 100644 --- a/plugins/stackhawk-data-seed/skills/stackhawk-data-seed/SKILL.md +++ b/plugins/stackhawk-data-seed/skills/stackhawk-data-seed/SKILL.md @@ -1,6 +1,6 @@ --- name: stackhawk-data-seed -version: 2.3.1 +version: 2.3.2 description: > Set up checked-in seed data so authenticated HawkScan can reach non-trivial paths. Drives the `hawk perch seed` preflight, designs the minimum seed diff --git a/plugins/wingman/.claude-plugin/plugin.json b/plugins/wingman/.claude-plugin/plugin.json index 4edd1c51..0d3b4e9a 100644 --- a/plugins/wingman/.claude-plugin/plugin.json +++ b/plugins/wingman/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "wingman", - "version": "2.3.1", + "version": "2.3.2", "description": "StackHawk umbrella plugin \u2014 installs the default StackHawk skill set (hawkscan, api, data-seed, optimize) in one command for the combined hawk CLI.", "author": { "name": "StackHawk", diff --git a/plugins/wingman/.codex-plugin/plugin.json b/plugins/wingman/.codex-plugin/plugin.json index 4edd1c51..0d3b4e9a 100644 --- a/plugins/wingman/.codex-plugin/plugin.json +++ b/plugins/wingman/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "wingman", - "version": "2.3.1", + "version": "2.3.2", "description": "StackHawk umbrella plugin \u2014 installs the default StackHawk skill set (hawkscan, api, data-seed, optimize) in one command for the combined hawk CLI.", "author": { "name": "StackHawk", From 5b4bdf8a124cab233b0096af9914b4d0d23fc297 Mon Sep 17 00:00:00 2001 From: KC Berg Date: Thu, 9 Jul 2026 15:41:22 -0600 Subject: [PATCH 13/29] feat(hawkscan): preflight runs 'hawk skills status' to surface stale skills MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds an early skills-freshness check to the preflight — recommends the user update (surfacing the exact /plugin update command) when the installed agent skills are out of date, before continuing. Pairs with the hawkscan-side removal of the ambient per-command staleness banner: freshness is now reported only on the explicit 'hawk skills status', which the skill drives once up front. Recommendation, not a hard stop; skipped gracefully if the subcommand predates the installed hawk. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01WWyT8CN77z9xrJhRLjrtJC --- .claude-plugin/marketplace.json | 12 ++++++------ .claude/skills/skill-authoring/SKILL.md | 2 +- .codex-plugin/plugin.json | 2 +- VERSION | 2 +- cursor/.cursor/rules/stackhawk-hawkscan.mdc | 8 ++++++++ gemini-extension.json | 2 +- plugin.json | 2 +- plugins/api/.claude-plugin/plugin.json | 2 +- plugins/api/.codex-plugin/plugin.json | 2 +- plugins/api/skills/api/SKILL.md | 2 +- plugins/hawkscan-ci/.claude-plugin/plugin.json | 2 +- plugins/hawkscan-ci/.codex-plugin/plugin.json | 2 +- plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md | 2 +- plugins/hawkscan/.claude-plugin/plugin.json | 2 +- plugins/hawkscan/.codex-plugin/plugin.json | 2 +- plugins/hawkscan/skills/hawkscan/SKILL.md | 10 +++++++++- plugins/optimize/.claude-plugin/plugin.json | 2 +- plugins/optimize/.codex-plugin/plugin.json | 2 +- plugins/optimize/skills/optimize/SKILL.md | 2 +- .../stackhawk-data-seed/.claude-plugin/plugin.json | 2 +- .../stackhawk-data-seed/.codex-plugin/plugin.json | 2 +- .../skills/stackhawk-data-seed/SKILL.md | 2 +- plugins/wingman/.claude-plugin/plugin.json | 2 +- plugins/wingman/.codex-plugin/plugin.json | 2 +- 24 files changed, 44 insertions(+), 28 deletions(-) diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index 80e510f1..f1ad7257 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -14,7 +14,7 @@ "category": "security", "source": "./plugins/hawkscan", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/hawkscan", - "version": "2.3.2" + "version": "2.3.3" }, { "name": "stackhawk-api", @@ -22,7 +22,7 @@ "category": "security", "source": "./plugins/api", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/api", - "version": "2.3.2" + "version": "2.3.3" }, { "name": "hawkscan-ci", @@ -30,7 +30,7 @@ "category": "security", "source": "./plugins/hawkscan-ci", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/hawkscan-ci", - "version": "2.3.2" + "version": "2.3.3" }, { "name": "stackhawk-data-seed", @@ -38,7 +38,7 @@ "category": "security", "source": "./plugins/stackhawk-data-seed", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/stackhawk-data-seed", - "version": "2.3.2" + "version": "2.3.3" }, { "name": "wingman", @@ -46,7 +46,7 @@ "category": "security", "source": "./plugins/wingman", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/wingman", - "version": "2.3.2" + "version": "2.3.3" }, { "name": "stackhawk-optimize", @@ -54,7 +54,7 @@ "category": "security", "source": "./plugins/optimize", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/optimize", - "version": "2.3.2" + "version": "2.3.3" } ] } diff --git a/.claude/skills/skill-authoring/SKILL.md b/.claude/skills/skill-authoring/SKILL.md index 1611481f..bb03566b 100644 --- a/.claude/skills/skill-authoring/SKILL.md +++ b/.claude/skills/skill-authoring/SKILL.md @@ -1,6 +1,6 @@ --- name: skill-authoring -version: 2.3.2 +version: 2.3.3 description: > Guides authoring and maintaining agent skills in this repository — enforcing Anthropic best practices, bumping versions, regenerating Cursor rules, and diff --git a/.codex-plugin/plugin.json b/.codex-plugin/plugin.json index 88ab9fd2..5a091de2 100644 --- a/.codex-plugin/plugin.json +++ b/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk", - "version": "2.3.2", + "version": "2.3.3", "description": "Dynamic application and API security testing (DAST) powered by HawkScan. Configure scans, interpret findings, and query the StackHawk platform API.", "author": { "name": "StackHawk", diff --git a/VERSION b/VERSION index e7034819..45674f16 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -2.3.2 \ No newline at end of file +2.3.3 \ No newline at end of file diff --git a/cursor/.cursor/rules/stackhawk-hawkscan.mdc b/cursor/.cursor/rules/stackhawk-hawkscan.mdc index 92f2a215..d22d7684 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan.mdc @@ -23,12 +23,20 @@ This skill requires **hawk v6.0.0 or newer**. Verify: ```bash hawk version hawk config --help >/dev/null 2>&1 || echo "MISSING: hawk config — upgrade hawk to v6.0.0+" +hawk skills status ``` If hawk is older than `6.0.0` or `hawk config --help` fails, stop and tell the user to upgrade before proceeding. Do not inline auth recipes from memory — they live in `hawk config show` and are stale by design when hardcoded. +**`hawk skills status`** reports whether the installed StackHawk agent skills are current. +If it lists anything out of date, surface the exact upgrade command it prints (e.g. +`/plugin marketplace update stackhawk && /plugin update hawkscan`) and recommend the user +update before continuing — a stale skill may be missing fixes for the very issues you'll hit. +It's a recommendation, not a hard stop: proceed if the user prefers. (If the subcommand isn't +recognized, the installed hawk predates it — skip this check.) + See `references/installation.md` for upgrade and install instructions. --- diff --git a/gemini-extension.json b/gemini-extension.json index 71cb35e2..f4fca1c8 100644 --- a/gemini-extension.json +++ b/gemini-extension.json @@ -1,5 +1,5 @@ { "name": "stackhawk", - "version": "2.3.2", + "version": "2.3.3", "description": "StackHawk security scanning and API reporting skills. AUTONOMOUS: After completing any code change, invoke the hawkscan skill to scan, fix all reported vulnerabilities (not just your changes), and rescan to verify. Also invoke for any mention of hawkscan, stackhawk, DAST, security scan." } diff --git a/plugin.json b/plugin.json index 6164dd44..d5322f8b 100644 --- a/plugin.json +++ b/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk", - "version": "2.3.2", + "version": "2.3.3", "description": "StackHawk security scanning and API reporting skills. Includes hawkscan (DAST scanning) and api (security posture reporting).", "author": { "name": "StackHawk", diff --git a/plugins/api/.claude-plugin/plugin.json b/plugins/api/.claude-plugin/plugin.json index f33d0605..55d5b61b 100644 --- a/plugins/api/.claude-plugin/plugin.json +++ b/plugins/api/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-api", - "version": "2.3.2", + "version": "2.3.3", "description": "Query the StackHawk platform API for security posture reporting, findings analysis, and app management. Uses the combined hawk CLI (hawk op \u2026) for all platform queries.", "author": { "name": "StackHawk", diff --git a/plugins/api/.codex-plugin/plugin.json b/plugins/api/.codex-plugin/plugin.json index 80950888..e28da4e7 100644 --- a/plugins/api/.codex-plugin/plugin.json +++ b/plugins/api/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-api", - "version": "2.3.2", + "version": "2.3.3", "description": "Query the StackHawk platform API for security posture reporting, findings analysis, and app management. Uses the combined hawk CLI (hawk op \u2026) for all platform queries.", "author": { "name": "StackHawk", diff --git a/plugins/api/skills/api/SKILL.md b/plugins/api/skills/api/SKILL.md index 5e0239a6..a5c7dcf9 100644 --- a/plugins/api/skills/api/SKILL.md +++ b/plugins/api/skills/api/SKILL.md @@ -1,6 +1,6 @@ --- name: api -version: 2.3.2 +version: 2.3.3 description: > Use this skill when a user or agent needs to query the StackHawk platform for security reporting, findings analysis, or app management. Triggers include: diff --git a/plugins/hawkscan-ci/.claude-plugin/plugin.json b/plugins/hawkscan-ci/.claude-plugin/plugin.json index 3e6571e1..085d9853 100644 --- a/plugins/hawkscan-ci/.claude-plugin/plugin.json +++ b/plugins/hawkscan-ci/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "hawkscan-ci", - "version": "2.3.2", + "version": "2.3.3", "description": "Configure HawkScan in your CI/CD pipeline. Provider-agnostic: detects GitHub Actions, GitLab CI, Jenkins, CircleCI, Azure Pipelines, Bitbucket, Buildkite, and other CI systems from repo files, edits the pipeline file in place to add a HawkScan job, and prompts for HAWK_API_KEY storage (CI-native secrets store or external secrets manager).", "author": { "name": "StackHawk", diff --git a/plugins/hawkscan-ci/.codex-plugin/plugin.json b/plugins/hawkscan-ci/.codex-plugin/plugin.json index abfb6ade..9076e3b0 100644 --- a/plugins/hawkscan-ci/.codex-plugin/plugin.json +++ b/plugins/hawkscan-ci/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "hawkscan-ci", - "version": "2.3.2", + "version": "2.3.3", "description": "Configure HawkScan in your CI/CD pipeline. Provider-agnostic: detects GitHub Actions, GitLab CI, Jenkins, CircleCI, Azure Pipelines, Bitbucket, Buildkite, and other CI systems from repo files, edits the pipeline file in place to add a HawkScan job, and prompts for HAWK_API_KEY storage (CI-native secrets store or external secrets manager).", "author": { "name": "StackHawk", diff --git a/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md b/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md index 4b18ce2e..b12ab09e 100644 --- a/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md +++ b/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md @@ -1,6 +1,6 @@ --- name: hawkscan-ci -version: 2.3.2 +version: 2.3.3 description: > Use when the user wants to configure HawkScan in their CI/CD pipeline — triggers on "set up hawkscan in CI", "add stackhawk to my pipeline", diff --git a/plugins/hawkscan/.claude-plugin/plugin.json b/plugins/hawkscan/.claude-plugin/plugin.json index 6179fdb2..4e3e04f9 100644 --- a/plugins/hawkscan/.claude-plugin/plugin.json +++ b/plugins/hawkscan/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "hawkscan", - "version": "2.3.2", + "version": "2.3.3", "description": "Configure, run, and interpret HawkScan DAST results inside Claude Code. Generates stackhawk.yml configs, runs scans via CLI or Docker, and transforms security findings into prioritized fix tasks for your coding agent.", "author": { "name": "StackHawk", diff --git a/plugins/hawkscan/.codex-plugin/plugin.json b/plugins/hawkscan/.codex-plugin/plugin.json index 5796150f..b93138ca 100644 --- a/plugins/hawkscan/.codex-plugin/plugin.json +++ b/plugins/hawkscan/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "hawkscan", - "version": "2.3.2", + "version": "2.3.3", "description": "Configure, run, and interpret HawkScan DAST results. Generates stackhawk.yml configs, runs scans via CLI or Docker, and transforms security findings into prioritized fix tasks for your coding agent.", "author": { "name": "StackHawk", diff --git a/plugins/hawkscan/skills/hawkscan/SKILL.md b/plugins/hawkscan/skills/hawkscan/SKILL.md index 25aac2c3..698d993d 100644 --- a/plugins/hawkscan/skills/hawkscan/SKILL.md +++ b/plugins/hawkscan/skills/hawkscan/SKILL.md @@ -1,6 +1,6 @@ --- name: hawkscan -version: 2.3.2 +version: 2.3.3 description: > Runs the HawkScan DAST security loop — configure, scan, fix all reported vulnerabilities (not just your changes), rescan to verify. Performs @@ -34,12 +34,20 @@ This skill requires **hawk v6.0.0 or newer**. Verify: ```bash hawk version hawk config --help >/dev/null 2>&1 || echo "MISSING: hawk config — upgrade hawk to v6.0.0+" +hawk skills status ``` If hawk is older than `6.0.0` or `hawk config --help` fails, stop and tell the user to upgrade before proceeding. Do not inline auth recipes from memory — they live in `hawk config show` and are stale by design when hardcoded. +**`hawk skills status`** reports whether the installed StackHawk agent skills are current. +If it lists anything out of date, surface the exact upgrade command it prints (e.g. +`/plugin marketplace update stackhawk && /plugin update hawkscan`) and recommend the user +update before continuing — a stale skill may be missing fixes for the very issues you'll hit. +It's a recommendation, not a hard stop: proceed if the user prefers. (If the subcommand isn't +recognized, the installed hawk predates it — skip this check.) + See `references/installation.md` for upgrade and install instructions. --- diff --git a/plugins/optimize/.claude-plugin/plugin.json b/plugins/optimize/.claude-plugin/plugin.json index fc306442..17536fe3 100644 --- a/plugins/optimize/.claude-plugin/plugin.json +++ b/plugins/optimize/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-optimize", - "version": "2.3.2", + "version": "2.3.3", "description": "Analyze a codebase and produce an optimal HawkScan setup (tech flags, scan-policy plugin selection, stackhawk.yml corrections), apply it as a non-destructive trial, run one trial scan, and promote or discard.", "author": { "name": "StackHawk", diff --git a/plugins/optimize/.codex-plugin/plugin.json b/plugins/optimize/.codex-plugin/plugin.json index 58ce6412..b04fe9c6 100644 --- a/plugins/optimize/.codex-plugin/plugin.json +++ b/plugins/optimize/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-optimize", - "version": "2.3.2", + "version": "2.3.3", "description": "Analyze a codebase and produce an optimal HawkScan setup (tech flags, scan-policy plugin selection, stackhawk.yml corrections), apply it as a non-destructive trial, run one trial scan, and promote or discard.", "author": { "name": "StackHawk", diff --git a/plugins/optimize/skills/optimize/SKILL.md b/plugins/optimize/skills/optimize/SKILL.md index 90bbe1ed..4c1ce9ba 100644 --- a/plugins/optimize/skills/optimize/SKILL.md +++ b/plugins/optimize/skills/optimize/SKILL.md @@ -1,6 +1,6 @@ --- name: optimize -version: 2.3.2 +version: 2.3.3 description: > Analyze a codebase and produce an optimal HawkScan setup — tech flags, scan-policy plugin selection, and stackhawk.yml corrections — then apply it as a non-destructive diff --git a/plugins/stackhawk-data-seed/.claude-plugin/plugin.json b/plugins/stackhawk-data-seed/.claude-plugin/plugin.json index 9ce16413..525c7ff5 100644 --- a/plugins/stackhawk-data-seed/.claude-plugin/plugin.json +++ b/plugins/stackhawk-data-seed/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-data-seed", - "version": "2.3.2", + "version": "2.3.3", "description": "Read a target repo and emit checked-in seed artifacts so HawkScan has data to scan. Generates SQL / HTTP / gRPC / Mongo / shell steps across the target repo and any upstream services it depends on, plus a manifest and a credentials handoff file that hawkscan consumes.", "author": { "name": "StackHawk", diff --git a/plugins/stackhawk-data-seed/.codex-plugin/plugin.json b/plugins/stackhawk-data-seed/.codex-plugin/plugin.json index 866576de..a9b16289 100644 --- a/plugins/stackhawk-data-seed/.codex-plugin/plugin.json +++ b/plugins/stackhawk-data-seed/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-data-seed", - "version": "2.3.2", + "version": "2.3.3", "description": "Read a target repo and emit checked-in seed artifacts so HawkScan has data to scan. Generates SQL / HTTP / gRPC / Mongo / shell steps across the target repo and any upstream services it depends on, plus a manifest and a credentials handoff file that hawkscan consumes.", "author": { "name": "StackHawk", diff --git a/plugins/stackhawk-data-seed/skills/stackhawk-data-seed/SKILL.md b/plugins/stackhawk-data-seed/skills/stackhawk-data-seed/SKILL.md index 403feb56..10c7cf4b 100644 --- a/plugins/stackhawk-data-seed/skills/stackhawk-data-seed/SKILL.md +++ b/plugins/stackhawk-data-seed/skills/stackhawk-data-seed/SKILL.md @@ -1,6 +1,6 @@ --- name: stackhawk-data-seed -version: 2.3.2 +version: 2.3.3 description: > Set up checked-in seed data so authenticated HawkScan can reach non-trivial paths. Drives the `hawk perch seed` preflight, designs the minimum seed diff --git a/plugins/wingman/.claude-plugin/plugin.json b/plugins/wingman/.claude-plugin/plugin.json index 0d3b4e9a..d0b4f2cb 100644 --- a/plugins/wingman/.claude-plugin/plugin.json +++ b/plugins/wingman/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "wingman", - "version": "2.3.2", + "version": "2.3.3", "description": "StackHawk umbrella plugin \u2014 installs the default StackHawk skill set (hawkscan, api, data-seed, optimize) in one command for the combined hawk CLI.", "author": { "name": "StackHawk", diff --git a/plugins/wingman/.codex-plugin/plugin.json b/plugins/wingman/.codex-plugin/plugin.json index 0d3b4e9a..d0b4f2cb 100644 --- a/plugins/wingman/.codex-plugin/plugin.json +++ b/plugins/wingman/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "wingman", - "version": "2.3.2", + "version": "2.3.3", "description": "StackHawk umbrella plugin \u2014 installs the default StackHawk skill set (hawkscan, api, data-seed, optimize) in one command for the combined hawk CLI.", "author": { "name": "StackHawk", From e1d647a464a1a8b0c72a61bd67e565b7f60ef16d Mon Sep 17 00:00:00 2001 From: KC Berg Date: Mon, 13 Jul 2026 09:01:21 -0600 Subject: [PATCH 14/29] docs(hawkscan): sync gate check count + base-path-mismatch across refs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Local pr-review caught three text-consistency gaps from commit a05e16a (adding base-path-mismatch) not fully propagated: - autonomous-loop.md Step 5 config-class list was missing base-path-mismatch (the unattended path — the one place a doc gap bites hardest). - 'four checks' naming (SKILL.md, scan-quality.md header/ToC, autonomous-loop.md) didn't reflect the added base-path resolve check — now 'five checks' (coverage evidence-only + base-path/auth/surface/health gating). - scan-quality.md said 'seven reason identifiers' but lists eight. Cursor rules regenerated (idempotent); 2.3.3 -> 2.3.4. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01WWyT8CN77z9xrJhRLjrtJC --- .claude-plugin/marketplace.json | 12 ++++++------ .claude/skills/skill-authoring/SKILL.md | 2 +- .codex-plugin/plugin.json | 2 +- VERSION | 2 +- .../rules/stackhawk-hawkscan-scan-quality.mdc | 8 ++++---- cursor/.cursor/rules/stackhawk-hawkscan.mdc | 5 +++-- gemini-extension.json | 2 +- plugin.json | 2 +- plugins/api/.claude-plugin/plugin.json | 2 +- plugins/api/.codex-plugin/plugin.json | 2 +- plugins/api/skills/api/SKILL.md | 2 +- plugins/hawkscan-ci/.claude-plugin/plugin.json | 2 +- plugins/hawkscan-ci/.codex-plugin/plugin.json | 2 +- plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md | 2 +- plugins/hawkscan/.claude-plugin/plugin.json | 2 +- plugins/hawkscan/.codex-plugin/plugin.json | 2 +- plugins/hawkscan/skills/hawkscan/SKILL.md | 7 ++++--- .../skills/hawkscan/references/autonomous-loop.md | 7 ++++--- .../skills/hawkscan/references/scan-quality.md | 8 ++++---- plugins/optimize/.claude-plugin/plugin.json | 2 +- plugins/optimize/.codex-plugin/plugin.json | 2 +- plugins/optimize/skills/optimize/SKILL.md | 2 +- .../stackhawk-data-seed/.claude-plugin/plugin.json | 2 +- .../stackhawk-data-seed/.codex-plugin/plugin.json | 2 +- .../skills/stackhawk-data-seed/SKILL.md | 2 +- plugins/wingman/.claude-plugin/plugin.json | 2 +- plugins/wingman/.codex-plugin/plugin.json | 2 +- 27 files changed, 46 insertions(+), 43 deletions(-) diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index f1ad7257..bca16232 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -14,7 +14,7 @@ "category": "security", "source": "./plugins/hawkscan", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/hawkscan", - "version": "2.3.3" + "version": "2.3.4" }, { "name": "stackhawk-api", @@ -22,7 +22,7 @@ "category": "security", "source": "./plugins/api", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/api", - "version": "2.3.3" + "version": "2.3.4" }, { "name": "hawkscan-ci", @@ -30,7 +30,7 @@ "category": "security", "source": "./plugins/hawkscan-ci", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/hawkscan-ci", - "version": "2.3.3" + "version": "2.3.4" }, { "name": "stackhawk-data-seed", @@ -38,7 +38,7 @@ "category": "security", "source": "./plugins/stackhawk-data-seed", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/stackhawk-data-seed", - "version": "2.3.3" + "version": "2.3.4" }, { "name": "wingman", @@ -46,7 +46,7 @@ "category": "security", "source": "./plugins/wingman", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/wingman", - "version": "2.3.3" + "version": "2.3.4" }, { "name": "stackhawk-optimize", @@ -54,7 +54,7 @@ "category": "security", "source": "./plugins/optimize", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/optimize", - "version": "2.3.3" + "version": "2.3.4" } ] } diff --git a/.claude/skills/skill-authoring/SKILL.md b/.claude/skills/skill-authoring/SKILL.md index bb03566b..723822b2 100644 --- a/.claude/skills/skill-authoring/SKILL.md +++ b/.claude/skills/skill-authoring/SKILL.md @@ -1,6 +1,6 @@ --- name: skill-authoring -version: 2.3.3 +version: 2.3.4 description: > Guides authoring and maintaining agent skills in this repository — enforcing Anthropic best practices, bumping versions, regenerating Cursor rules, and diff --git a/.codex-plugin/plugin.json b/.codex-plugin/plugin.json index 5a091de2..2ecf7b37 100644 --- a/.codex-plugin/plugin.json +++ b/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk", - "version": "2.3.3", + "version": "2.3.4", "description": "Dynamic application and API security testing (DAST) powered by HawkScan. Configure scans, interpret findings, and query the StackHawk platform API.", "author": { "name": "StackHawk", diff --git a/VERSION b/VERSION index 45674f16..fd06a926 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -2.3.3 \ No newline at end of file +2.3.4 \ No newline at end of file diff --git a/cursor/.cursor/rules/stackhawk-hawkscan-scan-quality.mdc b/cursor/.cursor/rules/stackhawk-hawkscan-scan-quality.mdc index 3563ceab..8329a31e 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan-scan-quality.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan-scan-quality.mdc @@ -33,7 +33,7 @@ across every config is an interactive operation only; never trigger one autonomo ## Contents - [Derive the expectation (fresh, every scan)](#derive-the-expectation-fresh-every-scan) -- [The four checks](#the-four-checks) +- [The five checks](#the-five-checks) - [On gaps: iterate the config](#on-gaps-iterate-the-config) - [Reporting rules (replace the old autonomy lock)](#reporting-rules-replace-the-old-autonomy-lock) - [Degradation](#degradation) @@ -67,9 +67,9 @@ it against. Build that expectation the same way every time, right after the scan back per the Degradation section below — detect the gap by the subcommand erroring or not existing, don't ask the user which version they have. -## The four checks +## The five checks -Run all four after every scan. Each has its own command, pass condition, and a stable +Run all five after every scan. Each has its own command, pass condition, and a stable reason identifier used when reporting a gap — this gate's own reporting (Step 4.5 in SKILL.md) uses these identifiers directly, so keep them exactly as written here across any edits to this file: `coverage-gap`, `spec-not-wired`, `surface-unscanned`, @@ -157,7 +157,7 @@ verify the app, retry once for free, and stop if it recurs. - **Performance-class signals are never gate gaps.** If health metrics point at slowness rather than unreachability or all-4xx, that's an `optimize` suggestion, not a quality-gate reason identifier. -- Keep every reported gap tied to one of the seven reason identifiers above so this gate's +- Keep every reported gap tied to one of the eight reason identifiers above so this gate's own reporting stays specific rather than parsing prose — the identifiers must stay stable across edits to this file. diff --git a/cursor/.cursor/rules/stackhawk-hawkscan.mdc b/cursor/.cursor/rules/stackhawk-hawkscan.mdc index d22d7684..b4b36f64 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan.mdc @@ -356,8 +356,9 @@ straight to Step 6's diagnosis / the environment-class path instead. It is a fee into config tuning, **not a governor**: gate state never blocks a finding from being reported and fixed; a thin scan that missed surface can still have found something real. -Run the four structural checks and derive the expectation fresh each time, exactly as -`references/scan-quality.md` describes — coverage, auth, surface-completeness, and health. +Run the five checks and derive the expectation fresh each time, exactly as +`references/scan-quality.md` describes — coverage (evidence-only), base-path resolve, auth, +surface-completeness, and health. On config-class gaps (`spec-not-wired`, `surface-unscanned`, `auth-validate-failed`, `auth-wall`, `all-4xx`, `base-path-mismatch`), loop back to Step 2b with an additive-only fix, batch every gap into one edit pass, and rescan once. Iteration cap: 2 interactive / 1 autonomous, per diff --git a/gemini-extension.json b/gemini-extension.json index f4fca1c8..a647d26c 100644 --- a/gemini-extension.json +++ b/gemini-extension.json @@ -1,5 +1,5 @@ { "name": "stackhawk", - "version": "2.3.3", + "version": "2.3.4", "description": "StackHawk security scanning and API reporting skills. AUTONOMOUS: After completing any code change, invoke the hawkscan skill to scan, fix all reported vulnerabilities (not just your changes), and rescan to verify. Also invoke for any mention of hawkscan, stackhawk, DAST, security scan." } diff --git a/plugin.json b/plugin.json index d5322f8b..cc62ccd2 100644 --- a/plugin.json +++ b/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk", - "version": "2.3.3", + "version": "2.3.4", "description": "StackHawk security scanning and API reporting skills. Includes hawkscan (DAST scanning) and api (security posture reporting).", "author": { "name": "StackHawk", diff --git a/plugins/api/.claude-plugin/plugin.json b/plugins/api/.claude-plugin/plugin.json index 55d5b61b..755ef838 100644 --- a/plugins/api/.claude-plugin/plugin.json +++ b/plugins/api/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-api", - "version": "2.3.3", + "version": "2.3.4", "description": "Query the StackHawk platform API for security posture reporting, findings analysis, and app management. Uses the combined hawk CLI (hawk op \u2026) for all platform queries.", "author": { "name": "StackHawk", diff --git a/plugins/api/.codex-plugin/plugin.json b/plugins/api/.codex-plugin/plugin.json index e28da4e7..6da84d58 100644 --- a/plugins/api/.codex-plugin/plugin.json +++ b/plugins/api/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-api", - "version": "2.3.3", + "version": "2.3.4", "description": "Query the StackHawk platform API for security posture reporting, findings analysis, and app management. Uses the combined hawk CLI (hawk op \u2026) for all platform queries.", "author": { "name": "StackHawk", diff --git a/plugins/api/skills/api/SKILL.md b/plugins/api/skills/api/SKILL.md index a5c7dcf9..692e1031 100644 --- a/plugins/api/skills/api/SKILL.md +++ b/plugins/api/skills/api/SKILL.md @@ -1,6 +1,6 @@ --- name: api -version: 2.3.3 +version: 2.3.4 description: > Use this skill when a user or agent needs to query the StackHawk platform for security reporting, findings analysis, or app management. Triggers include: diff --git a/plugins/hawkscan-ci/.claude-plugin/plugin.json b/plugins/hawkscan-ci/.claude-plugin/plugin.json index 085d9853..1e2d4811 100644 --- a/plugins/hawkscan-ci/.claude-plugin/plugin.json +++ b/plugins/hawkscan-ci/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "hawkscan-ci", - "version": "2.3.3", + "version": "2.3.4", "description": "Configure HawkScan in your CI/CD pipeline. Provider-agnostic: detects GitHub Actions, GitLab CI, Jenkins, CircleCI, Azure Pipelines, Bitbucket, Buildkite, and other CI systems from repo files, edits the pipeline file in place to add a HawkScan job, and prompts for HAWK_API_KEY storage (CI-native secrets store or external secrets manager).", "author": { "name": "StackHawk", diff --git a/plugins/hawkscan-ci/.codex-plugin/plugin.json b/plugins/hawkscan-ci/.codex-plugin/plugin.json index 9076e3b0..32d5bd34 100644 --- a/plugins/hawkscan-ci/.codex-plugin/plugin.json +++ b/plugins/hawkscan-ci/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "hawkscan-ci", - "version": "2.3.3", + "version": "2.3.4", "description": "Configure HawkScan in your CI/CD pipeline. Provider-agnostic: detects GitHub Actions, GitLab CI, Jenkins, CircleCI, Azure Pipelines, Bitbucket, Buildkite, and other CI systems from repo files, edits the pipeline file in place to add a HawkScan job, and prompts for HAWK_API_KEY storage (CI-native secrets store or external secrets manager).", "author": { "name": "StackHawk", diff --git a/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md b/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md index b12ab09e..d752d75a 100644 --- a/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md +++ b/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md @@ -1,6 +1,6 @@ --- name: hawkscan-ci -version: 2.3.3 +version: 2.3.4 description: > Use when the user wants to configure HawkScan in their CI/CD pipeline — triggers on "set up hawkscan in CI", "add stackhawk to my pipeline", diff --git a/plugins/hawkscan/.claude-plugin/plugin.json b/plugins/hawkscan/.claude-plugin/plugin.json index 4e3e04f9..a257fbdf 100644 --- a/plugins/hawkscan/.claude-plugin/plugin.json +++ b/plugins/hawkscan/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "hawkscan", - "version": "2.3.3", + "version": "2.3.4", "description": "Configure, run, and interpret HawkScan DAST results inside Claude Code. Generates stackhawk.yml configs, runs scans via CLI or Docker, and transforms security findings into prioritized fix tasks for your coding agent.", "author": { "name": "StackHawk", diff --git a/plugins/hawkscan/.codex-plugin/plugin.json b/plugins/hawkscan/.codex-plugin/plugin.json index b93138ca..fb9a537f 100644 --- a/plugins/hawkscan/.codex-plugin/plugin.json +++ b/plugins/hawkscan/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "hawkscan", - "version": "2.3.3", + "version": "2.3.4", "description": "Configure, run, and interpret HawkScan DAST results. Generates stackhawk.yml configs, runs scans via CLI or Docker, and transforms security findings into prioritized fix tasks for your coding agent.", "author": { "name": "StackHawk", diff --git a/plugins/hawkscan/skills/hawkscan/SKILL.md b/plugins/hawkscan/skills/hawkscan/SKILL.md index 698d993d..cd58e4ad 100644 --- a/plugins/hawkscan/skills/hawkscan/SKILL.md +++ b/plugins/hawkscan/skills/hawkscan/SKILL.md @@ -1,6 +1,6 @@ --- name: hawkscan -version: 2.3.3 +version: 2.3.4 description: > Runs the HawkScan DAST security loop — configure, scan, fix all reported vulnerabilities (not just your changes), rescan to verify. Performs @@ -367,8 +367,9 @@ straight to Step 6's diagnosis / the environment-class path instead. It is a fee into config tuning, **not a governor**: gate state never blocks a finding from being reported and fixed; a thin scan that missed surface can still have found something real. -Run the four structural checks and derive the expectation fresh each time, exactly as -`references/scan-quality.md` describes — coverage, auth, surface-completeness, and health. +Run the five checks and derive the expectation fresh each time, exactly as +`references/scan-quality.md` describes — coverage (evidence-only), base-path resolve, auth, +surface-completeness, and health. On config-class gaps (`spec-not-wired`, `surface-unscanned`, `auth-validate-failed`, `auth-wall`, `all-4xx`, `base-path-mismatch`), loop back to Step 2b with an additive-only fix, batch every gap into one edit pass, and rescan once. Iteration cap: 2 interactive / 1 autonomous, per diff --git a/plugins/hawkscan/skills/hawkscan/references/autonomous-loop.md b/plugins/hawkscan/skills/hawkscan/references/autonomous-loop.md index 29af898f..6986b87e 100644 --- a/plugins/hawkscan/skills/hawkscan/references/autonomous-loop.md +++ b/plugins/hawkscan/skills/hawkscan/references/autonomous-loop.md @@ -50,9 +50,10 @@ Capture `scan.id` from the JSON output — you'll need it for rescan. **5. Quality gate** -Before triaging findings, run the quality gate (`scan-quality.md`'s four checks — coverage, -auth, surface-completeness, health) against this scan. On config-class gaps -(`spec-not-wired`, `surface-unscanned`, `auth-validate-failed`, `auth-wall`, `all-4xx`), loop +Before triaging findings, run the quality gate (`scan-quality.md`'s five checks — coverage, +base-path resolve, auth, surface-completeness, health) against this scan. On config-class gaps +(`spec-not-wired`, `surface-unscanned`, `auth-validate-failed`, `auth-wall`, `all-4xx`, +`base-path-mismatch`), loop back to config tuning with a single additive-only batched fix and rescan once — the autonomous cap here is **1** config-fix iteration (interactive workflows get 2; this unattended loop gets 1). On environment-class gaps (`env-unreachable`: connection failures, diff --git a/plugins/hawkscan/skills/hawkscan/references/scan-quality.md b/plugins/hawkscan/skills/hawkscan/references/scan-quality.md index 13eacd07..2cad6e61 100644 --- a/plugins/hawkscan/skills/hawkscan/references/scan-quality.md +++ b/plugins/hawkscan/skills/hawkscan/references/scan-quality.md @@ -27,7 +27,7 @@ across every config is an interactive operation only; never trigger one autonomo ## Contents - [Derive the expectation (fresh, every scan)](#derive-the-expectation-fresh-every-scan) -- [The four checks](#the-four-checks) +- [The five checks](#the-five-checks) - [On gaps: iterate the config](#on-gaps-iterate-the-config) - [Reporting rules (replace the old autonomy lock)](#reporting-rules-replace-the-old-autonomy-lock) - [Degradation](#degradation) @@ -61,9 +61,9 @@ it against. Build that expectation the same way every time, right after the scan back per the Degradation section below — detect the gap by the subcommand erroring or not existing, don't ask the user which version they have. -## The four checks +## The five checks -Run all four after every scan. Each has its own command, pass condition, and a stable +Run all five after every scan. Each has its own command, pass condition, and a stable reason identifier used when reporting a gap — this gate's own reporting (Step 4.5 in SKILL.md) uses these identifiers directly, so keep them exactly as written here across any edits to this file: `coverage-gap`, `spec-not-wired`, `surface-unscanned`, @@ -151,7 +151,7 @@ verify the app, retry once for free, and stop if it recurs. - **Performance-class signals are never gate gaps.** If health metrics point at slowness rather than unreachability or all-4xx, that's an `optimize` suggestion, not a quality-gate reason identifier. -- Keep every reported gap tied to one of the seven reason identifiers above so this gate's +- Keep every reported gap tied to one of the eight reason identifiers above so this gate's own reporting stays specific rather than parsing prose — the identifiers must stay stable across edits to this file. diff --git a/plugins/optimize/.claude-plugin/plugin.json b/plugins/optimize/.claude-plugin/plugin.json index 17536fe3..a1cf258f 100644 --- a/plugins/optimize/.claude-plugin/plugin.json +++ b/plugins/optimize/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-optimize", - "version": "2.3.3", + "version": "2.3.4", "description": "Analyze a codebase and produce an optimal HawkScan setup (tech flags, scan-policy plugin selection, stackhawk.yml corrections), apply it as a non-destructive trial, run one trial scan, and promote or discard.", "author": { "name": "StackHawk", diff --git a/plugins/optimize/.codex-plugin/plugin.json b/plugins/optimize/.codex-plugin/plugin.json index b04fe9c6..e3d88001 100644 --- a/plugins/optimize/.codex-plugin/plugin.json +++ b/plugins/optimize/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-optimize", - "version": "2.3.3", + "version": "2.3.4", "description": "Analyze a codebase and produce an optimal HawkScan setup (tech flags, scan-policy plugin selection, stackhawk.yml corrections), apply it as a non-destructive trial, run one trial scan, and promote or discard.", "author": { "name": "StackHawk", diff --git a/plugins/optimize/skills/optimize/SKILL.md b/plugins/optimize/skills/optimize/SKILL.md index 4c1ce9ba..aacdf36d 100644 --- a/plugins/optimize/skills/optimize/SKILL.md +++ b/plugins/optimize/skills/optimize/SKILL.md @@ -1,6 +1,6 @@ --- name: optimize -version: 2.3.3 +version: 2.3.4 description: > Analyze a codebase and produce an optimal HawkScan setup — tech flags, scan-policy plugin selection, and stackhawk.yml corrections — then apply it as a non-destructive diff --git a/plugins/stackhawk-data-seed/.claude-plugin/plugin.json b/plugins/stackhawk-data-seed/.claude-plugin/plugin.json index 525c7ff5..ea9f7bcb 100644 --- a/plugins/stackhawk-data-seed/.claude-plugin/plugin.json +++ b/plugins/stackhawk-data-seed/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-data-seed", - "version": "2.3.3", + "version": "2.3.4", "description": "Read a target repo and emit checked-in seed artifacts so HawkScan has data to scan. Generates SQL / HTTP / gRPC / Mongo / shell steps across the target repo and any upstream services it depends on, plus a manifest and a credentials handoff file that hawkscan consumes.", "author": { "name": "StackHawk", diff --git a/plugins/stackhawk-data-seed/.codex-plugin/plugin.json b/plugins/stackhawk-data-seed/.codex-plugin/plugin.json index a9b16289..41134578 100644 --- a/plugins/stackhawk-data-seed/.codex-plugin/plugin.json +++ b/plugins/stackhawk-data-seed/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-data-seed", - "version": "2.3.3", + "version": "2.3.4", "description": "Read a target repo and emit checked-in seed artifacts so HawkScan has data to scan. Generates SQL / HTTP / gRPC / Mongo / shell steps across the target repo and any upstream services it depends on, plus a manifest and a credentials handoff file that hawkscan consumes.", "author": { "name": "StackHawk", diff --git a/plugins/stackhawk-data-seed/skills/stackhawk-data-seed/SKILL.md b/plugins/stackhawk-data-seed/skills/stackhawk-data-seed/SKILL.md index 10c7cf4b..6d4e507e 100644 --- a/plugins/stackhawk-data-seed/skills/stackhawk-data-seed/SKILL.md +++ b/plugins/stackhawk-data-seed/skills/stackhawk-data-seed/SKILL.md @@ -1,6 +1,6 @@ --- name: stackhawk-data-seed -version: 2.3.3 +version: 2.3.4 description: > Set up checked-in seed data so authenticated HawkScan can reach non-trivial paths. Drives the `hawk perch seed` preflight, designs the minimum seed diff --git a/plugins/wingman/.claude-plugin/plugin.json b/plugins/wingman/.claude-plugin/plugin.json index d0b4f2cb..60b8a7d2 100644 --- a/plugins/wingman/.claude-plugin/plugin.json +++ b/plugins/wingman/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "wingman", - "version": "2.3.3", + "version": "2.3.4", "description": "StackHawk umbrella plugin \u2014 installs the default StackHawk skill set (hawkscan, api, data-seed, optimize) in one command for the combined hawk CLI.", "author": { "name": "StackHawk", diff --git a/plugins/wingman/.codex-plugin/plugin.json b/plugins/wingman/.codex-plugin/plugin.json index d0b4f2cb..60b8a7d2 100644 --- a/plugins/wingman/.codex-plugin/plugin.json +++ b/plugins/wingman/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "wingman", - "version": "2.3.3", + "version": "2.3.4", "description": "StackHawk umbrella plugin \u2014 installs the default StackHawk skill set (hawkscan, api, data-seed, optimize) in one command for the combined hawk CLI.", "author": { "name": "StackHawk", From cc4759f79d9ac6209ce3ea78a0b1f6cf12fb00fd Mon Sep 17 00:00:00 2001 From: KC Berg Date: Mon, 13 Jul 2026 09:28:45 -0600 Subject: [PATCH 15/29] docs(skills): drop hawkop-lib version gating and design-lineage refs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The agent only knows the hawk CLI version (preflight requires >= 6.0.0), not the embedded hawkop-lib version, so "hawk op >= 0.11.7" gates were unknowable and unnecessary — scan uris/config ship with the combined CLI. Reframe to behavior-based degradation ("skip if the subcommand errors"). Also remove struck-design lineage that a fresh user has no context for: "(replace the old autonomy lock)" heading, "no longer treats exit 0 as success", "superseded by", legacy standalone-binary config file, and the deprecated --app-id alias note. Anchor rename propagated to the in-file ToC; cursor rules regenerated. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_012sAkR2BzkbLxzYUuGzLAQa --- .../.cursor/rules/stackhawk-api-hawk-op.mdc | 5 ++-- cursor/.cursor/rules/stackhawk-api.mdc | 3 +-- .../rules/stackhawk-hawkscan-scan-quality.mdc | 27 ++++++++++--------- cursor/.cursor/rules/stackhawk-hawkscan.mdc | 6 ++--- plugins/api/skills/api/SKILL.md | 3 +-- .../api/references/hawk-op-shortcuts.md | 5 ++-- plugins/hawkscan/skills/hawkscan/SKILL.md | 6 ++--- .../hawkscan/references/app-discovery.md | 9 +++---- .../hawkscan/references/scan-quality.md | 27 ++++++++++--------- 9 files changed, 44 insertions(+), 47 deletions(-) diff --git a/cursor/.cursor/rules/stackhawk-api-hawk-op.mdc b/cursor/.cursor/rules/stackhawk-api-hawk-op.mdc index 5fb69fc3..96a89135 100644 --- a/cursor/.cursor/rules/stackhawk-api-hawk-op.mdc +++ b/cursor/.cursor/rules/stackhawk-api-hawk-op.mdc @@ -53,8 +53,7 @@ hawk op status # Confirm auth ``` `hawk init` writes credentials to `~/.hawk/hawk.properties`. -`hawk op` reads that file plus the `HAWK_API_KEY` env var. The legacy -standalone-binary config file is no longer used. +`hawk op` reads that file plus the `HAWK_API_KEY` env var. **CI/CD only:** For pipeline use, set `HAWK_API_KEY` directly as a secret. Org ID and output format can be set via env vars: @@ -303,4 +302,4 @@ call — useful when building a reporting pipeline. - **`--app` takes a name or a UUID.** A UUID resolves exactly; a name resolves by case-insensitive exact match. If a name matches **more than one** app, `hawk op` errors and lists the matches — disambiguate by passing that app's UUID to `--app` (grab it from - `hawk op app list`). (`--app-id` still works as a deprecated hidden alias but isn't needed.) + `hawk op app list`). diff --git a/cursor/.cursor/rules/stackhawk-api.mdc b/cursor/.cursor/rules/stackhawk-api.mdc index 02a45825..6bbb128e 100644 --- a/cursor/.cursor/rules/stackhawk-api.mdc +++ b/cursor/.cursor/rules/stackhawk-api.mdc @@ -33,8 +33,7 @@ Before making any calls, check what's available: - valid org + JWT → proceed. - installed but unconfigured → `hawk init --browser` (interactive; the combined binary has no `hawk op init` — `hawk init` writes `~/.hawk/hawk.properties`, which - `hawk op` reads along with the `HAWK_API_KEY` env var; the legacy - standalone-binary config file is no longer used). + `hawk op` reads along with the `HAWK_API_KEY` env var). - not installed → instruct the user to install the `hawk` CLI (docs: [docs.stackhawk.com](https://docs.stackhawk.com)) and stop; do **not** fall back to curl. diff --git a/cursor/.cursor/rules/stackhawk-hawkscan-scan-quality.mdc b/cursor/.cursor/rules/stackhawk-hawkscan-scan-quality.mdc index 8329a31e..cfef8899 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan-scan-quality.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan-scan-quality.mdc @@ -35,7 +35,7 @@ across every config is an interactive operation only; never trigger one autonomo - [Derive the expectation (fresh, every scan)](#derive-the-expectation-fresh-every-scan) - [The five checks](#the-five-checks) - [On gaps: iterate the config](#on-gaps-iterate-the-config) -- [Reporting rules (replace the old autonomy lock)](#reporting-rules-replace-the-old-autonomy-lock) +- [Reporting rules](#reporting-rules) - [Degradation](#degradation) --- @@ -63,9 +63,10 @@ it against. Build that expectation the same way every time, right after the scan Spring, `swagger-jsdoc`/`@nestjs/swagger` for Node, `drf-yasg`/`drf-spectacular` for Django REST Framework, `swashbuckle` for .NET). Surface the recommendation; don't apply it unrequested. -- `hawk op scan uris` and `hawk op scan config` require hawk op ≥ 0.11.7. Older CLIs fall - back per the Degradation section below — detect the gap by the subcommand erroring or not - existing, don't ask the user which version they have. +- `hawk op scan uris` and `hawk op scan config` ship with the combined `hawk` CLI the + preflight already requires. If either subcommand errors or isn't recognized on the + installed CLI, fall back per the Degradation section below — detect it by the command + failing, don't ask the user which version they have. ## The five checks @@ -142,7 +143,7 @@ clean one. Environment-class gaps never enter this loop at all — see the Signal classes note above: verify the app, retry once for free, and stop if it recurs. -## Reporting rules (replace the old autonomy lock) +## Reporting rules - **Findings are always reported and always fixable**, independent of gate state. The gate never withholds, delays, or downgrades a finding — a scan that under-covered the surface @@ -150,10 +151,10 @@ verify the app, retry once for free, and stop if it recurs. - **Never say the scan is "done and secure"** (or any equivalent framing) while a config-class or environment-class gap is still open. Say plainly what was scanned, what wasn't, and why — the reason identifiers exist so this statement is specific, not vague. -- **The autonomous post-code-change loop no longer treats exit 0 as silent success.** It - runs this gate after its scan, and if gaps remain once the autonomous iteration cap (1) is - reached, it reports them explicitly — reason identifiers, coverage route-diff evidence, - and next steps — instead of accepting a clean exit code at face value. +- **The autonomous post-code-change loop reports gate gaps rather than treating exit 0 as + success.** It runs this gate after its scan, and if gaps remain once the autonomous + iteration cap (1) is reached, it reports them explicitly — reason identifiers, coverage + route-diff evidence, and next steps — instead of accepting a clean exit code at face value. - **Performance-class signals are never gate gaps.** If health metrics point at slowness rather than unreachability or all-4xx, that's an `optimize` suggestion, not a quality-gate reason identifier. @@ -163,9 +164,9 @@ verify the app, retry once for free, and stop if it recurs. ## Degradation -`hawk op scan uris` and `hawk op scan config` need hawk op ≥ 0.11.7. Detect the absence by -the subcommand erroring or not existing at all — don't ask the user to check their version, -just degrade silently and keep going: +`hawk op scan uris` and `hawk op scan config` ship with the combined `hawk` CLI. If either +subcommand errors or isn't recognized on the installed CLI, degrade silently and keep going — +detect it by the command failing, don't ask the user to check their version: - **Coverage (check 1)** degrades from a full scanned-vs-expected route diff to a single before/after number: `url_count` from `hawk op scan get `. You lose the @@ -175,7 +176,7 @@ just degrade silently and keep going: confirm a surface's config exists and its spec block is present, and use the `hawk validate config` output for each file to confirm the config is structurally sound. - **Auth and health (checks 2 and 4)** are unaffected — `hawk op scan metrics` and - `hawk validate auth` work on all hawk op versions and remain the primary signal either way. + `hawk validate auth` remain the primary signal either way. `hawk op scan stats` does not exist yet (it's a follow-up ticket) but will eventually add measured, per-path engine statistics rather than the grep- and `url_count`-based estimates diff --git a/cursor/.cursor/rules/stackhawk-hawkscan.mdc b/cursor/.cursor/rules/stackhawk-hawkscan.mdc index b4b36f64..76936eaf 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan.mdc @@ -58,8 +58,8 @@ The `api` skill wraps read-only StackHawk platform lookups via the `hawk` CLI (` | Triage a finding | `hawk op scan triage --scan --hash --status false-positive --note ""` | | Bulk triage from file | `hawk op scan triage --scan --from-file triage.yaml` | | Annotate w/o triage perm | `hawk op finding note --scan --hash --note "..."` | -| Get scanned URIs | `hawk op scan uris --format json` (hawk op ≥ 0.11.7; skip if absent) | -| Get effective scan config | `hawk op scan config ` (hawk op ≥ 0.11.7; skip if absent) | +| Get scanned URIs | `hawk op scan uris --format json` (skip if the subcommand errors) | +| Get effective scan config | `hawk op scan config ` (skip if the subcommand errors) | These reads require the combined `hawk` CLI; the `api` skill covers setup (`hawk init --browser` or the `HAWK_API_KEY` env var). @@ -370,7 +370,7 @@ At the cap, or once checks are clean, proceed to Step 4 regardless — findings reported and fixable. Never say a scan is "done and secure" while a gate gap is open; state plainly what was scanned, what wasn't, and why, using the reason identifiers. -→ Full check definitions, commands, degradation without hawk op ≥ 0.11.7, and the +→ Full check definitions, commands, degradation when a subcommand is unavailable, and the iteration/reporting rules: [`references/scan-quality.md`](references/scan-quality.md) --- diff --git a/plugins/api/skills/api/SKILL.md b/plugins/api/skills/api/SKILL.md index 692e1031..10c2676f 100644 --- a/plugins/api/skills/api/SKILL.md +++ b/plugins/api/skills/api/SKILL.md @@ -40,8 +40,7 @@ Before making any calls, check what's available: - valid org + JWT → proceed. - installed but unconfigured → `hawk init --browser` (interactive; the combined binary has no `hawk op init` — `hawk init` writes `~/.hawk/hawk.properties`, which - `hawk op` reads along with the `HAWK_API_KEY` env var; the legacy - standalone-binary config file is no longer used). + `hawk op` reads along with the `HAWK_API_KEY` env var). - not installed → instruct the user to install the `hawk` CLI (docs: [docs.stackhawk.com](https://docs.stackhawk.com)) and stop; do **not** fall back to curl. diff --git a/plugins/api/skills/api/references/hawk-op-shortcuts.md b/plugins/api/skills/api/references/hawk-op-shortcuts.md index 43a379e4..2550197f 100644 --- a/plugins/api/skills/api/references/hawk-op-shortcuts.md +++ b/plugins/api/skills/api/references/hawk-op-shortcuts.md @@ -47,8 +47,7 @@ hawk op status # Confirm auth ``` `hawk init` writes credentials to `~/.hawk/hawk.properties`. -`hawk op` reads that file plus the `HAWK_API_KEY` env var. The legacy -standalone-binary config file is no longer used. +`hawk op` reads that file plus the `HAWK_API_KEY` env var. **CI/CD only:** For pipeline use, set `HAWK_API_KEY` directly as a secret. Org ID and output format can be set via env vars: @@ -297,4 +296,4 @@ call — useful when building a reporting pipeline. - **`--app` takes a name or a UUID.** A UUID resolves exactly; a name resolves by case-insensitive exact match. If a name matches **more than one** app, `hawk op` errors and lists the matches — disambiguate by passing that app's UUID to `--app` (grab it from - `hawk op app list`). (`--app-id` still works as a deprecated hidden alias but isn't needed.) + `hawk op app list`). diff --git a/plugins/hawkscan/skills/hawkscan/SKILL.md b/plugins/hawkscan/skills/hawkscan/SKILL.md index cd58e4ad..6ec0e533 100644 --- a/plugins/hawkscan/skills/hawkscan/SKILL.md +++ b/plugins/hawkscan/skills/hawkscan/SKILL.md @@ -69,8 +69,8 @@ The `api` skill wraps read-only StackHawk platform lookups via the `hawk` CLI (` | Triage a finding | `hawk op scan triage --scan --hash --status false-positive --note ""` | | Bulk triage from file | `hawk op scan triage --scan --from-file triage.yaml` | | Annotate w/o triage perm | `hawk op finding note --scan --hash --note "..."` | -| Get scanned URIs | `hawk op scan uris --format json` (hawk op ≥ 0.11.7; skip if absent) | -| Get effective scan config | `hawk op scan config ` (hawk op ≥ 0.11.7; skip if absent) | +| Get scanned URIs | `hawk op scan uris --format json` (skip if the subcommand errors) | +| Get effective scan config | `hawk op scan config ` (skip if the subcommand errors) | These reads require the combined `hawk` CLI; the `api` skill covers setup (`hawk init --browser` or the `HAWK_API_KEY` env var). @@ -381,7 +381,7 @@ At the cap, or once checks are clean, proceed to Step 4 regardless — findings reported and fixable. Never say a scan is "done and secure" while a gate gap is open; state plainly what was scanned, what wasn't, and why, using the reason identifiers. -→ Full check definitions, commands, degradation without hawk op ≥ 0.11.7, and the +→ Full check definitions, commands, degradation when a subcommand is unavailable, and the iteration/reporting rules: [`references/scan-quality.md`](references/scan-quality.md) --- diff --git a/plugins/hawkscan/skills/hawkscan/references/app-discovery.md b/plugins/hawkscan/skills/hawkscan/references/app-discovery.md index d184f0d5..70f985e8 100644 --- a/plugins/hawkscan/skills/hawkscan/references/app-discovery.md +++ b/plugins/hawkscan/skills/hawkscan/references/app-discovery.md @@ -1,10 +1,9 @@ # App Discovery — Docs-First Source Table -This file's discovery procedure has been superseded by `references/scan-planning.md` — see -Step 1a in SKILL.md for the full workflow (per-surface detection, route-inventory -derivation, gap recommendations, and the user-confirmed summary required before the first -scan). What remains here is the docs-first source table below, which SKILL.md links to -directly and which stands on its own. +The full discovery workflow lives in `references/scan-planning.md` — see Step 1a in SKILL.md +(per-surface detection, route-inventory derivation, gap recommendations, and the +user-confirmed summary required before the first scan). This file holds the docs-first source +table below, which SKILL.md links to directly and which stands on its own. ## Read the repo's own docs first diff --git a/plugins/hawkscan/skills/hawkscan/references/scan-quality.md b/plugins/hawkscan/skills/hawkscan/references/scan-quality.md index 2cad6e61..994f7511 100644 --- a/plugins/hawkscan/skills/hawkscan/references/scan-quality.md +++ b/plugins/hawkscan/skills/hawkscan/references/scan-quality.md @@ -29,7 +29,7 @@ across every config is an interactive operation only; never trigger one autonomo - [Derive the expectation (fresh, every scan)](#derive-the-expectation-fresh-every-scan) - [The five checks](#the-five-checks) - [On gaps: iterate the config](#on-gaps-iterate-the-config) -- [Reporting rules (replace the old autonomy lock)](#reporting-rules-replace-the-old-autonomy-lock) +- [Reporting rules](#reporting-rules) - [Degradation](#degradation) --- @@ -57,9 +57,10 @@ it against. Build that expectation the same way every time, right after the scan Spring, `swagger-jsdoc`/`@nestjs/swagger` for Node, `drf-yasg`/`drf-spectacular` for Django REST Framework, `swashbuckle` for .NET). Surface the recommendation; don't apply it unrequested. -- `hawk op scan uris` and `hawk op scan config` require hawk op ≥ 0.11.7. Older CLIs fall - back per the Degradation section below — detect the gap by the subcommand erroring or not - existing, don't ask the user which version they have. +- `hawk op scan uris` and `hawk op scan config` ship with the combined `hawk` CLI the + preflight already requires. If either subcommand errors or isn't recognized on the + installed CLI, fall back per the Degradation section below — detect it by the command + failing, don't ask the user which version they have. ## The five checks @@ -136,7 +137,7 @@ clean one. Environment-class gaps never enter this loop at all — see the Signal classes note above: verify the app, retry once for free, and stop if it recurs. -## Reporting rules (replace the old autonomy lock) +## Reporting rules - **Findings are always reported and always fixable**, independent of gate state. The gate never withholds, delays, or downgrades a finding — a scan that under-covered the surface @@ -144,10 +145,10 @@ verify the app, retry once for free, and stop if it recurs. - **Never say the scan is "done and secure"** (or any equivalent framing) while a config-class or environment-class gap is still open. Say plainly what was scanned, what wasn't, and why — the reason identifiers exist so this statement is specific, not vague. -- **The autonomous post-code-change loop no longer treats exit 0 as silent success.** It - runs this gate after its scan, and if gaps remain once the autonomous iteration cap (1) is - reached, it reports them explicitly — reason identifiers, coverage route-diff evidence, - and next steps — instead of accepting a clean exit code at face value. +- **The autonomous post-code-change loop reports gate gaps rather than treating exit 0 as + success.** It runs this gate after its scan, and if gaps remain once the autonomous + iteration cap (1) is reached, it reports them explicitly — reason identifiers, coverage + route-diff evidence, and next steps — instead of accepting a clean exit code at face value. - **Performance-class signals are never gate gaps.** If health metrics point at slowness rather than unreachability or all-4xx, that's an `optimize` suggestion, not a quality-gate reason identifier. @@ -157,9 +158,9 @@ verify the app, retry once for free, and stop if it recurs. ## Degradation -`hawk op scan uris` and `hawk op scan config` need hawk op ≥ 0.11.7. Detect the absence by -the subcommand erroring or not existing at all — don't ask the user to check their version, -just degrade silently and keep going: +`hawk op scan uris` and `hawk op scan config` ship with the combined `hawk` CLI. If either +subcommand errors or isn't recognized on the installed CLI, degrade silently and keep going — +detect it by the command failing, don't ask the user to check their version: - **Coverage (check 1)** degrades from a full scanned-vs-expected route diff to a single before/after number: `url_count` from `hawk op scan get `. You lose the @@ -169,7 +170,7 @@ just degrade silently and keep going: confirm a surface's config exists and its spec block is present, and use the `hawk validate config` output for each file to confirm the config is structurally sound. - **Auth and health (checks 2 and 4)** are unaffected — `hawk op scan metrics` and - `hawk validate auth` work on all hawk op versions and remain the primary signal either way. + `hawk validate auth` remain the primary signal either way. `hawk op scan stats` does not exist yet (it's a follow-up ticket) but will eventually add measured, per-path engine statistics rather than the grep- and `url_count`-based estimates From 49103589944bcb1a34587eb3a714a2ff2f38118e Mon Sep 17 00:00:00 2001 From: KC Berg Date: Mon, 13 Jul 2026 09:50:34 -0600 Subject: [PATCH 16/29] docs(hawkscan): surface spec resolve-check in pre-scan summary; note gate blind spot MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Base-path mismatch is the documented worst-case failure (wall of 404s), yet the only pre-scan defense was a manual curl resolve-check buried in openapi-specs.md. Fold its result into the mandatory user-confirmation summary in scan-planning.md so the highest-value check is no longer skippable. Also document in scan-quality.md that the gate re-derives its no-spec expectation with the same greps discovery uses, so it inherits discovery's detection blind spots — the user-confirmed discovery summary is the real backstop, not the gate. ENG-704. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_012sAkR2BzkbLxzYUuGzLAQa --- .../.cursor/rules/stackhawk-hawkscan-scan-planning.mdc | 10 +++++++--- .../.cursor/rules/stackhawk-hawkscan-scan-quality.mdc | 7 +++++++ .../skills/hawkscan/references/scan-planning.md | 10 +++++++--- .../skills/hawkscan/references/scan-quality.md | 7 +++++++ 4 files changed, 28 insertions(+), 6 deletions(-) diff --git a/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc b/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc index b370bafa..9bcfcee4 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc @@ -184,9 +184,13 @@ For each surface: **Before the first scan of a fresh repo, present a one-screen summary and get the user's confirmation.** The summary should state, per surface: the surface type, which config file -covers it, the scan order and why, the auth approach, and the expected route count with how -it was derived (the same grep/spec-count evidence gathered above). Confirming this up front -is cheap; discovering the scan order or auth assumption was wrong after a scan has already +covers it, the scan order and why, the auth approach, the expected route count with how it +was derived (the same grep/spec-count evidence gathered above), and — for any REST surface +with a wired spec — the result of the mandatory resolve-check (e.g. "3/3 sample spec paths +resolved against host"; see `openapi-specs.md`). The resolve-check is the one thing that +catches a base-path mismatch before it wastes a whole scan, so its result belongs in the +summary rather than left to chance. Confirming this up front is cheap; discovering the scan +order, auth assumption, or a spec that 404s every path was wrong after a scan has already run is not. ## Cross-checking diff --git a/cursor/.cursor/rules/stackhawk-hawkscan-scan-quality.mdc b/cursor/.cursor/rules/stackhawk-hawkscan-scan-quality.mdc index cfef8899..6d74d2a3 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan-scan-quality.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan-scan-quality.mdc @@ -68,6 +68,13 @@ it against. Build that expectation the same way every time, right after the scan installed CLI, fall back per the Degradation section below — detect it by the command failing, don't ask the user which version they have. +Because the no-spec expectation is re-derived with the **same** per-framework greps discovery +uses, the gate inherits discovery's blind spots: a surface neither discovery nor this +re-derivation detects (a second gRPC service, a dynamically-mounted router) won't be flagged +as unscanned — the gate can't miss-diff against a surface it never counted. This is why the +user-confirmed discovery summary (`scan-planning.md`) is the real backstop for surface +completeness, not the gate alone. + ## The five checks Run all five after every scan. Each has its own command, pass condition, and a stable diff --git a/plugins/hawkscan/skills/hawkscan/references/scan-planning.md b/plugins/hawkscan/skills/hawkscan/references/scan-planning.md index 06a25cca..8a83dccf 100644 --- a/plugins/hawkscan/skills/hawkscan/references/scan-planning.md +++ b/plugins/hawkscan/skills/hawkscan/references/scan-planning.md @@ -178,9 +178,13 @@ For each surface: **Before the first scan of a fresh repo, present a one-screen summary and get the user's confirmation.** The summary should state, per surface: the surface type, which config file -covers it, the scan order and why, the auth approach, and the expected route count with how -it was derived (the same grep/spec-count evidence gathered above). Confirming this up front -is cheap; discovering the scan order or auth assumption was wrong after a scan has already +covers it, the scan order and why, the auth approach, the expected route count with how it +was derived (the same grep/spec-count evidence gathered above), and — for any REST surface +with a wired spec — the result of the mandatory resolve-check (e.g. "3/3 sample spec paths +resolved against host"; see `openapi-specs.md`). The resolve-check is the one thing that +catches a base-path mismatch before it wastes a whole scan, so its result belongs in the +summary rather than left to chance. Confirming this up front is cheap; discovering the scan +order, auth assumption, or a spec that 404s every path was wrong after a scan has already run is not. ## Cross-checking diff --git a/plugins/hawkscan/skills/hawkscan/references/scan-quality.md b/plugins/hawkscan/skills/hawkscan/references/scan-quality.md index 994f7511..24c17118 100644 --- a/plugins/hawkscan/skills/hawkscan/references/scan-quality.md +++ b/plugins/hawkscan/skills/hawkscan/references/scan-quality.md @@ -62,6 +62,13 @@ it against. Build that expectation the same way every time, right after the scan installed CLI, fall back per the Degradation section below — detect it by the command failing, don't ask the user which version they have. +Because the no-spec expectation is re-derived with the **same** per-framework greps discovery +uses, the gate inherits discovery's blind spots: a surface neither discovery nor this +re-derivation detects (a second gRPC service, a dynamically-mounted router) won't be flagged +as unscanned — the gate can't miss-diff against a surface it never counted. This is why the +user-confirmed discovery summary (`scan-planning.md`) is the real backstop for surface +completeness, not the gate alone. + ## The five checks Run all five after every scan. Each has its own command, pass condition, and a stable From 9b9f56d5a0eeebd8b1cf89d93f7c7843478bcaaf Mon Sep 17 00:00:00 2001 From: KC Berg Date: Mon, 13 Jul 2026 14:16:57 -0600 Subject: [PATCH 17/29] docs(hawkscan): parameterize runnable examples to prevent literal-match; record rule MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit LLMs copy runnable code blocks verbatim, so an app-specific literal baked into a command is a trap — it runs against a different repo and silently does the wrong thing. The resolve-check curl in openapi-specs.md used literal /authors and /api/v1/authors; replace with / placeholders that fail loudly on verbatim paste, keeping the concrete values as labeled e.g. comments. Add one framing line to scan-planning.md's derivation table (illustrative starting points — not literal src/ or openapi.yaml). Record the underlying principle in the skill-authoring skill: runnable blocks use placeholders that fail loudly; concrete values live in prose/comments as e.g.; literals only when genuinely universal. ENG-704. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_012sAkR2BzkbLxzYUuGzLAQa --- .claude/skills/skill-authoring/SKILL.md | 26 +++++++++++++++++++ .../stackhawk-hawkscan-openapi-specs.mdc | 15 ++++++----- .../stackhawk-hawkscan-scan-planning.mdc | 5 +++- .../hawkscan/references/openapi-specs.md | 15 ++++++----- .../hawkscan/references/scan-planning.md | 5 +++- 5 files changed, 52 insertions(+), 14 deletions(-) diff --git a/.claude/skills/skill-authoring/SKILL.md b/.claude/skills/skill-authoring/SKILL.md index 723822b2..bab869ee 100644 --- a/.claude/skills/skill-authoring/SKILL.md +++ b/.claude/skills/skill-authoring/SKILL.md @@ -106,6 +106,32 @@ immediately before the first `##` section. Format: Shorter files may omit the ToC. +### Examples vs. placeholders (avoid literal-match traps) + +Agents copy **runnable code blocks verbatim** far more readily than they adapt them. A +concrete app-specific value baked into a command is therefore a trap: the agent runs it +literally against a different codebase and silently does the wrong thing (curls a route that +doesn't exist, greps a directory that isn't there). + +The rule that prevents this: + +- **Runnable code blocks use ``s that fail loudly if pasted as-is.** Prefer + `curl "$HOST"` over `curl "$HOST/authors"` — a literal paste of the placeholder + form breaks visibly, while a literal paste of `/authors` hits the wrong route silently. Put + the concrete value in an adjacent comment as a clearly-labeled `e.g.` +- **Concrete examples live in prose or comments, marked `e.g.`** There they aid comprehension + without being executable, so literal-matching them is harmless. +- **Literal values are fine only when they are genuinely universal** — e.g. probing the real, + fixed served-spec paths (`/openapi.json`, `/v3/api-docs`) is correct precisely because those + are the same across every app of that framework. If a value changes per codebase, it is a + placeholder, not a literal. +- **Command tables (grep/derivation recipes) are illustrative starting points.** Say so once + near the table, and point them at the repo's real source root and filenames rather than a + literal `src/` or `openapi.yaml`. + +The test: *"if an agent runs this exact text against a completely different repo, does it fail +loudly or mislead silently?"* Silent-mislead means parameterize it. + --- ## Creating a New Plugin diff --git a/cursor/.cursor/rules/stackhawk-hawkscan-openapi-specs.mdc b/cursor/.cursor/rules/stackhawk-hawkscan-openapi-specs.mdc index 86c64464..c6325453 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan-openapi-specs.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan-openapi-specs.mdc @@ -143,14 +143,17 @@ source. This is the check that catches base-path mismatch, stale specs, and wron 1. **Count sanity.** Spec operation count should be in the same ballpark as the route grep from discovery. A spec with far fewer/more paths than the code has is suspect. -2. **Resolve check.** Take 2–3 spec paths and request them **as the scanner will build them** - — effective `host` + operation path — against the running app: +2. **Resolve check.** Take 2–3 real operation paths **from the spec you're wiring** and + request each **as the scanner will build it** — effective `host` + operation path — against + the running app, both as the spec declares it and with the app's detected base/context path + prepended. Substitute your own values for the placeholders; do not run them literally: ```bash - # effective base the scan uses (app.host, incl. any path component) - curl -s -o /dev/null -w "%{http_code} %{url_effective}\n" "$SCAN_HOST/authors" - curl -s -o /dev/null -w "%{http_code} %{url_effective}\n" "$SCAN_HOST/api/v1/authors" + # = an operation path copied from the spec (e.g. /authors) + # = the app's detected base/context path, or empty if none (e.g. /api/v1) + curl -s -o /dev/null -w "%{http_code} %{url_effective}\n" "$SCAN_HOST" + curl -s -o /dev/null -w "%{http_code} %{url_effective}\n" "$SCAN_HOST" ``` - If the spec-derived URL 404s while a base/context-path-prefixed variant returns a real + If the spec-derived URL 404s while the base/context-path-prefixed variant returns a real status (200/401/403), the base path is wrong — fix `app.host` or the spec before scanning. A raw container 404 (e.g. Tomcat's default page) rather than an application response is a strong tell that the request never reached the app's routing. diff --git a/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc b/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc index 9bcfcee4..f5050f37 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc @@ -61,7 +61,10 @@ For anything the docs didn't answer, find the surfaces the way a developer would manifests, entry points, and route definitions. Identify **every** distinct API surface in the repo — a service can have more than one (e.g. a REST API plus an internal gRPC service). For each surface found, record its type and derive a route inventory using the matching -technique: +technique. The commands below are **illustrative starting points**, not fixed incantations: +point them at the repo's real source root and its actual spec/schema filenames — don't run +`src/`, `openapi.yaml`, or `schema.graphql` literally — and adjust the pattern to what the +code actually uses. | Surface | Detection | Route inventory / derivation | |---------|-----------|-------------------------------| diff --git a/plugins/hawkscan/skills/hawkscan/references/openapi-specs.md b/plugins/hawkscan/skills/hawkscan/references/openapi-specs.md index 8a2fa070..310d66ef 100644 --- a/plugins/hawkscan/skills/hawkscan/references/openapi-specs.md +++ b/plugins/hawkscan/skills/hawkscan/references/openapi-specs.md @@ -137,14 +137,17 @@ source. This is the check that catches base-path mismatch, stale specs, and wron 1. **Count sanity.** Spec operation count should be in the same ballpark as the route grep from discovery. A spec with far fewer/more paths than the code has is suspect. -2. **Resolve check.** Take 2–3 spec paths and request them **as the scanner will build them** - — effective `host` + operation path — against the running app: +2. **Resolve check.** Take 2–3 real operation paths **from the spec you're wiring** and + request each **as the scanner will build it** — effective `host` + operation path — against + the running app, both as the spec declares it and with the app's detected base/context path + prepended. Substitute your own values for the placeholders; do not run them literally: ```bash - # effective base the scan uses (app.host, incl. any path component) - curl -s -o /dev/null -w "%{http_code} %{url_effective}\n" "$SCAN_HOST/authors" - curl -s -o /dev/null -w "%{http_code} %{url_effective}\n" "$SCAN_HOST/api/v1/authors" + # = an operation path copied from the spec (e.g. /authors) + # = the app's detected base/context path, or empty if none (e.g. /api/v1) + curl -s -o /dev/null -w "%{http_code} %{url_effective}\n" "$SCAN_HOST" + curl -s -o /dev/null -w "%{http_code} %{url_effective}\n" "$SCAN_HOST" ``` - If the spec-derived URL 404s while a base/context-path-prefixed variant returns a real + If the spec-derived URL 404s while the base/context-path-prefixed variant returns a real status (200/401/403), the base path is wrong — fix `app.host` or the spec before scanning. A raw container 404 (e.g. Tomcat's default page) rather than an application response is a strong tell that the request never reached the app's routing. diff --git a/plugins/hawkscan/skills/hawkscan/references/scan-planning.md b/plugins/hawkscan/skills/hawkscan/references/scan-planning.md index 8a83dccf..d4d84c5a 100644 --- a/plugins/hawkscan/skills/hawkscan/references/scan-planning.md +++ b/plugins/hawkscan/skills/hawkscan/references/scan-planning.md @@ -55,7 +55,10 @@ For anything the docs didn't answer, find the surfaces the way a developer would manifests, entry points, and route definitions. Identify **every** distinct API surface in the repo — a service can have more than one (e.g. a REST API plus an internal gRPC service). For each surface found, record its type and derive a route inventory using the matching -technique: +technique. The commands below are **illustrative starting points**, not fixed incantations: +point them at the repo's real source root and its actual spec/schema filenames — don't run +`src/`, `openapi.yaml`, or `schema.graphql` literally — and adjust the pattern to what the +code actually uses. | Surface | Detection | Route inventory / derivation | |---------|-----------|-------------------------------| From 7d8495c4ff7bc7b72c3f1b4384abfef47ef3602d Mon Sep 17 00:00:00 2001 From: KC Berg Date: Mon, 13 Jul 2026 16:05:46 -0600 Subject: [PATCH 18/29] =?UTF-8?q?fix(hawkscan):=20make=20discovery=20expli?= =?UTF-8?q?citly=20read-only=20=E2=80=94=20derive=20host/port=20by=20readi?= =?UTF-8?q?ng,=20not=20by=20starting=20the=20app?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Skill-eval discovery cells (firefly-iii, memos) fail a BLOCKING read-only guard on every model — the agent stands up the app/container during discovery (docker compose up, php artisan serve) even when told not to, to learn host/port. A three-way eval comparison (HEAD vs this session's base af3f3d1 vs old pre-rewrite main) shows the behavior is identical across all three: not caused by recent edits, long-standing skill-wide. Fix: state in SKILL.md Step 1a and scan-planning.md that discovery is read-only and static — host, port, and run command come from reading docker-compose.yml port mappings, .env/.env.example, and the README; starting the app is a scan-time step (Step 1c), never part of discovery. This is also better real-world behavior: no need to spin up Docker to read a port that's declared in config. ENG-704. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_012sAkR2BzkbLxzYUuGzLAQa --- cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc | 5 ++++- cursor/.cursor/rules/stackhawk-hawkscan.mdc | 6 ++++++ plugins/hawkscan/skills/hawkscan/SKILL.md | 6 ++++++ .../hawkscan/skills/hawkscan/references/scan-planning.md | 5 ++++- 4 files changed, 20 insertions(+), 2 deletions(-) diff --git a/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc b/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc index f5050f37..1e37c612 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan-scan-planning.mdc @@ -53,7 +53,10 @@ states: | `docs/` setup / quickstart / architecture pages | deeper API and service-boundary detail | Harvest: run command + host/port, which API surfaces exist and where their source lives -(useful in a monorepo), and any documented spec location or dev/test credential. +(useful in a monorepo), and any documented spec location or dev/test credential. Read these +from the docs and config — discovery never starts the app to learn them. A +`docker-compose.yml` port mapping or an `.env`/`.env.example` `APP_URL` gives you the host +and port without running anything; starting the app is a scan-time step, not a discovery one. ### Pass 2 — explore code per surface type diff --git a/cursor/.cursor/rules/stackhawk-hawkscan.mdc b/cursor/.cursor/rules/stackhawk-hawkscan.mdc index 76936eaf..7b05b85f 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan.mdc @@ -136,6 +136,12 @@ command, host/port, and auth shape needed to reach each one. Prefer the repo's o over guessing, explore code to fill gaps, and ask the user directly for whatever remains unresolved — never stall or invent. +**Discovery is read-only and static.** Determine the run command, host, and port by +*reading* config and docs — `docker-compose.yml` port mappings, `.env`/`.env.example` +(e.g. `APP_URL`), the README — not by launching anything. You never need to start the app, +run a container, or bring up a server to discover *what* to scan; standing up the target is +a scan-time step (Step 1c), not part of discovery. + → Full discovery workflow — per-surface detection, route-inventory derivation, gap recommendations, and the user-confirmed summary required before the first scan: [`references/scan-planning.md`](references/scan-planning.md) diff --git a/plugins/hawkscan/skills/hawkscan/SKILL.md b/plugins/hawkscan/skills/hawkscan/SKILL.md index 6ec0e533..afd590d3 100644 --- a/plugins/hawkscan/skills/hawkscan/SKILL.md +++ b/plugins/hawkscan/skills/hawkscan/SKILL.md @@ -147,6 +147,12 @@ command, host/port, and auth shape needed to reach each one. Prefer the repo's o over guessing, explore code to fill gaps, and ask the user directly for whatever remains unresolved — never stall or invent. +**Discovery is read-only and static.** Determine the run command, host, and port by +*reading* config and docs — `docker-compose.yml` port mappings, `.env`/`.env.example` +(e.g. `APP_URL`), the README — not by launching anything. You never need to start the app, +run a container, or bring up a server to discover *what* to scan; standing up the target is +a scan-time step (Step 1c), not part of discovery. + → Full discovery workflow — per-surface detection, route-inventory derivation, gap recommendations, and the user-confirmed summary required before the first scan: [`references/scan-planning.md`](references/scan-planning.md) diff --git a/plugins/hawkscan/skills/hawkscan/references/scan-planning.md b/plugins/hawkscan/skills/hawkscan/references/scan-planning.md index d4d84c5a..4579cd46 100644 --- a/plugins/hawkscan/skills/hawkscan/references/scan-planning.md +++ b/plugins/hawkscan/skills/hawkscan/references/scan-planning.md @@ -47,7 +47,10 @@ states: | `docs/` setup / quickstart / architecture pages | deeper API and service-boundary detail | Harvest: run command + host/port, which API surfaces exist and where their source lives -(useful in a monorepo), and any documented spec location or dev/test credential. +(useful in a monorepo), and any documented spec location or dev/test credential. Read these +from the docs and config — discovery never starts the app to learn them. A +`docker-compose.yml` port mapping or an `.env`/`.env.example` `APP_URL` gives you the host +and port without running anything; starting the app is a scan-time step, not a discovery one. ### Pass 2 — explore code per surface type From 56d2cb91d547321419f63863dc3850cab88f8496 Mon Sep 17 00:00:00 2001 From: KC Berg Date: Mon, 13 Jul 2026 16:34:02 -0600 Subject: [PATCH 19/29] fix(evals): narrow discovery guard to real container-start verbs (ENG-708) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The read-only discovery guard denied any command containing the bare word `docker` (also podman/nerdctl), which false-failed read-only inspection the agent legitimately uses to read port mappings during discovery — `docker ps`, `docker images`, `docker compose config`, `docker inspect`. For a Docker-packaged app (firefly-iii) that single denial flips the BLOCKING discovery_stayed_read_only check and fails an otherwise-correct, fully read-only discovery. Narrow to commands that actually start a container/app (docker run/start/create, docker compose up/run, docker-compose up, podman/nerdctl run/start, podman-compose up); allow read-only inspection. Adds test_denies_container_start + test_allows_container_readonly_inspection. ENG-708. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_012sAkR2BzkbLxzYUuGzLAQa --- evals/harnesses/claude-code/discovery_guard.py | 9 ++++++++- tests/lib/test_discovery_guard.py | 14 ++++++++++++++ 2 files changed, 22 insertions(+), 1 deletion(-) diff --git a/evals/harnesses/claude-code/discovery_guard.py b/evals/harnesses/claude-code/discovery_guard.py index 42b23ed7..28ab96c3 100644 --- a/evals/harnesses/claude-code/discovery_guard.py +++ b/evals/harnesses/claude-code/discovery_guard.py @@ -24,7 +24,14 @@ def main(): deny("hawk scan is out of scope for discovery") if re.search(r"\bgit\s+(commit|push)\b", low): deny("git commit/push not allowed") - if re.search(r"\b(docker|docker-compose|podman|nerdctl)\b", low) \ + # Only deny commands that actually START a container/app — not read-only + # inspection (`docker ps`, `docker images`, `docker compose config`, `docker + # inspect`), which the agent legitimately uses to read port mappings and + # config during discovery. Matching the bare word `docker` denied those too. + if re.search(r"\bdocker\s+(run|start|create|compose\s+up|compose\s+run)\b", low) \ + or re.search(r"\bdocker-compose\s+(up|run|start)\b", low) \ + or re.search(r"\b(podman|nerdctl)\s+(run|start|create)\b", low) \ + or re.search(r"\bpodman-compose\s+(up|run|start)\b", low) \ or re.search(r"\b(npm|pnpm|yarn)\s+(start|run\s+dev|run\s+serve|serve)\b", low) \ or re.search(r"\b(bootrun|runserver|uvicorn|gunicorn|hypercorn|nodemon)\b", low) \ or re.search(r"spring-boot:run|flask\s+run|mix\s+phx\.server|air\b|./gradlew\s+bootrun", low) \ diff --git a/tests/lib/test_discovery_guard.py b/tests/lib/test_discovery_guard.py index d8afd70e..8ed24a2d 100644 --- a/tests/lib/test_discovery_guard.py +++ b/tests/lib/test_discovery_guard.py @@ -194,6 +194,20 @@ def test_denies_go_and_cargo_run(self): event = {"tool_name": "Bash", "tool_input": {"command": c}} self.assertEqual(run_guard(event).returncode, 2, c) + def test_denies_container_start(self): + for c in ("docker compose up -d", "docker-compose up", "docker run -p 8080:8080 img", + "podman run img", "nerdctl start c", "podman-compose up"): + event = {"tool_name": "Bash", "tool_input": {"command": c}} + self.assertEqual(run_guard(event).returncode, 2, c) + + def test_allows_container_readonly_inspection(self): + # Reading config/port mappings during discovery is read-only — must NOT be + # denied just because the word "docker" appears (regression: ENG-708). + for c in ("docker ps", "docker images", "docker compose config", + "docker compose config --services", "docker inspect firefly", "docker compose ls"): + event = {"tool_name": "Bash", "tool_input": {"command": c}} + self.assertEqual(run_guard(event).returncode, 0, c) + # --- extended mutation vectors --- def test_denies_sed_in_place(self): event = {"tool_name": "Bash", "tool_input": {"command": "sed -i 's/a/b/' f.txt"}} From 862dce98fe12b1f442460271ca6f35314f164704 Mon Sep 17 00:00:00 2001 From: KC Berg Date: Mon, 13 Jul 2026 16:55:01 -0600 Subject: [PATCH 20/29] test(evals): add scan-quality gate reasoning cells (hw-21 base-path-mismatch, hw-22 auth-wall) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The post-scan quality gate had no eval coverage — it needs a completed scan, which CI can't run. Cover the gate's *reasoning* instead: two cells inline a synthetic completed-scan result and ask the agent to apply the gate. No harness change needed — uses applies_to-scoped process-checks (the same mechanism the discovery cells use). - hw-21: spec paths all 404 while /api/v1-prefixed variants resolve → agent must name the base-path-mismatch signal. - hw-22: auth-wall flag + 22/254 coverage on a reachable app → agent must name auth-wall (config-class) and report the coverage evidence. - Shared: must not declare the scan "done and secure" while a gap is open. Checks are scoped via applies_to so they don't touch the other 24 cells. Bumped the hardcoded prompt count in test_cli_resilience (24→26). ENG-704. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_012sAkR2BzkbLxzYUuGzLAQa --- evals/hawkscan/process-checks.json | 40 ++++++++++++++++++++++++++++++ evals/hawkscan/prompts.yaml | 29 ++++++++++++++++++++++ tests/lib/test_cli_resilience.py | 2 +- 3 files changed, 70 insertions(+), 1 deletion(-) diff --git a/evals/hawkscan/process-checks.json b/evals/hawkscan/process-checks.json index 80a12a34..6d6fcd59 100644 --- a/evals/hawkscan/process-checks.json +++ b/evals/hawkscan/process-checks.json @@ -348,6 +348,46 @@ "applies_to": ["firefly-iii", "wikijs", "memos", "dawarich"], "severity": "blocking", "failure_guidance": "Discovery is observational; any guard denial means the agent acted outside scope." + }, + { + "id": "gate_identifies_base_path_mismatch", + "phase": "step_4_5", + "description": "For the base-path-mismatch scenario, the agent names the base/context-path resolution gap (the gate's base-path-mismatch signal) as the cause of the systematic 404s.", + "type": "output_contains", + "applies_to": ["hw-21"], + "signals": ["base-path-mismatch", "base path", "base-path", "context path", "context-path"], + "severity": "blocking", + "failure_guidance": "A spec whose paths all 404 while a prefixed variant resolves is a base-path/context-path mismatch, not a missing spec or an environment problem. Naming it correctly is what routes to the right fix (align app.host and the spec base path)." + }, + { + "id": "gate_identifies_auth_wall", + "phase": "step_4_5", + "description": "For the auth-wall scenario, the agent names the auth-wall signal (authenticated paths returning a wall of 401/403) rather than misreading it as an environment problem or coverage-only.", + "type": "output_contains", + "applies_to": ["hw-22"], + "signals": ["auth-wall", "auth wall", "authentication wall"], + "severity": "blocking", + "failure_guidance": "A cluster of 401/403 across authenticated paths on a reachable app is a config-class auth-wall, fixed by correcting the authentication block — not an environment-class gap." + }, + { + "id": "gate_coverage_evidence_reported", + "phase": "step_4_5", + "description": "For the auth-wall scenario, the agent reports the coverage evidence (routes reached vs. expected) rather than ignoring the under-coverage.", + "type": "output_contains", + "applies_to": ["hw-22"], + "signals": ["22", "254", "coverage", "reached"], + "severity": "warning", + "failure_guidance": "Coverage is evidence-only but must still be reported (e.g. 22 of 254 routes reached) so the gap is specific." + }, + { + "id": "gate_no_false_done_secure", + "phase": "step_4_5", + "description": "The agent does NOT declare the scan done or the app secure while a gate gap is open.", + "type": "output_negative", + "applies_to": ["hw-21", "hw-22"], + "anti_patterns": ["done and secure", "complete and secure", "scan is complete and the app is secure"], + "severity": "warning", + "failure_guidance": "The gate reporting rule forbids claiming the scan is done or the app secure while a config- or environment-class gap remains open." } ] } diff --git a/evals/hawkscan/prompts.yaml b/evals/hawkscan/prompts.yaml index 5e13f022..1c950ea5 100644 --- a/evals/hawkscan/prompts.yaml +++ b/evals/hawkscan/prompts.yaml @@ -105,6 +105,35 @@ invocation_type: negative prompt: What types of vulnerabilities does HawkScan detect? notes: Research question; no scan intent, no code change +- id: hw-21 + should_trigger: true + invocation_type: explicit + prompt: | + Using the hawkscan skill's post-scan quality gate (Step 4.5 / scan-quality.md), evaluate a scan that just completed. You do NOT have live API access in this exercise — reason over the data provided below; do not try to run `hawk op` commands. + + Scanned config (stackhawk.yml): + app: + applicationId: ${APP_ID} + env: Development + host: http://localhost:8080 + openApiConf: + path: /v3/api-docs + The wired spec declares 60 operations under paths like `/authors`, `/books`, `/orders`. + The completed scan's scanned-URI list shows 60 requests and EVERY one returned 404. A manual check shows the app actually serves those routes under `/api/v1` — `/api/v1/authors` returns 200/401 while bare `/authors` returns 404. + + Apply the quality gate: name the gate signal, say whether it is config-class or environment-class, and give the single correct fix. Do not declare the scan done or the app secure. + notes: Gate reasoning — base-path mismatch (spec paths 404 while prefixed variants resolve) +- id: hw-22 + should_trigger: true + invocation_type: explicit + prompt: | + Using the hawkscan skill's post-scan quality gate (Step 4.5 / scan-quality.md), evaluate a completed scan. You do NOT have live API access in this exercise — reason over the data below; do not try to run `hawk op` commands. + + The scanned config has an `authentication:` block configured (OAuth2 bearer), and the app is confirmed up and reachable. + The scan's metrics flag an auth-wall — a large cluster of 401/403 responses across the authenticated paths. The scanned-URI list shows only 22 of an expected 254 routes were reached. + + Apply the quality gate: name the primary gate signal, say whether it is config-class or environment-class, and give the correct next step, noting the coverage evidence. Do not weaken the scan (e.g. widening excludePaths or raising failureThreshold) to make it pass, and do not claim the app is done or secure. + notes: Gate reasoning — auth-wall (config-class) + coverage-gap evidence - id: firefly-iii should_trigger: true invocation_type: implicit diff --git a/tests/lib/test_cli_resilience.py b/tests/lib/test_cli_resilience.py index 406b3282..9d6fefda 100644 --- a/tests/lib/test_cli_resilience.py +++ b/tests/lib/test_cli_resilience.py @@ -36,7 +36,7 @@ def test_main_survives_launch_crash(monkeypatch, tmp_path): assert (out / "cell.json").exists() assert (out / "summary.json").exists() cell = json.loads((out / "cell.json").read_text()) - assert len(cell["results"]) == 24 # all hawkscan prompts graded + assert len(cell["results"]) == 26 # all hawkscan prompts graded # positive prompts failed with a harness note; at least one note mentions the crash assert any("command not found" in r.get("note", "") for r in cell["results"]) From 08855168f3b41390c17ce1f15d894b7f0d8f4275 Mon Sep 17 00:00:00 2001 From: KC Berg Date: Mon, 13 Jul 2026 17:13:45 -0600 Subject: [PATCH 21/29] fix(evals): scope gate reasoning cells to their own checks; drop fragile no-secure check MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit hw-21/hw-22 are reasoning-only (no live scan), but grade() applied the global scan-flow checks (hawk version, hawk op app list, scan --json-output...) which a paper exercise never performs — failing the cells flakily by model. Mirror the discovery-cell precedent: add an own_checks_only cell flag routing these through grade_discovery (own applies_to checks + expected only), same as answer_key cells. Also drop gate_no_false_done_secure: output_negative substring-matched the CORRECT behavior — the agent quoting the rule ("never claim \"done and secure\"") tripped it. Telling a genuine false claim from a quoted rule needs the semantic rubric grader (not run in CI), so the check produced false failures. Keep the robust positive checks (names base-path-mismatch / auth-wall / coverage evidence). ENG-704. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_012sAkR2BzkbLxzYUuGzLAQa --- evals/cli.py | 7 +++++++ evals/hawkscan/process-checks.json | 10 ---------- evals/hawkscan/prompts.yaml | 2 ++ evals/lib/models.py | 1 + 4 files changed, 10 insertions(+), 10 deletions(-) diff --git a/evals/cli.py b/evals/cli.py index e9a12754..aa9e7a5a 100644 --- a/evals/cli.py +++ b/evals/cli.py @@ -70,6 +70,13 @@ def main() -> None: run.error = run.error or f"judge failed: {type(e).__name__}: {e}" res = grade_discovery(p, run, cfg.checks, judge_checks, platform=args.harness, skill=args.skill) + elif p.own_checks_only: + # Reasoning-only cell (e.g. post-scan gate): grade on its own + # applies_to checks + expected, not the global scan-flow checks + # (preflight/step1/scan) that a paper exercise never performs. + from evals.lib.grading import grade_discovery + res = grade_discovery(p, run, cfg.checks, [], + platform=args.harness, skill=args.skill) else: did = adapter.detect_trigger(run, args.skill) res = grade(p, run, cfg.checks, platform=args.harness, skill=args.skill, diff --git a/evals/hawkscan/process-checks.json b/evals/hawkscan/process-checks.json index 6d6fcd59..bf6a6f47 100644 --- a/evals/hawkscan/process-checks.json +++ b/evals/hawkscan/process-checks.json @@ -378,16 +378,6 @@ "signals": ["22", "254", "coverage", "reached"], "severity": "warning", "failure_guidance": "Coverage is evidence-only but must still be reported (e.g. 22 of 254 routes reached) so the gap is specific." - }, - { - "id": "gate_no_false_done_secure", - "phase": "step_4_5", - "description": "The agent does NOT declare the scan done or the app secure while a gate gap is open.", - "type": "output_negative", - "applies_to": ["hw-21", "hw-22"], - "anti_patterns": ["done and secure", "complete and secure", "scan is complete and the app is secure"], - "severity": "warning", - "failure_guidance": "The gate reporting rule forbids claiming the scan is done or the app secure while a config- or environment-class gap remains open." } ] } diff --git a/evals/hawkscan/prompts.yaml b/evals/hawkscan/prompts.yaml index 1c950ea5..9c5dbb7d 100644 --- a/evals/hawkscan/prompts.yaml +++ b/evals/hawkscan/prompts.yaml @@ -123,6 +123,7 @@ Apply the quality gate: name the gate signal, say whether it is config-class or environment-class, and give the single correct fix. Do not declare the scan done or the app secure. notes: Gate reasoning — base-path mismatch (spec paths 404 while prefixed variants resolve) + own_checks_only: true - id: hw-22 should_trigger: true invocation_type: explicit @@ -134,6 +135,7 @@ Apply the quality gate: name the primary gate signal, say whether it is config-class or environment-class, and give the correct next step, noting the coverage evidence. Do not weaken the scan (e.g. widening excludePaths or raising failureThreshold) to make it pass, and do not claim the app is done or secure. notes: Gate reasoning — auth-wall (config-class) + coverage-gap evidence + own_checks_only: true - id: firefly-iii should_trigger: true invocation_type: implicit diff --git a/evals/lib/models.py b/evals/lib/models.py index c94a0a8b..fc83f836 100644 --- a/evals/lib/models.py +++ b/evals/lib/models.py @@ -47,6 +47,7 @@ class PromptConfig(BaseModel): expected: list[ExpectedCheck] = [] target_repo: TargetRepo | None = None # clone into the cell cwd (read-only run) answer_key: str | None = None # path, relative to the suite dir + own_checks_only: bool = False # grade only this cell's applies_to checks (reasoning cells) — skip global scan-flow checks class Verdict(str, Enum): From d71144aa3572ef47d1937b2b6381bddb043fdbad Mon Sep 17 00:00:00 2001 From: KC Berg Date: Tue, 14 Jul 2026 13:11:26 -0600 Subject: [PATCH 22/29] =?UTF-8?q?test(evals):=20firefly-iii=20answer=20key?= =?UTF-8?q?=20=E2=80=94=20in-repo=20run=20path=20is=20the=20correct=20disc?= =?UTF-8?q?overy=20answer?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per the devkit/eval author's guidance: evals exist to catch regressions, not to pin one phrasing, and the scan-planning discovery drives a slightly different (more repo-accurate) answer. The firefly-iii key led run_command/host with docker-compose that isn't in the cloned repo — so an agent that correctly reports the in-repo path (`php artisan serve` on :8000) and notes Docker packaging is a separate image was judged "partial" against external knowledge. Reframe so the in-repo local-dev path OR the external Docker image both count as correct. Behavior-neutral for the skill; adjusts eval ground truth to the accurate target. ENG-704. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_012sAkR2BzkbLxzYUuGzLAQa --- evals/hawkscan/answer-keys/firefly-iii.json | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/evals/hawkscan/answer-keys/firefly-iii.json b/evals/hawkscan/answer-keys/firefly-iii.json index 74fb8911..42b62413 100644 --- a/evals/hawkscan/answer-keys/firefly-iii.json +++ b/evals/hawkscan/answer-keys/firefly-iii.json @@ -7,14 +7,14 @@ "evidence": "composer.json: \"php\": \">=8.5\", \"laravel/framework\": \"^13\"." }, "run_command": { - "value": "Official Docker image `fireflyiii/core` via docker compose against a database (MySQL/MariaDB/PostgreSQL/SQLite), served on container port 8080; or local dev with `php artisan serve` (Laravel dev server on 127.0.0.1:8000). A database is required.", + "value": "In-repo local dev: `composer install`, generate app + Passport keys, run migrations (a database is required — MySQL/MariaDB/PostgreSQL/SQLite), then `php artisan serve` (Laravel dev server on 127.0.0.1:8000). Production packaging ships as the separate `fireflyiii/core` Docker image (container port 8080); there is no docker-compose in this repo. Either identifying the in-repo `artisan serve` path OR the external Docker image is a correct answer.", "must_hit": false, - "evidence": "docker-compose examples; php artisan serve" + "evidence": "No docker-compose/Dockerfile in the repo; `php artisan serve` for local dev; `fireflyiii/core` image is built/published from separate packaging." }, "host": { - "value": "http://localhost:8080 (official Docker image) or http://127.0.0.1:8000 (php artisan serve). APP_URL defaults to http://localhost.", + "value": "http://127.0.0.1:8000 (`php artisan serve`, the in-repo local run) OR http://localhost:8080 (the separate `fireflyiii/core` Docker image). APP_URL defaults to http://localhost. Any of these is correct.", "must_hit": true, - "evidence": ".env.example APP_URL=http://localhost (line 301)." + "evidence": ".env.example APP_URL=http://localhost; Laravel dev server default 127.0.0.1:8000; fireflyiii/core image exposes 8080." }, "api_style": { "value": "REST. Base path /api/v1/. The OpenAPI/Swagger spec ships in the separate firefly-iii/api-docs repo, not served from this app.", From 79a0dea248135b7c2cea7dd05a0ba99a9ce839d1 Mon Sep 17 00:00:00 2001 From: KC Berg Date: Wed, 15 Jul 2026 10:36:30 -0600 Subject: [PATCH 23/29] feat(hawkscan): StackHawk-branded progress narration across the loop MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Users reported the autonomous loop runs too quietly. Add a canonical "Progress output" convention to SKILL.md: prefix every status line with `StackHawk | ` (ASCII only, no emoji — renders in Cursor, Windows cmd, PowerShell), emit one line entering each phase (discovery, config, scan, quality gate, findings, each fix batch, rescan, report) with the key number, and before a scan/rescan set a duration expectation + let hawk's native progress stream through (the honest "live progress" — the agent can't interject mid-synchronous scan). Status output, never a prompt. Propagate branded lines through autonomous-loop.md's phases (scan start/complete, gate result, findings, rescan) which were previously silent or plain. Greppable prefix so CI/users can filter loop status. ENG-704. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_012sAkR2BzkbLxzYUuGzLAQa --- cursor/.cursor/rules/stackhawk-hawkscan.mdc | 19 ++++++++++++-- plugins/hawkscan/skills/hawkscan/SKILL.md | 19 ++++++++++++-- .../hawkscan/references/autonomous-loop.md | 25 ++++++++++++------- 3 files changed, 50 insertions(+), 13 deletions(-) diff --git a/cursor/.cursor/rules/stackhawk-hawkscan.mdc b/cursor/.cursor/rules/stackhawk-hawkscan.mdc index 7b05b85f..7ea262ca 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan.mdc @@ -88,6 +88,20 @@ names deliberately. Findings have a lifecycle (NEW, FALSE_POSITIVE, RISK_ACCEPTE --- +## Progress output + +Keep the terminal informed so a running loop never looks silent. Prefix every status line with +`StackHawk | ` — **ASCII only, no emoji** (must render in Cursor, Windows `cmd`, and PowerShell; +keep the branded lines themselves plain ASCII). Emit one short line **entering each phase** — +discovery, config, scan, quality gate, findings, each fix batch, rescan, report — with what's +happening plus the key number (routes, coverage, findings). Before a scan/rescan, note it takes +a few minutes and let hawk's own progress stream through (don't suppress it); announce +completion after. Status output, **not** a prompt — never pause for input. E.g. +`StackHawk | Scanning http://localhost:8080 - a few minutes; progress streams below`, then +`StackHawk | 3 findings (2 High, 1 Medium) - fixing all`. Per-phase lines: `references/autonomous-loop.md`. + +--- + ## Phase 0: App Setup & Verification Run Phase 0 **once** when onboarding a new application (`stackhawk.yml` being created for @@ -171,7 +185,8 @@ strategy, frontend-vs-backend scenarios, and config templates: ```bash hawk create app --name "" --env ``` - Resolve `` with the Env Name Algorithm above. Announce the created app ID and URL. + Resolve `` with the Env Name Algorithm above. Announce it (Progress output): + `StackHawk | Created app () - `. 6. **Env exists?** Determine env name via Env Name Algorithm. Run `hawk op env list --app --format json`. Reuse if exists; otherwise run `hawk op env create --app --env --host `. @@ -466,7 +481,7 @@ initialized; skill is active. **Guard rails summary:** - One scan at a time — never run `hawk scan` or `hawk rescan` with `&` or `nohup` - Max one fix-rescan cycle per task — if findings remain after fixing, report them -- Always announce: "Running security scan...", "Found N vulnerabilities, fixing...", "Rescanning..." +- Always narrate with the `StackHawk | ` prefix at every phase (see **Progress output**) — never let the loop run silent - Interruptible — stop immediately if the user says to - Report gate gaps — never silently accept exit 0 from a thin scan - Never claim "done and secure" while gate gaps are open diff --git a/plugins/hawkscan/skills/hawkscan/SKILL.md b/plugins/hawkscan/skills/hawkscan/SKILL.md index afd590d3..af4820d3 100644 --- a/plugins/hawkscan/skills/hawkscan/SKILL.md +++ b/plugins/hawkscan/skills/hawkscan/SKILL.md @@ -99,6 +99,20 @@ names deliberately. Findings have a lifecycle (NEW, FALSE_POSITIVE, RISK_ACCEPTE --- +## Progress output + +Keep the terminal informed so a running loop never looks silent. Prefix every status line with +`StackHawk | ` — **ASCII only, no emoji** (must render in Cursor, Windows `cmd`, and PowerShell; +keep the branded lines themselves plain ASCII). Emit one short line **entering each phase** — +discovery, config, scan, quality gate, findings, each fix batch, rescan, report — with what's +happening plus the key number (routes, coverage, findings). Before a scan/rescan, note it takes +a few minutes and let hawk's own progress stream through (don't suppress it); announce +completion after. Status output, **not** a prompt — never pause for input. E.g. +`StackHawk | Scanning http://localhost:8080 - a few minutes; progress streams below`, then +`StackHawk | 3 findings (2 High, 1 Medium) - fixing all`. Per-phase lines: `references/autonomous-loop.md`. + +--- + ## Phase 0: App Setup & Verification Run Phase 0 **once** when onboarding a new application (`stackhawk.yml` being created for @@ -182,7 +196,8 @@ strategy, frontend-vs-backend scenarios, and config templates: ```bash hawk create app --name "" --env ``` - Resolve `` with the Env Name Algorithm above. Announce the created app ID and URL. + Resolve `` with the Env Name Algorithm above. Announce it (Progress output): + `StackHawk | Created app () - `. 6. **Env exists?** Determine env name via Env Name Algorithm. Run `hawk op env list --app --format json`. Reuse if exists; otherwise run `hawk op env create --app --env --host `. @@ -477,7 +492,7 @@ initialized; skill is active. **Guard rails summary:** - One scan at a time — never run `hawk scan` or `hawk rescan` with `&` or `nohup` - Max one fix-rescan cycle per task — if findings remain after fixing, report them -- Always announce: "Running security scan...", "Found N vulnerabilities, fixing...", "Rescanning..." +- Always narrate with the `StackHawk | ` prefix at every phase (see **Progress output**) — never let the loop run silent - Interruptible — stop immediately if the user says to - Report gate gaps — never silently accept exit 0 from a thin scan - Never claim "done and secure" while gate gaps are open diff --git a/plugins/hawkscan/skills/hawkscan/references/autonomous-loop.md b/plugins/hawkscan/skills/hawkscan/references/autonomous-loop.md index 6986b87e..c5e9a827 100644 --- a/plugins/hawkscan/skills/hawkscan/references/autonomous-loop.md +++ b/plugins/hawkscan/skills/hawkscan/references/autonomous-loop.md @@ -8,11 +8,13 @@ ## The Loop steps -After completing a code change, announce and execute these steps in order. +After completing a code change, announce and execute these steps in order. Narrate every +phase with the `StackHawk | ` prefix (SKILL.md **Progress output**) so the loop is never +silent — ASCII only, no emoji. **1. Announce** -Say: "Implementation complete. Running security scan against the application." +Say: `StackHawk | Implementation complete - running a security scan against the app` **2. Configure** @@ -42,11 +44,14 @@ timeout 30 hawk validate config stackhawk.yml || echo "Validate timed out — en **4. Scan** +Announce before launching: `StackHawk | Scanning - this takes a few minutes; scan progress streams below`. Run it synchronously and let hawk's own progress print (don't suppress it): + ```bash hawk scan --json-output ``` -Capture `scan.id` from the JSON output — you'll need it for rescan. +Capture `scan.id` from the JSON output — you'll need it for rescan. Then announce completion: +`StackHawk | Scan complete - URLs scanned (scan )`. **5. Quality gate** @@ -61,7 +66,8 @@ timeouts, app unreachable mid-scan), never edit the config: verify the app is up scan once for free, and if the same environment-class signal recurs on the retry, report it as an environment problem and stop — do not retry again or reach for a config edit. At the cap, or once the gate is clean, proceed to the next step regardless — findings are always -reported and fixable independent of gate state. +reported and fixable independent of gate state. Announce the gate result: +`StackHawk | Quality gate: coverage /, auth , `. **6. If findings exist** @@ -70,7 +76,7 @@ Run the Step 5 triage filter first (per-path `status` field): - Prioritize `ASSIGNED` paths before `NEW` of the same severity - Fix `NEW` paths in severity order -Announce: "Found [N] actionable vulnerabilities (+ [M] skipped due to prior triage). Fixing all actionable ones." +Announce: `StackHawk | [N] actionable findings (+ [M] skipped via prior triage) - fixing all` **Fix ALL findings — not just ones related to your recent changes.** DAST scans the entire running application. Pre-existing vulnerabilities are just as exploitable as new ones. @@ -80,7 +86,8 @@ Commit format: `fix: resolve [CWE-XXX] [vulnerability type] found by HawkScan` **7. Rescan** -Rescan is the default for all fix-verify cycles: +Rescan is the default for all fix-verify cycles. Announce before launching: +`StackHawk | Rescanning to verify fixes - a few minutes; progress streams below`. ```bash hawk rescan --scan-id --json-output @@ -111,9 +118,9 @@ second config-fix iteration. **9. Report** -- If clean **and no gate gaps remain open**: "Rescan complete. Zero new findings. All security issues have been resolved." +- If clean **and no gate gaps remain open**: `StackHawk | Rescan clean - 0 new findings; all security issues resolved on the scanned surface` - If clean **but gate gaps remain open** (cap reached with config-class gaps still unresolved, or an environment-class gap that recurred): report the gaps by reason identifier, the coverage evidence, and the remaining next steps. Do not say "all security issues have been resolved" or otherwise imply the app is done and secure while a gap is open. -- If findings remain: "Rescan found [N] remaining issues that require manual review:" and list them. +- If findings remain: `StackHawk | Rescan: [N] issues remain and need manual review` — then list them. - If findings were filtered by triage state: "Skipped [X] findings already triaged as RISK_ACCEPTED / FALSE_POSITIVE." - If findings were marked FALSE_POSITIVE in step 6: "Marked [N] findings as false positive — review at https://app.stackhawk.com/scans/\". @@ -123,7 +130,7 @@ second config-fix iteration. - **One scan at a time per app/env.** Never dispatch a second `hawk scan` or `hawk rescan` while one is still running. Run scan commands synchronously — never with `&` or `nohup`. Wait for the exit code. - **Max one fix-rescan cycle per task.** If the rescan still has findings after fixing, report the remaining issues rather than looping indefinitely. The user can ask for a follow-up. -- **Always announce what you're doing.** The developer should see "Running security scan...", "Found N vulnerabilities, fixing...", "Rescanning to verify..." in the output. +- **Always announce what you're doing** with the `StackHawk | ` prefix at every phase (SKILL.md **Progress output**) — the developer should see scan start, gate result, findings, and rescan stream by, never a silent gap. ASCII only, no emoji. - **Interruptible.** If the user interrupts or says to stop, stop immediately. - **Don't block on scan failures.** If `hawk scan` exits with code 1 (config error, app unreachable), report the error rather than retrying in a loop — with one exception: an environment-class failure (connection refused, timeout, unreachable) gets exactly one free retry after verifying the app is up. If the same environment-class signal recurs on that retry, report it and stop; don't retry a second time. - **Report gate gaps — never silently accept exit 0 from a thin scan.** From d1c4528523b340a13ede83f20e2bec9a1d244979 Mon Sep 17 00:00:00 2001 From: KC Berg Date: Wed, 15 Jul 2026 11:16:02 -0600 Subject: [PATCH 24/29] docs(hawkscan): drop internal "dogfood" framing from openapi-specs (public repo) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The shipped reference said "(in dogfood: 22 scanned URIs against 254 routes)" — internal-testing language in a customer-facing skill. Reword to a generic illustration ("well under 10% of routes reached on a typical REST API") that keeps the point (spec-less scans reach little) without the internal framing. ENG-704. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_012sAkR2BzkbLxzYUuGzLAQa --- cursor/.cursor/rules/stackhawk-hawkscan-openapi-specs.mdc | 5 +++-- plugins/hawkscan/skills/hawkscan/references/openapi-specs.md | 5 +++-- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/cursor/.cursor/rules/stackhawk-hawkscan-openapi-specs.mdc b/cursor/.cursor/rules/stackhawk-hawkscan-openapi-specs.mdc index c6325453..1bc098e7 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan-openapi-specs.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan-openapi-specs.mdc @@ -179,5 +179,6 @@ tooling (step 2) so the spec regenerates from code instead of drifting. visit — no methods, no request bodies, no path/query parameters — so a seedPaths-only scan can GET a handful of collection endpoints but never exercises POST/PUT/PATCH, parameterized routes, or request-body validation. That is why a spec-less scan collapses to a fraction of -the real surface (in dogfood: 22 scanned URIs against 254 routes). Only use seedPaths as a -supplement to a spec, or as a genuine last resort when even a hand-derived spec is impossible. +the real surface — on a typical REST API that can be well under 10% of the routes reached. +Only use seedPaths as a supplement to a spec, or as a genuine last resort when even a +hand-derived spec is impossible. diff --git a/plugins/hawkscan/skills/hawkscan/references/openapi-specs.md b/plugins/hawkscan/skills/hawkscan/references/openapi-specs.md index 310d66ef..539064dd 100644 --- a/plugins/hawkscan/skills/hawkscan/references/openapi-specs.md +++ b/plugins/hawkscan/skills/hawkscan/references/openapi-specs.md @@ -173,5 +173,6 @@ tooling (step 2) so the spec regenerates from code instead of drifting. visit — no methods, no request bodies, no path/query parameters — so a seedPaths-only scan can GET a handful of collection endpoints but never exercises POST/PUT/PATCH, parameterized routes, or request-body validation. That is why a spec-less scan collapses to a fraction of -the real surface (in dogfood: 22 scanned URIs against 254 routes). Only use seedPaths as a -supplement to a spec, or as a genuine last resort when even a hand-derived spec is impossible. +the real surface — on a typical REST API that can be well under 10% of the routes reached. +Only use seedPaths as a supplement to a spec, or as a genuine last resort when even a +hand-derived spec is impossible. From b86ab6316b4093c941672e7a2d73f805066f3be7 Mon Sep 17 00:00:00 2001 From: KC Berg Date: Thu, 16 Jul 2026 09:53:06 -0600 Subject: [PATCH 25/29] =?UTF-8?q?fix(evals):=20classify=20read-only=20guar?= =?UTF-8?q?d=20denials=20=E2=80=94=20app-start=20warns,=20hard=20fouls=20b?= =?UTF-8?q?lock?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cross-model inconsistency in the hawkscan discovery matrix (firefly-iii/memos sonnet blocking-fail while opus passes) traced to discovery_stayed_read_only being blocking on ANY guard denial. But a denial has two flavors: a blocked-and- recovered app-START attempt (php artisan serve / go run / docker run) is a soft, model-personality foul — the guard blocks it, discovery still completes read-only and correct; a write/scan/network-egress denial is a hard foul (discovery left its lane). Add a `guard_scope` field to guard_clean checks ("hard" | "app_start" | "all", default "all" — backward compatible). Scope discovery_stayed_read_only to "hard" (still blocking, now only real out-of-scope actions), and add discovery_no_app_start scoped to "app_start" at WARNING severity so the reflex is surfaced without flapping the matrix across models. +5 test cases. ENG-708. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_012sAkR2BzkbLxzYUuGzLAQa --- evals/hawkscan/process-checks.json | 15 +++++++++++-- evals/lib/grading.py | 33 +++++++++++++++++++++------ tests/lib/test_guard_clean_check.py | 35 +++++++++++++++++++++++++++++ 3 files changed, 74 insertions(+), 9 deletions(-) diff --git a/evals/hawkscan/process-checks.json b/evals/hawkscan/process-checks.json index bf6a6f47..d72cc3e4 100644 --- a/evals/hawkscan/process-checks.json +++ b/evals/hawkscan/process-checks.json @@ -343,11 +343,22 @@ { "id": "discovery_stayed_read_only", "phase": "app-discovery", - "description": "The read-only guard denied nothing (no writes/scans/app-start/remote push).", + "description": "Discovery took no HARD out-of-scope action — no file writes, no scans, no network egress, no remote push. A blocked-and-recovered app-start attempt is graded separately (discovery_no_app_start), not here.", "type": "guard_clean", + "guard_scope": "hard", "applies_to": ["firefly-iii", "wikijs", "memos", "dawarich"], "severity": "blocking", - "failure_guidance": "Discovery is observational; any guard denial means the agent acted outside scope." + "failure_guidance": "A write/scan/network-egress denial means discovery genuinely left its read-only lane — a real regression." + }, + { + "id": "discovery_no_app_start", + "phase": "app-discovery", + "description": "The agent did not try to START the app/container during discovery — host/port and run command should be read from config, not learned by launching anything.", + "type": "guard_clean", + "guard_scope": "app_start", + "applies_to": ["firefly-iii", "wikijs", "memos", "dawarich"], + "severity": "warning", + "failure_guidance": "Weaker models reflexively try `php artisan serve` / `go run` / `docker run` to learn the host; the guard blocks it and discovery recovers, so this is a soft foul (warning), not a blocking failure. Ideal discovery derives host/port from docker-compose/.env/README instead." }, { "id": "gate_identifies_base_path_mismatch", diff --git a/evals/lib/grading.py b/evals/lib/grading.py index 0c37c329..fa23526c 100644 --- a/evals/lib/grading.py +++ b/evals/lib/grading.py @@ -28,6 +28,24 @@ def _haystack(run: ParsedRun) -> str: return " ".join([*run.bash_commands, run.output_text]).lower() +# Read-only guard denials come in two flavors during discovery. An app-START +# attempt (`php artisan serve`, `go run`, `docker run`) is BLOCKED by the guard and +# the agent recovers — a soft, model-personality foul (opus resists it, weaker +# models reflex it), so it shouldn't hard-fail a cell whose discovery still +# completed read-only. A write / scan / network-egress denial is a HARD foul: +# discovery genuinely left its read-only lane. A check picks which flavor it asserts +# via `guard_scope`: "hard" (writes/scans/egress only), "app_start", or "all" (default). +_APP_START_DENY = "starting the app/server/container" + + +def _scoped_denials(denials: list[str], scope: str) -> list[str]: + if scope == "hard": + return [d for d in denials if _APP_START_DENY not in d] + if scope == "app_start": + return [d for d in denials if _APP_START_DENY in d] + return list(denials) # "all" + + def run_process_checks(run: ParsedRun, checks: list[dict]) -> list[ProcessCheckResult]: haystack = _haystack(run) # A "don't RUN command X" anti-pattern is about actual execution, so it scans @@ -77,16 +95,17 @@ def run_process_checks(run: ParsedRun, checks: list[dict]) -> list[ProcessCheckR else: passed = anti_hit is None # no preference expressed; only anti-patterns matter elif ctype == "guard_clean": - # Passes iff the read-only guard denied nothing during the run. - passed = not run.guard_denials + # Passes iff the guard denied nothing in this check's scope (see + # _scoped_denials). Default scope "all" keeps prior behavior. + rel = _scoped_denials(run.guard_denials, check.get("guard_scope", "all")) + passed = not rel + # Surface WHAT was denied (in scope) so a failure is diagnosable from the + # artifact without re-running the (costly) cell. + if rel: + anti_hit = "; ".join(d.strip() for d in rel)[:500] else: passed = signal_hit is not None and (anti_hit is None if antis else True) - # Surface WHAT the read-only guard denied so a guard_clean failure is - # diagnosable from the artifact without re-running the (costly) cell. - if ctype == "guard_clean" and run.guard_denials: - anti_hit = "; ".join(d.strip() for d in run.guard_denials)[:500] - results.append(ProcessCheckResult( id=check["id"], passed=passed, severity=check.get("severity", "warning"), diff --git a/tests/lib/test_guard_clean_check.py b/tests/lib/test_guard_clean_check.py index 0a4bb8d8..9038609d 100644 --- a/tests/lib/test_guard_clean_check.py +++ b/tests/lib/test_guard_clean_check.py @@ -36,5 +36,40 @@ def test_pass_has_no_anti_found(self): self.assertIsNone(res[0].anti_found) +APP_START = "Denied (read-only discovery): starting the app/server/container is out of scope for read-only discovery" +WRITE = "Denied (read-only discovery): Write would modify files" + + +class GuardScopeClassification(unittest.TestCase): + """A blocked-and-recovered app-START attempt is a soft foul; a write/scan/egress + denial is a hard foul. `guard_scope` lets a check assert one flavor.""" + + def _check(self, scope): + return [{"id": "c", "type": "guard_clean", "severity": "blocking", "guard_scope": scope}] + + def test_hard_scope_ignores_app_start(self): + run = ParsedRun(output_text="done", guard_denials=[APP_START]) + self.assertTrue(run_process_checks(run, self._check("hard"))[0].passed) + + def test_hard_scope_fails_on_write(self): + run = ParsedRun(output_text="done", guard_denials=[WRITE]) + self.assertFalse(run_process_checks(run, self._check("hard"))[0].passed) + + def test_hard_scope_fails_on_mixed(self): + run = ParsedRun(output_text="done", guard_denials=[APP_START, WRITE]) + res = run_process_checks(run, self._check("hard"))[0] + self.assertFalse(res.passed) + self.assertIn("Write would modify files", res.anti_found) + self.assertNotIn("starting the app", res.anti_found) # app-start not surfaced under hard scope + + def test_app_start_scope_flags_app_start(self): + run = ParsedRun(output_text="done", guard_denials=[APP_START]) + self.assertFalse(run_process_checks(run, self._check("app_start"))[0].passed) + + def test_app_start_scope_ignores_write(self): + run = ParsedRun(output_text="done", guard_denials=[WRITE]) + self.assertTrue(run_process_checks(run, self._check("app_start"))[0].passed) + + if __name__ == "__main__": unittest.main() From f45a5c225505fce45d0d0448397d71a619a22375 Mon Sep 17 00:00:00 2001 From: KC Berg Date: Thu, 16 Jul 2026 10:09:10 -0600 Subject: [PATCH 26/29] fix(evals): disable agy in CI + stop grading claude-code-only discovery cells as fails MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two matrix-consistency fixes surfaced running the full cross-agent matrix: 1. agy is OAuth-only and can't authenticate headless (always red). Comment out the eval-agy job, drop `agy` from the platform options, and remove it from the report job's `needs`. `platform: all` now = claude-code + codex + cursor. Re-enable later per the header note once agy creds work in CI. 2. target_repo (discovery) cells clone + read-only-guard a real repo — only the claude-code harness supports that; codex/cursor short-circuit them. But the harness still GRADED the short-circuited cell, so its expected-signal checks failed against empty output → 4 phantom blocking fails per codex/cursor hawkscan run (the bulk of their low scores). Skip those cells on non-claude harnesses in cli.py so they render "not present", not failed. ENG-708. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_012sAkR2BzkbLxzYUuGzLAQa --- .github/workflows/skill-evals.yml | 206 +++++++++++++++--------------- evals/cli.py | 6 + 2 files changed, 111 insertions(+), 101 deletions(-) diff --git a/.github/workflows/skill-evals.yml b/.github/workflows/skill-evals.yml index fc98d120..603d7d01 100644 --- a/.github/workflows/skill-evals.yml +++ b/.github/workflows/skill-evals.yml @@ -17,7 +17,7 @@ on: required: true default: "all" type: choice - options: [all, claude-code, codex, agy, cursor] + options: [all, claude-code, codex, cursor] # agy disabled (OAuth-only, can't auth in CI) rubric: description: "Also run the qualitative rubric grader (extra ANTHROPIC_API_KEY cost)" required: false @@ -256,105 +256,109 @@ jobs: retention-days: 30 # ── Antigravity (agy) — replaces Gemini ─────────────────────────────────── - eval-agy: - name: agy / ${{ matrix.skill }} / ${{ matrix.model }} - runs-on: ubuntu-latest - needs: validate-config - if: | - inputs.platform == 'all' || - inputs.platform == 'agy' - strategy: - fail-fast: false - matrix: - skill: [hawkscan, api, stackhawk-data-seed, hawkscan-ci, optimize] - model: [default] - - steps: - - uses: actions/checkout@v7 - - uses: astral-sh/setup-uv@v5 - - name: Install Claude Code CLI (native, rubric grader) - run: | - curl -fsSL https://claude.ai/install.sh | bash - echo "$HOME/.local/bin" >> "$GITHUB_PATH" - - - name: Install agy CLI - run: | - # /cli/install.sh is the real bootstrapper; /install-cli returns the - # site's HTML landing page (piping that into bash is what broke before). - curl -fsSL https://antigravity.google/cli/install.sh | bash - echo "$HOME/.local/bin" >> "$GITHUB_PATH" # installer drops `agy` here - continue-on-error: true # don't abort the job — evals records any launch failure - - - name: Verify agy CLI - run: agy --version - continue-on-error: true # if unavailable, the eval run captures it as a per-prompt error - - - name: Install StackHawk plugins - env: - # NOTE: agy ignores API-key env vars and forces interactive Google OAuth - # (verified empirically; upstream antigravity-cli#78 open). This mapping is - # kept for if/when key auth lands; headless CI needs the OAuth-token - # injection below. - ANTIGRAVITY_API_KEY: ${{ secrets.AGY_API_KEY }} - run: | - echo y | agy plugin install plugins/hawkscan - echo y | agy plugin install plugins/api - echo y | agy plugin install plugins/stackhawk-data-seed - echo y | agy plugin install plugins/hawkscan-ci - echo y | agy plugin install plugins/optimize - continue-on-error: true # depends on agy CLI; best-effort so evals still runs - - # HawkScan 6 ships as a single self-contained native binary (no JDK) that - # provides BOTH scanning (`hawk`) and the platform/API CLI (`hawk op …`). - # Install it from the download manifest (source of truth for version + URL) - # into /usr/local/bin (already on PATH). One install covers the hawkscan - # skill's CLI path AND the api skill + hawkscan Step-1 dedup checks. - - name: Install hawk CLI (combined binary, manifest-driven) - run: | - set -euo pipefail - MANIFEST=https://download.stackhawk.com/hawkdocs/hawk.manifest.json - url="$(curl -fsSL "$MANIFEST" | jq -r '.latest.assets[] - | select(.asset.group == "linux-x64" and (.url | endswith("/hawk"))) | .url')" - [ -n "$url" ] || { echo "ERROR: no linux-x64 hawk binary in manifest"; exit 1; } - echo "Installing hawk from $url" - curl -fsSL "$url" -o /tmp/hawk - chmod +x /tmp/hawk - sudo mv /tmp/hawk /usr/local/bin/hawk - continue-on-error: true # absence is captured per-prompt in the eval traces - - name: Verify hawk CLI - run: | - hawk version - hawk op --help - continue-on-error: true # absence is captured per-prompt in the eval traces - - - name: Run ${{ matrix.skill }} evals - env: - # agy ignores API-key env vars and forces OAuth (upstream #78); kept for - # if/when key auth lands. Headless CI needs OAuth-token injection (TODO). - ANTIGRAVITY_API_KEY: ${{ secrets.AGY_API_KEY }} - ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} # rubric grader (claude) - HAWK_API_KEY: ${{ secrets.HAWK_API_KEY }} - HAWK_FORMAT: json - run: | - MODEL_ARGS=() - if [ "${{ matrix.model }}" != "default" ]; then MODEL_ARGS=(--model "${{ matrix.model }}"); fi - RUBRIC=""; if [ "${{ inputs.rubric }}" = "true" ]; then RUBRIC="--rubric"; fi - uv run evals --harness agy --skill ${{ matrix.skill }} "${MODEL_ARGS[@]}" $RUBRIC - # No continue-on-error: the CLI now decides job color (exit 1 only on a - # PLUMBING FAILURE — 0 prompts ran cleanly). agy can't auth in headless CI - # (OAuth-only, upstream #78), so it exits 1 here and the job goes RED, which - # is correct: red = "couldn't run, needs work." The digest still renders agy's - # 🚫 cell because Upload results + the report job are both `if: always()`. - - - name: Upload results - if: always() - uses: actions/upload-artifact@v7 - with: - name: eval-agy-${{ matrix.skill }}-${{ matrix.model }} - path: evals/harnesses/agy/results/${{ matrix.skill }}/ - retention-days: 30 - - # ── Cursor ──────────────────────────────────────────────────────────────── + # --- eval-agy DISABLED: agy is OAuth-only and cannot authenticate in headless CI. + # Re-enable by uncommenting the job below, re-adding `agy` to the platform options, + # and adding `eval-agy` back to the report job's `needs:` list. See git history. + +# eval-agy: +# name: agy / ${{ matrix.skill }} / ${{ matrix.model }} +# runs-on: ubuntu-latest +# needs: validate-config +# # agy is OAuth-only and cannot authenticate in headless CI (always red), so it is +# # EXCLUDED from `all` — run it explicitly with platform: agy only when needed. +# if: inputs.platform == 'agy' +# strategy: +# fail-fast: false +# matrix: +# skill: [hawkscan, api, stackhawk-data-seed, hawkscan-ci, optimize] +# model: [default] +# +# steps: +# - uses: actions/checkout@v7 +# - uses: astral-sh/setup-uv@v5 +# - name: Install Claude Code CLI (native, rubric grader) +# run: | +# curl -fsSL https://claude.ai/install.sh | bash +# echo "$HOME/.local/bin" >> "$GITHUB_PATH" +# +# - name: Install agy CLI +# run: | +# # /cli/install.sh is the real bootstrapper; /install-cli returns the +# # site's HTML landing page (piping that into bash is what broke before). +# curl -fsSL https://antigravity.google/cli/install.sh | bash +# echo "$HOME/.local/bin" >> "$GITHUB_PATH" # installer drops `agy` here +# continue-on-error: true # don't abort the job — evals records any launch failure +# +# - name: Verify agy CLI +# run: agy --version +# continue-on-error: true # if unavailable, the eval run captures it as a per-prompt error +# +# - name: Install StackHawk plugins +# env: +# # NOTE: agy ignores API-key env vars and forces interactive Google OAuth +# # (verified empirically; upstream antigravity-cli#78 open). This mapping is +# # kept for if/when key auth lands; headless CI needs the OAuth-token +# # injection below. +# ANTIGRAVITY_API_KEY: ${{ secrets.AGY_API_KEY }} +# run: | +# echo y | agy plugin install plugins/hawkscan +# echo y | agy plugin install plugins/api +# echo y | agy plugin install plugins/stackhawk-data-seed +# echo y | agy plugin install plugins/hawkscan-ci +# echo y | agy plugin install plugins/optimize +# continue-on-error: true # depends on agy CLI; best-effort so evals still runs +# +# # HawkScan 6 ships as a single self-contained native binary (no JDK) that +# # provides BOTH scanning (`hawk`) and the platform/API CLI (`hawk op …`). +# # Install it from the download manifest (source of truth for version + URL) +# # into /usr/local/bin (already on PATH). One install covers the hawkscan +# # skill's CLI path AND the api skill + hawkscan Step-1 dedup checks. +# - name: Install hawk CLI (combined binary, manifest-driven) +# run: | +# set -euo pipefail +# MANIFEST=https://download.stackhawk.com/hawkdocs/hawk.manifest.json +# url="$(curl -fsSL "$MANIFEST" | jq -r '.latest.assets[] +# | select(.asset.group == "linux-x64" and (.url | endswith("/hawk"))) | .url')" +# [ -n "$url" ] || { echo "ERROR: no linux-x64 hawk binary in manifest"; exit 1; } +# echo "Installing hawk from $url" +# curl -fsSL "$url" -o /tmp/hawk +# chmod +x /tmp/hawk +# sudo mv /tmp/hawk /usr/local/bin/hawk +# continue-on-error: true # absence is captured per-prompt in the eval traces +# - name: Verify hawk CLI +# run: | +# hawk version +# hawk op --help +# continue-on-error: true # absence is captured per-prompt in the eval traces +# +# - name: Run ${{ matrix.skill }} evals +# env: +# # agy ignores API-key env vars and forces OAuth (upstream #78); kept for +# # if/when key auth lands. Headless CI needs OAuth-token injection (TODO). +# ANTIGRAVITY_API_KEY: ${{ secrets.AGY_API_KEY }} +# ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} # rubric grader (claude) +# HAWK_API_KEY: ${{ secrets.HAWK_API_KEY }} +# HAWK_FORMAT: json +# run: | +# MODEL_ARGS=() +# if [ "${{ matrix.model }}" != "default" ]; then MODEL_ARGS=(--model "${{ matrix.model }}"); fi +# RUBRIC=""; if [ "${{ inputs.rubric }}" = "true" ]; then RUBRIC="--rubric"; fi +# uv run evals --harness agy --skill ${{ matrix.skill }} "${MODEL_ARGS[@]}" $RUBRIC +# # No continue-on-error: the CLI now decides job color (exit 1 only on a +# # PLUMBING FAILURE — 0 prompts ran cleanly). agy can't auth in headless CI +# # (OAuth-only, upstream #78), so it exits 1 here and the job goes RED, which +# # is correct: red = "couldn't run, needs work." The digest still renders agy's +# # 🚫 cell because Upload results + the report job are both `if: always()`. +# +# - name: Upload results +# if: always() +# uses: actions/upload-artifact@v7 +# with: +# name: eval-agy-${{ matrix.skill }}-${{ matrix.model }} +# path: evals/harnesses/agy/results/${{ matrix.skill }}/ +# retention-days: 30 +# +# # ── Cursor ──────────────────────────────────────────────────────────────── eval-cursor: name: cursor / ${{ matrix.skill }} / ${{ matrix.model }} runs-on: ubuntu-latest @@ -438,7 +442,7 @@ jobs: # ── PR comment ──────────────────────────────────────────────────────────── report: name: Eval report (run summary + PR comment) - needs: [validate-config, eval-claude-code, eval-codex, eval-agy, eval-cursor] + needs: [validate-config, eval-claude-code, eval-codex, eval-cursor] if: always() runs-on: ubuntu-latest permissions: diff --git a/evals/cli.py b/evals/cli.py index aa9e7a5a..5dbf01cf 100644 --- a/evals/cli.py +++ b/evals/cli.py @@ -46,6 +46,12 @@ def main() -> None: out_dir = RESULTS_ROOT / args.harness / "results" / args.skill out_dir.mkdir(parents=True, exist_ok=True) for p in prompts: + if p.target_repo is not None and args.harness != "claude-code": + # Discovery (target_repo) cells clone + read-only-guard a real repo, which + # only the claude-code harness supports; codex/cursor/agy short-circuit them. + # Skip entirely so an un-run cell renders as "not present" in the matrix, + # not a phantom blocking failure that tanks the score for those harnesses. + continue try: run = adapter.launch(p.prompt, args.skill, p.id, plugin_dirs, model=args.model, load_skill=True, From 501701692e3fc50837516de3aa067e394c67282b Mon Sep 17 00:00:00 2001 From: KC Berg Date: Fri, 17 Jul 2026 07:19:16 -0600 Subject: [PATCH 27/29] =?UTF-8?q?fix(evals):=20data-seed=20=E2=80=94=20tie?= =?UTF-8?q?r=20deep=20checks=20extended=20+=20add=20ds-17=20Phase-2=20reas?= =?UTF-8?q?oning=20fixture?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The data-seed matrix weakness (haiku/sonnet/codex-o3 at 12/16) traced almost entirely to phase2_minimal_seed_proposed, a blocking output_contains with 15 generic substrings. The cells run in an EMPTY env (0/16 target_repo, not a git repo, hawk perch seed unavailable), so the agent can only DOCUMENT the multi-step workflow, never reach a real Phase-2 proposal. Pass/fail then hinged on whether a model's prose incidentally contained a signal word (opus hit "minimum seed"; sonnet hit none) — a phrasing lottery correlated with verbosity, not skill quality. Two fixes, matching how hawkscan handles live-only behavior: 1. Tier the phase-2/phase-3/phase-4 execution+emission checks `extended` so single-turn CI cells grade only what an empty-env turn can achieve (trigger, preflight, discovery, honesty, anti-patterns); the deep flow is graded only in full_auto/dogfood against a real repo. 2. Add ds-17: an own_checks_only Phase-2 reasoning fixture that inlines a repo digest (users/orgs/api_keys + a bearer-JWT login) and grades the minimal-seed PROPOSAL (ds17_minimal_seed_proposed blocking + credential/confirmation-gate warnings) — reachable and meaningful in one turn. Skill unchanged: the weakness was an eval mismatch, not a skill deficiency. ENG-708. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_012sAkR2BzkbLxzYUuGzLAQa --- evals/stackhawk-data-seed/process-checks.json | 83 +++++++++++++++++-- evals/stackhawk-data-seed/prompts.yaml | 14 ++++ 2 files changed, 90 insertions(+), 7 deletions(-) diff --git a/evals/stackhawk-data-seed/process-checks.json b/evals/stackhawk-data-seed/process-checks.json index 74da6f85..b2783cb3 100644 --- a/evals/stackhawk-data-seed/process-checks.json +++ b/evals/stackhawk-data-seed/process-checks.json @@ -101,7 +101,8 @@ "example test org" ], "severity": "blocking", - "failure_guidance": "Phase 2 is a dialog gate. Emitting artifacts without confirming the proposal violates the skill's contract \u2014 the user may want different entities or counts." + "failure_guidance": "Phase 2 is a dialog gate. Emitting artifacts without confirming the proposal violates the skill's contract \u2014 the user may want different entities or counts.", + "tier": "extended" }, { "id": "phase3_emits_to_data_seed_dir", @@ -114,7 +115,8 @@ "data-seed/credentials.env.example" ], "severity": "warning", - "failure_guidance": "The output contract is data-seed/. Missing these files means downstream consumers (hawkscan reads .data-seed-credentials.env, the future Runner reads manifest.yaml) have nothing to consume." + "failure_guidance": "The output contract is data-seed/. Missing these files means downstream consumers (hawkscan reads .data-seed-credentials.env, the future Runner reads manifest.yaml) have nothing to consume.", + "tier": "extended" }, { "id": "phase3_no_legacy_bootstrap_dir", @@ -138,7 +140,8 @@ "data-seed/manifest.yaml" ], "severity": "warning", - "failure_guidance": "The manifest is the contract between the skill and the future Runner. No manifest = no replay = no value." + "failure_guidance": "The manifest is the contract between the skill and the future Runner. No manifest = no replay = no value.", + "tier": "extended" }, { "id": "phase3_manifest_self_validated", @@ -154,7 +157,8 @@ "no circular" ], "severity": "warning", - "failure_guidance": "A half-broken manifest is worse than a clear error. Self-validation must run before any file is written to disk." + "failure_guidance": "A half-broken manifest is worse than a clear error. Self-validation must run before any file is written to disk.", + "tier": "extended" }, { "id": "phase3_credentials_emitted", @@ -165,7 +169,8 @@ ".data-seed-credentials.env" ], "severity": "warning", - "failure_guidance": "The credentials file is the handoff to hawkscan. Without it the seed is unusable." + "failure_guidance": "The credentials file is the handoff to hawkscan. Without it the seed is unusable.", + "tier": "extended" }, { "id": "phase3_gitignore_updated", @@ -177,7 +182,8 @@ ".data-seed-credentials.env" ], "severity": "warning", - "failure_guidance": "If the credentials file is not gitignored, the suggested commit in Phase 4.2 may publish secrets to the repo." + "failure_guidance": "If the credentials file is not gitignored, the suggested commit in Phase 4.2 may publish secrets to the repo.", + "tier": "extended" }, { "id": "phase3_idempotent_seed_steps", @@ -213,7 +219,8 @@ "Invoke hawkscan" ], "severity": "warning", - "failure_guidance": "Without the summary, the user doesn't know what files exist, what to commit, or how to continue to the next step (hawkscan)." + "failure_guidance": "Without the summary, the user doesn't know what files exist, what to commit, or how to continue to the next step (hawkscan).", + "tier": "extended" }, { "id": "antipattern_no_stackhawk_yml_written", @@ -266,6 +273,68 @@ ], "severity": "warning", "failure_guidance": "Unlike hawkscan, this skill is explicitly NOT autonomous. Treating it as a per-commit safety net wastes the user's time and dirties their tree." + }, + { + "id": "ds17_minimal_seed_proposed", + "phase": "phase_2", + "description": "Given an inline repo digest, the agent proposes a MINIMUM seed (counts + entities, ~one user / one org) rather than an exhaustive dump.", + "type": "output_contains", + "applies_to": [ + "ds-17" + ], + "signals": [ + "minimum", + "minimal", + "one user", + "1 user", + "single user", + "one org", + "1 org", + "single org", + "one organization" + ], + "severity": "blocking", + "failure_guidance": "Phase 2's core is proposing the SMALLEST seed that unblocks an authenticated scan. No minimal-set framing means the agent didn't do the Phase-2 design." + }, + { + "id": "ds17_credential_identified", + "phase": "phase_2", + "description": "Agent identifies the one credential an authenticated HawkScan would use (the bearer/JWT token from the login flow).", + "type": "output_contains", + "applies_to": [ + "ds-17" + ], + "signals": [ + "bearer", + "jwt", + "token", + "api key", + "api_key", + "credential", + "login" + ], + "severity": "warning", + "failure_guidance": "The seed exists to let an authenticated scan in; the proposal should name the credential HawkScan authenticates with." + }, + { + "id": "ds17_confirmation_gate", + "phase": "phase_2", + "description": "Agent surfaces the proposal for confirmation before emitting anything (Phase 2 is a dialog gate).", + "type": "output_contains", + "applies_to": [ + "ds-17" + ], + "signals": [ + "confirm", + "ok to proceed", + "before i", + "review", + "sign off", + "does this look", + "let me know" + ], + "severity": "warning", + "failure_guidance": "Phase 2 must present the proposal for sign-off, not emit artifacts unprompted." } ] } diff --git a/evals/stackhawk-data-seed/prompts.yaml b/evals/stackhawk-data-seed/prompts.yaml index 57173411..419baf22 100644 --- a/evals/stackhawk-data-seed/prompts.yaml +++ b/evals/stackhawk-data-seed/prompts.yaml @@ -78,3 +78,17 @@ invocation_type: negative prompt: Start the docker-compose stack and run the integration tests notes: Environment startup — skill explicitly forbids running services +- id: ds-17 + should_trigger: true + invocation_type: explicit + own_checks_only: true + prompt: | + Using the stackhawk-data-seed skill's Phase 2 (minimum-seed design), propose the seed for this service. You do NOT have shell or CLI access in this exercise — reason over the digest below and output only the proposal (do not write files or run commands). + + Repo digest: + - PostgreSQL datastore. Tables: `users` (id, email, password_hash, org_id), `organizations` (id, name), `api_keys` (id, user_id, token_hash). + - Auth: POST /login takes {email, password} and returns a bearer JWT. Routes under /api/v1 require that bearer token. + - No fixtures/seed checked in; users and organizations are empty on a fresh database. + + Propose the MINIMUM seed an authenticated HawkScan needs to reach the API: which entities and how many, and the one credential HawkScan would authenticate with. Present it for confirmation before emitting anything. + notes: Phase-2 reasoning fixture — minimal-seed proposal from an inline digest (own_checks_only; no live repo / hawk perch needed) From ed76c16b3b07f4b95e111b38ac685104885a6c75 Mon Sep 17 00:00:00 2001 From: KC Berg Date: Fri, 17 Jul 2026 15:42:58 -0600 Subject: [PATCH 28/29] =?UTF-8?q?fix(hawkscan-ci):=20sharpen=20trigger=20d?= =?UTF-8?q?escription=20=E2=80=94=20add=20Do-NOT=20exclusions=20+=20hand-o?= =?UTF-8?q?ff?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit hawkscan-ci over-triggered in the eval matrix on doc-only, research, and local-scan prompts (hci-21 "update the README about CI scanning" fired on all 4 models; hci-22 "what CI providers does HawkScan support?"; hci-17 "run a scan before deploy"). Root cause: the frontmatter description listed CI+scan trigger phrases but had no "Do NOT trigger for" clause (skill-authoring requires one; the base hawkscan skill has one). Add explicit exclusions — documentation-only changes, informational/research questions, and running a local scan (that's the hawkscan skill; hawkscan-ci only edits CI pipeline config). Also reframe the opener to "WIRE ... into a CI/CD pipeline config file" and soften "requires a working local scan path" → if no stackhawk.yml exists yet, still trigger and route local-config to hawkscan (fixes the hci-14 hand-off). Mirrored into the cursor MAPPINGS description; regenerated. Description 1001 chars (<=1024). Trigger behavior is soft/model-dependent — will verify empirically. ENG-704. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_012sAkR2BzkbLxzYUuGzLAQa --- .../.cursor/rules/stackhawk-hawkscan-ci.mdc | 2 +- .../hawkscan-ci/skills/hawkscan-ci/SKILL.md | 26 ++++++++++--------- scripts/generate-cursor-rules.sh | 2 +- 3 files changed, 16 insertions(+), 14 deletions(-) diff --git a/cursor/.cursor/rules/stackhawk-hawkscan-ci.mdc b/cursor/.cursor/rules/stackhawk-hawkscan-ci.mdc index 7882599a..52cf2fbe 100644 --- a/cursor/.cursor/rules/stackhawk-hawkscan-ci.mdc +++ b/cursor/.cursor/rules/stackhawk-hawkscan-ci.mdc @@ -1,6 +1,6 @@ --- description: > - Use when the user wants to configure HawkScan in their CI/CD pipeline — triggers on "set up hawkscan in CI", "add stackhawk to my pipeline", "scan in CI", "configure github actions / gitlab / jenkins / circleci for hawkscan", "wire hawkscan into ci/cd". Provider-agnostic: detects the CI system, edits the pipeline file in place to add a HawkScan job, prompts for HAWK_API_KEY storage (CI-native secrets store or external secrets manager), wires commit-SHA + branch traceability. Defers all local-scan concerns (stackhawk.yml, auth, findings, triage) to stackhawk-hawkscan. + Use when the user wants to WIRE HawkScan into a CI/CD pipeline config file — triggers on "set up hawkscan in CI", "add stackhawk to my pipeline", "scan in CI", "configure github actions / gitlab / jenkins / circleci for hawkscan", "wire hawkscan into ci/cd". Provider-agnostic: detects the CI system, edits the pipeline file in place to add a HawkScan job, prompts for HAWK_API_KEY storage (CI-native secrets store or external secrets manager), wires commit-SHA + branch traceability. Defers all local-scan concerns (stackhawk.yml, auth, findings, triage) to stackhawk-hawkscan. If no local stackhawk.yml exists yet, still trigger and route local-config work to stackhawk-hawkscan. Do NOT trigger for documentation-only changes, informational/research questions about CI, or running a local scan (that is stackhawk-hawkscan) — this skill only edits CI pipeline config. globs: - ".github/workflows/**" - ".gitlab-ci.yml" diff --git a/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md b/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md index d752d75a..5bc2cf00 100644 --- a/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md +++ b/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md @@ -2,18 +2,20 @@ name: hawkscan-ci version: 2.3.4 description: > - Use when the user wants to configure HawkScan in their CI/CD pipeline — - triggers on "set up hawkscan in CI", "add stackhawk to my pipeline", - "scan in CI", "configure github actions / gitlab / jenkins / circleci - for hawkscan", "wire hawkscan into ci/cd", or any provider-named - variant. Provider-agnostic: detects the CI system from repo files, - edits the pipeline file in place to add a HawkScan job, prompts the - user to set HAWK_API_KEY in their CI's native secrets engine (or an - organizationally-approved external secrets manager), and wires - commit-SHA + branch traceability. Defers every local-scan concern - (stackhawk.yml, auth, findings, triage) to the hawkscan skill — - requires a working local scan path before activating. Explicit - trigger only; no autonomous code-change hook. + Use when the user wants to WIRE HawkScan into a CI/CD pipeline config + file — triggers on "set up hawkscan in CI", "add stackhawk to my + pipeline", "scan in CI", "configure github actions / gitlab / jenkins / + circleci for hawkscan", "wire hawkscan into ci/cd", or any provider-named + variant. Provider-agnostic: detects the CI system from repo files, edits + the pipeline file in place to add a HawkScan job, prompts for HAWK_API_KEY + storage in the CI's native secrets store (or an approved external + manager), and wires commit-SHA + branch traceability. If no local + stackhawk.yml exists yet, still trigger and route the local-config work to + the hawkscan skill. Do NOT trigger for: documentation-only changes (e.g. + editing a README about CI scanning); informational or research questions + ("what CI providers does HawkScan support?"); or running / performing a + scan — a local scan is the hawkscan skill, whereas this skill only edits + CI pipeline config. Explicit trigger only; no autonomous code-change hook. --- # HawkScan CI Skill diff --git a/scripts/generate-cursor-rules.sh b/scripts/generate-cursor-rules.sh index 255c6662..23229617 100755 --- a/scripts/generate-cursor-rules.sh +++ b/scripts/generate-cursor-rules.sh @@ -31,7 +31,7 @@ MAPPINGS=( "api/skills/api/SKILL.md|stackhawk-api|Use when querying the StackHawk platform for security reporting, findings analysis, or app management. Triggers include \"stackhawk api\", \"security posture\", \"findings report\", \"show me findings\", \"untriaged findings\", \"which apps\", \"scan history\", \"security dashboard\", \"triage\", \"what needs attention\". Uses the combined hawk CLI (hawk op …) for all platform queries. Do NOT use for running scans (use the stackhawk-hawkscan rule for \"scan my app\", \"hawkscan\", \"stackhawk.yml\", \"DAST\") or for fixing/remediating code or vulnerabilities — this skill only reads and reports platform data.||false" "api/skills/api/references/hawk-op-shortcuts.md|stackhawk-api-hawk-op|hawk op shortcuts for the StackHawk API: user intent → single hawk op command mapping for scan drill-down, app/user/team/policy/repo/oas/config/secret/audit listing, hosted scan control, env management. All platform queries go through hawk op — no raw REST fallback.||false" "api/skills/api/references/reporting-recipes.md|stackhawk-api-recipes|StackHawk API reporting recipes: org security posture summary, app deep dive (scan -> alerts -> findings), stale apps detection, scan diff (what changed since last scan). Pre-built compositions using hawk op commands. Prefer hawk op shortcuts (stackhawk-api-hawk-op) for the deep-dive chain; use these recipes for the per-env untriaged posture view.||false" - "hawkscan-ci/skills/hawkscan-ci/SKILL.md|stackhawk-hawkscan-ci|Use when the user wants to configure HawkScan in their CI/CD pipeline — triggers on \"set up hawkscan in CI\", \"add stackhawk to my pipeline\", \"scan in CI\", \"configure github actions / gitlab / jenkins / circleci for hawkscan\", \"wire hawkscan into ci/cd\". Provider-agnostic: detects the CI system, edits the pipeline file in place to add a HawkScan job, prompts for HAWK_API_KEY storage (CI-native secrets store or external secrets manager), wires commit-SHA + branch traceability. Defers all local-scan concerns (stackhawk.yml, auth, findings, triage) to stackhawk-hawkscan.|.github/workflows/**,.gitlab-ci.yml,Jenkinsfile*,.circleci/config.yml,azure-pipelines*.yml,bitbucket-pipelines.yml,.buildkite/pipeline.yml,.travis.yml,appspec.yml,buildspec.yml|false" + "hawkscan-ci/skills/hawkscan-ci/SKILL.md|stackhawk-hawkscan-ci|Use when the user wants to WIRE HawkScan into a CI/CD pipeline config file — triggers on \"set up hawkscan in CI\", \"add stackhawk to my pipeline\", \"scan in CI\", \"configure github actions / gitlab / jenkins / circleci for hawkscan\", \"wire hawkscan into ci/cd\". Provider-agnostic: detects the CI system, edits the pipeline file in place to add a HawkScan job, prompts for HAWK_API_KEY storage (CI-native secrets store or external secrets manager), wires commit-SHA + branch traceability. Defers all local-scan concerns (stackhawk.yml, auth, findings, triage) to stackhawk-hawkscan. If no local stackhawk.yml exists yet, still trigger and route local-config work to stackhawk-hawkscan. Do NOT trigger for documentation-only changes, informational/research questions about CI, or running a local scan (that is stackhawk-hawkscan) — this skill only edits CI pipeline config.|.github/workflows/**,.gitlab-ci.yml,Jenkinsfile*,.circleci/config.yml,azure-pipelines*.yml,bitbucket-pipelines.yml,.buildkite/pipeline.yml,.travis.yml,appspec.yml,buildspec.yml|false" "hawkscan-ci/skills/hawkscan-ci/references/execution-shapes.md|stackhawk-hawkscan-ci-execution-shapes|HawkScan CI execution shapes reference: native action (stackhawk/hawkscan-action) vs Docker image (stackhawk/hawkscan) vs CLI download; pinning strategy; per-provider quick-reference recipes for GitHub Actions, GitLab CI, Jenkins, CircleCI, Azure Pipelines, Bitbucket, Buildkite, Travis, AWS CodeBuild; SARIF / Code Scanning notes.||false" "hawkscan-ci/skills/hawkscan-ci/references/app-startup-patterns.md|stackhawk-hawkscan-ci-app-startup|HawkScan CI app-startup patterns reference: Docker Compose + wait-for-it, GitLab services keyword, GitHub Actions services map, build-then-run (Node, JVM), run-a-built-image, scan-existing-host (ephemeral env / preview deployment), networking gotchas (--network host vs host.docker.internal), health-check endpoint conventions.||false" "hawkscan-ci/skills/hawkscan-ci/references/failure-semantics.md|stackhawk-hawkscan-ci-failure|HawkScan CI failure semantics reference: exit codes 0/1/42, failureThreshold tuning, block-on-42 vs warn-only vs scheduled-baseline modes, retry strategy (don't retry 42), caching strategy (cache CLI/image, never findings), scheduled-vs-PR-trigger tradeoffs.||false" From b0f52012d35f5c08349e9514cade0041c9f4f4e3 Mon Sep 17 00:00:00 2001 From: KC Berg Date: Mon, 20 Jul 2026 06:31:24 -0600 Subject: [PATCH 29/29] chore(release): bump version 2.3.4 -> 2.4.0 Minor release: the scan-planning discovery + post-scan quality gate + OpenAPI spec-accuracy rewrite of the hawkscan skill, StackHawk-branded loop progress output, and the skill-eval matrix-consistency work (guard-denial classification, non-claude discovery-cell skip, agy disabled in CI, data-seed Phase-2 reasoning fixture, hawkscan-ci trigger exclusions). bump-version.sh --minor updated VERSION + all platform manifests + SKILL.md frontmatter atomically; cursor rules regenerated (idempotent). ENG-704. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_012sAkR2BzkbLxzYUuGzLAQa --- .claude-plugin/marketplace.json | 12 ++++++------ .claude/skills/skill-authoring/SKILL.md | 2 +- .codex-plugin/plugin.json | 2 +- VERSION | 2 +- gemini-extension.json | 2 +- plugin.json | 2 +- plugins/api/.claude-plugin/plugin.json | 2 +- plugins/api/.codex-plugin/plugin.json | 2 +- plugins/api/skills/api/SKILL.md | 2 +- plugins/hawkscan-ci/.claude-plugin/plugin.json | 2 +- plugins/hawkscan-ci/.codex-plugin/plugin.json | 2 +- plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md | 2 +- plugins/hawkscan/.claude-plugin/plugin.json | 2 +- plugins/hawkscan/.codex-plugin/plugin.json | 2 +- plugins/hawkscan/skills/hawkscan/SKILL.md | 2 +- plugins/optimize/.claude-plugin/plugin.json | 2 +- plugins/optimize/.codex-plugin/plugin.json | 2 +- plugins/optimize/skills/optimize/SKILL.md | 2 +- .../stackhawk-data-seed/.claude-plugin/plugin.json | 2 +- .../stackhawk-data-seed/.codex-plugin/plugin.json | 2 +- .../skills/stackhawk-data-seed/SKILL.md | 2 +- plugins/wingman/.claude-plugin/plugin.json | 2 +- plugins/wingman/.codex-plugin/plugin.json | 2 +- 23 files changed, 28 insertions(+), 28 deletions(-) diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index bca16232..5f044101 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -14,7 +14,7 @@ "category": "security", "source": "./plugins/hawkscan", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/hawkscan", - "version": "2.3.4" + "version": "2.4.0" }, { "name": "stackhawk-api", @@ -22,7 +22,7 @@ "category": "security", "source": "./plugins/api", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/api", - "version": "2.3.4" + "version": "2.4.0" }, { "name": "hawkscan-ci", @@ -30,7 +30,7 @@ "category": "security", "source": "./plugins/hawkscan-ci", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/hawkscan-ci", - "version": "2.3.4" + "version": "2.4.0" }, { "name": "stackhawk-data-seed", @@ -38,7 +38,7 @@ "category": "security", "source": "./plugins/stackhawk-data-seed", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/stackhawk-data-seed", - "version": "2.3.4" + "version": "2.4.0" }, { "name": "wingman", @@ -46,7 +46,7 @@ "category": "security", "source": "./plugins/wingman", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/wingman", - "version": "2.3.4" + "version": "2.4.0" }, { "name": "stackhawk-optimize", @@ -54,7 +54,7 @@ "category": "security", "source": "./plugins/optimize", "homepage": "https://github.com/stackhawk/agent-skills/tree/main/plugins/optimize", - "version": "2.3.4" + "version": "2.4.0" } ] } diff --git a/.claude/skills/skill-authoring/SKILL.md b/.claude/skills/skill-authoring/SKILL.md index bab869ee..a4f1ca31 100644 --- a/.claude/skills/skill-authoring/SKILL.md +++ b/.claude/skills/skill-authoring/SKILL.md @@ -1,6 +1,6 @@ --- name: skill-authoring -version: 2.3.4 +version: 2.4.0 description: > Guides authoring and maintaining agent skills in this repository — enforcing Anthropic best practices, bumping versions, regenerating Cursor rules, and diff --git a/.codex-plugin/plugin.json b/.codex-plugin/plugin.json index 2ecf7b37..dd44546b 100644 --- a/.codex-plugin/plugin.json +++ b/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk", - "version": "2.3.4", + "version": "2.4.0", "description": "Dynamic application and API security testing (DAST) powered by HawkScan. Configure scans, interpret findings, and query the StackHawk platform API.", "author": { "name": "StackHawk", diff --git a/VERSION b/VERSION index fd06a926..9183195a 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -2.3.4 \ No newline at end of file +2.4.0 \ No newline at end of file diff --git a/gemini-extension.json b/gemini-extension.json index a647d26c..7e2386ef 100644 --- a/gemini-extension.json +++ b/gemini-extension.json @@ -1,5 +1,5 @@ { "name": "stackhawk", - "version": "2.3.4", + "version": "2.4.0", "description": "StackHawk security scanning and API reporting skills. AUTONOMOUS: After completing any code change, invoke the hawkscan skill to scan, fix all reported vulnerabilities (not just your changes), and rescan to verify. Also invoke for any mention of hawkscan, stackhawk, DAST, security scan." } diff --git a/plugin.json b/plugin.json index cc62ccd2..38d6368a 100644 --- a/plugin.json +++ b/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk", - "version": "2.3.4", + "version": "2.4.0", "description": "StackHawk security scanning and API reporting skills. Includes hawkscan (DAST scanning) and api (security posture reporting).", "author": { "name": "StackHawk", diff --git a/plugins/api/.claude-plugin/plugin.json b/plugins/api/.claude-plugin/plugin.json index 755ef838..9fdb379c 100644 --- a/plugins/api/.claude-plugin/plugin.json +++ b/plugins/api/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-api", - "version": "2.3.4", + "version": "2.4.0", "description": "Query the StackHawk platform API for security posture reporting, findings analysis, and app management. Uses the combined hawk CLI (hawk op \u2026) for all platform queries.", "author": { "name": "StackHawk", diff --git a/plugins/api/.codex-plugin/plugin.json b/plugins/api/.codex-plugin/plugin.json index 6da84d58..50f77b1a 100644 --- a/plugins/api/.codex-plugin/plugin.json +++ b/plugins/api/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-api", - "version": "2.3.4", + "version": "2.4.0", "description": "Query the StackHawk platform API for security posture reporting, findings analysis, and app management. Uses the combined hawk CLI (hawk op \u2026) for all platform queries.", "author": { "name": "StackHawk", diff --git a/plugins/api/skills/api/SKILL.md b/plugins/api/skills/api/SKILL.md index 10c2676f..44637d42 100644 --- a/plugins/api/skills/api/SKILL.md +++ b/plugins/api/skills/api/SKILL.md @@ -1,6 +1,6 @@ --- name: api -version: 2.3.4 +version: 2.4.0 description: > Use this skill when a user or agent needs to query the StackHawk platform for security reporting, findings analysis, or app management. Triggers include: diff --git a/plugins/hawkscan-ci/.claude-plugin/plugin.json b/plugins/hawkscan-ci/.claude-plugin/plugin.json index 1e2d4811..63b28a6d 100644 --- a/plugins/hawkscan-ci/.claude-plugin/plugin.json +++ b/plugins/hawkscan-ci/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "hawkscan-ci", - "version": "2.3.4", + "version": "2.4.0", "description": "Configure HawkScan in your CI/CD pipeline. Provider-agnostic: detects GitHub Actions, GitLab CI, Jenkins, CircleCI, Azure Pipelines, Bitbucket, Buildkite, and other CI systems from repo files, edits the pipeline file in place to add a HawkScan job, and prompts for HAWK_API_KEY storage (CI-native secrets store or external secrets manager).", "author": { "name": "StackHawk", diff --git a/plugins/hawkscan-ci/.codex-plugin/plugin.json b/plugins/hawkscan-ci/.codex-plugin/plugin.json index 32d5bd34..b73cced8 100644 --- a/plugins/hawkscan-ci/.codex-plugin/plugin.json +++ b/plugins/hawkscan-ci/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "hawkscan-ci", - "version": "2.3.4", + "version": "2.4.0", "description": "Configure HawkScan in your CI/CD pipeline. Provider-agnostic: detects GitHub Actions, GitLab CI, Jenkins, CircleCI, Azure Pipelines, Bitbucket, Buildkite, and other CI systems from repo files, edits the pipeline file in place to add a HawkScan job, and prompts for HAWK_API_KEY storage (CI-native secrets store or external secrets manager).", "author": { "name": "StackHawk", diff --git a/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md b/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md index 5bc2cf00..90820aa4 100644 --- a/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md +++ b/plugins/hawkscan-ci/skills/hawkscan-ci/SKILL.md @@ -1,6 +1,6 @@ --- name: hawkscan-ci -version: 2.3.4 +version: 2.4.0 description: > Use when the user wants to WIRE HawkScan into a CI/CD pipeline config file — triggers on "set up hawkscan in CI", "add stackhawk to my diff --git a/plugins/hawkscan/.claude-plugin/plugin.json b/plugins/hawkscan/.claude-plugin/plugin.json index a257fbdf..56f2c42c 100644 --- a/plugins/hawkscan/.claude-plugin/plugin.json +++ b/plugins/hawkscan/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "hawkscan", - "version": "2.3.4", + "version": "2.4.0", "description": "Configure, run, and interpret HawkScan DAST results inside Claude Code. Generates stackhawk.yml configs, runs scans via CLI or Docker, and transforms security findings into prioritized fix tasks for your coding agent.", "author": { "name": "StackHawk", diff --git a/plugins/hawkscan/.codex-plugin/plugin.json b/plugins/hawkscan/.codex-plugin/plugin.json index fb9a537f..20e6085c 100644 --- a/plugins/hawkscan/.codex-plugin/plugin.json +++ b/plugins/hawkscan/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "hawkscan", - "version": "2.3.4", + "version": "2.4.0", "description": "Configure, run, and interpret HawkScan DAST results. Generates stackhawk.yml configs, runs scans via CLI or Docker, and transforms security findings into prioritized fix tasks for your coding agent.", "author": { "name": "StackHawk", diff --git a/plugins/hawkscan/skills/hawkscan/SKILL.md b/plugins/hawkscan/skills/hawkscan/SKILL.md index af4820d3..bb90b53d 100644 --- a/plugins/hawkscan/skills/hawkscan/SKILL.md +++ b/plugins/hawkscan/skills/hawkscan/SKILL.md @@ -1,6 +1,6 @@ --- name: hawkscan -version: 2.3.4 +version: 2.4.0 description: > Runs the HawkScan DAST security loop — configure, scan, fix all reported vulnerabilities (not just your changes), rescan to verify. Performs diff --git a/plugins/optimize/.claude-plugin/plugin.json b/plugins/optimize/.claude-plugin/plugin.json index a1cf258f..ead71432 100644 --- a/plugins/optimize/.claude-plugin/plugin.json +++ b/plugins/optimize/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-optimize", - "version": "2.3.4", + "version": "2.4.0", "description": "Analyze a codebase and produce an optimal HawkScan setup (tech flags, scan-policy plugin selection, stackhawk.yml corrections), apply it as a non-destructive trial, run one trial scan, and promote or discard.", "author": { "name": "StackHawk", diff --git a/plugins/optimize/.codex-plugin/plugin.json b/plugins/optimize/.codex-plugin/plugin.json index e3d88001..e7cc0148 100644 --- a/plugins/optimize/.codex-plugin/plugin.json +++ b/plugins/optimize/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-optimize", - "version": "2.3.4", + "version": "2.4.0", "description": "Analyze a codebase and produce an optimal HawkScan setup (tech flags, scan-policy plugin selection, stackhawk.yml corrections), apply it as a non-destructive trial, run one trial scan, and promote or discard.", "author": { "name": "StackHawk", diff --git a/plugins/optimize/skills/optimize/SKILL.md b/plugins/optimize/skills/optimize/SKILL.md index aacdf36d..22e0556e 100644 --- a/plugins/optimize/skills/optimize/SKILL.md +++ b/plugins/optimize/skills/optimize/SKILL.md @@ -1,6 +1,6 @@ --- name: optimize -version: 2.3.4 +version: 2.4.0 description: > Analyze a codebase and produce an optimal HawkScan setup — tech flags, scan-policy plugin selection, and stackhawk.yml corrections — then apply it as a non-destructive diff --git a/plugins/stackhawk-data-seed/.claude-plugin/plugin.json b/plugins/stackhawk-data-seed/.claude-plugin/plugin.json index ea9f7bcb..b710b96b 100644 --- a/plugins/stackhawk-data-seed/.claude-plugin/plugin.json +++ b/plugins/stackhawk-data-seed/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-data-seed", - "version": "2.3.4", + "version": "2.4.0", "description": "Read a target repo and emit checked-in seed artifacts so HawkScan has data to scan. Generates SQL / HTTP / gRPC / Mongo / shell steps across the target repo and any upstream services it depends on, plus a manifest and a credentials handoff file that hawkscan consumes.", "author": { "name": "StackHawk", diff --git a/plugins/stackhawk-data-seed/.codex-plugin/plugin.json b/plugins/stackhawk-data-seed/.codex-plugin/plugin.json index 41134578..f8500265 100644 --- a/plugins/stackhawk-data-seed/.codex-plugin/plugin.json +++ b/plugins/stackhawk-data-seed/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackhawk-data-seed", - "version": "2.3.4", + "version": "2.4.0", "description": "Read a target repo and emit checked-in seed artifacts so HawkScan has data to scan. Generates SQL / HTTP / gRPC / Mongo / shell steps across the target repo and any upstream services it depends on, plus a manifest and a credentials handoff file that hawkscan consumes.", "author": { "name": "StackHawk", diff --git a/plugins/stackhawk-data-seed/skills/stackhawk-data-seed/SKILL.md b/plugins/stackhawk-data-seed/skills/stackhawk-data-seed/SKILL.md index 6d4e507e..fce1406c 100644 --- a/plugins/stackhawk-data-seed/skills/stackhawk-data-seed/SKILL.md +++ b/plugins/stackhawk-data-seed/skills/stackhawk-data-seed/SKILL.md @@ -1,6 +1,6 @@ --- name: stackhawk-data-seed -version: 2.3.4 +version: 2.4.0 description: > Set up checked-in seed data so authenticated HawkScan can reach non-trivial paths. Drives the `hawk perch seed` preflight, designs the minimum seed diff --git a/plugins/wingman/.claude-plugin/plugin.json b/plugins/wingman/.claude-plugin/plugin.json index 60b8a7d2..c406b15a 100644 --- a/plugins/wingman/.claude-plugin/plugin.json +++ b/plugins/wingman/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "wingman", - "version": "2.3.4", + "version": "2.4.0", "description": "StackHawk umbrella plugin \u2014 installs the default StackHawk skill set (hawkscan, api, data-seed, optimize) in one command for the combined hawk CLI.", "author": { "name": "StackHawk", diff --git a/plugins/wingman/.codex-plugin/plugin.json b/plugins/wingman/.codex-plugin/plugin.json index 60b8a7d2..c406b15a 100644 --- a/plugins/wingman/.codex-plugin/plugin.json +++ b/plugins/wingman/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "wingman", - "version": "2.3.4", + "version": "2.4.0", "description": "StackHawk umbrella plugin \u2014 installs the default StackHawk skill set (hawkscan, api, data-seed, optimize) in one command for the combined hawk CLI.", "author": { "name": "StackHawk",