Skip to content

Commit 3bfcab6

Browse files
bomly-guyclaude
andcommitted
docs: add Feature Checklist to CLAUDE.md and AGENTS.md
Capture the follow-ups that have come up across every reachability commit so far so future feature work doesn't need them re-requested. The checklist covers seven surfaces — CLI, MCP, plugin command, logging, caching, smoke tests, and documentation — with concrete file paths and what to do at each one. Specific patterns codified: - MCP: every new flag on scan/explain/diff must be added to the matching ScanRequest/ExplainRequest/DiffRequest plus the registerXxxTool wiring in internal/mcp, and the mcpOptionsAdapter.cloneWithOverrides path in internal/cli. - Plugin command: a new component class needs PluginKind constant, Manifest field + clone helper, pluginKindFilter entry, --<kind>s flag, builtInPluginInfos iteration, descriptor accessors, and renderPluginListTables / renderPluginInfo rendering. - Logging: INFO at boundaries (stage start, per-unit completion, final summary), DEBUG for command lines + cache decisions + byte counts, WARN never aborts. - Caching: schema version + input fingerprint + runtime version + runner name in the key, 24h default TTL, non-fatal failures, CacheDir/CacheTTL/DisableCache fields on the component. - Smoke tests: real public repo pinned via --url --ref, never local Go/npm/etc. modules under test/smoke/testdata; update normalizeJSON for new volatile fields; regenerate goldens. - Documentation: make generate, feature page under docs/, ARCHITECTURE.md decision-log, CLAUDE.md / AGENTS.md package map updates. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
1 parent 2a367df commit 3bfcab6

2 files changed

Lines changed: 140 additions & 0 deletions

File tree

AGENTS.md

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -137,6 +137,76 @@ Core passes these env vars. Plugin discovery: `~/.bomly/plugins/bomly-*` overrid
137137
- Fake binaries (npm, go, Gradle, plugin) are built in `TestMain` — see `internal/cli/root_test_main_test.go`.
138138
- No test conditionally skipped without a recorded reason.
139139

140+
## Feature Checklist
141+
142+
When adding a new user-visible feature (new CLI flag, new component class, new pipeline stage, new analyzer, etc.), walk this checklist before requesting review. The surfaces forgotten most often are **MCP**, **plugin command**, and **smoke test**.
143+
144+
### CLI surface
145+
146+
- Flag declared in `internal/cli/opts/flag_options.go` with override propagation in `applyFlagOverrides`.
147+
- Config field added to `internal/config/config.go` `Resolved` (with `doc:`/`env:`/`default:` tags) and `File` (with `yaml:` tag and matching pointer/slice shape).
148+
- Flag interactions (requires / conflicts / modifies semantics) get a check in `config.Validate` plus a unit test in `internal/config/validate_test.go`. Error messages must be actionable (`"--audit requires --enrich"`, not `"invalid combination"`).
149+
- If the flag drives a pipeline stage, propagate the value through `internal/cli/opts/options.go`'s `PipelineRequest` builder.
150+
- If the flag accepts a selector list, register an `available<Thing>Options` helper in `flag_options.go` for shell completion.
151+
152+
### MCP
153+
154+
Every new flag on `bomly scan` / `bomly explain` / `bomly diff` must be reachable from the matching MCP tool. AI agents won't get the feature otherwise.
155+
156+
- Add the field to `ScanRequest` / `ExplainRequest` / `DiffRequest` in `internal/mcp/server.go`.
157+
- Register the `mcplib.WithBoolean` / `WithString` argument in `tool_scan.go` / `tool_explain.go` / `tool_diff.go`. Mirror the CLI flag's help text and call out any prerequisite ("requires enrich").
158+
- Wire the field through `mcpOptionsAdapter` in `internal/cli/mcp_cmd.go`. Add it to the `mcpOverrides` struct (so future additions stay one-line) and apply it in `cloneWithOverrides`.
159+
160+
### Plugin command
161+
162+
When adding a new component class (a new sibling of Detector / Matcher / Auditor / Analyzer):
163+
164+
- Add a `PluginKind*` constant in `sdk/plugin.go` and accept it in `sdk/validate.go::ValidateMetadata`.
165+
- Add the descriptor pointer to `internal/plugin/types.go::Manifest` plus a `clone<Kind>Descriptor` helper that deep-copies every slice field.
166+
- In `internal/cli/plugin_cmd.go`:
167+
- Extend `pluginKindFilter` and add a `--<kind>s` filter flag.
168+
- Iterate the new descriptors in `builtInPluginInfos`; emit one `PluginInfo` per registered instance.
169+
- Add a `<kind>PluginInfo` constructor and the matching local clone helper.
170+
- Extend `pluginInfoEcosystems`, `pluginInfoPackageManagers`, and `pluginInfoFeatures` with the new case.
171+
- Add a new section to `renderPluginListTables` with sensible columns. If the descriptor exposes axes the existing kinds don't (e.g. analyzers have `SupportedLanguages`), add new columns and corresponding `pluginInfo<X>` / `join<X>` helpers.
172+
- Update `renderPluginInfo` to emit any new lines when present.
173+
174+
External plugin install/load (gRPC handshake, runtime descriptor fetch) is a separate, larger change and can land in a follow-up PR. Built-in listing is the minimum bar.
175+
176+
### Logging
177+
178+
Analyzers, matchers, auditors, and any new long-running stage must be observable at `-v` (INFO) and debuggable at `-vv` (DEBUG):
179+
180+
- **INFO** at natural boundaries: stage start (with key inputs — module count, item count, runner name, cache enabled), per-major-unit completion (cache hit/miss, counts per outcome, duration), final summary (totals, overall duration).
181+
- **DEBUG** for low-level detail: discovered inputs (module roots, manifest paths), exact command lines including args and working dir, cache key components, byte counts of subprocess output, branch decisions worth reproducing.
182+
- **WARN** for recoverable errors (analyzer failed, cache write failed). Never abort the pipeline for these; degrade and continue.
183+
184+
When invoking subprocesses, the DEBUG line MUST include the binary path, args, and working dir so a user with `-vv` can copy/paste the command to reproduce outside Bomly.
185+
186+
### Caching
187+
188+
If a new analyzer / matcher / detector produces deterministic output for a fixed `(input, schema version)` pair, wrap it with `internal/matchers/cache.FileCache`:
189+
190+
- Cache key folds: schema version (so we can bump and invalidate), input fingerprint (lockfile content hash), runtime version when the underlying tool is sensitive to it, and the runner name when multiple implementations exist.
191+
- Default location: `~/.cache/bomly/<area>/<subarea>/`.
192+
- Default TTL: 24h (matches OSV / EOL).
193+
- Cache failures are non-fatal — log a warning and proceed.
194+
- Expose `CacheDir`, `CacheTTL`, and `DisableCache` fields on the component for tests + opt-out.
195+
196+
### Smoke tests
197+
198+
- Use a **real public repo** pinned to a specific tag or commit SHA via `--url --ref`. Do not add local Go modules / npm packages / etc. under `test/smoke/testdata/`. The only acceptable testdata files are SBOM fixtures and similar non-project inputs.
199+
- The pinned ref must exercise the feature meaningfully. For reachability that means a repo with at least one symbol-tier reachable advisory; for a new ecosystem detector that means a repo whose lockfile actually parses.
200+
- Update `test/smoke/helpers_test.go::normalizeJSON` (or the more specific normalizers it calls) to scrub any new volatile fields (timestamps, line numbers, file paths under temp clone dirs) before they reach goldens.
201+
- Run `make smoke ARGS="-update"` to regenerate goldens. Commit the regenerated `.golden.json` in the same PR.
202+
203+
### Documentation
204+
205+
- `make generate` regenerates `docs/CONFIG_REFERENCE.md`, `docs/schemas/*`, and `docs/SUPPORT_MATRIX.md` from struct tags. Run it whenever `internal/config/config.go`, `internal/output/*`, or `sdk/catalog.go` / `sdk/support_matrix.go` change.
206+
- Add or update a feature page under `docs/` (e.g. `docs/REACHABILITY.md`) with quick-start usage, semantics, ecosystem coverage, output shape, and limitations. Be explicit about safety caveats (e.g. "tier-3 unreachable does not mean safe").
207+
- `docs/ARCHITECTURE.md`: update the pipeline diagram if the stage list changed; add a decision-log entry for non-obvious design choices.
208+
- `CLAUDE.md` and `AGENTS.md`: update the architecture tree and package-boundary list when introducing a new internal package.
209+
140210
## Reference Docs
141211

142212
| Doc | Covers |

CLAUDE.md

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -127,6 +127,76 @@ Cache failures are non-fatal — log a warning and continue.
127127

128128
**Testing helpers**: `t.TempDir()`, `testutil.BuildGoBinary()`, `httptest.NewServer()`. Shared fake-binary setup lives in `internal/cli/root_test_main_test.go`. No tests may be conditionally skipped without a recorded reason.
129129

130+
## Feature Checklist
131+
132+
When adding a new user-visible feature (new CLI flag, new component class, new pipeline stage, new analyzer, etc.), walk this checklist before requesting review. Reviewers will ask for everything that applies, and the surface that gets forgotten most often is **MCP** + **plugin command** + **smoke test**.
133+
134+
### CLI surface
135+
136+
- [ ] Flag declared in `internal/cli/opts/flag_options.go` with override propagation in `applyFlagOverrides`.
137+
- [ ] Config field added to `internal/config/config.go` `Resolved` (with `doc:`/`env:`/`default:` tags) and `File` (with `yaml:` tag and matching pointer/slice shape).
138+
- [ ] When the flag interacts with another flag (requires it, conflicts with it, modifies its semantics), add a check to `config.Validate` and a unit test in `internal/config/validate_test.go`. Keep validation errors actionable (`"--audit requires --enrich"`, not `"invalid combination"`).
139+
- [ ] If the flag drives a pipeline stage, propagate the value through `internal/cli/opts/options.go`'s `PipelineRequest` builder.
140+
- [ ] Shell completion: register an `available<Thing>Options` helper in `flag_options.go` if the flag accepts a selector list.
141+
142+
### MCP
143+
144+
Every new flag on `bomly scan` / `bomly explain` / `bomly diff` must be reachable from the matching MCP tool. AI agents won't get the feature otherwise.
145+
146+
- [ ] Add the field to `ScanRequest` / `ExplainRequest` / `DiffRequest` in `internal/mcp/server.go`.
147+
- [ ] Register the `mcplib.WithBoolean` / `WithString` argument in `tool_scan.go` / `tool_explain.go` / `tool_diff.go` with a description that mirrors the CLI flag's help text. If the flag requires another flag, say so in the description ("requires enrich").
148+
- [ ] Wire the field through the `mcpOptionsAdapter` in `internal/cli/mcp_cmd.go`. Add it to `mcpOverrides` (single struct; no positional-arg churn) and apply it in `cloneWithOverrides`.
149+
150+
### Plugin command
151+
152+
When adding a new component class (a new sibling of Detector / Matcher / Auditor / Analyzer):
153+
154+
- [ ] Add a `PluginKind*` constant in `sdk/plugin.go` and accept it in `sdk/validate.go::ValidateMetadata`.
155+
- [ ] Add the descriptor pointer to `internal/plugin/types.go::Manifest` and a `clone<Kind>Descriptor` helper that deep-copies every slice field.
156+
- [ ] In `internal/cli/plugin_cmd.go`:
157+
- Extend `pluginKindFilter` with the new kind plus a `--<kind>s` filter flag.
158+
- Iterate the new descriptors in `builtInPluginInfos` and emit one `PluginInfo` per registered instance.
159+
- Add a `<kind>PluginInfo` constructor and the matching local clone helper.
160+
- Extend `pluginInfoEcosystems`, `pluginInfoPackageManagers`, and `pluginInfoFeatures` with the new case.
161+
- Add the new section to `renderPluginListTables` with sensible columns. If the descriptor exposes axes the existing kinds don't (e.g. analyzers have `SupportedLanguages`), add new columns and corresponding `pluginInfoLanguages` / `joinLanguages` helpers.
162+
- Update `renderPluginInfo` to emit any new "Languages" / "Tiers" / etc. lines when present.
163+
164+
External plugin install/load (gRPC handshake, runtime descriptor fetch) is a separate, larger change and can land in a follow-up PR. Built-in listing is the minimum bar.
165+
166+
### Logging
167+
168+
Analyzers, matchers, auditors, and any new long-running stage must be observable at `-v` (INFO) and debuggable at `-vv` (DEBUG). The expected pattern:
169+
170+
- **INFO** at the natural boundaries: stage start (with key inputs — module count, item count, runner name, cache enabled), per-major-unit completion (cache hit/miss, counts per outcome, duration), final summary (totals, overall duration).
171+
- **DEBUG** for low-level detail: discovered inputs (module roots, manifest paths), exact command lines including args and working dir, cache key components, byte counts of subprocess output, branch decisions worth reproducing.
172+
- **WARN** for recoverable errors (analyzer failed, cache write failed). Never abort the pipeline for any of these; degrade and continue.
173+
174+
When invoking subprocesses, the DEBUG line MUST include the binary path, args, and working dir so a user with `-vv` can copy/paste the command to reproduce outside Bomly.
175+
176+
### Caching
177+
178+
If a new analyzer / matcher / detector produces deterministic output for a fixed `(input, schema version)` pair, wrap it with `internal/matchers/cache.FileCache`:
179+
180+
- Cache key folds: schema version (so we can bump and invalidate), input fingerprint (lockfile content hash), runtime version when the underlying tool is sensitive to it, and the runner name when multiple implementations exist.
181+
- Default location: `~/.cache/bomly/<area>/<subarea>/`.
182+
- Default TTL: 24h (matches OSV / EOL).
183+
- Cache failures are non-fatal — log a warning and proceed.
184+
- Expose `CacheDir`, `CacheTTL`, and `DisableCache` fields on the component for tests + opt-out.
185+
186+
### Smoke tests
187+
188+
- Use a **real public repo** pinned to a specific tag or commit SHA via `--url --ref`. Do not add local Go modules / npm packages / etc. under `test/smoke/testdata/`. The only acceptable testdata files are SBOM fixtures and similar inputs that aren't full project trees.
189+
- The pinned ref must exercise the feature meaningfully. For reachability that means a repo with at least one symbol-tier reachable advisory; for a new ecosystem detector that means a repo whose lockfile actually parses.
190+
- Update `test/smoke/helpers_test.go::normalizeJSON` (or the more specific normalizers it calls) to scrub any new volatile fields (timestamps, line numbers, file paths under temp clone dirs) before they reach goldens.
191+
- Run `make smoke ARGS="-update"` to regenerate goldens. Commit the regenerated `.golden.json` in the same PR.
192+
193+
### Documentation
194+
195+
- [ ] `make generate` regenerates `docs/CONFIG_REFERENCE.md`, `docs/schemas/*`, and `docs/SUPPORT_MATRIX.md` from struct tags. Run it whenever `internal/config/config.go`, `internal/output/*`, or `sdk/catalog.go` / `sdk/support_matrix.go` change.
196+
- [ ] Add or update a feature page under `docs/` (e.g. `docs/REACHABILITY.md`) with quick-start usage, semantics, ecosystem coverage, output shape, and limitations. Be explicit about safety caveats (e.g. "tier-3 unreachable does not mean safe").
197+
- [ ] `docs/ARCHITECTURE.md`: update the pipeline diagram if the stage list changed; add a decision-log entry for non-obvious design choices.
198+
- [ ] `CLAUDE.md` and `AGENTS.md`: update the architecture tree and package-boundary list when introducing a new internal package.
199+
130200
## Release
131201

132202
Draft releases are created automatically after merges to `main` based on commit message prefixes:

0 commit comments

Comments
 (0)