feat: select tests for non-Rust inputs via metadata.affected rules - #53
Conversation
cargo-affected selects tests by LLVM line-coverage overlap with the
diff. A test that reads a non-Rust file at runtime — an insta `.snap`,
a doc `.md` checked by a sync test, an `include_str!` target — has no
coverage row for that file, so editing it overlapped nothing and
selected no test. The change shipped but its guarding test never ran.
Close the gap with declarative rules in the manifest cargo-affected
already loads via `cargo metadata`:
[[workspace.metadata.affected.rule]]
globs = ["**/*.snap"]
filterset = "binary_id(=mycrate::integration)"
When a changed path matches a rule's globs, the rule's nextest filterset
is resolved with `cargo nextest list -E` and those tests are
force-selected. `[[package.metadata.affected.rule]]` covers single-crate
projects. The filterset speaks the full nextest filter language, so a
rule can target a binary, a test-name pattern, or any combination.
Rule selections form a disjoint `config` category in the summary and
JSON report (affected + config + new + stranded = selected), so they
never inflate the coverage-overlap counts.
Cache-neutrality: `[*.metadata]` is cargo's tool escape hatch and never
affects the build, so it's stripped from the manifest fingerprint.
Editing a rule no longer invalidates the coverage cache, making config
iteration free. Manifests without metadata still hash by raw bytes (no
churn on upgrade); only metadata-carrying manifests round-trip through
toml_edit, and only the metadata subtree is removed, so edits within it
leave the hash stable.
No rules means no behavior change and no extra `nextest list` call.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
cargo-affected-bot
left a comment
There was a problem hiding this comment.
One small correctness nit on user-facing strings, plus an offer.
The TABLE constant in src/config.rs hardcodes [workspace.metadata.affected], but it's interpolated into five different user-facing messages (parse error, invalid-glob error, builder error, filterset-resolution error, no-tests warning) that fire equally for rules loaded from [package.metadata.affected] via the single-crate fallback. A user on a single-crate project who hits any of these and greps their Cargo.toml for [workspace.metadata.affected] will come up empty. Suggested a one-line fix that mirrors the [*.metadata] shorthand already used in the README's "Input rules" section.
Separately, CLAUDE.md's architecture list doesn't yet mention src/config.rs. Happy to push a one-line addition to that entry if you'd like — there are several open doc-only PRs against CLAUDE.md already (#29, #36, #47), so I held off rather than racing them.
The `TABLE` constant is interpolated into five user-facing strings (the parse error, invalid-glob error, glob-builder error, filterset-resolution error, and no-tests warning), all of which fire equally for rules loaded from `[package.metadata.affected]` via the single-crate fallback. Hardcoding `[workspace.metadata.affected]` would send a single-crate user grepping for a table their `Cargo.toml` doesn't contain. Use the `[*.metadata.affected]` shorthand the README already uses. Also add `config.rs` to the CLAUDE.md architecture list and note the `[*.metadata]` fingerprint exclusion and the disjoint `config` selection category on the `fingerprint.rs` / `selection.rs` entries. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…3086) `cargo affected run` selects tests by Rust-line coverage overlap, so a file a test reads at runtime (never recorded in coverage) maps to no test and slips through. This adds `[workspace.metadata.affected]` rules so those inputs still select their guarding tests. ## Rules - **`tests/**/*.snap` → `binary_id(=worktrunk::integration)`**. insta snapshots are read at test time and never appear in coverage. - **`src/**/*.snap` → `binary_id(=worktrunk) | binary_id(=worktrunk::bin/wt)`**. The ~57 snapshots under `src/` belong to the lib and `wt`-binary unit tests (split 35 `worktrunk__*` / 19 `wt__*`). Routing by snapshot location is the finest granularity available without a snapshot→test map. - **README.md, `docs/**/*.md`, `dev/*.toml`, `src/cli/mod.rs`, `src/llm.rs`, `Taskfile.yaml`, `skills/**/*.md` → `test(/readme_sync/)`**. `tests/integration_tests/readme_sync.rs` reads each of these at runtime via `fs::read_to_string` and asserts the committed copies stay in sync; a change to any of them must rerun that module. `docs/demos/*.snap` are hyphen-named demo recordings, not insta test inputs, so the snapshot globs are scoped to `tests/` and `src/` to leave them out. ## Notes The rules are read by cargo-affected's input-rules feature ([max-sixty/cargo-affected#53](max-sixty/cargo-affected#53), released in 0.3.0); worktrunk's CI installs cargo-affected from its default branch, so this takes effect immediately. The `[*.metadata]` block is excluded from cargo-affected's fingerprint, so editing a rule is cache-neutral (no coverage re-collect). Each filterset was verified to resolve against the current suite via `cargo nextest list -E`. > _This was written by Claude Code on behalf of max_ --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
cargo-affected selects tests by LLVM line-coverage overlap with the diff, so a test that reads a non-Rust file at runtime (an insta
.snap, a doc.mdchecked by a sync test, aninclude_str!target) maps to no coverage row. Editing that file overlapped nothing and selected no test: the change shipped but its guarding test never ran.This adds declarative input→test rules, carried in the manifest cargo-affected already loads via
cargo metadata:When a changed path matches a rule's globs, the rule's nextest filterset is resolved with
cargo nextest list -Eand those tests are force-selected.[[package.metadata.affected.rule]]covers single-crate projects. The filterset speaks the full nextest filter language, so a rule can target a binary, a test-name pattern, or any combination.What to look at
src/config.rs(new) — reads and compiles rules from thecargo metadataJSON, resolves matched globs to concrete tests via the filterset. No rules → empty map and nonextest listcall, so a project without the table pays nothing.src/fingerprint.rs— strips[*.metadata]from the manifest hash so editing a rule doesn't invalidate the coverage cache. Metadata-free manifests still hash by raw bytes (no churn on upgrade); only metadata-carrying ones round-trip through toml_edit, and only the metadata subtree is removed.src/selection.rs— rule hits fold into a disjointconfigselection category (affected + config + new + stranded = selected), so they never inflate the coverage-overlap counts.Trade-off
Rules ride in the fingerprinted manifest, but the metadata exclusion makes rule edits cache-neutral. First-time addition of any metadata can move the fingerprint once; the stable guarantee is edit-invariance, covered by
manifest_metadata_excluded_from_fingerprint.Testing
Unit tests cover config parsing (including the
[package.metadata]fallback and malformed-table errors), glob compilation, and the fingerprint edit-invariance invariant. Three functional tests exercise the end-to-end miss→fix on a synthetic crate that reads a file at runtime, the committed-added-input path, and the inert-when-no-glob-matches case. Verified live on a real workspace: a.snapedit that previously selected 0 tests now selects the integration suite, and editing a rule keepscache=hit-exact.