Skip to content

feat: select tests for non-Rust inputs via metadata.affected rules - #53

Merged
max-sixty merged 2 commits into
mainfrom
declarative-input-rules
Jun 15, 2026
Merged

feat: select tests for non-Rust inputs via metadata.affected rules#53
max-sixty merged 2 commits into
mainfrom
declarative-input-rules

Conversation

@max-sixty

Copy link
Copy Markdown
Owner

cargo-affected selects tests by LLVM line-coverage overlap with the diff, so a test that reads a non-Rust file at runtime (an insta .snap, a doc .md checked by a sync test, an include_str! target) maps to no coverage row. Editing that file overlapped nothing and selected no test: the change shipped but its guarding test never ran.

This adds declarative input→test rules, carried in the manifest cargo-affected already loads via cargo metadata:

[[workspace.metadata.affected.rule]]
globs = ["**/*.snap"]
filterset = "binary_id(=mycrate::integration)"

When a changed path matches a rule's globs, the rule's nextest filterset is resolved with cargo nextest list -E and those tests are force-selected. [[package.metadata.affected.rule]] covers single-crate projects. The filterset speaks the full nextest filter language, so a rule can target a binary, a test-name pattern, or any combination.

What to look at

  • src/config.rs (new) — reads and compiles rules from the cargo metadata JSON, resolves matched globs to concrete tests via the filterset. No rules → empty map and no nextest list call, so a project without the table pays nothing.
  • src/fingerprint.rs — strips [*.metadata] from the manifest hash so editing a rule doesn't invalidate the coverage cache. Metadata-free manifests still hash by raw bytes (no churn on upgrade); only metadata-carrying ones round-trip through toml_edit, and only the metadata subtree is removed.
  • src/selection.rs — rule hits fold into a disjoint config selection category (affected + config + new + stranded = selected), so they never inflate the coverage-overlap counts.

Trade-off

Rules ride in the fingerprinted manifest, but the metadata exclusion makes rule edits cache-neutral. First-time addition of any metadata can move the fingerprint once; the stable guarantee is edit-invariance, covered by manifest_metadata_excluded_from_fingerprint.

Testing

Unit tests cover config parsing (including the [package.metadata] fallback and malformed-table errors), glob compilation, and the fingerprint edit-invariance invariant. Three functional tests exercise the end-to-end miss→fix on a synthetic crate that reads a file at runtime, the committed-added-input path, and the inert-when-no-glob-matches case. Verified live on a real workspace: a .snap edit that previously selected 0 tests now selects the integration suite, and editing a rule keeps cache=hit-exact.

This was written by Claude Code on behalf of max

cargo-affected selects tests by LLVM line-coverage overlap with the
diff. A test that reads a non-Rust file at runtime — an insta `.snap`,
a doc `.md` checked by a sync test, an `include_str!` target — has no
coverage row for that file, so editing it overlapped nothing and
selected no test. The change shipped but its guarding test never ran.

Close the gap with declarative rules in the manifest cargo-affected
already loads via `cargo metadata`:

    [[workspace.metadata.affected.rule]]
    globs = ["**/*.snap"]
    filterset = "binary_id(=mycrate::integration)"

When a changed path matches a rule's globs, the rule's nextest filterset
is resolved with `cargo nextest list -E` and those tests are
force-selected. `[[package.metadata.affected.rule]]` covers single-crate
projects. The filterset speaks the full nextest filter language, so a
rule can target a binary, a test-name pattern, or any combination.

Rule selections form a disjoint `config` category in the summary and
JSON report (affected + config + new + stranded = selected), so they
never inflate the coverage-overlap counts.

Cache-neutrality: `[*.metadata]` is cargo's tool escape hatch and never
affects the build, so it's stripped from the manifest fingerprint.
Editing a rule no longer invalidates the coverage cache, making config
iteration free. Manifests without metadata still hash by raw bytes (no
churn on upgrade); only metadata-carrying manifests round-trip through
toml_edit, and only the metadata subtree is removed, so edits within it
leave the hash stable.

No rules means no behavior change and no extra `nextest list` call.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@cargo-affected-bot cargo-affected-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One small correctness nit on user-facing strings, plus an offer.

The TABLE constant in src/config.rs hardcodes [workspace.metadata.affected], but it's interpolated into five different user-facing messages (parse error, invalid-glob error, builder error, filterset-resolution error, no-tests warning) that fire equally for rules loaded from [package.metadata.affected] via the single-crate fallback. A user on a single-crate project who hits any of these and greps their Cargo.toml for [workspace.metadata.affected] will come up empty. Suggested a one-line fix that mirrors the [*.metadata] shorthand already used in the README's "Input rules" section.

Separately, CLAUDE.md's architecture list doesn't yet mention src/config.rs. Happy to push a one-line addition to that entry if you'd like — there are several open doc-only PRs against CLAUDE.md already (#29, #36, #47), so I held off rather than racing them.

Comment thread src/config.rs Outdated
The `TABLE` constant is interpolated into five user-facing strings (the
parse error, invalid-glob error, glob-builder error, filterset-resolution
error, and no-tests warning), all of which fire equally for rules loaded
from `[package.metadata.affected]` via the single-crate fallback.
Hardcoding `[workspace.metadata.affected]` would send a single-crate user
grepping for a table their `Cargo.toml` doesn't contain. Use the
`[*.metadata.affected]` shorthand the README already uses.

Also add `config.rs` to the CLAUDE.md architecture list and note the
`[*.metadata]` fingerprint exclusion and the disjoint `config` selection
category on the `fingerprint.rs` / `selection.rs` entries.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@max-sixty
max-sixty merged commit d48d261 into main Jun 15, 2026
9 checks passed
@max-sixty
max-sixty deleted the declarative-input-rules branch June 15, 2026 03:13
max-sixty added a commit to max-sixty/worktrunk that referenced this pull request Jun 15, 2026
…3086)

`cargo affected run` selects tests by Rust-line coverage overlap, so a
file a test reads at runtime (never recorded in coverage) maps to no
test and slips through. This adds `[workspace.metadata.affected]` rules
so those inputs still select their guarding tests.

## Rules

- **`tests/**/*.snap` → `binary_id(=worktrunk::integration)`**. insta
snapshots are read at test time and never appear in coverage.
- **`src/**/*.snap` → `binary_id(=worktrunk) |
binary_id(=worktrunk::bin/wt)`**. The ~57 snapshots under `src/` belong
to the lib and `wt`-binary unit tests (split 35 `worktrunk__*` / 19
`wt__*`). Routing by snapshot location is the finest granularity
available without a snapshot→test map.
- **README.md, `docs/**/*.md`, `dev/*.toml`, `src/cli/mod.rs`,
`src/llm.rs`, `Taskfile.yaml`, `skills/**/*.md` →
`test(/readme_sync/)`**. `tests/integration_tests/readme_sync.rs` reads
each of these at runtime via `fs::read_to_string` and asserts the
committed copies stay in sync; a change to any of them must rerun that
module.

`docs/demos/*.snap` are hyphen-named demo recordings, not insta test
inputs, so the snapshot globs are scoped to `tests/` and `src/` to leave
them out.

## Notes

The rules are read by cargo-affected's input-rules feature
([max-sixty/cargo-affected#53](max-sixty/cargo-affected#53),
released in 0.3.0); worktrunk's CI installs cargo-affected from its
default branch, so this takes effect immediately. The `[*.metadata]`
block is excluded from cargo-affected's fingerprint, so editing a rule
is cache-neutral (no coverage re-collect). Each filterset was verified
to resolve against the current suite via `cargo nextest list -E`.

> _This was written by Claude Code on behalf of max_

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants