Skip to content

feat(testing): nightly mutation lane over protocol session, parser boundaries, storage atomicity, and mirror state #336

Description

@cyanheads

Use case

Line coverage is high across the framework, but coverage measures execution, not assertion strength. Four subsystems carry logic where a passing suite over an incorrect implementation is plausible, because the tests assert shapes and happy paths more than they pin state transitions:

Subsystem Why mutation testing pays here
src/mcp-server/protocolSession.ts + sessionStore cancellation/capability restore Correctness is a cross-request state machine; an off-by-one in registration or an inverted session check still returns plausible results
src/utils/parsing/inputBudget.ts boundaries Every guard is a comparison; flipping > to >= changes the contract and passes most shape assertions
src/storage/core + provider atomicity Batch capacity accounting and partial-commit prevention are arithmetic that a wrong constant satisfies
src/services/mirror state transitions ready(), checkpointing, and interrupted-rebuild handling are exactly where a survived mutant maps to a real bug (see #286)

Proposed API

A nightly (not per-PR) mutation lane scoped to those four paths, with a survived-mutant budget rather than a score target.

  • Scope by file glob, never the whole src/ tree — a full-repo run is hours and mostly noise
  • Nightly schedule in .github/workflows/, separate from the CI matrix
  • Report survived mutants as the artifact; each one is triaged into either a new test or an accepted-mutant entry

Run it manually against the four paths first and read the surviving mutants before wiring any schedule. If the first run's survivors are dominated by equivalent mutants, the lane is not worth its runtime and this should close.

Alternatives considered

Option Why not
Raise coverage thresholds Already high on these paths; the gap is assertion strength, not execution
Per-PR mutation testing Runtime is far past what a PR gate tolerates, and the signal is not per-change
Property-based tests everywhere Already used where the state space is modelable (tests/fuzz/session-store.model.fuzz.test.ts); mutation testing answers a different question — whether existing assertions bind

Additional context

Related: #286 (mirror ready() stays true after an interrupted rebuild) is the kind of defect a survived mutant on the mirror state transitions would have surfaced.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestsurplus-token-ideaWorth exploring when token budget allows

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions