Skip to content

test(vitest): retire the subprocess-stub kill-criterion experiment - #2255

Merged
thymikee merged 3 commits into
mainfrom
claude/vitest-retire-1823-experiment
Sep 3, 2026
Merged

test(vitest): retire the subprocess-stub kill-criterion experiment#2255
thymikee merged 3 commits into
mainfrom
claude/vitest-retire-1823-experiment

Conversation

@thymikee

@thymikee thymikee commented Sep 3, 2026

Copy link
Copy Markdown
Member

Summary

Retires #1823's subprocess-stub kill-criterion experiment after its observation window completed without timeout-shaped failures.

The former project is now absent from both configuration and source commentary. Mutation exclusions are named for their current purpose (MUTATION_EXCLUDED_TESTS), and the gate-model fixture mirrors the active fuzz-worker project rather than preserving the retired topology.

The affected real-subprocess tests continue to run normally in unit-core and remain excluded from mutation execution. No runtime behavior changes.

Validation

  • Tested exact commit dc8102909a.
  • pnpm check:affected --run passed every runnable local check for the cleanup; the final fixture correction passed the focused 7-test gate-model suite.
  • GitHub-authoritative device, coverage, and native lanes are running on the exact head.
  • Repository search finds no remaining subprocess-stub or SUBPROCESS_STUB_TESTS references.
  • No planted-red proof applies: this is a behavior-preserving ownership rename and removal of retired commentary.

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 2.59 MB 2.59 MB 0 B
JS gzip 871.3 kB 871.3 kB 0 B
npm bundled raw 2.59 MB 2.59 MB 0 B
npm bundled gzip 871.3 kB 871.3 kB 0 B
npm tarball 1.02 MB 1.02 MB +7 B
npm unpacked 3.47 MB 3.47 MB 0 B
npm clean-installed 3.47 MB 3.47 MB 0 B

npm unpacked components

Component Base Current Diff
JS / dist source 2.75 MB 2.75 MB 0 B
Apple runner source/project 544.4 kB 544.4 kB 0 B
Apple snapshot presentation source 33.8 kB 33.8 kB 0 B
macOS helper source 54.8 kB 54.8 kB 0 B
Android helper artifacts 43.3 kB 43.3 kB 0 B
Other package files 46.3 kB 46.3 kB 0 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 31.2 ms 30.5 ms -0.6 ms
CLI --help 80.1 ms 80.3 ms +0.2 ms

Top changed chunks: no changes in the largest emitted chunks.

Top changed packed files

No changed packed files.

@thymikee

thymikee commented Sep 3, 2026

Copy link
Copy Markdown
Member Author

BLOCKED at 6154523e24efebe25863ec21580c24238d0d35c0: docs/agents/testing.md still describes #1823 as an ongoing 20-run revert-on-failure experiment, contradicting this PR's resolved vitest.config.ts guidance. Update it to state the kill criterion was met, the files now run unserialized in unit-core, and the list remains solely for mutation-lane exclusion. No other finding: exact-head CI is green, merge state clean, no behavioral change or size impact.

@thymikee

thymikee commented Sep 3, 2026

Copy link
Copy Markdown
Member Author

Addressed the BLOCKED finding at 6154523.

docs/agents/testing.md contradicted the resolved vitest.config.ts guidance — rewrote the
"Real-subprocess-spawn tests" section: it no longer describes an ongoing 20-run revert-on-failure
experiment. It now states the #1823 kill criterion was met, SUBPROCESS_STUB_TESTS members run
un-serialized in unit-core's default forks pool, and the list survives solely as the
mutation-lane exclusion (SERIALIZED_TESTS) — matching the vitest.config.ts comment's language.

Checked PR #2247 (claude/docs-pr-rules-validation-lifecycle, head 72b7564), which also touches
this section: its copy still says "reverted if a timeout-shaped failure appears within 20
consecutive CI runs," i.e. it has not resolved this yet either, so there was no existing wording to
match byte-for-byte.

Validation: pnpm check:agent-guidance and pnpm format:check pass. No planted red — this is a
documentation-wording fix only.

Head: 3bdb15c. The full affected-gate result for this exact head
follows from the serialized gate stage.

@thymikee

thymikee commented Sep 3, 2026

Copy link
Copy Markdown
Member Author

Follow-up at 5bef6fa: dropped the whole "Real-subprocess-spawn tests" section from docs/agents/testing.md.

That section was only restating declaration-site comments (SUBPROCESS_STUB_TESTS / SERIALIZED_TESTS in vitest.config.ts, mutation exclusion in scripts/mutation/test-scope.ts). Once #1823's kill criterion is met, the files are ordinary unit-core members — testing.md has no remaining trap that is not already owned there. The "no unit-test retry layer" sentence went with it; that policy is not subprocess-specific and was not encoded as a gate.

pnpm check:agent-guidance and pnpm format:check are green on this head. Docs-only deletion, no planted red.

@thymikee

thymikee commented Sep 3, 2026

Copy link
Copy Markdown
Member Author

BLOCKED at 5bef6fa10d9989d5a8f3a7f49128df29f0ebdfb1. The stale subprocess-stub experiment text is fixed and CI is clean, but deleting the whole section also deletes the independent normative rule: "There is no unit-test retry layer — fix or remove flakes." That policy is not subprocess-specific; restore it under the general Speed rules or its actual owner. No other finding.

@thymikee

thymikee commented Sep 3, 2026

Copy link
Copy Markdown
Member Author

Resolved the remaining blocker at f6123387ce84c46b6f914042d0fb94088ce66da7: restored the independent no-unit-retry policy under general Speed rules while keeping the obsolete subprocess experiment section deleted. Fresh install/build, formatting, and the full local pnpm check:affected --run all pass. Exact-head GitHub checks are running.

#1823's kill criterion was met (~64 consecutive genuine Coverage-job
completions since dbc4f2f with zero timeout-shaped failures), so
the subprocess-stub project is gone for good rather than mid-experiment.
Rewrite the vitest.config.ts comments to state that resolved outcome
instead of framing it as an ongoing revert-on-first-failure trial.
@thymikee
thymikee force-pushed the claude/vitest-retire-1823-experiment branch from f612338 to 73169fb Compare September 3, 2026 13:14
@thymikee

thymikee commented Sep 3, 2026

Copy link
Copy Markdown
Member Author

Rebased onto current main after #2247 and #2242 merged. Their upstream documentation now owns the durable no-retry and evidence policies, so the redundant follow-up commits dropped cleanly. PR #2255 is again a single declaration-site comment change in vitest.config.ts. Rebased head: 73169fb3770b19458e88c950a2bc60de3aaac93c. Full local pnpm check:affected --run passes on this exact tree; fresh GitHub checks are running.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Sep 3, 2026
@thymikee

thymikee commented Sep 3, 2026

Copy link
Copy Markdown
Member Author

READY at 73169fb377c88bcab8ed84d5d7dfa4ea8b922d09. The declaration-site wording is current, the obsolete duplicated guidance is now upstream via #2247, full local affected validation passed, all exact-head GitHub checks including iOS/Android/macOS/Linux smoke and coverage are green, merge state is clean, and ready-for-human is applied. No remaining finding.

@thymikee

thymikee commented Sep 3, 2026

Copy link
Copy Markdown
Member Author

Removed the retired project from the source vocabulary at bd6f2e2c23: renamed the mutation exclusion seam for what it owns today, removed the historical topology comments, and generalized the gate-model fixture. Repository search now finds zero subprocess-stub / SUBPROCESS_STUB_TESTS traces. pnpm check:affected --run passed every runnable local check; exact-head GitHub CI is running.

@thymikee

thymikee commented Sep 3, 2026

Copy link
Copy Markdown
Member Author

Follow-up at dc8102909a: removed the synthetic extra-project name. The gate-model fixture now uses the real active fuzz-worker project and exactly mirrors the current coverage split (--project=!fuzz-worker plus test:fuzz-worker). Focused gate-model suite: 7/7 passed.

@thymikee
thymikee merged commit e3c44ea into main Sep 3, 2026
20 of 21 checks passed
@thymikee
thymikee deleted the claude/vitest-retire-1823-experiment branch September 3, 2026 15:34
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-09-03 15:35 UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant