Skip to content

[CI] perf(tests): reuse Vitest transforms between test runs - #26095

Merged
nwparker merged 5 commits into
mainfrom
nwparker/vitest-transform-cache
Oct 7, 2026
Merged

nwparker merged 5 commits into
mainfrom
nwparker/vitest-transform-cache

Conversation

@nwparker

@nwparker nwparker commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

ELI5

Repeated local test runs can reuse Vitest's transformed modules instead of rebuilding them. A frozen 997-file comparison took 69.31 seconds with caching disabled and 58.07 seconds with a warmed cache: 16.21% less time.

What Changed

Enable the supported fsModuleCache option in the existing shared test options, covering Bun, Node runtime contracts, the isolated measurement project, and the stock Node fallback. Test isolation, setup files, mock execution, worker limits, runtime routing, assertions, and deadlines stay the same.

Why

The measured sample evaluates about 12,600 modules roughly 354,000 times across isolated test files. Vitest can persist transform results without sharing evaluated module state or weakening isolation. This is a one-line use of Vitest's supported cache, avoiding a custom loader or a broad migration to another test API.

The improvement below is for repeated local runs. CI currently has no shared transform-cache restore, so this PR makes no CI or full-suite speed claim. The requested 2× full-suite target remains unmet.

Linked Issue

Maintainer-requested test performance work; no separate issue.

Visual Proof

N/A — test configuration only; no product UI or interaction change.

Testing

The measurement source was frozen at 0acf039b5d5050e4f40b2c247944cfc7ee652c96 plus exactly this cache setting. Both arms used identical source/configuration, 16 workers, the same 997 file arguments, reporter, launcher, and whole-command timing. Explicit CLI overrides selected cache off/on. Three pairs ran in alternating order; no timed run was discarded.

Pair Cache disabled Warmed cache Time reduction
1 70.707 s 58.016 s 17.95%
2 67.775 s 58.939 s 13.04%
3 69.439 s 57.257 s 17.54%
Mean 69.307 s 58.071 s 16.21%

The separately recorded empty-cache priming run passed in 67.006 s; it is not part of the warmed mean. Every run retained all 997 files and 9,713 cases: 9,642 passed, 71 existing skips, with no failures or lifecycle errors. Exact case names/statuses matched after normalizing only the generated temporary-directory suffix in two existing parameterized OpenCode worker titles; all raw reports were retained. Resolved cache flags were verified for every project.

The representative cache contained 15,804 files / 301.3 MiB. Vitest has no automatic size eviction. Developers can disable it with --fsModuleCache=false or clear it with pnpm exec vitest --config config/vitest.config.ts --clearCache. The cache is ignored, stays inside node_modules for normal use, and adds no remote cache trust or packaged application content.

  • Locally measured on macOS with the repository's pinned Bun and Vitest versions.
  • Ten runtime-focused files / 55 cases passed with cache enabled through both the Bun coordinator and stock Node, with identical case/status maps; includes real SQLite workers, PTY shutdown and process supervision.
  • Final integrated head 80f072895284 passes full pnpm tc (Node, CLI and web), changed-code quality checks, and formatting.
  • On final integrated head 80f072895284, six invalidation/isolation stages passed through both coordinators: 168 checks across 12 invocations. Reused one owned cache; required actual warm-cache read evidence. Source edits, define/alias changes, fresh environment-driven mock execution, changed mock-factory source, and file-to-index resolution relocation all observed the new values. Actual Bun/Node identities and fresh mutable module state were asserted. No tracked source changed during this probe.
  • Full PR CI and final review. Head 80f072895284 integrates main’s equivalent fixture repair fix(native-chat): repair main's web typecheck after the @-mention prop rename #26097. The duplicate [CI] test(native-chat): update the resume fixture for mention menus #26102 is closed without merging; conflict resolution kept main’s exact fixture bytes. The diff against integrated main is again only the cache option; fresh CI qualified this source. The first CI attempt stopped at that inherited compile error before unit shards ran; no unit success is inferred from it.
  • No implementation-mirroring test added for a one-line dependency option; qualification exercises existing cases and an ignored cache invalidation probe.

Final full CI run 37584606858 checked out 98e375b7326e (parents integrated main c8dcffce and final head 80f072895284), with all tested source bytes equal to the final head. All five actual assignment artifacts cover 11,407 physical files exactly once: 11,272 passed, 135 existing skips, zero failed/unhandled/missing/duplicated. Case totals are 110,641 passed, one expected failure and 1,035 existing skips. Static/typechecking, relay integration, headless persistence, selection evidence and aggregate verification passed. Final review has no unresolved threads. This establishes correctness coverage, not a CI-speed comparison.

Review

Independent source review found no change to isolation, Node/Bun routing, setup, or packaging. Cached JavaScript has the same local trust boundary as installed dependencies; this does not authenticate third-party cache artifacts. Linux is covered by PR CI; Windows filesystem-locking behavior is not locally measured.

Agent skill upstream boundary

  • Not applicable; no upstream skill content copied.

Checklist

  • This PR is small and focused.
  • Explained behavior, mechanism, alternative choice, measurements and limits.
  • N/A visual proof explained above.
  • Self-reviewed for correctness, security and performance.
  • Cross-platform and SSH impact considered: only the local test runner changes; no runtime or wire protocol change.
  • Full CI checks pass; local targeted checks above passed.

@coderabbitai

coderabbitai Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 086454ea-b6aa-4ae4-81a2-ff3ba0cb08f1
📥 Commits

Reviewing files that changed from the base of the PR and between a3ecee0 and 099cbb1.

📒 Files selected for processing (1)
  • config/vitest.config.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 5 remain after this review.


📝 Walkthrough

Walkthrough

The shared Vitest test options now set fsModuleCache to true.

Priority: ⬇️ Low

Merge Risk: ⚪ Minimal · up to 099cb

This turns on Vitest's filesystem module cache so repeated local test runs go faster. The cache can be disabled or cleared if it ever behaves unexpectedly. I found no concrete risk, so it is low risk to merge once CI passes.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 1…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely describes the main change: reusing Vitest transforms between test runs.
Description check ✅ Passed The description covers the template’s key sections and explains the change, rationale, measurements, testing, visual-proof status, and compatibility considerations. It identifies the missing issue as …
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

…form-cache

# Conflicts:
#	src/renderer/src/components/native-chat/native-chat-composer-field-resume.test.tsx
@nwparker
nwparker merged commit b6f3428 into main Oct 7, 2026
30 checks passed
@nwparker nwparker changed the title perf(tests): reuse Vitest transforms between test runs [CI] perf(tests): reuse Vitest transforms between test runs Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant