cltest: share one go-build cache across captured runs#2035
Open
cpunion wants to merge 1 commit into
Open
Conversation
Every RunAndCapture created a fresh temp GOCACHE and deleted it after, so each of the ~170 golden-dir run tests re-typechecked the entire dependency graph from scratch. Share one per-process temp cache instead: isolation from the developer cache is unchanged (still a fresh temp dir per test process), and the go build cache is content+flag keyed, so sharing across build configs is safe. Measured on darwin/arm64 (controlled A/B, run-mode _testrt suite, 63 dirs): 157.8s -> 119.6s (-24%). Full ./cl and ./ssa suites green with the change. The win should be larger on CI mac runners, where the per-test cache churn hits slow disks; the mac coverage job's cl package alone runs 37-45 minutes there.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This was referenced Jul 6, 2026
6716a99 to
4d90875
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI-time investigation follow-up: the stage-5 branches' mac coverage job was running the
clpackage at 37-45 minutes, right at the 45m go-test alarm.What this PR does
RunAndCapturecreated a fresh tempGOCACHEper test and deleted it afterwards, so each of the ~170 golden-dir run tests re-typechecked the whole dependency graph from scratch. This shares one per-process temp cache instead:sync.Oncepackage state) and tests inside a binary run serially, so no new concurrent access surface is introduced across the parallel test processes;Measurements (single-variable A/B: this one-file diff vs its parent, no
-pflags)./cland./ssasuites green with the change; this PR's CI is green twice (pre- and post-scope-trim).What was considered and rejected first
ssaTestFrom*suites as duplicates ofcl'sTestRunAndTestFrom*: rejected — they verify the same golden specs, but per-package coverage attribution means they carry thessapackage's coverage: 87.5% with them, 51.3% without (measured).t.Parallel()on the golden subtests: rejected for now —cltest's capture path swapsos.Stdout/Stderr,os.Chdirs, and mutatesGOCACHEglobally, and LLVM installs in-process signal handlers; under a cold cache this reproducibly fails (including a hardfatal error: semasleep on Darwin signal stack). Parallelizing needs a capture refactor (explicitcmd.Dir/Env/Stdout), tracked separately.Scope note
This PR is the cache sharing only. On the much heavier stage-5 branches, warm caches let the four heavy test packages' compile phases overlap fully, and the concurrent clang/lld burst killed 16GB ubuntu runners mid-run; those branches carry a
go test -p 2mitigation locally. That cap is deliberately NOT part of this PR — it lowers the collision probability rather than provably removing the peak; the structural fix (splitting heavy packages into separate CI jobs) is the tracked follow-up if the deaths reappear.🤖 Generated with Claude Code