perf: send only cache misses to the classloader-isolated workers. - #1804
perf: send only cache misses to the classloader-isolated workers.#1804ianbrandt wants to merge 1 commit into
Conversation
Isolating kotlin-metadata to workers meant `InMemoryCache` contents had to cross a classloader boundary, so `ExplodeJarTask` and `FindKotlinMagicTask` serialized every cache hit into the worker as a JSON seed and the worker parsed it back into a deep copy. A `noIsolation` worker previously shared those objects by reference in the daemon heap, so a warm cache cost one full copy of the compile classpath analysis per task. Partition in the daemon instead: keep hits by reference, hand the worker only what it has to compute, and merge before writing the reports. On a 40-project build that shares one compile classpath, this takes buildHealth from 24s to 15s, against 14s on 3.17.0. A JFR recording puts 82% of the removed CPU in Moshi, with the ASM analysis unchanged.
|
This PR keeps both of the things you flagged in your comment on #1800: |
|
Yikes, this is not something I was looking at when doing the isolation in #1719, my mistake. The concept of what you are doing here makes sense, thanks for looking into this and improving it. Note that I did not look at the code in detail. |
|
I developed this PR with the help of Claude Code, not having seen the Code of Conduct's strict no-LLM/AI policy. Closing. |
Isolating kotlin-metadata to workers means
InMemoryCachecontents have to cross a classloader boundary.ExplodeJarTaskandFindKotlinMagicTaskdo that by serializing every cache hit into the worker as a JSON seed, which the worker parses back into a deep copy, so a warm cache costs one full copy of the compile classpath analysis per task. This partitions in the daemon instead: hits stay there by reference, the worker receives only the artifacts it has to analyze, and the task merges the two before writing its reports.On 40 synthetic
java-libraryprojects that all declare the same compile classpath, where 68 of the 80explodeJartasks have no cache misses at all:3.17.0 is there as the pre-isolation reference. It is the last release that used a
noIsolationworker and shared the cache by reference.A JFR recording attributes the difference.
com.squareup.moshiaccounts for 82% of the CPU samples that disappear, 1049 down to 639 against a total drop of 500, whilecom.autonomousapps.internal.asmis unchanged at 80 to 99. So the bytecode analysis is untouched and what goes away is the copying. GC pause time falls from 1054ms to 608ms, about 5% of the wall-clock difference.The smallest
-Xmxthat completes drops too, from 640m to 576m on the 10-jar fixture and from 2560m to 2048m on the 14-jar one.