8372584: [Linux]: Replace reading proc to get thread user CPU time with clock_gettime - #621
8372584: [Linux]: Replace reading proc to get thread user CPU time with clock_gettime#621mmm-choi wants to merge 4 commits into
Conversation
|
👋 Welcome back mmm-choi! A progress list of the required criteria for merging this PR into |
|
❗ This change is not yet ready to be integrated. |
|
This backport pull request has now been updated with issue from the original commit. |
Webrevs
|
|
|
|
/approval request Fixes the long-standing defect JDK-8210452 (created in 2018): on Linux, ThreadMXBean.getCurrentThreadUserTime() parses /proc on every call, making it 30-400x slower than getCurrentThreadCpuTime() and worse under concurrency. This replaces that with a single clock_gettime(). I verified it in 25 on two builds identical except for this change: mean latency drops ~24x on the in-tree JMH microbenchmark, with a large tail-latency reduction. Risk is low: the change is Linux-only, is a net code simplification, has been in mainline since JDK 26 with no regressions filed against it, and passes the full thread-CPU-time test set (ThreadMXBean, nsk monitoring, JFR) on x86_64 and aarch64. There is also concrete downstream demand for it. Depends on JDK-8372625 (PR #620). |
|
Hi @mmm-choi |
|
Yes, the values themselves change, not just the speed, but callers will see more precise numbers after updating. The old user time path read utime from I don't think this breaks any documented contract. The spec for these methods only says "The returned value is of nanoseconds precision but not necessarily nanoseconds accuracy" and doesn't promise any particular clock or resolution: https://docs.oracle.com/en/java/javase/25/docs/api/java.management/java/lang/management/ThreadMXBean.html#getCurrentThreadUserTime() Also, from what I understand (please correct me if I'm wrong), on any modern kernel, getCurrentThreadCpuTime() (user+sys) was already using clock_gettime in 25, and only the user time query was still falling back to /proc. The reason is that clock_gettime only returns the user+sys total, so user-only time had no way to come from it, and /proc was the only place that split user and sys apart. This change uses the CPUCLOCK_VIRT clock id to get user-only time from clock_gettime too, so the JVM moves the user value onto clock_gettime rather than leaving it split across two sources. ThreadUserTime.java also still passes, including its checks that user time is monotonic and never exceeds user+sys. On the demand, it came from customer that is profiling per-thread CPU time who can't leave it running because of the /proc cost. That is also why the original JDK-8210452 was filed. |
|
The parent pull request that this pull request depends on has now been integrated and the target branch of this pull request has been updated. This means that changes from the dependent pull request can start to show up as belonging to this pull request, which may be confusing for reviewers. To remedy this situation, simply merge the latest changes from the new target branch into this pull request by running commands similar to these in the local repository for your personal fork: git checkout JDK-8372584
git fetch https://git.openjdk.org/jdk25u-dev.git master
git merge FETCH_HEAD
# if there are conflicts, follow the instructions given by git merge
git commit -m "Merge master"
git push |
This is a backport of JDK-8372584 (858d2e43) to 25u.
This is the second of three related backports and is the substantive change. It must be applied after JDK-8372625 (#620):
supports_fast_thread_cpu_timedual-path logic. This is the prerequisite./procparsing withclock_gettime. This is the optimization.Why this backport (this is an enhancement)
While it is classified as an enhancement, it addresses a long-standing and severe performance defect, JDK-8210452 (reported in 2018), where
getCurrentThreadUserTime()is 30x to 400x slower thangetCurrentThreadCpuTime()on Linux and degrades further under concurrency.The legacy implementation opens, reads, and
sscanf-parses/proc/self/task/<tid>/staton every call. That is roughly 3 syscalls plus a full-page (4096-byte) kernel allocation per sample, of which over 99% is waste, all just to extract one integer. This change replaces that with a singleclock_gettime()using theCPUCLOCK_VIRTclock-id bit, which is the de-facto kernel ABI glibc has relied on for over 20 years. There is more background at https://norlinder.nu/posts/User-CPU-Time-JVM/.There is concrete downstream demand for this. The
/procoverhead currently makes continuous self-instrumentation of per-thread CPU usage prohibitive for production users. The change is also a net code simplification (the diff removes more than it adds) and is fully integrated in mainline (JDK 26).Not a clean backport
Manual resolution was required for one hunk. Mainline commit openjdk/jdk@80ab094 JDK-8347707 (
os::snprintfstandardization, not in 25u) had changedsnprintftoos::snprintf_checkedinside the/proc-reading function that this change deletes, so the deletion context did not match. I resolved it by taking mainline's side (the newclock_gettimebody) in full. The resulting code, including the newget_thread_clockid()helper and theThreadMXBeanBenchmicrobenchmark, is byte-for-byte identical to mainline.How the fix was validated and checked for regressions
I built two
releaseimages that are identical except for these three commits:master(the exact commit this stack is based on)masterplus JDK-8372625, JDK-8372584, and JDK-8373557I confirmed via
nmonlibjvm.sothat the only difference is this code. The baseline exports the oldslow_thread_cpu_timeand noos::Linux::thread_cpu_time(clockid_t), and the fixed build is the inverse. After resolution I also diffed the thread-CPU-time region ofos_linux.cppandos_linux.hppagainst the mainline commit and confirmed it is identical, apart from retaining_pthread_setname_np, which is unrelated and absent from mainline only because of JDK-8368124, which is not in 25u.Functional and regression testing: The thread-CPU-time test set passes on both this commit and the full 3-commit stack, on linux-x86_64 and linux-aarch64:
java/lang/management/ThreadMXBean/ThreadUserTime.javajava/lang/management/ThreadMXBean/ThreadCpuTime.javajava/lang/management/ThreadMXBean/VirtualThreads.javavmTestbase/nsk/monitoring/ThreadMXBean/GetThreadCpuTime(10)vmTestbase/nsk/monitoring/ThreadMXBean/isThreadCpuTimeSupported(5)vmTestbase/nsk/monitoring/ThreadMXBean/isCurrentThreadCpuTimeSupported(5)jdk/jfr/event/runtime/TestThreadCpuTimeEvent.javajdk/jfr/event/profiling/*(CPU-time sampler)These exercise every consumer of the modified path: JMX (
ThreadMXBean), JVMTI (nsk), the JFR thread-CPU-time event, and the JFR CPU-time sampler.ThreadUserTime.javain particular asserts the key invariants the change must preserve.getCurrentThreadUserTime()returns-1before enabling, returns a non-negative value after enabling, and per-thread user time is monotonic and never exceeds the corresponding CPU (user+sys) time.Performance, reproduced in 25u. Measured with the exact JMH microbenchmark added by this commit (
ThreadMXBeanBench,SampleTime,@Fork(10) @Warmup(2x5s) @Measurement(5x5s), single thread), compiled against JMH 1.37 and run as the same benchmarks jar on both images:/proc)clock_gettime)(baseline n=6,586,868 samples, fixed n=5,230,948.) The common case drops from about 19 us to under 1 us, which is a single syscall versus a
/procopen+read+parse, and the worst-case sample drops from about 10 ms to about 1.5 ms. This matches the shape of the original PR's result (openjdk/jdk#28556) and confirms the tail-latency reduction that motivates the change.Risk
This is a Linux-only behavioral change (proc-derived to
clock_gettime(CPUCLOCK_VIRT)for user time), with no change to other platforms. The shared-file edits in this stack are comment-only. It was reviewed in tip and is covered by the existing regression testThreadUserTime.javaplus the added microbenchmark.Testing
Built
releaseon linux-x86_64 and linux-aarch64. The thread-CPU-time test set passes (see the table above).Progress
Issue
Reviewers
Reviewing
Using
gitCheckout this PR locally:
$ git fetch https://git.openjdk.org/jdk25u-dev.git pull/621/head:pull/621$ git checkout pull/621Update a local copy of the PR:
$ git checkout pull/621$ git pull https://git.openjdk.org/jdk25u-dev.git pull/621/headUsing Skara CLI tools
Checkout this PR locally:
$ git pr checkout 621View PR using the GUI difftool:
$ git pr show -t 621Using diff file
Download this PR as a diff file:
https://git.openjdk.org/jdk25u-dev/pull/621.diff
Using Webrev
Link to Webrev Comment