[Issue]: rocr: GPU timestamp bidirectional epoch correction on Windows

[woct0rdho]

Problem Description

PR #6282 has implemented an epoch correction when AQL timestamp has a positive epoch offset compared to D3DKMT. However, it's also possible to have a negative epoch offset. When a negative epoch offset happens, currently GpuAgent::TranslateTime outputs time.start = 0 and time.end = 0, so downstream benchmarking code may get stuck or behave wrong, such as torch.compile autotune and triton.jit autotune.

A possible fix:

--- a/projects/rocr-runtime/runtime/hsa-runtime/core/runtime/amd_gpu_agent.cpp
+++ b/projects/rocr-runtime/runtime/hsa-runtime/core/runtime/amd_gpu_agent.cpp
@@ -3098,7 +3098,14 @@ void GpuAgent::TranslateTime(core::Signal* signal, hsa_amd_profiling_dispatch_ti
   uint64_t start, end;
   signal->GetRawTs(false, start, end);

-  if ((start == 0) || (end == 0) || (start < t0_.GPUClockCounter) || (end < t0_.GPUClockCounter)) {
+  if ((start == 0) || (end == 0)
+#ifndef _WIN32
+      || (start < t0_.GPUClockCounter) || (end < t0_.GPUClockCounter)
+#endif
+  ) {
     debug_print("Signal %p time stamps may be invalid (start=%" PRIu64 ", end=%" PRIu64 ", t0=%" PRIu64 ").\n",
                 &signal->signal_, start, end, t0_.GPUClockCounter);
     time.start = 0;
@@ -3116,7 +3123,11 @@ void GpuAgent::TranslateTime(core::Signal* signal, hsa_amd_profiling_async_copy_
   uint64_t start, end;
   signal->GetRawTs(true, start, end);

-  if ((start == 0) || (end == 0) || (start < t0_.GPUClockCounter) || (end < t0_.GPUClockCounter)) {
+  if ((start == 0) || (end == 0)
+#ifndef _WIN32
+      || (start < t0_.GPUClockCounter) || (end < t0_.GPUClockCounter)
+#endif
+  ) {
     debug_print("Signal %p async copy time stamps may be invalid (start=%" PRIu64 ", end=%" PRIu64 ", t0=%" PRIu64 ").\n",
                 &signal->signal_, start, end, t0_.GPUClockCounter);
     time.start = 0;
@@ -3192,14 +3203,15 @@ uint64_t GpuAgent::TranslateTime(uint64_t tick) {
   }

 #ifdef _WIN32
-  // Detect epoch mismatch: only trigger when translated time is in the future,
-  // which proves AQL timestamps have an epoch offset from D3DKMT's GPU clock.
-  // If TranslateTime is called long after dispatch, system_tick <= now, so no
+  // Detect epoch mismatch: only trigger when translated time is before t0 or in the future,
+  // which proves AQL timestamps have an epoch offset from D3DKMT's GPU clock.
+  // If TranslateTime is called long after dispatch, t0 <= system_tick <= now, so no
   // false offset is computed.  Retries on subsequent calls until detected.
   if (gpu_clock_offset_ == 0) {
-    int64_t now = int64_t(os::TimeNanos());
-    if (int64_t(system_tick) > now) {
-      gpu_clock_offset_ = int64_t(double(int64_t(system_tick) - now) / ratio);
+    const int64_t now = int64_t(os::TimeNanos());
+    const int64_t translated_tick = int64_t(system_tick);
+    if ((translated_tick < int64_t(t0_.SystemClockCounter)) || (translated_tick > now)) {
+      gpu_clock_offset_ = int64_t(double(translated_tick - now) / ratio);
       // Re-translate this first event with the corrected offset.
       elapsed = int64_t(ratio * double((int64_t(tick) - gpu_clock_offset_) - int64_t(t1_.GPUClockCounter)));
       system_tick = uint64_t(elapsed) + t1_.SystemClockCounter;

Operating System

Windows 11 26200.8894

CPU

Ryzen AI MAX+ 395

GPU

Radeon 8060S

ROCm Version

rocm 7.15.0a20260721

ROCm Component

ROCR-Runtime

Steps to Reproduce

I'm not sure how to manually create a negative epoch offset. Maybe it happens when the computer is powered on for some time, or maybe it's because of some driver update. When it happens, an immediate phenomenon is that torch.compile autotune and triton.jit autotune get stuck.

(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support

No response

Additional Information

No response

cc @gandryey