Repository navigation
Failure of BenchmarkTools #19
Description
BenchmarkTools are not working correctly:
julia> using TropicalNumbers, CUDA, BenchmarkTools, LinearAlgebra, CuTropicalGEMM
julia> a = Tropical.(CUDA.randn(4096, 4096));
julia> @btime $a * $a;
3.375 μs (7 allocations: 256 bytes)
julia> @benchmark $a * $a
BenchmarkTools.Trial: 158 samples with 8 evaluations.
Range (min … max): 3.554 μs … 1.733 s ┊ GC (min … max): 0.00% … 0.07%
Time (median): 3.976 μs ┊ GC (median): 0.00%
Time (mean ± σ): 13.475 ms ± 137.779 ms ┊ GC (mean ± σ): 0.06% ± 0.01%
█ ▄
█▁▁▁▁▁▁▁▁▁▁▁▄▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█ ▄
3.55 μs Histogram: log(frequency) by time 13.5 ms <
Memory estimate: 256 bytes, allocs estimate: 7.Comparing to results directly from the C-CUDA tests, the result of @ benchmark is correct.
Activity
GiggleLiu commentedon Sep 28, 2023
Is it possible that the issue is related to CuStream? Because CUDA.synchronize() function takes CuStream as the input argument.
Maybe you need to pass CUDA.jl CuStream to the C code.
In your current code, the stream id seems to be the default value 0.
GiggleLiu commentedon Sep 28, 2023
This one works. Using the
CuContextas the input will synchronize all streams in the context, which is a heavy API.It would be great if the stream id can be an input.
GiggleLiu commentedon Oct 14, 2023
The current benchmark in the RAEDME does not look good. When we read benchmark, we always read the min-time, because it reflects the true performance.
GiggleLiu commentedon Sep 8, 2026
Resolved by the stream-control work merged in PR #27 and released as v0.1.2. The current matmul! API accepts a CuStream (defaulting to CUDA.stream()), and the README benchmark examples use CUDA.@sync so BenchmarkTools measures completed GPU work. This closure does not claim a new GPU run in the current maintenance environment.