Clarification for running GPU kernel evolution search on MI300

[chaitanyamanem]

Hi SimpleTES team,

Thank you for releasing the paper and code. I am working on reproducing and extending the GPU kernel optimization experiments on AMD MI300.

I have already evaluated the released TriMul best kernel on MI300 and get around 1390.182 µs, which is close to the paper’s reported MI300 latency of 1.352 ± 0.004 ms.

However, when I run the full evolutionary search process directly on MI300 for TriMul, the best kernel I currently find is around 1841.433 µs (1.841 ms), which is noticeably slower than both the released artifact and the paper result. I am trying to understand whether this is due to search-budget/settings differences, software stack differences, or expected stochastic variation.

From the paper/repo, I understand that the GPU kernel tasks used C=32 chains and K=8 local candidates, and that the original search was mainly run on H200.

Could you clarify the following minimal details needed to reproduce the search setup?

1. What exact L or --max-generations was used for the reported GPU kernel search runs?

2. Were the reported kernels obtained from a single search run, or selected from multiple independent runs/seeds?

3. For running the search directly on MI300, are there any recommended settings or caveats for:

   • Triton version
   • ROCm version
   • PyTorch version
   • evaluator server settings
   • compiler/eval GPU split
   • any MI300-specific environment variables

4. Are the released best kernels expected to be the result of H200 search only, or was any hardware-specific tuning/search performed for MI300?

Any guidance would be very helpful. I would like to run the full SimpleTES search on MI300 and compare the discovered kernels against the released artifacts.

[galeselee]

Hi @chaitanyamanem

Thank you very much for your detailed report and for your interest in our work. We really appreciate that you are following SimpleTES, reproducing the GPU kernel optimization experiments, and further extending the evaluation to AMD MI300.

Below, we provide the details of our search setup and evaluation environment, which we hope will be helpful.

1. For the reported GPU kernel search runs, we used L=200 and --max-generations=25600. In addition, for each SimpleTES task, we performed one best-restart run: after the initial search, we took the best solution found during this search as the initial solution and ran another search with the same setting, i.e., C=32, K=8, and L=200.
2. For the C=32, K=8, L=200 setting, we get our result program from a single run. However, we also tried multiple runs with C=32, K=16, L=100. In those experiments, we observed relatively large variance; for TriMul on H200, the best results were generally in the range of roughly 1.080 – 1.4 ms. Based on our ablation study in Paper Section 4, we would recommend using a relatively large L for the TriMul GPU kernel task if the search budget allows.
3. For MI300, our environment:

• MI300: AMD devcloud image rocm-7-2-software-gpu-mi300x1-192gb-devcloud-atl1
• Triton: 3.6.0
• PyTorch: 2.8
• ROCm: 7.2
• Compiler/eval GPU split: for the TriMul task, we found 2:6 to provide a reasonable balance between search efficiency and evaluation throughput. (It only effects evolving efficiency)
• MI300-specific environment variables: none. We only used MI300 for evaluation.

4. Regarding hardware-specific tuning: the released best kernels were obtained from searches performed on H200. We did not perform additional hardware-specific tuning or search on MI300.

[chaitanyamanem]

Hi @galeselee

Thanks for the clarification. I want to confirm two points because I may be misunderstanding the configuration.

Our understanding from the paper/code is that:

--max-generations = C × K × L

So for:

C = 32, K = 8, --max-generations = 25600

this would imply:

L = 25600 / (32 × 8) = 100

However, your message says the reported GPU kernel search used L = 200 and --max-generations = 25600. Could you clarify how L is defined in your runs, and how it maps to --max-generations?

Second, I want to clarify the restart setting. You mentioned that for each SimpleTES task, you performed one best-restart run, where the best solution from the initial search was used as the initial solution for another search with the same settings. But you also mentioned that for C=32, K=8, L=200, the result program came from a single run.

Could you clarify whether the reported TriMul result was obtained from:

1. one search run only with C=32, K=8, L=200, and --max-generations=25600, or
2. one initial search plus one best-restart search, each with C=32, K=8, L=200, and --max-generations=25600?

Under our current interpretation of --max-generations = C × K × L, C=32, K=8, L=200 would correspond to --max-generations=51200, which is why I want to clarify the mapping.

I am trying to reproduce the exact reported setting, so confirming these two details would be very helpful.

[galeselee]

Hi, @chaitanyamanem

Thanks for pointing this out, and sorry for the confusion. This was a typo in my previous message. The correct setting is: C = 32, K = 8, L = 200 and therefore --max-generations = C × K × L = 32 × 8 × 200 = 51200

Our final TriMul result was obtained as follows. We first ran one search with --max-generations=51200, using the naive PyTorch implementation as the initial solution. Then, we took the best performance program(1.107ms on H200) from this first search as the initial solution and ran a second search with another --max-generations=51200, got the final best result(1.020ms on H200).

[chaitanyamanem]

Hi @galeselee ,

Thank you for your earlier responses. They were very helpful.

I am trying to reproduce the paper’s lr&bsz SLDBench result with gpt-oss-120b. From §3.6.1, I understand the reported SimpleTES setting is:

C = 16
L = 20
K = 16
total evaluator budget = 16 × 20 × 16 = 5120
reported lr&bsz test R² = 0.712, averaged over 5 seeds

I am using the released datasets/scaling_law/lr_bsz_scaling_law files. Could you please clarify the remaining settings needed to match the reported run?

• Which selector was used for lr&bsz: rpucg, balance, or another policy?
• What values were used for num_inspirations / inspiration range and selector-specific params such as rpucg_gamma or puct_c?
• What generation settings were used for gpt-oss-120b: temperature, max tokens, backend (litellm or vllm_token_forcing), and any decoding/seed settings?
• Were reflection summaries and failure-pattern memory enabled?
• Was best-solution restart used for this task, or was it one independent SimpleTES evolution per seed?
• For the “5 seeds” result, does each seed mean one full evolution run producing one best program, followed by one held-out test evaluation of that program, with the five test R² scores averaged?

Once again, thank you very much for all your earlier responses. I look forward to your clarification on these details.

[galeselee]

Hi, @chaitanyamanem
Thanks for your continued interest in this work. If possible, could you also share the final reproduction results you obtained for the GPU Kernel task on MI300?

For your questions:

1. rpucg is the current core selector used in SimpleTES.
2. num_inspirations is usually set to either 3 or 5. We use puct_c = 1, and keep rpucg_gamma at its default value.
3. The temperature is set to 1.0. If you are able to use vllm_token_forcing, we use the following budget configuration:
   a. context_window = 49152
   b. reasoning_budget = 33616
   c. response_budget = 15536

4 & 5. Reflection summaries, failure-pattern summaries, and best-solution restarts are enabled for every task.
6. For the detailed lr & bsz settings used in SLDBench, @linhaowei1 could you help answer this and check the above answer.

[linhaowei1]

1. The selector is rpucg. The parameters (e.g., rpucg_gamma, puct_c, temperature, and max_tokens) are fixed to the same values used in the other tasks. vllm_token_forcing is used. Reflection and failure-pattern are enabled.

2. No best-solution restart is used.

3. Yes, the “5 seeds” results are obtained as described. One thing to note is that when selecting the law used for LR and batch-size selection, we actually choose the law with the best test (R^2).