[Perf] CUDA target defaults to precise transcendentals — 4x slower than Vulkan's default on the same GPU; downstream_args --use_fast_math silently ignored

[tekintatar]

Summary

The same Slang source compiles to very different transcendental implementations per target, with no warning:

• Vulkan / Metal: drivers compile exp / pow to fast hardware approximations by default.
• CUDA (NVRTC): exp / pow (and atan2, even exp2) compile to precise software routines by default — a pow-chain kernel runs 4.4x slower than with fast math, and 2x slower than the Vulkan default on the same physical GPU.

SlangCompilerOptions.floating_point_mode = fast fixes it (and is how we resolved a persistent CUDA-vs-Vulkan benchmark gap), so this is
partly a documentation/defaults report:

1. The cross-backend default inconsistency is surprising: identical Slang code is "fast math" on two backends and "precise math" on the
   third. A note in the docs (or aligned defaults) would save users a long investigation — kernels dense in transcendentals mysteriously
   lose to Vulkan on the same GPU.
2. downstream_args = ["--use_fast_math"] is accepted without error but has no effect on the generated CUDA code or timing — either
   it should reach NVRTC or it should be rejected.

Environment

• slangpy 0.42.0 (NVRTC 13.0 via CUDA 13 toolchain)
• Ubuntu 24.04, NVIDIA L40S, driver 580.126.16
• Python 3.12

Reproduction

python repro_fastmath.py <backend> <mode> — a 64-iteration dependent
pow chain per pixel at 3456^2.

Observed output:

cuda/default   (NVIDIA L40S): pow x64 chain at 3456^2 ->  1.49 ms
cuda/fast      (NVIDIA L40S): pow x64 chain at 3456^2 ->  0.34 ms
cuda/nvrtcfast (NVIDIA L40S): pow x64 chain at 3456^2 ->  1.50 ms   <- --use_fast_math ignored
vulkan/default (NVIDIA L40S): pow x64 chain at 3456^2 ->  0.75 ms

Full repro script (repro_fastmath.py)

# Minimal repro: transcendentals on the CUDA backend default to precise
# software routines — 2x slower than Vulkan's DEFAULT (hardware approx)
# on the same GPU, 4.4x slower than CUDA with floating_point_mode=fast
# (which works). downstream_args = ["--use_fast_math"] is accepted but
# silently changes nothing.
#
#   python repro_fastmath.py cuda default
#   python repro_fastmath.py cuda fast        # floating_point_mode
#   python repro_fastmath.py cuda nvrtcfast   # downstream --use_fast_math
#   python repro_fastmath.py vulkan default   # reference: hardware approx
#
# slangpy 0.42.0.

import sys
import time

import numpy as np
import slangpy as spy

SHADER = """
void pow_loop(uint2 tid, Texture2D<float4> src,
              [format("rgba16f")] RWTexture2D<float4> dst, int iters)
{
    uint w, h; src.GetDimensions(w, h);
    if (tid.x >= w || tid.y >= h) return;
    float acc = src[tid].x + 0.5;
    for (int i = 0; i < iters; ++i)
        acc = pow(abs(acc) + 0.5, 1.7) * 0.31 + src[tid].y * 1e-3;
    dst[tid] = float4(acc, 0.0, 0.0, 1.0);
}
"""

S, N, ITERS = 3456, 20, 64


def main() -> None:
    backend = sys.argv[1] if len(sys.argv) > 1 else "cuda"
    mode = sys.argv[2] if len(sys.argv) > 2 else "default"

    opts: dict = {"include_paths": [spy.SHADER_PATH]}
    if mode == "fast":
        opts["floating_point_mode"] = spy.SlangFloatingPointMode.fast
    elif mode == "nvrtcfast":
        opts["downstream_args"] = ["--use_fast_math"]
    device = spy.Device(type=getattr(spy.DeviceType, backend),
                        compiler_options=opts)
    module = spy.Module.load_from_source(device, "fastmath_repro", SHADER)

    rng = np.random.default_rng(3)
    usage = spy.TextureUsage.shader_resource | spy.TextureUsage.unordered_access
    src = device.create_texture(width=S, height=S, format=spy.Format.rgba16_float,
                                usage=usage,
                                data=rng.random((S, S, 4)).astype(np.float16))
    dst = device.create_texture(width=S, height=S, format=spy.Format.rgba16_float,
                                usage=usage)
    grid = spy.grid((S, S))

    module.pow_loop(grid, src, dst, ITERS)
    device.wait_for_idle()
    t0 = time.perf_counter()
    for _ in range(N):
        module.pow_loop(grid, src, dst, ITERS)
    device.wait_for_idle()
    ms = (time.perf_counter() - t0) * 1000 / N
    print(f"{backend}/{mode} ({device.info.adapter_name}): "
          f"pow x{ITERS} chain at {S}^2 -> {ms:6.2f} ms")


if __name__ == "__main__":
    main()

• CUDA default is 2x slower than the Vulkan default and 4.4x slower than CUDA with floating_point_mode = fast.
• downstream_args = ["--use_fast_math"] produces timing identical to the default (1.50 vs 1.49 ms).

Expected

Either consistent cross-target defaults for transcendental precision, or a prominent note on the CUDA target's behavior; and
downstream_args that don't reach the downstream compiler should error rather than silently no-op.

Workaround (for anyone else hitting this)

Set floating_point_mode = spy.SlangFloatingPointMode.fast in the device's compiler options for the CUDA backend — this matches what the Vulkan and Metal drivers already do to the same source, so precision behavior becomes consistent across backends rather than looser.

[nv-slang-bot]

Triage

• Status: Triaged → handed to a fixer coworker; fix incoming. Confirmed reproducible from source (no CUDA GPU in the triage environment, so the timing repro will be re-run by the fixer on hardware).
• Root cause (the actionable bug): User-supplied compiler_options["downstream_args"] are forwarded to the Slang compiler only on the D3D12 path. In src/sgl/device/shader.cpp the forwarding loop is gated if (device_type == DeviceType::d3d12) and tags each arg for the "dxc" downstream compiler (shader.cpp:322-326, and again in the link path at :1540-1543). The CUDA branch never iterates downstream_args, so ["--use_fast_math"] is accepted by the API and then silently dropped — matching your observation that nvrtcfast timing equals default. This same per-target gate means downstream args are dropped for Vulkan/Metal/WGPU/CPU too.
• Why floating_point_mode = fast works but --use_fast_math doesn't: floating_point_mode is a first-class Slang compiler option applied per-target (shader.cpp:397-398), so Slang emits the correct NVRTC flag itself. downstream_args is a raw pass-through list that simply isn't wired for the CUDA target.
• Planned fix: Add a CUDA branch forwarding each arg via session_options.add(CompilerOptionName::DownstreamArgs, "nvrtc", arg) (CLI-equivalent -Xnvrtc <flag>). The "nvrtc" pass-through tag is already used for the OptiX include at shader.cpp:334, and the CUDA target is SLANG_PTX (:429) → NVRTC is the downstream compiler, so this is the correct incantation. Considering pairing it with a warning when args target a backend that has no forwarding path (so nothing is silently ignored — addressing the "should error rather than no-op" request). Report 1 (cross-backend transcendental default) will be handled as a documentation note: the CUDA target's default keeps precise transcendentals while the Vulkan/Metal drivers approximate by default. We do not plan to silently flip the default (it changes numerical results); the default-policy question can be raised upstream in the Slang compiler if maintainers want it.
• Next action / blocker: No blocker. Fix in progress; a PR will follow and link back here. Reproduction on the CUDA/L40S box will confirm nvrtcfast drops from ~1.50 ms toward the ~0.34 ms of floating_point_mode=fast before the PR is marked ready.

_{🤖 Generated by an automated SlangPy coworker — may be inaccurate. A human maintainer should verify.}

[nv-slang-bot]

Triaged → fix in draft PR #1061, held pending review.

• Status: Root-caused and fixed in draft Forward downstream_args to NVRTC for the CUDA target #1061 (Fixes #1058); held as draft pending maintainer review.
• Link: Forward downstream_args to NVRTC for the CUDA target #1061
• Verdict: Confirmed against source — compiler_options.downstream_args were forwarded to the Slang compiler only inside the device_type == d3d12 gate (tagged "dxc"), in both the compile and link paths. On CUDA they were accepted without validation and silently dropped, so downstream_args=["--use_fast_math"] never reached NVRTC. The fix adds a CUDA branch forwarding each arg with the "nvrtc" pass-through tag (the same tag already used in-tree for the OptiX include path). Part 1 (CUDA's precise-transcendental default vs. Vulkan/Metal driver approximations) is not a plumbing bug — floating_point_mode is plumbed correctly, which is why floating_point_mode=fast already works; it's addressed as a docstring note, and the default is intentionally left unchanged (changing it would alter numeric results).
• Next-action: A maintainer should run the repro (repro_fastmath.py cuda nvrtcfast) on a CUDA box to confirm timing drops from ~1.50 ms to ~0.34 ms (matching cuda fast). The added regression test is CUDA-gated and needs a CUDA host / CI runner.
• Blocker: None. No CUDA GPU in the automated environment, so behavioral confirmation of the numeric speedup is pending a maintainer / CI run on CUDA hardware.

_{🤖 Generated by an automated SlangPy coworker — may be inaccurate. A human maintainer should verify.}

[nv-slang-bot]

Update: fix verified — draft PR #1061 is green.

• Status: CI passes on all 14 checks, including the CUDA build/test jobs. The regression test exercises the fix on real CUDA hardware: downstream_args=["--use_fast_math"] now reaches NVRTC and the kernel runs, while an invalid NVRTC option is correctly rejected (proving the args are actually forwarded).
• Link: Forward downstream_args to NVRTC for the CUDA target #1061 (draft, awaiting maintainer review)
• Verdict: Part 2 (the actionable bug) fixed for both the compile-time and link-time downstream_args paths. Part 1 (transcendental defaults) addressed as a docstring note; the default is unchanged.
• Next-action: Maintainer review to promote from draft. Optionally run repro_fastmath.py cuda nvrtcfast to observe the ~1.50 ms → ~0.34 ms timing improvement.
• Blocker: None.

_{🤖 Generated by an automated SlangPy coworker — may be inaccurate. A human maintainer should verify.}

[nv-slang-bot]

Update: the code fix landed via #1095, not #1061. Residual scope: the docs point plus regression coverage.

My earlier comments on this issue pointed at draft PR #1061 as the incoming fix. @skallweitNV noted it was made obsolete by #1095, which I verified and then closed #1061 as a duplicate — so that pointer is stale. Correcting the trail:

• Status: Part 2 (downstream_args silently dropped on CUDA) is fixed on main by Forward CUDA downstream compiler arguments #1095 — src/sgl/device/shader.cpp:384-386 (compile) and :1636-1638 (link) now forward each arg with the "nvrtc" tag. downstream_args=["--use_fast_math"] reaches NVRTC on the cuda target.
• Link: Forward CUDA downstream compiler arguments #1095 (merged 2026-08-07). Duplicate draft Forward downstream_args to NVRTC for the CUDA target #1061 closed unmerged.
• Verdict: On Part 1, source inspection shows no floating_point_mode plumbing defect — it is applied for every target via TargetDesc::floatingPointMode and the per-target compiler-option entries (shader.cpp:458), with an explicit link override at :1623. Whether the cross-backend default inconsistency itself counts as a product defect is a maintainer call, not something I can settle from the source; I'm only ruling out a plumbing bug.
• Correction to my own earlier framing (and to this issue's title): the ratios are two different comparisons. Per your repro_fastmath.py numbers, CUDA default (1.49 ms) is ≈2× slower than the Vulkan default (0.75 ms); the 4.4× is CUDA default vs CUDA floating_point_mode=fast (0.34 ms) — not vs Vulkan. The title's "4x slower than Vulkan's default" conflates the two, and my previous comment repeated that. Apologies for propagating it.
• Next-action: Two residuals for a maintainer to decide, both maintainer-gated: (a) document on SlangCompilerOptions.floating_point_mode that the CUDA/NVRTC target emits precise transcendentals by default and fast enables approximations; (b) add regression coverage — Forward CUDA downstream compiler arguments #1095 landed the forwarding fix without a test, and main currently has none referencing downstream_args, so a future refactor could silently re-drop it and reproduce Part 2 exactly. I have both written on the closed Forward downstream_args to NVRTC for the CUDA target #1061 branch and can open a small docs + test PR on request; not doing so unprompted.
• Blocker: None. No code-behavior change outstanding — the residual is documentation and test coverage.

_{🤖 Generated by an automated SlangPy coworker — may be inaccurate. A human maintainer should verify.}