Skip to content

[BUG] CuTe DSL: SIGSEGV compiling integer ** (int32/int64), including 2 ** 3 #3735

Description

@JayceSu98

Which component has the problem?

CuTe DSL

Bug Report

Describe the bug

Compiling a CUDA kernel containing x ** y, where both operands are loaded
from integer CUDA Tensors, terminates the Python process with SIGSEGV (exit
status 139) inside cute.compile().

The reproducer uses only small positive inputs: x = 2, y = 3, with expected
output 8. It does not involve zero bases, negative exponents, overflow, or
Python-valued constant evaluation. Compilation does not return, and the kernel
is not launched.

This reproduces for both int32 and int64 with native NVIDIA CuTeDSL wheels:

  • 4.5.2;
  • 4.8.0 (final release, cu12);
  • 4.8.0.dev0 (cu12), tested separately before checking the final release.

For 4.8.0 final, the frontend and compiler were freshly installed into an
isolated target from matching 4.8.0 wheels. No source changes or custom compiler
forks were used.

Steps/Code to reproduce bug

Save this as minimal_integer_pow.py in an environment with native NVIDIA
CuTeDSL and CUDA-enabled PyTorch:

import argparse

import cutlass
import cutlass.cute as cute
import torch
from cutlass.cute.runtime import from_dlpack


@cute.kernel
def pow_kernel(a: cute.Tensor, b: cute.Tensor, out: cute.Tensor):
    i, _, _ = cute.arch.thread_idx()
    x = a[i]
    y = b[i]
    out[i] = x ** y


@cute.jit
def launch(a: cute.Tensor, b: cute.Tensor, out: cute.Tensor):
    pow_kernel(a, b, out).launch(grid=(1, 1, 1), block=(128, 1, 1))


def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("--dtype", choices=("int32", "int64"), default="int32")
    args = parser.parse_args()
    print("CuTeDSL", cutlass.__version__, flush=True)
    print("PyTorch", torch.__version__, "CUDA runtime", torch.version.cuda,
          flush=True)
    print("GPU", torch.cuda.get_device_name(0), flush=True)
    print("dtype", args.dtype, "runtime inputs: a=2, b=3", flush=True)
    dtype = getattr(torch, args.dtype)
    a = torch.full((128,), 2, dtype=dtype, device="cuda")
    b = torch.full((128,), 3, dtype=dtype, device="cuda")
    out = torch.full_like(a, -123)
    handles = [from_dlpack(t, assumed_align=16) for t in (a, b, out)]
    print("COMPILE_BEGIN", flush=True)
    compiled = cute.compile(launch, *handles)
    print("COMPILE_OK", flush=True)
    compiled(*handles)
    torch.cuda.synchronize()
    result = out.cpu()
    assert torch.equal(result, torch.full_like(result, 8)), result
    print("PASS: synchronized output = 8", flush=True)


if __name__ == "__main__":
    main()

Run each dtype in a separate process:

python minimal_integer_pow.py --dtype int32
echo "exit=$?"
python minimal_integer_pow.py --dtype int64
echo "exit=$?"

Observed with 4.8.0 final:

CuTeDSL 4.8.0
PyTorch 2.7.0+cu128 CUDA runtime 12.8
GPU NVIDIA A100-PCIE-40GB
dtype int32 runtime inputs: a=2, b=3
COMPILE_BEGIN
PROCESS_EXIT_STATUS 139

COMPILE_OK is never reached. The shell reports Segmentation fault.
The int64 case also exits with status 139 during compilation.
4.5.2 additionally prints !!!!!!! Segfault encountered !!!!!!!.

Expected behavior

The kernel should compile and produce 8 for all elements. If runtime integer
exponentiation is not supported by this compilation path, an explicit compiler
diagnostic would be preferable to a process-level crash.

Environment details (please complete the following information):

  • Environment location: Docker, Linux x86_64.
  • GPU: NVIDIA A100-PCIE-40GB (SM80).
  • Python: 3.10.
  • PyTorch: 2.7.0+cu128; PyTorch CUDA runtime: 12.8.
  • CUDA Toolkit: 12.9.
  • NVIDIA driver: 610.43.02.
  • CuTeDSL: 4.5.2; 4.8.0 final with nvidia-cutlass-dsl, libs-base,
    libs-core, and libs-cu12 all at 4.8.0; separately 4.8.0.dev0 cu12.
  • Reproduced with file caching disabled (CUTE_DSL_ENABLE_FILE_CACHING=0).
  • The reproducer compiles an @cute.jit launcher, not a bare @cute.kernel.

The container originally had older user-space driver libraries. Matching host
driver libraries were selected for these test processes without replacing the
system installation. The CUDA execution controls below successfully compiled,
launched, synchronized, and checked device output with that same setup.

Additional context

Control cases use the same Tensor-loaded inputs 2 and 3, the same
128-thread block, and the same DLPack/compilation bridge. Successful cases
synchronize the device and check every output, rather than only compiling.

Expression 4.5.2 int32 / int64 4.8.0 final cu12 int32 / int64
x ** y Both SIGSEGV during compilation Both SIGSEGV during compilation
x + y, x * y, x // y, x % y Both pass Both pass
x - y, x / y, -x, abs(x) Both pass Not tested
cutlass.min, cutlass.max, ==, < Both pass Not tested
&, |, ^, ~, <<, >> Both pass Not tested
float32 x ** y Pass, output 8 Pass, output 8

Integer true division controls use float32 output. These are bounded controls,
not exhaustive coverage of all scalar operations or input domains.

The 4.5.2 pre-lowering IR contains math.ipowi on the loaded integer values.
An earlier GDB run stops inside the native NVIDIA _cutlass_ir.so, but does
not provide useful symbols. We have not identified a specific internal pass
root cause or established that every affected version has the same root cause.

We found the older floating-point power discussion #2296, but it concerns
v ** 1.3, increased compilation time, and a suspected runtime hang, rather
than this positive-integer compilation SIGSEGV. We could not find a matching
public issue or a documented fix for this integer-power reproducer.

Could you help clarify whether Tensor-loaded integer ** is intended to be
supported, and whether this crash matches a known issue or existing fix?

Activity

  1. yunweili3 commented on Oct 13, 2026

    @yunweili3

    Reproduced on B200 (sm_100a) with your script on the 4.8.0 wheel (torch 2.14 / CUDA 13.0) and on 4.5.3: exit 139 for int32 and int64, COMPILE_OK never reached. A few more data points that narrow it down:

    • The crash is in PassManager.run (Python frame base_dsl/compiler.py:254 via faulthandler), inside _cutlass_ir.so, with no diagnostic. Enabling MLIR IR printing on the pass manager produces no dump at all before the crash, and the pipeline is the single closed cute-to-nvvm{...} pass, so the failing sub-pass isn't visible from the outside.
    • The IR handed to the pipeline is just %5 = math.ipowi %3, %4 : i32, which is valid.
    • x ** 3 (constant exponent) crashes the same way; so does ** on TensorSSA vectors.
    • The identical source compiles and runs correctly on the host path (a @cute.jit function with a ** b and no kernel prints 8, 81, 2 ** 40), so only the device lowering of math.ipowi is broken. float ** int (math.fpowi) and float ** float (math.powf) are fine in kernels, as you saw.

    Since the Python front end can avoid the op entirely, I opened PR #3738: a ** b and cute.math.ipowi are now expanded in Python into square-and-multiply (arith.muli chain for a constant exponent, scf.while for a dynamic one, vectors included) with the semantics of math.ipowi (wrapping overflow, 1/-1/0 for negative exponents of signed values). With that patch your script passes for both dtypes on B200 and x ** 3 lowers to two mul.lo.s32. The compiler-side crash itself is still worth a fix on NVIDIA's end, but kernels no longer hit it.

  2. frokaikan commented on Oct 14, 2026

    @frokaikan

    Thanks for the report. I'm looking into this now and investigating the root cause. I'll post an update here once I know more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    CuTe DSLbugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions