[RFC] Make `cuda.core` a required dependency

[leofang]

Recent works in cuda.core/cuda.bindings have reached to a point where we can replace most, if not all, numba-cuda internal wrappers to driver, nvvm, linker, … by cuda.core. It has been the plan to make cuda.core a required (not just optional) dependency of numba-cuda (and cuda.bindings would become a transitive dependency). There are multiple benefits, such as

• separation of concerns: numba-cuda focuses on the compiler stack and leave CUDA’s Python exposure to CUDA Python
• support CUDA wheels out of box (so issue 65 and issue 66 do not need any effort on the numba-cuda side)
• better performance (since cuda-bindings is based on Cython instead of ctypes)
• support Windows (which pynvjitlink does not, as of today)
• coverage of latest CUDA features
• …

This RFC is created to track necessary efforts:

• [FEA] Make cuda.core.Buffer recognized by numba-cuda #152
• [FEA] Make cuda.core.utils.StridedMemoryView recognized by numba-cuda #153
• [FEA]: Make pathfinder usable as a full replacement for cuda_paths.py in numba-cuda cuda-python#1036
• [MNT] Replace all internal Linker-related classes by cuda.core.(experimental).Linker #129
• [RFC] Archive this project in favor of cuda.core.(experimental).Linker rapidsai/pynvjitlink#130
• Support NVVM IRs as input to Program cuda-python#452
• [FEA] Make cuda.core.Stream recognized by numba-cuda by supporting the __cuda_stream__ protocol #151
• [MNT] Drop CUDA_USE_NVIDIA_BINDING and always use cuda.core (and fallback to cuda.bindings) #154
• [RFE] Better support for CUDA Minor Version Compatibility #262
• Deprecate/remove cuda_paths.py entirely #302

[realarnavgoel]

@leofang Does cuda.core._launcher support option in LaunchConfig to select cuLaunchCooperativeKernel or cuLaunchKernel depending on the user needs for grid.sync() or other deadlock-free guarantees when performing non-local operations in each GPU (e.g. barrier between GPUs) of a multi-GPU application ?

numba-cuda supports this automatically and selects the appropriate mechanism in the dispatcher: https://github.com/NVIDIA/numba-cuda/blob/main/numba_cuda/numba/cuda/dispatcher.py#L409

[leofang]

Does cuda.core._launcher support option in LaunchConfig to select cuLaunchCooperativeKernel or cuLaunchKernel depending on the user needs for grid.sync() or other deadlock-free guarantees when performing non-local operations in each GPU (e.g. barrier between GPUs) of a multi-GPU application ?

The beauty of the new cuLaunchKernelEx API is it is not our (cuda.core, and maybe even Numba) job to choose which API to use. We just need to add a new option to LaunchConfig to hook with CU_LAUNCH_ATTRIBUTE_COOPERATIVE as you suggested, so that the driver knows to launch it cooperatively. Please open an issue in NVIDIA/cuda-python to track it.

[realarnavgoel]

Does cuda.core._launcher support option in LaunchConfig to select cuLaunchCooperativeKernel or cuLaunchKernel depending on the user needs for grid.sync() or other deadlock-free guarantees when performing non-local operations in each GPU (e.g. barrier between GPUs) of a multi-GPU application ?

The beauty of the new cuLaunchKernelEx API is it is not our (cuda.core, and maybe even Numba) job to choose which API to use. We just need to add a new option to LaunchConfig to hook with CU_LAUNCH_ATTRIBUTE_COOPERATIVE as you suggested, so that the driver knows to launch it cooperatively. Please open an issue in NVIDIA/cuda-python to track it.

Created an issue: NVIDIA/cuda-python#496