Skip to content

Audit Wheel Dependencies #3091

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
tarang-jain wants to merge 5 commits into
base: main
Choose a base branch
from
+10 −14
Open

Audit Wheel Dependencies #3091

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
4d4d611
drop dependencies.yaml
tarang-jain Jul 21, 2026
44b0639
audit more deps
tarang-jain Jul 22, 2026
f8bb1af
style
tarang-jain Jul 22, 2026
e8882f8
pylibraft only needs to depend on libraft
tarang-jain Jul 22, 2026
2c23173
Revert "pylibraft only needs to depend on libraft"
tarang-jain Jul 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions ci/build_wheel.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash
# SPDX-FileCopyrightText: Copyright (c) 2023-2026, NVIDIA CORPORATION.
# SPDX-FileCopyrightText: Copyright (c) 2023-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

set -euo pipefail
Expand Down Expand Up @@ -58,8 +58,10 @@ if [[ ${package_name} != "libraft" ]]; then
)
fi

SKBUILD_CMAKE_ARGS="-DUSE_NCCL_RUNTIME_WHEEL=ON"
export SKBUILD_CMAKE_ARGS
if [[ ${package_name} == "raft-dask" ]]; then
SKBUILD_CMAKE_ARGS="-DUSE_NCCL_RUNTIME_WHEEL=ON"
export SKBUILD_CMAKE_ARGS
fi

sccache --stop-server 2>/dev/null || true

Expand Down
14 changes: 5 additions & 9 deletions conda/recipes/libraft/recipe.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION.
# SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
schema_version: 1

Expand Down Expand Up @@ -192,17 +192,15 @@ outputs:
- cmake ${{ cmake_version }}
- ${{ compiler("c") }}
host:
- ${{ pin_subpackage("libraft-headers", exact=True) }}
- ${{ pin_subpackage("libraft-headers-only", exact=True) }}

@bdice bdice Jul 23, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this changed? What is the specific difference in host/run pinnings that you are aiming to produce?

@bdice bdice Jul 23, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary of the impact from AI:

libraft-headers depends on libraft-headers-only, so selecting it also installs the latter's cuda-version, librmm, and rapids-logger dependencies. Relative to that shared closure, libraft-headers additionally brings cuda-profiler-api, libcublas-dev, libcurand-dev, libcusolver-dev, libcusparse-dev, nccl >=2.19, and ucxx 0.52.\*.

Consequently, replacing libraft-headers with libraft-headers-only avoids the four CUDA development packages plus nccl and ucxx. The runtime CUDA libraries are not removed from libraft or libraft-static: both packages continue to directly require libcublas, libcurand, libcusolver, and libcusparse. cuda-profiler-api is also directly removed from those two packages in PR 3091, rather than being removed solely through the header-package substitution.

So basically this means that depending on libraft is no longer enough to build other libraries against libraft, because you don't have CUDA math library dev packages. This is a regression.

@tarang-jain tarang-jain Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that the libraft package only contains a few source files in the raft_runtime directory:

add_library(
    raft_objs OBJECT
    src/raft_runtime/random/rmat_rectangular_generator_int64_double.cu
    src/raft_runtime/random/rmat_rectangular_generator_int64_float.cu
    src/raft_runtime/random/rmat_rectangular_generator_int_double.cu
    src/raft_runtime/random/rmat_rectangular_generator_int_float.cu
    src/raft_runtime/solver/lanczos_solver_int64_double.cu
    src/raft_runtime/solver/lanczos_solver_int64_float.cu
    src/raft_runtime/solver/lanczos_solver_int_double.cu
    src/raft_runtime/solver/lanczos_solver_int_float.cu
  )

I think any downstream package should depend on libraft-headers and not libraft

@tarang-jain tarang-jain Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the important question is -- what do we need the libraft pkg to represent -- only the compiled source files or all the packages required to include any header in this repo?

@tarang-jain tarang-jain Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the description here:
https://pypi.org/project/libraft-cu12/#:~:text=RAFT%20is%20a,GPU%20with%20Dask.

I think this PR is in the right direction -- libraft's shared library should be C++ only. I'll wait for others / PICs to weigh in.

@divyegala divyegala Jul 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bdice yes, this is the right direction IMO. The main artifact of RAFT is libraft-headers split, not libraft

- cuda-version =${{ cuda_version }}
- cuda-profiler-api
- libcublas-dev
- libcurand-dev
- libcusolver-dev
- libcusparse-dev
run:
- ${{ pin_subpackage("libraft-headers", exact=True) }}
- ${{ pin_subpackage("libraft-headers-only", exact=True) }}
- ${{ pin_compatible("cuda-version", upper_bound="x", lower_bound="x") }}
- cuda-profiler-api
- libcublas
- libcurand
- libcusolver
Expand Down Expand Up @@ -242,17 +240,15 @@ outputs:
- cmake ${{ cmake_version }}
- ${{ compiler("c") }}
host:
- ${{ pin_subpackage("libraft-headers", exact=True) }}
- ${{ pin_subpackage("libraft-headers-only", exact=True) }}
- cuda-version =${{ cuda_version }}
- cuda-profiler-api
- libcublas-dev
- libcurand-dev
- libcusolver-dev
- libcusparse-dev
run:
- ${{ pin_subpackage("libraft-headers", exact=True) }}
- ${{ pin_subpackage("libraft-headers-only", exact=True) }}
- ${{ pin_compatible("cuda-version", upper_bound="x", lower_bound="x") }}
- cuda-profiler-api
- libcublas
- libcurand
- libcusolver
Expand Down
2 changes: 0 additions & 2 deletions dependencies.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,6 @@ files:
- build_common
- depends_on_librmm
- depends_on_rapids_logger
- depends_on_nccl
py_run_libraft:
output: pyproject
pyproject_dir: python/libraft
Expand All @@ -92,7 +91,6 @@ files:
- cuda_wheels
- depends_on_librmm
- depends_on_rapids_logger
- depends_on_nccl
py_build_pylibraft:
output: pyproject
pyproject_dir: python/pylibraft
Expand Down
Loading