rapids-xgboost: Stop publishing conda packages.

[trivialfis]

v26.08 will be the last version for RAPIDS to publish a conda package of XGBoost in the rapidsai channel: rapidsai/docs#814 .

Aside from external users, currently cuML and nvforest use the rapids-xgboost package in their CI.

Acceptance Criteria

• Merge the RSN. Add RSN for rapids-xgboost removal docs#814
• Update cuml and nvforest to use xgboost from conda forge.
  • Replace rapids-xgboost with xgboost in the CI. nvforest#185
  • Replace rapids-xgboost with xgboost in the CI. NVIDIA/cuml#8440
• Add deprecation notice in 26.08:
  • Add deprecation warning for rapids-xgboost xgboost-feedstock#151
  • Rebuild RAPIDS XGBoost 26.08 w/CCCL 3.4.3-rc0 & Deprecate RAPIDS XGBoost in 26.08 xgboost-feedstock#152
• Archive the xgboost conda package recipe fork and stop publishing the package in 26.10. https://github.com/rapidsai/xgboost-feedstock
• Remove rapids-xgboost package from integration
  • Delete recipe from https://github.com/rapidsai/integration/tree/main/conda/recipes/rapids-xgboost
  • Remove rapids-xgboost from rapids metapackage: https://github.com/rapidsai/integration/blob/3ca79d0fea7ed9339bcee66556186085958ff963/conda/recipes/rapids/recipe.yaml#L45
  • Update README and CI scripts, grep for any remaining xgboost references
• delete existing rapids-xgboost 26.10 packages on rapidsai-nightly channel

[jameslamb]

Thanks so much for writing this up, I think these tracking issues are really helpful.

I just added an item... there are already 26.10 nightlies of rapids-xgboost up on the rapidsai-nightly channel: https://anaconda.org/channels/rapidsai-nightly/packages/rapids-xgboost/overview

We should delete those once everything else is done, to get earlier warning of any other things depending on these packages that may have been missed. @ me (or anyone in RAPIDS Ops) when you're ready for that and we can do it in the Anaconda UI.

[trivialfis]

TL;DR

Removing existing xgboost packages from prior versions of the rapidsai channel is a breaking change for existing workflows, we want to mitigate the impact somehow.

Context

This issue concerns breaking changes to existing workflows with old rapids versions. It's not about future rapids releases. Deleting a package from a channel prevents users from re-creating existing conda environments and containers.

We have two xgboost packages (not accounting for sub-packages) in the rapidsai channel, namely, xgboost and the meta package rapids-xgboost, and the following methods to install one of them.

1. rapidsai::xgboost
2. Put rapidsai before conda-forge in the environment file, install xgboost, and let conda resolve based on channel priority.
3. Install rapids-xgboost.

In addition, we have two potential breaking changes: conda resolution failures and the lack of RMM integration. I will focus on the conda issue and mention the RMM issue at the end.

Removing XGBoost from the rapidsai channel is not a huge problem for 2, since conda will just fetch xgboost from conda-forge. If a user uses 1 or 3, the change will cause conda resolution to fail. @betatim raised the possibility that the rapids meta package might have a frozen version specifier in a production environment or in Colab.

On the other hand, if we don't remove the xgboost package from rapidsai, rapids users with strict channel priority will be stuck with xgboost<=3.3.0. Essentially, we manufactured a name conflict. The worst part is that the user might not be aware of this constraint, and XGBoost might be frozen by accident.

As a result, the existing xgboost packages must be removed from the main label in the channel. The remaining question is how to limit the impact. @bdice proposed that we move the xgboost package and friends to a special deprecation label rather than the main label, and patch the rapids-xgboost meta package to reroute accordingly.

Doing a case-by-case analysis:

• 1 will fail. There's nothing we can do. The middle ground might be that we suggest others specify a label explicitly, like rapidsai/label/deprecated for rapids<26.10 if we opt to move the packages instead of deleting them.
• 2 is fine, as mentioned, conda will fetch xgboost from conda forge. Whether the meta package rapids version is frozen should not affect xgboost.
• 3 depends. We might retain the rapids-xgboost meta package, let it reroute to rapidsai/label/deprecated for rapids<26.10. Alternatively, we move the rapids-xgboost into the deprecation label as well.

In summary, a special label suggested by @bdice helps downstream users 1 mitigate the breaking change and prevents issues with 3. Also, the RMM compatibility is kept for 1 and 3 since we are using existing packages. There's no RMM fix for 2, users will have to migrate accordingly.

What's next

Decide whether the suggested mitigation is necessary, and how to implement it. I can carry out the work if someone could point me to how the existing packages are managed.

[bdice]

Removing XGBoost from the rapidsai channel is not a huge problem for 2, since conda will just fetch xgboost from conda-forge.

This is true for flexible priority, but not for strict priority. We encourage (and support) strict priority in RAPIDS. This piece also applies only for strict priority. See edits inline (additions in bold, deletions in strikethrough).

On the other hand, if we don't remove the xgboost package from rapidsai, rapids users with strict channel priority will be stuck with xgboost<=3.3.0 even if they don't specifically request the xgboost package from the rapids channel due to channel priority.

---

Adding more specificity to "the xgboost package and friends": we mean xgboost, libxgboost, py-xgboost. We also have a few older builds of r-xgboost, dask-xgboost, and xgboost-proc which should probably be moved as well (though I'm less certain about what impact this would have or what mitigations might be needed -- I don't think those are used at all?).

---

3 depends. We might retain the rapids-xgboost meta package, let it reroute to rapidsai/label/deprecated for rapids<26.10 and to conda-forge for rapids>=26.10.

This is close. I think we should move rapids-xgboost to the deprecated label too. But whether we move it or not, we should patch all of its historical releases to depend on the corresponding rapidsai/label/deprecated::xgboost package. I do not think we should publish anything new for rapids>=26.10, we should simply remove xgboost from the rapids metapackage and delete rapids-xgboost recipes so that we no longer publish it.

---

I can carry out the work if someone could point me to how the existing packages are managed.

I think the next thing I would do is have an agent create a local replica of the rapidsai channel with all of the (meta)packages named above, and ask an agent to write a script that can produce patched packages to be uploaded by one of our conda channel admins (it would be good to run all of these ideas by @msarahan too). We could also set a lower bound, like only patching metapackage releases for 24.02 and newer.

[trivialfis]

Thank you for the replies, @bdice . I will use gdoc next time. ;-(

[trivialfis]

I think the next thing I would do is have an agent create a local replica of the rapidsai channel with all of the (meta)packages named above,

Let me talk with the build team first.