CUDA EP as a standalone plugin package (onnxruntime-ep-cuda) — explores #196 follow-up

[hmaarrfk]

Exploratory follow-up to #196, implementing @cbourjau's suggestion (comment): use upstream's new execution provider plugin system to decouple the expensive CUDA build from Python.

It would virtually remove the need for staged outputs, I think. We would compile one Python-independent cuda package per platform, and N fairly cheap Python-dependent frontends that depend on the former.

This branch contains everything from #196 plus the plugin reorganization. If #196 merges first this rebases to just the plugin commits.

Upstream mechanics (onnxruntime 1.28.0, docs/cuda_plugin_ep/)

• onnxruntime_BUILD_CUDA_EP_AS_PLUGIN=ON builds onnxruntime_providers_cuda_plugin — a standalone shared module with the canonical provider filename (libonnxruntime_providers_cuda.so / onnxruntime_providers_cuda.dll) registered under the canonical name CUDAExecutionProvider. The legacy in-tree CUDA provider is not built in this mode.
• The plugin is Python-independent (links the internal static libs + cudart/cublas/cublasLt/driver only) and loads into any onnxruntime core ≥ 1.24.4 via load-time API version negotiation (plugin-ep-cuda/MIN_ONNXRUNTIME_VERSION).
• cuDNN and cuFFT are build-time header dependencies but loaded lazily at runtime — the plugin has no hard link on either.
• Upstream itself distributes this as separate onnxruntime-ep-cuda12 / onnxruntime-ep-cuda13 wheels.

Recipe reorganization

• New output onnxruntime-ep-cuda (one per platform × CUDA version, no Python dimension): configures with BUILD_CUDA_EP_AS_PLUGIN=ON, ENABLE_PYTHON=OFF, builds only the plugin target, ships just the provider library (lib/ on unix, Library/bin/ on win, matching upstream's install rule).
• CUDA disappears from the staged outputs: onnxruntime-build/-cpp/python become CPU-only. The CUDA python matrix (previously 5 python versions × 2 CUDA versions × 3 platforms of full CUDA rebuilds) is gone.
• Usage until deeper integration lands:

  import onnxruntime as ort
  ort.register_execution_provider_library(
      "CUDAExecutionProvider", f"{os.environ['CONDA_PREFIX']}/lib/libonnxruntime_providers_cuda.so")
  sess = ort.InferenceSession("model.onnx", providers=[("CUDAExecutionProvider", {}), "CPUExecutionProvider"])

• run_constraints: onnxruntime >=1.24.4 documents the minimum core; __cuda run dep avoids needless downloads.

Exploratory CI restriction (revert before merge)

Per-recipe EXPLORATORY skips limit the matrix to the 6 linux/win CUDA jobs — the CPU/osx/novec outputs are byte-identical to #196 and already green there. Removing the top-level exploratory skips restores them.

Open questions for review

1. Auto-registration: the CPU python wheel doesn't carry cuda-plugin-ep=1 build info, so the bundled auto-registration path is inert. Options: patch onnxruntime/__init__.py to also probe $CONDA_PREFIX/lib, ship a small onnxruntime-ep-cuda-python shim (upstream's onnxruntime_ep_cuda module), or document explicit registration.
2. cuDNN as hard run dep: kept for now (via cudnn host run-exports) since most users want Conv; upstream's design treats it as optional. Could be relaxed to run_constrained.
3. Package naming: onnxruntime-ep-cuda (one name, CUDA version via variant/build string) vs upstream's onnxruntime-ep-cuda12/13 split names.
4. Whether -cpp CUDA consumers need anything beyond Env::RegisterExecutionProviderLibrary (the C++ path works with the plugin as-is).
5. CPU python frontends still use the staged megabuild from Migrate to rattler-build with staged outputs (refresh of #171 for 1.28.0) #196 — with CUDA gone it's much cheaper, but per-python full CPU rebuilds would be the alternative if we want to drop staging entirely.

CC @cbourjau — this is the shape you suggested; keen on your take on the open questions.

[conda-forge-admin]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/recipe.yaml) and found it was in an excellent condition.

I do have some suggestions for making it better though...

For recipe/recipe.yaml:

• ℹ️ License is not an SPDX identifier (or a custom LicenseRef) nor an SPDX license expression.

Documentation on acceptable licenses can be found conda-forge.org > Docs > Maintainer Documentation > Contributing packages > SPDX Identifiers and Expressions.

_{This message was generated by GitHub Actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/31313606262. Examine the logs at this URL for more detail.}

[cbourjau]

I think we should start with a clean slate here. A lot has changed upstream. We should be able to do a clean rattler-build migration without any staging outputs when using the EP-plugin.

[hmaarrfk]

I think we should start with a clean slate here. A lot has changed upstream. We should be able to do a clean rattler-build migration without any staging outputs when using the EP-plugin.

You know I’m using 100% Claude.

do you prefer to use an AI yourself?

I’ve had a lot of success with conda-forge stuff and Claude.

I’m mostly trying to find the good ideas In your cleanups before trying to cleanup the whole shared library situation ;)

ill give Claude your prompt, but I just don’t want you to think I’m adding too much human value to this particular effort

(I do stand by our previous discussion regarding unvendorinf. That was before I learned how to use AI lol)