Add configurable ONNX provider selection - #23
Merged
Conversation
Add a simple provider option for NeuralCornerDetector and NeuralEmbedder. The default auto mode tries available accelerators first and falls back to CPU, while cpu and cuda allow callers to force predictable behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0a8ed24a-df35-426a-80b6-e40c4f5f7308
Replace the public cuda provider value with gpu so callers can request any available accelerator without learning ONNX Runtime provider names. Keep CUDA as an install/runtime detail behind the gpu extra and provider resolver. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0a8ed24a-df35-426a-80b6-e40c4f5f7308
Move ONNX Runtime out of required dependencies so the gpu extra can install the accelerator runtime without the CPU wheel overriding it. Add a cpu extra for the normal install path and keep gpu as the public accelerator install path. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0a8ed24a-df35-426a-80b6-e40c4f5f7308
Constrain the Linux GPU extra to onnxruntime-gpu<1.27 after testing on train4070 showed 1.27 requires libcudart.so.13 while the CUDA 12 runtime is still common. This keeps provider='gpu' installable on the CUDA 12 host used for validation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0a8ed24a-df35-426a-80b6-e40c4f5f7308
Preload NVIDIA runtime libraries when CUDA or TensorRT providers are requested, add CUDA runtime dependencies to the gpu extra, and reject provider='gpu' sessions that silently initialize CPU-only. This matches the public contract that gpu means acceleration is required. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0a8ed24a-df35-426a-80b6-e40c4f5f7308
Leave the default gpu extra free to install the latest ONNX Runtime GPU wheel so newer CUDA systems are not pinned back. Add gpu-cu12 as the compatibility path for CUDA 12 hosts like train4070. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0a8ed24a-df35-426a-80b6-e40c4f5f7308
Remove ONNX Runtime extras so CollectorVision does not choose or conflict between CPU and GPU runtime wheels. Keep import lightweight, and raise actionable neural-inference guidance when no backend is installed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0a8ed24a-df35-426a-80b6-e40c4f5f7308
Detect when both onnxruntime and onnxruntime-gpu distributions are installed and warn once per process. This catches the common conflict where both distributions provide the same onnxruntime module and one can hide GPU providers. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0a8ed24a-df35-426a-80b6-e40c4f5f7308
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #22.
This adds a small provider option to both neural runtime classes:
provider="auto"is the default and prefers installed accelerator providers before CPUprovider="cpu"forces CPU for predictable behaviorprovider="gpu"requires an accelerated ONNX Runtime provider and raises a clear error if none are availableThe implementation keeps the public API intentionally small (
auto/cpu/gpu) and hides ONNX Runtime provider names behind an internal resolver. That matches CollectorVision's simple quickstart-oriented API while still making GPU acceleration possible for users with the right runtime installed.CollectorVision no longer declares ONNX Runtime as a dependency. Users install exactly one ONNX Runtime backend appropriate for their machine (
onnxruntime,onnxruntime-gpu, DirectML/ROCm variants, etc.), and CollectorVision gives an actionable error the first time neural inference is used without a backend installed. It also warns if bothonnxruntimeandonnxruntime-gpudistributions are present because they provide the same Python module and can conflict.Rationale
Before this change,
NeuralCornerDetectorandNeuralEmbedderboth hardcodedproviders=["CPUExecutionProvider"], so installingonnxruntime-gpuwas not enough to use GPU acceleration through CollectorVision. The new default behavior is accelerator-first because that is what most users expect when a capable provider is installed, but it still falls back to CPU if automatic accelerator session creation fails.Explicit
gpudoes not fall back silently because a user forcing acceleration should get a visible setup error if no accelerator provider is available or the runtime/drivers are not correct. ONNX Runtime can still use CPU as a graph fallback behind an accelerator provider when only part of a model is supported.I did not add broad ONNX Runtime provider-list plumbing to the main API because it would expose too many magic strings for a library that is intended to stay easy to use.
gpuis the meaningful user-facing choice across CoreML, CUDA, DirectML, ROCm, and future accelerator providers; we can add advanced provider-list support later if a concrete use case needs it.Testing on
train4070exposed two packaging/runtime issues that shaped this final approach:onnxruntimedependency, so GPU installs can accidentally install bothonnxruntimeandonnxruntime-gpu, causing the CPU wheel to win and hide CUDA.onnxruntime-gpu1.27 requires CUDA 13 runtime libraries ontrain4070, while 1.26 plus CUDA/cuDNN Python runtime packages works on CUDA 12. CollectorVision should not globally pin that decision because newer CUDA users should be able to use newer ONNX Runtime builds.Validation
uv run --with pytest python -m pytest tests/test_onnx_providers.py tests/test_interfaces.pyuv run --with ruff ruff check collector_vision/onnx_providers.py collector_vision/detectors/neural.py collector_vision/embedders/neural.py tests/test_onnx_providers.pyNeuralEmbedderandNeuralCornerDetectorwithprovider="cpu",provider="gpu"on CoreML, and defaultprovider="auto"collector_visionsuccessfully and raises actionable backend-install guidance whenNeuralEmbedder()is constructedonnxruntimeandonnxruntime-gpudistributions are installedtrain4070Linux/CUDA/RTX 4070, freshpip install -e .followed by manualpip install "onnxruntime-gpu<1.27" nvidia-cudnn-cu12 nvidia-cuda-runtime-cu12exposedCUDAExecutionProvider, passed targeted tests, and ran real detector + embedder inference with active providers["CUDAExecutionProvider", "CPUExecutionProvider"]