Add a runtime-provider layer that sits below model backends and above device detection.
Candidate providers:
- PyTorch eager /
torch.compile where useful.
- ONNX Runtime CPU, CUDA, ROCm, DirectML, OpenVINO providers.
- TensorRT or OpenVINO SDK adapters where model export and licensing allow it.
- Intel XPU / IPEX route if it can be tested without contaminating default installs.
Rules:
- Backends request a runtime capability; they do not own provider installation or global detection.
- Provider code must be optional and import-light until selected.
- Provider selection must be explicit or driven by documented
auto policy.
Acceptance criteria:
- Proposed API for runtime provider selection is documented.
- At least one provider adapter has tests with mocked availability.
- Backend code remains provider-agnostic unless a model genuinely needs a provider-specific graph.
Add a runtime-provider layer that sits below model backends and above device detection.
Candidate providers:
torch.compilewhere useful.Rules:
autopolicy.Acceptance criteria: