fix: restore custatevec import compatibility#334
Conversation
Reproduced the import behavior across cuQuantum versions to pin down exactly which environments hit this. The failure is real and reproducible, but it's narrower than "deployed GPU containers" in general — it's specifically cuQuantum < 25.03. Root cause confirmed
Three regimes (each verified by running the import probe)
The PR's failing container is the top row; a fresh modern install is the bottom row. Reproduce the failing env (isolated, no GPU needed for the import probe)uv run --no-project --python 3.12 --with "cuquantum-python-cu12==24.11.0" python - <<'PY'
import importlib
for m in ["cuquantum.custatevec", "cuquantum.bindings.custatevec"]:
try:
importlib.import_module(m); print("OK ", m)
except Exception as e:
print("FAIL", m, "->", type(e).__name__, e)
PY
# OK cuquantum.custatevec
# FAIL cuquantum.bindings.custatevec -> ModuleNotFoundError: No module named 'cuquantum.bindings.custatevec'Other pre-25.03 versions that reproduce it: Confirmed on an RTX 4090 box at the other end of the rangeWith So Implication for the shimThe shim is fixing a genuine, reproducible bug — but only for environments stuck on cuquantum < 25.03. Worth noting: PECOS's own pin is Doc follow-up (separate from this PR)
python -c "from cuquantum import custatevec; print('OK')"On any 26.x install that prints a scary Generated with Claude Code (Opus 4.8). Reproduced on an RTX 4090 / CUDA 13 workstation. |
65d2f13 to
be57215
Compare
There was a problem hiding this comment.
Pull request overview
Adds a cuQuantum import-compatibility shim so PECOS’s CuStateVec implementation can work with both newer cuquantum.bindings.custatevec installs and older legacy cuquantum.custatevec installs, plus a unit test and a docs update to reflect the modern import surface.
Changes:
- Introduce a
custatevecimport shim that preferscuquantum.bindings.custatevecand falls back to legacycuquantum.custatevec. - Route the cuStateVec state + gate modules through the shared shim rather than importing
cuquantum.bindings.custatevecdirectly. - Add a GPU-free unit test to lock in import preference/fallback behavior; update CUDA setup docs to use the modern import.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| python/quantum-pecos/src/pecos/simulators/custatevec/_cuquantum_compat.py | Adds the shared import shim for custatevec. |
| python/quantum-pecos/src/pecos/simulators/custatevec/state.py | Switches cuStateVec state implementation to use the shim. |
| python/quantum-pecos/src/pecos/simulators/custatevec/gates_one_qubit.py | Switches one-qubit gate implementation to use the shim. |
| python/quantum-pecos/src/pecos/simulators/custatevec/gates_two_qubit.py | Switches two-qubit gate implementation to use the shim. |
| python/quantum-pecos/src/pecos/simulators/custatevec/gates_meas.py | Switches measurement gate implementation to use the shim. |
| python/quantum-pecos/tests/pecos/unit/test_cuquantum_compat.py | Adds unit tests validating preferred vs fallback import behavior without GPU runtime. |
| docs/user-guide/cuda-setup.md | Updates troubleshooting command to use the modern cuquantum.bindings.custatevec import. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Re-review at
|
| cu12 version | from cuquantum.bindings import custatevec raises |
narrowed shim |
|---|---|---|
| 24.03.0.post1 | ModuleNotFoundError(name='cuquantum.bindings') |
catches → falls back ✓ |
| 24.08.0 | ModuleNotFoundError(name='cuquantum.bindings') |
catches → falls back ✓ |
| 24.11.0 | ImportError (bindings pkg present, custatevec absent) |
misses → propagates ✗ |
On 24.11 the ImportError propagates, pecos.simulators.__init__ turns it into CuStateVec = None, and you're back to the original "gpu-state-vector unavailable" symptom — for the most recent pre-25.03 release, i.e. exactly the stale-base-image case this PR targets. (Loading the actual _cuquantum_compat.py from a981a8d1 against the 24.11 wheel reproduces this.)
The new test passes but doesn't cover this
test_falls_back_to_legacy_custatevec mocks cuquantum as a bare module with cuquantum.bindings deleted, which raises ModuleNotFoundError — the 24.03/24.08 shape. It never exercises the real 24.11 shape (cuquantum.bindings present, custatevec absent → ImportError), so it's green while 24.11 is broken. A faithful mock:
cuquantum = ModuleType("cuquantum"); cuquantum.__path__ = []; cuquantum.custatevec = legacy
bindings = ModuleType("cuquantum.bindings"); bindings.__path__ = []
sys.modules["cuquantum"] = cuquantum; sys.modules["cuquantum.bindings"] = bindingsLoading the shim under that raises the same ImportError. (test_propagates_non_missing_module_errors is good — keep it.)
Suggested direction
Cleaner long-term fix: drop the legacy fallback and fail loud when cuQuantum is below the bindings-era floor — an actionable "upgrade cuquantum-python to >= …" message, surfaced on every CuStateVec construction path (not just QuantumSimulator("CuStateVec") — a direct CuStateVec(2) currently becomes None(2)). This regression is itself a sign that threading the right exception across cuQuantum layouts is fragile.
If you'd rather keep the shim, catch ImportError (the parent) and fall back only for the known absence shapes, re-raising everything else:
ModuleNotFoundErrorwithnamein{cuquantum.bindings, cuquantum.bindings.custatevec}(24.03/24.08), or- plain
ImportErrormatchingcannot import name 'custatevec' from 'cuquantum.bindings'(24.11); - re-raise any other
ImportErrorso missing.so/ partial installs surface their real cause.
Don't blanket-catch (ModuleNotFoundError, ImportError) and always fall back — that re-opens the masking problem on modern installs. Add a 24.11-shaped test for the new branch.
Also (not merge-blockers, but worth folding in)
- Line endings:
state.pyis mixed now —devis 148 CRLF lines, the PR adds 6 LF-only lines. Normalize. - The docs fix is good (
from cuquantum import custatevec→cuquantum.bindings), but it's still a raw cuQuantum probe; a PECOS-level check tracks what users actually care about:python -c "from pecos.simulators import CuStateVec; print('OK' if CuStateVec is not None else 'NOT AVAILABLE')". - Volta/V100 & versions (for the docs/packaging story): legacy
cuquantum.custatevecis removed at 25.9.0 (not 26.x); and cuStateVec drops Volta/V100 (CC 7.0) at 25.09 — Volta is only in cu12 25.03–25.06. So the docs' "CC 7.0+" claim and any cu12 floor need a Volta policy (e.g. pincuquantum-python-cu12>=25.3,<25.9for V100, or drop CC 7.0 for Python CuStateVec). Splitting[cuda]into[cuda12]/[cuda13]would make this explicit.
All exception/version facts above were reproduced on isolated wheel probes; the legacy path itself was separately confirmed to run a correct Bell state on an RTX 4090 with cu12 24.11, so the runtime works — the issue is purely the import guard.
Authored with Claude Code (Claude Opus 4.8) and cross-checked with Codex (GPT-5.5). The P0 regression and the version/Volta facts were independently reproduced by both via isolated wheel probes; the legacy path was confirmed end-to-end on an RTX 4090 / CUDA 13 workstation.
|
I encouraged the bots to think about dropping support for older versions of cuquantum but keeping support for both CUDA 12 and 13. And then failing loud if older versions of cuquantum are used and having useful error messages. I think the newer cuquantum drops support of "Volta", which seems to be the only downside of not supporting the older cuquantum, which I guess are RTX 2000s series... and others that came out around 2017-2018. |
Since it's beyond what I understand and have access to (gpus), can I leave the rest of this PR to you to take to the finish line? |
Yeah, sure. Can do. |
…t guard (Codex review)
…x (Codex round-2 nits)
| This is the single place the cuStateVec simulator imports CuPy and the cuQuantum | ||
| bindings. It never raises at import time: when CuPy or a bindings-era cuQuantum | ||
| (``cuquantum.bindings``, introduced in cuQuantum 25.03) is missing, the names are | ||
| set to ``None`` and a classified, actionable reason is recorded. ``CuStateVec`` | ||
| stays importable so ``import pecos`` works without CUDA; the failure surfaces | ||
| (loudly) only when the user constructs/uses ``CuStateVec``, via | ||
| ``require_custatevec()``. |
| # Python cuQuantum CuStateVec backend. Import always succeeds if the package is | ||
| # present; CuPy / cuQuantum availability is checked at construction time (like the | ||
| # Rust CudaStateVec below), so users get a clear error only when they use it. | ||
| try: | ||
| import cupy | ||
| import cuquantum | ||
|
|
||
| from pecos.simulators.custatevec.state import ( | ||
| CuStateVec, | ||
| ) | ||
| from pecos.simulators.custatevec.state import CuStateVec | ||
| except ImportError: | ||
| CuStateVec = None |
| # Test 3: cuQuantum | ||
| print_info "Test 3: Testing cuQuantum..." | ||
| CUQUANTUM_TEST=$(python3 -c " | ||
| import sys | ||
| try: | ||
| from cuquantum import custatevec | ||
| from cuquantum.bindings import custatevec | ||
| print('cuStateVec imported successfully') | ||
| sys.exit(0) | ||
| except Exception as e: |
…floor in setup_cuda.sh
Summary
Make the Python
CuStateVecbackend fail loud on unsupported cuQuantum, and support both CUDA 12 and CUDA 13.CuStateVecnow requires the bindings-era cuQuantum API (cuquantum.bindings, cuQuantum >= 25.03) and never uses the legacy top-levelcuquantum.custatevec.CuStateVec = None.import pecosstays non-fatal without CUDA; the failure surfaces as a clear, actionable error at construction (which package to install/upgrade) — mirroring the existing RustCudaStateVecpattern.[cuda]extra is replaced by explicit[cuda12]/[cuda13]extras (Python cuStateVec/MPS path), declared mutually exclusive via[tool.uv] conflictsso each resolves to its latest independently.Background
cuquantum.bindings.custatevecwas introduced in cuQuantum 25.03 (top-levelcuquantum.custatevecdeprecated then, removed at 25.9.0). The previous code imported the new path unconditionally, so deployed environments with an older pre-pin cuQuantum sawgpu-state-vectorreported as unavailable. Rather than fall back to the legacy API, this PR requires the bindings era and fails loud below it — a one-linepipupgrade for users, independent of CUDA/hardware.Changes
_cuquantum_compat.py(new): the single place CuPy +cuquantum.bindingsare imported. Never raises at import; records a classified reason (not-installed / pre-25.03 / present-but-broken / cupy-missing). Exposescusv,ComputeType,cudaDataType,custatevec_available(),custatevec_unavailable_reason(),require_custatevec().state.py/ gate modules: import CuPy + cuQuantum via the compat;CuStateVec.__init__callsrequire_custatevec()first.simulators/__init__.pydrops the import-time CuPy/cuQuantum preflight (the class is always importable now).[cuda12](cuquantum-python-cu12>=25.3.0) /[cuda13](cuquantum-python-cu13>=25.9.0), uv conflicts, root extras+groups mirrored,uv.lockregenerated. The Rust GPU backend (pecos-rslib-cuda) is no longer bundled in these extras — install it separately.pecos cuda setup-python/pecos setupand the Justfile auto-detect the CUDA major;setup_cuda.sh, README,DEVELOPMENT.md,simulators.md, andcuda-setup.mdupdated (CC 7.5+ default; V100/Volta via the cappedcuquantum-python-cu12>=25.3,<25.9).test_cuquantum_compat.pyrewritten (available / too-old / missing / cupy-missing / broken-import cases).test_statevec.pyandtest_backend_seed_determinism.pygate oncustatevec_available()instead ofCuStateVec is None.Notes for reviewers
state.pyis not rewritten. Its large diff is a one-time CRLF → LF normalization (the file was committed with CRLF, violating.gitattributes eol=lf) plus the small import/require_custatevec()edits. Review with whitespace hidden to see the real change.[cuda12]resolves to current cuQuantum, so V100 users pincuquantum-python-cu12>=25.3,<25.9(documented).Validation
test_cuquantum_compat.pyunit tests;test_statevec.py/ seed-determinism on an RTX 4090 (CUDA 13) — correct Bell state.import pecosnon-fatal;CuStateVec(2)andQuantumSimulator("CuStateVec")raise the actionable error.uv lock --check,cargo clippy -p pecos-cli -D warnings, and fullpre-commitall clean.Developed and reviewed with Claude Code (Claude Opus 4.8) and cross-reviewed with Codex (GPT-5.5).