feat: support Windows with a Rust Davidson eigensolver - #101
feat: support Windows with a Rust Davidson eigensolver#101Mostafa-Atallah2020 wants to merge 11 commits into
Conversation
This change is to follow other Qiskit addon source layout patterns more closely.
Coverage Report for CI Build 33094731853Coverage decreased (-0.2%) to 76.029%Details
Uncovered Changes
Coverage RegressionsNo coverage regressions found. Coverage Stats
💛 - Coveralls |
|
Thanks a lot, this already looks like it is in great shape! I made some adjustments mostly to the repo organization to follow other Qiskit addons more closely. I also ran a |
mrossinek
left a comment
There was a problem hiding this comment.
Review of the Rust Davidson eigensolver. Three of the inline findings compound into what I think is a release blocker, so I'll summarize the interaction here and leave the details on the lines.
The compounding failure
lib.rs:98 (stall criterion), lib.rs:107 (preconditioner clamp), and lib.rs:124 (cnorm < lindep → converged = true) interact badly on exactly the inputs this package solves. For the Pauli commutators forward.py passes in, the matrix diagonal is entirely zero (1287/1313 sampled operators), so cycle 0 gives theta ≈ 1e-16, the clamp flips the sign of the preconditioner shift, theta stays pinned near zero, and the stall criterion then fires on cycle 1 and returns (true, ~0).
90.3% of realistic commutator inputs (1186/1313) return a grossly wrong eigenvalue flagged as converged. Because forward.py:189 trusts the converged flag and skips the triangle-inequality fallback, the resulting error bounds are under-estimated — unsafe in the wrong direction for this package's purpose. The old pyscf implementation returned wrong values on some of these too, but reported converged=False, so the fallback caught them. The flag flip is the regression.
Test fixtures bake in the regression
tests/expected_fwd_bounds.pickle (binary, so no inline comment possible): 1080 of 4632 expected values changed, moving from 2.000000 — the triangle-inequality fallback, used when pyscf correctly reported non-convergence — to smaller values such as 1.111140, produced by the new solver claiming convergence with a too-small eigenvalue. The suite therefore passes while the bounds are wrong. I'd suggest not regenerating these fixtures until the solver is fixed, then diffing against the pre-regeneration values to confirm they come back to the fallback where they should. Same concern applies to expected_fwd_tightened_bounds.pickle and expected_merged_bounds.pickle if they were regenerated in the same pass.
Suggested ordering
lib.rs:107— preserve the sign ofd, use a relative floor.lib.rs:98— drop the stall criterion, keep only the residual test.lib.rs:124—converged = false(or restart with a fresh random vector).- Regenerate fixtures and confirm the reverted values.
The remaining inline comments are independent robustness items in the new Rust code.
| let ritz_image = &images_mat * &y; | ||
| let residual = &ritz_image - ritz.scale(theta); | ||
|
|
||
| if (eigval - prev).abs() < tol || residual.norm() < tol { |
There was a problem hiding this comment.
Critical: |eigval - prev| < tol declares false convergence.
This stall criterion fires while the residual is still large — a Davidson iteration whose Ritz value stops moving is not necessarily near an eigenvalue, it may just be stuck (see the preconditioner issue on line 107, which pins theta near zero and makes this fire on cycle 1).
It is the dominant exit path: 131 of 200 random test operators leave the loop here rather than through the residual test. Removing it and keeping only residual.norm() < tol drops the worst converged-result error from 3e-3 to 2.7e-8 on a 200-operator sweep at tol=1e-6.
| if (eigval - prev).abs() < tol || residual.norm() < tol { | |
| if residual.norm() < tol { |
With the stall criterion gone, prev becomes unused and can be dropped along with line 102.
| // Diagonal (Jacobi) preconditioner, clamping near-zero shifts to `tol`. | ||
| let mut correction = residual; | ||
| for i in 0..dim { | ||
| let mut d = diag[i] - C64::new(theta, 0.0); |
There was a problem hiding this comment.
Critical: the clamp on line 108-110 corrupts zero-diagonal operators.
For the Pauli commutators forward.py actually passes in, the diagonal is entirely zero — 1287 of 1313 sampled operators. That makes cycle 0 give theta ≈ 1e-16, so:
d = diag[i] - theta ≈ -1.4e-16
d.norm() < tol → true
d = +tol → sign flipped, magnitude inflated ~1e8
The correction is driven in the wrong direction, theta stays pinned at ~0, and the stall criterion on line 98 then fires on cycle 1 and reports success.
Reproducer:
op = SparsePauliOp(
['YYZYXIZ', 'YXYZZZZ', 'YIZXIXZ', 'XYYXXIY'],
[0.976j, -1.107j, -0.126j, -1.179j],
)
# returns (True, -2.1e-16); true minimum is -2.59690.3% of realistic commutator inputs (1186/1313) return a grossly wrong eigenvalue flagged as converged.
The clamp needs to preserve the sign of d and use a floor relative to the operator scale rather than the convergence tolerance — tol is a residual threshold and has no business as a magnitude floor on a matrix entry. Something along these lines:
let mut d = diag[i] - C64::new(theta, 0.0);
let floor = 1e-12 * scale.max(1.0); // scale from e.g. diag.amax() or ||A||
if d.norm() < floor {
d = C64::new(if d.re < 0.0 { -floor } else { floor }, 0.0);
}For an all-zero diagonal the preconditioner is degenerate no matter how it's clamped, so it may be cleaner to detect that case up front and skip preconditioning entirely (correction = residual), which is a valid, if slower, Davidson variant.
| let s_mat = columns_to_matrix(&s); | ||
| correction -= &s_mat * (s_mat.adjoint() * &correction); | ||
| let cnorm = correction.norm(); | ||
| if cnorm < lindep { |
There was a problem hiding this comment.
Major: cnorm < lindep should not set converged = true.
A correction vector that vanishes after orthogonalization means the subspace can no longer be expanded — the iteration is stuck, not converged. Reporting success here converts what was a safe fallback into a silently wrong bound.
Any purely diagonal operator (I/Z Paulis only) hits this:
op = SparsePauliOp(
['ZIIII', 'IZIII', 'IIZII', 'ZZIII', 'IIIZI'],
[1.0, 0.7, 0.4, 0.3, 0.9],
)
# returns (True, 0.185); true minimum is -2.7I checked the old pyscf implementation on this input: it returned the same wrong value, but with converged=False, so forward.py:189 fell back to the triangle inequality and the bound stayed valid. The flag flip is the regression, not the numerics.
| if cnorm < lindep { | |
| if cnorm < lindep { | |
| converged = false; |
Restarting with a fresh random vector (bounded by a retry count) would recover more of these cases, but simply reporting non-convergence restores the safe behaviour.
| let mut y = DVector::zeros(self.dim); | ||
| for row in 0..self.dim { | ||
| let mut acc = C64::default(); | ||
| for k in self.indptr[row] as usize..self.indptr[row + 1] as usize { |
There was a problem hiding this comment.
No CSR length or bounds validation — malformed input panics instead of raising.
indptr[row + 1] is read for row up to dim - 1 with no check that indptr.len() == dim + 1, and indices[k] indexes x unchecked. Since dim arrives as a parameter independent of the arrays (davidson_smallest line 148), a caller can easily desynchronize them.
Verified failure modes, all surfacing as pyo3_runtime.PanicException rather than a Python exception:
indptrshorter thandim + 1→index out of bounds: the len is 2 but the index is 2indices[k] >= dim→ out-of-bounds onxdiagorseedshorter thandim→ panic in the preconditioner /DVectorconstruction
A panic across the FFI boundary is much worse than an exception — it's not catchable as a normal error and the message is opaque. Worth validating in davidson_smallest and returning PyValueError: indptr.len() == dim + 1, indptr non-decreasing with indptr[dim] == indices.len() == data.len(), every indices[k] < dim, and diag.len() == seed.len() == dim.
| } | ||
|
|
||
| // Collapse the subspace to the current best estimate before it exceeds `max_space`. | ||
| if s.len() >= max_space { |
There was a problem hiding this comment.
max_space <= 1 silently degenerates rather than erroring.
With max_space <= 1 the condition s.len() >= max_space is true on every cycle, so the subspace collapses back to a single vector before it can ever grow to two. A 2D subspace never forms, Rayleigh-Ritz reduces to the Rayleigh quotient of the current vector, and the result is reported as converged.
max_space is user-facing through get_extremal_eigenvalue, so this is reachable from the public API. Worth rejecting max_space < 2 with a PyValueError in davidson_smallest alongside the CSR validation.
| diag_im: PyReadonlyArray1<f64>, | ||
| seed_re: PyReadonlyArray1<f64>, | ||
| seed_im: PyReadonlyArray1<f64>, | ||
| dim: usize, |
There was a problem hiding this comment.
dim = 0 returns (True, 0.0).
An empty matrix has no eigenvalues, but the loop body never executes in a way that changes eigval, so the function reports a fabricated converged eigenvalue of exactly 0.0. That is indistinguishable from a legitimate result and will propagate into a bound. Reject dim == 0 with a PyValueError — cheap to add next to the other validation.
|
|
||
| // Orthonormalize the correction against the subspace (modified Gram-Schmidt). | ||
| let s_mat = columns_to_matrix(&s); | ||
| correction -= &s_mat * (s_mat.adjoint() * &correction); |
There was a problem hiding this comment.
Single classical Gram-Schmidt pass, though the comment on line 120 says "modified Gram-Schmidt".
correction -= &s_mat * (s_mat.adjoint() * &correction) projects against all basis vectors simultaneously using the original correction — that is classical Gram-Schmidt, not MGS, which subtracts sequentially and re-reads the partially-updated vector each time.
With a subspace up to max_space vectors, one classical pass loses orthogonality progressively, which degrades the Rayleigh-Ritz projection in later cycles. Two options: repeat the projection twice (classical Gram-Schmidt with re-orthogonalization, which is numerically comparable to MGS and keeps the efficient matrix form), or loop the columns for true MGS. Either way the comment should match what the code does.
| correction -= &s_mat * (s_mat.adjoint() * &correction); | |
| correction -= &s_mat * (s_mat.adjoint() * &correction); | |
| correction -= &s_mat * (s_mat.adjoint() * &correction); |
Closes #84.
qiskit-addon-slcwas Linux/macOS-only because ofpyscf, which has no Windows wheel and cannot be built from source there. It was used in a single place: thepyscf.lib.davidson1call inget_extremal_eigenvalue. This replaces that with a compiled Rust Davidson eigensolver (built onnalgebra, no BLAS/LAPACK), dropspyscfentirely, and adds awindows-latestrunner to the test matrix so the suite is exercised on Windows in CI.cc @mrossinek @aeddins-ibm - this is the Rust route you suggested on #88. Compared with #85 (which made
pyscfoptional) it removespyscfoutright and proves the full workflow runs on Windows in CI, compared with the numpy/scipy version in #88 it keepspyscf-level speed without any dependency.