Raise NonlinearSolveBase floor for operator Jacobians

[ChrisRackauckas-Claude]

Important

Please ignore this draft until it has been reviewed by @ChrisRackauckas.

Summary

• declare NonlinearSolveBase as a direct dependency of OrdinaryDiffEqNonlinearSolve
• require NonlinearSolveBase 2.34.1 or newer, the first tested floor with the operator-Jacobian support this sublibrary uses
• bump the sublibrary patch version to 2.3.2

Root cause

PR #3885 began passing SciMLOperators Jacobian prototypes into NonlinearSolve. Its implementation requires newer NonlinearSolveBase, but the package metadata raised only the umbrella NonlinearSolve floor. Registry compatibility for that umbrella release still permits NonlinearSolveBase 2.33.0.

The deployed downgrade action therefore selects 2.33.0 and fails in safe_similar while constructing a cache for a SciMLOperators.MatrixOperator. A frozen-environment bisect identified 9a2f7e7b79141951df24a0f2191a8aa20e94f49a (the commit for #3885) as the first bad commit; 3b799e583 was the adjacent passing control.

Source prerequisite and temporary stack

#3971 fixes the independent resize path that dropped configured nonlinear verbosity. It takes version 2.3.1 and must land before this compatibility-floor patch.

GitHub cannot use a fork-only branch as this PR's upstream base, so the currently published branch temporarily contains #3971 followed by this PR's focused commit. The focused top commit changes only the direct dependency, its 2.34.1 compatibility floor, the corresponding import, and this package's 2.3.2 version. After #3971 merges, this branch must be rebased onto updated master so the final #3917 diff contains only that focused commit.

#3927 is intentionally outside this stack. Its unique SparseConnectivityTracer = "1" compatibility metadata can follow independently as 2.3.3 after these two patches.

Shared-workflow prerequisite

This package patch deliberately does not duplicate SciMLOperators, NonlinearSolveBase, or downstream MuladdMacro floors across unrelated sublibraries.

The currently deployed downgrade action removes path-source packages before minimum resolution, so their transitive constraints are invisible until the action appends those sources afterward. SciML/.github #118 fixes that shared workflow by exposing source constraints before resolution and restoring the original project for the final locked test. Its v1 release is required for every path-source downgrade lane to consume this floor correctly.

Validation

• The exact More RK Methods #118-reconciled downgrade graph selected NonlinearSolveBase 2.34.1, SciMLOperators 1.24.3, LinearSolve 4.3.0, MuladdMacro 0.2.4, and SciMLLogging 1.10.1; its manifest SHA-256 was dcb935a041618f14baa966fc95f7348071b145bf5bbc31d3c2882bc8b0a34880.
• The official locked ODEDIFFEQ_TEST_GROUP=Core package suite exited 0: Newton 6/6, sparse DAE initialization 27/27, linear nonlinear solver 44/44, linear solver 113/113, split solver 10/10, mass matrix 225 passed with 42 pre-existing broken, W-operator 18/18, DAE initialization 19/19, CheckInit 4/4, nested AD 5/5, Jacobian reuse 8/8, homotopy 25/25, sparse Jacobian 11/11, and matrix-free W-operator 14/14. The manifest hash was unchanged after the run.
• The deterministic resize/verbosity regression passed 2/2 and the three focused sparse/resize testsets passed 3/3, 4/4, and 4/4 under the constrained graph.
• Whole-repository Runic 1.7.0, git diff --check, and parsing every tracked Project.toml passed with the final 2.3.2 bump.
• On Preserve nonlinear verbosity when resizing caches #3971 without this floor, the current NonlinearSolve, BDF, and SDIRK downgrade lanes all reproduce the same NonlinearSolveBase 2.33.0 safe_similar(::MatrixOperator) failure. The earlier Raise NonlinearSolveBase floor for operator Jacobians #3917 NonlinearSolve downgrade lane passed after selecting 2.34.1.

Process

The incompatibility was reproduced under the deployed downgrade workflow, bisected with a frozen graph, reduced to the missing direct dependency floor, validated under the reconciled exact graph, then rebased as a focused commit on top of the independently tested #3971 prerequisite.

[ChrisRackauckas-Claude]

Follow-up diagnosis before the next branch update:

• The current PR text saying NonlinearSolveBase 2.34.1 requires LinearSolve 5 is incorrect. Its registered weak compatibility is LinearSolve 4.3–5.
• In CI's alldeps mode, that range selects LinearSolve 4.3.0 and PureKLU 1.1.1. PureKLU requires MuladdMacro >= 0.2.4, while the NonlinearSolve, BDF, and SDIRK sublibraries each pinned their direct floor to 0.2.1. That exact-floor conflict is the resolver failure.
• The local follow-up raises those three MuladdMacro floors to 0.2.4 and keeps the conservative NonlinearSolveBase 2.35 floor because it also enforces the source tree's SciMLOperators 1.24 generation.

The recursive alldeps resolver now succeeds for all three sublibraries with NonlinearSolveBase 2.35.0, SciMLOperators 1.24.3, LinearSolve 4.3.0, PureKLU 1.1.1, and MuladdMacro 0.2.4. The exact locked NonlinearSolve regression has passed 44/44; broader package tests are still running before I update the branch and PR body.

[ChrisRackauckas-Claude]

Correction to my previous follow-up: do not add the proposed duplicated MuladdMacro floors or raise this package to NonlinearSolveBase 2.35.

Those apparent extra floors came from the shared downgrade action resolving registered dependencies after removing local [sources] packages. The action then reattached source packages whose transitive constraints had never participated in that resolution.

SciML/.github #118 fixes that resolution boundary. With its reconciler around the exact deployed action and only this PR's NonlinearSolveBase = "2.34.1" metadata change, the real Core lane selected the coherent minimum stack (SciMLOperators 1.24.3, NonlinearSolveBase 2.34.1, LinearSolve 4.3.0, MuladdMacro 0.2.4) and the locked test passed all 107 assertions. The branch and PR description now reflect this smaller two-file package fix and its shared-workflow prerequisite.

[ChrisRackauckas-Claude]

Four-hour CI checkpoint (2026-07-16 19:38 EDT)

The checks that directly exercise this dependency-floor change are green:

• OrdinaryDiffEqNonlinearSolve locked downgrade: pass (1h11m)
• OrdinaryDiffEqNonlinearSolve Core on Julia LTS/current/pre: pass
• OrdinaryDiffEqNonlinearSolve ModelingToolkit: pass
• All completed non-GPU root CI lanes, documentation, downstream, Runic, spellcheck, benchmarks, and the root downgrade lane: pass

The aggregate PR status is still red, but each completed failure was reproduced or matched against unmodified master and is independent of this patch:

1. 15 other sublibrary downgrade lanes: they use the current shared v1 workflow (f8bbdfe), which removes path sources before minimum-version resolution and therefore hides their source compat. The focused shared-workflow fix is SciML/.github#118; its own CI is green, and the exact deployed downgrade action plus that helper passed OrdinaryDiffEqCore locally with only this PR's package change. v1 must advance before these lanes can become green. The target NLS downgrade lane already passes.
2. BDF Core inference: clean master reproduces the failure; 9a2f7e7b7 is the first bad commit. NonlinearSolve#1089 is the focused upstream cache-inference fix. Applying its three-file patch to the registered-equivalent NLSBase stack made the unchanged BDF inference file pass locally.
3. QA lanes: clean master reproduces the NLS QA failure. The public-boundary error (11 Core/Differentiation imports) is the existing architecture issue #3776. The separate missing SparseConnectivityTracer test compat introduced by NonlinearSolveAlg: reuse the ODE WOperator matrix-free for Krylov linsolves #3885 is fixed narrowly in #3927; local and hosted Aqua compat checks pass 4/4, and that PR's QA now fails only on ExplicitImports: drive sublibrary QA checks to green (ei_broken markers) #3776.
4. Root GPU: the branch has the exact master signature (24 pass, 137 DAE GPU failures, 11 broken). This is tracked by #3922 / #3923, with the CUDA QR ldiv! stack addressed upstream by Fix 3-arg QR ldiv! for multi-column right-hand sides JuliaGPU/CUDA.jl#3199. No local GPU is available, so this classification uses the real hosted GPU runs.

One unrelated StochasticDiffEq NoncommutativeConvergence lane is still queued without a runner after four hours; I am not representing the matrix as fully complete.

At this checkpoint the worktree is clean, the branch is 0 commits behind upstream/master, and the draft PR is mergeable. No additional code change is indicated here. Required coordination is: merge #118 and advance the shared v1, then rerun this PR; whichever of #3917 and #3927 lands second must rebase and reconcile their shared OrdinaryDiffEqNonlinearSolve version bump.

[ChrisRackauckas-Claude]

Independent confirmation from the master-CI investigation at 4e1a434 (Downgrade Sublibraries run 29629348539):

The OrdinaryDiffEqNonlinearSolve, OrdinaryDiffEqBDF, and OrdinaryDiffEqSDIRK downgrade jobs all die with exactly the failure this PR targets — the downgrade env resolves NonlinearSolveBase v2.33.0 (the BDF job log shows ⌅ [be0214bd] NonlinearSolveBase v2.33.0 alongside NonlinearSolve v4.21.0), and then:

MethodError: no method matching similar(::SciMLOperators.MatrixOperator{Float64, Matrix{Float64}, ...})
Stacktrace:
  [1] safe_similar(::SciMLOperators.MatrixOperator{...})
    @ NonlinearSolveBase.Utils ~/.julia/packages/NonlinearSolveBase/Gv4mJ/src/utils.jl:117
  [2] ...
  [3] construct_jacobian_cache(...)

reached from reuse_jac_prototype/init in lib/OrdinaryDiffEqNonlinearSolve/src/utils.jl (the #3885 operator jac_prototype).

Version archaeology agrees with the floor chosen here: safe_similar(::AbstractSciMLOperator) = x was added in SciML/NonlinearSolve.jl#1012 (commit c5e2a74e, 2026-07-08, "Handle SciMLOperator Jacobian prototypes") and first shipped in the registered NonlinearSolveBase 2.34.0 (release-prep NonlinearSolve.jl#1034, 2026-07-11). The registry's NonlinearSolve compat only floors NonlinearSolveBase at 2.31.0 until NonlinearSolve 4.21.1 (which floors it at 2.35.0 but also requires LinearSolve >= 4.3), so a direct NonlinearSolveBase dep + floor as done here is the right minimal fix; bumping the NonlinearSolve floor alone could not force it.

Note this fixes 3 of the 9 failing downgrade jobs; the four resolution failures are #3905's SciMLOperators floors, TaylorSeries is #3958, DiffEqDevTools at 4e1a434 was a runner-communication flake.

🤖 Generated with Claude Code

https://claude.ai/code/session_015WU5V4PXt8SZ3BpjxdtM7d

[ChrisRackauckas-Claude]

Post-rebase publication scratchpad:

1. Temporarily stacked the focused Raise NonlinearSolveBase floor for operator Jacobians #3917 dependency-floor commit on Preserve nonlinear verbosity when resizing caches #3971 because an upstream PR cannot use a fork-only prerequisite branch as its base. The final branch still needs one rebase onto master after Preserve nonlinear verbosity when resizing caches #3971 merges.
2. Kept Add SparseConnectivityTracer test compat #3927 outside the stack; its unique SparseConnectivityTracer = "1" compatibility can follow as 2.3.3.
3. Reconciled the exact minimum graph with Reconcile path-source constraints in downgrade tests .github#118: NonlinearSolveBase 2.34.1, SciMLOperators 1.24.3, LinearSolve 4.3.0, MuladdMacro 0.2.4, and SciMLLogging 1.10.1. Manifest SHA-256 before testing: dcb935a041618f14baa966fc95f7348071b145bf5bbc31d3c2882bc8b0a34880.
4. Ran the official locked ODEDIFFEQ_TEST_GROUP=Core package suite to exit 0: Newton 6/6, sparse DAE initialization 27/27, linear nonlinear solver 44/44, linear solver 113/113, split solver 10/10, mass matrix 225 passed with 42 pre-existing broken, W-operator 18/18, DAE initialization 19/19, CheckInit 4/4, nested AD 5/5, Jacobian reuse 8/8, homotopy 25/25, sparse Jacobian 11/11, and matrix-free W-operator 14/14. The post-test manifest SHA-256 remained dcb935a041618f14baa966fc95f7348071b145bf5bbc31d3c2882bc8b0a34880.
5. Verified the deterministic resize/verbosity regression 2/2 and the focused sparse/resize assertions 3/3, 4/4, and 4/4 under the constrained graph.
6. Ran whole-repository Runic 1.7.0, git diff --check, and TOML parsing successfully with version 2.3.2.
7. Immediately before publication, fetched both remotes and verified origin/master and fork/master were still the tested 298da19f44b0c2fa6b63ba729ab7d64d19266e6b. The old PR branch tip was still 032176e058e5d57cfffb2a477da13b6dcda737ab; its only two non-master commits were the expected ChrisRackauckas-Claude dependency-floor commit and merge commit. Replaced it with --force-with-lease pinned to that exact old SHA; no plain force push was used.

CI expectation: until this floor is present, the NonlinearSolve, BDF, and SDIRK downgrade lanes resolve NonlinearSolveBase 2.33.0 and fail in safe_similar(::SciMLOperators.MatrixOperator). Those failures are the target of this PR, not regressions from #3971.

[ChrisRackauckas-Claude]

Scheduled four-hour CI checkpoint — 2026-07-21 01:07 EDT

Head 3bf63c966d2274ffdad918755f6eb305bb9b1a3a is unchanged. The PR remains draft and GitHub reports it mergeable (unstable only because checks are not all green).

• 170 checks total: 138 success, 28 failure, 3 skipped, 1 queued.
• Comparing check names against stacked source-fix PR Preserve nonlinear verbosity when resizing caches #3971, 17 failures are shared, Raise NonlinearSolveBase floor for operator Jacobians #3917 makes Preserve nonlinear verbosity when resizing caches #3971's failing OrdinaryDiffEqNonlinearSolve downgrade lane green, and Raise NonlinearSolveBase floor for operator Jacobians #3917 adds 11 downstream downgrade-resolution failures.
• Those 11 added failures stop in dependency resolution before tests. They are the path-source floor propagation gap addressed by the already-documented prerequisite SciML/.github#118: the deployed wrapper pins an older NonlinearSolveBase, LinearSolve, or MuladdMacro before incorporating OrdinaryDiffEqNonlinearSolve's local source constraints.
• The remaining shared failures are unchanged baseline categories from Preserve nonlinear verbosity when resizing caches #3971: four downgrade lanes (DiffEqDevTools, OrdinaryDiffEqBDF, OrdinaryDiffEqSDIRK, and OrdinaryDiffEqTaylorSeries), 11 QA lanes, the pre-existing BDF inference failure, and the broad GPU baseline (24 pass, 137 gpu_error failures, 11 broken).
• The only unfinished check is StochasticDiffEq NoncommutativeConvergence. It remains queued with required labels self-hosted, Linux, X64, and high-memory, no assigned runner, and no conclusion.

No source-bearing CPU test exposed a new #3917 regression. The exact published Core graph already passed locally as recorded in the publication evidence. No branch change or rerun was made at this checkpoint.