Skip to content

Precompile a mass matrix DAE in the OrdinaryDiffEqBDF workload#3966

Merged
ChrisRackauckas merged 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:bdf-precompile-mass-matrix-dae
Jul 20, 2026
Merged

Precompile a mass matrix DAE in the OrdinaryDiffEqBDF workload#3966
ChrisRackauckas merged 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:bdf-precompile-mass-matrix-dae

Conversation

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member

Please ignore until reviewed by @ChrisRackauckas.

Problem

OrdinaryDiffEqBDF's @compile_workload only precompiles pure-ODE problems (lorenz). Nothing in it reaches the algebraic variable detection or the DAE initialization solve, so every mass matrix FBDF user pays that inference on their first solve — even though DAEs are one of the main reasons to reach for BDF.

This came out of a broader TTFX investigation into stiff ODE solves. The interesting part of that investigation was a negative result: using OrdinaryDiffEq invalidates ~3300 MethodInstances, but the FBDF solve path's pkgimage survives them intact (staleinstances = 0 on the ODE path). First-solve cost is almost entirely precompile coverage misses, not invalidation. This PR closes one such gap.

Change

Adds an index-1 Robertson mass matrix problem to the workload, gated behind a new PrecompileMassMatrixDAE preference (default true) so it can be disabled alongside the existing specialization toggles.

Measurements

Julia 1.12.6, first solve of a Robertson mass matrix DAE distinct from the one in the workload (so this measures genuine cache reuse, not an exact-signature hit):

first solve
before 4.21s, 3.94s
after 0.32s, 0.34s, 0.34s

Cost: OrdinaryDiffEqBDF precompile time goes from ~12s to ~17.5s.

Non-DAE path is unaffected (lorenz FBDF first solve stays at 0.0016s).

Solution correctness cross-checked against Rodas5P at abstol=reltol=1e-10: max component difference 1.8e-10, algebraic constraint residual exactly 0.

Tests

ODEDIFFEQ_TEST_GROUP=Core on this branch vs. clean master — identical results:

testset master this branch
DAE Convergence 26/26 26/26
DAE AD 16/16 16/16
DAE Event 4/4 4/4
DAE derivative_discontinuity! 5/5 5/5
DAE Initialization 35 pass, 2 broken 35 pass, 2 broken
BDF Inference 1 error 1 error

The BDF Inference Tests error is pre-existing on master (248c2fd6f), not introduced here — it reproduces on a clean checkout with this change stashed. It is an @inferred failure on build_nlsolver with NonlinearSolveAlg(TrustRegion(...)) at lib/OrdinaryDiffEqBDF/test/inference_tests.jl:19, and is being investigated separately.

Runic-formatted.

🤖 Generated with Claude Code

The BDF workload only precompiled pure-ODE problems, so the algebraic
variable detection and DAE initialization solve were never covered. Every
mass matrix FBDF user paid that inference on their first solve.

Adding an index-1 Robertson mass matrix problem takes the first solve of an
unrelated mass matrix DAE from 4.2s to 0.33s, at the cost of ~5.5s of
package precompile time. Gated behind a PrecompileMassMatrixDAE preference
so it can be turned off alongside the existing specialization toggles.

Measured on Julia 1.12.6, first solve of a Robertson DAE distinct from the
one in the workload:

  before: 4.21s, 3.94s
  after:  0.32s, 0.34s, 0.34s

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

CI failure triage

5 jobs are red. All 5 are pre-existing on master and none are attributable to this diff, which only touches OrdinaryDiffEqBDF's @compile_workload.

job status evidence
downgrade-sublibraries / lib/OrdinaryDiffEqBDF pre-existing fails in master's last completed Downgrade Sublibraries run (29733969835)
downgrade-sublibraries / lib/OrdinaryDiffEqNonlinearSolve pre-existing same run
downgrade-sublibraries / lib/OrdinaryDiffEqTaylorSeries pre-existing same run
sublibrary-ci / lib/OrdinaryDiffEqBDF [Core] pre-existing reproduced locally on clean master with this change stashed — the BDF Inference Tests @inferred error described in the PR body
sublibrary-ci / lib/DelayDiffEq [QA] pre-existing, unrelated ExplicitImports: `nlsolvefail` is not public in `OrdinaryDiffEqNonlinearSolve` but it was imported ... at lib/DelayDiffEq/src/fpsolve/fpsolve.jl:10:37

Master's own recent Sublibrary CI runs are all cancelled (concurrency supersede), so there is no green CI baseline to diff against for the last two — hence the local reproduction for the BDF one.

Follow-ups filed / suggested

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

Triage update — 2 further failures, both also pre-existing

Two more jobs went red since the previous comment. Both verified as pre-existing, so the count is now 7/7 not attributable to this diff.

sublibrary-ci / lib/OrdinaryDiffEqBDF [QA] — reproduced on a clean 248c2fd6f worktree locally, with byte-identical results (17 passed, 0 failed, 2 errored) and the same six flagged names:

all_qualified_accesses_are_public:
  - CompositeControllerCache is not public in OrdinaryDiffEqCore
      (lib/OrdinaryDiffEqBDF/src/controllers.jl:169:74)
all_explicit_imports_are_public:
  - build_nlsolver, du_alias_or_new, markfirststage!, nlsolve!, nlsolvefail
      not public in OrdinaryDiffEqNonlinearSolve
      (lib/OrdinaryDiffEqBDF/src/OrdinaryDiffEqBDF.jl:56-57)

Worth stating explicitly since this PR does add two imports: neither Diagonal (public in LinearAlgebra) nor ODEFunction (exported by SciMLBase) appears in the flagged set, and no_stale_explicit_imports passes, so the new imports are clean on both checks.

sublibrary-ci / lib/OrdinaryDiffEqBDF [Core] / Julia lts — same BDF Inference Tests @inferred failure as the Julia 1 job, already reproduced on clean master and fixed upstream by SciML/SciMLBase.jl#1455.

Summary

category jobs cause
downgrade-sublibraries (BDF, NonlinearSolve, TaylorSeries) 3 red in master's last completed Downgrade run
BDF Core (Julia 1 + lts) 2 LinearProblem/operator inference regression → SciMLBase#1455
BDF QA + DelayDiffEq QA 2 OrdinaryDiffEqNonlinearSolve internals (nlsolvefail et al.) not declared public

The two QA failures share one root cause and would be fixed together by promoting the OrdinaryDiffEqNonlinearSolve nlsolver interface (build_nlsolver, nlsolve!, nlsolvefail, markfirststage!, du_alias_or_new) to public API with docstrings and docs entries, plus CompositeControllerCache in OrdinaryDiffEqCore. Happy to do that as a separate focused PR if wanted — it is orthogonal to this one.

@ChrisRackauckas
ChrisRackauckas marked this pull request as ready for review July 20, 2026 17:16
@ChrisRackauckas
ChrisRackauckas merged commit 5766b7c into SciML:master Jul 20, 2026
127 of 137 checks passed
ChrisRackauckas added a commit that referenced this pull request Jul 21, 2026
Releases unreleased changes on master across four sublibraries:
- OrdinaryDiffEqCore 4.7.1 -> 4.8.0 (minor: restores isdiscretecache public API (#3907); also recompute non-lazy interpolation stages after a truncated step (#3964))
- OrdinaryDiffEqBDF 2.3.1 -> 2.3.2 (precompile a mass matrix DAE in the workload (#3966))
- OrdinaryDiffEqExtrapolation 2.3.0 -> 2.3.1 (drop stale BaseThreads/Sequential imports (#3945))
- OrdinaryDiffEqRosenbrock 2.4.1 -> 2.4.2 (SIMD-fuse RosenbrockCache stage loops; document public HybridExplicitImplicitRK (#3938))



Claude-Session: https://claude.ai/code/session_01SFmcAmLrGPrGtzwP333mX8

Co-authored-by: ChrisRackauckas-Claude <accounts@chrisrackauckas.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants