SciMLTesting 2.4.0 default-on reexport audit breaks existing run_qa callers

[ChrisRackauckas-Claude]

Summary

Registered SciMLTesting v2.4.0 changes the public run_qa default from check_reexports = false to check_reexports = true. That dependency-only update adds a failing test to unchanged callers, including a deliberate facade package. The public_reexports audit itself is useful; the problem is that its strict default shipped in the existing 2.x line before consumers were migrated.

This is independently reproducible on clean EasyModelAnalysis and Optimization checkouts. Optimization also has an older v2.3 rendered-documentation failure tracked in SciML/Optimization.jl#1278; v2.4 adds a second, separate failure. A fix for the v2.4 default alone will therefore not make Optimization QA fully green.

Exact release boundary

• SciMLTesting #27, merge 61011e57e868769ca23a5b42419b8246c8a948e3, added public_reexports and the check_reexports keyword with the default off. Its PR description explicitly says the audit remained opt-in "to avoid a breaking fleet-wide change" and set the source version to 2.4.0.
• SciMLTesting #28, merge 2881f4c15fb715f0934009a800d42765751587b1, changed the default to on and changed the source version from 2.4.0 to 2.5.0. The PR description says a plain run_qa(pkg) now fails on unapproved reexports.
• Commit 58dc92c then changed only the source version from 2.5.0 back to 2.4.0.
• General registration 56ff08e1af1903b0317b12467d8094a4d9fd236d added the first and only v2.4.0 entry with tree 7f694d4107f2babd9bd918f56e69564eb7ff44b7; that exactly equals 58dc92c^{tree}. There was no earlier registered v2.4.0 tree that got overwritten. For comparison, v2.3.0 is tree 6ebdbe314d25a3dfa1958e93356e61d7f3c0033d.

Changing the default of a public function so an unchanged, previously passing invocation now runs an additional failing check is incompatible observable behavior. The intended strict default therefore requires a v3.0.0 release under SemVer, not v2.4.0 (nor the originally intended v2.5.0). Adding the opt-in audit was an appropriate minor addition; flipping its default is the breaking step.

Clean local reproductions

All runs used Julia 1.12.6, current General, clean detached worktrees, and the packages' existing SciMLTesting/SciML test-group dispatchers.

EasyModelAnalysis

Checkout: SciML/EasyModelAnalysis.jl@cf9b9ea62635f2a8102834983bfce08563f2cf99.

GROUP=QA JULIA_PKG_PRECOMPILE_AUTO=0 JULIA_DEPOT_PATH=<fresh-depot> \
  julia +1.12 --startup-file=no --project=. -e 'using Pkg; Pkg.test()'

• Registered SciMLTesting v2.4.0: 18 pass, 1 fail, 1 existing broken (20 total). The sole failure is No unapproved public reexports, reporting exactly 1,127 symbols. Public API documentation passes.
• After pinning SciMLTesting = "=2.3.0" in both the root and test/qa/Project.toml: 18 pass, 1 existing broken (19 total); the test suite passes. Pinning both environments matters because the outer dispatcher loads SciMLTesting before activating the QA environment.

EasyModelAnalysis intentionally uses @reexport for DifferentialEquations, ModelingToolkit, Distributions, and Plots, so removing these bindings is a public-API decision rather than a mechanical QA cleanup. The same v2.4 failure is visible in EasyModelAnalysis PR #310's QA job.

OptimizationBase

Checkout: SciML/Optimization.jl@34a8fd5cb3484c1c7ce471f7964d0bf81bc9796a.

GROUP=OptimizationBase_QA JULIA_PKG_PRECOMPILE_AUTO=0 JULIA_DEPOT_PATH=<fresh-depot> \
  julia +1.12 --startup-file=no --project=. -e 'using Pkg; Pkg.test()'

• Registered SciMLTesting v2.4.0: 18 pass, 2 fail, 1 existing broken (21 total). Public API documentation reports 254 unrendered names and No unapproved public reexports reports 256 names.
• After pinning exact v2.3.0 in the root, lib/OptimizationBase/Project.toml, and lib/OptimizationBase/test/qa/Project.toml: 18 pass, 1 fail, 1 existing broken (20 total). Only rendered documentation fails (262 names); the reexport test does not exist.

Thus the v2.3→v2.4 boundary adds exactly the reexport failure. CI corroboration: clean master run 29760862950 resolved v2.3 and failed rendered docs; current v2.4 OptimizationBase QA job 88558443059 fails both checks.

Relationship to the older documentation regression

Optimization.jl #1278 already isolates the earlier v2.2→v2.3 failure to SciMLTesting commit 044d89a, which made rendered public API docs mandatory by default. That issue records the v2.2 passing control and v2.3 failing control. It still needs its own substantive documentation/ownership resolution; reverting the v2.4 reexport default will remove only Optimization's additional failure.

Requested rollout

A SemVer-compatible rollout would preserve check_reexports = false as the default throughout 2.x, keep the audit available through explicit check_reexports = true, migrate packages deliberately, and make the strict default part of v3.0.0. Packages migrated during 2.x can opt in explicitly, so enforcement need not wait.

Please do not resolve this by marking tests broken, pinning downstream packages indefinitely, or generating catch-all allowlists. Facade packages need a deliberate review of which reexports are public API; ordinary packages need actual ownership/import fixes. Optimization's older rendered-doc failure should continue to be fixed substantively under #1278.