Skip to content

Migrate to SciMLTesting v1.2 (off ParallelTestRunner)#144

Merged
ChrisRackauckas merged 1 commit into
SciML:mainfrom
ChrisRackauckas-Claude:sciml-testing-rollout
Jun 15, 2026
Merged

Migrate to SciMLTesting v1.2 (off ParallelTestRunner)#144
ChrisRackauckas merged 1 commit into
SciML:mainfrom
ChrisRackauckas-Claude:sciml-testing-rollout

Conversation

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Contributor

Replaces the ParallelTestRunner-based test/runtests.jl with SciMLTesting v1.2 explicit-args run_tests.

Why

SciML parallelizes tests at the CI-matrix level per GROUP (this repo already uses grouped-tests.yml@v1 and a test/test_groups.toml). ParallelTestRunner's in-process worker parallelism is therefore redundant, and that family of runners is a recurring source of breakage on new Julia releases. This swaps the runner for the shared SciMLTesting harness while keeping the test matrix and exact group semantics.

What changed

  • test/runtests.jl: find_tests/runtests + the hand-written GROUP-delete dispatch -> using SciMLTesting; run_tests(; core, qa, umbrellas). The XLA/CUDA memory withenv wrapping is preserved (pinned to one job per process, since each GROUP cell is now its own CI process).
  • QA files moved into test/qa/ (qa_tests.jl, doctests.jl) with a new test/qa/Project.toml sub-env. Aqua, Documenter, and ExplicitImports now live in that sub-env and are removed from the root test env.
  • Root test/Project.toml: removed ParallelTestRunner; added SciMLTesting and SafeTestsets.
  • test/test_groups.toml: unchanged — its Core/QA/GPU keys still drive the CI matrix.

Behavior-preserving group mapping

  • Core -> functional suite (models/{fno,deeponet,nomad}_tests, layers/{spectral_conv,spectral_kernel}_tests).
  • QA -> Aqua/ExplicitImports + doctests, in the test/qa sub-env.
  • GPU -> the self-hosted CUDA cell still runs the full functional + QA suite, expressed as umbrellas = Dict("GPU" => ["Core", "QA"]). The Reactant tests pick up CUDA through reactant_device() as before.
  • All (bare local Pkg.test()) -> Core then QA.

Each test file now runs inside its own @safetestset (file-level isolation replaces the old per-file worker isolation). The test bodies and assertions are unchanged.

Verification

  • Static: every Project.toml/test_groups.toml parses; all runtests.jl/test .jl files parse with no error nodes; Runic-clean.
  • Runtime: the GROUP dispatch (Core / QA / GPU / All) was exercised against SciMLTesting v1.2 locally with recording thunks and routes exactly as the old runtests.jl did (Core->Core, QA->QA-in-subenv, GPU->Core+QA, All->Core+QA). The full ML suite (Reactant/Enzyme/CUDA + a custom Reactant git rev) is left to per-PR CI to validate.

Ignore until reviewed by @ChrisRackauckas.

🤖 Generated with Claude Code

Replace the ParallelTestRunner-based test/runtests.jl (find_tests +
runtests + a hand-written GROUP-delete dispatch) with SciMLTesting v1.2
explicit-args run_tests. SciML parallelizes tests at the CI-matrix level
per GROUP (grouped-tests.yml@v1), so the in-process worker parallelism of
ParallelTestRunner is redundant and a recurring source of new-Julia-release
breakage.

Behavior-preserving:
- Core: the functional suite (models/{fno,deeponet,nomad}_tests,
  layers/{spectral_conv,spectral_kernel}_tests).
- QA: Aqua/ExplicitImports + doctests, moved into test/qa/ with their own
  test/qa/Project.toml sub-env (Aqua, Documenter, ExplicitImports live there
  now, out of the root test env).
- GPU: the self-hosted CUDA cell still runs the full functional + QA suite,
  expressed as the umbrella GPU => ["Core", "QA"]. The Reactant tests pick up
  CUDA through reactant_device() as before.
- All (bare local Pkg.test()): Core then QA, as before.

test_groups.toml is unchanged; its Core/QA/GPU keys still drive the CI matrix.
Each test file runs inside its own @safetestset (file-level isolation replaces
the old per-file worker isolation). The XLA/CUDA memory env wrapping is kept,
pinned to one job per process since each GROUP cell is now its own CI process.

GROUP dispatch (Core/QA/GPU/All) verified against SciMLTesting v1.2 locally.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ChrisRackauckas ChrisRackauckas marked this pull request as ready for review June 15, 2026 02:14
@ChrisRackauckas ChrisRackauckas merged commit 0413c53 into SciML:main Jun 15, 2026
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants