Add opt-in test group environment isolation#23
Conversation
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
|
Scratchpad update for commit 48aa317:
The stacked ModelingToolkit consumer will remain separate and will depend on SciMLTesting 2.3. |
|
Terminal CI follow-up: all 12 checks are green, including Julia LTS/current/prerelease on Linux, macOS, and Windows, plus downgrade, Runic, and spelling. This matches the local full-suite results of 299/299 on Julia 1.10 and 314/314 on Julia 1.12. No CI-specific failure appeared. |
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Resolve the combined additive public API version as 2.5.0. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Keep version 2.5.0 for the additive isolation API. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
|
Follow-up audit for While reproducing the MTSL #495 IntegrationTest failure, I found that a fresh isolated group process fixed the loaded-module collision but could silently stop testing a path-developed upstream package. A standalone fresh The follow-up preserves the active test sandbox's path-tracked packages in isolated children:
The regression develops an unregistered Real-graph validation used the stacked MTK isolation change from SciML/ModelingToolkit.jl#4749, with clean-main MTSL and this SciMLTesting checkout developed by path. A narrowed Final validation after merging current
The merged additive API version remains 2.5.0. Every new commit includes the required Chris co-author trailer. |
Summary
Motivation
Julia cannot unload a package after a test group activates a different environment. Aggregate suites could therefore combine a package module loaded from one environment with source resolved from another. This is reproducible when downstream groups require incompatible LinearSolve versions.
Implementation notes
The child re-enters the outer test entrypoint with the member group selected. Entry-point discovery prefers Base.PROGRAM_FILE and falls back to the external run_tests call site for eval-driven tests. Isolation is lazy and opt-in, so existing callers and directly selected groups retain their current behavior.
Tests performed locally
Process
I reproduced the cross-environment package-version leak, reduced it to a same-process activation mismatch, implemented the opt-in dispatcher path, added regression fixtures, ran both supported local Julia test suites, and ran final repository-wide formatting and hygiene checks.
Review
Ignore this PR until reviewed by @ChrisRackauckas.