test: isolate test_pybamm_import from sys.modules mutation (fixes #5402) - #5538
Open
gaoflow wants to merge 2 commits into
Open
test: isolate test_pybamm_import from sys.modules mutation (fixes #5402)#5538gaoflow wants to merge 2 commits into
gaoflow wants to merge 2 commits into
Conversation
…from sys.modules mutation
`test_pybamm_import` was mutating `sys.modules` in-process to pretend that
all optional dependencies were absent, then unloading `pybamm` itself so
the next `importlib.import_module("pybamm")` would re-run the top-level
import logic. The hand-rolled try/finally restored entries afterwards,
but the mutation window left the pytest worker in a partially-consistent
state for the duration of the test — and worse, any partial import state
cached during the re-import (logging handlers, import locks, optional-
dependency fallbacks already evaluated in other modules) could not be
undone by restoring `sys.modules`. With pytest-xdist that surfaces as
order-dependent CI flakiness in unrelated tests; pytest-forked did not
help because fork() inherits the parent's already-cached import state
(see pybamm-team#5402 for the full diagnosis).
This commit removes the in-process mutation:
* `test_pybamm_import` now runs the `import pybamm` check inside a
fresh `subprocess.run([sys.executable, "-c", ...])`. The script
marks every present optional dependency (and any already-imported
sub-modules) as missing inside its own throw-away interpreter,
then imports `pybamm`. The parent worker's `sys.modules` is never
touched. A non-zero exit code propagates `stdout`/`stderr` into
the pytest failure message.
* `test_import_optional_dependency` uses pytest's `monkeypatch.setitem`
to mask `sys.modules` entries. The mutation is automatically reverted
by pytest's fixture teardown even on failure — there is no need for
this test to run in a subprocess because it does not unload `pybamm`.
Fixes pybamm-team#5402.
Match the precedent already in `src/pybamm/codegen/compilation.py` and `tests/unit/test_solvers/test_solution.py`: the script run via `subprocess.run([sys.executable, "-c", ...])` is a literal string constructed in the test itself, not user input, so the standard `# nosec B404` (import) and `# nosec B603` (call) markers are appropriate. No behavior change.
gaoflow
force-pushed
the
fix/test-pybamm-import-sys-modules-isolation
branch
from
June 18, 2026 22:28
2aea61a to
3e6e199
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Isolate
test_pybamm_importandtest_import_optional_dependencyfromsys.modulesmutationFixes #5402.
Problem
tests/unit/test_util.py::TestUtil::test_pybamm_importmutatessys.modulesin-process to pretend that all optional dependencies are absent, then unloads
pybammitself so the nextimportlib.import_module("pybamm")re-runs thetop-level import logic. The hand-rolled
try/finallyrestores entriesafterwards, but for the duration of the test the pytest worker is in a
partially-consistent state, and any partial import state cached during the
re-import (logging handlers, import locks, optional-dependency fallbacks
already evaluated in other modules) cannot be undone by writing back to
sys.modules.With
pytest-xdistthat surfaces as order-dependent CI flakiness inunrelated tests — the issue author confirmed in #5402 that simply
@pytest.mark.skip-ing this single test made all CI tests pass consistently.@pytest.mark.forkeddid not help becausefork()inherits the parent'salready-cached import state.
Fix
Remove the in-process mutation entirely.
test_pybamm_importnow runs theimport pybammcheck inside a freshsubprocess.run([sys.executable, "-c", ...]). The script masks every presentoptional dependency (and any already-imported sub-modules) inside its own
throw-away interpreter, then imports
pybamm. The parent worker'ssys.modulesis never touched. A non-zero exit code propagatesstdoutand
stderrinto the pytest failure message so debugging stays easy.This is strictly stronger isolation than
pytest.mark.forked: a spawnedPython interpreter starts with a clean slate, while
fork()copies theparent's already-cached import state.
test_import_optional_dependencyuses pytest'smonkeypatch.setitemto mask
sys.modulesentries. The mutation is automatically reverted bypytest's fixture teardown even on failure — there is no need for this
test to run in a subprocess because it does not unload
pybamm.Verification
The refactored tests preserve the exact original intent:
test_pybamm_importstill asserts thatimport pybammsucceeds when everypresent optional dependency is marked missing.
test_import_optional_dependencystill asserts thatpybamm.util.import_optional_dependency(pkg)raisesModuleNotFoundErrorwith the expected message.Negative-test sanity (deliberately mask
numpyinstead of an optional dep):so the subprocess approach still surfaces a regression if
pybammeverstarts to require an optional dep at import time.
ruff check tests/unit/test_util.pyandruff format --check tests/unit/test_util.pyare both clean.Why this scope
The smallest change that fixes the root cause: only
test_util.pyis touched.No production code is altered. The two adjacent tests that exhibit the same
in-process-mutation anti-pattern are fixed together so the file is internally
consistent.