Skip to content

[ML] Move the RDataLoader tests to pytest - #23819

Open
siliataider wants to merge 8 commits into
root-project:masterfrom
siliataider:rdataloader-tests-pytest
Open

siliataider wants to merge 8 commits into
root-project:masterfrom
siliataider:rdataloader-tests-pytest

Conversation

@siliataider

@siliataider siliataider commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

This Pull request:

Changes:

Moves the RDataLoader tests from unittest to pytest to make them shorter and easier to read and to extend.
The test logic is unchanged.

  • Much shorter tests: 8341 --> 5152 lines (-38%), with 115 tests in 89 test functions instead of 122
  • One file per test class: ml_dataloader.py is split into ml_dataloader_single_dataframe.py, ml_dataloader_multiple_dataframes.py, ml_dataloader_undersampling.py, ml_dataloader_oversampling.py, ml_dataloader_save.py, ml_dataloader_implicit_mt.py and ml_dataloader_from_files.py
  • Fixtures for the input data: each test can directly ask for a common input built with the fixtures instead of writing its own
    • The helpers and the shared fixtures are in the existing conftest.py
  • No more file cleanup code: each test runs in its own tmp_path, created and deleted by pytest
    • This removes all the try/except/finally blocks and the os.removes that existed only to delete temporary files
    • Tests can't clash on file names anymore
  • One test class for both lazy and eager loading: the copy-pasted lazy/eager classes are merged into TestDataLoaderSingleDataframe and TestDataLoaderMultipleDataframes, which run all of their tests twice (for both loading modes) using a parametrized fixture

Checklist:

  • tested changes locally

@siliataider siliataider self-assigned this Oct 23, 2026
@siliataider siliataider added the in:ML Everything under ROOT/ML label Oct 23, 2026
@siliataider
siliataider marked this pull request as draft October 23, 2026 15:54
@siliataider
siliataider force-pushed the rdataloader-tests-pytest branch from e650945 to 38118fb Compare October 23, 2026 16:09
@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Test Results

    24 files      24 suites   3d 22h 50m 58s ⏱️
 3 889 tests  3 887 ✅ 0 💤 2 ❌
83 773 runs  83 771 ✅ 0 💤 2 ❌

For more details on these failures, see this check.

Results for commit 74e1241.

♻️ This comment has been updated with latest results.

@siliataider
siliataider force-pushed the rdataloader-tests-pytest branch 4 times, most recently from deb114f to 4cf4f42 Compare October 29, 2026 17:02
@siliataider
siliataider marked this pull request as ready for review October 29, 2026 17:09
The test classes no longer derive from unittest.TestCase, their names get the "Test" prefix fot  pytest to collect.

Assisted-by: Claude Opus 5.5
Each test now runs in its own temporary directory which pytest creates and cleans up.
The next commits will use these.
the fixtures are dupliucated in this commit because the tests are duplicated for lazy and eager loading modes,
the test classes will be merged in a future commit and so will the fixtures.

Assisted-by: Claude Opus 5.5
This remooves the try/except/finally blocks in favour of using the fixtures defined previously.

Assisted-by: Claude Opus 5.5
…etrized test each

The eager and lazy versions of the tests were near copies of each other
They are now merged into the sam etest class and use a parametrized fixture to switch between modes.
The tests that only existed for one of the modes still only run for it only.

Assisted-by: Claude Opus 5.5
@siliataider
siliataider force-pushed the rdataloader-tests-pytest branch from 4cf4f42 to 74e1241 Compare October 30, 2026 09:35

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI-Assisted in:ML Everything under ROOT/ML

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant