Skip to content

test: reorganize suite by module under test and remove duplicates - #31

Merged
vboussot merged 5 commits into
mainfrom
test/suite-reorg
Jul 6, 2026
Merged

test: reorganize suite by module under test and remove duplicates#31
vboussot merged 5 commits into
mainfrom
test/suite-reorg

Conversation

@vboussot

@vboussot vboussot commented Jul 5, 2026

Copy link
Copy Markdown
Member

Summary

Reorganize tests/unit from per-bugfix files into per-module files, without adding or losing any test behaviour: 60 → 38 unit files, net −460 lines, per-module coverage byte-identical before/after.

What changed

  • New tests/conftest.py: an autouse fixture sets the mandatory KONFAI_config_file / KONFAI_CONFIG_MODE defaults via monkeypatch.setenv (per AGENTS.md §7), plus shared write_config and image_attributes fixtures. This removes the 12 copy-pasted module-level os.environ.setdefault blocks and their # noqa: E402 trails.
  • Tests regrouped by the unit they cover: the audit/fixes grab-bags (test_audit_fixes.py, test_*_fixes.py, test_*_audit.py, test_data_pipeline_audit.py) and single-test files are redistributed into test_config.py, test_patching.py, test_data_manager.py, test_transform.py, test_augmentation.py, test_network.py, test_blocks.py, test_schedulers.py, test_models.py, test_dataset.py, test_measure.py, test_main_cli.py. Regression comments and AUDIT.md references are preserved.
  • 5 exact duplicates merged into their surviving twin, keeping every unique assertion:
    • config missing-file error (path + mode + hint checks folded into one test)
    • invalid-YAML error (×2 → 1, "Invalid YAML syntax" assertion kept)
    • invalid-Literal error (options-mentioned assertion kept)
    • Data._split tail dedup (no-duplication/subset assertions folded into the shard-balance test)

Verification

  • Nominal accounting: pytest-collected test IDs before (340) vs after (335) — the diff is exactly the 5 intended duplicates; nothing lost, nothing double-counted, nothing renamed.
  • Coverage proof: both trees run against the same konfai source in the same env — the --cov=konfai table is identical line-for-line, module-by-module (8844 stmts, 3254 miss, 63%), proving the removed tests covered no unique line.
  • Full suite green: 338 passed (unit + integration) in ~44 s; ruff check/format clean on every touched file (pre-existing violations in untouched files left as-is).

No test was added and none weakened; built-in measures/models intentionally stay untested as agreed.

Follow-up commits

  • test: merge the trainer-related files (test_trainer_checkpoints / test_early_stopping / test_ema_convention / test_resume_lr_override / test_resume_https_checkpointtest_trainer.py) and the runtime files (test_runtime_guards / test_runtime_progress_ddptest_runtime.py) — all 31 tests moved verbatim, collected IDs unchanged; fix every remaining ruff violation in tests/ and wire pytest.mark.integration so -m selection works.
  • ci: gate the core test suite in ruff: the CI lint/format jobs and both pixi task blocks now target konfai konfai-apps/konfai_apps tests.

Final state: 36 unit files (was 60), ruff check/format --check clean over the whole repo including tests, 338 tests passing.

vboussot added 3 commits July 5, 2026 19:04
Group tests by the unit they cover instead of by bug-fix batch:
- add tests/conftest.py (autouse KONFAI env defaults, write_config and
  image_attributes fixtures), replacing 12 copy-pasted module-level
  os.environ.setdefault blocks that violated the monkeypatch rule
- fold the audit/fixes grab-bags and single-test files into per-module
  files: config, patching, data_manager, transform, augmentation,
  network, blocks, schedulers, models, dataset, measure, main_cli
- merge 5 duplicated tests into their surviving twin, keeping every
  assertion (config missing-file/invalid-yaml/invalid-literal errors,
  Data._split tail dedup)

38 unit files instead of 60; 343 -> 338 collected tests (the 5 exact
duplicates); per-module coverage is byte-identical before/after.
- fold test_trainer_checkpoints/test_early_stopping/test_ema_convention/
  test_resume_lr_override/test_resume_https_checkpoint into test_trainer.py
  and test_runtime_guards/test_runtime_progress_ddp into test_runtime.py,
  all tests moved verbatim (18 and 13 collected IDs unchanged)
- fix the remaining ruff violations across tests/ (import order, unused
  noqa, ClassVar on mutable class defaults, zip strict, raw regex match)
  and reformat the flagged files
- mark tests/integration with pytest.mark.integration so -m selection works
Extend ruff check/format targets with tests/ in the CI jobs and in both
pixi task blocks so test-suite hygiene is gated like source code.
@vboussot
vboussot merged commit 16e5830 into main Jul 6, 2026
29 checks passed
@vboussot
vboussot deleted the test/suite-reorg branch July 6, 2026 09:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant