test: add v2 regression guards and supersession markers

[ajcasagrande]

Test-only carve from the agentx branch: three regression guards plus one integration-conftest env hunk. No src/ changes; every imported symbol already exists on main.

What each guard pins

• tests/unit/common/test_dataset_models_prereq.py (new, 4 tests) — pins the Turn / TurnMetadata prerequisite plumbing: default-empty prerequisites, TurnPrerequisite(kind=PrerequisiteKind.SPAWN_JOIN, ...) carried on both models, and Turn.metadata() copying prerequisites + branch_ids through. Imports only aiperf.common.enums.PrerequisiteKind and aiperf.common.models.{Turn, TurnMetadata, TurnPrerequisite}.
• tests/unit/orchestrator/test_orchestrator_execution.py (new, 1 test) — supersession marker for the v1 orchestrator execution suite removed by feat: multi-tier SLO search — resolve N tier boundaries in one job #1035. Asserts the v1 methods (_resolve_strategy, _execute, _execute_loop, _create_sweep_strategy, _create_confidence_strategy) stay absent from MultiRunOrchestrator, and that the v2 replacements (MultiRunOrchestrator.execute, aiperf.cli_runner._strategy.build_strategy) exist. The module docstring maps every v1 test concern to its v2 coverage so a re-port is forced (not silently skipped) if the v1 model ever returns.
• tests/unit/common/config/test_loader.py (new, 1 test) — supersession marker for the removed v1 aiperf.common.config.loader. Asserts the module stays gone (importlib only, no src dependency) and documents where each v1 loader behavior is now covered (tests/unit/config/test_loader_edge_cases.py / test_loader_adversarial.py / test_end_to_end_config_flow.py).
• tests/integration/conftest.py (+7) — os.environ.setdefault("MLFLOW_ALLOW_FILE_STORE", "true"). MLflow 3.x raises on file:// tracking stores unless this opt-out is set; the MLflow integration tests (tests/integration/post_processors/test_mlflow_live_correctness.py, tests/integration/plot/test_plot_mlflow_upload.py) use file:// stores under tmp_path. setdefault so explicit overrides win.

Evidence

• Targeted run: uv run pytest -n auto tests/unit/common/test_dataset_models_prereq.py tests/unit/orchestrator/test_orchestrator_execution.py tests/unit/common/config/test_loader.py — 6 passed.
• uv run pytest --collect-only -q tests/unit/ — 13874 collected, no errors.
• ruff format + ruff check on the four touched files — clean.
• make check-ruff-baselined — OK (0 new); make check-ergonomics — OK (0 new).
• Symbols re-verified on main (031407401): PrerequisiteKind.SPAWN_JOIN, Turn/TurnMetadata prerequisites+branch_ids fields and Turn.metadata(), MultiRunOrchestrator.execute, cli_runner._strategy.build_strategy, and aiperf.common.config absent from src/.

🤖 Generated with Claude Code

Summary by CodeRabbit

• Tests
  • Updated integration setup so file-based tracking works reliably in test environments.
  • Added coverage for dataset turn prerequisites and metadata handling.
  • Added checks confirming legacy config-loader and orchestrator v1 APIs are no longer present.

[github-actions]

Try out this PR

Quick install:

pip install --upgrade --force-reinstall git+https://github.com/ai-dynamo/aiperf.git@c7e1a7b7dcb83cd037ce71677e95158fb2809521

Recommended with virtual environment (using uv):

uv venv --python 3.12 && source .venv/bin/activate
uv pip install --upgrade --force-reinstall git+https://github.com/ai-dynamo/aiperf.git@c7e1a7b7dcb83cd037ce71677e95158fb2809521

---

Last updated for commit: c7e1a7b • Browse code

[coderabbitai]

Walkthrough

This PR adds test-only changes: a default environment variable for MLflow file-store support in integration test setup, and three new/updated unit test modules that verify config loader v1 removal, dataset model prerequisite handling, and orchestrator v1 execution method removal.

Changes

Test suite additions

Layer / File(s)	Summary
Integration test MLflow file-store setup tests/integration/conftest.py	Sets MLFLOW_ALLOW_FILE_STORE to "true" by default so file-based tracking under tmp_path avoids MLflow 3.x maintenance mode.
Config loader v1-removal guard test tests/unit/common/config/test_loader.py	Adds a test asserting that importing aiperf.common.config.loader raises ModuleNotFoundError, documenting v1 loader removal and mapping to v2 coverage.
Dataset model prerequisite tests tests/unit/common/test_dataset_models_prereq.py	Adds tests verifying Turn defaults to empty prerequisites, prerequisites and branch_id are retained on Turn and TurnMetadata, and Turn.metadata() copies prerequisites and branch_ids.
Orchestrator v1-to-v2 execution guard test tests/unit/orchestrator/test_orchestrator_execution.py	Adds a test confirming MultiRunOrchestrator no longer has v1 execution methods, while v2 execute and build_strategy remain available.

Estimated code review effort: 2 (Simple) | ~10 minutes

Poem

Hop, hop, through tests both new and true,
No more v1 stragglers hiding in view,
File-stores unlocked, prerequisites in line,
Orchestrators guarded, everything fine,
This bunny approves with a twitch of the nose! 🐰

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the test-only v2 regression guards and supersession markers added by this PR.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

🧹 Nitpick comments (1)

tests/unit/common/test_dataset_models_prereq.py (1)

7-30: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Missing return type hints on test functions.

All four test functions lack -> None annotations, unlike the sibling guard-test files in this PR.

✏️ Proposed fix

-def test_turn_defaults_empty_prerequisites():
+def test_turn_defaults_empty_prerequisites() -> None:
     t = Turn()
     assert t.prerequisites == []


-def test_turn_carries_prerequisites():
+def test_turn_carries_prerequisites() -> None:
     p = TurnPrerequisite(kind=PrerequisiteKind.SPAWN_JOIN, branch_id="b1")
     t = Turn(prerequisites=[p])
     assert len(t.prerequisites) == 1
     assert t.prerequisites[0].branch_id == "b1"


-def test_turn_metadata_carries_prerequisites():
+def test_turn_metadata_carries_prerequisites() -> None:
     p = TurnPrerequisite(kind=PrerequisiteKind.SPAWN_JOIN, branch_id="b1")
     m = TurnMetadata(prerequisites=[p])
     assert m.prerequisites == [p]


-def test_turn_metadata_copied_from_turn():
+def test_turn_metadata_copied_from_turn() -> None:
     p = TurnPrerequisite(kind=PrerequisiteKind.SPAWN_JOIN, branch_id="b1")
     t = Turn(prerequisites=[p], branch_ids=["b1"])
     m = t.metadata()
     assert m.prerequisites == [p]
     assert m.branch_ids == ["b1"]

Based on coding guidelines, which state: "Add type hints on ALL functions (params and return types)".

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/unit/common/test_dataset_models_prereq.py` around lines 7 - 30, All
four test functions in test_dataset_models_prereq.py are missing explicit return
type annotations. Update test_turn_defaults_empty_prerequisites,
test_turn_carries_prerequisites, test_turn_metadata_carries_prerequisites, and
test_turn_metadata_copied_from_turn to use -> None so they match the typing
style used in the sibling test files and comply with the function type-hint
guideline.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tests/unit/common/test_dataset_models_prereq.py`:
- Around line 7-30: All four test functions in test_dataset_models_prereq.py are
missing explicit return type annotations. Update
test_turn_defaults_empty_prerequisites, test_turn_carries_prerequisites,
test_turn_metadata_carries_prerequisites, and
test_turn_metadata_copied_from_turn to use -> None so they match the typing
style used in the sibling test files and comply with the function type-hint
guideline.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 49a2e831-acfc-4f37-ba05-57d61affb4fe

📥 Commits

Reviewing files that changed from the base of the PR and between 0314074 and c7e1a7b.

📒 Files selected for processing (4)

• tests/integration/conftest.py
• tests/unit/common/config/test_loader.py
• tests/unit/common/test_dataset_models_prereq.py
• tests/unit/orchestrator/test_orchestrator_execution.py

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!