Refactor scenario configuration to support suites and enhance CLI

[atti92]

Feature: Implement Test Suites for Scenario Organization

Closes: #50

Description

This PR introduces the concept of "Test Suites" to the verification tool, replacing the flat list of scenarios. This allows for better organization of test cases (e.g., grouping by "Basic Conformance" vs "Edge Cases") and enables users to execute specific groups of tests selectively via the CLI.

Key Changes

Configuration Restructuring

• Updated AppConfig and default.yaml to replace the top-level scenarios list with a suites dictionary.
• Introduced SuiteConfig and SuiteScenario models to handle suite-specific settings and overrides.

CLI Enhancements

• Added a new -s / --suite argument to the CLI.
• Users can now run specific suites (e.g., python -m openutm_verification --suite basic_conformance).

Execution Logic

• Updated the scenarios() dependency provider to iterate through configured suites.
• Implemented logic to handle suite-level overrides for data files (telemetry, flight declaration, etc.).
• Updated the execution context to track the current suite_name.

Dependency Injection Refactoring

• Improved the dependency decorator and DependencyResolver to better handle Annotated types (like ScenarioId) and fix type hinting issues.
• Updated RunContext to include suite information.

Reporting

• Added suite_name field to ScenarioResult to ensure this context is captured in the final JSON/HTML reports.

Documentation

• Updated VerificationFlowDiagram.md to reflect the new suite-based execution flow.

Breaking Changes

• The structure of config.yaml has changed. The top-level scenarios key is deprecated and replaced by suites. Existing configuration files will need to be updated to the new format:

suites:
  my_suite_name:
    scenarios:
      - name: my_scenario_id
        # ... overrides

[atti92]

@RomanPszonka You think we can merge this, or you would like some changes?

[Copilot]

Pull request overview

This PR refactors the verification tool's scenario configuration from a flat list to a hierarchical suite-based structure, enabling better test organization and selective execution via CLI. The changes introduce the concept of "Test Suites" which group related scenarios together (e.g., "basic_conformance" and "extra") and allow per-scenario configuration overrides within each suite.

Key Changes

• Introduced SuiteConfig and SuiteScenario models to support hierarchical test organization with suite-level data file overrides
• Enhanced CLI with --suite flag to selectively run specific test suites
• Refactored dependency injection system to support Annotated types as dependency keys, changing from Type to object type parameters

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
config/default.yaml	Restructured configuration from flat scenarios dict to nested suites structure with scenario lists
src/openutm_verification/core/execution/config_models.py	Added SuiteConfig and SuiteScenario models; updated AppConfig to use suites dict and added target_suites runtime field; extended RunContext with suite context
src/openutm_verification/core/execution/dependency_resolution.py	Changed dependency system types from Type to object to support Annotated types; updated CONTEXT default with suite fields
src/openutm_verification/core/execution/dependencies.py	Refactored scenarios() to iterate through suites; updated data_files() to apply suite-level overrides; added context setting with suite information
src/openutm_verification/core/execution/execution.py	Added suite_name enrichment to scenario results from execution context
src/openutm_verification/core/reporting/reporting_models.py	Added optional suite_name field to ScenarioResult for tracking test suite membership
src/openutm_verification/core/templates/report_template.html	Updated scenario display to show suite name badge when present
src/openutm_verification/cli/parser.py	Added --suite argument supporting multiple suite selection
src/openutm_verification/cli/__init__.py	Added suite validation and target_suites configuration before execution
VerificationFlowDiagram.md	Updated flow diagram to reflect suite-based iteration

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.