Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -1,2 +1,10 @@
# Set default text file line endings to LF
* text eol=lf

# Binary files
*.png binary
*.jpg binary
*.jpeg binary
*.gif binary
*.ico binary

4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@ This toolkit provides a configuration-driven framework for running automated con
* **Live Data Integration**: Support for OpenSky Network live flight data in test scenarios
* **Configuration-Driven**: YAML-based configuration for easy customization and environment management

## Documentation

For detailed information about the verification scenarios, please refer to the [Scenario Documentation](docs/index.md).

## Quick Start

### Prerequisites
Expand Down
3 changes: 3 additions & 0 deletions config/default.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,12 @@ data_files:
flight_declaration: "config/bern/flight_declaration.json" # Path to flight declarations JSON file
# geo_fence: "config/geo_fences.json" # Path to geo-fences

# List of test scenario IDs to execute
suites:
basic_conformance:
scenarios:
# fire_response is a placeholder scenario and not yet implemented.
- name: fire_response
- name: F1_happy_path
trajectory: "config/bern/trajectory_f1.json"
- name: F2_contingent_path
Expand Down
Binary file added docs/.DS_Store
Binary file not shown.
9 changes: 9 additions & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# OpenUTM Verification Scenarios

## Overview

This section documents the various test scenarios used to verify UTM functionality.

## Scenarios

* [Fire Response](scenarios/fire_response.md)
29 changes: 29 additions & 0 deletions docs/scenarios/fire_response.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Fire Response

**Demonstration Goals:** UTM services allow a diverse set of users to appropriately coordinate their responses to an emergency.

<table style="width:100%; border-collapse: collapse;">
<thead>
<tr style="background-color: #00684a; color: white;">
<th style="padding: 10px; text-align: left; width: 50%;">Details</th>
<th style="padding: 10px; text-align: center; width: 50%;">Map</th>
</tr>
</thead>
<tbody>
<tr>
<td style="background-color: #d1f0e6; vertical-align: top; padding: 15px; color: black;">
<ul style="margin-top: 0;">
<li>A new recreational flyer launches a VLOS flight for the first time in a park.</li>
<li>The fire department is alerted to a possible fire and sends out a drone to quickly assess the scale of the fire. The recreational flyer is notified of the fire department drone and decides to begin landing.</li>
<li>While landing, the recreational flyer briefly leaves the operation area triggering a notification to the fire department UTMSP but quickly returns to his area once notified and completes landing. Following the notification, the Fire Department uses RID to identify the recreational operator and following landing a member of the fire department can interact with the pilot advising him on other locations he might be able to fly in further from the fire.</li>
<li>When the fire is verified, the fire department informs all operators nearby of the fire via dynamic restrictions. A news organisation comes to cover the fire and is informed of the fire area via the dynamic restriction. After seeing the dynamic restriction constraint, the news operator plans their operation around it to avoid a conflict.</li>
<li>Seeing a nearby drone, the fire department is quickly able to determine that it belongs to a news organisation (with previously established processes with the fire department).</li>
<li>In response to a Fire Department request an air ambulance is dispatched to site and the news operator in the area makes accommodations for the helicopter after being informed of its imminent arrival via a dynamic restriction.</li>
</ul>
</td>
<td style="vertical-align: top; padding: 0; text-align: center; background-color: white;">
<img src="fire_response.png" alt="Fire Response Map" style="width: 100%; height: auto; display: block;">
</td>
</tr>
</tbody>
</table>
Binary file added docs/scenarios/fire_response.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
7 changes: 6 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,8 @@ dependencies = [
"loguru>=0.7.2",
"pydantic>=2.11.7",
"pydantic-settings>=2.10.1",
"websocket-client==1.9.0"
"websocket-client==1.9.0",
"markdown>=3.10",
]

[project.scripts]
Expand Down Expand Up @@ -73,6 +74,10 @@ build-backend = "hatchling.build"
packages = [
"src/openutm_verification",
]
# (artifacts configuration removed; docs are included via force-include)

[tool.hatch.build.targets.wheel.force-include]
"docs/scenarios" = "openutm_verification/docs/scenarios"

[tool.pytest.ini_options]
pythonpath = [
Expand Down
1 change: 1 addition & 0 deletions src/openutm_verification/core/execution/config_models.py
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,7 @@ def resolve_paths(self, config_file_path: Path) -> None:

class RunContext(TypedDict):
scenario_id: str
docs: Optional[str]
suite_scenario: Optional[SuiteScenario]
suite_name: Optional[str]

Expand Down
17 changes: 14 additions & 3 deletions src/openutm_verification/core/execution/dependencies.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from typing import Callable, Generator, Iterable, TypeVar, cast
from typing import Callable, Generator, Iterable, Optional, TypeVar, cast

from loguru import logger

Expand All @@ -16,6 +16,16 @@
T = TypeVar("T")


def get_scenario_docs(scenario_id: str) -> Optional[str]:
docs_path = SCENARIO_REGISTRY[scenario_id].get("docs")
if docs_path and docs_path.exists():
try:
return docs_path.read_text(encoding="utf-8")
except Exception as e:
logger.warning(f"Failed to read docs file {docs_path}: {e}")
else:
logger.warning(f"Docs file not found: {docs_path}")
Comment thread
atti92 marked this conversation as resolved.
return None
def scenarios() -> Iterable[tuple[str, Callable[..., ScenarioResult]]]:
"""Provides scenarios to run with their functions.

Expand Down Expand Up @@ -50,13 +60,14 @@ def scenarios() -> Iterable[tuple[str, Callable[..., ScenarioResult]]]:
logger.info("=" * 100)
logger.info(f"Running scenario: {scenario_id}")

scenario_func = SCENARIO_REGISTRY[scenario_id]
scenario_func = SCENARIO_REGISTRY[scenario_id].get("func")
docs_content = get_scenario_docs(scenario_id)

CONTEXT.set({
"scenario_id": scenario_id,
"suite_scenario": suite_scenario,
"suite_name": suite_name,
"docs": None
"docs": docs_content
})
yield scenario_id, scenario_func
else:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@

T = TypeVar("T")


DEPENDENCIES: dict[object, Callable[..., ContextManager[object]]] = {}
CONTEXT: ContextVar[RunContext] = ContextVar(
"context",
Expand All @@ -16,6 +17,7 @@
"scenario_id": "",
"suite_scenario": None,
"suite_name": None,
"docs": None
},
),
)
Expand Down
6 changes: 6 additions & 0 deletions src/openutm_verification/core/execution/execution.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
ScenarioResult,
Status,
)
from openutm_verification.utils.paths import get_docs_directory


def _sanitize_config(data: Any) -> Any:
Expand Down Expand Up @@ -79,11 +80,13 @@ def run_verification_scenarios(config: AppConfig, config_path: Path):
duration_seconds=0,
steps=[],
error_message=str(e),
docs=None,
)

# Enrich result with context data
context_data = CONTEXT.get()
result.suite_name = context_data.get("suite_name")
result.docs = context_data.get("docs")

scenario_results.append(result)
logger.info(f"Scenario {scenario_id} finished with status: {result.status}")
Expand All @@ -95,6 +98,8 @@ def run_verification_scenarios(config: AppConfig, config_path: Path):
failed_scenarios = sum(1 for r in scenario_results if r.status == Status.FAIL)
overall_status = Status.FAIL if failed_scenarios > 0 else Status.PASS

docs_dir = get_docs_directory()

report_data = ReportData(
run_id=config.run_id,
tool_version=version("openutm-verification"),
Expand All @@ -112,6 +117,7 @@ def run_verification_scenarios(config: AppConfig, config_path: Path):
passed=sum(1 for r in scenario_results if r.status == Status.PASS),
failed=failed_scenarios,
),
docs_dir=str(docs_dir) if docs_dir else None,
)

logger.info(f"Verification run complete with overall status: {overall_status}")
Expand Down
8 changes: 7 additions & 1 deletion src/openutm_verification/core/execution/scenario_runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@
import time
from dataclasses import dataclass, field
from functools import wraps
from typing import Any, Callable, List, Optional, ParamSpec, Protocol, TypeVar, cast, overload
from pathlib import Path
from typing import Any, Callable, List, Optional, ParamSpec, Protocol, TypedDict, TypeVar, cast, overload

from loguru import logger

Expand All @@ -23,6 +24,11 @@ class ScenarioState:
telemetry_data: Optional[Any] = None


class ScenarioRegistry(TypedDict):
func: Callable[..., Any]
docs: Optional[Path]


_scenario_state: contextvars.ContextVar[Optional[ScenarioState]] = contextvars.ContextVar("scenario_state", default=None)


Expand Down
28 changes: 28 additions & 0 deletions src/openutm_verification/core/reporting/reporting.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
import shutil
from pathlib import Path

import markdown
from jinja2 import Environment, FileSystemLoader, select_autoescape
from loguru import logger

Expand Down Expand Up @@ -44,6 +46,29 @@ def _generate_json_report(report_data: ReportData, output_dir: Path, base_filena
return report_path


def _copy_docs_images(report_data: ReportData, output_dir: Path):
"""
Copies images from the docs source directory to the report output directory.
"""
if not report_data.docs_dir:
return

source_dir = Path(report_data.docs_dir)
extensions = {".png", ".jpg", ".jpeg", ".gif", ".svg"}

for file_path in source_dir.rglob('*'):
if file_path.is_file() and file_path.suffix.lower() in extensions:
# Preserve directory structure
relative_path = file_path.relative_to(source_dir)
dest_path = output_dir / relative_path
dest_path.parent.mkdir(parents=True, exist_ok=True)
try:
shutil.copy2(file_path, dest_path)
logger.debug(f"Copied image {relative_path} to report directory")
except Exception as e:
logger.warning(f"Failed to copy image {file_path}: {e}")


def _generate_html_report(report_data: ReportData, output_dir: Path, base_filename: str):
"""
Generates an HTML report from the collected scenario results using a Jinja2 template.
Expand All @@ -55,12 +80,15 @@ def _generate_html_report(report_data: ReportData, output_dir: Path, base_filena
"""
# Generate visualizations for scenarios with flight data
_generate_visualizations(report_data, output_dir, base_filename)
# Copy images referenced in docs
_copy_docs_images(report_data, output_dir)

template_dir = Path(__file__).parent.parent / "templates"
env = Environment(
loader=FileSystemLoader(template_dir),
autoescape=select_autoescape(enabled_extensions=("html", "xml"), default_for_string=True, default=True),
)
env.filters["markdown"] = lambda text: markdown.markdown(text) if text else ""
Comment thread
atti92 marked this conversation as resolved.
template = env.get_template("report_template.html")

html_content = template.render(report_data=report_data)
Expand Down
2 changes: 2 additions & 0 deletions src/openutm_verification/core/reporting/reporting_models.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ class ScenarioResult(BaseModel):
telemetry_data: Optional[Any] = None
visualization_2d_path: Optional[str] = None
visualization_3d_path: Optional[str] = None
docs: Optional[str] = None


class ReportSummary(BaseModel):
Expand All @@ -70,3 +71,4 @@ class ReportData(BaseModel):
config: Dict[str, Any]
results: List[ScenarioResult]
summary: ReportSummary
docs_dir: Optional[str] = None
6 changes: 6 additions & 0 deletions src/openutm_verification/core/templates/report_template.html
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,12 @@ <h2>Scenario Results ({{ report_data.results|length }} executed)</h2>
<h4>Error Message:</h4>
<pre>{{ result.error_message }}</pre>
{% endif %}
{% if result.docs %}
<div class="scenario-docs">
<h4>Documentation:</h4>
{{ result.docs | markdown | safe }}
</div>
{% endif %}
<h4>Steps:</h4>
<table class="steps-table">
<thead>
Expand Down
10 changes: 7 additions & 3 deletions src/openutm_verification/scenarios/registry.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,17 +14,19 @@ def run_my_scenario(client, scenario_id):
"""

from functools import wraps
from pathlib import Path
from typing import Any, Callable, ParamSpec, TypeVar

from loguru import logger

from openutm_verification.core.execution.scenario_runner import ScenarioContext
from openutm_verification.core.execution.scenario_runner import ScenarioContext, ScenarioRegistry
from openutm_verification.core.reporting.reporting_models import (
ScenarioResult,
Status,
)
from openutm_verification.utils.paths import get_docs_directory

SCENARIO_REGISTRY = {}
SCENARIO_REGISTRY: dict[str, ScenarioRegistry] = {}
T = TypeVar("T")
P = ParamSpec("P")

Expand Down Expand Up @@ -77,7 +79,9 @@ def decorator(func: Callable[P, Any]) -> Callable[P, ScenarioResult]:
def wrapper(*args: P.args, **kwargs: P.kwargs) -> ScenarioResult:
return _run_scenario_simple(scenario_id, func, args, kwargs)

SCENARIO_REGISTRY[scenario_id] = wrapper
docs_dir = get_docs_directory()
docs_path = docs_dir / f"{scenario_id}.md" if docs_dir else None
SCENARIO_REGISTRY[scenario_id] = {"func": wrapper, "docs": docs_path}
return wrapper

return decorator
9 changes: 9 additions & 0 deletions src/openutm_verification/scenarios/test_fire_response.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
from openutm_verification.core.clients.flight_blender.flight_blender_client import FlightBlenderClient
from openutm_verification.core.execution.config_models import DataFiles
from openutm_verification.scenarios.registry import register_scenario


@register_scenario("fire_response")
def test_fire_response(fb_client: FlightBlenderClient, data_files: DataFiles) -> None:
"""Runs the Fire Response scenario."""
pass
26 changes: 26 additions & 0 deletions src/openutm_verification/utils/paths.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
from pathlib import Path


def get_docs_directory() -> Path | None:
"""
Determines the directory containing documentation and images.

Strategies:
1. Installed package location: openutm_verification/docs/
2. Development location: project_root/docs/
"""
# 1. Try installed package location: src/openutm_verification/docs/scenarios/
# This file is in src/openutm_verification/utils/
package_root = Path(__file__).parent.parent
docs_dir = package_root / "docs" / "scenarios"

if docs_dir.exists():
return docs_dir

# 2. Try development location: project_root/docs/scenarios/
# src/openutm_verification/utils/ -> src/openutm_verification/ -> src/ -> root/ (4 levels)
docs_dir = Path(__file__).parents[3] / "docs" / "scenarios"
if docs_dir.exists():
return docs_dir

return None
Loading