Add option to create markdown formatted docs for each scenario. - #59
Conversation
|
A sample documentation can be like the following image, source, Section 8.3.2, it describes in text and maps what is happening in the scenario the PDF has additional examples, we should be able to build this type of description (among others): |
There was a problem hiding this comment.
Pull request overview
This PR adds support for attaching markdown documentation to verification scenarios, which is automatically rendered in HTML reports. The implementation introduces a documentation discovery system that searches for markdown files in multiple locations (docs/scenarios directory or side-by-side with scenario files) and integrates them into the reporting pipeline.
Key Changes:
- Added markdown documentation support with automatic discovery and HTML rendering
- Introduced docs path resolution utilities and updated scenario registry to track documentation paths
- Extended the execution context and reporting models to propagate documentation content through the system
Reviewed changes
Copilot reviewed 18 out of 21 changed files in this pull request and generated 16 comments.
Show a summary per file
| File | Description |
|---|---|
| src/openutm_verification/utils/paths.py | New utility to locate docs directory in both installed and development environments |
| src/openutm_verification/scenarios/registry.py | Updated to discover and track documentation files for each scenario |
| src/openutm_verification/scenarios/test_fire_response.py | New placeholder scenario with accompanying markdown documentation |
| src/openutm_verification/core/reporting/reporting_models.py | Extended ScenarioResult and ReportData models with optional docs fields |
| src/openutm_verification/core/templates/report_template.html | Added documentation section rendering with markdown-to-HTML conversion |
| src/openutm_verification/core/reporting/reporting.py | Added markdown filter and image copying functionality for docs |
| src/openutm_verification/core/execution/execution.py | Integrated docs directory path and propagated docs content to results |
| src/openutm_verification/core/execution/dependency_resolution.py | Updated CONTEXT default to include docs field |
| src/openutm_verification/core/execution/dependencies.py | Added get_scenario_docs function and updated scenarios generator |
| src/openutm_verification/core/execution/config_models.py | Extended RunContext TypedDict with docs field |
| src/openutm_verification/core/execution/scenario_runner.py | Added ScenarioRegistry TypedDict definition |
| pyproject.toml | Added markdown dependency and configured docs packaging with force-include |
| config/default.yaml | Added fire_response scenario to execution list |
| docs/scenarios/fire_response.md | New markdown documentation for fire response scenario |
| docs/index.md | New index page for scenario documentation |
| Dockerfile | Changed to editable install mode |
| .gitattributes | Added binary file declarations for images |
| docs/.DS_Store | macOS system file (should not be committed) |
Comments suppressed due to low confidence (3)
src/openutm_verification/core/execution/dependencies.py:1
- Import of 'base64' is not used.
import base64
src/openutm_verification/core/execution/dependencies.py:2
- Import of 'mimetypes' is not used.
import mimetypes
src/openutm_verification/core/execution/dependencies.py:3
- Import of 're' is not used.
import re
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Fixes #47
Closes #48
Visible on github: https://github.com/atti92/utm_verification/blob/scenario-markdown-docs/docs/index.md
Description:
This PR introduces the ability to attach Markdown documentation to verification scenarios. This documentation is automatically read during execution and rendered as HTML within the generated verification reports. This allows for better context and explanation of test scenarios directly in the output.
Key Changes:
Documentation Support:
Added markdown library dependency.
Updated ScenarioResult model to include an optional docs field.
Modified the HTML report template (report_template.html) to render the docs field using a custom Jinja2 markdown filter.
Registry & Execution:
Updated register_scenario decorator to accept an optional docs argument (filename).
Enhanced dependencies.py to resolve and read the documentation file relative to the scenario function's location.
Propagated documentation content through RunContext to the final ScenarioResult.
Example:
How to Test:
uv run openutm-verify --config config/default.yaml/add_flight_declaration.md.