Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,3 +47,9 @@ Describe the testing strategy for this change and include any relevant details a
If the change is covered by the CI test suite, state that explicitly.
If manual testing was performed, describe the testing process and results.
-->


## Checklist

- [ ] Architecture/docs impact considered. Updated `ARCHITECTURE.md` and/or
developer docs where needed, or explained why not.
2 changes: 2 additions & 0 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ jobs:
steps:
- name: Checkout Myna
uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Install uv
uses: astral-sh/setup-uv@v7
with:
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/pre-commit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: '3.14'
Expand Down
11 changes: 9 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,8 +109,11 @@ Branch names use:
```

Examples: `docs/agent-harness`, `fix/database-paths`, `feat(cli): add dry-run flag`.
Complete `.github/pull_request_template.md` and state behavior impact, risk, and
testing strategy.
When asked to draft or open a pull request, first inspect and complete
`.github/pull_request_template.md`. Preserve the template headings and checklist, and
state behavior impact, risk, and testing strategy. If the requested PR body is scoped
to a subset of commits while the branch contains other commits, state that scope
explicitly in the body.

## Security and Data Handling

Expand All @@ -131,3 +134,7 @@ testing strategy.
- Public behavior changes, compatibility risks, or known gaps are called out.
- Docs and examples are updated when commands, inputs, outputs, or extension points
change.
- If code changes touch `src/myna/core/workflow/`, `src/myna/core/components/`,
`src/myna/core/app/`, `src/myna/core/context.py`, `src/myna/database/`, CLI behavior,
app extension patterns, or subsystem boundaries, update `ARCHITECTURE.md` and
relevant developer docs, or state why no docs update was needed.
37 changes: 23 additions & 14 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Status

- Last verified: 2026-06-01 during the documentation-harness update
- Last verified: 2026-06-01 during the workflow-context update
- Audience: maintainers, contributors, and coding agents
- Scope: describes current architecture; does not replace API docs or user docs

Expand Down Expand Up @@ -44,6 +44,7 @@ The main user workflows are:
| `uv.lock` | Locked `uv` dependency graph | Commit updates when dependency declarations change |
| `src/myna/` | Python package source | Runtime code lives here |
| `src/myna/core/` | Workflow orchestration, components, file abstractions, metadata, app base helpers, utilities | Prefer generic workflow behavior here |
| `src/myna/core/context.py` | Explicit workflow context shared by orchestration, components, app wrappers, metadata, and sync helpers | Prefer this over workflow-specific environment variables for new shared behavior |
| `src/myna/core/workflow/` | `config`, `run`, `sync`, `status`, and input loading stages | CLI entrypoint delegates here |
| `src/myna/core/components/` | Workflow component classes and lookup table | Component string keys are user-facing compatibility surface |
| `src/myna/core/files/` | Output file classes and validation/sync value extraction | Add new output formats here |
Expand All @@ -62,7 +63,7 @@ The main user workflows are:
| `docs/api-docs/` | Generated API docs from `scripts/group_docs.py` | Ignored by git; regenerate before docs builds |
| `scripts/group_docs.py` | Generates and groups LazyDocs API documentation | CI runs this before `mkdocs build --strict` |
| `scripts/check_dev_tools.py` | Checks local development tool availability and writable caches | Run first in new agent or container shells |
| `scripts/check_docs_harness.py` | Validates required agent-harness docs and links | Runs in pre-commit and CI |
| `scripts/check_docs_harness.py` | Validates required agent-harness docs, links, and architecture-doc updates for sensitive code changes | Runs in pre-commit and CI |
| `.pre-commit-config.yaml` | Local quality hooks | Includes Ruff, codespell, license headers, and docs harness check |
| `.github/workflows/CI.yml` | Package build, default tests, pylint, API docs, MkDocs, and external-app example CI | External-app job runs in a container |
| `.github/workflows/pre-commit.yml` | Pre-commit CI | Runs hooks on pull requests |
Expand All @@ -87,11 +88,17 @@ On import, `src/myna/core/__init__.py` sets:
- `MYNA_INSTALL_PATH` to the installed `myna` package root;
- `MYNA_APP_PATH` to the installed `myna/application` directory.

Workflow commands set additional runtime environment variables, including `MYNA_INPUT`,
`MYNA_CONFIG_INPUT`, `MYNA_RUN_INPUT`, `MYNA_SYNC_INPUT`, `MYNA_STEP_NAME`,
`MYNA_STEP_CLASS`, and `MYNA_STEP_INDEX`. Usage of runtime environment variables
is planned to be deprecated--the `*_INPUT` variables are marked in code as
future deprecation targets. Prefer using `MynaApp` class parameters for new features.
Workflow run and sync state is carried through `myna.core.context.WorkflowContext`.
This context includes the active input file, current step, step class, step index, and
previous-step information. `myna.core.context` resolves that explicit context first and
still falls back to legacy `MYNA_*` environment variables so direct stage-script
invocation remains compatible in Myna 1.x. That env-var fallback is deprecated and
scheduled for removal in Myna 2.0. `MynaApp` consumes that shared context resolution
rather than reading workflow env vars directly.

`myna config` still sets `MYNA_INPUT` and the deprecated `MYNA_CONFIG_INPUT` while it
extracts metadata. New shared workflow code should prefer `WorkflowContext`,
`MynaApp` attributes, or `get_workflow_input_file()` over direct `os.environ` reads.

## Main Concepts and Domain Model

Expand All @@ -107,8 +114,8 @@ future deprecation targets. Prefer using `MynaApp` class parameters for new feat
`data_requirements`, `input_requirement`, `output_requirement`, and hierarchical
`types` such as `build`, `build_region`, `part`, `region`, and `layer`.
- **Application wrapper**: Tool-specific code under `src/myna/application/<app>/<class>/`
that may provide `configure.py`, `execute.py`, and `postprocess.py` stages. Wrappers
commonly use `myna.core.app.MynaApp`.
that may provide `configure.py`, `execute.py`, and `postprocess.py` stage modules.
Wrappers commonly use `myna.core.app.MynaApp`.
- **Database adapter**: A subclass of `myna.core.db.Database` under
`src/myna/database/` that loads metadata and syncs outputs for a supported data
source.
Expand Down Expand Up @@ -138,7 +145,7 @@ flowchart TD
Component --> Cases[case directories and myna_data.yaml]
CLI --> Run[myna run]
Run --> ComponentRun[Component.run_component]
ComponentRun --> AppStages[application configure/execute/postprocess scripts]
ComponentRun --> AppStages[application configure/execute/postprocess modules]
AppStages --> Outputs[component output files]
CLI --> Sync[myna sync]
Sync --> FileValidation[file validation]
Expand All @@ -150,9 +157,11 @@ the input file, resolves the database adapter with `return_datatype_class`, extr
component metadata requirements, creates case directories, writes per-case
`myna_data.yaml`, records expected output paths, and writes the configured input. For
`run`, Myna reloads the input before each step, applies step settings to a component,
sets step environment variables, and runs the available app-stage scripts. For `sync`,
Myna validates component outputs and delegates supported sync behavior to the selected
database adapter.
and calls available app-stage modules in-process through `Component.run_component`.
Stage command-line arguments from the input file are still exposed through `sys.argv`
while each stage runs, preserving `argparse`-based app wrappers. For `sync`, Myna
validates component outputs and delegates supported sync behavior to the selected
database adapter while providing the active input file through `WorkflowContext`.

## Dependency Boundaries

Expand Down Expand Up @@ -222,7 +231,7 @@ Common validation commands:
| External examples | `uv run pytest -m "examples and not parallel"` | Requires external tools and example fixtures |
| Generate API docs | `uv run scripts/group_docs.py` | Writes ignored `docs/api-docs/` |
| Build docs | `uv run mkdocs build --strict` | Run after API docs generation for parity with CI |
| Docs harness | `uv run python scripts/check_docs_harness.py` | Verifies agent-doc structure, links, and compactness |
| Docs harness | `uv run python scripts/check_docs_harness.py` | Verifies required agent docs, links, and architecture-doc updates for sensitive code changes |
| Pre-commit | `uv run pre-commit run --all-files` | May require network the first time hooks install |

CI behavior:
Expand Down
65 changes: 43 additions & 22 deletions docs/developer_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,33 +119,54 @@ input file, just provide an arbitrary name for the application, e.g., "test".
## Developing new applications

Once a new component is implemented, you will have to implement a corresponding
application (app) to use with `myna run`. Applications consist of up to three scripts:

1. `config.py`: Script that configures the files within each Myna case folder generated
during `myna config`. At the end of this script, each case folder should be a valid
case directory for whatever model is being run.
2. `execute.py`: Script that executes the model for each case.
3. `postprocess.py`: Script that converts the output of the model into the required
myna file format for the component. This may be part of execute.py, as well.

These scripts are run sequentially and are mainly separated for clarity of the app
functionality and for handling runtime issues. *Technically* all functionality can
be in one script, however, this may get confusing so it is recommended to split
functions into three scripts. If the model is not Python-based, these scripts can
simply wrap other scripts as needed. All steps are optional and if one of these
scripts is not present, it will be ignored.
application (app) to use with `myna run`. Applications consist of up to three stage
modules:

1. `configure.py`: Module that configures the files within each Myna case folder
generated during `myna config`. At the end of this stage, each case folder should be a
valid case directory for whatever model is being run.
2. `execute.py`: Module that executes the model for each case.
3. `postprocess.py`: Module that converts the output of the model into the required
myna file format for the component. This may be part of `execute.py`, as well.

These stage modules are imported and called sequentially by `Component.run_component`.
Each module should define a function named for the stage (`configure`, `execute`, or
`postprocess`) or a `main()` function. The stages are mainly separated for clarity of
the app functionality and for handling runtime issues. *Technically* all functionality
can be in one stage, however, this may get confusing so it is recommended to split
functions into three stages. If the model is not Python-based, these stages can simply
wrap other commands as needed. All steps are optional and if one of these modules is
not present, it will be ignored.

Many of the already implemented apps use the `argparse` library to parse
user-specified inputs. In the input file, `configure`, `execute`, and
`postprocess` allow users to pass options to each of the scripts
for the app. Any parameters that you wish to have accessible to users are
intended to be adjusted through such options, which are passed to the script via
command line in the format `--key value` or `--key` for Boolean flags. For Boolean
flags, the assumed behavior is False if the flag is not passed and True if the flag is
passed.
`postprocess` allow users to pass options to each stage for the app. Any parameters
that you wish to have accessible to users are intended to be adjusted through such
options, which are exposed to the active stage through `sys.argv` in the format
`--key value` or `--key` for Boolean flags. For Boolean flags, the assumed behavior is
False if the flag is not passed and True if the flag is passed.

Applications that derive from `MynaApp` should read workflow state from app attributes
such as `self.input_file`, `self.step_name`, `self.last_step_name`, and
`self.step_index`. Avoid reading `MYNA_*` environment variables directly in new app
code; those names are only populated temporarily while a stage is running for
compatibility with existing wrappers and direct stage invocation in Myna 1.x. That
env-var fallback is deprecated and will be removed in Myna 2.0.

Because stage modules are imported and executed in-process, they should restore any
global process state they modify, especially the current working directory. Prefer
`myna.core.utils.working_directory()` over bare `os.chdir()` calls.

It is likely that your app will require a `template` directory, or a set of input
files for your model that get copied into every case. If you are using a template
directory, then the intended functionality is that during `config.py` the template
directory, then the intended functionality is that during `configure.py` the template
folder is copied into each of the case directory *and then updated*. Updating the files
inside the original template folder should be avoided.

## Versioning applications

There is base `MynaApp` functionality to extract a version number from process output
given a regex pattern to match. This can be used to have branching behavior in a Myna
application based on the version of the executable that is being used. Adding in branching
logic instead of updating the logic to handle only the most recent version of a code
is generally preferable for backwards compatibility.
6 changes: 5 additions & 1 deletion docs/documentation.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,11 @@ The documentation harness is intentionally small:
- `scripts/check_dev_tools.py` verifies that the current shell can run the repository's
development toolchain.
- `scripts/check_docs_harness.py` verifies required headings, required `AGENTS.md`
links, relative link targets, and the compact size of `AGENTS.md`.
links, relative link targets, compact size of `AGENTS.md`, and architecture-doc
updates when architecture-sensitive code paths change. It also verifies required PR
template guidance and PR template headings. These checks enforce repository design
decisions: the required documentation headers and related harness policies are
not optional formatting preferences.

Run the harness check with:

Expand Down
3 changes: 2 additions & 1 deletion examples/cases/microstructure_region/input.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ steps:
ry: 0.48e-3
rz: 0.16e-3
pad-xy: 0.48e-3
pad-z: 0.08e-3
pad-z: 0.48e-3
pad-sub: 0.64e-3
refine-layer: 1
refine-region: 2
Expand All @@ -30,6 +30,7 @@ steps:
nd: 250
mu: 21
std: 3
template: ./template
execute:
np: 2
data:
Expand Down
28 changes: 28 additions & 0 deletions examples/cases/microstructure_region/template/analysis.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
{
"Regions": {
"XY": {
"units": "Cells",
"zBounds": [360,360],
"printStats": ["GrainTypeFractions"],
"printPerGrainStats": ["Size","IPFZ-RGB"],
"printPoleFigureData": true,
"printInversePoleFigureData": true
},
"XZ": {
"units": "Cells",
"yBounds": [200,200],
"printStats": ["GrainTypeFractions"],
"printPerGrainStats": ["Size","IPFZ-RGB"],
"printPoleFigureData": true,
"printInversePoleFigureData": true
},
"YZ": {
"units": "Cells",
"xBounds": [200,200],
"printStats": ["GrainTypeFractions"],
"printPerGrainStats": ["Size","IPFZ-RGB"],
"printPoleFigureData": true,
"printInversePoleFigureData": true
}
}
}
38 changes: 38 additions & 0 deletions examples/cases/microstructure_region/template/inputs.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
{
"SimulationType": "FromFile",
"MaterialFileName": "",
"GrainOrientationFile": "",
"RandomSeed": 0.0,
"Domain": {
"CellSize": 2.5,
"TimeStep": 0.0625,
"NumberOfLayers": 10,
"LayerOffset": 20.0
},
"Nucleation": {
"Density": 250.0,
"MeanUndercooling": 21.0,
"StDev": 3.0
},
"TemperatureData": {
"TemperatureFiles": [
],
"LayerwiseTempRead": true
},
"Substrate": {
"MeanSize": 12.3
},
"Printing": {
"PathToOutput": "./",
"OutputFile": "exaca",
"PrintBinary": true,
"Interlayer": {
"Fields": [
"GrainID",
"LayerID",
"GrainMisorientation",
"MeltTimeStep"
]
}
}
}
10 changes: 10 additions & 0 deletions examples/cases/microstructure_region/template/runCase.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/bin/bash
cd ${0%/*} || exit 1 # run from this directory

# source the correct version of ExaCA based
# on {{EXACA_BIN_PATH}} and {{EXACA_EXEC}} specified
# in the app's configure.py script
EXE="{{EXACA_BIN_PATH}}/{{EXACA_EXEC}}"

# Run the ExaCA case
mpiexec -n {{RANKS}} $EXE inputs.json > exaca_run.log 2>&1
Loading
Loading