docs: documentation for user agents

AGENTS.md

@@ -0,0 +1,32 @@
+# AGENTS.md
+
+Guidance for AI coding agents working in this repository.
+
+*multiplex* is a modular framework for prototyping LLM-based mutation testing. One run mutates a single Java method (located via tree-sitter), generates mutants with an LLM, splices each mutant back into the source file, and runs the target project's tests to see which mutants are killed.
+
+## Documentation map
+
+Task-scoped docs live in `docs/` — start at [docs/README.md](docs/README.md) and load only the file relevant to your task instead of reading source:
+
+- [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) — pipeline, approaches, execution backends, output artifacts.
+- [docs/MODULE_REFERENCE.md](docs/MODULE_REFERENCE.md) — every function's signature, behavior, and side effects.
+- [docs/CONFIG.md](docs/CONFIG.md) — full `config.yml` schema.
+- [docs/EXTENDING.md](docs/EXTENDING.md) — checklists for adding approaches, execution backends, or LLM providers.
+- [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) — setup, conventions, CI, known bugs/gotchas.
+
+## Commands
+
+```bash
+uv sync -p 3.13                  # install deps (pin 3.13: tiktoken fails to build on 3.14)
+uv run pytest tests              # all tests — must run from repo root (relative resource paths)
+uv run pytest tests/checks/test_compiles.py::test_name      # one test
+uv run ruff check .              # lint (matches CI)
+uv run ./multiplex ./path/to/config.yml                     # run the tool
+```
+
+## Critical rules
+
+- **Import convention**: modules inside `multiplex/` import each other **without** the package prefix (`from model import Model`); tests import **with** it (`from multiplex.checks... import ...`). Match the file you are editing. Rationale in `docs/DEVELOPMENT.md`.
+- Adding an approach touches three places (package + prompt keys + dispatch); all `system_prompts` config keys are read unconditionally at startup. See `docs/EXTENDING.md`.
+- tree-sitter versions are pinned; do not bump them casually.
+- Check `docs/DEVELOPMENT.md` § Known issues before building on `execute/maven.py` (broken) or the STPA mutant loop (off-by-one); `execute/defects4j.py` is the reference execution backend.

CLAUDE.md

@@ -0,0 +1,35 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Overview
+
+*multiplex* is a modular framework for prototyping and comparing LLM-based mutation testing approaches. One run mutates a single Java method (located via tree-sitter), generates mutants with an LLM, injects each mutant back into the source file, and runs the target project's tests to determine which mutants are killed.
+
+## Documentation map
+
+Detailed docs live in `docs/` — load only the file relevant to the task instead of reading source:
+
+- [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) — end-to-end pipeline, approaches, execution backends, output artifacts.
+- [docs/MODULE_REFERENCE.md](docs/MODULE_REFERENCE.md) — every function's signature, behavior, and side effects.
+- [docs/CONFIG.md](docs/CONFIG.md) — full `config.yml` schema.
+- [docs/EXTENDING.md](docs/EXTENDING.md) — checklists for adding approaches, execution backends, or LLM providers.
+- [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) — setup, conventions, CI, and the current list of known bugs/gotchas.
+
+## Commands
+
+```bash
+uv sync -p 3.13                  # install deps (pin 3.13: tiktoken fails to build on 3.14)
+uv run pytest tests              # all tests — must run from repo root (relative resource paths)
+uv run pytest tests/checks/test_compiles.py::test_name      # one test
+uv run ruff check .              # lint
+uv run ./multiplex ./path/to/config.yml                     # run the tool
+```
+
+## Critical conventions
+
+- **Import convention**: the tool runs as a directory (`uv run ./multiplex`), so `multiplex/` itself is on `sys.path`. Modules inside `multiplex/` import each other **without** the package prefix (`from model import Model`, `from util.io import ...`); tests import **with** it (`from multiplex.checks... import ...`). Match the style of the file you are editing.
+- Pipeline entry point and dispatch: `multiplex/__main__.py`. Adding an approach touches three places (new `approach/<name>/controller.py` package, prompt keys in config **and** the `prompts` dict in `__main__.py`, dispatch branch) — see docs/EXTENDING.md.
+- All `system_prompts` config keys are read unconditionally at startup; a missing key is an immediate KeyError.
+- `<projectroot>/output/` is wiped at the start of every run; the target source file is edited in place and restored from a `.orig` backup.
+- Before relying on `execute/maven.py` or the STPA mutant loop, check docs/DEVELOPMENT.md § Known issues — both have verified bugs (Maven backend is broken; use `execute/defects4j.py` as reference).

docs/ARCHITECTURE.md

@@ -0,0 +1,103 @@
+# Architecture
+
+*multiplex* is a modular framework for prototyping LLM-based mutation testing.
+One run mutates **one Java method** in a target project and evaluates the
+generated mutants against that project's test suite.
+
+## Pipeline
+
+Orchestrated top-to-bottom by `multiplex/__main__.py`:
+
+```
+config.yml
+   │
+   ▼
+1. Load YAML config; read all 9 system_prompts keys into a dict
+   │
+   ▼
+2. Back up target source file to <filename>.orig
+   Delete and recreate <projectroot>/output/          ← destroys previous results
+   │
+   ▼
+3. util/extract_method.py (tree-sitter)
+   Find method by name + start line
+   → writes output/original_method.java
+   → returns (start_byte, end_byte) offsets into the source file
+   │
+   ▼
+4. approach/<name>/controller.main(model, output_path, prompts)
+   One or more LLM calls (via model.Model / LiteLLM)
+   → writes mutant files to output/<approach>-mutants/mutant_N.java
+   │
+   ▼
+5. execute/{maven,defects4j}.run_mutants(...)
+   For each mutant file:
+     a. restore source file from .orig backup
+     b. util/rewrite_method.py splices mutant text into the file
+        at the saved byte offsets
+     c. checks/compilable.py        — tree-sitter parse-error scan
+        checks/syntactic_equivalence.py — AST equality vs original (ignores comments)
+     d. run the project's tests (Defects4J backend only; see Status below)
+   → writes output/<approach>-mutants/mutant_summary.csv (Defects4J)
+   │
+   ▼
+6. Restore original source file from .orig backup
+```
+
+Key mechanism: the method's **byte offsets** captured in step 3 are reused in
+step 5b to splice each mutant into a pristine copy of the file. Anything that
+changes the file between those steps invalidates the offsets.
+
+## Mutant-generation approaches (`multiplex/approach/`)
+
+Each approach is a package with a `controller.py` exposing
+`main(model, output_dir, prompts)`. Dispatch is an if/elif chain in
+`__main__.py` keyed on `mutation.approach`.
+
+| Approach | Strategy | LLM calls |
+|----------|----------|-----------|
+| `basic` | Single system prompt, ask for a mutant 10 times | 10 |
+| `hazop` | Chain: describe method line-by-line → mutate descriptions using HAZOP guidewords (NO/MORE/LESS/...) → implement each deviation | 2 + 1 per deviation |
+| `stpa` | Chain: describe control flow as GraphViz DOT → identify Unsafe Control Actions (UCAs) → implement each UCA | 2 + 1 per UCA |
+| `mutahunter` | Single call with AST + line-numbered source; LLM returns YAML of line-level mutations spliced in locally (prompts not bundled — licensing) | 1 |
+| `llmorpheus` | tree-sitter query finds mutation sites (conditions, loop headers, call args), each replaced by `<PLACEHOLDER>`; LLM proposes 3 replacements per site | 1 per placeholder |
+
+Intermediate artifacts are files in `output/` — approaches communicate between
+their own steps via files, not in-memory state (see artifact table below).
+
+## Execution backends (`multiplex/execute/`)
+
+Selected by `project.runtool`:
+
+- `d4j` → `defects4j.py` — the complete backend. Baselines the unmutated
+  project first (aborts if its tests fail), times the baseline run, then runs
+  each compilable mutant's tests with a timeout of 5× baseline. A surviving
+  mutant is one whose test run reports `Failing tests: 0`. Requires the
+  `defects4j` CLI and a `JDK_11` env var (see DEVELOPMENT.md).
+- `mvn` → `maven.py` — **currently broken/incomplete**; do not rely on it
+  (details in DEVELOPMENT.md § Known issues).
+
+## Output artifacts
+
+Everything lands under `<projectroot>/output/` (wiped at the start of each run):
+
+| Artifact | Written by |
+|----------|-----------|
+| `original_method.java` | extract_method (step 3); read by every approach |
+| `hazop-descriptions.txt`, `hazop-mutated-descriptions.txt` | hazop chain steps |
+| `control_diagram.txt`, `ucas.csv` | stpa chain steps |
+| `placeholders/{N_placeholder.java, N_orig.java, placeholders.json}` | llmorpheus site finder |
+| `<approach>-mutants/mutant_N.java` | every approach's final step |
+| `<approach>-mutants/mutant_summary.csv` | defects4j backend; columns `MUTANT, EQUIVALENCE, COMPILABLE, SURVIVES` |
+| `<approach>-test/<mutant>_test.txt` | defects4j backend; per-mutant test output |
+
+## LLM access (`multiplex/model.py`)
+
+All LLM traffic goes through `Model.make_request(messages)`, a thin wrapper
+over LiteLLM `completion()`. Model/endpoint come from config; the API key is
+read from the environment variable *named* by `llm.token_env_var` (never stored
+in config). Azure endpoints get `AZURE_AI_API_BASE`/`AZURE_AI_API_KEY` set
+specially. To support another provider, replace or extend `model.py`.
+
+LLM responses are treated as fenced code: approaches strip a leading
+```` ```java ```` fence and keep everything before the next ```` ``` ```` fence.

docs/CONFIG.md

@@ -0,0 +1,56 @@
+# Configuration Reference
+
+*multiplex* takes a single YAML config file:
+`uv run ./multiplex ./path/to/config.yml`.
+Template: [`examples/config.yml`](../examples/config.yml).
+
+All keys below are read unconditionally by `multiplex/__main__.py` — a missing
+key raises `KeyError` at startup, even for approaches you are not running.
+
+## `project`
+
+| Key | Type | Meaning |
+|-----|------|---------|
+| `projectroot` | path | Root of the Java project under test. `output/` is created (and **wiped every run**) inside it. |
+| `filename` | path | The `.java` source file containing the method under test. Backed up to `<filename>.orig` during the run. |
+| `method` | string | Name of the method (or constructor) to mutate. |
+| `line` | int | 1-based line number of the method **name identifier** in `filename` (not annotations above it). Both `method` and `line` must match for extraction to succeed; disambiguates overloads. |
+| `runtool` | `mvn` \| `d4j` | Execution backend. `d4j` (Defects4J) is the complete one; `mvn` is currently broken (DEVELOPMENT.md § Known issues). |
+
+## `mutation`
+
+| Key | Type | Meaning |
+|-----|------|---------|
+| `approach` | `basic` \| `hazop` \| `stpa` \| `mutahunter` \| `llmorpheus` | Mutant-generation approach. Any other value prints "Invalid approach" and continues to the execution phase (which then fails on the missing mutants dir). |
+
+## `llm`
+
+| Key | Type | Meaning |
+|-----|------|---------|
+| `model` | string | LiteLLM model string, e.g. `ollama/gpt-oss:20b`. |
+| `endpoint` | URL | LLM API endpoint, e.g. `http://127.0.0.1:11434`. If the string contains `azure`, Azure env vars are set automatically. |
+| `token_env_var` | string or empty | **Name** of the environment variable holding the API key — never the key itself. Leave empty for keyless local endpoints (e.g. Ollama). The named var must exist or startup fails with KeyError. |
+
+Note: the top-level README shows this key as `tokenenvvar`; the code and
+example config use `token_env_var`. `token_env_var` is correct.
+
+## `system_prompts`
+
+All nine keys are required (multi-line YAML strings, typically `|` blocks).
+User prompts are constructed in code; only system prompts live in config.
+
+| Key | Used by |
+|-----|---------|
+| `basic_generate_mutants` | basic |
+| `hazop_describe_process` | hazop step 1 (line-by-line description) |
+| `hazop_identify_deviations` | hazop step 2 (guideword mutation; must instruct CSV output `number, original_rule, GUIDEWORD, changed_rule`) |
+| `hazop_implement_deviations` | hazop step 3 (code generation) |
+| `stpa_describe_control_flow` | stpa step 1 (GraphViz DOT control diagram) |
+| `stpa_identify_ucas` | stpa step 2 (numbered UCA list) |
+| `stpa_implement_ucas` | stpa step 3 (code generation) |
+| `mutahunter_generate_mutants` | mutahunter — **not bundled**; users must paste Mutahunter's own prompt text (licensing restriction). Response must be YAML with `mutants: [{mutated_code, line_number}]`. |
+| `llmorpheus_system` | llmorpheus (per-placeholder replacement generation) |
+
+Output-format expectations matter: downstream parsing is brittle string/CSV/
+YAML/regex handling (see MODULE_REFERENCE.md), so prompts must pin the exact
+output format the next step expects.

docs/DEVELOPMENT.md

@@ -0,0 +1,93 @@
+# Development Guide
+
+## Setup and commands
+
+Dependency management is [uv](https://github.com/astral-sh/uv); Python >= 3.13.
+
+```bash
+uv sync -p 3.13                  # install deps incl. dev group (pytest, ruff, pylint)
+uv run pytest tests              # all tests — run from the REPO ROOT
+uv run pytest tests/checks/test_compiles.py                # one file
+uv run pytest tests/checks/test_compiles.py::test_name     # one test
+uv run ruff check .              # lint (CI uses --output-format=github)
+uv run ./multiplex ./path/to/config.yml                    # run the tool
+```
+
+- **Pin Python 3.13** (as CI does): `requires-python` allows >= 3.13, but
+  litellm's `tiktoken` dependency fails to build on 3.14.
+- Tests must run from the repo root: they reference `tests/resources/...`
+  relatively.
+- Running the tool needs a reachable LLM endpoint and a target Java project;
+  `d4j` runs additionally need the `defects4j` CLI (expected under
+  `./defects4j/framework/bin`) and a `JDK_11` env var pointing at a JDK 11 home.
+
+CI (GitHub Actions, on push/PR to `main`): `test.yml` runs
+`uv run pytest tests` on Python 3.13; `ruff.yml` runs `uv run ruff check`.
+
+## Import convention (critical)
+
+The tool is executed as a directory (`uv run ./multiplex`), which puts
+`multiplex/` itself on `sys.path`:
+
+- **Inside `multiplex/`**: flat imports, no package prefix —
+  `from model import Model`, `from util.io import write_to_file`,
+  `import approach.basic.controller as basic`.
+- **Inside `tests/`**: package-prefixed imports —
+  `from multiplex.checks.syntactic_equivalence import check_mutant_equivalent`.
+
+Using the wrong style in either place breaks at import time. This also means
+`multiplex/` modules are not importable as `multiplex.x` **and** `x` in the
+same process; keep the two worlds separate.
+
+## Conventions
+
+- tree-sitter is the universal Java analysis tool (extraction, compilability,
+  equivalence, placeholder finding). Versions are pinned
+  (`tree-sitter==0.23.2`, `tree-sitter-java==0.23.5`); the code depends on that
+  API — do not bump casually.
+- Approaches communicate between their own steps via files in `output/`, not
+  in-memory state; artifact names are prefixed with the approach name.
+- LLM code responses are unfenced by stripping a leading ```` ```java ````
+  fence and truncating at the next ```` ``` ```` fence
+  (`removeprefix` + `split`, see EXTENDING.md for the exact two lines).
+- Mutant files are named `mutant_<N>.java` in `output/<approach>-mutants/`.
+- Status/progress reporting is `print()`-based throughout (no logging config,
+  one `logging.warning` in extract_method).
+
+## Known issues and gotchas (verified 2026-07)
+
+Bugs — fix or work around, don't replicate:
+
+1. **`execute/maven.py` is broken**: `run_mutants` calls `rewrite_method` with
+   5 args (signature takes 4 → TypeError), never calls its own
+   `_execute_mutant` (so `mvn test` never runs), and passes a directory to
+   `check_mutant_compilable` (which parses a single file). Use
+   `execute/defects4j.py` as the reference implementation.
+2. **STPA off-by-one**: `approach/stpa/code_generator.py` loops
+   `range(0, ucas_count - 1)`, silently skipping the last UCA.
+3. **Method-not-found crash**: `extract_method_from_file` returns `None` on a
+   miss, but `__main__.py` unpacks it as a tuple → opaque TypeError. Symptom:
+   check `project.method`/`project.line` in the config (`line` must be the
+   line of the method-name identifier).
+4. **README key mismatch**: README shows `llm.tokenenvvar`; the code reads
+   `llm.token_env_var` (the example config is correct).
+5. **`tests/execute/test_defects4j.py` is an empty stub** — the execute layer
+   has zero test coverage.
+
+Behavioral gotchas — by design (or at least current design), be aware:
+
+- `<projectroot>/output/` is **deleted without prompting** at every run start
+  (the confirmation prompt in `__main__.py` is commented out, answer
+  hard-coded to `"y"`).
+- The target source file is mutated **in place** during execution; it is
+  backed up to `<filename>.orig` and restored at run end, but a crash mid-run
+  can leave a mutant in the working tree (the `.orig` file remains for manual
+  restore).
+- "Compilable" means *parses without tree-sitter ERROR nodes* — no compiler
+  runs; type errors and unresolved symbols count as compilable.
+- An invalid `mutation.approach` value only prints "Invalid approach" and
+  falls through to the execution phase.
+- All nine `system_prompts` keys are read at startup regardless of the chosen
+  approach — a missing key is a KeyError before anything runs.
+- Mutahunter prompts are not bundled (licensing); the user pastes them into
+  the config.