docs: consolidate public guide flow

Author: zzzbitz

README.md

@@ -88,22 +88,19 @@ your agent runs from `@runs/...`; alternatively set
 prepare workspace -> run your agent -> write result/output_*.csv -> evaluate
 ```
 
-## Agent Integration
+Local runs live under a run root written `@runs/<agent>/<mode>/`. The `@runs/`
+prefix is just the naming convention for this directory, and it is git-ignored so
+your runs are never committed.
 
-PrepBench prepares workspaces and scores result tables; you run the agent. For
-each case, point your agent at the prepared workspace, let it read only the files
-exposed there, and have it write final tables under `result/output_*.csv`.
-Intermediate code, notebooks, logs, or workflow JSON files may stay in the
-workspace, but they are not scored.
+PrepBench is an honor-system benchmark: the repository ships evaluator and
+simulator assets, but your agent should read only the files exposed in its
+prepared workspace.
 
-In `interactive` and `workflow` mode, the agent may call the local user
-simulator. In `workflow` mode, the agent also reads `workflow_prompt.yaml` for
-the workflow operator instructions and uses the workflow executor to create the
-final result tables.
+## Quickstart
 
-## Prepare a Workspace
+Prepare one case workspace, run your agent in it, then evaluate.
 
-Create one case workspace under a run root:
+**1. Prepare a workspace**
 
 ```bash
 python scripts/prepare_run.py \
@@ -112,48 +109,61 @@ python scripts/prepare_run.py \
   --run-root @runs/my_agent/clarified
 ```
 
-The workspace layout is:
-
 ```text
 @runs/my_agent/clarified/case_001/
   query.md
   inputs/
   result/
 ```
 
-`interactive` workspaces also contain `clarification_guide.md`. `workflow`
-workspaces contain both `clarification_guide.md` and `workflow_prompt.yaml`.
+`interactive` workspaces also contain `clarification_guide.md`; `workflow`
+workspaces add `workflow_prompt.yaml`. Workspace files are symlinks where
+possible, so setup is cheap.
+
+**2. Run your agent**
 
-Repeat this command for each case you want to run. Workspace files are symlinks
-where possible, so setup is cheap.
+Point your agent at the workspace path (not copied file contents). It reads
+`query.md` and `inputs/`, may write code or other working files, and must write
+final tables to `result/output_*.csv`. See [Agent Integration](#agent-integration)
+for what each mode exposes.
 
-Prepare every GT case for a complete mode run:
+**3. Evaluate**
 
 ```bash
-python scripts/prepare_run.py \
+python scripts/evaluate_submission.py \
   --mode clarified \
-  --all \
-  --run-root @runs/my_agent/clarified
+  --run-root @runs/my_agent/clarified \
+  --case case_001
 ```
 
-`--all` uses the evaluator GT case set, then requires matching `data/case_xxx`
-directories for workspace setup.
+Results land in `evaluation/summary.json` and `evaluation/summary.csv` under the
+run root. See [docs/EVALUATION.md](docs/EVALUATION.md) for the scoring rules,
+summary fields, and full-run behavior.
 
-PrepBench is an honor-system benchmark. The repository still contains evaluator
-and simulator assets, but the model-under-test should only read files exposed in
-its prepared workspace.
+## Run the Full Benchmark
 
-## Run Your Agent
+Prepare every GT case with `--all`, then evaluate the run root without `--case`:
 
-Run your agent inside the case workspace. Give it the workspace path, not copied
-file contents. It can inspect `query.md`, read `inputs/`, write code or other
-working files, and finally write result tables:
-
-```text
-@runs/my_agent/<mode>/<case_id>/result/output_*.csv
+```bash
+python scripts/prepare_run.py --mode clarified --all --run-root @runs/my_agent/clarified
+python scripts/evaluate_submission.py --mode clarified --run-root @runs/my_agent/clarified
 ```
 
-For `interactive`, the agent may use `clarification_guide.md` and the Python API:
+`--all` uses the evaluator GT case set and requires matching `data/case_xxx`
+directories. A full run exits with code 0 only when every case passes; missing
+folders or result tables are reported as `NOT_FOUND`.
+
+## Agent Integration
+
+You run the agent; PrepBench only prepares workspaces and scores result tables.
+Point your agent at the prepared workspace (not copied file contents), let it read
+only the files exposed there, and have it write final tables under
+`result/output_*.csv`. Intermediate code, notebooks, logs, or workflow JSON may
+stay in the workspace but are not scored. See the [Public Modes](#public-modes)
+table for what each mode exposes.
+
+In `interactive` and `workflow` mode, the agent calls the local user simulator to
+clarify the request before writing results:
 
 ```python
 from simulator import LocalUserSimulatorAPI
@@ -166,49 +176,15 @@ reply = api.ask(
 )
 ```
 
-For comparable interactive runs, keep the simulator backend fixed. A practical
-default is official `deepseek-v4-flash` in non-thinking mode with
-`PREPBENCH_SIMULATOR_TEMPERATURE=0`; see `docs/USER_SIMULATOR.md` for
-provider-specific settings.
-
-For `workflow`, the agent may read `workflow_prompt.yaml`, write prep code as an
-intermediate artifact, convert that code into a workflow JSON file, and execute
-the workflow from the workspace. The workflow executor reads from `./inputs` and
-writes to `./result`, so the generated workflow produces the same
-`result/output_*.csv` files that the evaluator scores.
-
-## Evaluate
-
-Evaluate a full mode run:
-
-```bash
-python scripts/evaluate_submission.py \
-  --mode clarified \
-  --run-root @runs/my_agent/clarified
-```
-
-For single-case debugging:
+Keep the simulator backend fixed for comparable runs (a practical default is
+`deepseek-v4-flash` in non-thinking mode with `PREPBENCH_SIMULATOR_TEMPERATURE=0`).
+See [docs/USER_SIMULATOR.md](docs/USER_SIMULATOR.md) for provider settings and the
+question budget.
 
-```bash
-python scripts/evaluate_submission.py \
-  --mode clarified \
-  --run-root @runs/my_agent/clarified \
-  --case case_001
-```
-
-The evaluator writes:
-
-```text
-@runs/my_agent/clarified/evaluation/summary.json
-@runs/my_agent/clarified/evaluation/summary.csv
-```
-
-When `--case` is omitted, the evaluator checks every GT case. Missing case
-folders or missing result tables are reported as `NOT_FOUND`. The command exits
-with code 0 only when every evaluated case passes.
-
-If you prepared only one case, pass `--case`. Omit `--case` only for a complete
-mode run.
+In `workflow` mode, the agent additionally reads `workflow_prompt.yaml`, converts
+its prep logic into a workflow JSON file, and runs the workflow executor (reads
+`./inputs`, writes `./result`) to produce the result tables. See
+[docs/WORKFLOW_EXECUTION.md](docs/WORKFLOW_EXECUTION.md).
 
 ## Minimal Smoke Tests
 
@@ -262,6 +238,3 @@ agent's workflow-generation path.
 
 **What does the evaluator score?** Only final result tables under
 `result/output_*.csv`.
-
-**Where should local runs go?** Use `@runs/<agent>/<mode>/`; this directory is
-ignored by git.

docs/DATASET.md

@@ -34,25 +34,9 @@ should run from prepared workspaces.
 ## Public Workspaces
 
 Participants should run agents from prepared workspaces rather than reading case
-files directly:
-
-```bash
-python scripts/prepare_run.py \
-  --mode clarified \
-  --case case_001 \
-  --run-root @runs/my_agent/clarified
-```
-
-For a complete mode run, prepare all evaluator cases:
-
-```bash
-python scripts/prepare_run.py \
-  --mode clarified \
-  --all \
-  --run-root @runs/my_agent/clarified
-```
-
-`--all` uses the GT case set and requires matching `data/case_xxx` directories.
+files directly. See the [README quickstart](../README.md#quickstart) for one
+case and [Run the Full Benchmark](../README.md#run-the-full-benchmark) for
+`--all`.
 
 Workspace contents by mode:
 

docs/USER_SIMULATOR.md

@@ -14,7 +14,7 @@ Install the repo with `python -m pip install -e .`, or set
 `PYTHONPATH=/path/to/prepbench/src`, so `simulator` is importable from
 `@runs/...` workspaces.
 
-Recommended DeepSeek setup for comparable runs:
+Recommended setup for comparable runs:
 
 ```bash
 PREPBENCH_SIMULATOR_BASE_URL=https://api.deepseek.com
@@ -24,14 +24,10 @@ PREPBENCH_SIMULATOR_TEMPERATURE=0
 PREPBENCH_SIMULATOR_API_KEY=your_api_key
 ```
 
-OpenRouter alternative:
-
-```bash
-PREPBENCH_SIMULATOR_BASE_URL=https://openrouter.ai/api/v1
-PREPBENCH_SIMULATOR_MODEL=deepseek/deepseek-v4-flash
-PREPBENCH_SIMULATOR_TEMPERATURE=0
-PREPBENCH_SIMULATOR_API_KEY=your_openrouter_api_key
-```
+To use another provider, change `PREPBENCH_SIMULATOR_BASE_URL`,
+`PREPBENCH_SIMULATOR_MODEL`, and `PREPBENCH_SIMULATOR_API_KEY` to match it. For
+OpenRouter, use `https://openrouter.ai/api/v1` and the namespaced model id
+`deepseek/deepseek-v4-flash`.
 
 Optional settings:
 
@@ -79,15 +75,11 @@ case_001 -> case_001
 
 ## Question Budget
 
-The default question budget is:
-
-```text
-max_questions = ceil(2.5 * ambiguity_count)
-```
+Two limits apply:
 
-The default cap is `25`, but the effective cap is never lower than the case's
-ambiguity count. A single `ask(...)` call may contain at most 10 questions by
-default.
+- **Per call:** a single `ask(...)` may contain at most 10 questions.
+- **Per case:** the total budget is `ceil(2.5 * ambiguity_count)`, capped at 25
+  but never lower than the case's own `ambiguity_count`.
 
 ## Question Boundaries