@@ -42,7 +42,7 @@ workspace. Your agent runs in that workspace and writes
4242| Case workspace | ` @runs/<agent>/<mode>/<case_id>/ ` |
4343| Candidate output | ` result/output_*.csv ` |
4444| Ground truth | ` src/evaluate/gt/case_xxx/ ` |
45- | Optional tools | ` simulator.LocalUserSimulatorAPI ` , ` py2flow.api.execute_flow_file ` |
45+ | Optional tools | user simulator, workflow executor |
4646
4747## Public Modes
4848
@@ -75,8 +75,9 @@ activated environment.
7575
7676PrepBench is intended to run from a source checkout because the dataset,
7777simulator assets, workflow prompt, and evaluator ground truth live in the repo.
78- The editable install makes ` simulator ` and ` py2flow ` importable when your agent
79- runs from ` @runs/... ` ; alternatively set ` PYTHONPATH=/path/to/prepbench/src ` .
78+ The editable install makes the simulator and workflow executor importable when
79+ your agent runs from ` @runs/... ` ; alternatively set
80+ ` PYTHONPATH=/path/to/prepbench/src ` .
8081
8182> ** Note:** The editable install registers a package named ` evaluate ` . If you
8283> have HuggingFace ` evaluate ` installed in the same environment, the two will
@@ -98,8 +99,8 @@ workspace, but they are not scored.
9899
99100In ` interactive ` and ` workflow ` mode, the agent may call the local user
100101simulator. In ` workflow ` mode, the agent also reads ` workflow_prompt.yaml ` for
101- the py2flow operator contract and uses the workflow executor to create the final
102- result tables.
102+ the workflow operator instructions and uses the workflow executor to create the
103+ final result tables.
103104
104105## Prepare a Workspace
105106
@@ -172,18 +173,10 @@ default is official `deepseek-v4-flash` in non-thinking mode with
172173provider-specific settings.
173174
174175For ` workflow ` , the agent may read ` workflow_prompt.yaml ` , write prep code as an
175- intermediate artifact, convert that code into a py2flow JSON DAG, and execute the
176- workflow from the workspace:
177-
178- ``` python
179- from py2flow.api import execute_flow_file
180-
181- execute_flow_file(flow_path = " workflow.json" )
182- ```
183-
184- The workflow executor defaults to ` ./inputs ` and ` ./result ` , so a workflow run in
185- the case workspace writes the same ` result/output_*.csv ` files that the evaluator
186- scores.
176+ intermediate artifact, convert that code into a workflow JSON file, and execute
177+ the workflow from the workspace. The workflow executor reads from ` ./inputs ` and
178+ writes to ` ./result ` , so the generated workflow produces the same
179+ ` result/output_*.csv ` files that the evaluator scores.
187180
188181## Evaluate
189182
0 commit comments