Skip to content

Rename input-file flags from --path to --input-path #69

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
johnjosephhorton merged 1 commit into from
Jul 9, 2026
Merged

Rename input-file flags from --path to --input-path #69

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 15 additions & 15 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ zwill survey create --name hello_world
zwill raw add \
--survey hello_world \
--id questionnaire \
--path "$ZWILL_REPO/examples/hello_world/raw/questionnaire.md" \
--input-path "$ZWILL_REPO/examples/hello_world/raw/questionnaire.md" \
--kind questionnaire \
--title "Hello World Questionnaire"
```
Expand Down Expand Up @@ -131,7 +131,7 @@ For normal use, put those records in JSONL and import the file instead. The comm
```bash
zwill respondent import \
--survey hello_world \
--path "$ZWILL_REPO/examples/hello_world/respondents.jsonl"
--input-path "$ZWILL_REPO/examples/hello_world/respondents.jsonl"
```

Answers work the same way. You can add them one at a time, and each answer is validated against both the respondent id and the question's declared options.
Expand All @@ -151,7 +151,7 @@ Or load the same answer records from JSONL:
```bash
zwill answer import \
--survey hello_world \
--path "$ZWILL_REPO/examples/hello_world/answers.jsonl"
--input-path "$ZWILL_REPO/examples/hello_world/answers.jsonl"
```

Inspect the respondent-by-question table:
Expand Down Expand Up @@ -297,7 +297,7 @@ One-shot generated analysis uses the same staged pattern as the executive report
```bash
zwill prob-results analysis-export --survey <survey> --job-id <probability_job_id> --path reports/<survey>/one-shot-marginals.html
zwill edsl-run --job .zwill/projects/default/practitioner_reports/<report_id>/job.edsl.json --path .zwill/projects/default/practitioner_reports/<report_id>/results.json.gz
zwill prob-results analysis-import --report-id <report_id> --path .zwill/projects/default/practitioner_reports/<report_id>/results.json.gz
zwill prob-results analysis-import --report-id <report_id> --input-path .zwill/projects/default/practitioner_reports/<report_id>/results.json.gz
zwill prob-results analysis-render --report-id <report_id> --path reports/<survey>/one-shot-marginals.html
```

Expand Down Expand Up @@ -350,7 +350,7 @@ For AgentList exports, choose answer traits with `--question` or `--questions`;
Inspect an exported AgentList:

```bash
zwill agent-list inspect --path agent_list.edsl.json
zwill agent-list inspect --input-path agent_list.edsl.json
```

Ask constructed agents a new question by exporting an EDSL job from the AgentList:
Expand All @@ -367,7 +367,7 @@ zwill agent-study export \
--path agent_study_job.edsl.json

zwill edsl-run --job agent_study_job.edsl.json --path agent_study_results.json.gz
zwill agent-study import --path agent_study_results.json.gz
zwill agent-study import --input-path agent_study_results.json.gz
zwill agent-study report --format table
```

Expand Down Expand Up @@ -404,7 +404,7 @@ Use `--leakage-exclusion <heldout_question>:<context_question>` for target-speci
Run with `zwill edsl-run`, then import and score:

```bash
zwill twin-results import --survey w158_ccpolicy --path twin_results.json.gz
zwill twin-results import --survey w158_ccpolicy --input-path twin_results.json.gz
zwill twin-results report --survey w158_ccpolicy --job-id <job_id>
zwill twin-results report --survey w158_ccpolicy --job-id <job_id> --format html --path twin_report.html
```
Expand Down Expand Up @@ -535,7 +535,7 @@ zwill twin-approach scaffold \

zwill twin-approach add \
--survey w158_ccpolicy \
--path baseline_context.approach.json
--input-path baseline_context.approach.json

zwill twin-approach add \
--survey w158_ccpolicy \
Expand Down Expand Up @@ -586,16 +586,16 @@ zwill twin-experiment init-plan \
--path policy_holdout_v1.json

zwill twin-experiment approve \
--path policy_holdout_v1.json \
--input-path policy_holdout_v1.json \
--approved-by <reviewer> \
--note "Approved held-out targets, context policy, leakage exclusions, sample size, models, and seed."

zwill twin-experiment export-plan \
--path policy_holdout_v1.json \
--input-path policy_holdout_v1.json \
--output-dir policy_holdout_v1_jobs
```

Plans start as drafts and must be approved before export. `export-plan` writes a `manifest.json`, one EDSL job JSON per arm, approved-plan provenance, and planned experiment records in `experiments.json`. Running remains explicit: use `zwill edsl-run --job <job>` for each exported job, then `zwill twin-results import --survey <survey> --path <results>`. After import, the existing comparison, plot, microdata, and report commands use the planned experiment records.
Plans start as drafts and must be approved before export. `export-plan` writes a `manifest.json`, one EDSL job JSON per arm, approved-plan provenance, and planned experiment records in `experiments.json`. Running remains explicit: use `zwill edsl-run --job <job>` for each exported job, then `zwill twin-results import --survey <survey> --input-path <results>`. After import, the existing comparison, plot, microdata, and report commands use the planned experiment records.

The approval review should check the held-out targets, construction approaches, context policy, target-specific leakage exclusions, respondent sample and seed, model list, and the prediction count formula: `respondents x held-out questions x approaches x models`. If a draft plan must be exported only for debugging, pass `--allow-unapproved`; this is intentionally visible in the command history.

Expand Down Expand Up @@ -723,7 +723,7 @@ zwill edsl-run \

zwill twin-experiment report-import \
--report-id <report_id> \
--path .zwill/projects/default/practitioner_reports/<report_id>/results.json.gz
--input-path .zwill/projects/default/practitioner_reports/<report_id>/results.json.gz

zwill twin-experiment report-render \
--report-id <report_id> \
Expand Down Expand Up @@ -792,7 +792,7 @@ zwill edsl-run \

zwill twin-results executive-summary-import \
--report-id <report_id> \
--path .zwill/projects/default/practitioner_reports/<report_id>/results.json.gz
--input-path .zwill/projects/default/practitioner_reports/<report_id>/results.json.gz

zwill twin-results executive-summary-render \
--report-id <report_id> \
Expand All @@ -819,7 +819,7 @@ zwill edsl-run \
--path .zwill/projects/default/practitioner_reports/<report_id>/results.json.gz

zwill twin-study practitioner-report-import \
--path .zwill/projects/default/practitioner_reports/<report_id>/results.json.gz
--input-path .zwill/projects/default/practitioner_reports/<report_id>/results.json.gz

zwill twin-study practitioner-report-render \
--report-id <report_id> \
Expand All @@ -843,7 +843,7 @@ zwill edsl-run \
--path .zwill/projects/default/practitioner_reports/<report_id>/results.json.gz

zwill twin-benchmark practitioner-report-import \
--path .zwill/projects/default/practitioner_reports/<report_id>/results.json.gz
--input-path .zwill/projects/default/practitioner_reports/<report_id>/results.json.gz

zwill twin-benchmark practitioner-report-render \
--report-id <report_id> \
Expand Down
40 changes: 20 additions & 20 deletions SPEC.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ Returns:
"warnings": [],
"errors": [],
"next_steps": [
"zwill raw add --survey hiring_study --id <id> --path <file-or-dir>",
"zwill raw add --survey hiring_study --id <id> --input-path <file-or-dir>",
"zwill question add --survey hiring_study ..."
]
}
Expand All @@ -149,7 +149,7 @@ State written:
zwill raw add \
--survey hiring_study \
--id questionnaire \
--path raw/questionnaire.pdf \
--input-path raw/questionnaire.pdf \
--kind questionnaire \
--title "Hiring Study Questionnaire"
```
Expand Down Expand Up @@ -262,7 +262,7 @@ Returns:
"warnings": [],
"errors": [],
"next_steps": [
"zwill answer import --survey hiring_study --path answers.jsonl"
"zwill answer import --survey hiring_study --input-path answers.jsonl"
]
}
```
Expand Down Expand Up @@ -314,7 +314,7 @@ Returns:
### Bulk Import Questions

```bash
zwill question import --survey hiring_study --path questions.jsonl
zwill question import --survey hiring_study --input-path questions.jsonl
```

Each JSONL row is one question:
Expand Down Expand Up @@ -383,7 +383,7 @@ Returns:
Bulk import respondents:

```bash
zwill respondent import --survey hiring_study --path respondents.jsonl
zwill respondent import --survey hiring_study --input-path respondents.jsonl
```

Input:
Expand Down Expand Up @@ -430,7 +430,7 @@ zwill agent-material add \
Bulk import agent material:

```bash
zwill agent-material import --survey hiring_study --path agent_material.jsonl
zwill agent-material import --survey hiring_study --input-path agent_material.jsonl
```

Input:
Expand Down Expand Up @@ -537,7 +537,7 @@ Returns:
### Bulk Import Answers

```bash
zwill answer import --survey hiring_study --path answers.jsonl
zwill answer import --survey hiring_study --input-path answers.jsonl
```

Input:
Expand Down Expand Up @@ -610,7 +610,7 @@ Users can attach markdown context to a survey. This is intended for source descr
Append markdown context from a file:

```bash
zwill context add --survey hiring_study --path source_context.md
zwill context add --survey hiring_study --input-path source_context.md
```

Or from a command argument:
Expand Down Expand Up @@ -644,7 +644,7 @@ Returns:
Replace existing context:

```bash
zwill context set --survey hiring_study --path source_context.md
zwill context set --survey hiring_study --input-path source_context.md
```

Show stored context:
Expand Down Expand Up @@ -727,7 +727,7 @@ This avoids presenting survey answers as cryptic generic traits. Users can overr
Inspect an exported AgentList:

```bash
zwill agent-list inspect --path hiring_agents.edsl.json
zwill agent-list inspect --input-path hiring_agents.edsl.json
```

The inspector reports agent count, trait keys, codebook keys, instruction coverage, mean instruction length, export metadata, and a small agent preview. Use `--format json` for machine-readable output.
Expand Down Expand Up @@ -760,7 +760,7 @@ The exported job is a normal EDSL `Jobs.to_dict()` serialization with the AgentL

```bash
zwill edsl-run --job hiring_agent_study.edsl.json --path hiring_agent_study_results.json.gz
zwill agent-study import --path hiring_agent_study_results.json.gz
zwill agent-study import --input-path hiring_agent_study_results.json.gz
```

Imported AgentStudy results are stored in two forms:
Expand Down Expand Up @@ -816,7 +816,7 @@ Import serialized EDSL `Results` from a probability job:
```bash
zwill prob-results import \
--survey hiring_study \
--path example_prob_job.json.gz
--input-path example_prob_job.json.gz
```

The command stores the raw Results file under `.zwill/projects/<project_id>/surveys/<survey>/probability_jobs/<job_id>/raw/` and extracts one predicted probability distribution per EDSL result row. The default `job_id` is a deterministic hash of the Results survey/scenario/model payload; it can be overridden with `--job-id`. JSON answers wrapped in markdown fences are accepted. Extracted rows are stored in `.zwill/projects/<project_id>/surveys/<survey>/probability_predictions.jsonl`.
Expand Down Expand Up @@ -866,7 +866,7 @@ Run the exported job with `zwill edsl-run`, then import and score the Results:
```bash
zwill twin-results import \
--survey hiring_study \
--path hiring_twin_results.json.gz
--input-path hiring_twin_results.json.gz

zwill twin-results report \
--survey hiring_study \
Expand Down Expand Up @@ -919,7 +919,7 @@ zwill twin-approach add \
--complete-cases \
--model openai:gpt-5.5

zwill twin-approach add --survey hiring_study --path approach.json
zwill twin-approach add --survey hiring_study --input-path approach.json
zwill twin-approach scaffold --survey hiring_study --approach-id baseline_context --path approach.json
zwill twin-approach list --survey hiring_study
zwill twin-approach show --survey hiring_study --approach-id baseline_context
Expand Down Expand Up @@ -957,7 +957,7 @@ Experiment plans compile one or more registered or inline approaches into EDSL j

```bash
zwill twin-experiment init-plan --survey hiring_study --plan-id holdout_v1 --heldout-questions q1,q2 --approach-id baseline_context --path holdout_v1.json
zwill twin-experiment export-plan --path holdout_v1.json --output-dir holdout_v1_jobs
zwill twin-experiment export-plan --input-path holdout_v1.json --output-dir holdout_v1_jobs
```

`init-plan` must write a valid editable JSON plan using existing survey question names, defaulting to the first multiple-choice question when no held-out question is provided. It must not mutate project state or export jobs.
Expand Down Expand Up @@ -1021,7 +1021,7 @@ zwill twin-experiment report-export \
--report-model openai:gpt-5.5

zwill edsl-run --job .zwill/projects/default/practitioner_reports/<report_id>/job.edsl.json --path .zwill/projects/default/practitioner_reports/<report_id>/results.json.gz
zwill twin-experiment report-import --report-id <report_id> --path .zwill/projects/default/practitioner_reports/<report_id>/results.json.gz
zwill twin-experiment report-import --report-id <report_id> --input-path .zwill/projects/default/practitioner_reports/<report_id>/results.json.gz
zwill twin-experiment report-render --report-id <report_id> --path experiment_report.html
```

Expand All @@ -1040,7 +1040,7 @@ Respondent metadata (panel covariates such as age, party, region) is included as
For a held-out question with `question_type: numeric`, `zwill edsl-export --survey <survey> --target numeric-twin-job --heldout-question <q>` builds a twin job in which the model predicts a **quantile distribution** (5th/25th/50th/75th/95th percentiles) of the value the respondent would give, honoring optional `numeric_min` / `numeric_max` bounds. Run it with `zwill edsl-run`, then:

```bash
zwill numeric-results import --survey <survey> --path numeric_results.json.gz
zwill numeric-results import --survey <survey> --input-path numeric_results.json.gz
zwill numeric-results report --survey <survey> --job-id <job_id> --format html --path numeric_report.html
```

Expand All @@ -1055,13 +1055,13 @@ Free-text (`free_text`) questions are validated by **coding them into themes** a
zwill edsl-export --survey <survey> --target open-codebook-job --heldout-question <free_text_q> \
--n-themes 8 --sample-answers 150 --model openai:gpt-5.5 --path codebook.edsl.json
zwill edsl-run --job codebook.edsl.json --path codebook_results.json.gz
zwill open-coding codebook-import --survey <survey> --path codebook_results.json.gz
zwill open-coding codebook-import --survey <survey> --input-path codebook_results.json.gz

# 2. Classify each respondent's answer into one codebook theme.
zwill edsl-export --survey <survey> --target open-coding-job --heldout-question <free_text_q> \
--coded-question-name <free_text_q>_coded --model openai:gpt-5.5 --path coding.edsl.json
zwill edsl-run --job coding.edsl.json --path coding_results.json.gz
zwill open-coding import --survey <survey> --path coding_results.json.gz
zwill open-coding import --survey <survey> --input-path coding_results.json.gz
```

`open-codebook-job` builds a single scenario that shows a sample of the answers (`--sample-answers`, default 150) and asks for at most `--n-themes` (default 8) mutually-exclusive themes. `codebook-import` stores it under `open_coding/<question>/codebook.json`. `open-coding-job` builds one scenario per respondent that classifies their actual answer into a codebook theme (or a reserved `unclassified` bucket). `open-coding import` writes a new `multiple_choice` question (`--coded-question-name`, default `<question>_coded`; options = theme codes, labels = theme labels) plus one coded answer per respondent, and returns a warning if more than 20% of answers were `unclassified` (the codebook may not fit) or if the results carried multiple codings per respondent (code with a single model). The coded question is then validated exactly like any other multiple-choice target.
Expand All @@ -1076,7 +1076,7 @@ Practitioner report generation follows the same export/run/import/render lifecyc

`zwill twin-benchmark practitioner-report-export` accepts the same `--manifest` or `--config` sources for cross-survey benchmark/meta reports. Cross-survey reports should be used to compare exercises and models, not as the default practitioner narrative for one uploaded survey.

The exported job is run through `zwill edsl-run` or by another agent using EDSL `job.run()`. `zwill twin-benchmark practitioner-report-import --path <results.json.gz>` stores the raw Results object under the matching report id and extracts the generated Markdown. `zwill twin-benchmark practitioner-report-render --report-id <report_id>` wraps the stored Markdown as standalone HTML using the stored benchmark payload; rendering must not call a frontier model. The HTML should embed source JSON and include a button that copies the Markdown report to the clipboard for LLM use. Public-facing report HTML should use Expected Parrot branding, should not mention the internal tool name, and should include canned boilerplate explaining why held-out questions are used as a proxy for new questions asked of instantiated digital twins. That boilerplate should explain that this is a high-bar test because survey designers typically avoid asking highly correlated questions.
The exported job is run through `zwill edsl-run` or by another agent using EDSL `job.run()`. `zwill twin-benchmark practitioner-report-import --input-path <results.json.gz>` stores the raw Results object under the matching report id and extracts the generated Markdown. `zwill twin-benchmark practitioner-report-render --report-id <report_id>` wraps the stored Markdown as standalone HTML using the stored benchmark payload; rendering must not call a frontier model. The HTML should embed source JSON and include a button that copies the Markdown report to the clipboard for LLM use. Public-facing report HTML should use Expected Parrot branding, should not mention the internal tool name, and should include canned boilerplate explaining why held-out questions are used as a proxy for new questions asked of instantiated digital twins. That boilerplate should explain that this is a high-bar test because survey designers typically avoid asking highly correlated questions.

`zwill twin-study practitioner-report` and `zwill twin-benchmark practitioner-report` remain one-step convenience commands. They should call the same export, EDSL run, import, and render machinery rather than using a separate hidden model-call path.

Expand Down
2 changes: 1 addition & 1 deletion examples/hello_world/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ This is the smallest useful `zwill` example: one survey question and five respon
```bash
zwill init
zwill survey create --name hello_world
zwill raw add --survey hello_world --id questionnaire --path examples/hello_world/raw/questionnaire.md --kind questionnaire --title "Hello World Questionnaire"
zwill raw add --survey hello_world --id questionnaire --input-path examples/hello_world/raw/questionnaire.md --kind questionnaire --title "Hello World Questionnaire"
zwill question add --survey hello_world --question-name favorite_color --question-type multiple_choice --question-text "Which color do you like best?" --question-option red --question-option blue --question-option green --role survey_item --source-raw questionnaire --source-note "Single hello-world test question."
zwill respondent add --survey hello_world --respondent-id r001 --weight 1.0 --metadata "sample_source=demo"
zwill respondent add --survey hello_world --respondent-id r002 --weight 1.0 --metadata "sample_source=demo"
Expand Down
4 changes: 2 additions & 2 deletions examples/hello_world/agent_list_study.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ zwill_cmd edsl-export \
--include-agent-material \
--agent-material-kind profile \
--path agent_list.edsl.json
zwill_cmd agent-list inspect --path agent_list.edsl.json
zwill_cmd agent-list inspect --input-path agent_list.edsl.json

zwill_cmd agent-study export \
--agent-list agent_list.edsl.json \
Expand All @@ -63,7 +63,7 @@ if [[ "${ZWILL_EXAMPLE_DRY_RUN:-0}" == "1" ]]; then
zwill_cmd edsl-run --job agent_study_job.edsl.json --path agent_study_results.json.gz --dry-run
else
zwill_cmd edsl-run --job agent_study_job.edsl.json --path agent_study_results.json.gz
zwill_cmd agent-study import --path agent_study_results.json.gz --replace
zwill_cmd agent-study import --input-path agent_study_results.json.gz --replace
zwill_cmd agent-study list
zwill_cmd agent-study report --format table
fi
Loading
Loading