Summary
A few small consistency/observability rough edges in the CLI surface:
(a) survey report --format html emits no JSON status envelope
Nearly every zwill command returns a JSON status envelope on stdout (even when it also writes a file), which makes the CLI reliably scriptable. survey report --format html breaks that contract — it writes the HTML to --path and prints 0 bytes to stdout:
zwill survey report --survey kora --format html --path r.html | wc -c # -> 0
zwill next --survey kora | wc -c # -> 474
A caller that pipes it to a JSON parser gets Expecting value: line 1 column 1. Suggest emitting the usual {status, data:{path,...}, next_steps} envelope like the other file-writing commands.
(b) edsl-run reports loaded_env.loaded_keys: [] even after loading a .env
The run envelope includes:
"loaded_env": { "path": "/Users/.../.env", "loaded_keys": [] }
loaded_keys is always empty, so you can't tell from the output whether the .env actually contributed keys or which ones. Suggest listing the key names loaded (never values), e.g. ["EXPECTED_PARROT_API_KEY","OPENAI_API_KEY"].
(c) No preflight key check scoped to the requested steps
During a smoke run, edsl-run loaded ~/.env (which had EXPECTED_PARROT_API_KEY but not OPENAI_API_KEY) and nothing warned that the conditional-baseline key was absent — you only discover it later when twin-validate --require-baseline / twin-baseline fails. A preflight check that, given the requested operation (twins vs. baseline vs. embeddings), reports which required keys are present/missing up front would save a round-trip.
Suggested fixes
- (a) Emit the standard JSON envelope from
survey report --format html.
- (b) Populate
loaded_env.loaded_keys with the loaded key names.
- (c) Add a preflight key/connectivity check (could wrap
ep check) keyed to the operation being run.
Summary
A few small consistency/observability rough edges in the CLI surface:
(a)
survey report --format htmlemits no JSON status envelopeNearly every
zwillcommand returns a JSON status envelope on stdout (even when it also writes a file), which makes the CLI reliably scriptable.survey report --format htmlbreaks that contract — it writes the HTML to--pathand prints 0 bytes to stdout:A caller that pipes it to a JSON parser gets
Expecting value: line 1 column 1. Suggest emitting the usual{status, data:{path,...}, next_steps}envelope like the other file-writing commands.(b)
edsl-runreportsloaded_env.loaded_keys: []even after loading a.envThe run envelope includes:
loaded_keysis always empty, so you can't tell from the output whether the.envactually contributed keys or which ones. Suggest listing the key names loaded (never values), e.g.["EXPECTED_PARROT_API_KEY","OPENAI_API_KEY"].(c) No preflight key check scoped to the requested steps
During a smoke run,
edsl-runloaded~/.env(which hadEXPECTED_PARROT_API_KEYbut notOPENAI_API_KEY) and nothing warned that the conditional-baseline key was absent — you only discover it later whentwin-validate --require-baseline/twin-baselinefails. A preflight check that, given the requested operation (twins vs. baseline vs. embeddings), reports which required keys are present/missing up front would save a round-trip.Suggested fixes
survey report --format html.loaded_env.loaded_keyswith the loaded key names.ep check) keyed to the operation being run.