Skip to content

CLI consistency/observability: survey report html has no JSON envelope; loaded_keys always empty; no preflight key check #37

Description

@johnjosephhorton

Summary

A few small consistency/observability rough edges in the CLI surface:

(a) survey report --format html emits no JSON status envelope

Nearly every zwill command returns a JSON status envelope on stdout (even when it also writes a file), which makes the CLI reliably scriptable. survey report --format html breaks that contract — it writes the HTML to --path and prints 0 bytes to stdout:

zwill survey report --survey kora --format html --path r.html | wc -c   # -> 0
zwill next --survey kora | wc -c                                        # -> 474

A caller that pipes it to a JSON parser gets Expecting value: line 1 column 1. Suggest emitting the usual {status, data:{path,...}, next_steps} envelope like the other file-writing commands.

(b) edsl-run reports loaded_env.loaded_keys: [] even after loading a .env

The run envelope includes:

"loaded_env": { "path": "/Users/.../.env", "loaded_keys": [] }

loaded_keys is always empty, so you can't tell from the output whether the .env actually contributed keys or which ones. Suggest listing the key names loaded (never values), e.g. ["EXPECTED_PARROT_API_KEY","OPENAI_API_KEY"].

(c) No preflight key check scoped to the requested steps

During a smoke run, edsl-run loaded ~/.env (which had EXPECTED_PARROT_API_KEY but not OPENAI_API_KEY) and nothing warned that the conditional-baseline key was absent — you only discover it later when twin-validate --require-baseline / twin-baseline fails. A preflight check that, given the requested operation (twins vs. baseline vs. embeddings), reports which required keys are present/missing up front would save a round-trip.

Suggested fixes

  • (a) Emit the standard JSON envelope from survey report --format html.
  • (b) Populate loaded_env.loaded_keys with the loaded key names.
  • (c) Add a preflight key/connectivity check (could wrap ep check) keyed to the operation being run.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions