Study-runner skill: force a delivered report; refresh stale baseline guidance

[johnjosephhorton]

What the transcript actually showed

Not a report build bug. When the user said "Let's validate, get one-shot marginals and then generate the full report," the agent replied "Standing by." When pushed ("why didn't you run the report?") it replied "Here whenever you need me." It never executed the commands — it went passive/idle (most likely triggered by the earlier baseline hang: 600s task → TaskStop → stale-notification cascade, now fixed in #77/#78).

The digital-twin-study-runner skill that drives the agent had no forcing terminal step and was stale, which is the zwill-side lever.

Fix

• New step 6 — "Build and deliver the full report — do not stop before this." Run zwill report build --survey <s> --path report_out, confirm report.html exists, and tell the user its path. Explicitly: never end a validation request with "standing by", and a "blocked" interpretation stage is not "no report" (the deterministic report is complete and viewable without the optional frontier-model narrative).
• Refresh the conditional-baseline description — it's XGBoost over embeddings + respondent covariates (not "embedding + logistic"), a strong deployable bar.
• Refresh the embedder guidance — auto tries Expected Parrot first (health-probed, fails over in seconds), then OpenAI, then local sentence-transformers, then a built-in lexical embedder, so it runs with no key and never hangs. The old text said it "needs OPENAI_API_KEY", which could make the agent needlessly skip it.
• Stop condition updated: the baseline always runs; flag the weaker lexical fallback rather than claiming it can't run.

Testing

pytest — 261 passed (docs-only change). Frontmatter untouched.

🤖 Generated with Claude Code

[greptile-apps]

Greptile Summary

This PR is a documentation-only update to the digital-twin-study-runner skill. It addresses an observed agent failure mode where the agent went passive after validation without ever generating the deliverable report.

• New Step 6 makes report generation a mandatory, terminal step: the agent must run zwill report build, confirm report_out/report.html exists, and tell the user its path — explicit instructions prevent it from ending with "standing by."
• Conditional-baseline description refreshed: now correctly describes XGBoost over embeddings plus respondent covariates, and documents the four-level embedder fallback chain (Expected Parrot → OpenAI key → local sentence-transformers → built-in lexical), removing the now-incorrect claim that OPENAI_API_KEY is required.
• Stop conditions updated: the old caveat about the baseline being unable to run without an embedding key is replaced with guidance on interpreting the weaker lexical fallback.

Confidence Score: 5/5

Documentation-only change; no executable code paths are touched and all 261 tests pass.

The change adds a mandatory terminal step and corrects two stale descriptions in the skill guidance document. There are no code changes, no schema changes, and no behavioral regressions possible from docs alone. The technical details added (XGBoost model, four-level embedder fallback, explicit path for the generated report) are consistent with what the PR description says the implementation actually does.

No files require special attention — the single changed file is a Markdown skill document.

Important Files Changed

Filename	Overview
zwill/skills/digital-twin-study-runner/SKILL.md	Documentation-only update: adds a mandatory Step 6 forcing report delivery, refreshes conditional-baseline description (XGBoost + covariates, auto-selecting embedder), and updates stop-conditions to reflect that the baseline always runs.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[twin-validate bundle\nvalidation_bundle/report.html] --> B{Frontier-model\nexecutive narrative\ngenerated?}
    B -- Yes --> C[zwill report build\n--survey id --path report_out]
    B -- No / blocked --> C
    C --> D[report_out/report.html\nreport_out/index.html\nwritten to disk]
    D --> E[Tell user the file path\nTask complete]

    style C fill:#2d6a4f,color:#fff
    style E fill:#1b4332,color:#fff
    style B fill:#555,color:#fff

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[twin-validate bundle\nvalidation_bundle/report.html] --> B{Frontier-model\nexecutive narrative\ngenerated?}
    B -- Yes --> C[zwill report build\n--survey id --path report_out]
    B -- No / blocked --> C
    C --> D[report_out/report.html\nreport_out/index.html\nwritten to disk]
    D --> E[Tell user the file path\nTask complete]

    style C fill:#2d6a4f,color:#fff
    style E fill:#1b4332,color:#fff
    style B fill:#555,color:#fff

Loading

_{Reviews (1): Last reviewed commit: "Study-runner skill: force a delivered re..." | Re-trigger Greptile}