Skip to content

Directives queued via evo direct are never delivered to engaged sessions (events/ -> delivered/ path broken) #58

Description

@alokwhitewolf

Follow-up to #49 (engagement detection). #49's fix opens the eligibility gate -- sessions whose .evo/ was created mid-run now correctly flip has_evo_engaged: true and pass the fanout filter. But the delivery half of the directive system is broken: queued directives never reach the agent's context as [EVO DIRECTIVE id=...] banners.

Repro

In a real claude --print PostTrainBench run on alpha.7 (the run that validated #49 end-to-end):

  1. Agent runs evo init, evo new, evo run baseline normally. Session record at .evo/run_0000/inject/sessions/<sid>.json lands has_evo_engaged: true (the hook-drain: session not registered when .evo/ is created mid-session under claude --print #49 fix works -- confirmed via on-disk check).

  2. From a separate shell on the same host, fire a directive:

    evo direct "The optimize skill body explicitly addresses serial / single-GPU workloads: ..."
    

    Output:

    directive queued (id=019E88B5250EFFF54A5B0ADFFAA3EB20,
                      fanout=1 sessions,
                      skipped_unengaged=0,
                      skipped_subagent=0)
    

    Fanout=1 confirms the session was eligible -- engagement check passed.

  3. Wait 3 minutes. Check .evo/run_0000/inject/ state:

    events/workspace.jsonl    -- contains the directive (825 bytes, 2 entries)
    delivered/                -- EMPTY
    acks/                     -- EMPTY
    
  4. Grep the agent's stream-json log for the banner:

    grep -oE "\[EVO DIRECTIVE id=[A-F0-9]+\]" /home/ubuntu/ptb-run.log
    -> no matches
    
  5. Agent never invoked evo ack <event_id>. Agent's prose continued the original bypass plan it was on before the directive fired -- proving the directive content never entered its context.

Diagnosis

Two halves of the directive system:

Half Status
Queue: evo direct enqueues into events/workspace.jsonl + filters eligible sessions working (alpha.7 #49 fix)
Delivery: hook-drain reads events on each hook fire, splices [EVO DIRECTIVE id=...] banner into hook response, moves event to delivered/ broken -- delivery never fires

PostToolUse hooks fire constantly during the run (every Bash, Read, Write tool call). Each fire SHOULD trigger drain.py to:

  1. Scan inject/events/workspace.jsonl for unread directives addressed to this session
  2. Splice each as an [EVO DIRECTIVE id=<event_id>] ... [END EVO DIRECTIVE] banner into the hook response's additionalContext (or equivalent host-specific mechanism)
  3. Write the event id to inject/delivered/<event_id>-<sid>.json (or similar marker) so it's not re-delivered

The delivered/ dir stays empty for the entire run -- the splice step never happens.

Suspected source area

plugins/evo/src/evo/drain.py (the Python drain hand-off from the Rust hook-drain binary). The Rust binary at bin/evo-hook-drain-rs/src/main.rs correctly hands off via handoff_to_drain(...) when should_handoff returns true. But the Python side either:

  • Isn't being invoked (the Rust handoff fails silently)
  • IS invoked but doesn't scan workspace-broadcast events (only per-session/per-exp targeted directives)
  • Scans events but the banner-splicing code path is broken for the additionalContext envelope

The session record has has_evo_engaged: true, so the eligibility check passes if it's even being evaluated. The fact that NOTHING ever lands in delivered/ suggests the scan isn't running at all -- not just a filter mismatch.

Why this matters

The entire evo direct mid-run injection mechanism is non-functional. Use cases broken:

  • Mid-run course correction (user observes the agent making a wrong call, fires a directive to redirect)
  • Operator interventions during long autonomous runs
  • Cross-agent / dashboard-driven nudges
  • The verifier subagent's [EVO DIRECTIVE ...] pre-experiment annotations (if those route through the same channel)

In our PostTrainBench run, the agent was reasoning itself out of using /evo:optimize ("instead of going through the full evo:optimize ceremony, I'll directly drive experiments"). The directive content was crafted specifically to course-correct this. Queue confirmed delivery (fanout=1). Agent never saw it. Run continued the bypass.

Bonus minor bug surfaced during diagnosis

evo direct-status <id> fails with git rev-parse --show-toplevel returned non-zero exit status 128 when invoked outside the workspace directory. Other evo subcommands (evo direct, evo status) work fine from arbitrary cwd. direct-status should be workspace-agnostic the same way -- or at least error with a clear "must be in a workspace" message instead of leaking the git internal failure.

Out of scope

  • Re-design of the additionalContext envelope shape
  • Per-host delivery (claude-code / codex / opencode / etc. -- focus on claude-code first since the repro is there)
  • Retry/backoff for failed deliveries

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions