Skip to content

dispatch_model.md

Latest commit

 

History

History
110 lines (83 loc) · 6.71 KB

dispatch_model.md

File metadata and controls

110 lines (83 loc) · 6.71 KB
Raw

Hierarchical Dispatch Model

The Workflow Server utilizes a Hierarchical Dispatch Model to execute workflows. This architecture leverages multi-agent delegation, where specialized agents spawn sub-agents to handle specific scopes of work, ensuring clear boundaries between high-level user interaction, workflow state management, and low-level task execution.

The model is host-agnostic: any IDE or agent harness that supports spawning background sub-agents can drive it (Cursor's Task tool is one such mechanism). For environments without sub-agent support, see "Environment Considerations" at the bottom of this document.

The Three Layers of Orchestration

1. Meta Orchestrator (Level 0)

  • Role: The user-facing top-level agent. It discovers workflows, handles high-level user goals, and dispatches the appropriate client workflows.
  • Responsibilities:
    • Finding and selecting workflows (discover, list_workflows).
    • Starting sessions via start_session({ agent_id }) (optionally with planning_slug once a planning folder exists).
    • Dispatching workflow orchestrators via start_session with parent_planning_slug to establish parent-child trace correlation.
    • Acting as the sole interface for user prompts (e.g., presenting checkpoints).
    • Never executes domain work or tracks detailed workflow state.

2. Workflow Orchestrator (Level 1)

  • Role: A persistent background sub-agent dedicated to managing a single client workflow from start to finish.
  • Responsibilities:
    • Evaluating state variables and determining the next_activity to execute.
    • Dispatching Activity Workers to perform the actual steps.
    • Managing Git artifacts, state persistence, and mechanical/semantic tracing.
    • Relaying checkpoints yielded by workers up to the Meta Orchestrator.
    • Checking workflow status via get_workflow_status.

3. Activity Worker (Level 2)

  • Role: An ephemeral sub-sub-agent dispatched to execute one specific activity.
  • Responsibilities:
    • Loading activity definitions via get_activity and executing steps sequentially.
    • Using domain-specific tools to write code, review PRs, or modify files.
    • Yielding execution when hitting a blocking checkpoint via yield_checkpoint.
    • Returning a structured result containing modified variables and created artifacts.

Mechanics of Dispatch

The dispatch process safely hands off execution from one layer to the next. Each session has a 6-character session_index (base32, deterministically derived from the planning slug); agents pass the index — not a token — on every authenticated call. The canonical state lives in the server-owned session.json (see State Management).

Dispatching the Workflow Orchestrator (L0 → L1)

When the Meta Orchestrator decides to start a workflow (e.g., work-package), it calls start_session with the parent's planning_slug:

start_session({
  workflow_id: "work-package",
  planning_slug: "<child_slug>",
  parent_planning_slug: "<meta_slug>",
  agent_id: "workflow-orchestrator"
})

This creates a child session under the child planning folder; the server snapshots the parent's session.json (after seal-verifying it) under the child's parentSession field for trace correlation and recursive parent traversal. A trace event in the parent's trace store links the two sessions. The response includes a session_index for the child session.

The Meta Orchestrator then uses the host's sub-agent spawn mechanism (e.g., Cursor's Task tool) to start a background workflow orchestrator:

Task({
  subagent_type: "generalPurpose",
  prompt: "You are a workflow orchestrator. Your session_index is: <child_index>..."
})

Dispatching the Activity Worker (L1 → L2)

The Workflow Orchestrator evaluates the workflow, determines the next activity to run, and passes its own session_index to the worker.

Unlike the L0 → L1 transition (which creates a new child session), the L1 → L2 transition shares the same session_index. The worker uses this index directly to call get_activity, get_technique, and next_activity; the server resolves the index to the same session.json the orchestrator is reading, so both agents see a single canonical state.

The Workflow Orchestrator uses the host's sub-agent spawn mechanism to spawn the Activity Worker:

Task({
  subagent_type: "generalPurpose",
  prompt: "You are an autonomous worker agent... session_index: <orchestrator_index>... Activity: implement..."
})

Workflow Status Polling

The meta orchestrator can poll the status of a dispatched workflow using get_workflow_status:

get_workflow_status({ session_index: "<child_index>" })

This returns:

  • status: active, blocked, or completed
  • current_activity: The activity the sub-agent is executing
  • completed_activities: Activities finished so far (derived from session.json + trace)
  • last_checkpoint: The most recent resolved checkpoint
  • parent: If the session was dispatched, the parent's session info derived from parentSession in session.json

The status is determined from the session state: blocked when activeCheckpoint is set in session.json, completed when the workflow has no more activities, and active otherwise.

Resuming Sub-Agents

When an agent pauses (e.g., waiting for a checkpoint resolution), it doesn't die. Hosts that support sub-agents typically expose a resume mechanism that re-enters a previously spawned sub-agent with new instructions appended to its existing context (Cursor's Task tool, for example, accepts a resume parameter keyed on the sub-agent's ID).

When the parent agent needs to wake the sub-agent back up, it calls the host's resume primitive:

Task({
  resume: "<sub_agent_id>",
  prompt: "The checkpoint has been resolved. The user selected option 'proceed'. Please continue."
})

This appends the new instructions directly to the sub-agent's existing context window, allowing it to seamlessly continue its execution loop without losing its memory of the codebase or workflow state.

Environment Considerations (Inline Fallback)

The Hierarchical Dispatch Model is optimised for hosts that support background sub-agents (e.g., Cursor's Task tool).

In environments that do not support sub-agent spawning, the top-level agent must execute the workflow inline. A single agent sequentially adopts the personas of the Meta Orchestrator, Workflow Orchestrator, and Activity Worker, executing all instructions within a single contiguous conversation thread. The server's enforcement layers (HMAC tokens, checkpoint gates, manifests, trace tokens) function identically in either mode — only the persona-switching mechanism changes.