[Feature Request]: Orchestrator mode harness — no self-work, live status table, serial default, Learning Loop before kill

[MohabMohie]

Related to #5160 (SHAFT multi-ticket already orchestrates) and #5126 (local-coding sequential dispatch). This ticket does not reopen those. It specifies the remaining orchestrator-mode harness contract so a multi-task session cannot drift back into the main thread doing the work.

✅ Pre-submission Checklist

• Searched existing issues. SHAFT profile: multi-ticket assignments orchestrate, load all issues, fewest PRs, sequential chunks #5160 and Orchestrator sequential dispatch for the local coding delegate #5126 shipped SHAFT sequential multi-ticket and local-coding dispatch. No open ticket requires a live status table, per-delegate Learning Loop, default serial + optional parallel cap, or automatic switch on any multi-task request (not only named SHAFT issues).
• Checked current policy: chaos-engine/skills/chaos-engine/SKILL.md (solo-or-orchestrate), chaos-engine/references/delegation.md, chaos-engine/references/roles.md, chaos-engine/profiles/shaft/entrypoint.md.
• This is harness guidance + contract tests, not a Java product API.

🎯 Problem Statement

When the owner gives ChaosEngine more than one task, the main thread still sometimes implements, skips a living status board, leaves finished subagents alive, or finishes a chunk without routing learnings into source-controlled Memory / issues. #5160 made SHAFT multi-issue assignments orchestrate sequentially. That is not enough:

• "Multiple tasks" is broader than "multiple GitHub issues."
• Portable text still allows up to four concurrent writers as the default shape, while the owner default is one writer at a time unless they ask to parallelize.
• Delegates are not required to end with the reflective Learning Loop; only the session wrap-up is.
• There is no required, updated-in-the-open status table the owner can read while the orchestrator stays available.

The owner must be able to talk to the orchestrator at any time. The orchestrator must not disappear into implementation.

💡 Proposed Solution

Codify orchestrator mode as the automatic mode whenever the owner has two or more in-flight tasks (unrelated work streams, or two or more SHAFT issues — existing #5160 rule stays). In that mode the harness details below are mandatory and tested.

Orchestrator does no task work

• The main thread does no product, guidance, test, or implementation work.
• It does not edit, run long jobs, or install.
• It keeps: status, owner commands, consult/decompose, dispatch, review gate, merge babysit, and Learning Loop routing of leftovers the delegates failed to persist.
• It stays available for the owner in the same conversation.

Delegation and lifetime

• Every scoped task is assigned to a subagent (own worktree for writers).
• Each subagent loads the ChaosEngine entrypoint and follows the full workflow for its chunk: research receipt → implement/verify → independent review as required → PR/checkpoint if it made a retained commit → reflective self-learning (Learning Loop) before it reports done.
• When that subagent finishes (success, halt, or hand-back), the orchestrator kills it and its descendants. Do not followup_task / resume a completed assignment. A new review or a new chunk is a new instance (Harness: a completed subagent is being re-invoked repeatedly with the same instruction, burning ~85k tokens per cycle #4083 is the prior burn).
• After a SHAFT chunk PR merges, destroy that writer and start the next from a fresh ChaosEngine/* branch off fetched origin/main (SHAFT profile: multi-ticket assignments orchestrate, load all issues, fewest PRs, sequential chunks #5160).

Default serial, optional parallel

• Default: one subagent at a time, ordered by dependency then importance/priority of the scoped tasks.
• On owner request: parallelize independent writers. Default hard cap is 4 concurrent writing agents. A read-only reviewer does not consume a slot.
• The owner may set a lower or equal cap (1–4). The harness must refuse a requested cap above 4.
• File-overlapping writers never run in parallel even when parallel is requested.

Automatic switch

• Two or more owner tasks in one request (or two or more SHAFT issues, per SHAFT profile: multi-ticket assignments orchestrate, load all issues, fewest PRs, sequential chunks #5160) automatically enter orchestrator mode. Do not wait for the owner to say "orchestrate."
• Switching: finish or hand over anything the main thread already holds; do not start an edit in the same breath as adopting orchestrator mode.
• A host with no subagent primitive cannot orchestrate; it stays sequential solo and still owes the review gate (existing portable rule).

Live status table

After every dispatch and after every subagent finishes, the orchestrator posts/updates a status table the owner can read without reconstructing the session. Minimum columns:

ID / work item	Mode stream	Status	Owner / agent	Dependency	Last update	Details / evidence	Next action

Status values at least: planned, in progress, blocked, review, completed, out of scope.

The table must include completed, in progress, planned, and out of scope rows (out of scope stays visible so it is not silently dropped). Enhance with: blocker, PR/issue links, HEAD SHA, Learning Loop result (memory / issue / nothing durable).

Nothing durable is dropped

The orchestrator verifies that every subagent's Learning Loop actually landed in a source-controlled place before killing it:

• Failures, traps, and gotchas → native Memory on the task branch, or a new GitHub issue after duplicate search.
• Actionable leftover / future work → its own standalone issue (constraint.every-actionable-learning-requires-its-own-standalone-github-issue).
• Worthy reflections that change procedure → Memory or guidance, never chat-only.
• Secrets stay out of .memory/memory (.memory/private/ only).
• "Nothing durable" is a valid explicit result and must be recorded in the status table, not used to skip the check.

If a delegate omitted this, the orchestrator files the missing issue/Memory as an owner-command (allowed) and records that recovery in the table. It still does not implement the leftover work.

🔄 Alternatives Considered

• Leave SHAFT profile: multi-ticket assignments orchestrate, load all issues, fewest PRs, sequential chunks #5160 as the whole contract: it does not require the status table, per-delegate Learning Loop, or a user-settable parallel cap, and it is SHAFT-issue-specific.
• Default to four parallel writers: faster, but the owner default is serial so they can interrupt and so file conflicts stay rare.
• Let the orchestrator implement "small" leftover hunks: that is how orchestrator mode dies; reject it.

🗂️ Area of the Framework

• Other: ChaosEngine harness (chaos-engine/skills/chaos-engine/SKILL.md, references/delegation.md, references/roles.md, SHAFT entrypoint.md, router/harness tests)

📈 Use Case & Impact

Every multi-task session. The owner talks to one orchestrator, sees a current board, and never loses a finding that only lived in a dead subagent transcript.

User Scenarios & Testing

User Story 1 - Auto-switch and no self-implementation (Priority: P1)

Independent Test: A harness/router contract test (or pinned phrase + scenario fixture) fails if orchestrator mode is not selected for two owner tasks, or if the orchestrator-mode text still allows the main thread to implement.

Acceptance Scenarios:

1. Given one owner request that names two unrelated tasks, When triage runs, Then mode is orchestrator without the owner saying "orchestrate."
2. Given orchestrator mode, When a product/guidance chunk is ready, Then it is dispatched to a subagent; the main thread does not edit those files.

User Story 2 - Serial default, parallel on request (Priority: P1)

Independent Test: Policy pins plus a dispatch test: default concurrency is 1; owner-requested cap is honored up to 4; 5 is refused.

Acceptance Scenarios:

1. Given three independent streams and no parallel request, When dispatch runs, Then only one writer is live.
2. Given the owner asks to parallelize with cap 3, When four independent streams exist, Then at most three writers are live.
3. Given two streams that share files, When parallel is requested, Then they still run sequentially.

User Story 3 - Kill after finish + Learning Loop (Priority: P1)

Independent Test: Delegation/lifecycle contract asserts: completed assignment is closed; a new chunk is a new instance; dispatch text requires Learning Loop before done.

Acceptance Scenarios:

1. Given a writer reports done, When the orchestrator accepts the report, Then that agent is terminated and the status table shows Learning Loop result.
2. Given a writer omitted an actionable leftover, When the orchestrator checks the report, Then it files/searches a standalone issue before marking the row complete.

User Story 4 - Status table (Priority: P1)

Independent Test: Guidance names the required columns and statuses; a fixture/example in tests or the playbook matches.

Acceptance Scenarios:

1. Given a multi-task session just entered orchestrator mode, When the first status is posted, Then planned, in-progress, completed, and out-of-scope sections/rows exist as applicable.
2. Given a subagent finishes, When the orchestrator replies, Then the table is updated in that same turn (not deferred).

Edge Cases

• Reviewer is never a work stream and does not consume a writer slot (existing rule).
• Host with no subagent primitive: stay solo sequential; still show the status table.
• Switching from solo mid-task: no new edit until the handoff is recorded.
• Parallel cap of 1 equals the default.
• Do not put secrets in the status table or Memory.

Functional Requirements

• FR-001: Document orchestrator mode in the portable entrypoint and/or delegation.md so every host sees: no self-implementation, stay available, serial default, optional parallel cap (default max 4, owner-settable), auto-switch on multiple tasks.
• FR-002: Require every dispatched writer to end with the Learning Loop; orchestrator verifies source-controlled landing before kill.
• FR-003: Require the live status table and update-after-each-agent rule.
• FR-004: Kill finished agents; no resume of a completed assignment.
• FR-005: Keep SHAFT SHAFT profile: multi-ticket assignments orchestrate, load all issues, fewest PRs, sequential chunks #5160 sequential multi-issue rule; this ticket generalizes and fills gaps, it does not loosen SHAFT to silent parallel.
• FR-006: Keep taxonomy labels valid (one primary, one lifecycle, at least one subsystem/module).
• FR-007: Pin the new phrases in existing harness tests (test_agent_router_contract.py / portable-core / SHAFT profile tests as applicable).

Success Criteria

• SC-001: A reviewer can point at the checked-in guidance for every bullet in the owner request.
• SC-002: Contract tests fail if those phrases or the serial-default / auto-switch / kill / Learning Loop / status-table requirements are removed.
• SC-003: SHAFT multi-issue sequential behavior from SHAFT profile: multi-ticket assignments orchestrate, load all issues, fewest PRs, sequential chunks #5160 remains required.

Assumptions

• "Tasks" means owner-requested work streams, including but not limited to GitHub issues.
• Main-thread merge babysit and owner-command issue filing remain orchestrator duties, not "task work."

Out of scope

• Implementing Java product behavior.
• Changing capability-level names or adding a vendor/model name.
• Raising the writer cap above 4.
• Reopening SHAFT profile: multi-ticket assignments orchestrate, load all issues, fewest PRs, sequential chunks #5160 or Orchestrator sequential dispatch for the local coding delegate #5126.

📝 Additional Context

Current anchors to update or extend, not replace blindly:

• chaos-engine/skills/chaos-engine/SKILL.md — Solo or orchestrate
• chaos-engine/references/delegation.md — Main-thread duties, agent lifetime
• chaos-engine/references/roles.md — Orchestrator
• chaos-engine/profiles/shaft/entrypoint.md — Multi-ticket assignment orchestration
• chaos-engine/references/work-github-playbook.md — Learned-lessons workflow

[MohabMohie]

orchestrator mode should also group tasks that will change close proximity files together to minimize the risk of conflicts or redoing the work. and to target the least number of PRs that make sense, so no one-to-one mapping between an open issue or given task and the PR that fixes it.

chaos-engine when starting a new task should start by checking RAG and ensuring the task has a clear spec, if not then we should create the spec and track it.