From 6ca1e62c525693d7b7c9ca3978df6a881ed48d16 Mon Sep 17 00:00:00 2001 From: Krish Date: Thu, 6 Aug 2026 02:19:40 -0400 Subject: [PATCH 01/23] docs: add harness CLI workbench contract and lock the phase 1 core plan --- docs/architecture/harness-cli-workbench.md | 536 ++++++++++++++++++ .../2026-07-29-harness-cli-phase1-core.md | 148 +++++ .../test/prompt-library-contracts.test.mjs | 6 +- 3 files changed, 689 insertions(+), 1 deletion(-) create mode 100644 docs/architecture/harness-cli-workbench.md create mode 100644 docs/plans/2026-07-29-harness-cli-phase1-core.md diff --git a/docs/architecture/harness-cli-workbench.md b/docs/architecture/harness-cli-workbench.md new file mode 100644 index 00000000..1e99d51a --- /dev/null +++ b/docs/architecture/harness-cli-workbench.md @@ -0,0 +1,536 @@ +# Harness CLI Workbench — Feature Plan + +Finalized 2026-07-29. Execution begins after PR #37 (knowledge layer M1–M4) merges. The live execution plan for the current phase is the single dated file under `docs/plans/` (Phase 1: `2026-07-29-harness-cli-phase1-core.md`); this document is the durable contract the phase plans derive from. + +## TL;DR + +The concrete target is five releases: **CLI modes → knowledge navigation → governed execution → durable runs/TUI → resource and plugin system**. + +The new command families are: `search`, `lookup`, `tree`, `checks`, `exec`, `bash`, `run`, `config`, `trust`, `resources`, `plugin`, and `tui`. + +## Boundary and invariants + +Harness owns deterministic capabilities; the host owns LLM reasoning. Constraints that bind every phase: + +- **CLI never calls an LLM.** An LLM host consumes Harness; Harness never consumes a model. +- **Knowledge-layer invariants (settled on PR #37, do not re-litigate):** read paths never create the learnings store; the ops-JSON sole writer is `consolidate --apply`; the store lives at `~/.harness/knowledge//` (local git, never pushed); promoted learnings are immutable targets in both lanes; human authority derives from on-disk verified evidence and must be at least as recent as any recorded governance decision. +- **Search, lookup, and tree comply with the read-path rule** — navigation never creates or mutates the store. +- **Resources and plugins never write the learnings store, the run journal, or evidence.** Plugins may contribute knowledge *sources*; ingestion still flows through the consolidation loop. +- **Gate (vocabulary):** a gate is the existing plan/verification gate mechanism (`lib/gate.mjs`) that blocks lifecycle transitions until named checks and evidence pass. Later phases expose and record gates; they do not reinvent them. +- **Enforcement classes:** every control in this plan is one of — **enforced** (blocks pre-execution), **detect-and-block** (best-effort detection that halts on trip), or **audit-only** (recorded, never blocks). Each control names its class where it is specified; "governed" always means at least detect-and-block. +- **TUI boundary:** the TUI consumes the kernel in-process through the same command registry as the CLI — one behavior path, no separate implementation, no CLI shell-out requirement. This is the settled answer to the earlier SDK question; the CLI JSON/JSONL contracts serve out-of-process consumers on their program side (CI, hooks, Copilot, Codex) — the model side of any host consumes the agent lane per the output-lanes contract below. +- **Three output lanes, produced at the source:** every command renders its one canonical result as ledger (human), envelope (programs/TUI), and agent (LLM) lanes — deterministically, never via a model pass, and never by converting one lane into another. See "Output lanes" below. + +## Output lanes: the three-audience contract + +Every workbench-dispatched command renders one canonical result three ways. All three +renderings are deterministic CLI work — never a model pass. No output is ever converted +from one audience's format into another's; each lane is produced at the source. + +| Lane | Audience | Format | Contract | +|---|---|---|---| +| Ledger | Human | Styled ledger rows (`lib/style.mjs` conventions: glyph/key/value, truecolor → 256 → ASCII degradation) | What the TUI panes and plain terminal output render | +| Envelope | Programs / TUI | Versioned JSON envelope; JSONL streaming for long-running operations | Summary scalars first, detail arrays after, so one payload serves both a one-line footer and an expanded view. Long operations stream row-per-event with distinct `cancelled` vs `timed-out` terminal outcomes | +| Agent | LLM | Budgeted plain text | Token/byte-capped at the source; injection-hardened; byte-metered | + +### Agent-lane requirements + +1. **Budgeted at the source.** Every agent rendering carries a hard local cap, following + the existing harness precedents: 2048-byte context pack, 220-token plan slice, + bounded `harness get` excerpts (40 lines / 2048 bytes), 1000-token repo map. + Truncation happens at item boundaries; the reported size never exceeds the budget. +2. **Deterministic.** Produced by the CLI from the canonical result — never by asking a + model to summarize the envelope. This preserves the system-wide guarantee that + `orient`/`recall` and every read surface involve no model, network, or embedding call. +3. **Hardened.** Wherever the content is retrieved text (knowledge, search hits, doc + excerpts), the rendering passes the existing data boundary: data-not-instructions + preamble framing, `inertLine` neutralization, and secret redaction. +4. **Metered.** Rendered bytes are measured and emitted with the command's event so + `harness report` token/utilization SLOs account for the agent lane's real cost. + +### The consumption rule + +Agents consume the agent lane, never the envelope. JSON is token-inefficient and its +arrays are unbounded, so envelope output must never enter model context. The envelope +belongs to the TUI and tooling; the ledger belongs to humans. This is what makes the +dual human/LLM promise real: zero tokens are ever spent translating tool output. + +### Boundary with the harness-evolution track + +The workbench defines the registry, envelope schema, and run-journal contracts; the +harness evolution blueprint (`knowledge/proposals/harness-evolution-blueprint.md`, §9) +commits its new surfaces (`knowledge status/promote/prune`, `index --structural`, the +structural query) to being conforming citizens of them — registry-dispatchable, +envelope-emitting, streaming-capable, and shipping an agent rendering per this contract. + +## Final command surface + +### Knowledge and workspace navigation + +```text +harness index [--status] +harness search +harness lookup +harness tree +harness get --docid | --path +harness orient --query +``` + +#### `search` + +```text +harness search "lease fencing" \ + --scope code,knowledge,plans \ + --match ranked \ + --explain +``` + +Features: + +- Scopes: + - `code` + - `knowledge` + - `learnings` + - `plans` + - `skills` + - `checks` + - `events` + - `runs` + - `all` +- Match modes: + - `ranked`: BM25/knowledge ranking + - `literal`: exact content search + - `regex`: regex content search + - `path`: file/path discovery + - `symbol`: indexed symbol search +- Filters: + - path/glob + - collection + - result type + - minimum score + - limit +- Pagination cursor +- Snapshot-specific search +- Retrieval explanation +- Explicit partial-result handling +- Source, freshness, score and provenance in every result +- Empty search returns success with zero results + +This command incorporates ranked search, grep-like content search, find-like path search, and symbol lookup without creating separate public commands for each tool. + +Federation semantics (`--scope` with multiple sources or `all`) are deterministic: per-source scores are normalized before merging; result identity for dedup is (source, entity id); ordering ties break stably by (score, source, id); cursors remain valid across sources; a failed source is reported explicitly in the result envelope, never silently dropped. + +#### `lookup` + +```text +harness lookup +``` + +Supported kinds: + +```text +file | symbol | document | plan | skill | check | run | event | resource | learning | episode +``` + +Features: + +- Exact entity retrieval +- Metadata and source provenance +- Bounded content preview +- Related entities +- Current index generation +- Structured not-found error + +`get` remains the document/file compatibility command; `lookup` handles the broader entity model. `lookup learning` and `lookup episode` are read-only views over the knowledge store and respect the read-path invariant. + +#### `tree` + +```text +harness tree workspace [path] --depth 3 +harness tree knowledge [collection] +harness tree run +harness tree resources +``` + +Features: + +- Workspace hierarchy respecting ignore rules +- Knowledge collections, documents, and the learnings store (episodes → learnings → primitives, with provenance and governance state) +- Run lifecycle and evidence hierarchy +- Resource origin and override hierarchy +- Depth, type and path filters +- Human, JSON and TUI rendering + +`tree` is the single navigation verb: `harness tree resources` is canonical, and the resources family does not carry a duplicate `resources tree` subcommand. + +### Checks and execution + +```text +harness checks list +harness checks show +harness checks run + +harness exec [options] -- +harness bash [options] --