feat: introduce Terminal Studio (TUI)

.agents/skills/use-appclaw-cli/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: use-appclaw-cli
 description: >
-  Use the AppClaw CLI to run YAML flows, start interactive playground, explore apps,
+  Use the AppClaw CLI to run YAML flows, start the interactive TUI shell, explore apps,
   record/replay sessions, configure devices, and troubleshoot. Trigger for any request
   involving appclaw commands, device setup, .env configuration, running flows, vision
   setup, or debugging execution failures.
@@ -91,17 +91,19 @@ appclaw --flow tests/flows/youtube-phased.yaml --env dev
 
 **No LLM key needed** unless the flow has steps that fall back to LLM parsing (unrecognized natural language).
 
-### 3. Playground — interactive REPL
+### 3. Terminal Studio (`--tui`, alias `--playground`)
 
 ```sh
-appclaw --playground
-appclaw --playground --platform ios --device-type simulator
-appclaw --playground --device "iPhone 17 Pro"
+appclaw --tui
+appclaw --tui --platform ios --device-type simulator
+appclaw --tui --device "iPhone 17 Pro"
 ```
 
-Type natural language commands that execute live on the device. Steps accumulate and can be exported to a YAML flow.
+The interactive mode. Type natural language commands that execute live on the device; steps accumulate and can be exported as a YAML flow or an `@appclaw/runner` spec via `/export`.
 
-**REPL commands:** `/help`, `/steps`, `/export`, `/clear`, `/device`, `/disconnect`
+`--playground` is an alias for `--tui` — the old playground REPL was removed. (`--json --playground` is different: a headless NDJSON bridge used by the VS Code / Cursor extension, not something to run by hand.)
+
+Full-screen Ink shell: platform/device picker, slash-command palette, goal REPL, settings, run history. `/stream` (Android only) mirrors the device screen **inside the terminal** — Kitty graphics on Ghostty/kitty/WezTerm, 24-bit ANSI half-blocks everywhere else — at ~5fps via `adb screencap`; `/stream-close` stops it. Requires an interactive terminal; incompatible with `--json`.
 
 ### 4. Explorer — PRD to test flows
 
@@ -317,7 +319,7 @@ Records successful trajectories to `~/.appclaw/trajectories.json` and reuses the
 - `appclaw "goal"` (agent mode — uses LLM credits, takes actions on device)
 - `appclaw --explore` (LLM credits + device crawling)
 - `appclaw --record` (agent mode + saves recording)
-- `appclaw --playground` (interactive device session)
+- `appclaw --tui` (interactive device session; goals typed inside it use LLM credits — `--playground` is an alias for it)
 
 Why: agent and explorer modes consume LLM API credits and take real actions on the connected device.
 
@@ -390,8 +392,8 @@ appclaw --flow tests/flows/youtube-phased.yaml --env dev
 ### Quick test on iOS simulator
 
 ```sh
-appclaw --platform ios --device-type simulator --playground
-# In REPL: type commands, test them, /export to YAML
+appclaw --platform ios --device-type simulator --tui
+# In the TUI: type commands, test them, /export to YAML
 ```
 
 ### Generate test flows from a PRD

.gitignore

@@ -3,6 +3,9 @@ dist/
 .env
 *.log
 .DS_Store
+# JetBrains workspace state — per-developer, never shared
+.idea/
+*.iml
 .claude/
 recordings/
 logs/

.kiro/steering/product.md

@@ -6,7 +6,7 @@ AppClaw is an agentic AI layer for mobile automation on Android and iOS. Users d
 
 - **Agent mode** — LLM-driven goal execution (e.g. `appclaw "Send a WhatsApp message to Mom"`)
 - **YAML flows** — declarative, zero-LLM automation steps defined in YAML files
-- **Playground** — interactive REPL for building flows live on a device
+- **Terminal Studio** (`--tui`, alias `--playground`) — the interactive mode: step recorder, device picker, command palette, run history, and in-terminal device stream
 - **Explorer** — generates YAML test flows from a PRD or app description
 - **Record/Replay** — capture and adaptively replay goal executions
 - **Report** — Express server serving HTML run reports

.kiro/steering/structure.md

@@ -36,7 +36,8 @@ appclaw/
 | `device/`        | Device setup pipeline — platform/device picker, iOS setup, Appium session         |
 | `memory/`        | Episodic memory — trajectory recording, fingerprinting, retrieval                 |
 | `explorer/`      | PRD → YAML flow generation, screen crawler                                        |
-| `playground/`    | Interactive REPL for building flows                                               |
+| `step-recorder/` | Shared step-recording helpers + the headless `--json --playground` NDJSON bridge  |
+| `tui/`           | Terminal Studio — multi-screen Ink app (`--tui`, alias `--playground`)            |
 | `recording/`     | Session recorder and adaptive replayer                                            |
 | `report/`        | Run artifact collection, HTML report rendering, Express server                    |
 | `sdk/`           | Public SDK — `GoalRunner`, `FlowRunner`, `StepRunner`, config builder             |

CLAUDE.md

@@ -27,12 +27,12 @@ The CLI routes to 6 modes based on flags:
 
 - **Interactive** (default) — prompts for platform/device/goal, runs agent loop
 - **YAML Flow** (`--flow file.yaml`) — declarative automation, zero LLM cost
-- **Playground** (`--playground`) — interactive REPL for building flows
+- **Terminal Studio** (`--tui`, or its alias `--playground`) — multi-screen Ink app: platform/device picker, slash-command palette, step recording, `/goal` agent runs, settings, run history, and device mirroring — `/stream` renders the screen inside the terminal (see Terminal Studio below). This is the interactive mode; the old `--playground` REPL was removed and the flag now routes here.
 - **Explorer** (`--explore`) — PRD → YAML test flow generation
 - **Record/Replay** (`--record`, `--replay`) — capture and replay sessions
 - **Report** (`--report`) — Express server serving HTML reports from `.appclaw/runs/`
 
-The interactive and goal-direct paths also accept `--export [path]` (optionally `--export-dir <dir>`) to write a replayable vitest spec when the goal completes. Path resolution: empty → `EXPORT_DIR/<goal-slug>.test.ts`; bare filename → `EXPORT_DIR/<name>`; anything with a directory hint → used verbatim. Implementation in `src/sdk/goal-export.ts` (translator + renderer) called from `src/index.ts` after the agent loop.
+The interactive and goal-direct paths also accept `--export [path]` (optionally `--export-dir <dir>`) to write a replayable `@appclaw/runner` spec when the goal completes. Path resolution: empty → `EXPORT_DIR/<goal-slug>.test.ts` (EXPORT_DIR defaults to `tests`, the runner's own testDir, so an export is runnable where it lands); bare filename → `EXPORT_DIR/<name>`; anything with a directory hint → used verbatim. Implementation in `src/sdk/goal-export.ts` (translator + renderer) called from `src/index.ts` after the agent loop.
 
 ### SDK (`src/sdk/`)
 
@@ -41,10 +41,10 @@ Public TypeScript API consumed by external tests (vitest/jest/mocha). Single ent
 - `app.run(instruction, options?)` — one natural-language step, non-throwing, returns `{ success, action, message }`. `options` (`RunOptions`) applies per-command overrides for this call only: `waitTimeout`/`waitInterval` (implicit-wait poll budget) and `scrollMode`/`scrollTimes` (scroll/swipe distance + count). Instance-wide defaults for all four live on `AppClawOptions`; per-call values win. Implicit wait: every element-bearing action polls its target until present (DOM re-reads page source, vision re-captures the screenshot) or the budget is exhausted — `WAIT_TIMEOUT`/`WAIT_INTERVAL` env, default 10s/300ms
 - `app.verify(claim)` — assertion. Throws `AppClawAssertionError` on failure (includes `claim`, `result`, and `screenContents` from DOM page-source in DOM mode — in vision mode the LLM's reason is already in `result.message`)
 - `app.runFlow(path)` — wraps the YAML flow engine
-- `app.runGoal(goal, { exportPath?, exportConfig? })` — wraps the agent loop. When `exportPath` is set, the trajectory is filtered with `keepOnlyFinalAttempt()` (drops the branch before any rejected `done`) then rendered as a vitest spec via `generateSdkTest()`
+- `app.runGoal(goal, { exportPath?, exportConfig? })` — wraps the agent loop. When `exportPath` is set, the trajectory is filtered with `keepOnlyFinalAttempt()` (drops the branch before any rejected `done`) then rendered as an `@appclaw/runner` spec via `generateSdkTest()`
 - `app.teardown()` — finalize report, close MCP
 
-Helpers in `src/sdk/goal-export.ts`: `keepOnlyFinalAttempt`, `instructionsFromHistory`, `decisionToInstruction`, `generateSdkTest`, `generateSdkTestFromInstructions` (the last is used by the playground's `/export *.test.ts`).
+Helpers in `src/sdk/goal-export.ts`: `keepOnlyFinalAttempt`, `instructionsFromHistory`, `decisionToInstruction`, `generateSdkTest`, `generateSdkTestFromInstructions` (the last is used by the step recorders' `/export *.test.ts` — see `packages/cli/src/step-recorder/`).
 
 ### Core Agent Loop (`src/agent/loop.ts`)
 
@@ -60,7 +60,7 @@ Supporting agent modules: planner (goal decomposition), recovery (checkpointing)
 
 ### Key Module Responsibilities
 
-- **`src/sdk/`** — Public TypeScript API for external tests. `index.ts` is the `AppClaw` class; `goal-export.ts` translates agent histories back to natural-language `app.run(...)` calls and renders vitest specs; `step-runner.ts` adapts the YAML flow engine to single-instruction calls; `screen-snapshot.ts` captures visible DOM text for assertion-error context.
+- **`src/sdk/`** — Public TypeScript API for external tests. `index.ts` is the `AppClaw` class; `goal-export.ts` translates agent histories back to natural-language `app.run(...)` calls and renders `@appclaw/runner` specs; `step-runner.ts` adapts the YAML flow engine to single-instruction calls; `screen-snapshot.ts` captures visible DOM text for assertion-error context.
 - **`src/llm/`** — Multi-provider LLM integration. `provider.ts` is the factory; `prompts.ts` builds system/user messages; `schemas.ts` defines action schemas. Tools from appium-mcp are dynamically converted to Vercel AI SDK format.
 - **`src/mcp/`** — Appium MCP client wrapper. Connects via stdio (subprocess) or SSE. Handles tool calling, element finding, screenshots, keyboard input.
 - **`src/perception/`** — Screen parsing. Android (`android-parser.ts`) and iOS (`ios-parser.ts`) XML parsers. `dom-trimmer.ts` compacts DOM for LLM token efficiency.
@@ -70,6 +70,17 @@ Supporting agent modules: planner (goal decomposition), recovery (checkpointing)
 - **`src/memory/`** — Episodic memory. Records successful trajectories to `~/.appclaw/trajectories.json`, retrieves relevant past experiences via fingerprinting.
 - **`src/report/`** — Execution reporting. `writer.ts` collects artifacts; `renderer.ts` generates HTML reports; `server.ts` serves them.
 - **`src/ui/terminal.ts`** — Rich terminal output (spinners, boxes, gradient headers, markdown rendering). JSON output mode for IDE integration (`json-emitter.ts`).
+- **`packages/cli/src/step-recorder/`** — Shared by every step-recording surface. `flow-builder.ts` renders a recorded `FlowStep[]` as YAML or an `@appclaw/runner` spec and resolves `/export` paths; `screen-info.ts` answers "what's on screen?" via one vision call; `memory-inspect.ts` backs `/memory`. `json-bridge.ts` is the headless NDJSON-over-stdio recorder behind `appclaw --json --playground`, which the VS Code / Cursor extension spawns — its wire protocol is a shipped contract (see `vscode-extension/src/bridge.ts`).
+
+### Terminal Studio (`packages/cli/src/tui/`)
+
+`appclaw --tui` (and its alias `appclaw --playground`) is a separate, multi-screen Ink app (distinct from the single-screen agent-run UI in `packages/cli/src/ui/ink/`) with its own observable store (`store.ts`, a small subscribe/snapshot pub-sub) and a screen router (`TuiApp.tsx`) switching between `screens/{Welcome,DevicePicker,Main,Settings,History}Screen.tsx`. `commands.ts` defines the `/`-prefixed command palette shown on the Main screen. Anything NOT starting with `/` is one deterministic instruction — `runOneInstruction()` — appended to `store.steps`; that recording is what `/list`, `/yaml`, `/edit` and `/export` operate on, and a step that fails is reported but not recorded. `/goal <text>` is the opt-in to the autonomous loop (a single flat `runAgent()` call, no multi-sub-goal planner — that stays on the plain `appclaw "goal"` path) and records nothing. Device listing goes straight through `adb`/`xcrun simctl` (`@appclaw/core/device/emulator-list.ts`), not through an MCP session, so the picker works before any Appium session exists.
+
+**Device mirroring** is Android-only, and `adb -s <serial>` is device-agnostic — the same path covers a running emulator, a physical phone and a headless emulator.
+
+`/stream` renders the device **inside the terminal** (`packages/cli/src/tui/stream/`). Frames go to the side panel on the main screen (`components/StreamPanel.tsx`), which draws only chrome and leaves a blank region so the command palette and prompt stay live alongside it; `frame-loop.ts` polls `adb exec-out screencap` every 200ms and paints that region with direct `process.stdout.write`s — never from React, since Ink rewrites its whole frame on every state change. `terminal-caps.ts` picks the backend from env (no capability query — Ink owns stdin in raw mode): **Kitty graphics** for Ghostty/kitty/WezTerm (`kitty.ts` sends a PNG by file path, `a=T,f=100,t=f,c=…,r=…`, so the terminal does the scaling), otherwise **24-bit ANSI half-blocks** (`halfblock.ts` downsamples the raw RGBA framebuffer from `screencap` with no `-p`, so no PNG decoder and no new dependency). `layout.ts` holds the geometry both sides agree on. Force a backend with `APPCLAW_STREAM_BACKEND=kitty|halfblock`.
+
+`/stream-close` stops it; device switches and `quit()` tear the frame loop down the same way, via `resetStream()`.
 
 ### Configuration (`src/config.ts`)
 

README.md

@@ -44,23 +44,26 @@ appclaw "open the settings app and turn on airplane mode"
 
 You'll need **Node.js 22+**, a connected device / emulator / simulator, and an **LLM API key** (Anthropic, OpenAI, Google, Groq, or local Ollama). `appclaw doctor` checks all of this in seconds and prints fix hints for anything missing (`--full` also spawns appium-mcp for a real handshake). Full setup → **[appclaw.in](https://appclaw.in)**.
 
+Terminal Studio (`appclaw --tui`) can mirror the device inside the terminal with `/stream` (Android only). It looks best on a terminal that speaks the kitty graphics protocol — Ghostty, kitty, WezTerm — and falls back to half-block characters elsewhere.
+
 ## What it can do
 
 - **Agent mode** — plain-English goals; the LLM drives the device (tap, type, swipe)
 - **YAML flows** — deterministic, zero-LLM automation with [structured selectors and state assertions](docs/structured-selectors.md)
-- **Test runner** — vitest-style specs across real devices; scaffold with `appclaw init`
+- **Terminal Studio** (`appclaw --tui`) — record steps one at a time, watch the device mirrored in the terminal, then `/export` a runnable spec
+- **Test runner** — vitest-style specs across real devices, run with `appclaw test`; scaffold with `appclaw init`
 - **SDK** — drive AppClaw from your own vitest / jest / mocha
-- **Playground, cloud devices, record & replay, PRD explorer**, and more
+- **Cloud devices, record & replay, PRD explorer**, and more
 
 Every mode is documented at **[appclaw.in](https://appclaw.in)**.
 
 ## Packages
 
-| Install                 | Package           | For                                                       |
-| ----------------------- | ----------------- | --------------------------------------------------------- |
-| `npm i -g @appclaw/cli` | `@appclaw/cli`    | the `appclaw` command — goals, flows, playground, reports |
-| via `appclaw init`      | `@appclaw/runner` | vitest-style test runner (`appclaw-runner`)               |
-| `npm i @appclaw/core`   | `@appclaw/core`   | the SDK / headless engine                                 |
+| Install                 | Package           | For                                                            |
+| ----------------------- | ----------------- | -------------------------------------------------------------- |
+| `npm i -g @appclaw/cli` | `@appclaw/cli`    | the `appclaw` command — goals, flows, Terminal Studio, reports |
+| via `appclaw init`      | `@appclaw/runner` | vitest-style test runner (`appclaw test`)                      |
+| `npm i @appclaw/core`   | `@appclaw/core`   | the SDK / headless engine                                      |
 
 ## Local development