feat: introduce Terminal Studio (TUI)

[Delta456]

This PR introduces a full pledged Terminal Studio (TUI) appclaw --tui which building a flow by hand: you type one instruction at a time, it runs on the device immediately, and every step that succeeds is recorded. When the flow looks right, /export turns it into a runnable spec. --playground is an alias for it.

Demo

https://jumpshare.com/s/YC7aCCBy57DHPjgj3ET0

NOTE: The link will expire after 24 hours

Why

The playground was a single-screen REPL: one prompt, no device picker, no way to see the device without alt-tabbing to the emulator window, and no record of what a session did. It had also grown to a 1,678-line file. Terminal Studio splits that into a screen router with its own store, and makes the recorded step list — the thing users actually want out of a session — a first-class object that can be listed, edited, exported and replayed.

The layout

Two columns. On the left, a scrollable transcript of everything that has run, the command palette, and the prompt. On the right, the device screen, mirrored live.

What's in it

Terminal Studio (packages/cli/src/tui/, 28 files)

• Screens: welcome → device picker → main, plus settings and history.
• A /-prefixed command palette. Anything not starting with / is one deterministic instruction, appended to the recorded step list; a failed step is reported but not recorded.
• /goal <text> is the explicit opt-in to the autonomous agent loop (a single flat runAgent(); the multi-sub-goal planner stays on the appclaw "goal" path).
• JSON session log per run at .appclaw/sessions/<id>.json, rewritten after every event so a crash still leaves it readable. Surfaced in /history.
• Device listing goes straight through adb / xcrun simctl (@appclaw/core/device/emulator-list.ts), so the picker works before any Appium session exists.

Command parity. All 15 playground commands are present — /clear /delete /edit /exit /export /help /insert /list /memory /meta /preview /q /quit /undo /yaml — plus 13 new: /clear-log /config /device /doctor /goal /history /platform /runs /session /settings /stream /stream-close /?.

Device stream (/stream, Android only for now). Renders the device inside the terminal, in the side panel, while the prompt stays live. Two backends, picked from env: kitty graphics (Ghostty, kitty, WezTerm) and a 24-bit ANSI half-block fallback. Frames are painted with direct process.stdout.writes rather than from React, because Ink rewrites its whole frame on every state change. Force with APPCLAW_STREAM_BACKEND=kitty|halfblock. adb -s <serial> is device-agnostic, so emulators, physical phones and headless emulators all take the same path.

Shared step-recording helpers move out of the deleted playground/ into packages/cli/src/step-recorder/, so Terminal Studio and the headless bridge share one implementation.

Testing

Six new test files cover the parts that broke repeatedly during development: dialog height arithmetic and wrapping, command palette rendering, main-screen layout at terminal sizes from 27 to 100 rows, tab completion (including the /stream vs /stream-close and /exit vs /export ambiguities), the stream backends, and the generated spec's shape.

[Copilot]

Pull request overview

This PR introduces Terminal Studio (appclaw --tui) as the new interactive, full-screen Ink-based step-recording experience (with --playground as an alias), adds an optional in-terminal Android device stream (/stream), and updates export/test generation and VS Code extension wiring to align with the new UX.

Changes:

• Add a multi-screen TUI (welcome → device picker → main, plus settings/history) with step recording, command palette, and session logging.
• Implement Android device mirroring inside the terminal via /stream, with kitty-graphics and ANSI half-block backends.
• Update docs, SDK export generation, CLI help/config defaults, and VS Code extension command labeling/launch args to match --tui.

Reviewed changes

Copilot reviewed 70 out of 72 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
vscode-extension/src/extension.ts	Updates the VS Code command to launch Terminal Studio (integrated terminal) using --tui.
vscode-extension/README.md	Renames the command in docs to “Start Terminal Studio”.
vscode-extension/package.json	Renames the command title to “Start Terminal Studio”.
tests/ui/tui-stream.test.ts	Adds tests for stream capture/encoding, kitty protocol sequences, placeholders, and layout math.
tests/ui/output-dialog.test.tsx	Adds regression tests for OutputDialog height/wrapping/scrolling behavior.
tests/ui/main-screen-layout.test.tsx	Adds whole-frame layout tests to prevent Ink row-overlap regressions.
tests/ui/ink-ui.test.tsx	Removes PlaygroundApp tests; adds FinalSummary and StreamPanel placeholder-width regressions.
tests/ui/complete-command.test.ts	Adds tab-completion tests for TUI slash commands.
tests/ui/command-palette.test.tsx	Adds CommandPalette height/row-budget and truncation tests.
tests/sdk/goal-export.test.ts	Adds tests asserting exports target @appclaw/runner spec shape (not vitest/SDK).
README.md	Documents Terminal Studio and /stream; updates “What it can do” and packages table.
packages/core/src/ui/terminal.ts	Updates interactive help banner to include --tui + --playground alias messaging.
packages/core/src/ui/step-printer.ts	Updates docs/comments to reflect step recorders (vs legacy playground) as shared consumers.
packages/core/src/sdk/types.ts	Updates SDK option docs to reference TUI (vs playground).
packages/core/src/sdk/step-runner.ts	Updates docs/comments to reference step recorders/TUI as the shared pipeline peer.
packages/core/src/sdk/index.ts	Updates docs/comments to reference TUI semantics for per-step execution.
packages/core/src/sdk/goal-export.ts	Switches generated output from vitest+SDK lifecycle to @appclaw/runner spec format.
packages/core/src/flow/vision-execute.ts	Updates docs/comments to reflect step recorders instead of playground.
packages/core/src/flow/run-yaml-flow.ts	Updates comments to reflect step recorder surface names.
packages/core/src/flow/run-instruction.ts	Updates comments to reflect SDK + step recorders sharing the pipeline.
packages/core/src/device/index.ts	Updates docs/comments for “always pick device” to reference step recorder surfaces.
packages/core/src/device/emulator-list.ts	Adds direct adb/simctl device listing for pre-session UI surfaces (TUI picker/doctor).
packages/core/src/config.ts	Changes default EXPORT_DIR to tests and updates rationale in schema docs.
packages/core/src/agent/loop.ts	Updates comments to reference step recorders for shared coordinate scaling behavior.
packages/core/src/agent/app-resolver.ts	Updates comments to reference step-recorder usage patterns.
packages/cli/src/ui/ink/PlaygroundApp.tsx	Removes legacy single-screen playground REPL UI.
packages/cli/src/ui/ink/playground-store.ts	Removes legacy playground store.
packages/cli/src/ui/ink/playground-runner.tsx	Removes legacy playground Ink launcher/renderer bridge.
packages/cli/src/ui/ink/components/PlaygroundBottomBar.tsx	Removes legacy playground bottom bar component.
packages/cli/src/tui/useLayout.ts	Adds shared layout helpers for row budgeting and windowing.
packages/cli/src/tui/TuiApp.tsx	Adds TUI router/screen switching and modal handling.
packages/cli/src/tui/stream/terminal-caps.ts	Adds env-based stream backend detection (kitty vs halfblock).
packages/cli/src/tui/stream/placeholder.ts	Implements kitty Unicode placeholder encoding helpers.
packages/cli/src/tui/stream/layout.ts	Centralizes geometry constants and layout calculations for the main screen + stream.
packages/cli/src/tui/stream/kitty.ts	Implements kitty graphics protocol escape sequence builders.
packages/cli/src/tui/stream/halfblock.ts	Implements ANSI half-block renderer from raw RGBA frames.
packages/cli/src/tui/stream/frame-loop.ts	Adds the capture→encode→present stream loop with backpressure and teardown.
packages/cli/src/tui/stream/capture.ts	Adds adb-based PNG/raw capture and raw screencap parsing.
packages/cli/src/tui/store.ts	Adds TUI observable store (routing, steps/meta, transcript, stream, settings, history, modals).
packages/cli/src/tui/session-log.ts	Adds crash-resilient JSON session log writing + listing utilities.
packages/cli/src/tui/screens/WelcomeScreen.tsx	Adds welcome/platform-picker screen.
packages/cli/src/tui/screens/SettingsScreen.tsx	Adds in-TUI settings editor (writes back to .env via actions).
packages/cli/src/tui/screens/MainScreen.tsx	Adds main two-column UI (transcript+palette left, stream panel right) and input handling.
packages/cli/src/tui/screens/HistoryScreen.tsx	Adds unified run/session history browser screen.
packages/cli/src/tui/screens/DevicePickerScreen.tsx	Adds device picker UI with refresh/doctor/error dismissal behaviors.
packages/cli/src/tui/input-history.ts	Adds session-scoped input history recall logic.
packages/cli/src/tui/highlight.ts	Adds minimal TS/YAML highlighter for OutputDialog previews.
packages/cli/src/tui/components/StreamPanel.tsx	Adds stream panel that renders status + either placeholders or half-block text frames.
packages/cli/src/tui/components/StatusBar.tsx	Adds fixed-height status/hints bar with constant row budgeting.
packages/cli/src/tui/components/ProgressDialog.tsx	Adds modal progress dialog for long-running setup operations.
packages/cli/src/tui/components/OutputDialog.tsx	Adds modal scrollable output viewer with wrapping + hanging indent + optional highlighting.
packages/cli/src/tui/components/Header.tsx	Adds shared “AppClaw” header component for TUI screens.
packages/cli/src/tui/components/ConfirmDialog.tsx	Adds blocking yes/no modal for quit-with-unexported-steps.
packages/cli/src/tui/components/CommandPalette.tsx	Adds command palette + input panel with fixed height budgeting and filtering.
packages/cli/src/tui/capture-console.ts	Adds console.log capture utilities for TUI rendering of legacy console output.
packages/cli/src/step-recorder/screen-info.ts	Adds shared Stark vision “getInfo” helper for step-recording surfaces.
packages/cli/src/step-recorder/memory-inspect.ts	Adds shared /memory command implementation for step-recording surfaces.
packages/cli/src/step-recorder/flow-builder.ts	Adds shared YAML/spec rendering + export path resolution for recorded steps.
packages/cli/src/index.ts	Adds --tui flag, routes --playground (interactive) to TUI, preserves --json --playground bridge.
packages/cli/src/cli/doctor.ts	Resets module-global tallies to make doctor re-entrant (needed for /doctor in TUI).
packages/cli/package.json	Adds wrap-ansi dependency for dialog wrapping.
package-lock.json	Updates lockfile for new dependency and metadata changes.
CLAUDE.md	Updates repo guidance to document Terminal Studio, step-recorder helpers, stream design, and export format.
.kiro/steering/structure.md	Updates architecture map to replace playground with step-recorder/ + tui/.
.kiro/steering/product.md	Updates product description to replace Playground with Terminal Studio.
.agents/skills/use-appclaw-cli/SKILL.md	Updates skill docs to reference --tui and clarify --json --playground bridge behavior.
.gitignore	Adds JetBrains .idea/ and *.iml ignores.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

🎉 This PR is included in version 2.5.0 🎉

The release is available on:

• v2.5.0
• GitHub release

Your semantic-release bot 📦🚀