Skip to content

feat: TaskYou Desktop — Tauri GUI with real executor terminal#583

Merged
bborn merged 15 commits into
mainfrom
feat/tauri-desktop-gui
Jun 11, 2026
Merged

feat: TaskYou Desktop — Tauri GUI with real executor terminal#583
bborn merged 15 commits into
mainfrom
feat/tauri-desktop-gui

Conversation

@bborn

@bborn bborn commented Jun 10, 2026

Copy link
Copy Markdown
Owner

TaskYou Desktop — native GUI port (Tauri 2) with a real executor terminal

Ports the TUI feature set into a desktop GUI under desktop/, built on Tauri 2 + React + xterm.js, reusing the existing daemon/CLI/HTTP stack as the single source of truth.

The terminal (the hard requirement)

The executor pane is a real terminal, not a capture-pane mirror:

  • xterm.js (VS Code's emulator) in the webview, backed by a real PTY on the Rust side (portable-pty, wezterm's PTY layer), streaming over Tauri Channels.
  • The PTY runs a tmux client attached to a grouped view session targeting the daemon session, focused on the task's task-{id} window. Grouped sessions share windows but keep independent focus, so the GUI, the TUI, and the daemon coexist without moving panes (join-pane stays TUI-only). The view session is destroyed on detach; the daemon window survives.
  • Fully interactive: keystrokes, mouse (tmux mouse on), and resize all flow through. You see the executor pane and shell pane exactly as the daemon laid them out.
  • desktop/src-tauri/tests/terminal_attach.rs proves the path end-to-end against a scratch tmux server: window content streams to the PTY, typed input echoes back through the executor window, resize works, teardown kills the view session and leaves the daemon session intact.

Go API extensions (internal/web)

  • Attachments: list/upload/download/delete; GET/PATCH /api/settings (credential-like keys excluded); GET /api/executors; POST /api/autocomplete (ghost text); GET /api/tasks/latest-logs (board sub-lines); PATCH /api/tasks/{id} now accepts permission_mode/effort_level; task JSON exposes port/worktree/pane/window/session fields.
  • GET /api/tasks/{id}/terminal-info + POST /api/tasks/{id}/session: interactive-session bootstrap, extracted from the TUI detail view into executor.EnsureTaskWindow so TUI and API share one implementation (no more command-builder divergence). ty serve constructs an executor to power it.

Upstream bugs found & fixed along the way

  • /api/board/stream never fired: it polls event_log, but nothing ever wrote that table. Task mutations now record events in the db layer, so SSE live updates work across processes (also un-deadens /api/events).
  • API timestamps were local time with a Z suffix — now real UTC (RFC3339).

GUI feature set

Kanban board (live SSE updates, keyboard parity with the TUI, filter bar with #id/[project]/text, column collapse, command palette, toasts + native notifications) · task detail (markdown, live execution log, dependencies with add/remove, attachments with drag-drop, PR/editor/branch links, executor switcher, permission badges) · new/edit forms (ghost-text autocomplete, attachments, executor/effort/permission, execute-immediately) · retry-with-feedback, status, confirm dialogs, help overlay · project + task-type CRUD, connection settings · sidecar supervisor (discovers ty, health-checks the API, spawns ty serve/ty daemon when missing, kills only what it spawned).

Verification

  • Go: go vet, full -race suite, golangci-lint v2.8.0 (CI's pin) — 0 issues
  • Rust: 9/9 tests incl. the real-tmux integration test; clippy + fmt clean
  • TS: strict tsc + production build
  • Manual end-to-end on an isolated stack (own WORKTREE_DB_PATH + scratch TMUX_TMPDIR): board live-updates via SSE, task created through the form, palette/filter/keyboard flows, and the desktop app attaching its terminal to (1) a synthetic executor window with typed input echoing back through tmux, and (2) a real claude session bootstrapped via POST /api/tasks/{id}/session — Claude Code's UI rendering live in the GUI terminal next to the shell pane. Screenshot evidence captured locally (/tmp/ty-tauri-qa-shots/); publishing to the QA bucket needs your sign-off.

Notes / known limits

  • tauri dev / pnpm tauri build from desktop/ (pnpm + rustup toolchain). Bundle target: macOS .app.
  • Webview hard-reloads (dev HMR) can orphan a grouped view session until app exit's kill_all sweep — harmless, dev-only.
  • Related-tasks (QMD) and macOS Spotlight export aren't in the GUI yet; everything else from the TUI inventory is.

🤖 Generated with Claude Code

bborn and others added 3 commits June 10, 2026 17:34
…plete, session bootstrap

Extends the HTTP API with everything a desktop GUI client needs:
- GET/PATCH /api/settings (secret-like keys excluded)
- attachment list/upload/download/delete endpoints
- GET /api/executors (availability + default)
- POST /api/autocomplete (ghost-text suggestions)
- GET /api/tasks/{id}/terminal-info + POST /api/tasks/{id}/session
- GET /api/tasks/latest-logs for board activity sub-lines
- PATCH /api/tasks/{id} accepts permission_mode and effort_level
- task JSON exposes port, worktree path, pane/window/session ids

The interactive session bootstrap is extracted from the TUI detail view
into executor.EnsureTaskWindow so the TUI and HTTP API share one
implementation; ty serve now constructs an executor to power it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Native desktop port of the TUI built on Tauri 2 + React + xterm.js:

- Kanban board with live SSE updates, keyboard parity, filtering,
  command palette, toasts + native notifications
- Task detail with markdown, live execution log, dependencies,
  attachments, and a REAL terminal: a Rust PTY (portable-pty) running a
  tmux client attached to the task's daemon window via a grouped view
  session (destroy-unattached) — no pane moves, daemon survives, fully
  interactive (keys, mouse, resize)
- New/edit task forms with ghost-text autocomplete, drag-drop
  attachments, executor/effort/permission controls
- Project + task type CRUD, app connection settings
- Sidecar supervisor: discovers the ty binary, health-checks the API,
  spawns ty serve / ty daemon when absent, cleans up owned children

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
QA fixes from end-to-end testing:
- record task events in event_log on every mutation — the SSE board
  stream polls this table but nothing ever wrote it, so /api/board/stream
  (and /api/events) never fired; live updates now work across processes
- API timestamps were local time with a Z suffix; format real UTC
- prevent shortcut keys (n/p/f/e/r/S) from leaking into the inputs they
  open; new-task form defaults to the first task type from the API
- set destroy-unattached during tmux attach (not before) to avoid the
  view session being GC'd pre-attach; add a real-tmux integration test
  covering attach → output stream → typed input echo → clean teardown

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bborn

bborn commented Jun 10, 2026

Copy link
Copy Markdown
Owner Author

QA evidence

Kanban board (live SSE updates — the Done card moved columns with no reload):

qa-board

Task detail — markdown body, dependencies, attachments, execution log:

qa-detail

The real terminal in the desktop app — PTY tmux client attached to the task window (executor pane + shell pane). The typed-live-in-the-gui lines are keystrokes typed in the GUI terminal, echoed back through the tmux pane (independently verified via capture-pane):

qa-desktop-terminal

Real claude executor running live in the GUI — session bootstrapped via POST /api/tasks/1/session, Claude Code rendering interactively in the executor pane next to the shell pane:

qa-desktop-claude

All runs on an isolated stack (dedicated WORKTREE_DB_PATH + scratch TMUX_TMPDIR tmux server) — the live daemon was untouched.

🤖 Generated with Claude Code

… drag-and-drop board

Addresses 'feels like a web app crammed into a shell' feedback:

- Styling migrated from hand-rolled CSS to Tailwind v4 + shadcn/ui
  (button/dialog/select/command/sonner/etc); palette is now cmdk,
  toasts are sonner with an Open action
- Native shell: overlay titlebar with traffic-light inset + drag
  region, window vibrancy (UnderWindowBackground) behind a translucent
  theme, forced dark appearance, real menu bar (App/File/Edit/View/
  Window) — standard Edit roles make ⌘C/⌘V/⌘Z work; New Task ⌘N,
  Settings ⌘,, Search ⌘P, Board ⌘1 route to the frontend
- Arrow cursors on controls, no overscroll, system font stack
- Kanban cards are draggable between columns (Agentation feedback):
  drop on In Progress executes, Done closes, Backlog/Blocked set status
- Agentation feedback toolbar in dev builds, syncing to agentation-mcp
  on :4747; app icon generated from the TaskYou logo

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bborn

bborn commented Jun 10, 2026

Copy link
Copy Markdown
Owner Author

UI overhaul — native feel + shadcn/ui

Follow-up addressing the "web app crammed into a shell" feedback (and both Agentation annotations, now resolved):

  • Tailwind v4 + shadcn/ui replace the hand-rolled CSS (cmdk palette, sonner toasts, Radix dialogs/selects)
  • Native shell: overlay titlebar with traffic-light inset + drag region (fixes the window-control overlap annotation), window vibrancy, forced dark appearance, real macOS menu bar with working ⌘C/⌘V/⌘Z and New Task ⌘N / Search ⌘P / Settings ⌘,
  • Drag-and-drop kanban (the other annotation): drop on In Progress to execute, Done to close
  • App icon from the TaskYou logo; Agentation feedback toolbar ships in dev builds, syncing to agentation-mcp

qa-native-shadcn-board

🤖 Generated with Claude Code

…, modal fixes

Design pass from Agentation feedback + 'more modern and delightful':

- Full light + dark mode: system-following with a titlebar toggle
  (system → light → dark), themed window chrome, light/dark xterm
  palettes that switch live, themed toasts
- Opaque dialog/popover surfaces (annotation: 'transparent modal,
  super hard to read') — vibrancy translucency now applies only to the
  app shell, never to overlays
- Settings project/type editors open in proper modals (annotation)
- Motion: FLIP layout animations as cards move between kanban columns,
  enter/exit transitions, animated view switches — all 140-220ms ease-out
- Optimistic status updates: drag/execute/close/archive reflect
  instantly, server refresh reconciles; task cards memoized
- React 19 (latest shadcn targets ref-as-prop; fixes broken title
  autofocus + ghost-text ref on React 18), Radix-correct dialog
  autofocus via onOpenAutoFocus
- Semantic surface tokens replace hard-coded white/[…] overlays so both
  themes render correctly

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bborn

bborn commented Jun 10, 2026

Copy link
Copy Markdown
Owner Author

Design pass: light/dark mode, motion, optimistic board

Both new Agentation annotations resolved (opaque modals; settings editors in dialogs), plus:

  • Full light + dark mode — follows the system, titlebar toggle, themed terminal palettes and toasts
  • Motion — FLIP animations as cards move between columns, animated view transitions, enter/exit on cards
  • Optimistic updates — drag/execute/close reflect instantly, server reconciles
  • React 19 + Radix-correct dialog autofocus

Light (system-following):
qa-light-mode

Dark:
qa-dark-mode

Opaque modal (was translucent and unreadable):
qa-opaque-modal

⚠️ Note: the GUI needs the ty binary from this branch (make install + ty daemon restart) — older installed binaries lack the new endpoints (terminal-info 404) and don't write the live-update event feed.

🤖 Generated with Claude Code

bborn and others added 9 commits June 10, 2026 20:31
Agentation feedback: 'this filter thing should work just like the TUI
version'. The filter bar now mirrors the TUI exactly:

- typing an unclosed [ opens a fuzzy project dropdown (colors + task
  counts); ↑/↓ navigate, Tab/Enter insert the [project] chip
- Enter applies the filter and returns keyboard control to the board;
  the bar stays visible while a filter is active
- Esc clears (also from board level), Backspace on empty closes,
  Backspace after ] deletes the whole [project] chip

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Per-column render cap (50) with a '+N more' expander — the Done
  column alone was rendering 663 animated cards (824 total); the board
  now renders at most ~130, and keyboard selection auto-expands past
  the cap (TUI hidden-done parity)
- Field-level memo equality on cards: API refreshes return fresh
  objects, so reference-based memo re-rendered the whole board on every
  SSE tick; cards now re-render only when their content actually changes
- Narrow store subscriptions (useAppSelector) so selection changes
  don't fan out through the whole tree
- View switches swap immediately with a 90ms fade-in — the exit-then-
  enter transition read as sluggish; dialog/select animations at 120ms
- Card hover transitions limited to paint-only properties

Measured: ~15ms per keyboard selection move (single frame) on a
300-visible-task board in dev mode.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
'Waiting for executor to start' showed even though the executor was
running. Cause: the TUI's detail view join-panes the executor pane into
its own session, destroying the daemon window — the GUI's window-name
scan found nothing and concluded the executor never started.

- terminal-info now falls back to locating the stored pane ID anywhere
  on the tmux server (pane IDs survive join-pane moves) and reports
  pane_borrowed_by when it lives outside a daemon window
- the GUI shows the truth ('attached to the TUI, will appear here when
  released') and auto-attaches once the TUI gives the pane back
- ensure-session returns 409 while borrowed instead of spawning a
  duplicate executor session

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
In-app toasts cover the focused case; system notifications now only
fire when the user is in another app (native etiquette).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Conflicts: internal/web/server.go (union of Sessions/autocomplete and
browser-relay fields) and internal/web/handlers.go (both sides added
port/worktree_path/has_executor to task JSON with matching names; kept
the GUI's additional pane/session fields).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… and desktop

ty-web is superseded: ty serve now hosts the same React app the Tauri
shell renders, embedded into the binary behind a 'ui' build tag.

- internal/web/ui embeds desktop/dist (make build-ui stages it; the
  Makefile adds -tags ui automatically when present); plain go build
  stays Node-free and serves a pointer page at /
- SPA fallback for client routes; /api/* keeps precedence
- API base auto-detects same-origin when served by ty itself
- Browser terminal fallback: without a Tauri PTY, the terminal pane
  attaches xterm.js to the existing /api/tasks/{id}/terminal WebSocket
  (capture-pane mirror + send-keys input). It addresses panes by ID, so
  it works even while the TUI has the pane borrowed

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…-gui

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… README downloads

The desktop app no longer requires a ty install: a UI-embedded ty is
bundled as a Tauri sidecar (externalBin) and the supervisor prefers it
(config override → sidecar → PATH → known paths). Download → open →
works; tmux and an executor CLI remain machine prerequisites.

- make desktop-bundle stages the sidecar for the host triple and builds
  the platform bundles
- .github/workflows/desktop.yml builds macOS (arm64 + x64 dmg) and
  Linux (AppImage + deb) on v* tags and attaches them to the GitHub
  release next to the goreleaser binaries (workflow_dispatch uploads
  artifacts for testing); no Windows — execution runs on tmux
- transparent window + vibrancy moved to tauri.macos.conf.json so Linux
  gets an opaque window; html keeps an opaque themed background except
  in the macOS shell
- README 'Desktop App' section: where to download, prerequisites,
  unsigned-app first-launch note

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…aunches

Two fixes that make the downloaded bundle actually work on a clean Mac:

- Finder-launched apps get a minimal PATH (no /opt/homebrew/bin, no user
  bins), so the bundled ty couldn't find tmux or executors even when
  installed. The app now loads the login shell's PATH at startup before
  anything resolves binaries.
- First-run gate: before booting, the app probes for tmux and the known
  executor CLIs (claude, codex, gemini, opencode, pi, openclaw). If
  either is missing it shows a setup checklist with copyable install
  commands, Re-check, and Continue anyway — instead of failing
  mysteriously at execution time.

Also: make desktop-sidecar target (cargo check/test in src-tauri need a
staged sidecar since externalBin landed).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bborn bborn merged commit e416558 into main Jun 11, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant