From 73c27c1a39bfc0b203ec6e5696188df886378343 Mon Sep 17 00:00:00 2001 From: Noel Kwan Date: Fri, 24 Apr 2026 09:39:45 +0800 Subject: [PATCH 1/7] feat: browser-side screenshot + clipboard for web frontend MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Pass 1 of desktop-app removal (see docs/plans/2026-04-23-remove-desktop-app-design.md). Reimplements the two native-only features that blocked dropping the Tauri shell: - Screenshot: html2canvas renders the DOM; new /api/save_screenshot route decodes the base64 data URL and writes a PNG to the temp dir, keeping the existing file-path contract that downstream Claude Code CLI depends on. - Clipboard image: DOM drop handler + navigator.clipboard.write replaces the Tauri drag-drop event + copy_image_file_to_clipboard. Tauri branches are kept (gated on isTauri) so the desktop target still works through Pass 1; Pass 2 will delete them. Cropping is dropped per design decision — html2canvas captures the full DOM. Co-Authored-By: Claude Opus 4.7 (1M context) --- .../2026-04-23-remove-desktop-app-design.md | 87 +++++++++++++++++++ package.json | 1 + pnpm-lock.yaml | 39 +++++++++ src-tauri/src/bin/server.rs | 23 +++++ src/App.svelte | 14 ++- src/App.test.ts | 12 +-- src/lib/Terminal.svelte | 49 +++++++++-- src/lib/native.ts | 23 +++++ 8 files changed, 230 insertions(+), 18 deletions(-) create mode 100644 docs/plans/2026-04-23-remove-desktop-app-design.md create mode 100644 src/lib/native.ts diff --git a/docs/plans/2026-04-23-remove-desktop-app-design.md b/docs/plans/2026-04-23-remove-desktop-app-design.md new file mode 100644 index 0000000..4f7c302 --- /dev/null +++ b/docs/plans/2026-04-23-remove-desktop-app-design.md @@ -0,0 +1,87 @@ +# Remove Desktop App — Design + +**Goal:** Delete the Tauri desktop target. The axum-backed web frontend (commit 4a699ae, #543) is the sole deliverable. The `src-tauri/` directory goes away entirely, along with every `@tauri-apps/*` dep and every `isTauri` branch in `src/`. + +**Motivation:** Web frontend is Playwright-testable end-to-end; the Tauri target isn't. Parity is achieved for 57/59 commands — the two remaining (`capture_app_screenshot`, `copy_image_file_to_clipboard`) are reimplemented with browser APIs in this plan. + +## Definition + +Rip out the desktop shell; keep one frontend (`src/`) and one backend (the current `src-tauri/src/bin/server.rs` binary, relocated out of `src-tauri/`). Rewire the two native-only features so the web frontend can capture screenshots and push images onto the clipboard without Tauri. + +## Constraints + +- `src-tauri/src/bin/server.rs` and most of `src-tauri/src/*.rs` (auto_worker, maintainer, pty_manager, emitter, state, storage, worktree, etc.) are **shared library code**, not Tauri code. They must survive the deletion. Only the Tauri-specific layers go: `lib.rs`, `tauri.conf.json`, `build.rs`, `capabilities/`, `icons/`, `commands.rs` (the `#[tauri::command]` wrappers — the underlying functions in `commands/*.rs` stay), `main.rs` (Tauri entry), and all `tauri*` crate deps. +- Crate must be renamed from `the-controller` (the Tauri binary) to a server-only crate. New layout: move the whole `src-tauri/` contents up to a top-level `server/` — or simpler, **delete the `src-tauri/` wrapper and promote its contents**. Final layout: `server/Cargo.toml`, `server/src/lib.rs` (the current `the_controller_lib` root, minus Tauri), `server/src/bin/server.rs` → becomes `server/src/main.rs` so `cargo run` from `server/` just works. +- Browser reimplementations: + - **Screenshot** — `html2canvas` captures the DOM. Returns a PNG dataURL; the feedback widget consumes a path today but can take a Blob/dataURL just as easily. No OS-level window capture, no cropping mode — the `cropped: true` path is dropped (document in plan, confirm with user if needed). Alternative `getDisplayMedia` requires a user gesture + permission prompt every time; skip it. + - **Clipboard image** — `navigator.clipboard.write([new ClipboardItem({ 'image/png': blob })])`. The terminal drag-drop handler already has the file path; `fetch(fileUrl)` → `blob()` → write. Needs HTTPS or localhost (already the case for dev). +- Tests: the `@tauri-apps/*` mocks in four test files (`App.test.ts`, `AgentDashboard.test.ts`, `clipboard.test.ts`, `backend.test.ts`) go away; those tests switch to mocking `fetch` / `navigator.clipboard` / `window.open` directly. +- Playwright harness already boots axum + vite; no changes needed except dropping the chat-daemon gap tracked in the parity plan (out of scope here — keep working the same way it does now). +- `list_archived_projects` is in HTTP but not Tauri — already an HTTP-only route, no action needed on deletion. +- No backwards compat shims. No feature flags. Straight cut-over: one PR, one commit series, one merge. + +## Migration plan + +### Pass 1 — browser reimplementations (land first, verify in web mode) + +1. Add `html2canvas` to `package.json` (runtime dep, not dev). +2. Add `src/lib/native.ts` exporting two browser-only helpers: + - `captureScreenshot(cropped: boolean): Promise` — calls `html2canvas(document.body)`, returns the canvas as a PNG blob. If `cropped` is true, ignore it (log a one-time warn and fall through to full capture) — cropping is dropped. + - `copyImageBlobToClipboard(blob: Blob): Promise` — writes via `navigator.clipboard.write([new ClipboardItem({ [blob.type]: blob })])`. +3. Rewrite `src/App.svelte:231` (feedback widget) to use `captureScreenshot(cropped)` directly and feed a Blob/dataURL to the feedback submission flow instead of a filesystem path. Find downstream consumers of the path and switch them to Blob. +4. Rewrite `src/lib/Terminal.svelte:252` (drag-drop image paste) to `fetch` the dropped file URL (or read the `DataTransferItem` as a Blob), then call `copyImageBlobToClipboard`. +5. Update the four affected test files to mock browser APIs instead of Tauri plugins. + +**Exit:** `pnpm test` green. Manual: in a browser, drag an image onto a terminal, confirm it lands on the system clipboard; click the feedback widget screenshot button, confirm a PNG reaches the submission handler. + +### Pass 2 — strip `@tauri-apps/*` from the frontend + +1. `src/lib/backend.ts` — delete the Tauri branches. `command()` always `fetch`, `listen()` always WebSocket, `openUrl()` always `window.open(url, "_blank", "noopener")`. Remove the `isTauri` helper entirely (grep first to confirm no other call sites). +2. `src/App.svelte:292-302` (`updateWindowTitle`) — this sets the window title to `"The Controller (commit, branch, localhost:port)"`. The web branch already does the right thing via `document.title = title`. Keep the function and the `document.title` assignment; delete only the `if (isTauri)` branch (and the `@tauri-apps/api/window` dynamic import). No other window-control UI in the file — confirmed by `grep -n "getCurrentWindow\|@tauri-apps/api/window" src/App.svelte`. +3. `src/lib/clipboard.ts` — delete the Tauri branch; keep only the `navigator.clipboard.read()` path. +4. `package.json` — remove `@tauri-apps/api`, `@tauri-apps/plugin-clipboard-manager`, `@tauri-apps/plugin-opener`, `@tauri-apps/cli`. Remove `tauri` script. +5. `pnpm install` to refresh the lockfile. + +**Exit:** `grep -r "@tauri-apps" src/ package.json` returns nothing. `pnpm build` green. `pnpm test` green. + +### Pass 3 — delete the Rust Tauri layer + +1. Delete `src-tauri/src/lib.rs`, `src-tauri/src/main.rs`, `src-tauri/src/commands.rs` (the Tauri wrappers — the delegate functions in `src-tauri/src/commands/*.rs` stay). Verify `server.rs` doesn't reach into `commands.rs`; it calls the domain functions directly. +2. Delete `src-tauri/build.rs`, `src-tauri/tauri.conf.json`, `src-tauri/capabilities/`, `src-tauri/icons/`. +3. Delete the `media.rs` module entirely (screenshot + clipboard image were its only callers, both now browser-side). If any other file pulls `tauri::AppHandle`, sweep and delete — the remaining modules (`auto_worker`, `maintainer`, etc.) should already be Tauri-free or only lightly coupled via the emitter abstraction. +4. `Cargo.toml`: + - Remove `tauri`, `tauri-build`, `tauri-plugin-opener`, `tauri-plugin-clipboard-manager` deps. + - Remove the `[build-dependencies] tauri-build` entry. + - Remove `server` feature gate — axum becomes unconditional. + - Rename the crate from `the-controller` to `the-controller-server`. Rename `the_controller_lib` usage in `server.rs` to match. + - Promote `src/bin/server.rs` to `src/main.rs`; delete the `[[bin]]` section. +5. Move `src-tauri/` contents up: `git mv src-tauri/Cargo.toml server/Cargo.toml`, same for `src/`, `tests/`, `test-data/`. Delete the now-empty `src-tauri/`. +6. Update `Cargo.lock` via `cargo build` in `server/`. +7. `dev.sh`, `scripts/*`, `playwright.config.ts`, `CLAUDE.md`, `README.md`, `ARCHITECTURE.md`, `CONTRIBUTING.md`, `agents.md`, any `docs/domain-knowledge.md` references to Tauri — sweep and update. `pnpm tauri dev` is gone; `dev.sh` already boots vite + axum for the web path, that becomes the only dev command. +8. Remove the `tauri` script from `package.json` (already gone in Pass 2) and any `@tauri-apps/cli` references in `.gitignore` / build scripts. + +**Exit:** `grep -r "tauri" --include="*.rs" --include="*.toml" --include="*.ts" --include="*.svelte" --include="*.json" --include="*.md"` returns only historical references in `docs/plans/*` (fine — plans are historical). `cargo build` in `server/` green. `cargo test` green. `pnpm test` green. `pnpm build` green. `dev.sh` starts app, browser at `localhost:1420` works end-to-end. + +### Pass 4 — docs + CI + +1. Update `CLAUDE.md`: replace "Tauri v2 + Svelte 5 desktop app" framing with "axum + Svelte 5 web app". Remove `pnpm tauri dev` under Dev Commands. Update the domain-knowledge reference to note that the "Tauri main thread blocking" lesson is historical context for why `spawn_blocking` is used throughout, not an active constraint. +2. Update `ARCHITECTURE.md`, `README.md`, `CONTRIBUTING.md`. +3. CI: any job running `cargo tauri build` / invoking `@tauri-apps/cli` goes away. Verify GitHub Actions workflows in `.github/` (check during implementation). +4. Pre-commit hook: confirm it doesn't call tauri tooling. + +## Validation + +Per `CLAUDE.md` task structure, each pass has its own verification before moving on: + +- **Pass 1:** `pnpm test` green after the two reimplementations; manual browser check for feedback screenshot + terminal image paste. +- **Pass 2:** `grep -r "@tauri-apps" src/ package.json` empty; `pnpm build && pnpm test` green. +- **Pass 3:** `cargo build && cargo test` from `server/` green; `grep -rE "tauri|@tauri" --include="*.rs" --include="*.ts" --include="*.svelte" --include="*.toml" --include="*.json" src/ server/ package.json Cargo.toml` empty (docs excluded); `dev.sh` boots cleanly; Playwright smoke `pnpm playwright test` green. +- **Pass 4:** no stale `pnpm tauri dev` / `cargo tauri` references in checked-in docs or CI config. + +Revert-test the semantic changes: if Pass 1's browser reimplementations are reverted, the feedback screenshot test and the terminal clipboard test fail. If they apply cleanly, they pass. + +## Decisions (resolved 2026-04-24) + +1. **Cropped screenshots** — dropped. `html2canvas(document.body)` only; the `cropped` parameter becomes unused, callers stop passing it, the Tauri `screencapture -i` behavior is gone. Tests updated accordingly. +2. **macOS app bundle distribution** — dropped intentionally. `.dmg` / `.app` goes away; users run `dev.sh` or open the server URL. +3. **Window title** (`src/App.svelte:292-302`) — logic is already ported. Web path calls `document.title = title`; Tauri path calls `getCurrentWindow().setTitle(title)` with the same string. Cleanup keeps `document.title`, drops the `isTauri` branch and the `@tauri-apps/api/window` import. No other window APIs used in `App.svelte`. diff --git a/package.json b/package.json index 9815d18..4c8c65e 100644 --- a/package.json +++ b/package.json @@ -31,6 +31,7 @@ "@xterm/addon-fit": "^0.11.0", "@xterm/addon-web-links": "^0.12.0", "@xterm/xterm": "^6.0.0", + "html2canvas": "^1.4.1", "mermaid": "^11.13.0" }, "pnpm": { diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index c017627..c77f0c2 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -53,6 +53,9 @@ importers: '@xterm/xterm': specifier: ^6.0.0 version: 6.0.0 + html2canvas: + specifier: ^1.4.1 + version: 1.4.1 mermaid: specifier: ^11.13.0 version: 11.13.0 @@ -879,6 +882,10 @@ packages: resolution: {integrity: sha512-BLrgEcRTwX2o6gGxGOCNyMvGSp35YofuYzw9h1IMTRmKqttAZZVU67bdb9Pr2vUHA8+j3i2tJfjO6C6+4myGTA==} engines: {node: 18 || 20 || >=22} + base64-arraybuffer@1.0.2: + resolution: {integrity: sha512-I3yl4r9QB5ZRY3XuJVEPfc2XhZO6YweFPI+UovAzn+8/hb3oJ6lnysaFcjVpkCPfVWFUDvoZ8kmVDP7WyRtYtQ==} + engines: {node: '>= 0.6.0'} + brace-expansion@5.0.4: resolution: {integrity: sha512-h+DEnpVvxmfVefa4jFbCf5HdH5YMDXRsmKflpf1pILZWRFlTbJpxeU55nJl4Smt5HQaGzg1o6RHFPJaOqnmBDg==} engines: {node: 18 || 20 || >=22} @@ -923,6 +930,9 @@ packages: crelt@1.0.6: resolution: {integrity: sha512-VQ2MBenTq1fWZUH9DJNGti7kKv6EeAuYr3cLwxUWhIu1baTaXh4Ib5W2CqHVqib4/MqbYGJqiL3Zb8GJZr3l4g==} + css-line-break@2.1.0: + resolution: {integrity: sha512-FHcKFCZcAha3LwfVBhCQbW2nCNbkZXn7KVUJcsT5/P8YmfsVja0FMPJr0B903j/E69HUphKiV9iQArX8SDYA4w==} + css.escape@1.5.1: resolution: {integrity: sha512-YUifsXXuknHlUsmlgyY0PKzgPOr7/FjCePfHNt0jxm83wHZi44VDMQ7/fGNkjY3/jV1MC+1CmZbaHzugyeRtpg==} @@ -1184,6 +1194,10 @@ packages: resolution: {integrity: sha512-Y22oTqIU4uuPgEemfz7NDJz6OeKf12Lsu+QC+s3BVpda64lTiMYCyGwg5ki4vFxkMwQdeZDl2adZoqUgdFuTgQ==} engines: {node: '>=18'} + html2canvas@1.4.1: + resolution: {integrity: sha512-fPU6BHNpsyIhr8yyMpTLLxAbkaK8ArIBcmZIRiBLiDhjeqvXolaEmDGmELFuX9I4xDcaKKcJl+TKZLqruBbmWA==} + engines: {node: '>=8.0.0'} + http-proxy-agent@7.0.2: resolution: {integrity: sha512-T1gkAiYYDWYx3V5Bmyu7HcfcvL7mUrTWiM6yOfa3PIphViJ/gFPbvidQ+veqSOHci/PxBcDabeUNCzpOODJZig==} engines: {node: '>= 14'} @@ -1438,6 +1452,9 @@ packages: symbol-tree@3.2.4: resolution: {integrity: sha512-9QNk5KwDF+Bvz+PyObkmSYjI5ksVUYtjW7AU22r2NKcfLJcXp96hkDWU3+XndOsUb+AQ9QhfzfCT2O+CNWT5Tw==} + text-segmentation@1.0.3: + resolution: {integrity: sha512-iOiPUo/BGnZ6+54OsWxZidGCsdU8YbE4PSpdPinp7DeMtUJNJBoJ/ouUSTJjHkh1KntHaltHl/gDs2FC4i5+Nw==} + tinybench@2.9.0: resolution: {integrity: sha512-0+DUvqWMValLmha6lr4kD8iAMK1HzV0/aKnCtWb9v9641TnP/MFb7Pc2bxoxQjTXAErryXVgUOfv2YqNllqGeg==} @@ -1483,6 +1500,9 @@ packages: undici-types@7.18.2: resolution: {integrity: sha512-AsuCzffGHJybSaRrmr5eHr81mwJU3kjw6M+uprWvCXiNeN9SOGwQ3Jn8jb8m3Z6izVgknn1R0FTCEAP2QrLY/w==} + utrie@1.0.2: + resolution: {integrity: sha512-1MLa5ouZiOmQzUbjbu9VmjLzn1QLXBhwpUa7kdLUQK+KQ5KA9I1vk5U4YHe/X2Ch7PYnJfWuWT+VbuxbGwljhw==} + uuid@11.1.0: resolution: {integrity: sha512-0/A9rDy9P7cJ+8w1c9WD9V//9Wj15Ce2MPz8Ri6032usz+NfePxx5AcN3bN+r6ZL6jEo066/yNYB3tn4pQEx+A==} hasBin: true @@ -2355,6 +2375,8 @@ snapshots: balanced-match@4.0.4: {} + base64-arraybuffer@1.0.2: {} + brace-expansion@5.0.4: dependencies: balanced-match: 4.0.4 @@ -2397,6 +2419,10 @@ snapshots: crelt@1.0.6: {} + css-line-break@2.1.0: + dependencies: + utrie: 1.0.2 + css.escape@1.5.1: {} cssstyle@4.6.0: @@ -2686,6 +2712,11 @@ snapshots: dependencies: whatwg-encoding: 3.1.1 + html2canvas@1.4.1: + dependencies: + css-line-break: 2.1.0 + text-segmentation: 1.0.3 + http-proxy-agent@7.0.2: dependencies: agent-base: 7.1.4 @@ -2996,6 +3027,10 @@ snapshots: symbol-tree@3.2.4: {} + text-segmentation@1.0.3: + dependencies: + utrie: 1.0.2 + tinybench@2.9.0: {} tinyexec@1.0.2: {} @@ -3029,6 +3064,10 @@ snapshots: undici-types@7.18.2: {} + utrie@1.0.2: + dependencies: + base64-arraybuffer: 1.0.2 + uuid@11.1.0: {} vite@6.4.1(@types/node@25.4.0): diff --git a/src-tauri/src/bin/server.rs b/src-tauri/src/bin/server.rs index d3222fe..c16c2ff 100644 --- a/src-tauri/src/bin/server.rs +++ b/src-tauri/src/bin/server.rs @@ -107,6 +107,7 @@ async fn main() { .route("/api/scaffold_project", post(scaffold_project)) .route("/api/stage_session", post(stage_session)) .route("/api/unstage_session", post(unstage_session)) + .route("/api/save_screenshot", post(save_screenshot)) .route("/ws", get(ws_upgrade)) .fallback(fallback_handler) .layer(CorsLayer::permissive()) @@ -1315,6 +1316,28 @@ async fn unstage_session( Ok(Json(Value::Null)) } +async fn save_screenshot(Json(args): Json) -> Result, (StatusCode, String)> { + use base64::{engine::general_purpose, Engine as _}; + let data_url = args["dataUrl"] + .as_str() + .ok_or_else(|| (StatusCode::BAD_REQUEST, "dataUrl required".to_string()))?; + let b64 = data_url + .split_once(',') + .map(|(_, tail)| tail) + .ok_or_else(|| (StatusCode::BAD_REQUEST, "invalid data URL".to_string()))?; + let bytes = general_purpose::STANDARD + .decode(b64) + .map_err(|e| (StatusCode::BAD_REQUEST, format!("invalid base64: {e}")))?; + let path = std::env::temp_dir().join("the-controller-screenshot.png"); + std::fs::write(&path, bytes).map_err(|e| { + ( + StatusCode::INTERNAL_SERVER_ERROR, + format!("write failed: {e}"), + ) + })?; + Ok(Json(Value::String(path.to_string_lossy().to_string()))) +} + // --- WebSocket --- async fn ws_upgrade( diff --git a/src/App.svelte b/src/App.svelte index f1f7be1..3535a84 100644 --- a/src/App.svelte +++ b/src/App.svelte @@ -2,6 +2,7 @@ import { onMount } from "svelte"; import { fromStore } from "svelte/store"; import { command, isTauri, listen } from "$lib/backend"; + import { captureScreenshotPath } from "$lib/native"; import Sidebar from "./lib/Sidebar.svelte"; import TerminalManager from "./lib/TerminalManager.svelte"; import Onboarding from "./lib/Onboarding.svelte"; @@ -227,13 +228,20 @@ async function captureScreenshot(direct: boolean, cropped: boolean) { try { - showToast(cropped ? "Select area to capture..." : "Capturing screenshot...", "info"); - const screenshotPath: string = await command("capture_app_screenshot", { cropped }); + showToast( + cropped && isTauri ? "Select area to capture..." : "Capturing screenshot...", + "info", + ); + let screenshotPath: string; + if (isTauri) { + screenshotPath = await command("capture_app_screenshot", { cropped }); + } else { + screenshotPath = await captureScreenshotPath(); + } if (direct) { await createScreenshotSession(screenshotPath); } else { - // Show session picker screenshotPickerState = { path: screenshotPath, preview: false }; } } catch (e) { diff --git a/src/App.test.ts b/src/App.test.ts index c68e725..3d986cc 100644 --- a/src/App.test.ts +++ b/src/App.test.ts @@ -125,7 +125,7 @@ describe("App screenshot flow", () => { hotkeyAction.set({ type: "screenshot-to-session", direct: true }); await waitFor(() => { - expect(command).toHaveBeenCalledWith("capture_app_screenshot", { cropped: false }); + expect(command).toHaveBeenCalledWith("capture_app_screenshot", expect.any(Object)); }); // Should directly create session without showing picker @@ -140,13 +140,13 @@ describe("App screenshot flow", () => { expect(screen.queryByText("Send Screenshot To")).not.toBeInTheDocument(); }); - it("Cmd+D (direct): captures cropped screenshot and spawns session for the-controller", async () => { + it("Cmd+D (direct): captures screenshot and spawns session for the-controller", async () => { setupMocks(); render(App); hotkeyAction.set({ type: "screenshot-to-session", direct: true, cropped: true }); await waitFor(() => { - expect(command).toHaveBeenCalledWith("capture_app_screenshot", { cropped: true }); + expect(command).toHaveBeenCalledWith("capture_app_screenshot", expect.any(Object)); }); await waitFor(() => { @@ -166,7 +166,7 @@ describe("App screenshot flow", () => { hotkeyAction.set({ type: "screenshot-to-session" }); await waitFor(() => { - expect(command).toHaveBeenCalledWith("capture_app_screenshot", { cropped: false }); + expect(command).toHaveBeenCalledWith("capture_app_screenshot", expect.any(Object)); }); // Session picker modal should appear @@ -244,13 +244,13 @@ describe("App screenshot flow", () => { }); }); - it("Cmd+Shift+D (picker): captures cropped screenshot and shows picker", async () => { + it("Cmd+Shift+D (picker): captures screenshot and shows picker", async () => { setupMocks(); render(App); hotkeyAction.set({ type: "screenshot-to-session", cropped: true }); await waitFor(() => { - expect(command).toHaveBeenCalledWith("capture_app_screenshot", { cropped: true }); + expect(command).toHaveBeenCalledWith("capture_app_screenshot", expect.any(Object)); }); await waitFor(() => { diff --git a/src/lib/Terminal.svelte b/src/lib/Terminal.svelte index eb12372..e8641a7 100644 --- a/src/lib/Terminal.svelte +++ b/src/lib/Terminal.svelte @@ -4,7 +4,7 @@ import { Terminal } from "@xterm/xterm"; import { FitAddon } from "@xterm/addon-fit"; import { WebLinksAddon } from "@xterm/addon-web-links"; - import { command, listen, openUrl } from "$lib/backend"; + import { command, isTauri, listen, openUrl } from "$lib/backend"; import { refreshProjectsFromBackend } from "./project-listing"; import { makeCustomKeyHandler } from "./terminal-keys"; import { clipboardHasImage } from "./clipboard"; @@ -27,6 +27,7 @@ let unlistenOutput: (() => void) | undefined; let unlistenStatus: (() => void) | undefined; let unlistenDragDrop: (() => void) | undefined; + let unlistenDomDrop: (() => void) | undefined; // Gate: suppress onData forwarding during initialization to prevent // xterm.js auto-responses to terminal queries (DA, DSR) from being @@ -243,19 +244,48 @@ // Listen for drag-and-drop file events (from Finder). // Gate on active session — this is a window-level event so all // mounted Terminal instances receive it; only the active one should act. - unlistenDragDrop = listen<{ paths: string[] }>("tauri://drag-drop", async (payload) => { - if (get(activeSessionId) !== sessionId) return; - - const imagePath = payload.paths.find(isImageFile); - if (imagePath) { + if (isTauri) { + unlistenDragDrop = listen<{ paths: string[] }>("tauri://drag-drop", async (payload) => { + if (get(activeSessionId) !== sessionId) return; + + const imagePath = payload.paths.find(isImageFile); + if (imagePath) { + try { + await command("copy_image_file_to_clipboard", { path: imagePath }); + await writeToPty("\x1b[200~\x1b[201~"); + } catch (err) { + console.error("Failed to handle dropped image:", err); + } + } + }); + } else if (containerEl) { + const handleDragOver = (e: DragEvent) => { + e.preventDefault(); + if (e.dataTransfer) e.dataTransfer.dropEffect = "copy"; + }; + const handleDrop = async (e: DragEvent) => { + e.preventDefault(); + if (get(activeSessionId) !== sessionId) return; + + const file = Array.from(e.dataTransfer?.files ?? []).find((f) => + f.type.startsWith("image/"), + ); + if (!file) return; try { - await command("copy_image_file_to_clipboard", { path: imagePath }); + const { copyImageBlobToClipboard } = await import("$lib/native"); + await copyImageBlobToClipboard(file); await writeToPty("\x1b[200~\x1b[201~"); } catch (err) { console.error("Failed to handle dropped image:", err); } - } - }); + }; + containerEl.addEventListener("dragover", handleDragOver); + containerEl.addEventListener("drop", handleDrop); + unlistenDomDrop = () => { + containerEl?.removeEventListener("dragover", handleDragOver); + containerEl?.removeEventListener("drop", handleDrop); + }; + } // Handle resize resizeObserver = new ResizeObserver(() => { @@ -348,6 +378,7 @@ unlistenOutput?.(); unlistenStatus?.(); unlistenDragDrop?.(); + unlistenDomDrop?.(); resizeObserver?.disconnect(); mutationObserver?.disconnect(); term?.dispose(); diff --git a/src/lib/native.ts b/src/lib/native.ts new file mode 100644 index 0000000..e69c9eb --- /dev/null +++ b/src/lib/native.ts @@ -0,0 +1,23 @@ +import { command } from "./backend"; + +export async function captureScreenshotDataUrl(): Promise { + const { default: html2canvas } = await import("html2canvas"); + const canvas = await html2canvas(document.body, { + logging: false, + useCORS: true, + backgroundColor: null, + }); + return canvas.toDataURL("image/png"); +} + +export async function captureScreenshotPath(): Promise { + const dataUrl = await captureScreenshotDataUrl(); + return await command("save_screenshot", { dataUrl }); +} + +export async function copyImageBlobToClipboard(blob: Blob): Promise { + const type = blob.type || "image/png"; + await navigator.clipboard.write([ + new ClipboardItem({ [type]: blob }), + ]); +} From db179ead3d6148074ff1ab713f2f4a775f0da59d Mon Sep 17 00:00:00 2001 From: Noel Kwan Date: Fri, 24 Apr 2026 09:46:05 +0800 Subject: [PATCH 2/7] refactor: strip @tauri-apps/* from frontend MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Pass 2 of desktop-app removal. The web frontend no longer references Tauri at all: - backend.ts: drop isTauri, delete all Tauri branches in command, listen, listenAsync, openUrl — fetch/WebSocket only. - App.svelte: captureScreenshot always uses captureScreenshotPath; updateWindowTitle always uses document.title; no more window plugin. - Terminal.svelte: drop Tauri drag-drop listener; the DOM drop handler is now unconditional. isImageFile helper removed (callers gone). - clipboard.ts: clipboardHasImage now uses navigator.clipboard.read() instead of the Tauri clipboard plugin. - package.json: remove @tauri-apps/{api,plugin-clipboard-manager, plugin-opener,cli} and the tauri script; description updated. - Test harness: vitest-setup no longer mocks isTauri; test files updated to mock document.title and $lib/backend openUrl instead of Tauri plugin mocks. The src-tauri/ Rust crate is still buildable (Pass 3 deletes it). Co-Authored-By: Claude Opus 4.7 (1M context) --- package.json | 7 +- pnpm-lock.yaml | 149 --------------------------------- src/App.svelte | 25 ++---- src/App.test.ts | 33 ++++---- src/lib/AgentDashboard.test.ts | 7 +- src/lib/Terminal.svelte | 34 ++------ src/lib/backend.test.ts | 18 +--- src/lib/backend.ts | 31 ------- src/lib/clipboard.test.ts | 33 ++++++-- src/lib/clipboard.ts | 8 +- vitest-setup.ts | 1 - 11 files changed, 53 insertions(+), 293 deletions(-) diff --git a/package.json b/package.json index 4c8c65e..621d8fc 100644 --- a/package.json +++ b/package.json @@ -1,14 +1,13 @@ { "name": "the-controller", "version": "0.5.0", - "description": "A Tauri desktop app for managing multiple Claude Code sessions", + "description": "A web app for managing multiple Claude Code sessions", "type": "module", "scripts": { "dev": "vite", "build": "vite build", "preview": "vite preview", "check": "svelte-check --tsconfig ./tsconfig.json", - "tauri": "tauri", "test": "vitest run", "test:e2e": "npx playwright test --project=e2e", "demo": "npx playwright test --project=demo", @@ -25,9 +24,6 @@ "@fontsource/geist-mono": "^5.2.7", "@fontsource/geist-sans": "^5.2.5", "@replit/codemirror-vim": "^6.3.0", - "@tauri-apps/api": "^2", - "@tauri-apps/plugin-clipboard-manager": "^2.3.2", - "@tauri-apps/plugin-opener": "^2", "@xterm/addon-fit": "^0.11.0", "@xterm/addon-web-links": "^0.12.0", "@xterm/xterm": "^6.0.0", @@ -42,7 +38,6 @@ "devDependencies": { "@playwright/test": "^1.58.2", "@sveltejs/vite-plugin-svelte": "^5.0.0", - "@tauri-apps/cli": "^2", "@testing-library/jest-dom": "^6.9.1", "@testing-library/svelte": "^5.3.1", "@testing-library/user-event": "^14.6.1", diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index c77f0c2..cb728cb 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -35,15 +35,6 @@ importers: '@replit/codemirror-vim': specifier: ^6.3.0 version: 6.3.0(@codemirror/commands@6.10.2)(@codemirror/language@6.12.2)(@codemirror/search@6.6.0)(@codemirror/state@6.5.4)(@codemirror/view@6.39.17) - '@tauri-apps/api': - specifier: ^2 - version: 2.10.1 - '@tauri-apps/plugin-clipboard-manager': - specifier: ^2.3.2 - version: 2.3.2 - '@tauri-apps/plugin-opener': - specifier: ^2 - version: 2.5.3 '@xterm/addon-fit': specifier: ^0.11.0 version: 0.11.0 @@ -66,9 +57,6 @@ importers: '@sveltejs/vite-plugin-svelte': specifier: ^5.0.0 version: 5.1.1(svelte@5.53.6)(vite@6.4.1(@types/node@25.4.0)) - '@tauri-apps/cli': - specifier: ^2 - version: 2.10.0 '@testing-library/jest-dom': specifier: ^6.9.1 version: 6.9.1 @@ -578,86 +566,6 @@ packages: svelte: ^5.0.0 vite: ^6.0.0 - '@tauri-apps/api@2.10.1': - resolution: {integrity: sha512-hKL/jWf293UDSUN09rR69hrToyIXBb8CjGaWC7gfinvnQrBVvnLr08FeFi38gxtugAVyVcTa5/FD/Xnkb1siBw==} - - '@tauri-apps/cli-darwin-arm64@2.10.0': - resolution: {integrity: sha512-avqHD4HRjrMamE/7R/kzJPcAJnZs0IIS+1nkDP5b+TNBn3py7N2aIo9LIpy+VQq0AkN8G5dDpZtOOBkmWt/zjA==} - engines: {node: '>= 10'} - cpu: [arm64] - os: [darwin] - - '@tauri-apps/cli-darwin-x64@2.10.0': - resolution: {integrity: sha512-keDmlvJRStzVFjZTd0xYkBONLtgBC9eMTpmXnBXzsHuawV2q9PvDo2x6D5mhuoMVrJ9QWjgaPKBBCFks4dK71Q==} - engines: {node: '>= 10'} - cpu: [x64] - os: [darwin] - - '@tauri-apps/cli-linux-arm-gnueabihf@2.10.0': - resolution: {integrity: sha512-e5u0VfLZsMAC9iHaOEANumgl6lfnJx0Dtjkd8IJpysZ8jp0tJ6wrIkto2OzQgzcYyRCKgX72aKE0PFgZputA8g==} - engines: {node: '>= 10'} - cpu: [arm] - os: [linux] - - '@tauri-apps/cli-linux-arm64-gnu@2.10.0': - resolution: {integrity: sha512-YrYYk2dfmBs5m+OIMCrb+JH/oo+4FtlpcrTCgiFYc7vcs6m3QDd1TTyWu0u01ewsCtK2kOdluhr/zKku+KP7HA==} - engines: {node: '>= 10'} - cpu: [arm64] - os: [linux] - - '@tauri-apps/cli-linux-arm64-musl@2.10.0': - resolution: {integrity: sha512-GUoPdVJmrJRIXFfW3Rkt+eGK9ygOdyISACZfC/bCSfOnGt8kNdQIQr5WRH9QUaTVFIwxMlQyV3m+yXYP+xhSVA==} - engines: {node: '>= 10'} - cpu: [arm64] - os: [linux] - - '@tauri-apps/cli-linux-riscv64-gnu@2.10.0': - resolution: {integrity: sha512-JO7s3TlSxshwsoKNCDkyvsx5gw2QAs/Y2GbR5UE2d5kkU138ATKoPOtxn8G1fFT1aDW4LH0rYAAfBpGkDyJJnw==} - engines: {node: '>= 10'} - cpu: [riscv64] - os: [linux] - - '@tauri-apps/cli-linux-x64-gnu@2.10.0': - resolution: {integrity: sha512-Uvh4SUUp4A6DVRSMWjelww0GnZI3PlVy7VS+DRF5napKuIehVjGl9XD0uKoCoxwAQBLctvipyEK+pDXpJeoHng==} - engines: {node: '>= 10'} - cpu: [x64] - os: [linux] - - '@tauri-apps/cli-linux-x64-musl@2.10.0': - resolution: {integrity: sha512-AP0KRK6bJuTpQ8kMNWvhIpKUkQJfcPFeba7QshOQZjJ8wOS6emwTN4K5g/d3AbCMo0RRdnZWwu67MlmtJyxC1Q==} - engines: {node: '>= 10'} - cpu: [x64] - os: [linux] - - '@tauri-apps/cli-win32-arm64-msvc@2.10.0': - resolution: {integrity: sha512-97DXVU3dJystrq7W41IX+82JEorLNY+3+ECYxvXWqkq7DBN6FsA08x/EFGE8N/b0LTOui9X2dvpGGoeZKKV08g==} - engines: {node: '>= 10'} - cpu: [arm64] - os: [win32] - - '@tauri-apps/cli-win32-ia32-msvc@2.10.0': - resolution: {integrity: sha512-EHyQ1iwrWy1CwMalEm9z2a6L5isQ121pe7FcA2xe4VWMJp+GHSDDGvbTv/OPdkt2Lyr7DAZBpZHM6nvlHXEc4A==} - engines: {node: '>= 10'} - cpu: [ia32] - os: [win32] - - '@tauri-apps/cli-win32-x64-msvc@2.10.0': - resolution: {integrity: sha512-NTpyQxkpzGmU6ceWBTY2xRIEaS0ZLbVx1HE1zTA3TY/pV3+cPoPPOs+7YScr4IMzXMtOw7tLw5LEXo5oIG3qaQ==} - engines: {node: '>= 10'} - cpu: [x64] - os: [win32] - - '@tauri-apps/cli@2.10.0': - resolution: {integrity: sha512-ZwT0T+7bw4+DPCSWzmviwq5XbXlM0cNoleDKOYPFYqcZqeKY31KlpoMW/MOON/tOFBPgi31a2v3w9gliqwL2+Q==} - engines: {node: '>= 10'} - hasBin: true - - '@tauri-apps/plugin-clipboard-manager@2.3.2': - resolution: {integrity: sha512-CUlb5Hqi2oZbcZf4VUyUH53XWPPdtpw43EUpCza5HWZJwxEoDowFzNUDt1tRUXA8Uq+XPn17Ysfptip33sG4eQ==} - - '@tauri-apps/plugin-opener@2.5.3': - resolution: {integrity: sha512-CCcUltXMOfUEArbf3db3kCE7Ggy1ExBEBl51Ko2ODJ6GDYHRp1nSNlQm5uNCFY5k7/ufaK5Ib3Du/Zir19IYQQ==} - '@testing-library/dom@10.4.1': resolution: {integrity: sha512-o4PXJQidqJl82ckFaXUeoAW+XysPLauYI43Abki5hABd853iMhitooc6znOnczgbTYmEP6U6/y1ZyKAIsvMKGg==} engines: {node: '>=18'} @@ -2075,63 +1983,6 @@ snapshots: transitivePeerDependencies: - supports-color - '@tauri-apps/api@2.10.1': {} - - '@tauri-apps/cli-darwin-arm64@2.10.0': - optional: true - - '@tauri-apps/cli-darwin-x64@2.10.0': - optional: true - - '@tauri-apps/cli-linux-arm-gnueabihf@2.10.0': - optional: true - - '@tauri-apps/cli-linux-arm64-gnu@2.10.0': - optional: true - - '@tauri-apps/cli-linux-arm64-musl@2.10.0': - optional: true - - '@tauri-apps/cli-linux-riscv64-gnu@2.10.0': - optional: true - - '@tauri-apps/cli-linux-x64-gnu@2.10.0': - optional: true - - '@tauri-apps/cli-linux-x64-musl@2.10.0': - optional: true - - '@tauri-apps/cli-win32-arm64-msvc@2.10.0': - optional: true - - '@tauri-apps/cli-win32-ia32-msvc@2.10.0': - optional: true - - '@tauri-apps/cli-win32-x64-msvc@2.10.0': - optional: true - - '@tauri-apps/cli@2.10.0': - optionalDependencies: - '@tauri-apps/cli-darwin-arm64': 2.10.0 - '@tauri-apps/cli-darwin-x64': 2.10.0 - '@tauri-apps/cli-linux-arm-gnueabihf': 2.10.0 - '@tauri-apps/cli-linux-arm64-gnu': 2.10.0 - '@tauri-apps/cli-linux-arm64-musl': 2.10.0 - '@tauri-apps/cli-linux-riscv64-gnu': 2.10.0 - '@tauri-apps/cli-linux-x64-gnu': 2.10.0 - '@tauri-apps/cli-linux-x64-musl': 2.10.0 - '@tauri-apps/cli-win32-arm64-msvc': 2.10.0 - '@tauri-apps/cli-win32-ia32-msvc': 2.10.0 - '@tauri-apps/cli-win32-x64-msvc': 2.10.0 - - '@tauri-apps/plugin-clipboard-manager@2.3.2': - dependencies: - '@tauri-apps/api': 2.10.1 - - '@tauri-apps/plugin-opener@2.5.3': - dependencies: - '@tauri-apps/api': 2.10.1 - '@testing-library/dom@10.4.1': dependencies: '@babel/code-frame': 7.29.0 diff --git a/src/App.svelte b/src/App.svelte index 3535a84..f560974 100644 --- a/src/App.svelte +++ b/src/App.svelte @@ -1,7 +1,7 @@