Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions docs/features/cua-cross-platform-computer-use/plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Update `plugins/cua/plugin.json`:
- `plugin:runtime/linux/${arch}/cua-driver`
- Keep plugin-local runtime candidates first.
- Update packaged download URL conventions to include target platform and arch.
- Update tool policies for upstream v0.6.7 tools.
- Update tool policies for upstream v0.7.1 tools.
- Keep the internal tool server declaration owned by the plugin host; do not add user-facing MCP
setup instructions.

Expand All @@ -36,9 +36,9 @@ Update `plugins/cua/vendor/cua-driver/upstream.json` from the old Swift fork met
Rust driver release:

- `source`: upstream `trycua/cua`.
- `tag`: `cua-driver-rs-v0.6.7`.
- `commit`: `d6dea4bc3c3a65ce821261752067cae8200fe5d6`.
- `version`: `0.6.7`.
- `tag`: `cua-driver-rs-v0.7.1`.
- `commit`: `7caf72bee2286f47a985c3121b56aaabdebd62b9`.
- `version`: `0.7.1`.
- Include the expected asset map and checksums source.
- Record Windows arm64 as supported and Linux arm64 as unsupported for this pinned DeepChat
integration.
Expand Down Expand Up @@ -104,13 +104,13 @@ Update `.github/workflows/build.yml` and `.github/workflows/release.yml`:

### Skill Docs

Adapt CUA skill docs from upstream v0.6.7 into DeepChat-specific docs:
Adapt CUA skill docs from upstream v0.7.1 into DeepChat-specific docs:

- Remove upstream manual install, PATH, and standalone MCP setup requirements.
- Describe the DeepChat tool surface and platform behavior.
- Add platform caveats for macOS permissions, Windows foreground/background dispatch, and Linux
pre-release limitations.
- Replace Swift-era tool names with v0.6.7 tool names.
- Replace Swift-era tool names with v0.7.1 tool names.
- Keep plugin support metadata aligned with the supported platform/arch matrix.

### Settings and Permission UX
Expand All @@ -136,7 +136,7 @@ Update and add focused tests for:
- CUA manifest hydration and visibility for supported platform/arch targets.
- Runtime packaging validation per platform and arch.
- Unsupported Linux arm64 behavior.
- Tool policy coverage for upstream v0.6.7 known tools.
- Tool policy coverage for upstream v0.7.1 known tools.
- Skill docs no longer asserting macOS-only or user-managed MCP-only language.
- Build and release workflow assertions for CUA on Windows x64/arm64, macOS, and Linux x64.

Expand Down
14 changes: 7 additions & 7 deletions docs/features/cua-cross-platform-computer-use/spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@ The current plugin is macOS-only:
- Skill docs, runtime permission wording, tests, and packaging docs assume macOS.

Upstream `trycua/cua` now publishes the Rust CUA driver as cross-platform release artifacts. The
latest verified driver release for this plan is `cua-driver-rs-v0.6.7`, published on
2026-06-24. DeepChat support for this feature is limited to the targets that have upstream release
latest verified driver release for this plan is `cua-driver-rs-v0.7.1`, published on
2026-07-07. DeepChat support for this feature is limited to the targets that have upstream release
assets and have been validated for bundled plugin packaging:

- macOS arm64 and x86_64, plus universal variants.
Expand Down Expand Up @@ -101,9 +101,9 @@ DeepChat must continue to own the integration boundary:

Pin the CUA runtime to a specific upstream release:

- Tag: `cua-driver-rs-v0.6.7`.
- Commit: `d6dea4bc3c3a65ce821261752067cae8200fe5d6`.
- Version: `0.6.7`.
- Tag: `cua-driver-rs-v0.7.1`.
- Commit: `7caf72bee2286f47a985c3121b56aaabdebd62b9`.
- Version: `0.7.1`.

The build step must stage release artifacts instead of relying on local Swift-only source builds.
Every staged asset must be validated before packaging:
Expand Down Expand Up @@ -145,7 +145,7 @@ still consuming verified upstream release artifacts.

## Tool Surface

The plugin policy and skill docs must match upstream v0.6.7 tool names.
The plugin policy and skill docs must match upstream v0.7.1 tool names.

Removed or renamed assumptions:

Expand Down Expand Up @@ -224,7 +224,7 @@ The packaged app must keep CUA usable after Electron packaging:
- Optional MCP capabilities not implemented by the CUA driver, such as prompts and resources, are
treated as absent capabilities and must not produce error-level log spam.
- Skill docs describe DeepChat usage and platform caveats, not upstream manual installer workflows.
- Tool policies cover all upstream v0.6.7 tools known to this integration.
- Tool policies cover all upstream v0.7.1 tools known to this integration.
- Packaging docs and tests no longer describe CUA as macOS-only.
- Build, lint, i18n, and focused test suites pass after implementation.

Expand Down
8 changes: 4 additions & 4 deletions docs/features/cua-cross-platform-computer-use/tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
## Task List

- [x] T01 - Update CUA upstream metadata
- Replace old Swift fork metadata with pinned `cua-driver-rs-v0.6.7` metadata.
- Replace old Swift fork metadata with pinned `cua-driver-rs-v0.7.1` metadata.
- Record supported and unsupported platform/arch targets.
- Add expected upstream asset names and checksum source.

Expand All @@ -30,7 +30,7 @@

- [x] T05 - Update CUA tool policies
- Remove Swift-era `screenshot` and `set_recording` assumptions.
- Add policies for v0.6.7 read-only, action, recording, session, update, and platform-specific
- Add policies for v0.7.1 read-only, action, recording, session, update, and platform-specific
tools.
- Add a test that fails when a known upstream tool lacks an explicit policy.

Expand All @@ -54,10 +54,10 @@
- Keep official plugin verification failing on missing expected artifacts.

- [x] T09 - Update DeepChat skill docs
- Adapt upstream v0.6.7 skill guidance to DeepChat's bundled integration.
- Adapt upstream v0.7.1 skill guidance to DeepChat's bundled integration.
- Remove manual installer, PATH, and user-managed MCP setup language.
- Add macOS, Windows, and Linux platform caveats.
- Replace old tool names with v0.6.7 tool names.
- Replace old tool names with v0.7.1 tool names.

- [x] T10 - Update settings and permission status
- Keep macOS accessibility and screen-capture permission handling.
Expand Down
48 changes: 48 additions & 0 deletions docs/issues/cua-computer-use-skill-name/spec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# CUA Computer Use Skill Name

## Issue

The official CUA plugin contributes its built-in agent skill as `cua-driver` from
`skills/cua-driver/SKILL.md`. DeepChat skills use the folder-format contract
`<skill-name>/SKILL.md`, and discovery warns when `SKILL.md` frontmatter `name` differs from the
folder name. The user-facing capability is Computer Use; `cua-driver` is the runtime and MCP server
implementation detail.

GitHub issue: https://github.com/ThinkInAIXYZ/deepchat/issues/1900

## Impact

- Users see or activate a runtime implementation name instead of the Computer Use capability.
- The plugin skill is easier to confuse with the MCP server id.
- Future skill sync/export behavior expects the folder and frontmatter name to match.
- Existing sessions that persisted `cua-driver` as an active skill should keep Computer Use enabled
after the rename.

## Root Cause

The plugin skill was adapted from upstream CUA's `cua-driver` skill package without renaming the
DeepChat-owned skill contribution to the product capability name.

## Fix Plan

- Rename the plugin skill id and folder to `computer-use`.
- Rename `SKILL.md` frontmatter and heading to `computer-use`.
- Migrate persisted active skill names from `cua-driver` to `computer-use`.
- Keep runtime id, MCP server id, binary names, and upstream metadata as `cua-driver`.
- Update focused plugin tests.

## Tasks

- [x] Rename the CUA plugin skill contribution.
- [x] Migrate old active skill state to `computer-use`.
- [x] Update tests and references.
- [x] Run focused tests and required repository checks.

## Validation

- `corepack pnpm exec vitest run test/main/presenter/pluginPresenter.test.ts`
- `corepack pnpm exec vitest run test/main/presenter/skillPresenter/skillPresenter.test.ts`
- `corepack pnpm run plugin:bundle -- --name cua --platform darwin --arch arm64`
- `corepack pnpm run format`
- `corepack pnpm run i18n`
- `corepack pnpm run lint`
49 changes: 49 additions & 0 deletions docs/issues/cua-driver-0-7-1-update/spec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# CUA Driver 0.7.1 Update

## Issue

DeepChat pins the bundled CUA runtime to `cua-driver-rs v0.6.7`, while upstream now publishes
`cua-driver-rs v0.7.1`. Packaged users cannot update DeepChat's managed helper through their own
PATH-installed `cua-driver`, so the bundled runtime metadata, minimum version, and tool policy need
to track the latest verified upstream release.

GitHub issue: https://github.com/ThinkInAIXYZ/deepchat/issues/1898

## Impact

- Packaged CUA can report itself stale even though the user cannot update the bundled helper.
- New read-only desktop capture support in `v0.7.1` is unavailable unless the policy allows it.
- Obsolete tool names remain in the allowlist/asklist if the policy is not synced.

## Root Cause

The official CUA plugin consumes pinned GitHub release assets from
`plugins/cua/vendor/cua-driver/upstream.json`. The pin and conservative tool policy are still based
on `cua-driver-rs-v0.6.7`.

## Fix Plan

- Update the pinned upstream metadata to `cua-driver-rs-v0.7.1`.
- Keep the existing DeepChat target matrix unchanged.
- Update the plugin runtime `minVersion`.
- Sync CUA tool policy with the `cua-driver 0.7.1 list-tools` output.
- Update the maintained CUA cross-platform contract docs.
- Run focused CUA checks plus required format, i18n, and lint commands.

## Tasks

- [x] Update release metadata and plugin minimum version.
- [x] Sync CUA policy and tests with the v0.7.1 tool surface.
- [x] Update maintained SDD contract docs.
- [x] Validate plugin runtime staging for representative targets.
- [x] Run required repository checks.

## Validation

- `pnpm exec vitest run test/main/presenter/pluginPresenter.test.ts`
- `pnpm run plugin:cua:build:mac:arm64`
- `pnpm run plugin:cua:build:win:x64`
- `pnpm run plugin:cua:build:linux:x64`
- `pnpm run format`
- `pnpm run i18n`
- `pnpm run lint`
13 changes: 4 additions & 9 deletions plugins/cua/plugin.json
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@
"mode": "user-confirmed",
"provider": "trycua",
"strategy": "bundled-plugin-helper",
"minVersion": "0.6.7",
"minVersion": "0.7.1",
"guideUrl": "https://cua.ai/docs/cua-driver/guide/getting-started/installation"
}
},
Expand All @@ -59,8 +59,8 @@
],
"skills": [
{
"id": "cua-driver",
"path": "skills/cua-driver/SKILL.md",
"id": "computer-use",
"path": "skills/computer-use/SKILL.md",
"scope": "agent"
}
],
Expand All @@ -83,13 +83,13 @@
"get_screen_size": "allow",
"get_window_state": "allow",
"get_accessibility_tree": "allow",
"get_desktop_state": "allow",
"get_cursor_position": "allow",
"get_config": "allow",
"get_recording_state": "allow",
"get_agent_cursor_state": "allow",
"check_for_update": "allow",
"health_report": "allow",
"debug_window_info": "allow",
"start_session": "allow",
"end_session": "allow",
"launch_app": "ask",
Expand All @@ -99,14 +99,9 @@
"right_click": "ask",
"double_click": "ask",
"drag": "ask",
"mouse_button_down": "ask",
"mouse_button_up": "ask",
"mouse_drag": "ask",
"parallel_mouse_drag": "ask",
"scroll": "ask",
"move_cursor": "ask",
"type_text": "ask",
"type_text_chars": "ask",
"press_key": "ask",
"hotkey": "ask",
"set_value": "ask",
Expand Down
7 changes: 1 addition & 6 deletions plugins/cua/policies/tool-policy.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,13 @@
"get_screen_size": "allow",
"get_window_state": "allow",
"get_accessibility_tree": "allow",
"get_desktop_state": "allow",
"get_cursor_position": "allow",
"get_config": "allow",
"get_recording_state": "allow",
"get_agent_cursor_state": "allow",
"check_for_update": "allow",
"health_report": "allow",
"debug_window_info": "allow",
"start_session": "allow",
"end_session": "allow",
"launch_app": "ask",
Expand All @@ -23,14 +23,9 @@
"right_click": "ask",
"double_click": "ask",
"drag": "ask",
"mouse_button_down": "ask",
"mouse_button_up": "ask",
"mouse_drag": "ask",
"parallel_mouse_drag": "ask",
"scroll": "ask",
"move_cursor": "ask",
"type_text": "ask",
"type_text_chars": "ask",
"press_key": "ask",
"hotkey": "ask",
"set_value": "ask",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# CUA Computer Use Workflow
# Computer Use Workflow

This skill uses DeepChat's plugin-provided Computer Use tools.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
name: cua-driver
name: computer-use
description: Drive native desktop apps through DeepChat's built-in Computer Use tools. Use when the user asks to operate, inspect, automate, or perform a GUI task in a real desktop application.
platforms:
- darwin
Expand All @@ -9,7 +9,7 @@ metadata:
deepchatFeature: computer-use
---

# cua-driver
# computer-use

Use DeepChat's plugin-provided Computer Use tools as the only action surface for this skill. Do not
ask the user to install `cua-driver`, configure an external server, or put anything on PATH for the
Expand Down Expand Up @@ -70,12 +70,13 @@ Use this fallback order:
pixels.
3. Use `get_window_state` with vision capture for broad visual confirmation when window contents or
active overlays are unclear.
4. Use at most one `zoom({ pid, window_id, x1, y1, x2, y2 })` for small text or dense icons.
4. Use `get_desktop_state` only for desktop-scope workflows where there is no stable target window.
5. Use at most one `zoom({ pid, window_id, x1, y1, x2, y2 })` for small text or dense icons.
Repeated zoom calls are a failure signal; return to the full-window snapshot or ask for
clarification.
5. Use pixel coordinates from the latest same-window state with `click({ pid, window_id, x, y })`,
6. Use pixel coordinates from the latest same-window state with `click({ pid, window_id, x, y })`,
or from the single zoom image with `click({ pid, window_id, x, y, from_zoom: true })`.
6. Re-snapshot after each action and compare the resulting state.
7. Re-snapshot after each action and compare the resulting state.

Ask the user only when visible candidates are ambiguous, the requested action is destructive, or the
target is outside the current visible window.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ Use these checks after enabling the CUA plugin:
- `launch_app` starts a target app and returns a `pid` when the platform can provide one.
- `list_windows` returns windows for that `pid`.
- `get_window_state` returns a screenshot or accessibility tree for a selected `window_id`.
- `get_desktop_state` returns a full-display snapshot where desktop-scope capture is supported.
- `click` or `set_value` works after a same-window snapshot.
- `start_recording`, `stop_recording`, and `get_recording_state` are permission-gated.
- Plugin disable removes the `cua-driver` tools after the tool surface refreshes.
20 changes: 10 additions & 10 deletions plugins/cua/vendor/cua-driver/upstream.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,29 +2,29 @@
"sourceKind": "upstream-release",
"upstreamRepo": "https://github.com/trycua/cua.git",
"upstreamSubdir": "libs/cua-driver/rust",
"tag": "cua-driver-rs-v0.6.7",
"commit": "2cba1e769264a18f5a9d5f4e419729eb7fc17962",
"version": "0.6.7",
"updatedAt": "2026-06-24",
"releaseUrl": "https://github.com/trycua/cua/releases/tag/cua-driver-rs-v0.6.7",
"tag": "cua-driver-rs-v0.7.1",
"commit": "7caf72bee2286f47a985c3121b56aaabdebd62b9",
"version": "0.7.1",
"updatedAt": "2026-07-07",
"releaseUrl": "https://github.com/trycua/cua/releases/tag/cua-driver-rs-v0.7.1",
"checksumsAsset": "checksums.txt",
"supportedTargets": ["darwin/arm64", "darwin/x64", "win32/x64", "win32/arm64", "linux/x64"],
"unsupportedTargets": ["linux/arm64"],
"assets": {
"darwin-arm64": {
"name": "cua-driver-rs-0.6.7-darwin-arm64.tar.gz"
"name": "cua-driver-rs-0.7.1-darwin-arm64.tar.gz"
},
"darwin-x64": {
"name": "cua-driver-rs-0.6.7-darwin-x86_64.tar.gz"
"name": "cua-driver-rs-0.7.1-darwin-x86_64.tar.gz"
},
"windows-x64": {
"name": "cua-driver-rs-0.6.7-windows-x86_64.zip"
"name": "cua-driver-rs-0.7.1-windows-x86_64.zip"
},
"windows-arm64": {
"name": "cua-driver-rs-0.6.7-windows-arm64.zip"
"name": "cua-driver-rs-0.7.1-windows-arm64.zip"
},
"linux-x64": {
"name": "cua-driver-rs-0.6.7-linux-x86_64-binary.tar.gz"
"name": "cua-driver-rs-0.7.1-linux-x86_64-binary.tar.gz"
}
},
"releasePolicy": "Stage pinned upstream release assets at build time. Do not run upstream installers or require PATH-installed runtime binaries for the bundled DeepChat plugin."
Expand Down
Loading