Skip to content

fix(computer-use): win32 input actions fail truthfully; targeted mouse-down presses - #5910

Closed
EvanProgramming wants to merge 1 commit into
Hmbown:mainfrom
EvanProgramming:fix/win32-cu-false-success
Closed

fix(computer-use): win32 input actions fail truthfully; targeted mouse-down presses#5910
EvanProgramming wants to merge 1 commit into
Hmbown:mainfrom
EvanProgramming:fix/win32-cu-false-success

Conversation

@EvanProgramming

Copy link
Copy Markdown
Contributor

Summary

Fixes #5908 (relates to #5896). The Windows computer-use backend reported {action_sent: true} even when the underlying PowerShell subprocess never delivered the input.

Three root causes, all in crates/tui/plugins/computer-use/src/backends/win32.mjs:

  1. User32 type never loaded in the action process. create() only ran Add-Type in a bootstrap process, but every ps() spawns a fresh powershell.exe, so [User32]::mouse_event/SendInput/GetCursorPos were undefined there and the call failed silently.
  2. withUser32 ignored the exit code. It just returned ps(...), so a nonzero exit still produced {action_sent: true}.
  3. left_mouse_down precedence bug. target ? A : "" + B put the mouse_event(LEFTDOWN) in the no-target branch, so a targeted call only sent SetCursorPos.

Changes

  • USER32_DEF prepends the User32 P/Invoke definition to every User32-backed action so each is self-contained in its own process.
  • withUser32 throws an ExecError on a nonzero exit — no more false success.
  • left_mouse_down rewritten so a targeted call both moves the cursor and presses.
  • cursor_position now routes through withUser32 (loads the type, fails truthfully instead of lying about a position).
  • create({ exec }) accepts an injectable runner so the backend is testable on any host without powershell.exe (mirrors the harmonyos backend's pattern).

Tests

New crates/tui/plugins/computer-use/tests/win32.test.mjs (runs on any host via a mocked runner):

  • User32 actions define the type in their own process.
  • A nonzero subprocess exit rejects the action (no false action_sent: true).
  • left_mouse_down({ target }) sends both SetCursorPos and LEFTDOWN.
  • cursor_position loads the type in-process and never reports a fake position.

Verified: node --test tests/win32.test.mjs → 4/4 pass; node --test tests/backends.test.mjs → 10/10 pass (no regression).

… mouse-down presses

The win32 backend returned {action_sent: true} even when the PowerShell
subprocess never delivered the input:

- User32 P/Invoke type was only loaded in the bootstrap process, but every
  action spawns a fresh powershell.exe, so [User32]:: calls were undefined
  there and failed silently.
- withUser32 ignored the subprocess exit code, so failures still reported
  success.
- left_mouse_down({target}) dropped the LEFTDOWN press due to a
  ternary/concatenation precedence bug (only SetCursorPos was sent).
- cursor_position also referenced User32 without loading it.

Fix: define the User32 type in every action's process (USER32_DEF), make
withUser32 throw on a nonzero exit, and rewrite left_mouse_down so a
targeted call both moves and presses. create() now accepts an injectable
runner so the backend is testable without powershell.exe; regression tests
cover type-in-process, the failure path, and move+press.

Closes Hmbown#5908. Relates to Hmbown#5896.
@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Thanks @EvanProgramming for taking the time to contribute.

This repository is observing a maintainer-managed PR intake gate in dry-run mode, so this pull request is staying open. This note helps maintainers prepare the allowlist before any enforcement is considered.

Please read CONTRIBUTING.md for the expected contribution shape. A maintainer can grant recurring PR access by commenting /lgtm on a pull request.

@Hmbown

Hmbown commented Sep 6, 2026

Copy link
Copy Markdown
Owner

Closing with credit — this bug (#5896) was also fixed independently and has just merged as #5903 (self-contained Add-Type prelude per action, psOk/psJson throwing on nonzero exit/spawn failure, targeted mouse-down always pressing after the optional move). Your take arrived while ours was in CI, so rather than let it sit: thank you — and specifically, your instance-local injectable runner is a cleaner test seam than the fake-powershell.exe-on-PATH approach we merged, and we may adopt that pattern for future backend tests. If you'd like to rebase your DI seam on top of main as a follow-up refactor PR, it would be welcome. Also happy to review your other PR (#5911) — that one's untouched territory.

@Hmbown Hmbown closed this Sep 6, 2026
ysm827 pushed a commit to ysm827/DeepSeek-TUI that referenced this pull request Sep 6, 2026
… exec })

Follow-up to the closed Hmbown#5910, per the maintainer's invitation after the
bug itself was fixed and merged in Hmbown#5903. This PR is refactor-only: it
keeps main's merged behavior and adds a cleaner test seam.

Previously the win32 backend's PowerShell runner (ps/psOk/psJson) lived at
module scope, so backend tests had to fake powershell.exe on PATH (or run
on Windows). Move the runner helpers into create() as instance-local
functions that default to the production runner but can be overridden with
create({ exec: { run } }) — the same injection pattern the harmonyos
backend already uses.

tests/win32.test.mjs now exercises the win32 backend on any host through
the injected runner: actions run without powershell, nonzero exits reject
(fail-truthful), and targeted mouse-down is verified to move + press with
the User32 type defined in-process. Verified on macOS with no Windows
involved: node --test tests/win32.test.mjs (4/4) and
tests/backends.test.mjs (10/10, no regression).

Refs Hmbown#5903, Hmbown#5910.

Signed-off-by: EvanProgramming <evangonggyf@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

win32 computer-use: input actions report success on PowerShell failure; left_mouse_down drops the press

2 participants