Skip to content

win32 computer-use: input actions report success on PowerShell failure; left_mouse_down drops the press #5908

Description

@EvanProgramming

Root cause

The Windows backend (crates/tui/plugins/computer-use/src/backends/win32.mjs) reports {action_sent: true} even when the underlying action never ran. Three independent defects:

  1. No in-process type definition. create() only loads the User32 P/Invoke type in the bootstrap PowerShell process. Every action runs in a fresh powershell.exe spawned by ps(), so [User32]::mouse_event / SendInput / GetCursorPos are undefined there and the call fails — but nobody checks.
  2. withUser32 never inspects the exit code. It just return ps(script, opts), so a nonzero exit is ignored and every User32 action returns success unconditionally (see Windows computer-use reports input success after PowerShell failure #5896).
  3. left_mouse_down precedence bug. target ? A : "" + B evaluates "\u201c" + B on the no-target branch, so a targeted call only emits SetCursorPos and silently drops mouse_event(LEFTDOWN).

Impact

A parked/crashed agent, or any Windows session, believes input (clicks, drags, typing, key combos) was delivered when it was not. The model proceeds on false confirmations — a silent, high-impact correctness failure.

Fix (in progress)

  • Make create() accept an injectable runner ({ exec }) so the backend is testable without powershell.exe.
  • Prepend the User32 type definition (USER32_DEF) to every User32-backed script so each action is self-contained.
  • withUser32 now throws an ExecError on a nonzero exit instead of reporting success.
  • Rewrite left_mouse_down so a targeted call both moves and presses.
  • cursor_position routes through withUser32 so it loads the type in-process and fails truthfully.
  • Regression tests in tests/win32.test.mjs (mocked runner): type defined in-process, failure path rejects, targeted left_mouse_down moves+presses.

Relates to #5896.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions