Skip to content

computer-use macOS: request_access reports screenshot/accessibility available when the host lacks the grants; failures do not name the missing permission #5917

Description

@Hmbown

What happens

request_access on the macOS backend claims capabilities the host process does not have. Driving the bundled server directly over stdio from a host that lacks Screen Recording and Accessibility grants (receipt 2026-09-06, crates/tui/plugins/computer-use at main 7fdb613b2):

request_access -> ok: true, capabilities: { screenshot: true, recording: true, accessibility_tree: true, clipboard: true, ... }
screenshot     -> ok: false, error: "screencapture exited 1: could not create image from display"
get_app_state  -> ok: false, error: "application not found — call list_apps for exact names/pids"   (Calculator had just launched: launched: true)
key cmd+q      -> ok: false, error: "osascript timed out"

The README promises the opposite: "request_access reports what is missing and every capability fails closed naming the missing tool or permission — it never guesses." Today it probes for the presence of screencapture/osascript, not for the grant, so an agent is told it can look and act, then fails on the first call with an error that does not name the missing permission (the key failure is a 25 s timeout, not "Accessibility not granted for ").

Fix

  • Screen Recording: probe with a real capture attempt (or CGPreflightScreenCaptureAccess via the JXA bridge) and report screenshot: false with missing: ["screen_recording"] and the host app name to grant it to.
  • Accessibility: probe AXIsProcessTrusted() through the bridge; when false report accessibility_tree, key, type, click as unavailable with missing: ["accessibility"], and make key/type fail immediately with that reason instead of timing out.
  • open_application returning launched: true followed by get_app_state "application not found" 1.5 s later: either wait for the process to appear before returning, or return the pid so the caller can target it.
  • Tests for the probe results shape on darwin with an injected runner (the same seam refactor(computer-use): injectable win32 runner for host-agnostic backend tests #5912 added for win32).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions