Skip to content

Align v2 focus and targeted keyboard with Codex helper - #4

Draft
cm2435-hcomp wants to merge 3 commits into
agent/background-driver-v2-swift-parityfrom
agent/background-driver-v2-focus-semantics
Draft

Align v2 focus and targeted keyboard with Codex helper#4
cm2435-hcomp wants to merge 3 commits into
agent/background-driver-v2-swift-parityfrom
agent/background-driver-v2-focus-semantics

Conversation

@cm2435-hcomp

@cm2435-hcomp cm2435-hcomp commented Jul 29, 2026

Copy link
Copy Markdown
Owner

What

Aligns the v2 Rust/macOS background driver with the recovered signed Codex
helper in two related areas: target-contained focus semantics and generic
targeted-keyboard delivery.

  • removes the public action-wide posture result and the global
    foreground/key-window/HID witness
  • keeps the durable per-target synthetic-focus controller and treats unrelated
    user activity as external QA evidence
  • preserves exact identity, freshness, settlement, effect verification, and
    concrete cleanup failures
  • retains non-activating launch and the exact-close settlement profile
  • adds a typed MacKeyboardEventTarget with separate application and effective
    dispatch PIDs
  • revalidates the exact observed focused AX element immediately before mutation
  • ports the helper's CPS SwiftKeyboard target-belief recipe
  • constructs the complete four-event sequence before dispatch: requested flags,
    key down, key up, and restored combined-session flags
  • timestamps and posts each event exactly once with direct CGEventPostToPid
  • removes SkyLight and fallible per-post cleanup from the v2 keyboard route
    while leaving legacy v1 tools unchanged
  • publishes press_key for exact visible/occluded current-Space targets,
    including Framework::Unknown
  • accepts an idempotent Chromium AX enablement write when a non-success status
    is followed by exact requested-state readback
  • regenerates the 0.7.1 CLI and MCP reference docs

Why

The previous posture contract failed correct background actions when the user
moved the hardware cursor or used another application. Binary inspection
showed that the Codex helper coordinates current target focus but does not
return an action-wide containment verdict, callback-drain proof, focus poison
result, or launch activation journal.

The previous press_key refusal was also our porting gap, not a TextEdit or
macOS limitation. The signed helper selects a generic route from exact
target/focus facts, uses an effective PID for out-of-process AX targets, builds
four aggregate-flags/key events, and posts them directly to that PID. It does
not gate the generic route on an AppKit/Chromium classifier or use SkyLight.

Validation

Automated gates:

  • cargo test -p platform-macos --lib: 187 passed
  • cargo test -p cua-driver-core --lib: 120 passed
  • cargo test -p cua-driver --bins: 69 passed
  • cargo check -p platform-macos --all-targets: passed
  • changed Rust files pass rustfmt --check
  • generated CLI/MCP docs are synchronized
  • retained regressions cover Unknown-framework capability publication,
    application/effective PID evidence, stale-focus refusal, exact four-event
    order, direct one-post-per-event transport, and exact AX enablement readback

Live-host evidence at 2082500ed6890f15546d9e80c7fabb38dc4d4bfc:

  • TextEdit/Unknown: 5/5 repetitions
  • Chromium: 5/5 repetitions
  • 30/30 Return, Left, and Command+A actions produced independent AX or DOM
    effects
  • 30/30 receipts used targeted_keyboard, four attempted/posted events,
    core_graphics_pid, no clipboard, and truthful dispatch_unverified
  • zero target-frontmost and zero target-top-window samples
  • Slack remained foreground while the hardware cursor occupied 194 distinct
    sampled positions; no keyboard action attempted a cursor warp

Exact support boundary

Proven: visible/current-Space TextEdit/Unknown and Chromium targets where
application PID equals dispatch PID.

Refused before dispatch: minimized, off-Space, unknown-state, stale-focus,
missing-effective-PID, and unproved exception states.

Still unproved: a live out-of-process focus owner where application and dispatch
PID differ, Catalyst-specific preparation, targeted Unicode-event text
fallback, and the remaining Chromium scroll/drag/menu cells.

Workspace-wide cargo fmt --all -- --check and strict
cargo clippy ... -D warnings remain red on the unchanged 0.7.1 base because
of broad pre-existing formatting drift and legacy warnings in core,
cursor-overlay, and older macOS modules. The changed keyboard files are
formatted and add no new clippy diagnostic.

@github-actions

github-actions Bot commented Jul 29, 2026

Copy link
Copy Markdown

Linux visual regression artifacts

Matrix jobs now run independently. Download visual artifacts from this workflow run.
Each background-GUI job uploads a .gif of the interaction plus two annotated PNGs (<app>.png raw, <app>-atspi.png with AT-SPI element boxes); the cua-driver-linux-som-overlays artifact adds <app>-som.png cua Set-of-Marks overlays:

  • cua-driver-linux-cursor-click-gif
  • cua-driver-linux-background-terminal-gif
  • cua-driver-linux-parallel-drag-xserver
  • cua-driver-linux-background-gui-chromium
  • cua-driver-linux-background-gui-tk
  • cua-driver-linux-background-gui-gtk3-gedit
  • cua-driver-linux-background-gui-gtk3-mousepad
  • cua-driver-linux-background-gui-gtk3-scite
  • cua-driver-linux-background-gui-gtk4-characters
  • cua-driver-linux-background-gui-qt5-manuskript
  • cua-driver-linux-background-gui-qt5-klog
  • cua-driver-linux-background-gui-qt5-openambit
  • cua-driver-linux-background-gui-qt6-kate
  • cua-driver-linux-background-gui-qt6-kcalc
  • cua-driver-linux-background-gui-qt6-okular
  • cua-driver-linux-background-gui-qt6-qownnotes
  • cua-driver-linux-background-gui-electron-zettlr
  • cua-driver-linux-background-gui-electron-joplin
  • cua-driver-linux-background-gui-electron-logseq
  • cua-driver-linux-som-overlays

Open workflow run and download artifacts

@cm2435-hcomp cm2435-hcomp changed the title Align v2 focus semantics with Codex helper Align v2 focus and targeted keyboard with Codex helper Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant