You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
- resolve browser defaults once per automation open request
- pass one presentation decision into reused-tab readiness
- document merged defaults and focused verification results
🤖 Co-authored by GPT-5 in Codex via T3 Code
Upstream commit `32af2f002` (`fix(preview): stabilize PiP viewport identity`) owns epoch-scoped runtime guest identity and keeps PiP, recording, renderer surfaces, and Electron tabs aligned on that identity.
15
15
16
+
Upstream commit `949feb61` (`feat(web): configurable browser defaults`) owns the persisted viewport, zoom, appearance, and automatic floating-preview defaults, plus applying those defaults when a browser tab is created. Upstream commit `cd096b9ad` (`feat(server): users can withhold browser access from agents`) owns whether preview tools and instructions are exposed to a provider session. The branch composes with those settings; it does not maintain another browser-default or access-control layer.
17
+
16
18
This branch does not add a second recording/PiP capture lifecycle or another hidden-preview lifetime mechanism. Its remaining background-specific code is limited to bounded, one-shot automation snapshots. That distinction is necessary: a fresh offscreen guest did not produce a frame or settle upstream recording startup within eight seconds, while staging the same guest at effectively transparent opacity produced a complete screenshot and semantic snapshot in under 100 milliseconds. Upstream's retrying frame loop therefore does not replace the automation snapshot presentation lease, exact-target CDP capture, request deadline propagation, or nullable semantic fallback.
17
19
18
20
Expected behavior:
19
21
20
22
- Every Electron automation operation has a bounded control-session lifetime. The desktop manager reserves response grace inside the requested timeout without making the execution budget shrink when the caller increases a short timeout, always finalizes controller and action-timeline state, and detaches a timed-out debugger session while still holding an acquired control permit when a CDP command may be pending. Session removal and debugger teardown are atomic with respect to new session acquisition and bound to the exact acquired session, so late interruption or snapshot cleanup cannot detach a healthy replacement. Operations already queued on the retired semaphore detect that stale session and retry against its replacement. A request that times out while queued behind another action does not detach that action's shared debugger session.
21
23
- Click, type, and wait operations clamp their caller-supplied timeout to the remaining renderer host budget before entering the desktop control-session boundary. Operations without a caller timeout use the remaining bounded request budget rather than restarting the desktop default after renderer readiness work.
22
24
- Snapshot collection keeps active-tab capture on CDP `Page.captureScreenshot` from the compositor surface. For an unselected tab, the renderer stages the still-mounted guest at effectively transparent opacity for two compositor frames, but only for the snapshot itself. The desktop manager captures that compositor surface without focusing the guest or calling `Page.bringToFront`; either activation call can make Electron promote the native guest over the host window and keep the T3 interface covered after staging ends. A separately bounded `webContents.capturePage` attempt provides a fallback, using `stayHidden: true` for background guests and normal visible-page capture for the foreground. Primary and fallback screenshot waits are clamped to the remaining control-session deadline, with budget reserved for fallback and result settlement, so a tight caller deadline can still return semantic data instead of being preempted by the outer session timeout. Every returned PNG, including resized output, is validated and bounded. Final screenshot failure or timeout is logged, an actually timed-out CDP capture resets the session before releasing its control permit, queued work reattaches before issuing its first command, and a capture skipped before CDP runs leaves the healthy session attached. The semantic page state, interactive elements, accessibility tree, diagnostics, and action timeline still return with `screenshot: null` instead of failing the complete snapshot.
23
-
- Desktop preview guests following the system color scheme create their CDP debugger session lazily, with initialization included in the automation operation deadline. This prevents an offscreen Chromium guest from leaving `Runtime.enable` pending while holding the synchronized session lock, which previously made every later evaluation or snapshot against that tab time out even after it became presentable. A persisted non-system color-scheme override is restored after webview registration or detached DevTools closes through a separately bounded recovery path; tabs following the system scheme stay detached until the next automation operation.
25
+
- Desktop preview guests following the system color scheme create their CDP debugger session lazily, with initialization included in the automation operation deadline. This prevents an offscreen Chromium guest from leaving `Runtime.enable` pending while holding the synchronized session lock, which previously made every later evaluation or snapshot against that tab time out even after it became presentable. `apps/web/src/browser/desktopTabLifetime.ts` passes the upstream browser appearance default through `DesktopPreviewCreateTabInputSchema` in `packages/contracts/src/ipc.ts`; `apps/desktop/src/preview/Manager.ts` normalizes that value. A non-system color-scheme override is restored after webview registration or detached DevTools closes through a separately bounded recovery path, while tabs following the system scheme stay detached until the next automation operation.
24
26
- Building on upstream's retained hidden guest, automation background snapshot presentation is reference-counted independently from the normal surface lease and composes with upstream's fitted-source content and corner-radius presentation. `PreviewAutomationHosts.tsx` passes the epoch-scoped runtime tab id into `previewAutomationPresentation.ts`; every surface lookup, staging marker, readiness check, diagnostic read, lease, and desktop capture targets that exact runtime guest, while selection and errors retain the stable server tab id. The presentation helper API has no state-derived or server-id compatibility fallback. Only a one-shot automation snapshot acquires this lease; upstream recording and picture-in-picture continue to use their shared frame-capture lifecycle, while navigation, color-scheme changes, evaluation, waits, and input operations do not acquire an automation presentation lease. Staging always restores the offscreen position and does not change the human-selected surface. The entire lease, including compositor-frame staging and desktop IPC, is bounded by the operation's remaining response budget and reports a typed timeout if it stalls. If the server epoch replaces the runtime guest while staging is pending, the snapshot fails immediately with `PreviewAutomationTargetUnavailableError` instead of waiting on the stale staging marker. If the user foregrounds the target in either surface while staging is pending, that visible presentation satisfies readiness. A never-presented tab does not depend on another browser surface having supplied a panel rectangle: automation staging falls back to a deterministic rectangle fitted inside the renderer viewport.
25
27
- A background snapshot that times out before desktop capture begins releases its presentation lease even when Chromium has paused compositor-frame callbacks. Once desktop capture starts, a timed-out snapshot retains its presentation lease until that capture settles, so response timeouts cannot tear down compositor staging beneath an in-flight capture. The desktop snapshot receives the operation's remaining timeout and bounds its control session accordingly.
26
-
- The shared preview contract treats snapshot screenshots as nullable. MCP snapshot responses omit image content when capture is unavailable while preserving structured semantic content and explicitly reporting `screenshot: null`; tool descriptions promise a PNG only when capture is available. The desktop snapshot IPC schema and preload adapter default an omitted `background` flag to `false`, preserving foreground-capture behavior for legacy callers.
28
+
- The shared preview contract treats snapshot screenshots as nullable. MCP snapshot responses omit image content when capture is unavailable while preserving structured semantic content and explicitly reporting `screenshot: null`; tool descriptions promise a PNG only when capture is available. In `packages/contracts/src/ipc.ts`, the branch's desktop snapshot schema and upstream's create-tab defaults coexist: snapshot calls default an omitted `background` flag to `false`, while tab creation carries viewport, zoom, and color-scheme defaults through the preload and desktop manager.
27
29
- The renderer automation consumer reserves response grace before the broker deadline and converts a stalled host operation into a typed `PreviewAutomationTimeoutError` instead of leaving the broker to surface a generic execution failure. Short caller-supplied timeouts retain their full execution budget, and the transition into grace reservation remains monotonic as requested timeouts increase. Requests that ask to open the inline preview use the request's remaining bounded visibility budget rather than a fixed two-second ceiling. Best-effort presentation settling uses a non-throwing remaining-budget read and clamps its 500-millisecond ceiling to that budget. Overlay status calls are themselves bounded by the remaining deadline and revalidate runtime guest identity after awaiting the desktop bridge, while overlay, navigation, and visibility polling clamp each sleep to the remaining deadline. Stable-presentation dwell also contracts to fit short deadlines instead of requiring an impossible fixed 100 milliseconds. Reused empty or failed tabs acknowledge without waiting for a browser surface those states intentionally hide. Visibility timeouts separately report the inline preview's selected tab, the right panel's active surface and open state, the active presentation kind, whether the requested browser surface was registered, and whether it had a presentation rectangle.
28
-
- A newly created preview tab applies its server snapshot and assigned tab id, applies the upstream 1280×800 automation viewport when the server snapshot still uses `fill`, initiates any requested selection, and acknowledges server-side creation without making the first call depend on cold React panel rendering, Electron overlay registration, or page readiness. Its initial URL continues loading exactly once in that same tab; status can report progress, while later wait, snapshot, or interaction operations own any attachment or page-readiness wait. Reopening an existing shown tab selects both the preview-state tab and its matching inline mini-player surface, then waits for stable presentation; while the request remains pending it reasserts that explicit selection across same-server route hydration or session reconciliation instead of accepting one transient visible frame. A server-epoch change denotes a replacement runtime guest and aborts the pending open with `PreviewAutomationTargetUnavailableError`; the old request never adopts the new guest. Reused tabs retain overlay, navigation, and requested-visibility readiness checks because their existing automation target should already be available. The deprecated `show` input remains an alias for `open`.
30
+
- `PreviewAutomationHosts.tsx` resolves `browserDefaults.ts` once at the start of each automation-open request. That single snapshot supplies the new-tab viewport and the automatic floating-preview preference, so creation and presentation cannot observe different settings during one request. Explicit `open` or its deprecated `show` alias remains authoritative; when both are omitted, `autoShowFloatingPreview` decides presentation. The resulting `shouldPresentPreview` value is passed unchanged to `previewAutomationOpenReadiness.ts`, so a reused rendered tab left in the background does not wait for visibility while an explicitly shown tab does. A newly created tab applies its server snapshot and assigned tab id, uses the configured viewport or the branch's deterministic 1280×800 fallback when the snapshot remains `fill`, initiates any requested selection, and acknowledges server-side creation without depending on cold React panel rendering, Electron overlay registration, or page readiness. Its initial URL continues loading exactly once in that same tab; later wait, snapshot, or interaction operations own attachment and page readiness. Reopening an existing shown tab selects both the preview-state tab and its matching inline mini-player surface, then waits for stable presentation and reasserts that selection across same-server route hydration or session reconciliation. A server-epoch change aborts the pending open with `PreviewAutomationTargetUnavailableError`; the old request never adopts the replacement runtime guest.
29
31
- The standard dev runner keeps local navigation and direct backend URLs on `127.0.0.1`. Browser modes use a single origin: they leave client HTTP/WebSocket URLs and generic `HOST` unset so remote sharing and origin-derived HMR keep working, while Vite's default listener and its default backend proxy use explicit IPv4 loopback. Explicit IPv6 backend binds proxy through IPv6 loopback. Desktop mode pins `HOST` and its renderer/backend URLs to `127.0.0.1`; server-only mode keeps direct HTTP/WebSocket URLs on the same IPv4 loopback.
30
32
- Retained browser guests subscribe only to render state for their own epoch-scoped runtime tab, plus their thread's active panel and mini-player selection. Presentation or background-capture updates for another tab do not rerender every mounted `HostedBrowserWebview`; selection changes override a stale surface-visible flag so background staging remains nearly transparent and exposes its readiness marker, and staging derives its viewport-fitted rectangle from the target tab's own stable rectangle.
31
33
- The primary pairing route watches for later URL-fragment changes while it remains mounted. Navigating an already-loaded `/pair` document to `/pair#token=...` claims each new token once, removes the secret fragment, and runs the normal pairing exchange without requiring a reload or a second desktop window. Multiple tokens received while an exchange is pending are serialized, and the submitting state remains active until every queued exchange settles.
@@ -70,13 +74,11 @@ vp test run scripts/dev-runner.test.ts apps/desktop/src/app/DesktopAppIdentity.t
70
74
71
75
Current verification:
72
76
73
-
- The focused command above passed all 236 tests, including remaining-host-budget propagation for timeout-bearing desktop inputs, non-throwing best-effort presentation settling, deadline-bounded overlay status and polling, post-status runtime identity validation, deadline-clamped navigation polling, handled delayed capture rejection after a presentation timeout, current-tab-only browser-surface selection, stale-visible background staging after selection changes, and serialized pairing submissions.
74
-
- The desktop manager's 35 focused tests additionally verify that a screenshot timeout detaches the poisoned debugger session before a queued evaluation reattaches and sends its first command.
75
-
- The runtime-id presentation subset (`previewAutomationPresentation`, `previewAutomationOpenReadiness`, `previewAutomationOverlayReadiness`, `previewAutomationRequestConsumer`, and `previewNavigationReadiness`) passed all 50 tests with explicit runtime guest identity, including immediate open, monotonic short-deadline handling, remaining desktop input budgets, non-throwing best-effort settling, bounded overlay status calls, post-status runtime replacement rejection, deadline-clamped presentation, overlay, navigation, and visibility polling, handled delayed capture rejection, and snapshot rejection after a server-epoch replacement.
76
-
- Desktop, server, contracts, and scripts typechecks completed without type errors. The web typecheck currently reaches only three `RegistryContext` typing errors that are also present on `upstream/main`.
77
-
- An isolated worktree `dev:desktop` using the user-data override paired successfully and stayed usable through repeated renderer/CDP inspection and raster capture. An archive worktree client loaded its seeded project against that host without the prior IndexedDB `VersionError` or host-interface disappearance.
78
-
- An isolated web client on ports `5744`/`13784` completed first-navigation pairing, loaded the seeded Preview Reliability thread, and rendered the right-panel surface chooser. Its non-Electron Browser surface was unavailable as described under current limitations.
79
-
- A second isolated web pass on ports `5744`/`13784` delayed the first browser-session exchange, injected another fragment token while it was pending, and verified from in-page request timestamps that the second exchange started only after the first finished; the client then loaded the authenticated app.
77
+
- The branch-focused suite passed 254 tests across 17 files on Windows. The two unchanged desktop path-fixture files in the command above currently produce eight POSIX-versus-Windows path assertion failures; the affected runtime behavior passes in the remaining focused suite.
78
+
- The incoming browser-default, provider-access, pull-request budget, tooltip, update-copy, and settings coverage passed 552 tests across 26 files. The focused open-policy and open-session subset passes 18 tests, including a shared explicit-over-default presentation decision and a reused rendered tab that remains in the background without a visibility wait.
79
+
- Desktop, server, contracts, and mobile typechecks pass. Web typecheck currently reports 16 existing errors in `BranchToolbarBranchSelector.tsx`, `ModelPickerContent.tsx`, `PreviewAutomationHosts.tsx` at the pre-existing registry access, `FontFamilyPicker.tsx`, `use-atom-command.ts`, and `use-atom-query-runner.ts`; none are in the browser-default integration changed here.
80
+
- An isolated web client on ports `5744`/`13784` paired and loaded successfully. Settings → Integrations exposed the browser defaults and agent-access setting; agent browser access could be disabled and restored. The non-Electron client correctly disabled desktop-only viewport, zoom, and appearance controls.
81
+
- Android paired with an isolated server, loaded the seeded project, created a thread, and received a response. The changed Android thread-settings header remains visually unverified because Metro first exhausted file handles and a clean restart then exposed an installed-dev-client `EventEmitter` runtime mismatch.
0 commit comments