diff --git a/CONFIGURATION.md b/CONFIGURATION.md index fef5b191..4bfcedb3 100644 --- a/CONFIGURATION.md +++ b/CONFIGURATION.md @@ -133,7 +133,8 @@ cursor movement. CLI commands and native app focus changes never warp pointer. ## `[animation]` -Controls scrolling-column focus and managed column-resize animation. +Controls scrolling-column focus and managed layout animations, including width +changes, column/window moves, stacking, and floating transitions. The macOS Reduce Motion preference also disables these animations, including workspace transitions and mouse-driven reordering. @@ -145,7 +146,8 @@ duration_ms = 125 | Setting | Default | Values/type | Description | | --- | --- | --- | --- | -| `enabled` | `true` | boolean | Enables visual scrolling and managed resize animation. | +| `enabled` | `true` | boolean | Enables visual scrolling and managed layout animations, including resizing, column/window moves, stacking, and floating transitions. | +| `experimental_window_representations` | `false` | boolean | Experimental horizontal ribbon: animate one-shot window representations and commit native positions behind them. Requires existing Screen Recording access; does not request permission or start a continuous stream. Missing captures, unsupported geometry, and workspace switches use native windows. | | `duration_ms` | `125` | integer from `0` to `2000` | Animation duration in milliseconds. Vertical workspace transitions use at least 180 ms when the usable viewport covers the physical display; otherwise they switch immediately to prevent reserved-area leaks. `0` disables animation even when `enabled = true`. | ## `[overview]` @@ -156,6 +158,7 @@ Controls the Overview scale and optional pixels inside window cards. [overview] zoom = 0.5 window_previews = false +experimental_surface_transitions = false window_corner_radius = 12 ``` @@ -163,6 +166,7 @@ window_corner_radius = 12 | --- | --- | --- | --- | | `zoom` | `0.5` | number from `0` to `0.75` | Scales workspaces and windows. Lower values show more of the neighboring workspaces. | | `window_previews` | `false` | boolean | Captures a card-sized still image when a window first becomes visible in the current Overview session. | +| `experimental_surface_transitions` | `false` | boolean | Requires `window_previews = true`. Prepares public one-shot ScreenCaptureKit samples for native-size-to-Overview zoom transitions. Normal ribbon navigation remains native. | | `window_corner_radius` | `12` | number from `0` to `64` | Rounds window cards and their borders in the Overview. | With the default `false`, Defi performs no Screen Recording permission check, @@ -172,8 +176,34 @@ image. Denial, revocation, protected content, and capture errors leave the icon-and-title cards fully usable and do not trigger repeated prompts in the same daemon session. -Previews are memory-only, contain no audio or cursor, and use at most 32 MiB -between Overview sessions. Defi validates the window and process identity before +The surface transition experiment prepares one-shot screenshots of the active +workspace's visible windows and up to two nearby offscreen windows at native +resolution. While the Overview is closed, preparation runs at most once per +second, and only one batch may run at a time. It never starts persistent capture +streams. Opening freezes one fixed screenshot set through the zoom and the +Overview session; these are snapshots, not live video. Native ribbon navigation +keeps the ordinary native animation backend. + +The retained NV12 buffers use public AVFoundation video layers without video +encoding or per-frame CGImage conversion. NV12 preserves luma resolution but +subsamples chroma. A conservative 256 MiB screenshot/renderer overlap estimate +bounds the selected window set; this is not a bound on total process or +WindowServer RAM. Inactive cards use one-shot captures in the existing 32 MiB +compact-preview cache. No pixels are written to disk and no frame history is +retained. Public screenshot capture may still involve macOS permission or +transient capture UI; this option does not suppress system privacy indicators. + +When permission, geometry, readiness, snapshot age or the budget prevents a +complete transition, the normal Overview opens instead. Changing the Overview +projection returns to ordinary card previews. A cold start still needs screenshot +preparation. Escape can zoom back when native frames and Overview offsets are +unchanged; selecting a different window/workspace uses the existing native path. +Snapshot freshness during typing or animated content, shadows, widgets and +native fullscreen windows remain limits of this experiment. + +Ordinary remembered previews are memory-only, contain no audio or cursor, and +use at most 32 MiB between Overview sessions. The surface experiment has the +additional budgets described above. Defi validates the window and process identity before reuse, requests a fresh image immediately, and removes the remembered image if that capture fails. Remembered previews are limited to 512 pixels on their longest edge and retained until macOS signals memory pressure or their window identity diff --git a/Sources/DefiConfig/Config.swift b/Sources/DefiConfig/Config.swift index db5c97b0..7d0852ad 100644 --- a/Sources/DefiConfig/Config.swift +++ b/Sources/DefiConfig/Config.swift @@ -140,6 +140,9 @@ public struct Config: Equatable, Sendable { guard overview.zoom.isFinite, (0...0.75).contains(overview.zoom) else { throw ConfigError.invalidValue("overview.zoom") } + guard !overview.experimentalSurfaceTransitions || overview.windowPreviews else { + throw ConfigError.invalidValue("overview.experimental_surface_transitions requires window_previews") + } guard overview.windowCornerRadius.isFinite, (0...64).contains(overview.windowCornerRadius) else { diff --git a/Sources/DefiConfig/ConfigModels.swift b/Sources/DefiConfig/ConfigModels.swift index 5a49beaf..d5254ef0 100644 --- a/Sources/DefiConfig/ConfigModels.swift +++ b/Sources/DefiConfig/ConfigModels.swift @@ -137,15 +137,19 @@ public func parseBorderColor(_ value: String) -> UInt32? { public struct AnimationConfig: Codable, Equatable, Sendable { public var enabled: Bool public var durationMS: Int + public var experimentalWindowRepresentations: Bool - public init(enabled: Bool = true, durationMS: Int = 125) { + public init(enabled: Bool = true, durationMS: Int = 125, + experimentalWindowRepresentations: Bool = false) { self.enabled = enabled self.durationMS = durationMS + self.experimentalWindowRepresentations = experimentalWindowRepresentations } enum CodingKeys: String, CodingKey { case enabled case durationMS = "duration_ms" + case experimentalWindowRepresentations = "experimental_window_representations" } public init(from decoder: Decoder) throws { @@ -153,27 +157,33 @@ public struct AnimationConfig: Codable, Equatable, Sendable { let defaults = AnimationConfig() enabled = try values.decodeIfPresent(Bool.self, forKey: .enabled) ?? defaults.enabled durationMS = try values.decodeIfPresent(Int.self, forKey: .durationMS) ?? defaults.durationMS + experimentalWindowRepresentations = try values.decodeIfPresent( + Bool.self, forKey: .experimentalWindowRepresentations) ?? defaults.experimentalWindowRepresentations } } public struct OverviewConfig: Codable, Equatable, Sendable { public var zoom: Double public var windowPreviews: Bool + public var experimentalSurfaceTransitions: Bool public var windowCornerRadius: Double public init( zoom: Double = 0.5, windowPreviews: Bool = false, + experimentalSurfaceTransitions: Bool = false, windowCornerRadius: Double = 12 ) { self.zoom = zoom self.windowPreviews = windowPreviews + self.experimentalSurfaceTransitions = experimentalSurfaceTransitions self.windowCornerRadius = windowCornerRadius } enum CodingKeys: String, CodingKey { case zoom case windowPreviews = "window_previews" + case experimentalSurfaceTransitions = "experimental_surface_transitions" case windowCornerRadius = "window_corner_radius" } @@ -183,6 +193,9 @@ public struct OverviewConfig: Codable, Equatable, Sendable { zoom = try values.decodeIfPresent(Double.self, forKey: .zoom) ?? defaults.zoom windowPreviews = try values.decodeIfPresent(Bool.self, forKey: .windowPreviews) ?? defaults.windowPreviews + experimentalSurfaceTransitions = try values.decodeIfPresent( + Bool.self, forKey: .experimentalSurfaceTransitions + ) ?? defaults.experimentalSurfaceTransitions windowCornerRadius = try values.decodeIfPresent(Double.self, forKey: .windowCornerRadius) ?? defaults.windowCornerRadius } diff --git a/Sources/DefiCore/Overview.swift b/Sources/DefiCore/Overview.swift index b48ed245..096ad9fb 100644 --- a/Sources/DefiCore/Overview.swift +++ b/Sources/DefiCore/Overview.swift @@ -76,6 +76,25 @@ public func interpolateOverviewViewport( ) } +public func overviewProjectionReordersExistingCards( + from: OverviewProjection, to: OverviewProjection +) -> Bool { + guard from.monitorID == to.monitorID else { return false } + let previous = Dictionary(uniqueKeysWithValues: from.workspaces.flatMap { workspace in + workspace.windows.map { ($0.windowID, (workspace.workspaceID, $0.layer, + $0.frame.x - workspace.frame.x, $0.frame.y - workspace.frame.y)) } + }) + return to.workspaces.contains { workspace in + workspace.windows.contains { card in + guard let old = previous[card.windowID] else { return false } + return old.0 != workspace.workspaceID || old.1 != card.layer + || (card.layer == .floating && ( + abs(old.2 - (card.frame.x - workspace.frame.x)) >= 0.5 + || abs(old.3 - (card.frame.y - workspace.frame.y)) >= 0.5)) + } + } +} + public func overviewProjectionResizesExistingCards( from: OverviewProjection, to: OverviewProjection ) -> Bool { @@ -304,7 +323,8 @@ public func projectOverview( viewport: OverviewViewport, layout: LayoutSettings, zoom: Double = 0.5, - workspaceGap: Double = 28 + workspaceGap: Double = 28, + includeOffscreenContent: Bool = false ) -> OverviewProjection { guard let monitor = snapshot.monitors.first(where: { $0.id == monitorID }), let monitorFrame = snapshot.monitorFrames[monitorID], @@ -343,7 +363,7 @@ public func projectOverview( width: workspaceWidth, height: workspaceHeight ) - guard workspaceFrame.intersects(projectionBounds) else { continue } + guard includeOffscreenContent || workspaceFrame.intersects(projectionBounds) else { continue } var workspace = originalWorkspace workspace.scrollOffset = viewport.horizontalOffsets[workspace.id] @@ -390,13 +410,13 @@ public func projectOverview( ) if projectedFrame.x + projectedFrame.width <= workspaceFrame.x { hiddenTiledWindowCountBefore += 1 - continue + if !includeOffscreenContent { continue } } if projectedFrame.x >= workspaceFrame.x + workspaceFrame.width { hiddenTiledWindowCountAfter += 1 - continue + if !includeOffscreenContent { continue } } - guard projectedFrame.intersects(workspaceFrame) else { continue } + guard includeOffscreenContent || projectedFrame.intersects(workspaceFrame) else { continue } let isFullscreen = snapshot.nativeFullscreenWindowIDs.contains(windowID) projectedWindows.append( OverviewWindowProjection( @@ -425,7 +445,7 @@ public func projectOverview( width: localFrame.width * contentScale, height: localFrame.height * contentScale ) - guard projectedFrame.intersects(workspaceFrame) else { continue } + guard includeOffscreenContent || projectedFrame.intersects(workspaceFrame) else { continue } let isFullscreen = snapshot.nativeFullscreenWindowIDs.contains(windowID) projectedWindows.append( OverviewWindowProjection( diff --git a/Sources/DefiDaemon/DaemonAnimation.swift b/Sources/DefiDaemon/DaemonAnimation.swift index bcfe3c48..f5319fb7 100644 --- a/Sources/DefiDaemon/DaemonAnimation.swift +++ b/Sources/DefiDaemon/DaemonAnimation.swift @@ -309,7 +309,7 @@ extension Daemon { return true } - func dispatchManagedResizeAnimation( + func dispatchManagedLayoutAnimation( monitorIDs: Set? = nil, skipping skippedWindowIDs: Set = [], forcingFloatingFrameWritesFor forcedFloatingWindowIDs: Set = [], @@ -329,7 +329,7 @@ extension Daemon { skipping: skippedWindowIDs, forcingFloatingFrameWritesFor: forcedFloatingWindowIDs, commandPerformance: commandPerformance, - source: "command-resize-animation" + source: "command-layout-animation" ) needsDesktopSync = true return true diff --git a/Sources/DefiDaemon/DaemonCommands.swift b/Sources/DefiDaemon/DaemonCommands.swift index f19b17e0..b095225e 100644 --- a/Sources/DefiDaemon/DaemonCommands.swift +++ b/Sources/DefiDaemon/DaemonCommands.swift @@ -388,6 +388,39 @@ extension Daemon { ribbonPrototype: overviewRibbonPrototypeRequested(in: rawCommand) ) } + if overviewState.isOpen, command.editsSelectedLayout, monitorIndex == nil { + let originatingSession = overviewToggleState.snapshot().sessionGeneration + deferredResponse?.deferResponse() + DispatchQueue.main.async { [weak self] in + guard let self else { deferredResponse?.fail("daemon unavailable"); return } + if let controller = overviewController, + overviewToggleState.isCurrentSession(originatingSession), + controller.sessionGeneration == originatingSession { + let accepted = controller.applyLayoutCommand(command) { + [weak self] command, windowID, appID, monitorID, workspaceID, generation in + NavigationActor.enqueue { [weak self] in + let operation = { + guard let self else { return CommandResponse.failure("daemon unavailable") } + guard self.overviewToggleState.isCurrentSession(generation) else { + return CommandResponse.failure("overview session changed; command was not applied") + } + return self.editLayoutFromOverview(command, intent: OverviewWindowIntent( + windowID: windowID, expectedAppID: appID, + sourceMonitorID: monitorID, sourceWorkspaceID: workspaceID)) + } + if let deferredResponse { deferredResponse.perform(operation) } + else { _ = operation() } + } + } + if !accepted { + deferredResponse?.fail("overview selection cannot apply this layout command") + } + } else { + deferredResponse?.fail("overview session changed; command was not applied") + } + } + return .success("queued for overview selection") + } let commandMonitorID: MonitorID? if let monitorIndex { guard let monitorID = monitorID(atAppKitIndex: monitorIndex) else { @@ -577,21 +610,21 @@ extension Daemon { let switchesWorkspace = command.activatesWorkspace let mutatesWorkspaceWindows = command.movesWindowBetweenWorkspaces let movesAcrossMonitors = command.movesWindowsAcrossMonitors - let resizesManagedLayout = command.resizesManagedLayout + let animatesManagedLayout = command.animatesManagedLayout let speculativeRibbonNavigation = isSpeculativeRibbonNavigation(command) if switchesWorkspace || mutatesWorkspaceWindows || movesAcrossMonitors - || resizesManagedLayout || speculativeRibbonNavigation + || animatesManagedLayout || speculativeRibbonNavigation { rearmPointerFocusTransition() } if switchesWorkspace || mutatesWorkspaceWindows || movesAcrossMonitors - || resizesManagedLayout + || animatesManagedLayout || speculativeRibbonNavigation { preemptMouseGesture() } - let animatedManagedResize = - resizesManagedLayout + let animatedManagedLayout = + animatesManagedLayout && animationsEnabled && config.animation.durationMS > 0 let previousWorkspaceID = commandMonitorID.flatMap { monitorID in @@ -757,8 +790,8 @@ extension Daemon { inFlightAnimations: inFlightAnimationMonitorIDs ) let dispatchedAnimation = - animatedManagedResize - ? dispatchManagedResizeAnimation( + animatedManagedLayout + ? dispatchManagedLayoutAnimation( monitorIDs: affectedMonitorIDs, forcingFloatingFrameWritesFor: movedFloatingWindowIDs, commandPerformance: commandPerformance diff --git a/Sources/DefiDaemon/DaemonConfiguration.swift b/Sources/DefiDaemon/DaemonConfiguration.swift index 4803aa04..9ce698d8 100644 --- a/Sources/DefiDaemon/DaemonConfiguration.swift +++ b/Sources/DefiDaemon/DaemonConfiguration.swift @@ -159,6 +159,7 @@ extension Daemon { ) let requiresLayout = state.applyConfiguration(nextConfig) config = nextConfig + platform.experimentalSurfaceRibbonEnabled = nextConfig.animation.experimentalWindowRepresentations Task { @MainActor in DefiSettingsRuntimeStatus.shared.updateConfiguration(nextConfig) } @@ -294,6 +295,14 @@ extension Daemon { self?.publishOverviewState() } }, + overviewCancelHandler: { [weak self] in + NavigationActor.enqueue { [weak self] in self?.handleCheatsheetInput(.dismiss) } + DispatchQueue.main.async { [weak self] in + self?.cheatsheetController?.close() + self?.overviewController?.close() + self?.publishOverviewState() + } + }, cheatsheetHandler: { [weak self] input in guard let self, self.hotKeyGeneration.withLock({ $0 == generation }) else { return } self.handleCheatsheetInput(input) diff --git a/Sources/DefiDaemon/DaemonDesktop.swift b/Sources/DefiDaemon/DaemonDesktop.swift index 20438c27..e4bf456d 100644 --- a/Sources/DefiDaemon/DaemonDesktop.swift +++ b/Sources/DefiDaemon/DaemonDesktop.swift @@ -82,6 +82,7 @@ extension Daemon { guard !restorationInFlight, !shouldShutdown else { return } let layoutStartedAt = ProcessInfo.processInfo.systemUptime var assignments: [FrameAssignment] = [] + var ribbonFrames: [FrameAssignment] = [] var borderAssignments: [FrameAssignment] = [] var nativeFullscreenPlaceholderAssignments: [FrameAssignment] = [] var hiddenWindowIDs = Set() @@ -147,6 +148,7 @@ extension Daemon { ) let sizedFrames = layout.map(preserveIntrinsicSize) if workspace.id == monitor.activeWorkspace && !overviewParksWindows { + ribbonFrames.append(contentsOf: sizedFrames) let strip = continuousStripFramesForActiveWorkspace( sizedFrames, viewport: viewport, @@ -302,6 +304,7 @@ extension Daemon { platform.stageWindowBorderSelection(selectedWindowID) platform.apply( platformAssignments, + ribbonFrames: ribbonFrames, hiddenWindowIDs: hiddenWindowIDs, skipping: skipped, asynchronousPositionTimeoutSeconds: positionTimeoutSeconds, diff --git a/Sources/DefiDaemon/DaemonDesktopSynchronization.swift b/Sources/DefiDaemon/DaemonDesktopSynchronization.swift index 089fbbad..d8ed8068 100644 --- a/Sources/DefiDaemon/DaemonDesktopSynchronization.swift +++ b/Sources/DefiDaemon/DaemonDesktopSynchronization.swift @@ -35,6 +35,14 @@ func shouldCommitNativeFocusSelection( nativeFocusAccepted && selectionChanged } +func nativeFocusAnimationMonitorID( + focusedMonitorID: MonitorID?, floating: Bool, + overviewOpen: Bool, mouseGestureActive: Bool, displayGeometryChanged: Bool +) -> MonitorID? { + guard !floating, !overviewOpen, !mouseGestureActive, !displayGeometryChanged else { return nil } + return focusedMonitorID +} + func validatedNativeActivationTimestamp( snapshot: DesktopSnapshot, resolvedActivation: UserInputTracker.ApplicationActivation?, @@ -72,6 +80,7 @@ func shouldCloseOverviewAfterNativeFocusChange( func desktopSnapshotWaitsForCommandAnimation( animationPending: Bool, latestCommandInputTimestamp: TimeInterval, + latestNativeFocusAnimationInputTimestamp: TimeInterval = 0, mouseFocusIntentTimestamp: TimeInterval?, keyboardFocusIntentTimestamp: TimeInterval?, mouseGestureActive: Bool = false, @@ -82,7 +91,7 @@ func desktopSnapshotWaitsForCommandAnimation( mouseFocusIntentTimestamp ?? 0, keyboardFocusIntentTimestamp ?? 0, applicationActivationTimestamp ?? 0 - ) <= latestCommandInputTimestamp + ) <= max(latestCommandInputTimestamp, latestNativeFocusAnimationInputTimestamp) } @NavigationActor @@ -806,7 +815,6 @@ extension Daemon { viewports: viewportsByMonitor ) } - snapScrollOffsetsToTargets() if tracesWindowCreation { platform.recordPerformanceTrace("sync-before-layout") } @@ -815,10 +823,25 @@ extension Daemon { && snapshot.leftMouseButtonDown && animationsEnabled && config.animation.durationMS > 0 - if animatesMouseReorder { - mouseReorderAnimationActive = true - beginFrameAnimationActivity() - } + let nativeAnimationMonitorID = nativeFocusAnimationMonitorID( + focusedMonitorID: nativelyFocusedMonitorID, + floating: focusedWindowIDForAlignment.flatMap { state.windows[$0]?.floating } == true, + overviewOpen: overviewState.isOpen, + mouseGestureActive: mouseResizeGestureActive, + displayGeometryChanged: displayGeometryChanged + ) + let nativeFocusRequiresMovement = nativeAnimationMonitorID.flatMap { id in + state.monitors.first { $0.id == id } + }.flatMap { monitor in + monitor.workspaces.first { $0.id == monitor.activeWorkspace } + }.map { abs($0.scrollOffset - $0.targetScrollOffset) >= 0.000_1 } == true + let animatesNativeFocus = nativeAnimationMonitorID != nil + && (nativeFocusRequiresMovement || nativelyActivatedWorkspace) + && animationsEnabled && config.animation.durationMS > 0 + let animatesFocusOrReorder = animatesNativeFocus || animatesMouseReorder + snapScrollOffsetsToTargets() + if animatesMouseReorder { mouseReorderAnimationActive = true } + if animatesFocusOrReorder { beginFrameAnimationActivity() } let nativeFocusSkippedWindowIDs: Set if nativeFocusWasPending { if let nativeFocusFrameMonitorID { @@ -849,9 +872,10 @@ extension Daemon { } else { nativeCursorWarpIsCurrentAfterCommit = nil } - if desktopSnapshotWaitsForCommandAnimation( + if !animatesNativeFocus, desktopSnapshotWaitsForCommandAnimation( animationPending: platform.hasPendingAnimatedFrameWrites, latestCommandInputTimestamp: latestCommandInputTimestamp, + latestNativeFocusAnimationInputTimestamp: latestNativeFocusAnimationInputTimestamp, mouseFocusIntentTimestamp: snapshot.mouseFocusIntentTimestamp, keyboardFocusIntentTimestamp: snapshot.keyboardFocusIntentTimestamp, mouseGestureActive: mouseResizeGestureActive, @@ -860,15 +884,24 @@ extension Daemon { ) { needsDesktopSync = true } else { + if animatesNativeFocus { + latestNativeFocusAnimationInputTimestamp = max( + snapshot.latestUserInputTimestamp, + snapshot.mouseFocusIntentTimestamp ?? 0, + snapshot.keyboardFocusIntentTimestamp ?? 0, + snapshot.applicationActivationTimestamp ?? 0 + ) + } applyCurrentLayout( + monitorIDs: animatesNativeFocus ? nativeAnimationMonitorID.map { [$0] } : nil, asynchronousPositions: true, updateVisibility: true, positionTimeoutSeconds: 0.05, - animationDuration: animatesMouseReorder + animationDuration: animatesFocusOrReorder ? TimeInterval(config.animation.durationMS) / 1_000 : 0, skipping: nativeFocusSkippedWindowIDs, - positionsOnly: animatesMouseReorder, + positionsOnly: animatesMouseReorder || (animatesNativeFocus && !nativelyActivatedWorkspace), stagesVisibleBeforeParking: nativelyActivatedWorkspace, cursorWarpWindowIDAfterCommit: nativeCursorWarpWindowID, cursorWarpInputTimestampAfterCommit: nativeCursorWarpInputTimestamp, @@ -876,9 +909,10 @@ extension Daemon { nativeCursorWarpIsCurrentAfterCommit, forceFloatingFrameWrites: displayGeometryChanged, forcingFloatingFrameWritesFor: relocatedFloatingWindowIDs, - source: nativelyActivatedWorkspace - ? "native-workspace" - : (animatesMouseReorder ? "mouse-reorder-animation" : "desktop-sync") + source: animatesNativeFocus + ? "native-focus-animation" + : (nativelyActivatedWorkspace ? "native-workspace" + : (animatesMouseReorder ? "mouse-reorder-animation" : "desktop-sync")) ) } if let guardedRemovalFocus { diff --git a/Sources/DefiDaemon/DaemonFloatingWindows.swift b/Sources/DefiDaemon/DaemonFloatingWindows.swift index 4c98da8e..ab7f6c63 100644 --- a/Sources/DefiDaemon/DaemonFloatingWindows.swift +++ b/Sources/DefiDaemon/DaemonFloatingWindows.swift @@ -44,6 +44,8 @@ func updateFloatingWindowFrames( } continue } + // Overview edits are logical until exit submits their native frames. + if overviewState.isOpen, overviewFloatingFrameWriteIDs.contains(window.id) { continue } if let externalFrame = externallyChangedFrames[window.id] { floatingWindowFrames[window.id] = externalFrame platform.acceptObservedFrame(externalFrame, for: window.id) diff --git a/Sources/DefiDaemon/DaemonFocus.swift b/Sources/DefiDaemon/DaemonFocus.swift index 207bc645..b54648ee 100644 --- a/Sources/DefiDaemon/DaemonFocus.swift +++ b/Sources/DefiDaemon/DaemonFocus.swift @@ -219,7 +219,9 @@ extension Daemon { self.pointerFocusAppliedCount += 1 self.platform.commitWindowBorderSelection(windowID) self.startScrollAnimationsIfNeeded() - _ = self.dispatchScrollAnimationIfNeeded() + if self.dispatchScrollAnimationIfNeeded() { + self.latestNativeFocusAnimationInputTimestamp = timestamp + } self.needsDesktopSync = true self.updateMenuBar() case .resumeDisplaced: diff --git a/Sources/DefiDaemon/DaemonOverview.swift b/Sources/DefiDaemon/DaemonOverview.swift index 5fd5b8b7..b7194848 100644 --- a/Sources/DefiDaemon/DaemonOverview.swift +++ b/Sources/DefiDaemon/DaemonOverview.swift @@ -26,6 +26,7 @@ enum OverviewToggleDecision: Equatable, Sendable { struct OverviewToggleSnapshot: Equatable, Sendable { let generation: UInt64 let actualIsOpen: Bool + let sessionGeneration: UInt64? let desiredIsOpen: Bool? } @@ -34,6 +35,8 @@ final class OverviewToggleState: @unchecked Sendable { private let lock = NSLock() private var generation: UInt64 = 0 private var actualIsOpen = false + private var sessionGeneration: UInt64? + private var completedSelectionSession: UInt64? private var desiredIsOpen: Bool? private var applyingGeneration: UInt64? @@ -47,6 +50,7 @@ final class OverviewToggleState: @unchecked Sendable { guard !targetIsOpen || !ribbonPrototype || screenCaptureAccessGranted else { return .denied } + completedSelectionSession = nil generation &+= 1 desiredIsOpen = targetIsOpen return .requested( @@ -61,6 +65,7 @@ final class OverviewToggleState: @unchecked Sendable { func requestClose() -> OverviewToggleRequest { lock.lock() defer { lock.unlock() } + completedSelectionSession = nil generation &+= 1 desiredIsOpen = false return OverviewToggleRequest( @@ -95,10 +100,12 @@ final class OverviewToggleState: @unchecked Sendable { } } - func recordControllerState(_ isOpen: Bool) { + func recordControllerState(_ isOpen: Bool, sessionGeneration: UInt64? = nil) { lock.lock() defer { lock.unlock() } actualIsOpen = isOpen + if isOpen { completedSelectionSession = nil } + self.sessionGeneration = sessionGeneration guard let applyingGeneration else { generation &+= 1 desiredIsOpen = nil @@ -108,12 +115,35 @@ final class OverviewToggleState: @unchecked Sendable { desiredIsOpen = nil } + func isCurrentSession(_ session: UInt64?) -> Bool { + lock.lock() + defer { lock.unlock() } + return session != nil && session == sessionGeneration + && actualIsOpen && desiredIsOpen != false + } + + func completeSelectionExit(_ session: UInt64) { + lock.lock() + defer { lock.unlock() } + guard actualIsOpen, desiredIsOpen != false, sessionGeneration == session else { return } + completedSelectionSession = session + } + + func selectionWarpIsCurrent(_ session: UInt64?) -> Bool { + lock.lock() + defer { lock.unlock() } + guard let session else { return false } + return completedSelectionSession == session + || (actualIsOpen && desiredIsOpen != false && sessionGeneration == session) + } + func snapshot() -> OverviewToggleSnapshot { lock.lock() defer { lock.unlock() } return OverviewToggleSnapshot( generation: generation, actualIsOpen: actualIsOpen, + sessionGeneration: sessionGeneration, desiredIsOpen: desiredIsOpen ) } @@ -145,6 +175,7 @@ extension Daemon { private func applyOverviewToggle(_ request: OverviewToggleRequest) { let snapshot = makeOverviewSnapshot(), config = config, layout = state.layout + platform.experimentalSurfaceRibbonEnabled = config.animation.experimentalWindowRepresentations DispatchQueue.main.async { [self] in guard overviewToggleState.beginApplying(request) else { return } let controller = overviewController ?? makeOverviewController() @@ -158,13 +189,16 @@ extension Daemon { borders: config.decorations.borders, animation: config.animation, zoom: config.overview.zoom, windowCornerRadius: config.overview.windowCornerRadius, - windowPreviewsEnabled: config.overview.windowPreviews) + windowPreviewsEnabled: config.overview.windowPreviews, + experimentalSurfaceTransitions: config.overview.experimentalSurfaceTransitions, + experimentalRibbonRepresentations: config.animation.experimentalWindowRepresentations) } else { controller.toggle(snapshot: snapshot, layout: layout, borders: config.decorations.borders, animation: config.animation, zoom: config.overview.zoom, windowCornerRadius: config.overview.windowCornerRadius, windowPreviewsEnabled: config.overview.windowPreviews, + experimentalSurfaceTransitions: config.overview.experimentalSurfaceTransitions, ribbonPrototype: request.ribbonPrototype) } } else { @@ -177,6 +211,7 @@ extension Daemon { private func presentOverview() { let snapshot = makeOverviewSnapshot(), config = config, layout = state.layout + platform.experimentalSurfaceRibbonEnabled = config.animation.experimentalWindowRepresentations DispatchQueue.main.async { [self] in let controller = overviewController ?? makeOverviewController() overviewController = controller @@ -185,9 +220,15 @@ extension Daemon { borders: config.decorations.borders, animation: config.animation, zoom: config.overview.zoom, windowCornerRadius: config.overview.windowCornerRadius, - windowPreviewsEnabled: config.overview.windowPreviews) + windowPreviewsEnabled: config.overview.windowPreviews, + experimentalSurfaceTransitions: config.overview.experimentalSurfaceTransitions, + experimentalRibbonRepresentations: config.animation.experimentalWindowRepresentations) } else { - controller.prepare(windowPreviewsEnabled: config.overview.windowPreviews) + controller.prepare(windowPreviewsEnabled: config.overview.windowPreviews, + snapshot: snapshot, layout: layout, zoom: config.overview.zoom, + experimentalSurfaceTransitions: config.overview.experimentalSurfaceTransitions, + experimentalRibbonRepresentations: config.animation.experimentalWindowRepresentations, + windowCornerRadius: config.overview.windowCornerRadius) } publishOverviewState() } @@ -209,13 +250,20 @@ extension Daemon { @MainActor private func makeOverviewController() -> OverviewController { OverviewController( focusWindow: { [weak self] windowID, appID, monitorID, workspaceID in + let overviewGeneration = self?.overviewController?.sessionGeneration + let offsets = self?.overviewController?.scrollOffsets ?? [:] NavigationActor.enqueue { self?.focusFromOverview(windowID: windowID, appID: appID, - monitorID: monitorID, workspaceID: workspaceID) + monitorID: monitorID, workspaceID: workspaceID, overviewGeneration: overviewGeneration, offsets: offsets) } }, focusWorkspace: { [weak self] monitorID, workspaceID in - NavigationActor.enqueue { self?.focusWorkspaceFromOverview(monitorID: monitorID, workspaceID: workspaceID) } + let overviewGeneration = self?.overviewController?.sessionGeneration + let offsets = self?.overviewController?.scrollOffsets ?? [:] + NavigationActor.enqueue { + self?.focusWorkspaceFromOverview(monitorID: monitorID, workspaceID: workspaceID, + overviewGeneration: overviewGeneration, offsets: offsets) + } }, drop: { [weak self] windowID, appID, sourceMonitorID, sourceWorkspaceID, target in NavigationActor.enqueue { @@ -231,24 +279,48 @@ extension Daemon { }, openStateChanged: { [weak self] isOpen in guard let self else { return } - overviewToggleState.recordControllerState(isOpen) + overviewToggleState.recordControllerState(isOpen, + sessionGeneration: overviewController?.sessionGeneration) overviewInputMode(isOpen) publishOverviewState() let parksWindows = overviewController?.usesWorkspaceParking == true let timestamp = ProcessInfo.processInfo.systemUptime NavigationActor.enqueue { [self] in + overviewExitPreparationActive = false overviewState.isOpen = isOpen + if isOpen { + overviewEditedMonitorIDs.removeAll() + overviewFloatingFrameWriteIDs.removeAll() + } overviewState.usesWorkspaceParking = parksWindows overviewOpenedAt = isOpen ? timestamp : nil - platform.setWindowBordersSuppressed(isOpen) - if parksWindows { - applyCurrentLayout(asynchronousPositions: true, updateVisibility: true, + platform.setOverviewPresentationActive(isOpen) + if parksWindows || (!isOpen && !overviewEditedMonitorIDs.isEmpty) { + applyCurrentLayout(monitorIDs: parksWindows ? nil : overviewEditedMonitorIDs, + asynchronousPositions: true, updateVisibility: true, positionTimeoutSeconds: 0.05, stagesVisibleBeforeParking: !isOpen, + forcingFloatingFrameWritesFor: overviewFloatingFrameWriteIDs, source: isOpen ? "overview-park" : "overview-restore") } + if !isOpen { + overviewEditedMonitorIDs.removeAll() + overviewFloatingFrameWriteIDs.removeAll() + } } }, presentationChanged: { [weak self] in self?.publishOverviewState() }, + idlePreparationRequested: { [weak self] in + NavigationActor.enqueue { [weak self] in self?.updateOverviewIfOpen() } + }, + waitsForNativeSelectionCommit: true, + layoutCommand: { [weak self] command, windowID, appID, monitorID, workspaceID, generation in + NavigationActor.enqueue { [weak self] in + guard let self, overviewToggleState.isCurrentSession(generation) else { return } + _ = editLayoutFromOverview(command, intent: OverviewWindowIntent( + windowID: windowID, expectedAppID: appID, + sourceMonitorID: monitorID, sourceWorkspaceID: workspaceID)) + } + }, commitScrollOffsets: { [weak self] offsets in NavigationActor.enqueue { [weak self] in guard let self else { return } @@ -263,9 +335,33 @@ extension Daemon { ) } + func editLayoutFromOverview(_ command: Command, intent: OverviewWindowIntent) -> CommandResponse { + guard overviewState.isOpen, !overviewExitPreparationActive else { + return .failure("overview is closed or committing selection") + } + do { + let floatingUpdates = try applyOverviewLayoutCommand(command, intent: intent, + viewports: viewportsByMonitor, state: &state) + overviewEditedMonitorIDs.insert(intent.sourceMonitorID) + floatingWindowFrames.merge(floatingUpdates) { _, new in new } + overviewFloatingFrameWriteIDs.formUnion(floatingUpdates.keys) + platform.recordPerformanceTrace("overview-layout window=\(intent.windowID.rawValue) command=\(command)") + commandGeneration &+= 1 + synchronizeScrollOffsets(state: &state, viewports: viewportsByMonitor) + snapScrollOffsetsToTargets() + persistTopology() + updateOverviewIfOpen() + return .success() + } catch { + platform.recordPerformanceTrace("overview-layout-rejected error=\(error)") + updateOverviewIfOpen() + return .failure(String(describing: error)) + } + } + @MainActor func publishOverviewState() { guard let controller = overviewController else { return } - let projection = OverviewPresentationState( + var projection = OverviewPresentationState( isOpen: controller.isOpen, usesWorkspaceParking: controller.usesWorkspaceParking, panelCount: controller.panelCount, retainedPanelCount: controller.retainedPanelCount, permission: controller.previewPermissionState.rawValue, @@ -274,7 +370,24 @@ extension Daemon { firstPreviewMs: controller.firstPreviewMs, lastPreviewMs: controller.lastPreviewMs, receivedPreviews: controller.receivedPreviewCount ) - NavigationActor.enqueue { [self] in overviewState = projection } + projection.renderPerformance = controller.renderPerformance + projection.sessionGeneration = controller.sessionGeneration + projection.surfaceCapture = controller.surfaceCaptureState + projection.surfaceStreams = controller.surfaceStreamCount + projection.surfacePoolBytes = controller.surfaceEstimatedPoolBytes + projection.surfaceTransitions = controller.surfaceTransitionCount + projection.surfaceFallbacks = controller.surfaceFallbackCount + projection.previewClosings = controller.previewClosingCount + projection.surfaceAcquireMs = controller.surfaceAcquireMs + projection.openingTransitionHistory = controller.openingTransitionHistory + projection.ribbonSurfaceTransitions = controller.ribbonSurfaceTransitions + projection.ribbonSurfaceFallbacks = controller.ribbonSurfaceFallbacks + projection.ribbonSurfaceFallbackReason = controller.ribbonSurfaceFallbackReason + projection.ribbonSurfacePresenting = controller.ribbonSurfacePresenting + NavigationActor.enqueue { [self] in + overviewState = projection + if overviewExitPreparationActive { overviewState.usesWorkspaceParking = false } + } } private func makeOverviewSnapshot() -> OverviewSnapshot { @@ -294,8 +407,12 @@ extension Daemon { windowID: WindowID, appID: String, monitorID: MonitorID, - workspaceID: WorkspaceID + workspaceID: WorkspaceID, + overviewGeneration: UInt64?, + offsets: [MonitorID: [WorkspaceID: Double]] ) { + guard overviewToggleState.isCurrentSession(overviewGeneration) else { return } + overviewEditedMonitorIDs.formUnion(applyOverviewScrollOffsets(offsets, state: &state)) let previousSelectedWindowID = activeMonitorID.flatMap { state.selectedWindowID(on: $0) } @@ -322,6 +439,11 @@ extension Daemon { invalidateSubmittedWorkspaceFocus() focus.queueWorkspace(nil) synchronizeScrollOffsets(state: &state, viewports: viewportsByMonitor) + if overviewState.isOpen { + prepareOverviewExit(on: focusedMonitorID, selectedWindowID: windowID, + inputTimestamp: inputTimestamp, overviewGeneration: overviewGeneration) + return + } startScrollAnimationsIfNeeded() let animated = dispatchScrollAnimationIfNeeded( monitorIDs: [focusedMonitorID] @@ -368,13 +490,18 @@ extension Daemon { } catch { platform.recordPerformanceTrace("overview-focus-rejected error=\(error)") updateOverviewIfOpen() + finishOverviewExit(overviewGeneration: overviewGeneration) } } private func focusWorkspaceFromOverview( monitorID: MonitorID, - workspaceID: WorkspaceID + workspaceID: WorkspaceID, + overviewGeneration: UInt64?, + offsets: [MonitorID: [WorkspaceID: Double]] ) { + guard overviewToggleState.isCurrentSession(overviewGeneration) else { return } + overviewEditedMonitorIDs.formUnion(applyOverviewScrollOffsets(offsets, state: &state)) let inputTimestamp = ProcessInfo.processInfo.systemUptime do { let selectedWindowID = try focusOverviewWorkspace( @@ -391,24 +518,77 @@ extension Daemon { latestCommandInputTimestamp = inputTimestamp platform.userInputTracker.record(timestamp: inputTimestamp) synchronizeScrollOffsets(state: &state, viewports: viewportsByMonitor) - snapScrollOffsetsToTargets() - applyCurrentLayout( - monitorIDs: [monitorID], - asynchronousPositions: true, - updateVisibility: true, - positionTimeoutSeconds: 0.05, - stagesVisibleBeforeParking: true, - focusWindowIDAfterCommit: selectedWindowID, - focusInputTimestampAfterCommit: selectedWindowID == nil - ? nil - : inputTimestamp, - source: "overview-workspace" - ) - persistTopology() - updateMenuBar() - updateOverviewIfOpen() + prepareOverviewExit(on: monitorID, selectedWindowID: selectedWindowID, + inputTimestamp: inputTimestamp, overviewGeneration: overviewGeneration) } catch { platform.recordPerformanceTrace("overview-workspace-rejected error=\(error)") + updateOverviewIfOpen() + finishOverviewExit(overviewGeneration: overviewGeneration) + } + } + + private func prepareOverviewExit( + on monitorID: MonitorID, selectedWindowID: WindowID?, inputTimestamp: TimeInterval, + overviewGeneration: UInt64? + ) { + // The overview covers these writes. Prepare final native geometry once, + // rather than animating or parking real windows on every overview arrow. + let restoresAllMonitors = overviewState.usesWorkspaceParking + let affectedMonitorIDs = overviewEditedMonitorIDs.union([monitorID]) + overviewExitPreparationActive = true + overviewState.usesWorkspaceParking = false + scrollAnimations = scrollAnimations.filter { !restoresAllMonitors && !affectedMonitorIDs.contains($0.key.monitorID) } + for index in state.monitors.indices where restoresAllMonitors || affectedMonitorIDs.contains(state.monitors[index].id) { + for workspaceIndex in state.monitors[index].workspaces.indices { + state.monitors[index].workspaces[workspaceIndex].scrollOffset = + state.monitors[index].workspaces[workspaceIndex].targetScrollOffset + } + } + let generation = commandGeneration + applyCurrentLayout(monitorIDs: restoresAllMonitors ? nil : affectedMonitorIDs, asynchronousPositions: true, + updateVisibility: true, positionTimeoutSeconds: 0.05, + stagesVisibleBeforeParking: true, focusWindowIDAfterCommit: selectedWindowID, + focusInputTimestampAfterCommit: selectedWindowID == nil ? nil : inputTimestamp, + cursorWarpWindowIDAfterCommit: config.input.mouseFollowsFocus ? selectedWindowID : nil, + cursorWarpInputTimestampAfterCommit: config.input.mouseFollowsFocus ? inputTimestamp : nil, + cursorWarpIsCurrentAfterCommit: { [weak self] in + guard let self else { return false } + return overviewToggleState.selectionWarpIsCurrent(overviewGeneration) + && commandGeneration == generation && state.selectedWindowID(on: monitorID) == selectedWindowID + }, + forcingFloatingFrameWritesFor: overviewFloatingFrameWriteIDs, + source: "overview-exit-prepare") + persistTopology() + updateMenuBar() + updateOverviewIfOpen() + let windowIDs = Set(state.monitors.filter { restoresAllMonitors || affectedMonitorIDs.contains($0.id) }.flatMap { + $0.workspaces.flatMap { $0.columns.flatMap(\.windows) + $0.floatingWindows } + }) + Task { @NavigationActor [weak self] in + for attempt in 0..<150 { + guard let self, overviewToggleState.isCurrentSession(overviewGeneration) else { return } + if commandGeneration != generation { + finishOverviewExit(overviewGeneration: overviewGeneration) + return + } + let ready = platform.overviewExitFramesReady(windowIDs: windowIDs) + if ready || attempt == 149 { + if !ready { platform.recordPerformanceTrace("overview-exit-prepare-timeout") } + finishOverviewExit(overviewGeneration: overviewGeneration, nativeFramesReady: ready) + return + } + try? await Task.sleep(for: .milliseconds(10)) + } + } + } + + private func finishOverviewExit(overviewGeneration: UInt64?, nativeFramesReady: Bool = false) { + DispatchQueue.main.async { [weak self] in + if nativeFramesReady, let overviewGeneration, + self?.overviewController?.sessionGeneration == overviewGeneration { + self?.overviewToggleState.completeSelectionExit(overviewGeneration) + } + self?.overviewController?.selectionCommitCompleted(sessionGeneration: overviewGeneration, nativeFramesReady: nativeFramesReady) } } @@ -470,6 +650,20 @@ extension Daemon { } struct OverviewPresentationState: Sendable { + var sessionGeneration: UInt64 = 0 + var renderPerformance = "none" + var surfaceCapture = "disabled" + var surfaceStreams = 0 + var surfacePoolBytes = 0 + var surfaceTransitions = 0 + var surfaceFallbacks = 0 + var previewClosings = 0 + var surfaceAcquireMs: Double = 0 + var openingTransitionHistory = "none" + var ribbonSurfaceTransitions = 0 + var ribbonSurfaceFallbacks = 0 + var ribbonSurfaceFallbackReason = "none" + var ribbonSurfacePresenting = false var isOpen = false var usesWorkspaceParking = false var panelCount = 0 diff --git a/Sources/DefiDaemon/DaemonStatusLifecycle.swift b/Sources/DefiDaemon/DaemonStatusLifecycle.swift index 867043c4..a94970a1 100644 --- a/Sources/DefiDaemon/DaemonStatusLifecycle.swift +++ b/Sources/DefiDaemon/DaemonStatusLifecycle.swift @@ -172,6 +172,9 @@ extension Daemon { return "running session=\(desktopSessionActive ? "active" : "inactive") monitors=\(latestMonitors.count)[\(displaySizes)] windows=\(managedCount) floating=\(floatingCount) workspace=\(workspace) focused=\(focused) fullscreenSuspended=\(state.nativeFullscreenWindowIDs.count)[\(fullscreenWindowIDs)] columnWidth=\(focusedColumnState) menuBar=\(config.menuBar.enabled ? "installed" : "missing") hotkeys=\(hotKeyState) bindings=\(bindingCount) captured=\(capturedHotKeyCount) processed=\(processedHotKeyCount) queued=\(pendingHotKeyCommands.count) tapReenables=\(tapReenableCount) cheatsheet=\(cheatsheetState.isVisible ? "open" : "closed") overview=\(overviewOpen ? "open" : "closed") overviewPanels=\(overviewPanels) overviewRetainedPanels=\(overviewState.retainedPanelCount) overviewPreviewConfig=\(config.overview.windowPreviews) overviewPermission=\(overviewPermission) overviewCaptures=\(overviewCaptures) overviewPreviews=\(overviewPreviews) overviewPreviewCacheMiB=\(overviewPreviewCacheMiB) overviewPreviewFailures=\(overviewPreviewFailures) overviewFirstPreviewMs=\(overviewFirstPreviewMs) overviewLastPreviewMs=\(overviewLastPreviewMs) overviewReceivedPreviews=\(overviewState.receivedPreviews) events=\(observedPlatformEventCount) focusDedup=\(ignoredRedundantNativeFocusCount) closeFocusPreserved=\(preservedWindowRemovalFocusCount) displayEvents=\(displayConfigurationEventCount) displayRetries=\(pendingDisplaySyncDeadlines.count) drift=\(targetMismatches.count)[\(driftDetails)] resize=\(resize) visibility=\(visibility) hidden=\(platform.hiddenWindowCount) borders=\(borders.visible) borderNodes=\(borders.allocated) borderDormant=\(borders.dormant) borderOpacity=\(borderOpacity) borderSurfaceMiB=\(borderSurfaceMiB) borderCapture=\(borders.captureEnabled) borderPlans=\(borders.appliedPlans) borderSkips=\(borders.skippedPlans) borderGeometry=\(borders.geometryUpdates) snapshots=\(snapshotPerformance.full)/\(snapshotPerformance.incremental)/\(snapshotPerformance.cached) appInventories=\(snapshotPerformance.applicationInventories) snapshotMs=\(snapshotMS) snapshotMaxMs=\(snapshotMaxMS) snapshotCG=\(snapshotPerformance.cgCopies)/\(snapshotCGMS)/\(snapshotCGMaxMS) axReads=\(attributeReads.batched)/\(attributeReads.fallback) parkingChecks=\(parking.checks) parkingRepairs=\(parking.repairs) initialChecks=\(initialSettlement.checks) initialRepairs=\(initialSettlement.repairs) settling=\(frameCommit.settling) deferredCommits=\(frameCommit.deferred) observedCommits=\(frameCommit.observed) observedCommitMaxMs=\(observedCommitMaxMS) slowApps=\(platform.latencySensitiveProcessCount) slowAppDetails=[\(platform.latencySensitiveProcessDescription)] axAppDetails=[\(platform.processLatencyDescription)] posWrites=\(platform.successfulPositionWriteCount) stalePos=\(platform.skippedStalePositionWriteCount) droppedFrames=\(platform.droppedPositionFrameCount) displayedRebases=\(displayedFrameRebaseCount) displayedDelta=\(displayedRebaseDelta) sizeWrites=\(platform.successfulSizeWriteCount) displayHz=\(displayHz) timerHz=\(timerHz) axPending=\(platform.hasPendingAnimatedFrameWrites) axFrameMs=\(axFrameMS) axFrameMaxMs=\(axFrameMaxMS) axSlowFrames=\(axFramePerformance.slowFrames) focusPending=\(platform.hasPendingFocusWrite) focusFast=\(focusPerformance.fastPaths) focusCancelled=\(focusPerformance.cancelled) focusRetries=\(focusPerformance.retries) focusMainMs=\(focusMainMS) focusRaiseMs=\(focusRaiseMS) focusActivateMs=\(focusActivateMS) animating=\(platform.hasPendingAnimatedFrameWrites) animationFrames=\(axFramePerformance.animationFrames) animationMs=\(coordinatorAnimationMS) commandMs=\(commandMS) frameMs=\(frameMS) focusMs=\(focusMS)" + " displayArrangement=\(displayArrangementStatus) displayPointerWarps=\(displayPointerRouter.warpCount)" + + " overviewRender=\(overviewState.renderPerformance) overviewSurface=\(overviewState.surfaceCapture) overviewSurfaceStreams=\(overviewState.surfaceStreams) overviewSurfacePoolMiB=\(String(format: "%.2f", Double(overviewState.surfacePoolBytes) / 1_048_576)) overviewSurfaceTransitions=\(overviewState.surfaceTransitions) overviewSurfaceFallbacks=\(overviewState.surfaceFallbacks) overviewPreviewClosings=\(overviewState.previewClosings) overviewSurfaceAcquireMs=\(String(format: "%.1f", overviewState.surfaceAcquireMs))" + + " overviewOpening=\(overviewState.openingTransitionHistory)" + + " ribbonSurfaceTransitions=\(overviewState.ribbonSurfaceTransitions) ribbonSurfaceFallbacks=\(overviewState.ribbonSurfaceFallbacks) ribbonSurfaceFallbackReason=\(overviewState.ribbonSurfaceFallbackReason) ribbonSurfacePresenting=\(overviewState.ribbonSurfacePresenting)" + " topologyObservers=\(platform.hasReliableWindowTopologyObservation) appWindowLists=\(snapshotPerformance.applicationWindowListReads)" + " appLifecycleObservers=\(platform.hasReliableApplicationLifecycleObservation) appInventoryInterval=\(Int(platform.recommendedApplicationInventoryRefreshInterval))" + " desktopObservers=\(platform.hasReliableDesktopObservation)" diff --git a/Sources/DefiDaemon/DefiDaemon.swift b/Sources/DefiDaemon/DefiDaemon.swift index 4347f2ff..ce5e76a9 100644 --- a/Sources/DefiDaemon/DefiDaemon.swift +++ b/Sources/DefiDaemon/DefiDaemon.swift @@ -118,6 +118,9 @@ final class Daemon { @MainActor var cheatsheetController: CheatsheetController? var cheatsheetHoldTask: Task? nonisolated let hotKeyGeneration = Mutex(0) + var overviewEditedMonitorIDs: Set = [] + var overviewFloatingFrameWriteIDs: Set = [] + var overviewExitPreparationActive = false var overviewOpenedAt: TimeInterval? nonisolated let menuBar: MenuBarState var lastPublishedWorkspaceState: WorkspaceStateSnapshot? @@ -169,6 +172,7 @@ final class Daemon { var lastAnimationDurationMS = 0.0 var lastCommandDurationMS = 0.0 var latestCommandInputTimestamp: TimeInterval = 0 + var latestNativeFocusAnimationInputTimestamp: TimeInterval = 0 var commandGeneration: UInt64 = 0 var deferredMouseFocusIntent: DeferredMouseFocusIntent? var consumedMouseFocusIntentTimestamp: TimeInterval = 0 diff --git a/Sources/DefiMacOS/AXFrameCoordinatorAnimation.swift b/Sources/DefiMacOS/AXFrameCoordinatorAnimation.swift index c5f7de64..162a7fac 100644 --- a/Sources/DefiMacOS/AXFrameCoordinatorAnimation.swift +++ b/Sources/DefiMacOS/AXFrameCoordinatorAnimation.swift @@ -28,10 +28,23 @@ extension AXFrameCoordinator { let animatedWrites = frame.writes.filter { frame.animatedWindowIDs.contains($0.key) } + var admittedFrame = frame + if frame.source == "command-layout-animation", + frame.monitorFrames.count == 1, frame.displayIDs.count <= 1 + { + admittedFrame.animationDuration = horizontalAnimationDuration( + for: animatedWrites, requested: frame.animationDuration, + refreshRateHz: frame.refreshRateHz, allowsSizeChanges: true) + } + let frame = admittedFrame let staticWrites = frame.writes.filter { !animatedWrites.keys.contains($0.key) } - let deferredParkingWrites = staticWrites.filter { $0.value.isParked } + let parkingPreparationIDs = ribbonParkingPreparationWindowIDs(frame) + let parkingPreparationWrites = staticWrites.filter { parkingPreparationIDs.contains($0.key) } + let deferredParkingWrites = staticWrites.filter { + $0.value.isParked && !parkingPreparationIDs.contains($0.key) + } let blockingStaticWrites = staticWrites.filter { !$0.value.isParked } // Horizontal motion uses live lane readiness at each display pulse. A // historical AX stall must not decimate an entire later ribbon animation. @@ -104,6 +117,7 @@ extension AXFrameCoordinator { } let sizeCommitCandidates = interpolatedWrites.filter { !lanePlan.deferredSizeWindowIDs.contains($0.key) + && !$0.value.animatesSize && !$0.value.isReentering && !$0.value.requiresVerifiedOffscreenWrite && asynchronousSizeWriteIsRequired( @@ -142,6 +156,20 @@ extension AXFrameCoordinator { ) var applied = 0 var stale = 0 + // A superseded native sample can leave an already-logically-offscreen + // column visible. Clear that column before its neighbor moves through it. + if !parkingPreparationWrites.isEmpty, isCurrent(generation: frame.generation) { + let preparationFrame = QueuedPositionFrame( + generation: frame.generation, source: frame.source, + writes: parkingPreparationWrites, animatedWindowIDs: [], animationDuration: 0, + refreshRateHz: frame.refreshRateHz, displayIDs: frame.displayIDs, + monitorFrames: frame.monitorFrames, initialProgressVelocity: 0, + stagesVisibleBeforeParking: false, successfulWrite: frame.successfulWrite, + completion: nil) + let result = applyFrame(preparationFrame, progress: 1, skippedProcesses: []) + applied += result.applied + stale += result.stale + } let stagingGroup = DispatchGroup() let stagingAccumulator = FrameResultAccumulator() let reentryWrites = loopWrites.filter { $0.value.isReentering } @@ -545,7 +573,7 @@ extension AXFrameCoordinator { for windowID in windowIDs ?? frame.animatedWindowIDs { guard let write = frame.writes[windowID] else { continue } retargetHorizontalVelocities[windowID] = - (write.point.x - write.fromPoint.x) * progressVelocity + ((write.animationPoint ?? write.point).x - write.fromPoint.x) * progressVelocity } lock.unlock() } diff --git a/Sources/DefiMacOS/AXFrameCoordinatorQueue.swift b/Sources/DefiMacOS/AXFrameCoordinatorQueue.swift index 7af30fe1..c21eadfc 100644 --- a/Sources/DefiMacOS/AXFrameCoordinatorQueue.swift +++ b/Sources/DefiMacOS/AXFrameCoordinatorQueue.swift @@ -134,19 +134,37 @@ extension AXFrameCoordinator { positionChanged: write.positionChanged, sizeChanged: write.sizeChanged, animatesSize: write.animatesSize, + usesLogicalRibbonPath: write.usesLogicalRibbonPath, synchronousSizeWriteSucceeded: write.synchronousSizeWriteSucceeded, enhancedUIWasEnabled: write.enhancedUIWasEnabled, timeoutSeconds: write.timeoutSeconds, isParked: write.isParked, isReentering: write.isReentering, - requiresVerifiedOffscreenWrite: write.requiresVerifiedOffscreenWrite + requiresVerifiedOffscreenWrite: write.requiresVerifiedOffscreenWrite, + animationPoint: write.animationPoint, + usesCommonRibbonOffset: write.usesCommonRibbonOffset ) count += 1 } + let ribbonWrites = writes.filter { $0.value.usesCommonRibbonOffset } + if frame.monitorFrames.count == 1, + let offset = commonRibbonOffset( + targets: ribbonWrites.mapValues { frameAnimationDestination($0, intermediate: true) }, + starts: ribbonWrites.filter { !$0.value.isReentering }.mapValues(\.fromPoint), + sizes: ribbonWrites.mapValues(\.fromSize), monitor: frame.monitorFrames[0]) + { + for (id, write) in ribbonWrites { + var aligned = write + let destination = frameAnimationDestination(write, intermediate: true) + aligned.fromPoint = CGPoint(x: destination.x + offset, y: write.fromPoint.y) + writes[id] = aligned + if aligned.fromPoint != write.fromPoint { count += 1 } + } + } // Reentry's logical origin must follow neighbors that finished moving while // this command was queued; its native parking point is not a strip origin. if frame.source == "command-animation", frame.monitorFrames.count == 1 { - for (windowID, write) in writes where write.isReentering && !write.sizeChanged + for (windowID, write) in writes where write.isReentering && !write.usesCommonRibbonOffset && !write.sizeChanged && write.fromPoint.y == write.point.y { let neighbor = writes.filter { @@ -170,7 +188,7 @@ extension AXFrameCoordinator { guard let write = writes[windowID], let previousVelocity = retargetHorizontalVelocities[windowID] else { return nil } - let delta = write.point.x - write.fromPoint.x + let delta = (write.animationPoint ?? write.point).x - write.fromPoint.x guard abs(delta) >= 0.5 else { return nil } return previousVelocity / delta } diff --git a/Sources/DefiMacOS/AXFrameCoordinatorWrites.swift b/Sources/DefiMacOS/AXFrameCoordinatorWrites.swift index f3d17177..60fc5e4f 100644 --- a/Sources/DefiMacOS/AXFrameCoordinatorWrites.swift +++ b/Sources/DefiMacOS/AXFrameCoordinatorWrites.swift @@ -144,7 +144,22 @@ extension AXFrameCoordinator { }, by: \.value.processID ).map { - ProcessWriteBatch(processID: $0.key, writes: $0.value) + var entries = $0.value + let deltas = entries.map { + frameAnimationDestination($0.value, intermediate: true).x - $0.value.fromPoint.x + } + if entries.allSatisfy({ $0.value.usesCommonRibbonOffset }), + let delta = deltas.first, abs(delta) >= 0.5, + deltas.allSatisfy({ $0 * delta > 0 }) + { + // AX writes are sequential within an application. Release space in + // front of the strip before advancing its following native windows. + entries.sort { + let a = $0.value.fromPoint.x, b = $1.value.fromPoint.x + return a == b ? $0.key.rawValue < $1.key.rawValue : (delta < 0 ? a < b : a > b) + } + } + return ProcessWriteBatch(processID: $0.key, writes: entries) }.sorted { $0.processID < $1.processID } } @@ -393,6 +408,31 @@ extension AXFrameCoordinator { ) } + func horizontalAnimationDuration( + for writes: [WindowID: AsyncPositionWrite], + requested: TimeInterval, + refreshRateHz: Double, + allowsSizeChanges: Bool = false + ) -> TimeInterval { + guard requested > 0, !writes.isEmpty, + writes.values.allSatisfy({ + (!$0.animatesSize || allowsSizeChanges) + && ($0.usesCommonRibbonOffset || (allowsSizeChanges && $0.usesLogicalRibbonPath) + || abs($0.point.y - $0.fromPoint.y) < 0.5) }) + else { return requested } + lock.lock() + let latency = writes.values.map { + (recentIntermediateProcessLatencySamplesMS[$0.processID]?.map(\.latencyMS).max() + ?? predictedProcessLatencyMS[$0.processID] ?? 0) / 1_000 + }.max() ?? 0 + lock.unlock() + // Give every participating native lane two intermediate writes plus its + // final write. A bounded common timeline avoids replacing the whole strip + // with a jump after a recent stall, without queuing more AX work. + let interval = 1 / min(max(refreshRateHz, 30), 120) + return max(requested, min(0.4, 3 * latency + 2 * interval)) + } + func animationSupportsIntermediateFrames( processIDs: Set, animationDuration: TimeInterval, @@ -607,7 +647,8 @@ extension AXFrameCoordinator { stale += batch.writes.count - index break } - var interpolated = interpolatedFrame( + let destination = frameAnimationDestination(item.value, intermediate: intermediate) + let interpolated = interpolatedFrame( from: Rect( x: item.value.fromPoint.x, y: item.value.fromPoint.y, @@ -615,25 +656,38 @@ extension AXFrameCoordinator { height: item.value.fromSize.height ), to: Rect( - x: item.value.point.x, - y: item.value.point.y, + x: destination.x, + y: destination.y, width: item.value.size.width, height: item.value.size.height ), progress: progress ) - // Keep logical strip origins intact; native windows park at a reachable anchor. - // ponytail: single-display projection; retain conservative multi-display staging. - if (intermediate || stagingReentry), frame.source == "command-animation", - item.value.fromPoint.y == item.value.point.y, - let monitor = frame.monitorFrames.first, frame.monitorFrames.count == 1 + let nativeRibbonSample = (intermediate || stagingReentry) + && (item.value.usesCommonRibbonOffset || item.value.usesLogicalRibbonPath) + && frame.monitorFrames.count == 1 + let ribbonSample = item.value.usesCommonRibbonOffset + ? Rect(x: interpolated.x, y: item.value.fromPoint.y, + width: item.value.fromSize.width, height: item.value.fromSize.height) + : interpolated + let projectsNativeStrip = (intermediate || stagingReentry) + && frame.source == "command-animation" + && item.value.fromPoint.y == item.value.point.y + && frame.monitorFrames.count == 1 + let nativeFrame = (nativeRibbonSample || projectsNativeStrip) ? nativeRibbonAnimationFrame( + ribbonSample, + monitor: frame.monitorFrames[0]) : interpolated + let point = CGPoint(x: nativeFrame.x, y: nativeFrame.y) + let parksRibbonSample = nativeRibbonSample && requiresVerifiedOffscreenWrite( + frame: nativeFrame, monitorFrames: frame.monitorFrames) + // A distant logical column stays at its verified strip anchor. Do not + // send identical offscreen positions at every display refresh. + if parksRibbonSample, !stagingReentry, !item.value.sizeChanged, !item.value.animatesSize, + let completed = completedPosition(for: item.key), + accessibilityWriter.pointDistance(completed, point) < 0.5 { - interpolated = continuousStripFramesForActiveWorkspace( - [FrameAssignment(windowID: item.key, frame: interpolated)], viewport: monitor, - preservingExitSide: true - ).frames[0].frame + continue } - let point = CGPoint(x: interpolated.x, y: interpolated.y) let size = CGSize( width: interpolated.width, height: interpolated.height @@ -726,17 +780,10 @@ extension AXFrameCoordinator { && ((!intermediate && progress >= 1) || readsLiveBorderPosition) ? accessibilityWriter.readSize(item.value.element) : nil - let clampedSourceFrame: Rect? = acceptedSize.flatMap { observedSize in - guard item.value.positionChanged, - abs(observedSize.width - size.width) >= 0.5 - || abs(observedSize.height - size.height) >= 0.5, - let source = accessibilityWriter.readPosition(item.value.element) - else { return nil } - return Rect( - x: source.x, y: source.y, - width: observedSize.width, height: observedSize.height - ) - } + let sizeWasClampedBeforeMove = item.value.positionChanged + && (acceptedSize.map { + abs($0.width - size.width) >= 0.5 || abs($0.height - size.height) >= 0.5 + } ?? false) var nativeStagingPosition: CGPoint? let verifyNativeStage: (() -> Bool)? = stagingReentry && item.value.isReentering && !item.value.isParked @@ -756,7 +803,7 @@ extension AXFrameCoordinator { || accessibilityWriter.applyPosition( item.value, point: point, - forceOffscreenAccess: (stagingReentry && item.value.isReentering) + forceOffscreenAccess: parksRibbonSample || (stagingReentry && item.value.isReentering) || (!intermediate && item.value.requiresVerifiedOffscreenWrite), verifyParkedPosition: !intermediate, enhancedUIManagedByBatch: managesEnhancedUI @@ -767,15 +814,11 @@ extension AXFrameCoordinator { ) let positionDurationMS = (ProcessInfo.processInfo.systemUptime - positionStartedAt) * 1_000 - // AppKit can clamp a resize to the source display before accepting - // the move. Retry once at the destination, only after a measured clamp. - if positionApplied, - let clampedSourceFrame, - frameCentersCrossDisplays( - from: clampedSourceFrame, - to: interpolated, - displayFrames: frame.monitorFrames - ), + // AppKit can clamp a resize at the source position, even on the same + // display. Retry the final size once after moving, only after a measured + // mismatch; intermediate samples must keep their bounded write budget. + if !intermediate, progress >= 1, positionApplied, + sizeWasClampedBeforeMove, isCurrent(generation: frame.generation) { sizeApplied = accessibilityWriter.applySize( @@ -788,7 +831,7 @@ extension AXFrameCoordinator { nativeStagingPosition = nil positionApplied = accessibilityWriter.applyPosition( item.value, point: point, - forceOffscreenAccess: (stagingReentry && item.value.isReentering) + forceOffscreenAccess: parksRibbonSample || (stagingReentry && item.value.isReentering) || (!intermediate && item.value.requiresVerifiedOffscreenWrite), verifyParkedPosition: !intermediate, enhancedUIManagedByBatch: managesEnhancedUI || defersEnhancedUIRestore, @@ -879,14 +922,13 @@ extension AXFrameCoordinator { } // A superseded write can still have moved the native window. Keep that // physical starting point; process lanes serialize it before replacement. - let requiresReadback = - item.value.isParked - || item.value.requiresVerifiedOffscreenWrite + let requiresReadback = !intermediate + && (item.value.isParked || item.value.requiresVerifiedOffscreenWrite) if positionApplied, item.value.positionChanged { let completedPoint = acceptedPosition ?? point recordCompletedPosition( completedPoint, windowID: item.key, - positionWasReadBack: !defersBorderReadback || acceptedPosition != nil + positionWasReadBack: acceptedPosition != nil || requiresReadback ) } else if let acceptedPosition { recordCompletedPosition(acceptedPosition, windowID: item.key) diff --git a/Sources/DefiMacOS/ExperimentalRibbonRenderer.swift b/Sources/DefiMacOS/ExperimentalRibbonRenderer.swift new file mode 100644 index 00000000..d0eae656 --- /dev/null +++ b/Sources/DefiMacOS/ExperimentalRibbonRenderer.swift @@ -0,0 +1,353 @@ +import AppKit +import AVFoundation +import DefiCore +import DefiModel +import QuartzCore + +// Opt-in surface presentation only. Native geometry and focus remain owned by +// the regular frame coordinator; this scene never writes to user windows. +@MainActor +final class ExperimentalRibbonRenderer { + static let shared = ExperimentalRibbonRenderer() + private let visibleWindowInfo: () -> [[String: Any]] + + init(visibleWindowInfo: @escaping () -> [[String: Any]] = { + CGWindowListCopyWindowInfo([.optionOnScreenOnly, .excludeDesktopElements], + kCGNullWindowID) as? [[String: Any]] ?? [] + }) { + self.visibleWindowInfo = visibleWindowInfo + } + + private struct Context { + let monitorID: MonitorID + let workspaceID: WorkspaceID + let frames: [WindowID: Rect] + let owners: [WindowID: Int32] + let screen: NSScreen + let viewport: Rect + } + private var contexts: [Context] = [] + private var backgrounds: [MonitorID: CGImage] = [:] + private var backgroundTask: Task? + private var scenes: [MonitorID: NSPanel] = [:] + private var cornerRadius: Double = 12 + private var presentedLayers: [WindowID: (CALayer, Rect)] = [:] + var isPresenting: Bool { !scenes.isEmpty } + var backgroundsReady: Bool { contexts.allSatisfy { backgrounds[$0.monitorID] != nil } } + private var completion: Task? + private var expectedNativeTargets: [WindowID: Rect] = [:] + private var generation: UInt64 = 0 + private(set) var requests: [OverviewSurfaceRequest] = [] + private(set) var transitions = 0 + private(set) var fallbacks = 0 + private(set) var lastFallback = "none" + + func backgroundImage(for monitorID: MonitorID) -> CGImage? { backgrounds[monitorID] } + + func disable() { + cancel(); contexts = []; requests = []; backgrounds = [:] + backgroundTask?.cancel(); backgroundTask = nil + } + + func capturedIDs(on monitorID: MonitorID) -> Set { + Set(requests.map(\.windowID)).intersection(contexts.first { + $0.monitorID == monitorID + }?.frames.keys.map { $0 } ?? []) + } + + func prepare(snapshot: OverviewSnapshot, layout: LayoutSettings, enabled: Bool, + cornerRadius: Double = 12, preloadWorkspace: Bool = false) { + self.cornerRadius = cornerRadius + guard enabled else { + disable() + return + } + var next: [Context] = [] + var candidates: [(request: OverviewSurfaceRequest, distance: Double)] = [] + for monitor in snapshot.monitors { + guard let screen = NSScreen.screens.first(where: { + ($0.deviceDescription[NSDeviceDescriptionKey("NSScreenNumber")] as? NSNumber)?.uint64Value == monitor.id.rawValue + }), let viewport = snapshot.monitorFrames[monitor.id], + let workspace = monitor.workspaces.first(where: { $0.id == monitor.activeWorkspace }), + workspace.floatingWindows.isEmpty, + Set(workspace.columns.flatMap(\.windows)).isDisjoint(with: snapshot.nativeFullscreenWindowIDs) + else { continue } + let frames = Dictionary(uniqueKeysWithValues: computeLayout(workspace: workspace, + viewport: viewport, windows: Array(snapshot.windows.values), settings: layout) + .map { ($0.windowID, $0.frame) }) + let owners = snapshot.windows.compactMapValues(\.processID) + next.append(Context(monitorID: monitor.id, workspaceID: workspace.id, + frames: frames, owners: owners, screen: screen, viewport: viewport)) + let visible = frames.filter { $0.value.ribbonIntersects(viewport) } + let offscreen = frames.filter { !visible.keys.contains($0.key) } + let ordered = offscreen.sorted { + let left = distance($0.value, viewport), right = distance($1.value, viewport) + return left == right ? $0.key.rawValue < $1.key.rawValue : left < right + } + let neighbours = [ordered.first(where: { $0.value.x + $0.value.width <= viewport.x }), + ordered.first(where: { $0.value.x >= viewport.x + viewport.width })].compactMap { $0 } + for (id, frame) in Array(visible) + (preloadWorkspace ? ordered : Array(neighbours)) { + guard let window = snapshot.windows[id], let owner = owners[id], + frame.width.isFinite, frame.height.isFinite, frame.width > 0, frame.height > 0 + else { continue } + // Keep visible windows and immediate neighbours sharp; NV12 reduces the + // capture pool without reducing luma resolution. + let scale = screen.backingScaleFactor + let width = window.frame.width * scale, height = window.frame.height * scale + guard width <= 16_384, height <= 16_384 else { continue } + candidates.append((OverviewSurfaceRequest(windowID: id, appID: window.appID, + processID: owner, width: max((Int(width.rounded(.up)) + 1) / 2 * 2, 2), + height: max((Int(height.rounded(.up)) + 1) / 2 * 2, 2)), visible[id] == nil ? distance(frame, viewport) + 1 : 0)) + } + } + contexts = next + let required = candidates.filter { $0.distance == 0 }.map(\.request) + var selected = required + for item in candidates.filter({ $0.distance > 0 }).sorted(by: { $0.distance < $1.distance }) { + if overviewSurfaceRequestsFit(selected + [item.request]) { selected.append(item.request) } + } + requests = selected.sorted { $0.windowID.rawValue < $1.windowID.rawValue } + let ids = Set(next.map(\.monitorID)) + backgrounds = backgrounds.filter { id, image in + guard ids.contains(id), let context = next.first(where: { $0.monitorID == id }) else { return false } + return image.width == Int(context.screen.frame.width) && image.height == Int(context.screen.frame.height) + } + guard CGPreflightScreenCaptureAccess(), backgroundTask == nil, next.contains(where: { backgrounds[$0.monitorID] == nil }) else { return } + let desktopRequests = next.filter { backgrounds[$0.monitorID] == nil }.map { + OverviewDesktopCaptureRequest(monitorID: $0.monitorID, + displayID: UInt32($0.monitorID.rawValue), width: Int($0.screen.frame.width), + height: Int($0.screen.frame.height)) + } + backgroundTask = Task { [weak self] in + let result = await captureOverviewImages(previews: [], desktops: desktopRequests, previewCompleted: { _ in }) + guard !Task.isCancelled, let self else { return } + backgrounds.merge(result.desktops) { _, new in new } + backgroundTask = nil + } + } + + // Runs synchronously on the main actor before the native submission starts. + // False leaves the original native animation fully functional. + func begin(assignments: [FrameAssignment], ribbonFrames: [FrameAssignment] = [], duration: TimeInterval, + borderStyle: WindowBorderStyle? = nil, selectedWindowID: WindowID? = nil) -> Bool { + let previousPresentation = presentedLayers.mapValues { entry -> CGRect in + (entry.0.presentation()?.frame ?? entry.0.frame).offsetBy(dx: entry.1.x, dy: entry.1.y) + } + guard duration > 0, !contexts.isEmpty, + !NSWorkspace.shared.accessibilityDisplayShouldReduceMotion else { lastFallback = "disabled-or-no-context"; cancel(); return false } + let targets = Dictionary(uniqueKeysWithValues: assignments.map { ($0.windowID, $0.frame) }) + let logicalTargets = Dictionary(uniqueKeysWithValues: + (ribbonFrames.isEmpty ? assignments : ribbonFrames).map { ($0.windowID, $0.frame) }) + var plans: [(Context, Double, Set)] = [] + for context in contexts { + let viewport = context.viewport + guard let destination = logicalTargets.first(where: { + context.frames[$0.key] != nil && $0.value.ribbonOverlapWidth(viewport) > 2 + }), let source = context.frames[destination.key] else { continue } + let delta = destination.value.x - source.x + guard abs(delta) > 0.5 else { continue } + let ids = Set(context.frames.filter { + $0.value.ribbonIntersects(viewport) || shifted($0.value, delta).ribbonIntersects(viewport) + }.keys) + guard backgrounds[context.monitorID] != nil, + ids.allSatisfy({ id in + guard let frame = context.frames[id], let target = logicalTargets[id] else { return false } + return abs(frame.width - target.width) < 1 && abs(frame.height - target.height) < 1 + }), OverviewSurfaceCapture.shared.frames(windowIDs: ids) != nil, + ids.allSatisfy({ OverviewSurfaceCapture.shared.displayLayer(for: $0) != nil }) + else { lastFallback = "capture-or-background-or-size"; fallbacks += 1; cancel(); return false } + plans.append((context, delta, ids)) + } + guard !plans.isEmpty else { lastFallback = "no-moving-plan"; cancel(); return false } + guard plans.allSatisfy({ context, _, ids in + !hasUnrepresentedVisibleWindow(in: context.viewport, represented: ids) + }) else { lastFallback = "unrepresented-window"; fallbacks += 1; cancel(); return false } + let native = nativeFrames() + for (context, _, ids) in plans { + guard let captures = OverviewSurfaceCapture.shared.frames(windowIDs: ids) else { + lastFallback = "missing-capture"; fallbacks += 1; cancel(); return false + } + for id in ids { + guard let actual = native[id], actual.owner == context.owners[id], + let source = context.frames[id], + previousPresentation[id] != nil || !source.ribbonIntersects(screenRect(context.screen)) + || abs(actual.frame.x - source.x) < 3 + else { lastFallback = "native-source-mismatch"; fallbacks += 1; cancel(); return false } + guard let capture = captures[id], + overviewSurfaceMatchesNativeSize(source: capture.sourceSize, native: actual.frame) + else { lastFallback = "capture-size-mismatch"; fallbacks += 1; cancel(); return false } + } + } + generation &+= 1; completion?.cancel(); completion = nil + OverviewSurfaceCapture.shared.freeze() + presentedLayers = [:] + CATransaction.begin(); CATransaction.setDisableActions(true) + var expected: [WindowID: (Rect, Int32)] = [:] + for (context, delta, ids) in plans { + guard let frames = OverviewSurfaceCapture.shared.frames(windowIDs: ids) else { cancel(); return false } + let area = context.viewport + let appKitArea = CGRect(x: area.x, y: (NSScreen.screens.first?.frame.height ?? 0) - area.y - area.height, + width: area.width, height: area.height) + let panel = scenes[context.monitorID] ?? NSPanel(contentRect: appKitArea, + styleMask: [.borderless, .nonactivatingPanel], backing: .buffered, defer: false) + panel.isOpaque = true; panel.backgroundColor = .black; panel.hasShadow = false + panel.ignoresMouseEvents = true; panel.hidesOnDeactivate = false + panel.isReleasedWhenClosed = false; panel.level = .statusBar + panel.collectionBehavior = [.canJoinAllSpaces, .stationary, .ignoresCycle] + panel.setAccessibilityLabel("Defi experimental ribbon") + panel.title = "Defi Experimental Ribbon" + let view = NSView(frame: CGRect(origin: .zero, size: appKitArea.size)) + view.wantsLayer = true + let root = CALayer(); root.frame = view.bounds; root.isGeometryFlipped = true + root.masksToBounds = true; view.layer = root; panel.contentView = view + let backdrop = CALayer(); backdrop.frame = root.bounds + let screenOrigin = screenRect(context.screen) + backdrop.contents = backgrounds[context.monitorID]?.cropping(to: CGRect( + x: area.x - screenOrigin.x, y: area.y - screenOrigin.y, width: area.width, height: area.height)) + root.addSublayer(backdrop) + let origin = context.viewport + for id in ids { + guard let logical = context.frames[id], frames[id] != nil, let actual = native[id] else { continue } + let frame = Rect(x: logical.ribbonIntersects(context.viewport) ? actual.frame.x : logical.x, + y: actual.frame.y, width: actual.frame.width, height: actual.frame.height) + guard let item = OverviewSurfaceCapture.shared.displayLayer(for: id) else { cancel(); return false } + item.removeFromSuperlayer(); item.removeAllAnimations() + item.transform = CATransform3DIdentity + item.anchorPoint = CGPoint(x: 0.5, y: 0.5) + // Reuse the decoded one-shot layer without submitting another video sample. + item.contentsGravity = .resize + item.frame = previousPresentation[id]?.offsetBy(dx: -origin.x, dy: -origin.y) + ?? CGRect(x: frame.x - origin.x, y: frame.y - origin.y, + width: frame.width, height: frame.height) + let container = CALayer(); container.frame = item.frame + item.frame = container.bounds + item.cornerRadius = cornerRadius; item.masksToBounds = true; container.addSublayer(item) + if let borderStyle, let appearance = overviewWindowBorderAppearance( + isSelected: id == selectedWindowID, style: borderStyle, scale: 1) { + let geometry = overviewWindowBorderGeometry( + cardFrame: Rect(x: 0, y: 0, width: frame.width, height: frame.height), + cardRadius: cornerRadius, width: appearance.width, placement: borderStyle.placement) + let border = CAShapeLayer() + border.path = CGPath(roundedRect: CGRect(x: geometry.frame.x, y: geometry.frame.y, + width: geometry.frame.width, height: geometry.frame.height), + cornerWidth: geometry.radius, cornerHeight: geometry.radius, transform: nil) + let color = appearance.color + border.strokeColor = NSColor(srgbRed: CGFloat((color >> 16) & 255) / 255, + green: CGFloat((color >> 8) & 255) / 255, blue: CGFloat(color & 255) / 255, + alpha: CGFloat(windowBorderAlpha(of: color)) / 255).cgColor + border.fillColor = nil; border.lineWidth = appearance.width + container.addSublayer(border) + } + root.addSublayer(container) + let start = container.position + CATransaction.begin(); CATransaction.setDisableActions(true) + container.position = CGPoint(x: (logicalTargets[id]?.x ?? logical.x + delta) - origin.x + frame.width / 2, + y: frame.y - origin.y + frame.height / 2) + CATransaction.commit() + presentedLayers[id] = (container, origin) + let motion = CABasicAnimation(keyPath: "position") + motion.fromValue = NSValue(point: start); motion.toValue = NSValue(point: container.position) + motion.duration = duration; motion.timingFunction = CAMediaTimingFunction(name: .easeInEaseOut) + container.add(motion, forKey: "ribbon") + if let target = targets[id], let owner = context.owners[id] { + expected[id] = (Rect(x: target.x, y: actual.frame.y, + width: actual.frame.width, height: actual.frame.height), owner) + } + } + panel.orderFrontRegardless(); scenes[context.monitorID] = panel + } + CATransaction.commit(); CATransaction.flush() + contexts = contexts.map { context in + Context(monitorID: context.monitorID, workspaceID: context.workspaceID, + frames: context.frames.merging(logicalTargets.filter { context.frames[$0.key] != nil }) { _, new in new }, + owners: context.owners, screen: context.screen, viewport: context.viewport) + } + // Supersession compares logical submissions; native clamping is only relevant + // to the final handoff check below. + expectedNativeTargets = targets.filter { expected[$0.key] != nil } + transitions += 1 + let token = generation + completion = Task { [weak self] in + do { try await Task.sleep(for: .seconds(duration)) } catch { return } + let deadline = CACurrentMediaTime() + 1 + while !Task.isCancelled, let self, generation == token { + let current = nativeFrames() + if expected.allSatisfy({ id, value in + current[id].map { $0.owner == value.1 && matches($0.frame, value.0) } ?? false + }) { cancel(); return } + if CACurrentMediaTime() > deadline { fallbacks += 1; cancel(); return } + do { try await Task.sleep(for: .milliseconds(50)) } catch { return } + } + } + return true + } + + func supersedeIfTargetsChanged(_ assignments: [FrameAssignment]) { + guard isPresenting else { return } + if assignments.contains(where: { assignment in + expectedNativeTargets[assignment.windowID].map { !matches($0, assignment.frame) } ?? false + }) { cancel() } + } + + func cancel() { + generation &+= 1; completion?.cancel(); completion = nil + for panel in scenes.values { panel.orderOut(nil); panel.close() } + scenes = [:]; presentedLayers = [:]; expectedNativeTargets = [:] + } + + private func distance(_ a: Rect, _ b: Rect) -> Double { + max(b.x - a.x - a.width, a.x - b.x - b.width, 0) + } + private func shifted(_ frame: Rect, _ dx: Double) -> Rect { + Rect(x: frame.x + dx, y: frame.y, width: frame.width, height: frame.height) + } + private func screenRect(_ screen: NSScreen) -> Rect { + Rect(x: screen.frame.minX, y: (NSScreen.screens.first?.frame.height ?? 0) - screen.frame.maxY, + width: screen.frame.width, height: screen.frame.height) + } + private func matches(_ a: Rect, _ b: Rect) -> Bool { + abs(a.x - b.x) < 3 && abs(a.y - b.y) < 3 && abs(a.width - b.width) < 3 && abs(a.height - b.height) < 3 + } + private func hasUnrepresentedVisibleWindow(in viewport: Rect, represented: Set) -> Bool { + let infos = visibleWindowInfo() + return infos.contains { info in + guard let id = info[kCGWindowNumber as String] as? NSNumber, + let owner = info[kCGWindowOwnerPID as String] as? NSNumber, + owner.int32Value != NSRunningApplication.current.processIdentifier, + let level = info[kCGWindowLayer as String] as? NSNumber, + level.intValue >= 0, level.intValue < NSWindow.Level.statusBar.rawValue, + let bounds = info[kCGWindowBounds as String] as? [String: Any], + let rect = CGRect(dictionaryRepresentation: bounds as CFDictionary), + rect.width > 1, rect.height > 1 else { return false } + let frame = Rect(x: rect.minX, y: rect.minY, width: rect.width, height: rect.height) + let windowID = WindowID(rawValue: id.uint64Value) + guard frame.ribbonIntersects(viewport), !represented.contains(windowID) else { return false } + // Only known managed windows may contribute a verified one-pixel parking strip. + let managed = contexts.contains { $0.owners[windowID] == owner.int32Value } + return !managed || frame.ribbonOverlapWidth(viewport) > 1.01 + } + } + + private func nativeFrames() -> [WindowID: (frame: Rect, owner: Int32)] { + var result: [WindowID: (Rect, Int32)] = [:] + for info in CGWindowListCopyWindowInfo(.optionAll, kCGNullWindowID) as? [[String: Any]] ?? [] { + guard let number = info[kCGWindowNumber as String] as? NSNumber, + let owner = info[kCGWindowOwnerPID as String] as? NSNumber, + let bounds = info[kCGWindowBounds as String] as? [String: Any], + let frame = CGRect(dictionaryRepresentation: bounds as CFDictionary) else { continue } + result[WindowID(rawValue: number.uint64Value)] = (Rect(x: frame.minX, y: frame.minY, + width: frame.width, height: frame.height), owner.int32Value) + } + return result + } +} + +private extension Rect { + func ribbonOverlapWidth(_ other: Rect) -> Double { + max(min(x + width, other.x + other.width) - max(x, other.x), 0) + } + func ribbonIntersects(_ other: Rect) -> Bool { + x + width > other.x && other.x + other.width > x + && y + height > other.y && other.y + other.height > y + } +} diff --git a/Sources/DefiMacOS/FrameCommit.swift b/Sources/DefiMacOS/FrameCommit.swift index 8b17c1a8..8af2e505 100644 --- a/Sources/DefiMacOS/FrameCommit.swift +++ b/Sources/DefiMacOS/FrameCommit.swift @@ -10,14 +10,19 @@ struct FrameWriteIntent: Equatable, Sendable { let size: Bool } +func frameWritePosition(target: CGPoint, from start: CGPoint, horizontalOnly: Bool) -> CGPoint { + CGPoint(x: target.x, y: horizontalOnly ? start.y : target.y) +} + func frameWriteIntent( reference: Rect, target: Rect, - positionsOnly: Bool + positionsOnly: Bool, + horizontalOnly: Bool = false ) -> FrameWriteIntent { FrameWriteIntent( position: abs(reference.x - target.x) >= 0.5 - || abs(reference.y - target.y) >= 0.5, + || (!horizontalOnly && abs(reference.y - target.y) >= 0.5), size: !positionsOnly && (abs(reference.width - target.width) >= 0.5 || abs(reference.height - target.height) >= 0.5) @@ -82,8 +87,7 @@ func shouldDeferAnimatedSizeUntilMovementCompletes( to target: Rect, displayFrames: [Rect] ) -> Bool { - abs(source.x - target.x) >= 0.5 - || frameCentersCrossDisplays( + frameCentersCrossDisplays( from: source, to: target, displayFrames: displayFrames @@ -101,12 +105,92 @@ struct AsyncPositionWrite: @unchecked Sendable { let positionChanged: Bool let sizeChanged: Bool let animatesSize: Bool + var usesLogicalRibbonPath = false let synchronousSizeWriteSucceeded: Bool let enhancedUIWasEnabled: Bool let timeoutSeconds: Float let isParked: Bool let isReentering: Bool let requiresVerifiedOffscreenWrite: Bool + var animationPoint: CGPoint? = nil + var usesCommonRibbonOffset = false +} + +// One visible native window anchors the entire strip. Parking anchors are +// deliberately excluded: their one-pixel exposure is not a logical position. +func commonRibbonOffset( + targets: [WindowID: CGPoint], starts: [WindowID: CGPoint], + sizes: [WindowID: CGSize], monitor: Rect +) -> Double? { + let candidates = targets.compactMap { id, target -> (WindowID, Double, Double)? in + guard let start = starts[id], let size = sizes[id], + size.width > 0 else { return nil } + let exposure = min(start.x + size.width, monitor.x + monitor.width) + - max(start.x, monitor.x) + guard exposure > 1.5 else { return nil } + return (id, exposure, start.x - target.x) + }.sorted { + $0.1 == $1.1 ? $0.0.rawValue < $1.0.rawValue : $0.1 > $1.1 + } + return candidates.first?.2 +} + +func nativeRibbonAnimationFrame(_ logical: Rect, monitor: Rect) -> Rect { + let exposure = min(logical.x + logical.width, monitor.x + monitor.width) + - max(logical.x, monitor.x) + guard exposure <= parkedSliverWidth else { return logical } + return resolveParkingPlacement( + for: logical, ownerFrame: monitor, allMonitorFrames: [monitor], + preferredSide: logical.x < monitor.x ? .left : .right, + preferredY: logical.y).frame +} + +func ribbonAnimationStart( + logical: Rect, offset: Double, observed: Rect, monitorFrames: [Rect] +) -> Rect? { + let start = Rect(x: logical.x + offset, y: observed.y, + width: observed.width, height: observed.height) + return ribbonAnimationTarget(logical: logical, from: start, + isParked: true, monitorFrames: monitorFrames) == nil ? nil : start +} + +func layoutRibbonAnimationStart( + previousLogical: Rect?, target: Rect, observed: Rect, monitorFrames: [Rect] +) -> Rect? { + guard monitorFrames.count == 1, let previousLogical, + requiresVerifiedOffscreenWrite(frame: observed, monitorFrames: monitorFrames) + else { return nil } + let start = Rect(x: previousLogical.x, y: observed.y, + width: observed.width, height: observed.height) + return ribbonAnimationTarget(logical: target, from: start, + isParked: true, monitorFrames: monitorFrames) == nil ? nil : start +} + +func ribbonAnimationTarget( + logical: Rect?, from: Rect, isParked: Bool, monitorFrames: [Rect] +) -> Rect? { + // The logical path may cross the display even when both native endpoints + // are parking anchors. Such columns must participate in the common timeline. + guard isParked, monitorFrames.count == 1, let logical else { return nil } + let monitor = monitorFrames[0] + guard min(from.x, logical.x) < monitor.x + monitor.width, + max(from.x + from.width, logical.x + logical.width) > monitor.x, + min(from.y, logical.y) < monitor.y + monitor.height, + max(from.y + from.height, logical.y + logical.height) > monitor.y + else { return nil } + return logical +} + +func ribbonParkingPreparationWindowIDs(_ frame: QueuedPositionFrame) -> Set { + guard frame.source == "command-animation" || frame.source == "command-layout-animation", + frame.monitorFrames.count == 1 else { return [] } + return Set(frame.writes.compactMap { id, write in + write.isParked && !frame.animatedWindowIDs.contains(id) ? id : nil + }) +} + +func frameAnimationDestination(_ write: AsyncPositionWrite, intermediate: Bool) -> CGPoint { + intermediate ? (write.animationPoint ?? write.point) : write.point } func positionOnlyAnimationWrite( @@ -124,12 +208,15 @@ func positionOnlyAnimationWrite( positionChanged: write.positionChanged, sizeChanged: write.sizeChanged, animatesSize: false, + usesLogicalRibbonPath: write.usesLogicalRibbonPath, synchronousSizeWriteSucceeded: true, enhancedUIWasEnabled: write.enhancedUIWasEnabled, timeoutSeconds: write.timeoutSeconds, isParked: write.isParked, isReentering: write.isReentering, - requiresVerifiedOffscreenWrite: write.requiresVerifiedOffscreenWrite + requiresVerifiedOffscreenWrite: write.requiresVerifiedOffscreenWrite, + animationPoint: write.animationPoint, + usesCommonRibbonOffset: write.usesCommonRibbonOffset ) } @@ -138,19 +225,19 @@ func frameApplicationReference( settlingReference: Rect?, completedPosition: CGPoint?, previousTarget: Rect?, + prefersCompletedPosition: Bool = false, pendingAnimation: Bool = false, nativeReference: @autoclosure () -> Rect? ) -> Rect? { if let pendingCorrection { return pendingCorrection } - if let settlingReference, let completedPosition { + if let completedPosition, + let reference = settlingReference ?? (prefersCompletedPosition ? previousTarget : nil) + { return Rect( - x: completedPosition.x, - y: completedPosition.y, - width: settlingReference.width, - height: settlingReference.height - ) + x: completedPosition.x, y: completedPosition.y, + width: reference.width, height: reference.height) } if pendingAnimation, let previousTarget { let position = completedPosition ?? nativeReference().map { @@ -214,12 +301,15 @@ func frameWritesPreservingSupersededAsyncSizes( positionChanged: newer.positionChanged, sizeChanged: true, animatesSize: debt.animatesSize, + usesLogicalRibbonPath: newer.usesLogicalRibbonPath, synchronousSizeWriteSucceeded: debt.synchronousSizeWriteSucceeded, enhancedUIWasEnabled: newer.enhancedUIWasEnabled, timeoutSeconds: newer.timeoutSeconds, isParked: newer.isParked, isReentering: newer.isReentering, - requiresVerifiedOffscreenWrite: newer.requiresVerifiedOffscreenWrite + requiresVerifiedOffscreenWrite: newer.requiresVerifiedOffscreenWrite, + animationPoint: newer.animationPoint, + usesCommonRibbonOffset: newer.usesCommonRibbonOffset ) } else { result[windowID] = newer @@ -258,7 +348,7 @@ struct QueuedPositionFrame: @unchecked Sendable { let source: String let writes: [WindowID: AsyncPositionWrite] let animatedWindowIDs: Set - let animationDuration: TimeInterval + var animationDuration: TimeInterval let refreshRateHz: Double let displayIDs: Set let monitorFrames: [Rect] diff --git a/Sources/DefiMacOS/FrameCommitPolicy.swift b/Sources/DefiMacOS/FrameCommitPolicy.swift index b568b5ad..0f6631e2 100644 --- a/Sources/DefiMacOS/FrameCommitPolicy.swift +++ b/Sources/DefiMacOS/FrameCommitPolicy.swift @@ -668,3 +668,23 @@ func commandPerformanceFramePlan( .intersection(availableWindowIDs) return (expectedWindowIDs, expectedWindowIDs.isEmpty == false) } + +func overviewExitFramesAreReady( + windowIDs: Set, hiddenWindowIDs: Set, + pendingWriteWindowIDs: Set, unresolvedWindowIDs: Set, + targets: [WindowID: Rect], observed: [WindowID: Rect], monitorFrames: [Rect] +) -> Bool { + guard pendingWriteWindowIDs.isDisjoint(with: windowIDs), + unresolvedWindowIDs.isDisjoint(with: windowIDs.subtracting(hiddenWindowIDs)) + else { return false } + return windowIDs.allSatisfy { id in + guard let target = targets[id], let actual = observed[id] else { return false } + if hiddenWindowIDs.contains(id) { + // Native apps may clamp a parked frame vertically. Exact geometry there + // must not cancel the visible zoom, but a parking leak remains unsafe. + return !monitorFrames.isEmpty + && requiresVerifiedOffscreenWrite(frame: actual, monitorFrames: monitorFrames) + } + return frameDistance(actual, target) <= 1 + } +} diff --git a/Sources/DefiMacOS/HotKeyModels.swift b/Sources/DefiMacOS/HotKeyModels.swift index 00dd8439..5faed265 100644 --- a/Sources/DefiMacOS/HotKeyModels.swift +++ b/Sources/DefiMacOS/HotKeyModels.swift @@ -44,10 +44,16 @@ public enum OverviewKeyAction: Equatable, Sendable { case right case up case down + case firstColumn + case lastColumn + case workspaceUp + case workspaceDown + case workspace(WorkspaceTarget) case moveUp case moveDown case select case cancel + case layout(Command) } func overviewKeyAction( @@ -55,14 +61,35 @@ func overviewKeyAction( modifierBits: UInt64, configuredCommand: String? = nil ) -> OverviewKeyAction? { - let commandName = configuredCommand?.split(whereSeparator: \.isWhitespace).first + let parts = configuredCommand?.split(whereSeparator: \.isWhitespace) ?? [] + let commandName = parts.first + if commandName == "move-window", keyCode == 125 { return .moveDown } + if commandName == "move-window", keyCode == 126 { return .moveUp } + if let configuredCommand, let command = try? parseCommand(configuredCommand) { + switch command { + case .switchWorkspace(let id): return .workspace(.named(id.rawValue)) + case .focusWorkspace(let target): + switch target { + case .relative(.up): return .workspaceUp + case .relative(.down): return .workspaceDown + default: return .workspace(target) + } + default: + if command.editsSelectedLayout { return .layout(command) } + } + } + if parts.count == 2 { + switch (parts[0], parts[1]) { + case ("focus-column", "first"): return .firstColumn + case ("focus-column", "last"): return .lastColumn + default: break + } + } let navigatesOverview = modifierBits == 0 || commandName == "focus-column" || commandName == "focus-window" return switch keyCode { - case 125 where commandName == "move-window": .moveDown - case 126 where commandName == "move-window": .moveUp case 123 where navigatesOverview: .left case 124 where navigatesOverview: .right case 125 where navigatesOverview: .down @@ -70,6 +97,7 @@ func overviewKeyAction( case 36 where modifierBits == 0: .select case 76 where modifierBits == 0: .select case 53 where modifierBits == 0: .cancel + case _ where commandName == "toggle-overview": .cancel default: nil } } diff --git a/Sources/DefiMacOS/HotKeys.swift b/Sources/DefiMacOS/HotKeys.swift index 8dc17c78..c32c589c 100644 --- a/Sources/DefiMacOS/HotKeys.swift +++ b/Sources/DefiMacOS/HotKeys.swift @@ -29,6 +29,7 @@ public final class HotKeyManager { private let tapReenabledHandler: TapReenabledHandler private let closeIntentHandler: CloseIntentHandler private let overviewHandler: OverviewHandler + private let overviewCancelHandler: (@Sendable () -> Void)? private let cheatsheetHandler: @NavigationActor @Sendable (CheatsheetInput) -> Void private let cheatsheetModifierBits: UInt64? private let userInputTracker: UserInputTracker @@ -70,6 +71,7 @@ public final class HotKeyManager { tapReenabledHandler: @escaping TapReenabledHandler = { _ in }, closeIntentHandler: @escaping CloseIntentHandler = { _, _ in }, overviewHandler: @escaping OverviewHandler = { _ in }, + overviewCancelHandler: (@Sendable () -> Void)? = nil, cheatsheetHandler: @escaping @NavigationActor @Sendable (CheatsheetInput) -> Void = { _ in }, registrationHandler: @escaping @NavigationActor @Sendable (Bool, HotKeyError?) -> Void = { _, _ in }, handler: @escaping Handler @@ -84,6 +86,7 @@ public final class HotKeyManager { self.tapReenabledHandler = tapReenabledHandler self.closeIntentHandler = closeIntentHandler self.overviewHandler = overviewHandler + self.overviewCancelHandler = overviewCancelHandler self.cheatsheetHandler = cheatsheetHandler cheatsheetModifierBits = try? Key( accelerator: "\(config.defaultKeyModifier)-a", @@ -140,6 +143,7 @@ public final class HotKeyManager { let tapReenabledHandler = self.tapReenabledHandler let closeIntentHandler = self.closeIntentHandler let overviewHandler = self.overviewHandler + let overviewCancelHandler = self.overviewCancelHandler let cheatsheetHandler = self.cheatsheetHandler let context = InputMonitor( bindings: bindings, @@ -158,6 +162,11 @@ public final class HotKeyManager { handler(invocation) } } deliverOverview: { action in + // Escape and the overview toggle must not queue behind slow AX work. + if action == .cancel, let overviewCancelHandler { + overviewCancelHandler() + return + } NavigationActor.enqueue { overviewHandler(action) } diff --git a/Sources/DefiMacOS/MacOSPlatform+EventObservation.swift b/Sources/DefiMacOS/MacOSPlatform+EventObservation.swift index c69c88d4..9954fb6c 100644 --- a/Sources/DefiMacOS/MacOSPlatform+EventObservation.swift +++ b/Sources/DefiMacOS/MacOSPlatform+EventObservation.swift @@ -25,11 +25,13 @@ extension MacOSPlatform { } public func cancelPendingFrameWrites() { + DispatchQueue.main.async { ExperimentalRibbonRenderer.shared.cancel() } frameSubmissionGeneration &+= 1 frameCoordinator.invalidate(reason: "mouse-gesture") } public func prepareForRestore() async { + DispatchQueue.main.async { ExperimentalRibbonRenderer.shared.cancel() } frameSubmissionGeneration &+= 1 invalidateFocusStateForDisplayChange() let focusDeadline = ProcessInfo.processInfo.systemUptime + 5 @@ -42,7 +44,9 @@ extension MacOSPlatform { } private func clearFrameState() { + DispatchQueue.main.async { ExperimentalRibbonRenderer.shared.cancel() } frameSubmissionGeneration &+= 1 + previousLogicalRibbonFrames.removeAll(keepingCapacity: true) targetFrames.removeAll(keepingCapacity: true) pendingFrameDebtWindowIDs.removeAll(keepingCapacity: true) pendingFrameCorrections.removeAll(keepingCapacity: true) diff --git a/Sources/DefiMacOS/MacOSPlatform+FrameApplication.swift b/Sources/DefiMacOS/MacOSPlatform+FrameApplication.swift index 4b623b79..2a12cc1b 100644 --- a/Sources/DefiMacOS/MacOSPlatform+FrameApplication.swift +++ b/Sources/DefiMacOS/MacOSPlatform+FrameApplication.swift @@ -40,6 +40,7 @@ extension MacOSPlatform { public func apply( _ assignments: [FrameAssignment], + ribbonFrames: [FrameAssignment] = [], hiddenWindowIDs: Set = [], skipping requestedSkippedWindowIDs: Set = [], asynchronousPositionTimeoutSeconds: Float = 0.016, @@ -65,11 +66,28 @@ extension MacOSPlatform { commandPerformance: CommandPerformanceContext? = nil, source: String = "platform" ) { + let previousRibbonFrames = previousLogicalRibbonFrames + previousLogicalRibbonFrames = Dictionary(uniqueKeysWithValues: + ribbonFrames.map { ($0.windowID, $0.frame) }) + if experimentalSurfaceRibbonEnabled && source != "command-animation" { + snapshotEngine.onMain { _ in + ExperimentalRibbonRenderer.shared.supersedeIfTargetsChanged(assignments) + } + } + let usesSurfaceRibbon = experimentalSurfaceRibbonEnabled && source == "command-animation" && positionsOnly + && snapshotEngine.onMain { platform in + ExperimentalRibbonRenderer.shared.begin(assignments: assignments, ribbonFrames: ribbonFrames, + duration: animationDuration, + borderStyle: platform.borderStyle, selectedWindowID: platform.borderSelectedWindowID) + } + let animationDuration = usesSurfaceRibbon ? 0 : animationDuration frameSubmissionGeneration &+= 1 let submissionGeneration = frameSubmissionGeneration let skippedWindowIDs = requestedSkippedWindowIDs.union( nativeFullscreenWindowIDs ) + let horizontalRibbonNavigation = source == "command-animation" && positionsOnly + && lastMonitorFrames.count == 1 && !ribbonFrames.isEmpty let applyStartedAt = ProcessInfo.processInfo.systemUptime let tracesInitialFrame = !newlyDiscoveredWindowIDs.isEmpty if tracesInitialFrame { @@ -115,6 +133,7 @@ extension MacOSPlatform { for: assignment.windowID ), previousTarget: previousTargetFrames[assignment.windowID], + prefersCompletedPosition: horizontalRibbonNavigation, pendingAnimation: unfinishedRibbonWindowIDs.contains(assignment.windowID), nativeReference: latestObservedFrames[assignment.windowID] ?? lastSnapshotWindows.first(where: { $0.id == assignment.windowID })?.frame @@ -123,7 +142,8 @@ extension MacOSPlatform { let intent = frameWriteIntent( reference: reference, target: assignment.frame, - positionsOnly: positionsOnly + positionsOnly: positionsOnly, + horizontalOnly: horizontalRibbonNavigation ) if intent.position || intent.size { writeIntents[assignment.windowID] = (intent.position, intent.size) @@ -170,6 +190,18 @@ extension MacOSPlatform { } } } + // Logical strip endpoints differ from the safe native parking anchors. + // Use them only on a single display: an offscreen logical path must never + // sweep a user window through another monitor's visible region. + let layoutRibbonAnimation = source == "command-layout-animation" + let ribbonTargets = ((source == "command-animation" && positionsOnly) || layoutRibbonAnimation) + && lastMonitorFrames.count == 1 + ? Dictionary(uniqueKeysWithValues: ribbonFrames.map { ($0.windowID, $0.frame) }) + : [:] + let sharedRibbonOffset = !layoutRibbonAnimation && lastMonitorFrames.count == 1 ? commonRibbonOffset( + targets: ribbonTargets.mapValues { CGPoint(x: $0.x, y: $0.y) }, + starts: animationStartPositions, sizes: animationStartSizes, + monitor: lastMonitorFrames[0]) : nil let newlyUnparkedWindowIDs = lastHiddenWindowIDs.subtracting(effectiveHiddenWindowIDs) var reenteringWindowIDs = Set() @@ -210,6 +242,42 @@ extension MacOSPlatform { reenteringWindowIDs.insert(assignment.windowID) } } + if let offset = sharedRibbonOffset { + for assignment in assignments { + guard let logical = ribbonTargets[assignment.windowID] else { continue } + // Unchanged parking anchors have no write intent, but their logical + // columns can still cross the display during an end-to-end traversal. + let observedFrame = referenceFrames[assignment.windowID] + ?? latestObservedFrames[assignment.windowID] ?? assignment.frame + guard let plannedFrame = ribbonAnimationStart(logical: logical, offset: offset, + observed: observedFrame, monitorFrames: lastMonitorFrames) else { continue } + let observed = CGPoint(x: observedFrame.x, y: observedFrame.y) + let planned = CGPoint(x: plannedFrame.x, y: plannedFrame.y) + animationStartPositions[assignment.windowID] = planned + animationStartSizes[assignment.windowID] = CGSize( + width: observedFrame.width, height: observedFrame.height) + if requiresVerifiedOffscreenWrite(frame: observedFrame, monitorFrames: lastMonitorFrames), + reentryStartRequiresStaging(observed: observed, planned: planned) + { + reenteringWindowIDs.insert(assignment.windowID) + } + } + } + if layoutRibbonAnimation { + for assignment in assignments { + guard let target = ribbonTargets[assignment.windowID], + let observed = referenceFrames[assignment.windowID], + let start = layoutRibbonAnimationStart( + previousLogical: previousRibbonFrames[assignment.windowID], target: target, + observed: observed, monitorFrames: lastMonitorFrames) + else { continue } + let point = CGPoint(x: start.x, y: start.y) + animationStartPositions[assignment.windowID] = point + if reentryStartRequiresStaging(observed: CGPoint(x: observed.x, y: observed.y), planned: point) { + reenteringWindowIDs.insert(assignment.windowID) + } + } + } for assignment in assignments where newlyDiscoveredWindowIDs.contains(assignment.windowID) && previousTargetFrames[assignment.windowID] == nil @@ -276,7 +344,7 @@ extension MacOSPlatform { let isParked = hiddenWindowIDs.contains(assignment.windowID) let intent = writeIntents[assignment.windowID] - let position = CGPoint(x: assignment.frame.x, y: assignment.frame.y) + let logicalPosition = CGPoint(x: assignment.frame.x, y: assignment.frame.y) let size = CGSize(width: assignment.frame.width, height: assignment.frame.height) guard let processID = processIDs[assignment.windowID], let application = applications[processID] @@ -287,7 +355,10 @@ extension MacOSPlatform { frame: assignment.frame, monitorFrames: lastMonitorFrames ) - let startPoint = animationStartPositions[assignment.windowID] ?? position + let startPoint = animationStartPositions[assignment.windowID] ?? logicalPosition + // Preserve the native vertical clamp through the final commit and parking repair. + let position = frameWritePosition( + target: logicalPosition, from: startPoint, horizontalOnly: horizontalRibbonNavigation) let startSize = animationStartSizes[assignment.windowID] ?? size let startFrame = Rect( x: startPoint.x, @@ -295,18 +366,17 @@ extension MacOSPlatform { width: startSize.width, height: startSize.height ) + let parkedRibbonTarget = ribbonAnimationTarget( + logical: ribbonTargets[assignment.windowID], from: startFrame, + isParked: isParked, monitorFrames: lastMonitorFrames) let wantsFrameAnimation = animationDuration > 0 - && (!isParked || shouldAnimateParkedRibbonWindow( - source: source, - from: startFrame, - monitorFrames: lastMonitorFrames - )) + && (!isParked || parkedRibbonTarget != nil) && transitionCrossesViewport( from: startFrame, - to: assignment.frame + to: parkedRibbonTarget ?? assignment.frame ) - && (intent?.position == true + && (intent?.position == true || parkedRibbonTarget != nil || (animateSizeChanges && intent?.size == true)) let animatesSize = wantsFrameAnimation @@ -320,15 +390,20 @@ extension MacOSPlatform { point: position, fromSize: startSize, size: size, - positionChanged: intent?.position == true, + positionChanged: intent?.position == true || parkedRibbonTarget != nil, sizeChanged: intent?.size == true, animatesSize: animatesSize, + usesLogicalRibbonPath: layoutRibbonAnimation && ribbonTargets[assignment.windowID] != nil + && lastMonitorFrames.count == 1 && wantsFrameAnimation, synchronousSizeWriteSucceeded: intent?.size != true, enhancedUIWasEnabled: enhancedUIByProcess[processID] == true, timeoutSeconds: asynchronousPositionTimeoutSeconds, isParked: isParked, isReentering: reenteringWindowIDs.contains(assignment.windowID), - requiresVerifiedOffscreenWrite: needsVerifiedOffscreenWrite + requiresVerifiedOffscreenWrite: needsVerifiedOffscreenWrite, + animationPoint: parkedRibbonTarget.map { CGPoint(x: $0.x, y: $0.y) }, + usesCommonRibbonOffset: sharedRibbonOffset != nil && ribbonTargets[assignment.windowID] != nil + && wantsFrameAnimation ) if isParked || needsVerifiedOffscreenWrite { parkingTargets[assignment.windowID] = write diff --git a/Sources/DefiMacOS/MacOSPlatform+FramePerformance.swift b/Sources/DefiMacOS/MacOSPlatform+FramePerformance.swift index 41273f84..57aba8eb 100644 --- a/Sources/DefiMacOS/MacOSPlatform+FramePerformance.swift +++ b/Sources/DefiMacOS/MacOSPlatform+FramePerformance.swift @@ -507,6 +507,15 @@ extension MacOSPlatform { ) } + public func overviewExitFramesReady(windowIDs: Set) -> Bool { + let participants = windowIDs.subtracting(nativeFullscreenWindowIDs) + return overviewExitFramesAreReady(windowIDs: participants, + hiddenWindowIDs: lastHiddenWindowIDs, + pendingWriteWindowIDs: frameCoordinator.pendingWindowIDs, + unresolvedWindowIDs: pendingFrameWindowIDs, + targets: targetFrames, observed: latestObservedFrames, monitorFrames: lastMonitorFrames) + } + public var hasPendingFocusWrite: Bool { focusWriter.isBusy } diff --git a/Sources/DefiMacOS/MacOSPlatform.swift b/Sources/DefiMacOS/MacOSPlatform.swift index 7ead14d8..91d5a221 100644 --- a/Sources/DefiMacOS/MacOSPlatform.swift +++ b/Sources/DefiMacOS/MacOSPlatform.swift @@ -10,6 +10,14 @@ import OSLog @NavigationActor public final class MacOSPlatform { + public var experimentalSurfaceRibbonEnabled = false { + didSet { + if oldValue && !experimentalSurfaceRibbonEnabled { + DispatchQueue.main.async { ExperimentalRibbonRenderer.shared.cancel() } + } + } + } + var previousLogicalRibbonFrames: [WindowID: Rect] = [:] nonisolated let snapshotEngine: SnapshotEngine @@ -457,6 +465,7 @@ public final class MacOSPlatform { activeNativeFullscreenWindowIDs = activeWindowIDs.intersection(windowIDs) let now = ProcessInfo.processInfo.systemUptime if !entered.isEmpty { + DispatchQueue.main.async { ExperimentalRibbonRenderer.shared.cancel() } frameSubmissionGeneration &+= 1 if entered.contains(where: frameCoordinator.isBusy(for:)) { frameCoordinator.invalidate(reason: "native-fullscreen") diff --git a/Sources/DefiMacOS/OverviewController.swift b/Sources/DefiMacOS/OverviewController.swift index d4708cfa..be0c8fe6 100644 --- a/Sources/DefiMacOS/OverviewController.swift +++ b/Sources/DefiMacOS/OverviewController.swift @@ -2,8 +2,11 @@ import AppKit import DefiConfig import DefiCore import DefiModel +import OSLog import QuartzCore +private let overviewOpeningLogger = Logger(subsystem: "com.quentin.defi", category: "OverviewOpening") + let overviewTransitionDuration: TimeInterval = 0.16 let overviewOpenFadeDuration: TimeInterval = 0.14 let overviewDesktopFadeDuration: TimeInterval = 0.25 @@ -138,8 +141,11 @@ public final class OverviewController: NSObject { private let focusWindowHandler: WindowHandler private let focusWorkspaceHandler: WorkspaceHandler private let dropHandler: DropHandler + private let layoutCommandHandler: @MainActor @Sendable (Command, WindowID, String, MonitorID, WorkspaceID, UInt64) -> Void private let activateMonitorHandler: MonitorHandler private let presentationChanged: @MainActor @Sendable () -> Void + private var idlePreparationRetry: Task? + private let idlePreparationRequested: @MainActor @Sendable () -> Void private let openStateHandler: OpenStateHandler private let scrollCommitHandler: ScrollCommitHandler private var panels: [MonitorID: OverviewPanel] = [:] @@ -149,17 +155,48 @@ public final class OverviewController: NSObject { private var viewports: [MonitorID: OverviewViewport] = [:] private var projections: [MonitorID: OverviewProjection] = [:] private var previewMonitorIDs: [WindowID: MonitorID] = [:] + private var selectionsByWorkspace: [WorkspaceID: OverviewSelection] = [:] private var selection: OverviewSelection? + private var hasDeferredSelection = false + private var selectionCommitPending = false + private var selectionAlignmentFinished = false + private var selectionNativeReady = false + private let notificationCenter: NotificationCenter + private let waitsForNativeSelectionCommit: Bool private var drag: OverviewDrag? private var edgeScrollTimer: Timer? private var edgeScrollDirection: Double? - private var sessionGeneration: UInt64 = 0 + public private(set) var sessionGeneration: UInt64 = 0 private var windowPreviewsEnabled = false + private var surfaceTransitionsEnabled = false + private var ribbonRepresentationsEnabled = false + private var ribbonCornerRadius: Double = 12 + private let surfaceCapture = OverviewSurfaceCapture.shared + private var surfaceWindowIDs = Set() + public private(set) var surfaceTransitionCount = 0 + public private(set) var surfaceFallbackCount = 0 + public private(set) var previewClosingCount = 0 + public private(set) var surfaceAcquireMs: Double = 0 + private var openingEvents: [String] = [] + public var openingTransitionHistory: String { openingEvents.isEmpty ? "none" : openingEvents.joined(separator: "|") } + public var surfaceCaptureState: String { surfaceCapture.state } + public var ribbonSurfaceTransitions: Int { ExperimentalRibbonRenderer.shared.transitions } + public var ribbonSurfaceFallbacks: Int { ExperimentalRibbonRenderer.shared.fallbacks } + public var ribbonSurfaceFallbackReason: String { ExperimentalRibbonRenderer.shared.lastFallback } + public var ribbonSurfacePresenting: Bool { ExperimentalRibbonRenderer.shared.isPresenting } + public var surfaceStreamCount: Int { surfaceCapture.streamCount } + var surfacePresentedFrameCount: Int { panels.values.reduce(0) { $0 + $1.surfacePresentedFrameCount } } + public var surfaceEstimatedPoolBytes: Int { surfaceCapture.estimatedPoolBytes } private var idlePreparationEnabled = true + private var isUnderMemoryPressure = false + private var surfacePreparationPending = false private var previewTask: Task? private var desktopCaptureRetryTask: Task? private var previewCache: [WindowID: NSImage] = [:] private let rememberedPreviews = OverviewPreviewCache() + private var idlePreviewTask: Task? + private var idlePreviewAttempted: [OverviewPreviewRequest] = [] + private var idlePreviewPriorityIDs: Set = [] private var previewRevealStartedAt: [WindowID: TimeInterval] = [:] private var previewCacheExpiry: DispatchWorkItem? private var memoryPressureSource: DispatchSourceMemoryPressure? @@ -173,9 +210,25 @@ public final class OverviewController: NSObject { private var windowCornerRadius = 12.0 private var viewportAnimations: [MonitorID: OverviewViewportAnimation] = [:] private var projectionAnimations: [MonitorID: OverviewProjectionAnimation] = [:] + private var lastRenderedRefresh: [MonitorID: TimeInterval] = [:] + private var renderedIntervals = 0 + private var renderedIntervalSeconds: TimeInterval = 0 + private var maximumRenderedGap: TimeInterval = 0 + private var lateRenderedRefreshes = 0 + private var renderedFrames = 0 + private var renderedSeconds: TimeInterval = 0 + public var renderPerformance: String { + let hz = renderedIntervalSeconds > 0 ? Double(renderedIntervals) / renderedIntervalSeconds : 0 + let drawMS = renderedFrames > 0 ? renderedSeconds * 1_000 / Double(renderedFrames) : 0 + return String(format: "%.1fHz/frames:%d/late:%d/maxGapMs:%.2f/meanFrameMs:%.2f", + hz, renderedFrames, lateRenderedRefreshes, maximumRenderedGap * 1_000, drawMS) + } private var viewportDisplayLinks: [MonitorID: CADisplayLink] = [:] + private var pendingViewportFrames = Set() private var displayLinkMonitorIDs: [ObjectIdentifier: MonitorID] = [:] + public var scrollOffsets: [MonitorID: [WorkspaceID: Double]] { viewports.mapValues(\.horizontalOffsets) } + private var nativeExitCanZoom = true public private(set) var isOpen = false private var ribbonPrototype = false public var usesRibbonPrototype: Bool { ribbonPrototype } @@ -204,33 +257,36 @@ public final class OverviewController: NSObject { activateMonitor: @escaping MonitorHandler, openStateChanged: @escaping OpenStateHandler, presentationChanged: @escaping @MainActor @Sendable () -> Void = {}, + idlePreparationRequested: @escaping @MainActor @Sendable () -> Void = {}, + waitsForNativeSelectionCommit: Bool = false, + notificationCenter: NotificationCenter = NSWorkspace.shared.notificationCenter, + layoutCommand: @escaping @MainActor @Sendable (Command, WindowID, String, MonitorID, WorkspaceID, UInt64) -> Void = { _, _, _, _, _, _ in }, commitScrollOffsets: @escaping ScrollCommitHandler ) { focusWindowHandler = focusWindow focusWorkspaceHandler = focusWorkspace dropHandler = drop + layoutCommandHandler = layoutCommand activateMonitorHandler = activateMonitor openStateHandler = openStateChanged self.presentationChanged = presentationChanged + self.idlePreparationRequested = idlePreparationRequested + self.waitsForNativeSelectionCommit = waitsForNativeSelectionCommit + self.notificationCenter = notificationCenter scrollCommitHandler = commitScrollOffsets super.init() let pressure = DispatchSource.makeMemoryPressureSource( - eventMask: [.warning, .critical], queue: .main + eventMask: [.normal, .warning, .critical], queue: .main ) pressure.setEventHandler { [weak self] in MainActor.assumeIsolated { - self?.rememberedPreviews.removeAll() - if let self, !self.isOpen { - self.releaseIdleOverviewResources() - self.closePanelsImmediately() - self.idlePreparationEnabled = false - } - self?.presentationChanged() + guard let self, let events = self.memoryPressureSource?.data else { return } + self.handleMemoryPressure(events) } } pressure.resume() memoryPressureSource = pressure - let center = NSWorkspace.shared.notificationCenter + let center = notificationCenter for name in [ NSWorkspace.screensDidSleepNotification, NSWorkspace.sessionDidResignActiveNotification, @@ -245,12 +301,40 @@ public final class OverviewController: NSObject { } } + func handleMemoryPressure(_ events: DispatchSource.MemoryPressureEvent) { + if events.contains(.critical) || events.contains(.warning) { + isUnderMemoryPressure = true + surfacePreparationPending = true + idlePreviewTask?.cancel() + idlePreviewTask = nil + idlePreviewAttempted = [] + if events.contains(.critical) { rememberedPreviews.removeAll() } + else { rememberedPreviews.retain(idlePreviewPriorityIDs, maximumBytes: 4 * 1_024 * 1_024) } + ExperimentalRibbonRenderer.shared.disable() + surfaceCapture.stop(state: "pressure") + idlePreparationEnabled = false + if !isOpen { + releaseIdleOverviewResources() + closePanelsImmediately() + } + overviewOpeningLogger.notice("memory-pressure critical=\(events.contains(.critical), privacy: .public) retainedBytes=\(self.rememberedPreviews.byteCount, privacy: .public)") + presentationChanged() + } else if events.contains(.normal), isUnderMemoryPressure { + isUnderMemoryPressure = false + idlePreparationEnabled = true + idlePreviewAttempted = [] + overviewOpeningLogger.notice("memory-pressure recovered; requesting idle preparation") + idlePreparationRequested() + } + } + isolated deinit { + idlePreviewTask?.cancel() desktopCaptureRetryTask?.cancel() previewCacheExpiry?.cancel() memoryPressureSource?.cancel() for link in viewportDisplayLinks.values { link.invalidate() } - NSWorkspace.shared.notificationCenter.removeObserver(self) + notificationCenter.removeObserver(self) } public func toggle( @@ -261,6 +345,7 @@ public final class OverviewController: NSObject { zoom: Double = 0.5, windowCornerRadius: Double = 12, windowPreviewsEnabled: Bool = false, + experimentalSurfaceTransitions: Bool = false, ribbonPrototype: Bool = false ) { if isOpen { @@ -276,14 +361,36 @@ public final class OverviewController: NSObject { animation: animation, zoom: ribbonPrototype ? 1 : zoom, windowCornerRadius: windowCornerRadius, - windowPreviewsEnabled: ribbonPrototype || windowPreviewsEnabled + windowPreviewsEnabled: ribbonPrototype || windowPreviewsEnabled, + experimentalSurfaceTransitions: experimentalSurfaceTransitions ) } } - public func prepare(windowPreviewsEnabled: Bool) { + public func prepare( + windowPreviewsEnabled: Bool, + snapshot: OverviewSnapshot? = nil, + layout: LayoutSettings = LayoutSettings(), + zoom: Double = 0.5, + experimentalSurfaceTransitions: Bool = false, + experimentalRibbonRepresentations: Bool = false, + windowCornerRadius: Double = 12 + ) { defer { presentationChanged() } + // A closing zoom still owns its texture; idle refresh must wait for its handoff. guard !isOpen, idlePreparationEnabled else { return } + if panels.values.contains(where: \.hasSurfaceScene) || ExperimentalRibbonRenderer.shared.isPresenting { + if idlePreparationRetry == nil { + idlePreparationRetry = Task { [weak self] in + do { try await Task.sleep(for: .milliseconds(250)) } catch { return } + guard let self else { return } + idlePreparationRetry = nil + if !isOpen { idlePreparationRequested() } + } + } + return + } + idlePreparationRetry?.cancel(); idlePreparationRetry = nil usesWorkspaceParking = overviewUsesWorkspaceParking( windowPreviewsEnabled: windowPreviewsEnabled, screenCaptureAccessGranted: windowPreviewsEnabled && CGPreflightScreenCaptureAccess() @@ -294,6 +401,19 @@ public final class OverviewController: NSObject { .map { MonitorID(rawValue: $0.uint64Value) } }) preparePanels(monitorIDs: monitorIDs) + surfaceTransitionsEnabled = experimentalSurfaceTransitions && windowPreviewsEnabled + ribbonRepresentationsEnabled = experimentalRibbonRepresentations + ribbonCornerRadius = windowCornerRadius + let preparesSurfaces = surfaceTransitionsEnabled || ribbonRepresentationsEnabled + if !preparesSurfaces { ExperimentalRibbonRenderer.shared.disable() } + let requests = snapshot.map { surfaceRequests(snapshot: $0, layout: layout, zoom: zoom) } ?? [] + surfaceCapture.prepare(requests, enabled: preparesSurfaces) + surfacePreparationPending = false + if surfaceTransitionsEnabled, let snapshot { + prepareIdlePreviews(snapshot: snapshot, layout: layout, zoom: zoom) + } else { + idlePreviewTask?.cancel(); idlePreviewTask = nil; idlePreviewAttempted = [] + } } private func preparePanels(monitorIDs: Set) { @@ -310,6 +430,9 @@ public final class OverviewController: NSObject { monitorID: monitorID, screen: screen, usesCapturedDesktop: !usesWorkspaceParking, delegate: self ) + panel.openingTransitionChanged = { [weak self] event in + self?.recordOpeningTransition(event, monitorID: monitorID) + } panels[monitorID] = panel if panel.usesCapturedDesktop { panel.loadWallpaperIfNeeded() } } @@ -322,9 +445,11 @@ public final class OverviewController: NSObject { animation: AnimationConfig = AnimationConfig(), zoom: Double = 0.5, windowCornerRadius: Double = 12, - windowPreviewsEnabled: Bool = false + windowPreviewsEnabled: Bool = false, + experimentalSurfaceTransitions: Bool = false ) { - idlePreparationEnabled = true + idlePreparationEnabled = !isUnderMemoryPressure + idlePreparationRetry?.cancel(); idlePreparationRetry = nil sessionGeneration &+= 1 self.snapshot = snapshot self.layout = layout @@ -335,6 +460,7 @@ public final class OverviewController: NSObject { overviewZoom = ribbonPrototype ? 1 : zoom self.windowCornerRadius = windowCornerRadius self.windowPreviewsEnabled = windowPreviewsEnabled + surfaceTransitionsEnabled = experimentalSurfaceTransitions && windowPreviewsEnabled usesWorkspaceParking = overviewUsesWorkspaceParking( windowPreviewsEnabled: windowPreviewsEnabled, screenCaptureAccessGranted: CGPreflightScreenCaptureAccess() @@ -347,13 +473,27 @@ public final class OverviewController: NSObject { previewTask = nil cancelDesktopCaptureRetry() previewCache.removeAll(keepingCapacity: true) + surfaceWindowIDs.removeAll() restoreRememberedPreviews(for: snapshot) + if surfaceTransitionsEnabled { surfaceCapture.freeze() } resetPreviewFadeAnimation() attemptedPreviewWindowIDs.removeAll(keepingCapacity: true) capturedDesktopMonitorIDs.removeAll(keepingCapacity: true) previewPendingCount = 0 previewPermissionState = windowPreviewsEnabled ? .notDetermined : .disabled selection = initialSelection(in: snapshot) + selectionsByWorkspace.removeAll(keepingCapacity: true) + if let selection { selectionsByWorkspace[selection.location.workspaceID] = selection } + hasDeferredSelection = false + selectionCommitPending = false + nativeExitCanZoom = true + lastRenderedRefresh.removeAll(keepingCapacity: true) + renderedIntervals = 0 + renderedIntervalSeconds = 0 + maximumRenderedGap = 0 + lateRenderedRefreshes = 0 + renderedFrames = 0 + renderedSeconds = 0 viewports = Dictionary(uniqueKeysWithValues: snapshot.monitors.map { monitor in ( monitor.id, @@ -371,14 +511,57 @@ public final class OverviewController: NSObject { firstPreviewMs = nil lastPreviewMs = nil receivedPreviewCount = 0 - updatePanels() + if surfaceTransitionsEnabled { + for (monitorID, panel) in panels { + if let background = ExperimentalRibbonRenderer.shared.backgroundImage(for: monitorID) { + panel.setDesktopImage(NSImage(cgImage: background, size: panel.window.frame.size)) + } + } + } + updatePanels(scheduleCaptures: false) + var surfacesByMonitor: [MonitorID: [WindowID: OverviewSurfaceFrame]] = [:] + for monitor in snapshot.monitors { + if let frames = currentSurfaceFrames(snapshot: snapshot, monitorID: monitor.id) { + surfacesByMonitor[monitor.id] = frames + } + } openStateHandler(true) let fadeDuration = animationsEnabled && !NSWorkspace.shared.accessibilityDisplayShouldReduceMotion ? overviewOpenFadeDuration : 0 - for panel in panels.values { - panel.show(fadeDuration: fadeDuration) + for (monitorID, panel) in panels { + let surfaceFrames = surfacesByMonitor[monitorID] + if surfaceTransitionsEnabled, animationsEnabled, !usesWorkspaceParking, + !NSWorkspace.shared.accessibilityDisplayShouldReduceMotion, + let projection = projections[monitorID], + let monitor = snapshot.monitors.first(where: { $0.id == monitorID }), + let screen = screen(for: monitorID), + let scene = OverviewSurfaceScene(projection: projection, + workspaceID: monitor.activeWorkspace, screen: screen, surfaces: surfaceFrames ?? [:], + windows: snapshot.windows, cornerRadius: windowCornerRadius, previews: previewCache) + { + let ids = Set(scene.nativeFrames.keys) + let capturedIDs = Set(surfaceFrames?.keys.map { $0 } ?? []) + surfaceWindowIDs.formUnion(capturedIDs) + attemptedPreviewWindowIDs.formUnion(capturedIDs) + panel.showSurfaceScene(scene, windowIDs: ids, duration: 0.22) + surfaceTransitionCount += 1 + } else { + if surfaceTransitionsEnabled { surfaceFallbackCount += 1 } + recordOpeningTransition("fallback(enabled:\(surfaceTransitionsEnabled),animation:\(animationsEnabled),parking:\(usesWorkspaceParking),reduceMotion:\(NSWorkspace.shared.accessibilityDisplayShouldReduceMotion),surfaces:\(surfaceFrames?.count ?? 0),previews:\(previewCache.count))", monitorID: monitorID) + panel.show(fadeDuration: fadeDuration) + } } + schedulePreviewsIfNeeded() + } + + private func recordOpeningTransition(_ event: String, monitorID: MonitorID) { + let elapsed = max(Int((CACurrentMediaTime() - openedAt) * 1_000), 0) + let entry = "session:\(sessionGeneration),monitor:\(monitorID.rawValue),ms:\(elapsed),\(event)" + openingEvents.append(entry) + if openingEvents.count > 8 { openingEvents.removeFirst(openingEvents.count - 8) } + overviewOpeningLogger.notice("\(entry, privacy: .public)") + presentationChanged() } public func update( @@ -388,9 +571,20 @@ public final class OverviewController: NSObject { animation: AnimationConfig = AnimationConfig(), zoom: Double = 0.5, windowCornerRadius: Double = 12, - windowPreviewsEnabled: Bool? = nil + windowPreviewsEnabled: Bool? = nil, + experimentalSurfaceTransitions: Bool? = nil, + experimentalRibbonRepresentations: Bool? = nil ) { guard isOpen else { return } + let previousSurfaceTransitionsEnabled = surfaceTransitionsEnabled + let previousRibbonRepresentationsEnabled = ribbonRepresentationsEnabled + if let experimentalSurfaceTransitions { + surfaceTransitionsEnabled = experimentalSurfaceTransitions + && (windowPreviewsEnabled ?? self.windowPreviewsEnabled) + } + if let experimentalRibbonRepresentations { + ribbonRepresentationsEnabled = experimentalRibbonRepresentations + } let previousSnapshot = self.snapshot let previousProjections = projections let windowPreviewsEnabled = ribbonPrototype ? true : windowPreviewsEnabled @@ -413,20 +607,33 @@ public final class OverviewController: NSObject { previewPermissionState = windowPreviewsEnabled ? .notDetermined : .disabled } self.snapshot = snapshot + selectionsByWorkspace = selectionsByWorkspace.filter { $0.value.isValid(in: snapshot) } pruneRememberedPreviews(for: snapshot) self.layout = layout borderStyle = WindowBorderStyle(config: borders) animationsEnabled = animation.enabled overviewZoom = ribbonPrototype ? 1 : zoom self.windowCornerRadius = windowCornerRadius + ribbonCornerRadius = windowCornerRadius + if previousSurfaceTransitionsEnabled != surfaceTransitionsEnabled + || previousRibbonRepresentationsEnabled != ribbonRepresentationsEnabled { + surfacePreparationPending = true + } + if surfacePreparationPending, !isUnderMemoryPressure { + surfacePreparationPending = false + surfaceCapture.prepare(surfaceRequests(snapshot: snapshot, layout: layout, zoom: zoom), + enabled: surfaceTransitionsEnabled || ribbonRepresentationsEnabled) + } var movedSelectionPositions: ( previous: OverviewTiledPosition, next: OverviewTiledPosition )? - if drag == nil && !ribbonPrototype { + if drag == nil && !ribbonPrototype, !hasDeferredSelection || selection?.isValid(in: snapshot) != true { + hasDeferredSelection = false let focusedSelection = initialSelection(in: snapshot) if focusedSelection != selection { selection = focusedSelection + if let focusedSelection { selectionsByWorkspace[focusedSelection.location.workspaceID] = focusedSelection } alignSelectionOnNextUpdate = true } else if let windowID = selection?.windowID, let previousPosition = previousSnapshot?.tiledPosition(of: windowID), @@ -450,6 +657,12 @@ public final class OverviewController: NSObject { edgeScrollDirection = nil } var viewportTargets: [MonitorID: OverviewViewport] = [:] + if drag == nil, let windowID = selection?.windowID, + let previousPosition = previousSnapshot?.tiledPosition(of: windowID), + let nextPosition = snapshot.tiledPosition(of: windowID), previousPosition != nextPosition { + movedSelectionPositions = (previousPosition, nextPosition) + alignSelectionOnNextUpdate = true + } for monitor in snapshot.monitors { let previousMonitor = previousSnapshot?.monitors.first { $0.id == monitor.id } if viewports[monitor.id] == nil { @@ -469,11 +682,23 @@ public final class OverviewController: NSObject { }), var viewport = viewports[monitor.id] { - cancelAnimations(on: monitor.id) - viewport.workspaceOffset += Double(previousIndex - activeIndex) + let rebase = Double(previousIndex - activeIndex) + viewport.workspaceOffset += rebase viewports[monitor.id] = viewport - viewport.workspaceOffset = 0 - viewportTargets[monitor.id] = viewport + if let animation = viewportAnimations[monitor.id], + abs(animation.to.workspaceOffset + rebase) < 0.000_001 { + // Matching activation changes the origin, not the visual timeline. + var from = animation.from, to = animation.to + from.workspaceOffset += rebase + to.workspaceOffset += rebase + viewportAnimations[monitor.id] = OverviewViewportAnimation( + from: from, to: to, startedAt: animation.startedAt, duration: animation.duration) + viewportTargets[monitor.id] = to + } else { + cancelAnimations(on: monitor.id) + viewport.workspaceOffset = 0 + viewportTargets[monitor.id] = viewport + } } guard let previousMonitor else { continue } for workspace in monitor.workspaces { @@ -540,13 +765,16 @@ public final class OverviewController: NSObject { let resizesCards = overviewProjectionResizesExistingCards( from: source, to: projection(for: panel, snapshot: snapshot) ) - if resizesCards { + let reordersCards = overviewProjectionReordersExistingCards( + from: source, to: projection(for: panel, snapshot: snapshot) + ) + if resizesCards || reordersCards { // Size and position changes use one projection timeline. if let viewportAnimation = viewportAnimations.removeValue(forKey: monitorID) { viewports[monitorID] = viewportAnimation.to } } - if resizesCards || movedSelectionPositions?.next.monitorID == monitorID { + if resizesCards || reordersCards || movedSelectionPositions?.next.monitorID == monitorID { animateProjection(on: monitorID, from: source) } } @@ -557,9 +785,27 @@ public final class OverviewController: NSObject { close(commitScrollOffsets: !ribbonPrototype) } + /// Called after the final native layout has settled underneath the overview. + public func selectionCommitCompleted(sessionGeneration generation: UInt64? = nil, nativeFramesReady: Bool = true) { + guard generation == nil || generation == sessionGeneration, + isOpen, selectionCommitPending else { return } + selectionNativeReady = true + nativeExitCanZoom = nativeFramesReady + finishSelectionCloseIfReady() + } + + private func finishSelectionCloseIfReady() { + guard selectionCommitPending, selectionNativeReady, selectionAlignmentFinished else { return } + close(commitScrollOffsets: false) + } + private func close(commitScrollOffsets: Bool) { guard isOpen else { return } + let closingPreviews = previewCache + let closingProjections = projections isOpen = false + hasDeferredSelection = false + selectionCommitPending = false ribbonPrototype = false previewCacheExpiry?.cancel() let expiry = DispatchWorkItem { [weak self] in @@ -570,7 +816,8 @@ public final class OverviewController: NSObject { } } previewCacheExpiry = expiry - DispatchQueue.main.asyncAfter(deadline: .now() + 60, execute: expiry) + // Retain textures only through the closing handoff, not for a minute afterward. + DispatchQueue.main.asyncAfter(deadline: .now() + 0.25, execute: expiry) sessionGeneration &+= 1 edgeScrollTimer?.invalidate() edgeScrollTimer = nil @@ -591,8 +838,36 @@ public final class OverviewController: NSObject { } openStateHandler(false) let closingPanels = Array(panels.values) + // Selection already commits focus/layout through the daemon. Whether this + // close commits viewport offsets must not disable a safe visual handoff. + let canZoomBack = nativeExitCanZoom && surfaceTransitionsEnabled && animationsEnabled + && !NSWorkspace.shared.accessibilityDisplayShouldReduceMotion + && snapshot?.monitors.allSatisfy({ monitor in + let viewport = viewports[monitor.id] ?? OverviewViewport() + return viewport.workspaceOffset == 0 && monitor.workspaces.allSatisfy { + viewport.horizontalOffsets[$0.id, default: $0.scrollOffset] == $0.scrollOffset + } + }) == true for panel in closingPanels { - panel.hide() + if canZoomBack && panel.hideSurfaceSceneIfUnchanged(duration: 0.22) { continue } + if nativeExitCanZoom, surfaceTransitionsEnabled, animationsEnabled, !usesWorkspaceParking, + !NSWorkspace.shared.accessibilityDisplayShouldReduceMotion, + let snapshot, let monitor = snapshot.monitors.first(where: { $0.id == panel.monitorID }), + let workspace = monitor.workspaces.first(where: { $0.id == monitor.activeWorkspace }), + workspace.floatingWindows.isEmpty, + let viewport = snapshot.monitorFrames[monitor.id], let screen = screen(for: monitor.id), + let projection = closingProjections[monitor.id], + let scene = OverviewPreviewClosingScene(projection: projection, + workspaceID: workspace.id, screen: screen, previews: closingPreviews, + cornerRadius: windowCornerRadius, + targets: computeLayout(workspace: workspace, viewport: viewport, + windows: Array(snapshot.windows.values), settings: layout), windows: snapshot.windows) + { + panel.hidePreviewScene(scene, duration: 0.22) + previewClosingCount += 1 + } else { + panel.hide() + } } } @@ -608,16 +883,33 @@ public final class OverviewController: NSObject { direction: action == .left ? -1 : 1) return } + guard !selectionCommitPending || action == .cancel else { return } switch action { case .cancel: close() case .select: chooseSelection() - case .left, .right, .up, .down: + case .left, .right, .up, .down, .firstColumn, .lastColumn, .workspaceUp, .workspaceDown, .workspace: navigate(action) case .moveUp, .moveDown: moveSelectionVertically(action) - } + case .layout(let command): + _ = applyLayoutCommand(command) + } + } + + @discardableResult + public func applyLayoutCommand(_ command: Command, + handler: (@MainActor @Sendable (Command, WindowID, String, MonitorID, WorkspaceID, UInt64) -> Void)? = nil + ) -> Bool { + guard isOpen, !ribbonPrototype, !selectionCommitPending, let snapshot, let selection, + let windowID = selection.windowID, let window = snapshot.windows[windowID], + !snapshot.nativeFullscreenWindowIDs.contains(windowID), window.transientOwnerID == nil, + !window.floating || command == .toggleFloating else { return false } + hasDeferredSelection = true + (handler ?? layoutCommandHandler)(command, windowID, window.appID, selection.location.monitorID, + selection.location.workspaceID, sessionGeneration) + return true } @objc private func closeForSystemTransition(_ notification: Notification) { @@ -625,9 +917,17 @@ public final class OverviewController: NSObject { } private func chooseSelection() { - guard let snapshot, let selection else { return } + guard !selectionCommitPending, let snapshot, let selection else { return } + selectionCommitPending = true + hasDeferredSelection = false + selectionAlignmentFinished = false + selectionNativeReady = !waitsForNativeSelectionCommit alignSelectionOnNextUpdate = true - guard focusSelection(selection, in: snapshot) else { return } + guard focusSelection(selection, in: snapshot) else { + selectionCommitPending = false + close(commitScrollOffsets: true) + return + } closeAfterSelectionAlignment() } @@ -646,19 +946,21 @@ public final class OverviewController: NSObject { guard animationsEnabled, !NSWorkspace.shared.accessibilityDisplayShouldReduceMotion else { - close(commitScrollOffsets: false) + selectionAlignmentFinished = true + finishSelectionCloseIfReady() return } let generation = sessionGeneration DispatchQueue.main.asyncAfter(deadline: .now() + overviewTransitionDuration) { [weak self] in guard let self, isOpen, sessionGeneration == generation else { return } - close(commitScrollOffsets: false) + selectionAlignmentFinished = true + finishSelectionCloseIfReady() } } private func navigate(_ action: OverviewKeyAction) { - guard let snapshot, + guard !selectionCommitPending, let snapshot, let target = navigationTarget( from: selection ?? initialSelection(in: snapshot), action: action, @@ -666,12 +968,28 @@ public final class OverviewController: NSObject { ), target != selection else { return } selection = target - alignSelectionOnNextUpdate = true - guard focusSelection(target, in: snapshot) else { return } + selectionsByWorkspace[target.location.workspaceID] = target + hasDeferredSelection = true + guard let monitor = snapshot.monitors.first(where: { $0.id == target.location.monitorID }), + let activeIndex = monitor.workspaces.firstIndex(where: { $0.id == monitor.activeWorkspace }), + let targetIndex = monitor.workspaces.firstIndex(where: { $0.id == target.location.workspaceID }) + else { return } + var viewport = viewportAnimations[monitor.id]?.to ?? viewports[monitor.id] ?? OverviewViewport() + viewport.workspaceOffset = Double(targetIndex - activeIndex) + if let windowID = target.windowID, let frame = snapshot.monitorFrames[monitor.id] { + var workspace = monitor.workspaces[targetIndex] + if let column = workspace.columns.firstIndex(where: { $0.windows.contains(windowID) }) { + workspace.focusedColumn = column + viewport.horizontalOffsets[workspace.id] = focusedColumnLeftScrollOffset( + workspace: workspace, viewport: frame, windows: Array(snapshot.windows.values), settings: layout) + } + } + animateViewport(on: monitor.id, to: viewport) updatePanels() } private func moveSelectionVertically(_ action: OverviewKeyAction) { + guard !selectionCommitPending else { return } guard let snapshot, case .window(let windowID, let monitorID, let workspaceID) = selection, let window = snapshot.windows[windowID], @@ -752,6 +1070,26 @@ public final class OverviewController: NSObject { let workspace = monitor.workspaces[workspaceIndex] switch action { + case .workspace(let target): + let destination: (Monitor, Workspace)? + switch target { + case .named(let name): + destination = snapshot.monitors.compactMap { candidate in + candidate.workspaces.first(where: { $0.id.rawValue == name }).map { (candidate, $0) } + }.first + case .position(let position): + destination = position > 0 && !monitor.workspaces.isEmpty + ? (monitor, monitor.workspaces[min(position - 1, monitor.workspaces.count - 1)]) : nil + case .relative: + destination = nil + } + guard let (destinationMonitor, destinationWorkspace) = destination else { return selection } + if let remembered = selectionsByWorkspace[destinationWorkspace.id], + remembered.windowID != nil, remembered.isValid(in: snapshot) { + return remembered + } + return firstSelection(in: destinationWorkspace, monitorID: destinationMonitor.id) + ?? .workspace(monitorID: destinationMonitor.id, workspaceID: destinationWorkspace.id) case .left, .right: let delta = action == .left ? -1 : 1 if case .window(let windowID, _, _) = selection, @@ -776,9 +1114,14 @@ public final class OverviewController: NSObject { ) } return firstSelection(in: workspace, monitorID: monitor.id) ?? selection - case .up, .down: - let delta = action == .up ? -1 : 1 - if case .window(let windowID, _, _) = selection, + case .firstColumn, .lastColumn: + guard let column = action == .firstColumn ? workspace.columns.first : workspace.columns.last, + column.windows.indices.contains(column.focusedWindow) else { return selection } + return .window(windowID: column.windows[column.focusedWindow], + monitorID: monitor.id, workspaceID: workspace.id) + case .up, .down, .workspaceUp, .workspaceDown: + let delta = action == .up || action == .workspaceUp ? -1 : 1 + if action == .up || action == .down, case .window(let windowID, _, _) = selection, let columnIndex = workspace.columns.firstIndex(where: { $0.windows.contains(windowID) }), @@ -795,14 +1138,19 @@ public final class OverviewController: NSObject { } let adjacentIndex = workspaceIndex + delta guard monitor.workspaces.indices.contains(adjacentIndex) else { return selection } + let adjacent = monitor.workspaces[adjacentIndex] + if let remembered = selectionsByWorkspace[adjacent.id], + remembered.location.monitorID == monitor.id, remembered.isValid(in: snapshot) { + return remembered + } return firstSelection( - in: monitor.workspaces[adjacentIndex], + in: adjacent, monitorID: monitor.id ) ?? .workspace( monitorID: monitor.id, workspaceID: monitor.workspaces[adjacentIndex].id ) - case .moveUp, .moveDown, .select, .cancel: + case .moveUp, .moveDown, .select, .cancel, .layout: return selection } } @@ -877,6 +1225,9 @@ public final class OverviewController: NSObject { for (monitorID, panel) in panels where requestedMonitorID == nil || requestedMonitorID == monitorID { let target = projection(for: panel, snapshot: snapshot) + let invalidated = panel.invalidateSurfaceScene(ifProjectionChanged: target) + surfaceWindowIDs.subtract(invalidated) + attemptedPreviewWindowIDs.subtract(invalidated) let projection = displayedProjection( target: target, on: monitorID, @@ -884,6 +1235,7 @@ public final class OverviewController: NSObject { reduceMotion: reduceMotion ) projections[monitorID] = projection + panel.view.surfaceWindowIDs = surfaceWindowIDs panel.view.update( snapshot: snapshot, projection: projection, @@ -958,7 +1310,9 @@ public final class OverviewController: NSObject { guard windowPreviewsEnabled, isOpen, previewTask == nil, previewPermissionState != .denied else { return } - let requests = visiblePreviewRequests().filter { + guard let snapshot else { return } + let requests = visiblePreviewRequests(snapshot: snapshot, projections: projections, + selection: selection).filter { !attemptedPreviewWindowIDs.contains($0.windowID) } let hasPendingDesktopCapture = panels.keys.contains { @@ -1055,8 +1409,12 @@ public final class OverviewController: NSObject { ? overviewDesktopFadeDuration : 0 ) } + if results.authorizationDeclined { + previewPermissionState = .denied + cancelDesktopCaptureRetry() + } finishPreviewBatch(generation: generation) - if shouldRetryDesktopCapture { + if shouldRetryDesktopCapture && !results.authorizationDeclined { scheduleDesktopCaptureRetry(generation: generation) } updatePanels(scheduleCaptures: false) @@ -1067,7 +1425,12 @@ public final class OverviewController: NSObject { !Task.isCancelled else { return } previewPendingCount = max(previewPendingCount - 1, 0) defer { presentationChanged() } - guard let monitorID = previewMonitorIDs[result.request.windowID], + guard let monitorID = snapshot?.monitors.first(where: { monitor in + monitor.workspaces.contains { workspace in + workspace.columns.contains { $0.windows.contains(result.request.windowID) } + || workspace.floatingWindows.contains(result.request.windowID) + } + })?.id, snapshot?.windows[result.request.windowID]?.appID == result.request.expectedAppID else { previewFailureCount += 1 @@ -1097,6 +1460,8 @@ public final class OverviewController: NSObject { for: window ) } + // Offscreen captures are ready before navigation, without running an idle fade link. + guard previewMonitorIDs[result.request.windowID] != nil else { return } // A replaced preview cross-fades from the remembered image instead of swapping. if previewRevealStartedAt[result.request.windowID] == nil { previewRevealStartedAt[result.request.windowID] = CACurrentMediaTime() @@ -1198,16 +1563,56 @@ public final class OverviewController: NSObject { } } - private func releaseIdleOverviewResources() { + func releaseIdleOverviewResources() { guard !isOpen else { return } defer { presentationChanged() } - for panel in panels.values { panel.discardImages() } + // Expiry may beat a delayed closing-animation continuation. Hiding first + // prevents cancellation from leaving an input-blocking panel on screen. + for panel in panels.values { panel.hide() } snapshot = nil projections.removeAll() } - private func visiblePreviewRequests() -> [OverviewPreviewRequest] { - guard let snapshot else { return [] } + // One-shot compact previews reuse the existing bounded cache. No inactive streams. + private func prepareIdlePreviews(snapshot: OverviewSnapshot, layout: LayoutSettings, zoom: Double) { + guard CGPreflightScreenCaptureAccess() else { return } + idlePreviewPriorityIDs = Set(snapshot.monitors.compactMap { monitor in + monitor.workspaces.first(where: { $0.id == monitor.activeWorkspace }) + .flatMap { firstSelection(in: $0, monitorID: monitor.id)?.windowID } + }) + pruneRememberedPreviews(for: snapshot) + let projections = panels.mapValues { panel in + projectOverview(snapshot: snapshot, monitorID: panel.monitorID, + bounds: Rect(x: 0, y: 0, width: panel.view.bounds.width, height: panel.view.bounds.height), + viewport: OverviewViewport(), layout: layout, zoom: zoom, includeOffscreenContent: true) + } + let requests = visiblePreviewRequests(snapshot: snapshot, projections: projections, + selection: initialSelection(in: snapshot)).map { request in + let (width, height) = overviewPreviewPixelSize(cardWidth: Double(request.width), + cardHeight: Double(request.height), scale: 1, maximumWidth: 512, maximumHeight: 512) + return OverviewPreviewRequest(windowID: request.windowID, expectedAppID: request.expectedAppID, + width: width, height: height, + blurFadeHeight: max(Int(Double(request.blurFadeHeight) * Double(height) / Double(request.height)), 1)) + } + guard requests != idlePreviewAttempted else { return } + idlePreviewAttempted = requests + idlePreviewTask?.cancel() + let missing = requests.filter { rememberedPreviews.images[$0.windowID] == nil } + guard !missing.isEmpty else { idlePreviewTask = nil; return } + idlePreviewTask = Task { [weak self] in + _ = await captureOverviewImages(previews: missing, desktops: [], previewCompleted: { result in + guard let self, !Task.isCancelled, let image = result.rememberedImage, + let window = snapshot.windows[result.request.windowID] else { return } + self.rememberedPreviews.store(NSImage(cgImage: image, + size: NSSize(width: result.request.width, height: result.request.height)), + byteCost: image.bytesPerRow * image.height, for: window) + }) + } + } + + private func visiblePreviewRequests(snapshot: OverviewSnapshot, + projections: [MonitorID: OverviewProjection], selection: OverviewSelection? + ) -> [OverviewPreviewRequest] { var candidates: [OverviewPreviewCandidate] = [] var anchor: (x: Double, y: Double)? for (monitorID, projection) in projections { @@ -1241,9 +1646,32 @@ public final class OverviewController: NSObject { )) } } - return overviewPreviewCaptureOrder( + return boundedOverviewPreviewRequests(overviewPreviewCaptureOrder( candidates, selectedMonitorID: selection?.location.monitorID, anchor: anchor - ) + )) + } + + private func surfaceRequests( + snapshot: OverviewSnapshot, layout: LayoutSettings, zoom: Double + ) -> [OverviewSurfaceRequest] { + ExperimentalRibbonRenderer.shared.prepare(snapshot: snapshot, layout: layout, + enabled: surfaceTransitionsEnabled || ribbonRepresentationsEnabled, + cornerRadius: ribbonCornerRadius, preloadWorkspace: ribbonRepresentationsEnabled) + return ExperimentalRibbonRenderer.shared.requests + } + + private func currentSurfaceFrames(snapshot: OverviewSnapshot, monitorID: MonitorID) -> [WindowID: OverviewSurfaceFrame]? { + guard surfaceTransitionsEnabled, animationsEnabled, !usesWorkspaceParking, + !NSWorkspace.shared.accessibilityDisplayShouldReduceMotion else { return nil } + ExperimentalRibbonRenderer.shared.cancel() + let projected = Set(projections[monitorID]?.workspaces.first(where: { + $0.workspaceID == snapshot.monitors.first(where: { $0.id == monitorID })?.activeWorkspace + })?.windows.map(\.windowID) ?? []) + let ids = projected.intersection(ExperimentalRibbonRenderer.shared.capturedIDs(on: monitorID)) + let start = CACurrentMediaTime() + let frames = surfaceCapture.availableFrames(windowIDs: ids) + surfaceAcquireMs = (CACurrentMediaTime() - start) * 1_000 + return frames } private func desktopCaptureRequests() -> [OverviewDesktopCaptureRequest] { @@ -1272,7 +1700,14 @@ public final class OverviewController: NSObject { to target: OverviewViewport ) { let current = viewports[monitorID] ?? target - guard current != target else { return } + guard viewportAnimations[monitorID]?.to != target else { return } + if current == target { + if viewportAnimations[monitorID] != nil { + cancelAnimations(on: monitorID) + updatePanels() + } + return + } guard animationsEnabled, !NSWorkspace.shared.accessibilityDisplayShouldReduceMotion, let link = displayLink(on: monitorID) @@ -1323,18 +1758,36 @@ public final class OverviewController: NSObject { target: self, selector: #selector(viewportDisplayLinkDidFire(_:)) ) + let refreshRate = Float(max(screen.maximumFramesPerSecond, 1)) + link.preferredFrameRateRange = CAFrameRateRange( + minimum: refreshRate, maximum: refreshRate, preferred: refreshRate) link.add(to: .main, forMode: .common) viewportDisplayLinks[monitorID] = link displayLinkMonitorIDs[ObjectIdentifier(link)] = monitorID return link } + private func requestViewportFrame(on monitorID: MonitorID) { + guard let link = displayLink(on: monitorID) else { + updatePanels(only: monitorID) + return + } + // Keep every precise delta, but render only the latest accumulated viewport + // once per refresh. The pending flag also submits the final momentum event. + pendingViewportFrames.insert(monitorID) + link.isPaused = false + } + @objc private func viewportDisplayLinkDidFire(_ link: CADisplayLink) { guard let monitorID = displayLinkMonitorIDs[ObjectIdentifier(link)] else { link.isPaused = true return } - let geometryChanged = viewportAnimations[monitorID] != nil + let started = CACurrentMediaTime() + let view = panels[monitorID]?.view + let drawsBefore = view?.drawCount ?? 0 + let geometryChanged = pendingViewportFrames.remove(monitorID) != nil + || viewportAnimations[monitorID] != nil || projectionAnimations[monitorID] != nil let reduceMotion = NSWorkspace.shared.accessibilityDisplayShouldReduceMotion if let animation = viewportAnimations[monitorID] { @@ -1358,10 +1811,25 @@ public final class OverviewController: NSObject { } if geometryChanged { updatePanels(only: monitorID) } let fadePending = updatePreviewFade(on: monitorID) + view?.displayIfNeeded() + if (view?.drawCount ?? 0) > drawsBefore { + renderedFrames += 1 + renderedSeconds += CACurrentMediaTime() - started + if let previous = lastRenderedRefresh[monitorID] { + let gap = started - previous + renderedIntervals += 1 + renderedIntervalSeconds += gap + maximumRenderedGap = max(maximumRenderedGap, gap) + if gap > link.duration * 1.5 { lateRenderedRefreshes += 1 } + } + lastRenderedRefresh[monitorID] = started + } if !fadePending, viewportAnimations[monitorID] == nil, projectionAnimations[monitorID] == nil { link.isPaused = true + lastRenderedRefresh[monitorID] = nil + presentationChanged() } } @@ -1369,9 +1837,12 @@ public final class OverviewController: NSObject { viewportAnimations[monitorID] = nil projectionAnimations[monitorID] = nil viewportDisplayLinks[monitorID]?.isPaused = !updatePreviewFade(on: monitorID) + if viewportDisplayLinks[monitorID]?.isPaused == true { lastRenderedRefresh[monitorID] = nil } } private func stopOverviewAnimations() { + lastRenderedRefresh.removeAll(keepingCapacity: true) + pendingViewportFrames.removeAll(keepingCapacity: true) viewportAnimations.removeAll(keepingCapacity: true) projectionAnimations.removeAll(keepingCapacity: true) for link in viewportDisplayLinks.values { link.invalidate() } @@ -1380,6 +1851,8 @@ public final class OverviewController: NSObject { } private func closePanelsImmediately() { + // Panel recreation must preserve the warmed window surfaces for the opening handoff. + // Capture lifetime is controlled by prepare() and memory pressure independently. stopOverviewAnimations() for panel in panels.values { panel.close() } panels.removeAll(keepingCapacity: true) @@ -1418,7 +1891,8 @@ extension OverviewController: OverviewViewDelegate { _ view: OverviewView, clickedAt point: NSPoint ) { - guard !ribbonPrototype else { return } + guard isOpen, !selectionCommitPending else { return } + guard isOpen, !selectionCommitPending, !ribbonPrototype else { return } guard let projection = projections[view.monitorID], let hit = projection.hitTest(OverviewPoint(x: point.x, y: point.y)), let snapshot @@ -1426,13 +1900,13 @@ extension OverviewController: OverviewViewDelegate { activateMonitorHandler(view.monitorID) switch hit { case .window(let windowID, let monitorID, let workspaceID): - guard let window = snapshot.windows[windowID] else { return } - close() - focusWindowHandler(windowID, window.appID, monitorID, workspaceID) + guard snapshot.windows[windowID] != nil else { return } + selection = .window(windowID: windowID, monitorID: monitorID, workspaceID: workspaceID) case .workspace(let monitorID, let workspaceID): - close() - focusWorkspaceHandler(monitorID, workspaceID) + selection = .workspace(monitorID: monitorID, workspaceID: workspaceID) } + if let selection { selectionsByWorkspace[selection.location.workspaceID] = selection } + chooseSelection() } func overviewView( @@ -1440,7 +1914,8 @@ extension OverviewController: OverviewViewDelegate { beganDragging windowID: WindowID, at screenPoint: NSPoint ) { - guard !ribbonPrototype else { return } + guard isOpen, !selectionCommitPending else { return } + guard isOpen, !selectionCommitPending, !ribbonPrototype else { return } guard let snapshot, let window = snapshot.windows[windowID], let location = snapshot.location(of: windowID), @@ -1467,6 +1942,7 @@ extension OverviewController: OverviewViewDelegate { _ view: OverviewView, draggedTo screenPoint: NSPoint ) { + guard isOpen, !selectionCommitPending else { return } updateDrag(at: screenPoint) } @@ -1474,6 +1950,7 @@ extension OverviewController: OverviewViewDelegate { _ view: OverviewView, endedDraggingAt screenPoint: NSPoint ) { + guard isOpen, !selectionCommitPending else { return } guard let drag else { return } edgeScrollTimer?.invalidate() edgeScrollTimer = nil @@ -1495,6 +1972,7 @@ extension OverviewController: OverviewViewDelegate { hasPreciseScrollingDeltas: Bool, at point: NSPoint ) { + guard isOpen, !selectionCommitPending else { return } guard let snapshot, let monitor = snapshot.monitors.first(where: { $0.id == view.monitorID }) else { return } @@ -1533,7 +2011,7 @@ extension OverviewController: OverviewViewDelegate { if hasPreciseScrollingDeltas { guard target != viewport else { return } viewports[view.monitorID] = target - updatePanels() + requestViewportFrame(on: view.monitorID) } else { animateViewport(on: view.monitorID, to: target) } @@ -1544,6 +2022,7 @@ extension OverviewController: OverviewViewDelegate { rightDraggedBy deltaX: Double, at point: NSPoint ) { + guard isOpen, !selectionCommitPending else { return } guard let snapshot, let monitor = snapshot.monitors.first(where: { $0.id == view.monitorID }), let viewport = viewports[view.monitorID], @@ -1568,7 +2047,7 @@ extension OverviewController: OverviewViewDelegate { horizontalWorkspaceID: workspaceID, maximumHorizontalOffset: maximumOffset ) - updatePanels() + requestViewportFrame(on: view.monitorID) } func overviewView( @@ -1576,6 +2055,7 @@ extension OverviewController: OverviewViewDelegate { pageWorkspace workspaceID: WorkspaceID, direction: Int ) { + guard isOpen, !selectionCommitPending else { return } guard let snapshot, let monitor = snapshot.monitors.first(where: { $0.id == view.monitorID }), let maximumOffset = maximumHorizontalOffset(for: workspaceID, on: monitor) @@ -1624,6 +2104,7 @@ extension OverviewController: OverviewViewDelegate { sourceWorkspaceID: WorkspaceID, target: OverviewDropTarget ) { + guard isOpen, !selectionCommitPending else { return } let location = target.location selection = .window( windowID: windowID, diff --git a/Sources/DefiMacOS/OverviewPanel.swift b/Sources/DefiMacOS/OverviewPanel.swift index ef759c98..8809cff5 100644 --- a/Sources/DefiMacOS/OverviewPanel.swift +++ b/Sources/DefiMacOS/OverviewPanel.swift @@ -26,6 +26,17 @@ final class OverviewPanel { let window: NSPanel let view: OverviewView private let desktopView: NSView + private let rootView: NSView + private let glassView: NSGlassEffectView + private var surfaceScene: OverviewSurfaceScene? + private var previewClosingScene: OverviewPreviewClosingScene? + var hasSurfaceScene: Bool { surfaceScene != nil || previewClosingScene != nil } + var surfacePresentedFrameCount: Int { surfaceScene?.presentedFrameCount ?? 0 } + private var surfaceTask: Task? + private var invalidatedSurfaceWindowIDs = Set() + private var surfaceGeneration: UInt64 = 0 + var openingTransitionChanged: ((String) -> Void)? + private var openingTransitionInProgress = false private var desktopImageTask: Task? init( @@ -63,7 +74,7 @@ final class OverviewPanel { .ignoresCycle, ] window.sharingType = .readOnly - let rootView = NSView(frame: NSRect(origin: .zero, size: screen.frame.size)) + rootView = NSView(frame: NSRect(origin: .zero, size: screen.frame.size)) rootView.wantsLayer = true rootView.layer?.backgroundColor = usesCapturedDesktop ? NSColor.black.cgColor @@ -77,7 +88,7 @@ final class OverviewPanel { desktopView.layer?.contentsScale = screen.backingScaleFactor desktopView.layer?.masksToBounds = true desktopView.autoresizingMask = [.width, .height] - let glassView = NSGlassEffectView( + glassView = NSGlassEffectView( frame: NSRect(origin: .zero, size: screen.frame.size) ) glassView.style = .regular @@ -109,6 +120,7 @@ final class OverviewPanel { } func show(fadeDuration: TimeInterval = 0) { + cancelSurfaceTransition() view.wantsLayer = true if fadeDuration > 0 { window.alphaValue = 0 @@ -150,10 +162,115 @@ final class OverviewPanel { } func hide() { + cancelSurfaceTransition(reason: "hide") orderOut() + discardImages() + } + + func showSurfaceScene(_ scene: OverviewSurfaceScene, windowIDs: Set, duration: TimeInterval) { + cancelSurfaceTransition() + surfaceScene = scene + openingTransitionInProgress = true + openingTransitionChanged?("started(\(scene.openingContentDescription))") + let generation = surfaceGeneration + view.suppressedSurfaceWindowIDs = windowIDs + glassView.alphaValue = 0 + rootView.layer?.addSublayer(scene.layer) + window.alphaValue = 1 + loadWallpaperIfNeeded() + window.orderFrontRegardless() + scene.animate(opening: true, duration: duration) + NSAnimationContext.runAnimationGroup { context in + context.duration = duration + glassView.animator().alphaValue = 1 + } + surfaceTask = Task { [weak self] in + do { try await Task.sleep(for: .seconds(duration)) } catch { return } + guard !Task.isCancelled, let self, surfaceGeneration == generation else { return } + openingTransitionInProgress = false + openingTransitionChanged?("completed") + glassView.alphaValue = 1 + view.suppressedSurfaceWindowIDs.subtract(scene.finishOpening()) + if scene.nativeFrames.isEmpty { + scene.layer.removeFromSuperlayer() + surfaceScene = nil + } + surfaceTask = nil + } + } + + func hideSurfaceSceneIfUnchanged(duration: TimeInterval) -> Bool { + guard let surfaceScene, let screen = window.screen, + surfaceScene.matchesNativeFrames(screen: screen) else { return false } + if openingTransitionInProgress { + openingTransitionInProgress = false + openingTransitionChanged?("interrupted(exit)") + } + surfaceTask?.cancel() + surfaceGeneration &+= 1 + let generation = surfaceGeneration + view.suppressedSurfaceWindowIDs = Set(surfaceScene.nativeFrames.keys) + rootView.layer?.addSublayer(surfaceScene.layer) + surfaceScene.animate(opening: false, duration: duration) + NSAnimationContext.runAnimationGroup { context in + context.duration = duration + glassView.animator().alphaValue = 0 + } + surfaceTask = Task { [weak self] in + do { try await Task.sleep(for: .seconds(duration)) } catch { return } + guard !Task.isCancelled, let self, surfaceGeneration == generation else { return } + hide() + } + return true + } + + func hidePreviewScene(_ scene: OverviewPreviewClosingScene, duration: TimeInterval) { + cancelSurfaceTransition() + previewClosingScene = scene + let generation = surfaceGeneration + rootView.layer?.addSublayer(scene.layer) + scene.animate(duration: duration) + NSAnimationContext.runAnimationGroup { context in + context.duration = duration + glassView.animator().alphaValue = 0 + } + surfaceTask = Task { [weak self] in + do { try await Task.sleep(for: .seconds(duration)) } catch { return } + guard !Task.isCancelled, let self, surfaceGeneration == generation else { return } + hide() + } + } + + private func cancelSurfaceTransition(reason: String = "replaced") { + if openingTransitionInProgress { + openingTransitionInProgress = false + openingTransitionChanged?("interrupted(\(reason))") + } + surfaceGeneration &+= 1 + surfaceTask?.cancel() + surfaceTask = nil + surfaceScene?.layer.removeFromSuperlayer() + surfaceScene = nil + previewClosingScene?.layer.removeFromSuperlayer() + previewClosingScene = nil + view.suppressedSurfaceWindowIDs = [] + glassView.layer?.removeAllAnimations() + glassView.alphaValue = 1 + } + + @discardableResult + func invalidateSurfaceScene(ifProjectionChanged projection: OverviewProjection) -> Set { + var ids = invalidatedSurfaceWindowIDs + invalidatedSurfaceWindowIDs = [] + if let surfaceScene, surfaceScene.projection != projection { + ids.formUnion(surfaceScene.nativeFrames.keys) + cancelSurfaceTransition(reason: "projection-changed") + } + return ids } func discardImages() { + cancelSurfaceTransition() desktopImageTask?.cancel() desktopImageTask = nil desktopView.layer?.contents = nil @@ -167,6 +284,7 @@ final class OverviewPanel { } func close() { + cancelSurfaceTransition() orderOut() discardImages() window.close() diff --git a/Sources/DefiMacOS/OverviewPreviewCache.swift b/Sources/DefiMacOS/OverviewPreviewCache.swift index 6509d796..81b221e2 100644 --- a/Sources/DefiMacOS/OverviewPreviewCache.swift +++ b/Sources/DefiMacOS/OverviewPreviewCache.swift @@ -48,4 +48,15 @@ final class OverviewPreviewCache { func removeAll() { entries.removeAll() } + + func retain(_ windowIDs: Set, maximumBytes: Int) { + var remaining = max(maximumBytes, 0) + for id in entries.keys.sorted(by: { $0.rawValue < $1.rawValue }) { + guard windowIDs.contains(id), let entry = entries[id], entry.byteCost <= remaining else { + entries[id] = nil + continue + } + remaining -= entry.byteCost + } + } } diff --git a/Sources/DefiMacOS/OverviewPreviewCapture.swift b/Sources/DefiMacOS/OverviewPreviewCapture.swift index 9b8cfd08..e83566dc 100644 --- a/Sources/DefiMacOS/OverviewPreviewCapture.swift +++ b/Sources/DefiMacOS/OverviewPreviewCapture.swift @@ -3,6 +3,7 @@ import CoreImage import DefiModel import Foundation import ScreenCaptureKit +import os // Small placeholders survive between sessions; full-resolution captures stay session-local. func compactOverviewPreview(_ image: CGImage) -> CGImage? { @@ -132,6 +133,7 @@ struct OverviewDesktopCaptureRequest: Sendable { struct OverviewCaptureResults: Sendable { let previews: [OverviewPreviewCaptureResult] let desktops: [MonitorID: CGImage] + var authorizationDeclined = false } func overviewPreviewCacheCanStore( @@ -153,25 +155,57 @@ func overviewPreviewOwnerMatches( capturedAppID == expectedAppID } -// Shared by window and desktop captures (docs/plans/overview.md): at most two at once. +// Workers may overlap image processing, but only one macOS capture/consent call +// may be outstanding. Cancellation does not dismiss a system consent dialog. let overviewPreviewMaximumConcurrentCaptures = 2 +let overviewCaptureLimiter = OverviewCaptureLimiter(limit: 1) +private let overviewCaptureLogger = Logger(subsystem: "com.quentin.defi", category: "overview-capture") actor OverviewCaptureLimiter { private var available: Int - private var waiters: [CheckedContinuation] = [] + private var waiters: [(UUID, CheckedContinuation)] = [] + private(set) var authorizationDeclined = false + var waitingCount: Int { waiters.count } init(limit: Int = overviewPreviewMaximumConcurrentCaptures) { available = limit } - func acquire() async { + @discardableResult + func acquire() async -> Bool { + guard !authorizationDeclined, !Task.isCancelled else { return false } if available > 0 { available -= 1 - return + return true } - await withCheckedContinuation { waiters.append($0) } + let id = UUID() + return await withTaskCancellationHandler { + await withCheckedContinuation { continuation in + if Task.isCancelled { continuation.resume(returning: false) } + else { waiters.append((id, continuation)) } + } + } onCancel: { + Task { await self.cancelWaiter(id) } + } + } + + func release(error: NSError? = nil) { + if let error { + overviewCaptureLogger.error("Capture failed: domain=\(error.domain, privacy: .public) code=\(error.code)") + // SCStreamErrorUserDeclined: no background retry after denied direct access. + // A deliberate daemon restart starts a new authorization attempt. + if error.domain == SCStreamErrorDomain && error.code == -3801 { + authorizationDeclined = true + overviewCaptureLogger.notice("Capture suspended until restart after authorization refusal") + for (_, waiter) in waiters { waiter.resume(returning: false) } + waiters.removeAll() + } + } + if waiters.isEmpty { available += 1 } + else { waiters.removeFirst().1.resume(returning: true) } } - func release() { - if waiters.isEmpty { available += 1 } else { waiters.removeFirst().resume() } + private func cancelWaiter(_ id: UUID) { + guard let index = waiters.firstIndex(where: { $0.0 == id }) else { return } + waiters.remove(at: index).1.resume(returning: false) } } @@ -181,8 +215,8 @@ func overviewPreviewPixelSize( cardWidth: Double, cardHeight: Double, scale: Double, - maximumWidth: Double = 1_600, - maximumHeight: Double = 1_200 + maximumWidth: Double = 1_024, + maximumHeight: Double = 768 ) -> (width: Int, height: Int) { let width = max(cardWidth * scale, 1) let height = max(cardHeight * scale, 1) @@ -200,6 +234,26 @@ struct OverviewPreviewCandidate { let centerY: Double } +// Bound the whole decoded batch, including offscreen cards, rather than only +// limiting each image. Preserve every preview at lower resolution when needed. +func boundedOverviewPreviewRequests(_ requests: [OverviewPreviewRequest], + maximumBytes: Int = 128 * 1024 * 1024) -> [OverviewPreviewRequest] { + let total = requests.reduce(0.0) { $0 + Double($1.width) * Double($1.height) * 4 } + guard maximumBytes > 0, total > 0 else { return [] } + let scale = min(sqrt(Double(maximumBytes) / total), 1) + var remaining = maximumBytes + return requests.compactMap { request in + let width = max(Int(Double(request.width) * scale), 1) + let height = max(Int(Double(request.height) * scale), 1) + let bytes = width * height * 4 + guard bytes <= remaining else { return nil } + remaining -= bytes + return OverviewPreviewRequest(windowID: request.windowID, expectedAppID: request.expectedAppID, + width: width, height: height, + blurFadeHeight: max(Int(Double(request.blurFadeHeight) * scale), 1)) + } +} + // Selected monitor first, nearest to the anchor outward; other monitors keep their order. func overviewPreviewCaptureOrder( _ candidates: [OverviewPreviewCandidate], @@ -227,6 +281,7 @@ func overviewPreviewCaptureOrder( func runOverviewPreviewCaptures( _ requests: [OverviewPreviewRequest], maximumConcurrent: Int = overviewPreviewMaximumConcurrentCaptures, + retainResults: Bool = true, completed: @escaping @Sendable (OverviewPreviewCaptureResult) async -> Void = { _ in }, capture: @escaping @Sendable (OverviewPreviewRequest) async -> OverviewPreviewCaptureResult @@ -244,7 +299,7 @@ func runOverviewPreviewCaptures( } var results: [(Int, OverviewPreviewCaptureResult)] = [] while let result = await group.next() { - results.append(result) + if retainResults { results.append(result) } await completed(result.1) guard !Task.isCancelled else { group.cancelAll() @@ -269,12 +324,21 @@ func captureOverviewImages( previewCompleted: @escaping @MainActor @Sendable (OverviewPreviewCaptureResult) -> Void ) async -> OverviewCaptureResults { let renderingContext = overviewPreviewRenderingContext - let limiter = OverviewCaptureLimiter() + // Old sessions may still be inside a non-cancellable macOS capture. Share + // slots across sessions rather than multiplying WindowServer work on reopen. + let limiter = overviewCaptureLimiter do { - let content = try await SCShareableContent.excludingDesktopWindows( - true, - onScreenWindowsOnly: false - ) + guard await limiter.acquire() else { return await OverviewCaptureResults(previews: [], desktops: [:], + authorizationDeclined: limiter.authorizationDeclined) } + let content: SCShareableContent + do { + try Task.checkCancellation() + content = try await SCShareableContent.excludingDesktopWindows(true, onScreenWindowsOnly: false) + } catch { + await limiter.release(error: error is CancellationError ? nil : error as NSError) + throw error + } + await limiter.release() // Desktop backgrounds capture alongside window previews instead of ahead of them. let desktopTask = Task { @MainActor in var desktops: [MonitorID: CGImage] = [:] @@ -292,18 +356,20 @@ func captureOverviewImages( configuration.height = request.height configuration.showsCursor = false configuration.capturesAudio = false - await limiter.acquire() + guard await limiter.acquire() else { break } // A worker cancelled while queued gives its slot back without capturing. guard !Task.isCancelled else { await limiter.release() break } - let image = try? await SCScreenshotManager.captureImage( - contentFilter: filter, - configuration: configuration - ) - await limiter.release() - if let image { desktops[request.monitorID] = image } + do { + let image = try await SCScreenshotManager.captureImage( + contentFilter: filter, configuration: configuration) + await limiter.release() + desktops[request.monitorID] = image + } catch { + await limiter.release(error: error as NSError) + } } return desktops } @@ -316,13 +382,14 @@ func captureOverviewImages( limiter: limiter ) return await withTaskCancellationHandler { - let previews = await runOverviewPreviewCaptures(requests, completed: { result in + let previews = await runOverviewPreviewCaptures(requests, retainResults: false, completed: { result in await previewCompleted(result) }) { request in await batch.capture(request) } let desktops = await desktopTask.value - return OverviewCaptureResults(previews: previews, desktops: desktops) + return await OverviewCaptureResults(previews: previews, desktops: desktops, + authorizationDeclined: limiter.authorizationDeclined) } onCancel: { desktopTask.cancel() } @@ -330,11 +397,11 @@ func captureOverviewImages( for request in requests where !Task.isCancelled { previewCompleted(OverviewPreviewCaptureResult(request: request, image: nil)) } - return OverviewCaptureResults( + return await OverviewCaptureResults( previews: requests.map { OverviewPreviewCaptureResult(request: $0, image: nil) }, - desktops: [:] + desktops: [:], authorizationDeclined: limiter.authorizationDeclined ) } } @@ -374,7 +441,9 @@ private final class OverviewScreenCaptureBatch { configuration.showsCursor = false configuration.capturesAudio = false do { - await limiter.acquire() + guard await limiter.acquire() else { + return OverviewPreviewCaptureResult(request: request, image: nil) + } guard !Task.isCancelled else { await limiter.release() return OverviewPreviewCaptureResult(request: request, image: nil) @@ -386,7 +455,7 @@ private final class OverviewScreenCaptureBatch { configuration: configuration ) } catch { - await limiter.release() + await limiter.release(error: error as NSError) throw error } await limiter.release() diff --git a/Sources/DefiMacOS/OverviewPreviewClosingScene.swift b/Sources/DefiMacOS/OverviewPreviewClosingScene.swift new file mode 100644 index 00000000..d3c90ef7 --- /dev/null +++ b/Sources/DefiMacOS/OverviewPreviewClosingScene.swift @@ -0,0 +1,94 @@ +import AppKit +import DefiCore +import DefiModel +import QuartzCore + +// Reuse the compact images already displayed when navigation discarded the +// opening surface scene. No capture, encoding, or native mutation at closing. +@MainActor +final class OverviewPreviewClosingScene { + let layer = CALayer() + private let entries: [(CALayer, CGRect, CGRect)] + private let cornerRadius: CGFloat + + init?(projection: OverviewProjection, workspaceID: WorkspaceID, screen: NSScreen, + previews: [WindowID: NSImage], cornerRadius: Double = 12, + targets: [FrameAssignment], windows: [WindowID: Window]) { + guard let workspace = projection.workspaces.first(where: { $0.workspaceID == workspaceID }) + else { return nil } + self.cornerRadius = max(cornerRadius, 0) + let primaryHeight = NSScreen.screens.first?.frame.height ?? 0 + let origin = CGPoint(x: screen.frame.minX, y: primaryHeight - screen.frame.maxY) + let viewport = CGRect(origin: origin, size: screen.frame.size) + let desired = Dictionary(uniqueKeysWithValues: targets.map { assignment in + (assignment.windowID, CGRect(x: assignment.frame.x, y: assignment.frame.y, + width: assignment.frame.width, height: assignment.frame.height)) + }).filter { $0.value.intersects(viewport) } + guard !desired.isEmpty else { return nil } + let infos = CGWindowListCopyWindowInfo(.optionAll, kCGNullWindowID) as? [[String: Any]] ?? [] + var observed: [WindowID: CGRect] = [:] + for info in infos { + guard let number = info[kCGWindowNumber as String] as? NSNumber, + let owner = info[kCGWindowOwnerPID as String] as? NSNumber, + let bounds = info[kCGWindowBounds as String] as? [String: Any], + let rect = CGRect(dictionaryRepresentation: bounds as CFDictionary) else { continue } + let id = WindowID(rawValue: number.uint64Value) + if windows[id]?.processID == owner.int32Value { observed[id] = rect } + } + CATransaction.begin() + CATransaction.setDisableActions(true) + defer { CATransaction.commit() } + layer.frame = CGRect(origin: .zero, size: screen.frame.size) + layer.isGeometryFlipped = true + layer.masksToBounds = true + var entries: [(CALayer, CGRect, CGRect)] = [] + for card in workspace.windows where desired[card.windowID] != nil { + guard let target = desired[card.windowID], let actual = observed[card.windowID], + abs(actual.minX - target.minX) <= 2, abs(actual.minY - target.minY) <= 2, + abs(actual.width - target.width) <= 2, abs(actual.height - target.height) <= 2, + let image = previews[card.windowID]?.cgImage(forProposedRect: nil, context: nil, hints: nil), + card.frame.width > 0, card.frame.height > 0 else { return nil } + let source = CGRect(x: card.frame.x, y: card.frame.y, + width: card.frame.width, height: card.frame.height) + let item = CALayer() + item.anchorPoint = .zero + item.bounds = CGRect(origin: .zero, size: source.size) + item.position = source.origin + item.contents = image + item.contentsGravity = .resizeAspectFill + item.contentsScale = screen.backingScaleFactor + item.cornerRadius = self.cornerRadius + item.cornerCurve = .continuous + item.masksToBounds = true + layer.addSublayer(item) + entries.append((item, source, target.offsetBy(dx: -origin.x, dy: -origin.y))) + } + guard entries.count == desired.count else { return nil } + self.entries = entries + } + + func animate(duration: TimeInterval) { + CATransaction.begin() + CATransaction.setDisableActions(true) + for (item, source, target) in entries { + item.position = target.origin + item.transform = CATransform3DMakeScale(target.width / source.width, target.height / source.height, 1) + let radius = CABasicAnimation(keyPath: "cornerRadius") + radius.fromValue = cornerRadius + item.cornerRadius = cornerRadius * source.width / target.width + radius.toValue = item.cornerRadius + let position = CABasicAnimation(keyPath: "position") + position.fromValue = NSValue(point: source.origin) + position.toValue = NSValue(point: target.origin) + let transform = CABasicAnimation(keyPath: "transform") + transform.fromValue = NSValue(caTransform3D: CATransform3DIdentity) + transform.toValue = NSValue(caTransform3D: item.transform) + let animation = CAAnimationGroup() + animation.animations = [position, transform, radius] + animation.duration = duration + animation.timingFunction = CAMediaTimingFunction(name: .easeInEaseOut) + item.add(animation, forKey: "overview-preview-close") + } + CATransaction.commit() + } +} diff --git a/Sources/DefiMacOS/OverviewSurfaceCapture.swift b/Sources/DefiMacOS/OverviewSurfaceCapture.swift new file mode 100644 index 00000000..c54fd562 --- /dev/null +++ b/Sources/DefiMacOS/OverviewSurfaceCapture.swift @@ -0,0 +1,249 @@ +import AppKit +import AVFoundation +import CoreMedia +import CoreVideo +import DefiCore +import DefiModel +import IOSurface +import ScreenCaptureKit + +// Experimental public-API path. Conservative screenshot/renderer overlap budget. +// No persistent streams, bitmap copies or frame history. +let overviewSurfacePoolBudget = 256 * 1_024 * 1_024 + +struct OverviewSurfaceRequest: Equatable { + let windowID: WindowID + let appID: String + let processID: Int32 + let width: Int + let height: Int + + var estimatedPoolBytes: Int { + // Four NV12 buffers, including row/height alignment allowance. GPU renderer + // allocations are additional and must be measured separately. + ((width + 255) / 256 * 256) * ((height + 15) / 16 * 16) * 6 + } +} + +func overviewSurfaceRequestsFit(_ requests: [OverviewSurfaceRequest]) -> Bool { + requests.reduce(0) { $0 + $1.estimatedPoolBytes } <= overviewSurfacePoolBudget +} + +func overviewReadySurfaceIDs(requested: Set, captured: Set, + displayReady: Set) -> Set { + requested.intersection(captured).intersection(displayReady) +} + +struct OverviewSurfaceFrame { + // Keep the buffer alive while its surface is assigned to a layer. + let buffer: CVPixelBuffer + let sourceSize: CGSize + var surface: IOSurface? { CVPixelBufferGetIOSurface(buffer)?.takeUnretainedValue() } + var width: Int { CVPixelBufferGetWidth(buffer) } + var height: Int { CVPixelBufferGetHeight(buffer) } +} + +func overviewSurfaceMatchesNativeSize(source: CGSize, native: Rect) -> Bool { + source.width.isFinite && source.height.isFinite + && abs(source.width - native.width) < 3 + && abs(source.height - native.height) < 3 +} + +// One-shot samples only: no persistent capture sessions or per-frame history. +@MainActor +final class OverviewSurfaceCapture { + static let shared = OverviewSurfaceCapture() + private struct Capture { + let request: OverviewSurfaceRequest + let frame: OverviewSurfaceFrame + } + private var captures: [WindowID: Capture] = [:] + private var displayLayers: [WindowID: AVSampleBufferDisplayLayer] = [:] + private var requested: [OverviewSurfaceRequest] = [] + private var desiredRequests: [OverviewSurfaceRequest] = [] + private var preparation: Task? + private var deferredPreparation: Task? + private var pendingRequests: [OverviewSurfaceRequest]? + private var preparationDeadline: Task? + private var generation: UInt64 = 0 + private var lastPreparationAt: TimeInterval = -.infinity + private var failedRequests: [OverviewSurfaceRequest] = [] + private(set) var state = "disabled" + var streamCount: Int { 0 } + // Conservative allowance for screenshot/renderer overlap, not total GPU RAM. + var estimatedPoolBytes: Int { captures.values.reduce(0) { $0 + $1.request.estimatedPoolBytes } } + + func prepare(_ requests: [OverviewSurfaceRequest], enabled: Bool) { + guard enabled, CGPreflightScreenCaptureAccess() else { + stop(state: enabled ? "permission" : "disabled"); return + } + guard !requests.isEmpty, overviewSurfaceRequestsFit(requests) else { + stop(state: requests.isEmpty ? "empty" : "budget"); return + } + desiredRequests = requests + let desired = Dictionary(uniqueKeysWithValues: requests.map { ($0.windowID, $0) }) + captures = captures.filter { desired[$0.key] == $0.value.request } + displayLayers = displayLayers.filter { captures[$0.key] != nil } + state = captures.count == requests.count ? "ready" : "warming" + let now = CACurrentMediaTime() + if requested == requests { + pendingRequests = nil + deferredPreparation?.cancel(); deferredPreparation = nil + if preparation != nil { return } + if failedRequests == requests && now - lastPreparationAt < 5 { return } + } + if preparation != nil || now - lastPreparationAt < 1 { + pendingRequests = requests + if deferredPreparation == nil { + let delay = max(0.1, 1 - (now - lastPreparationAt)) + deferredPreparation = Task { [weak self] in + do { try await Task.sleep(for: .seconds(delay)) } catch { return } + guard let self, let latest = pendingRequests else { return } + pendingRequests = nil; deferredPreparation = nil + prepare(latest, enabled: true) + } + } + return + } + pendingRequests = nil + deferredPreparation?.cancel(); deferredPreparation = nil + generation &+= 1 + let token = generation + preparation?.cancel() + preparationDeadline?.cancel() + preparationDeadline = Task { [weak self] in + do { try await Task.sleep(for: .seconds(3)) } catch { return } + guard let self, generation == token, preparation != nil else { return } + generation &+= 1 + preparation?.cancel(); preparation = nil + failedRequests = requests; state = "timeout" + preparationDeadline = nil + } + requested = requests + lastPreparationAt = now + preparation = Task { [weak self] in + guard let self else { return } + do { + guard await overviewCaptureLimiter.acquire() else { throw SurfaceCaptureError.captureUnavailable } + let content: SCShareableContent + do { + try Task.checkCancellation() + content = try await SCShareableContent.excludingDesktopWindows(true, onScreenWindowsOnly: false) + } catch { + await overviewCaptureLimiter.release(error: error is CancellationError ? nil : error as NSError) + throw error + } + await overviewCaptureLimiter.release() + for request in requests { + guard !Task.isCancelled, generation == token else { return } + guard let window = content.windows.first(where: { + UInt64($0.windowID) == request.windowID.rawValue + && $0.owningApplication?.bundleIdentifier == request.appID + && $0.owningApplication?.processID == request.processID + }) else { throw SurfaceCaptureError.missingWindow } + let config = SCStreamConfiguration() + config.width = request.width; config.height = request.height + config.pixelFormat = kCVPixelFormatType_420YpCbCr8BiPlanarFullRange + config.colorMatrix = CGDisplayStream.yCbCrMatrix_ITU_R_709_2 + config.showsCursor = false; config.capturesAudio = false + config.ignoreShadowsSingleWindow = true + guard await overviewCaptureLimiter.acquire() else { throw SurfaceCaptureError.captureUnavailable } + guard !Task.isCancelled, generation == token else { + await overviewCaptureLimiter.release() + return + } + let sample: CMSampleBuffer + do { + sample = try await SCScreenshotManager.captureSampleBuffer( + contentFilter: SCContentFilter(desktopIndependentWindow: window), configuration: config) + } catch { + await overviewCaptureLimiter.release(error: error as NSError) + throw error + } + await overviewCaptureLimiter.release() + guard !Task.isCancelled, generation == token else { return } + guard let buffer = CMSampleBufferGetImageBuffer(sample) else { + throw SurfaceCaptureError.missingPixels + } + let frame = OverviewSurfaceFrame(buffer: buffer, sourceSize: window.frame.size) + guard frame.surface != nil, frame.width == request.width, frame.height == request.height else { + throw SurfaceCaptureError.missingPixels + } + // Decode the replacement separately: warming must leave the currently + // usable layer intact if opening cancels this refresh. + let layer = AVSampleBufferDisplayLayer() + layer.videoGravity = .resize + guard enqueueWindowSurface(frame, on: layer) else { throw SurfaceCaptureError.missingPixels } + for _ in 0..<30 where !layer.isReadyForDisplay { + try await Task.sleep(for: .milliseconds(10)) + } + guard !Task.isCancelled, generation == token else { return } + guard layer.isReadyForDisplay else { throw SurfaceCaptureError.missingPixels } + guard desiredRequests.contains(request) else { continue } + captures[request.windowID] = Capture(request: request, frame: frame) + displayLayers[request.windowID] = layer + } + guard generation == token else { return } + preparationDeadline?.cancel(); preparationDeadline = nil + preparation = nil; failedRequests = [] + state = captures.count == desiredRequests.count ? "ready" : "warming" + if state == "warming" { + // A skipped request may have become desired again while another was capturing. + prepare(desiredRequests, enabled: true) + } + } catch { + guard generation == token else { return } + preparationDeadline?.cancel(); preparationDeadline = nil + preparation = nil; failedRequests = requests + state = captures.isEmpty ? "failed" : "partial" + } + } + } + + // An opening scene owns one fixed image set, preventing refresh swaps during zoom. + func freeze() { + deferredPreparation?.cancel(); deferredPreparation = nil; pendingRequests = nil + generation &+= 1 + preparation?.cancel(); preparation = nil + preparationDeadline?.cancel(); preparationDeadline = nil + state = captures.isEmpty ? "empty" : "ready" + } + + func displayLayer(for windowID: WindowID) -> AVSampleBufferDisplayLayer? { + guard let layer = displayLayers[windowID], layer.isReadyForDisplay else { return nil } + return layer + } + + // Keep the last compatible photo while a refresh is pending or fails. Request + // changes still invalidate owner/dimensions, and the fixed pool remains bounded. + func frames(windowIDs: Set) -> [WindowID: OverviewSurfaceFrame]? { + guard !windowIDs.isEmpty else { return nil } + var result: [WindowID: OverviewSurfaceFrame] = [:] + for id in windowIDs { + guard let capture = captures[id] else { return nil } + result[id] = capture.frame + } + return result + } + + // Opening the overview can animate a subset. Ribbon replacement still uses + // frames(windowIDs:) so an incomplete capture never replaces native windows. + func availableFrames(windowIDs: Set) -> [WindowID: OverviewSurfaceFrame]? { + let ready = overviewReadySurfaceIDs(requested: windowIDs, + captured: Set(captures.keys), + displayReady: Set(windowIDs.filter { displayLayer(for: $0) != nil })) + return frames(windowIDs: ready) + } + + func stop(state: String = "disabled") { + deferredPreparation?.cancel(); deferredPreparation = nil; pendingRequests = nil + generation &+= 1 + preparation?.cancel(); preparation = nil + preparationDeadline?.cancel(); preparationDeadline = nil + captures = [:]; displayLayers = [:]; requested = []; failedRequests = [] + lastPreparationAt = -.infinity + self.state = state + } + + private enum SurfaceCaptureError: Error { case missingWindow, missingPixels, captureUnavailable } +} diff --git a/Sources/DefiMacOS/OverviewSurfaceScene.swift b/Sources/DefiMacOS/OverviewSurfaceScene.swift new file mode 100644 index 00000000..e04c246c --- /dev/null +++ b/Sources/DefiMacOS/OverviewSurfaceScene.swift @@ -0,0 +1,191 @@ +import AppKit +import AVFoundation +import DefiCore +import DefiModel +import QuartzCore + +// A short-lived layer scene: Core Animation moves existing textures rather than +// repainting the overview or sending frame writes to applications on every refresh. +@MainActor +final class OverviewSurfaceScene { + let layer = CALayer() + let presentedFrameCount = 1 + private(set) var openingContentDescription = "none" + private var entries: [(WindowID, CALayer, CGRect, CGRect)] + private var fallbackIDs: Set = [] + private let frames: [WindowID: OverviewSurfaceFrame] + private(set) var nativeFrames: [WindowID: CGRect] + let projection: OverviewProjection + private let nativeOwners: [WindowID: Int32] + private let cornerRadius: CGFloat + + init?( + projection: OverviewProjection, workspaceID: WorkspaceID, screen: NSScreen, + surfaces: [WindowID: OverviewSurfaceFrame], windows: [WindowID: Window], cornerRadius: Double = 12, + previews: [WindowID: NSImage]? = nil + ) { + guard let workspace = projection.workspaces.first(where: { $0.workspaceID == workspaceID }), + !workspace.windows.isEmpty, (!surfaces.isEmpty || previews != nil), + surfaces.keys.allSatisfy({ OverviewSurfaceCapture.shared.displayLayer(for: $0) != nil }) + else { return nil } + let workspaceIDs = previews == nil + ? Set(workspace.windows.map(\.windowID)).intersection(surfaces.keys) + : Set(workspace.windows.map(\.windowID)) + let infos = CGWindowListCopyWindowInfo(.optionAll, kCGNullWindowID) as? [[String: Any]] ?? [] + let primaryHeight = NSScreen.screens.first?.frame.height ?? 0 + let screenOrigin = CGPoint(x: screen.frame.minX, y: primaryHeight - screen.frame.maxY) + var frames: [WindowID: CGRect] = [:] + var order: [WindowID: Int] = [:] + var owners: [WindowID: Int32] = [:] + for (index, info) in infos.enumerated() { + guard let number = info[kCGWindowNumber as String] as? NSNumber, + let bounds = info[kCGWindowBounds as String] as? [String: Any], + let rect = CGRect(dictionaryRepresentation: bounds as CFDictionary) + else { continue } + let id = WindowID(rawValue: number.uint64Value) + guard workspaceIDs.contains(id), + let owner = info[kCGWindowOwnerPID as String] as? NSNumber, + windows[id]?.processID == owner.int32Value else { continue } + owners[id] = owner.int32Value + frames[id] = rect.offsetBy(dx: -screenOrigin.x, dy: -screenOrigin.y) + order[id] = index + } + guard previews != nil || workspaceIDs.allSatisfy({ frames[$0] != nil }) else { return nil } + self.frames = surfaces.filter { workspaceIDs.contains($0.key) } + nativeFrames = frames + nativeOwners = owners + self.projection = projection + self.cornerRadius = max(cornerRadius, 0) + CATransaction.begin() + CATransaction.setDisableActions(true) + defer { CATransaction.commit() } + layer.frame = CGRect(origin: .zero, size: screen.frame.size) + layer.isGeometryFlipped = true + layer.masksToBounds = true + var entries: [(WindowID, CALayer, CGRect, CGRect)] = [] + var cachedCount = 0 + // Public CGWindow order is front to back; preserve floating/native stacking. + for card in workspace.windows.sorted(by: { + order[$0.windowID, default: 0] > order[$1.windowID, default: 0] + }) { + guard let source = frames[card.windowID], source.width > 0, source.height > 0 else { continue } + let target = CGRect(x: card.frame.x, y: card.frame.y, + width: card.frame.width, height: card.frame.height) + let item: CALayer + if let frame = surfaces[card.windowID], + overviewSurfaceMatchesNativeSize(source: frame.sourceSize, native: Rect( + x: source.minX, y: source.minY, width: source.width, height: source.height)), + let displayLayer = OverviewSurfaceCapture.shared.displayLayer(for: card.windowID) { + item = displayLayer + _ = enqueueWindowSurface(frame, on: displayLayer) + } else { + // Only visible native windows need the opening fallback. Parked columns + // must not fly into the overview from their one-pixel strip anchors. + guard previews != nil, source.intersects(layer.bounds) else { continue } + item = CALayer() + item.contents = previews?[card.windowID]?.cgImage(forProposedRect: nil, context: nil, hints: nil) + if item.contents != nil { cachedCount += 1 } + item.backgroundColor = NSColor(calibratedWhite: 0.12, alpha: 1).cgColor + if item.contents == nil { + // Keep the window's silhouette legible while its preview loads. Only + // the small symbol is rasterized, never a window-sized placeholder. + item.borderColor = NSColor.white.withAlphaComponent(0.35).cgColor + item.borderWidth = 2 + let symbol = CALayer() + let image = NSImage(systemSymbolName: "macwindow", accessibilityDescription: nil)? + .withSymbolConfiguration(NSImage.SymbolConfiguration(pointSize: 48, weight: .regular) + .applying(.init(paletteColors: [.white]))) + symbol.contents = image?.cgImage(forProposedRect: nil, context: nil, hints: nil) + symbol.contentsGravity = .resizeAspect + symbol.frame = CGRect(x: (source.width - 64) / 2, + y: (source.height - 64) / 2, width: 64, height: 64) + item.addSublayer(symbol) + } + fallbackIDs.insert(card.windowID) + } + item.removeFromSuperlayer(); item.removeAllAnimations() + item.contentsGravity = .resize + item.contentsScale = screen.backingScaleFactor + item.transform = CATransform3DIdentity + item.anchorPoint = .zero + item.bounds = CGRect(origin: .zero, size: source.size) + item.position = source.origin + item.cornerRadius = self.cornerRadius + item.cornerCurve = .continuous + item.masksToBounds = true + layer.addSublayer(item) + entries.append((card.windowID, item, source, target)) + } + guard !entries.isEmpty else { return nil } + self.entries = entries + openingContentDescription = "surfaces:\(entries.count - fallbackIDs.count),cached:\(cachedCount),cards:\(fallbackIDs.count - cachedCount)" + nativeFrames = frames.filter { id, _ in entries.contains { $0.0 == id } } + } + + // Compact/placeholder layers only bridge the opening. The regular card owns + // progressive image delivery afterward; never keep a stale fallback on top. + func finishOpening() -> Set { + let released = fallbackIDs + for (id, item, _, _) in entries where released.contains(id) { item.removeFromSuperlayer() } + entries.removeAll { released.contains($0.0) } + nativeFrames = nativeFrames.filter { !released.contains($0.key) } + fallbackIDs = [] + return released + } + + func animate(opening: Bool, duration: TimeInterval) { + CATransaction.begin() + CATransaction.setDisableActions(true) + for (_, item, source, target) in entries { + let start = opening ? source : target + let end = opening ? target : source + let presented = item.presentation() + let startPosition = presented?.position ?? start.origin + let startTransform = presented?.transform ?? CATransform3DMakeScale( + start.width / source.width, start.height / source.height, 1 + ) + // Transform a fixed-size image; this retains one texture through the zoom. + item.bounds = CGRect(origin: .zero, size: source.size) + item.position = end.origin + item.transform = CATransform3DMakeScale( + end.width / source.width, end.height / source.height, 1 + ) + let radius = CABasicAnimation(keyPath: "cornerRadius") + radius.fromValue = presented?.cornerRadius ?? cornerRadius * source.width / start.width + item.cornerRadius = cornerRadius * source.width / end.width + radius.toValue = item.cornerRadius + let position = CABasicAnimation(keyPath: "position") + position.fromValue = NSValue(point: startPosition) + position.toValue = NSValue(point: end.origin) + let transform = CABasicAnimation(keyPath: "transform") + transform.fromValue = NSValue(caTransform3D: startTransform) + transform.toValue = NSValue(caTransform3D: item.transform) + let group = CAAnimationGroup() + group.animations = [position, transform, radius] + group.duration = duration + group.timingFunction = CAMediaTimingFunction(name: .easeInEaseOut) + item.add(group, forKey: "overview-surface") + } + CATransaction.commit() + } + + func matchesNativeFrames(screen: NSScreen) -> Bool { + let infos = CGWindowListCopyWindowInfo(.optionAll, kCGNullWindowID) as? [[String: Any]] ?? [] + let primaryHeight = NSScreen.screens.first?.frame.height ?? 0 + let origin = CGPoint(x: screen.frame.minX, y: primaryHeight - screen.frame.maxY) + var current: [WindowID: CGRect] = [:] + for info in infos { + guard let id = info[kCGWindowNumber as String] as? NSNumber, + let bounds = info[kCGWindowBounds as String] as? [String: Any], + let rect = CGRect(dictionaryRepresentation: bounds as CFDictionary) + else { continue } + let windowID = WindowID(rawValue: id.uint64Value) + if nativeFrames[windowID] != nil { + guard let owner = info[kCGWindowOwnerPID as String] as? NSNumber, + nativeOwners[windowID] == owner.int32Value else { return false } + current[windowID] = rect.offsetBy(dx: -origin.x, dy: -origin.y) + } + } + return current == nativeFrames + } +} diff --git a/Sources/DefiMacOS/OverviewTitleCache.swift b/Sources/DefiMacOS/OverviewTitleCache.swift new file mode 100644 index 00000000..85f3fd2e --- /dev/null +++ b/Sources/DefiMacOS/OverviewTitleCache.swift @@ -0,0 +1,78 @@ +import AppKit +import DefiModel + +// Rasterize small, unchanged labels once; moving a card must not invoke the +// typesetter or resolve application icon representations on every refresh. +@MainActor +func overviewLabelBitmap(size: CGSize, scale: CGFloat, draw: () -> Void) -> NSImage? { + guard size.width.isFinite, size.height.isFinite, scale.isFinite, + size.width > 0, size.height > 0, scale > 0 else { return nil } + let width = Int(ceil(size.width * scale)) + let height = Int(ceil(size.height * scale)) + guard width <= 16_384, height <= 256, + let context = CGContext(data: nil, width: width, height: height, + bitsPerComponent: 8, bytesPerRow: width * 4, space: CGColorSpaceCreateDeviceRGB(), + bitmapInfo: CGImageAlphaInfo.premultipliedLast.rawValue) else { return nil } + context.translateBy(x: 0, y: CGFloat(height)) + context.scaleBy(x: scale, y: -scale) + NSGraphicsContext.saveGraphicsState() + NSGraphicsContext.current = NSGraphicsContext(cgContext: context, flipped: true) + draw() + NSGraphicsContext.restoreGraphicsState() + guard let image = context.makeImage() else { return nil } + return NSImage(cgImage: image, size: size) +} + +@MainActor +final class OverviewTitleCache { + struct Label { + let image: NSImage + let intrinsicSize: CGSize + } + private struct Entry { + let text: String + let fontSize: CGFloat + let maximumWidth: CGFloat + let scale: CGFloat + let label: Label + let bytes: Int + } + private var entries: [WindowID: Entry] = [:] + private let maximumBytes: Int + private(set) var byteCount = 0 + private(set) var rasterizationCount = 0 + + init(maximumBytes: Int = 4 * 1024 * 1024) { self.maximumBytes = maximumBytes } + + func prune(windowIDs: Set) { + for id in Array(entries.keys) where !windowIDs.contains(id) { + byteCount -= entries.removeValue(forKey: id)?.bytes ?? 0 + } + } + + func label(for id: WindowID, text: String, fontSize: CGFloat, + maximumWidth: CGFloat, scale: CGFloat) -> Label? { + if let entry = entries[id], entry.text == text, entry.fontSize == fontSize, + entry.maximumWidth == maximumWidth, entry.scale == scale { return entry.label } + let attributes: [NSAttributedString.Key: Any] = [ + .font: NSFont.systemFont(ofSize: fontSize, weight: .medium), + .foregroundColor: NSColor.white.withAlphaComponent(0.9), + ] + let title = text as NSString + let intrinsic = title.size(withAttributes: attributes) + let size = CGSize(width: min(intrinsic.width, maximumWidth), height: intrinsic.height) + guard let image = overviewLabelBitmap(size: size, scale: scale, draw: { + title.draw(in: CGRect(origin: .zero, size: size), withAttributes: attributes) + }) else { return nil } + rasterizationCount += 1 + let label = Label(image: image, intrinsicSize: intrinsic) + let bytes = Int(ceil(size.width * scale)) * Int(ceil(size.height * scale)) * 4 + byteCount -= entries.removeValue(forKey: id)?.bytes ?? 0 + guard bytes <= maximumBytes else { return label } + if byteCount + bytes > maximumBytes { entries.removeAll(); byteCount = 0 } + entries[id] = Entry(text: text, fontSize: fontSize, maximumWidth: maximumWidth, + scale: scale, label: label, bytes: bytes) + byteCount += bytes + return label + } +} diff --git a/Sources/DefiMacOS/OverviewView.swift b/Sources/DefiMacOS/OverviewView.swift index 17420934..4cc7100b 100644 --- a/Sources/DefiMacOS/OverviewView.swift +++ b/Sources/DefiMacOS/OverviewView.swift @@ -21,6 +21,10 @@ final class OverviewView: NSView { private var desktopFadeLink: CADisplayLink? var hasDesktopImage: Bool { desktopImage != nil } private var previews: [WindowID: NSImage] = [:] + var surfaceWindowIDs = Set() + var suppressedSurfaceWindowIDs = Set() { + didSet { if oldValue != suppressedSurfaceWindowIDs { needsDisplay = true } } + } private var previewOpacities: [WindowID: Double] = [:] // Image being replaced, kept underneath until the new preview finishes fading in. private var outgoingPreviews: [WindowID: NSImage] = [:] @@ -30,6 +34,40 @@ final class OverviewView: NSView { private var leftDragStarted = false private var rightDragPoint: NSPoint? private var iconCache: [String: NSImage] = [:] + // The scrim is unchanged while cards move; retain a 512-byte strip instead + // of evaluating a gradient across every card on every refresh. + private let titleScrimImage = overviewLabelBitmap(size: CGSize(width: 1, height: 128), scale: 1) { + NSGradient( + colorsAndLocations: + (NSColor.black.withAlphaComponent( + overviewTitleScrimAlpha(progress: 0, opacity: 1) + ), 0), + (NSColor.black.withAlphaComponent( + overviewTitleScrimAlpha(progress: 0.25, opacity: 1) + ), 0.25), + (NSColor.black.withAlphaComponent( + overviewTitleScrimAlpha(progress: 0.5, opacity: 1) + ), 0.5), + (NSColor.black.withAlphaComponent( + overviewTitleScrimAlpha(progress: 0.75, opacity: 1) + ), 0.75), + (NSColor.black.withAlphaComponent( + overviewTitleScrimAlpha(progress: 0.9, opacity: 1) + ), 0.9), + (NSColor.black.withAlphaComponent( + overviewTitleScrimAlpha(progress: 0.97, opacity: 1) + ), 0.97), + (NSColor.clear, 1) + )?.draw( + from: NSPoint(x: 0.5, y: 0), + to: NSPoint(x: 0.5, y: 128), + options: [] + ) + } + private let titleCache = OverviewTitleCache() + var titleRasterizationCount: Int { titleCache.rasterizationCount } + private(set) var presentationUpdateCount = 0 + private(set) var drawCount = 0 override var isFlipped: Bool { true } @@ -45,6 +83,9 @@ final class OverviewView: NSView { func discardPreviewImages() { previews.removeAll(keepingCapacity: false) outgoingPreviews.removeAll(keepingCapacity: false) + previewOpacities.removeAll() + titleCache.prune(windowIDs: []) + iconCache.removeAll() } func setDesktopImage(_ image: NSImage?, fadeDuration: TimeInterval = 0) { @@ -99,6 +140,8 @@ final class OverviewView: NSView { || self.borderStyle != borderStyle || self.windowCornerRadius != windowCornerRadius || self.previews != visiblePreviews || self.previewOpacities != visibleOpacities else { return false } + presentationUpdateCount += 1 + if self.snapshot != snapshot { titleCache.prune(windowIDs: Set(snapshot.windows.keys)) } self.snapshot = snapshot self.projection = projection self.selection = selection @@ -148,6 +191,7 @@ final class OverviewView: NSView { override func draw(_ dirtyRect: NSRect) { guard let projection, let snapshot else { return } + drawCount += 1 for workspace in projection.workspaces { drawWorkspace(workspace, snapshot: snapshot) } @@ -283,7 +327,7 @@ final class OverviewView: NSView { operation: .sourceOver, fraction: fraction, respectFlipped: true, - hints: [.interpolation: NSImageInterpolation.high] + hints: [.interpolation: NSImageInterpolation.medium] ) } } else { @@ -326,6 +370,7 @@ final class OverviewView: NSView { _ card: OverviewWindowProjection, snapshot: OverviewSnapshot ) { + guard !suppressedSurfaceWindowIDs.contains(card.windowID) else { return } guard let window = snapshot.windows[card.windowID] else { return } let frame = nsRect(card.frame) guard needsToDraw(frame) else { return } @@ -336,6 +381,14 @@ final class OverviewView: NSView { ) NSColor(calibratedWhite: card.isNativeFullscreen ? 0.19 : 0.15, alpha: 1).setFill() path.fill() + if surfaceWindowIDs.contains(card.windowID), let preview = previews[card.windowID] { + NSGraphicsContext.saveGraphicsState() + path.addClip() + preview.draw(in: frame, from: .zero, operation: .sourceOver, fraction: 1, + respectFlipped: true, hints: [.interpolation: NSImageInterpolation.medium]) + NSGraphicsContext.restoreGraphicsState() + return + } let iconSize = overviewWindowTitleIconSize(cardHeight: frame.height) let titleBandHeight = overviewWindowTitleBandHeight(iconSize: iconSize) let titleFadeHeight = overviewPreviewBlurFadeHeight( @@ -355,7 +408,7 @@ final class OverviewView: NSView { operation: .sourceOver, fraction: 1, respectFlipped: true, - hints: [.interpolation: NSImageInterpolation.high] + hints: [.interpolation: NSImageInterpolation.medium] ) } // Cross-fades keep the scrim steady; only a first reveal fades it in. @@ -366,34 +419,12 @@ final class OverviewView: NSView { operation: .sourceOver, fraction: opacity, respectFlipped: true, - hints: [.interpolation: NSImageInterpolation.high] - ) - NSGradient( - colorsAndLocations: - (NSColor.black.withAlphaComponent( - overviewTitleScrimAlpha(progress: 0, opacity: scrimOpacity) - ), 0), - (NSColor.black.withAlphaComponent( - overviewTitleScrimAlpha(progress: 0.25, opacity: scrimOpacity) - ), 0.25), - (NSColor.black.withAlphaComponent( - overviewTitleScrimAlpha(progress: 0.5, opacity: scrimOpacity) - ), 0.5), - (NSColor.black.withAlphaComponent( - overviewTitleScrimAlpha(progress: 0.75, opacity: scrimOpacity) - ), 0.75), - (NSColor.black.withAlphaComponent( - overviewTitleScrimAlpha(progress: 0.9, opacity: scrimOpacity) - ), 0.9), - (NSColor.black.withAlphaComponent( - overviewTitleScrimAlpha(progress: 0.97, opacity: scrimOpacity) - ), 0.97), - (NSColor.clear, 1) - )?.draw( - from: NSPoint(x: frame.midX, y: frame.minY), - to: NSPoint(x: frame.midX, y: frame.minY + titleFadeHeight), - options: [] + hints: [.interpolation: NSImageInterpolation.medium] ) + titleScrimImage?.draw( + in: NSRect(x: frame.minX, y: frame.minY, width: frame.width, height: titleFadeHeight), + from: .zero, operation: .sourceOver, fraction: scrimOpacity, + respectFlipped: true, hints: [.interpolation: NSImageInterpolation.medium]) NSGraphicsContext.restoreGraphicsState() } let title = (window.title.isEmpty ? window.appID : window.title) as NSString @@ -401,15 +432,24 @@ final class OverviewView: NSView { .font: NSFont.systemFont(ofSize: min(13, max(frame.height * 0.09, 10)), weight: .medium), .foregroundColor: NSColor.white.withAlphaComponent(0.9), ] + let label = titleCache.label(for: card.windowID, text: title as String, + fontSize: min(13, max(frame.height * 0.09, 10)), + maximumWidth: max(frame.width - iconSize - 28, 1), + scale: windowBackingScale) let titleLayout = overviewWindowTitleLayout( cardFrame: frame, iconSize: iconSize, - titleSize: title.size(withAttributes: titleAttributes), + titleSize: label?.intrinsicSize ?? title.size(withAttributes: titleAttributes), blurHeight: titleBandHeight ) let icon = icon(for: window) icon.draw(in: titleLayout.iconFrame) - title.draw(in: titleLayout.titleFrame, withAttributes: titleAttributes) + if let label { + label.image.draw(in: titleLayout.titleFrame, from: .zero, operation: .sourceOver, + fraction: 1, respectFlipped: true, hints: [.interpolation: NSImageInterpolation.high]) + } else { + title.draw(in: titleLayout.titleFrame, withAttributes: titleAttributes) + } if card.isNativeFullscreen { let label = "Full Screen" as NSString label.draw( @@ -510,18 +550,29 @@ final class OverviewView: NSView { if let cached = iconCache[window.appID] { return cached } let icon = NSImage(systemSymbolName: "app", accessibilityDescription: window.appID) ?? NSImage(size: NSSize(width: 24, height: 24)) - iconCache[window.appID] = icon + iconCache[window.appID] = rasterizedIcon(icon) if let processID = window.processID { Task { @MainActor [weak self] in let loaded = await Task.detached(priority: .utility) { NSRunningApplication(processIdentifier: processID)?.icon }.value guard let self, let loaded else { return } - self.iconCache[window.appID] = loaded + self.iconCache[window.appID] = self.rasterizedIcon(loaded) self.needsDisplay = true } } - return icon + return iconCache[window.appID] ?? icon + } + + private var windowBackingScale: CGFloat { window?.backingScaleFactor ?? 2 } + + private func rasterizedIcon(_ image: NSImage) -> NSImage { + let size = CGSize(width: 24, height: 24) + return overviewLabelBitmap(size: size, scale: 2) { + image.draw(in: CGRect(origin: .zero, size: size), from: .zero, + operation: .sourceOver, fraction: 1, respectFlipped: true, + hints: [.interpolation: NSImageInterpolation.high]) + } ?? image } private func overviewBorderColor(_ value: UInt32) -> NSColor { diff --git a/Sources/DefiMacOS/PlatformEventMonitor.swift b/Sources/DefiMacOS/PlatformEventMonitor.swift index 2d1b8095..01cd82c1 100644 --- a/Sources/DefiMacOS/PlatformEventMonitor.swift +++ b/Sources/DefiMacOS/PlatformEventMonitor.swift @@ -238,6 +238,7 @@ final class PlatformEventMonitor { guard let self else { return } guard self.desktopSessionActive else { return } if eventStartsMouseFocusInteraction(event.type) { + ExperimentalRibbonRenderer.shared.cancel() let rawWindowID = event.cgEvent?.getIntegerValueField( .mouseEventWindowUnderMousePointerThatCanHandleThisEvent diff --git a/Sources/DefiMacOS/PlatformPresentation.swift b/Sources/DefiMacOS/PlatformPresentation.swift index 9cfb86c7..3a3915c7 100644 --- a/Sources/DefiMacOS/PlatformPresentation.swift +++ b/Sources/DefiMacOS/PlatformPresentation.swift @@ -182,6 +182,10 @@ extension MacOSPlatform { public func refreshWindowBorders() { enqueuePresentation { $0.presentRefreshWindowBorders() } } public func hideWindowBorders() { enqueuePresentation { $0.presentHideWindowBorders() } } + public func setOverviewPresentationActive(_ active: Bool) { + snapshotEngine.overviewPresentationActive = active + setWindowBordersSuppressed(active) + } public func setWindowBordersSuppressed(_ suppressed: Bool) { enqueuePresentation { $0.presentSetWindowBordersSuppressed(suppressed) } } diff --git a/Sources/DefiMacOS/SnapshotEngine.swift b/Sources/DefiMacOS/SnapshotEngine.swift index 6f125331..bb7421ee 100644 --- a/Sources/DefiMacOS/SnapshotEngine.swift +++ b/Sources/DefiMacOS/SnapshotEngine.swift @@ -345,6 +345,11 @@ final class SnapshotEngine: @unchecked Sendable { set { read { $0.applicationWindowCounts = newValue } } } + var overviewPresentationActive: Bool { + get { read { $0.overviewPresentationActive } } + set { read { $0.overviewPresentationActive = newValue } } + } + var lastSnapshotWindows: [Window] { get { read { $0.lastSnapshotWindows } } set { read { $0.lastSnapshotWindows = newValue } } @@ -793,11 +798,18 @@ extension SnapshotEngine { return .unmatched(title: title) } let windowID = WindowID(rawValue: UInt64(resolvedWindowID)) - let sizeConstraints = onMain { platform in - if let ownedWindowID = platform.borderManager.ownedSurfaceWindowID { - platform.borderBoundsProvider.probe(ownedWindowID: ownedWindowID) + let previousWindow = lastSnapshotWindows.first { + $0.id == windowID && $0.processID == processID + } + let sizeConstraints = windowSizeConstraintsForSnapshot( + previousWindow: previousWindow, overviewActive: overviewPresentationActive + ) { + onMain { platform in + if let ownedWindowID = platform.borderManager.ownedSurfaceWindowID { + platform.borderBoundsProvider.probe(ownedWindowID: ownedWindowID) + } + return platform.borderBoundsProvider.sizeConstraints(for: windowID) } - return platform.borderBoundsProvider.sizeConstraints(for: windowID) } let monitorID = monitor(containing: frame, monitors: monitors)?.id return .discovered( @@ -1289,6 +1301,7 @@ private struct Storage { var newlyDiscoveredWindowIDs = Set() var accessibilitySessionResetPending = false var hasCompletedWindowSnapshot = false + var overviewPresentationActive = false var lastSnapshotWindows: [Window] = [] var lastSnapshotWindowIDs = Set() var lastSnapshotProcessIDs = Set() @@ -1383,3 +1396,16 @@ struct SnapshotObservations: Equatable, @unchecked Sendable { var frameIncludesUnscopedRefresh = false var destroyedWindowIDs = Set() } + +func windowSizeConstraintsForSnapshot( + previousWindow: Window?, overviewActive: Bool, + freshRead: () -> WindowSizeConstraints? +) -> WindowSizeConstraints? { + // Optional native metadata must not interrupt the overview's presentation + // thread. The ordinary discovery path refreshes it again after closing. + guard overviewActive else { return freshRead() } + return previousWindow.map { + WindowSizeConstraints(minimumWidth: $0.minimumTiledWidth, + maximumWidth: $0.maximumTiledWidth, maximumHeight: $0.maximumTiledHeight) + } +} diff --git a/Sources/DefiMacOS/WindowSurfaceVideoLayer.swift b/Sources/DefiMacOS/WindowSurfaceVideoLayer.swift new file mode 100644 index 00000000..64df6d2c --- /dev/null +++ b/Sources/DefiMacOS/WindowSurfaceVideoLayer.swift @@ -0,0 +1,27 @@ +import AVFoundation +import CoreMedia +import CoreVideo + +// Wrap existing pixels for the public video renderer; no compression or bitmap copy. +@MainActor +func enqueueWindowSurface(_ frame: OverviewSurfaceFrame, on layer: AVSampleBufferDisplayLayer) -> Bool { + var format: CMVideoFormatDescription? + guard CMVideoFormatDescriptionCreateForImageBuffer(allocator: kCFAllocatorDefault, + imageBuffer: frame.buffer, formatDescriptionOut: &format) == noErr, + let format else { return false } + var timing = CMSampleTimingInfo(duration: .invalid, presentationTimeStamp: .zero, + decodeTimeStamp: .invalid) + var sample: CMSampleBuffer? + guard CMSampleBufferCreateReadyWithImageBuffer(allocator: kCFAllocatorDefault, + imageBuffer: frame.buffer, formatDescription: format, sampleTiming: &timing, + sampleBufferOut: &sample) == noErr, let sample, + let attachments = CMSampleBufferGetSampleAttachmentsArray(sample, createIfNecessary: true), + CFArrayGetCount(attachments) > 0 else { return false } + // Core Media specifies that this array contains mutable CF dictionaries. + let attachment = unsafeBitCast(CFArrayGetValueAtIndex(attachments, 0), to: CFMutableDictionary.self) + CFDictionarySetValue(attachment, + Unmanaged.passUnretained(kCMSampleAttachmentKey_DisplayImmediately).toOpaque(), + Unmanaged.passUnretained(kCFBooleanTrue).toOpaque()) + layer.sampleBufferRenderer.enqueue(sample) + return true +} diff --git a/Sources/DefiModel/Command.swift b/Sources/DefiModel/Command.swift index 8e794271..e03a607c 100644 --- a/Sources/DefiModel/Command.swift +++ b/Sources/DefiModel/Command.swift @@ -44,6 +44,20 @@ public enum Command: Equatable, Codable, Sendable { case toggleOverview case runStartupCommands + public var editsSelectedLayout: Bool { + switch self { + case .cycleWidth, .maximizeColumn, .moveColumn, .moveWindow, + .joinWindow, .unjoinWindows, .toggleFloating: + true + default: + false + } + } + + public var animatesManagedLayout: Bool { + editsSelectedLayout || resizesManagedLayout + } + public var resizesManagedLayout: Bool { switch self { case .cycleWidth, .maximizeColumn, .joinWindow, .unjoinWindows, diff --git a/Sources/DefiRuntime/OverviewRuntime.swift b/Sources/DefiRuntime/OverviewRuntime.swift index 3247649c..375aba00 100644 --- a/Sources/DefiRuntime/OverviewRuntime.swift +++ b/Sources/DefiRuntime/OverviewRuntime.swift @@ -50,6 +50,52 @@ public enum OverviewRuntimeError: Error, Equatable, Sendable { case classificationChanged(WindowID) } +/// Applies a local layout edit without activating the overview's selected workspace. +@discardableResult +public func applyOverviewLayoutCommand( + _ command: Command, + intent: OverviewWindowIntent, + viewports: [MonitorID: Rect], + state: inout RuntimeState +) throws -> [WindowID: Rect] { + guard command.editsSelectedLayout else { throw OverviewRuntimeError.invalidDropTarget } + _ = try validateOverviewWindow(intent, state: state) + guard state.windows[intent.windowID]?.transientOwnerID == nil else { + throw OverviewRuntimeError.transientWindow(intent.windowID) + } + guard !state.nativeFullscreenWindowIDs.contains(intent.windowID) else { + throw OverviewRuntimeError.nativeFullscreen(intent.windowID) + } + guard state.windows[intent.windowID]?.floating != true || command == .toggleFloating else { + throw OverviewRuntimeError.invalidDropTarget + } + var next = state + _ = focusWindow(intent.windowID, state: &next) + try reduce(command, on: intent.sourceMonitorID, state: &next, viewports: viewports) + guard let source = state.workspaceLocation(for: intent.sourceWorkspaceID), + let destination = next.workspaceLocation(for: intent.sourceWorkspaceID) + else { throw OverviewRuntimeError.unknownWorkspace(intent.sourceWorkspaceID) } + var floatingFrames: [WindowID: Rect] = [:] + if command == .toggleFloating, state.windows[intent.windowID]?.floating == false, + let viewport = viewports[intent.sourceMonitorID], + let tiledFrame = computeLayout( + workspace: state.monitors[source.monitorIndex].workspaces[source.workspaceIndex], + viewport: viewport, windows: Array(state.windows.values), settings: state.layout + ).first(where: { $0.windowID == intent.windowID })?.frame { + let frame = rebasedFloatingFrame(tiledFrame, from: viewport, to: viewport) + next.windows[intent.windowID]?.frame = frame + floatingFrames[intent.windowID] = frame + } + // Local edits cannot transfer windows or workspaces. Preserve the original + // topology and activation, including an active empty trailing workspace. + state.monitors[source.monitorIndex].workspaces[source.workspaceIndex] = + next.monitors[destination.monitorIndex].workspaces[destination.workspaceIndex] + state.windows = next.windows + state.pendingNativeFullscreenWidthResetWindowIDs = next.pendingNativeFullscreenWidthResetWindowIDs + state.suspendedTiledPlacements[intent.windowID] = next.suspendedTiledPlacements[intent.windowID] + return floatingFrames +} + @discardableResult public func focusOverviewWindow( _ intent: OverviewWindowIntent, diff --git a/Tests/DefiConfigTests/ConfigTests.swift b/Tests/DefiConfigTests/ConfigTests.swift index f62e4f5e..b322b565 100644 --- a/Tests/DefiConfigTests/ConfigTests.swift +++ b/Tests/DefiConfigTests/ConfigTests.swift @@ -103,8 +103,10 @@ struct ConfigTests { #expect(config.layout.defaultColumnWidth == 0.8) #expect(config.animation.enabled) #expect(config.animation.durationMS == 125) + #expect(!config.animation.experimentalWindowRepresentations) #expect(config.overview.zoom == 0.5) #expect(config.overview.windowPreviews == false) + #expect(config.overview.experimentalSurfaceTransitions == false) #expect(config.overview.windowCornerRadius == 12) #expect(config.decorations.borders.enabled) #expect(config.decorations.borders.width == 4) @@ -135,6 +137,7 @@ struct ConfigTests { [overview] zoom = 0.25 window_previews = true + experimental_surface_transitions = true window_corner_radius = 18 """.utf8 ) @@ -142,9 +145,17 @@ struct ConfigTests { #expect(config.overview.zoom == 0.25) #expect(config.overview.windowPreviews) + #expect(config.overview.experimentalSurfaceTransitions) #expect(config.overview.windowCornerRadius == 18) } + @Test + func `Surface transitions require explicit preview opt in`() { + #expect(throws: ConfigError.self) { + try Config.decode(Data("[overview]\nexperimental_surface_transitions = true".utf8)) + } + } + @Test func `Rejects invalid overview zoom`() { let data = Data( @@ -245,6 +256,7 @@ struct ConfigTests { [animation] enabled = true duration_ms = 120 + experimental_window_representations = true [workspaces] names = ["dev", "web"] @@ -263,6 +275,7 @@ struct ConfigTests { #expect(config.layout.gaps == 4) #expect(config.animation.durationMS == 120) + #expect(config.animation.experimentalWindowRepresentations) #expect(config.workspaces.defaultName == "dev") #expect(config.keys["hyper-1"] == "workspace dev") #expect(config.keys["hyper-shift-1"] == "move-column-to-workspace-name dev") diff --git a/Tests/DefiCoreTests/OverviewTests.swift b/Tests/DefiCoreTests/OverviewTests.swift index 649fe911..6db0469f 100644 --- a/Tests/DefiCoreTests/OverviewTests.swift +++ b/Tests/DefiCoreTests/OverviewTests.swift @@ -6,6 +6,22 @@ struct OverviewTests { let monitorID = MonitorID(rawValue: 1) let monitorFrame = Rect(x: 0, y: 0, width: 1_000, height: 800) + @Test func floatingMovementUsesWorkspaceRelativePosition() { + func projection(workspaceX: Double, windowX: Double) -> OverviewProjection { + OverviewProjection(monitorID: monitorID, workspaces: [OverviewWorkspaceProjection( + workspaceID: WorkspaceID(rawValue: "test"), + frame: Rect(x: workspaceX, y: 0, width: 1000, height: 800), + windows: [OverviewWindowProjection(windowID: WindowID(rawValue: 1), + frame: Rect(x: windowX, y: 100, width: 200, height: 200), + layer: .floating, isNativeFullscreen: false, canDrag: true)])]) + } + let source = projection(workspaceX: 0, windowX: 100) + #expect(overviewProjectionReordersExistingCards(from: source, + to: projection(workspaceX: 0, windowX: 200))) + #expect(!overviewProjectionReordersExistingCards(from: source, + to: projection(workspaceX: -50, windowX: 50))) + } + @Test func resizedCardsAnimateSizeAndPositionTogether() { func projection(x: Double, width: Double) -> OverviewProjection { @@ -71,6 +87,12 @@ struct OverviewTests { #expect(projection.workspaces.count < monitor.workspaces.count) #expect(projection.workspaces.contains(where: { $0.workspaceID == active.id })) #expect(snapshot.monitors[0].workspaces[4].scrollOffset == 0.25) + let captureProjection = projectOverview(snapshot: snapshot, monitorID: monitorID, + bounds: monitorFrame, viewport: OverviewViewport(horizontalOffsets: [active.id: 100]), + layout: LayoutSettings(), includeOffscreenContent: true) + #expect(captureProjection.workspaces.count == monitor.workspaces.count) + #expect(captureProjection.workspaces.flatMap(\.windows).map(\.windowID) == [WindowID(rawValue: 1)]) + #expect(snapshot.monitors[0].workspaces[4].scrollOffset == 0.25) } @Test @@ -468,6 +490,12 @@ struct OverviewTests { card(first, x: 100, column: 1), ]) + #expect(overviewProjectionReordersExistingCards(from: source, to: target)) + #expect(!overviewProjectionReordersExistingCards(from: source, to: source)) + #expect(!overviewProjectionReordersExistingCards(from: source, to: projection([ + card(first, x: -50, column: 0), card(second, x: 50, column: 1) + ]))) // Scrolling has its own timeline. + let middle = interpolateOverviewProjection(from: source, to: target, progress: 0.5) let windows = try #require(middle.workspaces.first?.windows) let movedFirst = try #require(windows.first(where: { $0.windowID == first })) diff --git a/Tests/DefiDaemonTests/DaemonCommandPolicyTests.swift b/Tests/DefiDaemonTests/DaemonCommandPolicyTests.swift index 22a6b70f..173f1c24 100644 --- a/Tests/DefiDaemonTests/DaemonCommandPolicyTests.swift +++ b/Tests/DefiDaemonTests/DaemonCommandPolicyTests.swift @@ -8,6 +8,55 @@ import Testing @testable import DefiDaemon struct DaemonCommandPolicyTests { + @Test func completedOverviewSelectionAllowsWarpUntilSuperseded() { + let state = OverviewToggleState() + state.recordControllerState(true, sessionGeneration: 1) + state.completeSelectionExit(1) + state.recordControllerState(false, sessionGeneration: 2) + #expect(state.selectionWarpIsCurrent(1)) + state.recordControllerState(true, sessionGeneration: 3) + #expect(!state.selectionWarpIsCurrent(1)) + _ = state.requestClose() + state.completeSelectionExit(3) + state.recordControllerState(false, sessionGeneration: 4) + #expect(!state.selectionWarpIsCurrent(3), "Cancellation cannot authorize a delayed warp") + } + + @Test func overviewSessionCancellationInvalidatesAlreadyQueuedSelection() { + let state = OverviewToggleState() + state.recordControllerState(true, sessionGeneration: 1) + #expect(state.isCurrentSession(1)) + _ = state.requestClose() + #expect(!state.isCurrentSession(1), "Cancellation must take effect before actor work resumes") + state.recordControllerState(false, sessionGeneration: 2) + state.recordControllerState(true, sessionGeneration: 3) + #expect(!state.isCurrentSession(1), "Reopening cannot revive an older selection") + #expect(state.isCurrentSession(3)) + } + + @Test(arguments: [20.0, 21.0]) + func nativeFocusAnimationSurvivesItsOwnSnapshotButYieldsToNewerInput(timestamp: Double) { + #expect(desktopSnapshotWaitsForCommandAnimation(animationPending: true, + latestCommandInputTimestamp: 10, latestNativeFocusAnimationInputTimestamp: 20, + mouseFocusIntentTimestamp: timestamp, keyboardFocusIntentTimestamp: nil) == (timestamp == 20)) + } + + @Test func changedNativeFocusAnimatesOnlyItsMonitor() { + let monitor = MonitorID(rawValue: 2) + #expect(nativeFocusAnimationMonitorID(focusedMonitorID: monitor, floating: false, + overviewOpen: false, mouseGestureActive: false, displayGeometryChanged: false) == monitor) + #expect(nativeFocusAnimationMonitorID(focusedMonitorID: nil, floating: false, + overviewOpen: false, mouseGestureActive: false, displayGeometryChanged: false) == nil) + } + + @Test(arguments: [(true, false, false, false), (false, true, false, false), + (false, false, true, false), (false, false, false, true)]) + func nativeFocusDoesNotAnimateDuringOtherInteractions(blockers: (Bool, Bool, Bool, Bool)) { + #expect(nativeFocusAnimationMonitorID(focusedMonitorID: MonitorID(rawValue: 2), + floating: blockers.0, overviewOpen: blockers.1, mouseGestureActive: blockers.2, + displayGeometryChanged: blockers.3) == nil) + } + @Test func supersededSubmittedWorkspaceFocusRequiresNativeCancellation() { #expect(workspaceFocusNeedsNativeCancellation( @@ -94,6 +143,7 @@ struct DaemonCommandPolicyTests { #expect(toggleState.snapshot() == OverviewToggleSnapshot( generation: 0, actualIsOpen: false, + sessionGeneration: nil, desiredIsOpen: nil )) diff --git a/Tests/DefiMacOSTests/DesktopE2ETests.swift b/Tests/DefiMacOSTests/DesktopE2ETests.swift index e65a0821..3ff94694 100644 --- a/Tests/DefiMacOSTests/DesktopE2ETests.swift +++ b/Tests/DefiMacOSTests/DesktopE2ETests.swift @@ -7,6 +7,7 @@ import DefiConfig import DefiCore import DefiModel import Synchronization +import ScreenCaptureKit import XCTest import class SwiftUI.NSHostingMenu @@ -18,6 +19,621 @@ private final class DesktopHotKeyObserver: Sendable { @MainActor final class DesktopE2ETests: XCTestCase { + func testOverviewKeyboardWorkspaceTransitionKeepsIntermediatePositions() async throws { + _ = try makePlatform() + guard !NSWorkspace.shared.accessibilityDisplayShouldReduceMotion else { + throw XCTSkip("Overview animation requires Reduce Motion to be disabled") + } + let screen = try XCTUnwrap(NSScreen.main) + let monitorID = MonitorID(rawValue: (screen.deviceDescription[ + NSDeviceDescriptionKey("NSScreenNumber")] as! NSNumber).uint64Value) + let windows = (0..<5).map { index in + Window(id: WindowID(rawValue: UInt64(index + 1)), appID: "test", title: "Vertical window \(index)", + frame: Rect(x: 0, y: 0, width: 700, height: 700), processID: getpid()) + } + let workspaces = windows.map { window in + Workspace(id: WorkspaceID(rawValue: "vertical-\(window.id.rawValue)"), + columns: [Column(window: window.id, width: .fraction(0.5))]) + } + func snapshot(_ active: Int) -> OverviewSnapshot { + OverviewSnapshot(monitors: [Monitor(id: monitorID, workspaces: workspaces, + activeWorkspace: workspaces[active].id)], monitorFrames: [monitorID: + Rect(x: 0, y: 0, width: screen.frame.width, height: screen.visibleFrame.height)], + windows: Dictionary(uniqueKeysWithValues: windows.map { ($0.id, $0) }), activeMonitorID: monitorID) + } + var nativeSelections: [WindowID] = [] + let controller = OverviewController(focusWindow: { id, _, _, _ in nativeSelections.append(id) }, + focusWorkspace: { _, _ in }, drop: { _, _, _, _, _ in }, activateMonitor: { _ in }, + openStateChanged: { _ in }, commitScrollOffsets: { _ in }) + controller.open(snapshot: snapshot(2), layout: LayoutSettings(), windowPreviewsEnabled: false) + defer { controller.close(); controller.close() } + func findView(_ view: NSView) -> OverviewView? { + if let overview = view as? OverviewView { return overview } + return view.subviews.compactMap(findView).first + } + let input = try XCTUnwrap(NSApp.windows.compactMap { $0.contentView.flatMap(findView) }.first) + func frame() throws -> CGRect { + let children = try XCTUnwrap(input.accessibilityChildren() as? [NSAccessibilityElement]) + return try XCTUnwrap(children.first { $0.accessibilityLabel() == "Vertical window 2" }).accessibilityFrame() + } + try await Task.sleep(for: .milliseconds(250)) + let before = try frame() + controller.handleKey(.down) + try await Task.sleep(for: .milliseconds(45)) + let awaitingActivation = try frame() + XCTAssertTrue(nativeSelections.isEmpty, "Overview arrows must not move native windows") + controller.update(snapshot: snapshot(2), layout: LayoutSettings()) + XCTAssertEqual(try frame().minY, awaitingActivation.minY, accuracy: 1, + "An unchanged runtime snapshot must preserve deferred navigation") + XCTAssertGreaterThan(abs(awaitingActivation.minY - before.minY), 1, + "Vertical keyboard navigation must animate before native activation completes") + controller.update(snapshot: snapshot(3), layout: LayoutSettings()) + XCTAssertEqual(try frame().minY, awaitingActivation.minY, accuracy: 1, + "Activating the workspace must preserve the displayed source position") + try await Task.sleep(for: .milliseconds(35)) + let intermediate = try frame() + XCTAssertGreaterThan(abs(intermediate.minY - awaitingActivation.minY), 1, + "Activation must preserve the ongoing animation timeline") + controller.handleKey(.up) + controller.update(snapshot: snapshot(2), layout: LayoutSettings()) + XCTAssertEqual(try frame().minY, intermediate.minY, accuracy: 1, + "Repeated arrows must resume from the displayed position") + for _ in 0..<10 { + try await Task.sleep(for: .milliseconds(20)) + controller.update(snapshot: snapshot(2), layout: LayoutSettings()) + } + XCTAssertEqual(try frame().minY, before.minY, accuracy: 1, + "Snapshot refreshes must not prevent the transition from finishing") + controller.handleKey(.down) + controller.handleKey(.up) + controller.update(snapshot: snapshot(2), layout: LayoutSettings()) + try await Task.sleep(for: .milliseconds(200)) + XCTAssertEqual(try frame().minY, before.minY, accuracy: 1, + "A reversal before the first refresh must cancel the superseded destination") + controller.handleKey(.down) + try await Task.sleep(for: .milliseconds(35)) + controller.update(snapshot: snapshot(1), layout: LayoutSettings()) + try await Task.sleep(for: .milliseconds(200)) + let actualTarget = projectOverview(snapshot: snapshot(1), monitorID: monitorID, + bounds: Rect(x: 0, y: 0, width: input.bounds.width, height: input.bounds.height), + viewport: OverviewViewport(), layout: LayoutSettings(), zoom: 0.5) + let expectedCard = try XCTUnwrap(actualTarget.workspaces.flatMap(\.windows).first { + $0.windowID == windows[2].id + }).frame + let expectedFrame = try XCTUnwrap(input.window).convertToScreen(input.convert( + NSRect(x: expectedCard.x, y: expectedCard.y, width: expectedCard.width, height: expectedCard.height), to: nil)) + XCTAssertEqual(try frame().minY, expectedFrame.minY, accuracy: 1, + "An unrelated workspace activation must supersede the predicted destination") + } + + func testOverviewNavigationRenderBudget() async throws { + _ = try makePlatform() + let screen = try XCTUnwrap(NSScreen.main) + let monitorID = MonitorID(rawValue: (screen.deviceDescription[ + NSDeviceDescriptionKey("NSScreenNumber")] as! NSNumber).uint64Value) + let controller = OverviewController(focusWindow: { _, _, _, _ in }, + focusWorkspace: { _, _ in }, drop: { _, _, _, _, _ in }, activateMonitor: { _ in }, + openStateChanged: { _ in }, commitScrollOffsets: { _ in }) + let panel = OverviewPanel(monitorID: monitorID, screen: screen, + usesCapturedDesktop: true, delegate: controller) + defer { panel.hide() } + var windows: [WindowID: Window] = [:] + let workspaces = (0..<5).map { row in + Workspace(id: WorkspaceID(rawValue: "render-\(row)"), columns: (0..<4).map { column in + let id = WindowID(rawValue: UInt64(row * 4 + column + 1)) + windows[id] = Window(id: id, appID: "com.apple.finder", title: "Overview render fixture \(id)", + frame: Rect(x: 0, y: 0, width: 1200, height: 1300), processID: getpid()) + return Column(window: id, width: .fraction(0.66)) + }) + } + let snapshot = OverviewSnapshot(monitors: [Monitor(id: monitorID, workspaces: workspaces, + activeWorkspace: workspaces[2].id)], monitorFrames: [monitorID: + Rect(x: 0, y: 0, width: screen.frame.width, height: screen.visibleFrame.height)], + windows: windows, activeMonitorID: monitorID) + let image = NSImage(size: NSSize(width: 1024, height: 1024)) + image.lockFocus() + NSGradient(starting: .systemBlue, ending: .systemOrange)!.draw(in: + NSRect(x: 0, y: 0, width: 1024, height: 1024), angle: 45) + image.unlockFocus() + let previews = windows.mapValues { _ in image } + panel.setDesktopImage(image) + panel.show() + var costs: [Double] = [] + for index in 0..<160 { + let projection = projectOverview(snapshot: snapshot, monitorID: monitorID, + bounds: Rect(x: 0, y: 0, width: panel.view.bounds.width, height: panel.view.bounds.height), + viewport: OverviewViewport(workspaceOffset: sin(Double(index) / 20) * 0.6), + layout: LayoutSettings(), zoom: 0.5) + let started = CACurrentMediaTime() + panel.view.update(snapshot: snapshot, projection: projection, selection: nil, drag: nil, + borderStyle: WindowBorderStyle(config: BordersConfig()), windowCornerRadius: 12, + previews: previews, previewOpacities: [:]) + panel.view.displayIfNeeded() + if index >= 20 { costs.append(CACurrentMediaTime() - started) } + if index % 20 == 0 { await Task.yield() } + } + let sorted = costs.sorted() + let mean = costs.reduce(0, +) / Double(costs.count) + let p95 = sorted[Int(Double(sorted.count - 1) * 0.95)] + print("DEFI_E2E overview-draw meanMs=\(mean * 1000) p95Ms=\(p95 * 1000) samples=\(costs.count)") + XCTAssertLessThan(p95, 1 / Double(max(screen.maximumFramesPerSecond, 60)), + "Warm overview drawing must fit one refresh budget") + XCTAssertLessThanOrEqual(panel.view.titleRasterizationCount, windows.count, + "Moving cards must reuse their rendered titles") + + // The same view is a real event source for the controller's installed panel. + controller.open(snapshot: snapshot, layout: LayoutSettings(), windowPreviewsEnabled: false) + defer { controller.close() } + func findView(_ view: NSView) -> OverviewView? { + if let overview = view as? OverviewView { return overview } + return view.subviews.compactMap(findView).first + } + let input = try XCTUnwrap(NSApp.windows.compactMap { window -> OverviewView? in + guard window !== panel.window, let root = window.contentView else { return nil } + return findView(root) + }.first) + let groups = try XCTUnwrap(input.accessibilityChildren() as? [NSAccessibilityElement]) + let group = try XCTUnwrap(groups.first { $0.accessibilityRole() == .group }) + let label = group.accessibilityLabel() + let before = group.accessibilityFrame() + let updatesBeforeScroll = input.presentationUpdateCount + for _ in 0..<100 { + controller.overviewView(input, scrolled: NSPoint(x: 0, y: -1), + hasPreciseScrollingDeltas: true, at: NSPoint(x: 100, y: 100)) + } + XCTAssertLessThan(input.presentationUpdateCount - updatesBeforeScroll, 5, + "Precise events must coalesce rather than project every input event") + try await Task.sleep(for: .milliseconds(100)) + let afterGroups = try XCTUnwrap(input.accessibilityChildren() as? [NSAccessibilityElement]) + let after = try XCTUnwrap(afterGroups.first { $0.accessibilityLabel() == label }).accessibilityFrame() + XCTAssertEqual(after.minY - before.minY, 100, accuracy: 0.5, + "The last pending frame must preserve all trackpad deltas") + } + + func testOverviewPressureRecoveryPreloadsAgainWithoutOpening() async throws { + _ = try makePlatform() + guard CGPreflightScreenCaptureAccess() else { + throw XCTSkip("Screen Recording permission unavailable to test process") + } + let screen = try XCTUnwrap(NSScreen.main) + let fixture = NSWindow(contentRect: NSRect(x: screen.frame.midX - 320, + y: screen.frame.midY - 220, width: 640, height: 440), + styleMask: [.titled], backing: .buffered, defer: false) + fixture.isReleasedWhenClosed = false + fixture.orderFrontRegardless() + defer { fixture.close() } + let id = WindowID(rawValue: UInt64(fixture.windowNumber)) + let content = try await SCShareableContent.excludingDesktopWindows(true, onScreenWindowsOnly: false) + let owner = try XCTUnwrap(content.windows.first(where: { $0.windowID == fixture.windowNumber })?.owningApplication) + let monitor = MonitorID(rawValue: (screen.deviceDescription[ + NSDeviceDescriptionKey("NSScreenNumber")] as! NSNumber).uint64Value) + let workspace = Workspace(id: WorkspaceID(rawValue: "pressure-preview"), + columns: [Column(window: id, width: .fraction(0.5))]) + let snapshot = OverviewSnapshot(monitors: [Monitor(id: monitor, + workspaces: [workspace], activeWorkspace: workspace.id)], + monitorFrames: [monitor: Rect(x: screen.frame.minX, y: 0, + width: screen.frame.width, height: 440)], + windows: [id: Window(id: id, appID: owner.bundleIdentifier, title: "Pressure preview", + frame: Rect(x: 0, y: 0, width: 640, height: 440), + processID: owner.processID)], activeMonitorID: monitor) + let recoveries = DesktopValue(0) + let controller = OverviewController(focusWindow: { _, _, _, _ in }, + focusWorkspace: { _, _ in }, drop: { _, _, _, _, _ in }, activateMonitor: { _ in }, + openStateChanged: { _ in }, idlePreparationRequested: { recoveries.value += 1 }, + notificationCenter: NotificationCenter(), commitScrollOffsets: { _ in }) + defer { controller.prepare(windowPreviewsEnabled: false) } + func prepare() { + controller.prepare(windowPreviewsEnabled: true, snapshot: snapshot, + layout: LayoutSettings(), experimentalSurfaceTransitions: true) + } + prepare() + for _ in 0..<80 where controller.rememberedPreviewMemoryBytes == 0 { + try await Task.sleep(for: .milliseconds(100)) + } + let bytes = controller.rememberedPreviewMemoryBytes + XCTAssertGreaterThan(bytes, 0) + controller.handleMemoryPressure(.warning) + XCTAssertEqual(controller.rememberedPreviewMemoryBytes, bytes) + XCTAssertLessThanOrEqual(bytes, 4 * 1_024 * 1_024) + controller.handleMemoryPressure(.critical) + XCTAssertEqual(controller.rememberedPreviewMemoryBytes, 0) + controller.handleMemoryPressure(.normal) + XCTAssertEqual(recoveries.value, 1) + prepare() + for _ in 0..<80 where controller.rememberedPreviewMemoryBytes == 0 { + try await Task.sleep(for: .milliseconds(100)) + } + XCTAssertGreaterThan(controller.rememberedPreviewMemoryBytes, 0, + "The same preview request must run again after eviction, without an overview opening") + XCTAssertFalse(controller.isOpen) + } + + func testOverviewAvailableSurfacesRetainReadyImageWhenAnotherIsMissing() async throws { + _ = try makePlatform() + guard CGPreflightScreenCaptureAccess() else { + throw XCTSkip("Screen Recording permission unavailable to test process") + } + let fixture = NSWindow(contentRect: NSRect(x: 100, y: 100, width: 640, height: 440), + styleMask: [.titled], backing: .buffered, defer: false) + fixture.isReleasedWhenClosed = false + fixture.orderFrontRegardless() + fixture.displayIfNeeded() + CATransaction.flush() + let capture = OverviewSurfaceCapture.shared + defer { capture.stop(); fixture.close() } + let content = try await SCShareableContent.excludingDesktopWindows(true, onScreenWindowsOnly: false) + let native = try XCTUnwrap(content.windows.first { $0.windowID == fixture.windowNumber }) + let owner = try XCTUnwrap(native.owningApplication) + let id = WindowID(rawValue: UInt64(native.windowID)) + capture.prepare([OverviewSurfaceRequest(windowID: id, + appID: owner.bundleIdentifier, processID: owner.processID, width: 640, height: 440)], enabled: true) + for _ in 0..<60 where capture.displayLayer(for: id) == nil { + try await Task.sleep(for: .milliseconds(100)) + } + XCTAssertNotNil(capture.displayLayer(for: id)) + let missing = WindowID(rawValue: UInt64.max) + XCTAssertNil(capture.frames(windowIDs: [id, missing]), + "Native ribbon replacement must remain all-or-nothing") + let available = try XCTUnwrap(capture.availableFrames(windowIDs: [id, missing])) + XCTAssertEqual(Set(available.keys), [id], + "A missing image must not cancel the ready overview texture") + XCTAssertNil(capture.availableFrames(windowIDs: [missing])) + let screen = try XCTUnwrap(fixture.screen) + let monitorID = MonitorID(rawValue: (screen.deviceDescription[ + NSDeviceDescriptionKey("NSScreenNumber")] as! NSNumber).uint64Value) + let primaryHeight = NSScreen.screens.first?.frame.height ?? 0 + let frame = Rect(x: fixture.frame.minX, y: primaryHeight - fixture.frame.maxY, + width: fixture.frame.width, height: fixture.frame.height) + let workspace = Workspace(id: WorkspaceID(rawValue: "opening-fallback"), + columns: [Column(window: id, width: .fraction(0.5))]) + let snapshot = OverviewSnapshot(monitors: [Monitor(id: monitorID, + workspaces: [workspace], activeWorkspace: workspace.id)], + monitorFrames: [monitorID: Rect(x: screen.frame.minX, y: primaryHeight - screen.frame.maxY, + width: screen.frame.width, height: screen.frame.height)], + windows: [id: Window(id: id, appID: owner.bundleIdentifier, title: "Opening fallback", + frame: frame, processID: owner.processID)], activeMonitorID: monitorID) + let projection = projectOverview(snapshot: snapshot, monitorID: monitorID, + bounds: Rect(x: 0, y: 0, width: screen.frame.width, height: screen.frame.height), + viewport: OverviewViewport(), layout: LayoutSettings(), zoom: 0.5) + let cached = NSImage(size: NSSize(width: 8, height: 8), flipped: false) { rect in + NSColor.red.setFill(); rect.fill(); return true + } + for previews: [WindowID: NSImage] in [[:], [id: cached]] { + let scene = try XCTUnwrap(OverviewSurfaceScene(projection: projection, + workspaceID: workspace.id, screen: screen, surfaces: [:], + windows: snapshot.windows, previews: previews)) + XCTAssertEqual(Set(scene.nativeFrames.keys), [id]) + let item = try XCTUnwrap(scene.layer.sublayers?.first) + XCTAssertEqual(item.contents != nil, !previews.isEmpty) + if previews.isEmpty { + XCTAssertGreaterThan(item.borderWidth, 0) + XCTAssertNotNil(item.sublayers?.first?.contents, + "An uncaptured window must remain recognizable during the zoom") + } else { + XCTAssertTrue(item.sublayers?.isEmpty ?? true) + } + // An unattached layer has no render-tree animation lifetime. Host the + // scene as production does before checking its running animation. + fixture.contentView?.wantsLayer = true + fixture.contentView?.layer?.addSublayer(scene.layer) + scene.animate(opening: true, duration: 0.22) + XCTAssertNotNil(item.animation(forKey: "overview-surface"), + "Opening must zoom even without a ready screenshot") + XCTAssertEqual(scene.finishOpening(), [id]) + XCTAssertTrue(scene.nativeFrames.isEmpty, "Closing must not reuse opening fallback layers") + XCTAssertTrue(scene.layer.sublayers?.isEmpty ?? true) + scene.layer.removeFromSuperlayer() + } + } + + func testOverviewSurfaceTransitionUsesFreshOwnedWindowAndReopensSafely() async throws { + _ = try makePlatform() + guard CGPreflightScreenCaptureAccess() else { + print("DEFI_E2E screen-recording=unavailable") + throw XCTSkip("Screen Recording permission unavailable to test process") + } + print("DEFI_E2E screen-recording=available") + let screen = try XCTUnwrap(NSScreen.main) + let recordsEvidence = ProcessInfo.processInfo.environment["DEFI_SURFACE_DEMO"] == "1" + var backdrop: NSWindow? + if recordsEvidence { + let background = NSWindow(contentRect: screen.frame, styleMask: .borderless, + backing: .buffered, defer: false) + background.isReleasedWhenClosed = false + background.backgroundColor = NSColor(calibratedRed: 0.08, green: 0.10, blue: 0.14, alpha: 1) + background.orderFrontRegardless() + backdrop = background + } + defer { backdrop?.close() } + let fixture = NSWindow(contentRect: NSRect(x: screen.frame.midX - 320, + y: screen.frame.midY - 220, width: 640, height: 440), + styleMask: [.titled], backing: .buffered, defer: false) + var nativeFrame = screen.visibleFrame + nativeFrame.size.width = 640 + fixture.setFrame(nativeFrame, display: false) + fixture.title = "Defi Surface Fixture" + fixture.isReleasedWhenClosed = false + fixture.animationBehavior = .none + let contentView = NSView() + contentView.wantsLayer = true + contentView.layer?.backgroundColor = NSColor(calibratedRed: 0.12, green: 0.17, blue: 0.24, alpha: 1).cgColor + let title = NSTextField(labelWithString: "NATIVE WINDOW → LIVE OVERVIEW") + title.font = NSFont.systemFont(ofSize: 23, weight: .semibold) + title.textColor = .white + let contentHeight = fixture.contentLayoutRect.height + title.frame = NSRect(x: 32, y: contentHeight - 90, width: 580, height: 40) + contentView.addSubview(title) + let description = NSTextField(labelWithString: "Direct ScreenCaptureKit surface · no bitmap conversion") + description.font = NSFont.systemFont(ofSize: 17) + description.textColor = .lightGray + description.frame = NSRect(x: 32, y: contentHeight - 132, width: 580, height: 30) + contentView.addSubview(description) + let dot = CALayer() + dot.backgroundColor = NSColor.systemTeal.cgColor + dot.frame = CGRect(x: 32, y: contentHeight / 2 - 36, width: 72, height: 72) + dot.cornerRadius = 36 + contentView.layer?.addSublayer(dot) + let movement = CABasicAnimation(keyPath: "position.x") + movement.fromValue = 68 + movement.toValue = 560 + movement.duration = 1.2 + movement.autoreverses = true + movement.repeatCount = .infinity + dot.add(movement, forKey: "live-content") + fixture.contentView = contentView + fixture.orderFrontRegardless() + fixture.displayIfNeeded() + CATransaction.flush() + defer { fixture.close() } + let content = try await SCShareableContent.excludingDesktopWindows(true, onScreenWindowsOnly: false) + let native = try XCTUnwrap(content.windows.first(where: { $0.windowID == fixture.windowNumber })) + let nativeFrameBounds = CGRect(x: fixture.frame.minX, + y: (NSScreen.screens.first?.frame.height ?? 0) - fixture.frame.maxY, + width: fixture.frame.width, height: fixture.frame.height) + var fixtureIsSettled = false + for _ in 0..<30 { + let info = (CGWindowListCopyWindowInfo(.optionIncludingWindow, + CGWindowID(fixture.windowNumber)) as? [[String: Any]])?.first(where: { + ($0[kCGWindowNumber as String] as? NSNumber)?.intValue == fixture.windowNumber + }) + if let bounds = info?[kCGWindowBounds as String] as? [String: Any], + CGRect(dictionaryRepresentation: bounds as CFDictionary) == nativeFrameBounds { + fixtureIsSettled = true; break + } + CATransaction.flush() + try await Task.sleep(for: .milliseconds(50)) + } + XCTAssertTrue(fixtureIsSettled, "Owned fixture must finish its native appearance before sampling") + let appID = try XCTUnwrap(native.owningApplication?.bundleIdentifier) + let processID = try XCTUnwrap(native.owningApplication?.processID) + let windowID = WindowID(rawValue: UInt64(native.windowID)) + let monitorID = MonitorID(rawValue: (screen.deviceDescription[ + NSDeviceDescriptionKey("NSScreenNumber")] as! NSNumber).uint64Value) + let workspace = Workspace(id: WorkspaceID(rawValue: "surface-fixture"), + columns: [Column(window: windowID, width: .fraction(nativeFrameBounds.width / screen.frame.width))]) + let snapshot = OverviewSnapshot( + monitors: [Monitor(id: monitorID, workspaces: [workspace], activeWorkspace: workspace.id)], + monitorFrames: [monitorID: Rect(x: screen.frame.minX, y: nativeFrameBounds.minY, + width: screen.frame.width, height: nativeFrameBounds.height)], + windows: [windowID: Window(id: windowID, appID: appID, title: fixture.title, + frame: Rect(x: nativeFrameBounds.minX, y: nativeFrameBounds.minY, + width: nativeFrameBounds.width, height: nativeFrameBounds.height), processID: processID)], + activeMonitorID: monitorID) + let surfaceLayout = LayoutSettings(outerTopGap: 0, outerRightGap: 0, + outerBottomGap: 0, outerLeftGap: 0) + let controller = OverviewController(focusWindow: { _, _, _, _ in }, + focusWorkspace: { _, _ in }, drop: { _, _, _, _, _ in }, activateMonitor: { _ in }, + openStateChanged: { _ in }, commitScrollOffsets: { _ in }) + defer { + controller.close() + controller.prepare(windowPreviewsEnabled: false) + } + controller.prepare(windowPreviewsEnabled: true, snapshot: snapshot, layout: surfaceLayout, + experimentalSurfaceTransitions: true) + for _ in 0..<60 where controller.surfaceCaptureState == "warming" { + try await Task.sleep(for: .milliseconds(100)) + } + XCTAssertEqual(controller.surfaceCaptureState, "ready") + try await Task.sleep(for: .milliseconds(150)) + let originalCapture = try XCTUnwrap(OverviewSurfaceCapture.shared.frames(windowIDs: [windowID])?[windowID]) + OverviewSurfaceCapture.shared.prepare([OverviewSurfaceRequest(windowID: windowID, + appID: "different.owner", processID: processID + 1, + width: originalCapture.width, height: originalCapture.height)], enabled: true) + XCTAssertNil(OverviewSurfaceCapture.shared.frames(windowIDs: [windowID]), + "A reused ID must stop exposing the previous owner's pixels immediately, including during throttle") + controller.prepare(windowPreviewsEnabled: true, snapshot: snapshot, layout: surfaceLayout, + experimentalSurfaceTransitions: true) + for _ in 0..<80 where controller.surfaceCaptureState != "ready" { + try await Task.sleep(for: .milliseconds(100)) + } + let capturedFrames = try XCTUnwrap(OverviewSurfaceCapture.shared.frames(windowIDs: [windowID])) + let projection = projectOverview(snapshot: snapshot, monitorID: monitorID, + bounds: Rect(x: 0, y: 0, width: screen.frame.width, height: screen.frame.height), + viewport: OverviewViewport(), layout: surfaceLayout, zoom: 0.5) + let previousScene = try XCTUnwrap(OverviewSurfaceScene(projection: projection, + workspaceID: workspace.id, screen: screen, surfaces: capturedFrames, + windows: snapshot.windows)) + previousScene.animate(opening: true, duration: 0) + let reusedLayer = try XCTUnwrap(OverviewSurfaceCapture.shared.displayLayer(for: windowID)) + XCTAssertFalse(CATransform3DIsIdentity(reusedLayer.transform)) + XCTAssertEqual(reusedLayer.cornerRadius * reusedLayer.transform.m11, 12, accuracy: 0.01, + "The overview texture must match the card's radius after scaling") + let reopenedScene = try XCTUnwrap(OverviewSurfaceScene(projection: projection, + workspaceID: workspace.id, screen: screen, surfaces: capturedFrames, + windows: snapshot.windows)) + XCTAssertTrue(CATransform3DIsIdentity(reusedLayer.transform), + "A reused Overview texture must start at native scale, even after a non-reversing close") + XCTAssertEqual(reusedLayer.cornerRadius, 12) + XCTAssertEqual(reusedLayer.bounds.size, nativeFrameBounds.size) + _ = reopenedScene + // Isolate the owned-window handoff from unrelated user windows. The + // production observer remains conservative whenever another window is visible. + var visibleWindowInfo: [[String: Any]] = [] + let renderer = ExperimentalRibbonRenderer(visibleWindowInfo: { visibleWindowInfo }) + renderer.prepare(snapshot: snapshot, layout: surfaceLayout, enabled: true) + defer { renderer.disable() } + for _ in 0..<120 where !renderer.backgroundsReady { + try await Task.sleep(for: .milliseconds(50)) + } + XCTAssertTrue(renderer.backgroundsReady, "Owned renderer background capture must finish before animation") + let originalRibbonFrame = fixture.frame + let target = Rect(x: nativeFrameBounds.minX + 60, y: nativeFrameBounds.minY, + width: nativeFrameBounds.width, height: nativeFrameBounds.height) + XCTAssertTrue(renderer.begin(assignments: [FrameAssignment(windowID: windowID, frame: target)], duration: 0.15), renderer.lastFallback) + XCTAssertTrue(renderer.isPresenting) + let firstPanels = Set(NSApplication.shared.windows.filter { + $0.title == "Defi Experimental Ribbon" + }.map(\.windowNumber)) + XCTAssertEqual(firstPanels.count, 1) + fixture.setFrame(originalRibbonFrame.offsetBy(dx: 60, dy: 0), display: true) + let retarget = Rect(x: target.x + 30, y: target.y, + width: target.width, height: target.height) + XCTAssertTrue(renderer.begin(assignments: [FrameAssignment(windowID: windowID, frame: retarget)], + duration: 0.15), renderer.lastFallback) + XCTAssertEqual(Set(NSApplication.shared.windows.filter { + $0.title == "Defi Experimental Ribbon" + }.map(\.windowNumber)), firstPanels, "Retargeting must reuse the visible overlay without a close/reopen") + fixture.setFrame(originalRibbonFrame.offsetBy(dx: 90, dy: 0), display: true) + for _ in 0..<30 where renderer.isPresenting { + CATransaction.flush() + try await Task.sleep(for: .milliseconds(50)) + } + XCTAssertFalse(renderer.isPresenting, "Proxy must yield after native convergence") + fixture.setFrame(originalRibbonFrame, display: true) + renderer.prepare(snapshot: snapshot, layout: surfaceLayout, enabled: true) + CATransaction.flush() + try await Task.sleep(for: .milliseconds(100)) + XCTAssertTrue(renderer.begin(assignments: [FrameAssignment(windowID: windowID, frame: target)], duration: 0.15)) + XCTAssertFalse(renderer.begin(assignments: [FrameAssignment(windowID: windowID, + frame: target)], duration: 0.15)) + XCTAssertFalse(renderer.isPresenting, "A no-plan fallback must dismiss an older proxy") + visibleWindowInfo = [[kCGWindowNumber as String: NSNumber(value: UInt64.max), + kCGWindowOwnerPID as String: NSNumber(value: Int32.max), + kCGWindowLayer as String: NSNumber(value: 0), + kCGWindowBounds as String: nativeFrameBounds.dictionaryRepresentation]] + XCTAssertFalse(renderer.begin(assignments: [FrameAssignment(windowID: windowID, frame: retarget)], duration: 0.15)) + XCTAssertEqual(renderer.lastFallback, "unrepresented-window") + XCTAssertFalse(renderer.isPresenting, "Unrepresented windows must remain visible on the native path") + visibleWindowInfo = [] + + // AppKit may normalize a prepared panel's frame. Rebuilding it must keep captures warm. + for panel in NSApplication.shared.windows where panel.title == "Defi Overview" { + panel.setFrame(panel.frame.offsetBy(dx: 1, dy: 0), display: false) + } + if recordsEvidence { + for _ in 0..<20 { + CATransaction.flush() + try await Task.sleep(for: .milliseconds(50)) + } + } + // A delayed idle refresh must not discard the last compatible snapshot. + try await Task.sleep(for: .seconds(3.2)) + let originalFrame = fixture.frame + for sample in 0..<2 { + controller.open(snapshot: snapshot, layout: surfaceLayout, windowPreviewsEnabled: true, + experimentalSurfaceTransitions: true) + XCTAssertEqual(controller.surfaceTransitionCount, sample + 1) + XCTAssertEqual(controller.surfaceFallbackCount, 0) + XCTAssertEqual(controller.surfaceStreamCount, 0) + XCTAssertLessThanOrEqual(controller.surfaceEstimatedPoolBytes, overviewSurfacePoolBudget) + print("DEFI_E2E surface sample=\(sample) acquireMs=\(controller.surfaceAcquireMs) poolBytes=\(controller.surfaceEstimatedPoolBytes)") + for _ in 0..<(recordsEvidence ? 40 : (sample == 0 ? 66 : 6)) { + CATransaction.flush() + try await Task.sleep(for: .milliseconds(50)) + } + XCTAssertEqual(controller.surfacePresentedFrameCount, 1, + "The zoom must retain one fixed screenshot, without live swaps or capture sessions") + XCTAssertEqual(fixture.frame, originalFrame, "Projection must not mutate native geometry") + controller.close() + // Discovery ticks during closing must not replace the texture being handed back. + controller.prepare(windowPreviewsEnabled: true, snapshot: snapshot, layout: surfaceLayout, + experimentalSurfaceTransitions: true) + CATransaction.flush() + if sample == 0 { + // Reopen before the previous closing task ends; it must not hide the new scene. + try await Task.sleep(for: .milliseconds(60)) + } else { + try await Task.sleep(for: .milliseconds(300)) + } + } + // Enter on the current selection must reverse the zoom just like toggling. + controller.open(snapshot: snapshot, layout: surfaceLayout, windowPreviewsEnabled: true, + experimentalSurfaceTransitions: true) + try await Task.sleep(for: .milliseconds(250)) + controller.handleKey(.select) + try await Task.sleep(for: .milliseconds(180)) + XCTAssertFalse(controller.isOpen) + XCTAssertEqual(controller.surfacePresentedFrameCount, 1, + "Enter must retain the surface scene during the closing zoom") + // Simulate resource expiry winning over a delayed closing continuation. + controller.releaseIdleOverviewResources() + XCTAssertTrue(NSApplication.shared.windows.filter { $0.title == "Defi Overview" }.allSatisfy { !$0.isVisible }, + "Memory cleanup must hide the overlay even when it cancels the closing task") + try await Task.sleep(for: .milliseconds(250)) + if recordsEvidence { try await Task.sleep(for: .seconds(1)) } + XCTAssertFalse(controller.isOpen) + XCTAssertTrue(NSApplication.shared.windows.filter { $0.title == "Defi Overview" }.allSatisfy { !$0.isVisible }) + // Workspace navigation invalidates the original full-resolution scene. Its + // already-displayed preview must still animate back to the target window. + let destination = NSWindow(contentRect: originalFrame, styleMask: [.titled], + backing: .buffered, defer: false) + destination.isReleasedWhenClosed = false + destination.title = "Defi Destination Fixture" + destination.backgroundColor = .systemBlue + destination.setFrame(originalFrame, display: true) + destination.orderFrontRegardless() + defer { destination.close() } + let destinationID = WindowID(rawValue: UInt64(destination.windowNumber)) + let destinationWorkspace = Workspace(id: WorkspaceID(rawValue: "destination"), + columns: [Column(window: destinationID, width: .fraction(nativeFrameBounds.width / screen.frame.width))]) + var destinationWindows = snapshot.windows + destinationWindows[destinationID] = Window(id: destinationID, appID: appID, + title: destination.title, frame: snapshot.windows[windowID]!.frame, processID: processID) + func navigationSnapshot(active: WorkspaceID) -> OverviewSnapshot { + OverviewSnapshot(monitors: [Monitor(id: monitorID, + workspaces: [workspace, destinationWorkspace], activeWorkspace: active)], + monitorFrames: snapshot.monitorFrames, windows: destinationWindows, + activeMonitorID: monitorID) + } + controller.prepare(windowPreviewsEnabled: true, snapshot: navigationSnapshot(active: workspace.id), + layout: surfaceLayout, experimentalSurfaceTransitions: true) + controller.open(snapshot: navigationSnapshot(active: workspace.id), layout: surfaceLayout, + windowPreviewsEnabled: true, experimentalSurfaceTransitions: true) + controller.handleKey(.down) + controller.update(snapshot: navigationSnapshot(active: destinationWorkspace.id), + layout: surfaceLayout, windowPreviewsEnabled: true) + for _ in 0..<30 where controller.previewCacheCount == 0 { + try await Task.sleep(for: .milliseconds(50)) + } + try await Task.sleep(for: .milliseconds(250)) + controller.handleKey(.select) + try await Task.sleep(for: .milliseconds(180)) + XCTAssertFalse(controller.isOpen) + XCTAssertTrue(NSApplication.shared.windows.filter { $0.title == "Defi Overview" }.contains { $0.isVisible }, + "Changing workspace must retain the displayed preview during the reverse zoom") + XCTAssertEqual(controller.previewClosingCount, 1) + try await Task.sleep(for: .milliseconds(300)) + XCTAssertTrue(NSApplication.shared.windows.filter { $0.title == "Defi Overview" }.allSatisfy { !$0.isVisible }) + XCTAssertEqual(destination.frame, originalFrame) + controller.prepare(windowPreviewsEnabled: false) + controller.open(snapshot: snapshot, layout: surfaceLayout, windowPreviewsEnabled: false) + XCTAssertTrue(ExperimentalRibbonRenderer.shared.requests.isEmpty) + controller.update(snapshot: snapshot, layout: surfaceLayout, windowPreviewsEnabled: false, + experimentalSurfaceTransitions: false, experimentalRibbonRepresentations: true) + XCTAssertTrue(ExperimentalRibbonRenderer.shared.requests.contains { $0.windowID == windowID }, + "Enabling representations in an open Overview must prepare the current native context") + for _ in 0..<80 where controller.surfaceCaptureState != "ready" { + try await Task.sleep(for: .milliseconds(100)) + } + XCTAssertEqual(controller.surfaceCaptureState, "ready") + controller.update(snapshot: snapshot, layout: surfaceLayout, + experimentalSurfaceTransitions: false, experimentalRibbonRepresentations: false) + XCTAssertTrue(ExperimentalRibbonRenderer.shared.requests.isEmpty) + controller.close() + controller.prepare(windowPreviewsEnabled: false) + XCTAssertEqual(controller.surfaceStreamCount, 0) + } + func testOverviewShowsBothMonitorsBeforeLoadingPreviews() throws { _ = try makePlatform() let screens = NSScreen.screens @@ -533,13 +1149,17 @@ final class DesktopE2ETests: XCTestCase { } XCTAssertTrue(pumpRunLoop(until: { !onNavigation { platform.hasPendingFrameWrites } }, timeout: 1)) let sizeWrites = onNavigation { platform.successfulSizeWriteCount } + // A recent stall must adapt the width animation instead of discarding it. + if let processID = window.processID { + platform.frameCoordinator.recordProcessLatencySamples([processID: 70]) + } onNavigation { platform.apply( [FrameAssignment(windowID: window.id, frame: target)], animationDuration: 0.08, animationRefreshRateHz: 120, animateSizeChanges: true, - source: "test-resize-animation" + source: "command-layout-animation" ) } XCTAssertTrue( pumpRunLoop( @@ -564,10 +1184,47 @@ final class DesktopE2ETests: XCTestCase { XCTAssertEqual(actual?.width ?? 0, target.width, accuracy: 2) XCTAssertGreaterThanOrEqual( onNavigation { platform.successfulSizeWriteCount } - sizeWrites, - 1 + 3, + "A resize animation must write intermediate sizes, not only its final size." ) } + func testAnimatedFullWidthConvergesFromBothViewportEdges() throws { + let platform = try makePlatform() + let snapshot = platform.snapshot(config: Config()) + let monitor = try XCTUnwrap(snapshot.monitors.first) + let window = try XCTUnwrap(testWindows(in: snapshot).first(where: { + $0.appID == "com.mitchellh.ghostty" + }) ?? testWindows(in: snapshot).first) + let original = window.frame + defer { + onNavigation { platform.apply([FrameAssignment(windowID: window.id, frame: original)]) } + pumpRunLoop(for: 0.3) + } + let full = Rect(x: monitor.frame.x, y: original.y, + width: monitor.frame.width, height: original.height) + for fraction in [0.5, 0.0, 0.25] { + let start = Rect(x: monitor.frame.x + monitor.frame.width * fraction, y: original.y, + width: monitor.frame.width / 2, height: original.height) + onNavigation { platform.apply([FrameAssignment(windowID: window.id, frame: start)]) } + XCTAssertTrue(pumpRunLoop(until: { !onNavigation { platform.hasPendingFrameWrites } }, timeout: 2)) + // Refresh the observed starting frame before the animated enlargement. + _ = platform.snapshot(config: Config()) + onNavigation { platform.apply([FrameAssignment(windowID: window.id, frame: full)], + animationDuration: 0.125, animationRefreshRateHz: 120, + animateSizeChanges: true, source: "command-layout-animation") } + XCTAssertTrue(pumpRunLoop(until: { !onNavigation { platform.hasPendingFrameWrites } }, timeout: 2)) + var actual: Rect? + let converged = pumpRunLoop(until: { + actual = platform.snapshot(config: Config()).windows.first { $0.id == window.id }?.frame + guard let actual else { return false } + return abs(actual.x - full.x) <= 2 && abs(actual.width - full.width) <= 2 + }, timeout: 1) + XCTAssertTrue(converged, + "Full width from \(fraction) must converge; actual=\(String(describing: actual)) trace=\(onNavigation { platform.frameCoordinatorTrace })") + } + } + func testUnhiddenOnePixelStripAnchorConvergesWithRealWindowFrame() throws { let platform = try makePlatform() let snapshot = platform.snapshot(config: Config()) @@ -1531,6 +2188,58 @@ final class DesktopE2ETests: XCTestCase { XCTAssertEqual(onNavigation { manager.capturedKeyCount }, 1) } + func testConfiguredHyperWorkspaceBindingsNavigateOverview() throws { + _ = try makePlatform() + let config = Config(modifierCombinations: ["hyper": "Alt + Cmd + Ctrl"], keys: [ + "hyper-1": "workspace dev", "hyper-2": "focus-workspace-position 2", + "hyper-3": "focus-workspace-name web"]) + let commands = DesktopValue<[HotKeyInvocation]>([]) + let actions = DesktopValue<[OverviewKeyAction]>([]) + let manager = onNavigation { HotKeyManager(config: config, + overviewHandler: { value in DispatchQueue.main.async { actions.value.append(value) } } + ) { value in DispatchQueue.main.async { commands.value.append(value) } } } + try onNavigation { try manager.start() } + onNavigation { manager.setOverviewModeEnabled(true) } + let expected: [OverviewKeyAction] = [.workspace(.named("dev")), + .workspace(.position(2)), .workspace(.named("web"))] + for (index, key) in [CGKeyCode(18), 19, 20].enumerated() { + let event = try XCTUnwrap(CGEvent(keyboardEventSource: CGEventSource(stateID: .hidSystemState), + virtualKey: key, keyDown: true)) + event.flags = [.maskAlternate, .maskCommand, .maskControl] + event.post(tap: .cghidEventTap) + XCTAssertTrue(pumpRunLoop(until: { actions.value.count == index + 1 }, timeout: 1)) + } + XCTAssertEqual(actions.value, expected) + XCTAssertTrue(commands.value.isEmpty, "Workspace shortcuts must not mutate the native workspace during overview") + } + + func testOverviewCancelBypassesBusyNavigationActor() throws { + _ = try makePlatform() + let closed = DesktopValue(false) + let manager = onNavigation { HotKeyManager(config: Config(), + overviewCancelHandler: { DispatchQueue.main.async { closed.value = true } } + ) { _ in } } + try onNavigation { try manager.start() } + onNavigation { manager.setOverviewModeEnabled(true) } + let entered = DispatchSemaphore(value: 0) + let release = DispatchSemaphore(value: 0) + NavigationActor.enqueue { + entered.signal() + release.wait() + } + defer { + release.signal() + onNavigation { manager.stop() } + } + XCTAssertEqual(entered.wait(timeout: .now() + 2), .success) + DispatchQueue.global(qos: .userInteractive).async { + guard let event = CGEvent(keyboardEventSource: nil, virtualKey: 53, keyDown: true) else { return } + event.post(tap: .cghidEventTap) + } + XCTAssertTrue(pumpRunLoop(until: { closed.value }, timeout: 1), + "Escape must reach the overview without waiting for the AX/navigation lane") + } + func testScrollWheelAdvancesUserInputTracker() throws { _ = try makePlatform() let tracker = UserInputTracker() diff --git a/Tests/DefiMacOSTests/FrameCommitTests.swift b/Tests/DefiMacOSTests/FrameCommitTests.swift index b9f4a4e3..aaa977d1 100644 --- a/Tests/DefiMacOSTests/FrameCommitTests.swift +++ b/Tests/DefiMacOSTests/FrameCommitTests.swift @@ -8,6 +8,34 @@ import Testing @testable import DefiMacOS struct FrameCommitTests { + @Test func overviewExitAcceptsVerifiedParkingDespiteNativeVerticalClamping() { + let visible = WindowID(rawValue: 1), parked = WindowID(rawValue: 2) + let monitor = Rect(x: 0, y: 0, width: 1512, height: 910) + let shown = Rect(x: 0, y: 37, width: 752, height: 902) + let target = Rect(x: 1511, y: 37, width: 752, height: 902) + let clamped = Rect(x: 1511, y: 30, width: 752, height: 902) + func ready(_ actual: Rect, pending: Set = []) -> Bool { + overviewExitFramesAreReady(windowIDs: [visible, parked], hiddenWindowIDs: [parked], + pendingWriteWindowIDs: pending, unresolvedWindowIDs: [parked], + targets: [visible: shown, parked: target], observed: [visible: shown, parked: actual], + monitorFrames: [monitor]) + } + #expect(ready(clamped), "A safe parked window must not disable the visible zoom handoff") + #expect(!ready(shown), "A parking leak must still prevent uncovering native windows") + #expect(!ready(clamped, pending: [parked]), "Outstanding writes must settle before uncovering") + #expect(!overviewExitFramesAreReady(windowIDs: [visible], hiddenWindowIDs: [], + pendingWriteWindowIDs: [], unresolvedWindowIDs: [visible], targets: [visible: shown], + observed: [visible: shown], monitorFrames: [monitor])) + #expect(!overviewExitFramesAreReady(windowIDs: [parked], hiddenWindowIDs: [parked], + pendingWriteWindowIDs: [], unresolvedWindowIDs: [parked], targets: [parked: target], + observed: [parked: clamped], monitorFrames: [monitor, + Rect(x: 1512, y: 0, width: 1512, height: 910)]), + "Parking must be safe on every monitor") + #expect(!overviewExitFramesAreReady(windowIDs: [visible], hiddenWindowIDs: [], + pendingWriteWindowIDs: [], unresolvedWindowIDs: [], targets: [visible: shown], + observed: [visible: clamped], monitorFrames: [monitor])) + } + @Test(arguments: [false, true]) func horizontalMotionPromotesOnlyIntermediateLaneWork(intermediate: Bool) { let observed = Mutex(nil) @@ -1015,6 +1043,139 @@ struct FrameCommitTests { #expect(positions == positions.sorted(by: targetX < 400 ? (>) : (<))) } + @Test + func interruptedRibbonPreservesColumnSpacingAcrossDifferentCompletionTimes() { + let first = WindowID(rawValue: 1), second = WindowID(rawValue: 2) + let coordinator = AXFrameCoordinator() + coordinator.recordCompletedPosition(CGPoint(x: 100, y: 40), windowID: first) + coordinator.recordCompletedPosition(CGPoint(x: 1000, y: 40), windowID: second) + var a = makeMotionWrite(fromX: 200, toX: -300) + var b = makeMotionWrite(fromX: 1000, toX: 500) + a.usesCommonRibbonOffset = true + b.usesCommonRibbonOffset = true + let frame = QueuedPositionFrame( + generation: 2, source: "command-animation", writes: [first: a, second: b], + animatedWindowIDs: [first, second], animationDuration: 0.125, + refreshRateHz: 120, displayIDs: [], + monitorFrames: [Rect(x: 0, y: 0, width: 1512, height: 910)], + initialProgressVelocity: 0, stagesVisibleBeforeParking: false, completion: nil) + let rebased = coordinator.rebaseFrameToCompletedPositionsLocked(frame).frame + let firstStart = rebased.writes[first]!.fromPoint.x + let secondStart = rebased.writes[second]!.fromPoint.x + #expect(secondStart - firstStart == 800) + #expect(firstStart == 100) + for progress in [0.0, 0.25, 0.5, 0.75, 1.0] { + let x = firstStart + (-300 - firstStart) * progress + let y = secondStart + (500 - secondStart) * progress + #expect(y - x == 800) + } + } + + @Test(arguments: [-400.0, 400.0]) + func ribbonWritesReleaseSpaceBeforeMovingNeighbor(delta: Double) { + let left = WindowID(rawValue: 2), right = WindowID(rawValue: 1) + var a = makeMotionWrite(fromX: 100, toX: 100 + delta) + var b = makeMotionWrite(fromX: 900, toX: 900 + delta) + a.usesCommonRibbonOffset = true + b.usesCommonRibbonOffset = true + let batches = AXFrameCoordinator().processWriteBatches( + [left: a, right: b], windowIDs: [left, right]) + #expect(batches.first?.writes.map(\.key) == (delta < 0 ? [left, right] : [right, left])) + } + + @Test + func horizontalRibbonOffsetIgnoresNativeVerticalClamping() { + let id = WindowID(rawValue: 1) + #expect(commonRibbonOffset( + targets: [id: CGPoint(x: -300, y: 41)], + starts: [id: CGPoint(x: 100, y: 37)], + sizes: [id: CGSize(width: 752, height: 902)], + monitor: Rect(x: 0, y: 0, width: 1512, height: 910)) == 400) + } + + @Test(arguments: [(-2000.0, -751.0), (-751.0, -751.0), (-700.0, -700.0), (400.0, 400.0), (3500.0, 1511.0)]) + func logicalRibbonMotionUsesNativeAnchorsOnlyOutsideViewport(x: Double, expected: Double) { + let logical = Rect(x: x, y: 37, width: 752, height: 902) + let native = nativeRibbonAnimationFrame(logical, + monitor: Rect(x: 0, y: 0, width: 1512, height: 910)) + #expect(native.x == expected) + #expect(native.y == logical.y) + #expect(native.width == logical.width) + #expect(native.height == logical.height) + } + + @Test(arguments: [-1.0, 1.0]) + func distantRibbonTraversalIncludesColumnsParkedAtBothEnds(direction: Double) { + let monitor = Rect(x: 0, y: 0, width: 1512, height: 910) + let start = Rect(x: direction < 0 ? 2400 : -1600, y: 37, width: 752, height: 902) + let target = Rect(x: direction < 0 ? -1600 : 2400, y: 37, width: 752, height: 902) + // The native reference can remain at exactly the same parking anchor, + // so no write intent or startPositions entry exists for this column. + let anchor = Rect(x: 1511, y: 37, width: 752, height: 902) + #expect(ribbonAnimationStart(logical: target, offset: start.x - target.x, + observed: anchor, monitorFrames: [monitor]) == start) + #expect(requiresVerifiedOffscreenWrite(frame: start, monitorFrames: [monitor])) + #expect(requiresVerifiedOffscreenWrite(frame: target, monitorFrames: [monitor])) + #expect(ribbonAnimationTarget(logical: target, from: start, + isParked: true, monitorFrames: [monitor]) == target) + #expect(ribbonAnimationTarget(logical: nil, from: start, + isParked: true, monitorFrames: [monitor]) == nil) + #expect(ribbonAnimationTarget(logical: target, from: target, + isParked: true, monitorFrames: [monitor]) == nil) + #expect(ribbonAnimationTarget(logical: target, from: start, + isParked: true, monitorFrames: [monitor, monitor]) == nil) + } + + @Test(arguments: ["command-animation", "workspace-transition"]) + func staticRibbonParkingPrecedesNextHorizontalMovement(source: String) { + let orphan = WindowID(rawValue: 1), moving = WindowID(rawValue: 2) + let base = makeMotionWrite(fromX: -369, toX: -751) + let parked = AsyncPositionWrite( + element: base.element, application: base.application, processID: base.processID, + fromPoint: base.fromPoint, point: base.point, fromSize: base.fromSize, size: base.size, + positionChanged: true, sizeChanged: false, animatesSize: false, + synchronousSizeWriteSucceeded: true, enhancedUIWasEnabled: false, + timeoutSeconds: 0.016, isParked: true, isReentering: false, + requiresVerifiedOffscreenWrite: true) + let frame = QueuedPositionFrame( + generation: 2, source: source, + writes: [orphan: parked, moving: makeMotionWrite(fromX: 387, toX: -369)], + animatedWindowIDs: [moving], animationDuration: 0.125, + refreshRateHz: 120, displayIDs: [], + monitorFrames: [Rect(x: 0, y: 0, width: 1512, height: 910)], + initialProgressVelocity: 0, stagesVisibleBeforeParking: false, completion: nil) + #expect(ribbonParkingPreparationWindowIDs(frame) == + (source == "command-animation" ? [orphan] : [])) + } + + @Test + func interruptedRibbonDoesNotMistakeUncommittedParkingTargetForAppliedPosition() { + let target = Rect(x: -1505, y: 37, width: 1506, height: 902) + let actual = CGPoint(x: -231, y: 37) + let reference = frameApplicationReference( + pendingCorrection: nil, settlingReference: nil, completedPosition: actual, + previousTarget: target, prefersCompletedPosition: true, nativeReference: nil) + #expect(reference?.x == Double(actual.x)) + #expect(frameWriteIntent(reference: reference!, target: target, positionsOnly: true).position) + #expect(reference?.width == target.width) + } + + @Test + func ribbonAvoidsRepeatedVerticalWritesForNativeHeightClamping() { + let native = Rect(x: -1505, y: 37, width: 1506, height: 902) + let target = Rect(x: -1505, y: 41, width: 1506, height: 869) + #expect(!frameWriteIntent(reference: native, target: target, + positionsOnly: true, horizontalOnly: true).position) + #expect(frameWriteIntent(reference: native, target: target, + positionsOnly: true).position) + let start = CGPoint(x: -231, y: native.y) + let destination = CGPoint(x: target.x, y: target.y) + #expect(frameWritePosition(target: destination, from: start, horizontalOnly: true) + == CGPoint(x: target.x, y: native.y)) + #expect(frameWritePosition(target: destination, from: start, horizontalOnly: false) + == destination) + } + @Test(arguments: [-300.0, 300.0]) func interruptedReentryKeepsItsGapWithTheRebasedNeighbor(offset: Double) { let neighbor = WindowID(rawValue: 1), entering = WindowID(rawValue: 2) @@ -1087,6 +1248,7 @@ struct FrameCommitTests { private func makeMotionWrite( fromX: Double, toX: Double, sizeChanged: Bool = false, processID: pid_t = 42, + animatesSize: Bool = false, isReentering: Bool = false, isParked: Bool = false, toY: Double = 40 ) -> AsyncPositionWrite { // Handles only: these tests never read or mutate the real desktop. @@ -1095,7 +1257,7 @@ struct FrameCommitTests { element: element, application: element, processID: processID, fromPoint: CGPoint(x: fromX, y: 40), point: CGPoint(x: toX, y: toY), fromSize: CGSize(width: 800, height: 700), size: CGSize(width: 900, height: 700), - positionChanged: true, sizeChanged: sizeChanged, animatesSize: false, + positionChanged: true, sizeChanged: sizeChanged, animatesSize: animatesSize, synchronousSizeWriteSucceeded: !sizeChanged, enhancedUIWasEnabled: false, timeoutSeconds: 0.016, isParked: isParked, isReentering: isReentering, requiresVerifiedOffscreenWrite: false @@ -1209,6 +1371,78 @@ struct FrameCommitTests { #expect(coordinator.pending?.animationDuration == 0.035) } + @Test + func `Horizontal ribbon retains common motion after a recent AX stall`() { + let coordinator = AXFrameCoordinator() + coordinator.running = true + let first = WindowID(rawValue: 1), second = WindowID(rawValue: 2) + coordinator.recordProcessLatencySamples([42: 120]) + coordinator.predictedProcessLatencyMS[43] = 2 + let writes = [ + first: makeMotionWrite(fromX: 900, toX: 100, processID: 42), + second: makeMotionWrite(fromX: 1800, toX: 1000, processID: 43), + ] + coordinator.submit(writes, source: "command-animation", animationDuration: 0.125, + refreshRateHz: 120, animatedWindowIDs: [first, second]) + let duration = coordinator.pending?.animationDuration ?? 0 + #expect(duration == 0.125) + + coordinator.recentIntermediateProcessLatencySamplesMS = [:] + coordinator.predictedProcessLatencyMS = [42: 2, 43: 2] + coordinator.submit(writes, source: "command-animation", animationDuration: 0.125, + refreshRateHz: 120, animatedWindowIDs: [first, second]) + #expect(coordinator.pending?.animationDuration == 0.125) + } + + @Test + func managedWidthAnimationRetainsIntermediateFramesAfterAnAXStall() { + let coordinator = AXFrameCoordinator() + coordinator.running = true + let selected = WindowID(rawValue: 1), neighbor = WindowID(rawValue: 2) + coordinator.recordProcessLatencySamples([42: 70, 43: 2]) + let resizing = makeMotionWrite(fromX: 500, toX: 0, sizeChanged: true, animatesSize: true) + let writes = [selected: resizing, + neighbor: makeMotionWrite(fromX: 1300, toX: 900, processID: 43)] + coordinator.submit(writes, source: "command-layout-animation", animationDuration: 0.125, + refreshRateHz: 120, monitorFrames: [Rect(x: 0, y: 0, width: 1512, height: 982)], + animatedWindowIDs: [selected, neighbor]) + #expect(coordinator.pending?.animationDuration == 0.125) + let duration = coordinator.horizontalAnimationDuration( + for: writes, requested: 0.125, refreshRateHz: 120, allowsSizeChanges: true) + #expect(duration >= 0.21 && duration <= 0.4) + #expect(coordinator.animationSupportsIntermediateFrames( + processIDs: [42, 43], animationDuration: duration, refreshRateHz: 120)) + #expect(coordinator.pending?.writes[selected]?.animatesSize == true) + coordinator.predictedProcessLatencyMS = [42: 2, 43: 2] + coordinator.recentIntermediateProcessLatencySamplesMS = [:] + coordinator.submit(writes, source: "command-layout-animation", animationDuration: 0.125, + refreshRateHz: 120, monitorFrames: [Rect(x: 0, y: 0, width: 1512, height: 982), + Rect(x: 1512, y: 0, width: 1512, height: 982)], + animatedWindowIDs: [selected, neighbor]) + #expect(coordinator.pending?.animationDuration == 0.125, + "Cross-display layout defers its safety decision until worker admission") + } + + @Test + func `Leaving ribbon window follows strip motion before final parking`() { + var leaving = makeMotionWrite(fromX: 100, toX: -759) + leaving.animationPoint = CGPoint(x: -900, y: leaving.point.y) + let middle = frameAnimationDestination(leaving, intermediate: true) + let parked = frameAnimationDestination(leaving, intermediate: false) + #expect(middle.x == -900) + #expect(parked.x == -759) + let neighbor = makeMotionWrite(fromX: 900, toX: -100) + let leavingFrame = interpolatedFrame( + from: Rect(x: 100, y: 0, width: 760, height: 700), + to: Rect(x: middle.x, y: 0, width: 760, height: 700), progress: 0.5) + let neighborFrame = interpolatedFrame( + from: Rect(x: 900, y: 0, width: 760, height: 700), + to: Rect(x: frameAnimationDestination(neighbor, intermediate: true).x, + y: 0, width: 760, height: 700), progress: 0.5) + #expect(neighborFrame.x - leavingFrame.x == 800) + #expect(positionOnlyAnimationWrite(leaving, holding: leaving.size).animationPoint == middle) + } + @Test func `Recent AX stalls prevent animation from restarting after a few fast writes`() { let coordinator = AXFrameCoordinator() @@ -1963,6 +2197,9 @@ struct FrameCommitTests { ) } let windowID = WindowID(rawValue: 1) + var replacement = write(point: CGPoint(x: 100, y: 40), + size: CGSize(width: 800, height: 600), positionChanged: true, sizeChanged: false) + replacement.animationPoint = CGPoint(x: -900, y: 40) let result = frameWritesPreservingSupersededAsyncSizes( active: [ windowID: write( @@ -1974,15 +2211,11 @@ struct FrameCommitTests { ], pending: [:], replacement: [ - windowID: write( - point: CGPoint(x: 100, y: 40), - size: CGSize(width: 800, height: 600), - positionChanged: true, - sizeChanged: false - ) + windowID: replacement ] ) + #expect(result[windowID]?.animationPoint == replacement.animationPoint) #expect(result[windowID]?.point == CGPoint(x: 100, y: 40)) #expect(result[windowID]?.size == CGSize(width: 800, height: 600)) #expect(result[windowID]?.positionChanged == true) @@ -2771,6 +3004,32 @@ struct FrameCommitTests { )) } + @Test func sameDisplayReflowKeepsSizeOnTheMovementTimeline() { + let display = Rect(x: 0, y: 0, width: 1_512, height: 982) + let source = Rect(x: 756, y: 25, width: 756, height: 910) + let target = Rect(x: 0, y: 25, width: 1_512, height: 910) + #expect(!shouldDeferAnimatedSizeUntilMovementCompletes( + from: source, to: target, displayFrames: [display])) + } + + @Test func reorderedParkedColumnStartsAtItsPreviousLogicalSlot() throws { + let monitor = Rect(x: 0, y: 25, width: 1_512, height: 910) + let previous = Rect(x: -1_512, y: 25, width: 756, height: 910) + let target = Rect(x: 756, y: 25, width: 756, height: 910) + let parked = nativeRibbonAnimationFrame(previous, monitor: monitor) + let start = try #require(layoutRibbonAnimationStart(previousLogical: previous, + target: target, observed: parked, monitorFrames: [monitor])) + #expect(start.x == previous.x) + #expect(start.x != parked.x) + let halfway = nativeRibbonAnimationFrame( + interpolatedFrame(from: start, to: target, progress: 0.5), monitor: monitor) + #expect(halfway.x < target.x) + #expect(layoutRibbonAnimationStart(previousLogical: nil, target: target, + observed: parked, monitorFrames: [monitor]) == nil) + #expect(layoutRibbonAnimationStart(previousLogical: previous, target: target, + observed: parked, monitorFrames: [monitor, Rect(x: 1_512, y: 0, width: 1_000, height: 800)]) == nil) + } + @Test func `Final only reentry keeps verified staging write`() { let fast = WindowID(rawValue: 1) @@ -2847,7 +3106,7 @@ struct FrameCommitTests { } @Test - func `Size clamp retry requires a cross-display move`() { + func `Frame centers distinguish same-display and cross-display movement`() { let displays = [ Rect(x: 0, y: 0, width: 1_000, height: 700), Rect(x: 1_000, y: 0, width: 1_000, height: 700), diff --git a/Tests/DefiMacOSTests/HotKeyTests.swift b/Tests/DefiMacOSTests/HotKeyTests.swift index fa76aadd..87c65160 100644 --- a/Tests/DefiMacOSTests/HotKeyTests.swift +++ b/Tests/DefiMacOSTests/HotKeyTests.swift @@ -22,6 +22,10 @@ private final class HotKeyInvocationRecorder: Sendable { @Suite struct OverviewHotKeyTests { + @Test func overviewToggleIsACancelActionWhileOverviewCapturesInput() { + #expect(overviewKeyAction(keyCode: 31, modifierBits: UInt64.max, + configuredCommand: "toggle-overview") == .cancel) + } @Test(arguments: [false, true]) func settingsTargetKeyEventsRespectTextFocus(textFocused: Bool) throws { let key = try Key(accelerator: "alt-left", aliases: [:]) @@ -46,6 +50,32 @@ struct OverviewHotKeyTests { } } + @Test(arguments: [(CGKeyCode(33), "focus-column first"), (CGKeyCode(30), "focus-column last")]) + func overviewCapturesRibbonEndpointBindings(keyCode: CGKeyCode, command: String) { + let hyper = hotKeyModifierBits([.maskAlternate, .maskCommand, .maskControl]) + #expect(overviewKeyAction(keyCode: keyCode, modifierBits: hyper, + configuredCommand: command) != nil, + "Endpoint bindings must not fall through to the native active workspace") + #expect(overviewKeyAction(keyCode: keyCode, modifierBits: hyper, + configuredCommand: command) == (command.hasSuffix("first") ? .firstColumn : .lastColumn)) + #expect(overviewKeyAction(keyCode: 0, modifierBits: hyper, + configuredCommand: command) == (command.hasSuffix("first") ? .firstColumn : .lastColumn)) + } + + @Test func overviewCapturesWorkspaceBindings() { + let hyper = hotKeyModifierBits([.maskAlternate, .maskCommand, .maskControl]) + #expect(overviewKeyAction(keyCode: 126, modifierBits: hyper, + configuredCommand: "focus-workspace up") == .workspaceUp) + #expect(overviewKeyAction(keyCode: 125, modifierBits: hyper, + configuredCommand: "focus-workspace down") == .workspaceDown) + #expect(overviewKeyAction(keyCode: 18, modifierBits: hyper, + configuredCommand: "workspace dev") == .workspace(.named("dev"))) + #expect(overviewKeyAction(keyCode: 19, modifierBits: hyper, + configuredCommand: "focus-workspace-position 2") == .workspace(.position(2))) + #expect(overviewKeyAction(keyCode: 20, modifierBits: hyper, + configuredCommand: "focus-workspace-name dev-secondary") == .workspace(.named("dev-secondary"))) + } + @Test(arguments: [false, true]) func interceptedModalKeysStillUpdateUserActivity(overview: Bool) throws { let tracker = UserInputTracker() @@ -66,7 +96,7 @@ struct OverviewHotKeyTests { } @Test - func `Overview captures navigation arrows but leaves move bindings active`() { + func `Overview routes navigation and local layout edits`() { let hyper = hotKeyModifierBits([ .maskAlternate, .maskCommand, @@ -87,7 +117,7 @@ struct OverviewHotKeyTests { keyCode: 123, modifierBits: hyper | hotKeyModifierBits([.maskShift]), configuredCommand: "move-column left" - ) == nil + ) == .layout(.moveColumn(.left)) ) #expect( overviewKeyAction( diff --git a/Tests/DefiMacOSTests/OverviewNavigationTests.swift b/Tests/DefiMacOSTests/OverviewNavigationTests.swift index 13c9d68a..afefa8ed 100644 --- a/Tests/DefiMacOSTests/OverviewNavigationTests.swift +++ b/Tests/DefiMacOSTests/OverviewNavigationTests.swift @@ -7,8 +7,160 @@ import Testing @MainActor struct OverviewNavigationTests { + @Test(arguments: [false, true]) + func cancellationNeverCommitsDeferredSelection(pendingSelection: Bool) { + let monitor = MonitorID(rawValue: .max) + let ids = [WindowID(rawValue: 1), WindowID(rawValue: 2)] + let workspace = Workspace(id: WorkspaceID(rawValue: "test"), columns: + ids.map { Column(window: $0, width: .fraction(0.5)) }) + var focuses: [WindowID] = [], drops = 0, edits = 0 + let controller = OverviewController(focusWindow: { id, _, _, _ in focuses.append(id) }, + focusWorkspace: { _, _ in }, drop: { _, _, _, _, _ in drops += 1 }, activateMonitor: { _ in }, + openStateChanged: { _ in }, waitsForNativeSelectionCommit: true, + notificationCenter: NotificationCenter(), layoutCommand: { _, _, _, _, _, _ in edits += 1 }, + commitScrollOffsets: { _ in }) + controller.open(snapshot: OverviewSnapshot(monitors: [Monitor(id: monitor, + workspaces: [workspace], activeWorkspace: workspace.id)], monitorFrames: [:], + windows: Dictionary(uniqueKeysWithValues: ids.map { id in + (id, Window(id: id, appID: "test", title: "Window", frame: Rect(x: 0, y: 0, width: 100, height: 100))) + })), layout: LayoutSettings()) + controller.handleKey(.right) + #expect(focuses.isEmpty) + if pendingSelection { + controller.handleKey(.select) + #expect(focuses == [ids[1]]) + controller.handleKey(.moveDown) + #expect(!controller.applyLayoutCommand(.maximizeColumn)) + #expect(drops == 0 && edits == 0) + } + controller.handleKey(.cancel) + #expect(!controller.isOpen) + #expect(focuses == (pendingSelection ? [ids[1]] : [])) + controller.selectionCommitCompleted() + #expect(!controller.isOpen) + } + + @Test(arguments: [WorkspaceTarget.named("b"), .position(2)]) + func directWorkspaceBindingSelectsItsFocusedWindow(target: WorkspaceTarget) { + let monitor = MonitorID(rawValue: .max) + let ids = (1...3).map { WindowID(rawValue: UInt64($0)) } + let a = Workspace(id: WorkspaceID(rawValue: "a"), columns: [Column(window: ids[0], width: .fraction(0.5))]) + let b = Workspace(id: WorkspaceID(rawValue: "b"), columns: [ + Column(window: ids[1], width: .fraction(0.5)), Column(window: ids[2], width: .fraction(0.5))], focusedColumn: 1) + var focused: [WindowID] = [] + let controller = OverviewController(focusWindow: { id, _, _, _ in focused.append(id) }, + focusWorkspace: { _, _ in }, drop: { _, _, _, _, _ in }, activateMonitor: { _ in }, + openStateChanged: { _ in }, notificationCenter: NotificationCenter(), commitScrollOffsets: { _ in }) + controller.open(snapshot: OverviewSnapshot(monitors: [Monitor(id: monitor, workspaces: [a, b], activeWorkspace: a.id)], + monitorFrames: [:], windows: Dictionary(uniqueKeysWithValues: ids.map { + ($0, Window(id: $0, appID: "test", title: "Window", frame: Rect(x: 0, y: 0, width: 100, height: 100))) + })), layout: LayoutSettings()) + defer { controller.close() } + controller.handleKey(.workspace(target)) + #expect(focused.isEmpty) + #expect(controller.isOpen) + controller.handleKey(.select) + #expect(focused == [ids[2]]) + } + @Test func layoutCommandUsesDeferredSelection() { + let monitor = MonitorID(rawValue: .max) + let ids = [WindowID(rawValue: 1), WindowID(rawValue: 2)] + let workspaces = ids.enumerated().map { index, id in + Workspace(id: WorkspaceID(rawValue: String(index)), columns: [Column(window: id, width: .fraction(0.5))]) + } + var edits: [WindowID] = [], nativeFocus: [WindowID] = [] + let controller = OverviewController(focusWindow: { id, _, _, _ in nativeFocus.append(id) }, + focusWorkspace: { _, _ in }, drop: { _, _, _, _, _ in }, activateMonitor: { _ in }, + openStateChanged: { _ in }, notificationCenter: NotificationCenter(), + layoutCommand: { command, id, app, display, workspace, _ in + #expect(command == .maximizeColumn) + #expect(app == "test") + #expect(display == monitor) + #expect(workspace == workspaces[1].id) + edits.append(id) + }, commitScrollOffsets: { _ in }) + controller.open(snapshot: OverviewSnapshot(monitors: [Monitor(id: monitor, + workspaces: workspaces, activeWorkspace: workspaces[0].id)], monitorFrames: [:], + windows: Dictionary(uniqueKeysWithValues: ids.map { id in + (id, Window(id: id, appID: "test", title: "Window", frame: Rect(x: 0, y: 0, width: 100, height: 100))) + })), layout: LayoutSettings(), windowPreviewsEnabled: false) + defer { controller.close(); controller.close() } + controller.handleKey(.workspaceDown) + controller.handleKey(.layout(.maximizeColumn)) + #expect(edits == [ids[1]]) + var ipcSelections: [WindowID] = [] + #expect(controller.applyLayoutCommand(.maximizeColumn) { _, id, _, _, _, _ in + ipcSelections.append(id) + }) + #expect(ipcSelections == [ids[1]]) + #expect(edits == [ids[1]], "IPC must use its execution handler without duplicating the keyboard mutation") + controller.close() + #expect(!controller.applyLayoutCommand(.maximizeColumn) { _, id, _, _, _, _ in + ipcSelections.append(id) + }) + #expect(ipcSelections == [ids[1]], "A closed controller cannot enqueue IPC mutation") + #expect(nativeFocus.isEmpty) + } + + @Test(arguments: [OverviewKeyAction.firstColumn, .lastColumn]) + func endpointNavigationUsesLocallySelectedWorkspace(action: OverviewKeyAction) { + let monitor = MonitorID(rawValue: .max) + let ids = (1...5).map { WindowID(rawValue: UInt64($0)) } + let windows = ids.map { Window(id: $0, appID: "test", title: "Window", + frame: Rect(x: 0, y: 0, width: 100, height: 100), processID: 1) } + let a = Workspace(id: WorkspaceID(rawValue: "native"), columns: [Column(window: ids[0], width: .fraction(0.5))]) + let b = Workspace(id: WorkspaceID(rawValue: "selected"), columns: [ + Column(windows: [ids[1], ids[2]], focusedWindow: 1, width: .fraction(0.5)), + Column(window: ids[3], width: .fraction(0.5)), Column(window: ids[4], width: .fraction(0.5))], focusedColumn: 1) + var focused: [WindowID] = [] + let controller = OverviewController(focusWindow: { id, _, _, _ in focused.append(id) }, + focusWorkspace: { _, _ in }, drop: { _, _, _, _, _ in }, activateMonitor: { _ in }, + openStateChanged: { _ in }, notificationCenter: NotificationCenter(), + commitScrollOffsets: { _ in }) + controller.open(snapshot: OverviewSnapshot(monitors: + [Monitor(id: monitor, workspaces: [a, b], activeWorkspace: a.id)], + monitorFrames: [:], windows: Dictionary(uniqueKeysWithValues: windows.map { ($0.id, $0) })), + layout: LayoutSettings(), windowPreviewsEnabled: false) + defer { controller.close(); controller.close() } + controller.handleKey(.workspaceDown) + controller.handleKey(action) + // Workspace bindings skip a stacked sibling and return to the remembered column. + controller.handleKey(.workspaceUp) + controller.handleKey(.workspaceDown) + #expect(focused.isEmpty) + controller.handleKey(.select) + #expect(focused == [action == .firstColumn ? ids[2] : ids[4]]) + } + + @Test func returningToWorkspaceKeepsItsLocalSelection() { + let monitor = MonitorID(rawValue: .max) + let ids = (1...3).map { WindowID(rawValue: UInt64($0)) } + let windows = ids.map { Window(id: $0, appID: "test", title: "Window", + frame: Rect(x: 0, y: 0, width: 100, height: 100), processID: 1) } + let a = Workspace(id: WorkspaceID(rawValue: "a"), columns: + ids.prefix(2).map { Column(window: $0, width: .fraction(0.5)) }) + let b = Workspace(id: WorkspaceID(rawValue: "b"), columns: + [Column(window: ids[2], width: .fraction(0.5))]) + var focused: [WindowID] = [] + let controller = OverviewController(focusWindow: { id, _, _, _ in focused.append(id) }, + focusWorkspace: { _, _ in }, drop: { _, _, _, _, _ in }, activateMonitor: { _ in }, + openStateChanged: { _ in }, notificationCenter: NotificationCenter(), + commitScrollOffsets: { _ in }) + controller.open(snapshot: OverviewSnapshot(monitors: + [Monitor(id: monitor, workspaces: [a, b], activeWorkspace: a.id)], + monitorFrames: [:], windows: Dictionary(uniqueKeysWithValues: windows.map { ($0.id, $0) })), + layout: LayoutSettings(), windowPreviewsEnabled: false) + defer { controller.close(); controller.close() } + controller.handleKey(.right) + controller.handleKey(.down) + controller.handleKey(.up) + #expect(focused.isEmpty) + controller.handleKey(.select) + #expect(focused == [ids[1]]) + } + @Test(arguments: [OverviewKeyAction.left, .right, .up, .down]) - func delayedActivationAfterRepeatedArrowsKeepsOverviewOpen(action: OverviewKeyAction) { + func delayedActivationAfterRepeatedArrowsKeepsOverviewOpen(action: OverviewKeyAction) async throws { // A nonexistent display exercises the controller without creating desktop panels. let monitorID = MonitorID(rawValue: .max) let workspaceID = WorkspaceID(rawValue: "test") @@ -38,6 +190,8 @@ struct OverviewNavigationTests { focusWindow: { id, _, _, _ in focused.append(id) }, focusWorkspace: { _, _ in }, drop: { _, _, _, _, _ in }, activateMonitor: { _ in }, openStateChanged: { _ in }, + waitsForNativeSelectionCommit: true, + notificationCenter: NotificationCenter(), commitScrollOffsets: { _ in } ) controller.open( @@ -57,15 +211,21 @@ struct OverviewNavigationTests { let lastWindowID = windows[startsAtEnd ? 0 : 2].id for _ in 0..<20 { controller.handleKey(action) } #expect(controller.isOpen) - #expect(focused.last == lastWindowID) + #expect(focused.isEmpty) NSWorkspace.shared.notificationCenter.post( name: NSWorkspace.didActivateApplicationNotification, object: nil, userInfo: [NSWorkspace.applicationUserInfoKey: NSRunningApplication.current] ) #expect(controller.isOpen) - #expect(focused == [windows[1].id, lastWindowID]) - controller.handleKey(.cancel) + #expect(focused.isEmpty) + controller.handleKey(.select) + #expect(focused == [lastWindowID]) + try await Task.sleep(for: .milliseconds(250)) + #expect(controller.isOpen, "The overview must cover pending native layout writes") + controller.selectionCommitCompleted(sessionGeneration: controller.sessionGeneration &+ 1) + #expect(controller.isOpen, "A different session cannot release the closing handoff") + controller.selectionCommitCompleted() #expect(controller.isOpen == false) } } diff --git a/Tests/DefiMacOSTests/OverviewPreviewTests.swift b/Tests/DefiMacOSTests/OverviewPreviewTests.swift index d600f07b..ae30034c 100644 --- a/Tests/DefiMacOSTests/OverviewPreviewTests.swift +++ b/Tests/DefiMacOSTests/OverviewPreviewTests.swift @@ -4,12 +4,197 @@ import DefiConfig import DefiCore import DefiModel import Testing +import ScreenCaptureKit @testable import DefiMacOS struct OverviewPreviewTests { + @MainActor @Test func liveExperimentChangesWaitForMemoryPressureRecovery() { + let controller = OverviewController(focusWindow: { _, _, _, _ in }, + focusWorkspace: { _, _ in }, drop: { _, _, _, _, _ in }, activateMonitor: { _ in }, + openStateChanged: { _ in }, notificationCenter: NotificationCenter(), commitScrollOffsets: { _ in }) + let snapshot = OverviewSnapshot(monitors: [], monitorFrames: [:], windows: [:]) + controller.open(snapshot: snapshot, layout: LayoutSettings(), windowPreviewsEnabled: false) + defer { controller.close(); controller.prepare(windowPreviewsEnabled: false) } + controller.handleMemoryPressure(.warning) + controller.update(snapshot: snapshot, layout: LayoutSettings(), + experimentalRibbonRepresentations: true) + #expect(controller.surfaceCaptureState == "pressure") + #expect(ExperimentalRibbonRenderer.shared.requests.isEmpty) + controller.handleMemoryPressure(.normal) + controller.update(snapshot: snapshot, layout: LayoutSettings()) + #expect(controller.surfaceCaptureState != "pressure", "Recovery must consume the pending preparation with unchanged settings") + } + + @MainActor @Test func pressureRecoveryRequestsPreloadingWithoutAnotherOpening() { + var preparations = 0 + let controller = OverviewController(focusWindow: { _, _, _, _ in }, + focusWorkspace: { _, _ in }, drop: { _, _, _, _, _ in }, activateMonitor: { _ in }, + openStateChanged: { _ in }, idlePreparationRequested: { preparations += 1 }, + notificationCenter: NotificationCenter(), commitScrollOffsets: { _ in }) + controller.handleMemoryPressure(.normal) + #expect(preparations == 0) + controller.handleMemoryPressure(.warning) + controller.handleMemoryPressure(.critical) + controller.handleMemoryPressure(.normal) + #expect(preparations == 1) + controller.handleMemoryPressure(.normal) + #expect(preparations == 1) + } + + @MainActor @Test func pressureReserveKeepsOnlyPreferredImagesWithinItsBudget() { + let cache = OverviewPreviewCache(maximumBytes: 100) + let ids = (1...3).map { WindowID(rawValue: UInt64($0)) } + let image = NSImage(size: NSSize(width: 10, height: 10)) + for id in ids { + cache.store(image, byteCost: 30, for: Window(id: id, appID: "test", title: "Test", + frame: Rect(x: 0, y: 0, width: 10, height: 10), processID: 42)) + } + cache.retain([ids[1], ids[2]], maximumBytes: 35) + #expect(Set(cache.images.keys) == [ids[1]]) + #expect(cache.byteCount == 30) + cache.removeAll() + #expect(cache.byteCount == 0) + } + @Test func missingOrUnreadyImageDoesNotCancelOtherOpeningSurfaces() { + let first = WindowID(rawValue: 1), missing = WindowID(rawValue: 2) + let warming = WindowID(rawValue: 3), unrelated = WindowID(rawValue: 4) + #expect(overviewReadySurfaceIDs(requested: [first, missing, warming], + captured: [first, warming, unrelated], displayReady: [first, unrelated]) == [first]) + #expect(overviewReadySurfaceIDs(requested: [missing], + captured: [first], displayReady: [first]).isEmpty) + } + @Test + func ribbonRejectsARescaledCaptureOfADifferentNativeSize() { + let native = Rect(x: 2, y: 37, width: 752, height: 902) + // A configured output of 1504 pixels can still contain a screenshot of a + // 1508-point source window. Matching output dimensions does not prove fidelity. + #expect(!overviewSurfaceMatchesNativeSize(source: CGSize(width: 1508, height: 902), native: native)) + #expect(overviewSurfaceMatchesNativeSize(source: CGSize(width: 752, height: 902), native: native)) + #expect(!overviewSurfaceMatchesNativeSize(source: CGSize(width: CGFloat.nan, height: 902), native: native)) + } + @Test + func deniedCaptureStopsAllQueuedAndFutureAttempts() async { + let limiter = OverviewCaptureLimiter(limit: 1) + #expect(await limiter.acquire()) + let queued = Task { await limiter.acquire() } + for _ in 0..<1_000 { + if await limiter.waitingCount == 1 { break } + await Task.yield() + } + #expect(await limiter.waitingCount == 1) + await limiter.release(error: NSError(domain: SCStreamErrorDomain, code: -3801)) + #expect(await queued.value == false) + #expect(await limiter.authorizationDeclined) + #expect(await limiter.acquire() == false) + } + + @Test + func cancelledWaiterFinishesWhileNativeCaptureIsStillPending() async { + let limiter = OverviewCaptureLimiter(limit: 1) + #expect(await limiter.acquire()) + let queued = Task { await limiter.acquire() } + for _ in 0..<1_000 { + if await limiter.waitingCount == 1 { break } + await Task.yield() + } + #expect(await limiter.waitingCount == 1) + queued.cancel() + #expect(await queued.value == false) + await limiter.release() + #expect(await limiter.acquire()) + await limiter.release() + } + + @Test + func ordinaryCaptureFailureDoesNotDisableAuthorization() async { + let limiter = OverviewCaptureLimiter(limit: 1) + #expect(await limiter.acquire()) + await limiter.release(error: NSError(domain: "test.transient", code: 1)) + #expect(await limiter.authorizationDeclined == false) + #expect(await limiter.acquire()) + await limiter.release() + } + @Test + func overlappingBatchesShareCaptureSlotsWithoutRetainingDeliveredImages() async { + let limiter = OverviewCaptureLimiter(limit: 2) + let probe = CaptureProbe() + let delivered = CaptureProbe() + let requests = (1...8).map { OverviewPreviewRequest(windowID: WindowID(rawValue: UInt64($0)), + expectedAppID: "test", width: 100, height: 80, blurFadeHeight: 20) } + let capture: @Sendable (OverviewPreviewRequest) async -> OverviewPreviewCaptureResult = { request in + await limiter.acquire() + await probe.begin() + try? await Task.sleep(for: .milliseconds(5)) + await probe.end() + await limiter.release() + return OverviewPreviewCaptureResult(request: request, image: nil) + } + let complete: @Sendable (OverviewPreviewCaptureResult) async -> Void = { _ in await delivered.begin() } + async let first = runOverviewPreviewCaptures(requests, retainResults: false, + completed: complete, capture: capture) + async let second = runOverviewPreviewCaptures(requests, retainResults: false, + completed: complete, capture: capture) + let results = await (first, second) + #expect(results.0.isEmpty && results.1.isEmpty) + #expect(await probe.maximum == 2) + #expect(await delivered.maximum == requests.count * 2) + } + @Test + func allWorkspacePreviewBatchFitsDecodedMemoryBudget() { + let requests = (1...100).map { OverviewPreviewRequest( + windowID: WindowID(rawValue: UInt64($0)), expectedAppID: "test", + width: 1600, height: 1200, blurFadeHeight: 100) } + let bounded = boundedOverviewPreviewRequests(requests, maximumBytes: 8 * 1024 * 1024) + #expect(bounded.count == requests.count) + #expect(bounded.reduce(0) { $0 + $1.width * $1.height * 4 } <= 8 * 1024 * 1024) + #expect(bounded[0].width < requests[0].width) + #expect(bounded.map(\.windowID) == requests.map(\.windowID)) + #expect(boundedOverviewPreviewRequests(requests, maximumBytes: 0).isEmpty) + } + @MainActor @Test func titleRasterCacheReusesLabelsAndBoundsMemory() throws { + let cache = OverviewTitleCache(maximumBytes: 12_000) + let id = WindowID(rawValue: 1) + let label = try #require(cache.label(for: id, text: "Finder", fontSize: 13, + maximumWidth: 200, scale: 2)) + for _ in 0..<120 { + let repeated = try #require(cache.label(for: id, text: "Finder", fontSize: 13, + maximumWidth: 200, scale: 2)) + #expect(repeated.image === label.image) + } + #expect(cache.rasterizationCount == 1) + let changed = try #require(cache.label(for: id, text: "Downloads", fontSize: 13, + maximumWidth: 200, scale: 2)) + #expect(changed.image !== label.image) + #expect(cache.rasterizationCount == 2) + for value in 2..<50 { + _ = cache.label(for: WindowID(rawValue: UInt64(value)), text: "Downloads", + fontSize: 13, maximumWidth: 200, scale: 2) + #expect(cache.byteCount <= 12_000) + } + cache.prune(windowIDs: []) + #expect(cache.byteCount == 0) + } + + @Test func surfaceCaptureBudgetIncludesProducerLatestAndDisplayedBuffers() { + let request = OverviewSurfaceRequest(windowID: WindowID(rawValue: 1), + appID: "test", processID: 1, width: 2_560, height: 1_600) + #expect(overviewSurfaceRequestsFit([request])) + #expect(overviewSurfaceRequestsFit([request, request])) + #expect(!overviewSurfaceRequestsFit(Array(repeating: request, count: 12))) + #expect(request.estimatedPoolBytes == 2_560 * 1_600 * 6) + } + + @MainActor @Test func disabledSurfaceCaptureDoesNotStartStreams() { + let capture = OverviewSurfaceCapture() + capture.prepare([], enabled: false) + #expect(capture.state == "disabled") + #expect(capture.streamCount == 0) + #expect(capture.estimatedPoolBytes == 0) + } + @Test func previewPixelSizeKeepsCardAspectWhenCapped() { let large = overviewPreviewPixelSize(cardWidth: 2_000, cardHeight: 1_040, scale: 2) - #expect(large.width <= 1_600 && large.height <= 1_200) + #expect(large.width <= 1_024 && large.height <= 768) #expect(abs(Double(large.width) / Double(large.height) - 2_000.0 / 1_040.0) < 0.01) let small = overviewPreviewPixelSize(cardWidth: 100, cardHeight: 60, scale: 2) #expect(small.width == 200 && small.height == 120) diff --git a/Tests/DefiMacOSTests/WindowBorderTests.swift b/Tests/DefiMacOSTests/WindowBorderTests.swift index 3b595322..42cb5831 100644 --- a/Tests/DefiMacOSTests/WindowBorderTests.swift +++ b/Tests/DefiMacOSTests/WindowBorderTests.swift @@ -9,6 +9,32 @@ import Testing @testable import DefiMacOS struct WindowBorderTests { + @Test func overviewReusesConstraintsWithoutBlockingMetadataReads() { + let previous = Window(id: WindowID(rawValue: 1), appID: "test", title: "Window", + frame: Rect(x: 0, y: 0, width: 800, height: 600), + minimumTiledWidth: 400, maximumTiledWidth: 1000, maximumTiledHeight: 900) + var reads = 0 + let fresh = WindowSizeConstraints(minimumWidth: 500, maximumWidth: 1100, maximumHeight: 950) + for _ in 0..<20 { + let reused = windowSizeConstraintsForSnapshot(previousWindow: previous, overviewActive: true) { + reads += 1 + return fresh + } + #expect(reused == WindowSizeConstraints(minimumWidth: 400, maximumWidth: 1000, maximumHeight: 900)) + } + #expect(reads == 0) + #expect(windowSizeConstraintsForSnapshot(previousWindow: nil, overviewActive: true) { + reads += 1 + return fresh + } == nil) + #expect(reads == 0) + #expect(windowSizeConstraintsForSnapshot(previousWindow: previous, overviewActive: false) { + reads += 1 + return fresh + } == fresh) + #expect(reads == 1) + } + @Test func pendingGeometryCoalescesProcessesAndCanScheduleAfterDelivery() { let first = WindowID(rawValue: 1), second = WindowID(rawValue: 2) diff --git a/Tests/DefiModelTests/ModelTests.swift b/Tests/DefiModelTests/ModelTests.swift index aeeaa46d..b6eedbde 100644 --- a/Tests/DefiModelTests/ModelTests.swift +++ b/Tests/DefiModelTests/ModelTests.swift @@ -109,6 +109,22 @@ struct ModelTests { } } + @Test(arguments: [ + (Command.moveColumn(.right), true), + (Command.moveWindow(.down), true), + (Command.cycleWidth(.next), true), + (Command.maximizeColumn, true), + (Command.toggleFloating, true), + (Command.joinWindow(.left), true), + (Command.unjoinWindows, true), + (Command.focusColumn(.right), false), + (Command.toggleOverview, false), + ]) + func selectedLayoutEditsAnimate(testCase: (command: Command, expected: Bool)) { + #expect(testCase.command.editsSelectedLayout == testCase.expected) + #expect(testCase.command.animatesManagedLayout == testCase.expected) + } + @Test(arguments: [ (Command.cycleWidth(.next), true, false), (Command.maximizeColumn, true, false), diff --git a/Tests/DefiRuntimeTests/OverviewRuntimeTests.swift b/Tests/DefiRuntimeTests/OverviewRuntimeTests.swift index 9cfcbcbc..809b1495 100644 --- a/Tests/DefiRuntimeTests/OverviewRuntimeTests.swift +++ b/Tests/DefiRuntimeTests/OverviewRuntimeTests.swift @@ -11,6 +11,72 @@ struct OverviewRuntimeTests { let secondWorkspace = WorkspaceID(rawValue: "2") let remoteWorkspace = WorkspaceID(rawValue: "remote") + @Test func overviewFloatingSelectionCannotEditAnUnrelatedTiledColumn() throws { + var state = makeState() + let id = WindowID(rawValue: 99) + var floating = window(id, monitorID: firstMonitor) + floating.floating = true + state.windows[id] = floating + state.monitors[0].workspaces[0].floatingWindows = [id] + let original = state + #expect(throws: OverviewRuntimeError.invalidDropTarget) { + try applyOverviewLayoutCommand(.cycleWidth(.next), intent: OverviewWindowIntent( + windowID: id, expectedAppID: "app", sourceMonitorID: firstMonitor, + sourceWorkspaceID: firstWorkspace), viewports: [:], state: &state) + } + #expect(state == original) + } + + @Test func overviewWidthEditClearsNativeFullscreenWidthReset() throws { + var state = makeState() + let id = WindowID(rawValue: 99) + state.windows[id] = window(id, monitorID: firstMonitor) + state.monitors[0].workspaces[0].columns = [Column(window: id, width: .fraction(0.5))] + state.pendingNativeFullscreenWidthResetWindowIDs.insert(id) + try applyOverviewLayoutCommand(.cycleWidth(.next), intent: OverviewWindowIntent( + windowID: id, expectedAppID: "app", sourceMonitorID: firstMonitor, + sourceWorkspaceID: firstWorkspace), viewports: [:], state: &state) + #expect(!state.pendingNativeFullscreenWidthResetWindowIDs.contains(id)) + } + + @Test(arguments: [Command.cycleWidth(.next), .maximizeColumn, .moveColumn(.right), .unjoinWindows]) + func layoutEditUsesOverviewSelectionWithoutActivatingItsWorkspace(command: Command) throws { + var state = makeState() + let ids = (11...13).map { WindowID(rawValue: UInt64($0)) } + state.monitors[0].workspaces[1].columns = ids.map { Column(window: $0, width: .fraction(0.5)) } + if command == .unjoinWindows { + state.monitors[0].workspaces[1].columns = [ + Column(windows: [ids[0], ids[1]], focusedWindow: 0, width: .fraction(0.5)), + Column(window: ids[2], width: .fraction(0.5))] + } + for id in ids { state.windows[id] = window(id, monitorID: firstMonitor) } + let originalMonitor = state.monitors[0] + let intent = OverviewWindowIntent(windowID: ids[1], expectedAppID: "app", + sourceMonitorID: firstMonitor, sourceWorkspaceID: secondWorkspace) + try applyOverviewLayoutCommand(command, intent: intent, + viewports: [firstMonitor: Rect(x: 0, y: 0, width: 1_000, height: 800)], state: &state) + #expect(state.monitors[0].activeWorkspace == originalMonitor.activeWorkspace) + #expect(state.monitors[0].workspaces[0] == originalMonitor.workspaces[0]) + #expect(state.monitors[1] == makeState().monitors[1]) + let edited = state.monitors[0].workspaces[1] + if command == .moveColumn(.right) { + #expect(edited.columns.map(\.windows) == [[ids[0]], [ids[2]], [ids[1]]]) + } else if command == .unjoinWindows { + #expect(edited.columns.map(\.windows) == [[ids[0]], [ids[1]], [ids[2]]]) + } else if command == .cycleWidth(.next) || command == .maximizeColumn { + #expect(edited.columns[1].width != originalMonitor.workspaces[1].columns[1].width) + #expect(edited.columns[0].width == originalMonitor.workspaces[1].columns[0].width) + } + let beforeStale = state + #expect(throws: OverviewRuntimeError.staleWindow(ids[1])) { + try applyOverviewLayoutCommand(command, + intent: OverviewWindowIntent(windowID: ids[1], expectedAppID: "other", + sourceMonitorID: firstMonitor, sourceWorkspaceID: secondWorkspace), + viewports: [:], state: &state) + } + #expect(state == beforeStale) + } + @Test func `Commits overview ribbon positions once without touching other workspaces`() { var state = makeState() diff --git a/docs/adr/0003-keep-overview-independent-from-capture.md b/docs/adr/0003-keep-overview-independent-from-capture.md index 41447bce..f7b8427e 100644 --- a/docs/adr/0003-keep-overview-independent-from-capture.md +++ b/docs/adr/0003-keep-overview-independent-from-capture.md @@ -10,6 +10,22 @@ Overview session, and never required for navigation, focus, or window movement. The last valid images may remain in a bounded memory-only cache while fresh captures are pending. They are never persisted. Missing, stale, protected, or denied previews fall back to identifiable cards. -The Overview may animate as one surface, but captured cards do not simulate the -motion of native windows. All committed focus and topology changes still pass +An opt-in `overview.experimental_surface_transitions` path may temporarily animate +cached snapshots from observed native frames into the active Overview ribbon. +Samples may be stale while a refresh is pending or has failed. +It uses public one-shot ScreenCaptureKit samples and Defi-owned Core Animation layers, never +native-window scale/opacity mutations. It has explicit pixel-buffer budgets, +no persistent capture sessions, no disk cache, and no frame history. Opening animates +ready surfaces, using the bounded preloaded preview cache for missing surfaces and +lightweight cards when no image exists yet. These fallback layers animate from +verified native geometry and are released at the end of opening so progressive +previews remain visible. Reverse surface animation is limited to unchanged +native geometry and Overview offsets; selection uses the existing native path. +An additional opt-in `animation.experimental_window_representations` experiment +tests horizontal ribbon continuity using those same bounded one-shot samples. +Defi-owned overlays animate while public Accessibility commits final native +positions; native windows remain the functional fallback. Interrupted transitions +reuse the existing panel and current presentation position. This experiment does +not authorize private compositor or native opacity mutations. +All committed focus and topology changes still pass through DefiRuntime, preserving the macOS authority established by ADR 0001. diff --git a/docs/research/macos-window-surface-transitions.md b/docs/research/macos-window-surface-transitions.md new file mode 100644 index 00000000..0835908e --- /dev/null +++ b/docs/research/macos-window-surface-transitions.md @@ -0,0 +1,158 @@ +# macOS window surface transitions: feasibility research + +**Question.** Can Defi get a near-imperceptible continuous ribbon/overview handoff at low RAM and CPU by animating cached graphics proxies, while ordinary managed windows remain native? + +**Research date:** 2026-10-03. Primary-source research and a bounded Defi prototype; measurements below describe this machine and workload only. + +## Current finding + +The implemented path uses bounded one-shot ScreenCaptureKit snapshots for Overview +and native Accessibility motion for ordinary ribbon navigation. The opt-in ribbon +representation experiment reuses those snapshots; it does not keep persistent +streams. The stream and disk-cache proposals below are earlier hypotheses, +superseded by the prototype correction later in this document. + +## Initial hypothesis + +The strongest low-risk experiment is a public ScreenCaptureKit prototype that captures only the currently relevant windows, retains at most the latest IOSurface per selected window, and displays that surface in a Defi-owned Metal/AppKit proxy during the handoff. Keep the proxy available before motion starts, then hand off to the real window at the endpoint. Do not start a capture stream for every window by default. Apple explicitly describes ScreenCaptureKit output as IOSurface-backed, uses idle status to indicate no new surface, and documents a default queue depth of three with a hard recommendation not to exceed eight. Each held surface reduces the producer's available surface pool; that bounds buffering but does not establish the cost of many simultaneous streams. [Apple's capture sample](https://developer.apple.com/documentation/screencapturekit/capturing-screen-content-in-macos?language=objc), [WWDC22: Meet ScreenCaptureKit](https://developer.apple.com/videos/play/wwdc2022/10156/), [queueDepth](https://developer.apple.com/documentation/screencapturekit/scstreamconfiguration/queuedepth) + +This is a plausible way to make a transition visually continuous, not evidence that it will be imperceptible. The proxy can be rendered and moved by Defi without waiting on the other app's Accessibility lane, but it is a frozen image unless refreshed. A stale image, missing menus/tooltips, capture permission, fullscreen behavior, and replacement timing at the endpoint all need an actual prototype. A fresh image at the exact handoff may itself be too slow. + +## What public APIs establish + +- `SCStream` is Apple's public API for capturing display or window content. A single-window `SCContentFilter` is supported, and Apple's WWDC example explicitly discusses window capture when the source is resized, occluded, moved off-screen, or minimized. This makes a prototype feasible even when a window is not currently unobscured; it does not promise every app/window type works identically. [SCStream](https://developer.apple.com/documentation/screencapturekit/scstream?language=objc), [WWDC22: Take ScreenCaptureKit to the next level](https://developer.apple.com/videos/play/wwdc2022/10155/?time=276), [SCContentFilter including windows](https://developer.apple.com/documentation/screencapturekit/sccontentfilter/init%28display%3Aincluding%3A%29?changes=l_1_1_8&language=objc) +- Apple's sample gets the `CVPixelBuffer` from the `CMSampleBuffer`, then obtains the backing `IOSurface`. The surface can be placed in an `NSView` layer. The WWDC session says a complete status is a new frame and idle means the content did not change, so there is no new IOSurface. These facts support holding the latest surface and reusing it without making a CPU bitmap copy on every animation frame. Apple's published sample uses a layer; Metal texture import/rendering is a reasonable implementation hypothesis, not demonstrated by those cited lines as a complete capture-to-Metal proxy pipeline. [Apple sample](https://developer.apple.com/documentation/screencapturekit/capturing-screen-content-in-macos?language=objc), [WWDC22 session transcript](https://developer.apple.com/videos/play/wwdc2022/10156/) +- The framework is designed for high-performance capture and uses GPU acceleration, but requires the user's Screen Recording consent. Persistent ScreenCaptureKit streams expose system capture-awareness UI. The one-shot experiments use existing access without requesting permission; transient indicator behavior for one-shot captures remains unverified. [ScreenCaptureKit overview](https://developer.apple.com/documentation/ScreenCaptureKit?language=objc), [WWDC23 privacy session](https://developer.apple.com/videos/play/wwdc2023/10053/?time=344) +- `queueDepth` defaults to three; Apple says deeper queues use more memory, and explicitly says not to exceed eight. WWDC describes the surfaces as a server-side pool and says retaining surfaces reduces the remaining pool; if they are not released quickly enough, capture stalls and drops frames. Hence retain no capture history in RAM. A full 3840×2160 BGRA surface is about 31.6 MiB before row alignment; at 2560×1600 it is 15.6 MiB. Three pool surfaces per stream are roughly 95 MiB and 47 MiB respectively, as a raw-size estimate, before compositor/GPU overhead. Multiple streams multiply that order of magnitude. [queueDepth](https://developer.apple.com/documentation/screencapturekit/scstreamconfiguration/queuedepth), [WWDC22: Take ScreenCaptureKit to the next level](https://developer.apple.com/videos/play/wwdc2022/10155/) +- Apple warns that frequently changing stream output dimensions can cause additional allocations and is not recommended. Keep a stream's dimensions stable, or recreate it at deliberate lifecycle boundaries rather than resizing it continuously with the ribbon. [WWDC22 single-window capture discussion](https://developer.apple.com/videos/play/wwdc2022/10155/?time=276) + +## Cost and storage recommendation + +Favor a **disk-backed, compressed thumbnail cache plus a tiny RAM working set** if Defi needs proxies for many windows. Keep the current/next window's newest surface (or one scaled proxy surface) in RAM; release IOSurfaces promptly; store occasional compressed snapshots on disk and decode lazily before navigation. Disk costs capacity and occasional decode latency, while uncompressed full-resolution IOSurfaces cost tens of MiB each. The disk proposal is an engineering inference; Apple does not prescribe a window-manager cache policy. + +For a 60 Hz ribbon, do not encode/decode from disk on the animation path. Warm likely-next proxy surfaces before a navigation gesture if cheap, but bound that speculation to one or two candidates. For a genuinely live proxy, test one persistent stream for the selected/current window first. Only expand to nearby windows after measurement. A 1–5 fps stream may keep snapshots fresh during idle at much lower nominal throughput than 60 fps, but frame rate alone does not prove low CPU, low power, or absence of ScreenCaptureKit/WindowServer problems. + +Avoid a short-lived `captureSampleBuffer` stream for each transition. AltTab maintainers reported that on macOS 26, their repeated one-shot capture path caused WindowServer instability; they described persistent per-window streams as the intended zero-copy option but cautioned that many always-running streams have CPU/power costs. That is first-party project experience, not an Apple guarantee, and applies to the reported versions/path. [AltTab issue #5786](https://github.com/lwouis/alt-tab-macos/issues/5786). A separate AltTab issue reports a 30 fps refresh prototype using private `CGSHWCaptureWindowList` was already laggy and failed to reach 60 fps, and investigated a private `SLSHWCaptureStreamCreateWithWindow`; it does not prove that private stream API is usable or stable. [AltTab issue #122](https://github.com/lwouis/alt-tab-macos/issues/122) + +## Private SkyLight/CGS window mutation and ownership + +Yabai's public source declares private `SLSSetWindowTransform` and related functions. A third-party CGS header documents `CGSSetWindowTransform` and claims strong transform restrictions (including no upscaling past the original frame). These are evidence that private symbols and experiments exist, not a supported contract or proof that a third-party process can apply reliable transforms to arbitrary app windows on current macOS. [yabai extern.h](https://github.com/asmvik/yabai/blob/master/src/misc/extern.h), [CGSInternal header](https://github.com/NUIKit/CGSInternal/blob/master/CGSWindow.h) + +Yabai documents a separate scripting-addition path: it injects into Dock, which it says owns the sole connection to WindowServer; some advanced operations need SIP partially disabled. Its docs list particular features such as window opacity, layers, animations, and Space operations as requiring that addition. They do **not** establish that every private SkyLight transform call requires disabled SIP, nor that SIP-off grants safe or supported arbitrary proxy transforms. The precise conclusion is: private APIs may be callable in some cases with SIP enabled, but the ability to mutate arbitrary third-party windows reliably without Dock injection is not established by these sources. [yabai SIP documentation](https://github.com/asmvik/yabai/wiki/Disabling-System-Integrity-Protection/0b75fef379c579096b2a51d6a0ddeb92ebe12130), [yabai command documentation](https://github.com/asmvik/yabai/blob/master/doc/yabai.asciidoc) + +Defi's own policy requires proof public APIs cannot meet a user-visible invariant before private mutation, confines it to `DefiMacOS`, and specifically approves third-party position/size mutation only for the opt-in experimental frame backend. A ribbon animation that privately scales/translates ordinary user windows would cross that policy boundary and is not the recommended path. A Defi-owned proxy window is different: Defi controls its own layer/window, and can animate it with public AppKit/Metal while the native window stays untouched. However, overlay ordering, focus, clicks, Spaces, and capture feedback still need validation. + +### Specific question: `CGSHWCaptureWindowList` + +There is credible implementation evidence that a normal app can resolve and call private `CGSHWCaptureWindowList` without injecting into Dock: the cmdcmd maintainer describes the symbol as `dlsym`-reachable from a normal process, documents the signature as taking a window-ID pointer plus count, and states its implementation has moved to it. The call shape accepts a list/count, so batching several windows in **one call** is a supported-by-observation experiment; the result is an array of `CGImage`s. It is not an IOSurface-returning API and its private ABI/semantics are not guaranteed. [cmdcmd issue #21](https://github.com/peterp/cmdcmd/issues/21), [cmdcmd v0.4.0 release](https://github.com/peterp/cmdcmd/releases) + +The cmdcmd project reports using it for current macOS live tile previews and reports that Screen Recording permission remains required and the app remains attributed in the menu bar. That is useful evidence of a non-Dock-injected route on the author's tested setup, but **not** a guarantee for all macOS releases, OS configurations, windows, or Defi's bundle/entitlements. It also says the batch call is polled, and notes a WindowServer memory leak above roughly 30 fps. That leak threshold is an author-reported observation, not an independently reproduced or Apple-documented limit. AltTab's older 30 fps prototype likewise missed 60 fps, so this path should be measured rather than assumed fast. [cmdcmd issue #21](https://github.com/peterp/cmdcmd/issues/21), [AltTab issue #122](https://github.com/lwouis/alt-tab-macos/issues/122) + +**Recommendation for Defi:** a one-shot-at-ribbon-open `CGSHWCaptureWindowList` batch is now the cheapest high-value *private* spike, if the team elects to evaluate read-only capture separately from the default product path. It avoids N stream startup and can produce an image for each window before animation. However, this captures pixel content rather than optional metadata, so Defi's private-API exception for read-only metadata does not automatically authorize it; the experimental-frame-backend opt-in exception names third-party position/size mutation, not screenshot capture. Do not silently ship this under either exception. The public control is a one-shot ScreenCaptureKit `SCScreenshotManager` capture at ribbon-open, with an explicit Screen Recording permission/failure path; it returns `CGImage`, not the capture stream's IOSurface. A single full-display capture can provide one system request, but only reflects the composited visible screen, so occluded/off-screen windows cannot be faithfully recovered by cropping it. Capturing individual windows uses a per-window filter and separate requests. [SCScreenshotManager](https://developer.apple.com/documentation/screencapturekit/scscreenshotmanager?changes=la_4_5_9&language=objc), [single-frame capture API](https://developer.apple.com/documentation/ScreenCaptureKit/SCScreenshotManager/captureImage%28contentFilter%3Aconfiguration%3AcompletionHandler%3A%29) + +## Visual continuity target + +The target is a continuous horizontal ribbon and an overview zoom-out in which content, title bars, clipping, and window geometry move together. The handoff between a native window and its graphical representation should preserve apparent position, scale, sharpness, stacking, and content freshness. A convincing graphics transform alone does not establish low capture latency or low memory cost; those properties require separate measurement. Captured proxies and native compositor transforms are alternative mechanisms to investigate, with the public-API and private-API constraints described above. + +## Defi prototype result + +The first opt-in `overview.experimental_surface_transitions` variant kept public per-window ScreenCaptureKit streams warm for the active workspace, froze their latest frames once into CGImages, and animated Defi-owned Core Animation layers between verified native bounds and overview bounds. It preserves native title bars and does not mutate native geometry for the animation. It is an overview handoff experiment, not a replacement for normal ribbon navigation. Disk caching was deliberately excluded from the handoff; compressed old thumbnails would not establish freshness at the native endpoint. + +Local verification and the focused native desktop test passed with Accessibility and Screen Recording available. The test covers fresh capture, panel recreation, rapid close/reopen cancellation, unchanged native geometry, and teardown. Computer Use inspected the installed overview and a subsequent ordinary click in the native TextEdit window. Still screenshots do not establish frame-by-frame imperceptibility. + +On the owned 640×440 fixture, the first run froze images in approximately 21 ms initially and 2 ms subsequently. On the installed full-height Retina TextEdit window, two opening freezes took 39.7 and 75.8 ms; estimated producer/retained buffers totaled 79.36 MiB, excluding frozen images and framework/compositor overhead. Both installed openings used the surface transition with zero fallbacks. A panel-recreation bug originally discarded warmed captures; the lifecycle fix and regression test preserve them independently of panel recreation. + +After that fix, the focused native fixture test measured 9.5 and 4.5 ms. A short paired idle sample on the installed TextEdit scene reported daemon CPU at 0% with capture disabled and 1.6–2.3% enabled. Both samples reported approximately 310 MiB process footprint after earlier captures, so this does not isolate incremental RAM: retained allocations, compression, and capture surfaces outside the daemon require a longer clean-process comparison. WindowServer/replayd CPU and energy were not isolated. Verification artifacts are under ignored `dist/verification/20261003-122542-517d199e/` (local/stage), `20261003-122731-08fb8265/` (focused desktop), and `surface-computer-use/` (installed trial). Original configuration, all monitor workspaces, focus, widths, scroll, and managed frames were restored; exactly one daemon remained. + +The user's ordinary multi-window workspace exceeded the 96 MiB buffer estimate and correctly used the ordinary overview fallback. Consequently this experiment does **not** yet meet the combined near-imperceptibility and low-resource requirements for the full ribbon. Shadows, live content changes during a frozen animation, cold starts, multiple displays, and total WindowServer/replayd cost remain unqualified. Small-window results must not be extrapolated to Retina workspaces. + +The follow-up variant renders the retained IOSurface directly, avoiding the synchronous CGImage conversion measured above. It updates layer content at a bounded 30 Hz cadence while the overview projection is unchanged, retains no frame history, falls back to ordinary previews when that projection changes, and estimates four buffers per window (producer, callback, and display overlap) within the same 96 MiB budget. Idle frames reuse unchanged content; stream errors invalidate it. The optional controlled native test uses an owned opaque backdrop and an animated fixture. An external native display recording captures the real scene; recording/encoding adds overhead. An SCRecordingOutput attempt produced incomplete files on this setup, so it was not retained as evidence or a test dependency. Measurements of this variant must be distinguished from the first variant above. [Apple capture sample](https://developer.apple.com/documentation/screencapturekit/capturing-screen-content-in-macos), [Apple recording API](https://developer.apple.com/documentation/screencapturekit/screcordingoutput). That addresses conversion latency and copies, but does not eliminate ScreenCaptureKit's producer pools. Compare a single display capture for the visible opening scene (occluded/offscreen content requires a separate path), and a bounded lower-resolution cache for inactive cards. Private batched capture and native compositor transforms remain hypotheses, with the policy and reliability constraints described above. + +## Direct-surface validation and remaining visual defect + +The final full-height 640-point Retina fixture passed local verification and the focused desktop test with Accessibility and Screen Recording available. The capture reference acquisition measured 0.008–0.023 ms; this is not total opening latency. The four-buffer estimate was 54,726,144 bytes (52.19 MiB). The test verifies changing surface submissions, unchanged native geometry, panel recreation, rapid reopen, and teardown; it does not verify pixel correctness of those submissions. + +Installed Computer Use inspected a real TextEdit overview surface and verified an ordinary native text-area click after closing it. Its buffer estimate was 69.63 MiB. Three short idle samples reported 0%, 1.4%, and 2.2% daemon CPU enabled, versus 0% disabled; both process footprints were approximately 280 MiB after capture, so incremental memory is not isolated. WindowServer/replayd and energy remain unmeasured. + +The real 60 fps screen recording of the animated owned fixture exposes a defect: the first overview opening shows background/empty content, while the second displays the animated window. Its cause remains unresolved; live buffer submission counters alone cannot establish correct pixels. The clip retains both openings, trimming only unrelated desktop lead-in/out. Therefore this prototype is not ready to claim an imperceptible native/proxy handoff or replace normal ribbon scrolling. It stays disabled by default. Disk storage can reduce inactive thumbnail cache residency, but cannot remove decoded display surfaces or live producer buffers. + +Final evidence: `dist/verification/20261003-131439-190206be/` (local/stage), `dist/verification/20261003-131609-b35d1fe2/` (focused desktop), `dist/verification/live-surface-computer-use/` (installed trial), and `dist/verification/live-surface-evidence/defi-live-surface-demo.mp4` (6.8 seconds, 408 frames). The original configuration and all monitor workspaces, logical focus, widths, scroll, and managed-frame convergence were restored after the trial. + +## Current multi-window workspace recording + +A follow-up recording used the user’s existing `dev-secondary` workspace with nine Ghostty windows, without changing their membership or widths. The experimental build reported `overviewSurface=budget`, zero streams, zero experimental transitions, and one fallback while the overview was open. The full-resolution per-window strategy therefore fails the resource constraint before it can demonstrate the intended dezoom on this ordinary workspace. The 60 fps recording at `dist/verification/current-windows-video/real-windows-experiment.mp4` contains two real openings/closings, but they use ordinary overview rendering and must not be presented as evidence of proxy-animation quality. + +The target includes normal ribbon navigation as well as overview. This spike does not implement ribbon proxy handoffs. A next prototype must bound capture residency by visible windows and immediate neighbours, independently of total workspace size, and validate exact native/proxy alignment, fresh content, entering/leaving clipping, asynchronous native settling, latest-wins cancellation, and native focus/click handoff. Display capture of the visible scene, selective per-window surfaces, and compressed inactive thumbnails require a measured comparison; disk-backed thumbnails alone do not provide fresh native endpoints. No claim of imperceptibility follows from the current implementation. + +The initial application bundle and configuration were restored, followed by verified workspace/focus/width/scroll/frame restoration. The evidence is under ignored `dist/verification/current-windows-video/`. Computer Use cannot directly inspect Ghostty in this environment because access to that application is blocked; the user-authorized native screen recording and Defi status provide the visual and runtime evidence here. + +## Cheapest public experiment + +1. Implement a disposable prototype path behind a feature flag: request one public ScreenCaptureKit single-window screenshot at ribbon-open, render its `CGImage` as a Defi-owned proxy, and animate that proxy. This tests the handoff with one native window and one capture, without stream setup. For many simultaneously visible windows, compare one full-display screenshot plus crops (cheap request count, but occlusion fidelity is lost) with per-window screenshots. If public one-shot latency is insufficient, switch the prototype to a single-window stream at stable reduced resolution/modest update rate and hold only its latest IOSurface. Separately compare the private batch capture spike described above only if private pixel capture is explicitly in scope. +2. Render the surface in a Defi-owned overlay/proxy and animate that proxy through a short ribbon movement while the native window remains at its original geometry. At the destination, hide the proxy and expose/focus the real window. Start with a stationary app and a typing/scrolling app to reveal stale-frame and cursor/menu differences. +3. Measure cold-start and warm-start handoff latency, frame-time misses, Defi/WindowServer/replayd CPU, power, and memory with 1, 3, then 10 windows. Compare one persistent stream against an occasional screenshot refresh. Test occluded and minimized states separately. Do not infer whole-ribbon scalability from a single window. +4. If the prototype passes, decide whether one or two recently used proxies should stay warm in RAM, while the broader snapshot cache stays compressed on disk. If it fails due to capture latency or system cost, keep the ribbon native-window-only and investigate a shorter staged transition rather than adding unbounded capture streams. + +This experiment is the cheapest because it tests the pivotal assumption—whether a Defi-owned proxy can bridge the animation and focus handoff—without private window mutation, an always-on stream per application window, or a long-lived cache architecture first. + + +## Shared live-video ribbon and overview trial (latest implementation) + +This implementation supersedes the overview-only experiments above. The opt-in path shares a bounded ScreenCaptureKit cache between normal horizontal ribbon navigation and Overview. Visible windows and at most one neighbour on each side are captured at native resolution. The current NV12 format preserves luma resolution and reduces the estimated pool size relative to BGRA, at the cost of chroma subsampling. A prewarmed AVSampleBufferDisplayLayer displays each uncompressed buffer directly; Defi performs no video encoding or per-frame CPU bitmap conversion. Readiness is checked before presentation. This public renderer supports uncompressed frames and exposes readiness for the first displayed frame. [Apple video layer documentation](https://developer.apple.com/documentation/avfoundation/avsamplebufferdisplaylayer), [Apple sample buffer renderer](https://developer.apple.com/documentation/avfoundation/avsamplebufferdisplaylayer/samplebufferrenderer). + +Ribbon proxies animate together in a workarea-sized panel. The normal Accessibility coordinator commits final native positions underneath, and the panel is removed only after public native bounds and owner PIDs converge. Supersession, mouse interaction, fullscreen and display changes cancel the scene. Native focus and workspace membership remain on the existing runtime path. Cached layers reset their transforms before reuse; configured borders follow the proxies. The wallpaper is prepared before opening. Inactive cards use one-shot compact captures in the existing 32 MiB cache, without additional streams. Changed Overview projections still return to ordinary preview rendering. + +Local build, non-desktop tests and workflow tests passed under `dist/verification/20261003-143355-b1404642/`. The focused native test `DesktopE2ETests/testOverviewSurfaceTransitionUsesFreshOwnedWindowAndReopensSafely` passed under `20261003-143607-27a8cfd7/`, with Accessibility and Screen Recording available. It covers animated owned-window content, ribbon/native convergence, Overview reopening, unchanged native Overview geometry, bounded capture residency and teardown. The full desktop suite was not run; desktop coverage is this focused test plus installed Computer Use and the real-window recording. Documentation was updated after these code checks. + +The installed trial used the user's existing nine-window Ghostty workspace and the existing Finder/T3/Xcode workspace. The former retained four streams with a 160.31 MiB four-buffer estimate; the latter retained three streams with a 120.23 MiB estimate. The compact preview cache reported 11.22 MiB when opened. Across the recorded trial, status reported seven GPU ribbon transitions with one fallback, and five GPU Overview openings with one fallback. These are aggregate counters across recording attempts, not a count of every motion in one clip. Geometry/freshness checks intentionally permit fallback; no universal imperceptibility follows from successful counters. The real video is `dist/verification/ribbon-evidence/real-ribbon-overview.mp4` (28 seconds, 1920×1080, 60 fps), with the native Finder/T3/Xcode ribbon sequence in its second half. It includes Overview navigation and both opening/closing scenes; it is not a synthetic window fixture. Status, trace and a frame contact sheet are alongside it. One initial command sequence was corrupted by terminal input buffering; the subsequent recording used a script file. Direct Computer Use access to Ghostty remains blocked, so its visual evidence is the authorized native recording. Computer Use exercised the installed Defi panel and a normal native Finder click after closing it. + +A short paired sample taken without recording/encoding found daemon CPU approximately 0–0.1% disabled versus 9.3–14.2% enabled, WindowServer 19.9–21.3% versus 55.2–57.3%, and replayd 0% versus 3.4–4.7%. These are percentages of one CPU core and short workload-specific samples, not an energy benchmark. Reported daemon footprint was about 24 versus 30 MiB, while WindowServer was about 1961 versus 2139–2168 MiB; externally hosted capture buffers make daemon footprint alone misleading. Other compositor activity and startup allocation prevent attributing the entire difference to Defi. The pool estimate is not a total memory bound. Capture preparation and renderer allocations may overlap during replacement. + +The trial establishes a functioning shared ribbon/Overview surface path on ordinary current windows, but does not yet meet a production guarantee of near-imperceptibility with low total resource cost. Cold preparation, fallback openings, chroma fidelity, missing native shadows, dynamic content timing, projection changes and multiple displays remain unqualified. Disk storage would help inactive compressed thumbnails; it would not remove the live producer pools or decoded display buffers required at handoff. A measured next comparison is a single-display capture for visible content plus selective offscreen captures, versus lower idle cadence with a fresh endpoint sample; neither has been demonstrated by this implementation. Native private compositor transforms remain outside the approved public-API trial. + +The original configuration, monitor workspace stack, logical focus, widths, scroll offsets and managed-frame convergence were restored after validation. The experiment is disabled by default and the tested build remains installed. Exactly one daemon is required after restoration. + + +## User feedback correction: snapshots for Overview, native ribbon + +The user observed persistent sharing badges, flicker, and disrupted ribbon animation. Runtime feedback showed 16 GPU ribbon transitions versus 36 fallbacks, despite the quieter controlled recording. Continuous per-window streams explain the persistent capture session; Apple exposes no general documented capture-indicator suppression option. One-shot screenshots avoid maintaining such a session, but the presence of transient system UI still needs empirical validation. [SCStreamConfiguration](https://developer.apple.com/documentation/screencapturekit/scstreamconfiguration), [SCScreenshotManager](https://developer.apple.com/documentation/screencapturekit/scscreenshotmanager). + +The user explicitly chose one-shot Overview snapshots and the native ribbon while the proxy ribbon is corrected. The daemon now keeps normal ribbon navigation on the native backend. The Overview cache uses public `captureSampleBuffer` screenshots instead of SCStream, refreshes at most once per second while closed, and freezes during the entire opening/session. There are no persistent capture streams. Snapshot content may lag ongoing typing or animation; this tradeoff was accepted explicitly and is not a live-content handoff guarantee. + +An owned native regression reproduced a separate reuse defect: after a non-reversing close, the next Overview scene kept the prior reduced layer transform. Assigning its native frame under that transform also produced incorrect bounds (about 1268×2716 instead of 640×1371 in the fixture). The scene now resets transform, bounds and position under disabled implicit Core Animation actions. The regression failed before the correction and passed after it, also checking zero streams and one fixed displayed snapshot. This addresses a concrete source of jumps at reopening; it does not by itself prove all observed flicker is gone. + + +A real repeated-opening recording exposed a new snapshot-specific expiry defect: the initial opening used the surface zoom, but the next opening fell back because the cached image aged while it was intentionally frozen. An owned fixture held open for over three seconds reproduced it. Snapshot idle validity is now paused during display and resumed after closing; discovery preparation waits until the closing scene has released its texture. The long-dwell/rapid-reopen regression failed before this correction and passed afterward. This preserves the explicitly static snapshot rather than falsely describing it as newly captured content. Computer Use observed the normal Finder traffic-light buttons in place of its persistent purple capture badge, with snapshots enabled and zero streams. A short sample without recording reported daemon CPU 0.3–0.4% enabled, versus 0% disabled; WindowServer was approximately 25% versus 16.4–16.6%. These remain workload-specific short samples, not a total energy or RAM benchmark. + +A subsequent abrupt-opening report reproduced a second expiry path: an idle screenshot older than three seconds was rejected even when its owner and dimensions still matched. The native regression now delays the idle refresh for 3.2 seconds before opening and requires both zoom openings to succeed. Compatible snapshots remain available until replacement; request changes still invalidate incompatible owner/dimensions and the capture pool remains bounded. This can show older content when refresh fails, but avoids a timer-driven switch to a non-zoom opening. Native ribbon navigation remains separate: its adaptive Accessibility latency guard may intentionally submit zero-duration coherent motion when a participating lane cannot produce intermediate frames safely. + +Repeated real-window openings still exposed warm-up fallbacks after removing expiry. Screenshot refresh now decodes into a separate replacement layer and publishes only after it is ready, retaining the usable layer if opening cancels preparation. The existing conservative four-buffer allowance covers the transient old/replacement snapshot overlap; no persistent stream or history is introduced. + +Enter selection previously closed with `commitScrollOffsets: false`, inadvertently disabling the reverse zoom even for unchanged native frames. Visual handoff eligibility now depends on animation settings, viewport alignment and verified native geometry independently of the scroll-commit policy. The owned native test invokes the actual Enter selection handler and asserts that the surface remains attached during closing before the panel disappears. It failed before the correction. + +A recorded workspace change exposed a separate closing limitation: projection updates intentionally discard the opening scene, leaving Enter with no reverse scene. Closing now attempts a short Core Animation scene made from the compact previews already displayed for the new active workspace. Native owner and geometry must match the computed targets within two points; otherwise the normal immediate close remains available. Images are reused and released with the closing panel, with no new screenshot, persistent stream or frame history. Compact previews can appear softer at native scale. `overviewPreviewClosings` reports this path. The owned native regression now navigates between two workspaces, updates the active snapshot, selects the destination and asserts a retained closing scene followed by panel removal without native geometry mutation; it failed before this correction. Real-window validation remains necessary for timing and fidelity. + +## Overview navigation rendering work (2026-10-03) + +An Animation Hitches/Time Profiler capture of the installed daemon showed repeated +AppKit typesetting and application-icon representation resolution in card drawing, +and a 33 ms polling task refreshing a fixed opening scene. The one-shot scene now +retains its original image set without a refresh task; projection changes still +invalidate it, and native bounds are checked before reverse zoom. + +Card titles use decoded bitmaps cached by window, text, font size, available width, +and backing scale, with a strict 4 MiB budget. Icons resolve to small 48 px bitmaps +once rather than repeatedly drawing application icon representations. Trackpad +and horizontal drag deltas accumulate immediately, but their latest viewport is +projected once per monitor display-link callback, including the final event. + +The same owned native renderer benchmark (140 warm, position-changing frames) +measured mean drawing time from 2.02 ms to 1.21–1.27 ms and p95 from 2.49 ms to +1.44–1.59 ms. This measures drawing work, not presented FPS, and does not establish +a universally stable 120 FPS. The native regression also feeds 100 precise scroll +events before yielding, checks that they are coalesced, and verifies the final +100-point movement. Preview, title-cache bounds/reuse, and reverse-zoom tests remain +separate checks. Raw profiling and benchmark artifacts are under the ignored +`dist/verification/overview-profile/` directory. + +The experimental surface capture paths reuse an existing Screen Recording grant and never request one. Overview's ordinary one-shot preview path may request Screen Recording access when previews are enabled and access is absent.