From cbbbadc9517ccd27d0802838656b482194135282 Mon Sep 17 00:00:00 2001 From: Alessandro De Blasis Date: Wed, 26 Aug 2026 13:18:46 +0300 Subject: [PATCH 1/2] fuzz: arm the terminal in morph, and hold it to one switch per chord morph sent its layout chords into an unarmed XAML island. The island drops synthesized keys until a real click lands on its own pixels, and #752 fixed that for the mouse-fuzz harnesses without reaching this one. The cost was invisible because nothing measured it: the harness kept sending, the app kept ignoring, and the oracle only ever read what did happen. Measured against a clean build, it converted 38% to 52% of the chords it sent - 13, 14 and 20 of 34, and 16 of 31. Run outside the suite, which minimizes other windows first, it reliably converted none at all: the startup probe reached a window that had never been clicked and traced no switch whatsoever. So arm before every chord, the same shape as Enable-Chords in mouse-fuzz-tab-close-selection.ps1. Three quarters across and down lands in the terminal under either layout; the window center only does while the strip is narrow, and this harness spends its run changing that. The ad-hoc re-arm click the tab-color branch already carried becomes a call to the same helper. It now converts 1.44 to 1.65 switches per chord - 49 of 34, 39 of 26 and 51 of 31 - so the floor moves from a third to one. That is a statement worth making: every layout chord begins a switch. The surplus above 1 is chords that arrived mid-switch, held as _pendingLayoutTarget and replayed on completion. The tail settle is waited on rather than slept through. Three times as many switches means the last one is far more likely to still be in the air when the finally kills the process, and the verdict reads the trace afterwards, so a fixed 1200ms started reporting a begin with no end as a switch that never finished. It polls for balance instead, which also returns sooner than the sleep did in the ordinary case. --- windows/scripts/fuzz-suite.ps1 | 2 +- windows/scripts/vtabs-morph-fuzz.ps1 | 89 ++++++++++++++++++++++------ 2 files changed, 73 insertions(+), 18 deletions(-) diff --git a/windows/scripts/fuzz-suite.ps1 b/windows/scripts/fuzz-suite.ps1 index 6a8fa85116..fc8a08e8e0 100644 --- a/windows/scripts/fuzz-suite.ps1 +++ b/windows/scripts/fuzz-suite.ps1 @@ -168,7 +168,7 @@ $Harnesses = [System.Collections.Generic.List[object]]@( [ordered]@{ name = 'tab-close-selection'; script = 'mouse-fuzz-tab-close-selection.ps1'; tags = @('tabs'); outDir = $true; seed = $true; minutes = 4 oracle = 'closes a randomly chosen OTHER tab in both strips and checks two things about the tab that was active, once the strip has stopped changing: UIA still reports it selected (matched on the tab title, which the harness seeds distinct because every tab here runs the same shell and would otherwise be named alike), and the fill painted under the selected row differs from every unselected row while the unselected rows all match each other. The second is what catches a selection fill left on a vacated slot. It compares no absolute color, so a build that paints the right row in the wrong color passes; it samples one pixel per row, so a fill that is misplaced by less than a row height passes; it settles before it measures, so a selection that is briefly wrong and right at rest is recorded as transientOff rather than failed; and it never closes the ACTIVE tab, so the successor-selection rule is not exercised. A strip that never stops changing is exit 1 rather than a verdict. Container RuntimeIds are measured over the same closes and reported as idDrift rather than asserted on: they name a slot, not a tab' } [ordered]@{ name = 'morph'; script = 'vtabs-morph-fuzz.ps1'; tags = @('tabs'); outDir = $false; seed = $true; minutes = 3 - oracle = 'randomized layout switching against a full strip, checked against a trace the product emits: no switch ends with a ghost on the morph layer and every begin is answered. The run is gated too, since those terms are all vacuous when nothing switched - a third of the layout chords it sends must begin a switch and no more than a fifth may be refused, so a build the chord never reaches leaves with 1 rather than a green verdict. That floor is sized to catch a run that switched nothing: a healthy run converts only 38% to 52% of its chords, the rest lost inside this harness, so it does not certify that any particular toggle worked and it clears the floor by a thin margin. A seed replays the sequence' } + oracle = 'randomized layout switching against a full strip, checked against a trace the product emits: no switch ends with a ghost on the morph layer and every begin is answered. The run is gated too, since those terms are all vacuous when nothing switched - every layout chord it sends must begin a switch and no more than a fifth may be refused, so a build the chord never reaches leaves with 1 rather than a green verdict. It arms the XAML island with a real click before each chord, which is what makes that floor affordable: a healthy run measures 1.44 to 1.65 switches per chord, the surplus being chords that arrived mid-switch and were replayed on completion. A seed replays the sequence' } [ordered]@{ name = 'inspector'; script = 'mouse-fuzz-inspector.ps1'; tags = @('inspector'); outDir = $true; seed = $false; minutes = 3 oracle = 'the inspector opens, renders something other than a flat surface, and closes; the tab-switch dismissal is gated but a missing second tab only warns' } [ordered]@{ name = 'dialogs'; script = 'mouse-fuzz-dialogs.ps1'; tags = @('dialogs'); outDir = $true; seed = $false; minutes = 2 diff --git a/windows/scripts/vtabs-morph-fuzz.ps1 b/windows/scripts/vtabs-morph-fuzz.ps1 index c5a09e41fa..31973c8cb2 100644 --- a/windows/scripts/vtabs-morph-fuzz.ps1 +++ b/windows/scripts/vtabs-morph-fuzz.ps1 @@ -23,9 +23,13 @@ # # All of that only says something when switches actually happened, and every # term of it is vacuous when none did. So the run is gated too: the desktop has -# to accept the layout chords, and a third of them have to begin a switch. A +# to accept the layout chords, and every one of them has to begin a switch. A # run that toggled nothing leaves with 1 - a chord nobody handled is a corpus # this harness could not establish, not a defect in the build. +# +# That gate is only affordable because the harness arms the XAML island before +# every chord. The island drops synthesized keys until a real click lands on +# its own pixels, and without that this harness lost most of what it sent. param( [string]$ExePath = (Join-Path $PSScriptRoot '..\Ghostty\bin\x64\Debug\net10.0-windows10.0.19041.0\Wintty.exe'), [int]$Seed = 0, @@ -205,6 +209,35 @@ $Colors = @('Red', 'Orange', 'Yellow', 'Green', 'Teal', 'Blue', 'Purple', 'Pink' # and the morph ghost has to copy something other than a default cmd tab. $Shells = @('powershell -NoLogo', 'cmd', 'powershell -NoLogo -Command "$host.UI.RawUI.WindowTitle=''fuzz''; cmd"') +# Clicking the terminal is what makes the app accept a chord at all: the XAML +# island drops synthesized keys until a real click lands on its own pixels, and +# a chrome interaction -- a tab color picked off a flyout -- moves focus off the +# terminal and un-arms it again. Same shape as Enable-Chords in +# mouse-fuzz-tab-close-selection.ps1, which learned it the same way. +# +# Three quarters across and down lands in the terminal under either layout: the +# vertical rail owns the left edge and the horizontal header the top, and +# neither reaches here. The window center does too, but only while the strip is +# narrow, and this harness spends its whole run changing that. +$script:ArmPid = 0 +$script:ArmHwnd = [IntPtr]::Zero +function Enable-Chords { + # Not a silent return. Arming before the window is known leaves the island + # unarmed, and that resurfaces later as a layout chord nobody handled - + # which reads as a broken keybinding rather than as a harness that was + # never allowed to type. + if (-not $script:ArmPid) { + throw 'FOREGROUND_MISS: asked to arm input before the window under test was known' + } + $rc = [MFz]::Rect($script:ArmHwnd) + $x = [int]($rc.L + ($rc.R - $rc.L) * 0.75) + $y = [int]($rc.T + ($rc.B - $rc.T) * 0.75) + if (-not [MFz]::Click([uint32]$script:ArmPid, $x, $y, $false)) { + throw "FOREGROUND_MISS: could not click the terminal at $x,$y to arm input" + } + Start-Sleep -Milliseconds 250 +} + function Get-UiaRoot([int64]$h) { return [System.Windows.Automation.AutomationElement]::FromHandle([IntPtr]::new($h)) } @@ -281,6 +314,10 @@ try { Write-Host ("target $([MFz]::Describe($hwnd))") + $script:ArmPid = $proc.Id + $script:ArmHwnd = $hwnd + Enable-Chords + # Seed the strip so the very first switches already have a crowd. foreach ($i in 1..7) { if (-not [MFz]::Chord($hwnd, $VK.T, $true, $false)) { @@ -292,6 +329,7 @@ try { # One deterministic toggle up front proves the oracle is alive before # minutes of fuzzing are spent on a build that cannot report it. + Enable-Chords if (-not [MFz]::Chord($hwnd, $VK.Comma, $true, $true)) { throw "FOREGROUND_MISS: probe toggle; foreground is $([MFz]::ForegroundNow())" } @@ -328,6 +366,7 @@ try { # nothing. [void][MFz]::Chord($hwnd, $VK.Esc, $false, $false) Start-Sleep -Milliseconds 90 + Enable-Chords $toggleAttempts++ if (-not [MFz]::Chord($hwnd, $VK.Comma, $true, $true)) { $chordMisses++ } # Sometimes toggle again before the switch has landed: the @@ -424,10 +463,7 @@ try { # layout chord only reaches the router from the # terminal surface. Without this the fuzz spends the # rest of the run sending toggles nobody handles. - $wr = [MFz]::Rect($hwnd) - [void][MFz]::Click([uint32]$proc.Id, - [int](($wr.L + $wr.R) / 2), [int](($wr.T + $wr.B) / 2), $false) - Start-Sleep -Milliseconds 200 + Enable-Chords } else { $act = 'tab-color-skipped' } } } catch { @@ -440,8 +476,23 @@ try { Start-Sleep -Milliseconds $rng.Next(160, 700) } - # Let the last switch land before reading the trace. - Start-Sleep -Milliseconds 1200 + # Let the last switch land before the finally kills the process, because + # the verdict reads the trace afterwards and a switch still in the air at + # kill time is a begin with no end - which the oracle would report as a + # switch that never finished. + # + # Waited on rather than slept through. A chord that arrives mid-switch is + # held as _pendingLayoutTarget and replayed when that switch completes, so + # the tail can be two switches deep, and arming the island made those + # chains common enough that a fixed settle started losing the race. + $settleDl = (Get-Date).AddSeconds(5) + while ((Get-Date) -lt $settleDl) { + Start-Sleep -Milliseconds 200 + $tail = @(Get-Content $log -ErrorAction SilentlyContinue) + $b = @($tail | Where-Object { $_ -like 'SWITCH begin*' }).Count + $e = @($tail | Where-Object { $_ -like 'SWITCH end*' -or $_ -like 'SWITCH cancel*' }).Count + if ($b -eq $e) { break } + } } finally { if ($proc -and -not $proc.HasExited) { try { $proc.Kill($true); [void]$proc.WaitForExit(3000) } catch { } } @@ -509,17 +560,21 @@ if ($failures.Count -gt 0) { # Both floors are proportions of the chords sent, because -Iterations decides # how many there are. # -# A third is under what a healthy run reaches, but not by much, because a -# healthy run converts far fewer chords than it sends: measured at 13 and 14 of -# 34 and at 16 of 31 against a clean build, so 38% to 52%. A chord landing -# inside the 340ms switch is dropped by design, and an iteration that touched -# the chrome can leave focus off the terminal for the next one. +# One switch per chord, because that is now what a chord does. Measured against +# a clean build at 49 switches for 34 chords, 39 for 26 and 51 for 31, so 1.44 +# to 1.65 - every chord lands, and a chord that arrives mid-switch is held as +# _pendingLayoutTarget and replayed on completion, which is where the surplus +# comes from. +# +# The surplus is the margin, and it is not guaranteed: two chords inside one +# switch share the single pending slot, so the second is lost and that run +# converts nearer to 1. Hence exactly 1 rather than something above it. # -# So this floor is sized for the case it exists to catch - a run that switched -# nothing at all - and the margin above it is thin: 38% of 34 chords clears a -# floor of 12 by one. Raise it once the harness stops losing chords, not -# before; a build is not at fault for a chord that never reached it. -$MinBeginRatio = 0.33 +# This floor only reads as a product statement because the harness now arms the +# XAML island before every chord. Before that it converted 38% to 52%, the loss +# was entirely its own, and a floor anywhere near this would have failed clean +# builds all day. +$MinBeginRatio = 1.0 $MaxChordMissRatio = 0.2 # These leave with 1, not 2. A chord the desktop refused, or one the router # never saw, is a corpus this harness could not establish rather than a defect From 3a9d48123cf972e2dfcf300483a60b58270aa6f8 Mon Sep 17 00:00:00 2001 From: Alessandro De Blasis Date: Wed, 26 Aug 2026 14:05:44 +0300 Subject: [PATCH 2/2] fuzz: measure the morph floor against the chords that landed Review of the floor at a ratio of one turned up three ways it blames the wrong thing. The floor divided by every chord attempted, including the ones Chord() refused and never delivered. At a third that slack was harmless; at one it charges a foreground steal to the build and then reports it as "the chords went out but the router never saw them" about chords that were never sent. Divide by the delivered ones instead. That is only a safe denominator because the miss cap above has already rejected a run that lost most of them - on its own, every chord refused would leave zero delivered and a floor of nothing, which is the hole the attempted count was there to close. Arming moved out of the tab-color catch. Enable-Chords throws where the click it replaced was best-effort, so a refused arming click was landing as one more 'tab-color-error' and leaving the island unarmed. The iterations after it would send chords nobody handled - none of them asserted on - and the run would go quietly useless until the next toggle threw. And Enable-Chords now checks the window before it measures it. Rect swallows GetWindowRect's return, so a window that has gone away comes back as zeros, which aimed the click at the top-left of the screen and reported a foreground miss for a window that no longer existed. The caller already treats that as PRODUCT_FAIL; say so here too. --- windows/scripts/vtabs-morph-fuzz.ps1 | 44 +++++++++++++++++++++------- 1 file changed, 34 insertions(+), 10 deletions(-) diff --git a/windows/scripts/vtabs-morph-fuzz.ps1 b/windows/scripts/vtabs-morph-fuzz.ps1 index 31973c8cb2..7b9ed02155 100644 --- a/windows/scripts/vtabs-morph-fuzz.ps1 +++ b/windows/scripts/vtabs-morph-fuzz.ps1 @@ -229,11 +229,23 @@ function Enable-Chords { if (-not $script:ArmPid) { throw 'FOREGROUND_MISS: asked to arm input before the window under test was known' } + # Rect swallows GetWindowRect's return, so a window that has gone away + # comes back as zeros rather than as an error. Left unchecked that aims the + # click at the top-left of the screen, hits whatever lives there, and + # reports a foreground miss for what is actually a window that no longer + # exists - a cause the caller reports as PRODUCT_FAIL. + if (-not [MFz]::IsWindow($script:ArmHwnd)) { + throw 'PRODUCT_FAIL: the window under test vanished before its input could be armed' + } $rc = [MFz]::Rect($script:ArmHwnd) + if ($rc.R -le $rc.L -or $rc.B -le $rc.T) { + throw "PRODUCT_FAIL: the window under test has a degenerate rect ($($rc.L),$($rc.T))-($($rc.R),$($rc.B))" + } $x = [int]($rc.L + ($rc.R - $rc.L) * 0.75) $y = [int]($rc.T + ($rc.B - $rc.T) * 0.75) if (-not [MFz]::Click([uint32]$script:ArmPid, $x, $y, $false)) { - throw "FOREGROUND_MISS: could not click the terminal at $x,$y to arm input" + throw ("FOREGROUND_MISS: could not click the terminal at $x,$y to arm input; " + + "foreground is $([MFz]::ForegroundNow())") } Start-Sleep -Milliseconds 250 } @@ -459,17 +471,21 @@ try { } else { $act = 'tab-color-miss' } Start-Sleep -Milliseconds 250 [void][MFz]::Chord($hwnd, $VK.Esc, $false, $false) - # UIA leaves focus on the swatch it clicked, and the - # layout chord only reaches the router from the - # terminal surface. Without this the fuzz spends the - # rest of the run sending toggles nobody handles. - Enable-Chords } else { $act = 'tab-color-skipped' } } } catch { $act = 'tab-color-error' [void][MFz]::Chord($hwnd, $VK.Esc, $false, $false) } + # Outside the catch on purpose. UIA leaves focus on the swatch it + # clicked, and the layout chord only reaches the router from the + # terminal surface, so this is what lets the rest of the run drive + # anything at all. Swallowed as one more 'tab-color-error' it would + # leave the island unarmed, and the iterations after it would send + # chords nobody handles while the actions table recorded them as + # having happened - none of those are asserted on, so the run would + # go quietly useless until the next toggle threw. + Enable-Chords } $actions[$act] = 1 + ($actions[$act] ?? 0) @@ -557,8 +573,9 @@ if ($failures.Count -gt 0) { # no ghost lines to find. A build whose layout chord never reaches the router # therefore reported nothing and left with 0. Gate the run itself. # -# Both floors are proportions of the chords sent, because -Iterations decides -# how many there are. +# Both are proportions rather than counts, because -Iterations decides how many +# chords there are. They divide by different things on purpose: the cap by +# every chord attempted, the floor by only those the desktop actually took. # # One switch per chord, because that is now what a chord does. Measured against # a clean build at 49 switches for 34 chords, 39 for 26 and 51 for 31, so 1.44 @@ -584,9 +601,16 @@ if ($chordMisses -gt $missCap) { throw ("FOREGROUND_MISS: $chordMisses of $toggleAttempts layout chords were refused " + "(cap $missCap) - another window held the foreground, so the switch was never driven") } -$beginFloor = [Math]::Max(1, [int][Math]::Ceiling($toggleAttempts * $MinBeginRatio)) +# Delivered, not attempted. A chord the desktop refused never reached the app, +# and the build cannot be asked to have answered it - counting it here would +# charge a foreground steal to the product and then say "the router never saw +# them" about chords that were never sent. Safe as a denominator only because +# the cap above has already rejected a run that lost most of them: on its own, +# every chord refused would leave zero delivered and a floor of nothing. +$delivered = $toggleAttempts - $chordMisses +$beginFloor = [Math]::Max(1, [int][Math]::Ceiling($delivered * $MinBeginRatio)) if ($begins -lt $beginFloor) { - throw ("only $begins switch(es) began for $toggleAttempts layout chords (floor " + + throw ("only $begins switch(es) began for $delivered delivered layout chords (floor " + "$beginFloor) - the chords went out but the router never saw them, so nothing " + 'above was measured') }