diff --git a/CHANGELOG.md b/CHANGELOG.md index ffd8555d5..eb125d153 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,9 @@ ## Unreleased +- Added strict `wait absent [timeoutMs]` polling for zero selector matches. Incomplete, + sparse, truncated, scoped, depth-limited, and Android unreadable captures cannot prove absence; + deadline diagnostics retain typed capture evidence and stable first-match details (#2236). - Fixed: `settings airplane on|off` now takes an Android device offline. It is applied through the connectivity service (`cmd connectivity airplane-mode`), which drives the radios, instead of writing `airplane_mode_on` and broadcasting `ACTION_AIRPLANE_MODE_CHANGED` — a broadcast Android diff --git a/docs/adr/0012-interactive-replay.md b/docs/adr/0012-interactive-replay.md index 96218d482..e873ec5fc 100644 --- a/docs/adr/0012-interactive-replay.md +++ b/docs/adr/0012-interactive-replay.md @@ -288,7 +288,10 @@ A recorded `id` never matches a node without that id. > `is exists` (existence assertion with no unique winner; wait-like semantics without the > guard-critical role), `is absent` (a strict one-capture absence observation has no resolved > winner; it records as an ordinary observation and its `predicate_failed` failure is always an -> action-failure, never an identity mismatch), every read-only `find` variant (fuzzy-locator resolution has no +> action-failure, never an identity mismatch), `wait absent` (strict zero-candidate polling has no +> resolved winner; its no-match success carries no `target-v1` or landmark annotation, and a +> `wait_target_present` deadline is an ordinary action-failure, never an identity mismatch or an +> ADR 0016 destination guard), every read-only `find` variant (fuzzy-locator resolution has no > selector-chain identity token for the classifier, and publication already refuses mutating `find` > as non-verifiable), and `wait text`/`wait stable`/duration waits/`wait @ref` (no element target, or > a session-local ref that ADR 0016 already refuses to publish; `wait @ref` is rejected rather than diff --git a/packages/ad-replay/src/internal/__tests__/target-verification.test.ts b/packages/ad-replay/src/internal/__tests__/target-verification.test.ts index a4df5f7de..9a4775b63 100644 --- a/packages/ad-replay/src/internal/__tests__/target-verification.test.ts +++ b/packages/ad-replay/src/internal/__tests__/target-verification.test.ts @@ -46,6 +46,16 @@ test('planPostResolutionTargetVerification: a non-selector wait form is inert (s ); }); +test('planPostResolutionTargetVerification: wait absent is not a landmark or identity guard', () => { + assert.deepEqual( + planPostResolutionTargetVerification({ + recorded: recorded(), + isSelectorWait: false, + }), + { kind: 'skip' }, + ); +}); + test('planPostResolutionTargetVerification: a selector wait with a recorded-unverifiable annotation refuses up front', () => { assert.deepEqual( planPostResolutionTargetVerification({ diff --git a/packages/contracts/src/client-system.ts b/packages/contracts/src/client-system.ts index a91043a81..cd3cecc36 100644 --- a/packages/contracts/src/client-system.ts +++ b/packages/contracts/src/client-system.ts @@ -10,6 +10,7 @@ export type WaitCommandTarget = text?: never; ref?: never; selector?: never; + absent?: never; stable?: never; quietMs?: never; timeoutMs?: never; @@ -19,6 +20,7 @@ export type WaitCommandTarget = durationMs?: never; ref?: never; selector?: never; + absent?: never; stable?: never; quietMs?: never; timeoutMs?: number; @@ -28,6 +30,7 @@ export type WaitCommandTarget = durationMs?: never; text?: never; selector?: never; + absent?: never; stable?: never; quietMs?: never; timeoutMs?: number; @@ -37,6 +40,17 @@ export type WaitCommandTarget = durationMs?: never; text?: never; ref?: never; + absent?: never; + stable?: never; + quietMs?: never; + timeoutMs?: number; + }) + | (SelectorSnapshotCommandOptions & { + absent: string; + durationMs?: never; + text?: never; + ref?: never; + selector?: never; stable?: never; quietMs?: never; timeoutMs?: number; @@ -47,6 +61,7 @@ export type WaitCommandTarget = text?: never; ref?: never; selector?: never; + absent?: never; quietMs?: number; timeoutMs?: number; }); diff --git a/packages/contracts/src/wait-runtime-plan.ts b/packages/contracts/src/wait-runtime-plan.ts index d33046d58..0ca8d7057 100644 --- a/packages/contracts/src/wait-runtime-plan.ts +++ b/packages/contracts/src/wait-runtime-plan.ts @@ -1,5 +1,5 @@ /** The normalized wait target, independent of the positional grammar that produced it. */ -export type WaitRuntimeTarget = 'sleep' | 'text' | 'ref' | 'selector' | 'stable'; +export type WaitRuntimeTarget = 'sleep' | 'text' | 'ref' | 'selector' | 'absent' | 'stable'; /** * Which wait shapes reach a device at all. A duration wait observes nothing, so it never asks diff --git a/packages/contracts/src/wait.ts b/packages/contracts/src/wait.ts index 1d6d47aeb..6ba07ad60 100644 --- a/packages/contracts/src/wait.ts +++ b/packages/contracts/src/wait.ts @@ -6,8 +6,9 @@ * `wait_capture_stalled` means no readable capture established an observation * before the deadline and is retriable. `wait_deadline_exceeded` means a later * capture consumed the remaining budget after at least one readable capture. - * `wait_target_absent` is the only ordinary absence verdict and therefore - * always carries readable-capture evidence. The remaining reasons describe + * `wait_target_absent` is the ordinary timeout reason for a selector that was + * never found. `wait_target_present` is the strict-absence timeout reason when + * valid captures still contain matches. The remaining reasons describe * stability and replay-landmark refusals. */ export const WAIT_REASONS = { @@ -15,6 +16,7 @@ export const WAIT_REASONS = { deadlineExceeded: 'wait_deadline_exceeded', runnerRestartExhausted: 'wait_runner_restart_exhausted', targetAbsent: 'wait_target_absent', + targetPresent: 'wait_target_present', stableTimeout: 'wait_stable_timeout', landmarkIdentityMismatch: 'wait_landmark_identity_mismatch', } as const; diff --git a/packages/maestro/src/internal/__tests__/export-label-projection.test.ts b/packages/maestro/src/internal/__tests__/export-label-projection.test.ts index 4401c11f5..b5d1c4f7b 100644 --- a/packages/maestro/src/internal/__tests__/export-label-projection.test.ts +++ b/packages/maestro/src/internal/__tests__/export-label-projection.test.ts @@ -40,6 +40,23 @@ test('keeps compound selectors that include label as hard export errors', () => ).toThrow(AppError); }); +test('does not export strict wait absent as Maestro notVisible', () => { + let thrown: unknown; + try { + exportReplayActionsToMaestro([action('wait', ['absent', 'label="Removed"', '1000'])], { + resolveSelector: (expression) => + projectSelectorExpression(expression, MAESTRO_SELECTOR_PROJECTION), + }); + } catch (error) { + thrown = error; + } + + expect(thrown).toBeInstanceOf(AppError); + if (!(thrown instanceof AppError)) return; + expect(thrown.message).toMatch(/unsupported|strict absence/i); + expect(thrown.message).not.toMatch(/notVisible/i); +}); + function action(command: string, positionals: string[]): SessionAction { return { ts: 0, command, positionals, flags: {} }; } diff --git a/packages/maestro/src/internal/export-flow.ts b/packages/maestro/src/internal/export-flow.ts index 23ae1b487..591f14883 100644 --- a/packages/maestro/src/internal/export-flow.ts +++ b/packages/maestro/src/internal/export-flow.ts @@ -322,6 +322,12 @@ function convertWaitAction( ], }; } + if (first === 'absent') { + return { + kind: 'unsupported', + message: 'strict wait absent requires zero selector matches and is unsupported by Maestro', + }; + } if (first === 'text' && second) { return { kind: 'commands', diff --git a/scripts/__tests__/eager-closure-budgets.ts b/scripts/__tests__/eager-closure-budgets.ts index c7bd3650b..1890c2de9 100644 --- a/scripts/__tests__/eager-closure-budgets.ts +++ b/scripts/__tests__/eager-closure-budgets.ts @@ -411,7 +411,11 @@ export const HUB_BUDGETS: Readonly> = Object.freeze({ // #2148 moves output-only CLI dependencies behind call-time imports and reduces the entry // closure by two modules. // #2146 splits one eagerly reached URL utility into its client and Metro owners. - 'src/cli.ts': 380, + // #2236 adds the typed pre-admission --scope/--depth refusal for `wait absent` to the existing + // wait command reader. That deliberately keeps the shared absence option contract and error + // modules on the CLI path; the measured two-module growth is the contract being loaded, not + // implementation or platform machinery being pulled in eagerly. + 'src/cli.ts': 382, 'src/platform-runtime.ts': 47, 'src/core/command-descriptor/registry.ts': 72, 'src/core/command-descriptor/platform-execution-entry.ts': 3, diff --git a/src/__tests__/client-wait.test.ts b/src/__tests__/client-wait.test.ts new file mode 100644 index 000000000..d9f37ecdc --- /dev/null +++ b/src/__tests__/client-wait.test.ts @@ -0,0 +1,23 @@ +import assert from 'node:assert/strict'; +import { test } from 'vitest'; +import { createAgentDeviceClient } from '../agent-device-client.ts'; +import { createTransport } from './client-transport-fixture.ts'; + +test('client.command.wait round-trips strict wait absent positionals', async () => { + const setup = createTransport(async () => ({ ok: true, data: { waitedMs: 0 } })); + const client = createAgentDeviceClient(setup.config, { transport: setup.transport }); + + await client.command.wait({ absent: 'label="Removed"', timeoutMs: 2500 }); + + assert.equal(setup.calls[0]?.command, 'wait'); + assert.deepEqual(setup.calls[0]?.positionals, ['absent', 'label="Removed"', '2500']); + + await assert.rejects( + async () => await client.command.wait({ absent: 'label="Removed"', depth: 2 }), + /wait absent does not support --depth/, + ); + await assert.rejects( + async () => await client.command.wait({ absent: 'label="Removed"', scope: 'Root' }), + /wait absent does not support --scope/, + ); +}); diff --git a/src/__tests__/command-doc-coverage.test.ts b/src/__tests__/command-doc-coverage.test.ts index 68f16e187..a518f9b57 100644 --- a/src/__tests__/command-doc-coverage.test.ts +++ b/src/__tests__/command-doc-coverage.test.ts @@ -184,6 +184,13 @@ describe('command reference doc coverage', () => { ); }); + test('commands.md publishes the wait absence unreadable-observation reason', () => { + assert.match( + markdown, + /`?predicate_failed`? means strict `wait absent` could not prove absence because no valid capture arrived/, + ); + }); + test('no stale waivers', () => { assert.deepEqual( findStaleUndocumentedWaivers( diff --git a/src/cli-schema/cli-help-command-usage.test.ts b/src/cli-schema/cli-help-command-usage.test.ts index 1e13d67c8..7025d2d2c 100644 --- a/src/cli-schema/cli-help-command-usage.test.ts +++ b/src/cli-schema/cli-help-command-usage.test.ts @@ -12,6 +12,13 @@ test('usageForCommand documents open --launch-args', async () => { assert.match(help, /--launch-console artifacts\/launch-console\.log/); }); +test('usageForCommand documents strict wait absent', async () => { + const help = await usageForCommand('wait'); + if (help === null) throw new Error('Expected wait help text'); + assert.match(help, /absent \[timeoutMs\]/); + assert.match(help, /strictly absent|zero selector matches/); +}); + test('usageForCommand documents screenshot web aliases and stabilization flags', async () => { const help = await usageForCommand('screenshot'); if (help === null) throw new Error('Expected screenshot help text'); diff --git a/src/cli-schema/cli-help-overview.ts b/src/cli-schema/cli-help-overview.ts index 72367136f..5838bc944 100644 --- a/src/cli-schema/cli-help-overview.ts +++ b/src/cli-schema/cli-help-overview.ts @@ -25,7 +25,7 @@ Loop: scroll [amount] --settle; back --settle acts, waits for quiet, and prints the UI diff. Continue from that diff. Run snapshot -i only when the diff lacks the next target or did not settle. - Verify a named expectation with the diff, wait text "...", wait , + Verify a named expectation with the diff, wait text "...", wait , wait absent , is, get, or find. A bare screenshot is not verification. End with: agent-device close diff --git a/src/cli-schema/cli-help-topics.test.ts b/src/cli-schema/cli-help-topics.test.ts index 29288b8ae..157a6b0ba 100644 --- a/src/cli-schema/cli-help-topics.test.ts +++ b/src/cli-schema/cli-help-topics.test.ts @@ -185,6 +185,10 @@ test('usageForCommand resolves workflow help topic', async () => { ); assert.match(help, /open -> snapshot -i -> settle -> verify -> close loop/); assert.match(help, /type never takes --settle/); + assert.match(help, /snapshot\/get\/is\/find answer read-only questions/); + assert.match(help, /--settle confirms local UI quieted/); + assert.match(help, /wait text "Expected result" or wait instead of polling/); + assert.match(help, /strict disappearance uses wait absent /); assert.match( help, /Chain confident consecutive steps with &&: press 'label="Search"' --settle && fill 'label="Search"' "query" --settle/, @@ -234,6 +238,11 @@ test('usageForCommand resolves workflow help topic', async () => { assert.match(help, /run serially within one session/); assert.match(help, /Wait failure contract:/); assert.match(help, /wait_target_absent: a readable capture ran and found no match/); + assert.match(help, /wait_target_present: wait absent timed out with matches/); + assert.match( + help, + /predicate_failed: wait absent had no valid capture; final observation\/diagnostic is preserved/, + ); assert.match(help, /wait_capture_stalled: no readable capture finished before the deadline/); assert.match(help, /wait_deadline_exceeded: a later capture used the remaining budget/); assert.match(help, /wait_landmark_identity_mismatch: a replay destination guard/); @@ -547,7 +556,10 @@ test('usageForCommand resolves manual QA help topic', async () => { assert.match(help, /label="Email" editable=true/); assert.match(help, /press 'label="Follow"' --settle/); assert.match(help, /Do not use placeholders such as @ref/); + assert.match(help, /wait text\/selector\/absent/); + assert.match(help, /wait absent 'label="Loading\.\.\."' 3000/); assert.match(help, /wait_target_absent: a readable capture ran and found no match/); + assert.match(help, /wait_target_present: wait absent timed out with matches/); assert.match(help, /wait_capture_stalled: no readable capture finished before the deadline/); }); diff --git a/src/cli-schema/cli-help.ts b/src/cli-schema/cli-help.ts index 4ce36f479..8f075211d 100644 --- a/src/cli-schema/cli-help.ts +++ b/src/cli-schema/cli-help.ts @@ -49,12 +49,15 @@ const EXAMPLE_LINES = [ 'agent-device snapshot -i', 'agent-device fill @e3 "test@example.com"', 'agent-device replay ./session.ad', + `agent-device wait absent 'label="Loading..."' 3000`, 'agent-device test ./suite --platform android', ] as const; const WAIT_FAILURE_CONTRACT = `Wait failure contract: - Read the verdict from error.details.reason in --json, not the message text. + Read error.details.reason in --json, not the message text. wait_target_absent: a readable capture ran and found no match. + wait_target_present: wait absent timed out with matches; details include matches and firstMatch. + predicate_failed: wait absent had no valid capture; final observation/diagnostic is preserved. wait_capture_stalled: no readable capture finished before the deadline -- retriable. wait_deadline_exceeded: a later capture used the remaining budget after an earlier readable one. wait_landmark_identity_mismatch: a replay destination guard found the selector but not the recorded identity. @@ -82,7 +85,7 @@ Loop: 3. Run press/fill/click/longpress --settle for each mutating step. 4. Treat a settled:true diff as the next observation. Do not add wait stable or another snapshot when the diff already shows the next target or expected result. 5. If --settle prints not settled, follow its hint before the next ref-based action. - 6. Verify named expectations with wait text/selector, get, is, find, or the settled diff. A bare screenshot/snapshot is not verification for a named expectation. + 6. Verify named expectations with wait text/selector/absent, get, is, find, or the settled diff. A bare screenshot/snapshot is not verification for a named expectation. 7. Close the session when the script ends. Command shapes: @@ -93,6 +96,7 @@ Command shapes: agent-device press 'label="Follow"' --settle agent-device fill @e13 "qa@example.com" --settle agent-device wait text "Order placed" 3000 + agent-device wait absent 'label="Loading..."' 3000 agent-device close --relaunch forces fresh app state; a deep link/URL open does not need it. Labels with an apostrophe or quote (label="Don't leave") are shell-quoting hazards: prefer the @ref from the latest snapshot/settle output over quoting the literal label. @@ -104,6 +108,7 @@ Targets: Recovery: Network/typeahead result missing: wait text "Expected result" or wait . + A target that should be gone/disappear: use wait absent ; wait exists ... is rejected in favor of the plain selector wait. Keyboard visible over the next target: the on-screen keyboard usually does not block presses, so press the target directly instead of dismissing. If the press fails or reports no visible effect, scroll the target into view or use keyboard enter when submission is wanted. Sparse or recovered accessibility snapshot: use screenshot as visual truth, leave the bad screen if needed, then retry snapshot -i. Non-hittable success hint: verify with the settled diff or snapshot; retarget by a better ref/selector if the UI did not change. @@ -125,10 +130,10 @@ Focused compatibility request: ${MAESTRO_COMPATIBILITY_ISSUE_URL}`, summary: 'Normal agent-device bootstrap, exploration, and validation loop', body: `agent-device help workflow -Command shapes, refs, selectors, waits, recovery, and platform limits for the default open -> snapshot -i -> settle -> verify -> close loop. +Command shapes, refs, selectors, waits, recovery, and platform limits for open -> snapshot -i -> settle -> verify -> close loop. Command shape: - Command lines only -- no prose, numbering, fences, pipes, or grep/head/tail/jq on agent-device output; raw output carries the refs/hints the next step needs. Subcommand first, then positionals, then flags: agent-device open com.example.app --session checkout --platform android --relaunch + Command lines only -- no prose, numbering, fences, pipes, or grep/head/tail/jq; raw output carries refs/hints for the next step. Order: subcommand, positionals, flags: agent-device open com.example.app --session checkout --platform android --relaunch Chain confident consecutive steps with &&: press 'label="Search"' --settle && fill 'label="Search"' "query" --settle. Fall back to one command at a time when a step is uncertain (ambiguous match, network-backed result, unseen screen). Refs look like @e12; use the exact ref from the latest snapshot -i, never a placeholder (@ref, @eN, @Label_Name). Pin with ~s (press @e12~s4); iOS rejects a stale pinned ref -- refresh with snapshot -i or use a selector. close = agent-device close. App back is back; system back is back --system. Taps are press/click. type never takes --settle: run type, then diff snapshot to verify. Known flow: batch --steps-file ./steps.json (help batch). @@ -141,11 +146,11 @@ Bootstrap: Apple CI: prepare ios-runner after boot/install, before replay/test (help prepare). Remote/cloud: connect -> open -> commands -> close -> disconnect (help remote). Reusable scripts, secret-safe fills, replay repair: help scripting. Snapshots and refs: - snapshot reads visible state; snapshot -i gets current interactive refs only -- the fast path before an interaction. Default text is agent-facing and token-efficient; --raw/--json only for the full provider tree. + snapshot reads visible state; snapshot -i gets current interactive refs only -- fast path before interaction. Default text is token-efficient; --raw/--json for full provider tree. Legend: @e12 [button] label="Add to cart" enabled hittable -> press @e12. [off-screen below] -> scroll down (a hint, not a ref). Refs stay valid until you press/click/fill/type/scroll/back/wait-for-async-UI, or otherwise change app state; open/--relaunch clears the stored snapshot outright. - Prefer --settle and continue from its settled diff when it shows the next target; refresh with snapshot -i only when you did not settle, it reported not settled, or its output lacks what you need. A known selector/label after a mutation is often enough, since interaction commands refresh state internally. - Truncated preview: snapshot -s @e12 (the current concrete ref), not get text. Missing target in a list: scroll down/up (not bottom/top unless the task wants the edge), then snapshot -i. TV/D-pad focus: help tv. + Prefer --settle and its diff when it shows next target; refresh with snapshot -i only when you did not settle, it reported not settled, or output lacks what you need. A known selector/label after a mutation is often enough, since interaction commands refresh state internally. + Truncated preview: snapshot -s @e12 (the current concrete ref), not get text. Missing list target: scroll down/up then snapshot -i. TV/D-pad focus: help tv. Selectors: id="field-email", label="Allow", role=button label="Search" -- not bare role keys (button="Search"); no CSS selectors/--selector/--text/raw x-y when refs/selectors exist. @@ -164,7 +169,7 @@ Session ordering: Read-only and waits: ${WAIT_FAILURE_CONTRACT} - snapshot/get/is/find answer read-only questions; snapshot -i only when refs are needed. --settle confirms local UI quieted; for results that arrive later (network/debounce), follow with wait text "Expected result" or wait instead of polling. + snapshot/get/is/find answer read-only questions; snapshot -i is for refs. --settle confirms local UI quieted; delayed results use wait text "Expected result" or wait instead of polling; strict disappearance uses wait absent . wait stable [quietMs] [timeoutMs] (defaults 500/10000) is the fallback for open/relaunch/navigation, or an intentionally-unsettled mutation -- not after a --settle whose diff already shows the change. Ambiguous find: add --first or --last. Navigation: @@ -212,7 +217,7 @@ Reusable open-to-destination scripts: agent-device press 'id="continue"' --settle agent-device wait 'role="heading" label="Screen X"' agent-device session save-script - session save-script [path] [--force] publishes the sole recorded open through the destination guard, omits close, and leaves the session active. The guard is a selector wait on a labeled/id-bearing landmark: its identity is captured while armed and re-verified after the wait resolves at replay time, so a reshuffled screen with the same label elsewhere fails closed instead of false-passing. A duration wait, wait stable, wait @ref, or a selector wait on an unlabeled element is not a destination guard. A second successful open aborts publication; start a fresh session to author again. + session save-script [path] [--force] publishes the sole recorded open through the destination guard, omits close, and leaves the session active. The guard is a selector wait on a labeled/id-bearing landmark: its identity is captured while armed and re-verified after the wait resolves at replay time, so a reshuffled screen with the same label elsewhere fails closed instead of false-passing. A duration wait, wait stable, wait absent, wait @ref, or a selector wait on an unlabeled element is not a destination guard. A second successful open aborts publication; start a fresh session to author again. Unparameterized fill/type inputs are literal .ad script content. For a sensitive fill, arm recording first, keep the live value in an env var, and name its replay placeholder explicitly: export AD_VAR_PASSWORD='' agent-device fill 'id="password"' "$AD_VAR_PASSWORD" --record-as PASSWORD @@ -917,7 +922,7 @@ First-slice loop: agent-device close --platform web Supported in agent-device web sessions: - open , snapshot -i, get text/attrs, is visible/hidden/exists/absent/focused/text, find text/selector, click/press @ref or selector, hover @ref or selector, fill/type @ref or selector, wait text/selector, network dump, audio probe, screenshot, record start/stop with WebM output, close, and replay scripts made from those commands. + open , snapshot -i, get text/attrs, is visible/hidden/exists/absent/focused/text, find text/selector, click/press @ref or selector, hover @ref or selector, fill/type @ref or selector, wait text/selector/absent, network dump, audio probe, screenshot, record start/stop with WebM output, close, and replay scripts made from those commands. hover moves the pointer without pressing so hover-gated UI (row toolbars, menus) appears; use --settle to read what it revealed, then act on the fresh refs. hover @ref hovers the browser element handle directly; pair --settle with a selector or coordinates (web refs carry no geometry, as with click @ref --settle). Web only: touch platforms have no hover state, so hover-gated flows there need a different entry point. Out of scope for agent-device web support: diff --git a/src/commands/capture/index.test.ts b/src/commands/capture/index.test.ts index 12208b065..c71816ed7 100644 --- a/src/commands/capture/index.test.ts +++ b/src/commands/capture/index.test.ts @@ -132,6 +132,33 @@ describe('capture command interface', () => { expectInvalidArgs(() => waitDaemonWriter({ text: 'Ready', ref: '@e1' }), 'exactly one'); }); + test('reads and writes strict wait absent targets', () => { + expect(waitCliReader(['absent', 'label="Removed"', '5000'], flags())).toMatchObject({ + absent: 'label="Removed"', + timeoutMs: 5000, + }); + expect(waitDaemonWriter({ absent: 'label="Removed"', timeoutMs: 5000 })).toMatchObject({ + command: 'wait', + positionals: ['absent', 'label="Removed"', '5000'], + }); + }); + + test('refuses wait absent scope and depth with typed invalid arguments', () => { + expectInvalidArgs( + () => waitCliReader(['absent', 'label="Removed"'], flags({ snapshotScope: 'Root' })), + '--scope', + ); + expectInvalidArgs( + () => waitCliReader(['absent', 'label="Removed"'], flags({ snapshotDepth: 2 })), + '--depth', + ); + expectInvalidArgs( + () => waitDaemonWriter({ absent: 'label="Removed"', scope: 'Root' }), + '--scope', + ); + expectInvalidArgs(() => waitDaemonWriter({ absent: 'label="Removed"', depth: 2 }), '--depth'); + }); + test('reads and writes wait stable with defaults', () => { expect(waitCliReader(['stable'], flags())).toMatchObject({ stable: true }); expect(waitDaemonWriter({ stable: true })).toMatchObject({ diff --git a/src/commands/capture/wait-command-contract.ts b/src/commands/capture/wait-command-contract.ts index 4579496ed..4d9e5eafb 100644 --- a/src/commands/capture/wait-command-contract.ts +++ b/src/commands/capture/wait-command-contract.ts @@ -1 +1,8 @@ -export const WAIT_KIND_VALUES = ['duration', 'text', 'ref', 'selector', 'stable'] as const; +export const WAIT_KIND_VALUES = [ + 'duration', + 'text', + 'ref', + 'selector', + 'absent', + 'stable', +] as const; diff --git a/src/commands/capture/wait.ts b/src/commands/capture/wait.ts index 1cad2085a..4a6f94508 100644 --- a/src/commands/capture/wait.ts +++ b/src/commands/capture/wait.ts @@ -2,6 +2,7 @@ import type { CliFlags } from '@agent-device/contracts/command'; import { PUBLIC_COMMANDS } from '../../command-catalog.ts'; import type { WaitCommandOptions } from '@agent-device/contracts/client'; import { parseWaitPositionals } from '../../core/wait-positionals.ts'; +import type { WaitParsed } from '../../core/wait-positionals.ts'; import { SELECTOR_SNAPSHOT_FLAGS } from '../cli-grammar/flag-groups.ts'; import { AppError } from '@agent-device/kernel/errors'; import { isValidSelectorExpression } from '@agent-device/selectors'; @@ -19,11 +20,13 @@ import { defineCommandFacet } from '../family/types.ts'; import { defineFieldCommandMetadata } from '../field-command-contract.ts'; import { messageOutput } from '../output-common.ts'; import { WAIT_KIND_VALUES } from './wait-command-contract.ts'; +import { absenceCaptureOptionRefusal } from '../../core/absence-observation.ts'; +import { absenceCaptureOptionError } from '../../core/absence-observation-errors.ts'; const WAIT_COMMAND_NAME = 'wait'; const waitCommandDescription = - 'Wait for a duration, text, snapshot ref, selector, or stable UI. Use text, ref, or selector for a specific readiness condition; stable waits until the UI stays quiet for the requested window.'; + 'Wait for a duration, text, snapshot ref, selector to appear, selector to be strictly absent, or stable UI. Use the strict absence target when zero selector matches are required; stable waits until the UI stays quiet for the requested window.'; const waitCommandMetadata = defineFieldCommandMetadata(WAIT_COMMAND_NAME, waitCommandDescription, { kind: enumField(WAIT_KIND_VALUES), @@ -31,6 +34,7 @@ const waitCommandMetadata = defineFieldCommandMetadata(WAIT_COMMAND_NAME, waitCo text: stringField(), ref: stringField(), selector: stringField(), + absent: stringField(), stable: booleanField(), quietMs: integerField(), timeoutMs: integerField(), @@ -44,7 +48,8 @@ const waitCommandDefinition = defineExecutableCommand(waitCommandMetadata, (clie ); const waitCliSchema = { - usageOverride: 'wait |text |@ref||stable [quietMs] [timeoutMs]', + usageOverride: + 'wait |text |@ref||absent [timeoutMs]|stable [quietMs] [timeoutMs]', positionalArgs: ['durationOrSelector', 'timeoutMs?'], allowsExtraPositionals: true, allowedFlags: [...SELECTOR_SNAPSHOT_FLAGS], @@ -60,7 +65,7 @@ export const waitDaemonWriter: DaemonWriter = direct(PUBLIC_COMMANDS.wait, (inpu export const waitCommandFacet = defineCommandFacet({ name: WAIT_COMMAND_NAME, text: { - summary: 'Wait for a duration, text, selector, or stable UI', + summary: 'Wait for a duration, text, selector, strict absence, or stable UI', }, metadata: waitCommandMetadata, definition: waitCommandDefinition, @@ -85,39 +90,66 @@ function readWaitOptionsFromPositionals( if (!parsed) { throw new AppError( 'INVALID_ARGS', - 'wait requires , text , @ref, [timeoutMs], or stable [quietMs] [timeoutMs].', + 'wait requires , text , @ref, [timeoutMs], absent [timeoutMs], or stable [quietMs] [timeoutMs].', ); } const base = { ...selectionOptionsFromFlags(flags), ...selectorSnapshotOptionsFromFlags(flags), }; - if (parsed.kind === 'sleep') return { ...base, durationMs: parsed.durationMs }; - if (parsed.kind === 'text') { - if (!parsed.text) throw new AppError('INVALID_ARGS', 'wait requires text.'); - return { ...base, text: parsed.text, ...readTimeoutOption(parsed.timeoutMs) }; - } - if (parsed.kind === 'ref') { - return { ...base, ref: parsed.rawRef, ...readTimeoutOption(parsed.timeoutMs) }; - } - if (parsed.kind === 'stable') { - return { - ...base, - stable: true, - ...readQuietOption(parsed.quietMs), - ...readTimeoutOption(parsed.timeoutMs), - }; - } - if (parsed.kind === 'invalid') { - throw new AppError('INVALID_ARGS', parsed.message); + switch (parsed.kind) { + case 'absent': + return readAbsentWaitOptions(base, parsed); + case 'sleep': + return { ...base, durationMs: parsed.durationMs }; + case 'text': + return readTextWaitOptions(base, parsed); + case 'ref': + return { ...base, ref: parsed.rawRef, ...readTimeoutOption(parsed.timeoutMs) }; + case 'stable': + return { + ...base, + stable: true, + ...readQuietOption(parsed.quietMs), + ...readTimeoutOption(parsed.timeoutMs), + }; + case 'invalid': + throw new AppError('INVALID_ARGS', parsed.message); + case 'selector': + return { + ...base, + selector: parsed.selectorExpression, + ...readTimeoutOption(parsed.timeoutMs), + }; + default: + return assertNever(parsed); } +} + +type WaitCliBaseOptions = ReturnType & + ReturnType; + +function readAbsentWaitOptions( + base: WaitCliBaseOptions, + parsed: Extract, +): WaitCommandOptions { + const refusedOption = absenceCaptureOptionRefusal(base); + if (refusedOption) throw absenceCaptureOptionError(refusedOption, 'wait'); return { ...base, - selector: parsed.selectorExpression, + absent: parsed.selectorExpression, ...readTimeoutOption(parsed.timeoutMs), }; } +function readTextWaitOptions( + base: WaitCliBaseOptions, + parsed: Extract, +): WaitCommandOptions { + if (!parsed.text) throw new AppError('INVALID_ARGS', 'wait requires text.'); + return { ...base, text: parsed.text, ...readTimeoutOption(parsed.timeoutMs) }; +} + // fallow-ignore-next-line complexity function waitPositionals(options: WaitCommandOptions): string[] { const targets = [ @@ -125,18 +157,27 @@ function waitPositionals(options: WaitCommandOptions): string[] { options.text !== undefined ? 'text' : undefined, options.ref !== undefined ? 'ref' : undefined, options.selector !== undefined ? 'selector' : undefined, + options.absent !== undefined ? 'absent' : undefined, options.stable !== undefined ? 'stable' : undefined, ].filter(Boolean); if (targets.length !== 1) { throw new AppError( 'INVALID_ARGS', - 'wait command requires exactly one of durationMs, text, ref, selector, or stable.', + 'wait command requires exactly one of durationMs, text, ref, selector, absent, or stable.', ); } if (options.durationMs !== undefined) return [String(options.durationMs)]; const timeout = optionalNumber(options.timeoutMs); if (options.text !== undefined) return ['text', options.text, ...timeout]; if (options.ref !== undefined) return [options.ref, ...timeout]; + if (options.absent !== undefined) { + const refusedOption = absenceCaptureOptionRefusal(options); + if (refusedOption) throw absenceCaptureOptionError(refusedOption, 'wait'); + if (!isValidSelectorExpression(options.absent)) { + throw new AppError('INVALID_ARGS', `Invalid wait absent selector: ${options.absent}`); + } + return ['absent', options.absent, ...timeout]; + } if (options.stable !== undefined) { const quiet = optionalNumber(options.quietMs); if (quiet.length === 0 && timeout.length > 0) { @@ -158,3 +199,7 @@ function readTimeoutOption(timeoutMs: number | null): { timeoutMs?: number } { function readQuietOption(quietMs: number | null): { quietMs?: number } { return quietMs === null ? {} : { quietMs }; } + +function assertNever(value: never): never { + throw new Error(`Unsupported wait positional target: ${String(value)}`); +} diff --git a/src/commands/interaction/runtime/selector-read.test.ts b/src/commands/interaction/runtime/selector-read.test.ts index 6f8566e75..2c83602c3 100644 --- a/src/commands/interaction/runtime/selector-read.test.ts +++ b/src/commands/interaction/runtime/selector-read.test.ts @@ -351,8 +351,8 @@ test('runtime selector convenience methods use explicit target helpers', async ( // mid-transition Android helper content verdicts) instead of aborting the // wait — the live-validated destination-guard gap from #1349's PR review. // (#1349's own in-loop landmark identity verification tests — the -// `target.recordedLandmark` cases — moved to `selector-wait.test.ts`, the -// 1:1 topology location for `selector-wait.ts`; #1478 P5 step 2 cell 7.) +// `target.recordedLandmark` cases — moved to `wait-selector.test.ts`, the +// 1:1 topology location for `wait-selector.ts`; #1478 P5 step 2 cell 7.) // --------------------------------------------------------------------------- function landmarkScreen(parentLabel: string) { diff --git a/src/commands/interaction/runtime/selector-wait.ts b/src/commands/interaction/runtime/selector-wait.ts index 2666e9feb..445871683 100644 --- a/src/commands/interaction/runtime/selector-wait.ts +++ b/src/commands/interaction/runtime/selector-wait.ts @@ -1,69 +1,24 @@ import { AppError } from '@agent-device/kernel/errors'; -import { WAIT_REASONS } from '@agent-device/contracts/wait'; import { findNodeByRef, normalizeRef, type SnapshotNode } from '@agent-device/kernel/snapshot'; -import { - annotationLocalIdentity, - buildAncestryChain, - buildIndexMap, - filterIdentitySet, - readNodeLocalIdentity, -} from '@agent-device/ad-script'; -import { - WAIT_LANDMARK_MISMATCH_REASON, - type TargetAnnotationV1, - type WaitLandmarkMismatchEvidence, -} from '@agent-device/contracts/replay'; +import type { TargetAnnotationV1 } from '@agent-device/contracts/replay'; import type { PublicPlatform } from '@agent-device/kernel/device'; -import { checkWaitText, type SelectorChainMatchList } from '@agent-device/selectors'; -import { - resolveSelectorPipeline, - type SelectorPipelineOutcome, -} from '../../../core/selector-pipeline.ts'; -import { SELECTOR_PIPELINE_POLICIES } from '../../../core/selector-pipeline-policy.ts'; -import { deriveSelectorCapturePolicy } from './selector-capture-policy.ts'; -import { findNodeByLabel, resolveRefLabel } from './selector-read-utils.ts'; -import { - createWaitPolling, - DEFAULT_WAIT_TIMEOUT_MS, - sleepWithWaitCancellation, - type WaitPollDeadline, - waitTimeoutError, -} from './wait-polling.ts'; - -/** - * The landmark check (#1349) needs the full candidate set, which the pipeline - * outcome carries in either shape. Wait's row never refuses, so this only ever - * adapts — it does not re-decide anything. - */ -function policyMatchList(outcome: SelectorPipelineOutcome): SelectorChainMatchList | undefined { - if (outcome.kind === 'ambiguous') { - return { - selector: outcome.selector, - selectorIndex: outcome.selectorIndex, - matchedNodes: outcome.matchedNodes, - }; - } - if (outcome.kind === 'target') { - return { - selector: outcome.selector, - selectorIndex: outcome.selectorIndex, - // Every candidate, not just the winner: the landmark check is satisfied - // when SOME match carries the recorded identity, so a first impostor - // must not hide a later genuine landmark (#1349). - matchedNodes: outcome.matchedNodes, - }; - } - return undefined; -} - -type WaitCommandContext = { +import type { CapturedSnapshot } from './selector-read-shared.ts'; +import { checkWaitText } from '@agent-device/selectors'; +import { resolveRefLabel } from './selector-read-utils.ts'; +import { sleepWithWaitCancellation } from './wait-polling.ts'; +import { waitForAbsent } from './wait-absent.ts'; +import { waitForSelector } from './wait-selector.ts'; +import { waitForStable } from './wait-stable.ts'; +import { waitForText } from './wait-text.ts'; + +export type WaitCommandContext = { session?: string; requestId?: string; signal?: AbortSignal; metadata?: Record; }; -type SelectorSnapshotOptions = { +export type SelectorSnapshotOptions = { depth?: number; scope?: string; raw?: boolean; @@ -88,6 +43,7 @@ export type WaitCommandOptions = WaitCommandContext & */ recordedLandmark?: TargetAnnotationV1; } + | { kind: 'absent'; selector: string; timeoutMs?: number | null } | { kind: 'stable'; quietMs?: number | null; timeoutMs?: number | null }; }; @@ -102,6 +58,7 @@ export type WaitCommandResult = node?: SnapshotNode; preActionNodes?: SnapshotNode[]; } + | { kind: 'absent'; waitedMs: number } | { kind: 'stable'; waitedMs: number; @@ -116,7 +73,7 @@ export type WaitForTextCommandOptions = WaitCommandContext & timeoutMs?: number | null; }; -type SelectorWaitRuntime = { +export type SelectorWaitRuntime = { backend: { platform: PublicPlatform; /** @@ -152,7 +109,7 @@ export type SelectorWaitOperations = { interactiveOnly?: boolean; includeHiddenContentHints?: boolean; }, - ) => Promise<{ snapshot: { nodes: SnapshotNode[] } }>; + ) => Promise; requireSnapshot: ( runtime: Runtime, requestedName: string | undefined, @@ -182,49 +139,54 @@ export function createSelectorWaitCommands( runtime: Runtime, options: WaitCommandOptions, ): Promise => { - if (options.target.kind === 'sleep') { - await sleepWithWaitCancellation(runtime, options, options.target.durationMs); - return { kind: 'sleep', waitedMs: options.target.durationMs }; - } - if (options.target.kind === 'ref') { - const capture = await operations.requireSnapshot(runtime, options.session); - const ref = normalizeRef(options.target.ref); - if (!ref) throw new AppError('INVALID_ARGS', `Invalid ref: ${options.target.ref}`); - const node = findNodeByRef(capture.snapshot.nodes, ref); - const text = node ? resolveRefLabel(node, capture.snapshot.nodes) : undefined; - if (!text) { - throw new AppError('COMMAND_FAILED', `Ref ${options.target.ref} not found or has no label`); - } - return await waitForText(operations, runtime, options, text, options.target.timeoutMs); - } - if (options.target.kind === 'selector') { - return await waitForSelector( - operations, - runtime, - options, - options.target.selector, - options.target.timeoutMs, - options.target.recordedLandmark, - ); - } - if (options.target.kind === 'stable') { - return await waitForStable( - operations, - runtime, - options, - options.target.quietMs, - options.target.timeoutMs, - ); + switch (options.target.kind) { + case 'sleep': + await sleepWithWaitCancellation(runtime, options, options.target.durationMs); + return { kind: 'sleep', waitedMs: options.target.durationMs }; + case 'ref': + return await waitForRef( + operations, + runtime, + options, + options.target.ref, + options.target.timeoutMs, + ); + case 'selector': + return await waitForSelector( + operations, + runtime, + options, + options.target.selector, + options.target.timeoutMs, + options.target.recordedLandmark, + ); + case 'absent': + return await waitForAbsent( + operations, + runtime, + options, + options.target.selector, + options.target.timeoutMs, + ); + case 'stable': + return await waitForStable( + operations, + runtime, + options, + options.target.quietMs, + options.target.timeoutMs, + ); + case 'text': + return await waitForTextTarget( + operations, + runtime, + options, + options.target.text, + options.target.timeoutMs, + ); + default: + return assertNever(options.target); } - const waitText = checkWaitText(options.target.text); - if (!waitText.ok) throw new AppError(waitText.code, waitText.message); - return await waitForText( - operations, - runtime, - options, - options.target.text, - options.target.timeoutMs, - ); }; const waitForTextCommand = async ( @@ -244,236 +206,34 @@ export function createSelectorWaitCommands( return { waitCommand, waitForTextCommand }; } -async function waitForSelector( +async function waitForRef( operations: SelectorWaitOperations, runtime: Runtime, options: WaitCommandOptions, - selectorExpression: string, + rawRef: string, timeoutMs: number | null | undefined, - recordedLandmark: TargetAnnotationV1 | undefined, -): Promise { - const polling = createWaitPolling(runtime, options, timeoutMs, SELECTOR_PIPELINE_POLICIES.wait); - const capturePolicy = deriveSelectorCapturePolicy(); - // ADR 0012 / #1349: the LAST poll whose capture matched the recorded - // selector without any match carrying the recorded landmark identity. A - // transient same-selector impostor (the previous screen mid-transition) - // must not abort a wait whose job is to wait through it, so the loop keeps - // polling; only the deadline turns this into the fail-closed refusal. - let landmarkMismatch: WaitLandmarkMismatchEvidence | undefined; - let deadline: WaitPollDeadline | undefined; - while (polling.hasTimeRemaining()) { - // Presence-only poll: skip scroll-hint derivation (#1270), same as waitForFindMatch. - const poll = await polling.capture( - async (signal) => - await operations.captureSnapshot( - runtime, - { ...options, signal }, - { - updateSession: true, - includeHiddenContentHints: false, - ...capturePolicy, - }, - ), - ); - if (poll.timedOut) { - deadline = poll.deadline; - break; - } - const capture = poll.value; - if (capture) { - const nodes = capture.snapshot.nodes; - // The wait row ignores occlusion and off-screen: a covered or scrolled-out - // element is present, and presence is the question this loop asks. - const outcome = await resolveSelectorPipeline( - SELECTOR_PIPELINE_POLICIES.wait, - nodes, - selectorExpression, - { platform: runtime.backend.platform }, - ); - // The wait row is `first-match`, so a multi-match screen resolves rather - // than refusing; the landmark check below is what decides satisfaction. - const matchList = policyMatchList(outcome); - if (matchList) { - const landmark = resolveLandmarkMatch(nodes, matchList, recordedLandmark); - if (landmark.kind === 'satisfied') { - return { - kind: 'selector', - selector: matchList.selector, - waitedMs: polling.waitedMs(), - node: landmark.node, - preActionNodes: nodes, - }; - } - landmarkMismatch = landmark.evidence; - } - } - await polling.sleepUntilNextPoll(); - } - if (deadline !== 'capture-stalled' && landmarkMismatch) { - throw new AppError( - 'COMMAND_FAILED', - `wait matched selector ${selectorExpression} but no candidate carried the recorded landmark identity`, - { reason: WAIT_LANDMARK_MISMATCH_REASON, ...landmarkMismatch }, - ); - } - throw waitTimeoutError(`wait timed out for selector: ${selectorExpression}`, polling, deadline); -} - -type LandmarkMatchOutcome = - | { kind: 'satisfied'; node: SnapshotNode } - | { kind: 'identity-mismatch'; evidence: WaitLandmarkMismatchEvidence }; - -/** - * #1349 landmark check: the wait is satisfied when SOME selector match - * carries the recorded identity (local identity + leaf-anchored ancestry - * prefix). Positional disambiguation signals are deliberately not consulted — - * a destination guard proves the landmark exists on the ready screen, not - * that it kept its list position. - */ -function resolveLandmarkMatch( - nodes: SnapshotNode[], - matchList: SelectorChainMatchList, - recorded: TargetAnnotationV1 | undefined, -): LandmarkMatchOutcome { - const firstMatch = matchList.matchedNodes[0]!; - if (!recorded) return { kind: 'satisfied', node: firstMatch }; - const byIndex = buildIndexMap(nodes); - const identitySet = filterIdentitySet( - matchList.matchedNodes, - byIndex, - annotationLocalIdentity(recorded), - recorded.ancestry, - ); - const member = identitySet[0]; - if (member) return { kind: 'satisfied', node: member }; - return { - kind: 'identity-mismatch', - evidence: { - matchCount: matchList.matchedNodes.length, - observed: readNodeLocalIdentity(firstMatch), - observedAncestry: buildAncestryChain( - firstMatch, - byIndex, - Math.max(recorded.ancestry.length, 1), - ).chain, - }, - }; +): Promise> { + const capture = await operations.requireSnapshot(runtime, options.session); + const ref = normalizeRef(rawRef); + if (!ref) throw new AppError('INVALID_ARGS', `Invalid ref: ${rawRef}`); + const node = findNodeByRef(capture.snapshot.nodes, ref); + const text = node ? resolveRefLabel(node, capture.snapshot.nodes) : undefined; + if (!text) throw new AppError('COMMAND_FAILED', `Ref ${rawRef} not found or has no label`); + return await waitForText(operations, runtime, options, text, timeoutMs); } -async function waitForText( +async function waitForTextTarget( operations: SelectorWaitOperations, runtime: Runtime, options: WaitCommandOptions, text: string, timeoutMs: number | null | undefined, -): Promise { - const polling = createWaitPolling(runtime, options, timeoutMs, SELECTOR_PIPELINE_POLICIES.wait); - let deadline: WaitPollDeadline | undefined; - while (polling.hasTimeRemaining()) { - const poll = await polling.capture( - async (signal) => await observeText(operations, runtime, { ...options, signal }, text), - ); - if (poll.timedOut) { - deadline = poll.deadline; - break; - } - const found = poll.value; - if (found) return { kind: 'text', text, waitedMs: polling.waitedMs() }; - await polling.sleepUntilNextPoll(); - } - throw waitTimeoutError(`wait timed out for text: ${text}`, polling, deadline); -} - -/** - * One poll's answer to "is this text on screen", from two sources with deliberately asymmetric - * authority: - * - * 1. the owner's native reading, when the bound runtime advertised it — a `true` here short- - * circuits the poll and skips the capture entirely, which is the whole benefit of the - * preferred operation; and - * 2. the canonical tree, always consulted when (1) did not answer `true`. - * - * So the fast path can only ever make a satisfied wait return sooner. It can never make a wait - * that the tree would satisfy fail, and it is never the reason a wait times out — the required - * tree path below it is complete on its own. - */ -async function observeText( - operations: SelectorWaitOperations, - runtime: Runtime, - options: WaitCommandOptions, - text: string, -): Promise { - if (runtime.backend.findText) { - const native = await runtime.backend.findText(backendContext(runtime, options), text); - if (native.found) return true; - } - return await snapshotContainsText(operations, runtime, options, text); -} - -async function snapshotContainsText( - operations: SelectorWaitOperations, - runtime: Runtime, - options: WaitCommandOptions, - text: string, -): Promise { - // Presence-only poll: skip scroll-hint derivation (#1270), same as waitForFindMatch. - const capture = await operations.captureSnapshot(runtime, options, { - updateSession: true, - includeHiddenContentHints: false, - }); - return Boolean(findNodeByLabel(capture.snapshot.nodes, text)); +): Promise> { + const waitText = checkWaitText(text); + if (!waitText.ok) throw new AppError(waitText.code, waitText.message); + return await waitForText(operations, runtime, options, waitText.text, timeoutMs); } -// The quiet-window loop itself lives in stable-capture.ts and is shared with -// the interaction `--settle` flag (#1101); this wrapper maps the loop outcome -// to wait's throwing semantics. -async function waitForStable( - operations: SelectorWaitOperations, - runtime: Runtime, - options: WaitCommandOptions, - quietMs: number | null | undefined, - timeoutMs: number | null | undefined, -): Promise> { - const timeout = timeoutMs ?? DEFAULT_WAIT_TIMEOUT_MS; - const quiet = quietMs ?? operations.stable.defaultQuietMs; - const outcome = await operations.stable.capture(runtime, options, { - quietMs: quiet, - timeoutMs: timeout, - }); - if (!outcome.settled) { - throw new AppError('COMMAND_FAILED', 'wait timed out waiting for a stable UI', { - reason: WAIT_REASONS.stableTimeout, - ...(outcome.stalled ? { captureStalled: true } : {}), - quietMs: quiet, - timeoutMs: timeout, - captures: outcome.captures, - nodeCount: outcome.nodeCount, - ...(outcome.stalled - ? { - hint: 'A snapshot capture stalled past the wait timeout, so no settle verdict is available. The UI may still be readable: retry, or use screenshot to inspect the surface.', - } - : {}), - }); - } - return { - kind: 'stable', - waitedMs: outcome.waitedMs, - captures: outcome.captures, - nodeCount: outcome.nodeCount, - ...(outcome.nodeCount < operations.stable.tinyTreeNodeCount - ? { hint: operations.stable.tinyTreeHint } - : {}), - }; -} - -function backendContext( - runtime: SelectorWaitRuntime, - options: WaitCommandContext, -): WaitCommandContext { - return { - session: options.session, - requestId: options.requestId, - signal: options.signal ?? runtime.signal, - metadata: options.metadata, - }; +function assertNever(value: never): never { + throw new Error(`Unsupported wait target: ${String(value)}`); } diff --git a/src/commands/interaction/runtime/wait-absent.test.ts b/src/commands/interaction/runtime/wait-absent.test.ts new file mode 100644 index 000000000..90ed4fba8 --- /dev/null +++ b/src/commands/interaction/runtime/wait-absent.test.ts @@ -0,0 +1,206 @@ +import assert from 'node:assert/strict'; +import { test } from 'vitest'; +import type { AgentDeviceBackend, BackendSnapshotResult } from '../../../backend.ts'; +import { createLocalArtifactAdapter } from '../../../io.ts'; +import { + createAgentDevice, + createMemorySessionStore, + localCommandPolicy, +} from '../../../runtime.ts'; +import { makeSnapshotState } from '../../../__tests__/test-utils/snapshot-builders.ts'; +import { createFakeClock } from './__tests__/test-utils/index.ts'; +import { AppError } from '@agent-device/kernel/errors'; + +const SELECTOR = 'label="Removed"'; + +function snapshot( + label?: string, + overrides?: Parameters[1], +): BackendSnapshotResult { + return { + snapshot: makeSnapshotState( + label === undefined + ? [] + : [{ index: 0, depth: 0, type: 'Button', identifier: 'removed', label }], + overrides, + ), + }; +} + +function unreadableCapture(): AppError { + return new AppError('COMMAND_FAILED', 'Android helper returned no readable app content.', { + androidSnapshotHelperFailureReason: 'empty-helper-output', + }); +} + +function absentDevice( + captures: Array, + platform: AgentDeviceBackend['platform'] = 'ios', +) { + let index = 0; + return createAgentDevice({ + backend: { + platform, + captureSnapshot: async () => { + const capture = captures[Math.min(index++, captures.length - 1)]; + if (capture instanceof AppError) throw capture; + return capture!; + }, + } satisfies AgentDeviceBackend, + artifacts: createLocalArtifactAdapter(), + sessions: createMemorySessionStore([{ name: 'default' }]), + policy: localCommandPolicy(), + clock: createFakeClock(), + }); +} + +async function waitAbsent(device: ReturnType, timeoutMs = 1000) { + return await device.selectors.wait({ + session: 'default', + target: { kind: 'absent', selector: SELECTOR, timeoutMs }, + }); +} + +test('wait absent succeeds immediately when the selector has zero matches', async () => { + const result = await waitAbsent(absentDevice([snapshot()])); + + assert.deepEqual(result, { kind: 'absent', waitedMs: 0 }); +}); + +test('wait absent polls until a present selector disappears', async () => { + let captures = 0; + const device = absentDevice([snapshot('Removed'), snapshot()]); + const originalCapture = device.backend.captureSnapshot; + device.backend.captureSnapshot = async (...args) => { + captures += 1; + if (!originalCapture) throw new Error('the test device must expose captureSnapshot'); + return await originalCapture(...args); + }; + + const result = await waitAbsent(device, 2000); + + assert.equal(result.kind, 'absent'); + assert.equal(captures, 2); + assert.equal(result.waitedMs >= 300, true); +}); + +test('wait absent reports stable first-match evidence when one match remains at the deadline', async () => { + const device = absentDevice([snapshot('Removed')]); + + await assert.rejects(waitAbsent(device, 500), (error: unknown) => { + assert.ok(error instanceof AppError); + assert.equal(error.details?.reason, 'wait_target_present'); + assert.equal(error.details?.timeoutMs, 500); + assert.equal(typeof error.details?.waitedMs, 'number'); + assert.equal(error.details?.matches, 1); + assert.deepEqual(error.details?.firstMatch, { + id: 'removed', + role: 'button', + label: 'Removed', + }); + assert.equal(error.details?.readableCaptures, 2); + return true; + }); +}); + +test('wait absent reports the complete match count at the deadline', async () => { + const device = absentDevice([ + { + snapshot: makeSnapshotState([ + { index: 0, depth: 0, type: 'Button', identifier: 'one', label: 'Removed' }, + { index: 1, depth: 0, type: 'Button', identifier: 'two', label: 'Removed' }, + ]), + }, + ]); + + await assert.rejects(waitAbsent(device, 500), (error: unknown) => { + assert.ok(error instanceof AppError); + assert.equal(error.details?.reason, 'wait_target_present'); + assert.equal(error.details?.matches, 2); + assert.equal((error.details?.firstMatch as { id?: string })?.id, 'one'); + assert.equal(error.details?.visibility, undefined); + return true; + }); +}); + +test('wait absent rides out sparse and truncated captures without counting them as readable', async () => { + const sparse = snapshot(undefined, { + snapshotQuality: { + state: 'sparse', + backend: 'private-ax', + reason: 'sparse tree', + reasonCode: 'sparse-tree', + }, + }); + const truncated = { snapshot: makeSnapshotState([]), truncated: true }; + const device = absentDevice([sparse, truncated, snapshot()]); + + const result = await waitAbsent(device, 2000); + + assert.equal(result.kind, 'absent'); + assert.equal(result.waitedMs >= 600, true); +}); + +test('wait absent excludes sparse and truncated polls from deadline readable-capture evidence', async () => { + const sparse = snapshot(undefined, { + snapshotQuality: { state: 'sparse', backend: 'private-ax', reasonCode: 'sparse-tree' }, + }); + const truncated = { snapshot: makeSnapshotState([]), truncated: true }; + const device = absentDevice([snapshot('Removed'), sparse, truncated]); + + await assert.rejects(waitAbsent(device, 500), (error: unknown) => { + assert.ok(error instanceof AppError); + assert.equal(error.details?.reason, 'wait_target_present'); + assert.equal(error.details?.readableCaptures, 1); + assert.equal(error.details?.matches, 1); + return true; + }); +}); + +test('wait absent rides out Android unreadable content and does not use it as absence', async () => { + const device = absentDevice([unreadableCapture(), snapshot()], 'android'); + + const result = await waitAbsent(device, 2000); + + assert.equal(result.kind, 'absent'); + assert.equal(result.waitedMs >= 300, true); +}); + +test('wait absent preserves the final typed diagnostic when no valid capture arrives', async () => { + const sparse = snapshot(undefined, { + snapshotQuality: { state: 'sparse', backend: 'private-ax', reasonCode: 'sparse-tree' }, + }); + const device = absentDevice([sparse]); + + await assert.rejects(waitAbsent(device, 500), (error: unknown) => { + assert.ok(error instanceof AppError); + assert.equal(error.details?.observation, 'sparse'); + assert.equal(error.details?.reason, 'predicate_failed'); + assert.equal(error.details?.readableCaptures, undefined); + return true; + }); +}); + +test('wait absent preserves a final truncated diagnostic when no valid capture arrives', async () => { + const device = absentDevice([{ snapshot: makeSnapshotState([]), truncated: true }]); + + await assert.rejects(waitAbsent(device, 500), (error: unknown) => { + assert.ok(error instanceof AppError); + assert.equal(error.details?.observation, 'truncated'); + assert.equal(error.details?.reason, 'predicate_failed'); + assert.equal(error.details?.readableCaptures, undefined); + return true; + }); +}); + +test('wait absent preserves a final Android unreadable diagnostic when no valid capture arrives', async () => { + const device = absentDevice([unreadableCapture()], 'android'); + + await assert.rejects(waitAbsent(device, 500), (error: unknown) => { + assert.ok(error instanceof AppError); + assert.equal(error.details?.observation, 'unreadable'); + assert.equal(error.details?.captureErrorReason, 'empty-helper-output'); + assert.equal(error.details?.readableCaptures, undefined); + return true; + }); +}); diff --git a/src/commands/interaction/runtime/wait-absent.ts b/src/commands/interaction/runtime/wait-absent.ts new file mode 100644 index 000000000..c5ac59eff --- /dev/null +++ b/src/commands/interaction/runtime/wait-absent.ts @@ -0,0 +1,137 @@ +import { WAIT_REASONS } from '@agent-device/contracts/wait'; +import { isUnreadableCaptureContentError } from '@agent-device/contracts/android-snapshot-quality'; +import { AppError } from '@agent-device/kernel/errors'; +import { + absenceCaptureOptionRefusal, + type AbsenceObservation, +} from '../../../core/absence-observation.ts'; +import { + absenceCaptureOptionError, + absenceObservationError, + absenceUnreadableError, +} from '../../../core/absence-observation-errors.ts'; +import { SELECTOR_PIPELINE_POLICIES } from '../../../core/selector-pipeline-policy.ts'; +import { resolveAbsenceObservationState } from '../../../core/absence-observation-resolution.ts'; +import { deriveSelectorCapturePolicy } from './selector-capture-policy.ts'; +import type { + SelectorWaitOperations, + SelectorWaitRuntime, + WaitCommandOptions, + WaitCommandResult, +} from './selector-wait.ts'; +import { + createWaitPolling, + type WaitFailureEvidence, + waitTimeoutError, + type WaitPollDeadline, +} from './wait-polling.ts'; + +type PresentObservation = Extract; + +export async function waitForAbsent( + operations: SelectorWaitOperations, + runtime: Runtime, + options: WaitCommandOptions, + selectorExpression: string, + timeoutMs: number | null | undefined, +): Promise> { + const refusedOption = absenceCaptureOptionRefusal(options); + if (refusedOption) throw absenceCaptureOptionError(refusedOption, 'wait'); + + const polling = createWaitPolling(runtime, options, timeoutMs, SELECTOR_PIPELINE_POLICIES.wait, { + isUnreadableError: isWaitAbsentUnreadableError, + preserveUnreadableOnStall: true, + }); + const capturePolicy = deriveSelectorCapturePolicy(); + let present: PresentObservation | undefined; + let deadline: WaitPollDeadline | undefined; + + while (polling.hasTimeRemaining()) { + const poll = await polling.capture(async (signal) => { + try { + const capture = await operations.captureSnapshot( + runtime, + { ...options, signal }, + { + updateSession: true, + includeHiddenContentHints: false, + ...capturePolicy, + }, + ); + const { observation } = await resolveAbsenceObservationState( + capture.snapshot, + selectorExpression, + runtime.backend.platform, + ); + if (observation.kind === 'sparse' || observation.kind === 'truncated') { + throw absenceObservationError(selectorExpression, observation, 'wait'); + } + return observation; + } catch (error) { + if (isUnreadableCaptureContentError(error)) { + throw absenceUnreadableError(selectorExpression, error, 'wait'); + } + throw error; + } + }); + + if (poll.timedOut) { + deadline = poll.deadline; + break; + } + const observation = poll.value; + if (observation?.kind === 'absent') { + return { kind: 'absent', waitedMs: polling.waitedMs() }; + } + if (observation?.kind === 'present') present = observation; + await polling.sleepUntilNextPoll(); + } + + // A runner restart is the authoritative deadline cause even when an earlier + // readable poll saw the target. Returning stale target-present evidence would + // hide the retriable restart and make callers stop retrying the wrong reason. + if (deadline === 'runner-restart-exhausted') { + throw waitTimeoutError( + `wait absent timed out for selector: ${selectorExpression}`, + polling, + deadline, + ); + } + if (present) throw waitTargetPresentError(selectorExpression, present, polling.failureEvidence()); + throw waitTimeoutError( + `wait absent timed out for selector: ${selectorExpression}`, + polling, + deadline, + ); +} + +function waitTargetPresentError( + selector: string, + observation: PresentObservation, + evidence: WaitFailureEvidence, +): AppError { + const multiple = observation.matches !== 1; + return new AppError( + 'COMMAND_FAILED', + `wait absent timed out for selector ${selector}: ${observation.matches} match${multiple ? 'es' : ''} remain`, + { + reason: WAIT_REASONS.targetPresent, + selector, + ...evidence, + matches: observation.matches, + firstMatch: observation.firstMatch, + }, + ); +} + +function isWaitAbsentUnreadableError(error: unknown): boolean { + if (!(error instanceof AppError)) return false; + const details = error.details; + return ( + details?.command === 'wait' && + details.predicate === 'absent' && + (details.observation === 'sparse' || + details.observation === 'truncated' || + details.observation === 'unreadable') + ); +} diff --git a/src/commands/interaction/runtime/wait-polling.ts b/src/commands/interaction/runtime/wait-polling.ts index f86b9bc82..ca5738135 100644 --- a/src/commands/interaction/runtime/wait-polling.ts +++ b/src/commands/interaction/runtime/wait-polling.ts @@ -43,6 +43,11 @@ type WaitPollingOptions = { signal?: AbortSignal; }; +export type WaitPollingClassification = { + isUnreadableError?: (error: unknown) => boolean; + preserveUnreadableOnStall?: boolean; +}; + type UnreadablePollTracker = { attempt: (capture: () => Promise) => Promise; recordReadableCapture: () => void; @@ -52,6 +57,7 @@ type UnreadablePollTracker = { type WaitFailurePolling = { failureEvidence: () => WaitFailureEvidence; + preserveUnreadableOnStall?: boolean; rethrowIfNeverReadable: () => void; }; @@ -65,11 +71,12 @@ export function createWaitPolling( options: WaitPollingOptions, requestedTimeoutMs: number | null | undefined, policy: SelectorPipelinePolicy, + classification: WaitPollingClassification = {}, ) { const budget = selectorPollBudget(policy); const timeoutMs = requestedTimeoutMs ?? budget.defaultTimeoutMs; const startedAtMs = now(runtime); - const unreadable = createUnreadablePollTracker(); + const unreadable = createUnreadablePollTracker(classification.isUnreadableError); let timeoutEvidence: Partial = {}; const remainingMs = () => Math.max(0, timeoutMs - (now(runtime) - startedAtMs)); @@ -115,6 +122,7 @@ export function createWaitPolling( waitedMs: now(runtime) - startedAtMs, ...timeoutEvidence, }), + preserveUnreadableOnStall: classification.preserveUnreadableOnStall, rethrowIfNeverReadable: unreadable.rethrowIfNeverReadable, sleepUntilNextPoll: async () => await sleepWithWaitCancellation(runtime, options, Math.min(budget.intervalMs, remainingMs())), @@ -167,7 +175,10 @@ export function waitTimeoutError( if (deadline === 'runner-restart-exhausted') { return waitRunnerRestartExhaustedError(message, evidence); } - if (deadline === 'capture-stalled') return waitCaptureStalledError(message, evidence); + if (deadline === 'capture-stalled') { + if (polling.preserveUnreadableOnStall) polling.rethrowIfNeverReadable(); + return waitCaptureStalledError(message, evidence); + } if (deadline === 'capture-truncated') return waitDeadlineExceededError(message, evidence); polling.rethrowIfNeverReadable(); @@ -200,7 +211,9 @@ function copyStringDetail( return typeof value === 'string' ? ({ [key]: value } as Pick) : {}; } -function createUnreadablePollTracker(): UnreadablePollTracker { +function createUnreadablePollTracker( + isUnreadableError: (error: unknown) => boolean = isUnreadableCaptureContentError, +): UnreadablePollTracker { let readableCaptureCount = 0; let lastUnreadableError: unknown; return { @@ -208,7 +221,7 @@ function createUnreadablePollTracker(): UnreadablePollTracker { try { return await capture(); } catch (error) { - if (!isUnreadableCaptureContentError(error)) throw error; + if (!isUnreadableError(error)) throw error; lastUnreadableError = error; return undefined; } diff --git a/src/commands/interaction/runtime/selector-wait.test.ts b/src/commands/interaction/runtime/wait-selector.test.ts similarity index 88% rename from src/commands/interaction/runtime/selector-wait.test.ts rename to src/commands/interaction/runtime/wait-selector.test.ts index c710015c8..e3825354a 100644 --- a/src/commands/interaction/runtime/selector-wait.test.ts +++ b/src/commands/interaction/runtime/wait-selector.test.ts @@ -8,11 +8,7 @@ import { localCommandPolicy, } from '../../../runtime.ts'; import { makeSnapshotState } from '../../../__tests__/test-utils/snapshot-builders.ts'; -import { - createFakeClock, - createSelectorDevice, - selectorReadSnapshot, -} from './__tests__/test-utils/index.ts'; +import { createFakeClock } from './__tests__/test-utils/index.ts'; import { computeTargetEvidence } from '../../../daemon/session-target-evidence.ts'; import { WAIT_LANDMARK_MISMATCH_REASON } from '@agent-device/contracts/replay'; import { AppError } from '@agent-device/kernel/errors'; @@ -52,36 +48,9 @@ test('runtime focused selector waits against a full snapshot', async () => { assert.equal(captureOptions?.interactiveOnly, false); }); -// A text wait has exactly one source of truth: the polled capture. The backend `findText` seam -// that short-circuited it on Apple was wait's second platform-execution path and retired with -// wait's ADR 0019 cutover, so the tree answer is the only answer — in both directions. -test('runtime wait resolves text from the polled snapshot', async () => { - const device = createSelectorDevice(selectorReadSnapshot(), { now: 10 }); - - const result = await device.selectors.wait({ - session: 'default', - target: { kind: 'text', text: 'Continue', timeoutMs: 100 }, - }); - - assert.deepEqual(result, { kind: 'text', text: 'Continue', waitedMs: 0 }); -}); - -test('runtime wait times out on text the polled snapshot does not carry', async () => { - const device = createSelectorDevice(selectorReadSnapshot(), { clock: createFakeClock() }); - - await assert.rejects( - async () => - await device.selectors.wait({ - session: 'default', - target: { kind: 'text', text: 'Ready', timeoutMs: 100 }, - }), - (error: Error) => error.message.includes('wait timed out for text: Ready'), - ); -}); - // --------------------------------------------------------------------------- // #1349 (relocated from `selector-read.test.ts` — this is the 1:1 topology -// location for `selector-wait.ts`, and #1478 P5 step 2 cell 7's pin): +// location for `wait-selector.ts`, and #1478 P5 step 2 cell 7's pin): // wait's in-loop landmark identity verification, threaded as // `target.recordedLandmark`. Polling semantics are preserved — a // same-selector impostor never aborts the wait; only the deadline turns diff --git a/src/commands/interaction/runtime/wait-selector.ts b/src/commands/interaction/runtime/wait-selector.ts new file mode 100644 index 000000000..1b146eb14 --- /dev/null +++ b/src/commands/interaction/runtime/wait-selector.ts @@ -0,0 +1,170 @@ +import { AppError } from '@agent-device/kernel/errors'; +import type { SnapshotNode } from '@agent-device/kernel/snapshot'; +import { + annotationLocalIdentity, + buildAncestryChain, + buildIndexMap, + filterIdentitySet, + readNodeLocalIdentity, +} from '@agent-device/ad-script'; +import { + WAIT_LANDMARK_MISMATCH_REASON, + type TargetAnnotationV1, + type WaitLandmarkMismatchEvidence, +} from '@agent-device/contracts/replay'; +import type { SelectorChainMatchList } from '@agent-device/selectors'; +import { + resolveSelectorPipeline, + type SelectorPipelineOutcome, +} from '../../../core/selector-pipeline.ts'; +import { SELECTOR_PIPELINE_POLICIES } from '../../../core/selector-pipeline-policy.ts'; +import { deriveSelectorCapturePolicy } from './selector-capture-policy.ts'; +import type { + SelectorWaitOperations, + SelectorWaitRuntime, + WaitCommandOptions, + WaitCommandResult, +} from './selector-wait.ts'; +import { createWaitPolling, type WaitPollDeadline, waitTimeoutError } from './wait-polling.ts'; + +/** + * The landmark check (#1349) needs the full candidate set, which the pipeline + * outcome carries in either shape. Wait's row never refuses, so this only ever + * adapts — it does not re-decide anything. + */ +function policyMatchList(outcome: SelectorPipelineOutcome): SelectorChainMatchList | undefined { + if (outcome.kind === 'ambiguous') { + return { + selector: outcome.selector, + selectorIndex: outcome.selectorIndex, + matchedNodes: outcome.matchedNodes, + }; + } + if (outcome.kind === 'target') { + return { + selector: outcome.selector, + selectorIndex: outcome.selectorIndex, + // Every candidate, not just the winner: the landmark check is satisfied + // when SOME match carries the recorded identity, so a first impostor + // must not hide a later genuine landmark (#1349). + matchedNodes: outcome.matchedNodes, + }; + } + return undefined; +} + +export async function waitForSelector( + operations: SelectorWaitOperations, + runtime: Runtime, + options: WaitCommandOptions, + selectorExpression: string, + timeoutMs: number | null | undefined, + recordedLandmark: TargetAnnotationV1 | undefined, +): Promise> { + const polling = createWaitPolling(runtime, options, timeoutMs, SELECTOR_PIPELINE_POLICIES.wait); + const capturePolicy = deriveSelectorCapturePolicy(); + // ADR 0012 / #1349: the LAST poll whose capture matched the recorded + // selector without any match carrying the recorded landmark identity. A + // transient same-selector impostor (the previous screen mid-transition) + // must not abort a wait whose job is to wait through it, so the loop keeps + // polling; only the deadline turns this into the fail-closed refusal. + let landmarkMismatch: WaitLandmarkMismatchEvidence | undefined; + let deadline: WaitPollDeadline | undefined; + while (polling.hasTimeRemaining()) { + // Presence-only poll: skip scroll-hint derivation (#1270), same as waitForFindMatch. + const poll = await polling.capture( + async (signal) => + await operations.captureSnapshot( + runtime, + { ...options, signal }, + { + updateSession: true, + includeHiddenContentHints: false, + ...capturePolicy, + }, + ), + ); + if (poll.timedOut) { + deadline = poll.deadline; + break; + } + const capture = poll.value; + if (capture) { + const nodes = capture.snapshot.nodes; + // The wait row ignores occlusion and off-screen: a covered or scrolled-out + // element is present, and presence is the question this loop asks. + const outcome = await resolveSelectorPipeline( + SELECTOR_PIPELINE_POLICIES.wait, + nodes, + selectorExpression, + { platform: runtime.backend.platform }, + ); + // The wait row is `first-match`, so a multi-match screen resolves rather + // than refusing; the landmark check below is what decides satisfaction. + const matchList = policyMatchList(outcome); + if (matchList) { + const landmark = resolveLandmarkMatch(nodes, matchList, recordedLandmark); + if (landmark.kind === 'satisfied') { + return { + kind: 'selector', + selector: matchList.selector, + waitedMs: polling.waitedMs(), + node: landmark.node, + preActionNodes: nodes, + }; + } + landmarkMismatch = landmark.evidence; + } + } + await polling.sleepUntilNextPoll(); + } + if (deadline !== 'capture-stalled' && landmarkMismatch) { + throw new AppError( + 'COMMAND_FAILED', + `wait matched selector ${selectorExpression} but no candidate carried the recorded landmark identity`, + { reason: WAIT_LANDMARK_MISMATCH_REASON, ...landmarkMismatch }, + ); + } + throw waitTimeoutError(`wait timed out for selector: ${selectorExpression}`, polling, deadline); +} + +type LandmarkMatchOutcome = + | { kind: 'satisfied'; node: SnapshotNode } + | { kind: 'identity-mismatch'; evidence: WaitLandmarkMismatchEvidence }; + +/** + * #1349 landmark check: the wait is satisfied when SOME selector match + * carries the recorded identity (local identity + leaf-anchored ancestry + * prefix). Positional disambiguation signals are deliberately not consulted — + * a destination guard proves the landmark exists on the ready screen, not + * that it kept its list position. + */ +function resolveLandmarkMatch( + nodes: SnapshotNode[], + matchList: SelectorChainMatchList, + recorded: TargetAnnotationV1 | undefined, +): LandmarkMatchOutcome { + const firstMatch = matchList.matchedNodes[0]!; + if (!recorded) return { kind: 'satisfied', node: firstMatch }; + const byIndex = buildIndexMap(nodes); + const identitySet = filterIdentitySet( + matchList.matchedNodes, + byIndex, + annotationLocalIdentity(recorded), + recorded.ancestry, + ); + const member = identitySet[0]; + if (member) return { kind: 'satisfied', node: member }; + return { + kind: 'identity-mismatch', + evidence: { + matchCount: matchList.matchedNodes.length, + observed: readNodeLocalIdentity(firstMatch), + observedAncestry: buildAncestryChain( + firstMatch, + byIndex, + Math.max(recorded.ancestry.length, 1), + ).chain, + }, + }; +} diff --git a/src/commands/interaction/runtime/selector-wait-stable.test.ts b/src/commands/interaction/runtime/wait-stable.test.ts similarity index 100% rename from src/commands/interaction/runtime/selector-wait-stable.test.ts rename to src/commands/interaction/runtime/wait-stable.test.ts diff --git a/src/commands/interaction/runtime/wait-stable.ts b/src/commands/interaction/runtime/wait-stable.ts new file mode 100644 index 000000000..15240a792 --- /dev/null +++ b/src/commands/interaction/runtime/wait-stable.ts @@ -0,0 +1,48 @@ +import { AppError } from '@agent-device/kernel/errors'; +import { WAIT_REASONS } from '@agent-device/contracts/wait'; +import type { + SelectorWaitOperations, + SelectorWaitRuntime, + WaitCommandOptions, + WaitCommandResult, +} from './selector-wait.ts'; +import { DEFAULT_WAIT_TIMEOUT_MS } from './wait-polling.ts'; + +export async function waitForStable( + operations: SelectorWaitOperations, + runtime: Runtime, + options: WaitCommandOptions, + quietMs: number | null | undefined, + timeoutMs: number | null | undefined, +): Promise> { + const timeout = timeoutMs ?? DEFAULT_WAIT_TIMEOUT_MS; + const quiet = quietMs ?? operations.stable.defaultQuietMs; + const outcome = await operations.stable.capture(runtime, options, { + quietMs: quiet, + timeoutMs: timeout, + }); + if (!outcome.settled) { + throw new AppError('COMMAND_FAILED', 'wait timed out waiting for a stable UI', { + reason: WAIT_REASONS.stableTimeout, + ...(outcome.stalled ? { captureStalled: true } : {}), + quietMs: quiet, + timeoutMs: timeout, + captures: outcome.captures, + nodeCount: outcome.nodeCount, + ...(outcome.stalled + ? { + hint: 'A snapshot capture stalled past the wait timeout, so no settle verdict is available. The UI may still be readable: retry, or use screenshot to inspect the surface.', + } + : {}), + }); + } + return { + kind: 'stable', + waitedMs: outcome.waitedMs, + captures: outcome.captures, + nodeCount: outcome.nodeCount, + ...(outcome.nodeCount < operations.stable.tinyTreeNodeCount + ? { hint: operations.stable.tinyTreeHint } + : {}), + }; +} diff --git a/src/commands/interaction/runtime/wait-text.test.ts b/src/commands/interaction/runtime/wait-text.test.ts new file mode 100644 index 000000000..bbfc7b2b1 --- /dev/null +++ b/src/commands/interaction/runtime/wait-text.test.ts @@ -0,0 +1,34 @@ +import assert from 'node:assert/strict'; +import { test } from 'vitest'; +import { + createFakeClock, + createSelectorDevice, + selectorReadSnapshot, +} from './__tests__/test-utils/index.ts'; + +// A text wait has exactly one source of truth: the polled capture. The backend `findText` seam +// that short-circuited it on Apple was wait's second platform-execution path and retired with +// wait's ADR 0019 cutover, so the tree answer is the only answer — in both directions. +test('runtime wait resolves text from the polled snapshot', async () => { + const device = createSelectorDevice(selectorReadSnapshot(), { now: 10 }); + + const result = await device.selectors.wait({ + session: 'default', + target: { kind: 'text', text: 'Continue', timeoutMs: 100 }, + }); + + assert.deepEqual(result, { kind: 'text', text: 'Continue', waitedMs: 0 }); +}); + +test('runtime wait times out on text the polled snapshot does not carry', async () => { + const device = createSelectorDevice(selectorReadSnapshot(), { clock: createFakeClock() }); + + await assert.rejects( + async () => + await device.selectors.wait({ + session: 'default', + target: { kind: 'text', text: 'Ready', timeoutMs: 100 }, + }), + (error: Error) => error.message.includes('wait timed out for text: Ready'), + ); +}); diff --git a/src/commands/interaction/runtime/wait-text.ts b/src/commands/interaction/runtime/wait-text.ts new file mode 100644 index 000000000..313730700 --- /dev/null +++ b/src/commands/interaction/runtime/wait-text.ts @@ -0,0 +1,73 @@ +import type { + SelectorWaitOperations, + SelectorWaitRuntime, + WaitCommandContext, + WaitCommandOptions, + WaitCommandResult, +} from './selector-wait.ts'; +import { findNodeByLabel } from './selector-read-utils.ts'; +import { SELECTOR_PIPELINE_POLICIES } from '../../../core/selector-pipeline-policy.ts'; +import { createWaitPolling, type WaitPollDeadline, waitTimeoutError } from './wait-polling.ts'; + +export async function waitForText( + operations: SelectorWaitOperations, + runtime: Runtime, + options: WaitCommandOptions, + text: string, + timeoutMs: number | null | undefined, +): Promise> { + const polling = createWaitPolling(runtime, options, timeoutMs, SELECTOR_PIPELINE_POLICIES.wait); + let deadline: WaitPollDeadline | undefined; + while (polling.hasTimeRemaining()) { + const poll = await polling.capture( + async (signal) => await observeText(operations, runtime, { ...options, signal }, text), + ); + if (poll.timedOut) { + deadline = poll.deadline; + break; + } + const found = poll.value; + if (found) return { kind: 'text', text, waitedMs: polling.waitedMs() }; + await polling.sleepUntilNextPoll(); + } + throw waitTimeoutError(`wait timed out for text: ${text}`, polling, deadline); +} + +async function observeText( + operations: SelectorWaitOperations, + runtime: Runtime, + options: WaitCommandOptions, + text: string, +): Promise { + if (runtime.backend.findText) { + const native = await runtime.backend.findText(backendContext(runtime, options), text); + if (native.found) return true; + } + return await snapshotContainsText(operations, runtime, options, text); +} + +async function snapshotContainsText( + operations: SelectorWaitOperations, + runtime: Runtime, + options: WaitCommandOptions, + text: string, +): Promise { + // Presence-only poll: skip scroll-hint derivation (#1270), same as waitForFindMatch. + const capture = await operations.captureSnapshot(runtime, options, { + updateSession: true, + includeHiddenContentHints: false, + }); + return Boolean(findNodeByLabel(capture.snapshot.nodes, text)); +} + +function backendContext( + runtime: SelectorWaitRuntime, + options: WaitCommandContext, +): WaitCommandContext { + return { + session: options.session, + requestId: options.requestId, + signal: options.signal ?? runtime.signal, + metadata: options.metadata, + }; +} diff --git a/src/core/absence-observation-errors.ts b/src/core/absence-observation-errors.ts index acc6b0fd5..bfa2e0b20 100644 --- a/src/core/absence-observation-errors.ts +++ b/src/core/absence-observation-errors.ts @@ -6,9 +6,12 @@ import { type AbsenceObservation, } from './absence-observation.ts'; -export function absenceCaptureOptionError(option: AbsenceCaptureOption): AppError { - return new AppError('INVALID_ARGS', absenceCaptureOptionMessage(option), { - command: 'is', +export function absenceCaptureOptionError( + option: AbsenceCaptureOption, + command: 'is' | 'wait' = 'is', +): AppError { + return new AppError('INVALID_ARGS', absenceCaptureOptionMessage(option, command), { + command, predicate: 'absent', rejectedOption: option, }); @@ -17,10 +20,11 @@ export function absenceCaptureOptionError(option: AbsenceCaptureOption): AppErro export function absenceObservationError( selector: string, observation: AbsenceObservation, + command: 'is' | 'wait' = 'is', ): AppError { const firstMatch = 'firstMatch' in observation ? observation.firstMatch : undefined; const details = { - command: 'is', + command, reason: INTERACTION_ERROR_REASONS.predicateFailed, predicate: 'absent', selector, @@ -34,7 +38,7 @@ export function absenceObservationError( const multiple = observation.matches > 1; return new AppError( 'COMMAND_FAILED', - `is absent failed for selector ${selector}: ${observation.matches} match${multiple ? 'es' : ''} found`, + `${command} absent failed for selector ${selector}: ${observation.matches} match${multiple ? 'es' : ''} found`, { ...details, ...(multiple ? { hint: 'Refine the selector to match no elements.' } : {}), @@ -43,7 +47,7 @@ export function absenceObservationError( } return new AppError( 'COMMAND_FAILED', - `is absent could not prove absence for selector ${selector}: ${ + `${command} absent could not prove absence for selector ${selector}: ${ observation.kind === 'sparse' ? 'capture was sparse' : 'capture was truncated' }`, { @@ -53,22 +57,30 @@ export function absenceObservationError( ); } -export function absenceUnreadableError(selector: string, error: unknown): AppError { +export function absenceUnreadableError( + selector: string, + error: unknown, + command: 'is' | 'wait' = 'is', +): AppError { const cause = asAppError(error); + const captureErrorReason = + typeof cause.details?.reason === 'string' + ? cause.details.reason + : typeof cause.details?.androidSnapshotHelperFailureReason === 'string' + ? cause.details.androidSnapshotHelperFailureReason + : undefined; return new AppError( 'COMMAND_FAILED', - `is absent could not prove absence for selector ${selector}: capture was unreadable`, + `${command} absent could not prove absence for selector ${selector}: capture was unreadable`, { - command: 'is', + command, reason: INTERACTION_ERROR_REASONS.predicateFailed, predicate: 'absent', selector, matches: 0, observation: 'unreadable', captureErrorCode: cause.code, - ...(typeof cause.details?.reason === 'string' - ? { captureErrorReason: cause.details.reason } - : {}), + ...(captureErrorReason ? { captureErrorReason } : {}), hint: 'Retry after the accessibility capture is readable.', }, cause, diff --git a/src/core/absence-observation-resolution.ts b/src/core/absence-observation-resolution.ts index f8fc1aa1c..ea123ebce 100644 --- a/src/core/absence-observation-resolution.ts +++ b/src/core/absence-observation-resolution.ts @@ -2,7 +2,7 @@ import type { Platform, PublicPlatform } from '@agent-device/kernel/device'; import type { SnapshotState } from '@agent-device/kernel/snapshot'; import { resolveSelectorPipeline } from './selector-pipeline.ts'; import { SELECTOR_PIPELINE_POLICIES } from './selector-pipeline-policy.ts'; -import { classifyAbsenceObservation } from './absence-observation.ts'; +import { classifyAbsenceObservation, type AbsenceObservation } from './absence-observation.ts'; import { absenceObservationError } from './absence-observation-errors.ts'; export type AbsenceObservationResult = { @@ -12,11 +12,16 @@ export type AbsenceObservationResult = { matches: 0; }; -export async function resolveAbsenceObservation( +export type ResolvedAbsenceObservation = { + selector: string; + observation: AbsenceObservation; +}; + +export async function resolveAbsenceObservationState( snapshot: SnapshotState, selectorExpression: string, platform: Platform | PublicPlatform, -): Promise { +): Promise { const matched = await resolveSelectorPipeline( SELECTOR_PIPELINE_POLICIES.readAny, snapshot.nodes, @@ -29,14 +34,24 @@ export async function resolveAbsenceObservation( : matched.kind === 'occluded' ? [matched.node] : []; - const observation = classifyAbsenceObservation(snapshot, matches); - if (observation.kind !== 'absent') { - throw absenceObservationError( + return { + selector: matched.kind === 'target' || matched.kind === 'ambiguous' ? matched.selector : selectorExpression, - observation, - ); + observation: classifyAbsenceObservation(snapshot, matches), + }; +} + +export async function resolveAbsenceObservation( + snapshot: SnapshotState, + selectorExpression: string, + platform: Platform | PublicPlatform, +): Promise { + const resolved = await resolveAbsenceObservationState(snapshot, selectorExpression, platform); + const observation = resolved.observation; + if (observation.kind !== 'absent') { + throw absenceObservationError(resolved.selector, observation); } return { predicate: 'absent', diff --git a/src/core/absence-observation.ts b/src/core/absence-observation.ts index e3a5b0cc7..5f5839f58 100644 --- a/src/core/absence-observation.ts +++ b/src/core/absence-observation.ts @@ -45,10 +45,14 @@ export function absenceCaptureOptionRefusal(options: { return undefined; } -export function absenceCaptureOptionMessage(option: AbsenceCaptureOption): string { +export function absenceCaptureOptionMessage( + option: AbsenceCaptureOption, + command: 'is' | 'wait' = 'is', +): string { + const surface = `${command} absent`; return option === 'scope' - ? 'is absent does not support --scope; it requires an unscoped capture' - : 'is absent does not support --depth; it requires a full-depth capture'; + ? `${surface} does not support --scope; it requires an unscoped capture` + : `${surface} does not support --depth; it requires a full-depth capture`; } export function classifyAbsenceObservation( diff --git a/src/core/wait-positionals.test.ts b/src/core/wait-positionals.test.ts index c47dc2db8..0967dcf23 100644 --- a/src/core/wait-positionals.test.ts +++ b/src/core/wait-positionals.test.ts @@ -32,8 +32,23 @@ test('a condition word followed by a selector-shaped value is rejected, not read assertInvalid(['open', 'label="Open"', '25000'], 'label="Open"'); }); -test('"exists" followed by a selector-shaped value is rejected and points at the selector form', () => { - assertInvalid(['exists', 'label="x"', '100'], "wait 'visible"); +test('"exists" followed by a selector-shaped value is rejected and points at the plain selector form', () => { + assertInvalid(['exists', 'label="x"', '100'], `wait 'label="x"'`); +}); + +test.each(['gone', 'disappears'])( + '%s routes to strict wait absent without becoming an alias', + (word) => { + assertInvalid([word, 'label="x"', '100'], `wait absent 'label="x"'`); + }, +); + +test('wait absent parses a selector and optional timeout', () => { + assert.deepEqual(parseWaitPositionals(['absent', 'label="Removed"', '2500']), { + kind: 'absent', + selectorExpression: 'label="Removed"', + timeoutMs: 2500, + }); }); test('unknown selector key is rejected with the supported-key list, not read as text', () => { diff --git a/src/core/wait-positionals.ts b/src/core/wait-positionals.ts index 2508c3992..71777eda5 100644 --- a/src/core/wait-positionals.ts +++ b/src/core/wait-positionals.ts @@ -8,12 +8,16 @@ import { splitSelectorFromArgs, } from '@agent-device/selectors'; -export type WaitInvalidReason = 'unknown-selector-key' | 'selector-shaped-text'; +export type WaitInvalidReason = + | 'unknown-selector-key' + | 'selector-shaped-text' + | 'absent-selector-required'; export type WaitParsed = | { kind: 'sleep'; durationMs: number } | { kind: 'ref'; rawRef: string; timeoutMs: number | null } | { kind: 'selector'; selectorExpression: string; timeoutMs: number | null } + | { kind: 'absent'; selectorExpression: string; timeoutMs: number | null } | { kind: 'text'; text: string; timeoutMs: number | null } | { kind: 'stable'; quietMs: number | null; timeoutMs: number | null } | { kind: 'invalid'; reason: WaitInvalidReason; message: string }; @@ -31,6 +35,7 @@ export function parseWaitPositionals(args: string[]): WaitParsed | null { const timeoutMs = parseTimeout(args.at(-1)); if (firstArg === 'text') return parseTextKeyword(args, timeoutMs); if (firstArg === 'stable') return parseStableKeyword(args); + if (firstArg === 'absent') return parseAbsentKeyword(args, timeoutMs); if (firstArg.startsWith('@')) return { kind: 'ref', rawRef: firstArg, timeoutMs }; return parseSelectorOrText(args, timeoutMs); } @@ -47,6 +52,34 @@ function parseStableKeyword(args: string[]): WaitParsed { return { kind: 'stable', quietMs, timeoutMs: stableTimeoutMs }; } +function parseAbsentKeyword(args: string[], timeoutMs: number | null): WaitParsed { + const selectorArgs = timeoutMs !== null ? args.slice(1, -1) : args.slice(1); + const split = splitSelectorFromArgs(selectorArgs); + if ( + selectorArgs.length > 0 && + split?.rest.length === 0 && + isValidSelectorExpression(split.selectorExpression) + ) { + return { + kind: 'absent', + selectorExpression: split.selectorExpression, + timeoutMs, + }; + } + return { + kind: 'invalid', + reason: 'absent-selector-required', + message: formatAbsentSelectorMessage(selectorArgs), + }; +} + +function formatAbsentSelectorMessage(selectorArgs: string[]): string { + const raw = selectorArgs.join(' '); + return raw.length > 0 + ? `wait absent requires one valid selector; received "${raw}". Use wait absent '' [timeoutMs].` + : 'wait absent requires [timeoutMs].'; +} + /** * The fallback path once `wait` isn't a keyword form, a sleep, or a `@ref`: either a full selector * expression, or literal text. #1800: a token shaped like a selector (a recognized key, or an @@ -131,8 +164,9 @@ function formatSelectorShapedMessage(argsWithoutTimeout: string[]): string { const offending = plainTokens[0]!; const conditionHint = CONDITION_WORDS.has(offending.toLowerCase()) ? ` "${offending}" describes a condition, not a selector key. Use the selector form: ` + - `wait 'visible ${selectorTokens.join(' ')}' [timeoutMs] to wait for it to appear, or ` + - `wait 'hidden ${selectorTokens.join(' ')}' [timeoutMs] to wait for it to disappear.` + (offending.toLowerCase() === 'gone' || offending.toLowerCase() === 'disappears' + ? `wait absent '${selectorTokens.join(' ')}' [timeoutMs] to wait for zero matches.` + : `wait '${selectorTokens.join(' ')}' [timeoutMs] to wait for it to appear.`) : ` "${offending}" is not a recognized selector key (${SELECTOR_KEY_NAMES.join(', ')}).`; return ( `"${raw}" mixes plain word "${offending}" with selector-shaped "${selectorTokens.join(' ')}", ` + diff --git a/src/daemon/__tests__/selector-recording.test.ts b/src/daemon/__tests__/selector-recording.test.ts index 1bfc1d34a..320ac9b3d 100644 --- a/src/daemon/__tests__/selector-recording.test.ts +++ b/src/daemon/__tests__/selector-recording.test.ts @@ -24,6 +24,13 @@ function req(command: string, flags: DaemonRequest['flags'] = {}): DaemonRequest return { token: 't', session: 'default', command, positionals: [], flags }; } +function waitAbsentReq(): DaemonRequest { + return { + ...req('wait'), + positionals: ['absent', 'label="Removed"', '5000'], + }; +} + /** A request as the replay runtime dispatches it: authored provenance stamped on `internal`. */ function planStepReq(command: string, flags: DaemonRequest['flags'] = {}): DaemonRequest { return { ...req(command, flags), internal: { replayPlanStep: true } }; @@ -110,3 +117,17 @@ test('outside a repair-armed session, get/is/find/wait all record normally', () 'wait', ]); }); + +test('wait absent records positionals without target-v1 annotation', () => { + const store = makeStore(); + store.set('default', makeIosSession('default')); + + recordIfSession(store, 'default', waitAbsentReq(), { waitedMs: 0 }); + + expect(store.get('default')!.actions[0]).toMatchObject({ + command: 'wait', + positionals: ['absent', 'label="Removed"', '5000'], + result: { waitedMs: 0 }, + }); + expect(store.get('default')!.actions[0]?.targetEvidence).toBeUndefined(); +}); diff --git a/src/daemon/__tests__/session-script-active-publication.test.ts b/src/daemon/__tests__/session-script-active-publication.test.ts index 2c8be47a5..8da160fd1 100644 --- a/src/daemon/__tests__/session-script-active-publication.test.ts +++ b/src/daemon/__tests__/session-script-active-publication.test.ts @@ -53,6 +53,7 @@ describe('ADR 0016 active publication contract', () => { ['stable', ['stable']], ['ref', ['@e7']], ['text', ['text', 'Screen X']], + ['absent', ['absent', 'label="Screen X"']], ])('rejects %s wait as the destination guard', (_kind, waitPositionals) => { expect(() => validateActivePublicationActions([action('open', ['Demo']), action('wait', waitPositionals)]), diff --git a/src/daemon/__tests__/session-script-writer.test.ts b/src/daemon/__tests__/session-script-writer.test.ts index 3960455ba..2748d2f12 100644 --- a/src/daemon/__tests__/session-script-writer.test.ts +++ b/src/daemon/__tests__/session-script-writer.test.ts @@ -69,6 +69,18 @@ test('write() with no boundary set (ordinary open/close --save-script) serialize expect(parsed.actions[1]?.positionals).toEqual(['label="Save"']); }); +test('write() round-trips wait absent positionals without adding an annotation', () => { + const root = mkdtempForTestSync('agent-device-script-writer-wait-absent-'); + const writer = new SessionScriptWriter(path.join(root, 'sessions')); + const session = makeAuthoringSession('default', { + actions: [action({ command: 'wait', positionals: ['absent', 'label="Removed"', '2500'] })], + }); + + const { script, parsed } = writeAndParse(writer, session); + expect(script).not.toContain('target-v1'); + expect(parsed.actions[0]?.positionals).toEqual(['absent', 'label="Removed"', '2500']); +}); + test('a boundary-sliced script still strips diagnostic snapshot actions', () => { const root = mkdtempForTestSync('agent-device-script-writer-snapshot-strip-'); const writer = new SessionScriptWriter(path.join(root, 'sessions')); diff --git a/src/daemon/__tests__/system-surface-disclosure.test.ts b/src/daemon/__tests__/system-surface-disclosure.test.ts index 7204847e1..21641741f 100644 --- a/src/daemon/__tests__/system-surface-disclosure.test.ts +++ b/src/daemon/__tests__/system-surface-disclosure.test.ts @@ -2,7 +2,8 @@ import { test, expect, vi, beforeEach } from 'vitest'; import { legacyDispatchCapture } from './legacy-snapshot-capture-fixture.ts'; import { handleFindCommands } from '../interaction/index.ts'; import { getRuntimeBindings } from './interaction-get-runtime-fixture.ts'; -import { dispatchFindReadOnlyViaRuntime, dispatchWaitViaRuntime } from '../selector-runtime.ts'; +import { dispatchFindReadOnlyViaRuntime } from '../selector-runtime.ts'; +import { dispatchWaitViaRuntime } from '../wait-runtime.ts'; import type { DaemonRequest, DaemonResponse } from '../types.ts'; import { ANDROID_SYSTEM_SURFACE_DISCLOSURE } from '../../core/android-system-surface-disclosure.ts'; import { snapshotRuntimeFixture } from './snapshot-runtime-fixture.ts'; diff --git a/src/daemon/__tests__/wait-conditional-selector.test.ts b/src/daemon/__tests__/wait-conditional-selector.test.ts index f6fb86aba..a397f870e 100644 --- a/src/daemon/__tests__/wait-conditional-selector.test.ts +++ b/src/daemon/__tests__/wait-conditional-selector.test.ts @@ -75,12 +75,16 @@ function harness(options: { return { findSelector, captureSnapshot, inspectFacts, bindDevice }; } -async function run(session: SessionState, runtime: ReturnType) { +async function run( + session: SessionState, + runtime: ReturnType, + positionals: string[] = ['id=runner-only', '200'], +) { const sessionStore = makeSessionStore('wait-conditional-selector-'); sessionStore.set(session.name, session); const req = { command: 'wait', - positionals: ['id=runner-only', '200'], + positionals, token: 'token', session: session.name, flags: {}, @@ -165,3 +169,17 @@ test('recording and no-app waits retain capture-owned evidence semantics', async expect(runtime.captureSnapshot).toHaveBeenCalled(); } }); + +test('strict wait absent skips the positive-only native selector fast path', async () => { + const runtime = harness({ found: true, nodes: [] }); + const response = await run(makeIosAppSession('wait-conditional-selector'), runtime, [ + 'absent', + 'id=runner-only', + '200', + ]); + + expect(response).toMatchObject({ ok: true, data: { waitedMs: expect.any(Number) } }); + if (response.ok) expect(response.data).not.toHaveProperty('kind'); + expect(runtime.findSelector).not.toHaveBeenCalled(); + expect(runtime.captureSnapshot).toHaveBeenCalled(); +}); diff --git a/src/daemon/__tests__/wait-runtime.test.ts b/src/daemon/__tests__/wait-runtime.test.ts index b85080a2a..ea73b788d 100644 --- a/src/daemon/__tests__/wait-runtime.test.ts +++ b/src/daemon/__tests__/wait-runtime.test.ts @@ -267,6 +267,28 @@ test('a stable wait binds the capture use once and polls through the bound opera expect(harness.bindDevice).toHaveBeenCalledWith(harness.device, waitWithoutActiveAppUse); }); +test.each([ + ['scope', { snapshotScope: 'Root' }], + ['depth', { snapshotDepth: 2 }], +])('strict wait absent refuses --%s before device admission', async (_option, flags) => { + const harness = waitRuntimeHarness(); + + const { response } = await runWait(['absent', 'label="Ready"', '400'], harness, [], flags); + + expect(response).toMatchObject({ + ok: false, + error: { + code: 'INVALID_ARGS', + details: { command: 'wait', predicate: 'absent' }, + }, + }); + if (!response.ok) { + expect(response.error.details?.rejectedOption).toBe(_option); + } + expect(harness.inspectFacts).not.toHaveBeenCalled(); + expect(harness.bindDevice).not.toHaveBeenCalled(); +}); + // --------------------------------------------------------------------------- // Facts are the only support authority: an unavailable exact-owner fact // refuses BEFORE any binding, and provider ownership never borrows the local @@ -533,6 +555,18 @@ test('a stalled capture reports capture-stalled with no readable captures', asyn expect(response.error.details?.readableCaptures).toBe(0); }); +test('strict wait absent reports capture-stalled when every capture stalls', async () => { + const stalling = stallingCapture(); + const harness = waitRuntimeHarness({ captureSnapshot: stalling.captureSnapshot }); + + const { response } = await runWait(['absent', 'label="Ready"', '150'], harness); + + expect(response.ok).toBe(false); + if (response.ok) return; + expect(response.error.details?.reason).toBe(WAIT_REASONS.captureStalled); + expect(response.error.details?.readableCaptures).toBe(0); +}); + test('a runner restart that exhausts the wait reports typed restart evidence', async () => { const captureSnapshot = vi.fn(async (input: CaptureSnapshotInput) => { const signal = input.signal; @@ -574,6 +608,68 @@ test('a runner restart that exhausts the wait reports typed restart evidence', a expect(response.error.details?.captureStalled).toBeUndefined(); }); +test('strict wait absent preserves runner-restart exhaustion as the deadline reason', async () => { + const captureSnapshot = vi.fn(async (input: CaptureSnapshotInput) => { + const signal = input.signal; + if (!signal) throw new Error('the poll deadline never reached the platform'); + await new Promise((resolve) => { + if (signal.aborted) return resolve(); + signal.addEventListener('abort', () => resolve(), { once: true }); + }); + throw new AppError('COMMAND_FAILED', 'request canceled', { + runnerRestarted: true, + runnerRestartReason: 'runner_readiness_preflight_failed_before_command_send', + runnerRestartCommand: 'snapshot', + }); + }); + const harness = waitRuntimeHarness({ captureSnapshot }); + + const { response } = await runWait(['absent', 'label="Ready"', '50'], harness); + + expect(response.ok).toBe(false); + if (response.ok) return; + expect(response.error.details?.reason).toBe(WAIT_REASONS.runnerRestartExhausted); + expect(response.error.details?.readableCaptures).toBe(0); +}); + +test('strict wait absent does not mask a runner restart after an earlier present capture', async () => { + let poll = 0; + const captureSnapshot = vi.fn(async (input: CaptureSnapshotInput) => { + if (poll++ === 0) { + return { + nodes: [{ index: 0, depth: 0, type: 'Button', label: 'Ready', hittable: true }], + backend: 'web' as const, + producer: 'agent-browser' as const, + }; + } + const signal = input.signal; + if (!signal) throw new Error('the poll deadline never reached the platform'); + await new Promise((resolve) => { + if (signal.aborted) return resolve(); + signal.addEventListener('abort', () => resolve(), { once: true }); + }); + throw new AppError('COMMAND_FAILED', 'request canceled', { + runnerRestarted: true, + runnerRestartReason: 'runner_readiness_preflight_failed_before_command_send', + runnerRestartCommand: 'snapshot', + }); + }); + const harness = waitRuntimeHarness({ captureSnapshot }); + + const { response } = await runWait(['absent', 'label="Ready"', '800'], harness); + + expect(response.ok).toBe(false); + if (response.ok) return; + expect(response.error.details).toMatchObject({ + reason: WAIT_REASONS.runnerRestartExhausted, + waitRunnerRestartExhausted: true, + runnerRestarted: true, + retriable: true, + readableCaptures: 1, + }); + expect(response.error.details?.reason).not.toBe(WAIT_REASONS.targetPresent); +}); + test('a readable capture that lacks the target stays target-absent, not capture-stalled', async () => { const harness = waitRuntimeHarness({ nodesPerPoll: [[{ index: 0, depth: 0, type: 'Button', label: 'Checkout', hittable: true }]], diff --git a/src/daemon/handlers/__tests__/wait-landmark-recording.test.ts b/src/daemon/handlers/__tests__/wait-landmark-recording.test.ts index 8ddc00bd3..7c772b895 100644 --- a/src/daemon/handlers/__tests__/wait-landmark-recording.test.ts +++ b/src/daemon/handlers/__tests__/wait-landmark-recording.test.ts @@ -13,7 +13,7 @@ */ import { test, expect, vi, beforeEach } from 'vitest'; import { legacyDispatchCapture } from '../../__tests__/legacy-snapshot-capture-fixture.ts'; -import { dispatchWaitViaRuntime } from '../../selector-runtime.ts'; +import { dispatchWaitViaRuntime } from '../../wait-runtime.ts'; import type { DaemonRequest } from '../../types.ts'; import { WAIT_LANDMARK_MISMATCH_REASON } from '@agent-device/contracts/replay'; import type { TargetAnnotationV1 } from '@agent-device/contracts/replay'; diff --git a/src/daemon/handlers/snapshot.ts b/src/daemon/handlers/snapshot.ts index 98aa6c89e..aef3bb55a 100644 --- a/src/daemon/handlers/snapshot.ts +++ b/src/daemon/handlers/snapshot.ts @@ -5,7 +5,7 @@ import { handleAlertCommand } from './snapshot-alert.ts'; import { handleSettingsCommand, parseSettingsArgs } from './snapshot-settings.ts'; import { dispatchSnapshotDiffViaRuntime } from '../snapshot-diff-runtime.ts'; import { dispatchSnapshotViaRuntime } from '../snapshot-runtime.ts'; -import { dispatchWaitViaRuntime } from '../selector-runtime.ts'; +import { dispatchWaitViaRuntime } from '../wait-runtime.ts'; import { resolveSessionDevice, withSessionlessRunnerCleanup } from '../snapshot-session.ts'; import type { BindDeviceRuntime, InspectDeviceRuntimeFacts } from '../request-runtime-binding.ts'; import type { PlatformResourceCleanup } from '@agent-device/contracts/platform-resource-cleanup'; diff --git a/src/daemon/selector-runtime.ts b/src/daemon/selector-runtime.ts index f90ddf418..3f17c8b51 100644 --- a/src/daemon/selector-runtime.ts +++ b/src/daemon/selector-runtime.ts @@ -1,17 +1,14 @@ -import { waitObservesDevice } from '@agent-device/contracts/wait-runtime-plan'; -import { parseWaitPositionals } from '../core/wait-positionals.ts'; -import type { WaitParsed } from '../core/wait-positionals.ts'; -import { AppError, asAppError } from '@agent-device/kernel/errors'; +import { asAppError } from '@agent-device/kernel/errors'; import type { SnapshotNode } from '@agent-device/kernel/snapshot'; -import type { DaemonRequest, DaemonResponse, SessionState } from './types.ts'; +import { absenceCaptureOptionError } from '../core/absence-observation-errors.ts'; +import { absenceCaptureOptionRefusal } from '../core/absence-observation.ts'; +import type { DaemonRequest, DaemonResponse } from './types.ts'; import { errorResponse } from './response.ts'; import { markSessionPartialRefsIssued, resolveRefStalenessWarning } from './session-snapshot.ts'; -import { resolveSessionDevice, withSessionlessRunnerCleanup } from './snapshot-session.ts'; import { checkElementTargetArgs, checkGetFormat, checkIsArgs, - checkWaitText, checkFindArgs, isReadOnlyFindAction, } from '@agent-device/selectors'; @@ -29,25 +26,13 @@ import { stripSelectorChain, toDaemonFindData, toDaemonGetData, - toDaemonWaitData, } from './selector-recording.ts'; import type { RecordedTargetCapture } from './session-target-evidence.ts'; -import type { TargetAnnotationV1 } from '@agent-device/contracts/replay'; -import { maybeWaitTimeoutSurfaceResponse } from './wait-current-surface.ts'; import { withSystemSurfaceDisclosure } from './system-surface-disclosure.ts'; import { createBoundSelectorRuntime, - createSelectorRuntimeForDevice, type SelectorRuntimeParams, } from './selector-runtime-backend.ts'; -import type { BindDeviceRuntime, InspectDeviceRuntimeFacts } from './request-runtime-binding.ts'; -import { - resolveBoundSelectorCapture, - type BoundSelectorOperations, -} from './selector-capture-binding.ts'; -import { dispatchConditionalWaitSelector } from './wait-conditional-selector.ts'; -import { absenceCaptureOptionRefusal } from '../core/absence-observation.ts'; -import { absenceCaptureOptionError } from '../core/absence-observation-errors.ts'; export async function dispatchFindReadOnlyViaRuntime( params: SelectorRuntimeParams, @@ -119,7 +104,7 @@ export async function dispatchFindReadOnlyViaRuntime( return withSystemSurfaceDisclosure(response, consumedSessionSnapshot(params)); } -function consumedSessionSnapshot(params: SelectorRuntimeParams) { +export function consumedSessionSnapshot(params: SelectorRuntimeParams) { // The capture runtime reports the consumed snapshot directly; sessionless selector routes have // no session record, so the stored-session read is only a fallback for pre-captured snapshots. return params.consumedSnapshot?.state ?? params.sessionStore.get(params.sessionName)?.snapshot; @@ -252,116 +237,8 @@ export async function dispatchIsViaRuntime( ); } -export async function dispatchWaitViaRuntime( - params: SelectorRuntimeParams & - Readonly<{ inspectFacts?: InspectDeviceRuntimeFacts; bindDevice?: BindDeviceRuntime }>, -): Promise { - const { req, sessionName, sessionStore } = params; - const parsed = parseWaitPositionals(req.positionals ?? []); - if (!parsed) return errorResponse('INVALID_ARGS', 'wait requires a duration or text'); - if (parsed.kind === 'invalid') return errorResponse('INVALID_ARGS', parsed.message); - const { session, device } = await resolveSessionDevice(sessionStore, sessionName, req.flags); - // ADR 0019: facts are the only support authority, through the selector family's one - // admit-then-bind entry. A duration wait observes nothing, so it never asks for a binding — - // exactly the cell legacy admission skipped by testing `parsed.kind !== 'sleep'`. - let waitOperations: BoundSelectorOperations | undefined; - if (waitObservesDevice(parsed.kind)) { - const bound = await resolveBoundSelectorCapture({ - command: 'wait', - device, - session, - inspectFacts: params.inspectFacts, - bindDevice: params.bindDevice, - }); - if (!bound.ok) return bound.response; - waitOperations = bound.operations; - } - // ADR 0012 / #1349, replay-only: the recorded landmark identity this wait must observe. - const recordedLandmark = req.internal?.replayLandmarkGuard; - // #1076 + ADR 0014: a wait @ref names an element from the retained ref-frame - // evidence, and its staleness is frame-derived rather than a property of the - // live polling capture the condition is checked against. Once the ref frame - // has expired any ref gets the frame-derived warning, else a pinned `@e12~s3` - // ref whose epoch no longer matches gets the precise generation-mismatch - // warning. The pin is split off HERE so the runtime and recording only ever - // see the plain `@e12` form. - let waitParsed = parsed; - let staleRefsWarning: string | undefined; - if (parsed.kind === 'ref') { - const versionedRef = parseVersionedRefPositional(parsed.rawRef); - if (!versionedRef.ok) return versionedRef.response; - waitParsed = { ...parsed, rawRef: versionedRef.ref }; - staleRefsWarning = resolveRefStalenessWarning({ - session, - ref: versionedRef.ref, - mintedGeneration: versionedRef.generation, - }); - } - if (waitParsed.kind === 'selector') { - const conditionalResponse = await dispatchConditionalWaitSelector({ - selectorExpression: waitParsed.selectorExpression, - operation: waitOperations?.findSelector, - recordedLandmark, - req, - session, - sessionName, - sessionStore, - logPath: params.logPath, - signal: params.signal, - }); - if (conditionalResponse) return conditionalResponse; - } - // Wait builds its runtime directly (no createBoundSelectorRuntime), so the consumed-snapshot slot - // must be initialized here too or sessionless waits have nowhere to report the capture from. - params.consumedSnapshot ??= {}; - const execute = async () => { - const runtime = createSelectorRuntimeForDevice({ - ...params, - session, - device, - bound: waitOperations, - }); - const response = await toDaemonResponse(async () => { - const result = await runtime.selectors.wait({ - session: sessionName, - requestId: req.meta?.requestId, - target: toWaitTarget(waitParsed, session, recordedLandmark), - }); - const recordedTarget = readRecordedResolutionTarget(result); - recordIfSession( - sessionStore, - sessionName, - req, - stripResolutionPayload(result), - recordedTarget, - 'landmark', - ); - const data = toDaemonWaitData(result); - return staleRefsWarning ? { ...data, warning: staleRefsWarning } : data; - }); - // Only a polling wait can fail with `targetAbsent`/`stableTimeout`, and only it holds a - // capture binding to describe the surface with. A duration wait has neither. - const enrichedResponse = waitOperations - ? await maybeWaitTimeoutSurfaceResponse( - { req, logPath: params.logPath, session, device, capture: waitOperations.capture }, - response, - ) - : response; - // Keep generic wait-surface details first so Android blocker detection can own the top-level message. - return await maybeAndroidForegroundBlockerResponse(params, enrichedResponse, 'wait'); - }; - // A pure sleep consumes no capture, so it never earns the system-surface disclosure below. - if (parsed.kind === 'sleep') return await execute(); - // Both a satisfied wait and a timeout consumed the polled capture stored on the session: - // when it is an occluding system surface, the outcome must disclose the occlusion. - return withSystemSurfaceDisclosure( - await withSessionlessRunnerCleanup(session, device, execute, params.platformResourceCleanup), - consumedSessionSnapshot(params), - ); -} - /** ADR 0012 decision 3 / #1349: a wait/is result's resolution payload, when the tree path produced one. */ -function readRecordedResolutionTarget( +export function readRecordedResolutionTarget( result: Record, ): RecordedTargetCapture | undefined { const node = result.node; @@ -402,41 +279,7 @@ function parseGetTarget(req: DaemonRequest): return { ok: true, target: { kind: 'selector', selector } }; } -function toWaitTarget( - // 'invalid' is rejected by the caller before this point; excluding it here makes the - // kind-by-kind narrowing below exhaustive without a runtime fallback branch (#1800). - parsed: Exclude, - session: SessionState | undefined, - recordedLandmark?: TargetAnnotationV1, -) { - if (parsed.kind === 'sleep') return { kind: 'sleep' as const, durationMs: parsed.durationMs }; - if (parsed.kind === 'selector') { - return { - kind: 'selector' as const, - selector: parsed.selectorExpression, - timeoutMs: parsed.timeoutMs, - ...(recordedLandmark ? { recordedLandmark } : {}), - }; - } - if (parsed.kind === 'ref') { - if (!session?.snapshot) { - throw new AppError('INVALID_ARGS', 'Ref wait requires an existing snapshot in session.'); - } - return { kind: 'ref' as const, ref: parsed.rawRef, timeoutMs: parsed.timeoutMs }; - } - if (parsed.kind === 'stable') { - return { - kind: 'stable' as const, - quietMs: parsed.quietMs, - timeoutMs: parsed.timeoutMs, - }; - } - const waitText = checkWaitText(parsed.text); - if (!waitText.ok) throw new AppError(waitText.code, waitText.message); - return { kind: 'text' as const, text: waitText.text, timeoutMs: parsed.timeoutMs }; -} - -async function toDaemonResponse( +export async function toDaemonResponse( task: () => Promise>, ): Promise { try { @@ -447,7 +290,7 @@ async function toDaemonResponse( } } -async function maybeAndroidForegroundBlockerResponse( +export async function maybeAndroidForegroundBlockerResponse( params: SelectorRuntimeParams, response: DaemonResponse, commandLabel: string, diff --git a/src/daemon/wait-runtime.ts b/src/daemon/wait-runtime.ts new file mode 100644 index 000000000..b017ef763 --- /dev/null +++ b/src/daemon/wait-runtime.ts @@ -0,0 +1,275 @@ +import { waitObservesDevice } from '@agent-device/contracts/wait-runtime-plan'; +import type { TargetAnnotationV1 } from '@agent-device/contracts/replay'; +import { AppError } from '@agent-device/kernel/errors'; +import { checkWaitText } from '@agent-device/selectors'; +import { parseWaitPositionals } from '../core/wait-positionals.ts'; +import type { WaitParsed } from '../core/wait-positionals.ts'; +import { absenceCaptureOptionError } from '../core/absence-observation-errors.ts'; +import { absenceCaptureOptionRefusal } from '../core/absence-observation.ts'; +import { parseVersionedRefPositional } from './ref-positionals.ts'; +import { errorResponse } from './response.ts'; +import { resolveRefStalenessWarning } from './session-snapshot.ts'; +import { resolveSessionDevice, withSessionlessRunnerCleanup } from './snapshot-session.ts'; +import { recordIfSession, stripResolutionPayload, toDaemonWaitData } from './selector-recording.ts'; +import { + resolveBoundSelectorCapture, + type BoundSelectorOperations, +} from './selector-capture-binding.ts'; +import { + consumedSessionSnapshot, + maybeAndroidForegroundBlockerResponse, + readRecordedResolutionTarget, + toDaemonResponse, +} from './selector-runtime.ts'; +import type { BindDeviceRuntime, InspectDeviceRuntimeFacts } from './request-runtime-binding.ts'; +import type { DaemonRequest, DaemonResponse, SessionState } from './types.ts'; +import { dispatchConditionalWaitSelector } from './wait-conditional-selector.ts'; +import { maybeWaitTimeoutSurfaceResponse } from './wait-current-surface.ts'; +import { withSystemSurfaceDisclosure } from './system-surface-disclosure.ts'; +import { + createSelectorRuntimeForDevice, + type SelectorRuntimeParams, +} from './selector-runtime-backend.ts'; + +type DispatchWaitParams = SelectorRuntimeParams & + Readonly<{ inspectFacts?: InspectDeviceRuntimeFacts; bindDevice?: BindDeviceRuntime }>; + +export async function dispatchWaitViaRuntime(params: DispatchWaitParams): Promise { + const { req, sessionName, sessionStore } = params; + const parsedOrResponse = parseWaitRequest(req); + if ('ok' in parsedOrResponse) return parsedOrResponse; + const parsed = parsedOrResponse; + const { session, device } = await resolveSessionDevice(sessionStore, sessionName, req.flags); + // ADR 0019: facts are the only support authority, through the selector family's one + // admit-then-bind entry. A duration wait observes nothing, so it never asks for a binding — + // exactly the cell legacy admission skipped by testing `parsed.kind !== 'sleep'`. + const boundOrResponse = await bindWaitOperations(params, parsed, session, device); + if (boundOrResponse && 'ok' in boundOrResponse) return boundOrResponse; + const waitOperations = boundOrResponse; + // ADR 0012 / #1349, replay-only: the recorded landmark identity this wait must observe. + const recordedLandmark = req.internal?.replayLandmarkGuard; + const normalized = normalizeWaitPositionals(parsed, session); + if ('ok' in normalized) return normalized; + const { waitParsed, staleRefsWarning } = normalized; + const conditionalResponse = await dispatchConditionalWaitIfNeeded( + params, + waitParsed, + session, + waitOperations, + recordedLandmark, + ); + if (conditionalResponse) return conditionalResponse; + // Wait builds its runtime directly (no createBoundSelectorRuntime), so the consumed-snapshot slot + // must be initialized here too or sessionless waits have nowhere to report the capture from. + params.consumedSnapshot ??= {}; + // A pure sleep consumes no capture, so it never earns the system-surface disclosure below. + if (parsed.kind === 'sleep') { + return await executeWaitRequest( + params, + waitParsed, + session, + device, + waitOperations, + staleRefsWarning, + recordedLandmark, + ); + } + // Both a satisfied wait and a timeout consumed the polled capture stored on the session: + // when it is an occluding system surface, the outcome must disclose the occlusion. + return withSystemSurfaceDisclosure( + await withSessionlessRunnerCleanup( + session, + device, + () => + executeWaitRequest( + params, + waitParsed, + session, + device, + waitOperations, + staleRefsWarning, + recordedLandmark, + ), + params.platformResourceCleanup, + ), + consumedSessionSnapshot(params), + ); +} + +function parseWaitRequest( + req: DaemonRequest, +): Exclude | DaemonResponse { + const parsed = parseWaitPositionals(req.positionals ?? []); + if (!parsed) return errorResponse('INVALID_ARGS', 'wait requires a duration or text'); + if (parsed.kind === 'invalid') return errorResponse('INVALID_ARGS', parsed.message); + if (parsed.kind !== 'absent') return parsed; + const refusedOption = absenceCaptureOptionRefusal({ + depth: req.flags?.snapshotDepth, + scope: req.flags?.snapshotScope, + }); + if (!refusedOption) return parsed; + const error = absenceCaptureOptionError(refusedOption, 'wait'); + return errorResponse(error.code, error.message, error.details); +} + +async function bindWaitOperations( + params: DispatchWaitParams, + parsed: Exclude, + session: SessionState | undefined, + device: SessionState['device'], +): Promise { + if (!waitObservesDevice(parsed.kind)) return undefined; + const bound = await resolveBoundSelectorCapture({ + command: 'wait', + device, + session, + inspectFacts: params.inspectFacts, + bindDevice: params.bindDevice, + }); + return bound.ok ? bound.operations : bound.response; +} + +type NormalizedWaitPositionals = { + waitParsed: Exclude; + staleRefsWarning?: string; +}; + +function normalizeWaitPositionals( + parsed: Exclude, + session: SessionState | undefined, +): NormalizedWaitPositionals | DaemonResponse { + if (parsed.kind !== 'ref') return { waitParsed: parsed }; + const versionedRef = parseVersionedRefPositional(parsed.rawRef); + if (!versionedRef.ok) return versionedRef.response; + return { + waitParsed: { ...parsed, rawRef: versionedRef.ref }, + staleRefsWarning: resolveRefStalenessWarning({ + session, + ref: versionedRef.ref, + mintedGeneration: versionedRef.generation, + }), + }; +} + +async function dispatchConditionalWaitIfNeeded( + params: DispatchWaitParams, + parsed: Exclude, + session: SessionState | undefined, + waitOperations: BoundSelectorOperations | undefined, + recordedLandmark: TargetAnnotationV1 | undefined, +): Promise { + if (parsed.kind !== 'selector') return null; + return await dispatchConditionalWaitSelector({ + selectorExpression: parsed.selectorExpression, + operation: waitOperations?.findSelector, + recordedLandmark, + req: params.req, + session, + sessionName: params.sessionName, + sessionStore: params.sessionStore, + logPath: params.logPath, + signal: params.signal, + }); +} + +async function executeWaitRequest( + params: DispatchWaitParams, + parsed: Exclude, + session: SessionState | undefined, + device: SessionState['device'], + waitOperations: BoundSelectorOperations | undefined, + staleRefsWarning: string | undefined, + recordedLandmark: TargetAnnotationV1 | undefined, +): Promise { + const { req, sessionName, sessionStore } = params; + const runtime = createSelectorRuntimeForDevice({ + ...params, + session, + device, + bound: waitOperations, + }); + const response = await toDaemonResponse(async () => { + const result = await runtime.selectors.wait({ + session: sessionName, + requestId: req.meta?.requestId, + target: toWaitTarget(parsed, session, recordedLandmark), + }); + const recordedTarget = readRecordedResolutionTarget(result); + recordIfSession( + sessionStore, + sessionName, + req, + stripResolutionPayload(result), + recordedTarget, + 'landmark', + ); + const data = toDaemonWaitData(result); + return staleRefsWarning ? { ...data, warning: staleRefsWarning } : data; + }); + const enrichedResponse = waitOperations + ? await maybeWaitTimeoutSurfaceResponse( + { req, logPath: params.logPath, session, device, capture: waitOperations.capture }, + response, + ) + : response; + return await maybeAndroidForegroundBlockerResponse(params, enrichedResponse, 'wait'); +} + +function toWaitTarget( + // 'invalid' is rejected by the caller before this point; excluding it here makes the + // kind-by-kind narrowing below exhaustive without a runtime fallback branch (#1800). + parsed: Exclude, + session: SessionState | undefined, + recordedLandmark?: TargetAnnotationV1, +) { + switch (parsed.kind) { + case 'sleep': + return { kind: 'sleep' as const, durationMs: parsed.durationMs }; + case 'selector': + return { + kind: 'selector' as const, + selector: parsed.selectorExpression, + timeoutMs: parsed.timeoutMs, + ...(recordedLandmark ? { recordedLandmark } : {}), + }; + case 'absent': + return { + kind: 'absent' as const, + selector: parsed.selectorExpression, + timeoutMs: parsed.timeoutMs, + }; + case 'ref': + return toRefWaitTarget(parsed, session); + case 'stable': + return { + kind: 'stable' as const, + quietMs: parsed.quietMs, + timeoutMs: parsed.timeoutMs, + }; + case 'text': { + return toTextWaitTarget(parsed); + } + default: + return assertNever(parsed); + } +} + +function assertNever(value: never): never { + throw new Error(`Unsupported wait positional target: ${String(value)}`); +} + +function toRefWaitTarget( + parsed: Extract, + session: SessionState | undefined, +) { + if (!session?.snapshot) { + throw new AppError('INVALID_ARGS', 'Ref wait requires an existing snapshot in session.'); + } + return { kind: 'ref' as const, ref: parsed.rawRef, timeoutMs: parsed.timeoutMs }; +} + +function toTextWaitTarget(parsed: Extract) { + const waitText = checkWaitText(parsed.text); + if (!waitText.ok) throw new AppError(waitText.code, waitText.message); + return { kind: 'text' as const, text: waitText.text, timeoutMs: parsed.timeoutMs }; +} diff --git a/src/mcp/__tests__/command-tools-input-docs.test.ts b/src/mcp/__tests__/command-tools-input-docs.test.ts new file mode 100644 index 000000000..a699c126b --- /dev/null +++ b/src/mcp/__tests__/command-tools-input-docs.test.ts @@ -0,0 +1,167 @@ +import assert from 'node:assert/strict'; +import { test } from 'vitest'; +import { listCommandTools } from '../command-tools.ts'; + +// Guidance no longer restates input fields in prose, so a tool's inputSchema is the only +// place its inputs are documented — for the model, for `--help`, and for the docs site. An +// undescribed property is therefore a silent gap rather than a cosmetic one. +// +// The baseline pins exact `tool.property` identities, not bare property names plus a total: +// a name-and-count baseline stays green when a gap migrates (describe `foo.text`, add an +// undescribed `bar.text`, and both the allowed-name set and the total are unchanged), and +// stale names keep authorizing gaps that appear later. Exact identities make every new or +// moved gap fail, and require deleting an entry to record a fix. +const UNDESCRIBED_TOOL_INPUTS = new Set([ + 'alert.action', + 'alert.timeoutMs', + 'audio.action', + 'audio.probeAction', + 'back.mode', + 'clipboard.action', + 'clipboard.text', + 'close.saveScript', + 'debug.action', + 'diff.depth', + 'diff.interactiveOnly', + 'diff.kind', + 'diff.out', + 'diff.raw', + 'diff.scope', + 'events.cursor', + 'events.limit', + 'find.action', + 'find.depth', + 'find.first', + 'find.last', + 'find.locator', + 'find.query', + 'find.raw', + 'find.timeoutMs', + 'find.value', + 'get.format', + 'install-from-source.retainPaths', + 'install-from-source.retentionMs', + 'is.predicate', + 'is.selector', + 'is.value', + 'keyboard.action', + 'logs.action', + 'logs.message', + 'logs.restart', + 'metro.action', + 'metro.bridgeScope', + 'metro.bundleUrl', + 'metro.installDependenciesIfNeeded', + 'metro.kind', + 'metro.launchUrl', + 'metro.listenHost', + 'metro.logPath', + 'metro.metroHost', + 'metro.metroPort', + 'metro.port', + 'metro.probeTimeoutMs', + 'metro.projectRoot', + 'metro.publicBaseUrl', + 'metro.reuseExisting', + 'metro.runtimeFilePath', + 'metro.startupTimeoutMs', + 'metro.statusHost', + 'metro.timeoutMs', + 'network.action', + 'network.include', + 'network.limit', + 'open.saveScript', + 'orientation.orientation', + 'perf.action', + 'perf.area', + 'perf.kind', + 'perf.subject', + 'push.app', + 'push.payload', + 'react-native.action', + 'record.action', + 'record.fps', + 'record.hideTouches', + 'record.path', + 'record.quality', + 'record.recordingScope', + 'reinstall.app', + 'replay.backend', + 'replay.env', + 'replay.force', + 'replay.maestro', + 'replay.path', + 'replay.resumeFrom', + 'replay.resumePlanDigest', + 'replay.saveScript', + 'replay.update', + 'screenshot.fullscreen', + 'screenshot.normalizeStatusBar', + 'screenshot.overlayRefs', + 'screenshot.stabilize', + 'screenshot.surface', + 'scroll.direction', + 'settings.app', + 'settings.latitude', + 'settings.longitude', + 'settings.mode', + 'settings.permission', + 'settings.setting', + 'settings.state', + 'snapshot.depth', + 'snapshot.forceFull', + 'snapshot.interactiveOnly', + 'snapshot.raw', + 'snapshot.scope', + 'swipe.pattern', + 'test.artifactsDir', + 'test.backend', + 'test.env', + 'test.failFast', + 'test.maestro', + 'test.paths', + 'test.recordVideo', + 'test.retries', + 'test.shardAll', + 'test.shardSplit', + 'test.timeoutMs', + 'test.update', + 'trace.action', + 'trace.path', + 'tv-remote.button', + 'wait.depth', + 'wait.durationMs', + 'wait.absent', + 'wait.kind', + 'wait.quietMs', + 'wait.raw', + 'wait.ref', + 'wait.scope', + 'wait.selector', + 'wait.stable', + 'wait.text', + 'wait.timeoutMs', +]); + +test('MCP tool inputs do not add undocumented properties', () => { + const undescribed: string[] = []; + for (const tool of listCommandTools()) { + for (const [key, schema] of Object.entries(tool.inputSchema.properties ?? {})) { + if (!schema.description) undescribed.push(`${tool.name}.${key}`); + } + } + + const added = undescribed.filter((entry) => !UNDESCRIBED_TOOL_INPUTS.has(entry)).sort(); + assert.deepEqual( + added, + [], + `These MCP tool inputs need a schema description: ${added.join(', ')}`, + ); + + const fixed = [...UNDESCRIBED_TOOL_INPUTS].filter((entry) => !undescribed.includes(entry)).sort(); + assert.deepEqual( + fixed, + [], + `These MCP tool inputs are documented now — remove them from UNDESCRIBED_TOOL_INPUTS: ${fixed.join(', ')}`, + ); +}); diff --git a/src/mcp/__tests__/command-tools-parity.test.ts b/src/mcp/__tests__/command-tools-parity.test.ts index 849172ad2..b0c4a6f7f 100644 --- a/src/mcp/__tests__/command-tools-parity.test.ts +++ b/src/mcp/__tests__/command-tools-parity.test.ts @@ -86,6 +86,30 @@ test('MCP open keeps device-selection evidence in structured content and JSON te assert.deepEqual(JSON.parse(result.content[0]?.text ?? '{}').selection, openResult.selection); }); +test('MCP advertises and round-trips the strict wait absent field', async () => { + let observedInput: Record | undefined; + const executor = createCommandToolExecutor({ + createClient: () => ({}) as AgentDeviceClient, + runCommand: async (_client, _name, input) => { + observedInput = input; + return { waitedMs: 0 }; + }, + }); + const waitTool = listCommandTools().find((tool) => tool.name === 'wait'); + assert.equal( + (waitTool?.inputSchema.properties?.absent as { type?: string } | undefined)?.type, + 'string', + ); + + const result = await executor.execute('wait', { + absent: 'label="Removed"', + timeoutMs: 5000, + }); + + assert.equal(result.isError, false); + assert.deepEqual(observedInput, { absent: 'label="Removed"', timeoutMs: 5000 }); +}); + test('MCP fill projects target-bound unconfirmed verification through its advertised schema', async () => { const fillResult = { targetKind: 'point', diff --git a/src/mcp/__tests__/command-tools.test.ts b/src/mcp/__tests__/command-tools.test.ts index 99cfb0f0e..23c8440f2 100644 --- a/src/mcp/__tests__/command-tools.test.ts +++ b/src/mcp/__tests__/command-tools.test.ts @@ -1017,146 +1017,6 @@ test('MCP forwards noRecord from a press tool call through to the executed comma ]); }); -// Guidance no longer restates input fields in prose, so a tool's inputSchema is the only -// place its inputs are documented — for the model, for `--help`, and for the docs site. An -// undescribed property is therefore a silent gap rather than a cosmetic one. -// -// The baseline pins exact `tool.property` identities, not bare property names plus a total: -// a name-and-count baseline stays green when a gap migrates (describe `foo.text`, add an -// undescribed `bar.text`, and both the allowed-name set and the total are unchanged), and -// stale names keep authorizing gaps that appear later. Exact identities make every new or -// moved gap fail, and require deleting an entry to record a fix. -const UNDESCRIBED_TOOL_INPUTS = new Set([ - 'alert.action', - 'alert.timeoutMs', - 'audio.action', - 'audio.probeAction', - 'back.mode', - 'clipboard.action', - 'clipboard.text', - 'close.saveScript', - 'debug.action', - 'diff.depth', - 'diff.interactiveOnly', - 'diff.kind', - 'diff.out', - 'diff.raw', - 'diff.scope', - 'events.cursor', - 'events.limit', - 'find.action', - 'find.depth', - 'find.first', - 'find.last', - 'find.locator', - 'find.query', - 'find.raw', - 'find.timeoutMs', - 'find.value', - 'get.format', - 'install-from-source.retainPaths', - 'install-from-source.retentionMs', - 'is.predicate', - 'is.selector', - 'is.value', - 'keyboard.action', - 'logs.action', - 'logs.message', - 'logs.restart', - 'metro.action', - 'metro.bridgeScope', - 'metro.bundleUrl', - 'metro.installDependenciesIfNeeded', - 'metro.kind', - 'metro.launchUrl', - 'metro.listenHost', - 'metro.logPath', - 'metro.metroHost', - 'metro.metroPort', - 'metro.port', - 'metro.probeTimeoutMs', - 'metro.projectRoot', - 'metro.publicBaseUrl', - 'metro.reuseExisting', - 'metro.runtimeFilePath', - 'metro.startupTimeoutMs', - 'metro.statusHost', - 'metro.timeoutMs', - 'network.action', - 'network.include', - 'network.limit', - 'open.saveScript', - 'orientation.orientation', - 'perf.action', - 'perf.area', - 'perf.kind', - 'perf.subject', - 'push.app', - 'push.payload', - 'react-native.action', - 'record.action', - 'record.fps', - 'record.hideTouches', - 'record.path', - 'record.quality', - 'record.recordingScope', - 'reinstall.app', - 'replay.backend', - 'replay.env', - 'replay.force', - 'replay.maestro', - 'replay.path', - 'replay.resumeFrom', - 'replay.resumePlanDigest', - 'replay.saveScript', - 'replay.update', - 'screenshot.fullscreen', - 'screenshot.normalizeStatusBar', - 'screenshot.overlayRefs', - 'screenshot.stabilize', - 'screenshot.surface', - 'scroll.direction', - 'settings.app', - 'settings.latitude', - 'settings.longitude', - 'settings.mode', - 'settings.permission', - 'settings.setting', - 'settings.state', - 'snapshot.depth', - 'snapshot.forceFull', - 'snapshot.interactiveOnly', - 'snapshot.raw', - 'snapshot.scope', - 'swipe.pattern', - 'test.artifactsDir', - 'test.backend', - 'test.env', - 'test.failFast', - 'test.maestro', - 'test.paths', - 'test.recordVideo', - 'test.retries', - 'test.shardAll', - 'test.shardSplit', - 'test.timeoutMs', - 'test.update', - 'trace.action', - 'trace.path', - 'tv-remote.button', - 'wait.depth', - 'wait.durationMs', - 'wait.kind', - 'wait.quietMs', - 'wait.raw', - 'wait.ref', - 'wait.scope', - 'wait.selector', - 'wait.stable', - 'wait.text', - 'wait.timeoutMs', -]); - // Retired-input regressions run the REAL command route (no runCommand // injection): field projection used to silently drop the removed `maxSize` // key before the daemon writers could refuse it, returning native-size @@ -1191,26 +1051,3 @@ test('MCP screenshot and record schemas do not advertise the retired maxSize inp assert.equal('maxSize' in (tool.inputSchema.properties ?? {}), false); } }); - -test('MCP tool inputs do not add undocumented properties', () => { - const undescribed: string[] = []; - for (const tool of listCommandTools()) { - for (const [key, schema] of Object.entries(tool.inputSchema.properties ?? {})) { - if (!schema.description) undescribed.push(`${tool.name}.${key}`); - } - } - - const added = undescribed.filter((entry) => !UNDESCRIBED_TOOL_INPUTS.has(entry)).sort(); - assert.deepEqual( - added, - [], - `These MCP tool inputs need a schema description: ${added.join(', ')}`, - ); - - const fixed = [...UNDESCRIBED_TOOL_INPUTS].filter((entry) => !undescribed.includes(entry)).sort(); - assert.deepEqual( - fixed, - [], - `These MCP tool inputs are documented now — remove them from UNDESCRIBED_TOOL_INPUTS: ${fixed.join(', ')}`, - ); -}); diff --git a/src/mcp/__tests__/router.test.ts b/src/mcp/__tests__/router.test.ts index 89b19ca1f..7b15fe7b4 100644 --- a/src/mcp/__tests__/router.test.ts +++ b/src/mcp/__tests__/router.test.ts @@ -81,6 +81,7 @@ test('server instructions are the compact workflow card, under the 2 KB client c // The card must name the start rule and the guide tool, and speak in tool properties. assert.match(MCP_SERVER_INSTRUCTIONS, /open \{app, foreground: true\}/); assert.match(MCP_SERVER_INSTRUCTIONS, /snapshot \{interactiveOnly: true\}/); + assert.match(MCP_SERVER_INSTRUCTIONS, /wait \{absent: selector\}/); assert.match(MCP_SERVER_INSTRUCTIONS, /Call help only/); }); diff --git a/src/mcp/server-guide.ts b/src/mcp/server-guide.ts index 63657f784..8e81c4f8e 100644 --- a/src/mcp/server-guide.ts +++ b/src/mcp/server-guide.ts @@ -16,7 +16,7 @@ export const MCP_SERVER_INSTRUCTIONS = `agent-device drives iOS, Android, tvOS, Start: known app -> call open {app, foreground: true} at once; do not probe with devices, apps, appstate, snapshot, or screenshot first. open returns the initial interactive snapshot with @refs. Unknown app id: devices, then apps, then open the discovered id; never invent ids. Existing session: continue from its state, do not reopen. -Loop: press/click/fill/longpress/hover/scroll/back with settle: true; the response is the settled UI diff, continue from it. snapshot {interactiveOnly: true} only when the diff lacks the next target or did not settle. Verify with wait {kind: "text", text}, wait {selector}, is, get, or find; a bare screenshot is not verification. End with close. +Loop: press/click/fill/longpress/hover/scroll/back with settle: true; the response is the settled UI diff, continue from it. snapshot {interactiveOnly: true} only when the diff lacks the next target or did not settle. Verify with wait {kind: "text", text}, wait {selector}, wait {absent: selector}, is, get, or find; a bare screenshot is not verification. End with close. Targets: copy refs byte-for-byte (@e12, @e12~s4; keep @ and any ~sN). Refs go stale after mutations. Prefer refs, then id/label/role selectors; coordinates last. On a sparse/AX-unavailable warning its refs and selectors are invalid: screenshot, read the image, press {x, y}, then snapshot the changed screen. diff --git a/website/docs/docs/commands.md b/website/docs/docs/commands.md index eca4b8d87..e6502c05e 100644 --- a/website/docs/docs/commands.md +++ b/website/docs/docs/commands.md @@ -402,6 +402,7 @@ agent-device wait 1500 agent-device wait text "Welcome back" agent-device wait @e12 agent-device wait 'role="button" label="Continue"' 5000 +agent-device wait absent 'label="Loading..."' 5000 agent-device alert agent-device alert get agent-device alert wait 3000 @@ -409,13 +410,15 @@ agent-device alert accept agent-device alert dismiss ``` -- `wait` accepts a millisecond duration, `text `, a snapshot ref (`@eN`), or a selector. +- `wait` accepts a millisecond duration, `text `, a snapshot ref (`@eN`), a selector, or strict `absent `. - `wait [timeoutMs]` polls until the selector resolves or the timeout expires. +- `wait absent [timeoutMs]` polls until a complete, readable capture has zero matches. It is strict absence, not a visibility check: hidden or off-screen matches still keep the wait pending. +- Strict absence rejects `--scope` and `--depth`. Sparse, truncated, incomplete, and Android unreadable-content captures do not count as readable captures and cannot satisfy the wait; they are ridden out until the deadline, and a run with no valid capture preserves its typed unreadable diagnostic. - `wait @ref [timeoutMs]` requires an existing session snapshot from a prior `snapshot` command. - `wait @ref` resolves the ref to its label/text from that stored snapshot, then polls for that text; it does not track the original node identity. - Because `wait @ref` is text-based after resolution, duplicate labels can match a different element than the original ref target. - `wait` shares the selector/snapshot resolution flow used by `click`, `fill`, `get`, and `is`. -- Wait failures carry a structured `error.details.reason` in `--json` output: `wait_target_absent` proves at least one readable capture saw no match; `wait_capture_stalled` means no readable capture arrived and is retriable; `wait_deadline_exceeded` means a later capture consumed the remaining budget after an earlier readable capture; `wait_landmark_identity_mismatch` is a replay destination-guard refusal; and `wait_stable_timeout` means the UI did not settle. Use `readableCaptures` and `waitedMs` instead of parsing error text. +- Wait failures carry a structured `error.details.reason` in `--json` output: `wait_target_absent` proves a positive wait never found a match; `wait_target_present` means strict `wait absent` reached its deadline with valid captures that still contained matches; `predicate_failed` means strict `wait absent` could not prove absence because no valid capture arrived, with the final observation/diagnostic preserved; `wait_capture_stalled` means no readable capture arrived and is retriable; `wait_deadline_exceeded` means a later capture consumed the remaining budget after an earlier readable capture; `wait_landmark_identity_mismatch` is a replay destination-guard refusal; and `wait_stable_timeout` means the UI did not settle. Use `readableCaptures`, `waitedMs`, `matches`, and `firstMatch` instead of parsing error text. `firstMatch` carries identity/text evidence only; absence failures do not claim visibility or rect evidence. - `alert` inspects or handles system alerts on iOS simulator, macOS desktop, and Android native/runtime permission dialogs. - `alert` without an action is equivalent to `alert get`. - Use `alert get` for an immediate cheap check. Use `alert wait ` only when a prompt may appear after async work. @@ -527,6 +530,7 @@ agent-device is text 'id="greeting"' "Welcome back" - `is exists` only checks whether the selector matches in the current snapshot. - `is absent` passes only when the selector has zero matches in one readable, complete, unscoped, full-depth accessibility capture. It does not mean hidden; `--scope` and `--depth` are rejected, and sparse, unreadable, or truncated captures fail closed. - `wait text` is a text-presence wait, not a hittability assertion. +- Strict `wait absent` is not exported to Maestro's lenient `notVisible` condition; Maestro export reports it as unsupported unless an exact zero-candidate primitive becomes available. - `is text ` compares the resolved element text against the expected value. - `is` does not accept snapshot refs like `@e3`; use a selector expression instead. - `is` accepts the same selector-oriented snapshot flags as `click`, `fill`, `get`, and `wait`; `is absent` rejects `--scope` and `--depth` because its proof must cover the complete unscoped tree.