Skip to content

Commit e633ee4

Browse files
committed
fix(gestures): fail pre-removal gesture forms at .ad parse time (#1216)
#1315 removed the timed forms of `swipe`, `gesture fling`, and `gesture swipe` and `gesture rotate`'s `velocity`, but shipped without the migration guide, the repository sweep, or the parse-time error that issue #1216's own checklist gates a removal on. The sweep finds what that left behind: both `06-swipe-gestures.ad` integration fixtures still carried the 5-argument swipe and would fail at replay, two tests still asserted the removed shapes, and two branches still read the retired positional. Argument arity for every public gesture syntax now lives in one table keyed off the canonical `GESTURE_KINDS`, so a new kind cannot skip it and a form removed from the CLI is removed from `.ad` in the same edit. Both callers read it: the CLI argv parse, and a new `.ad` preflight. A stale script now fails when it is parsed — before the replay executes any device action — naming the line and computing its rewrite, instead of running up to that step and failing as a repairable divergence. The preflight checks arity only: `${VAR}` tokens resolve after planning, and interpolation never splits a token, so the count is decidable while the values are not. Deleting the dead duration read in `readSwipeGeometry` would have left `replay export` emitting no duration, handing Maestro's 400ms default to a gesture the script runs at 100ms, so the export now states `duration: 100`. `.ad` positional gesture parsing is NOT removed. Its only remaining callers are the CLI argv parse and the `.ad` line parse, both the current public syntax rather than a bridge to an older one, so there is nothing to migrate off. ADR 0013 records that and drops the "compatibility" framing that made it read as debt. Both migrated fixtures verified on real devices with the repo's own CLI: iOS simulator 34.9s, Android emulator 45.9s.
1 parent 14be01b commit e633ee4

14 files changed

Lines changed: 469 additions & 56 deletions

File tree

CHANGELOG.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,9 @@
1313
- `gesture swipe preset` no longer accepts a trailing `durationMs` positional; use `gesture pan` for timed movement.
1414
- `gesture rotate degrees [x] [y]` no longer accepts a trailing `velocity` positional; rotation pacing is derived from `degrees`.
1515
- MCP/Node schemas no longer advertise `velocity` or `durationMs` on `swipe`/`fling`/`gesture swipe`; `durationMs` remains on `gesture pan` and `gesture transform`.
16+
- A `.ad` script that still carries a removed positional now fails when the script is parsed, before the replay executes any device action, naming the line and its rewrite (for example `swipe accepts 4 arguments: x1 y1 x2 y2 (line 6). The trailing durationMs positional was removed: use "gesture pan 197 650 0 -350 300" ...`). Previously the script ran up to that step and then failed as a replay divergence.
17+
- Published the [gesture migration guide](https://agent-device.dev/docs/migrating-gestures) covering CLI, Node.js, MCP, and saved `.ad` recordings, plus the deprecation policy the next such removal follows (#1216).
18+
- `replay export` now writes an explicit `duration: 100` for `swipe` — the canonical fling duration — instead of omitting it and letting Maestro apply its own 400ms default. Maestro flows replayed by `agent-device` are unaffected; a timed Maestro `swipe` still normalizes to `gesture pan` with the `endpoint-hold` profile.
1619
- Breaking: the deprecated `rotate` CLI command alias has been removed. Use `orientation` instead; invoking `rotate` now fails with `rotate was renamed to orientation; for the two-finger gesture use: gesture rotate`.
1720
- Breaking (ADR 0014, session ref-frame lifetime): a mutation through an `@ref` now expires the session's ref frame, so a later ref mutation without a fresh observation fails closed with a typed `details.reason` (`ref_frame_expired`, `ref_generation_mismatch`, `plain_ref_requires_complete_frame`, or `ref_not_issued`) instead of acting on a possibly-navigated screen. A ref-oriented sequence that performs several mutations must re-`snapshot` between them, consume an honestly issued `--settle` ref in pinned `@eN~s<gen>` form, or use selectors. Enforcement applies on every platform, not just iOS. Legacy hand-written `.ad` scripts that reuse several bare refs from one snapshot must capture between mutations or use selectors.
1821
- Ref reads resolve against the authorized ref frame's source tree, so an internal read-only capture (including Android freshness) can no longer retarget an admitted `@ref` by positional coincidence. Read-only ref consumers keep the structured staleness warning while the frame retains the ref's evidence.

docs/adr/0013-unified-gesture-plans.md

Lines changed: 19 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -96,9 +96,25 @@ Platform adapters consume the canonical plan:
9696
The `Interactor` and backend expose one compositional `performGesture(plan)` primitive instead of a
9797
method per semantic alias. The old scalar Apple and Android multi-touch executors and the
9898
public-command alias-to-positionals-to-reparse route are deleted. `.ad` keeps its established
99-
positional syntax through one named replay compatibility codec; CLI, Node.js, and MCP send
100-
structured input. Providers should compose transport/device bindings with the shared platform
101-
adapter rather than reimplement the interaction runtime.
99+
positional syntax through one named codec; CLI, Node.js, and MCP send structured input. Providers
100+
should compose transport/device bindings with the shared platform adapter rather than reimplement
101+
the interaction runtime.
102+
103+
That `.ad` codec is the script format's own syntax, not a compatibility shim, and is not scheduled
104+
for removal (amended 2026-07, issue #1216). Its only remaining callers are the CLI argv parse and
105+
the `.ad` line parse — both the current public syntax — so there is nothing left to migrate off.
106+
A structured `.ad` gesture payload was rejected: it would make recordings unreadable and ungreppable
107+
for no behavioral gain.
108+
109+
Argument arity for both callers comes from one table (`PUBLIC_GESTURE_SYNTAX`), so a form removed
110+
from the CLI is removed from `.ad` in the same edit. A `.ad` script that still carries a removed
111+
positional fails when the script is parsed, before the replay executes any device action, with the
112+
offending line and its rewrite. Arity is the only thing checked at parse time: `${VAR}` tokens
113+
resolve after planning, and interpolation never splits a token, so the count is decidable while the
114+
values are not. The removal process these inputs follow — announce, warn for one minor release,
115+
publish the migration guide, prove the repository is clean, then remove the branch and its tests —
116+
is documented in the public
117+
[gesture migration guide](https://agent-device.dev/docs/migrating-gestures).
102118

103119
Repeated coordinate swipes are bounded at the public command contract and daemon trust boundary.
104120
Individual count and pause limits prevent pathological fields, while the combined planned gesture

src/compat/maestro/__tests__/export-flow.test.ts

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ screenshot "./artifacts/checkout"
4343
test('exports coordinate gestures and sleep waits with warnings', () => {
4444
const result = exportReplayScriptToMaestro(`open com.example.app
4545
click 120 240
46-
swipe 200 700 200 200 300 --count 2
46+
swipe 200 700 200 200 --count 2
4747
wait 500
4848
`);
4949

@@ -52,8 +52,8 @@ wait 500
5252
[
5353
'launchApp',
5454
{ tapOn: { point: '120,240' } },
55-
{ swipe: { start: '200,700', end: '200,200', duration: 300 } },
56-
{ swipe: { start: '200,700', end: '200,200', duration: 300 } },
55+
{ swipe: { start: '200,700', end: '200,200', duration: 100 } },
56+
{ swipe: { start: '200,700', end: '200,200', duration: 100 } },
5757
{ waitForAnimationToEnd: { timeout: 500 } },
5858
],
5959
]);

src/compat/maestro/export-flow.ts

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
import type { SessionAction } from '../../daemon/types.ts';
2+
import { GESTURE_FLING_DURATION_MS } from '../../contracts/gesture-plan.ts';
23
import { parseSelectorChain, type Selector } from '../../selectors/index.ts';
34
import type { SelectorTerm } from '../../selectors/parse.ts';
45
import { AppError } from '../../kernel/errors.ts';
@@ -39,7 +40,7 @@ type ActionConverter = (action: SessionAction) => ConvertedAction;
3940
type SwipeGeometry = {
4041
start: string;
4142
end: string;
42-
duration?: number;
43+
duration: number;
4344
};
4445

4546
const TEXT_SELECTOR_KEYS = new Set<string>(MAESTRO_TEXT_SELECTOR_KEYS);
@@ -362,13 +363,18 @@ function convertSwipeAction(action: SessionAction): ConvertedAction {
362363
};
363364
}
364365

366+
/**
367+
* `swipe` carries no duration positional, so the export states the canonical
368+
* fling duration rather than letting Maestro apply its own 400 ms default to a
369+
* gesture the script runs at 100 ms.
370+
*/
365371
function readSwipeGeometry(action: SessionAction): SwipeGeometry | undefined {
366-
const [x1, y1, x2, y2, duration] = action.positionals;
372+
const [x1, y1, x2, y2] = action.positionals;
367373
if (!isNumber(x1) || !isNumber(y1) || !isNumber(x2) || !isNumber(y2)) return undefined;
368374
return {
369375
start: formatMaestroPoint(x1, y1),
370376
end: formatMaestroPoint(x2, y2),
371-
...(duration && isNumber(duration) ? { duration: Number(duration) } : {}),
377+
duration: GESTURE_FLING_DURATION_MS,
372378
};
373379
}
374380

src/contracts/gesture-normalization.test.ts

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
import assert from 'node:assert/strict';
22
import { test } from 'vitest';
33
import {
4+
describeReplayGestureArityError,
45
gesturePayloadFromPositionals,
56
gesturePayloadToPositionals,
67
normalizePublicGesture,
@@ -57,6 +58,62 @@ test('gesture recording codec round-trips fling with distance', () => {
5758
assert.deepEqual(gesturePayloadFromPositionals(gesturePayloadToPositionals(payload)), payload);
5859
});
5960

61+
test('a retired trailing positional reports its migration, not a bare usage line', () => {
62+
assert.throws(() => swipePayloadFromPositionals(['197', '650', '197', '300', '300']), {
63+
code: 'INVALID_ARGS',
64+
message:
65+
'swipe accepts 4 arguments: x1 y1 x2 y2. The trailing durationMs positional was removed: use "gesture pan 197 650 0 -350 300" for the same timed drag, or "swipe 197 650 197 300" for a default-duration swipe.',
66+
});
67+
assert.throws(() => gesturePayloadFromPositionals(['fling', 'down', '100', '200', '80', '500']), {
68+
code: 'INVALID_ARGS',
69+
message: /trailing durationMs positional was removed: use "gesture fling down 100 200 80"/,
70+
});
71+
assert.throws(() => gesturePayloadFromPositionals(['swipe', 'left', '500']), {
72+
code: 'INVALID_ARGS',
73+
message: /trailing durationMs positional was removed: use "gesture swipe left"/,
74+
});
75+
assert.throws(() => gesturePayloadFromPositionals(['rotate', '35', '100', '200', '800']), {
76+
code: 'INVALID_ARGS',
77+
message: /trailing velocity positional was removed: use "gesture rotate 35 100 200"/,
78+
});
79+
});
80+
81+
test('a malformed line is a usage error rather than a migration claim', () => {
82+
assert.throws(() => swipePayloadFromPositionals(['1', '2', '3', '4', '5', '6']), {
83+
code: 'INVALID_ARGS',
84+
message: 'swipe accepts 4 arguments: x1 y1 x2 y2.',
85+
});
86+
assert.throws(() => swipePayloadFromPositionals(['1', '2', '3', '4', '--unknown']), {
87+
code: 'INVALID_ARGS',
88+
message: 'swipe accepts 4 arguments: x1 y1 x2 y2.',
89+
});
90+
});
91+
92+
test('the .ad preflight names the offending line and leaves live syntax alone', () => {
93+
assert.equal(
94+
describeReplayGestureArityError('swipe', ['197', '650', '197', '300', '300'], 'line 6'),
95+
'swipe accepts 4 arguments: x1 y1 x2 y2 (line 6). The trailing durationMs positional was removed: use "gesture pan 197 650 0 -350 300" for the same timed drag, or "swipe 197 650 197 300" for a default-duration swipe.',
96+
);
97+
assert.equal(
98+
describeReplayGestureArityError('gesture', ['pan', '110', '443', '48', '0', '500'], 'line 2'),
99+
undefined,
100+
);
101+
assert.equal(describeReplayGestureArityError('click', ['120', '240'], 'line 2'), undefined);
102+
});
103+
104+
test('the .ad preflight defers to dispatch for values it cannot know yet', () => {
105+
// `${VAR}` tokens resolve after planning, and interpolation never splits a
106+
// token, so only the argument count is decidable at parse time.
107+
assert.equal(
108+
describeReplayGestureArityError('swipe', ['${X1}', '${Y1}', '${X2}', '${Y2}'], 'line 3'),
109+
undefined,
110+
);
111+
assert.equal(
112+
describeReplayGestureArityError('gesture', ['${KIND}', '1', '2', '3'], 'line 4'),
113+
undefined,
114+
);
115+
});
116+
60117
test('pinch and rotate syntax rejects a partial origin', () => {
61118
assert.throws(() => gesturePayloadFromPositionals(['pinch', '1.5', '100']), {
62119
code: 'INVALID_ARGS',

0 commit comments

Comments
 (0)