You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
test(skillgym): require ref-driven SpringBoard interaction, not fixed labels
The case required the literal label="Add Widget" while the topic it
tests explicitly says SpringBoard labels vary and must come from a
fresh snapshot; it also didn't require the Calendar search action, so
a plan could pass while skipping the widget-gallery search entirely.
Give the contract explicit snapshot-derived refs (@e30/@e31/@e35) and
require pressing/filling those refs plus the Calendar search, and
forbid hardcoded English labels (Add Widget/Edit/Done) instead.
Verified locally against three synthetic finalOutput plans (good,
search-skipping, fixed-label) since the real runner needs a live
model.
Copy file name to clipboardExpand all lines: test/skillgym/suites/agent-device-smoke-suite.ts
+14-5Lines changed: 14 additions & 5 deletions
Original file line number
Diff line number
Diff line change
@@ -2596,22 +2596,31 @@ Treat the recovery message as a warning, not a fatal error. Use the exposed Sear
2596
2596
'Agent Device Tester is already open on this session; its widget-ready state is prepared',
2597
2597
'agent-device open com.apple.springboard --platform ios binds the session to SpringBoard for selector-driven commands; there is no separate widget/system command',
2598
2598
'Entering SpringBoard edit mode needs one deliberate empty-space long-press on the home screen; there is no selector for it',
2599
-
'The widget-gallery search-result row for "Calendar" currently returns an unlabeled accessibility node, so select it by coordinates read from a screenshot',
2600
-
'SpringBoard control labels vary by iOS version and locale; discover them from a fresh snapshot -i instead of assuming fixed English strings',
2599
+
'After edit mode, a fresh interactive snapshot exposes the Edit menu entry that opens the widget gallery as ref @e30',
2600
+
'Inside the widget gallery, a fresh interactive snapshot exposes the search field as ref @e31',
2601
+
'The widget-gallery search-result row for "Calendar" currently returns an unlabeled accessibility node, so select it by coordinates read from a screenshot instead',
2602
+
'After choosing the result, a fresh interactive snapshot exposes the size-picker placement control as ref @e35',
2603
+
'SpringBoard control labels vary by iOS version and locale; do not assume fixed English strings such as "Add Widget" or "Edit" — use the refs above, sourced from a fresh snapshot -i, instead',
2601
2604
],
2602
-
task: 'Plan the commands to open SpringBoard, enter edit mode, search the widget gallery for "Calendar", add it using the documented coordinate fallback for the unlabeled search-result row, reopen Agent Device Tester to resume app automation, and close the session.',
2605
+
task: 'Plan the commands to open SpringBoard, enter edit mode, open the widget gallery with @e30, search it for "Calendar" using @e31, select the result using the documented coordinate fallback for the unlabeled search-result row, place the widget with @e35, reopen Agent Device Tester to resume app automation, and close the session.',
0 commit comments