feat(flow): add structured YAML selectors and property assertions - #56
Merged
saikrishna321 merged 3 commits intoAug 15, 2026
Merged
Conversation
中文 变更: - 为 YAML Flow 增加精确、包含和正则选择器,以及状态、索引、空间和树关系匹配。 - 将结构化目标接入点击、输入、等待、滚动和断言执行路径,并输出匹配元素与失败诊断。 - 增加属性、数量、值和尺寸断言,补齐报告展示、敏感值脱敏、文档和回归测试。 验证: - npm run typecheck - npm test(22 个测试文件,427 项测试) - npm run format:check - git diff --check English Changes: - Add exact, contains, and regex YAML selectors with state, index, spatial, and tree-relation matching. - Wire structured targets through tap, type, wait, scroll, and assertion execution with matched-element diagnostics. - Add property, count, value, and dimension assertions plus report rendering, secret redaction, documentation, and regression coverage. Verification: - npm run typecheck - npm test (22 files, 427 tests) - npm run format:check - git diff --check
变更: - 保留带 resource-id 的无文本、不可点击 Android 节点,允许结构化 selector 唯一定位并按中心坐标点击 - Android session 创建后通过 driver settings 写入 idle、selector 与 action acknowledgment timeout,失败时清理 session - 补充中文 OpenSpec、回归测试及 selector/runner 文档 验证: - openspec validate fix-android-structured-selector-device-stability --strict - npm run typecheck - npm test(24 个测试文件,433 项通过) - git diff --cached --check fix(android): stabilize structured selectors and page-source reads Changes: - retain id-bearing Android nodes without labels or clickable state so structured selectors can resolve and tap them by center coordinates - apply Android idle, selector, and action acknowledgment timeouts through driver settings after session creation, cleaning up on failure - add Chinese OpenSpec artifacts, regression coverage, and selector/runner documentation Verification: - openspec validate fix-android-structured-selector-device-stability --strict - npm run typecheck - npm test (24 test files, 433 tests passed) - git diff --cached --check
中文 变更: - 覆盖结构化 scrollAssert 与上游锚点滚动参数同时存在的执行路径。 - 防止 rebase 冲突处理再次丢失 selector 或 target。 验证: - npm run build -w @appclaw/core - npx vitest run tests/flow/structured-selector-runtime.test.ts tests/flow/run-instruction-proximity.test.ts tests/flow/parse-yaml-flow.test.ts(44 项通过) English Changes: - Cover execution when structured scrollAssert selectors coexist with upstream anchored-scroll arguments. - Prevent future conflict resolution from dropping either selector or target data. Verification: - npm run build -w @appclaw/core - npx vitest run tests/flow/structured-selector-runtime.test.ts tests/flow/run-instruction-proximity.test.ts tests/flow/parse-yaml-flow.test.ts (44 tests passed)
qmkCamel
marked this pull request as ready for review
August 11, 2026 09:21
saikrishna321
approved these changes
Aug 15, 2026
github-actions Bot
pushed a commit
that referenced
this pull request
Aug 15, 2026
## [2.3.0](v2.2.0...v2.3.0) (2026-08-15) ### Features * **flow:** add structured YAML selectors and property assertions ([#56](#56)) ([10363cd](10363cd))
|
🎉 This PR is included in version 2.3.0 🎉 The release is available on:
Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds a deterministic accessibility-tree lane to YAML flows while preserving the existing natural-language/vision lane:
text,id,accessibilityId,type,hint, orvalueStructured targets fail deterministically when the accessibility tree cannot resolve them; they do not silently fall back to vision. Existing semantic string targets remain backward compatible.
Android stability fixes
Real-device validation exposed two Android-specific gaps that are included here:
This makes id-only child containers addressable and prevents dynamic apps from waiting for an idle accessibility tree before each page-source read.
Compatibility
scrollAsserttarget/target-proximity behavior from feat: Feat/anchored scroll assert #55scrollAssertusing both a structured selector and an anchored targetDocumentation
docs/structured-selectors.mdselector/assertion reference and migration guidancedocs/runner-architecture.mdAndroid hierarchy and post-session settings detailsfix-android-structured-selector-device-stabilityValidation
npm run typechecknpm test— 24 files, 448 tests passednpm run format:checkopenspec validate fix-android-structured-selector-device-stability --strictcom.smile.gifmaker: 8/8 steps passed; compound selectors, state/geometry assertions, andrightOfrelation verified (20260811T091013-9f73ee)com.kwai.gif.qmk.edge: two 8/8 structured-flow runs passed before the v2.2.0 rebase (20260811T083033-e6c872,20260811T083117-0a7d83)The post-rebase range-diff retains the Android fix unchanged; the only integration delta is preserving both the upstream anchored-scroll arguments and the new structured selector, covered by the focused compatibility test above.