Skip to content

fix: detect iOS TextView as editable - #51

Merged
saikrishna321 merged 1 commit into
appclawhq:mainfrom
qmkCamel:agent/fix-ios-krn-textview
Jul 31, 2026
Merged

saikrishna321 merged 1 commit into
appclawhq:mainfrom
qmkCamel:agent/fix-ios-krn-textview

Conversation

@qmkCamel

Copy link
Copy Markdown
Contributor

Summary

  • treat XCUIElementTypeTextView as editable in the normalized iOS page-source parser
  • mark TextView nodes as editable="true" in the trimmed DOM and include them in editableCount
  • add a regression test based on a real React Native/KRN composer page-source shape

Root cause

XCUITest exposes UIKit/React Native multiline inputs as XCUIElementTypeTextView. AppClaw's two iOS editable-type allowlists covered TextField, SecureTextField, TextEditor, and SearchField, but omitted TextView.

As a result, a visible and enabled composer such as kitchen-composer-text-input was parsed with editable=false, the trimmed screen reported editableCount=0, and YAML type steps failed with no editable field on screen even when the accessibility ID was exact.

Impact

YAML flows can now resolve and type into iOS multiline inputs used by UIKit, React Native, and KRN composers without falling back to direct XCUITest scripts.

Verification

  • npm test — 18 test files, 365 tests passed
  • Prettier check passed for all changed files
  • parsed a captured real-device KRN XML source with the patched build:
    • type=TextView
    • editable=true
    • action=type
    • target resolver returned kitchen-composer-text-input
    • trimmed DOM reported editableCount=1

Environment note

A full local CLI rerun against the device was attempted, but the current main dependency stack crashed inside appium-ios-remotexpc during WDA port forwarding, before AppClaw received any page source. The parser and resolver behavior was therefore verified against the captured real-device XML in addition to the regression test.

@qmkCamel
qmkCamel marked this pull request as ready for review July 31, 2026 04:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a gap in AppClaw’s iOS page-source normalization and DOM trimming so multiline iOS inputs exposed as XCUIElementTypeTextView are treated as editable targets (enabling YAML type steps to resolve and type into them).

Changes:

  • Extend the iOS editable-type allowlist to include TextView in parseIOSPageSource().
  • Extend the iOS DOM trimmer’s editable-type detection to mark TextView nodes as editable="true" (affecting editableCount and trimmed output).
  • Add a vitest regression test using a captured real-world React Native/KRN composer XML shape.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
tests/flow/ios-textview-editable.test.ts Adds regression coverage ensuring TextView is parsed/trimmed as editable and resolvable for type targeting.
packages/core/src/perception/ios-parser.ts Treats TextView as an editable element type so parsed UI elements get editable: true and action: 'type'.
packages/core/src/perception/dom-trimmer.ts Marks iOS TextView nodes as editable in trimmed DOM output and includes them in editableCount.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@saikrishna321
saikrishna321 merged commit 7215eef into appclawhq:main Jul 31, 2026
4 of 5 checks passed
github-actions Bot pushed a commit that referenced this pull request Jul 31, 2026
## [2.0.1](v2.0.0...v2.0.1) (2026-07-31)

### Bug Fixes

* detect iOS TextView as editable ([#51](#51)) ([7215eef](7215eef))
@github-actions

Copy link
Copy Markdown

🎉 This PR is included in version 2.0.1 🎉

The release is available on:

Your semantic-release bot 📦🚀

qmkCamel pushed a commit to qmkCamel/AppClaw that referenced this pull request Aug 4, 2026
中文:合并上游 PR appclawhq#51、appclawhq#52 以及 2.0.1、2.0.2 发布提交,保持 PR appclawhq#50 基于最新主分支。

English: Merge upstream PRs appclawhq#51 and appclawhq#52 plus the 2.0.1 and 2.0.2 release commits so PR appclawhq#50 is based on the latest main.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants