feat(testing): add MeshtasticUITests accessibility-tree navigation driver - #2180
Open
bruschill wants to merge 6 commits into
Open
feat(testing): add MeshtasticUITests accessibility-tree navigation driver#2180bruschill wants to merge 6 commits into
bruschill wants to merge 6 commits into
Conversation
…iver
Adds a new UI test target for driving the app through real
XCUIApplication element queries — the same path VoiceOver and a real
user's taps take — rather than in-process shortcuts.
- MeshtasticUITests/AccessibilityDriver.swift: NavigationStep enum
(tab, tapIdentifier, tapButtonLabeled, waitForIdentifier, pause) +
a runner that fails loudly (not silently) when a step's target
never appears, since an unreachable target is itself a finding
(broken navigation or an accessibility regression), not something
to skip past.
- MeshtasticUITests/TabBarNavigationUITests.swift: a real regression
test exercising the driver — walks all five root tabs and asserts
each is reachable through the accessibility tree. Bootstraps via
the existing --meshtastic-marketing-seed launch mode (skips
onboarding, disables BLE discovery/autoconnect) so tab switches
aren't racing a connect attempt or blocked behind a first-launch
sheet.
- ContentView.swift: tag each root tab bar button with a
.accessibilityIdentifier("tab-*"). Confirmed via a live accessibility
hierarchy dump during development that SwiftUI's value-based
Tab(value:) API does not currently propagate this identifier to the
underlying UITabBarButton (Xcode 26.6/iOS 18) — NavigationStep.tab
therefore matches on the visible title as a documented, working
fallback. The identifiers are left in place: harmless today, and
picked up automatically if a future SwiftUI version fixes the gap.
- project.yml: new MeshtasticUITests target (bundle.ui-testing,
TEST_TARGET_NAME: Meshtastic) wired into the Meshtastic scheme's
test action. Regenerated Meshtastic.xcodeproj with the pinned
XcodeGen 2.46.0 (verified zero drift both before and after this
change).
Verified: build-for-testing succeeds, TabBarNavigationUITests passes
end-to-end against a real simulator run (all 5 tabs reached). Full
MeshtasticTests suite run for regression: same 3 pre-existing failures
(CoreDataMigrationServiceTests x2, IntervalConfigurationDetailedTests
.allCases_count) reproduce identically on a clean upstream/main
checkout, confirming they're unrelated to this change.
Adds a "UI Tests (MeshtasticUITests)" section to docs/developer/testing.md: what the target is for, why it deliberately uses XCTestCase instead of Swift Testing (XCUITest has no Swift Testing equivalent), a usage example for AccessibilityDriver.run(), the known SwiftUI Tab(value:) identifier-propagation gap that makes tab lookup match on visible title today, and how to run the target. Regenerated the bundled docs via `bash scripts/build-docs.sh --output Meshtastic/Resources/docs` per the repo's docs-staleness convention.
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughAdds the ChangesUI testing
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant TabBarNavigationUITests
participant AccessibilityDriver
participant Meshtastic
TabBarNavigationUITests->>Meshtastic: launch app
TabBarNavigationUITests->>AccessibilityDriver: run tab steps
AccessibilityDriver->>Meshtastic: query and tap root tabs
Meshtastic-->>AccessibilityDriver: expose accessibility elements
AccessibilityDriver-->>TabBarNavigationUITests: report missing targets via XCTFail
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@Meshtastic/Resources/docs/markdown/developer/testing.md`:
- Around line 141-151: Reconcile the contradictory CLI guidance in the testing
documentation by narrowing the earlier “no CLI test runner” statement or
replacing it with the existing xcodebuild command reference. Keep the UI test
instructions consistent, then regenerate the bundled documentation after
updating the testing content.
- Around line 121-139: Pin the UI test locale to English by adding
-AppleLanguages and -AppleLocale launch arguments to testSomeFlowIsReachable,
and configure the Meshtastic test scheme/action with the same settings. Keep
NavigationStep.tab title matching unchanged unless migrating the tabs to stable
identifiers.
In `@MeshtasticUITests/TabBarNavigationUITests.swift`:
- Around line 28-35: Pin the UI test locale to English before launching the app
in the setup containing the --meshtastic-marketing-seed launch argument, so
NavigationStep.tab can reliably resolve the visible English titles. Keep the
existing title-based tab lookup and navigation assertions unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 2d4cfd16-c14a-4ce2-8589-b5700b404093
📒 Files selected for processing (10)
Meshtastic.xcodeproj/project.pbxprojMeshtastic.xcodeproj/xcshareddata/xcschemes/Meshtastic.xcschemeMeshtastic/Resources/docs/developer/testing.htmlMeshtastic/Resources/docs/index.jsonMeshtastic/Resources/docs/markdown/developer/testing.mdMeshtastic/Views/ContentView.swiftMeshtasticUITests/AccessibilityDriver.swiftMeshtasticUITests/TabBarNavigationUITests.swiftdocs/developer/testing.mdproject.yml
CodeRabbit flagged two issues on PR meshtastic#2180: - docs/developer/testing.md's "Running Tests" section claimed "there is no CLI test runner — tests require Xcode", directly contradicting the new "Running UI Tests" section a few paragraphs down, which documents the xcodebuild test invocation actually used to verify this PR and run CI. Reworded to point at that section instead of denying a CLI path exists. - NavigationStep.tab matches tab bar buttons by visible English title (a documented SwiftUI Tab(value:) accessibilityIdentifier propagation gap), which made testAllRootTabsAreReachable() locale-dependent — it'd flake on a non-English simulator/device. Pinned the test's launch to English via -AppleLanguages "(en)" and -AppleLocale en_US alongside the existing --meshtastic-marketing-seed argument, and updated the AccessibilityDriver doc example to show the same pattern. Regenerated bundled docs via scripts/build-docs.sh. Verified with xcodebuild test against MeshtasticUITests/TabBarNavigationUITests (iPhone 17 simulator) — passes. swiftlint lint clean on the changed test file.
…tests # Conflicts: # Meshtastic/Resources/docs/index.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed?
Adds a new
MeshtasticUITeststarget (XCUITest, wired into theMeshtasticscheme's test action) built around a small, reusable accessibility-tree-driven navigator:AccessibilityDriver.swift—NavigationStep(tab,tapIdentifier,tapButtonLabeled,waitForIdentifier,pause) and arun(_:app:)that drivesXCUIApplicationthrough real element queries and taps — the same path VoiceOver and a real user's touches take. A step whose target never appears fails the test loudly (not silently skipped) — an unreachable target is itself a finding.TabBarNavigationUITests.swift— a real regression test built on the driver: walks all five root tabs (Messages, Nodes, Map, Settings, Connect) and asserts each is reachable through the accessibility tree.ContentView.swift— tags each root tab bar button with.accessibilityIdentifier("tab-*"). While building this I confirmed (by dumping the live accessibility hierarchy during a real run) that SwiftUI's value-basedTab(value:)API does not currently propagate that identifier down to the underlyingUITabBarButtonon Xcode 26.6/iOS 18 — the driver's.tabstep therefore matches on the visible title as a documented, working fallback. The identifiers are left in place: harmless today, and picked up automatically if a future SwiftUI version fixes the gap.project.yml— newMeshtasticUITeststarget (bundle.ui-testing,TEST_TARGET_NAME: Meshtastic). RegeneratedMeshtastic.xcodeprojwith the pinned XcodeGen 2.46.0.docs/developer/testing.md— new "UI Tests (MeshtasticUITests)" section covering what the target is for, why it deliberately usesXCTestCaseinstead of Swift Testing (XCUITest has no Swift Testing equivalent — this is an intentional, documented exception to the "noXCTAssert*" rule above it, not drift from it), a usage example, theTab(value:)identifier gap, and how to run it.This is deliberately not the same mechanism as
MarketingCapture(Meshtastic/Persistence/MarketingCapture.swift), which navigates in-process viaRouter— fast and great for a fixed, curated App Store screenshot list, but it bypasses real touch dispatch, hit-testing, and accessibility traits entirely.Why did it change?
This is infrastructure: a reusable way to drive the app through its actual accessibility tree rather than in-process shortcuts, so a passing UI test also proves the target is genuinely reachable and tappable the way VoiceOver and a real user experience it — catching hit-testing/accessibility-trait regressions that router-driven navigation can't.
TabBarNavigationUITestsis the first real consumer and gives permanent regression coverage for the root tab bar. (A follow-up PR builds a PR-review screenshot-capture pipeline on top of this driver — kept separate since this piece stands on its own.)How is this tested?
xcodebuild build-for-testingsucceeds for theMeshtasticscheme.TabBarNavigationUITests.testAllRootTabsAreReachablerun against a real simulator (iPhone 17): passes, all 5 tabs reached via real accessibility-tree taps.MeshtasticTestssuite run for regression: same 3 pre-existing failures (CoreDataMigrationServiceTestsx2,IntervalConfigurationDetailedTests.allCases_count) reproduce identically on a cleanupstream/maincheckout, confirming they're unrelated to this change.xcodegen generatewith the pinned 2.46.0 verified to produce zero drift against the committed project both before and after this change.Screenshots/Videos (when applicable)
N/A — this PR adds test infrastructure, not user-visible UI.
Checklist
docs/user/ordocs/developer/, and updated accordingly (see copilot-instructions.md for the view → doc page mapping). If no doc update is needed, add theskip-docs-checklabel.Summary by CodeRabbit
New Features
Tests
Documentation
xcodebuildcommands) and how accessibility-based tab navigation works.