Skip to content

test: run App Intents through the system with AppIntentsTesting - #191

Merged
henrywang merged 1 commit into
mainfrom
test/189-app-intents-testing
Oct 10, 2026
Merged

henrywang merged 1 commit into
mainfrom
test/189-app-intents-testing

Conversation

@henrywang

Copy link
Copy Markdown
Owner

Closes #189

What

  • New BerthlyUITests/AppIntentsSystemTests.swift: two tests that drive Berthly's intents through the system, the path Shortcuts and Spotlight use.
    • Open Container: resolves the mock worker container through Berthly's entity query, runs the intent, and checks the detail pane shows "worker".
    • Start Container: starts the stopped worker, then checks the detail pane shows the Stop button.
  • Guards: #if canImport(AppIntentsTesting), because the framework ships only with Xcode 27, plus @available(macOS 27, *).
  • Safety: each test launches Berthly in mock mode first. The system routes intents to the running instance, so a system-launched instance never acts on a real daemon.
  • project.pbxproj: sets DEVELOPMENT_TEAM on the BerthlyUITests target only.

Why the signing change (spike result)

AppIntentsTesting answers only a test runner signed by the same development team as the app. Apple's WWDC26 session "Validate your App Intents adoption with AppIntentsTesting" states this, and I verified it locally:

Runner signing Result
Team-signed both tests pass
Ad-hoc (the previous default) AppIntentsServicesSecurityErrorDomain code 800, "Your app does not have permission to perform this"
Everything ad-hoc, as ci.yml builds same error

CI behavior

  • macOS 26 leg: compiles the file out (Xcode 26).
  • macOS 27 leg: builds ad-hoc, so the tests catch the code 800 error and XCTSkip. The run stays green.

So like BerthlyE2ETests, these run only on a developer's Mac, with no extra arguments: a normal ⌘U or xcodebuild test on Xcode 27 / macOS 27 includes them. Running them on CI would need a development certificate in CI secrets, which is not part of this PR.

They stay in BerthlyUITests, not BerthlyE2ETests. They use mock data, and scripts/e2e.sh re-signs its runner ad-hoc, which would make them skip.

Verification

  • Local, project signing (team runner): both tests pass.
  • Local, CI-style ad-hoc signing: both skip, and the run succeeds.
  • Full BerthlyUITests with the team-signed runner: 84 tests, 0 failures.
  • swiftlint lint --strict clean.

Local gotcha

If stale copies of app.berthly.Berthly are registered with LaunchServices (old DerivedData builds, an installed release), macOS may not see the intents, and the tests fail on "exactly one container named worker". Unregister the stale copies with lsregister -u (see the PR #188 description).

🤖 Generated with Claude Code

Adds AppIntentsSystemTests (Open and Start Container via IntentDefinitions), compiled only with Xcode 27. The system answers only a runner signed by the app's team, so the UI test target now sets DEVELOPMENT_TEAM; CI builds ad-hoc and the tests skip there on the code 800 permission error.
@henrywang henrywang added this to the v1.5.0 milestone Oct 10, 2026
@henrywang
henrywang merged commit 2ec2b6c into main Oct 10, 2026
7 checks passed
@henrywang
henrywang deleted the test/189-app-intents-testing branch October 10, 2026 12:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Spike: cover App Intents with AppIntentsTesting on the macOS 27 CI leg

1 participant