Add Xcode Cloud screenshot fetch scripts and tests#17
Draft
gsbernstein wants to merge 9 commits into
Draft
Conversation
Introduce a Python CLI that downloads Xcode Cloud test result bundles via the App Store Connect API, extracts UI test screenshots with xcresulttool when available, and can post a summary comment to GitHub PRs. Add ci_post_xcodebuild.sh so extraction can run directly on the Xcode Cloud Mac using CI_RESULT_BUNDLE_PATH, avoiding a separate macOS runner. Include pytest coverage with mocked API responses for the fetch path. Co-authored-by: Greg <gsbernstein@users.noreply.github.com>
Expose a trigger-and-fetch CLI command that starts a workflow via the App Store Connect API, polls until the build run completes, and then downloads screenshots in one blocking invocation. Co-authored-by: Greg <gsbernstein@users.noreply.github.com>
Document the macOS-only xcresulttool extraction requirement prominently and make the recommended PR path extract-on-Xcode-Cloud, upload to a public bucket, then embed stable URLs in GitHub comments. Co-authored-by: Greg <gsbernstein@users.noreply.github.com>
Imgur only needs a free Client-ID for anonymous uploads, which is much lighter than S3. Auto-detect prefers Imgur when IMGUR_CLIENT_ID is set. Co-authored-by: Greg <gsbernstein@users.noreply.github.com>
Add BedtimeUITests with ScreenshotTests that capture home and settings screens via XCTAttachment (.keepAlways) for Xcode Cloud PR previews. Support -ui_testing launch argument with mock sleep data so simulator runs do not require HealthKit authorization. Remove the pytest suite for the Python fetch/upload helpers; those scripts remain as optional post-build tooling. Co-authored-by: Greg <gsbernstein@users.noreply.github.com>
Add trigger-screenshots.sh for branch/tag pushes that start a dedicated screenshots/* Xcode Cloud workflow without App Store Connect credentials. Parse What to test notes from commit messages and auto-detect PR context from CI_PULL_REQUEST_* or screenshots/pr-N branch names in ci_post_xcodebuild. Co-authored-by: Greg <gsbernstein@users.noreply.github.com>
Compare new screenshots against a per-PR baseline stored on the screenshot-baselines branch. Only changed and new screenshots appear in the PR comment table; unchanged runs get a short no-changes note. The same comment is updated in place each build via a hidden marker. Co-authored-by: Greg <gsbernstein@users.noreply.github.com>
Replace the screenshot-baselines JSON branch with sticky GitHub commit comments. Main builds embed Imgur URLs in the comment metadata; PR builds compare against CI_PULL_REQUEST_TARGET_COMMIT and post a before/after table on the PR head commit. Co-authored-by: Greg <gsbernstein@users.noreply.github.com>
Post terminal success/failed/no_screenshots commit comments from ci_post even when xcodebuild fails. Add hidden build-status metadata, comment-build and wait-for-commit-report CLI commands, and poll instructions in trigger-screenshots.sh. Co-authored-by: Greg <gsbernstein@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Xcode Cloud screenshot automation with Imgur + sticky GitHub commit comments, including a closed agent feedback loop.
Agent loop
Terminal commit comments
ci_post_xcodebuild.shalways posts a sticky commit comment whenGITHUB_TOKEN+CI_COMMITare set:successfailedCI_XCODEBUILD_EXIT_CODE != 0, extraction error, or missing Imgurno_screenshotsHidden metadata for agents:
<!-- bedtime-build-status {"status":"failed",...} -->Storage
CI_PULL_REQUEST_TARGET_COMMIT)Secrets
IMGUR_CLIENT_ID(success path)GITHUB_TOKEN(commit comments, success + failure)