artie — Remote Test Executor

A CLI + server pair that lets CI jobs inside Tartelet macOS VMs dispatch Android Gradle tests and iOS Xcode tests (XCUITest, Screenshot tests) to a bare-metal Mac on the LAN. Results stream back as if the tests ran locally — no changes to build.gradle.kts or Xcode project files required.

artie — pronounced ar-tee. A Remote Test Executor (RTE).

Why artie exists

Android — the HVF problem

Inside a Tartelet macOS VM on Apple Silicon, nested hardware virtualisation (HVF) is not exposed by Apple's Virtualization.framework. The Android emulator needs HVF to boot; it simply cannot start inside the VM. Bare-metal Macs on the same LAN can run the emulator natively. artie bridges the two without touching your Gradle configuration.

iOS — rendering fidelity and performance

iOS Simulators can technically run inside a Tartelet VM, but two problems make it impractical:

Screenshot test flakiness. Snapshot comparison tests (swift-snapshot-testing, etc.) are pixel-sensitive. Rendering in a virtualised GPU produces images that differ from reference snapshots captured on real hardware, causing false failures.
Speed. A UI test suite that takes 4 min on bare metal typically takes 12–18 min inside a VM due to CPU and GPU overhead.

artie offloads both types of tests to the same pool of bare-metal Macs.

How it works

Tartelet VM                              Bare-metal Mac
──────────────────────────────           ─────────────────────────────────────
artie execute \                  artie-server (daemon)
  --tasks :UITests:…      ─────────────▶│  :9443  gRPC/TLS   (events, logs)
                                         │  :9444  HTTPS/TLS  (tarball, artifacts)
artie xcodebuild \                      │
  --scheme MyApp          ─────────────▶│  Bonjour _artie._tcp
  --destination "…"                     │  advertises capabilities:
                                         │    android: aosp/arm64-v8a/api-34
                                         │    ios:     simulator/arm64/18.0

Both commands follow the same flow:
  1. Bonjour discover → score by queue depth → pick best server
  2. gRPC stream opened (ExecuteGradle or ExecuteXcodebuild)
  3. HTTPS PUT project tarball in 8 parallel parts
  4. Server runs ./gradlew … or xcodebuild …
  5. LogLine / TaskStarted / TaskFinished / TestCaseResult stream back
  6. ArtifactReady events → HTTPS GET artifacts in 4 parallel parts
  7. exit with the tool's own exit code

Authentication: every gRPC call carries Authorization: Bearer <secret>. TLS encrypts the wire; the shared secret authenticates the caller.

Repository layout

artie/
├── proto/
│   ├── artie.proto          gRPC service (ExecuteGradle + ExecuteXcodebuild)
│   └── generate.sh          Runs protoc; copies stubs into both sub-projects
│
├── artie-cli/               Swift Package — CLI binary (Mint-compatible)
│   └── Sources/ArtieCLI/
│       ├── ArtieCLI.swift   @main entry point
│       ├── Commands/        execute / xcodebuild subcommands
│       ├── Discovery/       Bonjour browser + server scoring
│       ├── Upload/          Parallel Content-Range PUTs with per-part retry
│       ├── Download/        Parallel artifact download with sha256 verification
│       ├── Tarball/         Project packager (tar + sha256)
│       └── Generated/       grpc-swift v2 stubs (do not edit)
│
├── artie-server/            Swift Package — server library + headless daemon
│   └── Sources/
│       ├── ArtieServer/     Core server library (no SwiftUI dependency)
│       │   ├── App/         ServerConfig, UpdateController
│       │   ├── Server/      gRPC server (:9443), HTTPS file server (:9444)
│       │   ├── Queue/       Actor-based FIFO queue, slot pool
│       │   ├── Execution/   GradleRunner, XcodebuildRunner (planned),
│       │   │                ArtifactWatcher, XcresultWatcher (planned),
│       │   │                SimulatorManager (planned)
│       │   ├── Bonjour/     Bonjour publisher, TXT record updated every 5 s
│       │   ├── Security/    Self-signed cert generation, HMAC-SHA256 signed URLs
│       │   ├── UI/          MenuBarView (status, Start/Stop/Restart), DashboardView, SettingsView
│       │   └── Generated/   grpc-swift v2 stubs (do not edit)
│       └── ArtieServerDaemon/  Headless executable (no SwiftUI)
│
├── scripts/
│   ├── build.sh             Build both binaries → dist/ (supports --clean)
│   ├── generate-certs.sh    Generate self-signed TLS certs
│   └── bootstrap.sh         One-shot setup: proto generation + certs
│
├── dist/                    Staged binaries (produced by build.sh)
│   ├── artie
│   └── artie-server
│
└── spec/artie-spec.md       Full design specification

Prerequisites

Every machine

macOS 15+
Xcode 16+ (Swift 5.10)

Proto generation (dev only)

brew install protobuf swift-protobuf protoc-gen-grpc-swift

Server machine — Android

Android SDK at $HOME/Library/Android/sdk
JDK (bundled with Android Studio): $HOME/Applications/Android Studio.app/Contents/jbr/Contents/Home
Android system images for the API levels you test against
avdmanager on PATH (for capability detection)

Server machine — iOS

Xcode 16+ at /Applications/Xcode.app
iOS Simulator runtimes for each version you test against
(install via Xcode → Settings → Platforms, or xcrun simctl runtime add iOS-18-0)

A single bare-metal Mac can serve both Android and iOS jobs simultaneously.

Installation

1. Bootstrap (first time, from repo root)

./scripts/bootstrap.sh

Generates proto stubs into both artie-cli/…/Generated/ and artie-server/…/Generated/.

2. Build

# Release build — produces dist/artie and dist/artie-server
./scripts/build.sh release

# Clean rebuild
./scripts/build.sh release --clean

Or build individually:

cd artie-cli   && swift build -c release
cd artie-server && swift build -c release --product ArtieServerDaemon

3. Install the CLI

cp dist/artie /usr/local/bin/artie

Mint

mint install num42/artie

4. Install and start the server

artie-server comes in two flavours — use whichever fits the machine:

Variant	When to use
`artie-server.app`	The bare-metal Mac has a logged-in user. The menu-bar app provides live monitoring and one-click start/stop/restart.
`artie-server` (headless daemon)	Fully unattended / SSH-only machines. No UI; controlled entirely via environment variables and CLI flags.

Headless daemon

scp dist/artie-server admin@build-mac.local:~/artie-server

# First launch — paths are persisted in UserDefaults for future restarts
ssh admin@build-mac.local \
  "ARTIE_SECRET=your-secret ~/artie-server \
   --java-home \"$HOME/Applications/Android Studio.app/Contents/jbr/Contents/Home\" \
   --android-home \"$HOME/Library/Android/sdk\" \
   --developer-dir /Applications/Xcode.app/Contents/Developer &"

# Subsequent launches (paths already saved)
ssh admin@build-mac.local "ARTIE_SECRET=your-secret ~/artie-server &"

macOS App

Copy artie-server.app to /Applications on the bare-metal Mac and double-click it. On first launch:

Open Settings (⌘,) and paste your shared secret (or click Generate New).
Optionally set ARTIE_SECRET, JAVA_HOME, ANDROID_HOME, and DEVELOPER_DIR in the app's launch environment (e.g. via a launchd plist) so values survive reinstalls.

The app starts the service automatically on launch. Use Restart Service in the menu bar whenever you change a setting.

On first launch (either variant) the server:

Generates a self-signed TLS cert at ~/Library/Application Support/Artie/certs/.
Starts gRPC on :9443 and the HTTPS file server on :9444.
Publishes _artie._tcp. via Bonjour with slot, queue, and capability metadata.
Detects Android API levels (avdmanager) and iOS simulator runtimes (xcrun simctl) automatically.

5. Configure the secret on clients

The CLI reads the secret in priority order:

Source	How to set
`--secret <value>`	Pass inline
`ARTIE_SECRET` env var	Export in shell or CI environment
`~/.config/artie/secret` file	Write once, reused automatically

mkdir -p ~/.config/artie
echo "your-secret" > ~/.config/artie/secret

Usage

Android — `artie execute`

Dispatches a Gradle task to a bare-metal Mac. The server's existing testOptions.managedDevices block runs unchanged.

# Single device
artie execute \
  --project Android \
  --tasks ":UITests:pixel5api34HammLocalDebugAndroidTest" \
  --out Android

# Two devices (both tasks run on the server in a single job)
artie execute \
  --project Android \
  --tasks ":UITests:pixel5api34HammLocalDebugAndroidTest" \
          ":UITests:pixel5api35HammLocalDebugAndroidTest" \
  --out Android

# Target a specific server, skip Bonjour
artie execute \
  --server build-mac-01.local:9443 \
  --project Android \
  --tasks ":UITests:pixel5api34HammLocalDebugAndroidTest" \
  --out Android

Option	Default	Description
`--project <path>`	`.`	Path to the Gradle project directory.
`--tasks <task…>`	(required)	One or more Gradle task paths passed verbatim to `./gradlew`.
`--out <path>`	`.`	Directory where JUnit XML and test reports are written.
`--secret <value>`	(see above)	Shared secret.
`--server <host:port>`	(auto)	Skip Bonjour; connect to a specific server.
`--discovery-window <s>`	`10`	Seconds to browse via Bonjour.
`--upload-parallelism <n>`	`8`	Concurrent tarball PUT parts.
`--download-parallelism <n>`	`4`	Concurrent artifact GETs.
`--max-server-wait <s>`	`300`	Abort if no slot is free. `0` = unlimited.
`--device <cap>`	(any)	Capability filter, e.g. `aosp/arm64-v8a/api-34`.
`--project-id <id>`	(dir name)	Identifier for server-side queue fairness.
`--timeout <s>`	`3600`	Gradle execution timeout.

iOS — `artie xcodebuild`

Dispatches an xcodebuild test job to a bare-metal Mac. The server builds the project, boots an iOS Simulator, runs the tests, and streams the .xcresult bundle and failure screenshots back.

UI tests

artie xcodebuild \
  --project iOS \
  --workspace MyApp.xcworkspace \
  --scheme MyApp \
  --destination "platform=iOS Simulator,OS=18.0,name=iPhone 16" \
  --test-plan UITests \
  --out TestResults

Screenshot tests

artie xcodebuild \
  --project iOS \
  --workspace MyApp.xcworkspace \
  --scheme MyApp \
  --destination "platform=iOS Simulator,OS=18.0,name=iPhone 16" \
  --test-plan ScreenshotTests \
  --out TestResults

# Failure screenshots land in TestResults/Attachments/ automatically.

Recording / updating reference snapshots

artie xcodebuild \
  --project iOS \
  --workspace MyApp.xcworkspace \
  --scheme MyApp \
  --destination "platform=iOS Simulator,OS=18.0,name=iPhone 16" \
  --test-plan ScreenshotTests \
  --xcodebuild-arg "-testenv SNAPSHOT_RECORD=1" \
  --out TestResults

# New reference images come back as artifacts in TestResults/.
# Commit them to the repo from there.

Test-without-building (faster if build already done)

artie xcodebuild \
  --project iOS \
  --workspace MyApp.xcworkspace \
  --scheme MyApp \
  --destination "platform=iOS Simulator,OS=18.0,name=iPhone 16" \
  --mode test-without-building \
  --xctestrun iOS/DerivedData/Build/Products/MyApp.xctestrun \
  --out TestResults

Run a single test class

artie xcodebuild \
  --project iOS \
  --workspace MyApp.xcworkspace \
  --scheme MyApp \
  --destination "platform=iOS Simulator,OS=18.0,name=iPhone 16" \
  --only-testing UITests/LoginTests \
  --out TestResults

Option	Default	Description
`--project <path>`	`.`	Directory to pack and dispatch. Must contain the workspace/project.
`--workspace <path>`	(see note)	`.xcworkspace` path relative to `--project`. Mutually exclusive with `--xcode-project`.
`--xcode-project <path>`	(see note)	`.xcodeproj` path relative to `--project`. Mutually exclusive with `--workspace`.
`--scheme <name>`	(required)	Xcode scheme to build and test.
`--destination <string>`	(required)	xcodebuild destination, e.g. `"platform=iOS Simulator,OS=18.0,name=iPhone 16"`.
`--configuration <name>`	`Debug`	Build configuration.
`--test-plan <name>`	(none)	Name of an `.xctestplan` in the scheme.
`--only-testing <target>`	(all)	Filter to a test target/class/method. Repeatable.
`--skip-testing <target>`	(none)	Exclude a test target/class/method. Repeatable.
`--xcodebuild-arg <arg>`	(none)	Extra argument passed verbatim to `xcodebuild`. Repeatable.
`--mode`	`build-and-test`	`build-and-test` or `test-without-building`.
`--xctestrun <path>`	(required for test-without-building)	Path to `.xctestrun` inside `--project`.
`--out <path>`	`.`	Directory where `.xcresult`, JUnit XML, and screenshots are written.
`--secret <value>`	(see above)	Shared secret.
`--server <host:port>`	(auto)	Skip Bonjour; connect to a specific server.
`--discovery-window <s>`	`10`	Seconds to browse via Bonjour.
`--upload-parallelism <n>`	`8`	Concurrent tarball PUT parts.
`--download-parallelism <n>`	`4`	Concurrent artifact GETs.
`--max-server-wait <s>`	`300`	Abort if no slot is free.
`--ios-runtime <version>`	(from destination)	Override iOS runtime version for capability filtering, e.g. `18.0`.
`--timeout <s>`	`3600`	xcodebuild execution timeout.

iOS artifacts

File	Description
`test.xcresult`	Full Xcode result bundle. Import into Xcode or parse with `xcresultkit`.
`test-results.junit.xml`	JUnit XML extracted from the xcresult; ready for CI parsers without Xcode tooling.
`Attachments/<name>.png`	Per-test-case failure screenshots, streamed back progressively as they appear.
`xcodebuild.log`	Raw xcodebuild output for debugging.

Exit codes

Code	Meaning
`0`	Tests passed.
`1`–`125`	Tool's own exit code (test failures, compilation errors, etc.).
`65`	xcodebuild `BUILD_FAILED`.
`124`	Timed out on the server.
`1` + `ARTIE_NO_SERVER`	No server discovered on the LAN.
`1` + `ARTIE_NO_IOS_RUNTIME`	No server has the required iOS simulator runtime.
`1` + `ARTIE_TIMEOUT_QUEUED`	Timed out waiting for a server slot.

Failure behaviour

Failure	What artie does
No server discovered	Exits `1` with `ARTIE_NO_SERVER`.
All servers full, `--max-server-wait` exceeded	Cancels the stream, exits `1` with `ARTIE_TIMEOUT_QUEUED`.
Server RPC error (UNAVAILABLE)	Retries once on the next-best ranked server.
Wrong or missing secret	Server returns `UNAUTHENTICATED`; artie exits `1`.
Upload checksum mismatch	Retries the failing part once.
Gradle exit != 0	Streams artifacts as usual, exits with the same code.
xcodebuild BUILD_FAILED	Streams partial logs, exits `65`.
Simulator boot timeout (>120 s)	Server kills the job, exits `124`.
No iOS runtime on server	Server returns `RESOURCE_EXHAUSTED`; client tries next server.

CI Integration

Add ARTIE_SECRET as a repository secret (Settings → Secrets and variables → Actions).

Android

- name: Android UI tests
  env:
    ARTIE_SECRET: ${{ secrets.ARTIE_SECRET }}
  run: |
    artie execute \
      --project Android \
      --tasks ":UITests:pixel5api34HammLocalDebugAndroidTest" \
      --out Android

iOS — UI tests

- name: iOS UI tests
  env:
    ARTIE_SECRET: ${{ secrets.ARTIE_SECRET }}
  run: |
    artie xcodebuild \
      --project iOS \
      --workspace MyApp.xcworkspace \
      --scheme MyApp \
      --destination "platform=iOS Simulator,OS=18.0,name=iPhone 16" \
      --test-plan UITests \
      --out TestResults

- name: Upload test results
  if: always()
  uses: actions/upload-artifact@v4
  with:
    name: test-results-ios
    path: TestResults/test.xcresult

iOS — Screenshot tests

- name: Screenshot tests
  env:
    ARTIE_SECRET: ${{ secrets.ARTIE_SECRET }}
  run: |
    artie xcodebuild \
      --project iOS \
      --workspace MyApp.xcworkspace \
      --scheme MyApp \
      --destination "platform=iOS Simulator,OS=18.0,name=iPhone 16" \
      --test-plan ScreenshotTests \
      --out TestResults

- name: Upload failure screenshots
  if: failure()
  uses: actions/upload-artifact@v4
  with:
    name: screenshot-diffs
    path: TestResults/Attachments/

Workarounds you can remove once artie is live

continue-on-error: true on test steps → drop it.
android.testoptions.manageddevices.emulator.gpu=swiftshader_indirect → drop it.
TODO comments about nested-HVF or VM GPU rendering → drop them.

Server configuration

Settings are persisted in UserDefaults. The macOS .app exposes a Settings window (⌘,); the daemon reads the same defaults plus environment variables.

Key	Default	Description
`grpcSecret`	(auto-generated UUID)	Shared secret. `ARTIE_SECRET` env var takes precedence.
`javaHome`	(env `JAVA_HOME`)	JDK path for Gradle subprocesses. `--java-home` on first launch.
`androidHome`	(env `ANDROID_HOME`)	Android SDK path. `--android-home` on first launch.
`developerDir`	(env `DEVELOPER_DIR`)	Xcode developer dir for all `xcodebuild`/`xcrun` calls. `--developer-dir` on first launch.
`slotsTotal`	`4`	Maximum concurrent jobs (Android + iOS combined).
`maxQueueDepth`	`16`	Maximum jobs waiting. Beyond this `RESOURCE_EXHAUSTED` is returned.
`maxConcurrentSimulators`	`2`	Hard cap on simultaneously booted iOS Simulators.
`simBootTimeout`	`120`	Seconds to wait for a Simulator to reach `Booted` before aborting.
`grpcPort`	`9443`	gRPC listener port.
`filesPort`	`9444`	HTTPS file server port.
`hostID`	(hostname)	Stable identifier in logs and client output.

Pinning the secret

# launchd plist or startup script
ARTIE_SECRET=your-fixed-secret ~/artie-server

Set the same value in GitHub Actions secrets and neither side needs to change again.

Checking advertised capabilities

dns-sd -B _artie._tcp.
dns-sd -L "build-mac-01" _artie._tcp.
# Look for android_caps=… and ios_caps=… in the TXT record

Adding a second server

Launch artie-server on another bare-metal Mac with the same secret. Clients discover it automatically via Bonjour. No other configuration needed. Servers that only have Xcode (no Android SDK) will only appear in discovery for iOS jobs, and vice versa.

artie-server macOS App

artie-server ships as a native macOS menu-bar app (artie-server.app) as well as a headless daemon (artie-server). Both run the same server stack; the app adds a live UI for monitoring and control without needing SSH access to the build machine.

Menu bar

The menu-bar icon shows a green dot when the service is running and a red dot when it is stopped. Clicking it opens a popover with a live snapshot updated every 5 seconds:

● Running
─────────────────────────────
Slots:  2/4 free
Queue:  1
─────────────────────────────
CPU: 38%   RAM: 1 204 MB
─────────────────────────────
Recent jobs
✓  clinic-android   :UITests:pixel5…   2:14
✓  clinic-ios       UITests            4:07
✗  kali-android     :UITests:pixel5…   0:53
─────────────────────────────
[Restart Service]
[Stop Service]
─────────────────────────────
[Copy Secret]  [Open Dashboard]  [Quit]

Starting, stopping, and restarting

The menu bar exposes full dynamic service control — no Terminal required:

State	Available actions
Running	Restart Service — stops and immediately restarts the gRPC + file server (picks up config changes). Stop Service — shuts down both listeners and unpublishes the Bonjour record; in-flight jobs finish before the port closes.
Stopped	Start Service — boots both listeners and re-publishes the Bonjour record.

Restart is the normal way to apply a config change (e.g. after updating the slot count or secret in Settings). The Bonjour record is re-published with the new values after restart.

Dashboard

Click Open Dashboard to open a full-window job browser:

Sidebar — all jobs from this session, sorted newest-first, each showing project, tasks or scheme, relative start time, duration, and a pass/fail/running indicator.
Detail pane — job ID, project, tasks or scheme, start time, outcome, and duration for the selected job.
Toolbar — live slots-free/total and queue-depth counters.

Settings (`⌘,`)

Authentication Secret — view, copy, edit, or generate a new UUID secret. Changes take effect immediately; running jobs are not interrupted. The ARTIE_SECRET environment variable always takes precedence over the stored value.

Log files

~/Library/Application Support/Artie/logs/artie-server.jsonl

Structured JSON, one line per event: ts, level, msg, source, job_id, host_id, platform, scheme or tasks, destination. Retained for 7 days.

Development

Proto changes

Edit proto/artie.proto.
Run ./proto/generate.sh — updates stubs in both sub-projects.
Adapt Swift source.

Running the server locally

cd artie-server
ARTIE_SECRET=dev-secret swift run ArtieServerDaemon

Point the CLI at it directly (skipping Bonjour):

# Android
ARTIE_SECRET=dev-secret artie execute \
  --server localhost:9443 \
  --project /path/to/android \
  --tasks assembleDebug \
  --out /tmp/results

# iOS
ARTIE_SECRET=dev-secret artie xcodebuild \
  --server localhost:9443 \
  --project /path/to/ios \
  --workspace MyApp.xcworkspace \
  --scheme MyApp \
  --destination "platform=iOS Simulator,OS=18.0,name=iPhone 16" \
  --out /tmp/results

Clean rebuild

./scripts/build.sh release --clean

Releases with Fastlane

CLI

cd artie-cli
bundle exec fastlane release version:1.2.0

Builds a release binary, code-signs it, notarizes the zip, and creates a GitHub release.

Server

cd artie-server
bundle exec fastlane release version:1.2.0

Archives the .app, signs and notarizes it, packages a .dmg, updates appcast.xml for Sparkle, and creates a GitHub release.

Required environment variables

Variable	Description
`CODESIGN_IDENTITY`	`Developer ID Application: num42 GmbH (TEAMID)`
`ASC_PROVIDER`	App Store Connect team ID
`ASC_API_KEY_PATH`	Path to an App Store Connect API key JSON file
`GITHUB_TOKEN`	GitHub personal access token for creating releases

Security

TLS encrypts all gRPC and file-transfer traffic. The server generates a self-signed cert on first launch; clients skip hostname verification (LAN-only deployment).
Every gRPC call is authenticated with a shared Authorization: Bearer <secret>. The server returns UNAUTHENTICATED on mismatch.
HMAC-SHA256 signed URLs for the file endpoint; tokens are scoped to a single URL + job ID + expiry and rotate daily with a 5-minute grace window.
Workspaces are wiped at job end. No secrets from the tarball persist on disk.
gRPC reflection is disabled in production builds.
File endpoint enforces: 2 GB max tarball, 500 MB max single artifact.

Name		Name	Last commit message	Last commit date
Latest commit History 15 Commits
.github/workflows		.github/workflows
artie-cli		artie-cli
artie-server		artie-server
dist		dist
proto		proto
scripts		scripts
spec		spec
.gitignore		.gitignore
Mintfile		Mintfile
Package.swift		Package.swift
README.md		README.md

Uh oh!

Folders and files

Latest commit

History

Repository files navigation

artie — Remote Test Executor

Why artie exists

Android — the HVF problem

iOS — rendering fidelity and performance

How it works

Repository layout

Prerequisites

Every machine

Proto generation (dev only)

Server machine — Android

Server machine — iOS

Installation

1. Bootstrap (first time, from repo root)

2. Build

3. Install the CLI

Mint

4. Install and start the server

Headless daemon

macOS App

5. Configure the secret on clients

Usage

Android — artie execute

iOS — artie xcodebuild

UI tests

Screenshot tests

Recording / updating reference snapshots

Test-without-building (faster if build already done)

Run a single test class

iOS artifacts

Exit codes

Failure behaviour

CI Integration

Android

iOS — UI tests

iOS — Screenshot tests

Workarounds you can remove once artie is live

Server configuration

Pinning the secret

Checking advertised capabilities

Adding a second server

artie-server macOS App

Menu bar

Starting, stopping, and restarting

Dashboard

Settings (⌘,)

Log files

Development

Proto changes

Running the server locally

Clean rebuild

Releases with Fastlane

CLI

Server

Required environment variables

Security

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases 4

Packages 0

Uh oh!

Uh oh!

Contributors

Uh oh!

Languages

Android — `artie execute`

iOS — `artie xcodebuild`

Settings (`⌘,`)

Packages