Setup: provide approved Appium and Android emulator toolchain across CLI, Java, MCP, and engine

[MohabMohie]

Objective

Implement the MOBILE_ANDROID setup profile promised by #4849 so CLI, Java, MCP, and engine runtime share one pinned Appium/Android owner. A reviewed plan must be able to install and verify Appium 3, the Inspector plugin, UiAutomator2, Android command-line tools, platform-tools, API platform, build-tools/aapt2, Emulator, one host-compatible system image, and a SHAFT-owned AVD; start/stop/logs must own the emulator and Appium server without hidden downloads or global environment mutation.

Parent: #4849

Current evidence (researched 2026-08-13)

The only current installer is package-private McpMobileToolchainService. It:

• downloads unverified Node and Android command-line-tools archives;
• runs mutable npm install, then Appium extension CLI installs;
• scripts y into sdkmanager --licenses without binding accepted license IDs to the reviewed plan;
• installs platform-tools, emulator, platforms;android-<api>, and a system image, but omits build-tools, so aapt2 is unavailable (the concrete Establish honest real-provider acceptance for Appium screen recording #4751 failure);
• deletes/replaces extraction targets without shared setup receipts/rollback;
• treats any adb ... device row as emulator readiness and never proves sys.boot_completed=1;
• starts Appium with --relaxed-security by default and owns neither durable lease nor log identity;
• is usable only through the MCP inspector-recording path. CLI/Java report MOBILE_ANDROID unsupported.

Current release pins are Node 24.19.0, Appium 3.6.0, Inspector plugin 2026.7.1, UiAutomator2 8.2.2, Android command-line tools 15859902, API 36, device pixel_8, tag google_apis, RAM 4096, cores 2. npm metadata confirms the three Appium packages support Node 24 and Appium 3.

Primary contracts:

• Appium installation and independent driver ownership: https://appium.io/docs/en/latest/quickstart/install/ and https://appium.io/docs/en/latest/quickstart/uiauto2-driver/
• Appium project dependency / APPIUM_HOME rules: https://appium.io/docs/en/latest/guides/managing-exts/
• Android package IDs, stable scripting, and explicit license prompts: https://developer.android.com/tools/sdkmanager
• AVD creation: https://developer.android.com/tools/avdmanager
• Emulator and acceleration diagnosis: https://developer.android.com/studio/run/emulator-commandline and https://developer.android.com/studio/run/emulator-acceleration
• aapt2 is supplied by build-tools;<version>: https://developer.android.com/tools/aapt2
• Official command-line-tools 15859902 SHA-256 values: Windows 90ae805d...fb04a, macOS x64 c5a6378a...050f3, macOS arm64 835b62a2...2c4e, Linux x64 4e4c464f...df583: https://developer.android.com/studio#command-tools

Resolved design

Ownership and packaging

1. Add AndroidSetupProvider, planner/manifest, installer, status verifier, and lifecycle owner to shaft-infrastructure; register it in InfrastructureSetupService.builtIn(). This is the only download/install/process owner.
2. Refactor the already verified portable Node owner out of reporting into a shared internal service; Appium and reporting must use the same Node lock/destination and retain reporting plan compatibility.
3. Package an exact Appium package.json/package-lock.json containing appium@3.6.0, appium-uiautomator2-driver@8.2.2, and appium-inspector-plugin@2026.7.1. Run npm ci --ignore-scripts --no-audit --no-fund in staging, verify exact versions and Appium driver/plugin lists, then atomically publish. Do not globally install npm packages and do not call mutable Appium extension installation after publication.
4. Package a canonical Android release manifest binding command-line-tools URL/SHA-256/size by supported host, API 36, exact build-tools revision, platform-tools, emulator, platforms;android-36, and system-image package by architecture. sdkmanager is the upstream package installer, but the exact package set and command-line-tools bootstrap are plan/receipt bound. Stable channel only; --no_https, alternate repository OS, and implicit upgrades are forbidden.
5. Use ShaftCachePaths: archives/npm cache in downloads/cache, versioned Appium and Android SDK under tools, AVD data/leases/logs under durable data. Reject relative roots, linked/reparse ancestors and non-regular executables before reads, writes, or execution.

Selection and public adapters

6. Add an immutable AndroidSetupRequest rather than overloading SetupOptions. It binds AVD name, API, device profile, image tag, host-compatible ABI, RAM, cores, and server port. Defaults reproduce the release manifest. Normalize and validate all values before URI/path creation. Expose additive coordinator and SHAFT.Infrastructure overloads.
7. Add CLI options on status|doctor|plan|install|verify|start: --api-level, --device-profile, --image-tag, --abi, --avd-name, --ram-mb, --cores, and --port. install reconstructs the request from the reviewed plan; repeated selectors are optional but must match. Persisted plan actions remain authoritative.
8. Replace MCP install/start code with a compatibility adapter over the shared coordinator/lifecycle. Preserve existing tool responses and proposal confirmation semantics, but translate the confirmed proposal into the exact plan/approval/license flow. Delete the duplicate downloader/extractor/npm/sdkmanager owner after parity proof.
9. Engine local Appium bootstrap uses the shared runtime only when setup mode is managed and a compatible receipt exists. An explicit non-local execution address always wins and triggers zero local setup. Child-only environment receives APPIUM_HOME, ANDROID_HOME, ANDROID_SDK_ROOT, ANDROID_AVD_HOME, and PATH additions; process-global environment remains untouched.

Licenses, integrity, offline, and privilege

10. Bind the Android SDK license agreement identifier and all installed package license IDs into plan actions. CLI/Java/MCP must receive explicit matching acceptance before the first directory, process, or network action. Remove unconditional "y\\n".repeat(...) consent.
11. EXTERNAL is read-only and executes no probes that can initialize caches. MANAGED/HYBRID installation is exact-plan only. offline=true succeeds only from a complete verified installed state or complete verified artifact/npm/sdk cache; otherwise it fails before directories/processes/network.
12. Hypervisor, BIOS virtualization, Linux KVM group access, Windows Hypervisor Platform, macOS host entitlement, USB/device trust, and JDK availability are diagnostic prerequisites. Use emulator -accel-check; never enable features, elevate, edit shell profiles, or install host drivers.
13. Appium binds only 127.0.0.1, does not enable --relaxed-security by default, has bounded/redacted logs, and is READY only after /status returns a compatible build. The emulator is READY only after its exact serial is online, getprop sys.boot_completed is 1, package manager responds, and the selected AVD/image identity matches.

Atomicity and lifecycle

14. One normalized-root JVM lock plus OS file lock protects shared Node, Appium, Android SDK package install, AVD creation, receipts, and leases. Recheck under the lock. Separate JVM/process acceptance must converge.
15. Archive extraction rejects absolute/traversal/drive/UNC/ADS paths, links/devices, duplicate/case-fold collisions, reserved Windows names, excessive entries/sizes/ratios, and linked ancestors. Downloads are size bounded and SHA-256 verified before extraction.
16. Each install action is atomic. Known-good destinations are quarantined only after replacement is verified; rollback/recovery preserves the last usable copy. A partial failure exposes the failed action and partial receipt but writes no final profile receipt. Retrying reuses verified completed actions.
17. start creates or reuses only SHAFT-owned leases. Start emulator first, wait for full boot, then start Appium and verify /status. On any later failure, tear down resources started by that call and retain logs. close/stop validates PID/start-instant/command/root/endpoint before terminating descendants then parent with one bounded deadline. Unknown ports/PIDs are never adopted or killed. logs reads only the lease-bound bounded files.

Caller matrix

Caller	Mode/precedence	Paths/environment	Mutation authority	Result
CLI	explicit remote > requested mode > external default	absolute SHAFT roots; child env only	plan digest + exact license IDs	JSON/text plan, receipt, lease/logs, exit 0/2/3/4/5
SHAFT.Infrastructure	API > properties > defaults; explicit remote wins	caller cwd irrelevant	explicit SetupApproval; privileged work rejected	plan/status/install/start/managed close
Driver bootstrap	remote sessions never require local assets	receipt-bound roots	no hidden install; auto-start only with prior approval/receipt	actionable remediation or owned endpoint
MCP recorder	confirmed proposal translated to reviewed plan	workspace never used for tools	exact digest/license confirmation	existing compatible response plus shared leases
Concurrent process	receipt identity wins	same user roots	lock + recheck	one publication, one compatible lease policy

State/failure matrix

State	Preflight	Mutation/process	Readiness/result	Recovery
External/system compatible	read-only exact probes only	none	READY with external ownership	user-managed
Missing, unapproved/license missing	validate plan/approval first	none	MISSING/denied	exact remediation
Managed online cold	disk/host/license/network/manifest	stage Node/Appium/SDK/AVD, verify, publish, receipt	READY only on exact versions/packages	retry reuses verified actions
Managed offline complete	verify every required cache/install	no network	install/reuse succeeds	stable
Managed offline incomplete/corrupt	verify all before mutation/process	none	deterministic failure	populate exact cache online
Partial install	preserve completed verified actions	no final profile receipt	failed action + partial receipt	retry exact plan
Healthy owned runtime	lease + endpoint + boot/status	refcount/reuse only	READY	close decrements/stops by policy
Stale/mismatched runtime	validate full identity	do not adopt/kill mismatch	DEGRADED	explicit recovery guidance
Missing acceleration/JDK/trust	diagnostic probe	no elevation/global change	MISSING/DEGRADED	user fixes host prerequisite

Acceptance-to-proof map

Requirement	Positive proof	Negative/mutation proof
Exact deterministic plan	golden full actions, request, licenses, digests per host	mutate order/version/source/hash/package/build-tools/AVD/port => approval/replan rejection before callbacks
aapt2 included	installed build-tools exact revision; aapt2 version and APK badging flow pass	remove build-tools action/package => unit + gated Android acceptance fail
Explicit licenses	approved IDs allow install	missing/wrong ID => zero roots/process/network/provider mutation
Integrity/extraction	official command-tools checksum; canonical npm lock; safe staged publish	bad hash, truncation, oversize, traversal/link/collision, rollback failure preserve known-good/no receipt
Offline	complete verified cache/install succeeds	cold/mixed/corrupt cache => zero process/network/write, including roots
Concurrency	two services/JVMs converge	three-waiter lock-map and overlapping file lock mutants fail
Exact status	Node/Appium/driver/plugin/sdk packages/aapt2/AVD exact => READY	fake, wrong version, partial, symlink, stale receipt => DEGRADED
Emulator boot	selected AVD reaches device + boot property + package manager	adb device without boot completion times out and cleans owned process
Appium lifecycle	local-only server /status, lease, logs, refcount	occupied port, PID reuse, mismatched command/root never adopted/killed; failure cleans tree
Adapter parity	CLI/Java/MCP plans and reports match	delete provider registration/options/compatibility delegation => boundary tests fail
Remote/default safety	explicit remote and EXTERNAL run zero local setup	mutant callback/directory/process fails no-mutation snapshots
CI/package adoption	packaged CLI plan/install/verify and gated emulator session	wrong classifier/provider/resource omission fails workflow/boundary checks

Sequential RED → GREEN delivery

• RED provider registration and complete deterministic Android plan, including build-tools and accepted license IDs.
• GREEN immutable manifests, request model, plan/status provider, and exact resource/hash tests.
• RED/GREEN shared Node refactor with reporting plan/status/install compatibility.
• RED/GREEN locked Appium npm install, exact probes, offline/integrity/rollback/concurrency.
• RED/GREEN Android command-line-tools bootstrap, sdk package set, AVD creation, full readiness.
• RED/GREEN lease-safe emulator/Appium lifecycle and bounded logs.
• RED/GREEN CLI/Java selection and lifecycle parity.
• RED/GREEN MCP compatibility migration; delete duplicate installer/downloader.
• RED/GREEN engine managed bootstrap and explicit-remote zero-local behavior.
• Add packaged CLI and real gated Android emulator acceptance, module/workflow omission guards, publication/reactor/docs validators.
• Independent adversarial correctness/failure/blast-radius review; resolve every blocking mutation proof before merge.
• Companion docs PR: setup commands, licenses, platform prerequisites, cache/offline/lifecycle/recovery, MCP migration, mobile examples.

Definition of done

MOBILE_ANDROID is no longer catalog-only. CLI, Java, MCP, and managed engine execution agree on one exact plan/report/receipt and one owned Appium+emulator lifecycle. A clean supported host can reach a real UiAutomator2 session with aapt2 present after explicit reviewed approval/licenses; external/default/remote/offline-denial paths perform zero hidden mutation; no duplicate installer remains in MCP; every acceptance above has mutation-sensitive automated proof and all required CI/publication/documentation gates are green.

[MohabMohie]

Implementation checkpoint (2026-08-14, solo RED→GREEN→REFACTOR pass)

Official pinned sources reused from the issue research and verified against current release metadata:

• Node 24.19.0: https://nodejs.org/dist/v24.19.0/
• Appium 3.6.0 npm archive: https://registry.npmjs.org/appium/-/appium-3.6.0.tgz
• Inspector plugin 2026.7.1: https://registry.npmjs.org/appium-inspector-plugin/-/appium-inspector-plugin-2026.7.1.tgz
• UiAutomator2 8.2.2: https://registry.npmjs.org/appium-uiautomator2-driver/-/appium-uiautomator2-driver-8.2.2.tgz
• Android command-line tools 15859902 archives: https://dl.google.com/android/repository/repository2-3.xml
• Stable Android package revisions resolved from Google's repository metadata: platform-tools 37.0.1; emulator 37.1.11; platforms;android-36 revision 2; build-tools;36.0.0; google_apis system images revision 7.

Implemented evidence includes exact plan/lock/checksum binding, explicit android-sdk-license approval, verified transaction receipts, safe ZIP extraction, exact SDK/Appium readiness, recoverable AVD publication, lease-safe emulator→Appium lifecycle, PID/start-instant/command identity, shared shutdown deadline, descendant cleanup, stale/partial lease handling, port rejection, linked-ancestor denial, JVM and separate-process convergence, CLI/facade adapters, MCP shared owner/lifecycle migration, and receipt-bound engine auto-start with explicit remote precedence and no hidden install.

Current green checks:

• shaft-infrastructure full suite: 93 tests, 0 failures/errors/skips.
• SetupCommandTest: 14 tests, 0 failures/errors/skips.
• MCP mobile toolchain + Inspector recording: 15 tests, 0 failures/errors/skips.
• ManagedAndroidBootstrapTest: 4 tests, 0 failures/errors/skips.
• InfrastructurePropertiesTests plus bootstrap regression: 10 tests, 0 failures/errors/skips (before the final focused teardown rerun, which was 4/4).

Remaining before merge: packaged CLI/CI omission guards, gated real emulator+UiAutomator2+aapt2 acceptance, publication/reactor/quality/docs validators, final solo adversarial audit, then PR auto-merge/watch and companion docs.

[MohabMohie]

Final requirement-to-evidence audit — 2026-08-14

Audited current default branches, merged artifacts, the complete #4898 design, caller/state matrices, acceptance-to-proof map, sequential delivery list, and definition of done. Every requirement is proven; no blocking or deferred #4898 work remains.

Resolved design requirements

#	Requirement	Authoritative evidence
1	One infrastructure owner	Engine PR #4913 adds and registers AndroidSetupProvider, setup service, lifecycle, health, runtime controller, and manager in shaft-infrastructure; CLI, MCP, and engine call that owner.
2	Shared portable Node	ReportingSetupServiceTest plus Android setup tests prove reporting compatibility and shared verified Node publication/locking. Infrastructure suite: 101 green, with only 2 platform-specific Windows skips.
3	Exact Appium project	Packaged canonical package.json/lock pins Appium 3.6.0, Inspector 2026.7.1, and UiAutomator2 8.2.2. Planner/install/status tests verify lock integrity, npm ci, exact package and extension lists, and reject wrong versions before publication.
4	Exact Android manifest	Six ordered actions bind command-line tools 15859902, platform-tools 37.0.1, Emulator 37.1.11, API 36 revision 2, build-tools 36.0.0/aapt2, host image revision 7, and the SHAFT AVD. Planner golden/mutation tests and real gate 31773936548 prove the package set.
5	SHAFT paths and path safety	Versioned cache/data layouts, regular-file checks, and linked/reparse-ancestor denial are covered by setup, extraction, runtime-layout, and lifecycle tests.
6	Typed selection model	Immutable AndroidSetupRequest validates and plan-encodes AVD name, API, device, tag, ABI, RAM, cores, and port; additive coordinator and SHAFT.Infrastructure overloads are covered by planner and facade tests.
7	CLI parity	Packaged CLI covers plan/install/verify/start/logs/stop and plan-authoritative selector reconstruction. Follow-up PR #4917 adds explicit managed-mode readiness to doctor/status/verify with strict RED→GREEN evidence; SetupCommandTest 16/16.
8	MCP migration	MCP proposal/confirmation delegates installation and runtime ownership to the shared coordinator/lifecycle. MCP compatibility and inspector-recording suites: 25/25; boundary checks prevent duplicate ownership from returning.
9	Managed engine bootstrap	Receipt-bound auto-start is enabled only for local managed Android. Explicit remote wins, missing receipt never installs, and owned environment remains child-only. Facade/bootstrap suites: 11/11.
10	Explicit licenses	android-sdk-license is plan-bound. Missing/wrong acceptance reaches zero roots/process/network/provider mutation; package-scoped consent replaces blanket sdkmanager --licenses.
11	External/offline safety	EXTERNAL is read-only. Cold, partial, corrupt, wrong-version, or linked offline state fails before write/process/network; complete exact installed state is reusable and receipt-bound.
12	Host prerequisites only diagnose	JDK, architecture, acceleration/KVM/hypervisor conditions use exact probes including emulator -accel-check; implementation never elevates, installs drivers, edits profiles, or enables firmware features.
13	Exact readiness and local security	Emulator readiness requires exact serial, boot property, package manager, AVD/image identity. Appium binds 127.0.0.1, rejects wrong /status version, and never adds --relaxed-security.
14	Locking/convergence	Normalized-root JVM plus OS locking, recheck under lock, three-waiter tests, and separate-process convergence prove one publication/receipt.
15	Integrity/extraction	Official SHA-256/size ceilings and ZIP traversal, absolute/drive/UNC/ADS, link/device, case-fold collision, reserved-name, entry/size/ratio, and linked-ancestor rejection have focused mutation tests.
16	Atomic install/recovery	Action staging/publication, quarantine rollback, partial receipts, no premature final receipt, verified retry reuse, and recoverable AVD directory + .ini pointer publication are covered.
17	Owned lifecycle	Emulator-first/Appium-second startup, durable refcount lease, PID/start/command/root/endpoint identity, occupied-port denial, one shared shutdown deadline, descendant cleanup, stale/partial recovery, reverse shutdown, and bounded lease-bound logs have focused tests. The real gate proves a later packaged CLI process can stop the earlier owned runtime.

Acceptance and delivery evidence

• Exact deterministic plan, aapt2, license, integrity, offline, concurrency, exact-status, boot, lifecycle, adapter-parity, remote/default-safety, and CI/package mutation rows are all asserted by the tests named in the solo adversarial audit on PR feat: add managed Android Appium setup #4913.
• Exact implementation head da16e83b9dd3c65c686713880beb8112e7b29d00 passed the real Ubuntu managed Android workflow: packaged CLI plan/install/start, real emulator, UiAutomator2 session, APK badging through aapt2, and later-process stop — https://github.com/ShaftHQ/SHAFT_ENGINE/actions/runs/31773936548
• PR feat: add managed Android Appium setup #4913 merged as 99804feab708449ffc533b2cb34b0338e32e6f3b; PR fix: inspect managed setup readiness from CLI #4917 merged as 9775e8d8b4b042a1d890d29b6e4c51583b99809a. Both are ancestors of current origin/main.
• feat: add managed Android Appium setup #4913 verification: infrastructure 101 tests, CLI 15/15, MCP 25/25, engine 11/11, validator regressions 128/128, full 19-module package, CodeQL, Codacy, Linux/Windows CLI, installers, module suites, quality/reactor/docs/agent gates — green.
• Companion docs PR docs: add managed Android infrastructure setup shafthq.github.io#973 documents commands, exact pins, licenses, prerequisites, cache/offline behavior, lifecycle/recovery, MCP migration, and managed engine examples. It merged as 079888598263e535c5d5aca21e57fca8cfad8c1b, is an ancestor of origin/master, and passed Docs Build, Chromium install, accessibility, Playwright stability, CodeQL, and hosting checks.
• Required local companion validators are green: yarn test, yarn typecheck, yarn build, and exact yarn test:playwright (22/22). An unrelated inconsistent pixel-sampling run was reproduced as non-deterministic without source changes and is separately tracked at Test: stabilize composited contrast pixel sampling shafthq.github.io#974; no threshold/test weakening was used.
• Final solo adversarial review re-read current source, tests, PR checks, merge ancestry, real-run head SHA, and docs state. No Setup: provide approved Appium and Android emulator toolchain across CLI, Java, MCP, and engine #4898 acceptance gap remains.

Definition of done

MOBILE_ANDROID is provider-backed across CLI, Java, MCP, and engine; all callers share one exact plan/report/receipt and owned Appium+emulator lifecycle. A clean gated host reached a real UiAutomator2 session with aapt2 after explicit reviewed approval/licenses. External/default/remote/offline-denial paths have mutation-sensitive zero-hidden-mutation proof. MCP has no duplicate installer owner. Required code, CI, publication, and companion-documentation gates are green.

Audit verdict: complete. Closing #4898 is now warranted.

[MohabMohie]

Closed after the final requirement-to-evidence audit above; implementation, real Android acceptance, managed-readiness follow-up, and companion documentation are merged and green.