Android companion app for Visual Mapper - enables local device control and sensor capture.
The Visual Mapper Android Companion app runs on your Android device and provides:
- Accessibility Service - Captures UI elements and screen state
- Local Action Execution - Performs taps, swipes, and gestures
- MQTT Communication - Sends sensor data directly to Home Assistant
- Flow Execution - Runs automation flows locally on the device
This app works alongside the Visual Mapper Server to provide a complete Android automation solution for Home Assistant.
| Feature | Description |
|---|---|
| UI Element Detection | Captures all visible UI elements via Accessibility Service |
| Sensor Publishing | Publishes element values as MQTT sensors |
| Action Execution | Taps, swipes, scrolls, and text input |
| Flow Runner | Executes multi-step automation flows |
| App Explorer | AI-assisted app navigation and mapping |
| Privacy Controls | Configurable app exclusions and consent management |
| Audit Logging | Tracks all accessibility service usage |
- Android 11 or higher
- Accessibility Service permission
- Network access to Visual Mapper server or MQTT broker
# Clone the repository
git clone https://github.com/botts7/visual-mapper-android.git
cd visual-mapper-android
# Build debug APK
./gradlew assembleDebug
# Install to connected device
adb install -r app/build/outputs/apk/debug/app-debug.apkDownload the latest APK from Releases.
- Install the app on your Android device
- Grant Accessibility Permission when prompted
- Configure Server URL - Enter your Visual Mapper server address
- Configure MQTT (optional) - For direct Home Assistant integration
visual-mapper-android/
├── app/
│ └── src/main/
│ ├── java/com/visualmapper/companion/
│ │ ├── accessibility/ # Accessibility service
│ │ ├── explorer/ # App exploration & AI
│ │ ├── mqtt/ # MQTT client
│ │ ├── sensor/ # Sensor capture
│ │ ├── security/ # Privacy & consent
│ │ ├── storage/ # Local database
│ │ └── ui/ # Activities & fragments
│ └── res/ # Resources
├── build.gradle.kts
└── README.md
| Repository | Description |
|---|---|
| visual-mapper | Main server application |
| visual-mapper-android | Android companion app (this repo) |
| visual-mapper-addon | Home Assistant add-on |
The app includes several privacy features:
- App Exclusions - Exclude sensitive apps from monitoring
- Consent Management - Per-app consent for data collection
- Audit Log - View all accessibility service activity
- PIN Lock - Optional app lock screen
MIT License - see LICENSE for details.
- Issues: GitHub Issues