PhoneDriver API

A Python-based mobile automation agent that uses Cloud Vision-Language APIs (Kimi, GPT-4V, Claude, etc.) to understand and interact with Android devices through visual analysis and ADB commands.

No GPU required! This fork replaces the original local Qwen3-VL model with API-based vision models.

⚠️ Security & Privacy Notice

USB debugging exposes your device to ADB-based attacks. Only enable it when actively using PhoneDriver-API, disable it immediately after use, and NEVER connect to untrusted computers or public charging stations while it is enabled.

Screenshots of your device are sent to cloud AI providers. Do NOT use this tool while sensitive personal, financial, or confidential information is visible on screen. Review your provider's data retention policy before use.

🎯 Project Overview

PhoneDriver-API is a cloud-powered mobile automation agent that turns natural-language instructions into ADB actions on Android devices. By pairing screenshots from your phone with a Vision-Language Model (VLM), it plans, executes, and verifies UI interactions without requiring a local GPU. This makes it easy to automate apps, settings, and workflows using only an API key and a USB-debugged device.

flowchart LR
    User([User Task]) --> PhoneAgent
    subgraph PhoneAgent["PhoneAgent"]
        ConfigResolver[ConfigResolver]
        TaskOrchestrator[TaskOrchestrator]
        ActionExecutor[ActionExecutor]
    end
    PhoneAgent --> VLM[Vision-Language Provider]
    VLM --> ADB[ADB]
    ADB --> Android[Android Device]
    Android --> Screenshot[Screenshot]
    Screenshot --> PhoneAgent

The diagram above shows how a user task flows through PhoneAgent's internal components to the Vision-Language Provider, which reasons over a device screenshot and returns an ADB action. ADB executes the action on the Android device, and the resulting screenshot is fed back to the agent for the next step.

🌟 Features

☁️ Cloud Vision Models: Use Kimi K2.5, GPT-4V, Claude 3.5 Sonnet, or other VLM APIs
🤖 ADB Integration: Controls Android devices via ADB commands
📝 Natural language tasks: Describe what you want in plain English
🌐 Web UI: Built-in Gradio interface for easy control
📱 Real-time feedback: Live screenshots and execution logs
🔌 Multi-Provider Support: Kimi Code, OpenRouter, Moonshot, OpenAI, and more

📋 Requirements

Python 3.10+
Android device with USB debugging & Developer Mode enabled
ADB (Android Debug Bridge) installed
API key from supported providers (Kimi Code, OpenAI, OpenRouter, etc.)

🚀 Quick Start

1. Install ADB

Windows:

# Download from https://developer.android.com/studio/releases/platform-tools
# Add to PATH

Linux/Ubuntu:

sudo apt update
sudo apt install adb

macOS:

brew install android-platform-tools

2. Clone & Install

git clone https://github.com/Yesssssbabe/PhoneDriver-API.git
cd PhoneDriver-API

# Create virtual environment
python -m venv venv

# Windows
venv\Scripts\activate

# Linux/macOS
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

3. Configure API Provider

Copy the example config and edit:

cp .env.example .env

IMPORTANT: Ensure .env is in your .gitignore file and NEVER commit API keys to version control. Keep your .env file secure and private.

Edit .env with your preferred provider:

Option A: Kimi Code (Recommended for China)

PROVIDER=kimi_code
KIMI_CODE_API_KEY=sk-kimi-xxxxx

Option B: OpenRouter (Supports multiple models)

PROVIDER=openrouter
OPENROUTER_API_KEY=sk-or-v1-xxxxx
MODEL=moonshotai/kimi-k2.5

Option C: OpenAI

PROVIDER=openai
OPENAI_API_KEY=sk-xxxxx
MODEL=gpt-4o

Option D: Moonshot AI

PROVIDER=moonshot
MOONSHOT_API_KEY=sk-xxxxx
MODEL=kimi-k2.5

4. Connect Your Device

Enable USB debugging on your Android device:

Settings → About Phone → Tap "Build Number" 7 times
Settings → Developer Options → Enable "USB Debugging"
Connect via USB and allow debugging

Verify connection:

adb devices

5. Run

Command Line:

python phone_agent.py "Open Settings"

Web UI:

python ui.py
# Navigate to http://localhost:7860

Configuration

Copy the example config file:

cp config.example.json config.json

Edit config.json to match your device and preferences. Screen resolution is auto-detected by default.

📁 Project Structure

PhoneDriver-API/
├── phone_agent.py          # Main CLI agent
├── ui.py                   # Gradio web interface
├── config.example.json     # Example device configuration
├── config.json             # Device configuration (created by user)
├── .env                    # API keys (create from .env.example)
├── requirements.txt        # Python dependencies
├── README.md              # English documentation (this file)
├── README_CN.md           # Simplified Chinese documentation
├── README_TW.md           # Traditional Chinese documentation
├── README_JP.md           # Japanese documentation
├── README_KR.md           # Korean documentation
├── README_ES.md           # Spanish documentation
├── LICENSE                # MIT License
├── providers/             # API provider implementations
│   ├── __init__.py
│   ├── base.py            # Base provider interface
│   ├── kimi_code.py       # Kimi Code API
│   ├── openrouter.py      # OpenRouter API
│   ├── openai_provider.py # OpenAI API
│   └── moonshot.py        # Moonshot AI API
└── utils/                 # Utility functions
    ├── __init__.py
    ├── adb.py             # ADB wrapper
    └── screenshot.py      # Screenshot capture

⚙️ Configuration

Screen Resolution

The agent auto-detects your device resolution. To verify:

adb shell wm size

Supported Providers

Provider	Model	Vision	Notes
Kimi Code	kimi-for-coding, kimi-k2.5	✅	Best for coding tasks
OpenRouter	moonshotai/kimi-k2.5, anthropic/claude-3.5-sonnet, etc.	✅	Multiple models
OpenAI	gpt-4o, gpt-4o-mini	✅	Reliable, higher cost
Moonshot	kimi-k2.5, kimi-vl	✅	Official Moonshot API

Environment Variables

Variable	Description	Required
Variable	Description	Required
----------	-------------	----------
`PROVIDER`	API provider (`kimi_code`, `openrouter`, `openai`, `moonshot`)	Yes
`KIMI_CODE_API_KEY`	Kimi Code API key	If using Kimi Code
`OPENROUTER_API_KEY`	OpenRouter API key	If using OpenRouter
`OPENAI_API_KEY`	OpenAI API key	If using OpenAI
`MOONSHOT_API_KEY`	Moonshot API key	If using Moonshot
`MODEL`	Model name (provider-specific)	Optional
`TEMPERATURE`	Sampling temperature (0.0–1.0)	Optional
`MAX_TOKENS`	Max tokens per API response	Optional
`MAX_RETRIES`	API retry attempts	Optional
`MAX_CYCLES`	Max agent cycles per task	Optional
`STEP_DELAY`	Delay between actions (seconds)	Optional
`AUTO_DETECT_RESOLUTION`	Auto-detect screen size via ADB	Optional
`CHECK_COMPLETION`	Enable task completion checks	Optional

📝 Usage Examples

Command Line

# Open an app
python phone_agent.py "Open Chrome"

# Perform a search
python phone_agent.py "Search for weather in New York"

# Change settings
python phone_agent.py "Open Settings and enable WiFi"

# Take a photo
python phone_agent.py "Open camera and take a photo"

Python API

from phone_agent import PhoneAgent

config = {
    "provider": "kimi_code",
    "api_key": "your-api-key",
    "screen_width": 1080,
    "screen_height": 2340
}

agent = PhoneAgent(config)
result = agent.execute_task("Open Settings")
print(result)

🔧 Troubleshooting

Device not detected

# Restart ADB server
adb kill-server
adb start-server
adb devices

Wrong tap locations

Resolution is auto-detected by default in both CLI and UI. If taps are incorrect, verify with:

adb shell wm size

Then manually set screen_width and screen_height in config.json.

API errors

Verify your API key is valid
Check if you have sufficient quota/credits
Ensure PROVIDER matches the API key type

Unicode logging errors on Windows

If you see UnicodeEncodeError in logs, run PowerShell as UTF-8:

[Console]::OutputEncoding = [System.Text.Encoding]::UTF8
python phone_agent.py "your task"

👥 Contributors

Yesssssbabe - Creator & Maintainer (@Yesssssbabe)

💬 Contact

Have questions or suggestions? Feel free to reach out!

WeChat: Scan the QR code below (add note: phonedriverapi)
GitHub Issues: Create an issue

Note: Please add phonedriverapi when sending friend request.

🙏 Acknowledgments

Project Contributors

@Yesssssbabe - Creator & Maintainer of PhoneDriver-API

Original Project

@OminousIndustries - Original PhoneDriver author

API Providers

Kimi by Moonshot AI
OpenRouter for unified API access

📄 License

MIT License - see LICENSE file for details.

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

Fork the repository
Create your feature branch (git checkout -b feature/AmazingFeature)
Commit your changes (git commit -m 'Add some AmazingFeature')
Push to the branch (git push origin feature/AmazingFeature)
Open a Pull Request

💡 Future Improvements

Support for more providers (Anthropic, Google Gemini, etc.)
Batch task processing
Task recording and replay
iOS support (via WebDriverAgent)
Multi-device coordination

🐛 Recent Improvements

Added config.example.json and automatic screen resolution detection
Refactored provider code to reduce duplication and add API retry logic
Fixed text input escaping using shlex.quote with a clipboard fallback
Fixed PNG screenshot saving (optimize=True instead of unsupported quality)
Added task completion checks and bounded action history
Improved device detection in adb devices parsing

⭐ Star this repo if you find it useful!

Name		Name	Last commit message	Last commit date
Latest commit History 18 Commits
.github		.github
providers		providers
tests		tests
utils		utils
.env.example		.env.example
.gitignore		.gitignore
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
README.md		README.md
README_CN.md		README_CN.md
README_ES.md		README_ES.md
README_JP.md		README_JP.md
README_KR.md		README_KR.md
README_TW.md		README_TW.md
config.example.json		config.example.json
phone_agent.py		phone_agent.py
requirements-lock.txt		requirements-lock.txt
requirements.txt		requirements.txt
ui.py		ui.py
wechat_qr.jpg		wechat_qr.jpg

Folders and files

Latest commit

History

Repository files navigation

PhoneDriver API

🎯 Project Overview

🌟 Features

📋 Requirements

🚀 Quick Start

1. Install ADB

2. Clone & Install

3. Configure API Provider

4. Connect Your Device

5. Run

Configuration

📁 Project Structure

⚙️ Configuration

Screen Resolution

Supported Providers

Environment Variables

📝 Usage Examples

Command Line

Python API

🔧 Troubleshooting

Device not detected

Wrong tap locations

API errors

Unicode logging errors on Windows

👥 Contributors

💬 Contact

🙏 Acknowledgments

Project Contributors

Original Project

API Providers

📄 License

🤝 Contributing

💡 Future Improvements

🐛 Recent Improvements

About

Resources

License

Contributing

Uh oh!

Stars

Watchers

Forks

Releases 2

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages