Gesture Mouse — Hand-Tracking Cursor Control

Control your computer's mouse using hand gestures, tracked in real time from your webcam with MediaPipe Hands and OpenCV.

Features

• Right hand moves the cursor — pinch your index finger and thumb together to "engage" and start moving; release the pinch to stop.
• Left hand clicks — pinch index and thumb to click; a quick second pinch triggers a double-click.
• Movement is scaled based on how close your hand is to the camera, so small hand movements go further when your hand is far from the webcam.
• Supports multi-monitor setups on Windows by reading the full virtual screen size.

Requirements

• Python 3.8+
• A webcam
• Windows (the script uses ctypes.windll to read virtual screen dimensions — see Cross-platform notes if you're on macOS/Linux)

Installation

git clone https://github.com/Jayavisaag/Gesture-Mouse.git
cd Gesture-Mouse
pip install -r requirements.txt

Usage

python main.py

• A window will open showing your webcam feed with hand landmarks drawn on it.
• Pinch your right thumb and index finger to move the mouse.
• Pinch your left thumb and index finger to click (pinch twice quickly to double-click).
• Press q to quit.

How it works

1. A background thread continuously captures frames from the webcam.
2. Each frame is processed by MediaPipe Hands, which detects up to two hands and labels them as "Left" or "Right".
3. The right hand's index fingertip position is mapped to screen coordinates (scaled by hand size and smoothed) and used to move the cursor via pyautogui.
4. The left hand's pinch gesture between thumb and index finger triggers a click or double-click.

Cross-platform notes

The virtual screen size lookup (ctypes.windll.user32...) is Windows-only. To run this on macOS or Linux, replace that section with pyautogui.size(), which returns the primary screen resolution (single-monitor).