GestureApp 🖐️

A real-time hand gesture recognition desktop app that turns your webcam into a full PC controller — built with MediaPipe, TensorFlow Lite, and PyQt6.

Control your mouse, system volume, media playback, and page scrolling entirely with hand gestures. No extra hardware needed.

---

✨ Features

Feature	Description
🖱️ Mouse Control	Move cursor, left click, right click
🔊 Volume Control	Thumb up / down to raise or lower system volume
🎵 Media Control	Fist to play/pause, swipe for next/previous track
📜 Scroll Control	Pointer finger up/down to scroll any window
⚡ Enable / Disable All	One-click bulk toggle for all control modules
📋 Action History Log	Timestamped log of the last 20 triggered actions
⚙️ Live Settings Panel	Sensitivity sliders — changes apply instantly and persist across sessions
📷 Camera Selector	Switch webcam source without restarting
📊 Rich Status Bar	Live FPS + per-hand gesture name and confidence %
🔒 Gesture Lock	Hold left-hand OK pinch 1.5s to lock / unlock all controls

---

🚀 Quick Start

1. Clone the repository

git clone https://github.com/vkDemon1/GestureApp.git
cd GestureApp

2. Create a virtual environment

python -m venv .venv
.venv\Scripts\activate

3. Install dependencies

pip install -r requirements.txt

4. Run

python -m app.main

Closing: Click the ✕ button or press ESC. The engine stops and the camera is released immediately.

---

🖱️ Gesture → Action Mapping

Right Hand

Gesture	Action
✋ Open Hand	Move mouse cursor
✊ Fist	Left click
☝️ Pointer (index finger up)	Scroll up / down
👌 OK Sign	Right click

Left Hand

Gesture	Action
✊ Fist	Play / Pause
👍 Thumb Up	Volume Up
👎 Thumb Down	Volume Down
👌 OK pinch (hold 1.5s)	Lock / Unlock all controls
↩ Counter-clockwise swipe	Previous track
↪ Clockwise swipe	Next track

Note: Thumb Up/Down (volume) and Fist (play/pause) are mutually exclusive — volume gestures never accidentally trigger play/pause.

---

⚙️ Settings Panel

Open the Settings tab to tune behaviour:

Setting	Default	Description
Mouse Sensitivity	1.8×	Cursor speed multiplier
Scroll Speed	3 lines	Lines scrolled per tick
Click Cooldown	500 ms	Minimum time between clicks
Volume Step	3%	Volume change per thumb gesture

All settings are saved automatically to config.json and restored on next launch.

---

🏗️ Project Structure

GestureApp/
├── app/
│   ├── core/
│   │   ├── gesture_engine.py    # QThread pipeline — camera → MediaPipe → signals
│   │   ├── hand_tracker.py      # MediaPipe Hands wrapper
│   │   └── controller.py        # Gesture → PC action dispatcher
│   ├── ui/
│   │   ├── main_window.py       # Root PyQt6 window
│   │   ├── camera_widget.py     # Live camera feed panel
│   │   ├── control_panel.py     # Module toggles + action history log
│   │   ├── settings_panel.py    # Sensitivity sliders + camera selector
│   │   └── styles.qss           # Dark glassmorphism theme
│   └── main.py                  # Entry point
│
├── model/
│   ├── keypoint_classifier/     # Static hand sign TFLite model + labels
│   └── point_history_classifier/# Dynamic swipe TFLite model + labels
│
├── tests/                       # Unit tests
├── requirements.txt
└── requirements-dev.txt

---

🧪 Running Tests

pip install -r requirements-dev.txt
pytest tests/ -v

---

📋 Requirements

• Python 3.10+
• Windows 10 / 11 (volume & media control use Windows APIs)
• Webcam

mediapipe >= 0.10.0
opencv-python >= 4.8.0
tensorflow >= 2.13.0
PyQt6 >= 6.5.0
pyautogui >= 0.9.54
pycaw >= 20181228
comtypes >= 1.2.0
numpy >= 1.24.0

---

📄 License

This project is licensed under the Apache License 2.0.

---

🙏 Credits

• Original gesture recognition pipeline by Kazuhito Takahashi
• English translation by Nikita Kiselov
• Rebuilt as a full PyQt6 desktop app with PC control, live settings, action log, and gesture conflict prevention