Turner

Overview

What Runs

• run.py: main local runner
• slidesync/server.py: FastAPI backend for SSE, camera selection, microphone selection, and dashboard helpers
• slidesync/gesture.py: live camera-based gesture worker
• transcript/transcript.py: microphone capture + AWS Transcribe streaming
• transcript/transcript_wrapper.py: bridges finalized transcripts into the semantic layer
• semantics/document_context.py: builds per-slide semantic context from PDF + script
• semantics/semantic_confidence.py: decides slide actions from finalized transcript chunks
• dashboard/: Next.js frontend

Requirements

• Python environment with pip install -r requirements.txt
• Node.js for the dashboard
• macOS camera permission enabled for Terminal if using gestures
• Keynote or PowerPoint open if you want live slide control
• AWS credentials for voice transcription
• Gemini Developer API key for semantic processing

Vercel Deployment: Dashboard UI

	URL
Dashboard	https://dashboard-theta-three-11.vercel.app
Inject mock state	https://instance-20260412-105657.tailf8cf11.ts.net/mock?slide=3&total=10

Open the dashboard link in a browser. To push mock slide state so the UI shows live data, hit the mock URL (change slide= to any number). The dashboard will update in real time.

---

Turner is a presentation control system that combines:

A touchless presentation plugin that allows you to control slides with gestures, voice, and semantic prediction. No physical clicker needed. Jump to any slide, hands-free, and enjoy your talk.

Speech Control: Gesture Control: Computer Vision Backend:

Environment Setup

Copy the template and fill in values:

cp example.env .env

Important variables in .env:

• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY
• AWS_REGION
• GEMINI_API_KEY
• GEMINI_MODEL
• GEMINI_RECURSIVE_MODEL
• BATCH_SIZE
• CAMERA_INDEX

The semantic layer uses the Gemini Developer API through google.genai.

Install

Python dependencies

pip install -r requirements.txt

Dashboard dependencies

cd dashboard
npm install
cd ..

MediaPipe model assets

Download the three required .task files:

mkdir -p gesture_controller/assets

curl -L -o gesture_controller/assets/gesture_recognizer.task \
  https://storage.googleapis.com/mediapipe-models/gesture_recognizer/gesture_recognizer/float16/latest/gesture_recognizer.task

curl -L -o gesture_controller/assets/hand_landmarker.task \
  https://storage.googleapis.com/mediapipe-models/hand_landmarker/hand_landmarker/float16/latest/hand_landmarker.task

curl -L -o gesture_controller/assets/pose_landmarker.task \
  https://storage.googleapis.com/mediapipe-models/pose_landmarker/pose_landmarker_lite/float16/1/pose_landmarker_lite.task

Run

Full local app

This starts:

• dashboard backend on port 8000
• transcription worker
• semantic routing
• optional gesture worker

python run.py --slide-path slides.pdf --script-path script.txt --mic 0

With camera preview:

python run.py --slide-path slides.pdf --script-path script.txt --mic 0 --gesture-preview

If you only want to skip gesture input:

python run.py --slide-path slides.pdf --script-path script.txt --mic 0 --no-gesture

--no-gesture disables the camera gesture worker, but the voice and semantic pipeline still start.

Dashboard frontend

cd dashboard
npm run dev

Open http://localhost:3000.

Inject mock state

To test the dashboard without a live presentation, push mock state via the VM:

https://instance-20260412-105657.tailf8cf11.ts.net/mock?slide=3&total=10

Change slide= to any number — the dashboard updates in real time.

Gesture debug viewer

python demos/gesture_debug.py

Transcription utilities

List microphones:

python transcript/transcript.py --list-mics

The dashboard also exposes:

• GET /microphones
• GET /microphone?index=N
• GET /cameras
• GET /camera?index=N

Notes About Startup

• The first run for a new slide deck and script may take noticeably longer because document_context.py builds semantic context and caches a wrapper pickle.
• Live transcript lines only appear for finalized AWS segments, not partials.
• Semantic processing can retry on Gemini 503 UNAVAILABLE responses. Transcript intake is decoupled from that retry path, but slide decisions may lag while retries are happening.

Current Behavior

• camera selection is hot-swappable through the dashboard backend
• microphone selection is hot-swappable through transcript.transcript.set_input_device(...) and the /microphone endpoint
• transcript processing is performed in the background so semantic retries do not block transcript intake

Tests

python -m pytest tests/test_swipe_detector_unit.py
python -m pytest tests/test_body_twist_detector_unit.py
python -m pytest tests/test_hand_geometry_unit.py
python -m pytest tests/test_controller.py
python -m pytest tests/test_gesture.py

Some tests require hardware or presentation software.

Documentation

• docs/gesture_controller.md
• docs/dashboard/integration.md
• docs/dashboard/microphone_integration.md
• docs/transcription/integration.md
• docs/semantics/usage.md
• docs/semantics/integration.md