VideoCraft Studio 🎬✨

AI-Powered Speech-Synced Video Editor & Dynamic Motion Generator

VideoCraft Studio is an automated desktop web application that synchronizes visual story beats directly to voiceover narration. By analyzing audio waveforms and speech pause boundaries with FFmpeg, the engine computes natural scene transition timestamps and applies cinematic Ken Burns camera motions to still imagery without requiring manual timeline slicing.

---

📋 Overview

The Problem

Traditional video editors (Premiere, DaVinci, CapCut) require manual scrubbing and cutting to align images or B-roll with voiceover pauses. For creators, educators, and storytellers producing narrated or faceless content, manual beat matching is tedious and time-consuming.

The Solution

VideoCraft Studio automates this workflow. It analyzes voiceover audio using signal silence-detection algorithms, identifies natural breath pauses and sentence boundaries, calculates optimal cut points, and pairs each audio segment with relevant imagery from a media pool while generating smooth Ken Burns pan-and-zoom motions.

---

✨ Key Features

• 🎙 Natural Speech-Pause Alignment: Employs FFmpeg signal analysis (silencedetect=noise=-26dB:d=0.15) to detect natural vocal breaks and calculate precise cut points.
• ⏱ Single-Track Interactive Timeline: Visual timeline with audio waveform representation, proportional scene blocks, and a real-time draggable playhead scrubber.
• 🎬 Real-Time Video Preview: 16:9 preview canvas with synchronized audio playback, live subtitles, and hardware-accelerated CSS Ken Burns motion previews (Zoom In, Zoom Out, Pan Left, Pan Right, Static Cut).
• 📋 Scalable Storyboard (200+ Scenes): Manage large projects with thumbnail previews, exact timecodes, motion indicators, keyword-based AI match scores (91% ✓), and drag-and-drop scene reordering.
• ⚙️ Scene Details Inspector: Fine-tune individual beats, edit narration scripts, adjust durations with automatic drift-free timestamp recalculation, and modify camera motion presets.
• ✨ Keyword-Based AI Image Matching: Analyzes narration script keywords to recommend and swap the most relevant image from your project media pool.
• 🚀 Multi-Threaded Video Rendering: Fast 10-scene preview rendering, full master video export (1080p, 720p, 4K, 30/60 FPS) with parallel CPU worker allocation, real-time terminal progress monitoring, and direct MP4 downloads.
• ⌨️ Keyboard Shortcuts & History: Undo/Redo history stack (Ctrl+Z / Ctrl+Shift+Z), Play/Pause (Space), and scene navigation (← / →).

---

📸 Screenshots

+-----------------------------------------------------------------------------------+
|  [Header: VideoCraft Studio]                     [Quick Demo]  [🚀 Export Video] |
+-----------------------------------------------------------------------------------+
|  [ Preview Player 16:9 ]          |  [ Scene Inspector & Storyboard ]             |
|                                   |  - Scene #04 (00:08.40 - 00:11.20)            |
|   [▶] Live Subtitles & Motion     |  - Narration: "The engine computes cadence..."|
|                                   |  - Motion: [ Zoom In ▾ ]  AI Match: [ 94% ]   |
+-----------------------------------┴-----------------------------------------------+
|  [ Audio Waveform & Interactive Single-Track Timeline Scrubber ]                 |
|  | Scene 1 (3.2s) | Scene 2 (2.8s) | Scene 3 (4.1s) | Scene 4 (2.8s) |           |
+-----------------------------------------------------------------------------------+

(To add actual screenshot images, create a docs/screenshots/ folder and link them here using standard Markdown syntax).

---

🛠️ Architecture & Pipeline

┌─────────────────┐       ┌─────────────────┐
│ Voiceover Audio │  ───▶ │ Image Media Pool│
└────────┬────────┘       └────────┬────────┘
         │                         │
         ▼                         │
┌───────────────────────────────┐  │
│ FFmpeg silencedetect Analysis │  │
│  - Noise threshold: -26 dB    │  │
│  - Min duration: 0.15s        │  │
└────────┬──────────────────────┘  │
         │                         │
         ▼                         ▼
┌───────────────────────────────────────────┐
│     Timestamp & Scene Beat Calculator     │
│  - Calculates scene durations             │
│  - Maps images to beats via keyword score │
└────────────────────┬──────────────────────┘
                     │
                     ▼
┌───────────────────────────────────────────┐
│         Interactive Studio GUI            │
│  - Waveform scrubber & preview canvas     │
│  - Storyboard reorder & motion inspector  │
└────────────────────┬──────────────────────┘
                     │
                     ▼
┌───────────────────────────────────────────┐
│       Multi-Threaded FFmpeg Engine        │
│  - Applies Ken Burns filter_complex       │
│  - Renders video segments in parallel     │
│  - Concat demuxer generates master MP4    │
└───────────────────────────────────────────┘

---

💻 Tech Stack

• Backend: Python 3.8+, FastAPI, Uvicorn
• Media Engine: FFmpeg (silencedetect, filter_complex, Ken Burns transformation)
• Frontend: Vanilla HTML5, Modern CSS3 (Dark Theme, CSS Grid/Flexbox), ES6+ JavaScript
• Testing: pytest (API routes, duration recalculation, engine tests)
• Distribution: Windows Batch, Shell script, Inno Setup installer, PyInstaller launcher

---

⚡ Quick Start

1. Prerequisites

• Python 3.8+
• FFmpeg installed and added to system PATH:
  • Windows: winget install Gyan.FFmpeg or download from gyan.dev
  • macOS: brew install ffmpeg
  • Linux: sudo apt update && sudo apt install ffmpeg

2. Installation

Clone the repository:

git clone https://github.com/imanandvardhan/VideoCraft-Studio.git
cd VideoCraft-Studio

Install dependencies:

pip install -r requirements.txt

3. Launching

On Windows:

run_app.bat
# or: python server.py

On macOS / Linux:

chmod +x run_app.sh
./run_app.sh
# or: python3 server.py

Navigate to http://localhost:8000 in your browser.

---

📂 Project Structure

VideoCraft-Studio/
├── server.py              # FastAPI server & REST API endpoints
├── video_engine.py        # FFmpeg speech analysis & multi-threaded rendering
├── desktop_launcher.py    # Desktop webview launcher wrapper
├── installer_script.iss   # Inno Setup Windows installer script
├── static/                # Front-end UI
│   ├── index.html         # Workspace UI, timeline, and modals
│   ├── app.css            # Dark theme & responsive design system
│   └── app.js             # Timeline scrubbing, audio sync & Undo/Redo stack
├── tests/                 # Test suite
│   ├── test_api.py        # API endpoint validation
│   ├── test_engine_run.py # Video engine execution tests
│   └── ...
├── outputs/               # Rendered master MP4 files
├── temp_render/           # Intermediate parallel render segments
├── uploads/               # Project session media files
├── requirements.txt       # Python dependencies
├── run_app.bat            # Windows startup script
├── run_app.sh             # Linux/macOS startup script
└── README.md

---

🛠️ API Reference

Method	Endpoint	Description
POST	/api/analyze-paths	Analyze voiceover audio and images using server filesystem paths
POST	/api/upload	Upload audio, images, and script via multipart form
POST	/api/quick-load	Load pre-packaged sample demo session
GET	/api/state	Fetch current session state, total duration, and scene counts
GET	/api/timeline	Fetch synchronized scene blocks and cut timestamps
POST	/api/timeline/update	Update individual scene motion or duration
POST	/api/timeline/save	Persist reordered storyboard and scene parameters
GET	/api/audio	Stream project audio track for preview synchronization
POST	/api/render	Trigger multi-threaded FFmpeg video export
GET	/api/render/status	Poll real-time rendering percentage and terminal logs
GET	/api/download/{filename}	Download generated MP4 video

---

🎯 Roadmap

• Add hardware-accelerated GPU encoding (NVENC / QuickSync / VAAPI).
• Direct integration with local Whisper models for automated speech-to-text transcript generation.
• Multi-track timeline supporting background music ducking and sound effects.
• Native export presets for vertical video platforms (9:16 for Reels / Shorts / TikTok).

---

⚠️ Known Limitations

• Audio files without distinct vocal pauses (e.g., continuous shouting or high background music) may result in fewer detected pause boundaries, requiring manual scene cuts in the editor.
• High-resolution (4K 60FPS) video exports require adequate CPU cores and RAM due to multi-threaded FFmpeg rendering.

---

📜 License

This project is licensed under the MIT License.