Skip to content

haithamelmengad/popcorn

BranchesTags

Repository files navigation

README • ⚖️ License


🍿
popcorn

🇨🇳 中文 • 🇯🇵 日本語 • 🇰🇷 한국어 • 🇪🇸 Español • 🇩🇪 Deutsch • 🇫🇷 Français • 🇧🇷 Português • 🇷🇺 Русский • 🇸🇦 العربية • 🇮🇹 Italiano • 🇳🇱 Nederlands • 🇹🇷 Türkçe • 🇻🇳 Tiếng Việt • 🇮🇳 हिन्दी

An agent skill that gives any coding agent the ability to watch and understand video. Works with Claude Code, Codex, and any MCP-compatible agent.

License Version Node MCP Compatible


Quick Start • How It Works • MCP Tools • Transcription • Configuration • Troubleshooting • License

Popcorn enables AI agents to watch and understand long-form videos by extracting transcripts, detecting scene changes, and returning key frames. Everything runs locally—no external APIs, no fees, complete privacy.

Quick Start

# Install FFmpeg (required)
brew install ffmpeg                    # macOS
sudo apt install ffmpeg                # Ubuntu/Debian

# Install Popcorn
git clone https://github.com/anthropics/popcorn.git
cd popcorn && npm install && npm run build

# Optional: Install a transcription backend
pip install mlx-whisper                # Apple Silicon (fastest)
pip install openai-whisper             # Any platform

Add to Claude Desktop config (~/Library/Application Support/Claude/claude_desktop_config.json):

{
  "mcpServers": {
    "popcorn": {
      "command": "node",
      "args": ["/path/to/popcorn/dist/index.js"]
    }
  }
}

Key Features:

  • Scene Detection — Captures frames at visual transitions, not fixed intervals
  • Local Transcription — 4 backend options (mlx-whisper, faster-whisper, whisper-cpp, whisper)
  • Inline Images — Returns key frames directly in MCP responses
  • Smart Presets — Auto-configures for screencasts, presentations, movies, interviews
  • Zero Config — Just pass a video path and it works
  • Privacy First — Everything runs locally, no data leaves your machine

Documentation

Getting Started

Guides

Reference

How It Works

Core Components:

  1. FFprobe — Extracts video metadata (duration, resolution, codecs)
  2. FFmpeg Scene Detection — Finds visual transitions using select='gt(scene,N)' filter
  3. Parallel Frame Extraction — Captures JPEGs at scene change timestamps
  4. Multi-Backend Transcription — Whisper variants convert audio to timestamped text
  5. Analysis Bundle — Results saved to .popcorn/ directory
  6. MCP Response — Returns metadata + inline base64 images
Video File ──▶ FFprobe ──▶ FFmpeg ──▶ Whisper ──▶ Analysis Bundle
                 │           │          │              │
                 ▼           ▼          ▼              ▼
              metadata    frames    transcript    MCP Response

MCP Tools

Tool Description
popcorn_analyze Main analysis — extracts frames, transcribes audio, returns results
popcorn_suggest Probe video metadata and get recommended settings
popcorn_presets List available video types and objectives
popcorn_backends Detect your system and show transcription options
popcorn_read Read transcript slices with time filtering

Basic Usage

{
  "tool": "popcorn_analyze",
  "arguments": {
    "path": "/path/to/video.mp4"
  }
}

With Presets

{
  "tool": "popcorn_analyze",
  "arguments": {
    "path": "/path/to/video.mp4",
    "videoType": "screencast",
    "objective": "detailed"
  }
}

Video Types

Type Best For Scene Detection
screencast Tutorials, coding sessions, UI demos Low threshold
presentation Slides, lectures, keynotes Slide transitions
movie Films, TV shows Balanced
interview Podcasts, talking heads Transcription priority
surveillance Security footage, dashcam High threshold
sports Live events, fast action High frame rate

Objectives

Objective Use When
summary Quick overview needed
detailed Don't miss anything
find_moment Searching for specific content
transcribe Audio/speech is most important
visual_only Only care about visuals
quick_scan Fast preview needed

Transcription Backends

Popcorn auto-detects your system and recommends the best backend.

Backend Comparison

Backend Speed Best For Install
mlx-whisper Fastest Apple Silicon (M1/M2/M3/M4) pip install mlx-whisper
faster-whisper Fast NVIDIA GPUs pip install faster-whisper
whisper-cpp Moderate Cross-platform brew install whisper-cpp
whisper Slow Most compatible pip install openai-whisper

Processing Times (60-min video)

Backend Time
mlx-whisper 3-8 min
faster-whisper 5-10 min
whisper-cpp 10-20 min
whisper 30-60 min

Force a Backend

{
  "tool": "popcorn_analyze",
  "arguments": {
    "path": "/path/to/video.mp4",
    "backend": "mlx-whisper"
  }
}

Configuration

All Parameters

Parameter Type Description
path string Required. Absolute path to video file
videoType string Video type preset
objective string Analysis objective preset
transcribe boolean Enable/disable transcription
backend string Transcription backend
model string Whisper model (tiny, base, small, medium, large)
language string Language code (e.g., en, es, fr)
frameMode string scene or interval
sceneThreshold number Scene sensitivity (0-1)
maxFrames number Maximum frames to extract
inlineFrames number Frames to return as base64

Output Structure

.popcorn/<video>_<timestamp>/
├── analysis.json          # Full metadata
├── transcript.txt         # Plain text
├── transcript.json        # Timestamped segments
├── transcript.chunks.json # LLM-friendly chunks
└── assets/
    ├── audio.wav
    └── frames/
        ├── scene_000001.jpg
        └── ...

Troubleshooting

FFmpeg not found

brew install ffmpeg          # macOS
sudo apt install ffmpeg      # Ubuntu/Debian

No transcription backend

pip install mlx-whisper      # Apple Silicon
pip install openai-whisper   # Any platform

Too few frames detected

{ "sceneThreshold": 0.15, "minSceneInterval": 2 }

Too many frames detected

{ "sceneThreshold": 0.5, "minSceneInterval": 10 }

See Troubleshooting Guide for more solutions.

Development

npm install          # Install dependencies
npm run build        # Build
npm run dev          # Development mode
npm start            # Run server

Project Structure

popcorn/
├── src/
│   ├── index.ts        # MCP server
│   ├── analyze.ts      # Analysis pipeline
│   ├── ffmpeg.ts       # Video processing
│   ├── transcribe.ts   # Multi-backend transcription
│   ├── presets.ts      # Video type presets
│   └── commands.ts     # Shell execution
├── docs/               # Documentation
└── skills/             # Agent skills

Contributing

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/amazing)
  3. Commit changes (git commit -m 'Add amazing feature')
  4. Push to branch (git push origin feature/amazing)
  5. Open a Pull Request

License

MIT License — see LICENSE for details.

Acknowledgments

Made with 🍿 for AI agents everywhere

About

Video analysis MCP plugin for Claude Code — scene detection, local transcription, keyframe extraction. All local, zero cost, full privacy.

Resources

Readme
MIT license

Contributing

Contributing
Activity

Stars

5 stars

Watchers

1 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages