Montai

AI-powered video editing tool that extracts storylines from unscripted footage and generates edited vlogs.

See the MANUAL.md for full documentation.

Install

npm install -g montai-cli

Install from source

git clone https://github.com/jysperm/Montai.git
cd Montai
npm ci && npm link

Prerequisites:

• Node.js >= 22 (v20 has a readline bug with CJK input)
• ffmpeg and ffprobe on PATH (brew install ffmpeg)
• Gemini for video analysis and editing (required) — set GEMINI_API_KEY from Google AI Studio
• Lyria 2 for music generation and Gemini TTS for voiceover generation (both are optional) — set GOOGLE_CLOUD_PROJECT and GOOGLE_APPLICATION_CREDENTIALS from Google Cloud Console

Interactive Story Editing

The montai story command opens an interactive session where you chat with AI to craft your storyline and timeline. You can use any language to iteratively refine the edit — adjust pacing, reorder scenes, add or remove clips, and tweak transitions — all through natural conversation.

Editing Capabilities

• Clip trimming — select segments from any analyzed video with precise start/end times
• Playback rate — speed up or slow down individual clips
• Volume control — adjust volume per clip
• Transitions — fade, slide, and wipe transitions between clips
• Crop — static crop to reframe shots
• Ken Burns — animated pan & zoom from one crop to another
• Rotation — rotate clips to fix footage shot in the wrong orientation
• Text overlays — title, subtitle, and caption styles at 6 positions, with fade, slide, and pop entrance animations
• Background music — add library music with volume and fade controls, auto-looping with crossfade
• AI music generation — generate instrumental background music
• AI voiceover generation — write a script and have the AI narrate it (Google Gemini-TTS, or macOS's built-in voices), then place the narration under the footage
• Mixed orientations — can also mix landscape and portrait shots in the same timeline, with automatic rotation adjustments
• Voiceover-driven editing — match video clips to narration recordings, selecting visuals that fit each spoken segment

Self-Feedback

• Inspect a single frame — view any moment of the current edit as an image to verify how an effect actually looks
• Watch the edited video — view a range (or the whole timeline) end-to-end to check pacing, transitions, and overall flow

Quick Start

1. Create a project directory and put your source videos in a footage/ subdirectory inside it.

Create montai.yaml:

assets:
  videos: ./footage
language: en
output:
  resolution: 1080p
  fps: 50
models:
  analysis: gemini-3.5-flash
  editing: gemini-3.5-flash
  musicGeneration: lyria-002 # Optional but recommended

2. Write your credentials to ~/.config/montai/env:

GEMINI_API_KEY=...
GOOGLE_CLOUD_PROJECT=...
GOOGLE_APPLICATION_CREDENTIALS=/path/to/google-cloud-vertex-key.json

3. Analyze the source media:

montai analyze

4. Start a new interactive story editing session:

montai story --new

Inside the story session, use /preview to start Remotion Studio to preview the edited video or /export to export FCPXML for Final Cut Pro:

> /preview
Auto preview: on
Remotion Studio: http://localhost:3000

> /export
Auto export: on
FCPXML exported

Commands

Command	Description	See Also
montai analyze	Transcode, upload, and analyze videos, music, and voiceovers	Analyze Media
montai analyze --re-run [file]	Re-analyze media file (omit for all)	Analyze Media
montai analyze --list	List media analysis status	Analyze Media
montai analyze --show <file>	Show one stored media analysis	Analyze Media
montai project	Show project overview and stats	Project Overview
montai story	Open an interactive story editing session (create new or open existing one)	Interactive Story Editing
montai story --list	List stories	Stories
montai story --sessions	List saved editing sessions	Sessions
montai story --resume [id]	Resume a prior editing session (omit to select interactively)	Sessions
montai preview [name]	Open Remotion Studio for preview (omit for all)	Live Preview
montai export [name]	Export FCPXML from a timeline (omit for all)	Export FCPXML
montai render [name]	Render MP4 through Remotion (omit for all)	Render with Remotion
montai [preview | export | render] --from-archived	Use archived videos as source	Work From Archived Clips
montai archive	Archive original video clips referenced by current timelines	Archive Source Clips
montai archive --encode [spec]	Encode archived clips instead of passthrough copy	Encoded Archive
montai clean	Remove regenerable cache files	Clean Cache

Debug logging for LLM calls via the DEBUG env var:

DEBUG=montai:*,-montai:*:verbose montai story    # print each LLM call
DEBUG=montai:* montai story                      # including full message contents

Export .fcpxml

montai export generates .fcpxml 1.11 files in the fcpxml/ directory, which can be imported into professional video editors. .fcpxml preserves clips, transitions, and text overlays, and is recommended over render for HDR projects.

Final Cut Pro (recommended)

First import your video files into Final Cut Pro, then use File → Import → XML to import the .fcpxml file. FCP will automatically link the media.

DaVinci Resolve

First import your video files into the Media Pool, then use File → Import → Timeline → Import AAF, EDL, XML, FCPXML to import the .fcpxml file. Resolve will automatically match the media to the timeline clips.

Archiving

Montai can be used to curate interesting segments from a large amount of raw footage. After creating stories, you may want to delete the original files to free up space. montai archive extracts the video segments referenced by all current story timelines into the archived/ directory (video files only). Timeline checkpoints created with /mark are not included unless restored into the current story timeline first.

By default, montai archive uses passthrough mode to preserve original quality without re-encoding. Use --encode to re-encode using project output settings, or --encode 720p,crf=20,fps=30,8bit to customize the encoding spec.

After archiving, use --from-archived on render, preview, or export to work from the archived clips instead of the original files. The time offsets are automatically remapped based on the archived filenames.

Output Compatibility

	Final Cut Pro	DaVinci Resolve	Remotion
Color depth	Passthrough 8/10-bit	Passthrough 8/10-bit	8-bit only
Color space	SDR and HDR	SDR and HDR	SDR only
Transitions	Fade, slide, wipe	Fade only	Fade, slide, wipe
Text overlays & animations	Full with known issues	Centered only, no animations	Full
Ken Burns	Yes, not working with rotated	Fallback to static crop	Yes
Mixed aspect ratios	Yes	Known issues	Yes
Audio fades	Yes	No	Yes
Render speed	Fast	Fast	Very slow