Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 14 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

# MasterSelects

<h3>Browser-based Video Compositor & 3D Engine</h3>
<h3>Browser-based Video Compositor, Vector Animation & 3D Engine</h3>

<br>

Expand All @@ -15,11 +15,11 @@
<p>
GPU-first editing with <b>30 effects</b>, <b>37 blend modes</b>, <b>79 AI tools</b>, <b>real 3D via Three.js</b>, and only <b>14 dependencies</b>.<br>
Built from scratch in <b>2,400+ lines of WGSL</b> and <b>138k lines of TypeScript</b>.<br>
Import <b>OBJ, glTF, GLB, FBX, PLY, SPLAT</b> assets directly into the timeline.
Import <b>.lottie, Lottie JSON, OBJ, glTF, GLB, FBX, PLY, SPLAT</b> assets directly into the timeline.
</p>

<p>
<a href="https://github.com/Sportinger/MasterSelects/releases"><img src="https://img.shields.io/badge/version-1.5.1-blue.svg" alt="Version"></a>
<a href="https://github.com/Sportinger/MasterSelects/releases"><img src="https://img.shields.io/badge/version-1.5.4-blue.svg" alt="Version"></a>
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-green.svg" alt="License"></a>
<a href="https://app.fossa.com/projects/custom%2b61097%2fmasterselects"><img src="https://app.fossa.com/api/projects/custom%2b61097%2fmasterselects.svg?type=shield" alt="FOSSA Status"></a>
</p>
Expand Down Expand Up @@ -56,14 +56,17 @@ Decoding depends on what the **browser** supports — the container is just the
<tr><td><b>Video codecs</b></td><td>H.264 (AVC), H.265 (HEVC)¹, VP8, VP9, AV1</td></tr>
<tr><td><b>Audio files</b></td><td>WAV, MP3, OGG, FLAC, AAC, M4A, WMA, AIFF, OPUS</td></tr>
<tr><td><b>Image</b></td><td>PNG, JPG/JPEG, WebP, GIF, BMP, SVG</td></tr>
<tr><td><b>Vector animation</b></td><td><code>.lottie</code> packages and Lottie JSON files (content-sniffed)</td></tr>
<tr><td><b>3D Models</b></td><td>OBJ, glTF, GLB, FBX — rendered via Three.js with lighting</td></tr>
<tr><td><b>Gaussian Splats</b></td><td>PLY, SPLAT</td></tr>
<tr><td><b>Download</b></td><td>YouTube, TikTok, Instagram, Twitter/X, Vimeo + <a href="https://github.com/yt-dlp/yt-dlp/blob/master/supportedsites.md">all yt-dlp sites</a> via Native Helper</td></tr>
<tr><th colspan="2">Export (Encode)</th></tr>
<tr><td><b>Containers</b></td><td>MP4, WebM</td></tr>
<tr><td><b>Video containers</b></td><td>MP4, WebM via WebCodecs / HTMLVideo; MOV, MKV, AVI, MXF via FFmpeg WASM</td></tr>
<tr><td><b>FFmpeg codecs</b></td><td>ProRes, DNxHR, FFV1, UTVideo, MJPEG</td></tr>
<tr><td><b>Video codecs</b></td><td>H.264, H.265¹, VP9, AV1 — GPU-accelerated via WebCodecs</td></tr>
<tr><td><b>Audio codecs</b></td><td>AAC (MP4), Opus (WebM)</td></tr>
<tr><td><b>Interchange</b></td><td>FCPXML (Final Cut Pro / DaVinci Resolve), PNG sequence</td></tr>
<tr><td><b>Image export</b></td><td>PNG, JPG/JPEG, WebP, BMP (current playhead frame)</td></tr>
<tr><td><b>Audio-only export</b></td><td>AAC or OGG/Opus depending on browser codec support</td></tr>
<tr><td><b>Interchange</b></td><td>FCPXML (Final Cut Pro / DaVinci Resolve)</td></tr>
</table>

¹ H.265 decode/encode depends on OS & hardware — full support on Windows, partial on macOS/Linux.
Expand Down Expand Up @@ -139,10 +142,11 @@ This requires the Native Helper to be running, a MasterSelects editor tab to be
| [**AI Integration**](docs/Features/AI-Integration.md) | Built-in OpenAI chat, 79 exported tool-callable edit actions, and local/native bridges for external agents |
| [**FlashBoard**](docs/Features/FlashBoard.md) | Node-based AI canvas for text-to-video, image-to-video, and image generation |
| [**Multicam AI**](docs/Features/Multicam-AI.md) | Sync cameras, transcribe footage, and generate Claude-powered multicam EDLs *(experimental)* |
| [**Export Pipeline**](docs/Features/Export.md) | WebCodecs Fast/Precise, FFmpeg WASM *(experimental / WIP)*, FCPXML, and PNG sequence export |
| [**Export Pipeline**](docs/Features/Export.md) | WebCodecs Fast/Precise, FFmpeg intermediates, image/audio-only export, FCPXML, and project-persistent presets |
| [**Live EQ & Audio**](docs/Features/Audio.md) | 10-band parametric EQ with real-time Web Audio preview |
| [**Download Panel**](docs/Features/Download-Panel.md) | YouTube, TikTok, Instagram, Twitter/X, Vimeo, and other yt-dlp-supported sites via Native Helper |
| [**Text & Solids**](docs/Features/Text-Clips.md) | 50 Google Fonts, stroke, shadow, solid color clips |
| [**Vector Animation**](docs/Features/Vector-Animation.md) | `.lottie` and Lottie JSON clips with loop controls, fit, and deterministic preview/export |
| [**Text & Solids**](docs/Features/Text-Clips.md) | 50 Google Fonts, stroke, shadow, and solid color clips |
| [**Proxy System**](docs/Features/Proxy-System.md) | GPU-accelerated proxies with resume and cache indicator |
| [**Output Manager**](docs/Features/Preview.md) | Multi-window outputs, source routing, corner pin warping, slice masks |
| [**Slot Grid**](docs/Features/Slot-Grid.md) | Resolume-style 12x4 grid with multi-layer live playback and slot-clip trims |
Expand Down Expand Up @@ -230,7 +234,7 @@ If something breaks, refresh. If it's still broken, [open an issue](https://gith

- **Frontend:** React 19, TypeScript, Zustand, Vite 7.2
- **Rendering:** WebGPU + 2,500+ lines of WGSL shaders
- **Video:** WebCodecs, mp4box, mp4-muxer, webm-muxer, HTMLVideo fallback, experimental FFmpeg WASM export path
- **Video:** WebCodecs, MediaBunny, mp4box, HTMLVideo fallback, and experimental FFmpeg WASM export path
- **Audio:** Web Audio API with 10-band live EQ, element-synced playback, drift correction, and waveform extraction
- **AI:** Built-in OpenAI editor chat with 79 exported tools, Native Helper HTTP bridge for Claude Code / external agents, Claude/Anthropic for experimental multicam EDLs, SAM2 via ONNX Runtime, MatAnyone2 via Native Helper, local Whisper via Hugging Face Transformers, and Kie.ai / hosted cloud / PiAPI-backed generation flows
- **Native:** Rust helper for Firefox storage backend, native decode/encode, and yt-dlp downloads
Expand Down Expand Up @@ -318,6 +322,7 @@ src/
│ ├── nativeHelper/ # Native decoder + WebSocket client
│ ├── layerBuilder/ # Layer building + video sync
│ ├── mediaRuntime/ # Media runtime bindings + playback
│ ├── vectorAnimation/ # Lottie metadata sniffing + runtime canvas playback
│ └── export/ # FCPXML export
├── shaders/ # WGSL (composite, effects, output, optical flow, slice)
├── hooks/ # React hooks (useEngine, useGlobalHistory, useMIDI, useTheme)
Expand Down
55 changes: 50 additions & 5 deletions docs/Features/Export.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

[Back to Index](./README.md)

Video export, audio export, transparent stacked-alpha export, FFmpeg WASM export, and FCPXML interchange.
Video export, image export, audio-only export, transparent stacked-alpha export, FFmpeg WASM intermediates, named export presets, and FCPXML interchange.

---

Expand All @@ -14,14 +14,37 @@ The export panel currently exposes three live encoder paths:
- `htmlvideo` for the precise HTML-video-seeking pipeline
- `ffmpeg` for the CPU FFmpeg WASM pipeline

There is also a separate FCPXML export action for NLE interchange.
FCPXML is exposed as a selectable export container for NLE interchange.

### Current Panel Layout

- A sticky summary bar at the top shows compact badges plus the primary export button.
- Clicking a summary badge smooth-scrolls to the related control group and briefly highlights it.
- A compact command row above `Basic` contains a project-persistent preset list plus `Load`, `Update`, and `Save`.
- The workflow picker (`WebCodecs`, `HTMLVideo`, `FFmpeg`) is its own section above `Basic`.
- `Basic` contains output naming and container selection. The container row is grouped by `Video`, `Image`, `Audio`, and `XML`, and switches output mode by selecting a deliverable directly.
- The `Video` group contains codec selection, resolution, frame rate, bitrate/rate controls, stacked-alpha, and range toggles.
- In `Image` mode the same middle group becomes an `Image` panel with format-aware resolution and quality controls, and it exports only the current playhead frame.
- The `Audio` group contains sample rate, bitrate, normalization, and audio-only range controls.
- Lower in the panel, legacy `Advanced Video`, `Advanced Audio`, and `Range & Summary` sections still exist for raw-value access.
- Export settings and preset selection now live in a shared store, so changes inside the Export tab participate in global undo/redo and are restored with the project.

### Export Presets

- Presets are stored per project, not in browser-only local storage.
- `Save` prompts for a name and creates a new preset or overwrites an existing preset with the same name.
- `Update` overwrites the currently selected preset without asking for a new name.
- `Load` restores the selected preset back into the live export settings.
- Presets and the current export settings are written into the project UI state and restored with the project.

---

## WebCodecs And HTMLVideo Export

`FrameExporter` is used for both the WebCodecs and HTMLVideo export buttons.

Canvas-backed sources such as text, solids, and Lottie are re-rendered for every export frame before capture, so the exported frame matches the current timeline time instead of reusing a stale first-frame texture.

### Fast Mode

- Uses WebCodecs sequential decoding for a single clip.
Expand Down Expand Up @@ -71,6 +94,9 @@ Supported codecs are checked at runtime:
- WebM is limited to VP9 or AV1.
- MP4 accepts the full codec list, but browser support is still checked with `VideoEncoder.isConfigSupported()`.
- Unsupported combinations are not silently promised by the docs; they must pass the runtime checks or be remapped by the encoder logic.
- The selected bitrate is passed into `VideoEncoder`, but in the WebCodecs path it is still a target, not a guaranteed final file bitrate.
- `rateControl = cbr` maps to `VideoEncoderConfig.bitrateMode = "constant"` and falls back to variable bitrate if constant mode is rejected during encoder configuration.
- Browser encoders can still undershoot the requested bitrate on simple material, so the panel treats file size as a target estimate rather than an exact promise.

---

Expand Down Expand Up @@ -113,6 +139,27 @@ Audio export is handled separately from the video encoder.

---

## Image Export

Image export renders a single composited frame at the current playhead position.

### Supported Formats

- PNG
- JPG
- WebP
- BMP

### Current Behavior

- The export does not use the In/Out range. It always renders only the current playhead frame.
- Custom resolution still applies before the image is written.
- PNG and BMP are exported losslessly.
- JPG and WebP expose a quality control in the panel.
- Audio is ignored while image export is active.

---

## FFmpeg Export

The FFmpeg path is a separate CPU-based export pipeline.
Expand All @@ -135,9 +182,7 @@ The FFmpeg path is a separate CPU-based export pipeline.
### Supported Containers

- MOV
- MP4
- MKV
- WebM
- AVI
- MXF

Expand All @@ -152,7 +197,7 @@ The FFmpeg path is a separate CPU-based export pipeline.

## FCPXML Export

FCPXML export is separate from video rendering.
FCPXML export is available through the container chooser as `.fcpxml`.

### What It Exports

Expand Down
13 changes: 9 additions & 4 deletions docs/Features/Media-Panel.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,15 @@ Import, organize, and manage media assets with folder structure, proxy generatio
| **Video** | MP4, WebM, MOV, AVI, MKV, WMV, M4V, FLV |
| **Audio** | WAV, MP3, OGG, FLAC, AAC, M4A, WMA, AIFF, OPUS |
| **Image** | PNG, JPG/JPEG, GIF, WebP, BMP, SVG |
| **Vector Animation** | `.lottie`, Lottie JSON (`.json`, content-sniffed) |

The panel also accepts a few specialized asset types that flow into the timeline as 3D clips:

- `model` files: OBJ, glTF/GLB, FBX
- `gaussian-splat` files: PLY, SPLAT

Lottie imports are treated as first-class media items. `.json` files are only accepted when their contents actually match Lottie structure, so arbitrary JSON data is not misclassified as animation.

### Import Methods

#### Import Button
Expand Down Expand Up @@ -377,7 +380,7 @@ Clicking transcript or analysis badges selects the corresponding clip in the tim
interface MediaFile {
id: string;
name: string;
type: 'video' | 'audio' | 'image';
type: 'video' | 'audio' | 'image' | 'lottie' | 'rive';
file?: File; // Undefined when needs reload
url: string;
parentId: string | null;
Expand Down Expand Up @@ -411,6 +414,7 @@ interface MediaFile {
// Analysis
analysisStatus?: AnalysisStatus;
analysisCoverage?: number;
vectorAnimation?: VectorAnimationMetadata;
// File System Access API
hasFileHandle?: boolean;
filePath?: string;
Expand All @@ -431,7 +435,7 @@ interface MediaFile {
### Drag Types
| Item Type | Drag Payload Kind | Data Transfer Key |
|-----------|-------------------|-------------------|
| Media file (video/image) | `media-file` | `application/x-media-file-id` |
| Media file (video/image/lottie) | `media-file` | `application/x-media-file-id` |
| Media file (audio) | `media-file` (marked as audio) | `application/x-media-file-id` |
| Composition | `composition` | `application/x-composition-id` |
| Text item | `text` | `application/x-text-item-id` |
Expand All @@ -450,7 +454,7 @@ interface MediaFile {
### Track Type Enforcement
| Media Type | Allowed Tracks |
|------------|----------------|
| Video/Image/Composition/Text/Solid/Mesh | Video tracks only |
| Video/Image/Lottie/Composition/Text/Solid/Mesh | Video tracks only |
| Audio | Audio tracks only |

---
Expand All @@ -474,7 +478,8 @@ On project load:
- Thumbnails restored from `Cache/thumbnails` by file hash
- Existing proxies detected automatically, including legacy media-id based storage
- Existing transcripts and analysis data loaded from the project folder
- Blob URLs regenerated for available files
- Dead blob/object URLs are regenerated for available files
- If a retained `File` object is still present, image/video thumbnails are rebuilt when needed after refresh
- Folder structure, expansion state, dock layout, and per-composition view state restored

### Media File IDs
Expand Down
5 changes: 5 additions & 0 deletions docs/Features/Project-Persistence.md
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,7 @@ The `setupAutoSync()` function (in `projectLifecycle.ts`) subscribes to store ch
- Clips or tracks change (timelineStore)
- YouTube panel state changes
- Dock layout changes
- Export settings or export presets change

### Manual Save
- `Ctrl+S` shortcut
Expand Down Expand Up @@ -240,6 +241,7 @@ When opening a project, the app automatically:
3. Checks read permission on restored handles
4. Scans the `Raw/` folder for missing files (exact filename match, case-insensitive)
5. Also checks stored IndexedDB handles for files not found in Raw
6. Regenerates missing object URLs for files that were restored successfully and rebuilds previews when the underlying `File` object is still available

### Reload All Button
In Media Panel toolbar:
Expand Down Expand Up @@ -301,6 +303,7 @@ interface ProjectFile {
- Nested composition references
- Text clip properties
- Solid clip color
- Vector animation settings (loop, end behavior, fit, animation selection, background)
- Transcript and analysis data per clip
- Scene description data

Expand All @@ -311,6 +314,7 @@ interface ProjectFile {
- Bitrate and file size
- hasAudio flag
- Proxy status
- Vector animation metadata (provider, animation names, default animation, frame count)
- Folder organization (folderId)
- `projectPath` when the file is copied into `Raw/`

Expand All @@ -321,6 +325,7 @@ interface ProjectFile {
- Transcript language preference
- View toggles: thumbnails, waveforms, proxy, transcript markers
- Changelog preferences (`showChangelogOnStartup`, `lastSeenChangelogVersion`)
- Export panel state: live export settings, named export presets, and the selected preset

### Other Persisted Panels
- YouTube panel state is saved in `project.json`
Expand Down
10 changes: 6 additions & 4 deletions docs/Features/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

Current feature documentation for the `staging` branch.

Version 1.5.1 | April 2026
Version 1.5.4 | April 2026

---

Expand All @@ -23,6 +23,7 @@ The docs in this folder were re-audited against the current codebase and now tra
| **AI Control** | OpenAI chat with 79 exported tools plus local/native bridge access for external agents |
| **AI Video Workspace** | Classic AI Video plus FlashBoard board-mode generation and media import |
| **3D Layers** | Three.js layers, camera clips, Gaussian splats, and splat effectors |
| **Vector Animation** | Lottie clips with deterministic canvas playback, looping, and export |
| **Audio** | Element-synced playback, drift correction, waveform extraction, EQ, and audio export |
| **Project Storage** | `project.json` source of truth, RAW-copy-first media flow, autosave, relink, backups |
| **Native Helper** | Firefox storage backend, yt-dlp download flow, local AI bridge, native jobs |
Expand Down Expand Up @@ -53,8 +54,9 @@ The docs in this folder were re-audited against the current codebase and now tra
| [Masks](./Masks.md) | Overlay mask editing, feathering, stored modes, and current limitations |
| [Text Clips](./Text-Clips.md) | Canvas-backed text rendering, typography controls, and timeline text items |
| [3D Layers](./3D-Layers.md) | Three.js scene path, native Gaussian splats, cameras, and splat effectors |
| [Vector Animation](./Vector-Animation.md) | Lottie import, runtime playback, looping, and export behavior |
| [Audio](./Audio.md) | Playback sync, EQ, waveform extraction, audio clip behavior, and export |
| [Export](./Export.md) | WebCodecs fast/precise export, FFmpeg WASM path, and interchange output |
| [Export](./Export.md) | WebCodecs fast/precise export, FFmpeg intermediates, image/audio-only export, FCPXML, and project-persistent presets |
| [Proxy System](./Proxy-System.md) | Proxy generation, on-disk frame layout, audio proxies, and warmup behavior |
| [Media Panel](./Media-Panel.md) | Import flow, RAW-copy promotion, folders, compositions, and relinking |
| [Project Persistence](./Project-Persistence.md) | Save/load model, IndexedDB handle cache, continuous save, interval save mode, relink, and project roots |
Expand Down Expand Up @@ -109,7 +111,7 @@ Native Helper Rust service with HTTP/WebSocket bridge, yt-dlp, helper-backed

## Audit Notes

- The authoritative app version is [`src/version.ts`](../../src/version.ts), currently `1.5.1`.
- The authoritative app version is [`src/version.ts`](../../src/version.ts), currently `1.5.4`.
- Preview quality is wired into engine-backed preview resolution through `useEngine()`; it does not affect export resolution or the HTML-only source monitor.
- `openComposition` and `searchVideos` are still the two known AI dispatch gaps.
- Gaussian AI tool definitions exist in code but are not exported through `AI_TOOLS` yet.
Expand All @@ -120,4 +122,4 @@ Native Helper Rust service with HTTP/WebSocket bridge, yt-dlp, helper-backed
## Version History

See [`src/version.ts`](../../src/version.ts) and [`src/changelog-data.json`](../../src/changelog-data.json) for the authoritative changelog.
Current version: 1.5.1.
Current version: 1.5.4.
Loading
Loading