Tare

Tare is a low-latency synchronized LAN audio platform for shared movie experiences.

A host computer plays a movie while connected mobile clients receive synchronized audio streams over a local WiFi network.

Users watch the projected video together while listening through their own headphones.

Different users can independently select different audio tracks and languages while remaining synchronized with the host video.

---

Goals

Tare is designed for:

• synchronized multi-device movie playback
• long-duration synchronization stability
• low-latency LAN streaming
• independent audio track selection
• seamless movie-night experiences

The system prioritizes:

• synchronization stability
• deterministic timing
• smooth playback
• predictable latency

---

Platform Support

Host

• macOS
• Windows

Client

• Android
• iOS

---

Architecture Overview

tare/
├── README.md
├── melos.yaml
└── packages/
    ├── tare_core/
    ├── tare_host/
    └── tare_client/

Packages

• tare_core: Pure Dart package containing the synchronization engine, packet protocols, clock negotiation, jitter buffering, and shared data models. No Flutter dependency allowed.
• tare_host: Flutter Desktop application responsible for video playback, audio extraction via FFmpeg, Opus encoding, UDP broadcasting, and synchronisation beacons.
• tare_client: Flutter Mobile application responsible for zero-configuration discovery, TCP session handshakes, receiving multicast UDP audio, decoding Opus packets, jitter buffering, and synchronized playback.

Core Technologies

• Flutter: Cross-platform application framework.
• media_kit: Hardware-accelerated Desktop video playback.
• FFmpeg: Real-time audio track extraction & pacing.
• Opus: Highly-optimized, low-latency audio codec.
• UDP Multicast: Primary transport for audio packets and synchronization beacons.
• TCP: Reliable control channel for session join handshakes and interactive zone events.

---

2. Architecture & Isolate Topology

To ensure zero stuttering and maximum UI responsiveness, heavy work is offloaded to background isolates.

graph TB
    subgraph Host ["Host Platform (Desktop)"]
        subgraph HostUI ["UI Isolate"]
            VPlayer["Video Player (media_kit)"]
            HostCtrl["Host Controller (Cubit)"]
            ReactionsUI["Floating Reactions Renderer"]
        end
        
        subgraph FFmpegIsolate ["FFmpeg Isolate"]
            Extractor["FFmpeg Audio Extractor"]
            Encoder["Opus Encoder (20ms frames)"]
        end
        
        subgraph BroadcasterIsolate ["Broadcaster Isolate"]
            USocket["UDP Socket (Multicast/Unicast)"]
            TSocket["TCP Server (Control & Handshake)"]
            BeaconGen["Sync Beacon Generator"]
        end
    end

    subgraph Client ["Client Platform (Mobile)"]
        subgraph ClientUI ["UI Isolate"]
            ClientCtrl["Client Session Cubit"]
            EmojiSender["Reaction Input & Visuals"]
            Playout["Audio Playout Thread"]
        end
        
        subgraph ReceiverIsolate ["Receiver Isolate"]
            CRXSocket["UDP Multicast Receiver"]
            CTCPClient["TCP Control Client"]
            OpusDec["Opus Decoder"]
            JBuffer["Jitter Buffer"]
            SEngine["Sync Engine (Drift Evaluator)"]
        end
    end

    %% Network Connections
    TSocket <--> |"TCP: Session Join / Control / Reactions"| CTCPClient
    USocket --> |"UDP Multicast: Audio Packets (Ogg-Opus)"| CRXSocket
    USocket --> |"UDP Multicast: Sync Beacons (Monotonic Clocks)"| CRXSocket
    
    %% Host Internal Flows
    VPlayer -->|"Monotonic Time & Track State"| HostCtrl
    HostCtrl -->|"FFmpeg Extraction Request"| Extractor
    Extractor -->|"Raw Audio"| Encoder
    Encoder -->|"Opus Audio Frames"| USocket
    HostCtrl -->|"Beacon Info"| BeaconGen
    BeaconGen -->|"Beacons"| USocket
    TSocket -->|"Reaction Events"| ReactionsUI
    
    %% Client Internal Flows
    CRXSocket -->|"Encoded Opus Frames"| OpusDec
    CRXSocket -->|"Sync Beacons"| SEngine
    OpusDec -->|"Decoded PCM"| JBuffer
    JBuffer -->|"Buffer Level / Pacing"| SEngine
    SEngine -->|"Speed / Seek Actions"| Playout
    CTCPClient -->|"Interactive Reactions"| EmojiSender

Loading

---

3. Session Setup & Joining Sequence

sequenceDiagram
    autonumber
    actor UserClient as Client User
    participant C_UI as Client UI Isolate
    participant C_RX as Client Receiver Isolate
    participant H_BC as Host Broadcaster Isolate
    participant H_UI as Host UI Isolate

    %% mDNS Discovery
    Note over C_UI, H_UI: 1. Zero-Configuration mDNS Discovery
    H_UI->>H_UI: Initialize Bonsoir Service
    H_UI->>H_UI: Advertise '_tare._tcp' Service
    C_UI->>C_UI: Start Bonsoir Service Discovery
    H_UI-->>C_UI: Service Resolved (Host IP & Control Port)

    %% TCP Handshake
    Note over C_UI, H_UI: 2. TCP Session Establishment
    UserClient->>C_UI: Enter Name & Tap Join
    C_UI->>C_RX: Launch TCP Connection
    C_RX->>H_BC: TCP Connect (Control Port)
    C_RX->>H_BC: Send JoinRequest(clientId, displayName, version)
    H_BC->>H_UI: Dispatch ClientJoinEvent
    H_UI->>H_UI: Register Client & Choose Audio Track
    H_BC->>C_RX: Send JoinAck(sessionCode, multicastAddress, availableTracks, hostVideoDelayUs)
    C_RX->>C_UI: Handshake Complete & State Updated

    %% Multicast Stream Joining
    Note over C_UI, H_UI: 3. Audio & Sync Multicast Engagement
    C_RX->>C_RX: Bind & Join Multicast Group (239.0.0.1)
    H_BC->>C_RX: UDP Multicast Stream: Audio Packets & Sync Beacons
    C_RX->>C_UI: Continuous Audio Playout Starts

Loading

---

4. Synchronization & Precision Drift Correction Loop

Tare operates a synchronization-first engine using monotonic clocks. It avoids absolute wall-clock times to protect against network jitter and clock skew.

graph TD
    %% Define styles for clarity
    classDef formula fill:#1f2937,stroke:#3b82f6,stroke-width:2px,color:#fff;
    classDef action fill:#065f46,stroke:#10b981,stroke-width:2px,color:#fff;
    classDef decision fill:#7c2d12,stroke:#f97316,stroke-width:2px,color:#fff;

    Start([Sync Beacon Received]) --> ReadLocalTime[Read Client Local Monotonic Clock:<br/>localMonotonicUs]
    ReadLocalTime --> Step1[Calculate Projected Host Position]
    
    Step1 --> Formula1["projectedHost = hostPosition + (localMonotonic + clockOffset - hostMonotonic)"]:::formula
    
    Formula2["effectiveLocal = localPlaybackPosition + jitterBufferDelay - userDelay"]:::formula
    
    Formula1 & Formula2 --> CalcDrift[Calculate Playout Drift]
    
    CalcDrift --> Formula3["drift = projectedHost - effectiveLocal"]:::formula
    
    Formula3 --> Evaluate{{"Evaluate Drift |drift|"}}
    
    Evaluate -->|"< 20ms (Micro Threshold)"| NoAction["Normal Speed (1.0x)<br/>Sync Decision: NONE"]:::action
    
    Evaluate -->|">= 120ms (Hard Seek Threshold)"| HardSeek["Emergency Hard Seek!<br/>Seek to: projectedHost + userDelay"]:::action
    
    Evaluate -->|"20ms <= |drift| < 120ms"| MicroAdjust["Precision Rate Adjust<br/>Capped at +/- 2% tempo speed"]:::action
    
    MicroAdjust --> FormulaRate["rate = 1.0 + clamp(drift / 5,000,000, -0.02, 0.02)"]:::formula
    
    NoAction & HardSeek & FormulaRate --> ApplyAdjustment([Apply Playout Speed/Seek in Audio Engine])

Loading

See System Architecture Documentation for full detailed derivations of drift estimation formulas and clock synchronisation algorithms.