Skip to content

RFC.md

Latest commit

 

History

History
68 lines (49 loc) · 2.9 KB

RFC.md

File metadata and controls

68 lines (49 loc) · 2.9 KB
Raw

RFC: MultixAI - Unified Protocol for AI and Media Streaming

1. Introduction

MultixAI is a protocol designed for low-latency, secure, and extensible streaming of AI data, audio, and video, while supporting modern web and meeting technologies through adapter services.

2. Goals and Use Cases

  • Real-time AI inference (LLMs, speech, vision)
  • Audio/video conferencing and streaming
  • Hybrid scenarios: live transcription, translation, moderation
  • Seamless interoperability with WebRTC/WebTransport

3. Architecture

  • Runs over QUIC/UDP or TCP; supports fallback to WebSocket
  • Multiplexed logical channels (audio, video, AI, control)
  • Adapter service enables bridging with WebRTC/WebTransport

4. Packet/Header Format

Field Size Description
Version/Flags 2B Protocol version, feature flags
Message Type 1B Request, response, stream, AUDIO, VIDEO, control
Session ID 4B Unique session identifier
Channel ID 4B Logical stream/channel within session
Media Type/Sub 1B AI, TEXT, AUDIO, VIDEO, reserved
Codec/Format 1B Opus, H.264, Protobuf, etc.
Seq Number 2B Media/stream packet ordering
Priority/QoS 1B Delivery guarantees, drop tolerance
Header Length 2B Extension headers length
Payload Length 4B Payload size
Extensions var TLV or CBOR-encoded key-value extension headers
Payload var Data (media, AI, control)

5. Communication Flow

  1. Session Setup: Handshake with capabilities, codecs, and authentication.
  2. Channel Management: Dynamic creation/removal of audio, video, AI, control channels.
  3. Data Transfer: Streaming, request/response, batching, with prioritized delivery.
  4. Adapter Service: Bridges WebRTC/WebTransport streams to MultixAI format and vice versa.

6. Security

  • Encrypted by default (TLS/DTLS/QUIC)
  • Token-based or mutual TLS authentication
  • Replay protection via nonces/timestamps

7. Extensibility

  • Versioned and flag-rich header for upgrades
  • Extension headers for new metadata or control info
  • Negotiable message/media types and codecs

8. Interoperability

MultixAI adapters SHOULD support:

  • Re-encapsulation of media/data from WebRTC/WebTransport
  • Metadata and signaling translation
  • Bidirectional flow for in-meeting AI features

9. References