Skip to content

01.md

Latest commit

 

History

History
331 lines (240 loc) · 20.5 KB

01.md

File metadata and controls

331 lines (240 loc) · 20.5 KB
Raw

MIP-01

Group Construction & Marmot Group Data Extension

review mandatory

This document defines how to create and configure MLS groups with Nostr integration, including the mandatory Marmot Group Data Extension. This is one of the core specifications that MUST be implemented by all projects wanting to be interoperable.

Creating Groups

Group Identity and Privacy

Every MLS group has a unique 32-byte ID (distinct from the nostr_group_id field of the Marmot Group Data Extension detailed below) that serves as its permanent identifier. This ID is:

  • Private: Never published to relays in any form
  • Permanent: Cannot be changed once the group is created
  • Random: Generated using secure randomness

Compatibility Requirements

Before creating a group, clients must verify compatibility with intended members by checking their published KeyPackage events. This ensures all members support the same:

  • Ciphersuite: Cryptographic algorithms used for the group
  • Capabilities: MLS features the group will use
  • Extensions: Additional functionality required by the group

Required MLS Extensions

All groups MUST include these MLS extensions:

Group State Changes

Group modifications follow a two-step process:

  1. Proposal: Any member can propose changes (add/remove members, update settings)
  2. Commit: Changes are applied to the group state. Admins can commit any proposals, while non-admin members can only create self-update Commits (updating their own key material). See MIP-03 for details.

While these are MLS operations, Commit events must be published to Nostr relays so all members can see and process the changes.

Marmot Group Data Extension

The marmot_group_data extension brings Marmot protocol specific data into authenticated and cryptographically verified MLS group state. This extension:

  • Is mandatory: MUST be included in all groups
  • Is cryptographically secure: Changes are authenticated and tamper-proof
  • Stores Nostr metadata: Contains information like group admins and Nostr-specific settings

This extension MUST be included as a required capability when creating any new group.

Extension Identifier

This extension uses the identifier 0xF2EE. Implementations MUST use this identifier when registering support for this extension in MLS capabilities negotiation. This extension MUST be included in individual client capabilities (via LeafNode.capabilities in KeyPackages) and as a group-level requirement (via GroupContext.extensions).

Purpose and Use Cases

The Marmot Group Data Extension serves several critical functions:

  1. Group Identity Mapping: Links MLS group encryption with Nostr's decentralized identity system
  2. Relay Agreement: Provides authenticated relay endpoints for group message distribution
  3. Access Control: Establishes cryptographically verified admin roles independent of MLS protocol roles
  4. Group Image Encryption: Enables secure, encrypted sharing of the group image with ChaCha20-Poly1305 and optional cleanup of old images via deterministic upload identity derivation
  5. Metadata Protection: Ensures group metadata changes are authenticated and verifiable by all members

TLS Serialization Requirements

CRITICAL: This extension MUST use TLS presentation language serialization (RFC 8446 Section 3) with proper length prefixes and byte alignment. All variable-length vectors use QUIC-style variable-length integer encoding (RFC 9000 Section 16) for length prefixes, as implemented by the tls_codec crate (v0.4+). This means length prefix size varies based on the content length:

Content Length Prefix Size Format
0-63 1 byte 0b00xxxxxx
64-16383 2 bytes 0b01xxxxxx xxxxxxxx
16384+ 4 bytes 0b10xxxxxx ...

Incorrect TLS serialization will cause interoperability failures.

Extension Fields

Field Specifications

Field TLS Type Length Constraints Description
version uint16 version Exactly 2 bytes Extension format version number. Current version is 2. Implementations MUST validate version compatibility before processing extension data.
nostr_group_id opaque nostr_group_id[32] Exactly 32 bytes A 32-byte identifier for the group used in Nostr protocol operations. This value is distinct from the MLS group ID and MAY be updated over time through group proposals. This identifier is used in the h tags when publishing group message events to Nostr relays. MUST be cryptographically random when initially generated.
name opaque name<0..2^16-1> 0-65535 bytes UTF-8 encoded group name. MUST be valid UTF-8. SHOULD be limited to 255 characters for practical display purposes. Empty string is permitted for unnamed groups.
description opaque description<0..2^16-1> 0-65535 bytes UTF-8 encoded group description. MUST be valid UTF-8. SHOULD be limited to 1000 characters for practical display purposes. Empty string is permitted.
admin_pubkeys opaque admin_pubkeys<0..2^16-1> 0-65535 bytes Variable-length vector containing concatenated raw 32-byte Nostr x-only public keys. Each public key MUST be a valid secp256k1 x-only public key. The total byte length MUST be a multiple of 32. Clients MUST verify admin status before processing proposals that modify group state, membership, or metadata. The array MUST NOT contain duplicate keys. At least one admin key MUST be present for most group operations. See Array Encoding.
relays RelayUrl relays<0..2^16-1> 0-65535 bytes Variable-length vector of individually length-prefixed relay URLs. Each element is a UTF-8 encoded WebSocket URL with its own QUIC varint length prefix. Each URL MUST be a valid WebSocket URI (ws:// or wss://). The vector SHOULD contain at least one relay URL and SHOULD NOT exceed 20 relays. See Array Encoding.
image_hash opaque image_hash<0..32> 0 or 32 bytes SHA-256 hash of the encrypted group image stored via Blossom protocol. The hash identifies the encrypted blob on the storage server. Empty (zero-length) when no image is set; exactly 32 bytes when set.
image_key opaque image_key<0..32> 0 or 32 bytes Image encryption seed. Used with HKDF to derive the ChaCha20-Poly1305 encryption key (see Image Encryption). MUST be cryptographically random when generated. Empty when no image is set; exactly 32 bytes when set.
image_nonce opaque image_nonce<0..12> 0 or 12 bytes ChaCha20-Poly1305 nonce for group image encryption. MUST be cryptographically random and unique for each image. Used with the derived encryption key for authenticated encryption/decryption. Empty when no image is set; exactly 12 bytes when set.
image_upload_key opaque image_upload_key<0..32> 0 or 32 bytes Upload seed for deriving the Nostr keypair used for Blossom upload authentication (see Image Upload Identity). Cryptographically independent from image_key. MUST be cryptographically random when generated. Empty when no image is set; exactly 32 bytes when set.

TLS Structure Definition

In TLS presentation language, the extension data MUST be structured as:

opaque RelayUrl<0..2^16-1>;        // UTF-8 encoded WebSocket URL

struct {
    uint16 version;                    // Version number (current: 2)
    opaque nostr_group_id[32];
    opaque name<0..2^16-1>;
    opaque description<0..2^16-1>;
    opaque admin_pubkeys<0..2^16-1>;   // Concatenated raw 32-byte x-only pubkeys
    RelayUrl relays<0..2^16-1>;        // Vector of length-prefixed URL strings
    opaque image_hash<0..32>;          // 0 or 32 bytes
    opaque image_key<0..32>;           // 0 or 32 bytes (encryption seed)
    opaque image_nonce<0..12>;         // 0 or 12 bytes
    opaque image_upload_key<0..32>;    // 0 or 32 bytes (upload seed)
} NostrGroupData;

Version Field

The version field enables forward-compatible evolution of the extension format:

  • Version 1: Initial versioned format (deprecated). Used image_key directly as the ChaCha20-Poly1305 encryption key and derived the upload keypair from image_key. Did not include image_upload_key field. Used hex-encoded admin pubkeys.
  • Version 2 (current): Uses raw 32-byte admin pubkeys. Uses image_key as a seed for HKDF derivation of the encryption key. Adds image_upload_key as a separate seed for upload keypair derivation, providing cryptographic independence between encryption and upload authentication. Variable-length image fields allow empty encoding when no image is set.
  • Version 0: Reserved and MUST be rejected as invalid. Version numbering starts at 1.
  • Future Versions: Higher version numbers indicate newer formats with additional fields appended at the end
  • Forward Compatibility: Implementations SHOULD gracefully handle unknown versions by parsing known fields and ignoring unknown trailing fields

Implementation Considerations

Array Encoding Specifications

Admin Public Keys Array: The admin_pubkeys field is a variable-length vector containing concatenated raw 32-byte x-only public keys with a single outer QUIC varint length prefix:

  • Outer length prefix: QUIC varint encoding the total byte count of all keys
  • Keys: Concatenated raw 32-byte x-only secp256k1 public keys (no per-element length prefix)
  • The total byte length MUST be a multiple of 32; divide by 32 to get the key count
  • Keys are packed sequentially with no separators or padding

Wire format example (admin_pubkeys with two keys):

[QUIC varint: 64][32 bytes of pubkey₁][32 bytes of pubkey₂]

Note: With 2 keys (64 total bytes), the outer QUIC varint prefix is 2 bytes (since 64 >= 64 threshold). With 1 key (32 bytes), the prefix is 1 byte.

Relays Array: The relays field is a variable-length vector of individually length-prefixed relay URLs:

  • Outer length prefix: QUIC varint encoding the total byte count of all inner elements (including their prefixes)
  • Each element: QUIC varint length prefix for that URL's byte count, followed by the UTF-8 encoded URL bytes
  • Each URL MUST be a valid WebSocket URI (ws:// or wss://)

Wire format example (relays with two URLs):

[QUIC varint: total inner bytes][QUIC varint: url₁ len][url₁ bytes][QUIC varint: url₂ len][url₂ bytes]

Image Encryption

The group image encryption uses ChaCha20-Poly1305 with key material derived via HKDF from seeds stored in the extension.

Encryption (v2): When encrypting a group image:

image_key = random(32)              // Cryptographically random encryption seed
image_nonce = random(12)            // Cryptographically random nonce
image_upload_key = random(32)       // Cryptographically random upload seed

// Derive encryption key from seed using HKDF (Extract then Expand)
prk = HKDF-Extract(salt=None, IKM=image_key)
encryption_key = HKDF-Expand(prk, "mip01-image-encryption-v2", 32)

encrypted_image = ChaCha20-Poly1305.encrypt(encryption_key, image_nonce, image_data)

The image_key field stores the seed, not the encryption key directly. This provides domain separation between the stored seed and the derived encryption key.

Image Upload Identity

The Nostr keypair for Blossom upload authentication MUST be deterministically derived from the image_upload_key seed:

prk = HKDF-Extract(salt=None, IKM=image_upload_key)
upload_secret = HKDF-Expand(prk, "mip01-blossom-upload-v2", 32)
upload_keypair = secp256k1_keypair_from_secret(upload_secret)

The image_upload_key is cryptographically independent from image_key, ensuring that the upload authentication keypair is fully separated from the encryption key material.

Update and Cleanup Workflow:

When an admin updates the group image:

  1. Generate new seeds: Create new random image_key (32 bytes), image_nonce (12 bytes), and image_upload_key (32 bytes)
  2. Derive encryption key: Compute prk = HKDF-Extract(salt=None, IKM=image_key), then encryption_key = HKDF-Expand(prk, "mip01-image-encryption-v2", 32)
  3. Encrypt new image: Use ChaCha20-Poly1305 to encrypt the new group image with the derived key
  4. Derive upload keypair: Compute prk = HKDF-Extract(salt=None, IKM=image_upload_key), then upload_secret = HKDF-Expand(prk, "mip01-blossom-upload-v2", 32) and derive the Nostr keypair
  5. Upload to Blossom: Upload the encrypted image using the derived keypair for authentication, obtaining new_image_hash = SHA256(encrypted_image)
  6. Update extension: Commit an MLS proposal updating image_hash, image_key, image_nonce, and image_upload_key fields
  7. Cleanup old image: After successful commit, derive the previous upload keypair from old_image_upload_key and request deletion of the old image blob from Blossom

Security Properties (see threat_model.md for threat assumptions and context):

  • Encryption independence: Image encryption keys persist across MLS epochs (unlike chat media in MIP-04 which uses rotating exporter secrets)
  • Cleanup capability: Any admin who has access to a previous image_upload_key can derive the upload keypair to delete that image blob
  • Domain separation: HKDF with distinct context strings ensures cryptographic independence between the encryption key, the upload secret, and the stored seeds
  • Key independence: image_key and image_upload_key are independently random, so compromise of one does not affect the other
  • Forward compatibility: Old members who leave cannot access new group images (they don't receive updated extension data)

Extension Lifecycle Management

Creation: When creating a new group, this extension MUST be populated with initial values and included in the group's required capabilities.

Updates: Any field in this extension can be updated through MLS Proposal/Commit mechanisms. All extension updates require admin authorization. Before processing any Commit that modifies this extension, implementations MUST verify the committer is listed in the current admin_pubkeys array. Note: Self-update Commits (where a member updates only their own key material) do not modify this extension and do not require admin authorization (see MIP-03).

Version Handling

Version Detection: Implementations MUST validate the extension format version:

MAX_SUPPORTED_VERSION = 2

function detect_version(extension_data) {
    if (extension_data.length < 2) {
        return ERROR_INVALID_EXTENSION
    }

    // Read version field (uint16, big-endian)
    version = read_uint16_be(extension_data[0:2])

    // Version 0 is reserved and invalid
    if (version == 0) {
        return ERROR_INVALID_EXTENSION
    }

    // Validate known versions
    if (version >= 1 && version <= MAX_SUPPORTED_VERSION) {
        if (validate_structure(extension_data, version)) {
            return version
        }
        return ERROR_INVALID_EXTENSION
    }

    // Handle future versions with forward compatibility
    if (version > MAX_SUPPORTED_VERSION) {
        log_warning("Unknown extension version, attempting forward compatibility")
        return VERSION_UNKNOWN_FUTURE
    }

    return ERROR_INVALID_EXTENSION
}

function validate_structure(data, version) {
    // All variable-length fields use QUIC varint length-prefixed encoding.
    // Minimum size with all variable fields empty (1-byte QUIC varint prefix each):
    //   version(2) + nostr_group_id(32) + name(1+0) + description(1+0) +
    //   admin_pubkeys(1+0) + relays(1+0) + image_hash(1+0) + image_key(1+0) +
    //   image_nonce(1+0) + image_upload_key(1+0)
    MIN_SIZE = 2 + 32 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1  // = 42 bytes

    if (data.length < MIN_SIZE) return false

    offset = 2 + 32  // Skip version and nostr_group_id

    // Validate each variable-length field in order
    variable_fields = [
        "name", "description", "admin_pubkeys", "relays",
        "image_hash", "image_key", "image_nonce", "image_upload_key"
    ]

    for field in variable_fields {
        (field_len, prefix_size) = read_quic_varint(data, offset)
        if (offset + prefix_size + field_len > data.length) return false
        offset += prefix_size + field_len
    }

    // All known fields consumed; ignore any trailing bytes (forward compatibility)
    return true
}

Forward Compatibility: The extension format is append-only: future versions may only add new fields at the end. Parsers MUST parse fields in the documented order. When encountering unknown future versions:

  • Parse known fields in order (version, nostr_group_id, name, description, admin_pubkeys, relays, image_hash, image_key, image_nonce, image_upload_key)
  • Ignore any unknown trailing fields beyond the last known field
  • Continue normal operation with available data
  • Log warning about unknown version for debugging

Implementation Guidelines

Requirements:

  • Register extension 0xF2EE as required capability in both individual capabilities (LeafNode.capabilities) and group requirements (GroupContext.extensions)
  • Use exact TLS structure definition provided
  • Use QUIC-style variable-length integer encoding for all vector length prefixes
  • Implement version detection algorithm for forward compatibility
  • Validate UTF-8 strings, public keys, and URLs
  • Serialize admin_pubkeys as concatenated raw 32-byte x-only public keys (no per-element prefix)
  • Serialize relays as a vector of individually QUIC varint length-prefixed URL strings
  • Verify admin permissions for all extension updates including version changes

Version-Specific Implementation:

  • New Groups: MUST create with version 2 format
  • Version 1 Support: Implementations SHOULD support reading version 1 extensions for backward compatibility. In version 1, image_key is used directly as the encryption key (not as an HKDF seed), image_upload_key is absent (the upload keypair is derived from image_key using context "mip01-blossom-upload-v1"), and admin pubkeys are hex-encoded strings
  • Future Versions: SHOULD implement forward compatibility for unknown versions
  • Error Handling: MUST gracefully handle version mismatches with clear error messages

Testing Requirements:

  • Test version detection and structure validation
  • Test forward compatibility with simulated future versions
  • Verify admin permission checks for version updates
  • Test structure validation for malformed extension data

Summary

This MIP defines the foundation for creating secure, Nostr-integrated groups:

Key Components

  1. Group Creation: Secure MLS group establishment with privacy protection
  2. Required Extensions: Standard MLS extensions plus Nostr Group Data
  3. Marmot Group Data Extension: Cryptographically secure bridge between MLS and Nostr
  4. Admin Controls: Authenticated admin permissions for group management

Implementation Requirements

MUST implement:

  • Required MLS extensions: required_capabilities, ratchet_tree, marmot_group_data
  • TLS serialization for Marmot Group Data Extension with versioning support
  • Version detection and forward compatibility
  • Admin verification for all extension updates including version changes
  • Compatibility checking before group creation
  • Private MLS group IDs (never publish to relays)
  • ChaCha20-Poly1305 encryption for group images with HKDF key derivation: HKDF-Extract(salt=None, IKM=image_key) then HKDF-Expand(prk, "mip01-image-encryption-v2", 32)
  • Deterministic upload keypair derivation using HKDF-Extract(salt=None, IKM=image_upload_key) then HKDF-Expand(prk, "mip01-blossom-upload-v2", 32) for Blossom authentication and cleanup