MVP for porting to Matrix

[mensch72]

This issue details the MVP to prototype porting vodle's backend from CouchDB/PouchDB to the Matrix protocol, focusing on mapping each poll to a distinct Matrix room and mapping user-specific data to user-owned private rooms. The goal is to make a feature-complete comparison between Matrix and the current sync/database model, surfacing opportunities and blockers for a full migration.

Why Matrix?
Matrix is an open standard for decentralized communication and event-driven data federation, offering device sync, end-to-end encryption, presence, and federation. The protocol brings real-time event streaming capabilities which could be leveraged for collaborative polling workflows, while private rooms offer device-synced state and privacy for user data.

Design & Architecture

• Poll Rooms
  • Each poll is a dedicated Matrix room, identified by a unique room alias.
  • Poll metadata (title, description, due date, options, tally logic) encoded via state events.
  • Votes, delegations, and poll actions (closing, reopening, revealing tallies) are emitted as timeline events with type m.room.message or custom event types (vodle.vote, vodle.delegate).
  • Room membership (invited, joined, left) grants or restricts voting rights and poll visibility.
  • State events updated for poll lifecycle changes (opening/closing/status/tally), versioned using event IDs.
  • Optionally, one can implement "bridging" to map poll DB state snapshots to Matrix events for migration/backfill ease.
• User Private Data Rooms
  • Each user gains one personal encrypted room (is_direct=true, only visible to themselves).
  • User-specific preferences, participation history, last state, and device-private keys are stored as custom state events (e.g., vodle.userdata).
  • Device sync using Matrix's multi-device capabilities; user room state converges across devices upon login/session refresh.
  • User private room serves as a replacement for local-only DB, reflecting ephemeral settings, keys, and per-user status.
  • Access control: one-member room (creator only); recommend encryption via Matrix E2EE.

Implementation requirements:

• Use matrix-js-sdk for JS client-side prototyping; recommend Synapse or Dendrite as the test homeserver.
• Write adapters or wrappers to translate between CouchDB/PouchDB JSON docs and Matrix event model.
  • Core: converting poll structures to Matrix state events, mapping votes/delegations as timeline events.
• Create UI flows:
  • Poll creation: create room, set metadata state events, invite participants.
  • Poll vote/delegate: send custom timeline events, synchronize updates via Matrix sync.
  • Poll closing/tally: set status event, publish results as state or timeline event.
  • User room setup: create/join private room, sync preferences, reflect login/logout activity.
• Offline/Online workflow:
  • Test Matrix event sync convergence across offline/online transitions (simulate airplane mode, reconnects).
  • Test federation: run two homeservers, simulate two users voting on same poll from different servers.
  • Measure latency, event order, and data accuracy for both poll and user room flows.
• Security/Privacy:
  • Use E2EE for user private rooms and for polls that require confidential votes.
  • Limit visibility for private poll rooms, restrict membership using room ACLs.
  • Experiment with history visibility (Matrix API) to restrict access to older votes if needed.
• Migration tooling:
  • Design scripts or utilities to export current poll data from CouchDB to JSON suitable for injection as Matrix events/state.
  • Document migration pain points—especially around event ordering, state replay, and indexing.

Success metrics:

• Demonstrate full poll lifecycle (create, vote, delegate, tally, close/reopen) using Matrix-only backend for at least 3 clients (3 distinct users/devices).
• Demonstrate user data sync/restore (settings/preferences/history) across at least 2 devices using user private room.
• Show resiliency to network partitions, offline mode, and federation splits/rejoins.
• Performance report: event sync latency, data throughput, data recovery/scalability for high poll volumes.
• Security: privacy of user room data, poll visibility restrictions, E2EE configuration and bot integration for key management.
• Migration report: summary of data schema mapping, lossless export, and minimum viable fidelity in poll migration.

Technical steps/Example snippets:

• Poll lifecycle in matrix-js-sdk:

import { createClient } from "matrix-js-sdk";
const client = createClient({ baseUrl: "https://your-homeserver", accessToken, userId });
// Create a poll
client.createRoom({ name: "Poll: Ice Cream", visibility: "private", invite: [userId1, userId2], alias: "#poll-1234" })
  .then(room => {
    client.sendStateEvent(room.roomId, "vodle.poll", {
      title: "Favorite flavor?", options: ["Vanilla", "Strawberry", "Lemon"], due: "2025-11-09T15:00:00Z"
    }, "");

    // Send a vote
    client.sendEvent(room.roomId, "vodle.vote", { voter: userId1, option: "Vanilla" });

    // Close poll
    client.sendStateEvent(room.roomId, "vodle.poll.status", { status: "closed" }, "");
  });

User private room/channel example:

client.createRoom({ name: "User Private Room", is_direct: true, invite: [userId], preset: "trusted_private_chat", encryption: true })
  .then(room => {
    client.sendStateEvent(room.roomId, "vodle.userdata", { settings: { lang: "de", darkmode: true } }, "");
  });

Listening for events (sync and event processing):

client.on("Room.timeline", (event, room, toStartOfTimeline) => {
    if(event.getType() === "vodle.vote") {
        // Handle vote logic
    } else if(event.getType() === "vodle.poll.status") {
        // Update poll status
    }
    // ...other event types
});
client.startClient();

Open questions to address:

How to efficiently query Matrix rooms for aggregated tallies, option counts, or historical vote events? Do room state events suffice or is a secondary index needed?
How to perform atomic/validated updates to poll state? (Room state events are "last write wins"—how does this interact with concurrent votes or delegated logic?)
How to migrate legacy poll data with self-contained event timelines and converged state?
What are failure modes when federation splits, and how do they impact cross-server poll sync?

Use this issue for user stories, architectural sketches, design docs, discussion of migration scripts, and prototype links. Include any missteps, data fidelity loss, and protocol gaps encountered during MVP work.

Related code in current repo for mapping reference:

src/app/data.service.ts
src/app/poll.service.ts
src/app/previewpoll/previewpoll.page.ts

Matrix Poll Rooms – Requirements & Workflow

• Room Architecture:
  • Each poll is a Matrix room (alias format: #poll-<poll_id>). All poll-related state and actions are stored as Matrix events.
  • Poll metadata is a state event (vodle.poll): contains title, description, close time, options.
  • Membership controls: invited users can vote/see results; optionally public poll rooms for open surveys.
  • Actions (vote, delegate, state changes) encoded as custom timeline events (vodle.vote, vodle.delegate, vodle.poll.status).
  • Poll lifecycle (open, close, tally, reveal): handled by emitting status state events.
  • Historical votes kept in timeline; current tally is recomputed from timeline on demand.
  • Add schema for event consistency, e.g., vodle.poll.status with status=open|closed|tallying|revealed.
  • E2EE integration for confidential polls; restrict read/write with server ACL and room encryption settings.
• Example matrix-js-sdk Poll Creation:

client.createRoom({ alias_name: "poll-icecream", invite: [user1, user2], preset: "private_chat", name: "Ice Cream Poll", encryption: true })
.then(room => {
  client.sendStateEvent(room.roomId, "vodle.poll", { title: "Best flavor?", options: ["chocolate","strawberry","lemon"], due: "2025-11-15T16:00:00Z" }, "");
  client.sendEvent(room.roomId, "vodle.vote", { voter: user1, option: "chocolate" });
  client.sendStateEvent(room.roomId, "vodle.poll.status", { status: "closed", timestamp: Date.now() }, "");
});

Matrix User Private Room – Requirements & Workflow

Room Architecture:
    Each user has a direct encrypted room (preset "trusted_private_chat", is_direct=true). Only the user joins; no remote guests allowed.
    Settings, app history, keys, device-specific state stored as state events (vodle.userdata).
    Prefer E2EE and local device key management for privacy and offline usability.
    State events reflect login activity, app preferences, and poll participation history; users retrieve their own state across devices.
Example matrix-js-sdk User Room Creation:

client.createRoom({ name: "User Private Room", preset: "trusted_private_chat", is_direct: true, invite: [userId], encryption: true })
.then(room => {
  client.sendStateEvent(room.roomId, "vodle.userdata", { settings: { lang: "de", colorscheme: "dark" } }, "");
});

Testing, Migration, and Federation

Validate via multi-device and multi-homeserver scenarios:
    Poll vote events propagate near-real-time; status changes converge reproducibly under partition/reconnect.
    Sync user room state across devices (phone + desktop).
Write a migration tool to export CouchDB docs, convert to event payloads, batch inject into Matrix rooms, and verify lossless conversion of structure, timestamps, and authorship.
Use federation splits: simulate two users on different Matrix homeservers voting then merging poll room histories, resolving partition events and state.
Security: verify E2EE restricts state/timeline visibility; measure event replay on device recovery (history visibility, leave/join rehydration).
Performance: log event sync latency, polling state recomputation time, event replay speed.

Open issues and research questions:

How to design secondary indices (for fast tally, delegation graph, historical search) atop Matrix?
How does "last write wins" room state affect poll close/reopen, delegated votes, and concurrent changes?
Which edge cases in migration lead to loss of data or timeline inconsistency?
How does Matrix federation resolve conflicting poll state on network partition or during recovery?
Is it practical to implement CRDT-based merge for poll tallies using Matrix event model?

Use this issue as a blueprint, log all architectural decisions, code samples, and test finding, and update with observations during real-world MVP prototyping.

Reference mapping sources in current vodle repo:

src/app/data.service.ts
src/app/poll.service.ts
src/app/previewpoll/previewpoll.page.ts