Skip to content

PROGRESS.md

Latest commit

 

History

History
130 lines (101 loc) · 4.34 KB

PROGRESS.md

File metadata and controls

130 lines (101 loc) · 4.34 KB
Raw

SyncDoc — Progress Tracker

Current Phase: Phase 3 — Production Infrastructure ✅

v3.0

What's Built

Infrastructure

  • PostgreSQL (via pg) — replaces SQLite; async connection pool; same schema
  • Redis pub/sub (via ioredis) — optional; when REDIS_URL is set, Yjs updates are published to ydoc:<docId> channel so all server instances relay them to their local clients
  • y-indexeddb — browser-side offline persistence; doc loads from IndexedDB instantly on open; edits queue offline and merge via CRDT on reconnect
  • Docker Compose — includes postgres:16 + redis:7 services with health checks

Offline flow

  1. User opens doc → IndexedDB loads last known state instantly (no server needed)
  2. WebSocket connects in background → server state merges with local state via CRDT
  3. User goes offline → edits continue writing to IndexedDB and the Yjs doc in memory
  4. User comes back online → WebSocket reconnects, queued updates sync to server
  5. Topbar shows "Offline — editing locally" while disconnected but IndexedDB is ready

Multi-server scaling flow

Server A (clients 1,2)          Redis           Server B (clients 3,4)
─────────────────────           ─────           ─────────────────────
Client 1 types
  → setupWSConnection broadcasts to clients on A
  → ydoc.on('update') fires
  → publishToRedis('ydoc:DOCID', update)   →   redisSub receives 'ydoc:DOCID'
                                                  → Y.applyUpdate(localYdoc)
                                                  → setupWSConnection broadcasts to clients on B

Project Structure

syncdoc/
├── docker-compose.yml        # app + postgres:16 + redis:7
├── Dockerfile
├── PROGRESS.md
│
├── server/
│   ├── .env.example          # DATABASE_URL, REDIS_URL, PORT, CLIENT_ORIGIN
│   └── src/
│       ├── index.js          # Express REST API (all async)
│       ├── collaboration.js  # y-websocket + Redis pub/sub
│       ├── db.js             # PostgreSQL via pg pool
│       └── utils.js          # generateDocId (4-letter)
│
└── client/
    └── src/
        ├── hooks/useYjsEditor.js   # + IndexeddbPersistence, offlineReady state
        └── components/
            └── Topbar.jsx          # shows "Offline — editing locally" status

Setup & Run

Development (with local Postgres + Redis via Docker)

# Start Postgres and Redis only
docker-compose up postgres redis -d

# Install deps and run servers
npm install
cp server/.env.example server/.env
# Edit server/.env — set DATABASE_URL and REDIS_URL

npm run dev

Development (Postgres + Redis already running)

cd syncdoc
npm install
cp server/.env.example server/.env
# DATABASE_URL=postgresql://syncdoc:syncdoc@localhost:5432/syncdoc
# REDIS_URL=redis://localhost:6379
npm run dev

Single-server without Redis

Leave REDIS_URL empty in .env. The server runs fine — just no cross-instance sync.

Production (full stack)

docker-compose up -d
# App: http://localhost:3001
# Postgres data: postgres_data volume
# Redis data: redis_data volume

Scale horizontally

docker-compose up -d --scale app=3
# Add a load balancer (nginx/caddy) in front with sticky sessions (or just round-robin — CRDT handles it)

Environment Variables

Variable Default Required
PORT 3001 No
NODE_ENV development No
DATABASE_URL — Yes
REDIS_URL — (single-server mode if unset) No
CLIENT_ORIGIN http://localhost:5173 Prod: Yes

Remaining Roadmap (Phase 4)

  • Auth — OAuth / magic-link
  • Access permissions — read-only share links
  • Comments — threaded inline comments
  • Document list — recently visited (localStorage)
  • Rate limiting middleware (express-rate-limit)
  • Prometheus metrics endpoint