FB Live Comments — System Design + Go Implementation

A working backend for the comment section of a live video — viewers post comments and everyone sees them stream in near-real-time, modeled on Facebook / Instagram Live. Built in Go with Postgres as the source of truth and a sharded Redis Cluster for cache + live fan-out over Server-Sent Events (SSE).

Part of the System Design Series — real-world system design problems studied end-to-end, each with a full design write-up and runnable code.

---

TL;DR

A single viral live stream can hold a large share of all concurrent viewers, so the design must never pin a hot video to one machine. The whole system follows one idea:

The subscription follows the viewers. Viewers do not follow the subscription.

A viewer connects to any edge server; that server subscribes to Redis only for the videos its own connected viewers are watching. This inversion is what lets the read path scale and makes reconnection trivial.

📖 Full design write-up: DESIGN.md — requirements, write/read paths, fan-out, reconnection, dual-write trade-offs, and the deferred hot-video fan-out tier.

---

What it demonstrates

• Durable write path → Postgres — Postgres is the source of truth; the POST ack means "persisted." Publish to Redis happens only after commit.
• Recent-window cache in Redis — only the newest ~N comments per video are cached (a contiguous sliding window, RPUSH + LTRIM); cache miss falls back to Postgres and backfills.
• Sharded pub/sub fan-out — SPUBLISH / SSUBSCRIBE with a {videoId} hash-tag so a video's cache and its fan-out channel land on the same Redis Cluster shard.
• On-demand SSE live tail — a reader subscribes to a video's shard when its first viewer joins and unsubscribes when the last leaves; the subscription set is dynamic, never pre-assigned.
• Read-your-own-write without sticky routing — the POST returns the persisted comment so the author renders it instantly; the later SSE echo is de-duped by comment id.
• Gap-free reconnect — SSE events carry id: = comment id; on reconnect the browser resends Last-Event-ID and the reader replays everything after it.
• Keyset (cursor) pagination for scroll-back history, served straight from Postgres — no OFFSET scans, no cache/DB merge.

Tech stack

Go 1.26 · gin (HTTP) · pgx/v5 (Postgres) · go-redis/v9 (Redis Cluster) · Server-Sent Events · Docker Compose · PostgreSQL 18 · Redis 7.4

Architecture at a glance

 POST /comments ─▶ Command Service (writer, :8081) ─▶ Postgres  (durable source of truth)
                                   │ publish AFTER commit
                                   ▼
                               Redis Cluster  ── cache: newest-N per video  +  live channel per video
                                   │ each edge subscribes ONLY to the videos its viewers watch
        ┌──────────────────────────┼──────────────────────────┐
     Edge A (SSE)               Edge B (SSE)      ...        Edge N (SSE)   (reader, :8080)
        │ SSE                       │ SSE                       │ SSE
     viewers                     viewers                     viewers

See DESIGN.md §3–§7 for the full diagram and rationale.

Repository map

fb-live-comments/
├── DESIGN.md            # full high-level design (start here for the "why")
├── README.md            # this file (the "what" + how to run)
├── writer/main.go       # Command/write service  :8081  — POST /comments
├── reader/main.go       # Read service           :8080  — GET / , /history , /stream (SSE)
├── web/index.html       # single-file frontend, served by the reader
├── internal/lib/        # shared wiring
│   ├── comment.go       #   comment model
│   ├── postgres.go      #   Postgres source-of-truth + keyset paging
│   └── redis.go         #   Redis Cluster cache + sharded pub/sub
├── docker-compose.yaml  # Redis Cluster: 3 masters, sharded pub/sub
└── infra/               # Postgres: 1 primary + 20 replicas with staged apply-delay
    ├── docker-compose.yml   #   (a separate replication-lag experiment)
    ├── generate.sh          #   regenerates the 20-replica compose file
    └── init-primary.sh      #   replicator role + writes table + pg_hba rules

Run it locally

1. Redis Cluster

docker compose up -d            # 3 masters on :7001-:7003; waits until healthy

2. Postgres

The app needs one Postgres with database app. DSN comes from PG_DSN (default postgres://postgres:pw@127.0.0.1:5432/app).

docker compose -f infra/docker-compose.yml up -d primary   # only the primary; skip the 20 replicas

If host port 5432 is in use, run Postgres on another port and point PG_DSN at it:

docker run -d --name fb-pg -e POSTGRES_PASSWORD=pw -e POSTGRES_DB=app -p 5442:5432 postgres:18
export PG_DSN="postgres://postgres:pw@127.0.0.1:5442/app"

3. Services (two terminals, from the repo root)

go run ./writer        # :8081
go run ./reader        # :8080

4. Use it

Open http://localhost:8080 in two tabs, enter a different email per tab (identity is per-tab via sessionStorage), pick a video id (or click game-1 / concert / cooking) → Join / Go Live. Comment in one tab; it appears live in the other. Your own messages show instantly (read-your-own-write); refresh keeps your user + video and reloads history.

Configuration

Env	Default	Used by
PG_DSN	postgres://postgres:pw@127.0.0.1:5432/app	writer, reader
REDIS_ADDRS	127.0.0.1:7001,127.0.0.1:7002,127.0.0.1:7003	writer, reader

pw is a throwaway local-dev password; there are no real credentials in this repo.

Intentionally out of scope

Kept simple to keep the core ideas legible: multiple reader instances, the dedicated edge/fan-out tier for hot videos, consistent hashing, and outbox/CDC for the dual write. See DESIGN.md §9–§10 for exactly where each one slots in.

---

Keywords: system design · distributed systems · scalability · real-time · live comments · live chat · fan-out · pub/sub · Redis Cluster · sharding · Server-Sent Events (SSE) · read-your-own-writes · keyset pagination · Postgres · Go (Golang) · backend engineering · system design interview.

License

MIT — learn from it, fork it, build on it.