Skip to content

Latest commit

 

History

History
116 lines (92 loc) · 5.65 KB

File metadata and controls

116 lines (92 loc) · 5.65 KB

BoltQ Documentation

BoltQ is two systems that share a process.

A work queue and pub/sub broker — the original BoltQ. Trusted backend clients connect over a binary TCP protocol, push jobs, consume them, ack them. Raft replicates the queue state.

A messaging backbone — a partitioned, replayable log with per-user authorisation and a WebSocket edge, built for end-user devices: chat, direct messages, presence, typing indicators, offline push.

The two are independent. You can run either alone or both together; enabling one does not change the behaviour of the other. The messaging subsystem is off by default.

Running the messaging subsystem in production? Start with STATUS.md — the honest inventory of what is verified, which gaps cost data today, and which design decisions are still open.


Part 1 — Queue and pub/sub

  1. Getting Started — Quick start guide, installation, first message
  2. Architecture — System design, components, data flow, concurrency model
  3. API Reference — Full HTTP REST API documentation
  4. Configuration — Config file, environment variables, tuning guide
  5. Persistence & WAL — Write-Ahead Log, disk mode, recovery process
  6. Monitoring — Prometheus metrics, Grafana dashboards, alerting
  7. Go SDK — Go client library reference and examples
  8. Node.js SDK — Node.js client library reference and examples
  9. CLI Reference — Command-line tool usage
  10. Deployment — Docker, Kubernetes, systemd, Nginx, production checklist
  11. Security — Security features, API key auth, TLS encryption

Part 2 — Messaging (chat, presence, push)

Start here

If you want to… Read
Understand why this subsystem exists at all Why a log
See how a message travels from one phone to another Message lifecycle
Build a chat app on BoltQ Building a chat app
Wire up authentication Authentication and authorisation
Connect a client Gateway protocol
Run this in production Production checklist
Plan for multiple regions Global HA
Upgrade an existing queue deployment Migrating from queues

Architecture

  • Why a log — why chat needs a replayable log rather than a queue, and what that changes.
  • The stream engine — segments, sparse indexes, sequence assignment, retention, crash recovery.
  • Message lifecycle — the full path of a message, including what happens when the recipient is offline.
  • Fan-out strategies — why a 3-person chat and a 50,000-person channel are delivered differently.
  • Cursors and multi-device — how four devices belonging to one person each keep their own read position.
  • Replication — how the log is copied to other nodes, quorum acknowledgement, and manual failover.
  • Durability — exactly what survives a crash, what does not, and why the defaults are what they are.
  • Ordering guarantees — what BoltQ promises about message order, and what it does not.

Guides

Reference

Operations


A note on scope

STATUS.md is the single place that records open work, bugs already found and fixed, and undecided design questions. Keep it current — it exists so nobody has to re-derive these conclusions later.

The Part 2 documents try to be honest about limits. Several things a global-scale messaging system eventually needs are not implemented — automatic leader election and failover, resharding, tiered storage to object storage, and follower reads. Where that is the case the documents say so plainly and describe what you would have to build or buy instead, rather than implying coverage that does not exist. See Global HA for the full list.