NA-0642: durable store-and-forward queue — SQLite, retention TTL, delete-on-ACKNOWLEDGED-pull (D578)

Cargo.toml

@@ -9,6 +9,8 @@ tokio = { version = "1", features = ["rt-multi-thread", "macros"] }
 serde = { version = "1", features = ["derive"] }
 serde_json = "1"
 uuid = { version = "1", features = ["v4"] }
+rusqlite = { version = "0.32", features = ["bundled"] }
+sha2 = "0.10"
 tower-http = { version = "0.5", features = ["trace"] }
 tracing = "0.1"
 tracing-subscriber = { version = "0.3", features = ["env-filter"] }

DECISIONS.md

@@ -70,3 +70,11 @@
   - **Decision:** qsl-server now retires legacy `/v1/push/:channel` and `/v1/pull/:channel?max=N` outright. Canonical header-carried routing on token-free `/v1/push` and `/v1/pull?max=N` is the only supported route-token ingress shape.
   - **Rationale:** The compatibility window from D-0009 existed only to get supported clients and operator guidance onto the safer header-based posture. That migration is now complete enough that continuing to accept URI-carried route tokens leaves a known passive-leak surface live without adding truthful transport value.
   - **References:** NA-0012; `src/lib.rs`; `tests/relay_smoke.rs`; `README.md`; `docs/server/DOC-SRV-003_Relay_Inbox_Contract_v1.0.0_DRAFT.md`; `packaging/runbook_ubuntu.md`; `scripts/check_relay_compatibility.sh`; `scripts/verify_remote.sh`; `scripts/aws_update_and_verify.sh`; `scripts/ci/test_relay_deploy_compatibility_guard.sh`; `TRACEABILITY.md`
+
+- **ID:** D-0011
+  - **Status:** Accepted
+  - **Date:** 2026-07-13
+  - **Goals:** G4, G5
+  - **Decision:** qsl-server's store-and-forward queue becomes DURABLE (embedded SQLite via rusqlite/bundled, WAL + synchronous=FULL, single-file store at the required `STORE_PATH`; route tokens persisted only as SHA-256 digests; payloads stored verbatim as opaque blobs), and the delivery contract adds an ACKNOWLEDGED-PULL mode per the qsl-protocol D578 design-lock (option B, operator-chosen): `GET /v1/pull?ack=lease` returns messages WITHOUT deleting and leases them for `PULL_LEASE_SECS` (default 60 s); `POST /v1/pull/ack {"ids":[...]}` deletes ONLY leased copies (idempotent; scoped to the route; unleased duplicate copies per the NA-0275 contract survive); un-acked leases expire and the messages reappear. The LEGACY pull (`GET /v1/pull?max=N`, no ack parameter) keeps its exact delete-on-deliver contract and response shape — the current non-acking qsc client is not stranded. The 5-minute idle-route discard (`ROUTE_IDLE_TTL_MS`) is RETIRED (warn-and-ignore) and replaced by an operator-tunable retention TTL for undelivered messages (`RETENTION_TTL_SECS`, default 7 days, ceiling 30 days); delivered+acked messages are still forgotten immediately — the relay is reliable, not an archive. Startup is fail-closed: `STORE_PATH` has no default.
+  - **Rationale:** The in-memory queue was demo-class: a restart dropped every queued message and idle routes discarded after 5 minutes, unacceptable for the DOC-PROG-003 self-host operator-path (Tier-1, step 1). Delete-on-pull loses a message if the puller crashes between pull and local persistence; the lease model closes that window without wire-semantic or E2EE change (payloads stay opaque, the relay stays blind, nothing precludes future E2EE read receipts as ordinary payloads). Route/pull semantics are a recorded-decision surface per D-0009/D-0010, so the contract change is recorded here; governance authority for the lane lives in qsl-protocol (NA-0642, D578, D-1265).
+  - **References:** qsl-protocol NA-0642 / QSL-DIR-2026-07-13-578 (D578) / D-1265; D-0009; D-0010; `src/store.rs`; `src/lib.rs`; `src/main.rs`; `packaging/systemd/relay.env.example`; `packaging/systemd/qsl-server.service` (StateDirectory); `packaging/runbook_ubuntu.md`; `tests/na0642_durability_restart.rs`; `tests/na0642_ack_contract.rs`; `tests/na0642_retention_lifecycle.rs`; `tests/na0642_retention_logging.rs`; `tests/na0642_backward_compat.rs`; `tests/na0642_concurrency.rs`; `tests/na0642_store_privacy.rs`

packaging/runbook_ubuntu.md

@@ -34,7 +34,15 @@ Edit `/etc/qsl-server/relay.env`:
 - `BIND_ADDR=127.0.0.1` (default safe bind)
 - `PORT=8080`
 - `RELAY_TOKEN=` (set a strong token out-of-band; do not commit tokens)
-- `MAX_BODY_BYTES`, `MAX_QUEUE_DEPTH` as needed
+- `STORE_PATH=/var/lib/qsl-server/relay.db` (REQUIRED since NA-0642 — the durable
+  SQLite queue; the packaged unit provisions `/var/lib/qsl-server` via
+  `StateDirectory=qsl-server`. This file IS the relay's data backup unit.)
+- `RETENTION_TTL_SECS=604800` (undelivered-message lifetime, default 7 days;
+  replaces the retired `ROUTE_IDLE_TTL_MS` idle discard — a leftover
+  `ROUTE_IDLE_TTL_MS` line is warned about and ignored)
+- `PULL_LEASE_SECS=60` (visibility timeout for acknowledged pulls, `?ack=lease`)
+- `MAX_BODY_BYTES`, `MAX_QUEUE_DEPTH` as needed (template default 257 — the
+  NA-0598 exact-4-MiB attachment needs 256 chunks + 1 manifest)
 
 Apply config:
 

packaging/systemd/qsl-server.service

@@ -16,6 +16,9 @@ PrivateTmp=true
 ProtectSystem=strict
 ProtectHome=true
 ReadWritePaths=/opt/qsl-server /etc/qsl-server
+# NA-0642: creates /var/lib/qsl-server (owned by the service user) and keeps
+# it writable under ProtectSystem=strict for the durable STORE_PATH.
+StateDirectory=qsl-server
 CapabilityBoundingSet=
 AmbientCapabilities=
 RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX

packaging/systemd/relay.env.example

@@ -2,7 +2,21 @@
 PORT=8080
 BIND_ADDR=127.0.0.1
 MAX_BODY_BYTES=1048576
-MAX_QUEUE_DEPTH=256
+MAX_QUEUE_DEPTH=257
+
+# Durable store-and-forward queue (REQUIRED; NA-0642). The SQLite database
+# file holding queued messages — back up this file to back up the relay.
+# ':memory:' is accepted only for deliberately ephemeral runs.
+STORE_PATH=/var/lib/qsl-server/relay.db
+
+# Undelivered-message retention in seconds (default 604800 = 7 days,
+# ceiling 2592000 = 30 days). Replaces the retired ROUTE_IDLE_TTL_MS
+# idle-route discard as the message-lifetime control.
+RETENTION_TTL_SECS=604800
+
+# Ack-mode pull lease (visibility timeout) in seconds for
+# GET /v1/pull?ack=lease (default 60, ceiling 3600).
+PULL_LEASE_SECS=60
 
 # Optional bearer token gate for /v1/push and /v1/pull.
 # Leave empty to disable auth.