NA-0642: durable store-and-forward queue — SQLite, retention TTL, delete-on-ACKNOWLEDGED-pull (D578) - #61
Merged
Merged
Conversation
… TTL, delete-on-ACKNOWLEDGED-pull (D578, D-0011) Store: rusqlite(bundled), WAL + synchronous=FULL, schema v1 (routes keyed by SHA-256 of the route token — raw tokens never persisted; messages: seq FIFO, msg_id non-unique per NA-0275, opaque BLOB body, enqueued_at, leased_until). STORE_PATH is required, fail-closed. Contract (D-0011, per the qsl-protocol D578 design-lock, option B): - legacy GET /v1/pull unchanged byte-for-byte (delete-on-deliver) — the current non-acking qsc client is not stranded - GET /v1/pull?ack=lease leases without deleting (PULL_LEASE_SECS, 60s) - POST /v1/pull/ack deletes only leased copies; idempotent; route-scoped - RETENTION_TTL_SECS (7d default) replaces the 5-min idle-route discard (ROUTE_IDLE_TTL_MS now warn-and-ignore); delivered+acked still forgotten Tests: hard-kill (SIGKILL) restart durability + crash-between-pull-and-ack lease survival; retention expiry + non-vacuity controls; ack semantics; legacy-shape guard; concurrency no-loss/no-dup; token-at-rest privacy. Packaging: relay.env.example (new envs + MAX_QUEUE_DEPTH 256->257 fix), StateDirectory=qsl-server, runbook config section. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
qsl-protocol-governed lane NA-0642 (QSL-DIR-2026-07-13-578, operator-approved design-lock: ack model B, rusqlite+bundled, all five sub-choices as proposed). Governance closeout lands in qsl-protocol at D-1265; this PR is the CODE half. Records D-0011 in this repo's DECISIONS.md (the pull-contract decision, per the D-0009/D-0010 surface).
What changes
src/store.rs): embedded SQLite (rusqlitebundled),journal_mode=WAL+synchronous=FULL(a 200 on push = fsynced), schema v1 with fail-closed version check. Route tokens are persisted only as SHA-256 digests; payloads stay opaque verbatim BLOBs;msg_idis deliberately NOT unique (NA-0275 duplicate-push contract).GET /v1/pull?ack=leasereturns without deleting and leases forPULL_LEASE_SECS(default 60 s);POST /v1/pull/ack {"ids":[...]}deletes only leased copies, idempotent, route-scoped; un-acked leases expire and messages reappear. LegacyGET /v1/pullis byte-identical — delete-on-deliver, same JSON shape (guarded by an exact-field-set test) — the current non-acking qsc client and the NA-0640 e2e are unaffected.RETENTION_TTL_SECS(default 604800 = 7 d, ceiling 30 d) governs undelivered-message lifetime, swept lazily on push/pull + a 60 s background task.ROUTE_IDLE_TTL_MSis retired warn-and-ignore (a stale relay.env cannot brick a restart). Delivered+acked messages are still forgotten — reliable, not an archive.STORE_PATHREQUIRED (no default;:memory:accepted as a deliberate ephemeral choice;ERR_INVALID_CONFIG_STORE_PATHon missing/unopenable). Rate buckets stay in-memory (abuse control, resets on restart by design); drain-release parity for route slots and buckets preserved.StateDirectory=qsl-serveradded to the unit — flagged scope judgment: required so the packagedProtectSystem=strictunit can write/var/lib/qsl-server; runbook §3 updated.Test evidence (cargo test: 100 passed, 0 failed, 25 binaries; fmt + clippy -D warnings clean; both CI guard scripts pass)
na0642_durability_restart.rs, real binary,Child::kill= SIGKILL, no graceful shutdown):pushed_message_survives_hard_kill_and_restart(32 KiB payload byte-identical after SIGKILL between push-200 and restart — proves the synchronous=FULL fsync, not a graceful flush);leased_message_survives_hard_kill_and_reappears_after_lease_expiry(SIGKILL mid-lease; lease survives restart, message invisible while live, reappears byte-identical after expiry, ack then deletes);legacy_delivery_does_not_resurrect_after_restart(delivered messages stay forgotten).ERR_BAD_ACK_MODE/_BODY/_IDS); route-scoped ack.event=retention_expired, no token/payload leak).config_semanticszero-env start → now fail-closed (STORE_PATH required);route_lifecycle_ttl*idle-discard tests retired → drain-release contracts carried intona0642_retention_lifecycle.rs;na0347retention-purge block moved to the retention mechanism.Not in this PR (recorded as owed in qsl-protocol)
qsc ack-client lane; qsc dev-dep pin bump + NA-0640 e2e re-run; the filed hardening debt (a)–(f) incl. README/DOC-SRV-003 idle-TTL text now behind the code; ENG-0014 bearer compare untouched.
🤖 Generated with Claude Code