feat: optional read replica, and one query to identify the caller

.env.example

@@ -28,6 +28,20 @@ ICHAT_DB_URL=jdbc:postgresql://localhost:5432/intellistream_chat
 ICHAT_DB_USERNAME=ichat_role
 ICHAT_DB_PASSWORD=CHANGE-ME
 
+# Optional read replica. Off by default, and off means there is no second pool at all. Enabled,
+# every @Transactional(readOnly = true) is served from the replica, while writes, Flyway and raw
+# JDBC access stay on the primary above.
+#
+# A replica lags, so a read-only transaction may see a slightly older world than the write that
+# just returned. That is fine for history and search, and it is why the Lucene reconcile sweeps
+# stay on the primary. Blank username/password inherit the primary's, which is what a streaming
+# replica normally wants. Size the read pool with SPRING_-style Hikari keys under
+# ichat.datasource.replica.hikari.*; the shipped config sets none, so it is Boot's default of 10.
+# ICHAT_DB_REPLICA_ENABLED=false
+# ICHAT_DB_REPLICA_URL=jdbc:postgresql://replica.internal:5432/intellistream_chat
+# ICHAT_DB_REPLICA_USERNAME=
+# ICHAT_DB_REPLICA_PASSWORD=
+
 # --- Keycloak / OIDC ---
 KEYCLOAK_ISSUER_URI=https://auth.your-domain/realms/ichat-realm
 KEYCLOAK_CLIENT_ID=ichat-client
@@ -239,10 +253,12 @@ ICHAT_ASSETS_UNBUNDLED=false
 # Optional: Vault / OpenBao
 # =============================================================================================
 
-# Credentials only. The processor reads secret/data/${path} and maps exactly five keys —
-# db.username, db.password, keycloak.client-id, keycloak.client-secret, keycloak.issuer-uri —
-# into their Spring property names before anything consumes them. Any other key in the record is
-# ignored on purpose, so tuning stays here in the env file.
+# Connection details and credentials. The processor reads secret/data/${path} and maps exactly ten
+# keys — db.url, db.username, db.password, db.replica-enabled, db.replica-url,
+# db.replica-username, db.replica-password, keycloak.client-id, keycloak.client-secret,
+# keycloak.issuer-uri — into their Spring property names before anything consumes them. Any other
+# key in the record is ignored on purpose, so tuning stays here in the env file. Every one of them
+# is optional: a record holding only db.password overrides only that.
 # ICHAT_VAULT_ENABLED=false
 # ICHAT_VAULT_URI=
 # ICHAT_VAULT_TOKEN=

AGENTS.md

@@ -58,6 +58,7 @@ src/main/java/ai/intellistream/chat/
 │                  CallService, TurnCredentialService, CallScheduler, CallSessionListener
 ├── config/        SecurityConfig (two filter chains), WebSocketConfig,
 │                  StompAuthorizationConfig, MultipartConfig, VaultEnvironmentPostProcessor,
+│                  ReadReplicaDataSourceConfig / ReadReplicaProperties (optional replica),
 │                  RegistrationAuthorizationRequestResolver
 ├── domain/        JPA entities — User, Channel, Message, Conversation,
 │                  Attachment / ConversationAttachment, MessageReaction / ConversationReaction,
@@ -108,6 +109,8 @@ Several autoconfigurations that lived inside `spring-boot-autoconfigure` in 3.x
   - A client subscribes once per **joined channel**, not once per rendered sidebar row, so notification coverage cannot narrow when the sidebar's rendering changes. The per-session SUBSCRIBE budget is 2000/min because over-budget frames are *dropped, not refused* — at the old 200 the tail of a large account's own subscriptions vanished with no error anywhere.
   - **Authorisation happens once, at SUBSCRIBE.** The broker never re-checks, so revoking access needs the subscription taken away too: `ChannelSubscriptionRevoker` (channels) and its conversation sibling do that on leave, kick, and channel delete. Evicting the access cache alone only stops the *next* subscribe.
 - **`CurrentUser`** is the single bridge between Spring Security principals and the domain `User`. It provisions/upserts a `User` row from the OIDC subject the first time it sees a principal. Always go through it; don't read JWT/OidcUser claims in controllers.
+  **It resolves in two steps, and the order matters.** `UserService.findUnchanged` is a read-only single-`select` lookup that answers only when the row already agrees with the token on *every* field the write path sets — handle, email, display name, admin. Anything else returns empty and falls through to `UserService.upsert`, which is unchanged and still the only thing that writes. This runs on every authenticated request, so the difference is the whole per-request database cost of being logged in: one replica-eligible read instead of two primary reads inside a writable transaction. Both halves read their claims through `UserService.claimsOf`, one shared `ClaimView` — a second copy of the claim-reading logic would let the two disagree about whether anything changed, which is a write on every request and no error anywhere. The fall-through is stable, not repeating: `upsert` settles the row into exactly the shape `findUnchanged` tests for. `UserProvisioningFastPathTest` pins the per-field cases, including that a revoked `admin` claim must not be served from the existing row.
+  One consequence worth knowing: with a read replica configured, the suspension backstop in `CurrentUser.resolve` reads a row that may be up to the replication lag old. A ban issued through `BanService` is unaffected — it updates `SuspensionRegistry` before writing the row, and the filter refuses the request before it reaches the backstop. What lags is what the registry cannot see: a `suspended_at` edited directly in psql, or a ban issued by another node.
 - **Channel types.** `PUBLIC` channels are joinable by anyone via `ChannelService.join` **unless archived**. `PRIVATE` channels require `ChannelService.invite` by an admin. The creator becomes the first `ADMIN` member automatically, and `join` promotes the first person into an *empty* channel — otherwise a channel everyone left could never have an admin again. Members can `leave`; the last admin leaving hands the role to the longest-standing remaining member rather than being refused.
 - **Slug rule.** `Channel.slug` is generated from the name in `ChannelService.create` **and `ChannelService.rename`**: lowercased, non-alphanumerics collapsed to `-`, trimmed to 80 chars. A name with no alphanumerics is rejected. Renaming moves the slug, which is safe because nothing user-facing resolves a channel by slug — pages are `/channels/{id}` and every API route is id-keyed.
 - **UI icons come from the SVG sprite, never from emoji.** `templates/fragments/icon-sprite.html` holds every symbol; use it as `<svg class="icon"><use href="#icon-name"/></svg>` (`.icon` = 20px buttons, `.icon-sm` = 14px inline markers), and add a new 24×24 symbol rather than reaching for a glyph. Emoji as icons look wrong for three reasons: they render in the font's own colours so they ignore the theme and can't be dimmed or turned red for a destructive action, they're drawn differently on every platform, and they vanish entirely on hosts with no emoji font. Real emoji stay emoji — reactions, the picker, and custom status are content. Letterforms (`B`, `I`, `S`, `{ }`) in the composer toolbar are labels, not icons; leave them. Note that a message's action row has **one builder and several readers**: `attachActions` in `js/chat/index.js` builds it for channel messages (including server-rendered ones — `channels.html` renders no action row at all) and `js/conversation.js` does the same for DMs. The desktop `⋯` overflow menu and the mobile long-press sheet in `chat-kit.js` both *derive* from the buttons that builder produced, reading their `data-action` and `title`. Add an action in the builder and the other two inherit it; add a second builder and they won't.
@@ -127,6 +130,9 @@ Several autoconfigurations that lived inside `spring-boot-autoconfigure` in 3.x
 - **Sidebar.** `SidebarService.joinedFor(user)` returns a `SidebarView` of **every channel the user is a member of** — not a ranked shortlist — alphabetical (case-insensitive, ties by id), excluding archived ones, rendered as Favourites then Channels. Entries carry `joined` and `favourite`; there is no `admin` flag, because being a channel admin is not something you need in a list you scan fifty times a day (the members panel and the cog show it). The markup is `templates/fragments/sidebar.html`, one copy shared by `channels.html` *and* `conversation.html`, in the **left** `<aside class="sidebar">`.
 - **WebSocket destinations.** Send to `/app/channels/{id}/send`, subscribe to `/topic/channels/{id}` (plus `/typing`); conversations mirror it at `/app/conversations/{id}/…` and `/topic/conversations/{id}`. The server persists, renders Markdown, then broadcasts the `MessageDto`. Per-user queues carry what only the sender or recipient should see: `/user/queue/notices` (slash-command replies and refusals), `/user/queue/conversation-alerts` (DM notifications). A thread reply rides the normal channel topic with `threadParticipants` on the DTO — deliberately *not* routed to the mention bell, which is for things addressed to you by name.
 - **Open-in-view is off** (`spring.jpa.open-in-view=false`). Touch lazy associations inside `@Transactional` boundaries on the service, never in the controller.
+- **An optional read replica turns `readOnly = true` into a routing decision.** `ichat.datasource.replica.enabled` (off by default, and off means the beans don't exist — Boot's ordinary single pool) adds a second Hikari pool in `ReadReplicaDataSourceConfig`, behind a `LazyConnectionDataSourceProxy` whose read-only variant is the replica. `@Transactional(readOnly = true)` then lands on the replica; writes, Flyway, and raw `DataSource`/`JdbcTemplate` access land on the primary. Note that a **Spring Data repository call from a non-transactional caller is a read-only transaction** — `SimpleJpaRepository` is annotated `@Transactional(readOnly = true)` at class level — so the controllers that hold a repository directly route to the replica too; only the raw-JDBC path opts out by never having a transaction at all. The laziness is the mechanism, not a tuning choice: Hibernate checks a connection out when the transaction begins, which is before the read-only flag exists, so a pool wired straight to the EntityManagerFactory would have taken a primary connection before anyone could route it. Two things follow.
+  - `readOnly = true` now means "this may read a slightly stale world", so it is **wrong for a read that decides to delete something** or that must see a write this process just made. The Lucene↔Postgres reconciles in `CleanupTasks` and `LuceneBootstrap` are the cases here, and all three carry a comment saying why they use a plain `@Transactional` — putting `readOnly = true` back turns replica lag into deleted search documents. Adding it to a service method that only reads is exactly right and needs no thought.
+  - The replica pool is marked read-only at the driver rather than per transaction, because `LazyConnectionDataSourceProxy` suppresses the per-transaction `setReadOnly` once a dedicated read-only DataSource is configured. Flyway is pinned to the writer with `@FlywayDataSource` rather than left to follow `@Primary`. Both are asserted in `ReadReplicaDataSourceConfigTest`; `ReadReplicaRoutingIT` asserts the routing itself against a real Postgres, since every link in the chain is framework internals a Boot upgrade could move.
 - **The STOMP channel executors are load-bearing.** `WebSocketConfig` sets them *unconditionally* via `registration.executor(...)`, with sizes constructor-injected (not `@Value` fields — those can be unset when the configurer callback runs). A missing inbound executor silently lands every `@MessageMapping` on the single-threaded heartbeat scheduler, which caps the whole server at one message in flight. `StompChannelDiagnostics` logs the resolved executors at startup; check that line before trusting any throughput number. See `scalability.md`.
 - **The message send path is the hot path** and is deliberately query-free: the domain `User` comes from the STOMP session (cached at CONNECT by `StompAuthorizationConfig`), the channel and write-access decision from `ChannelAccessCache`, and mentions from what `syncMentions` already resolved. A body with no `@` never reaches mention resolution at all. The exception is a **broadcast** handle (`@channel`/`@here`/`@everyone`), which costs one member fetch plus a batched INSERT per 500 recipients — ordinary messages are untouched. If you add work to `ChatWebSocketController.send` or `MessageService.postWithMentions`, check `benchmark/write-stages.sh` afterwards.
 - **`ChannelAccessCache` caches only *positive* access decisions, and everything that can turn one false has to say so.** Membership used to be add-only; it isn't — `ChannelService.leave` and `removeMember` exist, and both call `evictMember`. A channel's own state changes too: rename, archive and unarchive all call `evictChannel`, and archiving matters most because the archived check lives inside `requireWriteAccess` **before** the cached short-circuit — behind it, the members with the warmest entries (the ones who post most) would keep posting into an archived channel for the whole TTL.

CHANGELOG.md

@@ -108,7 +108,19 @@ against anything: there is no released version for something to have been fixed
 - Runs a workspace of a thousand people in **under a gigabyte of memory on a single core**.
 - **Flyway migrations**, **health endpoints**, and configuration through environment variables —
   every tunable listed in `.env.example` — with an optional **Vault/OpenBao** backend for the
-  five credentials.
+  database and Keycloak credentials.
+- **An optional read replica.** Point `ICHAT_DB_REPLICA_URL` at a standby and every
+  `@Transactional(readOnly = true)` is served from it, while writes and migrations stay on the
+  primary — 90–95% of queries on a read-heavy mix, and all of them on an endpoint that only reads.
+  Off by default, and off means the second pool does not exist rather than sitting idle — a
+  deployment that never asks for one gains no proxy and no new way to fail. The replica can be
+  configured entirely from Vault alongside the primary's credentials.
+- **Identifying the caller costs one read instead of three.** Resolving a logged-in principal used
+  to run two queries inside a writable transaction on every single request, to re-derive a row that
+  had not changed since the request before. It now takes a read-only single-`select` fast path and
+  only falls back to the full upsert when the token actually disagrees with the stored row. This is
+  a saving on its own and the thing that makes the replica worthwhile, since the old cost fell on
+  every request regardless of how little work it did.
 - **An AlmaLinux installer** and a separate **SELinux hardening script**, both verified end to
   end on AlmaLinux 10.2 with SELinux enforcing.
 - **Container quick start** with `podman compose up -d`.

scalability.md

@@ -324,6 +324,40 @@ pool of 10 — worth knowing before you copy a number from here into production.
 acquire time at 13,000 posts/second is 0.03 ms with zero pending — the pool is not a constraint at
 this rate, and raising it would not help. Postgres itself sat at ~10% CPU.
 
+### Read replica
+
+`ichat.datasource.replica.enabled=true` plus a URL adds a second Hikari pool against a standby, and
+every `@Transactional(readOnly = true)` is served from it. Writes, Flyway and raw JDBC access stay
+on the primary. Sized separately through `ichat.datasource.replica.hikari.*`.
+
+Most of the read volume moves. A `GET /channels/{id}` runs about fourteen read-only transactions
+(the `CurrentUser` lookup, sidebar, conversations, the message page and its attachments, reactions,
+reply counts and polls, membership, notification level) against a single primary read — the one
+inside `markRead`'s write. An endpoint that only reads puts nothing on the primary at all. Call it
+**90–95% of `SELECT`s on a read-heavy mix**, and 100% for the pure-read endpoints.
+
+That depends on `CurrentUser` resolving a settled principal through `UserService.findUnchanged`
+rather than `upsert` — see the `CurrentUser` bullet in AGENTS.md. Before that fast path existed,
+every authenticated request spent two `SELECT`s inside a writable transaction just to identify the
+caller, which pinned the figure nearer 80% and made small API calls a roughly even split. It is
+the single change that decides whether a replica is worth having: it is a fixed per-request cost,
+so it dominates exactly the light endpoints a replica is otherwise best at absorbing.
+
+WebSocket sends contribute almost no read volume either way — the send path is deliberately
+query-free, and the user is resolved once at CONNECT rather than per frame. Both pools are named
+(`ichat-writer`, `ichat-reader`), so measure the real split per pool rather than trusting this
+paragraph.
+
+Note what the numbers above say about when this is worth doing: at 17,000 messages/second Postgres
+is at ~10% CPU, so the write path is nowhere near needing relief. **The case for a replica here is
+read volume and availability, not write throughput** — a workspace whose history, search hydration
+and sidebar reads dominate the query mix, or a deployment that wants the primary reserved for
+writes. Adding one to chase message throughput will not move the number this document reports.
+
+The cost is that a replica lags, so a read-only transaction may see a slightly older world than the
+write that just returned. Reads that decide to *delete* something must therefore stay on the
+primary; the Lucene reconcile sweeps do, deliberately. See the read-replica bullet in AGENTS.md.
+
 ### Kernel
 
 The stock kernel is provisioned for a workstation, not 10⁵ sockets. Persist these in

src/main/java/ai/intellistream/chat/cleanup/CleanupTasks.java

@@ -142,7 +142,13 @@ private void sweepDir(String label, Path root, Supplier<Set<String>> liveKeys) {
     /** CLEAN-3: reconcile the Lucene index with the messages table — index the missing, drop the stale. */
     @Scheduled(fixedDelayString = "${ichat.cleanup.reconcile-ms:3600000}",
                initialDelayString = "${ichat.cleanup.initial-delay-ms:300000}")
-    @Transactional(readOnly = true)
+    // Deliberately NOT readOnly = true. This method reads nothing but it deletes on what it read,
+    // and readOnly is what routes a transaction to the read replica when one is configured
+    // (ReadReplicaDataSourceConfig). A replica lags; a message committed a moment ago is in the
+    // index and not yet in the replica's copy of `messages`, which reads here as "stale" — and
+    // the sweep would delete the search document for a message that exists. The read ordering
+    // below (N3) closes the same window for the primary; only a primary read closes it at all.
+    @Transactional
     public void reconcileSearchIndex() {
         if (!props.isEnabled()) return;
         // Snapshot the INDEX first, then the DB (N3). A message posted+indexed between the two reads
@@ -202,7 +208,9 @@ public void reconcileSearchIndex() {
      */
     @Scheduled(fixedDelayString = "${ichat.cleanup.reconcile-ms:3600000}",
                initialDelayString = "${ichat.cleanup.initial-delay-ms:300000}")
-    @Transactional(readOnly = true)
+    // Not readOnly = true, for the reason spelled out on reconcileSearchIndex above: this one
+    // deletes too, so it has to read the primary.
+    @Transactional
     public void reconcileConversationSearchIndex() {
         if (!props.isEnabled()) return;
         // Index first, then the DB — see reconcileSearchIndex above (N3).