Skip to content

Remember where each person has read up to in a conversation - #346

Merged
grytchat merged 6 commits into
mainfrom
claude/read-state-server
Oct 27, 2026
Merged

grytchat merged 6 commits into
mainfrom
claude/read-state-server

Conversation

@sivert-io

Copy link
Copy Markdown
Member

Finding 2 from the beta.12 testing. Signing in on a new machine showed no unread anywhere except announcements — and that one survived only because it was a mention, and mentions already live in the database (GRYT-747).

This is the server half. The client half is useless without it and comes next; I would not merge this expecting a visible change.

The finding was wrong about the starting point, in a way that matters

It said the client "emits messages:read" and that "the client half already exists".

messages:read is a plugin capability name — src/plugins/api.ts maps "message:created" to it. It is not a socket event and nothing emits it.

And there is no read position anywhere, on either side. useUnreadTracker.ts in the client says so in its own first comment:

Counted from when this window connected, not from a server-side marker.

It is an in-memory Map. Not localStorage, not a file. So unread also resets on every restart, not only on a new machine — the same missing piece, and probably the more annoying half.

What this adds

One table, one row per person per conversation they have opened, updated in place.

CREATE TABLE IF NOT EXISTS read_markers (
  server_user_id TEXT NOT NULL,
  conversation_id TEXT NOT NULL,
  read_through_at TEXT NOT NULL,
  read_through_message_id TEXT NOT NULL,
  PRIMARY KEY (server_user_id, conversation_id)
);

createSchema runs on every start with CREATE TABLE IF NOT EXISTS, so an existing database picks this up on the next boot. No migration step, nothing to back out.

Two handlers, read:list and read:set, following mentions: same rate-limit shape, and access re-checked on both read and write.

The decisions worth your attention

The marker is the message's timestamp, not the time of the read. Deliberately. Whoever runs the server learns which message you reached — something everyone in that channel can already see — and not when you were reading it. It costs nothing, because the counting query needs the position anyway. If you would rather it recorded read times for some other feature later, that is a decision to make on purpose rather than inherit from me.

Channels only. Read state on a sealed or MLS conversation is a read receipt on an encrypted message, which is a different promise from an unread badge. Nothing here touches DMs.

No foreign key to messages, unlike mentions. A mention row is its message, so a cascade is right there. A marker is a position, and deleting the message somebody stopped at must not forget where they were. There is a test for exactly this.

Every comparison is on the (created_at, message_id) pair. created_at is ISO to the millisecond, so two messages in one conversation can share one, and message ids are randomUUID() with no order of their own. On created_at alone a busy channel loses or repeats a message. There is a test that inserts three messages in the same millisecond.

The marker only moves forwards. Two devices reading the same conversation race, and the one that finishes last is not the one that read furthest. ON CONFLICT … DO UPDATE … WHERE rather than a plain upsert.

A conversation with no marker is left out of the counts and the client reads absent as caught up — joining a server should not light up every channel in it. That is a product choice as much as a technical one, and the alternative is defensible.

One thing your own test caught

mergeGuest.test.ts walks every table for a column that can hold a server user id and fails on any the merge does not account for. It caught read_markers.server_user_id and told me what to do about it:

AssertionError: add these to AUTHOR_COLUMNS, or handle them in mergeGuestIntoAccount
+ [ 'read_markers.server_user_id' ]

That is a good guard and it earned its keep here. The two tables beside it keep the account's row where both hold one; read markers keep the further of the two instead, since merging two of your own identities should not lose read progress. Covered both ways round, plus a marker only the guest held.

Testing

17 tests: 10 on the db module, 6 over the socket, 1 in mergeGuest.test.ts. Both access cases are covered — a marker cannot be written for a conversation you cannot reach, and one left behind by a deleted channel is not handed back on read.

npx tsc --noEmit clean. eslint --flag unstable_native_nodejs_ts_config clean on the new files. check-comment-length.mts clean.

One thing you should know about running the suite on Windows: most of this repo's db tests fail their after() hook with EPERM on rmSync, because there is no close for the SQLite connection and Windows will not unlink an open file. That is pre-existing and invisible on CI, which is ubuntu — the same shape as the client issue in Gryt-chat/client#1094. My two test files wrap their own cleanup in a try/catch, matching what conversationRefusalEvent.test.ts already does, so they report their results rather than their tidying.

What to look at

  • The schema, since it is the part that is awkward to change later. Particularly the missing foreign key and the composite comparison — both are deliberate and both have a test, but both are the kind of thing that looks like an oversight.
  • Whether read:set should reply with the whole state. It does, so two devices on one account converge without either guessing what the other did. The cost is a count query per channel opened, rate-limited to 60 a minute. A quieter design would acknowledge and let the client apply its own change optimistically.
  • The count query's plan. It joins read_markers to messages and leans on idx_messages_conv_time, which already exists. I have not looked at EXPLAIN QUERY PLAN on a database with real volume, and that is the one thing here that could behave differently at scale.

Not done here

The client half: switching the tracker from "count since connect" to "count after marker", sending read:set, and consuming the replay. check-unread-tracker in the client will need rewriting, since what it tests is the model this replaces.

🤖 Generated with Claude Code

sivert-io and others added 5 commits October 27, 2026 17:20
Finding 2 from the beta.12 testing: signing in on a new machine showed no unread
anywhere except announcements, which survived only because it was a mention and
mentions already live in the database.

There was no read position anywhere, on either side. The client's tracker says so
itself -- "counted from when this window connected, not from a server-side
marker" -- and it is an in-memory Map, so unread also resets on every restart,
not only on a new machine. Nothing emitted anything either: `messages:read`
exists in both repos only as a plugin capability name.

This is the server half. read_markers holds one row per person per conversation
they have opened, updated in place.

The marker is the message's own created_at and id, never the time anybody
looked. Whoever runs the server learns which message you reached, which everyone
in the channel can already see, and not when you were reading it. That was a
deliberate choice over a read timestamp, and it costs nothing: the id and the
timestamp together are what the counting query needs anyway.

Notes on the shape, since the schema is the part that is awkward to change:

- No foreign key to messages, unlike mentions. A mention row *is* its message
  and should go when the message does. A marker is a position, and deleting the
  message somebody stopped at must not forget where they were.
- created_at is ISO to the millisecond, so two messages in one conversation can
  share one. Every comparison is on the (created_at, message_id) pair, because
  message ids are random UUIDs and carry no order of their own.
- The marker only ever moves forwards. Two devices reading the same conversation
  race, and the one that finishes last is not the one that read furthest.
- A conversation with no marker is left out of the counts, and the client reads
  absent as caught up: joining a server should not light up every channel in it.
- Thread replies are left to the thread's own count, and your own messages are
  not news to you.

Handlers follow mentions: same rate-limit shape, and access re-checked on both
read and write, so a marker cannot say whether a guessed conversation exists and
one left behind by a deleted channel is not handed back.

Channels only. Read state on a sealed or MLS conversation is a read receipt on
an encrypted message, which is a different promise from an unread badge.

16 tests, 10 on the db and 6 over the socket, including both access cases. The
client half is still to come and is useless without this.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
mergeGuest.test.ts caught this: it walks every table for a column that can hold a
server user id and fails on any that the merge does not account for. read_markers
was one, and the message it prints says what to do about it.

The two tables next to it keep the account's row where both hold one. Read markers
keep the further of the two instead: merging two of your own identities should not
lose read progress. Guest rows behind the account's go first, so whatever is left
is strictly ahead and the account's row gives way to it.

Added to the handled-apart list in that test, and a case covering it both ways
round, plus a marker only the guest held.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CI's lint failed on five `unknown`s in src/socket/handlers/readMarkers.test.ts:
the fake socket's emit, what it records, and the HandlerContext cast, all copied
from conversationRefusalEvent.test.ts.

eslint.config.mts carries a per-file list for this, commented "tests build loose
payloads on purpose", and every other handler test in that directory is on it.
Added there rather than typed around, since the payloads are deliberately loose.

Mine passed locally because I linted a file list and left that one file out of
it. CI runs `eslint .`, so that is what I should have run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Mark as read in the client has a conversation and no message id, since
the channel may not be open. With none named, the marker moves to the
newest message there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
grytchat added a commit to Gryt-chat/client that referenced this pull request Oct 27, 2026
Pairs with Gryt-chat/server#346. read:set goes out when a channel is
opened, when a message lands in the open one, and from Mark as read.
Without a message id the server reads up to its newest message, so
opening a channel and marking it read need no id. It ignores a DM.

The server's counts replace this window's only for conversations it
has a marker for. A channel never opened keeps what the window counted,
where before the reply wiped it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
grytchat added a commit to Gryt-chat/client that referenced this pull request Oct 27, 2026
* Client half of server-backed read state, unfinished

Pairs with Gryt-chat/server#346. Pushed as a branch rather than a PR because it
is incomplete, and handed over rather than carried on.

Done:

- useUnreadTracker gains setServerUnread(host, counts), which replaces what the
  window had counted. A conversation the server leaves out has no marker and
  reads as caught up, not as everything unread. The doc comment no longer says
  counting starts when the window connected, because it no longer does.
- registerServerSocketEvents asks for read:list on the same two moments it asks
  for mentions -- a reload and a rejoin -- and applies the reply.

Not done:

- Nothing emits read:set yet, so nothing is ever recorded as read. The site is
  serverView's channel-opened effect, which calls markChannelRead; the newest
  message id has to come from useChat, which is destructured below it, so the
  emit needs its own effect further down rather than a line in that one.
- check-unread-tracker still tests the old model and will need rewriting.
- Mark as read, in MarkAsReadItem.tsx, has no message id to send. Either the
  server accepts read:set without one as "everything here", or the client finds
  the newest id for a conversation it may not have open.
- Channels only, by the same argument as the server side: a DM's marker would be
  a read receipt on an encrypted message.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat: tell the server what you've read, so unread survives a restart

Pairs with Gryt-chat/server#346. read:set goes out when a channel is
opened, when a message lands in the open one, and from Mark as read.
Without a message id the server reads up to its newest message, so
opening a channel and marking it read need no id. It ignores a DM.

The server's counts replace this window's only for conversations it
has a marker for. A channel never opened keeps what the window counted,
where before the reply wiped it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Sivert Gullberg Hansen <sivertgullberg@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Sivert <316918682+grytchat@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@grytchat
grytchat merged commit 03e271b into main Oct 27, 2026
5 checks passed
@grytchat
grytchat deleted the claude/read-state-server branch October 27, 2026 16:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants