Mentions show up when they arrive: mentions and responses change domains (migration 0032)

docs/d1-polling-cache-operations.md

@@ -6,6 +6,40 @@ tables. Relevant source writes and counter changes commit or roll back together.
 Missing state and unsafe counter overflow reject writes rather than silently
 lose notifications. No application mutation caller needs an invalidation hook.
 
+### Mentions and responses domains (migration 0032)
+
+`0032_mentions_change_domain.sql` adds two counters to `change_state` and six
+triggers (insert, update, delete on `mentions_in` and `mentions_out`), so the
+tracked set is now 36 `change_*` triggers over twelve tables (27 from 0024, three
+from 0026 on `read_state`, six here):
+
+| Domain | Moves when | Read by |
+|---|---|---|
+| `mentions` | any `mentions_out` change; a `mentions_in` change to a row whose status is, or was, `verified` or `gone` | Studio mentions view (both lists) and, with `reading`, each mention's source card |
+| `responses` | a `mentions_in` change to a row that is, or was, `verified` and not hidden | public HTML: the responses list under an item (`listPublicResponses`) |
+
+`responses` is in the public HTML cache key (`publicHtmlRevision`), so a newly
+verified response, a hide or a show reaches the item page at once instead of
+after the edge window (60 s plus up to 300 s stale-while-revalidate). A pending
+claim, its failure and the daily prune of failed claims move neither counter:
+anyone can POST a claim, and if claims rolled the key a stranger could empty
+the edge cache at the receiver's rate cap. They still write the state row once,
+like every tracked table, and are refused if it is missing.
+
+Apply 0032 before deploying a Worker that reads these columns: `/api/changes`
+selects every domain by name and fails without them, which stops Studio from
+loading new data, and public pages fall back to the plain cache key (the
+pre-0.46.1 60-second behaviour) until the migration lands. The first deploy
+after 0032 changes every public cache key once, so each page renders fresh on
+its next read.
+
+Studio already polled the mentions view every 15 seconds, re-reading both lists
+in full and every row's source card on each tick. Those reads are now gated on
+the counters like every other owner read, so an unchanged tick costs one
+`/api/changes` read per collection. Polls stop while the tab or installed PWA is
+hidden and run at once on `visibilitychange` back to visible and on window
+focus (`src/ui/polling.ts`), for every domain.
+
 The feed uses the existing `MEDIA` R2 binding. Objects under `__cache/feed/` are
 internal XML artifacts, separate from attachment paths. Keys include software/
 renderer version, request origin and mount. No new secret, bucket or Durable
@@ -69,7 +103,11 @@ write totals separately; these changes do not establish a free-plan capacity.
 ## Database restores
 
 Pause requests and disable cron triggers before restoring/replacing D1. Apply any missing migrations if
-the backup predates `0024_change_state.sql`. Then run:
+the backup predates `0024_change_state.sql`, and in particular `0032_mentions_change_domain.sql`:
+the reset script below names the `mentions` and `responses` columns and fails on a
+database without them. A restore brings back mention rows as they were at the backup,
+so a response verified or hidden since then is undone on the public page too; the new
+epoch makes every Studio and public cache entry re-read. Then run:
 
 ```sh
 npx wrangler d1 execute DB --remote --file scripts/reset-change-epoch.sql

e2e/polling-cache.spec.ts

@@ -75,3 +75,49 @@ test('a buffered query response keeps its pre-fetch revision and receives a late
     await expect(page.getByText(marker, { exact: false }).first()).toBeVisible();
   } finally { release(); }
 });
+
+// Delivery was never the delay: live mentions verify in seconds. What lagged
+// was every view of them. A verified mention written by the receiver (here the
+// fixture's stand-in for verifyMention) must reach an open mentions view within
+// one poll, and a view resumed from the background must check at once.
+test('a newly verified mention reaches the open mentions view in one poll, and a resumed view checks at once', async ({ page, request }) => {
+  test.setTimeout(60_000);
+  await page.clock.install();
+  await page.goto('/studio/login'); await page.locator('[name=password]').fill('test-password');
+  await page.getByRole('button', { name: 'log in', exact: true }).click(); await expect(page.locator('#composer-text')).toBeVisible();
+  const marker = `Live mention target ${crypto.randomUUID()}`;
+  const item = await page.evaluate(async content_md => {
+    const created = await (await fetch('/api/items', { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ content_md, kind: 'thread' }) })).json() as { id: string };
+    await fetch(`/api/items/${created.id}/publish`, { method: 'POST' });
+    return created.id;
+  }, marker);
+  await page.goto('/studio/mentions');
+  await expect(page.locator('.mention-group').first()).toBeVisible();
+  await page.clock.pauseAt(new Date(Date.now() + 10_000));
+  const group = page.locator('.mention-group').filter({ hasText: marker });
+  await expect(group).toHaveCount(0);
+  const id = `live${Date.now()}`;
+  expect((await request.post('http://127.0.0.1:8789/__fixture/verified-mention', { data: { id, item, origin: 'https://responder.example/' } })).status()).toBe(204);
+  await page.clock.runFor(15_000);
+  await expect(group).toContainText('1 responses');
+  await expect(group).toContainText('Live responder');
+  await expect(group.getByRole('button', { name: 'hide from page', exact: true })).toBeVisible();
+
+  // Backgrounded: no reads at all, however long it stays hidden.
+  await page.evaluate(() => {
+    Object.defineProperty(document, 'visibilityState', { configurable: true, get: () => 'hidden' });
+    document.dispatchEvent(new Event('visibilitychange'));
+  });
+  const reads: string[] = [];
+  page.on('request', r => { if (r.method() === 'GET' && new URL(r.url()).pathname.startsWith('/api/')) reads.push(new URL(r.url()).pathname); });
+  expect(await page.evaluate(async id => (await fetch(`/api/mentions/${id}`, { method: 'PATCH', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ hidden: true }) })).ok, id)).toBe(true);
+  await page.clock.runFor(60_000);
+  expect(reads).toEqual([]);
+  // Resumed: the check runs on the visibility change, not at the next tick.
+  await page.evaluate(() => {
+    Object.defineProperty(document, 'visibilityState', { configurable: true, get: () => 'visible' });
+    document.dispatchEvent(new Event('visibilitychange'));
+  });
+  await expect(group.getByRole('button', { name: 'show on page', exact: true })).toBeVisible();
+  expect(reads).toContain('/api/changes');
+});

migrations/0032_mentions_change_domain.sql

@@ -0,0 +1,85 @@
+-- Two revision domains for Webmentions, so their views stop waiting out a
+-- timer: `mentions` for the Studio (GET /mentions, both directions) and
+-- `responses` for public HTML (the responses list under an item, which reads
+-- mentions_in through listPublicResponses).
+--
+-- They are separate because their response dependencies differ, and because
+-- the public one must not move on unverified claims. Anyone can POST a claim;
+-- if every pending row rolled the public cache key, a stranger could empty the
+-- edge cache at the receiver's rate cap. So:
+--   mentions  moves on any mentions_out change, and on a mentions_in change
+--             that touches a row the Studio lists (status verified or gone,
+--             before or after).
+--   responses moves on a mentions_in change that touches a row the public list
+--             shows (status verified and not hidden, before or after): a new
+--             verification, a hide, a show, a re-claim that sends it back to
+--             pending, a withdrawal found on re-verification.
+-- A pending claim, and its failure, move neither; they still pay the one
+-- state-row write every tracked table pays, and still need the state row.
+--
+-- Same shape as 0024's guards: SELECT RAISE ... WHERE for the missing row, and
+-- every CASE closed inside parentheses, never by END and a semicolon, which
+-- D1's remote executor takes as the end of the trigger (session 37).
+ALTER TABLE change_state ADD COLUMN mentions INTEGER NOT NULL DEFAULT 0 CHECK(typeof(mentions)='integer' AND mentions BETWEEN 0 AND 9007199254740991);
+ALTER TABLE change_state ADD COLUMN responses INTEGER NOT NULL DEFAULT 0 CHECK(typeof(responses)='integer' AND responses BETWEEN 0 AND 9007199254740991);
+
+CREATE TRIGGER change_mentions_in_insert
+AFTER INSERT ON mentions_in
+BEGIN
+  SELECT RAISE(ABORT, 'missing change state') WHERE NOT EXISTS(SELECT 1 FROM change_state WHERE id=1);
+  UPDATE change_state SET
+    mentions=mentions+(CASE WHEN NEW.status IN ('verified','gone') THEN 1 ELSE 0 END),
+    responses=responses+(CASE WHEN NEW.status='verified' AND NEW.hidden=0 THEN 1 ELSE 0 END)
+  WHERE id=1;
+END;
+
+CREATE TRIGGER change_mentions_in_update
+AFTER UPDATE ON mentions_in
+WHEN OLD."attempts" IS NOT NEW."attempts" OR OLD."error" IS NOT NEW."error" OR OLD."first_seen" IS NOT NEW."first_seen" OR OLD."hidden" IS NOT NEW."hidden" OR OLD."id" IS NOT NEW."id" OR OLD."last_seen" IS NOT NEW."last_seen" OR OLD."relation" IS NOT NEW."relation" OR OLD."source" IS NOT NEW."source" OR OLD."source_author_json" IS NOT NEW."source_author_json" OR OLD."source_id" IS NOT NEW."source_id" OR OLD."source_kind" IS NOT NEW."source_kind" OR OLD."source_origin" IS NOT NEW."source_origin" OR OLD."source_page" IS NOT NEW."source_page" OR OLD."source_version" IS NOT NEW."source_version" OR OLD."status" IS NOT NEW."status" OR OLD."target" IS NOT NEW."target" OR OLD."target_item_id" IS NOT NEW."target_item_id" OR OLD."verified_at" IS NOT NEW."verified_at"
+BEGIN
+  SELECT RAISE(ABORT, 'missing change state') WHERE NOT EXISTS(SELECT 1 FROM change_state WHERE id=1);
+  UPDATE change_state SET
+    mentions=mentions+(CASE WHEN OLD.status IN ('verified','gone') OR NEW.status IN ('verified','gone') THEN 1 ELSE 0 END),
+    responses=responses+(CASE WHEN (OLD.status='verified' AND OLD.hidden=0) OR (NEW.status='verified' AND NEW.hidden=0) THEN 1 ELSE 0 END)
+  WHERE id=1;
+END;
+
+CREATE TRIGGER change_mentions_in_delete
+AFTER DELETE ON mentions_in
+BEGIN
+  SELECT RAISE(ABORT, 'missing change state') WHERE NOT EXISTS(SELECT 1 FROM change_state WHERE id=1);
+  UPDATE change_state SET
+    mentions=mentions+(CASE WHEN OLD.status IN ('verified','gone') THEN 1 ELSE 0 END),
+    responses=responses+(CASE WHEN OLD.status='verified' AND OLD.hidden=0 THEN 1 ELSE 0 END)
+  WHERE id=1;
+END;
+
+-- The Studio lists every outbound row, status and retry time included, so any
+-- change is a mentions change. Outbound rows never reach a public page.
+CREATE TRIGGER change_mentions_out_insert
+AFTER INSERT ON mentions_out
+BEGIN
+  SELECT RAISE(ABORT, 'missing change state') WHERE NOT EXISTS(SELECT 1 FROM change_state WHERE id=1);
+  UPDATE change_state SET
+    mentions=mentions+(1)
+  WHERE id=1;
+END;
+
+CREATE TRIGGER change_mentions_out_update
+AFTER UPDATE ON mentions_out
+WHEN OLD."attempts" IS NOT NEW."attempts" OR OLD."created" IS NOT NEW."created" OR OLD."endpoint" IS NOT NEW."endpoint" OR OLD."id" IS NOT NEW."id" OR OLD."item_id" IS NOT NEW."item_id" OR OLD."last_error" IS NOT NEW."last_error" OR OLD."next_attempt_at" IS NOT NEW."next_attempt_at" OR OLD."status" IS NOT NEW."status" OR OLD."target" IS NOT NEW."target" OR OLD."target_version" IS NOT NEW."target_version" OR OLD."version" IS NOT NEW."version"
+BEGIN
+  SELECT RAISE(ABORT, 'missing change state') WHERE NOT EXISTS(SELECT 1 FROM change_state WHERE id=1);
+  UPDATE change_state SET
+    mentions=mentions+(1)
+  WHERE id=1;
+END;
+
+CREATE TRIGGER change_mentions_out_delete
+AFTER DELETE ON mentions_out
+BEGIN
+  SELECT RAISE(ABORT, 'missing change state') WHERE NOT EXISTS(SELECT 1 FROM change_state WHERE id=1);
+  UPDATE change_state SET
+    mentions=mentions+(1)
+  WHERE id=1;
+END;

openapi.json

@@ -187,6 +187,16 @@
                 "type": "integer",
                 "minimum": 0,
                 "maximum": 9007199254740991
+              },
+              "mentions": {
+                "type": "integer",
+                "minimum": 0,
+                "maximum": 9007199254740991
+              },
+              "responses": {
+                "type": "integer",
+                "minimum": 0,
+                "maximum": 9007199254740991
               }
             },
             "required": [
@@ -196,7 +206,9 @@
               "hoppers",
               "signals",
               "settings",
-              "feed"
+              "feed",
+              "mentions",
+              "responses"
             ]
           }
         },

scripts/e2e-server.ts

@@ -76,6 +76,20 @@ const proxy = createServer(async (request, response) => {
       response.end('<iframe src="' + escaped + '"></iframe>');
       return;
     }
+    // Disposable stand-in for a sender whose claim this blyg has just verified:
+    // writes one verified inbound mention straight into the 8787 fixture
+    // database, as verifyMention's UPDATE would. Fixture-only; the Worker ships
+    // no such route (e2e/polling-cache.spec.ts, live mentions).
+    if (path === "/__fixture/verified-mention" && request.method === "POST") {
+      const chunks: Buffer[] = [];
+      for await (const chunk of request) chunks.push(Buffer.from(chunk));
+      const { id, item, origin } = JSON.parse(Buffer.concat(chunks).toString("utf8")) as { id: string; item: string; origin: string };
+      const now = new Date().toISOString(), source = `${origin}t/${id}/`;
+      await db.prepare("INSERT INTO mentions_in (id, source, target, target_item_id, status, relation, source_origin, source_id, source_kind, source_version, source_author_json, source_page, first_seen, last_seen, verified_at, hidden) VALUES (?, ?, ?, ?, 'verified', 'stub', ?, ?, 'thread', 1, ?, ?, ?, ?, ?, 0)")
+        .bind(id, source, `http://127.0.0.1:8787/t/${item}/`, item, origin, id, JSON.stringify({ name: "Live responder" }), source, now, now, now).run();
+      response.writeHead(204); response.end();
+      return;
+    }
     if (!path.startsWith("/notes/b/") && !path.startsWith("/api/")) { response.writeHead(404); response.end("outside forwarding ranges"); return; }
     const chunks: Buffer[] = [];
     for await (const chunk of request) chunks.push(Buffer.from(chunk));

scripts/reset-change-epoch.sql

@@ -2,4 +2,5 @@
 -- automatically during ordinary upgrades: cursors should survive deployments.
 INSERT INTO change_state(id) VALUES(1)
 ON CONFLICT(id) DO UPDATE SET epoch=excluded.epoch,
-  items=0, reading=0, subscriptions=0, hoppers=0, signals=0, settings=0, feed=0;
+  items=0, reading=0, subscriptions=0, hoppers=0, signals=0, settings=0, feed=0,
+  mentions=0, responses=0;

sdk/generated/types.gen.ts

@@ -42,6 +42,8 @@ export type ChangeState = {
         signals: number;
         settings: number;
         feed: number;
+        mentions: number;
+        responses: number;
     };
 };
 

sdk/generated/zod.gen.ts

@@ -37,7 +37,9 @@ export const zChangeState = z.object({
         hoppers: z.int().gte(0).lte(9007199254740991),
         signals: z.int().gte(0).lte(9007199254740991),
         settings: z.int().gte(0).lte(9007199254740991),
-        feed: z.int().gte(0).lte(9007199254740991)
+        feed: z.int().gte(0).lte(9007199254740991),
+        mentions: z.int().gte(0).lte(9007199254740991),
+        responses: z.int().gte(0).lte(9007199254740991)
     })
 });
 

src/change-state.ts

@@ -1,5 +1,8 @@
 /** Fixed domains shared by the API and Studio. Adding a response dependency
- * requires its trigger and receiving witness, not a caller invalidation hook. */
-export const CHANGE_DOMAINS = ['items', 'reading', 'subscriptions', 'hoppers', 'signals', 'settings', 'feed'] as const;
+ * requires its trigger and receiving witness, not a caller invalidation hook.
+ * `mentions` is the Studio's view of both mention tables; `responses` is the
+ * narrower public one (verified, unhidden inbound rows), so unverified claims
+ * cannot roll the public HTML cache key (migration 0032). */
+export const CHANGE_DOMAINS = ['items', 'reading', 'subscriptions', 'hoppers', 'signals', 'settings', 'feed', 'mentions', 'responses'] as const;
 export type ChangeDomain = typeof CHANGE_DOMAINS[number];
 export type ChangeState = { epoch: string; domains: Record<ChangeDomain, number> };

src/html-cache.ts

@@ -14,10 +14,14 @@ export function publicHtmlRequest(request: Request, mount: string): boolean {
     (PATH_SHAPE.test(page.slice(1)) && !RESERVED_PATHS.includes(page.slice(1)));
 }
 
-// Every domain a public page can read. Reading and signals are the owner's
-// private state, so moving them must not empty the cache. Inbound mentions
-// have no domain yet and still wait out the 60-second window.
-const PUBLIC_DOMAINS: readonly ChangeDomain[] = ['items', 'subscriptions', 'hoppers', 'settings', 'feed'];
+// Every domain a public page can read. Reading, signals and mentions are the
+// owner's private state, so moving them must not empty the cache. `responses`
+// is the one mentions fact a page shows: the responses list under an item
+// (listPublicResponses: verified and not hidden). Its triggers (0032) move it
+// only when a row enters, leaves or changes inside that set, so a verified
+// response or a hide/show appears at once, while a stranger's unverified claim,
+// or a flood of them, cannot roll the key and empty the cache.
+const PUBLIC_DOMAINS: readonly ChangeDomain[] = ['items', 'subscriptions', 'hoppers', 'settings', 'feed', 'responses'];
 
 /** A token that moves whenever a publish, withdrawal or setting can change public HTML. */
 export function publicHtmlRevision(changes: ChangeState): string {

src/mentions/store.ts

@@ -309,8 +309,10 @@ export async function pendingInboundCount(db: D1Database, now: number = Date.now
  */
 export async function pruneFailedInbound(db: D1Database, now: number = Date.now()): Promise<number> {
   const cutoff = new Date(now - FAILED_INBOUND_RETENTION_MS).toISOString();
-  const res = await db.prepare("DELETE FROM mentions_in WHERE status = 'failed' AND last_seen < ?").bind(cutoff).run();
-  return res.meta.changes ?? 0;
+  // Counted from the rows deleted, not meta.changes: D1 adds the change-state
+  // trigger's write to that count (0032), which keeps only its zero/nonzero meaning.
+  const res = await db.prepare("DELETE FROM mentions_in WHERE status = 'failed' AND last_seen < ? RETURNING id").bind(cutoff).all();
+  return res.results.length;
 }
 
 /**

src/ui/catalog.tsx

@@ -750,10 +750,6 @@ export function MentionsPage() {
                   now
                 </p>
                 <ResponseVisibility id={id} mode={mode} defaultShowing={!!settings?.show_responses_default} disabled={!item} />
-                <p className="muted small">
-                  The public page can take about a minute to reflect a change
-                  here: it is cached at the edge.
-                </p>
                 {rows.map((mention) => (
                   <MentionRow key={mention.id} mention={mention} />
                 ))}