fix(notifications): paginate and index the notification list endpoints#564
Draft
tagpro wants to merge 1 commit into
Draft
fix(notifications): paginate and index the notification list endpoints#564tagpro wants to merge 1 commit into
tagpro wants to merge 1 commit into
Conversation
The notification tab loaded slowly because both list endpoints returned the entire notification table. GET /notification and GET /notification/unread ran `select(Notification).order_by(timestamp desc)` with no limit, so every page load fetched every row, pydantic-validated each one, serialised the lot to JSON and shipped it to the browser. On the instance this was found on the table had ~437,000 rows. The table also had no index beyond the primary key on `id`, so the ORDER BY sorted the whole table on every load and the unread filter was a full scan. Backend: - Both list endpoints now take `limit` (default 50, max 200) and `offset` (>= 0), threaded through the service into `.limit()/.offset()` in the SQL. Ordering stays timestamp descending. The number of matching notifications is returned in the `X-Total-Count` response header (exposed via CORS) so the UI can paginate. - GET /notification takes an optional `read` filter, so the UI can page through read notifications rather than fetching everything and filtering client-side. - New migration indexes the table for these queries: an index on `timestamp` descending for the ORDER BY, and a partial index for the unread query. - New PATCH /notification/read marks every unread notification read in a single statement. The UI used to do this by looping a PATCH over every notification it held, which only worked because it held all of them. Frontend: - The notifications page now loads a page of 50 with "load more", and does not fetch read notifications at all until that section is expanded. Tests: - Repository tests covering the limit cap, offset, timestamp-descending ordering, the unread query, and the bulk mark-as-read, seeded with more rows than the page size so pagination is actually exercised. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019wrZPxJDip5x3mzjWnpwos
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The notifications tab is very slow to load. Both list endpoints return the entire notification table:
So every visit to the tab fetches every row, pydantic-validates each one, serialises the whole list to JSON and ships it to the browser. On the instance where I hit this, the
notificationtable had ~437,000 rows — the tab was loading all of them on every page load and on every 30s poll.The table also has no index beyond the primary key on
id, so theORDER BY timestamp DESCsorts the whole table on every load, and the unread query (WHERE read = false) is a full scan.Changes
Backend
GET /notificationandGET /notification/unreadnow takelimit(default 50,le=200) andoffset(ge=0), threaded through the service into.limit()/.offset()in the SQL. Ordering is unchanged (timestamp descending).X-Total-Countresponse header (and exposed via CORS) so the UI knows how many pages there are.GET /notificationtakes an optionalreadfilter. The UI previously built its "read" list by fetching everything and filtering client-side, which is not possible once the endpoint is paginated.b3d51c7fa9e2, chained offe60ae827ed98) adds an index ontimestamp DESCfor the ORDER BY, and a partial index (WHERE read = false) for the unread query.PATCH /notification/readmarks every unread notification as read in a singleUPDATE. "Mark All as Read" used to work by looping a PATCH over every notification the client held — that only worked because the client held all of them, and would silently mark just the current page now.Frontend
Tests
limitcaps the number returned,offsetskips correctly, ordering is timestamp descending, the unread query returns only unread rows and respects limit/offset, and the bulk mark-as-read reaches past the loaded page. Seeded with more rows than the page size so pagination is actually exercised.tests/, async SQLite viaaiosqlite, plus atestdependency group), since there wasn't one yet.Not in this PR
The reason that instance had 437k notifications is a separate root cause: identical failure notifications are written every import cycle, so the table floods. That's worth fixing on its own; this PR is limited to making the endpoint bounded and indexed, which is what makes the tab slow regardless of how the rows got there.
Verification
uv run pytest— 14 passeduv run ruff check ./media_manager— cleannpm run lint(inweb/) — cleanX-Total-Count,limit=500is rejected with 422, offset pages don't overlap, the unread page never contains read rows, andPATCH /notification/readempties the unread list.alembic upgrade e60ae827ed98:b3d51c7fa9e2 --sqlrenders the expected DDL, including the partial index.🤖 Generated with Claude Code
https://claude.ai/code/session_019wrZPxJDip5x3mzjWnpwos