Repository navigation
chore(perf): add a perf host app for web performance testing - #3333
Draft
NikolaGoranNaumovski wants to merge 9 commits into
Draft
NikolaGoranNaumovski wants to merge 9 commits into
NikolaGoranNaumovski wants to merge 9 commits into
Conversation
A minimal chat page for measuring the SDK in the browser: SDK defaults only, one channel list, configured entirely by URL params (list, type, cap, page size). Pins the livestream message cap (maxLoadedItems: 230) and ships dev, prod and profiling builds. Lives in a private perf/* workspace that examples:build doesn't pick up.
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
|
Size Change: 0 B Total Size: 849 kB ℹ️ View Unchanged
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #3333 +/- ##
==========================================
+ Coverage 87.02% 87.72% +0.70%
==========================================
Files 547 545 -2
Lines 16251 16217 -34
Branches 5097 5077 -20
==========================================
+ Hits 14143 14227 +84
+ Misses 2108 1990 -118 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Prints ready-to-open perf host links (prod, dev, profiling) with every required URL parameter, plus the pinned web settings: livestream cap 230 via maxLoadedItems, messaging uncapped, no offline store. Aliases: react, react-web, browser.
… run/check commands Runner: perf run (seed → verify → N iterations → CSV row, per channel type), M-LAUNCH with a network/client split, M-FRAMES from streamed PipelineReporter traces (dropped % and per second), M-CPU with an idle floor, gates (markers-contiguous, no-anchor-jump, pagination-stalled, live-achieved, live-delivered, nothing-drew, receipts-changed, rows-changed), and perf check ml-08-oldest. Scenarios: CL-01, CL-06–CL-11, ML-01, ML-08 (+ scrolled-up), ML-09–ML-12, ML-15. Host: settle detector, workspace navigation bridge, message/channel/live state readers, idle().
…ive runs - navigation: open and close threads through the workspace navigation, jump to a message (quote or pinned route), switch between channels settling on each one's name - settling: a configurable watched region (the channel list for reconnects) and a settle armed while the tab is hidden; DOM changes counted while hidden - state readers: loaded rows and markers for a channel or the open thread, paginator flags, reply counts, read receipts, unread counts, channel-list order and row changes, which channels still exist, key interactions for M-INP, kind checks for the kitchen sink - validation switches (?leak=, ?drop_events=) for deliberate failures, logged in the settings Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Scenarios: message-list variants (ML-02…ML-07 with the kinds-rendered gate, ML-16), channel-list variants (CL-02…CL-05, CL-13), memory (CL-12, ML-17) and channel switching (CS-01, CS-02), soak (SK-01 with SK-02 read off the same window, sk-01-60m), threads (TH-01…TH-03), jumps (ML-13 and the pinned route), composer M-INP (CP-01 and its control, CP-02), reconnect (OF-02 with OF-03, and the message-list variant), and the browser-specific runs (WB-01 hidden tab, WB-02 resize, WB-03 startup cost). Shared pieces: - history.ts: the scroll-to-oldest window and its gates, also on a thread's paginator - memory.ts: 1 Hz sampling, forced GC, detached nodes, a chart per run - rawcdp.ts: a Chrome driven over raw CDP, for a tab Playwright would keep visible - live-specs.ts: every chatperf live command as data - streamed traces of any category; multi-seed datasets, per-iteration reseeds, extra rows read off one window, self-navigating scenarios; a recorded_at column for the nightly comparison Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A recording now also keeps the chat API responses and the WebSocket greeting the page received (paths only; the API key travels in query strings and is never stored). `perf renders --replay --offline` serves them through Playwright's request and WebSocket routing, with a placeholder key and an unsigned stand-in token: no Stream app, no credentials, no Ruby. `--baseline write|check` stores or enforces untargeted rows per commit. Fixtures for ML-09, ML-11 and CL-07 are re-recorded with their backend, and ML-08 is added. The PR workflow runs the contract test and the offline render gate, so it also runs for forks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…port - `perf run --set nightly|signoff|channel-list|message-list|leaks|desktop` (WEB-SCENARIOS.md §7); consecutive scenarios on an untouched dataset share one seed - `perf contract [--update]` and its test: every scenario file against a pinned copy of `chatperf scenarios --json` (canonical live rates and durations, seedable datasets, metrics), naming the file that drifted; re-pinning is a deliberate change, and the pin is generated, so Prettier leaves it alone - `perf compare`: the last 12 hours of rows against their 7-day median, alerting above +15 % - `perf report`: the sweep report, with the baseline deltas, standard ÷ virtualized, dev ÷ prod, the threshold flags and proposed M-FRAMES bands - `yarn workspace @stream-io/react-perf-runner test` runs the contract and compare tests Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Runs the nightly set against the live test app on a self-hosted `perf-nightly` runner, which holds chatperf, Ruby and the test app's credentials in its own environment (no GitHub secrets). Results accumulate in PERF_RESULTS_DIR outside the checkout, and `perf compare` fails the job when a row is more than 15 % worse than its 7-day median. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…on switches Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A minimal chat page for measuring the SDK in the browser: SDK defaults only, one channel list, configured entirely by URL params (list, type, cap, page size). Pins the livestream message cap (maxLoadedItems: 230) and ships dev, prod and profiling builds. Lives in a private perf/* workspace that examples:build doesn't pick up.
The entire perf web plan and scenarios can be found here