Distinguish per-key from whole-queue rate limiting in matching hook#10943
Closed
rkannan82 wants to merge 5 commits into
Closed
Distinguish per-key from whole-queue rate limiting in matching hook#10943rkannan82 wants to merge 5 commits into
rkannan82 wants to merge 5 commits into
Conversation
Per-fairness-key rate limiting should not suppress WCI scaling because other keys may still need workers. Only whole-queue rate limits indicate that adding more workers genuinely cannot help. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add SyncMatchOutcomeFairnessKeyRateLimited to hook enum instead of collapsing per-key rate limiting into NotMatched. Let hooks decide how to handle each outcome. - Rename syncMatchPerKeyRateLimited to syncMatchFairnessKeyRateLimited. - Use switch in MatchTaskImmediately so new enum values are handled explicitly. - Replace wholeQueueLimited bool return from findMatch with a rateLimitKind enum for a clearer API. - Remove hook-consumer-specific references from comments. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
rkannan82
force-pushed
the
kannan/wci-per-key-rate-limit
branch
from
3b77442 to
45d47b7
Compare
rkannan82
changed the title
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…temporal into kannan/wci-per-key-rate-limit
dnr
reviewed
Jul 7, 2026
| // A poller was available but whole-queue rate limiting blocked the match. | ||
| SyncMatchOutcomeRateLimited | ||
| // A poller was available but per-fairness-key rate limiting blocked the match. | ||
| SyncMatchOutcomeFairnessKeyRateLimited |
Contributor
There was a problem hiding this comment.
I'd rather not put this in the api like this:
- the existing fairness key rate limit is an interim feature that should be replaced by flow control features, which will be harder to handle this way
- at this point, the fairness key rate limit will block the whole queue (priority level, at least), so there's nothing different to be done from a scaler pov
Contributor
Author
There was a problem hiding this comment.
Ok. Closing this PR.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Distinguishes per-fairness-key rate limiting from whole-queue rate limiting by introducing a new enum constant called
SyncMatchOutcomeFairnessKeyRateLimited. This is now propagated to the matching hook API.Why
PR #10045 added
SyncMatchOutcomeRateLimitedbut didn't distinguish whole-queue from per-key rate limits. These have different scaling implications: whole-queue limits mean more workers can't help, but per-key limits doesn't mean you cannot scale up.How did you test it?
Unit tests:
TestMatchTaskImmediatelyFairnessKeyRateLimitedverifies per-key rate limiting returns the distinct outcome; existingTestMatchTaskImmediatelyRateLimitedcovers the whole-queue case.🤖 Generated with Claude Code