feat: ai rate limiting redis support - #12751
Conversation
dosubot
Bot
added
size:XL
beardnick
changed the title
|
@Baoyuantop PTAL |
|
@beardnick many thx for your contribution, some CI tests are failed, you can fix them |
There was a problem hiding this comment.
Look like the changes in this PR should be limited to the ai-rate-limiting plugin and minor tweaks to the limit-count plugin. The current PR modifies too much code and test cases for other rate limiting plugins, expanding its scope. We should minimize unnecessary code changes.
|
Hi @beardnick, please check these comments. |
Baoyuantop
added
the
wait for update
|
Hi @beardnick, following up on the previous review comments. Please let us know if you have any updates. Thank you. |
There was a problem hiding this comment.
Pull request overview
This PR extends the ai-rate-limiting plugin to support Redis / Redis Cluster backed counters (to share token quotas across APISIX replicas and persist across restarts), refactors the limit-count Redis backends to support log-phase token commits via timers, and updates docs/tests accordingly.
Changes:
- Add
policy+ Redis/Redis Cluster configuration toai-rate-limitingand pass those settings through tolimit-count. - Refactor
limit-countRedis implementations (single + cluster) and introduce a shared Lua-script utility. - Add Redis / Redis Cluster coverage to
ai-rate-limitingtests and document the new configuration options in EN/ZH docs.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| t/plugin/ai-rate-limiting.t | Adds test cases for ai-rate-limiting with policy=redis and policy=redis-cluster. |
| docs/en/latest/plugins/ai-rate-limiting.md | Documents Redis/Redis Cluster-backed quota sharing and new config fields. |
| docs/zh/latest/plugins/ai-rate-limiting.md | Same as EN docs, in Chinese. |
| apisix/plugins/limit-count/util.lua | New shared Redis EVAL script helpers for limit-count Redis backends. |
| apisix/plugins/limit-count/limit-count-redis.lua | Adds log-phase support via timers and uses the shared util script. |
| apisix/plugins/limit-count/limit-count-redis-cluster.lua | Same as single-Redis backend, for Redis Cluster. |
| apisix/plugins/ai-rate-limiting.lua | Adds schema fields for Redis policies and forwards Redis configuration into limit-count. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
moonming
left a comment
moonming
left a comment
There was a problem hiding this comment.
Hi @beardnick, thank you for the AI rate limiting Redis support! This has received 26 reviews, showing significant community interest.
Since this has extensive review history:
- Could you rebase on the latest master?
- Please confirm all 26 review comments have been addressed
- Provide a summary of the key design decisions made during the review process
The feature itself is important — token-based rate limiting for AI workloads needs a distributed backend like Redis. Looking forward to getting this across the finish line!
dosubot
Bot
added
the
size:XXL
|
Hi @beardnick, I re-run CI, but some tests are still failing. Please fix it. |
The failed tests don't seem related to my PR. |
5e0096b to
0a59598
Compare
|
RFC-001 proposes a new ai-rate-limiting-redis plugin that clones ai-rate-limiting and replaces the local memory counter with Redis single-node backend to resolve the multi-replica rate limiting issue. Includes: - Full schema definition (inherited + 10 new Redis properties) - Redis Lua Script atomic counter design - Access Phase / Log Phase dual-phase implementation design - Complete ai-proxy-multi integration analysis and sequence diagram - 4 key integration caveats (instance name consistency, fallback counter behavior, placeholder restore on reject, header behavior on fallback) - Behavioral comparison table vs original plugin - Fail-open design rationale for Redis unavailability - Test plan covering multi-node, fallback, and Redis failure scenarios Ref: apache/apisix#12482, apache/apisix#12751 https://claude.ai/code/session_01Nw29Vgt1yXKJkGFLJnUpSN
…iting-redis-support
Resolve conflict in apisix/plugins/limit-count/init.lua by keeping both: - upstream's dry_run peek logic for local policy - PR's log_phase_incoming support for Redis policies Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
In OpenResty, return values from ngx.timer.at callbacks are discarded. Add core.log.error for Redis connection failures and token deduction failures in log_phase_incoming_thread, so operators have visibility when log-phase Redis operations fail silently. Apply the same fix to both limit-count-redis.lua and limit-count-redis-cluster.lua. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…e-limiting-redis-support Merge master into ai-rate-limiting
dosubot
Bot
added
size:XL
|
This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If you think that's incorrect or this pull request should instead be reviewed, please simply write any comment. Even if closed, you can still revive the PR at any time or discuss it on the dev@apisix.apache.org list. Thank you for your contributions. |
|
I noticed that this has been implemented in #13670. It's a pity that I didn't manage to merge this PR. I'll find other opportunities to contribute to APISIX. Thanks for your review — I really learned a lot from it. I'll close this PR. |
Description
Which issue(s) this PR fixes:
Fixes #12482
Notice
limit-count-redis.luaandlimit-count-redis-cluster.luafiles to ensure they now support rate-limiting during the log phase.limit-conn-redis.luafile as a guide while implementing rate-limiting in the log phase.require("lib.test_redis").flush_all()function to every Redis rate-limiting test case. Without this addition, the tests could fail unpredictably.limit-req-redis.tandlimit-req-redis-cluster.ttest cases. After thorough verification, the correct behavior is[200, 403, 403, 403]. Previously, the results appeared as[403, 403, 403, 403]due to Redis not being properly cleaned beforehand.Checklist