[Feature] Add Rate Limit Configuration with Different Tiers (Free, Premium, Admin)

[KarenZita01]

Description

The basic rate limiting middleware implemented in Issue #6 applies a single, global limit to all requests. As the platform scales with different user tiers (free, premium, admin) and API key tiers (Issue #20), a more sophisticated rate limiting system is needed that applies different limits based on user role or authentication type.

This issue enhances the rate limiting system to support multiple tiers:

Free Tier: 60 requests per minute (approx 1 req/s). Applied to unauthenticated requests and users with free role. Includes basic endpoints only (system health, meter reads). Does NOT include admin endpoints or data exports.

Premium Tier: 600 requests per minute (approx 10 req/s). Applied to users with premium role or API keys with standard or premium tier. Includes all non-admin endpoints.

Admin Tier: 6000 requests per minute (approx 100 req/s). Applied to users with admin role. Includes all endpoints including admin. Monitoring/job endpoints have separate higher limits.

Internal Tier: Unlimited (or effectively very high, 60000/min). Applied to internal services (webhook callbacks, event listener sync) that communicate via a service-to-service API key or internal network.

The rate limiter should: identify the tier based on req.user.role (from JWT auth, Issue #5) or req.apiKey.tier (from API key auth, Issue #20) or fall back to free for unauthenticated requests; apply separate rate limit counters per tier (using Redis for distributed rate limiting, Issue #12); return appropriate Retry-After headers; log rate limit violations with user identity for abuse monitoring; and provide a rate limit status endpoint (GET /api/system/rate-limits) for users to check their current usage and limits.

Technical Context & Impact

• Dependencies: Depends on authentication (Issue [Testing] Expand Test Coverage with Integration Tests, API Tests, and Mock Blockchain Layer #5), API key system (Issue [Testing] Add Load Testing with k6 to Identify Performance Bottlenecks Under Stress #20), and Redis caching (Issue [Performance] Add Redis Caching Layer for Frequent Blockchain Data Queries #12) for distributed rate limit counters. May use express-rate-limit with custom key generator and skip function, or implement custom rate limiting logic using Redis INCR with TTL.
• Architecture: Update src/middleware/rateLimiter.js to accept tier configuration. Store tier definitions in src/config/rateLimits.js. Use Redis-backed counters for production, in-memory Map for development.
• Impact: Essential for fair resource allocation, abuse prevention, and monetization (free vs premium tiers). Proper rate limiting prevents a single user from degrading API performance for others.

Step-by-Step Implementation Guide

1. Create Rate Limit Configuration: Write src/config/rateLimits.js exporting tier definitions: { free: { windowMs: 60000, max: 60 }, premium: { windowMs: 60000, max: 600 }, admin: { windowMs: 60000, max: 6000 }, internal: { windowMs: 60000, max: 60000 } }. Include separate limits for write-heavy endpoints if needed.
2. Create Tier Detection Function: Write determineTier(req) that checks req.apiKey?.tier first (API key auth), then req.user?.role (JWT auth), else returns 'free'. Consider allowing admin role to override API key tier.
3. Update Rate Limiter Middleware: Modify the rate limiter from Issue [Middleware] Add Request Logging, Error Handling, Rate Limiting, and Request Validation Middleware #6. Use express-rate-limit with a custom keyGenerator that includes tier, user ID/IP, and endpoint group. Use a custom skip function to exempt internal services. Store counters in Redis (fallback to memory store).
4. Implement Redis-Backed Store: Create a Redis store adapter for rate limit counters. Use Redis.INCR and set TTL on first request in the window. For distributed environments, this ensures accurate counts across multiple API instances.
5. Create Rate Limit Status Endpoint: Write GET /api/system/rate-limits returning { tier, limit, remaining, resetTime, retryAfter }. Protected by authentication. This allows clients to proactively manage their request rate.
6. Per-Endpoint Overrides: Add support for endpoint-specific overrides. For example, the export endpoint (Issue [Feature] Create Export Endpoint for CSV/JSON Data Downloads with Streaming #23) may have lower limits than simple reads. Use route-level configuration via rateLimit({ tier: 'premium', max: 60 }).
7. Write Tests: Create tests/unit/rateLimiter.test.js testing tier detection, counter increments, limit enforcement. Create tests/integration/rateLimiter.test.js testing full flow: authenticate as different roles, exceed limits, check Retry-After headers, verify rate limit status endpoint.

Verification & Testing Steps

1. Call an API endpoint without authentication 61 times in one minute — verify the 61st request returns 429 with Retry-After header.
2. Authenticate as a premium user and make 601 requests in one minute — verify the 601st returns 429. Verify that free tier limit (60) does not apply to premium users.
3. Call GET /api/system/rate-limits with a valid JWT — verify the response shows the correct tier (premium), remaining count, and reset time.
4. As an admin, make requests rapidly — verify the admin limit (6000/min) applies, which is much higher than free/premium.
5. As an internal service with a service-to-service API key, make requests — verify no rate limiting is applied (or very high limit).
6. Check the application logs for rate limit violation entries — verify they include user ID/IP and tier information for abuse monitoring.

[Muhammadjazuli]

I would appreciate the opportunity to work on this issue and contribute to this project. I have experience with React, JavaScript, Next.js, and backend development with NestJS. I am confident in my ability to understand the requirements and deliver a high-quality solution within the expected timeline.