Added Artemis AI changes

[grafail]

Overview

This update introduces several improvements and refactorings across the API, monitoring, and routing components. The changes focus on performance, maintainability, and robustness, especially in rate limiting, model routing, and metrics collection.

---

Key Changes by Area

1. API Handlers (pkg/api/handlers.go)

• Rate Limiter Improvements:
  Switched to sync.RWMutex for better concurrency. Added expiry and cleanup for unused client limiters to prevent memory leaks. Improved logic for creating and cleaning up per-client rate limiters.

• Validation & Security:
  Centralized valid model and task type definitions for faster lookups. Added a setCommonHeaders function to standardize security and cache headers across responses.

• Request Handling:
  Simplified request body parsing and error handling. Improved error messages for invalid JSON and oversized requests.

• Environment Variable Parsing:
  Fixed bug in getEnvAsInt (now correctly reads environment variables and parses integers).

---

2. Parallel Query Handling (pkg/api/parallel.go)

• Model Validation:
  Centralized valid model definitions for parallel queries. Improved error handling for invalid models.

• Performance:
  Pre-sized response maps for efficiency. Improved error response structure and logging for parallel model queries.

---

3. Monitoring & Metrics (pkg/monitoring/monitoring.go)

• Performance & Accuracy:
  Added running sums and counts for efficient average duration calculation. Limited stored request durations to the most recent 100 for each model. Pre-allocated maps for better performance.

• Metrics Reporting:
  Average durations are now calculated using running totals, improving accuracy and efficiency.

---

4. Model Router (pkg/router/router.go)

• Maintainability:
  Centralized the list of all known model types to avoid duplication. Improved comments and documentation for clarity.

• Concurrency & Robustness:
  Enhanced locking and TTL checks for model availability updates. Improved fallback logic for model selection and error handling. More robust random selection of available models.

• Logging:
  Added more detailed logging for routing decisions and error fallbacks.

---

Impact

• Performance: Reduced memory usage and improved concurrency in rate limiting and metrics.
• Security: Consistent application of security headers.
• Maintainability: Centralized configuration and validation logic.
• Reliability: More robust error handling and fallback mechanisms.