Flawed LLM Cost Estimation for Versioned and Pinned Models

[archittmittal]

Severity

Medium

Impact

Cost estimations on the dashboard show $0.00 for the majority of production runs.

Description

In packages/sdk/agentscope/_pricing.py, the SDK uses an exact dictionary match to find model prices. However, developers routinely specify pinned model versions (e.g. gpt-4o-2024-05-13, claude-3-5-sonnet-20240620). Because these versioned names do not match the keys in PRICING_TABLE exactly, the pricing calculator returns 0.0 cost.

Sub-Issues

• Sub-Issue 6.1: Substring/Pattern-Matching Price Resolution
  Implement pattern matching or prefix-based checks (e.g. checking if model_name.startswith("gpt-4o")) to map versioned models to their base model prices.
• Sub-Issue 6.2: Configurable Pricing Table
  Allow developers to pass custom pricing definitions or override the pricing table via environment variables or the configuration SDK.