feat(sdk): Add Claude 3 and Gemini 1.5 Flash to pricing table - #33
feat(sdk): Add Claude 3 and Gemini 1.5 Flash to pricing table#33HeaTTap wants to merge 3 commits into
Conversation
- Add pricing lookup mappings for claude-3-haiku, claude-3-opus, and gemini-1.5-flash - Support pricing lookup matching for minor and date-pinned versions - Add unit tests verifying cost calculation for new models and date-pinned model names Closes PxA-Labs#29
github-actions
Bot
added
pkg: sdk
type: feature
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (2)
📝 WalkthroughWalkthroughThe SDK adds Claude 3 and Gemini 1.5 Flash pricing. Model lookup supports case-insensitive matching for versioned identifiers. Cost calculation resolves rates, handles missing models and token counts, and rejects negative counts. Tests cover aliases and validation. ChangesModel pricing support
Estimated code review effort: 2 (Simple) | ~10 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
packages/sdk/tests/test_sdk.py (1)
15-27: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winAdd regression tests for lookup boundaries and casing.
The new tests cover lowercase versioned names, but not the documented case-insensitive behavior or rejection of unrelated names that merely share a prefix. Add cases such as an uppercase date-pinned identifier and an invalid non-delimited suffix.
Suggested additions
+ assert calculate_cost("CLAUDE-3-HAIKU-20240307", 1_000_000, 1_000_000) == 1.50 + assert calculate_cost("claude-3-haikuish", 100, 100) == 0.0🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/sdk/tests/test_sdk.py` around lines 15 - 27, Extend the calculate_cost regression tests to verify case-insensitive matching with an uppercase date-pinned model identifier and to reject an unrelated name with a non-delimited suffix that merely shares a supported model prefix. Keep the existing pricing assertions and use the established expected result or error behavior for each boundary case.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/sdk/agentscope/_pricing.py`:
- Around line 49-53: Update the token normalization in the pricing calculation
so prompt_tokens and completion_tokens remain zero when None, but raise
ValueError for either negative count before computing input_cost or output_cost.
Use the existing input_tokens and output_tokens flow to validate both values
before the price calculations.
- Around line 25-27: Update the model-prefix matching loop in the pricing lookup
to require a hyphen separator immediately after the matched key, preventing
unrelated names such as gpt-4orange or claude-3-haikuish from matching. Preserve
the existing longest-key-first ordering and return behavior for valid versioned
identifiers.
---
Nitpick comments:
In `@packages/sdk/tests/test_sdk.py`:
- Around line 15-27: Extend the calculate_cost regression tests to verify
case-insensitive matching with an uppercase date-pinned model identifier and to
reject an unrelated name with a non-delimited suffix that merely shares a
supported model prefix. Keep the existing pricing assertions and use the
established expected result or error behavior for each boundary case.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: a115e9d5-1576-4590-9092-883bbf754d8d
📒 Files selected for processing (2)
packages/sdk/agentscope/_pricing.pypackages/sdk/tests/test_sdk.py
…ct negative token counts
archittmittal
left a comment
archittmittal
left a comment
There was a problem hiding this comment.
Thank you for the contribution! The changes look great and solve Issue #29 and Sub-Issue 6.1.
One minor request: could you please expand the test cases in packages/sdk/tests/test_sdk.py to also cover:
- Case-insensitivity verification using an uppercase date-pinned model name (e.g.,
CLAUDE-3-HAIKU-20240307). - Ensuring that non-delimited suffixes that share a model prefix but are unrelated (e.g.,
claude-3-haikuish) do not match and return0.0.
Once these tests are added, we will merge the PR!
|
Added the requested test cases in a commit here. Please pull from the branch to include in your PR: Additional test cases added:
You can pull the branch with: Or just add these test cases manually to your |
… test cases Signed-off-by: HeaTTap <heatver@gmail.com>
|
Thanks @archittmittal! I have expanded the test suite in
|
purvanshjoshi
left a comment
purvanshjoshi
left a comment
There was a problem hiding this comment.
Reviewed on behalf of @purvanshjoshi. All CI checks pass (Test Python SDK, Test FastAPI Server, Lint, CodeQL). Verified the current head addresses every prior review comment:
- Hyphen-separator prefix guard (
startswith(f{key}-)) — rejectsgpt-4orange/claude-3-haikuishfalse matches (CodeRabbit inline + archittmittal request). - Negative token counts now raise
ValueError;Nonetokens default to 0 (CodeRabbit inline). - Regression tests cover uppercase date-pinned identifiers and non-delimited suffix rejection (both requests).
Approving. One non-blocking note for a follow-up: the new prefix matching means gpt-4o-mini now resolves to gpt-4o pricing (previously 0.0), which will overstate cost ~10x for that model. Consider adding an explicit gpt-4o-mini entry (or ordering a more specific key first) — this also applies to the equivalent logic in #66.
3 participants
Description
This PR adds model pricing mappings for
claude-3-haiku,claude-3-opus, andgemini-1.5-flashtopackages/sdk/agentscope/_pricing.pyas requested in issue #29.Key Changes:
claude-3-haiku: $0.25 input / $1.25 output per 1M tokensclaude-3-opus: $15.00 input / $75.00 output per 1M tokensgemini-1.5-flash: $0.075 input / $0.30 output per 1M tokensget_model_pricinghelper supporting prefix-based lookup for minor and date-pinned model versions (e.g.,claude-3-haiku-20240307,gemini-1.5-flash-001,gpt-4o-2024-05-13).packages/sdk/tests/test_sdk.pywith unit tests for new model rates and date-pinned version matching.Closes #29
Summary by CodeRabbit
New Features
Bug Fixes