Fix HuggingFace usage extraction: defaultapi_flavor and generic huggingface provider - #452
Fix HuggingFace usage extraction: defaultapi_flavor and generic huggingface provider#452adtyavrdhn wants to merge 13 commits into
defaultapi_flavor and generic huggingface provider#452Conversation
The generated huggingface_* providers only exposed api_flavor 'chat', so extract_usage with the library default flavor raised ValueError. Duplicate the copied OpenAI chat extractor as the implicit default flavor, keeping 'chat' for backward compatibility.
HuggingFace defaults to server-side auto routing, in which case no routed provider URL is available and provider resolution found nothing. Add a hand-written huggingface provider with the same OpenAI-chat-shaped extractors. Its api_pattern matches only the /v1 router path so routed huggingface_* providers keep winning URL matches. It has no priced models, so the public extract_usage model lookup raises LookupError; callers extracting via the provider object directly are unaffected.
There was a problem hiding this comment.
No issues found across 21 files
Not reviewed (too large): tests/dataset/usages.json (~12,384 lines), packages/js/src/data.ts (~428 lines), packages/python/genai_prices/data.py (~266 lines), prices/data_slim.json (~2 lines), prices/data.json (~2 lines) - if these are generated or fixture files, add them to ignored paths to exclude them from future reviews.
…dence The generic provider is extraction-only: with server-side auto routing the serving provider is unknown, so it has no per-model prices and price calculation always fails. JS matches api_pattern with unanchored RegExp.test, so the routed-vs-generic resolution precedence is now pinned in the JS tests as well.
…xtraction # Conflicts: # README.md # packages/js/src/__tests__/extractUsage.test.ts # prices/data.json # prices/data_slim.json # prices/src/prices/source_huggingface.py # tests/dataset/usages.json # tests/test_extract_usage.py
…xtraction # Conflicts: # prices/new_data/v2/data.json # prices/new_data/v2/data_slim.json
…xtraction # Conflicts: # prices/new_data/v2/data.json # prices/new_data/v2/data_slim.json
There was a problem hiding this comment.
All reported issues were addressed across 14 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
default api_flavor and generic huggingface providerdefaultapi_flavor and generic huggingface provider
|
Closing in favor of #634, which implements the accepted default |
Part of #324 (gaps 2 and 3). Unblocks the HuggingFace migration in pydantic/pydantic-ai#4818.
What this fixes
defaultextractor and the existing explicitchatextractor for every routedhuggingface_*provider. Keeping both flavors preserves existing callers while making the library default work.huggingfaceprovider for Hugging Face's server-side auto-routed/v1endpoint, where the downstream provider is intentionally unknown. It has no models or fallback pricing because assigning a downstream price without provider identity would be ambiguous. Its explicit empty model path makes the public Python API return usage withmodel=Noneinstead of attempting an impossible model lookup; JavaScript returns the equivalentmodel: null.output_reasoning_tokens, whileoutput_tokensremains the aggregate completion total.The implementation adopts #327's generator-root-cause analysis and its backward-compatible “duplicate
defaultandchat” design option. Both #327 and this PR are by @adtyavrdhn; #327 remains open as the original plan/design record.Evidence
huggingface_hubgeneratedChatCompletionOutputUsagetype guaranteeprompt_tokens,completion_tokens, andtotal_tokens: chat-completion schema,huggingface_hubgenerated type.cached_tokensand Nebius/Qwen response with null nested detail objects.The generic extractor deliberately does not reinterpret Together's provider-specific top-level
cached_tokens: HF's formal schema does not define it, response shapes vary by routed provider, and mapping both top-level and nested forms would be additive in the current data-driven contract and could double-count. The exact recorded Together shape is pinned in Python and JavaScript tests to make that behavior explicit.Validation
make buildmake lintmake typecheckmake testnpm run citests/dataset/extract_usages.py(golden corpus up to date)git diff --check