Repository navigation
Drop legacy options from HF Inference api - #40892
Conversation
There was a problem hiding this comment.
📋 Review Summary
This PR updates _InferenceAPIHandler.run_inference to send only {"inputs": batch} to the Hugging Face router feature-extraction endpoint, removing the legacy nested "options" field in the JSON payload. The change is straightforward and fixes the payload compatibility issue.
🔍 General Feedback
- With
inference_argsno longer forwarded in_InferenceAPIHandler.run_inference, theoptionsdictionary (wait_for_modelanduse_cache) populated inInferenceAPIEmbeddings.get_ptransform_for_processing(lines 268-274) is now dead code. Consider cleaning that up or passing those options via HTTP headers (x-wait-for-model,x-use-cache) if supported by the router.
💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #40892 +/- ##
===========================================
+ Coverage 0 79.83% +79.83%
===========================================
Files 0 494 +494
Lines 0 83575 +83575
===========================================
+ Hits 0 66722 +66722
- Misses 0 16853 +16853
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Assigning reviewers: R: @claudevdm for label python. Note: If you would like to opt out of this review, comment Available commands:
The PR bot will only process comments in the main thread (not review comments). |
Cherrypick #40892 Drop legacy options from HF Inference api
fixes: #30799