[feat]: add prompt_embeds support#2511
Conversation
|
@huisunCompiler please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
Contributor License AgreementContribution License AgreementThis Contribution License Agreement (“Agreement”) is agreed to by the party signing below (“You”),
|
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds new ModelBuilder pass configuration options related to KV cache and prompt-embeds generation, and verifies those options are forwarded to the underlying GenAI builder.
Changes:
- Added
use_cacheandhidden_states_layerspass config parameters toModelBuilder. - Added a unit test ensuring these options (and existing ones) are forwarded to
create_model.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| test/passes/onnx/test_model_builder.py | Adds a test that captures create_model(**kwargs) to verify new options are passed through. |
| olive/passes/onnx/model_builder.py | Introduces new pass config params (use_cache, hidden_states_layers) with descriptions for text-encoder/prompt-embeds scenarios. |
| assert fake_builder.create_model.call_args.kwargs["input_path"] == str(test_model_path) | ||
|
|
||
|
|
||
| def test_model_builder_prompt_embeds_options_forwarded(tmp_path, monkeypatch): |
| type_=list[int], | ||
| required=False, | ||
| description=( | ||
| "Hugging Face hidden_states layer indices to concatenate into prompt_embeds " |
Support use_cache and hidden_states_layers