Repository navigation
Python: Bug: Anthropic, Bedrock, Google AI and Vertex AI tool formatters ignore include_in_function_choices=False #14615
Description
Describe the bug
A parameter marked with {"include_in_function_choices": False} is hidden from the model by the OpenAI formatters. The Anthropic, Bedrock, Bedrock agent, Google AI and Vertex AI formatters ignore the flag. They still send the parameter in the tool schema.
The decorator docs say a parameter with this flag "will not be included in the function choices". In function_calling_utils.py both OpenAI formatters filter on include_in_function_choices. The other formatters read metadata.parameters without the filter.
Files on current main:
python/semantic_kernel/connectors/ai/anthropic/services/utils.pypython/semantic_kernel/connectors/ai/bedrock/services/model_provider/utils.pypython/semantic_kernel/connectors/ai/google/google_ai/services/utils.pypython/semantic_kernel/connectors/ai/google/vertex_ai/services/utils.pypython/semantic_kernel/agents/bedrock/action_group_utils.py(Bedrock agent action group schema)
To Reproduce
I ran this on microsoft/semantic-kernel main at 7a92cd4 (Python 3.13).
from typing import Annotated
from semantic_kernel import Kernel
from semantic_kernel.functions import kernel_function
from semantic_kernel.connectors.ai.function_calling_utils import kernel_function_metadata_to_function_call_format as oa
from semantic_kernel.connectors.ai.anthropic.services.utils import kernel_function_metadata_to_function_call_format as an
from semantic_kernel.connectors.ai.google.google_ai.services.utils import kernel_function_metadata_to_google_ai_function_call_format as ga
from semantic_kernel.connectors.ai.google.vertex_ai.services.utils import kernel_function_metadata_to_vertex_ai_function_call_format as va
from semantic_kernel.connectors.ai.bedrock.services.model_provider.utils import update_settings_from_function_choice_configuration as bd
from semantic_kernel.connectors.ai.bedrock.bedrock_prompt_execution_settings import BedrockChatPromptExecutionSettings
from semantic_kernel.agents.bedrock.action_group_utils import kernel_function_metadata_to_bedrock_function_schema as ba
from semantic_kernel.connectors.ai.function_choice_type import FunctionChoiceType
from semantic_kernel.connectors.ai.function_call_choice_configuration import FunctionCallChoiceConfiguration
class P:
@kernel_function(name="lookup", description="Look up a user")
def lookup(
self,
query: Annotated[str, "What to look up"],
token: Annotated[str, "Internal token", {"include_in_function_choices": False}] = "",
) -> str:
return query
meta = Kernel().add_plugin(P(), "p").functions["lookup"].metadata
print("OpenAI :", list(oa(meta)["function"]["parameters"]["properties"]))
print("Anthropic :", list(an(meta)["input_schema"]["properties"]))
print("GoogleAI :", list(ga(meta)["parameters"]["properties"]))
print("VertexAI :", list(type(va(meta)).to_dict(va(meta))["parameters"]["properties"]))
s = BedrockChatPromptExecutionSettings()
bd(FunctionCallChoiceConfiguration(available_functions=[meta]), s, FunctionChoiceType.AUTO)
print("Bedrock :", list(s.tools[0]["toolSpec"]["inputSchema"]["json"]["properties"]))
print("BedrockAgent :", list(ba(meta)["parameters"]))Output:
OpenAI : ['query']
Anthropic : ['query', 'token']
GoogleAI : ['query', 'token']
VertexAI : ['token', 'query']
Bedrock : ['query', 'token']
BedrockAgent : ['query', 'token']
Expected behavior
All formatters hide token, as the OpenAI one does. The model should only see query.
Impact
The parameter name and description are sent to the model on these providers. The model can then try to fill a value that the caller meant to supply itself. It also adds tokens to every request.
Fix
The fix is small. Filter on include_in_function_choices in these formatters, for both properties and required. I have a branch with tests that fail on main and pass with the change: https://github.com/VANDRANKI/semantic-kernel/tree/fix/non-openai-formatters-include-in-function-choices
I can open a PR from this branch if the maintainers want it.
Activity