Declarative elicitation: ask for tool parameters by annotating them - #4691
Declarative elicitation: ask for tool parameters by annotating them#4691jlowin wants to merge 14 commits into
Conversation
A parameter annotated Annotated[T, Elicit(...)] is elicited rather than model-supplied, hidden from the input schema, and resolved before the body runs. Independent questions batch into one round; a callable question takes its values by name, which also orders the asks. The engine is private and expected to move into uncalled-for.
The page was organized around which protocol era you were on; declared parameters work on both, so it now leads with the choice between declaring, asking imperatively, and driving the rounds yourself.
marvin-context-protocol
Bot
added
feature
There is no ordering knob. A question that must wait almost always has something to say about what it waits for — a confirmation quoting the details being the clearest case — so writing it that way fixes the wording and the timing together.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 79d9f58f2c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| `None` is what a decline leaves behind. Any other metadata in the | ||
| `Annotated` is preserved so `Field(...)` constraints still shape the schema. | ||
| """ | ||
| type_arg = get_args(annotation)[0] |
There was a problem hiding this comment.
When an elicited parameter includes additional metadata such as Annotated[int, Field(gt=0), Elicit(...)], selecting only get_args(annotation)[0] discards the Field metadata. Because the parameter is also removed from the tool-validation schema, the generated elicitation schema accepts values such as -1 and no later validation restores the declared constraint. Rebuild the response annotation with all non-Elicit metadata after removing any optional None arm.
Useful? React with 👍 / 👎.
| if context._is_modern_protocol(): | ||
| return await _resolve_across_rounds(specs, arguments, context) | ||
| return await _resolve_in_process(specs, arguments, context) |
There was a problem hiding this comment.
When an Elicit-annotated tool runs with task=True, the worker Context has no live request context, so _is_modern_protocol() returns false and this chooses _resolve_in_process(). That path calls Context.elicit(), which deliberately raises for every background task, even though the task input loop supports InputRequiredResult re-entry. Treat background tasks like the modern protocol here so declarative elicitation produces NeedsInput instead of making all tasked calls fail.
Useful? React with 👍 / 👎.
Elicit becomes generic and a resolver returns `T | Elicit[T]`: a value fills the parameter without asking, an Elicit performs the ask. elicit_type states the type where the annotation is out of view, and a resolver's declared Elicit[T] is checked against its parameter at registration. Also fixes 3.10, where get_type_hints still applies implicit-Optional, so a `= None` parameter arrived wrapped in a Union and the buried-marker guard misfired on the legitimate optional form.
…notation-1d4770 # Conflicts: # docs/servers/elicitation.mdx
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3ad16cd578
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if not found: | ||
| return {} | ||
|
|
||
| available = set(signature.parameters) |
There was a problem hiding this comment.
When a resolver names a Context or Depends parameter already declared on the tool, this check accepts it because available contains every signature parameter. Elicitation runs before resolve_dependencies(), however, so ElicitParam.resolve() receives only model arguments and prior elicited values and raises KeyError for that parameter. Either resolve these values before invoking the resolver or reject this wiring at registration and require the resolver to declare its own dependency.
Useful? React with 👍 / 👎.
| # component turns into this leg's result. | ||
| user_kwargs = { | ||
| **user_kwargs, | ||
| **await resolve_elicitations(elicit_params, user_kwargs, get_context()), |
There was a problem hiding this comment.
On the modern protocol, this unconditional resolution also activates Elicit for prompts and resources because their implementations call the same without_injected_parameters() helper. Only FunctionTool.run() catches NeedsInput and can return an InputRequiredToolResult, so a prompt such as name: Annotated[str, Elicit("Name?")] has its argument hidden but get_prompt fails with NeedsInput instead of eliciting; resource templates fail similarly. Gate this behavior to tool wrappers, or provide compatible handling for every component that uses this shared wrapper.
AGENTS.md reference: AGENTS.md:L44-L51
Useful? React with 👍 / 👎.
| # `InputRequiredResult` would each overwrite the other's `request_state` | ||
| # and the call would never converge. Reject the combination outright | ||
| # rather than let it fail confusingly at run time. | ||
| if _contains_input_required(output_type) and find_elicit_parameters(fn): |
There was a problem hiding this comment.
When a tool with an Elicit parameter omits its return annotation but its body returns an InputRequiredResult, output_type is inspect.Parameter.empty, so this registration guard does not detect the forbidden combination. On the modern protocol the declarative answer lets the body run, the body's request state replaces the declarative state, and the next leg asks for the declarative parameter again; the two questions then alternate until the client raises its input-round-limit error. Add a runtime conflict check when such a tool body returns InputRequiredResult.
Useful? React with 👍 / 👎.
The declarative form describes the same request as the imperative one, so it takes the same keywords. The difference is await versus return.
jlowin
changed the title
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1a49190d00
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| f"The resolver for {self.name!r} depends on {param_name!r}, " | ||
| "which could not be resolved" | ||
| ) from value.error | ||
| outcome = self.marker.message(**bound, **injected) |
There was a problem hiding this comment.
When a supported synchronous resolver performs blocking work, such as a database query or HTTP request, invoking it directly here blocks the server's event loop and stalls unrelated requests for the duration. Sync tool bodies already honor run_in_thread for this reason; synchronous resolver callables should likewise be dispatched through the threadpool while async resolvers continue to be awaited normally.
Useful? React with 👍 / 👎.
| # can see. Matching the DI engine's own tolerance, treat the function as | ||
| # having none rather than failing every tool with an odd annotation. | ||
| logger.debug("Could not read annotations of %r: %s", _fn_name(fn), e) | ||
| return {} |
There was a problem hiding this comment.
For a supported functools.partial tool, inspect.signature() exposes the remaining parameter annotations, but typing.get_type_hints(partial_obj) raises TypeError; returning an empty mapping here therefore silently disables every Elicit marker. For example, partially binding the first argument of a function whose second argument is Annotated[str, Elicit("Where?")] leaves that argument visible and required in the tool schema instead of eliciting it. Fall back to the resolved signature or the wrapped callable's annotations rather than treating this failure as proof that no markers exist.
Useful? React with 👍 / 👎.
call_tool and call_tool_mcp take input_responses and request_state, and allow_input_required hands back the ask instead of resolving it against the client's own handlers. CallToolResult carries it on input_required, mirroring the server's InputRequiredToolResult.
allow_input_required was the SDK session's own vocabulary. drive=True (the default) answers each ask from the client's handlers and returns a terminal result; drive=False hands the ask back so the caller can answer it on a later call, which is what an app whose user replies minutes later actually needs.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5fe1887fa0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| dependency_params = get_dependency_parameters(fn) | ||
| elicit_params = find_elicit_parameters(fn) |
There was a problem hiding this comment.
When a parameter is declared as x: Annotated[str, Elicit(...)] = Depends(provider), both scanners select it, so elicitation asks the user for x and then resolve_dependencies() overwrites that answer with the dependency value; declining is also treated as optional because the Depends object counts as a default. Reject the overlap during registration so a prompt cannot be shown when its answer will never reach the tool body.
AGENTS.md reference: AGENTS.md:L147-L150
Useful? React with 👍 / 👎.
|
|
||
| Answers do not float free of the questions that produced them. Each one is recorded against the exact text the user was shown, so it can only ever satisfy the question it actually answered. | ||
|
|
||
| That guard earns its keep on the modern protocol, where a call spans rounds and earlier answers travel back and forth with the request. Deploy reworded copy in the middle of someone's booking, or retry a call with a different argument feeding one of the questions, and the wording shifts underneath them — so FastMCP drops the stale answer and asks again rather than crediting someone with an answer to a question they were never shown. There is nothing to configure. It is worth knowing about because it explains the one behaviour that surprises people: a question you expected to be remembered coming back around. |
There was a problem hiding this comment.
On modern connections, the SDK's RequestStateBoundary cryptographically binds requestState to the tool name and a digest of the original arguments. Retrying a continuation with any changed argument is therefore rejected as Invalid or expired requestState before _decode_state() or the question digest can drop the stale answer and re-ask it. Restrict this example to wording changes with identical arguments, or explain that changing arguments requires restarting the exchange without the old state.
Useful? React with 👍 / 👎.
No elicitation handler and a tool that asks now yields the question on CallToolResult.input_required instead of an 'Elicitation not supported' error. A handler still answers inline where one is registered, and legacy still errors, because there the server is blocked mid-call and there is no result to hand back. No opt-in flag: not passing a handler already says it.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ecc05fa504
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if _return_ask and isinstance(first, mcp_types.InputRequiredResult): | ||
| # Internal contract with `call_tool`, which sets `_return_ask` | ||
| # when this client has no handler and narrows the result back | ||
| # out. Public callers never see anything but a CallToolResult. | ||
| return cast("mcp_types.CallToolResult", first) |
There was a problem hiding this comment.
When callers use the public call_tool_mcp() API without an elicitation handler, _return_ask defaults to false and only call_tool() sets it, so the first InputRequiredResult is sent into _drive_input_required() instead of being returned. The session's unsupported-elicitation callback then causes an MCPError rather than exposing the request, making the newly added input_responses and request_state parameters unusable for manually driving this raw API and contradicting its documented no-handler behavior. Base this decision on whether a handler exists or expose a supported public opt-out from automatic driving.
Useful? React with 👍 / 👎.
Migration guides now map the SDK's Resolve to Elicit rather than claiming FastMCP has no resolver injection; the FastMCP 3 guide leads its ctx.elicit era-gate section with declaring the parameter, since that is the fix that serves both eras. Client docs cover answering the rounds yourself, including the two same-named ElicitResult types. Also fixes a pre-existing broken anchor in the FastMCP 3 guide.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 16cd68d520
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| request_state=request_state, | ||
| # With no handler there is nothing here that can answer, so ask for | ||
| # the question itself instead of failing. | ||
| _return_ask=self._elicitation_callback is None, |
There was a problem hiding this comment.
When a modern tool returns an InputRequiredResult containing a sampling or roots request, a client configured with the corresponding sampling_handler or roots callback—but no elicitation handler—sets this flag to true and returns the intermediate request instead of invoking _drive_input_required(). This regresses the existing transparent handling of those request types, even though the driver dispatches elicitation, sampling, and roots alike; decide whether to return manually based on the embedded requests and their available handlers rather than only _elicitation_callback.
Useful? React with 👍 / 👎.
| annotation = _unwrap_optional(annotation) | ||
| if get_origin(annotation) is not Annotated: | ||
| return None | ||
| return next((m for m in get_args(annotation)[1:] if isinstance(m, Elicit)), None) |
There was a problem hiding this comment.
When the annotation is a PEP 695 or typing_extensions.TypeAliasType alias such as type Destination = Annotated[str, Elicit("Where?")], get_origin() is not Annotated and get_args() is empty, so both marker checks silently miss it. The resulting tool exposes destination as a required model-supplied argument and never elicits it; unwrap TypeAliasType.__value__ before checking optional and Annotated wrappers.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2d4b824a53
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if request.response_type is not None: | ||
| return request.response_type | ||
| return self.response_type |
There was a problem hiding this comment.
When an Elicit explicitly supplies response_type, this branch blindly prefers it over the parameter annotation, and the resulting value is injected after the tool's argument validation has already run. For example, x: Annotated[int, Elicit("Number?", response_type=str)] accepts a string and invokes the body with that string despite the int contract, potentially causing failures or incorrect behavior. Validate compatibility at registration or validate the settled value against the parameter's declared type before injection.
Useful? React with 👍 / 👎.
|
Holding this for chrisguidry/uncalled-for#12 |
jlowin
added
the
DON'T MERGE
1 participant
Asking a user for input means picking a mechanism and writing the exchange yourself. On handshake connections that's
ctx.elicit(). On the modern protocol, where the back-channel is gone, it's returning anInputRequiredResultand hand-rolling the state machine — buildingElicitRequestparams, stashing values intorequest_state, parsing them back out next round. The two are mutually exclusive, so serving both eras means writing both paths and branching on the protocol version.This adds a third option: declare which parameters come from the user and let FastMCP do the asking. Those parameters leave the input schema, get filled before the body runs, and — because the ask is no longer baked into the body's control flow — the framework picks the transport, so one function serves both eras.
An
Elicittakes the samemessageandresponse_typeasctx.elicit(), because it describes the same request — youawaitthe imperative one andreturnthis one. A fixed question can go straight in the annotation; anything more is a resolver, a function returningT | Elicit[T]where a plain value means the answer was already known and nobody is asked.The model sees one parameter,
traveller. The rest are the user's to answer, and the round structure falls out of what each question refers to rather than being written:destination,dateandcabinquote nothing, so they go out together instead of costing a round trip each.airportis only asked of the newcomer — the resolver already knew the returning traveller's home airport.confirmquotes two earlier answers, so it lands last; a confirmation that names what it confirms is ordered correctly by saying so, with no ordering knob involved. Decliningcabinleaves its default and the booking proceeds; drop the default and declining fails the call instead.Signature mistakes surface at import, not on the first production call: a resolver naming a value the tool doesn't have, two questions waiting on each other, or a resolver whose declared
Elicit[T]can't fit its parameter. Answers are pinned to a digest of the exact text the client was shown, so redeploying reworded copy mid-conversation re-asks rather than crediting someone with an answer to a question they never saw.The guard pattern stays, and the docs draw the line. Declared parameters resolve on every round, so a question built by a live search runs that search again and can offer flights that no longer exist by the time the answer arrives. When the work is expensive or non-deterministic, drive the rounds from the body and carry the result in
request_state. Combining the two in one tool is rejected at registration — one call, one input channel.The engine lives in a private module and is expected to move into uncalled-for once its
Annotatedpath can inject a value rather than only run a dependency for its side effects.Elicitis exported fromfastmcp.elicitationso that move stays invisible.