Problem
#1553 documents that structured-output examples using @generative stubs / format= are unreliable on thinking models unless the caller explicitly sets ModelOption.THINKING: False — the output parser otherwise receives the combined think-block-plus-answer text.
#1587 (Granite 4.2 defaults) independently confirmed the underlying mechanism: granite4.2 thinks by default unless reasoning_effort="none" / chat_template_kwargs.enable_thinking=False is sent, and produces a disproportionately long think block even for trivial prompts (observed: a 64-token capped reply was ~45 thinking tokens with thinking on, 2 with it off).
Proposal
When a caller passes format=SomeModel (or otherwise requests constrained/structured output) to generate_from_raw / act / a @generative stub, and the caller has not explicitly set ModelOption.THINKING, default it to False. This fixes the "parser receives combined think+answer text" failure mode structurally, instead of requiring every caller to remember to set the option themselves.
This is a default-behavior change (an explicit ModelOption.THINKING: True would still opt back in), so it needs maintainer sign-off on the direction before implementation — filing as a proposal to discuss, not a ready-to-merge PR.
Related
Problem
#1553 documents that structured-output examples using
@generativestubs /format=are unreliable on thinking models unless the caller explicitly setsModelOption.THINKING: False— the output parser otherwise receives the combined think-block-plus-answer text.#1587 (Granite 4.2 defaults) independently confirmed the underlying mechanism:
granite4.2thinks by default unlessreasoning_effort="none"/chat_template_kwargs.enable_thinking=Falseis sent, and produces a disproportionately long think block even for trivial prompts (observed: a 64-token capped reply was ~45 thinking tokens with thinking on, 2 with it off).Proposal
When a caller passes
format=SomeModel(or otherwise requests constrained/structured output) togenerate_from_raw/act/ a@generativestub, and the caller has not explicitly setModelOption.THINKING, default it toFalse. This fixes the "parser receives combined think+answer text" failure mode structurally, instead of requiring every caller to remember to set the option themselves.This is a default-behavior change (an explicit
ModelOption.THINKING: Truewould still opt back in), so it needs maintainer sign-off on the direction before implementation — filing as a proposal to discuss, not a ready-to-merge PR.Related
ModelOption.THINKINGonOllamaModelBackendTHINKING→reasoning_effortmapping this proposal would build on