Repository navigation
reasoningEffort: executor-level guard for deprecated sampling params is Python-only (cross-runtime parity gap) #523
Description
Summary
reasoningEffort was added to ModelOptions in TypeSpec (fc07e78) and flows to all seven runtimes as a generated model field. However, the executor-level behavior that acts on it exists only in the Python runtime. The other runtimes carry the field but do nothing with it at call time, which is a cross-runtime parity gap.
What Python does (the reference behavior)
In runtime/python/prompty/prompty/providers/openai/executor.py:
_is_reasoning_model(model_id)— name-based detection of OpenAI reasoning models (o1/o3/o4/o5…, GPT-5 family)._OPENAI_REASONING_UNSUPPORTED = ("temperature", "top_p", "top_k", "frequency_penalty", "presence_penalty").- A live-call capability guard applied in the executor arg builders just before the SDK call:
- Reasoning models: drop the scalar sampling params above, each with a
DeprecationWarning("… does not support '{key}'; dropping it. Use reasoningEffort instead."). - Non-reasoning models: drop
reasoning_effort(only valid on reasoning models).
- Reasoning models: drop the scalar sampling params above, each with a
This is deliberately in the executor (a live-call concern), not in the pure to_wire option mappers.
The gap
reasoningEffort appears only in generated model files (model_options, session_start_payload, generated tests) in Go, Rust, C#, Java, TypeScript, and Swift — none of their execution layers translate reasoningEffort into the provider request or guard the deprecated sampling params. Verified via git grep: no executor/provider references outside Python.
Runtimes with execution layers that should reach parity: Rust (src/pipeline, src/engine), C# (Prompty.Foundry executors), TypeScript (packages/foundry). Model-only runtimes are out of scope until they grow a live executor.
Why it matters
This is exactly the class of gap the live-integration vectors are meant to surface: newer reasoning models reject temperature/top_p, so a runtime that forwards them will get live API errors while Python silently adapts. Cross-runtime behavior should match.
Proposed resolution
- Define the reasoning-model detection + unsupported-sampling-param set once (ideally data-driven from TypeSpec metadata so it isn't re-hardcoded per runtime — see note below).
- Port the executor-level guard to Rust, C#, and TypeScript executors (drop unsupported sampling params on reasoning models; drop
reasoningEfforton non-reasoning models). - Add shared conformance coverage so the guard's behavior is asserted cross-runtime rather than per-runtime.
Note on enforcement
The per-runtime name-based detection is duplicated logic. Consider whether the reasoning-model classification and its unsupported-param set belong in the TypeSpec model surface (so all runtimes derive the same table) rather than each runtime hardcoding OpenAI naming conventions. Tracked thought, not a blocker.
Acceptance criteria
- Rust, C#, and TypeScript executors guard deprecated sampling params for reasoning models and translate
reasoningEffort, matching Python. - Shared conformance asserts the behavior across runtimes.
Related: introduced in PR #522 (Python-only guard 09f5ce2). Turn-engine coverage backlog is separate (#519).
Activity