fix(backends): deep-merge extra_body.chat_template_kwargs in merge_model_options

[planetf1]

Pull Request

Issue

Fixes #1539

Description

Impact

A backend-level extra_body default (e.g. a thinking-mode toggle set at construction time) gets silently dropped, with no error or warning, the moment a per-call extra_body is also present — even one that's about something else entirely:

backend = OpenAIBackend(
    model_id="gpt-4o", base_url="...",
    model_options={"extra_body": {"chat_template_kwargs": {"enable_thinking": False}}},
)
# ... later, any call that passes its own unrelated extra_body:
m.instruct("...", model_options={"extra_body": {"some_unrelated_field": 123}})
# -> enable_thinking is gone. The model thinks again, silently.

This isn't a rare edge case: it fires on every intrinsic/adapter call, because OpenAIBackend._generate_from_intrinsic always builds its own extra_body (to write chat_template_kwargs["adapter_name"]) — so a session that sets a thinking default and also uses any adapter loses that default the first time the adapter is called.

Root cause

merge_model_options merges option dicts with a flat, top-level dict.update. That's correct for scalar options, but wrong for extra_body, whose value is itself a dict that different call sites write into independently.

Fix

merge_model_options now special-cases extra_body: when both sides have a dict there, their chat_template_kwargs sub-dicts deep-merge instead of one replacing the other (new ModelOption._merge_extra_body, mirroring the deep-merge OpenAIBackend._merge_user_extra_body already does one layer down). Every other option key — and every other key inside extra_body — still flat-overwrites exactly as before; only the one broken case changes.

Also updates docs/docs/integrations/openai.md, which documented this exact clobbering as a known limitation — that caveat is now gone.

Testing

• Tests added to the respective file if code was changed
• New code has 100% coverage if code was added
• Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

New tests:

• test_model_options.py, test_options.py — unit coverage on the merge functions: unrelated-key preservation, chat_template_kwargs deep-merge, conflict resolution, non-mutation, non-dict fallback.
• test_openai_intrinsics_unit.py::test_construction_time_extra_body_default_survives_intrinsic_call — the impact scenario end to end through a real intrinsic call: a construction-time thinking default, an unrelated per-call extra_body, and the adapter's own adapter_name all survive together in the final request. Confirmed this test fails on pre-fix code (KeyError: 'enable_thinking') and passes post-fix.

Full run: uv run pytest test/ -m "not qualitative" → 4286 passed, 20 skipped, 4 xpassed, 1 unrelated pre-existing flake (test_tracing_backend.py::test_stream_e2e, a live-Ollama timing assertion — passes on rerun). ruff format/ruff check/mypy clean; markdownlint-cli2 clean on the docs change.

Attribution

• AI coding assistants used

---

Adding a new component, requirement, sampling strategy, or tool?

If your PR adds or modifies one of the types below, check the matching box. A checklist of type-specific review items will be posted as a comment.

• Component
• Requirement
• Sampling Strategy
• Tool

NOTE: Please ensure you have an issue that has been acknowledged by a core contributor and routed you to open a pull request against this repository. Otherwise, please open an issue before continuing with this pull request.

[jakelorocco]

For openai backends at least, I believe this causes construction-time enable_thinking to silently override per-call ModelOption.THINKING.