Skip to content

Rename model-client chat() → response(), add solve_physics_problem() and Batch API support - #2

Merged
sherryzyh merged 8 commits into
mainfrom
refactor/model-client-response-api
Jun 15, 2026
Merged

Rename model-client chat() → response(), add solve_physics_problem() and Batch API support#2
sherryzyh merged 8 commits into
mainfrom
refactor/model-client-response-api

Conversation

@sherryzyh

Copy link
Copy Markdown
Owner

Summary

This branch reworks the model-client layer in two substantive ways and ships a formatting cleanup:

  1. Rename the low-level call chat()response() and add a high-level solve_physics_problem() entry point, aligning the client surface with OpenAI's responses.create.
  2. Add synchronous Batch API support across OpenAI, Anthropic, and Gemini (~50% of sync cost), with provider-agnostic status/result types.

6 commits · 44 files · +2,075 / −374. Branch is a clean fast-forward over main (no divergence).

⚠️ Breaking change: the public client method chat() is renamed to response(). A deprecated chat() alias is kept that warns and forwards, so existing callers keep working but should migrate.

What changed

1. API rename + high-level entry point (111d25c, dff9d01)

  • chat()response(); prompt param user_promptinput; new keyword-only instructions (system prompt).
  • When instructions is omitted, every provider except OpenAI falls back to a short DEFAULT_INSTRUCTIONS system prompt (via _resolve_instructions; OpenAI overrides to stay input-only). An explicit empty string suppresses it.
  • System prompt is injected at each provider's native site: Anthropic top-level system, OpenAI instructions, openai-compatible/Ollama system message, Gemini system_instruction.
  • New BaseModelClient.solve_physics_problem() dispatches on input type: str question, PhysicsProblem (formatted prompt + image_path images), or PhysicsQuestionSemantics (NotImplementedError TODO, lazily imported to keep core off semantics at import time). Output gated by the new PhysicsOutputMode enum (only ANSWER_TEXT implemented).
  • New core-only modes.py and prompts.py; annotation worker's physics-expert prompt is centralized via instructions.
  • Cookbooks, docs/CORE.md, README, and RELEASE_NOTES updated to the new call.

2. Batch API support (2563141, 9e719e1, b7e7052)

  • BaseModelClient gains a synchronous batch lifecycle: submit_batchpoll_batchretrieve_batch_results, plus a free-text build_batch_request(...) mirroring response() (complements the existing structured build_batch_structured_request).
  • New prkit.core.model_clients.batch_types: BatchState, BatchStatus, BatchItemStatus, BatchResult normalize each provider's status enum and per-request results.
  • Each provider's request-body construction is now shared between response() and the batch builders (_build_responses_body / _build_messages_params) to prevent drift. OpenAI o-family models drop temperature at build time. Unsupported providers raise NotImplementedError.
  • Gemini correlation fix: batches are submitted as an uploaded keyed JSONL file (File API, src=<file>) rather than inline requests. Inline Gemini responses carry no per-request key and cannot be correlated; the keyed-JSONL path returns {"key": ..., "response": {...}} that maps reliably to each request. Dead inline submit/retrieve paths removed.
  • Gated live smoke test (skipped unless GEMINI_API_KEY/GOOGLE_API_KEY set) submitting a real batch with distinct ids + unique answers, asserting each answer lands under the correct custom_id — the identity check offline mocks can't make.

3. Formatting (b6504a5)

  • black applied to anthropic.py, openai.py, and tests/uq/conftest.py (pre-existing non-formatted files; CI runs black --check).

New files

  • src/prkit/core/model_clients/batch_types.py, modes.py, prompts.py
  • tests/prkit/core/model_clients/test_batch.py, test_batch_live.py, test_prompts.py

Testing

  • black --check src tests → clean (242 files).
  • pytest tests/ -x -q1378 passed, 1 skipped, 84% coverage.
  • Live Gemini batch correlation validated by the gated smoke test (requires API key).

🤖 Generated with Claude Code

sherryzyh and others added 8 commits June 14, 2026 13:42
Align the low-level model-client call with OpenAI's responses.create:
rename chat() -> response(), rename the prompt parameter user_prompt ->
input, and add a keyword-only `instructions` (system prompt) parameter.
When `instructions` is omitted every provider except OpenAI falls back to
a short DEFAULT_INSTRUCTIONS system prompt (resolved via the new
_resolve_instructions helper, which OpenAI overrides to stay input-only);
an explicit empty string suppresses it. Each provider injects the system
prompt at its native site (Anthropic top-level `system`, OpenAI
`instructions`, openai-compatible/Ollama system message, Gemini
`system_instruction`). chat_structured threads `instructions` through; a
deprecated chat() alias warns and forwards to response().

Add a high-level BaseModelClient.solve_physics_problem() that dispatches on
the input type: a plain str question, a PhysicsProblem (formatted into a
prompt plus its image_path images), or a PhysicsQuestionSemantics
(NotImplementedError TODO, lazy-imported to keep core off semantics at
import time); any other type raises TypeError. Output is gated by the new
PhysicsOutputMode enum (only ANSWER_TEXT implemented).

Add core-only modes.py and prompts.py; the semantics _format_problem now
reuses format_problem_context for its shared header. Centralize the
annotation worker's physics-expert system prompt via `instructions` and
update the calls.py retry path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Switch the model-client examples in the cookbooks, docs/CORE.md, README,
and RELEASE_NOTES to the new response(input=...) call, document the
`instructions` parameter and DEFAULT_INSTRUCTIONS fallback, and add a
solve_physics_problem() usage section. Also fixes a pre-existing typo in
the README quickstart.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a synchronous batch job lifecycle (submit_batch/poll_batch/
retrieve_batch_results) and a free-text build_batch_request() to
BaseModelClient, complementing the existing structured batch builder.
New batch_types module (BatchState/BatchStatus/BatchItemStatus/
BatchResult) normalizes each provider's status enum and per-request
results. Each provider's request-body construction is now shared
between response() and the batch builders to prevent drift; o-family
OpenAI drops temperature at build time.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Submit Gemini batches as an uploaded keyed JSONL file (File API, src=<file>)
instead of inline requests, so results return as documented keyed JSONL and
correlate to each request by key. Verified against google-genai: inline
responses carry no key/metadata, so inline correlation is impossible -- the
inline submit/retrieve paths are removed.

- submit_batch: serialize requests to JSONL, upload, submit src=<file name>
- retrieve_batch_results: file-only keyed-JSONL path
- remove dead _parse_gemini_inline_response; trim _extract_gemini_text to dicts
- tests: assert keyed JSONL upload + src is the file name; drop inline test
- CHANGELOG: note Gemini file-based submission

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Integration test (skipped unless GEMINI_API_KEY/GOOGLE_API_KEY set) that
submits a real batch with distinct ids + unique answers and asserts each
answer lands under the correct custom_id -- the identity check offline mocks
can't make. Deterministic arithmetic prompts + thinking-aware token budget so
it validates correlation, not model instruction-following.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
files.upload(...).name is typed str | None, but batches.create(src=...)
requires a non-None value. Raise a clear RuntimeError if the upload
returns no name instead of passing Optional[str] through.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The pre-commit config ran ruff/black/whitespace but omitted mypy and
pytest, so type and test failures only surfaced in CI -- as the Gemini
batch Optional[str] mypy error did on this branch. Add local hooks that
invoke the project venv to run `mypy src/prkit` and `pytest tests/prkit`,
matching .github/workflows/ci.yml, so both fail at commit time instead.
Scoped via files: filters so they fire only when src/prkit or tests/prkit
is staged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@sherryzyh
sherryzyh merged commit 0bca542 into main Jun 15, 2026
3 checks passed
@sherryzyh
sherryzyh deleted the refactor/model-client-response-api branch June 15, 2026 18:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant