Rename model-client chat() → response(), add solve_physics_problem() and Batch API support - #2
Merged
Merged
Conversation
Align the low-level model-client call with OpenAI's responses.create: rename chat() -> response(), rename the prompt parameter user_prompt -> input, and add a keyword-only `instructions` (system prompt) parameter. When `instructions` is omitted every provider except OpenAI falls back to a short DEFAULT_INSTRUCTIONS system prompt (resolved via the new _resolve_instructions helper, which OpenAI overrides to stay input-only); an explicit empty string suppresses it. Each provider injects the system prompt at its native site (Anthropic top-level `system`, OpenAI `instructions`, openai-compatible/Ollama system message, Gemini `system_instruction`). chat_structured threads `instructions` through; a deprecated chat() alias warns and forwards to response(). Add a high-level BaseModelClient.solve_physics_problem() that dispatches on the input type: a plain str question, a PhysicsProblem (formatted into a prompt plus its image_path images), or a PhysicsQuestionSemantics (NotImplementedError TODO, lazy-imported to keep core off semantics at import time); any other type raises TypeError. Output is gated by the new PhysicsOutputMode enum (only ANSWER_TEXT implemented). Add core-only modes.py and prompts.py; the semantics _format_problem now reuses format_problem_context for its shared header. Centralize the annotation worker's physics-expert system prompt via `instructions` and update the calls.py retry path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Switch the model-client examples in the cookbooks, docs/CORE.md, README, and RELEASE_NOTES to the new response(input=...) call, document the `instructions` parameter and DEFAULT_INSTRUCTIONS fallback, and add a solve_physics_problem() usage section. Also fixes a pre-existing typo in the README quickstart. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a synchronous batch job lifecycle (submit_batch/poll_batch/ retrieve_batch_results) and a free-text build_batch_request() to BaseModelClient, complementing the existing structured batch builder. New batch_types module (BatchState/BatchStatus/BatchItemStatus/ BatchResult) normalizes each provider's status enum and per-request results. Each provider's request-body construction is now shared between response() and the batch builders to prevent drift; o-family OpenAI drops temperature at build time. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Submit Gemini batches as an uploaded keyed JSONL file (File API, src=<file>) instead of inline requests, so results return as documented keyed JSONL and correlate to each request by key. Verified against google-genai: inline responses carry no key/metadata, so inline correlation is impossible -- the inline submit/retrieve paths are removed. - submit_batch: serialize requests to JSONL, upload, submit src=<file name> - retrieve_batch_results: file-only keyed-JSONL path - remove dead _parse_gemini_inline_response; trim _extract_gemini_text to dicts - tests: assert keyed JSONL upload + src is the file name; drop inline test - CHANGELOG: note Gemini file-based submission Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Integration test (skipped unless GEMINI_API_KEY/GOOGLE_API_KEY set) that submits a real batch with distinct ids + unique answers and asserts each answer lands under the correct custom_id -- the identity check offline mocks can't make. Deterministic arithmetic prompts + thinking-aware token budget so it validates correlation, not model instruction-following. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
files.upload(...).name is typed str | None, but batches.create(src=...) requires a non-None value. Raise a clear RuntimeError if the upload returns no name instead of passing Optional[str] through. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The pre-commit config ran ruff/black/whitespace but omitted mypy and pytest, so type and test failures only surfaced in CI -- as the Gemini batch Optional[str] mypy error did on this branch. Add local hooks that invoke the project venv to run `mypy src/prkit` and `pytest tests/prkit`, matching .github/workflows/ci.yml, so both fail at commit time instead. Scoped via files: filters so they fire only when src/prkit or tests/prkit is staged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This branch reworks the model-client layer in two substantive ways and ships a formatting cleanup:
chat()→response()and add a high-levelsolve_physics_problem()entry point, aligning the client surface with OpenAI'sresponses.create.6 commits · 44 files · +2,075 / −374. Branch is a clean fast-forward over
main(no divergence).What changed
1. API rename + high-level entry point (
111d25c,dff9d01)chat()→response(); prompt paramuser_prompt→input; new keyword-onlyinstructions(system prompt).instructionsis omitted, every provider except OpenAI falls back to a shortDEFAULT_INSTRUCTIONSsystem prompt (via_resolve_instructions; OpenAI overrides to stay input-only). An explicit empty string suppresses it.system, OpenAIinstructions, openai-compatible/Ollama system message, Geminisystem_instruction.BaseModelClient.solve_physics_problem()dispatches on input type:strquestion,PhysicsProblem(formatted prompt +image_pathimages), orPhysicsQuestionSemantics(NotImplementedErrorTODO, lazily imported to keep core off semantics at import time). Output gated by the newPhysicsOutputModeenum (onlyANSWER_TEXTimplemented).modes.pyandprompts.py; annotation worker's physics-expert prompt is centralized viainstructions.docs/CORE.md,README, andRELEASE_NOTESupdated to the new call.2. Batch API support (
2563141,9e719e1,b7e7052)BaseModelClientgains a synchronous batch lifecycle:submit_batch→poll_batch→retrieve_batch_results, plus a free-textbuild_batch_request(...)mirroringresponse()(complements the existing structuredbuild_batch_structured_request).prkit.core.model_clients.batch_types:BatchState,BatchStatus,BatchItemStatus,BatchResultnormalize each provider's status enum and per-request results.response()and the batch builders (_build_responses_body/_build_messages_params) to prevent drift. OpenAI o-family models droptemperatureat build time. Unsupported providers raiseNotImplementedError.src=<file>) rather than inline requests. Inline Gemini responses carry no per-request key and cannot be correlated; the keyed-JSONL path returns{"key": ..., "response": {...}}that maps reliably to each request. Dead inline submit/retrieve paths removed.GEMINI_API_KEY/GOOGLE_API_KEYset) submitting a real batch with distinct ids + unique answers, asserting each answer lands under the correctcustom_id— the identity check offline mocks can't make.3. Formatting (
b6504a5)blackapplied toanthropic.py,openai.py, andtests/uq/conftest.py(pre-existing non-formatted files; CI runsblack --check).New files
src/prkit/core/model_clients/batch_types.py,modes.py,prompts.pytests/prkit/core/model_clients/test_batch.py,test_batch_live.py,test_prompts.pyTesting
black --check src tests→ clean (242 files).pytest tests/ -x -q→ 1378 passed, 1 skipped, 84% coverage.🤖 Generated with Claude Code