Add OptoPrimeV3 and OPROv3 multimodal optimizers#77
Merged
Conversation
allenanie
force-pushed
the
feature/optoprime-v3
branch
from
94b15bb to
bb5e444
Compare
allenanie
force-pushed
the
feature/llm-backbone
branch
from
9f8153d to
73b0347
Compare
Stacked on the multimodal backbone branch. These optimizers build prompts as multimodal Content (text + images) via the backbone Chat/UserTurn/AssistantTurn primitives and require an mm_beta LLM. - opto/optimizers/optoprime_v3.py: OptoPrimeV3 (subclasses OptoPrime), OptimizerPromptSymbolSet variants, ProblemInstance, and value_to_image_content. - opto/optimizers/opro_v3.py: OPROv3 (subclasses OptoPrimeV3) with a smaller prompt symbol set. - opto/optimizers/__init__.py: export OptoPrimeV3 and OPROv3. - tests/llm_optimizers_tests/test_optoprime_v3.py. Fixes a pre-existing bug in ProblemInstance: content fields passed as plain strings (feedback/context) crashed __repr__/to_content_blocks. Added a __post_init__ that normalizes fields via ContentBlockList.ensure, and made __repr__ include the Context section so it matches to_content_blocks.
Mirror the backbone-branch test gating: real LLM optimizer-step tests now run only when RUN_LIVE_LLM_TESTS=1, so they don't fail against CI's text-only stub.
allenanie
force-pushed
the
feature/optoprime-v3
branch
from
bb5e444 to
ee504a2
Compare
Member
|
LGTM |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Adds the v3 optimizers that build prompts as multimodal
Content(interleaved text + images) using the backboneChat/UserTurn/AssistantTurnprimitives.PR 2 of 2 (stacked). Targets
feature/llm-backbone(#76), which provides the backbone/LLM layer this depends on. Merge #76 first, then this re-targetsexperimental.Changes
opto/optimizers/optoprime_v3.py—OptoPrimeV3(subclassesOptoPrime),OptimizerPromptSymbolSetvariants,ProblemInstance, andvalue_to_image_content. Requires anmm_betaLLM (asserted at construction).opto/optimizers/opro_v3.py—OPROv3(subclassesOptoPrimeV3) with a smaller prompt symbol set.opto/optimizers/__init__.py— exportOptoPrimeV3andOPROv3.tests/llm_optimizers_tests/test_optoprime_v3.py— offline tests pass; live optimizer-step tests are opt-in viaRUN_LIVE_LLM_TESTS=1.Bug fix
ProblemInstanceis a@dataclasswith no normalization, so passingfeedback/contextas plain strings (as the tests and callers do) crashed__repr__/to_content_blockswith'str' object has no attribute 'to_text'. Added a__post_init__that normalizes content fields viaContentBlockList.ensure, and made__repr__include the Context section so it matchesto_content_blocks.Verification
pytest tests/llm_optimizers_tests/test_optoprime_v3.py— 13 pass / 4 skip.OptoPrimeV3/OPROv3instantiate and import against the existingoptoprime.py/optoprime_v2.pyonexperimental(no changes to those files required).