Conversation
…string recovery_strategy _fix_common_issues() called .lower() on whatever the model emitted for recovery_strategy. Weak models sometimes return a list or dict there, so the AttributeError escaped the best-effort fix path and took down the whole Workforce call instead of falling back. Non-string values are now dropped so the schema default (None) applies.
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
| # and let schema validation fail into the caller's fallback | ||
| # values, same as an unfixable string value. | ||
| if isinstance(strategy, str): | ||
| strategy = strategy.lower() |
There was a problem hiding this comment.
I ran this against dfcf5194 in a clean container (python:3.11-slim, pip install -e . 'mcp<2.0.0').
The isinstance guard looks right, and it matches what the TaskAssignResult branch already does at :374. The new test is the part I would look at again: test_string_recovery_strategy_normalization_unchanged passes with the whole TaskAnalysisResult branch deleted, so it does not pin the normalization it is named for. 5 passed at dfcf5194, 5 passed with lines 384-407 removed.
That is not something this PR introduced. strategy is a local, and it reaches fixed_data only inside the partial-match loop at :404, so when strategy.lower() is already valid nothing is written back and the original casing goes to validation:
input _fix returns result path taken
RETRY 'RETRY' retry _fix_common_issues -> default instance
rety 'rety' retry _fix_common_issues -> default instance
replan 'replan' replan validated on first try
REPLAN 'REPLAN' retry _fix_common_issues -> default instance
Decompose 'Decompose' retry _fix_common_issues -> default instance
So REPLAN and Decompose fail validation and fall through to the default instance, which is RETRY. All three cases in the test miss this: RETRY and rety land on that same default and RETRY is what it returns, and replan is already valid so it never enters _fix_common_issues. 'retry'.startswith('rety') is False, so rety is not a partial match either.
One line inside your isinstance block covers it:
strategy = strategy.lower()
fixed_data['recovery_strategy'] = strategyI applied that and re-ran: REPLAN gives replan, Decompose gives decompose, and your 5 tests still pass.
Not checked: the async paths, and the rest of the workforce suite.
|
Good catch on all three points — thanks for running it in a clean container. Confirmed on my side: at Tests reworked to pin paths instead of outcomes: On async paths: |
|
Thanks, that matches what I measured. 397c28f puts the write-back exactly where the lowered value was being dropped, and asserting |
Related Issue
Closes #4318
Description
_fix_common_issuescalls.lower()onTaskAnalysisResult.recovery_strategyunconditionally. When a weak model emits a non-string value for that field (e.g."recovery_strategy": ["retry"]), the best-effort fix path itself raisesAttributeError, which escapesparse_structured_response(onlyValidationErroris caught) and crashes the Workforce task-analysis flow instead of usingfallback_values.This PR guards the normalization with an
isinstance(strategy, str)check. Non-string values are left untouched, so validation fails into the existing fallback chain (callerfallback_values→ default instance) — exactly how unfixable string values already behave. Failure analysis keeps its documented default-to-RETRY behavior, and string normalization (lowercase + partial match) is unchanged.What is the purpose of this pull request?
Testing evidence
New regression tests in
test/workforce/test_structured_output_handler.py(fail onmasterwithAttributeError, pass with this fix):Existing suite unaffected: the failures in
test/workforce/test_workforce.pyon my machine are pre-existing missing-API-key errors, identical before and after this patch (11 passed / 7 failed in both cases).Checklist
pyproject.tomland runuv lock