fix: show updated agent answer before each feedback prompt when verbose=False - #6386
fix: show updated agent answer before each feedback prompt when verbose=False#6386Julien-ser wants to merge 4 commits into
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Team Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (2)
Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review. 📝 WalkthroughWalkthrough
ChangesVerbosity-aware agent answer display in human feedback
Suggested reviewers: Merge Risk: ⚪ Minimal · up to Human-feedback prompts now show the agent answer when verbose output is disabled, including repeated feedback cycles. The stated regression and integration coverage supports merge readiness. 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
lib/crewai/src/crewai/core/providers/human_input.py (1)
342-386: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winRender the supplied answer before branching on training mode.
handle_feedback*now passesanswerfor non-verbose prompts, but_prompt_input*only prints it in the non-training branch. In training mode, the answer is dropped and the user is asked to judge result quality without seeing the result.Proposed fix
try: + if answer is not None: + output_str = HumanInputProvider._get_output_string(answer) + result_content = Text() + result_content.append(output_str, style="green") + formatter.console.print( + Panel( + result_content, + title="✅ Agent Final Answer", + border_style="green", + padding=(1, 2), + ) + ) + if crew and getattr(crew, "_train", False): prompt_text = ( "TRAINING MODE: Provide feedback to improve the agent's performance.\n\n" "This will be used to train better versions of the agent.\n" "Please provide detailed feedback about the result quality and reasoning process." ) title = "🎓 Training Feedback Required" else: - if answer is not None: - output_str = HumanInputProvider._get_output_string(answer) - result_content = Text() - result_content.append(output_str, style="green") - formatter.console.print( - Panel( - result_content, - title="✅ Agent Final Answer", - border_style="green", - padding=(1, 2), - ) - ) prompt_text = (Apply the same move in
_prompt_input_async.Also applies to: 414-457
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@lib/crewai/src/crewai/core/providers/human_input.py` around lines 342 - 386, The answer display in _prompt_input is still gated by the training-mode branch, so handle_feedback callers in training mode never see the agent result before giving feedback. Move the answer-rendering logic (the HumanInputProvider._get_output_string and Panel print) to run before the training/non-training branch, and apply the same fix in _prompt_input_async so both paths show the supplied answer consistently.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@lib/crewai/src/crewai/core/providers/human_input.py`:
- Around line 342-386: The answer display in _prompt_input is still gated by the
training-mode branch, so handle_feedback callers in training mode never see the
agent result before giving feedback. Move the answer-rendering logic (the
HumanInputProvider._get_output_string and Panel print) to run before the
training/non-training branch, and apply the same fix in _prompt_input_async so
both paths show the supplied answer consistently.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: a50ddd51-8cb9-4a08-b157-30aa0fda95a7
📒 Files selected for processing (1)
lib/crewai/src/crewai/core/providers/human_input.py
ErenAta16
left a comment
ErenAta16
left a comment
There was a problem hiding this comment.
Reviewed the diff against human_input.py on main (confirmed this is the only file touched, +68/-8, via the PR's files API - my local shallow clone briefly showed a much larger diff against a stale cached main, but that was a clone artifact on my end, not something in this PR).
This fixes a real usability gap (#6072): when verbose=False, the human-in-the-loop feedback loop re-prompts the operator for input after each iteration without ever showing what the agent's updated answer actually was, so the operator is asked to give feedback "blind" on a response they can't see. The fix threads a show_answer/answer flag through _prompt_input (both sync and async paths) that's derived from agent.verbose or crew.verbose, and prints the current answer immediately before the feedback prompt only when verbose is off (when verbose is on, the answer is already visible from the normal execution log, so it correctly avoids double-printing).
Checked that both the sync (_handle_regular_feedback) and async (_handle_regular_feedback_async) iteration loops were updated consistently - they were, including the initial pre-loop prompt and every iteration inside the loop. No existing test file covers this provider directly, but the logic change is small, symmetric across both code paths, and matches the linked issue's exact complaint. LGTM.
|
This PR is stale because it has been open for 45 days with no activity. |
5ed84ef to
51cfc09
Compare
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
|
Updated: rebased onto current The gap was real. The answer panel this PR adds was rendered only in the non-training branch of Hoisted the rendering above the training/normal branch in both the sync and async paths, and pulled it into a Added a parametrized regression test over After the fix, both pass, along with |
There was a problem hiding this comment.
🟡 Changes recommended
The new test assertion is likely brittle across Rich versions (title type), and the new answer-panel styling is inconsistent with existing final-answer rendering conventions.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR fixes the human_input=True feedback loop UX when verbose=False by ensuring the agent’s latest answer is re-rendered before each subsequent feedback prompt, so operators can see what changed between iterations.
Changes:
- Plumbs a
show_answerflag through the regular feedback loop to re-display the updatedAgentFinishoutput before each new feedback prompt when non-verbose. - Adds a helper to render the “✅ Agent Final Answer” Rich panel from the human-input provider path.
- Adds a unit test asserting the answer panel is rendered by
_prompt_inputin both normal and training modes.
File summaries
| File | Description |
|---|---|
| lib/crewai/src/crewai/core/providers/human_input.py | Adds answer-panel rendering and propagates show_answer so non-verbose human feedback prompts always show the latest answer. |
| lib/crewai/tests/agents/test_agent.py | Adds coverage to ensure _prompt_input renders the answer panel in both training and non-training prompts. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 2
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| rendered = [ | ||
| call.args[0] | ||
| for call in formatter.console.print.call_args_list | ||
| if getattr(call.args[0], "title", None) == "✅ Agent Final Answer" | ||
| ] |
| result_content = Text() | ||
| result_content.append( | ||
| HumanInputProvider._get_output_string(answer), style="green" | ||
| ) |
ErenAta16
left a comment
ErenAta16
left a comment
There was a problem hiding this comment.
Read the updated branch, and the training-mode gap is closed properly rather than
patched over. Two things I checked before saying so.
I went looking for the obvious follow-on hole, that the training path gets the
answer on its first prompt and then loses it on subsequent ones, the way the
regular path would without show_answer. It does not exist:
_handle_training_feedback and _handle_training_feedback_async take the
feedback once and go straight to return improved_answer, with no iteration
loop. So the single prompt they make is the only one, and it now carries the
answer through _prompt_input(context.crew, answer=...). Nothing further is
needed on that side, which is worth stating explicitly since the asymmetry in the
diff (regular gets show_answer, training does not) reads like an oversight and
is not one.
The is_verbose guard is the part I would not have thought of and it is the right
call:
is_verbose = context.agent.verbose or bool(context.crew and getattr(context.crew, "verbose", False))
feedback = self._prompt_input(context.crew, answer=None if is_verbose else formatted_answer)Rendering the panel unconditionally would have printed the answer twice for
anyone running verbose, which is the usual outcome when a fix like this is done
narrowly. Deriving it from both the agent and the crew, with getattr for the
crew being absent, covers the combinations without assuming a crew exists.
The sync and async paths stay in step through all of it: same guard, same
show_answer=not is_verbose, same threading into the loop. That symmetry is the
thing most likely to rot later, so having both edited together in one diff is
worth more than it looks.
show_answer: bool = False defaulting off is also the safe direction for the two
_handle_regular_feedback entry points, since any other caller keeps today's
behaviour rather than inheriting a new panel.
Nothing blocking from me. The 217-commit rebase was the right call too; the branch
was old enough that a reviewer could not otherwise tell which failures were yours.
When human_input=True but verbose=False, the feedback panel said 'Provide feedback on the Final Result above' but the result was never displayed — the AgentLogsExecutionEvent is verbose-gated while the human-input gate fires unconditionally. Fix: handle_feedback / handle_feedback_async detect when the executor is non-verbose and pass the AgentFinish to _prompt_input / _prompt_input_async. Both methods now print a green 'Agent Final Answer' panel before the feedback prompt when the caller supplies the answer, so the operator always sees the result they are asked to review. Training mode is unaffected — the answer panel is skipped in that path. Fixes crewAIInc#6072
…verbose=False Addresses CodeRabbit feedback from PR crewAIInc#6073: when iterating through multiple rounds of human feedback in non-verbose mode, only the initial answer was shown. On subsequent rounds, the user was prompted again without seeing the newly generated response. Pass show_answer=not is_verbose to _handle_regular_feedback / _handle_regular_feedback_async so that _prompt_input re-displays the current answer before each feedback prompt whenever verbose output is suppressed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The answer panel added by this PR was rendered only in the non-training branch of _prompt_input / _prompt_input_async, so with crew._train set the operator was still asked to judge output they could not see -- the same gap this PR set out to close, just on the training path. The training prompt explicitly asks for "detailed feedback about the result quality and reasoning process", so the result has to be on screen there as much as in normal mode. Hoist the rendering above the training/normal branch in both the sync and async paths, and extract it into a _render_answer_panel helper rather than repeat the same panel construction a third and fourth time. Adds a parametrized regression test covering _train=True and _train=False; the _train=True case fails without this change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JRC5ACRhgrzdagKjPUbYEH
…itle assertion) - Use bright_green for the answer text, matching the existing final-answer rendering in console_formatter.py (lines 622 and 1220). The panel border stays green as it is there, so the same "✅ Agent Final Answer" panel now looks identical in the verbose and non-verbose flows. - Coerce the Rich panel title with str() before comparing. Rich stores Panel.title as either a str or a Text depending on version and config, so the equality check could silently match nothing on some versions and leave the test passing vacuously. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xx1JiCDAZcBC71R2HN7Eyx
51cfc09 to
e023a14
Compare
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
Fixes #6072.
Problem
When
human_input=Trueis set andverbose=False, the agent's initial answer was displayed before the first feedback prompt, but on every subsequent iteration the user was prompted again without seeing what the agent had produced in the latest round. The operator had no way to know what changed between feedback cycles.Fix
Pass
show_answer=not is_verbosethrough to_handle_regular_feedback/_handle_regular_feedback_async. On each loop iteration, after_invoke_loop()/_ainvoke_loop()produces a new answer,_prompt_inputre-renders the updated answer in a Rich panel before asking for the next round of input.Changes
lib/crewai/src/crewai/core/providers/human_input.pyhandle_feedback/handle_feedback_async— passshow_answer=not is_verbose_handle_regular_feedback/_handle_regular_feedback_async— acceptshow_answer: bool = False; on subsequent iterations, passanswer=answer if show_answer else Noneto the prompt helpers_prompt_input/_prompt_input_async— already acceptanswerand render the panel (added in the initial commit)Rebased from
This branch was rebased cleanly onto current
mainfrom the original work in #6073.AI assistance
This fix was developed with the help of Claude Code (
🤖 Generated with Claude Code). All changed lines have been reviewed and understood.Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com