feat: added supervisor mode - #9
Conversation
|
Warning Review limit reached
Next review available in: 44 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughAdds billing, technical, and sales investigators with bounded tool execution; introduces a DBOS supervisor that plans, runs, and synthesizes investigations; and lets WebSocket clients select supervised execution through a task mode. ChangesSupervised investigation execution
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Client
participant server.py
participant supervisor_workflow
participant Investigators
participant OpenAI
Client->>server.py: Submit task with mode
server.py->>supervisor_workflow: Start supervised workflow
supervisor_workflow->>OpenAI: Create investigation plan
supervisor_workflow->>Investigators: Run planned objectives in parallel
Investigators->>OpenAI: Execute bounded tool-assisted investigation
Investigators-->>supervisor_workflow: Return findings or failure
supervisor_workflow->>OpenAI: Synthesize successful findings
OpenAI-->>supervisor_workflow: Return customer reply
supervisor_workflow-->>server.py: Return workflow result
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (2)
harness/supervisor.py (1)
105-105: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueAdd
strict=Truetozip.
resultsis 1:1 withstepshere, so making it explicit is free and guards future edits.🛠️ Suggested change
- for step, result in zip(steps, results): + for step, result in zip(steps, results, strict=True):🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@harness/supervisor.py` at line 105, Update the zip call in the loop over steps and results to pass strict=True, preserving the existing 1:1 pairing while validating that both iterables have equal length.Source: Linters/SAST tools
harness/investigators.py (1)
12-13: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueModel id is duplicated across modules.
MODEL = "gpt-5.6-luna"is declared identically inharness/supervisor.py(Line 12). Pull it fromconfig.settingsso investigator and supervisor can't drift.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@harness/investigators.py` around lines 12 - 13, Replace the local MODEL declaration in investigators.py with the shared model setting from config.settings, matching the approach used by supervisor.py. Update references in the investigator flow to use that imported setting and remove the duplicated literal so both modules remain synchronized.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@harness/investigators.py`:
- Around line 33-40: Correct the inaccurate comment above _TOOL_FNS to reflect
that getCharges is available to the runner. Update the getCharges and
searchKnowledgeBase handlers to safely read required arguments without raising
KeyError for malformed or empty model arguments, so run_investigator can return
the resulting tool error to the model instead of failing the DBOS step.
- Around line 87-104: Update run_investigator to explicitly signal MAX_STEPS
exhaustion when the final response still contains function calls, using a short
error or truncation marker that supervisor_workflow records as subagent.failed
instead of successful empty findings. Also remove per-call timeout arguments
from client.responses.create and configure the timeout on the client or via
client.with_options(timeout=...).
In `@harness/supervisor.py`:
- Around line 98-101: Bound the untrusted steps produced by plan_step before the
asyncio.gather dispatch in the supervisor flow. Cap the number of planned steps
using the existing workflow limit or gate investigate_step calls with a shared
semaphore, ensuring concurrent investigators cannot grow without limit while
preserving the current result collection behavior.
---
Nitpick comments:
In `@harness/investigators.py`:
- Around line 12-13: Replace the local MODEL declaration in investigators.py
with the shared model setting from config.settings, matching the approach used
by supervisor.py. Update references in the investigator flow to use that
imported setting and remove the duplicated literal so both modules remain
synchronized.
In `@harness/supervisor.py`:
- Line 105: Update the zip call in the loop over steps and results to pass
strict=True, preserving the existing 1:1 pairing while validating that both
iterables have equal length.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 25da43bf-54e9-40f7-8538-565b129e492a
📒 Files selected for processing (3)
harness/investigators.pyharness/supervisor.pyserver.py
| calls = [item for item in resp.output if item.type == "function_call"] | ||
| if not calls: | ||
| return resp.output_text | ||
|
|
||
| # A tool fired -> upgrade to the list form so we can feed the result back. | ||
| if isinstance(input_items, str): | ||
| input_items = [{"role": "user", "content": input_items}] | ||
| input_items += [item.model_dump(exclude={"status"}) for item in resp.output] | ||
| for call in calls: | ||
| result = _TOOL_FNS[call.name](json.loads(call.arguments)) | ||
| input_items.append( | ||
| { | ||
| "type": "function_call_output", | ||
| "call_id": call.call_id, | ||
| "output": json.dumps(result), | ||
| } | ||
| ) | ||
| return resp.output_text if resp else "" |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🌐 Web query:
openai python client responses.create timeout parameter
💡 Result:
The OpenAI Python client does not have a timeout parameter directly within the create method (e.g., chat.completions.create) [1][2]. Instead, you must manage timeouts either at the client level or by using the client's with_options method to override settings for a specific request [3][4][5]. Default Configuration By default, the OpenAI Python client has a timeout of 600 seconds (10 minutes) [6][7][5]. How to Configure Timeouts 1. Client-Level Configuration: Set a default timeout for all requests when initializing the OpenAI client [3][8][5]. client = OpenAI(timeout=20.0) # Sets a 20-second timeout for all requests [3][5] 2. Per-Request Configuration: Use the client.with_options method to apply a specific timeout to a single call [3][5]. # Overrides the client default for this specific request response = client.with_options(timeout=5.0).chat.completions.create( model="gpt-4", messages=[{"role": "user", "content": "Hello!"}]) 3. Fine-Grained Control: You can pass an httpx.Timeout object to either the client or with_options for more detailed control over connection, read, write, and pool timeouts [3][7][5]. import httpx client = OpenAI( timeout=httpx.Timeout(60.0, read=5.0, write=10.0, connect=2.0)) Important Notes - If a request exceeds the specified timeout, the client will raise an APITimeoutError [3][8]. - Setting timeout=None will disable timeouts entirely, which may cause requests to hang indefinitely [5]. - Do not attempt to use parameters like request_timeout inside the create method, as this will result in a TypeError [9].
Citations:
- 1: https://github.com/openai/openai-python/blob/main/src/openai/resources/chat/completions/completions.py
- 2: https://github.com/openai/openai-python/blob/main/src/openai/resources/completions.py
- 3: https://github.com/openai/openai-python/blob/main/README.md
- 4: https://github.com/openai/openai-python/blob/2384e276969d5908eebff2ca12bdfcd69b3cfa93/README.md
- 5: https://mintlify.wiki/openai/openai-python/concepts/timeouts
- 6: https://github.com/openai/openai-python/blob/v1.98.0/src/openai/_constants.py
- 7: https://mintlify.wiki/openai/openai-python/api/client
- 8: https://pypi.org/project/openai/1.97.0/
- 9: https://community.openai.com/t/is-request-timeout-parameter-real/377446
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== locate files =="
git ls-files | rg '(^|/)investigators\.py$|(^|/)supervisor\.py$|(^|/)requirements.*\.txt$|(^|/)pyproject\.toml$|uv\.lock$|poetry\.lock$|Pipfile\.lock$' || true
echo
echo "== investigators outline =="
ast-grep outline harness/investigators.py --view expanded || true
echo
echo "== investigators relevant lines =="
cat -n harness/investigators.py | sed -n '1,130p'
echo
echo "== supervisor relevant search ====="
rg -n "completed|failed|finding|investigate|investigator|subagent|synthesi|completion" harness/supervisor.py harness -g '*.py' || trueRepository: inesaranab/agent-harness
Length of output: 10458
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== supervisor full file =="
cat -n harness/supervisor.py | sed -n '1,150p'
echo
echo "== runtime relevant file search =="
cat -n harness/runtime.py | sed -n '1,190p'
echo
echo "== pyproject deps =="
cat pyproject.tomlRepository: inesaranab/agent-harness
Length of output: 13830
Surface investigator step-limit failures instead of returning empty findings.
When run_investigator exits the MAX_STEPS loop, supervisor_workflow treats the normal return as a successful subagent result, so synthesis receives empty findings even though the last response still contained function calls. Raise a short error or return an explicit truncation marker so the supervisor fan-in can record subagent.failed.
Also, avoid per-call timeout arguments on client.responses.create (the OpenAI create API does not accept timeout=); configure client-level timeout or client.with_options(timeout=...) instead.
🧰 Tools
🪛 ast-grep (0.44.1)
[info] 100-100: use jsonify instead of json.dumps for JSON output
Context: json.dumps(result)
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@harness/investigators.py` around lines 87 - 104, Update run_investigator to
explicitly signal MAX_STEPS exhaustion when the final response still contains
function calls, using a short error or truncation marker that
supervisor_workflow records as subagent.failed instead of successful empty
findings. Also remove per-call timeout arguments from client.responses.create
and configure the timeout on the client or via client.with_options(timeout=...).
1 participant
Summary by CodeRabbit