[giladsamuels-cognition]

Summary

Test-only PR: 15 new test_*.py files under tests/ (mirroring src/agentlab/). They target the modules with the lowest coverage that can be tested without outside services. No production code changes. Every external call is mocked: OpenAI/Anthropic clients, requests, litellm get_model_info/completion_cost, HF AutoTokenizer, os.system git calls, and time.sleep. No new secrets, services, or dependencies are needed (pytest-cov was only used to measure coverage).

Coverage (pytest -n 8 -m 'not pricy' tests/ --cov=agentlab, MiniWoB running): TOTAL 32% → 49%

module	before	after
agents/agent_utils.py	0%	100%
agents/most_basic_agent/most_basic_agent.py	0%	100%
agents/generic_agent/reproducibility_agent.py	0%	76%
analyze/archive_studies.py	0%	76%
analyze/episode_to_html.py	0%	90%
analyze/inspect_results.py	31%	55%
benchmarks/abstract_env.py	56%	100%
benchmarks/setup_benchmark.py	not imported	100%
benchmarks/osworld_axtree_preprocessing.py	not imported	87%
experiments/exp_utils.py	66%	99%
llm/chat_api.py	40%	88%
llm/huggingface_utils.py	22%	100%
llm/llm_utils.py	42%	70%
llm/prompt_templates.py	47%	100%
llm/tracking.py	36%	96%

Full suite: 286 passed / 8 failed / 2 errors, compared with 50 passed / 8 failed / 2 errors on main in the same environment. The failing and erroring tests are exactly the same set before and after this PR:

• tests/llm/test_litellm_api.py and tests/llm/test_response_api.py fail at import: response_api.py imports anthropic.types.Completion, which the installed anthropic==2.3.0 doesn't have.
• 5 tests in tests/agents/test_agent.py and 3 test_launch_system_* tests in tests/experiments/test_launch_exp.py fail in the MiniWoB/browser runs.

Pre-existing issues found while writing tests. These are documented, not fixed:

• AnthropicChatModel.__call__ checks hasattr(TRACKER, "instance") and then calls the tracker. After any set_tracker() block exits, instance is None, so the call raises TypeError and the model goes into its 60 s retry loop. The tests therefore call it inside set_tracker().
• archive_studies.search_for_reasons_to_archive raises ZeroDivisionError when a summary has n_completed == "0/0".
• inspect_results.map_err_key drops the last character of messages that have no === logs === marker, because str.find returns -1.

Devin-Org: engineering

Link to Devin session: https://devin-gtm.devinenterprise.com/sessions/0876f728718b48e6af96237770d6fd9b
Open in Devin Desktop: https://devin-gtm.devinenterprise.com/desktop/session/0876f728718b48e6af96237770d6fd9b?variant=devin
Requested by: @giladsamuels-cognition

---

Devin Review

Status	Commit
⚪ Not started	—

Run Devin Review

Enable auto-review to review your pull requests automatically.

---

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[devin-ai-integration]

🔍 Devin Review: 2 flags

Not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)