Add trace-based behavioral tests with Monocle Test Tools

[imohammedansari]

Summary

Adds a behavioral test suite that asserts against Open Deep Research's Monocle execution traces: which agent ran, what it was asked, what it returned, and its token and duration cost. Additive only, under tests/monocle/, with no app-code changes.

Why

The repo's existing tests are evaluation harnesses that score final reports over a benchmark with LangSmith and LLM judges. They exercise output quality, not what a real agent run actually did: which agent handled the request, what it returned, and what it cost. This adds that behavioral coverage, so if a later prompt, model, or route change alters routing, output, or token cost, a test catches it.

How it works

It uses Monocle Test Tools. The offline tests load a recorded trace from file with with_trace_source("file", trace_path=...), which is fast, needs no keys, and is deterministic. They then assert with the fluent API: called_agent, contains_output / contains_any_output, under_token_limit, under_duration. The live test drives the real agent end to end and asserts on structure and budget only.

The point of loading from file is that the same assertions run in both modes. You pin a check against a recorded trace while iterating (fast, no keys, deterministic), then run the identical check against a live run for real regression coverage. The offline tests and the live test here share the same fluent assertions for exactly this reason.

Open Deep Research runs its search inside the model call (OpenAI-native web search), so its traces carry no agentic.tool spans; behavior is validated through the agent invocation, the output, and the workflow budget.

Changes (all under tests/monocle/)

• test_opendeepresearch.py: 4 offline file-loaded tests, one per curated question (Earth's seasons, renewable vs nonrenewable energy, ocean tides, TCP vs UDP), plus 1 live test (TCP vs UDP, guarded by OPENAI_API_KEY).
• conftest.py: Monocle setup, .env loading, run_opendeepresearch().
• traces/: 4 recorded traces, one per question.
• requirements.txt, README.md.