Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,11 @@ LANGSMITH_API_KEY=
LANGSMITH_PROJECT=
LANGSMITH_TRACING=

# Optional Monocle observability (opt-in; requires the extra: pip install "open_deep_research[monocle]").
MONOCLE_TRACING=false # set to true to enable
MONOCLE_EXPORTERS=file # file, console, okahu, s3, blob, gcs (default: file)
OKAHU_API_KEY=okh_xxxxxxxx # required only for the `okahu` exporter

# Only necessary for Open Agent Platform
SUPABASE_KEY=
SUPABASE_URL=
Expand Down
18 changes: 18 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,24 @@ Open Deep Research supports a wide range of search tools. By default it uses the

See the fields in the [configuration.py](https://github.com/langchain-ai/open_deep_research/blob/main/src/open_deep_research/configuration.py) for various other settings to customize the behavior of Open Deep Research.

#### Monocle Tracing

Open Deep Research also supports [Monocle](https://github.com/monocle2ai/monocle), an OpenTelemetry-based tracer for agentic applications. It records each run end-to-end: LLM calls, agent steps, and tool invocations, with inputs, outputs, timings, and token counts.

Install the optional extra and add the following to your `.env` file:

```bash
pip install "open_deep_research[monocle]"
```

```bash
MONOCLE_TRACING=true
MONOCLE_EXPORTERS=file # file, console, okahu, s3, blob, gcs (default: file)
OKAHU_API_KEY=okh_xxxxxxxx # required only for the `okahu` exporter
```

Each run writes one trace file to `.monocle/`; open it in the [Monocle VS Code extension](https://marketplace.visualstudio.com/items?itemName=OkahuAI.monocle-apptrace). Connect to [Okahu](https://www.okahu.ai) to analyze traces across runs (via the `okahu` exporter).

### 📊 Evaluation

Open Deep Research is configured for evaluation with [Deep Research Bench](https://huggingface.co/spaces/Ayanami0730/DeepResearch-Leaderboard). This benchmark has 100 PhD-level research tasks (50 English, 50 Chinese), crafted by domain experts across 22 fields (e.g., Science & Tech, Business & Finance) to mirror real-world deep-research needs. It has 2 evaluation metrics, but the leaderboard is based on the RACE score. This uses LLM-as-a-judge (Gemini) to evaluate research reports against a golden set of reports compiled by experts across a set of metrics.
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ dependencies = [

[project.optional-dependencies]
dev = ["mypy>=1.11.1", "ruff>=0.6.1"]
monocle = ["monocle_apptrace"]

[build-system]
requires = ["setuptools>=73.0.0", "wheel"]
Expand Down
28 changes: 28 additions & 0 deletions src/open_deep_research/deep_researcher.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
"""Main LangGraph implementation for the Deep Research agent."""

import asyncio
import os
from typing import Literal

from langchain.chat_models import init_chat_model
Expand Down Expand Up @@ -52,6 +53,33 @@
think_tool,
)

def _setup_monocle_tracing() -> None:
"""Optional Monocle observability, gated by MONOCLE_TRACING (see .env.example). No-op when off."""
if os.getenv("MONOCLE_TRACING", "").strip().lower() not in ("1", "true", "yes", "on"):
return
allowed = ("file", "console", "okahu", "s3", "blob", "gcs")
# App owns MONOCLE_EXPORTERS: validate here so a typo fails fast, then forward as-is.
exporters = os.getenv("MONOCLE_EXPORTERS", "").strip() or "file"
selected = [e.strip() for e in exporters.split(",") if e.strip()]
unknown = [e for e in selected if e not in allowed]
if unknown:
raise ValueError(
f"MONOCLE_EXPORTERS has unknown exporter(s): {', '.join(unknown)}. Allowed: {', '.join(allowed)}."
)
if "okahu" in selected and not os.getenv("OKAHU_API_KEY"):
raise ValueError("Monocle 'okahu' exporter is selected but OKAHU_API_KEY is not set.")
try:
from monocle_apptrace import setup_monocle_telemetry
except ImportError as exc:
raise RuntimeError(
"MONOCLE_TRACING is enabled but monocle_apptrace is not installed. "
'Install the \'monocle\' extra: pip install "open_deep_research[monocle]".'
) from exc
setup_monocle_telemetry(workflow_name="open-deep-research", monocle_exporters_list=exporters)


_setup_monocle_tracing()

# Initialize a configurable model that we will use throughout the agent
configurable_model = init_chat_model(
configurable_fields=("model", "max_tokens", "api_key"),
Expand Down