Skip to content

feat: add Tavily search provider option to Agentic-RAG-Pipeline#35

Open
tavily-integrations wants to merge 1 commit into
hoangsonww:masterfrom
Tavily-FDE:feat/tavily-migration/agentic-rag-google-cse
Open

feat: add Tavily search provider option to Agentic-RAG-Pipeline#35
tavily-integrations wants to merge 1 commit into
hoangsonww:masterfrom
Tavily-FDE:feat/tavily-migration/agentic-rag-google-cse

Conversation

@tavily-integrations

Copy link
Copy Markdown

Summary

Adds Tavily as a configurable web search provider alongside the existing Google Custom Search Engine (CSE) in the Agentic-RAG-Pipeline sub-app. This is an additive change — Google CSE remains the default and is fully intact.

What changed

  • Agentic-RAG-Pipeline/core/tools.py: Added TavilyWebSearch class using tavily-python SDK, exposing the same search(q, num) interface and {title, url, snippet} return format as the existing WebSearch class.
  • Agentic-RAG-Pipeline/app.py: Extended provider selection logic to check SEARCH_PROVIDER env var. When set to "tavily" and TAVILY_API_KEY is present, uses TavilyWebSearch; otherwise falls through to Google CSE as before.
  • Agentic-RAG-Pipeline/.env.example: Added SEARCH_PROVIDER and TAVILY_API_KEY entries alongside existing CSE vars.
  • Agentic-RAG-Pipeline/requirements.txt: Added tavily-python>=0.3.0.

How to use

Set these environment variables to switch to Tavily:

SEARCH_PROVIDER=tavily
TAVILY_API_KEY=tvly-your-key

To keep using Google CSE (the default), no changes are needed.

Notes for reviewers

  • Google CSE code paths, env vars (CSE_API_KEY, CSE_ENGINE_ID), and dependencies are untouched.
  • TavilyWebSearch maps Tavily's content field to snippet for interface compatibility.

Automated Review

  • Passed after 1 attempt(s)
  • Final review: The migration is correct and complete. TavilyWebSearch properly mirrors the WebSearch interface, uses the right Tavily SDK call (TavilyClient.search(query=..., max_results=...)) with correct response field mapping (contentsnippet), and the synchronous client is appropriate for this non-async app. app.py correctly implements the SEARCH_PROVIDER env-var gate while preserving the Google CSE fallback path. All four planned files are updated. Two minor issues: the tavily-python pin lacks an upper bound (inconsistent with all other deps), and there is no warning log when SEARCH_PROVIDER=tavily is set but TAVILY_API_KEY is absent (silent fallthrough to CSE/disabled).

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces Tavily Search as an optional web search provider alongside Google CSE. The changes include a new TavilyWebSearch wrapper class in core/tools.py, updated environment variable examples, and logic in app.py to toggle between search providers. Feedback suggests improving error reporting when a specific search provider is requested but its API key is missing, and adding an upper version bound to the tavily-python dependency in requirements.txt to ensure consistency and long-term stability.

Comment on lines +39 to +46
if search_provider == "tavily" and tavily_key:
web = TavilyWebSearch(api_key=tavily_key)
print("[web] Tavily Search enabled.")
elif cse_key and cse_engine:
web = WebSearch(api_key=cse_key, engine_id=cse_engine)
print("[web] Google Programmable Search enabled.")
else:
print("[web] Web search disabled (set CSE_API_KEY & CSE_ENGINE_ID to enable).")
print("[web] Web search disabled (set CSE_API_KEY & CSE_ENGINE_ID, or TAVILY_API_KEY to enable).")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

When SEARCH_PROVIDER is explicitly set to 'tavily' but TAVILY_API_KEY is missing, the application currently falls back to Google CSE (if configured) or disables search entirely without a specific warning. It would be more helpful to the user to explicitly report that the requested provider is misconfigured.

Suggested change
if search_provider == "tavily" and tavily_key:
web = TavilyWebSearch(api_key=tavily_key)
print("[web] Tavily Search enabled.")
elif cse_key and cse_engine:
web = WebSearch(api_key=cse_key, engine_id=cse_engine)
print("[web] Google Programmable Search enabled.")
else:
print("[web] Web search disabled (set CSE_API_KEY & CSE_ENGINE_ID to enable).")
print("[web] Web search disabled (set CSE_API_KEY & CSE_ENGINE_ID, or TAVILY_API_KEY to enable).")
if search_provider == "tavily":
if tavily_key:
web = TavilyWebSearch(api_key=tavily_key)
print("[web] Tavily Search enabled.")
else:
print("[web] Tavily Search requested but TAVILY_API_KEY is missing.")
elif cse_key and cse_engine:
web = WebSearch(api_key=cse_key, engine_id=cse_engine)
print("[web] Google Programmable Search enabled.")
else:
print("[web] Web search disabled (set CSE_API_KEY & CSE_ENGINE_ID, or TAVILY_API_KEY to enable). ")

pydantic>=2.7.0,<3.0.0
python-dotenv>=1.0.1,<2.0.0
numpy>=1.26.4,<3.0.0
tavily-python>=0.3.0

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The tavily-python dependency is missing an upper bound, which is inconsistent with the other dependencies in this file. Adding an upper bound (e.g., <1.0.0) helps ensure stability by preventing automatic upgrades to potentially breaking major versions.

tavily-python>=0.3.0,<1.0.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant