Skip to content

feat(examples): add web_search_chatbot example (#259)#287

Open
Diyaaa-12 wants to merge 5 commits into
Mr-DooSun:mainfrom
Diyaaa-12:feat/259-web-search-chatbot
Open

feat(examples): add web_search_chatbot example (#259)#287
Diyaaa-12 wants to merge 5 commits into
Mr-DooSun:mainfrom
Diyaaa-12:feat/259-web-search-chatbot

Conversation

@Diyaaa-12

@Diyaaa-12 Diyaaa-12 commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Adds a new examples/web_search_chatbot/ following the simple_chatbot layout, with a PydanticAI agent using duckduckgo_search_tool() for real web search and a keyless StubChatbot fallback.

  • DI selector keys off settings.llm_model_name alone, matching the other chatbot examples (no new WEB_SEARCH_ENABLED flag).
  • Adds the pydantic-ai-duckduckgo extra to pyproject.toml.
  • Unit tests cover stub flow, structural real-agent flow (offline via TestModel(call_tools=[])), and tool-invocation (offline via FunctionModel + a monkeypatched fake search tool).
  • Adds a copy-flow smoke ExampleCase so the example is verified to boot post-copy under make quickstart.

Related Issue

Change Summary

  • New examples/web_search_chatbot/ example: PydanticAI Agent + duckduckgo_search_tool() for real web search, with a keyless StubChatbot fallback.
  • WebSearchChatbotContainer / bootstrap_web_search_chatbot_domain added following the auto-discovery naming convention; all other files keep the generic chatbot_* names.
  • Added pydantic-ai-duckduckgo extra to pyproject.toml.
  • Added unit tests (tests/unit/web_search_chatbot/) and a copy-flow smoke ExampleCase (tests/integration/examples/test_copyflow_smoke.py).
  • Added README.md for the example, following the chatbot_with_memory README format.
  • Reverted an earlier ci.yml change that installed the pydantic-ai-duckduckgo extra in the test job — it un-skipped two pre-existing latent test failures (tracked and fixed separately in tests: latent failures in envs with pydantic-ai installed (embedding openai guard + classification settings monkeypatch) #288).

Type of Change

  • feat: New feature
  • fix: Bug fix
  • refactor: Code restructuring
  • docs: Documentation
  • chore: Build/tooling
  • test: Tests
  • ci: CI/CD
  • perf: Performance
  • style: Code style

Checklist

  • Architecture rules followed (no Domain -> Infrastructure imports)
  • Tests pass
  • Linting passes (ruff check src/)

How to Test

pytest tests/unit/web_search_chatbot/ -v
pytest tests/integration/examples/test_copyflow_smoke.py -v
ruff check .
ruff format --check .

Self-Structured Review Evidence

Reviewed against docs/ai/shared/skills/review-pr.md § Self-Structured Review Checklist and docs/ai/shared/security-checklist.md.

Coverage (OK/SKIP):

  • F/G/H/I reasoning guards — OK, re-verified file:line and test output from live tool calls this session, not memory.
  • Contract shape — OK, no OpenAPI/frontend-handoff surface touched.
  • SEC — Auth: POST /v1/chat has no auth dependency — OK, matches the existing unauthenticated simple_chatbot/chatbot_with_memory pattern this repo already ships, not a new gap introduced here.
  • SEC — LLM API Security §12 credential management, provider/model validation, structured output validation — OK (unchanged from simple_chatbot, no new credential paths).
  • REG/ARCH §5 — OK, unit tests + copy-flow smoke ExampleCase added for the new surface.
  • Dependencies/Configuration §5 — OK, no hardcoded secrets, pydantic-ai-duckduckgo is a scoped optional extra.

Findings (OPEN):

  • SEC-1 [When applicable][HIGH] — Prompt Injection / Input Validation (§12): the agent's search-tool results (external, untrusted DuckDuckGo content) flow back into the LLM context without the escape_for_prompt_xml boundary-tagging + detect_prompt_injection runtime guardrail pattern used by the RAG/classifier agents.
    • Basis: infrastructure/chatbot/pydantic_ai_chatbot.py registers duckduckgo_search_tool() directly with no output-side escaping or guardrail call; docs/ai/shared/security-checklist.md §12 requires this for any dynamic content reaching the prompt.
    • Closure: Deferred-with-rationale. This example is explicitly scoped by issue examples: agentic web-search tool for PydanticAI agents (tool-calling follow-up to simple-chatbot) #259 as a standalone, non-production teaching example — README already carries a Production Considerations note pointing to guardrails.py and flagging that search results are untrusted content. The issue explicitly keeps the guardrailed RAG agent untouched and defers a shared _core web-search infra abstraction (which is where a reusable escaping/guardrail wrapper would land) to a future issue once there's a second consumer.

Verdict: No OPEN BLOCKING or intent/contract-breaking HIGH — SEC-1 is closed via documented deferral, not left open.


Governor Footer

Adds a new examples/web_search_chatbot/ following the simple_chatbot
layout, with a PydanticAI agent using duckduckgo_search_tool() for
real web search and a keyless StubChatbot fallback.

- DI selector keys off settings.llm_model_name alone, matching the
  other chatbot examples (no new WEB_SEARCH_ENABLED flag).
- Adds the pydantic-ai-duckduckgo extra to pyproject.toml.
- Unit tests cover stub flow, structural real-agent flow (offline via
  TestModel(call_tools=[])), and tool-invocation (offline via
  FunctionModel + a monkeypatched fake search tool).
- Adds a copy-flow smoke ExampleCase so the example is verified to
  boot post-copy under make quickstart.
The web_search_chatbot unit tests instantiate PydanticAIChatbot
directly (via TestModel/FunctionModel), which lazily imports
duckduckgo_search_tool in __init__. CI's test job wasn't installing
the pydantic-ai-duckduckgo extra, so those tests failed with
ImportError in CI even though they passed locally.
…ision

ChatMessageModel was using __tablename__ = 'chatbot_message', the
same name as simple_chatbot's model, on the same SQLAlchemy
Base.metadata. When the full test suite collects both example
packages in one process, this raises InvalidRequestError. Renamed
to 'web_search_chatbot_message', matching the unique-per-example
convention used by chatbot_with_memory and chatbot_with_guardrails.
@Diyaaa-12

Copy link
Copy Markdown
Contributor Author

Investigated the remaining CI / test failures — none are related to this PR's changes:

  • test_pydantic_ai_embedding_adapter.py::TestOpenAIBatchSplitting::* (4 failures) — ModuleNotFoundError: No module named 'tiktoken' / missing openai extra, in the embedding adapter's OpenAI batch-splitting tests.
  • test_classification_stub_fallback.py::test_core_container_llm_model_is_test_model (2 errors) — AttributeError: property 'llm_model_name' of 'Settings' object has no setter, from a test trying to monkeypatch.setattr(settings, "llm_model_name", None) on what is a read-only computed property.

I confirmed both fail identically on main with none of this PR's changes applied (checked out clean, ran the same two test files). They look like pre-existing environment/test issues unrelated to web_search_chatbot — happy to open separate issues for them if useful, but wanted to flag rather than silently work around them in this PR.

The one failure that was caused by this PR — a chatbot_message table name collision with simple_chatbot when the full suite collects both in one process — is fixed in 7001678.

@Diyaaa-12

Copy link
Copy Markdown
Contributor Author

Hi @Mr-DooSun! Governor Footer link fixed, remaining CI failures are the pre-existing unrelated ones flagged above

@Mr-DooSun Mr-DooSun left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — web_search_chatbot example (#259)

Verdict: Request changes — one blocking item (CI is red as a direct effect of this PR's ci.yml change). Everything else is small, and the implementation itself is solid.

Verified good — please keep as-is

  • Container / selector / bootstrap / router wiring is an exact pattern match with simple_chatbot (_chatbot_selector keyed on settings.llm_model_name; wire(modules=[chatbot_router]) with the imported module object). Do not change.
  • Copy-flow compliance verified on this head: tools/check_examples_copyflow.py → 0 violations across 246 files; test_example_survives_copy_to_src[web_search_chatbot] and the completeness meta-test pass.
  • Unit tests run 2 passed / 2 skipped in a no-pydantic-ai env; request validation (prompt 1–1000 chars, Path(ge=1)) is right; the web_search_chatbot_message table name correctly avoids the sibling collision.
  • Security: the prompt-injection deferral (SEC-1 in the PR body) is accepted — search-tool output guardrails stay out of scope per #259, and the README production note covers it. Do not add guardrails in this PR.

Blocking — CI

See the inline comment on .github/workflows/ci.yml:66. Short version: the new extra installs pydantic_ai in CI for the first time, un-skipping six latent pre-existing test failures (tracked in #288 — please don't fix those here). Recommended path: revert the ci.yml line + add the duckduckgo importorskip guards (inline comment on the test file).

The failure is deterministic — re-running CI (e.g. the empty chore: trigger CI re-run commit at HEAD) cannot help. Please also drop that empty commit when you rebase: this repo merges with merge commits, so it would be preserved in main history permanently.

Summary-level findings (files outside the diff)

  1. [MEDIUM][REG/GOV] Only if the maintainer decides to keep the ci.yml extra: make check-full (Makefile:132) and its "CI-parity" descriptions (AGENTS.md, .claude/rules/commands.md) must gain the same extra in the same PR — otherwise the documented local parity loop cannot reproduce CI, and the two new agent tests run in no documented local entrypoint. With the recommended revert, no action is needed.
  2. [LOW][GOV] docs/reference.md "Optional dependency extras" table: please add a pydantic-ai-duckduckgo row (installs pydantic-ai-slim[duckduckgo]ddgs; enables this example's real search tool). The table's pre-existing openai/otel omissions are handled separately — adding just your row is enough.
  3. [DRIFT] examples/README.md: add a web_search_chatbot/ row to the Available examples index table — every other example has one (suggested: | web_search_chatbot/ | Agentic web-search tool use (DuckDuckGo, keyless) | LLM-calling | ✅ #287 |).

PR body

  • The dangling Fixes # line can be removed.
  • After the fixes, please re-verify the "Tests pass" checkbox and the Governor Footer final-verdict: merge-ready against a green CI run — both are currently stale.

Thanks for the contribution — the DI/bootstrap/copy-flow work is exactly right, and the offline FunctionModel tool-invocation test is a nice pattern. Once the CI line is resolved and the README runbook actually boots the real path, this is a strong addition to the examples set.

Comment thread .github/workflows/ci.yml Outdated
Comment thread tests/unit/web_search_chatbot/chatbot_test.py
Comment thread examples/web_search_chatbot/README.md Outdated
Comment thread examples/web_search_chatbot/README.md Outdated
Comment thread examples/web_search_chatbot/README.md Outdated
Comment thread examples/web_search_chatbot/README.md Outdated
Comment thread examples/web_search_chatbot/README.md Outdated
@Diyaaa-12 Diyaaa-12 force-pushed the feat/259-web-search-chatbot branch from 4031451 to 96718f7 Compare July 14, 2026 09:20
@Diyaaa-12

Copy link
Copy Markdown
Contributor Author

Hi @Mr-DooSun! Addressed all the review feedback:

  • Reverted the ci.yml extra (CI is green now)
  • Added duckduckgo importorskip guards to both agent tests
  • Fixed README: UTF-8 encoding, setup instructions (admin+openai extras), camelCase in both response examples, table name doc, typo
  • Removed the dangling Fixes # line
  • Dropped the empty chore: trigger CI re-run commit via rebase

All checks are passing now. Let me know if anything else needs adjustment!

@Mr-DooSun Mr-DooSun left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review — head 96718f7

Verdict: PASS on the code — posting as Comment (not Approve) because two summary-level items from the previous review are still unaddressed.

Verified fixed at this head — thank you, all confirmed

  • duckduckgo importorskip guards on both agent tests ✔
  • README: valid UTF-8 ✔ · table name matches the model (web_search_chatbot_message) ✔ · real-LLM runbook now installs the right extras and wipes quickstart.db ✔ · DuckDuckGo typo ✔ · camelCase response keys ✔
  • ci.yml extra install reverted → CI fully green at this head (all 6 jobs) ✔
  • Empty re-run commit dropped ✔ · dangling Fixes # removed ✔

All 7 resolved inline threads were re-verified against this head — every resolution is substantively justified.

Still open — carried over from the previous review's summary section

These items reference files outside this PR's diff, so no resolvable inline threads could anchor to them — which is likely why they slipped past the thread-resolution sweep. Please use this checklist as the tracker:

  • docs/reference.md — add the new extra to the "Optional dependency extras" table (after the pydantic-ai-google row, currently line 77):
    | `pydantic-ai-duckduckgo` | DuckDuckGo search tool for PydanticAI (`ddgs`) | `examples/web_search_chatbot/` real search tool |
    
  • examples/README.md — add an index row to the Available examples table (LLM-calling group, after the chatbot_with_memory/ row, currently line 116):
    | `web_search_chatbot/` | Agentic web-search tool use (DuckDuckGo, keyless) | LLM-calling | ✅ [#287](https://github.com/Mr-DooSun/fastapi-agent-blueprint/pull/287) |
    
  • .github/workflows/ci.yml — restore the removed blank line so the file drops out of the diff entirely (see the inline comment; leftover from the revert in 96718f7).
  • PR body — the intro ("Also fixes CI's test job…") and the Change Summary bullet ("Updated .github/workflows/ci.yml's test job to install the pydantic-ai-duckduckgo extra") still describe the change that 96718f7 reverted. Please update both — this repo merges with merge commits, so the PR body is preserved in history.

Please keep as-is

  • The ci.yml revert direction is correct — do not re-add the extra to the test job (the latent failures it un-skips are tracked in #288).
  • The importorskip guards, DI/selector/bootstrap wiring, unique table name, and the offline FunctionModel tool-invocation test are all right — do not change.

Once the four checkboxes land, I'll re-review and this becomes an Approve — the implementation itself is merge-quality.

Comment thread .github/workflows/ci.yml
- Add pydantic-ai-duckduckgo row to docs/reference.md's Optional
  dependency extras table.
- Add web_search_chatbot/ row to examples/README.md's Available
  examples index table.
- Restore the blank line in ci.yml left over from the earlier
  pydantic-ai-duckduckgo extra revert, so ci.yml drops out of the
  diff entirely.
@Diyaaa-12

Copy link
Copy Markdown
Contributor Author

Hi @Mr-DooSun! Addressed the remaining four items:

  • Added the pydantic-ai-duckduckgo row to docs/reference.md
  • Added the web_search_chatbot/ row to examples/README.md
  • Restored the blank line in ci.yml (now drops out of the diff entirely)
  • Updated the PR body intro + Change Summary bullet to reflect the revert

All checks are green. Let me know if anything else is needed!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

examples: agentic web-search tool for PydanticAI agents (tool-calling follow-up to simple-chatbot)

2 participants