Skip to content

feat: migrate LLM backend from OpenAI gpt-5-nano to Claude Sonnet #8

Description

@panchgonzalez

Background

Production is hitting OpenAI 429 insufficient_quota errors with gpt-5-nano. This causes silent failures in the lead classification pipeline. Beyond the quota issue, gpt-5-nano is underpowered for the 3-stage pipeline (triage → research → scoring) — classification quality suffers at the cheap tier.

Switching to Claude Sonnet (claude-sonnet-4-6) removes the OpenAI dependency entirely and gives us a significantly more capable model for lead scoring.

Proposed Change

This is a provider swap — pydantic-ai supports Anthropic natively, so no framework changes are needed.

Files to modify

  1. pyproject.toml

    • Change pydantic-ai-slim[openai,duckduckgo]pydantic-ai-slim[anthropic,duckduckgo]
  2. src/leads_agent/agent.py (lines 9-10, 105-106)

    • Remove OpenAIProvider, OpenAIChatModel, OpenAIChatModelSettings imports
    • Add AnthropicModel from pydantic_ai.models.anthropic
    • Update agent_factory() to instantiate AnthropicModel(model_name=llm_model_name) — no base_url needed
  3. src/leads_agent/config.py (lines 52-54)

    • Change default LLM_MODEL_NAME from gpt-5-nanoclaude-sonnet-4-6
    • Remove LLM_BASE_URL field (Anthropic client doesn't take a custom base URL)
    • Rename OPENAI_API_KEYANTHROPIC_API_KEY
  4. .env.example

    • Update env var name: OPENAI_API_KEYANTHROPIC_API_KEY
    • Update default model: gpt-5-nanoclaude-sonnet-4-6
    • Remove LLM_BASE_URL line (or leave as optional for Ollama use cases)
  5. src/leads_agent/core/init_wizard.py (lines 45-47)

    • Update interactive setup prompt default to claude-sonnet-4-6

Out of Scope

  • No changes to agent prompts or system instructions
  • No changes to output schemas or Pydantic models
  • No changes to Slack integration or Docker configuration (beyond .env)

Testing

  1. Set ANTHROPIC_API_KEY in .env
  2. Run docker compose up locally
  3. Send a test lead through the Slack trigger
  4. Confirm all 3 stages (triage, research, scoring) complete without errors in logs
  5. Verify / reaction appears on the Slack message as expected

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions