Skip to content

context.md

Latest commit

 

History

History
92 lines (75 loc) · 3.13 KB

context.md

File metadata and controls

92 lines (75 loc) · 3.13 KB
Raw

Voice-to-Voice Claude Extension - Project Context

Project Goal

Build a browser extension that enables voice-based interaction with Claude AI, creating a conversational interface similar to Siri or Google Assistant.

Core Functionality

  • Voice Input: User speaks into the extension
  • Processing: Speech is transcribed to text and sent to Claude API
  • Voice Output: Claude's text response is converted to speech and played back conversationally

User Flow

  1. User speaks into the extension
  2. Speech gets transcribed to text (Speech-to-Text)
  3. Text gets sent to Claude's API
  4. Claude's response comes back as text
  5. Text gets converted to speech (Text-to-Speech)
  6. Speech plays back to user

Development Roadmap

  • Phase 1: Browser extension (starting point)
  • Phase 2: Expand to other formats (desktop app, mobile app, web app)

Technical Decisions

Speech-to-Text (STT)

Option: Browser's Web Speech API (chosen for MVP)

  • Performance: Pretty solid for English, comparable to Siri/Google Assistant
  • Speed: Real-time, very fast
  • Pros: Free, built-in, no additional API needed
  • Cons:
    • Only works reliably in Chrome/Edge (Firefox support is spotty)
    • Requires internet connection
    • Can be inconsistent with accents or background noise
    • Google powers it (privacy considerations)

Text-to-Speech (TTS)

Option: Browser's Web Speech Synthesis API (for MVP, plan to upgrade)

  • Performance: Robotic and unnatural sounding
  • Speed: Instant, no lag
  • Pros: Free, built-in, no additional API needed
  • Cons:
    • Voices vary wildly by OS/browser
    • Not very conversational or human-like
    • Can sound monotone, especially for longer responses

Future upgrade options:

  • ElevenLabs API
  • OpenAI's TTS API

Strategy

Start with free browser APIs to get working prototype, then upgrade TTS later for better conversational feel.

Project Scope

MVP Components

  1. Extension UI (popup with mic button) - ~30 mins
  2. Web Speech API integration for recording - ~1 hour
  3. Claude API integration for processing - ~30 mins
  4. TTS integration for playback - ~30 mins
  5. Error handling and polish - ~1-2 hours

Estimated Time:

  • Experienced with JS/extensions: 2-3 hours
  • Newer to extensions: 5-8 hours

Challenges to Address

  • Handling speech recognition failures
  • Managing Claude API response times
  • Implementing playback controls (stop/pause)
  • Error handling for network issues
  • Making Claude responses more conversational (may need to adjust prompts)

Key Considerations

Conversational Feel

  • Challenge: Claude tends to give detailed responses, while Siri/Google Assistant are short and punchy
  • Potential solutions:
    • Prompt Claude to give shorter, more conversational responses
    • Implement interrupt/stop functionality for long responses

Browser Compatibility

  • Primary target: Chrome/Edge (best Web Speech API support)
  • Firefox: Limited support, may need fallback

Next Steps

  1. Set up browser extension boilerplate
  2. Implement basic UI with mic button
  3. Wire up speech-to-text
  4. Integrate Claude API
  5. Add text-to-speech playback
  6. Test and refine
  7. Plan TTS upgrade for better voice quality