Skip to content

RAG-5: Semantic embedding fallback for conceptual queries (stretch goal) #70

Description

@nonogv

Context

Stretch goal. Depends on #67 (RAG-2). Only worth implementing if BM25 proves insufficient in practice.

Motivation

BM25 handles keyword-heavy queries well ("Operator ratio", "Glue Compressor attack"). It struggles with conceptual queries like "how do I make it warmer?" or "why does my mix sound muddy?" — queries where the answer is in the docs but no keywords overlap.

What

If retrieveLiveDocs returns no results above the BM25 threshold, optionally embed the query using the user's already-configured provider API key:

Provider Embedding model
OpenAI text-embedding-3-small (~$0.02/1M tokens)
Gemini text-embedding-004
Anthropic No embedding API — skip, fall back to web_search

Pre-computed chunk embeddings stored in data/live-docs-embeddings.bin, generated by npm run scrape-docs --embed. At query time: embed the query, cosine-similarity against chunk embeddings in-process, return top-k above threshold.

Constraints

  • Only fires when BM25 returns nothing — never adds latency to keyword-matched queries
  • Uses the user's existing API key — no new credential required
  • --embed flag on the scraper is opt-in; the extension works fine without embeddings
  • Embeddings file is gitignored (too large) — generated locally or in CI with --embed

Decision gate

Implement only after RAG-3 ships and real usage shows BM25 gaps. Track missed queries in a separate issue before committing to this.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestideasSpeculative ideas to considerv2v2 power features

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions