feat: Vector/semantic search backend (sqlite-vss / Qdrant / pgvector)

[tgosoul2019]

Context

Suggested by multiple LLMs (Claude 3.7, Grok) during protocol evaluation (see kcp-outreach/prompts/llm-scorecard.md).

Problem

Current search uses SQLite FTS5 (BM25 full-text search). This is excellent for keyword search but misses semantic similarity — e.g., searching rate limiting won't find an artifact titled throttling strategies even if the content is semantically equivalent.

Proposed Solution

Add a pluggable vector search backend to KCPNode:

node = KCPNode(
    user_id="alice",
    search_backend="sqlite-vss",  # or "qdrant", "pgvector", "chroma"
    embedding_model="text-embedding-3-small"  # or local Ollama
)
results = node.search("rate limiting", mode="semantic")
results = node.search("rate limiting", mode="hybrid")  # FTS5 + vector

Candidate Backends

Backend	Mode	Notes
sqlite-vss	Local	Zero-infra, same SQLite file, natural fit for local mode
chromadb	Local/Server	Python-native, popular in AI ecosystem
qdrant	Server	Production-grade, open-source
pgvector	Server	PostgreSQL extension, natural fit for Hub mode

Design Principles

• FTS5 remains the default — zero additional dependencies
• Vector search is opt-in via search_backend param
• Embedding generation should support: OpenAI, HuggingFace local, Ollama
• Local mode default: sqlite-vss (keeps zero-infra philosophy)

Acceptance Criteria

• KCPNode accepts optional search_backend and embedding_model params
• At minimum: sqlite-vss local backend working end-to-end
• node.search(query, mode='semantic'|'keyword'|'hybrid')
• Tests for Python SDK (sdk/python/)
• SPEC.md §4.3 updated to reflect semantic discovery

Related

• RFC KCP-001 §4.3 (Discovery)
• Suggested during LLM eval: Claude 3.7 Sonnet, Grok 2

[m13v]

sqlite-vss is the right default for local mode. we use sqlite-vec (its successor) in production for a knowledge extraction database and it handles the zero-infra local-first use case well. one thing to be aware of: sqlite-vss uses IVF indexing which requires a training step on your data before queries are accurate. sqlite-vec switched to brute-force kNN which is simpler and accurate on small to medium datasets (under 500k vectors). for the pluggable design, having FTS5 as the default with vector search as opt-in is correct. in practice about 70% of agent queries are specific enough that BM25 handles them well. the hybrid mode is where you see the biggest quality jump, combining the precision of keyword matching with the recall of semantic search. for the embedding generation, supporting Ollama as a provider alongside OpenAI is smart since it lets users keep everything local.

[m13v]

our sqlite-vec implementation with local embeddings and hybrid search over a knowledge database: https://github.com/m13v/ai-browser-profile/blob/main/ai_browser_profile/embeddings.py and the database schema with structured categories: https://github.com/m13v/ai-browser-profile/blob/main/ai_browser_profile/db.py

[totto]

Thanks @m13v — this is exactly the kind of production experience that should shape the design.

The sqlite-vss → sqlite-vec migration makes sense. The IVF training requirement in sqlite-vss is a real footgun for local-first use: KCP nodes will often have small, sparse corpora where IVF training is either impractical or gives poor results. Brute-force kNN at <500k vectors is a non-issue for the workloads this targets.

Your 70% BM25 / 30% hybrid split matches what we've observed in practice. The plan is FTS5 as the mandatory default, vector search as an explicit opt-in, and hybrid as the quality upgrade path — your numbers validate that order.

Appreciated the reference implementation. The structured categories pattern in your DB schema is interesting — we're looking at whether KCP's manifest taxonomy (declaration types, relationship types) maps naturally onto a category dimension for retrieval tuning.

Will update the issue to reflect sqlite-vec as the target rather than sqlite-vss.