Intent
Measure and optionally improve recall after #94 while keeping lexical-only PostgreSQL fully usable.
Design
Retrieve independent lexical and vector candidates, deduplicate by canonical source identity, and fuse ranks with Reciprocal Rank Fusion; default k=60 with configurable candidate/final limits. Never combine heterogeneous raw scores. Return optional provenance for ranks, RRF contributions, provider/model, rerank position, and timing.
Reranking is a separate opt-in OpenAI-v1-compatible chat-completions adapter. It receives redacted bounded snippets plus opaque IDs and must return a strictly validated ID order. Use temperature zero, budgets, timeouts, retries, circuit breaking, and RRF/lexical fallback.
Deliverables
- Hybrid retrieval/configuration and backward-compatible metadata.
- Strict reranker adapter.
- Versioned offline evaluation fixtures/command/report.
- Recall@k, MRR, nDCG@k, latency, provider usage, failure, and fallback metrics.
- Explicit rollout switch.
Non-goals
Vendor-specific rerank APIs, online learning, automatic model selection, telemetry upload, default hybrid/reranked results, or parity dependency.
Dependencies
Blocked by #89 and #94. Does not block parity or #225; blocks #226.
Acceptance tests
Golden queries reproducibly compare lexical, vector, RRF, and reranked modes. Tests cover ties, duplicates, missing embeddings, profile/dimension mismatch, provider timeout/429/5xx/auth/malformed/unknown IDs, budgets, redaction, deterministic order, fallback, and public response compatibility. Reports pin dataset/config and capture query plans.
Documentation
Document fusion math, knobs, provenance, evaluation method, privacy/cost/latency, provider setup, circuit breaker/fallback, staged enablement, and rollback.
Rollout
Evaluate offline first. Require no material relevance regression and an explicit latency/cost decision before opt-in canarying. Lexical remains the default until product evidence says otherwise.
Failure handling
Provider or validation failure returns RRF or lexical results with degraded metadata; recall must not fail solely because inference is unavailable. Never accept unknown/duplicate IDs or expose snippets in logs.
Intent
Measure and optionally improve recall after #94 while keeping lexical-only PostgreSQL fully usable.
Design
Retrieve independent lexical and vector candidates, deduplicate by canonical source identity, and fuse ranks with Reciprocal Rank Fusion; default
k=60with configurable candidate/final limits. Never combine heterogeneous raw scores. Return optional provenance for ranks, RRF contributions, provider/model, rerank position, and timing.Reranking is a separate opt-in OpenAI-v1-compatible chat-completions adapter. It receives redacted bounded snippets plus opaque IDs and must return a strictly validated ID order. Use temperature zero, budgets, timeouts, retries, circuit breaking, and RRF/lexical fallback.
Deliverables
Non-goals
Vendor-specific rerank APIs, online learning, automatic model selection, telemetry upload, default hybrid/reranked results, or parity dependency.
Dependencies
Blocked by #89 and #94. Does not block parity or #225; blocks #226.
Acceptance tests
Golden queries reproducibly compare lexical, vector, RRF, and reranked modes. Tests cover ties, duplicates, missing embeddings, profile/dimension mismatch, provider timeout/429/5xx/auth/malformed/unknown IDs, budgets, redaction, deterministic order, fallback, and public response compatibility. Reports pin dataset/config and capture query plans.
Documentation
Document fusion math, knobs, provenance, evaluation method, privacy/cost/latency, provider setup, circuit breaker/fallback, staged enablement, and rollback.
Rollout
Evaluate offline first. Require no material relevance regression and an explicit latency/cost decision before opt-in canarying. Lexical remains the default until product evidence says otherwise.
Failure handling
Provider or validation failure returns RRF or lexical results with degraded metadata; recall must not fail solely because inference is unavailable. Never accept unknown/duplicate IDs or expose snippets in logs.