Skip to content

Document ts_rank as BM25 approximation and validate before pg_bm25 #12

Description

@sid-ak

Category: Retrieval Performance
Priority: Medium
Phase: Phase 4 (Hybrid retrieval + reranking)

Problem
Postgres tsvector/ts_rank approximates BM25 but diverges from true term-frequency/length-normalized scoring, which can skew RRF fusion on keyword-heavy queries.

Solution
Document ts_rank as the explicit approximation and validate against the eval harness first; only adopt pg_bm25 (ParadeDB) if evals show a real gap.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions