feat(search): optional BM25 relevance ranking (--rank)#11
Merged
Conversation
Slice 1 (engine + CLI) of the ranking feature. Opt-in; changes only the ORDER of matches, never which match, and preserves exact substring semantics. Design discovery mid-build (see ops/RELEVANCE_RANKING_DESIGN.md correction): the "one ORDER BY bm25" plan doesn't work for the default path — literal searches go through _direct_scan_search (Python substring scan, so "bob" matches "bobcat") and never touch FTS5, so the index's bm25() is unreachable there. Routing them through FTS5 would change WHICH results match (token vs substring), breaking the "only order changes" promise. Resolution (Option B): a Python BM25 scorer applied to the matched set after matching — uniform on both the substring and whole-word paths, exact-substring-preserving, deterministic. - indexer._rank_matches: BM25 over the matched set (term rarity/IDF, saturated TF, length normalization); ties keep original order. - indexer.search_with_index: applies it when config["rank"] and not use_context (context regroups by file, which ranking would scramble). - api.search: new rank=False param, threaded via search_config; docstring. - cli: --rank flag + help text; prints a note if --rank is used without an index (index-gated — ranking lives on the indexed path). - tests: unit scorer (ordering, determinism, empty/no-terms) + integration through search() (same matches reordered; no-index no-op is safe). Scope: CLI + engine. MCP wiring and GUI checkbox are follow-up slices. No version bump (Unreleased). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Robert D. Schoening <bobschoening@gmail.com>
exbuf
marked this pull request as ready for review
July 21, 2026 00:50
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Optional relevance ranking —
peekdocs --rank <terms>(CLI) /search(..., rank=True)(API). Orders matches by BM25 instead of the default file-then-line order. Slice 1 of 3 (engine + CLI); MCP wiring and a GUI checkbox are follow-up slices.Opt-in, and it changes only the order — never which matches are returned — preserving peekdocs's exact substring matching and its deterministic story.
The mid-build discovery (why this isn't the "one ORDER BY" we expected)
The design note's premise was "ranking is a one-line
ORDER BY bm25(), the index already has it." Wiring it up proved that half-wrong:_direct_scan_search— a Python substring scan (so "bob" matches "bobcat") that never touches FTS5, so the index'sbm25()is unreachable there.bm25()only lives on the whole-word_fts5_fast_searchpath.bm25()would change which results match (token vs substring) — breaking the "only order changes" promise.Resolution (Option B, chosen with the maintainer): a Python BM25 scorer (
_rank_matches) applied to the matched set after matching — uniform on both paths, exact-substring-preserving, deterministic.How it ranks
Classic BM25 over the matched paragraphs: term rarity (IDF), term frequency with saturation (repetition can't dominate), and length normalization (density beats size). So a short on-point passage outranks a file that just repeats a common word — not by match-count-per-file.
Behavior / constraints
--rankwithout an index prints a note and returns file-order results (graceful no-op, verified).Verification
TestRanking(unit scorer: ordering, determinism, empty/no-terms; integration: same-matches-reordered, no-index safe no-op).Docs
CHANGELOG (Unreleased), USER_GUIDE CLI flag table. The ops design note is corrected to record the substring-path discovery and the Option-B resolution.
🤖 Generated with Claude Code