Skip to content

Fix KPI catalog search dropping matches on digit+letter periods - #2

Merged
fredolss merged 2 commits into
mainfrom
fix/kpi-catalog-search-tokenizer
Sep 23, 2026
Merged

fredolss merged 2 commits into
mainfrom
fix/kpi-catalog-search-tokenizer

Conversation

@fredolss

Copy link
Copy Markdown
Owner

Summary

  • CatalogSearch's term tokenizer regex ([\p{L}\p{N}]+) glued a number directly onto an adjacent word into a single token, e.g. "14day" instead of "14" + "day". Since MatchesAllTerms requires every query term to equal a whole token, list_kpi_screener_options returned zero matches for "RSI 14" even though kpiId 159 ("[RSI] 14day") is exactly that KPI.
  • Split the regex into [\p{L}]+|[\p{N}]+ so digits and letters tokenize separately, fixing this for both KpiScreenerCatalog and KpiHistoryCatalog (they share CatalogSearch.MatchesAllTerms). Verified live: "RSI 14" now returns exactly kpiId 159, correctly excluding the unrelated RSI trend variants (1day/3day/5day/7day).
  • Also moves CLAUDE.md's content into AGENTS.md with CLAUDE.md now just @-importing it, so the guidance isn't duplicated across both files Claude Code reads (this was already staged in the working tree before this session's investigation started).

Test plan

  • Added LookupMatchesNumericTermGluedToLetterSuffixInDescription regression test covering the "RSI 14" case
  • dotnet test — all 165 tests pass (157 unit + 8 integration), no regressions

🤖 Generated with Claude Code

fredolss and others added 2 commits September 23, 2026 18:48
Claude Code reads both CLAUDE.md and AGENTS.md; keeping the actual
guidance in AGENTS.md and having CLAUDE.md just @-import it avoids
maintaining two copies of the same document.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
CatalogSearch's term regex ([\p{L}\p{N}]+) glued a leading number
directly onto an adjacent word into one token, e.g. "14day" instead of
"14" + "day". Since MatchesAllTerms requires every query term to equal
a whole token, searching list_kpi_screener_options for "RSI 14" found
zero matches even though kpiId 159 ("[RSI] 14day") is exactly that
KPI - the "14" query term never equaled any single token.

Split the regex into [\p{L}]+|[\p{N}]+ so digits and letters tokenize
separately. This fixes both KpiScreenerCatalog and KpiHistoryCatalog,
which share this search helper.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@fredolss
fredolss merged commit db89179 into main Sep 23, 2026
7 checks passed
@fredolss
fredolss deleted the fix/kpi-catalog-search-tokenizer branch September 23, 2026 17:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant