Skip to content

Fix zero-vector items corrupting queryItems results - #140

Open
OsamaAnsar wants to merge 1 commit into
Stevenic:mainfrom
OsamaAnsar:fix/query-zero-vector-nan
Open

OsamaAnsar wants to merge 1 commit into
Stevenic:mainfrom
OsamaAnsar:fix/query-zero-vector-nan

Conversation

@OsamaAnsar

Copy link
Copy Markdown

What was wrong

ItemSelector.normalizedCosineSimilarity returns NaN for a zero-norm vector (deliberate, and covered by existing tests). Since #124, LocalIndex.queryItems picks the top K with a bounded min-heap that compares with > / <. Every comparison with NaN is false, so a NaN entry at the heap root can never be replaced, and the final heap.sort((a, b) => b.distance - a.distance) then uses an inconsistent comparator. A stored zero-vector item can therefore push real matches out of the results, or leave the best match out of first place.

Repro on current main (built lib/, Node 22): items z=[0,0,0], a=[1,0,0], b=[0.9,0.1,0], c=[0,1,0], query [1,0,0]. The independent numpy oracle gives cosine [nan, 1.0, 0.9938837346736189, 0.0] for z, a, b, c, i.e. the expected order is a, b, c, z (NaN last).

zero first,  topK=2 -> z:NaN a:1.0000                      (expected a, b)   b is missing
zero first,  topK=3 -> z:NaN a:1.0000 b:0.9939             (expected a, b, c)
zero second, topK=2 -> a:1.0000 z:NaN                      (expected a, b)   b is missing
zero second, topK=4 -> b:0.9939 z:NaN a:1.0000 c:0.0000    (expected a, b, c, z)   best match not first
zero last,   topK=4 -> a:1.0000 b:0.9939 c:0.0000 z:NaN    (ok)

With this change the same script prints:

zero first,  topK=2 -> a:1.0000 b:0.9939
zero first,  topK=3 -> a:1.0000 b:0.9939 c:0.0000
zero second, topK=2 -> a:1.0000 b:0.9939
zero second, topK=4 -> a:1.0000 b:0.9939 c:0.0000 z:NaN
zero last,   topK=4 -> a:1.0000 b:0.9939 c:0.0000 z:NaN

The fix

The heap and the final sort compare a rank instead of the raw score: the same value, except NaN becomes -Infinity. Zero-vector items therefore rank below every real match, and the top K is always chosen from the real scores. The final sort compares explicitly (b.rank - a.rank would itself be NaN for two -Infinity ranks).

Choices I made, in case you prefer otherwise:

  • normalizedCosineSimilarity / cosineSimilarity are untouched; they still return NaN.
  • A zero-vector item is still returned (with score: NaN) when fewer than K items have a real score, which matches what callers got before. I did not drop them from the results, to avoid changing result counts. Skipping NaN items entirely would be a one-line change if you would rather have that.
  • A zero-norm query vector makes every score NaN; behavior there is unchanged (the first K items in index order are returned).

Tests

New queryItems > zero-norm vectors specs in src/LocalIndex.spec.ts:

  • all 24 insertion orders of the four items above, for topK 1 to 5, must return a, b, c, z truncated to topK
  • the zero vector keeps a NaN score when it is returned
  • 20 items with 10 zero vectors interleaved: the true top 5 is returned

Verification:

  • yarn lint: 0 errors
  • yarn test: 596 passing (593 on main)
  • Revert check: with only src/LocalIndex.ts reverted to main, the 3 new tests fail; with the fix restored all 596 pass.

docs/changelog.md has a new Unreleased entry. No public API change, so llms.txt is untouched.

CONTRIBUTING suggests opening an issue for bugs; I went straight to a PR with the repro above, but I'm happy to open an issue as well if you prefer.

normalizedCosineSimilarity returns NaN for a zero-norm vector, and every
comparison with NaN is false. In the bounded min-heap used by queryItems
a NaN entry at the root could never be replaced, so a zero-vector item
could displace real matches from the top K, and the final sort used an
inconsistent comparator that could leave the best match out of first
place.

The heap and the final sort now compare a rank in which NaN becomes
-Infinity, so zero-vector items rank below every real match. The score
reported for such an item is still NaN.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant