Skip to content

Fix thread safety issue with ordinal terms collector - #1006

Merged
aprudhomme merged 1 commit into
mainfrom
fix_ordinal_terms_collector_thread_safety
Jul 22, 2026
Merged

Fix thread safety issue with ordinal terms collector#1006
aprudhomme merged 1 commit into
mainfrom
fix_ordinal_terms_collector_thread_safety

Conversation

@aprudhomme

Copy link
Copy Markdown
Contributor

Problem

GlobalOrdinalLookup instances are cached per IndexReader and shared across all concurrent search requests. The cached object held a single SortedDocValues / SortedSetDocValues instance, which is not thread-safe — Lucene's term dictionary uses mutable LZ4 decompression buffers internally (blockBuffer, blockInput, currentCompressedBlockStart/End).

When two requests both reach the reduce() phase of an OrdinalTermsCollectorManager at the same time, their lookupGlobalOrdinal calls operate on the same TermsDict decompression state simultaneously. One thread's decompressBlock() corrupts the buffers the other is reading, producing:

  java.lang.ArrayIndexOutOfBoundsException: arraycopy: source index -59 out of bounds for byte[259]
      at org.apache.lucene.util.compress.LZ4.decompress(LZ4.java:135)
      at org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$TermsDict.decompressBlock(...)
      at org.apache.lucene.index.MultiDocValues$MultiSortedDocValues.lookupOrd(...)
      at com.yelp.nrtsearch.server.search.GlobalOrdinalLookup$SortedLookup.lookupGlobalOrdinal(...)
      at com.yelp.nrtsearch.server.search.collectors.additional.TermsCollectorManager.fillBucketResult(...)

The OrdinalMap (used for segment-to-global ordinal mapping during collection) is thread-safe — it is backed by read-only packed integer arrays. Only the term lookup path is affected.

Fix

Separate the thread-safe part (OrdinalMap) from the non-thread-safe part (SortedDocValues / SortedSetDocValues):

  • GlobalOrdinalLookup (cached, shared) now holds only the OrdinalMap and value count. getSegmentMapping() is unchanged.
  • A new createTermLookup(IndexReader) method returns a lightweight per-request TermLookup that holds its own private array of segment-level doc values. It uses the cached OrdinalMap to route lookupGlobalOrdinal(ord) to the correct segment and local ordinal.
  • OrdinalTermsCollectorManager.reduce() now calls createTermLookup to get a fresh, request-private TermLookup before resolving ordinals to strings.

Getting per-segment SortedDocValues via DocValues.getSorted(leafReader, field) is cheap (a pointer into already-loaded segment data). The expensive OrdinalMap.build() call remains cached as before.

Testing

  • All existing GlobalOrdinalLookupTest and OrdinalTermsCollectorManagerTest tests pass unchanged.
  • Added testConcurrentTermLookup: 8 threads × 500 iterations each calling createTermLookup and resolving ordinals concurrently from the same cached GlobalOrdinalLookup, asserting no errors or incorrect results.

@aprudhomme
aprudhomme requested a review from sarthakn7 July 21, 2026 23:24
@aprudhomme
aprudhomme merged commit 3d11670 into main Jul 22, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants