Skip to content

Fix InMemoryKache::getOrPut race condition in high concurrency#393

Open
ryansmartpadpro wants to merge 2 commits into
MayakaApps:mainfrom
smartpadpro-team:fix/getOrPut-concurrency
Open

Fix InMemoryKache::getOrPut race condition in high concurrency#393
ryansmartpadpro wants to merge 2 commits into
MayakaApps:mainfrom
smartpadpro-team:fix/getOrPut-concurrency

Conversation

@ryansmartpadpro

Copy link
Copy Markdown

Summary

Fixes #239.

getOrPut had a race condition under high concurrency: a creation deferred could complete and remove itself from creationMap (via its finally block) between the creationMutex.withLock exit and the trailing get(key) call. If the entry was also evicted by concurrent cache pressure in that window, get(key) would find nothing and return null.

The fix captures the deferred inside the lock (before it can remove itself from creationMap) and awaits it directly via getFromCreation(key, deferred), returning the value from the deferred's completion state rather than re-looking it up through the map.

Before:

creationMutex.withLock {
    if (creationMap[key] == null && map[key] == null) {
        @Suppress("DeferredResultUnused")
        internalPutAsync(key, creationFunction)
    }
}
return get(key)  // deferred may have already completed and been evicted

After:

val deferred = creationMutex.withLock {
    if (creationMap[key] == null && map[key] == null) {
        internalPutAsync(key, creationFunction)
    } else {
        creationMap[key]  // capture existing deferred while holding the lock
    }
}
return deferred?.let { getFromCreation(key, it) } ?: get(key)

Test plan

  • Added getOrPutHighConcurrency regression test: launches 20,001 coroutines each calling getOrPut with a unique key against a maxSize = 100 cache. The resulting LRU evictions reliably open the race window. The test uses InMemoryKache directly (not testInMemoryKache) so creation deferreds run on Dispatchers.Default real threads — testInMemoryKache overrides creationScope to the single-threaded test dispatcher which hides the race.
  • Confirmed the test fails on the unfixed code and passes with the fix.
  • Full jvmTest suite passes.

🤖 Generated with Claude Code

ryansmartpadpro and others added 2 commits June 25, 2026 12:16
The getOrPut function had a race condition where concurrent calls could result in
a NullPointerException. When multiple threads called getOrPut simultaneously on
different keys with a small cache size:

1. Thread A starts async creation and captures the deferred in creationMap
2. Thread A releases the creationMutex lock
3. The async task completes and calls creationMap.remove(key) in finally block
4. Thread A calls get(key) which looks in creationMap but finds nothing
5. Result: null is returned, causing NPE when caller expects a value

The fix captures the deferred reference while holding the lock, ensuring we can
wait for it even after it's been removed from creationMap:

    val deferred = creationMutex.withLock {
        if (creationMap[key] == null && map[key] == null) {
            internalPutAsync(key, creationFunction)
        } else {
            creationMap[key]
        }
    }
    return deferred?.let { getFromCreation(key, it) } ?: get(key)

This guarantees that getOrPut returns the successfully created value or null,
but never returns null when a value was successfully created.

Fixes MayakaApps#239
Tests that getOrPut never returns null under high concurrency when
more unique keys than maxSize causes LRU evictions. Uses the real
default creationScope (Dispatchers.Default) so creation deferreds
run on actual threads — testInMemoryKache overrides it to the
single-threaded test dispatcher which hides the race.

Closes MayakaApps#239

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

InMemoryKache::getOrPut fails in high concurrency context

1 participant