Skip to content

[InMemory] Fixed deadlock - #559

Merged
mhutchinson merged 1 commit into
transparency-dev:mainfrom
mhutchinson:fix/in-memory
Jun 16, 2026
Merged

[InMemory] Fixed deadlock#559
mhutchinson merged 1 commit into
transparency-dev:mainfrom
mhutchinson:fix/in-memory

Conversation

@mhutchinson

Copy link
Copy Markdown
Contributor

Interesting bug found by accident when trying to benchmark in memory:

  1. Client calls Witness.Update, which calls Persistence.Update. The in-memory persistence acquires a write lock on its global mutex to safely update the checkpoint.
  2. Persistence.Update calls the callback function provided to perform consistency checks.
  3. To verify the signature on the previous checkpoint, the callback needs the log's public key. It calls Persistence.Log, which attempts to acquire a read lock on the same global mutex
  4. Deadlock!

This introduces a sync.Map for the logs (which are largely intialized up-front), and uses a mutex for the checkpoint state.

Interesting bug found by accident when trying to benchmark in memory:
  1. Client calls `Witness.Update`, which calls `Persistence.Update`. The in-memory persistence acquires a write lock on its global mutex to safely update the checkpoint.
  2. `Persistence.Update` calls the callback function provided to perform consistency checks.
  3. To verify the signature on the previous checkpoint, the callback needs the log's public key. It calls `Persistence.Log`, which attempts to acquire a read lock on the same global mutex
  4. Deadlock!

This introduces a sync.Map for the logs (which are largely intialized up-front), and uses a mutex for the checkpoint state.
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 54.54545% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 24.63%. Comparing base (26841b7) to head (1cec140).

Files with missing lines Patch % Lines
persistence/inmemory/inmemory.go 54.54% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #559      +/-   ##
==========================================
+ Coverage   24.35%   24.63%   +0.28%     
==========================================
  Files          27       27              
  Lines        2012     2005       -7     
==========================================
+ Hits          490      494       +4     
+ Misses       1417     1406      -11     
  Partials      105      105              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mhutchinson
mhutchinson merged commit f6b6fd3 into transparency-dev:main Jun 16, 2026
13 checks passed
@mhutchinson
mhutchinson deleted the fix/in-memory branch June 16, 2026 15:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants