Skip to content

Serialize libpq access to the shared replication connection#102

Merged
lukashes merged 1 commit into
mainfrom
bugfix/pgconn-mutex
Jul 12, 2026
Merged

Serialize libpq access to the shared replication connection#102
lukashes merged 1 commit into
mainfrom
bugfix/pgconn-mutex

Conversation

@lukashes

Copy link
Copy Markdown
Owner

Problem

The receive loop and the flush worker share one PGconn: the main thread calls PQgetCopyData/PQconsumeInput, the worker sends standby feedback via PQputCopyData/PQflush. libpq forbids using one connection from two threads: internal buffers are reallocated with no locking, and a blocked send even reads input from the writer thread (pqSendSome calls pqReadData on EWOULDBLOCK). Under load this race wedged the connection: PQflush stuck in select() forever and the process hung instead of failing fast.

Solution

  • Add std.Io.Mutex to ReplicationProtocol; every libpq call after startup takes it: the receive path, the feedback path, and the PQerrorMessage reads on error paths.
  • The poll() wait stays outside the lock, so feedback never queues behind an idle wait. Contention is negligible: the worker locks once per flush interval to send 34 bytes.
  • lockUncancelable on purpose: the critical sections are microseconds, and a cancelation point inside a libpq call would let shutdown interrupt the final LSN commit.
  • A plain mutex, not RwLock: there is exactly one reader, and "reading" still mutates PGconn (buffer reallocs, error state), so shared locking would recreate the race.

Simpler alternative to #92: threading stays as is, only the connection access is serialized.

@github-actions

Copy link
Copy Markdown
Contributor

📊 Benchmark Results

Current run is the minimum over 3 passes, compared against the base branch (main @ 206ef00), built on the same runner.

Benchmark Baseline Current Δ Time Allocs Status
Converter DELETE 126.35μs 126.21μs -0.1% 12 → 12 ➡️
Converter INSERT 121.10μs 120.79μs -0.2% 12 → 12 ➡️
Converter UPDATE 172.12μs 171.71μs -0.2% 21 → 21 ➡️
JsonSerializer 41.27μs 41.21μs -0.2% 3 → 3 ➡️
KafkaProducer produce 275.58μs 291.20μs +5.7% 1 → 1 🔴 slower
KafkaProducer sendMessage 343.57μs 374.57μs +9.0% 0 → 0 🔴 slower
PgOutputDecoder 94.29μs 93.35μs -1.0% 6 → 6 ➡️
getPartitionKeyValue boolean 16.19μs 15.99μs -1.2% 1 → 1 ➡️
getPartitionKeyValue integer 50.15μs 50.24μs +0.2% 3 → 3 ➡️
getPartitionKeyValue not found 0.03μs 0.03μs +0.0% 0 → 0 ⚪ noise
getPartitionKeyValue string 18.02μs 18.04μs +0.1% 1 → 1 ➡️
matchStreams found 17.35μs 17.16μs -1.1% 1 → 1 ➡️
matchStreams not found 0.13μs 0.05μs -59.7% 0 → 0 ⚪ noise

Summary: ➡️ 9 neutral · 🔴 2 slower · ⚪ 2 ignored (sub-μs)

Thresholds: <1μs ignore · 1–20μs 15% · 20–50μs 10% · ≥50μs 5%. Measured on a shared CI runner — treat small deltas as noise. Informational only; this check never fails the build.

@lukashes lukashes merged commit a93ff4b into main Jul 12, 2026
5 checks passed
@lukashes lukashes deleted the bugfix/pgconn-mutex branch July 12, 2026 08:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant