Skip to content

Validate REPLICA IDENTITY FULL for delete-tracking streams at startup#104

Merged
lukashes merged 2 commits into
mainfrom
feat/validate-replica-identity
Jul 12, 2026
Merged

Validate REPLICA IDENTITY FULL for delete-tracking streams at startup#104
lukashes merged 2 commits into
mainfrom
feat/validate-replica-identity

Conversation

@lukashes

Copy link
Copy Markdown
Owner

Closes #79.

Problem

The docs require REPLICA IDENTITY FULL, but nothing checked it. With the default identity the pipeline runs fine, and only DELETE events silently drop the non-key columns from the old row, contradicting the documented format. The InvalidReplicaIdentity error was declared in the validator but never used.

Solution

  • Add PostgresValidator.checkReplicaIdentityFull: reads pg_class.relreplident and fails with an actionable ALTER TABLE ... REPLICA IDENTITY FULL message unless the identity is f.
  • Run it per stream during startup validation, but only when the stream tracks DELETE (Stream.tracksDelete). Replica identity is irrelevant for INSERT, and UPDATE emits only the new row, so an insert/update-only stream is not rejected and does not pay FULL's extra UPDATE WAL.
  • Fail fast at startup rather than guarding at runtime: a wrong identity is a config/setup problem the operator must fix.

Test

  • Integration: FULL table passes, a default-identity table errors with InvalidReplicaIdentity, and the check reports ConnectionFailed when not connected.
  • Unit: Stream.tracksDelete is false for insert/update-only and true when delete is present, proving delete-free streams skip the check.

Docs (README, AGENTS.md) updated: FULL is needed on tables whose stream tracks DELETE, not on every tracked table.

@github-actions

Copy link
Copy Markdown
Contributor

📊 Benchmark Results

Current run is the minimum over 3 passes, compared against the base branch (main @ a93ff4b), built on the same runner.

Benchmark Baseline Current Δ Time Allocs Status
Converter DELETE 127.29μs 126.56μs -0.6% 12 → 12 ➡️
Converter INSERT 121.42μs 120.76μs -0.5% 12 → 12 ➡️
Converter UPDATE 172.77μs 172.71μs -0.0% 21 → 21 ➡️
JsonSerializer 40.93μs 41.07μs +0.3% 3 → 3 ➡️
KafkaProducer produce 299.02μs 283.01μs -5.3% 1 → 1 🟢 faster
KafkaProducer sendMessage 345.37μs 349.44μs +1.2% 0 → 0 ➡️
PgOutputDecoder 84.22μs 93.77μs +11.3% 6 → 6 🔴 slower
getPartitionKeyValue boolean 16.31μs 16.13μs -1.1% 1 → 1 ➡️
getPartitionKeyValue integer 50.69μs 50.31μs -0.8% 3 → 3 ➡️
getPartitionKeyValue not found 0.03μs 0.03μs +0.0% 0 → 0 ⚪ noise
getPartitionKeyValue string 18.41μs 18.27μs -0.8% 1 → 1 ➡️
matchStreams found 17.32μs 17.27μs -0.3% 1 → 1 ➡️
matchStreams not found 0.05μs 0.05μs +0.0% 0 → 0 ⚪ noise

Summary: 🟢 1 faster · ➡️ 9 neutral · 🔴 1 slower · ⚪ 2 ignored (sub-μs)

Thresholds: <1μs ignore · 1–20μs 15% · 20–50μs 10% · ≥50μs 5%. Measured on a shared CI runner — treat small deltas as noise. Informational only; this check never fails the build.

@lukashes lukashes merged commit 6da9661 into main Jul 12, 2026
4 checks passed
@lukashes lukashes deleted the feat/validate-replica-identity branch July 12, 2026 19:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

REPLICA IDENTITY is not validated at startup

1 participant