feat: composite primary key support in mysql_replicator (#7) - #55
Merged
Conversation
`primary_key` now accepts a comma-separated list of columns. A row's id
becomes the array of those column values, used for change detection and as
the Elasticsearch document _id (values joined by ","). A single-column key
(the default `id`) behaves exactly as before.
- in_mysql_replicator: parse `primary_key` as an array, key the change-
detection hash on the composite id via `extract_id`, check every component
for nil, and emit delete records as {col => val} for each key column.
- out_mysql_replicator_elasticsearch: build the document _id from one or more
key columns (`join_id`); backward compatible with single keys and with
mysql_replicator_multi (which keeps a single-column primary key).
Reimplements the idea from #7 (by @k4200) on the current codebase and fixes
its `table_hash[row[id]]` lookup bug. Adds unit tests for both plugins.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements composite primary key support for
mysql_replicator, requested in #7.primary_keynow accepts a comma-separated list of columns:Change detection (insert/update/delete) then keys on the combination of
those columns, and the Elasticsearch document
_idbecomes their values joinedby
,(e.g.10,7). A single-columnprimary_key(the defaultid) behavesexactly as before.
Changes
in_mysql_replicator:primary_keyparsed as an array; a row's id isextract_id(row)(the array of key-column values), used as the change-detection hash key; missing-key check covers every component; delete records
are emitted as
{col => val}for each key column.out_mysql_replicator_elasticsearch: builds the_idfrom one or morekey columns via
join_id. Backward compatible with single keys and withmysql_replicator_multi(which still uses a single-column key).Relationship to #7
This reimplements the idea from #7 (by @k4200) on the current codebase
(which has diverged substantially since 2015) and fixes a bug in the
original: its update branch read
table_hash[row[id]](alwaysnil) instead oftable_hash[id]. #7 can be closed as superseded by this PR.Tests
primary_keydefaults to['id'], parsestenant_id,id, andextract_idreturns single/composite values._id(10,7).Verified locally on Ruby 3.4 (input
18 tests, ES output19 tests, 0 failures).Scope note:
mysql_replicator_multiis unchanged (its integer-gap deletedetection assumes a single numeric key); composite support there could be a
follow-up.
🤖 Generated with Claude Code