Use Case Description
I want to replicate data from MySQL into ClickHouse for specific tables, but only for rows that fall within a defined date range (for example, created_at BETWEEN '2026-01-01' AND '2026-03-31').
Currently, mysql_ch_replicator supports filtering by databases and tables, but it does not appear to support row-level filtering based on column values such as timestamps. This limits scenarios where only partial historical data needs to be synced.
Proposed Solution
Introduce a configuration option (e.g., row_filter or where_clause) that allows users to specify a SQL condition for initial replication.
Example:
tables:
- name: orders
row_filter: "created_at >= '2026-01-01' AND created_at < '2026-04-01'"
## Additional Context
This feature would be useful for compliance-driven environments where only a subset of historical data is required in ClickHouse.
It would also reduce load during initial replication by avoiding unnecessary rows.
Similar functionality exists in other replication tools (e.g., Debezium supports snapshot filters), so adding it here would align mysql_ch_replicator with broader ecosystem practices.
Use Case Description
I want to replicate data from MySQL into ClickHouse for specific tables, but only for rows that fall within a defined date range (for example,
created_at BETWEEN '2026-01-01' AND '2026-03-31').Currently, mysql_ch_replicator supports filtering by databases and tables, but it does not appear to support row-level filtering based on column values such as timestamps. This limits scenarios where only partial historical data needs to be synced.
Proposed Solution
Introduce a configuration option (e.g.,
row_filterorwhere_clause) that allows users to specify a SQL condition for initial replication.Example: