Skip to content

Support row-level filtering by date range during initial replication #260

Description

@simplecrm-github

Use Case Description

I want to replicate data from MySQL into ClickHouse for specific tables, but only for rows that fall within a defined date range (for example, created_at BETWEEN '2026-01-01' AND '2026-03-31').
Currently, mysql_ch_replicator supports filtering by databases and tables, but it does not appear to support row-level filtering based on column values such as timestamps. This limits scenarios where only partial historical data needs to be synced.

Proposed Solution

Introduce a configuration option (e.g., row_filter or where_clause) that allows users to specify a SQL condition for initial replication.
Example:

tables:
  - name: orders
    row_filter: "created_at >= '2026-01-01' AND created_at < '2026-04-01'"


## Additional Context
This feature would be useful for compliance-driven environments where only a subset of historical data is required in ClickHouse.

It would also reduce load during initial replication by avoiding unnecessary rows.

Similar functionality exists in other replication tools (e.g., Debezium supports snapshot filters), so adding it here would align mysql_ch_replicator with broader ecosystem practices.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions