Information consultation

[Jimerson21]

Hello, nice to greet you.

I would like to know what are the available commands and use case for each one.

usage: mysql_ch_replicator [-h] [--config CONFIG] [--db DB] [--target_db TARGET_DB] [--wait_initial_replication WAIT_INITIAL_REPLICATION] [--initial_only INITIAL_ONLY]
{run_all,binlog_replicator,db_replicator,monitoring,db_optimizer}

positional arguments:
{run_all,binlog_replicator,db_replicator,monitoring,db_optimizer}   run mode

options:
  -h, --help            show this help message and exit
  --config CONFIG       config file path
  --db DB               source database(s) name
  --target_db TARGET_DB target database(s) name, if not set will be same as source
  --wait_initial_replication WAIT_INITIAL_REPLICATION
  --initial_only INITIAL_ONLY don't run realtime replication, run initial replication only

Thank you!

[bakwc]

We have two primary commands:

run_all - this is for realtime replication (first it copy what already exists in mysql into clickhouse, and then run a realtime replication, for every change in mysql it makes it on clickhouse).
This command automatically launches several processes that do all the job: binlog_replicator, db_replicator for each of the database and db_optimizer.

db_replicator - with a flag --initial_only is for single time data copy, when you only need copy data once and don't need realtime updates.

Then there is also a monitoring command - it show a difference between last transaction in mysql, and last transaction in clickhouse. You should also pass a --db flag to use it. It could be used to ensure that there is no lag, usually difference should be zero. If it's growing - that means some problem.

Last two commands are for internal usage and not intended to run manually.

binlog_replicator - it copy binlog from mysql server into a local file on server where it runs. Later another process (binlog_replicator read this files and insert all transactions into mysql)

db_optimizer - it calls OPTIMIZE on ch tables once in a few hours, this is needed to avoid clikhouse tables growing infinitely in terms of disk usage.

[Jimerson21]

Thank you very much @bakwc for the information, it is now much clearer what I can do with this incredible tool.