Quayen
Start free
Free tools

Is your MySQL ready for change data capture?

Run one query, paste the result, and see whether your MySQL server's binary log is ready for CDC to BigQuery, with the exact fix for your host for anything that isn't.

1. Run this query

SHOW GLOBAL VARIABLES WHERE Variable_name IN
  ('version', 'log_bin', 'binlog_format', 'binlog_row_image',
   'gtid_mode', 'binlog_expire_logs_seconds', 'expire_logs_days');

Checked in your browser; nothing is sent anywhere.

Paste the query result to see what to change.

What MySQL needs for change data capture

  • Binary logging on (log_bin = ON). Managed services turn it on with automated backups (RDS) or point-in-time recovery (Cloud SQL).
  • Row-based format (binlog_format = ROW) so each change records the actual row values. See binlog_format and binlog_row_image explained.
  • Full row images (binlog_row_image = FULL) so updates include every column.
  • Enough retention, at least 3 days, so a paused pipeline can resume without a full re-copy. See how long to keep binary logs.
  • A user with SELECT, REPLICATION SLAVE and REPLICATION CLIENT.

These are the same checks Quayen's connection test runs. For error messages you may already be seeing, see common MySQL CDC errors.

What you can paste

The checker reads the table printed by the mysql command line, vertical (\G) output, tab-separated output copied from a GUI such as MySQL Workbench or DBeaver, and my.cnf lines like binlog_format = ROW.

Questions

What MySQL settings does change data capture need?

log_bin ON, binlog_format ROW, binlog_row_image FULL, binary logs kept for at least a day (3 days recommended), and a user with SELECT, REPLICATION SLAVE and REPLICATION CLIENT.

Do I need GTIDs for MySQL CDC?

Not for Quayen. It works with gtid_mode ON or OFF.

How long should MySQL keep binary logs for CDC?

At least 3 days, ideally 7, so a pipeline that stops on a Friday evening can catch up on Monday without re-copying tables.

Why doesn't binlog_expire_logs_seconds change retention on Amazon RDS?

RDS and Aurora manage retention with 'binlog retention hours'. Set it with CALL mysql.rds_set_configuration('binlog retention hours', 72); and check it with CALL mysql.rds_show_configuration;.

Is what I paste sent anywhere?

No. The check runs entirely in your browser.

Your MySQL data in BigQuery today

Free during early access. Set it up yourself in about 15 minutes, or we'll do it with you on a call.

Start for free Book a guided setup