MySQL Replication Formats

MySQL Replication: Statement based Replication vs Row based Replication

Before discussing the replication formats in MySQL it is necessary to discuss how replication works in MySQL. Why is replication required? The answer depends it may be for the process of scaling MySQL for reads, or to maintain disaster copy. Lets proceed by discussing the replication process. Replication process consists of the following steps:

1. MySQL Master writes any changes that occur to the 'binlog'. binlog is a log that contains any updates/inserts/deletes made on the master MySQL.

2. Slave's I/O thread reads the binlog from the master and writes the events in its 'relaylog'.3. The MySQL thread reads the events from the relaylog and applies those events to the slave.4. Steps 1 till 3 are repeated so the slave is synchronized with the master all the time.

Below is the diagram that will aid you in understanding the process of replication in MySQL.

Now that we are aware to some extent about the process of replication, lets dive deep into the replication formats. By replication format I mean the format in which events are recorded in master's binlog. There are three types of replication formats:

1. SBR or Statement based Replication

In this format of replication, the MySQL master, records the events as SQL statements into the binlog. The statements are picked up by the MySQL slaves and replayed in the same way as they are played at master

2. RBR or Row based Replication

In this format of replication, the MySQL master, records the events as actual rows that indicate how the rows are changed at the master.

3. MBR or Mixed Mode Replication

This format of replication is a mix of RBR and SBR. MySQL switches the format in real-time depending on the type of event. By default it used SBR and for unsafe statements it switches to RBR.

Of course there are pros and cons of each format, we will primarily address the RBR and SBR.

Advantages of statement-based replication

Proven technology that has existed in MySQL since 3.23. Less data written to log files. When updates or deletes affect many rows, this results in much less

storage space required for log files. This also means that taking and restoring from backups can be accomplished more quickly.

Log files contain all statements that made any changes, so they can be used to audit the database.

Disadvantages of statement-based replication

Statements that are unsafe for SBR. Not all statements which modify data (such as INSERT DELETE, UPDATE, and REPLACE statements) can be replicated using statement-based replication. Any nondeterministic behavior is difficult to replicate when using statement-based replication. Examples of such DML (Data Modification Language) statements include the following:

o A statement that depends on a UDF or stored program that is nondeterministic, since the value returned by such a UDF or stored program or depends on factors other than the parameters supplied to it. (Row-based replication, however, simply replicates the value returned by the UDF or stored program, so its effect on table rows and data is the same on both the master and slave.)

o DELETE and UPDATE statements that use a LIMIT clause without an ORDER BY are nondeterministic.

o Statements using any of the following functions cannot be replicated properly using statement-based replication:

LOAD_FILE() UUID() , UUID_SHORT() USER() FOUND_ROWS() SYSDATE() (unless both the master and the slave are started with the --

sysdate-is-now option) GET_LOCK() IS_FREE_LOCK() IS_USED_LOCK()

MASTER_POS_WAIT() RAND() RELEASE_LOCK() SLEEP() VERSION()

However, all other functions are replicated correctly using statement-based replication, including NOW() and so forth.

Statements that cannot be replicated correctly using statement-based replication are logged with a warning like the one shown here:

o [Warning] Statement is not safe to log in statement format.

o A similar warning is also issued to the client in such cases. The client can display it using SHOW WARNINGS.

INSERT ... SELECT requires a greater number of row-level locks than with row-based replication.

UPDATE statements that require a table scan (because no index is used in the WHERE clause) must lock a greater number of rows than with row-based replication.

For InnoDB: An INSERT statement that uses AUTO_INCREMENT blocks other nonconflicting INSERT statements.

For complex statements, the statement must be evaluated and executed on the slave before the rows are updated or inserted. With row-based replication, the slave only has to modify the affected rows, not execute the full statement.

If there is an error in evaluation on the slave, particularly when executing complex statements, statement-based replication may slowly increase the margin of error across the affected rows over time.

Deterministic UDFs must be applied on the slaves. Table definitions must be (nearly) identical on master and slave.

Advantages of row-based replication

All changes can be replicated. This is the safest form of replication.

The mysql database is not replicated. The mysql database is instead seen as a node-specific database. Row-based replication is not supported on tables in this database. Instead, statements that would normally update this information—such as GRANT, REVOKE and the manipulation of triggers, stored routines (including stored procedures), and views—are all replicated to slaves using statement-based replication.

For statements such as CREATE TABLE ... SELECT, a CREATE statement is generated from the table definition and replicated using statement-based format, while the row insertions are replicated using row-based format.

The technology is the same as in most other database management systems; knowledge about other systems transfers to MySQL.

Fewer row locks are required on the master, which thus achieves higher concurrency, for the following types of statements:

o INSERT ... SELECT o INSERT statements with AUTO_INCREMENT o UPDATE or DELETE statements with WHERE clauses that do not use keys or do not

change most of the examined rows.

Fewer row locks are required on the slave for any INSERT, UPDATE, or DELETE statement.

Disadvantages of row-based replication

RBR tends to generate more data that must be logged. To replicate a DML statement (such as an UPDATE or DELETE statement), statement-based replication writes only the statement to the binary log. By contrast, row-based replication writes each changed row to the binary log. If the statement changes many rows, row-based replication may write significantly more data to the binary log; this is true even for statements that are rolled back. This also means that taking and restoring from backup can require more time. In addition, the binary log is locked for a longer time to write the data, which may cause concurrency problems.

Deterministic UDFs that generate large BLOB values take longer to replicate with row-based replication than with statement-based replication. This is because the BLOB column value is logged, rather than the statement generating the data.

You cannot examine the logs to see what statements were executed, nor can you see on the slave what statements were received from the master and executed.

However, you can see what data was changed using mysqlbinlog with the options --base64-output=DECODE-ROWS and --verbose.

For tables using the MyISAM storage engine, a stronger lock is required on the slave for INSERT statements when applying them as row-based events to the binary log than when applying them as statements. This means that concurrent inserts on MyISAM tables are not supported when using row-based replication.

After comparison of both the formats, Row based Replication in MySQL is the way to go because:

Its the safest form of replication It is safer when it comes to replicating triggers and stored procedures. It requires fewer locks and hence its much faster and achieves high concurrancy. Regarding the drawback of having large binlog, I would say that not all applications may issue

updates that effect thousands of rows.

Please Note: The choice of which format to choose depends on the application and requirements. If its heavy IO intended many writes , updates or deletes happen on bulk data then its best to choose mixed formatting as it switches to RBR in case of unsafe statements .

How to read binlog in SBR :mysqlbinlog <bin_log_name>

How to read binlog in RBR :mysqlbinlog -v --base64-output=DECODE-ROWS <bin_log_name>

How to change binlog formatting : This needs instance bounce

yinst set mysql_config_multi.binlog_format='ROW';yinst set mysql_config_multi.binlog_format='STATEMENT';yinst set mysql_config_multi.binlog_format='MIXED’;

ROW VS STATEMENT BASED REPLICTION EXAMPLE 1 :

mysql> insert into test1 values('RAANI');Query OK, 1 row affected (0.00 sec)


mysql> commit;Query OK, 0 rows affected (0.00 sec)

mysql> select * from rajani.test1;+-------+| user |+-------+| RAANI || RAANI |+-------+2 rows in set (0.00 sec)

mysql> select * from rajani.test1;+-------+| user |+-------+| RAANI || RAANI |+-------+2 rows in set (0.00 sec)

mysql> delete from rajani.test1;Query OK, 2 rows affected (0.00 sec)

mysql> exitBye[root@rivershiver backup]#

*******************On slave mysql> insert into test1 values('RAANI');Query OK, 1 row affected (0.00 sec)


mysql> select * from rajani.test1;

+-------+| user |+-------+| RAANI || RAANI || RAANI || RAANI |+-------+4 rows in set (0.00 sec)


mysql> select * from rajani.test1;Empty set (0.00 sec)

mysql>

****************NOW ROW BASEDroot@rivershiver backup]# yinst set mysql_config_multi.binlog_format='ROW';

mysql> \! hostname rivershiver.corp.gq1.yahoo.commysql> show variables like 'bin%';+-----------------------------------------+-------+| Variable_name | Value |+-----------------------------------------+-------+| binlog_cache_size | 32768 || binlog_direct_non_transactional_updates | OFF || binlog_format | ROW || binlog_stmt_cache_size | 32768 |+-----------------------------------------+-------+4 rows in set (0.00 sec)

mysql> use rajani;Database changedmysql> insert into test1 values('RAANI');Query OK, 1 row affected (0.02 sec)

mysql> select * from rajani.test1;+-------+| user |+-------+| RAANI |+-------+

1 row in set (0.00 sec)

mysql> delete from rajani.test1;Query OK, 1 row affected (0.02 sec)

mysql> \! hostnameacquirewire.corp.gq1.yahoo.commysql> show variables like 'bin%';+-----------------------------------------+-------+| Variable_name | Value |+-----------------------------------------+-------+| binlog_cache_size | 32768 || binlog_direct_non_transactional_updates | OFF || binlog_format | ROW || binlog_stmt_cache_size | 32768 |+-----------------------------------------+-------+4 rows in set (0.00 sec)

mysql> use rajani;Database changedmysql> insert into test1 values('RA');Query OK, 1 row affected (0.04 sec)

mysql> select * from test1;+-------+| user |+-------+| RAANI || RA |+-------+2 rows in set (0.00 sec)

mysql> select * from test1;+------+| user |+------+| RA |+------+1 row in set (0.00 sec)

EXAMPLE2 : USING USER() FUNCTION TO INSERT DATA :

mysql> use rajani

Database changedmysql> insert into test1( select user() );Query OK, 1 row affected, 1 warning (0.02 sec)Records: 1 Duplicates: 0 Warnings: 1

mysql> show warnings;+-------+------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+| Level | Code | Message |+-------+------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+| Note | 1592 | Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. Statement is unsafe because it uses a system function that may return a different value on the slave. |+-------+------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+1 row in set (0.00 sec)

mysql> select * from test1;+----------------+| user |+----------------+| root@localhost |+----------------+1 row in set (0.00 sec)

mysql> insert into test1 values('RAJANI');Query OK, 1 row affected (0.00 sec)


mysql> select * from test1;+----------------+| user |+----------------+| root@localhost || RAJANI |+----------------+2 rows in set (0.00 sec)

mysql>

-- On slave mysql> show slave status\G*************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: rivershiver.corp.gq1.yahoo.com Master_User: repl Master_Port: 3306 Connect_Retry: 60 Master_Log_File: mysqld-bin.000018 Read_Master_Log_Pos: 1574 Relay_Log_File: mysqld-relay-bin.000044 Relay_Log_Pos: 1721 Relay_Master_Log_File: mysqld-bin.000018 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 1574 Relay_Log_Space: 1922 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 1735853001 row in set (0.00 sec)

mysql> select * from test1;+--------+| user |+--------+| || RAJANI |+--------+2 rows in set (0.00 sec)

**********************************RBR

mysql> insert into test1( select user() );Query OK, 1 row affected (0.00 sec)Records: 1 Duplicates: 0 Warnings: 0


mysql> select * from test1;+----------------+| user |+----------------+| root@localhost || RAJANI || root@localhost |+----------------+3 rows in set (0.00 sec)

mysql> \! hostnameacquirewire.corp.gq1.yahoo.commysql> select * from test1;+----------------+| user |+----------------+| || RAJANI || root@localhost |+----------------+3 rows in set (0.00 sec)

mysql>

***********WITH MIXED replication :

On master :::

mysql> use rajaniDatabase changedmysql> \! hostnamerivershiver.corp.gq1.yahoo.com



mysql> select * from rajani.test1;+----------------+| user |+----------------+| root@localhost |+----------------+1 row in set (0.00 sec)

mysql> delete from rajani.test1;Query OK, 1 row affected (0.00 sec)


*****ON salve : mysql>mysql> use rajaniDatabase changedmysql> \! hostnameacquirewire.corp.gq1.yahoo.com


mysql> select * from rajani.test1;+----------------+| user |+----------------+| root@localhost |+----------------+1 row in set (0.00 sec)


mysql> select * from rajani.test1;+----------------+| user |+----------------+| root@localhost || root@localhost |+----------------+2 rows in set (0.00 sec)


Binlog : [root@rivershiver data]# mysqlbinlog -v --base64-output=DECODE-ROWS mysqld-bin.000023/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;/*!40019 SET @@session.max_insert_delayed_threads=0*/;/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;DELIMITER /*!*/;# at 4#131206 21:30:18 server id 173585300 end_log_pos 107 Start: binlog v 4, server v 5.5.30-log created 131206 21:30:18# Warning: this binlog is either in use or was not closed properly.# at 107#131206 21:30:36 server id 173585300 end_log_pos 177 Query thread_id=3

exec_time=0 error_code=0SET TIMESTAMP=1386365436/*!*/;SET @@session.pseudo_thread_id=3/*!*/;SET @@session.foreign_key_checks=1, @@session.sql_auto_is_null=0, @@session.unique_checks=1, @@session.autocommit=1/*!*/;SET @@session.sql_mode=0/*!*/;SET @@session.auto_increment_increment=1, @@session.auto_increment_offset=1/*!*/;/*!\C utf8 *//*!*/;SET @@session.character_set_client=33,@@session.collation_connection=33,@@session.collation_server=33/*!*/;SET @@session.lc_time_names=0/*!*/;SET @@session.collation_database=DEFAULT/*!*/;BEGIN

/*!*/;# at 177# at 225#131206 21:30:36 server id 173585300 end_log_pos 225 Table_map: `rajani`.`test1` mapped to number 33#131206 21:30:36 server id 173585300 end_log_pos 270 Write_rows: table id 33 flags: STMT_END_F### INSERT INTO rajani.test1### SET### @1='root@localhost'# at 270#131206 21:30:36 server id 173585300 end_log_pos 297 Xid = 29COMMIT/*!*/;# at 297#131206 21:30:50 server id 173585300 end_log_pos 367 Query thread_id=3

exec_time=0 error_code=0SET TIMESTAMP=1386365450/*!*/;BEGIN/*!*/;# at 367#131206 21:30:50 server id 173585300 end_log_pos 456 Query thread_id=3

exec_time=0 error_code=0use `rajani`/*!*/;SET TIMESTAMP=1386365450/*!*/;delete from rajani.test1/*!*/;# at 456#131206 21:30:50 server id 173585300 end_log_pos 483 Xid = 30COMMIT/*!*/;DELIMITER ;# End of log fileROLLBACK /* added by mysqlbinlog */;/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;[root@rivershiver data]#

EXAMPLE4 : USING uuid() FUNCTION TO INSERT DATA :

mysql> \! hostnamerivershiver.corp.gq1.yahoo.commysql> show variables like "binlog_format";+---------------+-----------+| Variable_name | Value |+---------------+-----------+| binlog_format | STATEMENT |+---------------+-----------+

1 row in set (0.00 sec)

mysql> insert into test1 set user=UUID();Query OK, 1 row affected, 2 warnings (0.00 sec)

mysql> show warnings;+---------+------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+| Level | Code | Message |+---------+------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+| Warning | 1265 | Data truncated for column 'user' at row 1 || Note | 1592 | Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. Statement is unsafe because it uses a system function that may return a different value on the slave. |+---------+------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+2 rows in set (0.00 sec)

mysql> select * from test1;+---------------------------+| user |+---------------------------+| dae9bf54-61cc-11e3-ad8d-f |+---------------------------+1 row in set (0.00 sec)

On slave :mysql> \! hostnameacquirewire.corp.gq1.yahoo.commysql> show variables like "binlog_format";+---------------+-----------+| Variable_name | Value |+---------------+-----------+| binlog_format | STATEMENT |+---------------+-----------+1 row in set (0.00 sec)

mysql> use rajaniDatabase changedmysql> select * from test1;

+---------------------------+| user |+---------------------------+| db631af0-61cc-11e3-b7b4-f | -- diff value+---------------------------+1 row in set (0.00 sec)

mysql>

Binlog : #131210 18:57:02 server id 173585300 end_log_pos 275 Query thread_id=4

exec_time=0 error_code=0use `rajani`/*!*/;SET TIMESTAMP=1386701822/*!*/;insert into test1 set user=UUID() --- Same statement got executed so its getting diff value . /*!*/;# at 275#131210 18:57:02 server id 173585300 end_log_pos 302 Xid = 48COMMIT/*!*/;DELIMITER ;# End of log fileROLLBACK /* added by mysqlbinlog */;/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;[root@rivershiver data]#

ROW BASED REPLICATION :mysql> \! hostnamerivershiver.corp.gq1.yahoo.commysql> show variables like "binlog_format";+---------------+-------+| Variable_name | Value |+---------------+-------+| binlog_format | ROW |+---------------+-------+1 row in set (0.00 sec)

mysql> use rajaniDatabase changedmysql> select * from test1;Empty set (0.00 sec)

mysql> insert into test1 set user=UUID();Query OK, 1 row affected, 1 warning (0.02 sec)

mysql> show warnings;+---------+------+-------------------------------------------+| Level | Code | Message |+---------+------+-------------------------------------------+| Warning | 1265 | Data truncated for column 'user' at row 1 |+---------+------+-------------------------------------------+1 row in set (0.00 sec)

mysql> select * from test1;+---------------------------+| user |+---------------------------+| f18e9b03-61cd-11e3-b6cc-f |+---------------------------+1 row in set (0.00 sec)

mysql>

On slave : mysql> \! hostnameacquirewire.corp.gq1.yahoo.commysql> show variables like "binlog_format";+---------------+-------+| Variable_name | Value |+---------------+-------+| binlog_format | ROW |+---------------+-------+1 row in set (0.00 sec)

mysql> use rajaniDatabase changedmysql> select * from test1;Empty set (0.00 sec)

mysql> select * from test1;+---------------------------+| user |+---------------------------+| f18e9b03-61cd-11e3-b6cc-f |+---------------------------+1 row in set (0.00 sec)

mysql>

Bin log :

SET TIMESTAMP=1386702290/*!*/;BEGIN/*!*/;# at 774# at 822#131210 19:04:50 server id 173585300 end_log_pos 822 Table_map: `rajani`.`test1` mapped to number 33#131210 19:04:50 server id 173585300 end_log_pos 878 Write_rows: table id 33 flags: STMT_END_F### INSERT INTO rajani.test1### SET### @1='f18e9b03-61cd-11e3-b6cc-f'# at 878#131210 19:04:50 server id 173585300 end_log_pos 905 Xid = 39COMMIT/*!*/;DELIMITER ;# End of log fileROLLBACK /* added by mysqlbinlog */;/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;[root@rivershiver data]#

Documents

MySQL Replication Formats