MySQL Replication Tips and Tricks - O'Reilly Mediaassets.en.oreilly.com/1/event/21/Replication Tricks _ Tips Presentatio… · 2009-04-23 | Lars Thalmann & Mats Kindahl | Replication

2009-04-23 | Lars Thalmann & Mats Kindahl | Replication Tricks and Tips | © MySQL AB 2007-9 | www.mysql.com 1

MySQL ReplicationTips and Tricks

Dr. Mats KindahlLead Developer, Replication

[email protected]

mysqlmusings.blogspot.com

Dr. Lars ThalmannDevelopment Manager, Replication and Backup

[email protected]

larsthalmann.blogspot.com

MySQL Conference and ExpoApril 23th, 2009


• Some old, some new• Some easy, some more advanced

• Meant to be “food for thoughts”

Tricks and Tips

L


• Check position it stopped on– SHOW SLAVE STATUS– Next event to transfer from master:

– Master_Log_File, Read_Master_Log_Pos– Next event to apply:

– With respect to master log: Relay_Master_Log_File, Exec_Master_Log_Pos

– With respect to relay log:

Relay_Log_File, Relay_Log_Pos• Use mysqlbinlog to read the contents

– mysqlbinlog master-log.000001– mysqlbinlog relay-log.000001

• Investigate the problem, and possibly:– Remove rows in database; START SLAVE– SET SQL_SLAVE_SKIP_COUNTER=1; START SLAVE

Slave stopped – now what?

M


$ mysqlbinlog --hexdump master-bin.000001

# at 235

#060420 20:16:02 server id 1 end_log_pos 351

# Position Timestamp Type Master ID

# 000000eb e2 cf 47 44 02 01 00 00 00

# Size Master Pos Flags

# 74 00 00 00 5f 01 00 00 10 00

Examining binlogmysqlbinlog --hexdump master-bin.000001Statement-based INSERT1/2: Query event header

L


# 000000fe 02 00 00 00 00 00 00 00

# 04 00 00 1a 00 00 00 40 |................|

# 0000010e 00 00 ... |.............std|

# 0000011e 04 08 ... |.......test.INSE|

# 0000012e 52 54 ... |RT.INTO.t1.VALUE|

# 0000013e 53 20 ... |S...A...B......X|

# 0000014e 27 2c ... |...Y......X...X.|

# 0000015e 29 |.|

# Query thread_id=2 exec_time=0 error_code=0

SET TIMESTAMP=1145556962;

INSERT INTO t1 VALUES ('A','B'), ('X','Y'), ('X','X');

Examining binlog mysqlbinlog --hexdump master-bin.000001

Statement-based INSERT 2/2: Query event data

L


$ mysqlbinlog --verbose master-bin.000001

BINLOG '

qZnvSRMBAAAAKQAAAAYCAAAAABAAAAAAAAAABHRlc3QAAnQxAAEDAAE=

qZnvSRcBAAAAJwAAAC0CAAAQABAAAAAAAAEAAf/+AwAAAP4EAAAA

'/*!*/;

### INSERT INTO test.t1

### SET

### @1=3

### INSERT INTO test.t1

### SET

### @1=4

Examining binlogmysqlbinlog --verbose master-bin.000001Reconstructing INSERT from row-based events

L


• Replicate to a slave server– So that all data is there

• Stop replication

– To stop changes to the slave

• Do backup on the slave– It does not matter if this is blocking since the master is not blocked

Replication to get online backup

MySQLServer

MySQLServer

Backup

M

This technique is usedby the MySQL Time Machine


Extra slave to initiate new slaves

MySQLServer

MySQLServer

MySQLServer

MySQLServer

New slaveExtra slave

MySQLServer

L


Dump data from a server “master” and restore it on another server “slave”:

mysqldump --host=master –uuser –-master-data --lock-all-tables --databases test |mysql --host=slave –uuser

This locks the database!

Using mysqldump to inititialize slave

M


To avoid locking the entire table:

mysqldump --host=master –uuser –-single-transaction -–master-data --databases test |mysql --host=slave –uuser

Note: only works for tables that can handle consistent snapshots, such as InnoDB

Using mysqldump to inititialize slave

M


• Write a script that, for each table t1:

SELECT * INTO OUTFILE ‘t1.txt’ FROM t1;DROP TABLE IF EXISTS t1;CREATE TABLE t1 …;LOAD DATA INFILE ‘t1.txt’ INTO TABLE t1;

• A version using INSERT-SELECT (works for row-based):

CREATE TEMORARY TABLE t1_tmp LIKE t1;INSERT INTO t1_tmp SELECT * FROM t1;DROP TABLE IF EXISTS t1;CREATE TABLE t1 SELECT * FROM t1_tmp;

For row-based replication, the temporary table is not transferred to the slave, so this will only send the data that is necessary to bootstrap the table.

Using replication to repopulate a table

M

Works with statement-based replication

Works with row-based replication


For point in time recovery, you can take the replication log and apply it into the MySQL.

mysqlbinlog \--read-from-remote-server \

--host=master -uroot --position=192 \--to-datetime=”2009-04-11 12:36:56” \master-bin.00002[2-4] |

mysql -uroot --host=slave

Point-in-time recovery using mysqlbinlog

L


Proxy to do multi-master

MySQLServer

MySQLServer

MySQLServer

MySQLProxy

MySQLServer

M

MySQLServer

MySQL Proxy writes a new binary log


Use a default engine for a server

SET GLOBAL STORAGE_ENGINE = INNODB;

CREATE TABLE ( uid INT AUTO_INCREMENT PRIMARY KEY, name VARCHAR(64));

CREATE TABLE ( uid INT AUTO_INCREMENT PRIMARY KEY, name VARCHAR(64)) ENGINE=INNODB;

Not replicated, so remember to set default engine on slaves!


Split replication stream on table

MySQLServer

MySQLServer

RelaySlave

MySQLServer

M

MySQLServer

MySQLServer

MySQLServer

RelaySlave

MySQLServer

ALTER TABLE xxx ENGINE = BLACKHOLESET GLOBAL STORAGE_ENGINE = BLACKHOLE

replicationdotable=east_coast replicationdotable=west_coast


MySQL Cluster multi-source replication

MySQLCluster

MySQLCluster

MySQLCluster

MySQLServer

MySQLServer

MySQLServer

MySQLServer

Let masters updatedifferent data

L


Multi-channel replication with fail-over

MySQLServerMySQL

ClusterMySQLClusterMySQL

Server

MySQLServer

MySQLServer

L


Slave having time-sharing masters

MySQLServer

MySQLServer

MySQLServer

Clockedswitching

1. Stop slave2. Save position3. Change master4. Start slave

M


Stop the slave and save positionusing sh(1)

M

socket="mysql-test/var/tmp/$1.sock"

mysql_exec () { # Read commands from stdin client/mysql --vertical --socket="$1" --batch \ --skip-column-names --batch –user=root}

stop_slave () { echo STOP SLAVE | mysql_exec "$1"; }

fetch_pos () { echo SHOW SLAVE STATUS | mysql_exec "$1" | grep '\<Master_$Host\|Port\|Log_File$\|\<Read_Master_Log_Pos' | cut -f2 -d:}

stop_slave "$socket"echo `fetch_pos "$socket"` >$1.savepos


Restore the slave position and start slaveusing sh(1)

M

...

start_slave () { echo START SLAVE | mysql_exec "$1"; }

change_to_pos () { mysql_exec "$1" <<EOF # Pass command using here-isCHANGE MASTER TO MASTER_HOST='$2', MASTER_PORT=$3, MASTER_LOG_FILE='$4', MASTER_LOG_POS=$5EOF}

cat $1.savepos | { read host port file pos change_to_pos $socket "$host" "$port" "$file" "$pos" start_slave "$socket"}

Server name


Rotate between several slavesusing sh(1)

M

cnt=1

while true; do save_pos_and_stop_slave mysqld.$cnt cnt=`expr $cnt % 5 + 1` restore_pos_and_start_slave mysqld.$cnt}

Server names are of theform “mysqld.X”

Server savepos files are of theform “mysqld.X.savepos”


CHANGE MASTER TO

MASTER_SSL = 1,

MASTER_SSL_CA = 'cacert.pem'

MASTER_SSL_CERT = ‘mycert.pem‘

MASTER_SSL_KEY = ‘mykey.pem'

Encrypted replication

M


• In STATEMENT format, you can set current db to filter statements:– If --binlog-ignore-db=db1 is used on server,

then the following INSERT is not logged:USE foo; INSERT into bar.t1 VALUES(1);

– If --replicate-ignore-db=db1 is used on slave,then the following INSERT is not applied:USE foo; INSERT into bar.t1 VALUES(1);

• In ROW format, you can not use this trick, since row-based always use the actual database:

– The following INSERT is logged and applied if db1 is not filtered:USE foo; INSERT into db1.t1 VALUES(1);

Using current database to filter

M


You can add a column on the slave:

Master: CREATE TABLE t1 (a INT, b INT);

Slave: ALTER TABLE t1 ADD ts TIMESTAMP;

Master: INSERT INTO t1(a,b) VALUES (10,20);

More columns on slave than master

M

Row-based replication need extra column last

Extra columns have tohave default values!


In row-based replication, you can remove columns on the slave:

Master: CREATE TABLE t1 (a INT, b INT, comments TEXT);

Slave: ALTER TABLE t1 DROP comments;

Master: INSERT INTO t1

VALUES (1,2,”Do not store this on slave”);

Less columns on slave than master

M

Only possible to remove columns from the end


Replication Heartbeat

MySQL Server

SE2SE1

Storage Engines

Master

Binlog

Replication MySQL Server

SE2SE1

Slave

RelayBinlog

Binlog

I/Othread

New in

6.0/5.4


Replication Heartbeat

MySQLSlave

MySQL

Master

New in

6.0/5.4

• Automatic checking of connection status• No more relay log rotates when the master is idle• Detection of master/slave disconnect configurable in millisecs

CHANGE MASTER SET master_heartbeat_period= val;

SHOW STATUS like 'slave_heartbeat period'

SHOW STATUS like 'slave_received_heartbeats'


Ignoring Servers in Circular Replication

Circular replication

If server A is removed from the circle, server B can be set to terminate A's events in the new circle

Server B> CHANGE MASTER TO MASTER_HOST=C ... IGNORE_SERVER_IDS=(A)

MySQLServer C

MySQLServer A

MySQLServer B

MySQLServer D

New in

6.0/5.4


Replication database

c1 c2 c3

14

25

36

rpl_db

rpl_db

copy

replication

master

Slave

c1 c2 c3

14

25

36

c1 c2 c3

14

25

36

test

L


1. Two options to only replicate this part:

1. On master:

[mysqld]...binlog-do-db=rpl_db

Note that this filters the binlog on master

– On slave:[mysqld]...replicate-do-db=rpl_db

This filters the replication execution on the slave

• Note that you need row-based for this to work

– In statement-based replication, the source tables are need to execute the statement.


L



CREATE DATABASE rpl_db;

USE test; CREATE TABLE t1 (a INT, b INT, c INT);

USE rpl_db; CREATE TABLE t1 (a INT, b INT, c INT);

USE test; # Not replicated

INSERT INTO t1 VALUES (1,2,3), (2,5,9);

USE rpl_db; # Replicated

INSERT INTO t1 VALUES (3,4,5), (4,6,11);

L



a b c a c

a c

rpl_db

rpl_db

trigger

replication

master

slave

L


Master:CREATE DATABASE rpl_db;

USE test;

CREATE TABLE t1 (a INT, b BLOB, c INT);

CREATE TRIGGER tr_t1 AFTER INSERT ON test.t1 FOR EACH ROW

INSERT INTO rpl_db.t1_v(a,c) VALUES(NEW.a,NEW.c);

USE rpl_db;

CREATE TABLE t1_v (a INT, c INT);

Use like this:USE test;

SET @blob = REPEAT('beef',100);

INSERT INTO t1 VALUES (1,@blob,3), (2,@blob,9);


L


Only replicate some rows to slave

a b c

14

25

36

rpl_db

rpl_db

trigger

replication

master

Slave

a b c

4 5 6

a b c

4 5 6

M


# Just replicate rows that has odd numbers in first column.

USE rpl_db;

CREATE TABLE t1_h (a INT, b BLOB, c INT);

--delimiter //

CREATE TRIGGER slice_t1_horiz AFTER INSERT ON test.t1

FOR EACH ROW

BEGIN

IF NEW.a MOD 2 = 1 THEN

INSERT INTO rpl_db.t1_h VALUES (NEW.a, NEW.b, NEW.c);

END IF;

END//

--delimiter ;

Only replicate some rows to master

M


Provides:

Execution of replication events on slave is to be N seconds behind master

User interface:

CHANGE MASTER TO MASTER_DELAY= N

N is a non-negative less than MAX_ULONG number of seconds to wait by the slave. (Attempts to set higher is rejected with error.)

Controlling of a delayed setup is via SHOW SLAVE STATUS “Seconds_behind_master”

Kudos: Kay Röpke, original patch; Sven, modifications

Feature PreviewTime-delayed replication

http://forge.mysql.com/wiki/ReplicationFeatures/DelayedReplication

L


Scriptable ReplicationHigh-level structure

http://forge.mysql.com/wiki/ReplicationFeatures/ScriptableReplication

Master Slave

BinaryLog

RelayLog

Dump I/O SQLClient

function befo local q = eve slave:query(q

end


end


end


end

Extension modules


The “oracle” algorithmPaul Tuckfield’s (Google/YouTube)

module(..., package.seeall); require “luasql.mysql”

pattern = {

["UPDATE%s+(%w+).*%s(WHERE.*)"] = "SELECT * FROM %1 %2",

["DELETE%s+FROM%s+(%w+).*%s(WHERE.*)"] = "SELECT * FROM %1 %2",

}

env = luasql.mysql()

con = env:connect("test", “root”, “”, “localhost”, mysql.port)

function before_write(event)

local line = event.query

if not line then return end

for pat,repl in pairs(pattern) do

local str = string.gsub(line, pat, repl)

if str then con:execute(str); break; end

end

end


Semi-synchronous ReplicationOriginal patch: Mark Callaghan and Wei Li, Google

Adoptions: Zhenxing He, Sun Microsystems

Application

MySQL Server

SE2SE1

Storage Engines

Master

Binlog

Replication MySQL Server

Application

Slave

RelayBinlog

L R R A

SE2SE1

Storage Engines

Logging/Replication

Semi-SyncReplicator

Relay Log/Applier

Semi-SyncReceiver

Ack


Install Plug-ins and PlayOn master:

INSTALL PLUGIN 'rpl_semi_sync_master' SONAME 'libsemisync_master.so';

SET rpl_semi_sync_master_enabled=1;SET rpl_semi_sync_master_timeout=1000;

On slave:INSTALL PLUGIN 'rpl_semi_sync_slave' SONAME 'libsemisync_slave.so';

SET rpl_semi_sync_slave_enabled=1;START SLAVE;

New in

6.0/5.4

milliseconds


Look into the DevZone

Dr. Mats KindahlLead Developer, Replication Technology

[email protected]

Dr. Lars ThalmannDevelopment Manager, Replication and Backup Technology

[email protected]

MySQL Replication DevZone

http://dev.mysql.com/replication

L

Documents

MySQL Replication Tips and Tricks - O'Reilly Mediaassets.en.oreilly.com/1/event/21/Replication Tricks _ Tips Presentatio… · 2009-04-23 | Lars Thalmann & Mats Kindahl | Replication