State of the art of MySQL replication and clustering

©Continuent 2012.

State of the Art in MySQL HA and Replication

Robert Hodges, CEOGiuseppe Maxia, Director of QA

1Tuesday, November 13, 12

©Continuent 2012

Introducing Continuent

2

• The leading provider of clustering and replication for open source DBMS

• Our Product: Continuent Tungsten

• Clustering - Commercial-grade HA, performance scaling and data management for MySQL

• Replication - Flexible, high-performance data movement


©Continuent 2012

Quick Continuent Facts

• Largest Tungsten clustering installation processes about 500M transactions each day

• Tungsten Replicator was application of the year at the 2011 MySQL User Conference

• Continuent is well known in the MySQL community for its clustering and replication

• Over 90 customers currently deployed

3


©Continuent 2012

A Few Continuent Customers

4


©Continuent 2012 5

Future-Proof HA and Replication for MySQL


©Continuent 2012

What is Future-Proo!ng?

• Future-proo!ng = creating systems that last while parts change and improve

• MySQL is not losing out to other solutions for data management

• The real problem is using MySQL as a building block with other technologies that are changing rapidly

6


©Continuent 2012

But Wait... Isn’t MySQL Dead??

7

MySQL“You were so open”

BornMay 25, 1995

Died2008, 2009, 2010,

2011, 2012, ...


©Continuent 2012

Oracle/MySQL 5.6 Replication (On the Way)

• Global transaction IDs

• Parallel slave threads

• Crash safe slaves

• Optimized row updates

• Time delay replication

• Informational log events

8


©Continuent 2012

Galera Synchronous Clusters (New Arrival in 2012)

• Multi-master for InnoDB

• Synchronous replication avoids data loss

• Automatic cluster membership management

• Simple node provisioning model

• Based on innovative state-machine work from Fernando Pedone

9


©Continuent 2012

Tungsten Async Replication (2011 Belle of the Ball)

• All 5.6 features including parallel replication for MySQL 5.0 onwards

• Optimized support for failover

• Programmable transaction !ltering

• Backup/restore integration

• Multi-master, fan-in, and star replication

• Replication to/from Oracle and batch loading of data warehouses

• Replication to MongoDB

10


©Continuent 2012

Not Bad for a Dying Market

11

MySQL,'88%'

NoSQL,'5%'

NewSQL,'2%'

My/NewSQL,'5%'

0"100"200"300"400"500"600"700"800"900"1000"

2011" 2012" 2013" 2014" 2015"

THE$TOTAL$MARKET$FOR$MYSQL,$NOSQL$$AND$NEWSQL$$

Text

(451Group)


©Continuent 2012

• Cloud and Big Data investment dwarf the MySQL marketplace

MySQL Is Part of Bigger Trends

12


©Continuent 2012

21st Century CRM/Call Center

13

CRM WebApplication

Telephony Data Streams

CRM WebApplication


Legacy Oracle

Fat ClientAccounting

Call Session Data

Hadoop Analytics

Online TransactionProcessing

Amazon Web ServicesOn-Premise


©Continuent 2012

Revolutionary Challenges for MySQL

• Failures in unstable cloud environments

• Zero-downtime maintenance

• Rapidly growing data volumes, esp. in cloud

• Distributing data to geographical regions

• Integration between MySQL, NoSQL, commercial RDBMS

• Supplying real-time analytics

• Technology upgrade/replacement

14


©Continuent 2012

Future-Proofed HA and Replication

15

Encapsulated, fault-tolerant, horizontally scalable, globally

accessible, integrated data

Data Fabric


©Continuent 2012

Data Fabric Design Patterns

• Fabric Connector

• Fault-Tolerant Data Service

• Sharded Data Service

• Multi-Site Data Service

• Real-Time Data Bridge

16


©Continuent 2012

• Single point of entry for applications

• Service catalog locates data within fabric

• Transparent connectivity, multiple protocols

• Security, auditing, performance management

1. Fabric Connector

17

FabricConnector

FabricConnector

ApplicationStack

ApplicationStack


©Continuent 2012

2. Fault-Tolerant Data Service

• Encapsulate redundant database copies

• Sync and async clustering models

• Protect against local DBMS failure

• Rolling maintenance of replicas

18

Master/Slave

Multi-Master


©Continuent 2012

3. Sharded Data Service

• Partition datasets based on RAM / storage speed / resident set size

• Multiple buckets per server

• Look-up methods for locating data in buckets

• Re-sharding / migration for high growth

19


©Continuent 2012

4. Multi-Site Data Service

• Robust protection against region/site failures

• Geographic distribution of data

• Primary/DR vs. multi-master

• Eventually consistent replication for SQL

20

Amazon East

Rackspace DFWAPAC TokyoUS East

EU West


©Continuent 2012

5. Real-Time Data Bridge

• Replicate from logs in real-time

• Enable apps to get data from one data source

• Heterogeneous transfer and transformation

• High performance, low application impact

21

Bridge

Bridge

Bridge


©Continuent 2012

Future-Proofed CRM/Call Center Data

22

CRM WebApplication

CRM WebApplication

Legacy Oracle


Bridge

Bridge

FabricConnector

FabricConnector

OLTP Primary

Data Marts

Bridge

Bridge HadoopAnalytics



FabricConnector

FabricConnector

FabricConnector

OLTP DR

Bridge

Call Session Data


©Continuent 2012

Future-Proofed CRM/Call Center Data

23

CRM WebApplication

CRM WebApplication

Legacy Oracle


Bridge

Bridge

FabricConnector

FabricConnector

OLTP Primary

Data Marts

Bridge

Bridge HadoopAnalytics



FabricConnector

FabricConnector

OLTP DR

Bridge

Call Session Data

FabricConnector


©Continuent 2012 24

Enabling Data Fabric With Tungsten


©Continuent 2012

Tungsten Fault Tolerant Data Service

25

Tungsten clusters combine o"-the-shelf open source DBMS servers into data services with:

• 24x7 data access • Scaling of load on replicas• Simple management commands

...without app changes or data migration

AmazonUS West

apache/php

GonzoPortal.com

Connector Connector


©Continuent 2012

Doesn’t Replication Do This Already?

26

Application

Libmysqlclient.a

Application

Libmysqlclient.a

Replicator

Db2

Replicator

Db3Db1

Replicator

Slave SlaveMaster


©Continuent 2012

Replication Limitations

Replication alone cannot enable constant access to data...

• What happens if the master fails?

• What happens if a slave is lagging too far behind to be usable?

• How do I maintain the DBMS without stopping apps?

27


©Continuent 2012

From Replicating to a Data Service

28

Application

Tungsten Connector

Application

Tungsten Connector

Replicator

Db2

Replicator

Db3Db1

Replicator

Slave SlaveMaster

Manager Manager Manager

Monitoring and Control

Monitoring and Control

Data Service: sjc


©Continuent 2012

Basic Operations

29

• Switch -- Promote a slave node to master

• Shun -- Kick node out of cluster

• Welcome -- Return node to cluster

• Backup -- Run a backup

• Restore -- [Re-]provision a node

• Recover -- Return a broken node to slave

• Heartbeat -- Check cluster liveness


©Continuent 2012

Automatic Failover

30

Application

Tungsten Connector


©Continuent 2012

Automatic Failover

31

Application

Tungsten Connector

1. Detect non-responsivemaster

2. Halt master connections


©Continuent 2012

Automatic Failover

32

Application

Tungsten Connector

3. Select most advanced slave and wait for pending transactions to apply


©Continuent 2012

Automatic Failover

33

Application

Tungsten Connector

4. Promote to master, reconfigure slave(s), and re-enable connections


©Continuent 2012

Automatic Failover

34

Application

Tungsten Connector

5. Recover failed master


©Continuent 2012

Rolling SQL Schema Upgrade

35

Final UpgradeSwitchSlave Upgrade Slave Upgrade

• Shun slave 1, run upgrade, and return to cluster. Discard and recover on failure

• Upgrade old master. Cluster schema is now fully upgraded

• Repeat for additional slave(s)

• Switch master to promote an upgraded slave


©Continuent 2012

Backup, Recovery, and Restore

36

• Backups

/cluster > datasource DB1 backup

• Recover

/cluster > datasource DB2 recover

• Restore

/cluster > datasource DB2 restore

• Out-of-box support for mysqldump/XtraBackup; simple integration for others



Data Fabric Connectivity


©Continuent 2012

Tungsten Connector Basics

• Any MySQL/PostgreSQL client can connect

• Initiates connections using JDBC

38

App NativeprotocolCOM_QUERY

COM_INIT_DBCOM_DROP_DB

Connector


©Continuent 2012

Tungsten Connector Basics

• Pass-through operation after connection

• Full transparency & low overhead for clients

39

App

Protocol packetCOM_QUERY

select * from t

ConnectorProtocol packet

OKResultSet Rows:1


©Continuent 2012

Simple Read/Write Splitting

• Very fast, easy-to-understand scaling model

• Transactions go to master

• Auto-commit SELECTs go to slaves, if available, but...

• Reads potentially outdated data

• Ephemeral SQL objects won’t be available

• May miss: select update_function()

40


©Continuent 2012

SmartScale Session Load Balancing

41

App

Connect/Write

Connector

TransactionHistory Log

Slavereplay

position

Session xSession y

Slave DBMS

MasterDBMS

Slave DBMS


©Continuent 2012


42

AppRead

Connector


Slavereplay

position

Session xSession y

Slave DBMS

MasterDBMS

Slave DBMS


©Continuent 2012


43

AppRead

Connector


Slavereplay

position

Session x

Slave DBMS

MasterDBMS

Slave DBMS



Multi-Site Data Service


©Continuent 2012

Multi-Master Pattern

45

• Updates on 2+ sites (active-active mode)

• Enables geographic distribution of data

• No failover necessary if network fails or site becomes unavailable

• Not all applications can handle multi-master

• Applications must avoid con!icts

• Careful testing required

• Restoration of broken systems may not be easy


©Continuent 2012

Simple Multi-Master Con!guration

46

NYC FrankfurtReplicator

fra (master)

nyc (slave)

Replicator

fra (slave)

nyc (master)

Database-to-Database


©Continuent 2012

Clustered Multi-Master Con!guration

47

NYC Frankfurt

master

slave fra

fra

nyc

nyc

master

slave


©Continuent 2012


48

NYC Frankfurt

slave

master fra

fra

nyc

nyc

master

slave

NYC Master Offline


©Continuent 2012


49

NYC Frankfurt

master

slave fra

fra

nyc

nyc

master

slave

NYC Replicator Offline


©Continuent 2012

Complex Multi-Master Topologies

50

StarReplication

AllMasters


©Continuent 2012

Primary/DR Pattern

51

• Updates go to primary site master

• DR site receives updates in real time

• Simple command promotion

• Tungsten connectors can route to primary and DR sites

• No application changes required

• Simplest way to protect against region/site failures


©Continuent 2012

Tungsten Composite Data Services

52

App Server+ Connector

App Server+ Connector

Master Slave

San Jose New York

sj_nyc



Real-Time Data Bridge


©Continuent 2012

Problem: Real-Time Order Ful!llment

54

Real-Time Replication

MySQL

Web-Facing Sales Purchase Ordering

Oracle

PHP/Perl

Java/JBoss

3M Order Items

1-2M transactions daily


©Continuent 2012

How Does Tungsten Do That?

55

MySQL Tungsten Master Replicator

Oracle

Service oracle

MySQLExtractorSpecial Filters* Transform enum to string

binlog_format=row

Tungsten Slave Replicator

Service oracle

Special Filters* Ignore extra tables* Map names to upper case* Optimize updates to remove unchanged columnsMySQL

Binlog


©Continuent 2012

Batch Loading to Data Warehouses

56

CsvFile

CsvFile

CsvFile. . .

London

Replicator

Service batch

VerticaData

Warehouse


MySQLMaster

THL


©Continuent 2012

Possibilities for Analytics...

57

OLTP Data

MySQL MasterComplex, near real-time reporting

Light-weight, real-time operational status

Web-facing mini-data marts for SaaS users


©Continuent 2012

Getting Started

58


©Continuent 2012

How Do I Obtain Tungsten?

59

• Register at Continuent to get downloads for Oracle replication: http://www.continuent.com/downloads/software

• Documentation is available on Continuent website

• MySQL replication is available from Tungsten Replicator project on code.google.comhttp://code.google.com/p/tungsten-replicator/


http://www.continuent.com/downloads/software

http://www.continuent.com/downloads/software

http://code.google.com/p/tungsten-replicator/

http://code.google.com/p/tungsten-replicator/

©Continuent 2012.

Continuent Web Page:http://www.continuent.com

Tungsten Replicator 2.0:http://code.google.com/p/tungsten-replicator

Our Blogs:http://scale-out-blog.blogspot.comhttp://datacharmer.org/bloghttp://www.continuent.com/news/blogs

560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel +1 (866) 998-3642 Fax +1 (408) 668-1009e-mail: [email protected]


http://www.continuent.com

http://www.continuent.com

http://code.google.com/p/tungsten-replicator

http://code.google.com/p/tungsten-replicator

http://scale-out-blog.blogspot.com

http://scale-out-blog.blogspot.com

http://datacharmer.blogspot.com

http://datacharmer.blogspot.com

http://continuent-tungsten.blogspot.com

http://continuent-tungsten.blogspot.com

mailto:[email protected]

mailto:[email protected]