24
Webinar: Scaling MySQL Catch 22 of Read Write Splitting January 17, 2013

Scaling MySQL: Catch 22 of Read Write Splitting

Embed Size (px)

DESCRIPTION

This webinar will explore the secrets, dos and don’ts, benefits and caveats behind Read-write Splitting.

Citation preview

Page 1: Scaling MySQL: Catch 22 of Read Write Splitting

Webinar: Scaling MySQL Catch 22 of Read Write Splitting

January 17, 2013

Page 2: Scaling MySQL: Catch 22 of Read Write Splitting

2

Agenda

1. Who We Are 2. The Scalability Problem

3. How We Solve it with Read/Write Splitting

4. Customer ROI/Case Studies

5. Q & A

(please type questions directly into the GoToWebinar side panel)

Page 3: Scaling MySQL: Catch 22 of Read Write Splitting

3

Who We Are

Presenters: Paul Campaniello,

VP of Global Marketing 25 year technology veteran with marketing experience at Mendix, Lumigent, Savantis and Precise.

Doron Levari, Founder A technologist and long-time

veteran of the database industry. Prior to founding ScaleBase, Doron

was CEO to Aluna.

Page 4: Scaling MySQL: Catch 22 of Read Write Splitting

4

Who We Are

ScaleBase allows apps

to cost-effectively scale

to an infinite number of users,

with NO disruption to the existing infrastructure

Page 5: Scaling MySQL: Catch 22 of Read Write Splitting

5

The ScaleBase Data Traffic Manager

• Database Scalability

– Scale out relational databases

to unlimited users

– Real-time elasticity

• Database Availability

– Enable high availability of all

apps

• Centralized Management

– Removes complexity and

provides a unified point of

management for distributed

database environments

• Improves performance

Requires NO changes to your existing infrastructure

Page 6: Scaling MySQL: Catch 22 of Read Write Splitting

6

Pain Points – The Scalability Problem

• Thousands of new online and mobile

apps launching every day

• Demand climbs for these apps and

databases can’t keep up

• App must provide uninterrupted

access and availability

• Database performance and

scalability is critical

Page 7: Scaling MySQL: Catch 22 of Read Write Splitting

7

Big Data = Big Scaling Needs

The 451 Group & Teradata

Big Data = Transactions + Interactions + Observations

BIG

DA

TA

ER

P

CR

M

WE

B

Petabytes

Terabytes

Gigabytes

Megabytes

Increasing Data Variety and Complexity

Purchase Detail

Purchase Record

Payment Record

Segmentation

Offer Details

Customer Touches

Support Contacts

Web Logs

Offer History A/B Testing

Dynamic Pricing

Affiliate Networks

Search Marketing

Behavioral

Targeting

Dynamic

Funnels

Sensors/RFID/Devices

User Click Stream

Mobile Web

Sentiment

User Generated Content

Social Interactions & Feeds

Spatial & GPS Coordinates

External

Demographics

Business Data

Feeds

HD Video, Audio, Images

Speech to Text

Product/Service Logs

SMS/MMS

Page 8: Scaling MySQL: Catch 22 of Read Write Splitting

8

Scalability Pain

You just lost

customers

Infrastructure Cost $

time

Large

Capital

Expenditure

Opportunity

Cost

Predicted Demand

Traditional Hardware

Actual Demand

Dynamic Scaling

Page 9: Scaling MySQL: Catch 22 of Read Write Splitting

9

Ongoing “Scaling MySQL” Series

• August 16 & September 20, 2012

– Scaling MySQL: ScaleUp versus Scale Out

• October 23, 2012

– Methods and challenges to Scale out MySQL

• December 13, 2012

– Benefits of Automatic Data Distribution

• Today

– Catch 22 of read-write splitting

Page 10: Scaling MySQL: Catch 22 of Read Write Splitting

10

The Database Engine is the Bottleneck...

• Every write operation is At Least 4 write operations inside the DB:

– Data segment

– Index segment

– Undo segment

– Transaction log

• And Multiple Activities in the DB engine memory:

– Buffer management

– Locking

– Thread locks/semaphores

– Recovery tasks

Page 11: Scaling MySQL: Catch 22 of Read Write Splitting

11

• Every write operation is At Least 4 write operations inside the DB:

– Data segment

– Index segment

– Undo segment

– Transaction log

• And Multiple Activities in the DB engine memory:

– Buffer management

– Locking

– Thread locks/semaphores

– Recovery tasks

The Database Engine is the Bottleneck

Now multiply

by 10TB accessed by

10000 concurrent

sessions

Page 12: Scaling MySQL: Catch 22 of Read Write Splitting

12

So… Let’s get to work!

• I’m growing, have more users, need to support more throughput thru scale-out

• Solution:

– Create replicated database servers

– Distribute the sessions across those database servers. Read / Write splitting:

– Reads use the slaves

– Writes go to the master

• Benefits:

– Better resource consumption

– Get more read throughput

– Get more write throughput

– Awesome!

Page 13: Scaling MySQL: Catch 22 of Read Write Splitting

13

Read / Write Splitting

Replication

Page 14: Scaling MySQL: Catch 22 of Read Write Splitting

14

If Done Alone…

• Application code needs to be changed

• Maintain 2 connection pools – Write – master

– Reads – A blend of all slaves

• Every flow, in its beginning, disclaims: – “I will do only reads”

– “I will do reads and writes”

– What’s the default?

• Result: – Writing code, maintaining code

– Maintaining database ops in the app: add/remove slaves, change IPs...

– Master database is far more occupied than it should be

– Reads are not well balanced

– What if replication breaks?

– Can I read stale data?

Page 15: Scaling MySQL: Catch 22 of Read Write Splitting

15

Read / Write Splitting with ScaleBase

• 0 code changes and 0 code maintenance

• Reads run faster

• Writes run faster

• Better resource utilization/load balancing

• Improved data consistency/transaction isolation

• Built-in failover with high availability

• Database aware, replication state aware, replication lag aware

• Real time monitoring and alerts

• Centralized management dashboard

Page 16: Scaling MySQL: Catch 22 of Read Write Splitting

16

Read / Write Splitting

Replication

Current

Replication

With ScaleBase

Application Experience

Page 17: Scaling MySQL: Catch 22 of Read Write Splitting

17

Read/Write Splitting

APPLICATION /

WEB SERVERS USERS

Replication

Write

Read

Page 18: Scaling MySQL: Catch 22 of Read Write Splitting

18

Scale Out with Amazon AWS RDS Read Replica

RDS Read Replicas

Current

RDS Read Replicas

With ScaleBase

Application Experience

Page 19: Scaling MySQL: Catch 22 of Read Write Splitting

19

Choose Your Scale-out Path

# of concurrent sessions

Dat

abas

e S

ize

1 DB?

Good for me!

Data Distribution

Read/Write Splitting

Page 20: Scaling MySQL: Catch 22 of Read Write Splitting

20

Scaling Out Achieves Unlimited Scalability

6000 12000

24000

36000

48000

60000

84000

500 500 1000

1500 1500 2000

2500

0

20000

40000

60000

80000

100000

120000

140000

160000

1 2 4 6 8 10 14

Thro

ugh

pu

t

Number of Databases

Throughput (TPM)

Total DB Size (MB)

# Connections

Page 21: Scaling MySQL: Catch 22 of Read Write Splitting

21

Detailed Scale Out Case Studies

• Device Apps App

• Availability

• Scalability

• Geo-clustering

• 100 Apps

• 300 MySQL DB

Solar Edge

• Next Gen Monitoring App

• Massive Scale

• Monitors real time data from thousands of distributed systems

Mozilla

• New Product/ Next Gen App/ AppStore

• Scalability

• Geo-sharding

AppDynamics

• Next gen APM company

• Scalability for the Netflix implementation

One of world's largest & most widely respected manufacturers of smart

phones and telecommunications hardware & software

Page 22: Scaling MySQL: Catch 22 of Read Write Splitting

22

Summary

• Database scalability is a significant problem

– App explosion, Big Data, Mobile

• Scale Up helps somewhat, but Scale Out provides

a long-term, cost-effective solution

• ScaleBase has an effective Scale Out

solution with a proven ROI

– Improves performance &

requires NO changes to

your existing infrastructure

• Choose your scale-out path....

– The ScaleBase platform enables

you to start with R/W splitting and

grow into automatic data distribution

Page 23: Scaling MySQL: Catch 22 of Read Write Splitting

23

Questions (please enter directly into the GTW side panel)

617.630.2800

www.ScaleBase.com

[email protected]

[email protected]

Page 24: Scaling MySQL: Catch 22 of Read Write Splitting

24

Thank You