MySQL Scaling for SCaLE 12x conference

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.1


Insert Picture Here

MySQL Scaling: Drinking from the fire hose, never being down & serving billionsDave StokesMySQL Community Manager

[email protected] @Stoker

Slideshare.net/davestokes/presentations

mailto:[email protected]


You want to get a bigger, badder MySQL --BUT what are the costs???

Budget Complexity Application Opportunity Sanity


Insert Picture Here

Simple Stuff

Insert Picture Here


Option 0

MySQL 5.5 was 20% faster than 5.1! 5.6 is another 10-15%--Make sure your config file matches your hardware--ANALYSE TABLE--Check for SLOW QUERIES, QUERIES w/o INDEXES--TUNE QUERIES

This is the least complex, less upsetting, and usually lowest budget cost option

Run the Latest, Greatest MySQL


Option 0.1

Cache for quick access● Session Data● Sports Scores● Opens up NoSQL access

to InnoDB later

This is the second least complex, less upsetting, and usually lowest budget cost option but does make you change your application to become 'cache aware'.

Add in a Memcached Layer


Insert Picture Here

What database service Levels do you Need to Provide?

Insert Picture Here


UpTime versus Response Time versus Access

Up Time– How much down time can you afford?

Response Time– How long is too long to wait?

Access– How many do you need to serve simultaneously

Things to be engineered/negotiated upfront


Insert Picture Here

Simple Level 1Read/Write Splitting

Insert Picture Here


Use MySQL Replication to Split Read/Writes


Read/Write Splitting

● More complex environment

● Application Changes

● Easier with PHP & Java

● Low opportunity and sanity costs

● Less risk of one server stopping operations

Most Common Scaling Technique


Read/Write Splitting—Easy Way

● mysqlserverclone

● mysqlfailover

● When master dies, most up to date slave (GTID) promoted automatically

● Mysqlrplcheck & mysqlrplshow

Use MySQL Utilities to copy master, set up failover


Insert Picture Here

Simple Level 2Moderately Easy, Better Availability

Insert Picture Here


Distributed Replicated Block Device (DRBD)

● More complex environment

● Proven, supported

● Please practice fail over and back

● Easy to manage butnot HA panacea

High Availability


CLOUD

● Slightly more complex environment

● Proven, supported

● Best for ephemeral data, may not make upper management comfortable (or you)

● Easy to manage butnot HA panacea

High Availability, Low Cost (maybe)


Insert Picture Here

Complex Level N^NNever, Ever Down

Insert Picture Here


MySQL ClusterFor 99.999% uptime, upper bound on response time

● Extremely Complex

● No single point of failure = lots of $$hardware

● Not general purpose MySQL

● Auto sharding

● Add/Subtract nodes at will

● Replicate between geographically separated locations

● Need a smart staff


MySQL Cluster Shards Data Automatically

● Single or multiple data node failures do not bring down a cluster

● Easy to add/remove data nodes


MySQL Cluster Topology

● Nodes

● MGMT – startup/add node

● Data

● Mysqld

● Can be engineered for nosingle point of failure

● Can replicate across data centers

● Expensive on all fronts


Insert Picture Here

Big Data

Insert Picture Here


Ask ten DBAs to define 'Big Data' and You Will Get At Least Eleven Answers

.BIG as massive amounts● Define MASSIVE

BIG as in fast● Define fast

BIG as all of the other three!!!BIG as lots of eyes

● How many simultaneous users


The Easiest Way to Get Speed

● NoSQL/Memcached to InnoDB/NBD

● 9x faster by skipping parser, optimizer

● Key/Value pair access and SQL access of same data


Easiest Way to Gain Size

● Columnar Storage Engines for MySQL

● Not InnoDB

● Unique features, quirks

● Need > 26,000,000,000 rows to start seeing 'sweet spot'

● Use new server, don't scrimp


Calpont

InfiniDBColumn-oriented

● MySQL front end● Multi threaded● High speed data loading● Automatic partitioning● Enterprise Features

● Partition Drop● UDF’s● Physical compression● MPP, multi-node scale-out● Distributed, Shared Nothing data cache● Auto Increment● Centralized database and operating system

monitoring● Notifications and Alerts● INSERT INTO...SELECT FROM...● Views on InfiniDB Tables


Infobright

● Similar Features to Calpont butdifferent implementation


Hadoop-- getting data from MySQL to Hadoop

● 80% of Hadoop Installations feed from MySQl

● Hadoop is made up of several components

● Twisty little maze, all alike

● Use Apache Bigtop

– Tested for interoperability● Cloudera & Horton Works have great tutorials

● Kinda 'SQL-ish'

● Squoop and MySQL Hadoop Applier to move data


Hadoop Applier

● Infobright


Partitioning and Sharding /Dividing and Conquering

● Partitions built-in to MySQL

● Nice management tool

– Rolling quarters/months

– Fast drops

● No change to application but higher complexity, low opportunity cost

● Sharding

● Much more complex application and operations

● Not off shelf

● Hard to re-engineer

● Works


Insert Picture Here

Hard Part – Thinking

Insert Picture Here


Goals – What are you trying to accomplish

● Define problem completely

● Who produces, who consumes data

● Negotiate performance rates

● Plan for failure, catastrophe

● Design for long life

● What does end solution look/feel like?

● Platforms

● Speed

● Access areas

● Security

● Serenity


Where Else to Seek Info

● MySQL Connect Conference

Sept. in San Francisco

Same time as OOW

Weekdays!

● Planet.mysql.com and forums.mysql.com

● Hadoop

● Horton Works and Cloudera both have wonderful tutorials

● Apache Bigtop project


MySQL Training

● MySQL Marinate

● Online Virtual Class run by Boston MySQL Users Group

● 1 chapter/week

● Free

● Oracle and other vendors

● Dozens of book

● Online


Insert Picture [email protected]@Stokerslideshare.net/davestokes

Insert Picture Here

mailto:[email protected]


Graphic Section Divider


Technology

MySQL Scaling for SCaLE 12x conference