CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity

  • View
    249

  • Download
    1

Embed Size (px)

DESCRIPTION

Presented at the CakePHP CakeFest 2014 Madrid This talk covers an overview of how to scale from 1 MySQL instance to cover needs of availability, access, and costs

Text of CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity

  • 1. Copyright 2013, Oracle and/or its affiliates. All rights reserved.1 Insert Picture Here MySQL Scaling Dave Stokes MySQL Community Manager David.Stokes@Oracle.com @stoker slideshare.net/davestokes 23rd August 2014

2. Copyright 2014, Oracle and/or its affiliates. All rights reserved.2 Safe Harbor Agreement .The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decision. The development, release, and timing of any features or functionality described for Oracles products remains at the sole discretion of Oracle. 3. Copyright 2013, Oracle and/or its affiliates. All rights reserved.3 Insert Picture Here MySQL Releases since 2010 Enterprise Monitor 2.2, 2.3 Cluster 7.1, 7.2, 7.3 Cluster Manager 1, 1.1, 1.3 Workbench 5.2, 6, 6.1 Database 5.5, 5.6, DMR 5.7 Backup 3.5, 3.7, 3.1 Oracle VM for MySQL Oracle Product Certifications Windows Installer Utilities 1.0.6 Audit Fabric Under promise/Over Deliver More Developers More QA More Support Making $$ Classes popular Instructor lead Self-paced We're hiring! 4. Copyright 2013, Oracle and/or its affiliates. All rights reserved.4 You want to get a bigger, badder MySQL -- BUT what are the costs??? Budget Complexity Application Opportunity Sanity. 5. Copyright 2014, Oracle and/or its affiliates. All rights reserved.5 Simple Stuff 6. Copyright 2014, Oracle and/or its affiliates. All rights reserved.6 Option 0 Run the Latest, Greatest MySQL MySQL 5.5 was 20% faster than 5.1! 5.6 is another 10-15% --Make sure your config file matches your hardware --ANALYSE TABLE --Check for SLOW QUERIES, QUERIES w/o INDEXES --TUNE QUERIES This is the least complex, less upsetting, and usually lowest budget cost option with NO changes to your application! 7. Copyright 2014, Oracle and/or its affiliates. All rights reserved.7 Option 0.1 Add in a Memcached Layer Cache for quick access Session Data Sports Scores Opens up NoSQL access to InnoDB later This is the second least complex, less upsetting, and usually lowest budget cost option but does make you change your application to become 'cache aware'. 8. Copyright 2014, Oracle and/or its affiliates. All rights reserved.8 Up Time versus Response Time versus Access Things to be engineered/negotiated upfront Up Time How much down time can you afford? Response Time How long is too long to wait? Access How many do you need to serve simultaneously. 9. Copyright 2013, Oracle and/or its affiliates. All rights reserved.9 Insert Picture Here Simple Level 1 - Splitting Reads 10. Copyright 2014, Oracle and/or its affiliates. All rights reserved.10 Use MySQL Replication to Split Read/Writes 11. Copyright 2014, Oracle and/or its affiliates. All rights reserved.11 READ Splitting Most Common Scaling Technique More complex environment Application Changes Easier with PHP & Java Low opportunity and sanity costs Less risk of one server stopping operations 12. Copyright 2014, Oracle and/or its affiliates. All rights reserved.12 Read SplittingEasy Way Use MySQL Utilities to copy master, set up failover mysqlserverclone mysqlfailover When master dies, most up to date slave (GTID) promoted automatically Mysqlrplcheck & mysqlrplshow 13. Copyright 2013, Oracle and/or its affiliates. All rights reserved.13 Insert Picture Here Simple Level 2 Moderately Easy, Better Availability 14. Copyright 2014, Oracle and/or its affiliates. All rights reserved.14 Distributed Replicated Block Device (DRBD) High Availability More complex environment Proven, supported Please practice fail over and back Easy to manage but not HA panacea 15. Copyright 2014, Oracle and/or its affiliates. All rights reserved.15 Cloud High Availability, low cost (sometimes) Slightly more complex environment Proven, supported Best for ephemeral data, may not make upper management comfortable (or you) Easy to manage but not a HA panacea 16. Copyright 2013, Oracle and/or its affiliates. All rights reserved.16 Insert Picture Here Complex Level N^N Never, Ever Down 17. Copyright 2014, Oracle and/or its affiliates. All rights reserved.17 MySQL Cluster For 99.999% up time, upper bound on response time Extremely Complex No single point of failure = lots of $$ hardware Not general purpose MySQL Auto sharding Add/Subtract nodes at will Replicate between geographically separated locations Need a smart staff 18. Copyright 2014, Oracle and/or its affiliates. All rights reserved.18 MySQL Cluster Shards Data Automatically Single or multiple data node failures do not bring down a cluster Easy to add/remove data nodes 19. Copyright 2014, Oracle and/or its affiliates. All rights reserved.19 MySQL Cluster Topology Nodes MGMT startup/add node Data Mysqld Can be engineered for no single point of failure Can replicate across data centers 20. Copyright 2013, Oracle and/or its affiliates. All rights reserved.20 Insert Picture Here Big Data 21. Copyright 2014, Oracle and/or its affiliates. All rights reserved.21 Ask ten DBAs to define 'Big Data' and You Will Get At Least Eleven Answers BIG as massive amounts Define MASSIVE BIG as in fast Define fast BIG as lots of eyes BIG as all of the the above!!! How many simultaneous users 22. Copyright 2014, Oracle and/or its affiliates. All rights reserved.22 The Easiest Way to Get Speed NoSQL/Memcached to InnoDB/NBD 9x faster by skipping parser, optimizer Key/Value pair access and SQL access of same data 23. Copyright 2014, Oracle and/or its affiliates. All rights reserved.23 Easiest Way to Gain Size Columnar Storage Engines for MySQL Not InnoDB Unique features, quirks Need > 26,000,000,000 rows to start seeing 'sweet spot' Use new server, don't scrimp See Calpont and Inforbright 24. Copyright 2014, Oracle and/or its affiliates. All rights reserved.24 Hadoop-- getting data from MySQL to Hadoop 80% of Hadoop Installations feed from MySQL Hadoop is made up of several components Twisty little maze, all alike Use Apache Bigtop Tested for interoperability Cloudera & Horton Works have great tutorials Kinda 'SQL-ish' Squoop and MySQL Hadoop Applier to move data 25. Copyright 2014, Oracle and/or its affiliates. All rights reserved.25 26. Copyright 2014, Oracle and/or its affiliates. All rights reserved.26 Partitioning and Sharding / Dividing and Conquering Partitions built-in to MySQL Nice management tool Rolling quarters/months Fast drops No change to application but higher complexity, low opportunity cost Sharding Much more complex application and operations Not off shelf Hard to re-engineer Works 27. Copyright 2014, Oracle and/or its affiliates. All rights reserved.27 MySQL Fabric Lets you shard and reshard Very new Looking for feedback Released in May 28. Copyright 2013, Oracle and/or its affiliates. All rights reserved.28 Insert Picture Here Wrap up 29. Copyright 2014, Oracle and/or its affiliates. All rights reserved.29 Goals What are you trying to accomplish Define problem completely Who produces, who consumes data Negotiate performance rates Plan for failure, catastrophe Design for long life What does end solution look/feel like? Platforms Speed Access areas Security Serenity 30. Copyright 2014, Oracle and/or its affiliates. All rights reserved.30 MySQL Marinate Free virtual class to learn MySQL run by the Boston MySQL Users Group 1 chapter a week Homework graded helpfully 31. Copyright 2014, Oracle and/or its affiliates. All rights reserved.31 Q&A David.Stokes@Oracle.com @stoker slideshare.net/davestokes