Lightning talks percona live mysql_2012

Lightning talksGiuseppe MaxiaContinuent, Inc

April 11, 2012

Wednesday, April 11, 12

LineupStewart Smith The 5 minutes tour of Drizzle

Jeff Mace What can possibly go wrong?

Nicolai Plum MAKE.DATABASES.FAST

Bill Karwin Schemadoc

Robert Hodges It's the application, stupid!

Robert Buck NuoDB overview

David Holoboff Diagnosis and possible fixes

Lawrence Schwartz OLTP and OLAP

Baron Schwartz Building a scalable system

Sheeri Cabral How to become a rock star DBA

Giuseppe Maxia Cruelty on database clusters

The 5 minutes tour of DrizzleStewart Smith

2008...

http://www.flickr.com/photos/shaylor/53819053/

2011...

Drizzle7

2012...

Drizzle7.1

What is Drizzle?

Modular architecture

Pluggable

Infrastructure Aware

Community Developed

Multicore

Concurrency

Web focus

EBCDIC!=

Enable others

Modern codebase

Nolinked-list innovation

Collaboration

Participation

Build businesses

Support & Consulting

Testing

Transactional

Logging

Replication

Multi-sourceReplication

In database tables

No bad data

strict_mode

No GOTCHAs

No implicit COMMIT

BSD Client Libraries

New in 7.1

Multi-source Replication

XtraBackup

IPv6 data type

ZeroMQ

Syslog is default

query_log plugin

Auth from tables

Server-side Javascript

HTTP JSON API

Percona InnoDB Patches

What could possibly go wrong?

Jeff Mace

©Continuent 2012.

What Could Possibly Go Wrong?

Je! Mace

©Continuent 2012 59

http://mingle2.com/blog/view/web-developer-mind

http://www.flickr.com/photos/brhefele/4026340146/

http://www.flickr.com/photos/themoly/4336260658/

©Continuent 2012

What can I possibly do?

• Relax

• Have a plan

• Keep it simple

http://www.flickr.com/photos/devenlaney/5767271943

MAKE.DATABASES.FASTNicolai Plum

Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices

MAKE.DATABASES.FAST

Nicolai Plum

The business

• Booking.com–Leading online hotel reservation agency–Part of the Priceline Group of Companies–Sustained rapid business growth–All powered by MySQL

The challenge

The problem…

• Hundreds of database servers• Many replication slaves• Few DBAs, some more sysadmins• Need to make tens of servers each day

If you do this manuallyYou are doing it wrong

Among our weapons are…

• Network OS installation• Puppet configuration management• Database cloning• and a fanatical devotion to automation!

Server installation

• Server Database• PXE netboot• CentOS kickstart

System configuration

• Now we have a server that can run Puppet• Puppet does

–OS setup–Software installation–LVM setup and filesystem creation (tailored by role)

• Prefer ongoing configuration management tools to single perfect installation tools

Database cloning

• Now we have a server ready to run a database• Copying databases based on LVM snapshots• create_clone program

–Copy via SSH and rsync–LVM snapshot–Fix my.cnf (server id!) automatically

Things we learned

• Waiting for actions to complete takes a lot of person time – automate batches of work as much as possible

• You really need a Server Database, and drive everything from it

• Most hardware can’t support being a clone source and in service, you need spares

• Online snapshots break in subtle ways; best stop the database.

Things we learned

DBAs are toolsmiths, not database installers

Thank you

• … if you like the way we work, we are hiring• http://www.booking.com/jobs

SchemadocBill Karwin

SchemadocBill Karwin, Percona Inc.

www.percona.com

Schema Can Be Unclear

CREATE TABLE AccountActivity ( Id INT PRIMARY KEY, Name VARCHAR(100), Status VARCHAR(20), EntityNum INT, Order INT);

www.percona.com

whose name?

www.percona.com

whose name?

what are the values?

www.percona.com

whose name?

what’s an entity?what’s a number?

www.percona.com

whose name?

what’s an entity?what’s a number?

commerce, or ordinal?

www.percona.com

Comments!

CREATE TABLE AccountActivity ( Id INT PRIMARY KEY, Name VARCHAR(100) COMMENT ‘person who placed the order’, Status VARCHAR(20) COMMENT ‘new, open, or closed’, EntityNum INT COMMENT ‘how many items in the order’, Order INT COMMENT ‘reference to the Orders table’) COMMENT ‘any change to e-commerce orders’;

www.percona.com

Length

• Limits increased in MySQL 5.5.3:• Per column: 1024 characters• Per index: 1024 characters• Per table: 2048 characters• Per partition: 80 characters

(MySQL 5.6 increases this to 1024)

www.percona.com

“Is there a tool for MySQL like javadoc for Java code?”

www.percona.com

developmentdba

documentation

www.percona.com

schemadoc

www.percona.com

$ schemadoc -v h=dbserver,D=EcommerceDB,u=root,p=xxxxLoading metadata...Running online analysis...done.Outputting to html.Rendering pages for tables. done.Rendering pages for views done.Rendering pages for procedures done.Rendering pages for triggers done.Rendering page for schema summary... done.Rendering page for schema link... done.Rendering page for deprecated-list... done.Rendering page for all-index... done.Rendering page for help-doc... done.Rendering page for allschema-frame... done.Rendering page for index... done.

www.percona.com

hooray.

www.percona.com

But Wait, There’s More

•Parses SQL dump file•Reads from a live MySQL instance•Document many schemas•Update one schema at a time

www.percona.com

Schema Analysis

•Number of indexes and columns per data type•Columns with same name but different type•ENUM columns that aren’t NOT NULL•Use of FLOAT or DOUBLE• Tables with one index per column• Tables with one index over all columns• Tables with no indexes• INT(N) other than default N• Lack of INT UNSIGNED columns

www.percona.com

Schema Analysis

• Tables that look like Entity-Attribute-Value• Tables that look like Polymorphic Associations•Superfluous primary key•Report table with least / most indexes• IP addresses stored in VARCHAR•Excessive use of VARCHAR(255),

report actual max string length• ...And supports plugins for other checks!

www.percona.com

yes, we get it.

www.percona.com

Future of Schemadoc

• Integrate with Percona Toolkit library•Read extra metadata (foreign keys)•Show table sizes and growth rates•Show SQL privileges (who has access?)•CLI for adding / updating comments•Batch-mode output•Redesign HTML output with modern look & feel

www.percona.com

Future of Schemadoc

www.percona.com

launchpad.net/schemadoc

It's the application, stupid!Robert Hodges

©Continuent 2012.

It’s the Application, Stupid!

(Or: Multi-Master Enlightenment in 5 Minutes or Less)

Robert Hodges, Continuent CEO

Multi-Master Replication Is This Easy

tungsten-sandbox-2.0.4 -m 5.1.54 \ -i $RELDIR/tungsten-replicator-2.0.5 \ -n 3 --topology=star --hub=2

HubSpoke Spoke

Multi-Master Apps Not So Much

Airline Reservations

HubSpoke Spoke

Who Is in Which Seat on Flight 3523?

Book Bobin Seat 11E

Airline Reservations

HubSpoke Spoke

Book Suein Seat 11E

Bump Bob from Flight 3523

Rules for Multi-Master Apps

1. Isolate destructive changes to a single master to avoid harmful con!icts

➡ “Change gate for !ight 3523 to C28”

2. Use non-destructive INSERTs for everything else

➡ “Please assign Sue to seat 11A”

Rules for Multi-Master Apps

3. Coordinate across masters to aggregate results

➡ “Generate "nal passenger manifest for !ight 3523”

4. Be prepared to apologize

➡ “Add 5000 frequent !yer miles to Bob’s account”

Multi-Master Enlightenment

Scalable multi-master applications are eventually consistent

1. Think through the whole problem: Business processes are often eventually consistent already

2. Follow design rules for eventually consistent applications

3. Use appropriate replication technology for MySQL*

* As in Tungsten :)

NuoDB overviewRobert Buck

The Elastically Scalable Database™

‣ SQL/ACID: Emergent Architecture

‣ Elastically Scale: Demand-driven, auto-balancing

‣ Single Logical Database: No sharding, no partitioning

‣ Run Anywhere: On-Premises / Private, Public, or Hybrid Cloud

‣ Store Anywhere: KV-Store backend (flexible, e.g. Files, S3, HDFS)

‣ Multi-Tenant: DBs share machines dynamically

‣ Geo-Distributed: Active/active across several data centers

‣ Extreme Availability: Massively redundant, permitting live upgrade

‣ Minimal Admin: No knobs, auto everything

NuoDB Summary

Emergent Database Architecture

“An emergent behavior can

appear when a number of simple agents operate in an environment,

forming more complex behaviors

as a collective.”

- Wikipedia

Instantly Elastic

Second Machine Instant Performance Increase

‣ Second machine more than doubles TPS due to caching effects

‣ Second machine is added to live database while it is running at 1,000’s of TPS

‣ Performance increase is immediate

‣ BTW - you can take either machine away and the database keeps running without data loss

Number of Nodes

Linearly Scalable

Technical Details:‣ 2-9 Tx engines‣ 1 storage manager‣ Best sustained TPS and #

clients combination‣ 50% updates

Nodes TPS

MySQL 1 3,000

NuoDB 1 4,500

NuoDB 9 27,000

NuoDB running on 9 nodes was approx. 9x faster than MySQL running on 1 node.

10000"

15000"

20000"

25000"

30000"

1" 2" 3" 4" 5" 6" 7" 8" 9"

... even on Amazon EC2

10000"

12000"

1" 2" 3" 4" 5" 6" 7" 8" 9"

Number of EC2 Nodes

‣ NuoDB scales linearly on EC2

‣ EC2 performance bound by AMZ TCP/IP traffic shaping

‣ Linearly scales at 50% of rate for physical commodity hardware

‣ Whereas, Amazon RDS runs on 1 node, and gets overloaded at 10+ connections

We Work with Your SQL Tools

SQuirreLSQL

MS Excel (and other MS tools)

DbVisualizer

NuoDB System Architecture

Application(JDBC, ODBC, SQL) NuoDB Storage

ManagerNuoDB Transaction

Engines

NuoDB Brokers

Tx Scalability Add as many of B, TE, or SM as you like

I/O Scalability Distributed IO, e.g. Hadoop HDFS

100% Uptime No single point of failure, geo-distributed

Optimal Utilization Multiple Databases on finite resources

Low People Costs Single Console Management

TM1TM1TM1TM1TM1TM1TM1TM1TM1TM1TM1TM1TM1TM1TE

‣ 100% SQL

‣ 100% ACID

‣ 100% Elastic Scaling

‣ If you have more questions...

‣ Prove it to yourself. Try it out!

‣ Download it at: www.nuodb.com

‣ Chat with us afterward

NuoDB Summary

NuoDBThe Elastically Scalable Database™

Diagnosis and possible fixes

David Holoboff

DIAGNOSIS ANDPOSSIBLE FIXES:

A MULTIPLE CHOICE PRIMERFOR BEGINNING DBAS

This talk is for entertainment value only. Any perceived similarity between what you will hear in the next five minutes and true life scenarios are merely coincidental, with

no intent of harm or liability.

Are you being asked the right question?

What do you do with a slow database server?

What do you do if swap memory is always being used up?

OLTP and OLAPLawrence Schwartz

OLTP and OLAPHave Your Cake and Eat it Too!

Lawrence Schwartz

Avoiding the B-tree Valley of Death

Read-intensive Write-intensive

OLAP OLTP

B-treeValley of

Columnar DBs HadoopMongoDB

However, Today’s database categories - OLAP, OLTP are compromises to get around dated, rigid indexing technology

“The real end-game for Big Data is to have transactional and analytic data on the same database.”

B-trees Perform Poorly on Disk

• Random Workloads Lead To Frequent Drive Head Movement

• MySQL Can’t Take Advantage of Full Drive I/O Potential

MySQL + InnoDB

• Continuous Rebalancing and Aggregation at Nodes

• Eliminates Fragmentation, Maximizes Disk I/O

MySQL + TokuDBBased on B-Tree Indexes Based on Fractal Tree Indexes

B-trees Can’t Handle Big Data

• Performance Drops to a Trickle above 50-100M Rows

• Delayed Checkpoints Diminish Performance

• 20x Insertion Rates at One Billion Rows

• Stand Up More Indexes for Better Query Performance

MySQL + TokuDBMySQL + InnoDB

B-trees Aren’t Agile

• Big Database Schema Changes Take Hours To Complete

• Changes Have to Wait for Scheduled Downtime

• Changes Complete in Seconds

• Allows Business' Data Models to Immediately Adapt

B-trees Don’t Compress Well

• Compression ~2x to ~2.5x Typical

• Compression Impacts Performance

• High Compression

• No Impact to Performance

TokuDB Eliminates Death Valley

Read-intensive Write-intensive

ge“While TokuDB® is effectively an operational database technology, it does blur the lines between operations and analytics”

OLAP OLTP

Fractal Tree® Indexesare the Solution

eCommercePersonalization

Cloud Assets

Logfile analysis

SocialNetworks

Real Time OLAP

Analytic OLTP

Building a scalable systemBaron Schwartz

How to become a rock star DBA

Sheeri Cabral

How to Become a Rock Star DBA

Sheeri Cabral, MozillaOracle ACE Director

MySQL Community Award Winner^2

Do what you love....

What you really love

Have a Twin Brother

forums.mysql.com

Tutorials / HOW-TOs

MariaDB/Percona Documentation

Make MisteaksMistakes

Teach others

You know more than you think you do!

Keep Learning

Organize

User Group

UnconferenceIgnite Event

Magazine

Connect People

Avoid burnout

Have a hobby

Have Support

Be tall...or not

Find Ways Others Can Help You

Some Help You Can Get

MySQL Community TeamVolunteers – UG video: Richard Laskey“What can I help you do?”Technocation for grants - meetup.com fees - Open SQL/DB Camp - Postgres Open - Open Source Bridge - Drizzle Day

(donate to www.technocation.org!)

It Takes a Community

Permission to use pictures granted by:

Ronald BradfordFrancesca Simon-Carboni PhotographyColin CharlesJeremy ColeJames Duncan DavidsonKeith LarsonGerry Narvaja

Cruelty on database clustersGiuseppe Maxia

Cruel things that you can inflict on your replicated

cluster ... and get away with it!Giuseppe MaxiaContinuent, Inc

I must confess

I am a serial killer

I kill

•database servers

I kill

•database servers

•client connections

I kill

•database servers

•transactions

I kill

•database servers

•transactions

•"lesystems

I am armed and dangerous

= "class 2" weapon

And of course I carry ...

(GASP) Perl!

I have a criminal mindset

• Like a bank robber

• or an airplane hijacker

• I look around, and watch for weak spots to exploit

Whenever I get a new database cluster

• I open "re

#!#!#!#!#!

when a cluster is working just "ne ...

• I pull the rug from under its feet

• I switch o# a master's database server

•or I delete its "les191

I am a Harry Potter fan

So I get carried away, and I use the unforgivable curses

$ kill -9

We have di#erent standpoints

•My colleagues say that I am cruel

• I call it QA

Have you ever reported a bug ...

• about a failure that happens after 9000 switches?

• Or a master that slows down when handling 512 slaves?

Have you ever reported a bug ...

• about a failure that happens after 9000 switches?

• Or a master that slows down when handling 512 slaves?

I DIDWednesday, April 11, 12

Things I do that make developers say "it is not fair"

• using the installer with wrong options

• running the installer without a mandatory option

• Using the wrong version of Java, or Ruby, or MySQL

• running the installer in a system with only 200 MB of free storage

• Counting the records after a performance test (and "nding that some are missing)

Why do I do cruel things to replication clusters?

• Because, most of the time:

• Developers are not users

• Users do what developers can't imagine

• Because, most of the time:

• Developers are not users

• Users do what developers can't imagine

•To make a fool-proof system you need to act like a fool

in the end, I get away with cruelty

I am not bad. I am just drawn that way

(Jessica Rabbit)

Lightning talks percona live mysql_2012

Technology

What's Lightning Talks (松山版)

BKK19-117 SWG Lightning talks

One Man Lightning Talks

Ignite (Lightning) Talks

Lightning Talks - Stanford University · Lightning Talks 1. Christina Delimitrou: “Tarcil: Reconciling Scheduling and Quality in Large Shared Clusters” 2. Nic McDonald: “Service-Oriented

30c3 lightning talks - phdays labyrinth

Lightning Talks: An Innovation Showcase

Lightning Talks: All EartCube Funded Projects

One Man Band - Drupal Lightning Talks

Atlanta Drupal Users Group - Lightning Talks - Sass

LMG Lightning Talks - Amazon Web Servicesconnect.linaro.org.s3.amazonaws.com/sfo17/Presentations...LMG Lightning Talks LMG linaro-android kernel topic branch updates Amit Pundir ENGINEERS

Lightning Talks - Platform Lab Talks/lightning-talks.pdfPISCES: A Programmable, Protocol-Independent Software Switch Muhammad Shahbaz, Sean Choi, Ben Pfaff, Changhoon Kim, Nick Feamster,

LCU14 102- PMWG Lightning Talks v2

UXSG2014 Lightning Talks - Design for impact (Annet Bruil)

Collaborations Workshop Lightning Talks

ICTI Lightning Talks - Aggregating Demand

Lightning Talks at IOGDC 2012

Grunt plugin talk - SeattleJS Lightning Talks

Shibuya.js Lightning Talks

ESWC 2009 Lightning Talks