Why Traditional Databases Fail so Miserably to Scale with E-Commerce Site Growth

Why Tradi*onal Databases Fail so Miserably to Scale With E-‐Commerce Growth

Dragon Slayer Consul*ng – Marc Staimer CDS

Dragon Slayer Consul9ng All Rights Reserved © 2014 2

[email protected]

Analyst/Consultant IT Industry Experience

Publish w/TT & GigaOM

Speak @ Trade Shows

Improve Vendors Marke9ng Help EUs w/Problems

Agenda

Storage Decisions Conference | © TechTarget

It’s What We Know Is True When It’s Not That Causes The Most Difficult Problems

Storage Decisions Conference | © TechTarget

If I Catch You, You’re Mine

SQL Database E-‐Commerce Performance Problem

!   E-‐Commerce mission cri9cal !   Slow performance and/or outages =

!   Lost sales !   Lost revenue !   Lost profits

8/5/14 Dragon Slayer Consul9ng All Rights Reserved © 2014 5

When The IOPS BoOleneck is Storage

!   > DRAM !   Fast with lots of random IOPS !   Expensive & vola9le

!   > HDDs !   Rela9vely inexpensive !   Not as many random IOPS so need lots of them !   > suppor9ng infrastructure !   > exper9se required (short-‐stroking)


The New Darling: Flash Storage

!   DDR DIMM

!   PCIe

!   SAS/SATA SSD

!   Hybrid array

!   All flash array (AFA)


Flash ShiRs SQL DBMS Performance BoOleneck

Fall 2014 Dragon Slayer Consul9ng All Rights Reserved © 2014 8

Making SQL DBMS Performance A “Scale” Problem

!   Complicated !   Requiring a lot of exper9se

!   Costs are excessive !   Many tasks are con9nuously manually labor-‐intensive !   Reduced Up9me

Common SQL Database Scaling Workarounds

!   Scale-‐Up !   Scale-‐Out

!   Sharding !   Read-‐only Slaves !   Peer-‐to-‐Peer Replica9on !   Linked Servers & Distributed Queries !   Distributed Par99on Views !   Data Par99on Rou9ng !   NoSQL Databases


Scale-‐Up

!   More cores !   More DRAM !   More system IO bandwidth !   More power


Scale-‐Up Pros & Cons

Pros !   Simpler – easier

!   No sodware changes !   Nothing addi9onal to manage !   Or train/learn

Cons !   Cost $$$$

!   > Server HW CapEx !   > Server HW OpEx !   (Servers oden proprietary) !   > DBMS licensing costs !   > HA costs


Scale-‐Out: Sharding

!   Breaks up the database !   Into separate instances !   Easy to understand !   Fairly common especially with

!   MySQL !   SQL Server !   PostgreSQL


Sharding Pros & Cons

Pros !   Data split along natural fault lines !   Highly scalable !   Many DBAs know how

!   Non-‐DBAs do not

Cons !   Mul9ple DBMS instances > license $ !   Requires lots of exper9se

!  App & SQL DBMS data structures !   Time consuming: labor intensive

!  Error prone !   Non-‐dynamic !   Rela9onship loss !   Too many SPOFs


Scale-‐Out: Read-‐Only Slaves

!   Master database that replicates !   To a series of slave databases

!   Slaves are read-‐only !   # of slaves varies !   Limited only by master’s replica9on ability

!  Speed !  Affect on DBMS performance


Slaved SQL DBMS

Read-‐Only Database Files

Read-‐Only Slaves Pros & Cons

Pros !   Nominal setup !   Simple !   App/DBMS transparent !   Incredibly Popular

Cons !   Master database bolleneck !   Not good for write intensive apps !   Master database SPOF

!   Patches/upgrades/fixes

!   Mul9ple DBMS instances = > $$$$


Scale-‐Out: Peer-‐to-‐Peer w/Replica*on

!   Mul9ple database copies !   Each database engine manages and maintains its own DBMS copy !   Replica9on updates other database engines when there’s a change


Peer-‐to-‐Peer w/Replica*on Pros & Cons

Pros !   Mul9ple database copies !   No SPOF !   Good for infrequent updates

Cons !   Complicated opera9ons !   Conflict resolu9on is difficult

!   Stewardship: labor-‐intensive !   Merge replica9on: high overhead



Scale-‐Out: Linked Servers & Distributed Queries

!   U9lized when: !   Databases are split into func9onal areas

!  Requiring minimal coupling

!   Or when spliong data by type


Client App

Server DBMS

Server 1 DBMS

Server 2

Linked Servers & Distributed Queries Pros & Cons

Pros !   Makes mul9-‐DBMS appear as 1 !   DBMS update frequency

!   Has no impact

!   DBMS split into func9onal areas !   Minimal coupling !   When spliong data by type

Cons !   Referen9al integrity constraints !   Queries spanning mul9ple DBMS

!   = > latencies < performance

!   Significant exper9se required !   Difficult to pull off successfully



Scale-‐Out: Distributed Par**on Views (DPV)

!   Table data par99oned among tables in numerous distributed databases !  Based on a par99oning key

!   Update-‐intensive apps best target for DPVs !  Generally good E-‐Commerce query performance !  Few rows impacted – likely processed in single DBMS


Distributed Par**on Views (DPV) Pros & Cons

Pros !   Good for update intensive Apps

!   App transparent !   Scale-‐out par99oned data

!   Table data par99oned into tables !   Mul9ple DBMS

Cons !   Data movement between par99ons

!   Slows performance !   Ltd by table’s par9onability !   Mul9ple SPOFs !   Significant exper9se required

!   Difficult to pull off successfully



Scale-‐Out: Data Dependent Rou*ng (DDR)

!   E-‐Commerce app or middleware responsible for DBMS query rou9ng !   Premise: they have deeper understanding of the data !   Popular when queries distributed across hundreds or more databases


Client Middleware or

data layer routed to SQL DBMS

w/data requested

Data Dependent Rou*ng (DDR) Pros & Cons

Pros !   Good when queries spread

!   Across 100s or > DBMS !   Apps know their own data

!   Beler query decisions

Cons !   Significant exper9se required

!   Deep understanding of apps !   Or middleware !   Strong architectural skills !   Massively complicated

!  Based on # of DBMS servers

!   ALL apps must be customized !   Mul9ple DBMS instances = > $$$$


Scale-‐Out: NoSQL Databases

!   Excellent for large data sets and analy9cs in-‐general !   OLAP !   Data Warehousing !   Unstructured data analy9cs


NoSQL Databases Pros & Cons

Pros !   Data Ingest !   Unstructured data analy9cs !   OLAP !   Data Warehousing !   Historical data

Cons !   Poor transac9onal performance !   Lack real-‐9me dashboards

!   & Real-‐9me repor9ng

!   Not good @ scanning data !   As it’s changing “NOW”

!   Lacks common SQL DBMS tools !   Technology s9ll immature


Conclusion & Summary

!   Tradi9onal scaling methodologies have serious issues ranging from !   Too much cost $$$$ !   Too complicated

!  Up front & ongoing

!   Too labor intensive !   Too much down9me

!  Trouble shoo9ng excessively difficult

!   Too much frustra9on


The Test of Three


© 2014 CLUSTRIX

Software launched

San Francisco San Jose Seattle New York London

Company

Offices Worldwide Top-Tier Venture Funding

1 Trillion+ transactions per month

100s of installations worldwide

Clustrix founded

Mission-critical production

deployments

2014 2006 2010 2013

ClustrixDB Overview 30

Without the right database, e-commerce success

leads to website slowdown or failure

Slowdowns Outages

Unhappy Customers

Lost Revenue


More Uptime

More Capacity

More Business

The first e-commerce database purpose-built for today’s online retailers


ClustrixDB Overview

Today’s E-Commerce Database •  Readily handles e-commerce growth

•  Easy accommodates traffic spikes •  Meets no-downtime demands

•  Supports on-the-fly updates •  Enables live e-commerce reporting

•  Integrates with e-commerce stack

•  Replaces MySQL with minimal effort


Serve fast-growing mobile and social channels

Business Advantages

Ready for Revenue Growth

Peak Performance at Peak Times

Capture More Business

Scale effortlessly with sales success

Avoid lost revenue during traffic spikes

Readily manage unpredictable demand

Analyze and act on real-time sales data

Better orchestrate omni-channel shopping


E-commerce leaders across retail, travel, digital services, and social commerce rely on ClustrixDB

Customer Highlights


E-Retailer Takes 600% Sales Spike in Stride

“Three years ago, Black Friday was horrible. We wasted money having database issues. With Clustrix, those issues are gone.”

Challenge ClustrixDB

•  Second-fastest growth rate in Internet Retailer Top 500

•  Loads spike 20X or more during holidays

•  Hours-long downtime incidents, with $20K-$60K in lost revenue per hour

•  Flexed up and down to handle 600% Cyber Monday sales spike with no downtime

•  Real-time catalog updates during flash sales

•  Live reporting and analytics on e-commerce business

Keith Bussey VP Technology


Online Textbook Company Graduates from MySQL

“We wouldn’t be able to grow our business as easily without ClustrixDB.”


•  3X revenue growth strained chaotic MySQL backend

•  Significant administrative overhead required for redundancy and uptime

•  Large schema changes needed system-wide downtime

•  100% increase in database load with no performance hits

•  Simplified operations with easy expansion and seamless datacenter failover

•  Went from 10 hours of downtime to no downtime with on-the-fly changes

Paolo Resmini VP of Engineering


Picture-Perfect Performance for Digital Photo Service

“Clustrix has given us one advantage over our competitors… the ability to stay open at peak times with great service levels.”


•  Highly seasonal business with 10X spike in December

•  Need to ensure uptime during peak loads for 30 million members

•  Reached capacity limits of conventional database

•  Gained performance and scale with no app rewrite

•  89% reduction in December service incidents

•  More time on innovation, less time on database worries

Graham Hobson CTO


Online Travel Site Increases Uptime, Lowers TCO

“MakeMyTrip frequently experiences 50% spikes in traffic that no longer cause slow page load times and costly downtime.”


•  Two server MySQL config with single point of failure

•  Slow page loads and costly downtime

•  Sharding / MySQL Cluster too expensive to implement and maintain

•  Drop-in replacement with no single point failure

•  $100K cost reduction with easy expansion

•  Handles massive spikes in traffic with faster web response times

Sanjay Kharb Director Technical Ops


Automatic, 100% fault tolerance

Flex up and down, in minutes

Massive, linear scalability

Technical Advantages

Capacity Availability Productivity

Extreme concurrency

No single point of failure

Battle-tested performance

Eliminates re-architecting the database

Plug-in MySQL compatibility

Self-managing operation


Clustrix Design!

Intelligent Data Distribution

Massively Parallel Query Processing

Shared Nothing Architecture

SQL

SQL

SQL

SQL

SQL


Query Compiler Data Map

Database Engine


Database Engine


Database Engine

Signs It’s Time for ClustrixDB

•  Capacity planning for a busy shopping season •  Gearing up for mobile and social demand •  Issues with website slowdowns or outages •  Scaling is too costly, complex and time consuming •  Database is a single point of failure •  Business wants new e-commerce capabilities


Thank You.

facebook.com/clustrix

ecommerce.clustrix.com

@clustrix

linkedin.com/clustrix
