Industry experts webinar slides (final v1.0)

Live Webinar - Sep 24, 2014

Industry Experts Examine the State of Databases

Speakers

Matt AslettResearch Director

The 451 Group

Seth ProctorCTO

NuoDB, Inc.

2

© 2014 by The 451 Group. All rights reserved

Company Overview

One company with 3 operating divisions

Syndicated research, advisory, professional services, datacenter certification, and events

Global focus

270+ staff 1,500+ client organizations:

enterprises, vendors, service providers, and investment firms

Organic and growth through acquisition

114

Relational zone

Non-relational zone

Lotus Notes

Objectivity

MarkLogic

InterSystemsCaché

McObject

Starcounter

ArangoDB

FoundationDB

Neo4J

InfiniteGraph

CouchDB

MongoDB

Oracle NoSQL

Redis

Handlersocket

RavenDB

AWS DynamoDBCloudant

Redis-to-go

RethinkDB

App EngineDatastore

SimpleDB

LevelDB

Accumulo

Iris CouchMongoLab

Compose

Cassandra

HBase

RiakCouchbase

Key: General purposeSpecialist analytic

BigTablesGraphDocumentKey value stores

-as-a-Service

Splice Machine

Actian IngresSAP Sybase ASE

EnterpriseDB

SQL Server

MySQL

InformixMariaDB

SAP HANA

IBMDB2

Database.com

ClearDB

Google Cloud SQL

RackspaceCloud Databases

AWS RDS

SQL Azure

FathomDB

HP Cloud RDB for MySQL

StormDB

Hadapt Teradata Aster

HPCC

ClouderaHortonworksMapR IBM

BigInsights

AWSEMR

Google Compute

Engine

Zettaset

NGDATA

451 Research: Data Platforms Landscape Map – September 2014

InfochimpsMetascale

MortarData

Rackspace

Qubole

Voldemort

Aerospike

Key value direct accessHadoop

Teradata

IBM PureDatafor Analytics

Pivotal GreenplumHP Vertica

InfiniDB

SAP Sybase IQ

IBM InfoSphere

Actian Vector

XtremeData

Kx Systems

Exasol

Actian Matrix

ParStream

Tokutek

ScaleDBMySQL ecosystemAdvanced clustering/sharding

VoltDB

ScaleArc

ContinuentTransLattice

NuoDB

Drizzle

JustOneDB

Pivotal SQLFire

Galera

CodeFutures

ScaleBase

Zimory Scale

Clustrix

TesoraMemSQL

GenieDB

Datomic New SQL databasesYarcData

FlockDB

AllegrographHypergraphDB

AffinityDB

Giraph

Trinity MemCachier

Redis LabsRedis Cloud

Redis LabsMemcached Cloud

FairCom

BitYota

IronCache

Grid/cache zone

Memcached

Ehcache

ScaleOutSoftware

IBM eXtreme

ScaleOracle

Coherence

GigaSpaces XAPGridGain

PivotalGemFire

CloudTran

InfiniSpan

Hazelcast

OracleExalytics

OracleDatabase

MySQL Cluster

Data cachingData gridSearch

Oracle Endeca Server Attivio

Elasticsearch

LucidWorksBig Data

Lucene/Solr

IBM InfoSphere Data Explorer

TowardsE-discovery

Towardsenterprise search

Appliances

DocumentumxDB

TaminoXML Server

Ipedo XMLDatabase

ObjectStore

LucidDB

MonetDB

Metamarkets Druid

Databricks/Spark

AWSElastiCache

Firebird

SciDBSQLite

Oracle TimesTensolidDB

Adabas

IBM IMS

UniDataUniVerse

WakandaDB

Altiscale Oracle Big Data Appliance

RainStor

OrientDB

Sparksee

ObjectRocket

Metamarkets

TreasureData

PostgreSQLPercona

vFabric Postgres

© 2014 by 451 Research LLC. All rights reserved

HyperDex

TIBCOActiveSpaces

TitanCloudBird

SAP Sybase SQL Anywhere

JethroData

CitusDB Pivotal HD

BigMemory

ActianVersant

DataStaxEnterprise

DeepDB

Infobright

FatDB

Google Cloud

Datastore

Heroku Postgres

GrapheneDBCassandra.io

Hypertable

BerkeleyDB

SqrrlEnterprise

MicrosoftHDInsight

HPAutonomy

OracleExadata

IBM PureData

RedisGreen

AWSElastiCachewith Redis

IBMBig SQL

Impala

ApacheDrill

Presto

MicrosoftSQL Server

PDW

ApacheTajo

ApacheHive

SPARQLBASE

MammothDB

Altibase HDBLogicBlox

SRCH2

TIBCOLogLogic

Splunk

TowardsSIEMLoggly Sumo

LogicLogentries

InfiniSQL

In-memory

JumboDB

ActianPSQLProgressOpenEdge

Kognitio

Altibase XDB

Savvis

SoftlayerVerizon

xPlenty

Stardog

MariaDBEnterprise

Apache StormApache S4

IBMInfoSphereStreams

TIBCOStreamBase

DataTorrent

AWSKinesis

Feedzai

GuavusLokad

SQLStream

Software AG

Stream processing

OpenStack Trove

1010data

Google BigQuery

AWSRedshift

TempoIQ

InfluxDB

MagnetoDB

WebScaleSQL

MySQL Fabric Spider

21 43 65

E

D

A

B

C

T-Systems

E

D

A

B

C

21 43 65

SQream

SpaceCurve

Postgres-XL

GoogleCloud

DataflowTrafodion

Hadapt

ObjectRocketRedis

DocumentDB

AzureSearch

Red Hat JBossData Grid

114

Relational zone

Non-relational zone

Lotus Notes

Objectivity

MarkLogic

InterSystemsCaché

McObject


MySQL

Hadapt

451 Research: Data Platforms Landscape Map – 5ish years ago

Grid/cache zone

ScaleOutSoftware

IBM eXtreme

ScaleTangosol

Coherence

GigaSpaces

GemStone

Data grid/cacheSearch

EndecaAttivio

LucidImagination

Vivisimo

TowardsE-discovery


DocumentumxDB

TaminoXML Server

Ipedo XMLDatabase

SQLite

Adabas

IBM IMS

UniDataUniVerse

PostgreSQL


TIBCOActiveSpaces

Versant

BerkeleyDB

Autonomy

LogLogicSplunk

TowardsSIEM

In-memory

ProgressApama

StreamBase

TIBCOSQLStream

Coral8

Stream processing

21 43 65

E

D

A

B

C

E

D

A

B

C

21 43 65

Terracotta Memcached

ProgressObjectStore

LuceneSolr

Aleri

BEA

IngresSybase ASE

EnterpriseDB

Firebird

Sybase SQL Anywhere

SQL Server

InformixIBMDB2

OracleDatabase

Oracle TimesTenIBM solidDB

Pervasive PSQLProgress OpenEdge

Kognitio

1010data

TeradataNetezza

GreenplumVertica

Calpont

Sybase IQ

IBM InfoSphere

VectorWiseInfobright

Kx Systems

ParAccel

MonetDB

Aster Data

114

Relational zone

Non-relational zone

Lotus Notes

Objectivity

MarkLogic

InterSystemsCaché

McObject

Starcounter

ArangoDB

FoundationDB

Neo4J

InfiniteGraph

CouchDB

MongoDB

Oracle NoSQL

Redis

Handlersocket

RavenDB

AWS DynamoDBCloudant

Redis-to-go

RethinkDB

App EngineDatastore

SimpleDB

LevelDB

Accumulo

Iris CouchMongoLab

Compose

Cassandra

HBase

RiakCouchbase


BigTablesGraphDocumentKey value stores

-as-a-Service

Splice Machine

Actian IngresSAP Sybase ASE

EnterpriseDB

SQL Server

MySQL

InformixMariaDB

SAP HANA

IBMDB2

Database.com

ClearDB

Google Cloud SQL

RackspaceCloud Databases

AWS RDS

SQL Azure

FathomDB

HP Cloud RDB for MySQL

StormDB

Hadapt Teradata Aster

HPCC

ClouderaHortonworksMapR IBM

BigInsights

AWSEMR

Google Compute

Engine

Zettaset

NGDATA

451 Research: Data Platforms Landscape Map – September 2014

InfochimpsMetascale

MortarData

Rackspace

Qubole

Voldemort

Aerospike

Key value direct accessHadoop

Teradata

IBM PureDatafor Analytics

Pivotal GreenplumHP Vertica

InfiniDB

SAP Sybase IQ

IBM InfoSphere

Actian Vector

XtremeData

Kx Systems

Exasol

Actian Matrix

ParStream

Tokutek

ScaleDBMySQL ecosystemAdvanced clustering/sharding

VoltDB

ScaleArc

ContinuentTransLattice

NuoDB

Drizzle

JustOneDB

Pivotal SQLFire

Galera

CodeFutures

ScaleBase

Zimory Scale

Clustrix

TesoraMemSQL

GenieDB

Datomic New SQL databasesYarcData

FlockDB

AllegrographHypergraphDB

AffinityDB

Giraph

Trinity MemCachier

Redis LabsRedis Cloud

Redis LabsMemcached Cloud

FairCom

BitYota

IronCache

Grid/cache zone

Memcached

Ehcache

ScaleOutSoftware

IBM eXtreme

ScaleOracle

Coherence

GigaSpaces XAPGridGain

PivotalGemFire

CloudTran

InfiniSpan

Hazelcast

OracleExalytics

OracleDatabase

MySQL Cluster

Data cachingData gridSearch

Oracle Endeca Server Attivio

Elasticsearch

LucidWorksBig Data

Lucene/Solr

IBM InfoSphere Data Explorer

TowardsE-discovery


Appliances

DocumentumxDB

TaminoXML Server

Ipedo XMLDatabase

ObjectStore

LucidDB

MonetDB

Metamarkets Druid

Databricks/Spark

AWSElastiCache

Firebird

SciDBSQLite

Oracle TimesTensolidDB

Adabas

IBM IMS

UniDataUniVerse

WakandaDB

Altiscale Oracle Big Data Appliance

RainStor

OrientDB

Sparksee

ObjectRocket

Metamarkets

TreasureData

PostgreSQLPercona

vFabric Postgres


HyperDex

TIBCOActiveSpaces

TitanCloudBird

SAP Sybase SQL Anywhere

JethroData

CitusDB Pivotal HD

BigMemory

ActianVersant

DataStaxEnterprise

DeepDB

Infobright

FatDB

Google Cloud

Datastore

Heroku Postgres

GrapheneDBCassandra.io

Hypertable

BerkeleyDB

SqrrlEnterprise

MicrosoftHDInsight

HPAutonomy

OracleExadata

IBM PureData

RedisGreen

AWSElastiCachewith Redis

IBMBig SQL

Impala

ApacheDrill

Presto

MicrosoftSQL Server

PDW

ApacheTajo

ApacheHive

SPARQLBASE

MammothDB

Altibase HDBLogicBlox

SRCH2

TIBCOLogLogic

Splunk

TowardsSIEMLoggly Sumo

LogicLogentries

InfiniSQL

In-memory

JumboDB

ActianPSQLProgressOpenEdge

Kognitio

Altibase XDB

Savvis

SoftlayerVerizon

xPlenty

Stardog

MariaDBEnterprise

Apache StormApache S4

IBMInfoSphereStreams

TIBCOStreamBase

DataTorrent

AWSKinesis

Feedzai

GuavusLokad

SQLStream

Software AG

Stream processing

OpenStack Trove

1010data

Google BigQuery

AWSRedshift

TempoIQ

InfluxDB

MagnetoDB

WebScaleSQL

MySQL Fabric Spider

21 43 65

E

D

A

B

C

T-Systems

E

D

A

B

C

21 43 65

SQream

SpaceCurve

Postgres-XL

GoogleCloud

DataflowTrafodion

Hadapt

ObjectRocketRedis

DocumentDB

AzureSearch

Red Hat JBossData Grid

NoSQL

Hadoop

NewSQL

DBaaS


Drivers for change

Developers

Agile

REST

JSON

Schemaless

Schema-on-read

Flexible

Architecture

Cloud

Scalable

Elastic

Virtual

Distributed

Flexible

Applications

Web

Social

Mobile

Always-on

Interactive

Local

Global


Drivers for change influence each other

Developers

Agile

REST

JSON

Schemaless

Schema-on-read

Flexible

Architecture

Cloud

Scalable

Elastic

Virtual

Distributed

Flexible

Applications

Web

Social

Mobile

Always-on

Interactive

Local

Global

New applications require distributed architecture

Distributed architecture encourages new development approaches

New development approaches demand new architecture

Distributed architecture enables new applications

New app requirements demand new development approaches

New dev approaches enable new lightweight apps


Drivers for change: applications

Social – increased interactivity generates data

Mobile – different form factors and access methods

Global – applications need to be immediately available everywhere

Local – need to deliver localized content

Applications

Web

Social

Mobile

Always-on

Interactive

Local

Global

Social, mobile, global, local all have implications for data connectivity


Drivers for change: development

Rapid development and continuous delivery is inconsistent with traditional database management processes

Need to unite application development and database management people/processes to achieve common goals

DevOps movement growing apace

Developers

Agile

REST

JSON

Schemaless

Schema-on-read

Flexible

Developers increasingly drive data management and database selection


Architecture

Cloud

Scalable

Elastic

Virtual

Distributed

Flexible

Drivers for change: architecture

Transitioning from a traditional database to a distributed database

Interactive applications means the pace of user growth and multiplicity of data types is too great for traditional relational databases to efficiently absorb.

Scalability Performance Relaxed consistency Agility Intricacy Necessity


Transitioning from on-premises computing to the cloud




Architecture

CloudElastic

Virtual

Distributed

Flexible

Scalable






Architecture

CloudElastic

Virtual

Distributed

Flexible

Scalable


Amazon’s top enterprise use cases are (in order of popularity starting with the most popular):• Development and test• New workloads• Supplement existing workloads with cloud• Migration of existing workloads to the cloud• Datacenter migration• All-in cloud

Top three adoption drivers for public cloud have no impact on the existing database landscape



Transitioning from a traditional database to a distributed databaseArchitecture

CloudElastic

Virtual

Distributed

Flexible

Scalable


Drivers for change: combined effect

Architecture

Cloud

Scalable

Elastic

Virtual

Distributed

Flexible

Applications

Web

Social

Mobile

Always-on

Interactive

Local

Global

Developers

Agile

REST

JSON

Schemaless

Schema-on-read

Flexible



DevelopersApplications

Architecture


DevelopersApplications

Architecture


NoSQL DBaaS HadoopNewSQL


New databases: similarities

NoSQL DBaaS HadoopNewSQL

Distributed architecture Agility Elasticity New application development projects


New databases: differences

NoSQL Non-relational data models.Trade-off consistency for availability

NewSQLAdds availability and flexibility tothe familiar relational data model

Hadoop Batch (and now interactive) analytic processing of unstructured data

DBaaSAny of the above, or traditional RDBMS, delivered as a service


Use cases

Approach Details Examples

NoSQLMongoDB, Couchbase,

Cassandra, Redis, Aerospike, Cloudant

Non-transactional operational applications, unstructured data,

lightweight query

NewSQLNuoDB, MemSQL,

Translattice, VoltDB, Splice Machine

Transactional operational apps, structured data, complex query,

operational intelligence

HadoopCloudera, MapR,

Hortonworks, Pivotal, IBM, Teradata

Non-transactional analytic applications, multi-structured data,

complex query

DBaaSObjectRocket, AWS

DynamoDB, AWS RDS, Altiscale, Qubole

Any of the above, or traditional RDBMS, delivered as a service


A quick word operational intelligence

It has become an accepted best practice that analytics should be performed on data stored in a separate database from that used to support operational, transactional systems• data management benefits • the need to avoid the performance limitations of traditional systems

STRUCTURED DATASTRUCTURED DATA

OPERATIONAL DATABASE

APPLICATIONS

DATA WAREHOUSE

AD HOC ANALYTICS

PRE-DEFINED REPORTING



The emergence of a new breed of relational database vendors taking advantage of hardware, memory and processor performance to support transactional and analytic workloads in the same instance • A rejection of the concept that it is necessary to wait for data to become

available in analytic databases


NEWSQLDATABASE

APPLICATIONS

DATA WAREHOUSE

AD HOC ANALYTICS


OPERATIONALINTELLIGENCE



This is not a matter of making the data warehouse redundant, but rather providing another source of business intelligence to complement that generated by the data warehouse• Providing users with a ‘live’ view of their operational data for rapid

decision making


NEWSQLDATABASE

APPLICATIONS

DATA WAREHOUSE

AD HOC ANALYTICS


OPERATIONALINTELLIGENCE

Webscale Distributed Database

Convergence

NoSQL systems adding structure & query expressivenessNon-ACID systems running limited transactionsSQL databases for JSON or RDFHDFS as the core for SQLEtc.

25

Why?

26

Simplicity …

27

… and because we can.

28

Multi-Model

How is your data used?Relational, Graph or Document not SQL, RDF or JSON

This is how to optimize workloadsDrives access patterns, storage models, caching, distribution, etc.Requires thought at the core architecture

Simplifying utilities like transactions or consistency models are general

29

DBaaS & Automation

DBaaS simplifies cloud modelsShould be a logical unit to operate

Enables auto-pilot operationCan be automated to simplify operations

Not a “nice to have”A system cannot scale to any massive size without some amount of self-awareness

30

Some Requirements for a Distributed Database

Scale in & out on-demandProvide resiliency and online upgradePresent a logical, single system viewSupport multiple models on mixed infrastructureRun in multiple locationsBe simple to use

31

Distributed Database Designs

Approach Shared DiskShared-Nothing/

ShardedSynchronous Replication

DurableDistributed Cache

Key Idea Sharing a file system.Independent databases for disjoint subsets of

data.

Committing data transactionally to multiple

locations before returning.

Replicating data in memory on-demand.

Topology

Example Oracle RACDB2 Pure Scale

*VoltDBMySQL Cluster

and most NoSQL/NewSQL solutions

Google F1

32

*Note: Most major web properties include custom-sharded MySQL or sharded PostgreSQL, including Facebook, GOOGLE, Wikipedia, Amazon, Flickr, Box.net, and Heroku.

Peer to Peer Architecture

33

Scale-out PerformanceMulti-TenancyContinuous Availability

No-knobs Admin

Breakthrough Capabilities

34

• NuoDB scales to over 100 server machines

• Scalability is instant and elastic • Scales-out and scales-in• TPS numbers exceed 10m TPS on

$100k of hardware• Also scales on AWS, GCE etc. Public

demo of 32 nodes with GOOGLE• Now showing linear scalability on

TPC-C type workloads (DBT-2)• Scalability demonstrated with

heavier duty customer applications (eg Axway, Dassault Systémes)

• Self-healing• No single point of failure• Fully distributed control• Arbitrarily redundant• Online backup• Online schema evolution• Rolling upgrades

• HP Moonshot Launch – 45 Micro servers in a 4U rack mount box

• NuoDB ran 72,000 databases on a single Moonshot box

• Uses proprietary “Database Hibernation” and “Database Bursting” technologies

• Zero admin UI• Demo showed the potential of

“Software Defined Database”• Moonshot is the foundation of

the HP relationship

• Active/Active • ACID Semantics• Transactional

Consistency • N-Way Redundant• Local User Latency• Asynch WAN Comms

• Auto-admin• Rules-driven• Auto-optimizing• Auto-backup

Geo-Distribution

HTAP on NuoDB

35

TE TE TE TE TE TE

SM SM

Long-running Analytical Queries

Read/Write OLTP Workload

• MVCC: workloads operate on live data without lock contention• Scale-out architecture: workloads can be distributed across, and appropriately

matched to, machine resources to ensure consistent throughput for diverse operational and analytical workloads

• Scale-out architecture: burst out analytics to appropriate hardware when needed; upon completion, those resources can be spun down until needed again

Single logical database

Our investment in NuoDB demonstrates our strong interest and belief in NuoDB’s

strategy and technologies for next-generation cloud-based services.

“”

Dominique Florack, Senior Executive VP, Products-R&D

36

Thank you!