Teradata QueryGrid to MongoDB Lightning Introduction

Teradata QueryGrid to MongoDB Lightning Introduction

Rich Charucki - Teradata

2

What is a Teradata Data Warehouse?

• Analytic database– In-memory, in-database

• Scale-out MPP– 30+ petabyte sites– 35PB, 4096 cores

• Self service BI– Dashboards, reports, OLAP– Predictive analytics

• Complex SQL– 20-50 way joins– 350 pages of SQL

• Real time access/load

• Mixed workloads

Datascientists

Powerusers

Sales,partners

1024 nodes

IntelCPUs

512GB

IntelCPUs

512GB

IntelCPUs

512GB

IntelCPUs

512GB

3

JSONPath inside SQL

Color Size Prod_ID Create_Time----- ----- ------- -------------------Blue Small 96 2013-06-17 20:07:27

SELECT box.MFG_Line.Product.Color AS "Color", box.MFG_Line.Product.Size AS "Size", box.MFG_Line.Product.Prod_ID AS "Prod_ID", box.MFG_Line.Product.Create_Time AS "Create_Time"

FROM mfgTable WHERE CAST(box.MFG_Line.Product.Create_Time AS TIMESTAMP) >= TIMESTAMP'2013-06-16 00:00:00' AND box.MFG_Line.Product.Prod_ID = 96;

4

Math and Stats

DataMining

BusinessIntelligence

Applications

Languages

Marketing

ANALYTIC TOOLS &

APPSUSERS

UNIFIED DATA ARCHITECTURE

MarketingExecutives

OperationalSystems

FrontlineWorkers

CustomersPartners

Engineers

DataScientists

BusinessAnalysts

INTEGRATED DATA WAREHOUSE

DISCOVERY PLATFORM

DATA LAKE

REAL TIME PROCESSING

ERP

SCM

CRM

Images

Audio and Video

Machine Logs

Text

Web and Social

SOURCES

5

MONGODB

NoSQLDatabase

Teradata and MongoDB: QueryGridIDW

TERADATA DATABASE

Discovery

ASTER DATABASE

Business users Data scientists

TERADATA ASTER SQL,

SQL-MR,SQL-GR

Teradata Systems

TERADATA DATABASEHADOOP

Push-down

to Hadoop

SAS, Perl, R, Python,

Ruby

LANGUAGES

6

Integration Export / Import

Direct Connect

7

Teradata and MongoDB

• Operational + Analytical

– Rich MongoDB applications

– Rich Teradata analytics

– Complementary

• Teradata pulls directly from MongoDB sharded clusters

• Teradata pushes back to MongoDB deployments

MongoDB Teradata

Operational Data

Analytics

8

Scale-out NoSQL + Scale-out DW SQL Application

Primary

Shard 1

Primary

Shard 2

Primary

Shard N

Primary

Shard 3

Query router Query router Query router

NoSQL

SQL

AMPAMP

PE

AMPAMP

PE

AMPAMP

PE

AMPAMP

PE

9

Query Router

Shard 1

Shard 2

Shard 3

Shard 4

Contract Phase

Teradatanode

PE

SQL

EAH

AMP

AMP

AMP

AMP

10

Contract Phase

Teradatanode

AMP

AMP

AMP

AMP

EAH

Query Router

Shard 1

Shard 2

Shard 3

Shard 4

PE

11

Data Export to Shards

Teradatanode

EAH

Query Router

Shard 1

Shard 2

Shard 3

Shard 4

AMP

AMP

AMP

AMP

PE

12

Import Data from Shards

Teradatanode

EAH

Query Router

Shard 1

Shard 2

Shard 3

Shard 4

AMP

AMP

AMP

AMP

PE

13

Back-office context to the Front-office operations

Use cases

14

Data Warehouse

eCommerce in Action: A Virtuous Circle

Buyer preferencesSales catalogCampaigns

Recent purchasesProfitability

Shard

Shard

Shard

Shard

Shard

Shard

Shard

Shard

15

Data Warehouse

Shard

Shard

Shard

Shard

Shard

Shard

Shard

Shardreal time

Call Center Efficiency: A Virtuous Circle

Trouble ticketsCustomer profilesPayment history

ClaimsNext best offer

web logs

16

• Context from the DW– Enriching MongoDB applications

• Integration– Import/export – Teradata QueryGrid

• Two scale out architectures– OLTP scale-out – Analytics scale-out

• JSON in the data warehouse

Conclusions

1717