Upload
mongodb
View
122
Download
1
Embed Size (px)
Citation preview
Teradata QueryGrid to MongoDB Lightning Introduction
Rich Charucki - Teradata
2
What is a Teradata Data Warehouse?
• Analytic database– In-memory, in-database
• Scale-out MPP– 30+ petabyte sites– 35PB, 4096 cores
• Self service BI– Dashboards, reports, OLAP– Predictive analytics
• Complex SQL– 20-50 way joins– 350 pages of SQL
• Real time access/load
• Mixed workloads
Datascientists
Powerusers
Sales,partners
1024 nodes
IntelCPUs
512GB
IntelCPUs
512GB
IntelCPUs
512GB
IntelCPUs
512GB
3
JSONPath inside SQL
Color Size Prod_ID Create_Time----- ----- ------- -------------------Blue Small 96 2013-06-17 20:07:27
SELECT box.MFG_Line.Product.Color AS "Color", box.MFG_Line.Product.Size AS "Size", box.MFG_Line.Product.Prod_ID AS "Prod_ID", box.MFG_Line.Product.Create_Time AS "Create_Time"
FROM mfgTable WHERE CAST(box.MFG_Line.Product.Create_Time AS TIMESTAMP) >= TIMESTAMP'2013-06-16 00:00:00' AND box.MFG_Line.Product.Prod_ID = 96;
4
Math and Stats
DataMining
BusinessIntelligence
Applications
Languages
Marketing
ANALYTIC TOOLS &
APPSUSERS
UNIFIED DATA ARCHITECTURE
MarketingExecutives
OperationalSystems
FrontlineWorkers
CustomersPartners
Engineers
DataScientists
BusinessAnalysts
INTEGRATED DATA WAREHOUSE
DISCOVERY PLATFORM
DATA LAKE
REAL TIME PROCESSING
ERP
SCM
CRM
Images
Audio and Video
Machine Logs
Text
Web and Social
SOURCES
5
MONGODB
NoSQLDatabase
Teradata and MongoDB: QueryGridIDW
TERADATA DATABASE
Discovery
ASTER DATABASE
Business users Data scientists
TERADATA ASTER SQL,
SQL-MR,SQL-GR
Teradata Systems
TERADATA DATABASEHADOOP
Push-down
to Hadoop
SAS, Perl, R, Python,
Ruby
LANGUAGES
6
Integration Export / Import
Direct Connect
7
Teradata and MongoDB
• Operational + Analytical
– Rich MongoDB applications
– Rich Teradata analytics
– Complementary
• Teradata pulls directly from MongoDB sharded clusters
• Teradata pushes back to MongoDB deployments
MongoDB Teradata
Operational Data
Analytics
8
Scale-out NoSQL + Scale-out DW SQL Application
Primary
Shard 1
Primary
Shard 2
Primary
Shard N
Primary
Shard 3
Query router Query router Query router
NoSQL
SQL
AMPAMP
PE
AMPAMP
PE
AMPAMP
PE
AMPAMP
PE
9
Query Router
Shard 1
Shard 2
Shard 3
Shard 4
Contract Phase
Teradatanode
PE
SQL
EAH
AMP
AMP
AMP
AMP
10
Contract Phase
Teradatanode
AMP
AMP
AMP
AMP
EAH
Query Router
Shard 1
Shard 2
Shard 3
Shard 4
PE
11
Data Export to Shards
Teradatanode
EAH
Query Router
Shard 1
Shard 2
Shard 3
Shard 4
AMP
AMP
AMP
AMP
PE
12
Import Data from Shards
Teradatanode
EAH
Query Router
Shard 1
Shard 2
Shard 3
Shard 4
AMP
AMP
AMP
AMP
PE
13
Back-office context to the Front-office operations
Use cases
14
Data Warehouse
eCommerce in Action: A Virtuous Circle
Buyer preferencesSales catalogCampaigns
Recent purchasesProfitability
Shard
Shard
Shard
Shard
Shard
Shard
Shard
Shard
15
Data Warehouse
Shard
Shard
Shard
Shard
Shard
Shard
Shard
Shardreal time
Call Center Efficiency: A Virtuous Circle
Trouble ticketsCustomer profilesPayment history
ClaimsNext best offer
web logs
16
• Context from the DW– Enriching MongoDB applications
• Integration– Import/export – Teradata QueryGrid
• Two scale out architectures– OLTP scale-out – Analytics scale-out
• JSON in the data warehouse
Conclusions
1717