Embed Size (px)
DESCRIPTION
Over the last few years we’ve seen a frenzy of interest and buzz around the area of Big Data. Beyond the hype, there is a solid base of growing use cases, which are becoming center stage to most businesses. 2012 was the year of awareness. There was a great amount of sharing from the early core developers of the analytic platforms – showing the rest of the world the capabilities of the tools and platforms that had been developed for special purpose high scale analytics. The big names at the core of open source analytics development include Facebook, eBay, Linkedin, Twitter – all blazing the trail with new approaches. These companies brought along with them a new and expanding interest in leveraging the same technologies for commercial interest. This talk is focused at how a growing number of enterprises that are already heavily invested in the use cases – but by volume, most customers now have some form of big data proof-of-concept underway. These proof of concepts typically start with a thesis of how competitive advantage can be gained through insight from the data. A proof of concept can quickly validate the theory, and helps sell further investment in the analytics platform, and it snowballs from there.
Citation preview
© 2009 VMware Inc. All rights reserved
Big Data: Now and in the Future
Is Your Cloud Ready for Big Data?
Richard McDougall
CTO, Application and Storage Services
2
Not Just for the Web Giants – The Intelligent Enterprise
3
Real-time analysis allows instant understanding of
market dynamics.
Retailers can have intimate understanding of their
customers needs.
Market Segment Analysis Personalized Customer Targeting`
4
The Emerging Pattern of Big Data Systems: Retail Example
Real-TimeStreams
Exa-scale Data Store
Parallel DataProcessing
Real-TimeProcessing
MachineLearning
Data Science
Cloud Infrastructure
5
Storage: Plan for Peta-scale Data Storage and Processing
2000 2003 2006 2009 2012 20150.01
0.1
1
10
100
1000
Online Apps
AnalyticsPB ofData
Analytics Rapidly Outgrows Traditional Data Size by 100x
6
Unprecedented Scale
“Data transparency, amplified by Social Networks
generates data at a scale never seen before”
- The Human Face of Big Data
We are creating an Exabyte of data every minute in 2013
Yottabyte by 2030
7
A single GE Jet Engine produces
10 Terabytes of data in one hour – 90 Petabytes per year.
Enabling early detection of faults, common mode failures, product engineering feedback.
Post Mortem Proactively Maintained Connected Product
8
The Emerging Pattern of Big Data Systems: Manufacturing
Exa-scale Data Store
Parallel DataProcessing
Real-TimeProcessing Machine
Learning
Data Science
Cloud Infrastructure
Real-TimeSensor
Analytics SupportProduct
Engineering
9
Cloud Infrastructure Supports Mixed Big Data Workloads
MachineLearning HadoopReal-Time
Analytics
Cloud Infrastructure
MachineLearning
Hadoop
Real-TimeAnalytics
Management
Network/Security
Storage/Availability
Compute
10
Software-defined Datacenter: Compute
Agility / Rapid deployment
Lower Capex
Isolation for resource control and security
1
2
3
Operational efficiency4
Management
The Core Values of Virtualization Apply to Big Data
Network/Security
Storage/Availability
Compute
11
Strong Isolation between Workloads is Key
Hungry Workload 1
Reckless Workload 2
NosyWorkload 3
Cloud Infrastructure
12
Management
Software-defined Datacenter: Storage
Requirements of Next Generation Storage
Network/Security
Storage/Availability
Compute
10x lower cost of storage
Handle explosive data growth
Support a variety ofapplication types
1
2
3
Solve the privacy andsecurity issues4
13
Software-defined Storage Enables Fundamental Economics
0.5 1 2 4 8 16 32 64 128 $-
$0.50
$1.00
$1.50
$2.00
$2.50
$3.00
$3.50
$4.00
$4.50
$5.00
$5.50
Cost per GB
Petabytes Deployed
TraditionalSAN/NAS
DistributedObject
StorageHDFSMAPRCEPH
Scale-out NASIsilon, NTAP
14
Management
Software-defined Datacenter: Network and Security
Automate secure network provisioning
Network & Security Requirements for Big Data
Network/Security
Storage/Availability
Compute
3
New high bandwidth network designs
1
Leverage Software-defined network security
2
15
Big Data Requires Extreme Bandwidth
16
Summary
17
Customers Winning from Consolidated Big Data Platforms
“Dedicated hardware makes no sense”
“Software-defined Datacenter enables rapid deployment multiple tenants and labs”
“Our mixed workloads include Hadoop, Database, ETL and
App-servers”
“Any performance penalties are minor”Management
Network/Security
Storage/Availability
Compute
18
Cloud Infrastructure is Ready for Big Data – Are you?
Cloud Infrastructure
19
Q&A
