Upload
emerson-louison
View
214
Download
1
Tags:
Embed Size (px)
Citation preview
1 HP Confidential©2011 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice
BIG DATA AND COLUMNAR DBMS
CJ Barton – Sales (412) 400-7771 [email protected] Carroll – SE (415) 505-3030 [email protected]
2 HP Confidential
Welcome/agenda• The Big Data Ecosystem
– Columnar DBMS– Unstructured Data– Hadoop
• Big Data Trends– Data Governance– Cloud– Mobile
• How to Survive in the Big Data World
• TCO Comparison
• Q&A
• Introduction– Announcement and what it means
• Big Data– Definition– Market Place– 3rd Party Validation
• Columnar DBMS- What is it?– Where is it in the market place?
3 HP Confidential3 HP Confidential
WHAT IS ‘BIG DATA’
4 HP Confidential
DEFINITION‘Big Data’ is a term applied to data sets whose size is beyond the ability of commonly used software tools to capture, manage, and process the data within a tolerable elapsed time. ‘Big data’ sizes are a constantly moving target currently ranging from a few dozen terabytes to many petabytes of data in a single data set.
EXAMPLES Web logs, RFID, sensor networks, social networks, Internet text and documents, Internet search indexing, call detail records, genomics, astronomy, biological research, military surveillance, medical records, photography archives, video archives, and large-scale eCommerce.Source: Wikipedia
5 HP Confidential
What’s going on in the industry?• 5 billion mobile phones in use in 2010
• 30 billion pieces of content shared every month on Facebook
• 40% projected growth in global data generated per year
• Budgets and IT staff relatively flat or declining
Source: McKinsey Global Institute – Big Data: The Next Frontier for Innovation, Competition and Productivity.
Figure 1: The Digital Universe 2009 - 2020
Growing by a Factor of 44
2009 0.8 ZB*
*Zettabyte = 1 trillion gigabytes
2020 35 ZB*
6 HP Confidential
Where are we seeing “Big Data”
FINANCIAL SERVICES
COMMUNICATIONS
CONSUMER MARKETINGHEALTHCARE
RETAILONLINE WEB AND GAMING
7 HP Confidential
There is strategic value in big data; with real-time analytics, organizations are able to maximize business value and efficiencies
What’s the value of Big Data? Opportunity to monetize ‘Big Data’ is everywhere
TECHNOLOGY
SensorsXML LOBs
IPV6
SOCIAL MEDIA
HEALTHCAREElectronic Patient Record
Medical Imaging GeneSequencing
COMPLIANCESarbanes-Oxley
HIPPABasel II
MOBILITY
GEOPHYSICAL EXPLORATION
ENTERPRISE
ERPCRMProducts
Customers
Suppliers
Partners
FINANCIAL SERVICESAlgorithmic Trading
High-frequency Trading
COMMUNICATIONS
Call Detail Records
8 HP Confidential
2011 Top Strategic Initiatives
1.Cloud Computing2.Mobile Applications3.Social Collaboration4.Video5.Next Generation Analytics6.Social Analytics7.Context Aware Computing8.Storage Class Memory9.Ubiquitous Computing10.Fabric Based Infrastructure
9 HP Confidential
We live in an analytics world• More data, and it comes in continuously
• No more overnight batch loading
• Mixed workloads and user variety accessing
• Must retain long history of data for compliance and analysis
• Need to customize and analyze diverse data/relations
• New Forms of Data for Mining (Logs, Social Media, Etc)
Creates a great opportunity!
10 HP Confidential
Power and benefits of real-time analytics• Create competitive differentiation via information and rich analytics
– Optimize user experiences via real-time campaign updates/management
– Customize interactions with constituents, clients, prospects via real-time engagement
• Reduce operational expense while improving critical Key Performance Indicators (KPIs)– Drastically reduce exposure to fraud and other nefarious business activities
• Understand brand sentiment and social trends– Proactively manage customer satisfaction and brand recognition
11 HP Confidential11 HP Confidential
WHAT ARE COLUMNAR DBMS AND HOW DO THEY SOLVE BIG DATA CHALLENGES?
12 HP Confidential
Data Base
OLTP 55-70%
Analytics 30-45%
Traditional Business and Technology Gap
Business WorkloadData Base
OLTP 95%
EDW5%
How IT is Deployed
One size does not fit all!!!!Cost UPComplexity UPPerformance DOWNScale LIMITED
13 HP Confidential
Next Generation Optimization
Data Base
OLTP 55-70% Purpose
Built Columnar 30-45%
Cost GREATLY REDUCEDComplexity GREATLY REDUCEDPerformance DRASTIC INCREASEScale LIMITLESS
14 HP Confidential
The Problem: Data, Access, Performance
Data volumes are growing at
increasing rates
Users ask questions iteratively on their
own
“Classic” DBMS are 30 years old and slowing you down
15 HP Confidential
Why Next Generation Analytics? • Legacy analytic methodologies are becoming obsolete
- Analysis based on summary data
- Poor performance- Application down-time - Batch-style loading and
querying- DB as a place to park the data- Canned SQL queries- 100+ control knobs to be
tweaked
Next-gen business models require next-gen analytics!
16 HP Confidential
Column-Store is Transformational & Shortest Time to Value
17 HP Confidential
The Forrester Wave™ Enterprise Data Warehousing Platforms, Q1 2011 With an increasing focus on performance,
scalability, optimized storage, and in-database analytics, Vertica Systems positions its EDW offerings as a robust platform for the most demanding enterprise analytics.Vertica’s customer momentum, coupled with its focus on enhancing its columnar-based EDW architecture, gives it a competitive advantage. Expect that Vertica will leverage these strengths, … to grow its share of the market among large enterprises looking for a high-performance massively parallel EDW.Source: Forrester Research, Inc., “The Forrester Wave™ Enterprise Data Warehousing Platforms, Q1 2011,” James G. Kobielus, 10 February 2011.
18 HP Confidential
Gartner Magic Quadrant for data warehouse database management systems – 2011
Source: Gartner, “Magic Quadrant for Data Warehouse Database Management Systems,” Donald Feinberg, Mark A. Beyer, 28 January 2011.
This Magic Quadrant graphic was published by Gartner, Inc. as part of a larger research note and should be evaluated in the context of the entire report. The Gartner report is available upon request from HP.
The Magic Quadrant is copyrighted January 28, 2011, by Gartner, Inc. and is reused with permission. The Magic Quadrant is a graphical representation of a marketplace at and for a specific time period. It depicts Gartner's analysis of how certain vendors measure against criteria for that marketplace, as defined by Gartner. Gartner does not endorse any vendor, product or service depicted in the Magic Quadrant, and does not advise technology users to select only those vendors placed in the "Leaders" quadrant. The Magic Quadrant is intended solely as a research tool, and is not meant to be a specific guide to action. Gartner disclaims all warranties, express or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.
19 HP Confidential
Row Oriented Columnar
DBMS COMPARISON
20 HP Confidential
Breaking traditional barriers to entry to managing Big DataVertica Analytics Platform
• Founded in 2005 by Michael Stonebraker – ‘Purpose Built’ analytic platform• Low-latency “Real Time” analytics• Powerful UDxF framework• 50–1000x faster performance than
traditional row-stores at ¼ the cost• Simple install/use with auto setup and tuning• Industry standard x86 hardware• Hybrid in-memory/on-disk architecture• Rich analytics – GIS, Event Series, GFI, Regression• Large scale, multi-use workloads
SPEED
SCALABILITY
SIMPLICITY
TCO