Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
The innovative HP Big Data Technology stackand the use cases
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Realtime Analytics of extreme data
Helmut SchmittSales Manager DACH
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.3
Big Data is a Massive Disruptor
“A 100 fold multiplication in the amount of data is a 10,000 fold multiplication in the number of patterns we can see in that data.”
Philip Evans: Boston Consulting Group Fellow, Ted Talk
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Industry-leading breath & depth of capabilities
ContextualSearch
DataExploration
Image/VideoAnalytics Geospatial
Analytics
SQL onHadoopAccelerated
AnalyticsSentiment
Analysis
PredicativeAnalytics
Haven Big Data Platform
Access Explore Enrich Analyze Predict Serve Act
Andmore…..
Core Big Data Business Capabilities
On-premise In the Cloud
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
DATA is an organization’s most strategic asset
Monetize
Differentiate
Personalize
Monitor
Meter
Optimize
Predict
…and more
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Monetize
Differentiate
Personalize
Monitor
Meter
Optimize
Predict
…and more
…and its greatest risk
Regulate
Comply
Control
Secure
Address
Ensure
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
The Big Data Balance Sheet
Monetize
Differentiate
Personalize
Monitor
Meter
Optimize
Predict
…and more
Regulate
Comply
Control
Secure
Address
Ensure
Assets Liabilities
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
We will be the trusted partner for every organization
Store
*Explore
*Govern
*
Protect
*
Serve
*
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.9
The Big Data flow
Store & Explore Serve Govern & Protect
Business resiliency
Operations Analytics
Predictive Maintenance
Smart Metering
Patient analytics
Fraud prevention
Records Management
Advertising analytics
Legal & Compliance
Structured enterprisedata repositories
Cloud-basedrepositories
Mobile & social media
Offsite or removable data repositories
Address business & operational objectivesEnterprise Content ManagementEnterprise Search & Collaboration
Legacy Data Cleanup
Legal HoldsInformation Archiving Records
ManagementeDiscovery
Address legal & compliance objectives
Backup & Recovery
Disaster Recovery
Address informationmanagement objectives
Business Resiliency
Long-Term Retention
Unstructured enterprisedata repositories
Data
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Used in association with ANPR
Vehicle Recognition
• Match Make and/or Model– Easy to train– Real-time matching
• Alert or Search for Vehicle without registration
• Validate database using ANPR result to identify illegal plated vehicles
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Core Capabilities – Built for Speed• We boost performance
Use to take Now takes
1 hour 3.6 Seconds
8 hours (overnight) Under 30 seconds
What 1000% means:
"When we did the first queries, they were done so fast, we thought they were broken.“
- Michael Relich, Guess?
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Secrets to Achieving Performance Increases
Columnar Storage
Compression MPP Scale-Out Distributed Query
Projections
Speeds Query Time by Reading Only Necessary Data
Lowers costly I/O to boost overall performance
Provides high scalability on
clusters with no name node or other
single point of failure
Any node can initiate the queries
and use other nodes for work. No single
point of failure
Combine high availability with
special optimizations for
query performance
CPU
Memory
Disk
CPU
Memory
Disk
CPU
Memory
Disk
A B D C E A
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.14
Query Optimization Comparison
Traditional Materialized Views
• Are secondary storage
• Are rigid: Practically limited to columns and query needs, more columns = more I/O
• Are mostly batch updated
• Provide high data latency
Vertica Projections
• Are primary storage – no base tables are required
• Can be segmented, partitioned, sorted, compressed and encoded to suit your needs
• Have a simple physical design
• Are efficient to load & maintain
• Are versatile – they can support any data model
• Allow you to work with the detailed data
• Provide near-real time low data latency
• Combine high availability with special optimizations for query performance
Traditional Indexes
• Are secondary storage pointing to base table data
• Support one clustered index at most – tough to scale out
• Require complex design choices
• Are expensive to update
• Provide high data latency
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.15
Analytical Features of VerticaVertica SQLStandard SQL-99 Conventions
Vertica Extended-SQLAdvanced Analytics with SQL
Vertica InnovationsAdvanced Analytics using Custom Logic
Vertica User Defined Extensions
Aggregate Sessionization Regression Testing Analytics• C++• Java• R
Connection• ODBC/JDBC• HIVE• Hadoop• Flex Zone
Analytical Time Series• Time slice• Interpolation (Constant & Linear)• Gap Filling• Aggregate
Statistical Modeling
Window Functions Event-based Windows• Conditional Change Event• Conditional True Event
Classification Algorithms
Graph Event Series Joins Page Rank
Monte Carlo Social Media/Pulse• Text Mining• Patterns/Trends
Text-mining
Geospatial Pattern Matching• Match, Define, Pattern Keywords• Funnel Analysis
Geospatial (Place)
Statistical
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.16
HP Vertica Distributed R
Challenge: Customers want to use R for analytics. However, R scalability is always a question
SOLUTION: HP Distributed RBenefit:
• Analyze data sets too large for standard R
• Perform complex analyses much more quickly (20x faster than Hadoop)
• Use familiar R environment to explore data, develop, and execute algorithms
• Operate on full data set (no down sampling)
R-based Analytics
Algorithm Use cases
Linear Regression (GLM)
Risk Analysis, Trend Analysis, etc.
Logistic Regression (GLM)
Customer Response modeling, Healthcare analytics (Disease analysis)
Random Forest Customer churn, Market campaign analysis
K-Means ClusteringCustomer segmentation, Fraud detection, Anomaly detection
Page Rank Identify influencers
CPU
Memory
Disk
CPU
Memory
Disk
CPU
Memory
Disk
R R R
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.17
• HP Vertica for SQL on Hadoop offers the only full-featured query engine on Hadoop
- Same Core Engine
- Hadoop Distribution Agnostic
- Enterprise-ready Solution
- World-class Enterprise Support and Services
- Open platform
- Ready for Haven
• Competitive price point
Introducing HP Vertica for SQL on Hadoop
Hadoop Storage
Vertica ANSI SQL
Data Exploration
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.18
One Query Engine to Serve it all
• Query data in place in Hadoop Formats
• Co-Locate and leverage existing Hadoop infrastructure
• HP Vertica performance on lower-cost infrastructure
• Single query engine across diverse formats and infrastructure
Query Engine
Format
File System Vertica (EXT4)
Vertica Optimized (ROS, Flex Tables)
HP VerticaANSI SQL
Hadoop (HDP, CDH, MapR NFS)
Hadoop (ORC, Parquet, et al)
Store Data in HP Vertica or any Hadoop Distribution
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.19
Which Version Is Right for You?
• For Hadoop environments only• Full MPP SQL engine• Includes JOINs, time series
analysis and Key Value• Management tools including
workload management, database designer and back-up and restore
• Hadoop Agnostic Compatibility• Flex Zone• Compression and Columnar Store• Java UDx
SQL on Hadoop
HP Vertica EE
Highly Optimized HP Vertica EXT4 file system
Accelerated Analytics , Live Aggregate projections, Geospatial and Sentiment Analysis
HP Vertica for SQL on Hadoop
• Discover Data
• Control Costs
• Leverage Hadoop Infrastructure
• No Frills, No Brainer
HP Vertica Enterprise Edition
• Boost Performance
• Faster Analytics
• Deeper Analytics
• Customize Analytics Infrastructure
• All the bells and whistles
C++ UDx / UDL
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.20
High End Scalability
Vertica Community edition:
Up to 3 nodes
Up to 1 Terabyte
Free for productive use
Scale up to Enterprise edition
Add nodes on the fly
Scale up to PB
Embed Hadoop
Think Big – Start Small
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.21
Leaders don’t make compromises
• Promotional Testing • Behavior Analytics
• Claims Analyses • Click Stream Analyses
• Patient Analyses • Network Analyses
• Clinical data Analyses • Customer Analytics
• Fraud Monitoring • Compliance Testing
• Financial Tracking • Loyalty Analysis
• Trading Analytics • Marketing Analytics
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.22
Communications, Media & Ent.
ConsumerWeb
Health & Life Sciences
RetailFinancial Services
Energy Public Sector
Clickstream Analytics ✓ ✓ ✓ ✓ ✓ ✓
Customer Analytics ✓ ✓ ✓ ✓ ✓ ✓ ✓
Hadoop Accelration
EDW Modernization
Fraud Detection ✓ ✓
Transaction Analytics ✓ ✓ ✓
Compliance ✓ ✓
Security ✓ ✓ ✓
Operations Analytics ✓
Sensor Data Analytics ✓ ✓
HP Vertica’s Top Use Cases & Verticals Click to view Use Case
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Big Data @ World TourPräsentationen:
14:20 - 14:50 Uhr- Big Data Anwendungsfälle – handfeste DemosBernd Mußmann
15:20 - 15:50 Uhr - Der innovative HP Big Data Technologiestack und seine EinsatzgebieteHelmut Schmitt
16:00 - 16:30 Uhr - Big Data Infrastrukturen und Services für das datenorientierte UnternehmenPhilipp Koik & Jochen Mohr
16:40 - 17:10 Uhr - Big Data as a Service – Herangehensweisen und BeispieleJens Scheffler
Ausstellung:
Cape-to-Cape
HP Big Data Referenzarchitektur
HP Big Data für die IT
HP Software Technologie-Stack:
HP Big Data Services
Transformation Experience Workshop:
GoalsUnderstanding what Big Data is Defining customer’s Big Data challenges Evaluating business and IT priorities Introducing HP Big Data solutionsBuilding a Big Data transformation roadmap
Customer Benefits
Understand the benefits, scope, scale and critical success factorsLeverage best practicesGain stakeholder commitmentEstablish a common understandingParticipants C-level, senior staff/initiative owners (5- 8 persons) 2-3 Sr. HP consultants, HP Sales
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HP Big Data Transformation WorkshopGoals
• Understanding what Big Data is • Defining your Big Data challenges • Evaluating business and IT priorities • Introducing HP Big Data • Building a customized Big Data transformation roadmap
Your Benefits
• Understand the benefits, scope, scale and critical success factors• Leverage best practices• Gain stakeholder commitment• Establish a common understanding, consensus and alignment
Participants :
• C-level, senior staff/initiative owners (5- 8 persons) • Senior HP consultants, HP Sales
Location & Time-slots :
• Reception/Check-in desk for Big Data Transformation Workshop, Level 1, Kap Europa
• Information desk for Transformation Workshops, Level 4- entrance of exhibition hall
• Session Options:-11:00 -11:30, 12:00 -12:30, 13:30-14:00, 14:20 – 14:50, 15:20 –15:50, 16:00 – 16:30, 16:40 – 17:10
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Rekordjäger Rainer Zietlow
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Thank you