View
333
Download
2
Category
Tags:
Preview:
DESCRIPTION
Bernard Doering, Senior Slaes Director DACH, Cloudera. Hadoop and the Future of Data Management. As Hadoop takes the data management market by storm, organisations are evolving the role it plays in the modern data centre. Explore how this disruptive technology is quickly transforming an industry and how you can leverage it today, in combination with MongoDB, to drive meaningful change in your business.
Citation preview
1
I o T E U R O P E A N C I T Y T O U R
S T U T T G A R T
2
Hadoop and the Future of Data Management Bernard Doering Regional Sales Director, Central Europe
33
Leading the Way in Data ManagementPowered by Hadoop2008CLOUDERA FOUNDED BY MIKE OLSONAMR AWADALLAH &JEFF HAMMERBACHER
2009HADOOP CREATOR
DOUG CUTTING JOINS CLOUDERA
2009CLOUDERA RELEASES CDH THE FIRST COMMERCIAL APACHE HADOOP DISTRIBUTION
2010CLOUDERA MANAGER:
FIRST MANAGEMENT APPLICATION FOR
HADOOP
2011CLOUDERA REACHES 100 PRODUCTION CUSTOMERS
2011CLOUDERA UNIVERSITY
EXPANDS TO 140 COUNTRIES
2012CLOUDERA ENTERPRISE 4THE STANDARD FOR HADOOP IN THE ENTERPRISE
2012CLOUDERA CONNECT
REACHES 300 PARTNERS
2014THE ENTERPRISEDATA HUBLAUNCHED
2013CLOUDERA IMPALACLOUDERA NAVIGATORCLOUDERA SEARCH
2013TOM REILLY JOINS AS CEO
OVER 800 PARTNERS IN CLOUDERA CONNECT
CDHCloudera Manager
CLOUDERA ENTERPRISE
4ASK BIGGER QUESTIONS
ENTERPRISEDATA HUB
Intel Confidential4
Big Deal: Cloudera + IntelIntel invests $740M in Cloudera As Intel’s largest data center venture capital investment, which represents
Intel’s commitment to Internet of Things and Big Data Supports Cloudera’s ability to remain independent
Intel & Cloudera drive innovation through open source Accelerate evolution of Hadoop by joining forces on foundational
technologies Enable open source developers to innovate in and on top of the Hadoop
platform
Intel enables CDH to run best on Intel Architecture – performance optimisation Enables Cloudera to make best use of Intel data center technologies Provides datacenter infrastructure for Cloudera development &
benchmarking at scale
Intel Confidential5
Big Goal: Converge on one open source platform
• Most stable, compatible, and mature Hadoop distribution
• Leading SQL functionality & performance (Impala)
• Deepest management and governance capabilities
• 150 Hadoop developers• 100 open source committers
• The only distribution with performance and security enhanced from the silicon up
• Leading security capabilities including encryption, access control, and auditing
• 50 Hadoop developers and 12 committers
• Long-standing committment to open source with 1000 developers working on Linux, KVM, Xen, Java, OpenStack, Hadoop
6
Data drives innovation – Internet of Things
INTELLIGENT CLOUD
Richer data to analyze
2.8 Zettabytes of data generated WW
in 20121
SMART CLIENTS
Richer user experiences
Richer data from devices
INTELLIGENT THINGS
Sources: (1) IDC Digital Universe 2020, (2) IDC
40 Zettabytes of data will be generated
WW in 20201
7
Big Data is All Data and All Paradigms
Transactional & Application Data
Machine Data Social Data
• Volume • Structured• Throughput
• Velocity • Semi-structured • Ingestion
• Variety• Highly unstructured • Veracity
Enterprise Content
• Variety• Highly unstructured• Volume
88
Which one of these people is likely to be carrying a bomb?
Do you have any liquids in your carry-on?
99
Is it possible to set rates based on actual risk for each particular house?
How big is your house? What are comparable insurance claims rates?
1010
Which new technologies actually improve patient health?
What’s our budget for new equipment?
1111
Can we correlate manufacturing data with customer satisfaction?
Can a robot weld this car better than a person?
12 ©2014 Cloudera, Inc. All rights reserved.12
Expanding Data Requires A New Approach
1980sBring Data to Compute
NowBring Compute to Data
Relative size & complexity
DataInformation-centric
businesses use all data:
Multi-structured, internal & external data
of all types
Compute
Compute
Compute
Process-centric businesses use:
• Structured data mainly• Internal data only• “Important” data only
Compute
Compute
Compute
Data
Data
Data
Data
14 ©2014 Cloudera, Inc. All rights reserved.
The Old Way: Moving Data to ComputeHuge Investment in Specialized Systems that Treat Data as a Commodity
SERVERSMARTSEDWS DOCUMENTS STORAGE SEARCH ARCHIVE
ERP, CRM, RDBMS, MACHINES FILES, IMAGES, VIDEOS, LOGS, CLICKSTREAMS EXTERNAL DATA SOURCES
Major ChallengesMissing Data• Leaving data behind• Risk and compliance• High cost of storage
Complex Architecture• Many special-purpose systems• Moving data around• No complete views
Cost of Analytics• Existing systems strained• No agility• “BI backlog”
Time to Data• Up-front modeling• Transforms slow• Transforms lose data
15 ©2014 Cloudera, Inc. All rights reserved.
The Old Way: Siloed Business FunctionsLack of Coordination Increases Opportunity Costs and Decreases Data Availability
TRANSACTIONALRISKMARKETING LENDING CREDIT CARDS INVESTMENT
CUSTOMER DATATRANSACTIONS MARKET DATA RESEARCHLOGS
BACK OFFICE
Major Challenges
Poor Visibility
Inefficiency
Extreme Cost
Complexity
16 ©2014 Cloudera, Inc. All rights reserved.
The New Way: Bringing Compute to DataMaximize Benefit from All Your Data for Mission-Critical Jobs and Innovation
SERVERS MARTS EDWS DOCUMENTS STORAGE SEARCH ARCHIVE
ERP, CRM, RDBMS, MACHINES FILES, IMAGES, VIDEOS, LOGS, CLICKSTREAMS EXTERNAL DATA SOURCES
Major BenefitsActive Compliance Archive• Full fidelity original data• Indefinite time, any source• Lowest cost storage
Diverse Analytic Platform• Bring applications to data• Combine different workloads on
common data (i.e. SQL + Search)• True analytic agility
Self-Service Exploratory BI• Simple search + BI tools• “Schema on read” agility• Reduce BI user backlog requests
Persistent Storage• One source of data for all analytics• Persist state of transformed data• Significantly faster & cheaper
18
Data ScienceExplorationETLAcceleration
Operational Efficiency Information Advantage
CheapStorage
Business IT
Your Journey to Achieve Full Potential
©2014 Cloudera, Inc. All Rights Reserved.
EDWOptimization
Consolidation 360° View
Advance from Strategy to ROI with Best Practices and Peak Performance
19 ©2014 Cloudera, Inc. All rights reserved.19
From Hadoop to an Enterprise Data Hub
Open SourceScalableFlexibleCost-Effective
✔
Managed ✖Open Architecture ✖Secure and Governed ✖
✔
✔
✔
3RD PARTYAPPS
STORAGE FOR ANY TYPE OF DATAUNIFIED, ELASTIC, RESILIENT, SECURE
CLOUDERA’S ENTERPRISE DATA HUB
BATCHPROCESSING
MAPREDUCE
ANALYTICSQL
IMPALA
SEARCHENGINE
SOLR
MACHINELEARNING
SPARK
STREAMPROCESSINGSPARK STREAMING
WORKLOAD MANAGEMENT YARN
FILESYSTEMHDFS
ONLINE NOSQLHBASE
DATAM
ANAG
EMEN
TCLO
UD
ERA NAVIG
ATOR
SYSTEMM
ANAG
EMEN
TCLO
UD
ERA MAN
AGER
SENTRY, SECURE
20
WEB/MOBILE APPLICATIONS
ONLINE SERVING SYSTEM
ENTERPRISE DATA WAREHOUSE
ENTERPRISE REPORTINGBI / ANALYTICSMACHINE
LEARNINGCONVERGED
APPLICATIONSCLOUDERA MANAGER
META DATA / ETL TOOLS
ENTERPRISE DATA HUB
©2014 Cloudera, Inc. All Rights Reserved.
The Modern Information ArchitectureData Architects System Operators Engineers Data Scientists Analysts Business Users
Customers & End Users
SYS LOGS WEB LOGS FILES RDBMS
21
Customer Success Across Industries
Financial &Business Services
Telecom & Technology
Healthcare &Life Sciences
Media &Information
Retail &Consumer
Energy & Public Sector
©2014 Cloudera, Inc. All rights reserved.
22
BI and Analytics Partners
Enabling The App Store of Big Data
SI, Cloud, MSP Partners
Database Partners
Resellers
Data Integration PartnersHardware Partners
©2014 Cloudera, Inc. All rights reserved.
Intel Confidential
Partnership
● Combine the rich data from MongoDB with other data sources in Cloudera
● Leverage data from Cloudera in operational apps on MongoDB
24
Example - Storage Archive
eCommerce App Storage Archive
● Clicks● Behavior● Etc.
MongoDB Connector for Hadoop
● Profile Data● Product Catalog● Clicks● Etc.
25
MongoDB Connector for Hadoop
Example - ETL
eCommerce App ETLData
Warehouse
● Existing Reporting● Clicks● Behavior● Etc.
● Profile Data● Product Catalog● Clicks● Etc.
26
Example - Recommendation Analysis
eCommerce App Analysis
● CTR Analysis● Patterns
● Better recommendations in real-time
● Profile Data● Product Catalog● Clicks● Etc.
MongoDB Connector for Hadoop
27
Operational Analytical
28
Thank you!Bernard Doeringbdoering@cloudera.comTel. +49 172 692 9837
28
29
I o T E U R O P E A N C I T Y T O U R
S T U T T G A R T
Recommended