Slide deck from Splunk and Hortonworks joint webinar on October 1, 2014. Title Building a Modern Data Architecture for Risk Management.
Citation preview
1. We do Hadoop together. Page 1 Hortonworks Inc. 2011 2014.
All Rights Reserved
2. Modern Data Architecture for Risk Management Page 2
Hortonworks Inc. 2011 2014. All Rights Reserved with Splunk and
Apache Hadoop
3. Quick Housekeeping Q&A box is available for your
questions Webinar will be recorded for future viewing Thank you for
joining! Page 3 Hortonworks Inc. 2011 2014. All Rights
Reserved
4. Your Presenters Brett Sheppard (@zettaforce) Director, Big
Data, Splunk Previously analyst at Gartner and DoD Big data and
enterprise architectures Steve Nunez Solution Principal,
Hortonworks Working with Hadoop since 2010 global insurance company
mining analytics from hundreds of millions of claims Solution
engineer in Silicon valley serving enterprise customers
5. Todays Topics Drivers for the Modern Data Architecture
Advanced Risk Management Analytics Demo: Risk Management with Hunk
and Hadoop Q&A Page 5 Hortonworks Inc. 2011 2014. All Rights
Reserved
6. Poll: Where are you in your Hadoop journey? 1. Researching
our options 2. Currently evaluating some software 3. Deep in a
trial 4. In production with a Hadoop cluster 5. Whats Hadoop?
7. Risk Management Challenges What are your banks biggest risk
management challenges? Respondents were asked to select no more
than three Source: FIS, April 2014
8. Regulatory Costs are Rising Source: NZZ (Switzerland)
9. A Data Architecture Under Pressure From New Data Business
Analy4cs RDBMS EDW MPP Hortonworks Inc. 2014 DATA SYSTEM
APPLICATIONS REPOSITORIES SOURCES Exis4ng Sources (CRM, ERP,
Clickstream, Logs) Custom Applica4ons Packaged Applica4ons 2.8 ZB
in 2012 85% from New Data Types 15x Machine Data by 2020 40 ZB by
2020 Source: IDC OLTP, ERP, CRM Systems Unstructured documents,
emails Server logs Clickstream Sen?ment, Web Data Sensor. Machine
Data Geoloca?on
10. Hadoop Within An Emerging Modern Data Architecture
Hortonworks Inc. 2014 DEV & DATA TOOLS Build & Test
OPERATIONS TOOLS Provision, Manage & Monitor DATA SYSTEM
REPOSITORIES SOURCES RDBMS EDW MPP OLTP, ERP, CRM Systems
Documents, Emails Web Logs, Click Streams Social Networks Machine
Generated Sensor Data Geoloca?on Data Governance & Integration
Security Operations Data Access Data Management APPLICATIONS
Business Analy4cs Custom Applica4ons Packaged Applica4ons
11. Hadoop: Typically Used For New Analytic Applications SCALE
SCOPE New Analytic Apps New types of data LOB-driven
12. Clickstream Capture and analyze website visitors data
trails and optimize your website Hortonworks Inc. 2014 Sensors
Discover patterns in data streaming automatically from remote
sensors and machines Server Logs Research logs to diagnose process
failures and prevent security breaches Hadoop Value: New types of
data Sentiment Understand how your customers feel about your brand
and products right now Geographic Analyze location-based data to
manage operations where they occur Unstructured Understand patterns
in files across millions of web pages, emails, and documents
13. New Analytic Applications For New Types Of Data Hortonworks
Inc. 2014 $ Supplier Consolidation Supply Chain and Logistics
Assembly Line Quality Assurance Proactive Maintenance Crowdsourced
Quality Assurance New Account Risk Screens Fraud Prevention Trading
Risk Maximize Deposit Spread Insurance Underwriting Accelerate Loan
Processing Call Detail Records (CDRs) Infrastructure Investment
Next Product to Buy (NPTB) Real-time Bandwidth Allocation New
Product Development 360 View of the Customer Analyze Brand
Sentiment Localized, Personalized Promotions Website Optimization
Optimal Store Layout Financial Services Retail Telecom
Manufacturing Healthcare Utilities, Oil & Gas Public Sector
Genomic data for medical trials Monitor patient vitals Reduce
re-admittance rates Store medical research data Recruit cohorts for
pharmaceutical trials Smart meter stream analysis Slow oil well
decline curves Optimize lease bidding Compliance reporting
Proactive equipment repair Seismic image processing Analyze public
sentiment Protect critical networks Prevent fraud and waste
Crowdsource reporting for repairs to infrastructure Fulfill open
records requests
14. And Incrementally Delivers A Data Lake A Modern Data
Architecture/Data Lake SCALE SCOPE RDBMS MPP EDW New Analytic Apps
New types of data LOB-driven Governance & Integration Security
Operations Data Access Data Management Data Lake An architectural
shift in the data center that uses Hadoop to deliver deeper insight
across a large, broad, diverse set of data at efficient scale
15. Example Journey Towards a Data Lake Hortonworks Inc. 2014
DATA VALUE Risk Management E.g., Fraud Reduction Operational
Excellence E.g., Network Maintenance Data Lake New Business E.g.,
Data as a Product Customer Intimacy E.g., 360 Degree View of the
Customer TBs PB PBs DATA LAKE An architectural shift in the data
center that uses Hadoop to deliver deep insight across a large,
broad, diverse set of data at efficient scale
16. Splunk, Part of the Modern Data Architecture Page 16
Hortonworks Inc. 2014 Bi-directional data integration between
Splunk & HDP Collect data from across the organization, deliver
it to Hadoop for refining data and batch analytics Output of Hadoop
jobs can be imported into Splunk Enterprise for rapid analysis and
visualization Archiving from Splunk Enterprise to Hadoop
17. Joint Value: New Types of Data = New Business Apps Unlock
new opportunities via analytic apps built around new types of data
Sentiment (social media) Clickstream Machine / sensor data Geo /
tracking data Web Logs Unstructured (video, pictures, free text)
Leverage Splunk platform for indexing / analyzing data for new
business applications LOB / Business IT oriented Page 17
Hortonworks Inc. 2011 2014. All Rights Reserved ENTERPRISE HADOOP
PLATFORM New Sources (sen4ment, clickstream, geo, sensor, )
18. Splunk, Hunk & Hortonworks YARN Ready Partner Certified
on latest release of Hortonworks Data Platform Existing Sandbox
tutorial Page 18 Hortonworks Inc. 2011 2014. All Rights
Reserved
19. Hunk + Hortonworks INTERACTIVE EXPLORATION Page 19
Hortonworks Inc. 2011 2014. All Rights Reserved FULL-FEATURED
ANALYTICS Explore, analyze and visualize data in HDP from one
integrated plaLorm Simply point Hunk at your HDP cluster(s) and
start exploring data immediately Search data, change perspec?ves
and preview results as MapReduce jobs run RICH DEVELOPER
ENVIRONMENT Build big data apps on data in HDP using standard web
languages and frameworks FAST TO DEPLOY AND DRIVE VALUE
20. Hortonworks Inc. 2014
21. Explore, Analyze, Visualize Raw Granular Data Security
& Compliance Digital Intelligence Business Risk Mgt. Analy?cs
360-degree Customer View Developer Platform (REST API, SDKs) Hadoop
Client Libraries Hunk Apps HDP Clusters NoSQL and Other Data Stores
Product and Service Analy?cs Internet of Things 2 1
22. Hunk Risk Management Analy?cs Preview results and
interac?vely search across one or more clusters Provides more
meaningful representa?on of underlying raw machine data Enables
non-technical users to build complex reports without learning the
search language Interac4ve Search Data Model Pivot
23. Interac?vely Ques?on Data in HDP Pause means stop fetching
results Stop means treat the current results as final and end the
MapReduce job
24. Hunk Applies Schema on the Fly Structure applied at search
?me No bri^le schema to work around Automa?cally find pa^erns and
trends
25. Dashboards for Self-Service Analy?cs Interac?ve Dashboards
and Charts Easy-to-use dashboard editor Chart overlay Pan and zoom
In-dashboard drilldown Embed charts and dashboards in 3rd party
apps
26. Hortonworks + Hunk = Business Value
27. Hortonworks + Hunk = Business Value Splunks Hunk is perhaps
the most promising technology to deliver a true interactive
experience. Especially powerful are Splunks capabilities for
discovering the structure of machine data and other unstructured
data on the fly.
28. About Splunk and Hortonworks
hortonworks.com/partner/splunk/ Get started with Hortonworks
Sandbox hortonworks.com/hadoop-tutorial/splunk-hunk/ Get started
with Hunk Sandbox splunk.com/hunk Follow us: @hortonworks @splunk
Tr y Now Question & Answer session will be conducted
electronically, using the panel to the right of your screen
29. Appendix Slides Hunk Demo Page 30 Hortonworks Inc. 2011
2014. All Rights Reserved
30. Risk Management Visual Analy?cs 31
31. Drill Down to Raw Data and Correlate Across Sources 32
32. Powerful Search with Point and Click Visualiza?ons 33
33. Easy to Configure Hunk with HDP 34
34. Define Virtual Indexes with Role-based Security 35