35
We do Hadoop together.

Splunk-hortonworks-risk-management-oct-2014

Embed Size (px)

DESCRIPTION

Slide deck from Splunk and Hortonworks joint webinar on October 1, 2014. Title Building a Modern Data Architecture for Risk Management.

Citation preview

  • 1. We do Hadoop together. Page 1 Hortonworks Inc. 2011 2014. All Rights Reserved
  • 2. Modern Data Architecture for Risk Management Page 2 Hortonworks Inc. 2011 2014. All Rights Reserved with Splunk and Apache Hadoop
  • 3. Quick Housekeeping Q&A box is available for your questions Webinar will be recorded for future viewing Thank you for joining! Page 3 Hortonworks Inc. 2011 2014. All Rights Reserved
  • 4. Your Presenters Brett Sheppard (@zettaforce) Director, Big Data, Splunk Previously analyst at Gartner and DoD Big data and enterprise architectures Steve Nunez Solution Principal, Hortonworks Working with Hadoop since 2010 global insurance company mining analytics from hundreds of millions of claims Solution engineer in Silicon valley serving enterprise customers
  • 5. Todays Topics Drivers for the Modern Data Architecture Advanced Risk Management Analytics Demo: Risk Management with Hunk and Hadoop Q&A Page 5 Hortonworks Inc. 2011 2014. All Rights Reserved
  • 6. Poll: Where are you in your Hadoop journey? 1. Researching our options 2. Currently evaluating some software 3. Deep in a trial 4. In production with a Hadoop cluster 5. Whats Hadoop?
  • 7. Risk Management Challenges What are your banks biggest risk management challenges? Respondents were asked to select no more than three Source: FIS, April 2014
  • 8. Regulatory Costs are Rising Source: NZZ (Switzerland)
  • 9. A Data Architecture Under Pressure From New Data Business Analy4cs RDBMS EDW MPP Hortonworks Inc. 2014 DATA SYSTEM APPLICATIONS REPOSITORIES SOURCES Exis4ng Sources (CRM, ERP, Clickstream, Logs) Custom Applica4ons Packaged Applica4ons 2.8 ZB in 2012 85% from New Data Types 15x Machine Data by 2020 40 ZB by 2020 Source: IDC OLTP, ERP, CRM Systems Unstructured documents, emails Server logs Clickstream Sen?ment, Web Data Sensor. Machine Data Geoloca?on
  • 10. Hadoop Within An Emerging Modern Data Architecture Hortonworks Inc. 2014 DEV & DATA TOOLS Build & Test OPERATIONS TOOLS Provision, Manage & Monitor DATA SYSTEM REPOSITORIES SOURCES RDBMS EDW MPP OLTP, ERP, CRM Systems Documents, Emails Web Logs, Click Streams Social Networks Machine Generated Sensor Data Geoloca?on Data Governance & Integration Security Operations Data Access Data Management APPLICATIONS Business Analy4cs Custom Applica4ons Packaged Applica4ons
  • 11. Hadoop: Typically Used For New Analytic Applications SCALE SCOPE New Analytic Apps New types of data LOB-driven
  • 12. Clickstream Capture and analyze website visitors data trails and optimize your website Hortonworks Inc. 2014 Sensors Discover patterns in data streaming automatically from remote sensors and machines Server Logs Research logs to diagnose process failures and prevent security breaches Hadoop Value: New types of data Sentiment Understand how your customers feel about your brand and products right now Geographic Analyze location-based data to manage operations where they occur Unstructured Understand patterns in files across millions of web pages, emails, and documents
  • 13. New Analytic Applications For New Types Of Data Hortonworks Inc. 2014 $ Supplier Consolidation Supply Chain and Logistics Assembly Line Quality Assurance Proactive Maintenance Crowdsourced Quality Assurance New Account Risk Screens Fraud Prevention Trading Risk Maximize Deposit Spread Insurance Underwriting Accelerate Loan Processing Call Detail Records (CDRs) Infrastructure Investment Next Product to Buy (NPTB) Real-time Bandwidth Allocation New Product Development 360 View of the Customer Analyze Brand Sentiment Localized, Personalized Promotions Website Optimization Optimal Store Layout Financial Services Retail Telecom Manufacturing Healthcare Utilities, Oil & Gas Public Sector Genomic data for medical trials Monitor patient vitals Reduce re-admittance rates Store medical research data Recruit cohorts for pharmaceutical trials Smart meter stream analysis Slow oil well decline curves Optimize lease bidding Compliance reporting Proactive equipment repair Seismic image processing Analyze public sentiment Protect critical networks Prevent fraud and waste Crowdsource reporting for repairs to infrastructure Fulfill open records requests
  • 14. And Incrementally Delivers A Data Lake A Modern Data Architecture/Data Lake SCALE SCOPE RDBMS MPP EDW New Analytic Apps New types of data LOB-driven Governance & Integration Security Operations Data Access Data Management Data Lake An architectural shift in the data center that uses Hadoop to deliver deeper insight across a large, broad, diverse set of data at efficient scale
  • 15. Example Journey Towards a Data Lake Hortonworks Inc. 2014 DATA VALUE Risk Management E.g., Fraud Reduction Operational Excellence E.g., Network Maintenance Data Lake New Business E.g., Data as a Product Customer Intimacy E.g., 360 Degree View of the Customer TBs PB PBs DATA LAKE An architectural shift in the data center that uses Hadoop to deliver deep insight across a large, broad, diverse set of data at efficient scale
  • 16. Splunk, Part of the Modern Data Architecture Page 16 Hortonworks Inc. 2014 Bi-directional data integration between Splunk & HDP Collect data from across the organization, deliver it to Hadoop for refining data and batch analytics Output of Hadoop jobs can be imported into Splunk Enterprise for rapid analysis and visualization Archiving from Splunk Enterprise to Hadoop
  • 17. Joint Value: New Types of Data = New Business Apps Unlock new opportunities via analytic apps built around new types of data Sentiment (social media) Clickstream Machine / sensor data Geo / tracking data Web Logs Unstructured (video, pictures, free text) Leverage Splunk platform for indexing / analyzing data for new business applications LOB / Business IT oriented Page 17 Hortonworks Inc. 2011 2014. All Rights Reserved ENTERPRISE HADOOP PLATFORM New Sources (sen4ment, clickstream, geo, sensor, )
  • 18. Splunk, Hunk & Hortonworks YARN Ready Partner Certified on latest release of Hortonworks Data Platform Existing Sandbox tutorial Page 18 Hortonworks Inc. 2011 2014. All Rights Reserved
  • 19. Hunk + Hortonworks INTERACTIVE EXPLORATION Page 19 Hortonworks Inc. 2011 2014. All Rights Reserved FULL-FEATURED ANALYTICS Explore, analyze and visualize data in HDP from one integrated plaLorm Simply point Hunk at your HDP cluster(s) and start exploring data immediately Search data, change perspec?ves and preview results as MapReduce jobs run RICH DEVELOPER ENVIRONMENT Build big data apps on data in HDP using standard web languages and frameworks FAST TO DEPLOY AND DRIVE VALUE
  • 20. Hortonworks Inc. 2014
  • 21. Explore, Analyze, Visualize Raw Granular Data Security & Compliance Digital Intelligence Business Risk Mgt. Analy?cs 360-degree Customer View Developer Platform (REST API, SDKs) Hadoop Client Libraries Hunk Apps HDP Clusters NoSQL and Other Data Stores Product and Service Analy?cs Internet of Things 2 1
  • 22. Hunk Risk Management Analy?cs Preview results and interac?vely search across one or more clusters Provides more meaningful representa?on of underlying raw machine data Enables non-technical users to build complex reports without learning the search language Interac4ve Search Data Model Pivot
  • 23. Interac?vely Ques?on Data in HDP Pause means stop fetching results Stop means treat the current results as final and end the MapReduce job
  • 24. Hunk Applies Schema on the Fly Structure applied at search ?me No bri^le schema to work around Automa?cally find pa^erns and trends
  • 25. Dashboards for Self-Service Analy?cs Interac?ve Dashboards and Charts Easy-to-use dashboard editor Chart overlay Pan and zoom In-dashboard drilldown Embed charts and dashboards in 3rd party apps
  • 26. Hortonworks + Hunk = Business Value
  • 27. Hortonworks + Hunk = Business Value Splunks Hunk is perhaps the most promising technology to deliver a true interactive experience. Especially powerful are Splunks capabilities for discovering the structure of machine data and other unstructured data on the fly.
  • 28. About Splunk and Hortonworks hortonworks.com/partner/splunk/ Get started with Hortonworks Sandbox hortonworks.com/hadoop-tutorial/splunk-hunk/ Get started with Hunk Sandbox splunk.com/hunk Follow us: @hortonworks @splunk Tr y Now Question & Answer session will be conducted electronically, using the panel to the right of your screen
  • 29. Appendix Slides Hunk Demo Page 30 Hortonworks Inc. 2011 2014. All Rights Reserved
  • 30. Risk Management Visual Analy?cs 31
  • 31. Drill Down to Raw Data and Correlate Across Sources 32
  • 32. Powerful Search with Point and Click Visualiza?ons 33
  • 33. Easy to Configure Hunk with HDP 34
  • 34. Define Virtual Indexes with Role-based Security 35