Upload
inside-analysis
View
116
Download
2
Embed Size (px)
DESCRIPTION
TechWise with Eric Kavanagh, Dr. Robin Bloor and Dr. Kirk Borne Live Webcast on July 23, 2014 Watch the archive: https://bloorgroup.webex.com/bloorgroup/lsr.php?RCID=59d50a520542ee7ed00a0c38e8319b54 Analytical applications are everywhere these days, and for good reason. Organizations large and small are using analytics to better understand any aspect of their business: customers, processes, behaviors, even competitors. There are several critical success factors for using analytics effectively: 1) know which kind of apps make sense for your company; 2) figure out which data sets you can use, both internal and external; 3) determine optimal roles and responsibilities for your team; 4) identify where you need help, either by hiring new employees or using consultants 5) manage your program effectively over time. Register for this episode of TechWise to learn from two of the most experienced analysts in the business: Dr. Robin Bloor, Chief Analyst of The Bloor Group, and Dr. Kirk Borne, Data Scientist, George Mason University. Each will provide their perspective on how companies can address each of the key success factors in building, refining and using analytics to improve their business. There will then be an extensive Q&A session in which attendees can ask detailed questions of our experts and get answers in real time. Registrants will also receive a consolidated deck of slides, not just from the main presenters, but also from a variety of software vendors who provide targeted solutions. Visit InsideAnlaysis.com for more information.
Citation preview
Grab some coffee and enjoy the pre-show banter before the top of the hour!
“How Can Analy,cs Improve Business?” TechWise Webcast | July 23, 2014
+
Guests
Dr. Robin Bloor Chief Analyst, The Bloor Group
Host: Eric Kavanagh CEO, The Bloor Group
Dr. Kirk Borne Data Scientist, George Mason University
PLUS: Will Gorman Chief Architect, Pentaho Steve Wilkes CTO, WebAction Frank Sanders Technical Director, MarkLogic Hannah Smalltree Director, Treasure Data
+
Analytics Can Help a Business: • Streamline operations • Improve marketing • Raise revenue • Identify opportunities • Assess plans
Executive Summary
+
Dr. Kirk Borne Data Scientist, George Mason University
Big Data Analytics for Data-to-Decisions Support
Kirk Borne George Mason University, Fairfax, VA ● www.kirkborne.net @KirkDBorne
Extrac,ng Knowledge, Insights, and Data-‐to-‐Decisions (D2D) from Big Data is hard!
The D2D Challenge**
!me
flux
1. Characterize and Contextualize first.
2. Collect and Curate each entity’s features.
…then Come to the data-driven decision!
• Data-to-Discoveries • Data-to-Decisions • Data-to-Dollars
Characteriza,on & Contextualiza,on Feature & Context Detection and Extraction:
• Identify and characterize features in the data: – Machine-generated – Human-generated – Crowdsourced? (= Tapping the Power of Human Cognition
to find patterns and anomalies in massive data!) • Extract the context of the data: the source, the channel,
the data user, the use cases, the value, the re-uses … where, when, who, how, what, why = Metadata!
• Curate these features for search, re-use, and D2D! • Find other parameters and features from other data
sources and databases – integrate all information to help characterize & contextualize (and ultimately make decision regarding) each new event.
• Report entity’s features & characteristics back to the database for search, retrieval, sharing, and reuse
• Individual (or groups of) entities (objects and/or events) are tagged and annotated ... – with new knowledge discovered – with related data/information of any kind – with common knowledge about those things – with inter-relationships between entities and their properties – with concepts – with context – i.e., assertions (e.g., classifications, interpretations, quality
flags, relationships, references, common knowledge, learned knowledge, inter-connectivity with other entities)
– with data collection parameters – with sensor channel descriptors
Semantics!
Provenance (for data curation)
Characterization via Tagging & Annotation
Data integration
Characteriza,on & Contextualiza,on Feature & Context Detection and Extraction:
• Identify and characterize features in the data: – Machine-generated – Human-generated – Crowdsourced? (= Tapping the Power of Human Cognition
to find patterns and anomalies in massive data!) • Extract the context of the data: the source, the channel,
the data user, the use cases, the value, the re-uses … where, when, who, how, what, why = Metadata!
• Curate these features for search, re-use, and D2D! • Find other parameters and features from other data
sources and databases – integrate all information to help characterize & contextualize (and ultimately make decision regarding) each new event.
Then what?
Then what? Get down to business with the Curated Collection of Characterizations and Contextualizations: • Data Analytics:
– Outlier / Anomaly / Novelty / Surprise detection – Clustering (= New Class discovery) – Correlation & Association discovery
• D2D: – Data-to-Discoveries – Data-to-Decisions – Data-to-Dollars
The Business Analyst-‐in-‐the-‐Loop Tags, annota,ons, features, and context – – These can be …
• measured (by observa,on), or • inferred through machine learning, or • provided by human analysts.
– The resul,ng synergy yields: • improved training sets, more accurate predic,ve models,
fewer false posi,ves/nega,ves, ac,ve learning, efficient human interven,ons
– Combining machine learning on Big Data with the power of human cogni,on for discovery (e.g., using Data Visualiza,on, Visual Analy,cs, Immersive Data Environments, or Crowdsourcing) therefore augments and accelerates discovery, insights, and D2D.
or all 3 of these processes simultaneously.
+
Dr. Robin Bloor Chief Analyst, The Bloor Group
The Data
Scientist &
The Business Analyst
Robin Bloor
The Data Analysis Budget
u Data Analysis is Business R&D
u The focus is on business process
u The outcome of successful R&D is a changed process
u Think of manufacturing for a useful example
Big Data Architecture
u Project manager
u Qualified statistician
u Domain Business expert
u Experienced data architect
u Software engineer
What is a Data Scientist?
(IT’S A TEAM)
The Impact of Machine Learning
Machine learning is changing the process (for the BUSINESS ANALYST & the DATA SCIENTIST) BUT the analytics team needs to understand IT!!
Take Note!
You can know more about a business
from its data than by any other
means
There are Two Issues for the Business Can you get the TECHNOLOGY right?
Can you get the PEOPLE right?
&
+
Will Gorman Chief Architect, Pentaho
© 2014, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 24
July 2014
Pentaho Business Analytics
Architected for the Future of Analytics
Will Gorman, Chief Architect
© 2014, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 25
Critical mass achieved • Over 1,500 commercial customers
• Over 10,000 production deployments
Innovation through open source • Open, pluggable, purpose-built for the future
• Early sustained leadership in big data ecosystem with technology innovation
Modern, cohesive data integration and business analytics platform • Full spectrum of advanced analytics for all key roles
• Embeddable, cloud-ready analytics
• Big data blending for analytics in real-time environments
• Broadest and deepest big data integration
WHAT WE DO
We enable the modern, big data-driven business
© 2014, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 26
Pentaho 5.1 Architected for the Future Simplified analytics @ scale for all users
© 2014, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 27
Evolving Big Data Architectures
Existing ETL Tool or PDI
EDW Data Marts
Analytics
Existing ETL Tool or PDI
Customer
Provisioning
Billing
Other BI Tools
© 2014, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 28
Evolving Big Data Architectures
Just-in-Time Integration P D I
PDI
Analytic DB
Location
Web
Social Media
Network
Existing Process or PDI Hadoop
Cluster
NoSQL
Existing ETL Tool or PDI
EDW Data Marts
Analytics
Existing ETL Tool or PDI
Customer
Provisioning
Billing
Other BI Tools
© 2014, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 29
Data integration
Business analytics +
The IT department
Lines of business +
Big data Any data +
Any data. Any environment. Any analytics.
The strength of Pentaho lies in the power of combination
© 2014, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 30
Thank You
blog.pentaho.com
@Pentaho
JOIN THE CONVERSATION. YOU CAN FIND US ON:
Pentaho Business Analytics
Facebook.com/Pentaho
+
Steve Wilkes CTO, WebAction
The Future of Data Driven Apps July 2014
WebAction® delivers the leading Real-time App Platform
enabling the next generation of Data Driven Apps for the Agile Enterprise
Acquire Store Process
Acquire Process in Memory Store
BI / Analytics RDBMS EDW
Structured Data
Machine Data
Location Click Stream
Structured Data
Machine Data
Location Click Stream
Data Driven Apps
RDBMS Hadoop
Batch Reactive
R E A LT I M E B A R R I E R
Proactive Real-time
Metadata
High Speed D
ata Acquisition
WActionStore
Distributed WAction Cache
Distributed DIM Processor
Tungsten Visualization Device Data
Big Data Infrastructure
Industry Data
Social Feeds
Transaction Data
Enterprise Applications
Enterprise Data Warehouse
RDBMS
Data Driven Apps
System/ IT Data
Security Event Processing
Cloud Application Control
Risk & Fraud Alerting
Quality of Service Management
Consumer Analytics
DataCenter Management
+
Frank Sanders Technical Director, MarkLogic
Slide 38 Copyright © 2010 MarkLogic® Corporation. All rights reserved. Slide 38 Copyright © 2011 MarkLogic® Corporation. All rights reserved.
© COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 38
Data Centered Approach is More Flexible
Slide 39 Copyright © 2010 MarkLogic® Corporation. All rights reserved. Slide 39 Copyright © 2011 MarkLogic® Corporation. All rights reserved.
© COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 39
Universal Index Powers Search & Analytics <SAR>
<title>
Suspicious vehicle…
<date>
2012-11-12Z
<type>
<threat>
suspicious activity
<category>
suspicious vehicle
<location>
<lat>
37.497075
<long>
-122.363319
<description>
A blue van…
<subject>
<subject>
<predicate>
<object>
IRIID
IRIID
isa
value
license-plate
ABC 123 <predicate>
<object>
observation/surveillance <type>
<triple>
<triple>
Unstructured full-text
Geospatial Values
Slide 40 Copyright © 2010 MarkLogic® Corporation. All rights reserved. Slide 40 Copyright © 2011 MarkLogic® Corporation. All rights reserved.
© COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 40
Fairfax County Police Events Application
Slide 41 Copyright © 2010 MarkLogic® Corporation. All rights reserved. Slide 41 Copyright © 2011 MarkLogic® Corporation. All rights reserved.
© COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 41
OECD Better Life Index
Slide 42 Copyright © 2010 MarkLogic® Corporation. All rights reserved. Slide 42 Copyright © 2011 MarkLogic® Corporation. All rights reserved.
© COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 42
MarkMail: Search-powered Visualization
+
Hannah Smalltree Director, Treasure Data
Copyright ©2014 Treasure Data. All Rights Reserved.
The Treasure Data Cloud Service
Store !
Cloud Storage!Managed, Monitored,
Scalable, Secure !
Web Mgmt. Console!View/query data, Access controls !
Analyze !
Query with SQL Multiple Query
Engines, Ad Hoc !!!
BI Tool Connectivity!Tableau, Most BI/Viz/
Analytics Tools !
Export !Query Results
or Datasets !Anytime !
Cloud Managed Service (SaaS) || <2 Week Setup || Flat monthly rate!
Collect !
Stream !Logs/Events in
Real-time !
Bulk Import!from Most Sources !
Copyright ©2014 Treasure Data. All Rights Reserved.
Specializing in Streaming “BIG” Data
Volume Velocity Variety
Examples: Clickstream, Web Access Logs, Mobile Data, App Logs, Event Logs, Sensors, Machine Data…
Copyright ©2014 Treasure Data. All Rights Reserved.
Big Data Analytics Use Cases
Use Case! Key Data Sources! Results! Treasure Example!
Website & "Mobile App "Behavior Analytics"
Mobile App Clicks "Web Clickstream"+ eComm, POS"
Increase sales and retail foot traffic within weeks"
Mobile Application Analytics"
Mobile Application Logs"
Increase Engagement (=Sales) by Iterating Quickly"
Product Behavior "& Sensor Analytics" Sensor Data"
Improved Product Development""
New Product/Service Development"
$216B Global Retailer
Video Games
Copyright ©2014 Treasure Data. All Rights Reserved.
Treasure Data In Your Analytics Environment
Collect" Store" Analyze"
Your"Server,"Device,"Gateway"etc…"
SQL"
Your BI, Visualization"Adv. Analytics"
Your Data Mart"Data Warehouse"
DBMS, etc."
Treasure Data Service"Streaming"
Export/Integrate"
Aggregates"
Copyright ©2014 Treasure Data. All Rights Reserved.
Resources
TreasureData.com!Datasheets, Case Studies, Whitepapers!
TDWI, 451, Analyst Whitepapers!Gartner Report: Cool Vendors in Big Data!
!Try the Starter Service For Free!
TreasureData.com/TryItNow!
+
Questions?
#TechWise or
USE THE Q&A
+
THANK YOU!
FIND THE ARCHIVE AT InsideAnalysis.com & Techopedia.com