Upload
future-insights
View
798
Download
0
Embed Size (px)
Citation preview
Pretty Pictures Zen, Data Visualization and the Art of Real-Time Decision-Making
Brandon Satrom @BrandonSatrom
SO. MUCH. DATA.
18% 25%
81%
!
"
#
1
2
4
3
Social Apps
$ % & ' ( $ ) * + , !
% & ' ) $ % ! ' ( ' (
$ % & ' ( $ ) * + , !
% & ' ) $ % ! ' ( ' (
App Analytics
$ % & ' ( $ ) * + , !
% & ' ) $ % ! ' ( ' (
$ % & ' ( $ ) * + , !
% & ' ) $ % ! ' ( ' (
DevOps
$ % & ' ( $ ) * + , !
% & ' ) $ % ! ' ( ' (
$ % & ' ( $ ) * + , !
% & ' ) $ % ! ' ( ' (
WHAT ARE WE DOING WITH ALL THIS DATA? $
-
15
30
45
60
What effect, good or bad, did the last release of your web or mobile
app have on customer conversion?
CONVERSION
15
30
45
60
Did that recent system outage negatively impact website
engagement? How do you plan to address it?
ENGAGEMENT
13
25
38
50
Why have only 40% of Android users installed the latest version of
your app?
ADOPTION
18
35
53
70
What’s causing all of those app crashes anyway?
BLOCKERS
QUESTIONS. QUESTIONS. QUESTIONS.
18
35
53
70
Did that new pair of shoes affect your speed or running style?
PERFORMANCE
15
30
45
60
How can your checkin history tell help you choose a restaurant or a beer or wine from this 3 page list?
PREFERENCES
13
25
38
50
What effect does meeting your step count goal for the day have on
your energy, diet or overall well-being?
GOALS
18
35
53
70
What impact does mood tracking or journaling have on your career
choices?
ASPIRATIONS
QUESTIONS. QUESTIONS. QUESTIONS.
Building better experiences for our customers?
DOES ANY OF THAT DATA MATTER IF WE’RE NOT…
1
2
3
Using insights to change and improve our behavior?
Improving the apps and systems we use to run our businesses?
HOW ARE WE MANAGING THE NOISE?
-
!
., /
0 1
2
3
4
5
6
7
89
:
;
?
=
>
@
#
A
B
D
E
F
G
H
300 billion
3 billion
1 billion
14 million
“BIGGER THAN THE INTERNET”
Internet Growth from 1993 - 2015
The Number of “Connected Things” by 2020
(Projected)
HOW DO WE SEPARATE THE SIGNAL FROM THE NOISE?
Tools for Data Ingestion Tools for Data Visualization Tools for Recommendation
THE RIGHT TOOLS + THE RIGHT APPS = TRUE INSIGHT
Apps that can be human-directed Apps that can learn Apps that take action
THE CONTINUUM OF DATA INSIGHT
INSIGHT
DIRECTIONHuman defines rules or conditions in the system in advance; System takes action when conditions are
met.
RECOMMENDATION
System makes suggestions based
on data & a human takes
action
VISUALIZATION
System provides visuals to help you
reason about data.
COLLECTION
System gathers data and stores it in some location
LEARNINGSystems that take action based on past behavior,
public information or other factors
Real-Time
SPEED OF DECISION-MAKING
SPEED OF DATA INGESTION
Data Collection
$ % & ' ( $ ) * + , !
% & ' ) $ % ! ' ( ' (
$ % & ' ( $ ) * + , !
% & ' ) $ % ! ' ( ' (
DATA COLLECTION/INGESTION SYSTEMS
1. Data is created (and stored locally)
2.Data is sent to another location for storage
3.The rest is up to you…
@
#
B
EH
J
KL
M
EXAMPLES OF DATA COLLECTION/INGESTION SYSTEMS
Microsoft Azure
AT&T M2X
Wolfram Data Drop
Telerik Backend Services
Parse
Building Collection Systems
$ % & ' ( $ ) * + , !
% & ' ) $ % ! ' ( ' (
$ % & ' ( $ ) * + , !
% & ' ) $ % ! ' ( ' (
• Data storage is foundational, so your options are endless…
• HOWEVER, if you have a choice, optimize for speed (of entry and retrieval)
• Real-time apps begin with real-time transport & storage
• Socket.io • Meteor • Firebase • MongoDB
CONSIDERATIONS FOR DATA COLLECTION SYSTEMS
CONSIDERATIONS FOR DATA COLLECTION SYSTEMS
• Even better, consider backends with built-in analytics capabilities
• InfluxDB + Grafana
• Wolfram Alpha
Data Visualization
$ % & ' ( $ ) * + , !
% & ' ) $ % ! ' ( ' (
$ % & ' ( $ ) * + , !
% & ' ) $ % ! ' ( ' (
DATA VISUALIZATION SYSTEMS
1. System aggregates and presents data in a consumable way.
2. Action taken in response, if any, is manual
3. Many Experiences insert “gamification” here to trigger action or improve engagement
,
N
|
• Favor tools that provide automated visualizations of your data…
•OR tools that make it easy to configure and analyze data.
• Automated Data Visualization Tools • Jupyter.Org • Grafana + InfluxDB •Wolfram Alpha •AT&T M2X
CONSIDERATIONS FOR DATA VISUALIZATION SYSTEMS
Data Recommendations
$ % & ' ( $ ) * + , !
% & ' ) $ % ! ' ( ' (
$ % & ' ( $ ) * + , !
% & ' ) $ % ! ' ( ' (
DATA RECOMMENDATION SYSTEMS
1. System processes data and suggests one or more actions
2. Human intervenes and takes action or redirects the decision
3. First popular in E-Commerce and Marketing Systems, but applicable elsewhere
18%
Likelihood to purchase a related product based on past history
25%
Category
81%
TYPES OF “PRODUCT RECOMMENDATION” SYSTEMS
MANUAL CROSS-SELL
CROWD-SOURCED
PERSONA- BASED
RULE- & ALGORITHM-
BASED
RECOMMENDATIONS - NOT JUST FOR E-COMMERCE
• Recommendations are applicable to nearly any problem we’re solving with software
• Many examples of recommender tools are popping up on Mobile
• Uses
• Prioritizing bugs based on crash report frequency/geo/other factors
• Suggesting web-pages for optimization based on automated funnel analysis
• Suggesting modification to watering frequency based on soil-moisture readings
78%Enterprise
Software Developers
19%Solo
Developers & Entrepreneurs
{{
• Note: not many tools exist outside of e-commerce/marketing for general use
• HOWEVER, a few open-source options and example applications do exist
• Consider looking at:
• LensKit • PredictionIO • Wikipedia SuggestBot • Cyclopath/Cycloplan
CONSIDERATIONS FOR RECOMMENDER SYSTEMS
Human-Directed Systems
$ % & ' ( $ ) * + , !
% & ' ) $ % ! ' ( ' (
$ % & ' ( $ ) * + , !
% & ' ) $ % ! ' ( ' (
HUMAN-DIRECTED SYSTEMS
1. System provides capability for defining rules or conditions in the system
2. Includes
• Rules & Workflow
• Triggers
• Data Tagging/Categorization
PQ
R
S T&
|
EXAMPLES OF HUMAN-DIRECTED SYSTEMS
Tagging (RunScribe)
Triggers (AT&T M2X)
Rules & Workflow (NodeRed)
Learning Systems
$ % & ' ( $ ) * + , !
% & ' ) $ % ! ' ( ' (
$ % & ' ( $ ) * + , !
% & ' ) $ % ! ' ( ' (
LEARNING SYSTEMS
1. Can apply insights based on publicly-available information or history
2. Takes action without intervention
3. Informs a human after the fact, who can refine and adjust the parameters for the next decision
)
UV
'
!
$X
*
♥
Z
[
=
• This is the wild-west of insight
• There are few tools available today to support this, out of the box
• Start with your recommendation and visualization tools and build from there
• Learning Systems Require:
• Enough Data Volume to be meaningful
• A Facility for automated decision-making and refinement (incl. manual)
•Rules++
CONSIDERATIONS FOR LEARNING SYSTEMS
1) \ 2 3] ^
Building Decision-Making Systems
$ % & ' ( $ ) * + , !
% & ' ) $ % ! ' ( ' (
$ % & ' ( $ ) * + , !
% & ' ) $ % ! ' ( ' (
“REAL-TIME”
• Real-time means…
• Speed of entry
• Speed of insight
• Speed of action@
#
B
EH
J
KL
M
BUILDING “REAL-TIME” APPS - TIPS
1. Use tools and transports that make real-time simple
• Meteor, Modulus.io & MongoDB
2. Push insights outward
• Push Notifications
• Triggers to other systems
3. Build with rules and workflow in mind
• Rules defined in advance
• Workflow between systems
4. Use public data to make decisions simple (or automatic)
@
#
B
EH
J
KL
M
Create an Arduino-based weather station that:
• Monitors environmental data from on-board sensors • Temp, pressure, wind speed & direction, rainfall, soil moisture, etc.
• Powered by a small solar panel with integrated battery monitoring • Posts all data to a cloud-hosted source • Can respond to environmental conditions and external inputs
SCENARIO. GARDEN WEATHER STATION
_
`V
|
a
Ingestion
• Store environmental data as a ContentType in Telerik Backend Services
• Store Battery Data (Charge, Voltage) as a separate ContentType
Visualization
• Create a monitoring dashboard that shows summary views for environmental and battery data
• Built with • Node.js, Express and Modulus (hosting) • Meteor for real-time communication • Kendo UI for web dashboard widgets/UI • NativeScript and UI for NativeScript for
Mobile
GARDEN WEATHER STATION - INGESTION AND VISUALIZATION
_
`V
|
a
∠
∠
Recommendations & Triggers
• Send a notification when: • Soil moisture falls below a certain level • Rainfall is greater than a certain amount • The temperature is nearing freezing • Device battery drops below a threshold or isn’t charging
enough over time • Recommendations can be combined with triggers to suggest
action
Human-Directed (Rules)
• Instruct the device to enter “low-power” mode when battery level is low
• Set thresholds for soil moisture, rainfall and wind speed • Trigger sprinkler system when moisture-level is low • Delay sprinkler system when rainfall surpasses a threshold
GARDEN WEATHER STATION - RECOMMENDATIONS & RULES
_
`V
|
a
∠
∠
GARDEN WEATHER STATION - LEARNING SYSTEMS
_
`V
|
a
Learning Systems
• Set a watering schedule for the week/day based on public forecast data
• Combine history and rules to adjust moisture and rain delay thresholds
• Re-position the solar panel (w/ servo) to obtain optimal charge • Self-monitor sensors and hardware and send notifications
when future malfunctions are likely
∠
THE CONTINUUM OF DATA INSIGHT
INSIGHT
DIRECTIONHuman defines rules or conditions in the system in advance; System takes action when conditions are
met.
RECOMMENDATION
System makes suggestions based
on data & a human takes
action
VISUALIZATION
System provides visuals to help you
reason about data.
COLLECTION
System gathers data and stores it in some location
LEARNINGSystems that take action based on past behavior,
public information or other factors
Real-Time
SPEED OF DECISION-MAKING
SPEED OF DATA INGESTION