Upload
lars-trieloff
View
1.225
Download
5
Tags:
Embed Size (px)
DESCRIPTION
My presentation from Big Data Munich: How decision automation based on big data and machine learning can help you run a better business and avoid common cognitive biases.
Citation preview
A U T O M AT E D D E C I S I O N M A K I N G
L A R S T R I E L O F F – B L U E Y O N D E R – I ’ M @ T R I E L O F F O N T W I T T E R
T H E F R O N T I E RW H A T N O B O D Y T E L L S Y O U A B O U T
W H O I S U S I N G B I G D ATA T O D AY
W H E R E B I G D ATA I S U S E D
Effective Use
Marketing
Finance
Everyone Else
S O U R C E : O W N , P O T E N T I A L LY B I A S E D O B S E R VAT I O N S
Potential Use
Marketing
Finance
Everyone Else
S O U R C E : O W N , P O T E N T I A L LY B I A S E D O B S E R VAT I O N S
W H E R E B I G D ATA S H O U L D B E U S E D
T H R E E A P P R O A C H E S
Faster DataMore Data Better Decisions
M O R E D ATAD I G I TA L M A R K E T I N G ’ S A P P R O A C H :
FA S T E R D ATAF I N A N C E ’ S A P P R O A C H
B U T W H AT A B O U T B E T T E R D E C I S I O N S ?
Physiological
Safety
Love/Belonging
Esteem
Self-Actualization
Collection
Storage
Analysis
Prediction
Decision
Collection
Storage
Analysis
Prediction
Decision
Physiological
Safety
Love/Belonging
Esteem
Self-Actualization
“… all others bring data.”
— W. E D W A R D S D E M I N G
H O W D ATA - D R I V E N D E C I S I O N S S H O U L D W O R K
C O M P U T E R C O L L E C T S
C O M P U T E R S T O R E S
H U M A N A N A LY Z E S
H U M A N P R E D I C T S
H U M A N D E C I D E S
H O W D ATA - D R I V E N D E C I S I O N S R E A L LY W O R K
C O M P U T E R C O L L E C T S
C O M P U T E R S T O R E S
H U M A N A N A LY Z E S
C O M M U N I C AT I O N B R E A K D O W N
H U M A N D E C I D E S
C O M M U N I C AT I O N B R E A K D O W N
Communication Breakdown, It's always the same, I'm having a nervous breakdown, Drive me insane!
— L E D Z E P P E L I N
• Drill-down analysis … misunderstood or distorted• Metrics dashboards … contradictory and confusing• Monthly reports … ignored after two iterations• In-house analyst teams … overworked and powerless
H O W D ATA - D R I V E N D E C I S I O N S R E A L LY W O R K
H T T P : / / D I L B E R T. C O M / S T R I P S / C O M I C / 2 0 0 7 - 0 5 - 1 6 /
H O W D E C I S I O N S R E A L LY S H O U L D W O R K
C O M P U T E R C O L L E C T S
C O M P U T E R S T O R E S
C O M P U T E R A N A LY Z E S
C O M P U T E R P R E D I C T S
C O M P U T E R D E C I D E S
99.9% of all business decisions can be automated
H O W D E C I S I O N S A R E B E I N G M A D E
H O W D E C I S I O N S A R E B E I N G M A D E
9 0 % N O D E C I S I O N I S M A D E
9 0 % N O D E C I S I O N I S M A D E
9 % D E C I S I O N F O L L O W S R U L E
9 % D E C I S I O N F O L L O W S R U L E
1 % H U M A N D E C I S I O N M A K I N G
1 % H U M A N D E C I S I O N M A K I N G
– R O B I N S H A R M A
“Making no decision is a decision. To do nothing. And nothing always brings you nowhere..”
B U S I N E S S R U L E S F O R B E G I N N E R S
Not doing anything is the simplest business rule in the world – and also the most popular
A D VA N C E D B U S I N E S S R U L E S
Computers are machines following rules. This means business rules are programs.
B U S I N E S S R U L E S A R E P R O G R A M S , J U S T N O T V E R Y G O O D P R O G R A M S .
• Business rules are like programs – written by non-programmers
• Business rules can be contradictory, incomplete, and complex beyond comprehension
• Business rules have no built-in feedback mechanism: “It is the rule, because it is the rule”
H U M A N D E C I S I O N M A K I N G
• • System 1: Fast, automatic, frequent, emotional, stereotypic, subconscious
• • System 2: Slow, effortful, infrequent, logical, calculating, conscious
D A N I E L K A H N E M A N N , T H I N K I N G FA S T A N D S L O W
H O W T O D E C I D E FA S TF R E Q U E N T D E C I S I O N M A K I N G M E A N S FA S T D E C I S I O N M A K I N G , M E A N S U S I N G H E U R I S T I C S O R C O G N I T I V E B I A S E S
Anchoring effect
IKEA effect
Confirmation bias
Bandwagon effect
Substitution
Availability heuristic Texas Sharpshooter Fallacy
Rhyme as reason effect
Over-justification effect
Zero-risk bias
Framing effect
Illusory correlationSunk cost fallacy
Overconfidence
Outcome bias
Inattentional Blindness
Benjamin Franklin effect
Hindsight bias
Gambler’s fallacy
Anecdotal evidenceNegativity bias
Loss aversion
Backfire effect
• Abraham Lincoln and John F. Kennedy were both presidents of the United States, elected 100 years apart.
• Both were shot and killed by assassins who were known by three names with 15 letters, John Wilkes Booth and Lee Harvey Oswald, and neither killer would make it to trial.
• Lincoln had a secretary named Kennedy, and Kennedy had a secretary named Lincoln.
• They were both killed on a Friday while sitting next to their wives, Lincoln in the Ford Theater, Kennedy in a Lincoln made by Ford.
• Abraham Lincoln and John F. Kennedy were both presidents of the United States, elected 100 years apart.
• Both were shot and killed by assassins who were known by three names with 15 letters, John Wilkes Booth and Lee Harvey Oswald, and neither killer would make it to trial.
• Lincoln had a secretary named Kennedy, and Kennedy had a secretary named Lincoln.
• They were both killed on a Friday while sitting next to their wives, Lincoln in the Ford Theater, Kennedy in a Lincoln made by Ford.
H O W C O M P U T E R S D E C I D E FA S TM A C H I N E L E A R N I N G O F F E R S A N A LT E R N A T I V E T O H U M A N C O G N I T I V E B I A S E S A N D C A N B E M A D E FA S T T H R O U G H B I G D A TA
K-Means Clustering
Naive BayesSupport Vector Machines Affinity Propagation
Least Angle Regression
Nearest Neighbors
Decision Trees
Markov Chain Monte Carlo
Spectral clustering
Restricted Bolzmann Machines
Logistic Regression
M A C H I N E L E A R N I N G M E A N S P R O G R A M S T H AT W R I T E P R O G R A M S
• A machine learning algorithm is a system that derives a set of rules based on a set of data
• It is based on systematic observation, double-checking and cross-validation
• There is no magic, just data – and without data there no magic either
Better decisions through predictive applications
H O W P R E D I C T I V E A P P L I C AT I O N S W O R K
C O L L E C T & S T O R E
A N A LY Z E C O R R E L AT I O N S
B U I L D D E C I S I O N M O D E L
D E C I D E & T E S T O P T I M I Z E
W H Y T E S T ?
– R A N D A L L M U N R O E
“Correlation doesn’t imply causation, but it does waggle its eyebrows suggestively and gesture furtively while mouthing ‘look
over there’”
S T O R Y T I M E(Not safe for vegetarians)
T H E G R O U N D B E E F D I L E M M A
T H E G R O U N D B E E F D I L E M M A
Yesterday Today Tomorrow Next Delivery Next Day
In Stock Demand
T H E G R O U N D B E E F D I L E M M A
• Order too much and you will have to throw meat away when it goes bad. You lose money and cows die in vain
• Order too little and you won’t serve all your potential customers. You lose money and customers stay hungry.
A C C U R AT E LY P R E D I C T D E M A N DC H A L L E N G E # 1
A U T O M AT E R E P L E N I S H M E N TC H A L L E N G E # 2
C O L L E C T S T O C K A N D
S A L E S
P R E D I C T D E M A N D
T R A D E O F F C O S T S
C R E AT E O R D E R S I N E R P
S Y S T E M
O P T I M I Z E & R E P E AT
Collect Data Predict Outcomes Drive Decisions
Optimize Returns
Trend Estimation Classification Event Prediction
Reduce risk and cost, increase profit and revenue
Batch and streaming data ingestion, batch and streaming decision delivery (with
realtime option)
powered byBlue Yonder Platform
Secure Micro Cloud Infrastructure
Multi-Tenant Runtime Environment
Flex Storage Auto APIs Data ServicesJob Control &ML Toolkit
Application RuntimeRelational, Flat File
and In-Memory Storage Service
RESTful APIs for easy integration and data
accessPluggable machine learning pipelines
HTML5 based Web UI and API Builder
Public data prepared for machine learning
@ B YA N A LY T I C S _ E N @ T R I E L O F F