Numbersense: Clearing the Fog of Big Data

Numbersense: Clearing the Fog

of Big Data

Kaiser FungINFORMS NYC Luncheon

9/18/2013

Monday, September 23, 2013

Big Data studies

! Observational data

! Co-opted

! Seemingly exhaustive N

! Fused data

! No controls


Flight Delay: the data

! U.S. domestic commercial flights

! 1987 to 2008

! 123 million records

! 29 variables

Wicklin (2009)Monday, September 23, 2013

Which airline had a lower delay rate?


Which airline had a lower delay rate?


Ask the “right” question


Alaska’s On-time Performance

10.9%Alaska is the industry leader in on-time flights


Flight Delay: the data

! U.S. domestic commercial flights

! 1987 to 2008

! 123 million records

! 29 variables

Wicklin (2009)

In which variable(s) does Simpson’s paradox lurk?

A priori?


!"#$%&'(#%)#')*#%+(#%)#(,'(&-$.%&'(#%+$+*/0#0%&'(#%12-34*/5%6"#(#%+(#%&'(#%6"#'(-#05%&'(#%)'-$60%',%7-#85%&'(#%3'&)*#9-6/5%&'(#%3'$:*-360%+$;%&'(#%3'$,20-'$<%="#(#%-0%*#00%3*+(-6/5%*#00%3'$0#$020%+$;%*#00%3'$:-;#$3#<


Big Data: Producers

user interactions

web logs:distributedservers

dataware house

datamarts

displays

Excel

Dashboards

Data Cubes

Models

Forecasts


user interactions

displays

Excel

Dashboards

Data Cubes

Models

Forecasts

web logs... datamarts

Strategies

Tactics

Plans

Big Data: Consumers


Moneyball



Statistics !=

Math

David S. Moore, The Basic Practice of Statistics, ~2007Monday, September 23, 2013

The Obesity Epidemic


The Obesity Epidemic


Quetelet’s Index (1830)


BMI Critics (2000-)


BMI Critics (2000-)


BMI Critics (2000-)


Taking eyes off the ball

>?*6"'2."%@A?%-0%+%;-(#36%&#+02(#&#$6%',%,+6%+$;%+%B#66#(%&#+02(#%',%+;-)'0-6/%6"+$%CDE5%-6%-0%$'6%+%;-0#+0#%3'((#*+6#<F


Taking eyes off the ball

>?*6"'2."%@A?%-0%+%;-(#36%&#+02(#&#$6%',%,+6%+$;%+%B#66#(%&#+02(#%',%+;-)'0-6/%6"+$%CDE5%-6%-0%$'6%+%;-0#+0#%3'((#*+6#<F


The more things change

35 % 39 %

1 % 25 %

74%

26%BMI

DXA

Obese

Obese

Not Obese

Not Obese

DXA Totals

BMI Totals

All Pa!ents

100%

26 % 48 %

0 % 26 %

36 % 64 % 26 % 74 %

74%

26%

DXA

Obese

Obese

Not Obese

Not Obese

DXA Totals

BMI Totals

Female Pa!ents

BMI

100%


n-U-isance

0

50

100

150

200

250

300

350

15 20 25 30 35 40

Mortality Ra!o

Body Mass Index

over-weight

obese extremely obese


Reinstall Windows


Trust, not Truth


A team of psychologists performed personality tests on 100 professionals, of which 30 were engineers and 70 were lawyers. Here is a brief description of one of the subjects:

Jack is a 45-year-old man. He is married and has four children. He is generally conservative, careful, and ambitious. He shows no interest in political or social issues and spends most of his free time on his many hobbies, which include

home carpentry, sailing, and mathematics.

Kahneman and Tversky (1974)

Embarrassment of Riches


What is the probability that Jack is one of the 30 engineers?

A. 10 - 40 %

B. 41 - 60 %

C. 61 - 80 %

D. 81 - 100 %

Kahneman and Tversky (1974); Vanity FairMonday, September 23, 2013

The Law of Small Numbers is even more relevant in the era of Big Data


Target knows your daughter is pregnant

... before you do


Customer Acquisition

“Right around the birth of a child... parents are exhausted and overwhelmed and their shopping patterns and brand loyalties are up for grabs.”


Customer Acquisition

“We knew that if we could identify them in their second trimester, there’s a good chance we could capture them for years.”


Brochure Design

“We started mixing in all these ads for things we knew pregnant women would never buy, so the baby ads looked random.”


Brochure Design

“We’d put an ad for wineglasses next to infant clothes. That way, it looked like all the products were chosen by chance.”


Brochure Design

“As long as a pregnant woman thinks she hasn’t been spied on, she’ll use the coupons.”


The Model

Buy baby products

soon3

4

21

Ward made 4 of 25 related purchases

Coco Bu!er Lo"on XXL Purse

Zinc/Magnesium Supplement

Bright Blue Rug

Pregnancy Score = 87%


Mad Dad

Model Says

14%

10 90

4%

6%

76%

20

80

Pregnant Not

Pregnant

Not

Reality

100

False posi!ve rate:

False nega!ve rate:

Posi!ve predic!ve value:

Incidence:

1490

410

= 16%

= 40%

620

= 30%

10100

= 10%

3x


Sending Mixed Messages

Model Says

14%

10 90

4%

6%

76%

20

80

Pregnant Not

Pregnant

Not

Reality

100

False posi!ve rate:

False nega!ve rate:

Posi!ve predic!ve value:

Incidence:

1490

410

= 16%

= 40%

620

= 30%

10100

= 10%

3x


Intuition+Data!!Data+TheoryTrust, ! Truth!!Humans+DataLaw of small numbers


Thank you!Twitter: @junkcharts!Gmail: JunkCharts


Documents

Numbersense: Clearing the Fog of Big Data