Embed Size (px)
Citation preview
Catherine Leigh
Fewer HiPPOs
More Lab Coats
Highest paid person’s opinion Data driven decisions
Change is a constant
Change is a constant
2 months ago
Today
Some Examples
BlockbusterReal world examples
Ford EdselReal world examples
Ford EdselReal world examples
OJ SimpsonReal world examples
Probabilities can changeReal world examples
OJ SimpsonReal world examples
Medicine
http://www.alzheimersresearchuk.org/
Real world examples
Where do you sit?
Bad Good
Misinterpretation
Getting it right
Ignoring data
No data
How to Get it Right
Change is a constant
Change is a constant
Hypothesis testing is a tool for managing uncertainty
Change is a constant
Hypothesis generalisation:
H0: nothing is different(the null hypothesis)
H1: something is different(the alternative hypothesis)
Change is constant
Hypothesis example with coins:
H0: this coin is unbiased (the null hypothesis)
H1: this coin is biased(the alternative hypothesis)
Change is constant
Hypothesis example with coins:Change is constant
If H0 was true, what is the chance of seeing a result this extreme?
Hypothesis example with coins:Change is constant
> pbinom(9, size=20, prob=0.5) * 2[1] 0.8238029
Hypothesis example with coins:Change is constant
> pbinom(9, size=20, prob=0.5) * 2[1] 0.8238029
82%
Hypothesis example with an AB test:
H0: a new logo will not affect bounce rate*(the null hypothesis)
H1: a new logo will affect bounce rate(the alternative hypothesis)
Change is constant
Hypothesis example with an AB test:Change is constant
Hypothesis example with an AB test:Change is constant
Bounces
2+ Page Views
Total
Total
1000
4000
5000
1152
3853
5005
2152
7853
10005
Hypothesis example with an AB test:Change is constant
> library(stats)> amazingLogoTest <-+ matrix(c(1000, 4000, 1152, 3853),+ nrow = 2,+ dimnames = list(visitorAction = c("Bounce", "Stay"),+ logo = c("Boring", "Best Logo Ever")))> fisher.test(amazingLogoTest, alternative = "less")Fisher's Exact Test for CountDatadata: amazingLogoTestp-value = 0.0001316alternative hypothesis: true odds ratio is less than 195 percent confidence interval:0.0000000 0.9070399sample estimates:odds ratio0.8361718
Hypothesis example with an AB test:Change is constant
> library(stats)> amazingLogoTest <-+ matrix(c(1000, 4000, 1152, 3853),+ nrow = 2,+ dimnames = list(visitorAction = c("Bounce", "Stay"),+ logo = c("Boring", "Best Logo Ever")))> fisher.test(amazingLogoTest, alternative = "less")Fisher's Exact Test for CountDatadata: amazingLogoTestp-value = 0.0001316alternative hypothesis: true odds ratio is less than 195 percent confidence interval:0.0000000 0.9070399sample estimates:odds ratio0.8361718
0.01%
Change is a constant
Hypothesis testing is a tool for managing uncertainty
You will need...
34
You will need...
35
> install.packages(“stats”)> library(stats)
