Sta 111 - Summer II 2017Probability and Statistical Inference

10. Bootstrap intervals

Lu Wang

Duke University, Department of Statistical Science

July 18, 2017

Outline

1. Bootstrapping = sampling with replacement from the observedsample

2. Bootstrap percentile intervals: middle XX% of the bootstrapdistribution

3. Bootstrap testing for a single statistic requires shifting the datato ensure that the null hypothesis is true

4. Summary

1

Rotten horrors

is a movie aggregator,where the audience isalso able to review andscore the movies. Wewant to estimate themedian audience score ofhorror movies onRottenTomatoes.com. Westart with a randomsample of 20 horrormovies.

2

Data

title audience_score1 Patrick 522 Demon Seed 433 Tormented 344 Under the Bed 125 Phantasm IV: Oblivion 416 Fright Night Part 2 427 House of 1000 Corpses 658 Creepshow 2 469 The Forsaken 44

10 All the Boys Love Mandy Lane 3411 Jason Lives: Friday the 13th Part VI 5712 Vampire's Kiss 4813 The Witches of Eastwick 6014 Yellowbrickroad 2815 Dying Breed 2716 Carrie 7317 Whoever Slew Auntie Roo? 5618 The Mangler 2319 Primal 2920 The Twilight Saga: New Moon 65

3

First look

The histogram below shows the distribution of the audience scores ofthese movies (ranging from 0 to 100). The median score in the sample is43.5. Can we apply CLT based methods we have learned so far toconstruct a confidence interval for the median RottenTomatoes score ofhorror movies. Why or why not?

audience scores

10 20 30 40 50 60 70 80

02

46

4

Bootstrapping

▶ An alternative approach to constructing confidence intervalsis bootstrapping.

▶ This term comes from the phrase “pulling oneself up by one’sbootstraps”, which is a metaphor for accomplishing animpossible task without any outside help.

▶ In this case the impossible task is estimating a populationparameter, and we’ll accomplish it using data from only thegiven sample.

5

Bootstrapping

▶ Bootstrapping works as follows:(1) take a bootstrap sample - a random sample taken with replacement

from the original sample, of the same size as the original sample

(2) calculate the bootstrap statistic - a statistic such as mean, median,proportion, etc. computed on the bootstrap samples

(3) repeat steps (1) and (2) many times to create a bootstrap distribution- a distribution of bootstrap statistics

▶ The XX% bootstrap confidence interval can be estimated bythe cutoff values for the middle XX% of the bootstrapdistribution

6

Bootstrap sample 1(1) Take a bootstrap sample:

title audience_score1 Vampire's Kiss 482 Phantasm IV: Oblivion 413 House of 1000 Corpses 654 Dying Breed 275 Whoever Slew Auntie Roo? 566 The Forsaken 447 The Twilight Saga: New Moon 658 The Twilight Saga: New Moon 659 Whoever Slew Auntie Roo? 56

10 The Twilight Saga: New Moon 6511 The Mangler 2312 Dying Breed 2713 Creepshow 2 4614 House of 1000 Corpses 6515 Whoever Slew Auntie Roo? 5616 Tormented 3417 Jason Lives: Friday the 13th Part VI 5718 Vampire's Kiss 4819 Primal 2920 The Witches of Eastwick 60

(2) Calculate the median of the bootstrap sample:23, 27, 27, 29, 34, 41, 44, 46, 48, 48, 56, 56, 56, 57, 60, 65, 65, 65, 65, 65median = (48 + 56) / 2 = 52(3) Record this value

7

Bootstrap sample 2(1) Take another bootstrap sample:

title audience_score1 Fright Night Part 2 422 Carrie 733 The Forsaken 444 The Mangler 235 Primal 296 Patrick 527 Jason Lives: Friday the 13th Part VI 578 The Mangler 239 Vampire's Kiss 48

10 All the Boys Love Mandy Lane 3411 The Twilight Saga: New Moon 6512 All the Boys Love Mandy Lane 3413 Yellowbrickroad 2814 Vampire's Kiss 4815 Tormented 3416 The Mangler 2317 Phantasm IV: Oblivion 4118 Patrick 5219 House of 1000 Corpses 6520 The Twilight Saga: New Moon 65

(2) Calculate the median of the bootstrap sample:23, 23, 23, 28, 29, 34, 34, 34, 41, 42, 44, 48, 48, 52, 52, 57, 65, 65, 65, 73median = (42 + 44) / 2 = 43(3) Record this value

8

Many more bootstrap samples

... repeat

9

The dot plot below is the bootstrap distribution of mediansconstructed using 100 simulations. What does each dot on the dotplot represent?

bootstrap medians

●● ● ●●● ●● ●●

● ●●●

●

●● ●

●●●

●●

●● ●

●

●

●●●●●

●

●

●● ●●●

● ●

● ●

● ●●

●

●

●

●

●

● ●

●●

●

●●●

●●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●●● ●

●

●

●

●

●

●

●

●

●●●

●●

30 35 40 45 50 55

(a) Score of a horror movie in the original sample(b) Score of a horror movie in the population(c) Median from one bootstrap sample from the original sample(d) Median from one sample from the population

10

The dot plot below shows the distribution of 100 bootstrapmedians. Estimate the 90% bootstrap confidence interval for themedian RT score of horror movies using the percentile method.

bootstrap medians

●● ● ●●● ●● ●●

● ●●●

●

●● ●

●●●

●●

●● ●

●

●

●●●●●

●

●

●● ●●●

● ●

● ●

● ●●

●

●

●

●

●

● ●

●●

●

●●●

●●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●●● ●

●

●

●

●

●

●

●

●

●●●

●●

30 35 40 45 50 55

(a) (29, 58.5)(b) (34, 57)

(c) (37.5, 52)(d) (40, 49.5)

11

Bootstrap testing for a median

▶ This is very similar to bootstrapping, i.e. we randomly samplewith replacement from our data, but this time we need to firstshift the data to ensure that the null hypothesis is true.

– To get p-value, we need to know the sampling distribution of our teststatistic when the null hypothesis is true.

– Normal distribution might not be a good approximation here. Weinstead perform a simulation under conditions in which we know thenull hypothesis is true.

– So we use our data to represent the population, but first we shift itover to ensure that the null hypothesis is true.

▶ The p-value is then defined as the proportion of simulationsthat yield a sample statistic at least as favorable to thealternative hypothesis as the original observed samplestatistic.

12

Back to movie scoresThe median of the original sample is 43.5. Do the data provideconvincing evidence that the median audience score of horror movies isgreater than 40? The dot plot below is the bootstrap distribution ofmedians constructed using 100 draws from the shifted data (the medianis shifted to be 40).

bootstrap medians

●● ● ●●● ●● ●●

● ●●●

●

●● ●

●●●

●●

●● ●

●

●

●●●●●

●

●

●● ●●●

● ●

● ●

● ●●

●

●

●

●

●

● ●

●●

●

●●●

●●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●●● ●

●

●

●

●

●

●

●

●

●●●

●●

25 30 35 40 45 50 55

H0 : median = 40

HA : median > 40

p-value: proportion of simulations where thesimulated bootstrap sample median is at leastas extreme as the one observed (43.5). → 20/ 100 = 0.20

13

Summary of Bootstrap method

1. Bootstrapping = sampling with replacement from theobserved sample

2. Bootstrap percentile intervals: middle XX% of the bootstrapdistribution

3. Bootstrap testing for a single statistic requires shifting thedata to ensure that the null hypothesis is true

14