1

Baseball, Shakespeare, andModern Statistical Theory

Bradley EfronStanford

2

What Is “Statistics”?

3

PROSTATE CANCER DATA (Microarray)(Singh et al. 2002)

-0.91-0.790.00-0.80-0.80-0.70-0.67-0.09-0.25gene6033

0.100.09-0.89-0.88-0.87-0.91-0.881.33-0.90gene6032

-1.18-0.82-1.18-1.17-0.92-0.91-0.790.100.35gene6031

.

.

-0.14-0.14-0.10-1.080.941.701.050.18-1.12gene5

-1.130.43-0.19-0.36-0.13-1.13-0.102.42-0.36gene4

-0.03-1.100.094.040.11-1.160.220.100.06gene3

3.57-0.82-0.27-0.830.25-0.75-0.16-0.85-0.84gene2

1.470.732.77-1.09-0.58-0.99-1.08-0.75-0.93gene1

“z”pat102pat101pat52pat51pat50pat49pat2pat1

TESTSTATISTICS

PROSTATE CANCERHEALTHY

4

The Puzzled Physicist

5

BAYES’ RULE (1763)

6

Corbet’s Butterflies(Malaysia 1943)

7

8

Proving the Magic Formula

9

Empirical Bayes (1952)

10

Shakespeare’s Word Counts

837 …104314632292434314376?

6 …543210

11

Shakespeare’s Missing Words

12

“Shall I Die?”

13

Eighteen Baseball Players

0.265.265.265Grand Average0.239.200.1567/4518. Alvis0.244.316.1788/4517. Munson0.249.286.2009/4516. Campaneris0.254.226.22210/4515. E Rodriguez0.254.264.22210/4514. Petrocelli

:::::0.277.222.33315/454. Johnstone0.281.276.35616/453. F Howard0.286.298.37817/452. F Robinson0.290.346.40018/451. Clemente

James-Stein“TRUTH”Avehits/ABNameObserved

14

15

Stein Estimation (1956)

16

17

Frequentist Hypothesis Testing

18

PROSTATE CANCER DATA (Microarray)(Singh et al. 2002)

-0.91-0.790.00-0.80-0.80-0.70-0.67-0.09-0.25gene6033

0.100.09-0.89-0.88-0.87-0.91-0.881.33-0.90gene6032

-1.18-0.82-1.18-1.17-0.92-0.91-0.790.100.35gene6031

.

.

-0.14-0.14-0.10-1.080.941.701.050.18-1.12gene5

-1.130.43-0.19-0.36-0.13-1.13-0.102.42-0.36gene4

-0.03-1.100.094.040.11-1.160.220.100.06gene3

3.57-0.82-0.27-0.830.25-0.75-0.16-0.85-0.84gene2

1.470.732.77-1.09-0.58-0.99-1.08-0.75-0.93gene1

“z”pat102pat101pat52pat51pat50pat49pat2pat1

TESTSTATISTICS

PROSTATE CANCERHEALTHY

19

20

False Discovery Rates (1995)

21

Modern Statistical Theory (2000+)

22

23

References