Embed Size (px)
Citation preview
Stick Tossing and Confidence Intervals
Asilomar - December 2006
Bruce CohenLowell High School, SFUSD
[email protected]://www.cgl.ucsf.edu/home/bic
David SklarSan Francisco State University
Ver. 0.5
An Old Problem:When a thin stick of unit length is “randomly” tossed onto a grid of parallel lines spaced one unit apart what is the probability that the stick lands crossing a grid line?
We would like to take a purely experimental and statistical approach to the problem of finding, or at least estimating, the desired probability.
Estimating a Probability
Our experiments will consist of tossing a stick some fixed number of times, keeping track of how many times the stick lands crossing a grid line (the data), and computing the percentage of times this event occurs (a statistic).
Basic statistical theory will help us understand how to interpret these results.
Sketch of a proof of a special case of the Central Limit Theorem
Where does the procedure for finding confidence intervals come from? Why does it work?
A mathematical model for the data
The mathematics of the model
Plan
Estimating a simple probability
Toss sticks, gather data
Estimating the probability
Estimating the uncertainty in the estimate of the probability
Confidence intervals and what they mean
Background material
The average and standard deviation of a list of numbers
Histograms, what they are and what they aren’t
The average and standard deviation of a histogram
The normal curve
Box models and histograms for the sum of the draws
The Central Limit Theorem
0.5555 55.6%
20 1636 36
36
estimated probability
Estimating the Probability: A Sample Calculation
Result: 20 line crossings in 36 tosses
0.0828 8.3%
Conclusions: Based on this data an approximate 68% confidence interval for the probability that the stick lands crossing a line is 55.6% 8.3%
an approximate 95% confidence interval is 55.6% 16.6%
# crossings
# tosses
Standard Error (SE) for the
estimated probability
est. prob. est. prob. of
of crossing not crossing
# tosses
20 36
47.3% 63.9%
72.2%39.0%
68% Confidence Intervals for 10 Experiments
47.3% 63.9%
58.8% 74.6%
44.5% 61.1%
cross prob SE20 55.6% 8.3%
24 66.7% 7.9%
19 52.8% 8.3%
70.9% 84.7%
55.9% 71.2%
28 77.8% 6.9%
23 63.9% 8.0%
61.7% 77.1%
58.8% 74.6%
55.9% 71.2%
25 69.4% 7.7%
24 66.7% 7.9%
27 75.0% 7.2%
23 63.9% 8.0%
21 58.3% 8.2%
67.8% 82.2%
50.1% 66.5%
60% 70% 80%40% 50%
estimated(36 tosses per experiment)
70.0%60.0%
62.5% 67.5%
Pooling the data
Result: 234 line crossings in 360 (independent) tosses
# crossings 234estimated probability 65.0%
# tosses 360
est. prob. est. prob. of
.650 .350of crossing not crossing
# tosses 360
0.025 2.5%
Conclusions: Based on this data an approximate 68% confidence interval for the probability that the stick lands crossing a line is 65.0% 2.5%
an approximate 95% confidence interval is 65.0% 5.0%
Standard Error (SE) for the
estimated probability
68% Confidence Intervals for 10 Experiments
47.3% 63.9%
58.8% 74.6%
44.5% 61.1%
70.9% 84.7%
55.9% 71.2%
cross prob error20 55.6% 8.3%
24 66.7% 7.9%
19 52.8% 8.3%
28 77.8% 6.9%
23 63.9% 8.0%
61.7% 77.1%
58.8% 74.6%
67.8% 82.2%
55.9% 71.2%
25 69.4% 7.7%
24 66.7% 7.9%
27 75.0% 7.2%
23 63.9% 8.0%
60% 70% 80%40% 50%
estimated(36 tosses per experiment)
62.5% 67.5%
21 58.3% 8.2%50.1% 66.5%
234 65.0% 2.5%
Some 95% Confidence Intervals
Where Does the Procedure for FindingConfidence Intervals Come From?
As with all “real world” applications of mathematics we begin with a Mathematical Model.
1 0? ??Box Model
The number of line crossings in n tosses of the stick is like the Sum of values of n draws at random with replacement from a box with two kinds of numbered tickets. Those numbered 1 correspond to the stick landing crossing a line, and those numbered 0 to not crossing. The percentage of tickets numbered 1 in the box is not known.
This unknown percentage corresponds to the probability that a stick lands crossing a line.
The n drawn tickets are a sample, and the % of 1’s in the sample is a statistic.
The set of tickets in the box is called the population, and the (unknown) % of 1’s in the population is a parameter.
Note: this kind of box is called a zero–one box.
The Mathematics of the Model
The goal for the rest of the talk is to develop the mathematics of the box model.
We first review some basic background material which we then use tounderstand the behavior of the sum of the draws from a box of knowncomposition. Finally we use this understanding to see why the confidencelevels come from areas under the normal curve.
The Average and Standard Deviation of a List of Numbers
Example List: 21, 28, 30, 30, 34, 37
deviation element average deviations 9, 2, 0, 0, 4, 7
The Standard Deviation (SD) mean of the squared deviations
2 2 2 2 2 29 2 0 0 4 7 5
6
The SD measures the spread of the list about the mean. It has the same units as the values in the list. It is a natural scale for the list: we are often more interested in how many SD’s a value is from the mean than in the value itself.
25 35
The average is the balance point.
The SD measures the spread.
The mean measures the “center” of the list.
sum of the valuesThe mean or average
number of elements
30
The Average and Standard Deviation of a List of NumbersFor a list consisting of just 0’s and 1’s we have:
sum of the values number of onesaverage fraction of 1's
number of elements number of elements
and with some algebra we can show that
SD mean of the squared deviations fractions of 1's fractions of 0's
# crossings sample # of 1's estimated probability sample fraction of 1's
# tosses sample size
est. prob. est. prob. of sample sample
of crossing not crossing fraction of 1's fraction of 0'sSE
# tosses sample size
We can now re-interpret the procedure for estimating our probability
sample average
sample SD
sample size
Properties of The Average and Standard Deviation
1. If we add a constant, B, to each element of a list the average of the new list is the old
average + B.
2. If we multiply each element of a list by a constant, A, the average of the new list is A times the old
average.
3. If we add a constant, B, to each element of a list the SD of the new list is the old SD.
4. If we multiply each element of a list by a constant, A, the SD of the new list is |A| times the old SD.
Standard Units
A list in standard units will have mean 0 and SD 1.
We are often more interested in how many SD’s a value is from the mean than in the value itself. For example: 37 is 1.4 SD’s above the average or 28 is 0.4 SD’s below the average.
deviation value average -value
SD SDz
The value of an element in Standard Units is the the number of SD’s it is above (positive), or below (negative) the mean. To convert a value to standard units use
List: 21, 28, 30, 30, 34, 37 with average 30 and SD 5Example
In Standard Units: -1.8, -0.4, 0, 0, 0.8, 1.4
For many lists roughly 68% of the values liewithin 1 SD of the mean and 95% lie within 2
SD’s.
value in standard units
Adding a constant to each element of a list or multiplying each element by a constant will not change the values of the elements in standard units.
From Lists to Histograms
23, 29, 30, 31, 35, 38, 40, 41, 42, 45, 46, 51, 52, 54, 55, 55, 57, 58, 59, 60, 61, 63, 69, 70, 70, 71, 71, 74, 75, 75, 82, 85, 86, 91, 91, 93. Note:
Example: 36 Exam Scores
20 - 3838 - 5050 - 7474 - 9090 - 100
class intervals
6 5#
16.713.9%
1.4 16 44.4 1.9 6 3
16.7 8.3
1.10.8
0.8
density(% /point)
A Histogram represents the percentages by areas (not by heights).
A histogram is not a bar chart.
Av = 59.1, SD = 18.9
Endpoint convention: class intervals contain left endpoints, but not right endpoints
13.9 % 18 pts density in % pt
De
nsi
ty (
% p
er
po
int)
0.0
0.5
2.0
1.5
1.0
scores20 40 60 80 100
13.9%
16.7%
44.4%
16.7% 8.3%
(0.8)
(1.0)
(1.9)
(1.4)
(0.8)
area in % (width in pts)(height in %/pt)
A Histogram is Not A Bar Chart
A Histogram represents the percentages by areas (not by heights).
A histogram is not a bar chart.
De
nsi
ty (
% p
er
po
int)
0.0
0.5
2.0
1.5
1.0
scores20 40 60 80 100
13.9%
16.7%
44.4%
16.7% 8.3%
(0.8)
(1.0)
(1.9)
(1.4)
(0.8)
Histogram of Scores
20 38 50 74 90 100scores
13.9%16.7%
44.4%
16.7%
% o
f to
tal p
ap
ers
01
04
03
02
0
Bar Chart of Scores
8.3%
De
nsi
ty (
% p
er
po
int)
0.0
0.5
2.0
1.5
1.0
scores
13.9%
16.7%
44.4%
16.7% 8.3%
(0.8)
(1.0)
(1.9)
(1.4)
(0.8)
20 40 60 80 100
The Average and Standard Deviation of a Histogram
To find the mean or average of a histogram first list the center of each class interval then multiply each by the area of the block above it and finally sum.
Histogram Av 29 .139 +44 .167 +62 .444 +82 .167 +95 .083 60.5
Class intervals: 20 to 38, 38 to 50, 50 to 74, 74 to 90, 90 to 100
To find the standard deviation of a histogram find the squared deviations of the center of each class interval, then multiply each by the area of its corresponding block, then sum, and finally take the square root.
2 2
2 2
2
29 60.5 .139 44 - 60.5 .167
SD 62 - 60.5 .444 82 - 60.5 .167
95 - 60.5 .083
19.0
SD = 19
Av = 60.5
For many histograms roughly 68% of the area lies within 1 SD of the mean and 95% lies within 2 SD’s.
[Note for the original data: Av = 59.1, SD = 18.9]
List of midpoints: 29, 44, 62, 82, 95
Histograms and Standard Units
De
nsi
ty (
% p
er
po
int)
scores
0.0
0.5
2.0
1.5
1.0
Av = 60.5
SD = 19
Standard Units
0 1 2 3-1-2-3
The Normal Curve
2
2
The equation for the
Standard Normal Curve is
1
2
z
y f z e
2
221
the family: 2
x
g x e
From: Freedman, Pisani, and Purves, Statistics, 3rd Ed.
The normal curve was discovered by Abraham De Moivre around 1720. Around 1870 Adolph Quetelet had the idea of using it as an ideal histogram to which histograms for data could be compared. Many histograms follow the normal curve and many do not.
Height(% per Std.U.)
Area(percent)
De
nsi
ty (
% p
er
po
int)
scores
0.0
0.5
2.0
1.5
1.0
Av = 60.5
SD = 19
Standard Units
0 1 2 3-1-2-3
Histograms, Standard Units, and the Normal curve
Data Histograms and Probability Histograms
Discrete data convention
From: Freedman, Pisani, and Purves, Statistics, 3rd ed.
Data Histograms and Probability Histograms forthe Sum of the Draws
The Central Limit Theorem
There are many Central Limit Theorems. We state two in terms of box models. The second is a special case of the first and it covers the model we are dealing with in our stick tossing problem. It goes back to the early eighteenth century.
When drawing at random with replacement from a box of numbered tickets (with bounded range), the probability histogram for the sum of the draws will follow the standard normal curve, even if the the contents of the box do not. The histogram must be put into standard units, and the number of draws must be reasonably large.
De Moivre – La Place version: When drawing at random with replacement from a zero-one box, the probability histogram for the sum of the draws will follow the standard normal curve, even if the the contents of the box do not. The histogram must be put into standard units, and the number of draws must be reasonably large.
The Normal Curve and Probability Histograms forthe Sum of the Draws
100
50
00 1
Histogram for the box
From: Freedman, Pisani, and Purves
1 0
provides a box model for counting the number of heads in n tosses of a fair coin.
The Normal Curve and Probability Histograms forthe Sum of the Draws
From: Freedman, …
The Normal Curve and Probability Histograms forthe Sum of the Draws
From: Freedman, …
Histogram for the box
1 2 9
The Central Limit Theorems
When drawing at random with replacement from a box of numbered tickets (with bounded range), the probability histogram for the sum (and average) of the draws will follow the standard normal curve, even if the the contents of the box do not. The histogram must be put into standard units, and the number of draws must be reasonably large.
De Moivre – La Place version: When drawing at random with replacement from a zero-one box, the probability histogram for the sum (and average) of the draws will follow the standard normal curve, even if the the contents of the box do not. The histogram must be put into standard units, and the number of draws must be reasonably large.
The probability histogram for the average of the draws, when put in standard units is the same as for the sum because multiplying each value of the sum by 1/(# of draws) won’t change the corresponding values in standard units.
Where Does the 68% Confidence Level Come From?
Sample Average
True Population Average
Pop. SD
# of drawsTrue SE for the
average of the drawsEstimated SE for the average
SD of the sample
# of draws
Standard units
1
Since the estimated SE for the average computed from sample is, on average, about equal to the true SE a 68% confidence interval will cover the true population mean whenever the sample mean is within 1 SE of the true mean. The probability of this happening is, by the central limit theorem, the area within 1 standard unit of 0 under the normal curve, and this area is about 68%.
How to Prove The De Moivre – La Place Version of The Central Limit Theorem
Show that the probability that the sum of n draws at random with replacement from a zero-one box is exactly k given by the binomial formula
!
; , , where 1! !
k n knb k n p p q q p
k n k
Then using “Stirling’s Formula”1
2! 2n nn n e
show ; ,
2
k n kn np nq
b k n pn n k k n k
Letting and recalling that 1
1; , 1 1
2 1 1
x np x nq
x k np q p
x xb k n p
np nqx xnpq
np nq
How to Prove The De Moivre – La Place Version of The Central Limit Theorem -- continued
2
1log 1 1
2
x np x nqx x x
np nq npq
Which implies
2
1
21 1
xx np x nq
npqx xe
np nq
2 2
21 1 12 2 2
1 1 1Hence ; ,
2 2 2
k npxznpq npqb k n p e e e
npq npq npq
Use the series for the log to show that, for x npq
The limiting processes in these steps require some care. Both k and n must go to infinity together in a fixed relationship to each other, and we need to understand why values of x for which |x|>npq are unimportant.
Bibliography
1. Freedman, Pisani, & Purves, Statistics, 3rd Ed., W.W. Norton, New York, 1998
2. W. Feller, An Introduction to Probability Theory and Its Applications, Volume I, 2nd Ed., John Wiley & Sons, New York, London, Sydney, 1957
3. F. Mosteller, Fifty Challenging Problems in Probability with Solutions, Addison-Wesley, Palo Alto, 1965.
4. http://www-history.mcs.st-andrews.ac.uk/Biographies/De_Moivre.html
5. R Development Core Team, R: A language and environment for statistical computing, R Foundation for Statistical Computing, Vienna, Austria, 2006, <http://www.R-project.org>
