11

Sampling Sampling Distribution TheoryDistribution Theorych6ch6

22

Two independent R.V.s have the joint p.m.f. = the Two independent R.V.s have the joint p.m.f. = the product of individual p.m.f.s. product of individual p.m.f.s. Ex6.1-1: X1is the number of spots on a fair die. Ex6.1-1: X1is the number of spots on a fair die.

ff11(x(x11)=1/6, x)=1/6, x11=1,2,3,4,5,6. =1,2,3,4,5,6. XX22is the number of heads on 4 indep. Tosses of a fair coin. is the number of heads on 4 indep. Tosses of a fair coin.

If XIf X1 1 and Xand X2 2 are indep. are indep.

If XIf X1 1 and Xand X2 2 have the same p.m.f., their joint p.m.f. is have the same p.m.f., their joint p.m.f. is f(xf(x11)*f(x)*f(x22). ). This collection of XThis collection of X11and Xand X22is a is a random sample of size random sample of size

n=2 from f(x). n=2 from f(x).

P(X1=1,2 & X2=3,4)

33

Linear Functions of Indep. R.V.sLinear Functions of Indep. R.V.s Suppose a function Y=XSuppose a function Y=X11+X+X22, S, S11={1,2,3,4,5,6}, ={1,2,3,4,5,6},

SS22={0,1,2,3,4}. ={0,1,2,3,4}. Y will have the support S={1,2,…,9,10}. Y will have the support S={1,2,…,9,10}. The p.m.f. g(y) of Y is The p.m.f. g(y) of Y is

The mathematical expectation (or expected value) The mathematical expectation (or expected value) of a function Y=u(Xof a function Y=u(X11,X,X22) is) is

If XIf X11and Xand X2 2 are indep.are indep.

44

Example Example Ex6.1-2: XEx6.1-2: X1 1 and Xand X2 2 are two indep. R.V. from casting a die are two indep. R.V. from casting a die

twice. twice. E(XE(X11)=E(X)=E(X22)=3.5; Var(X)=3.5; Var(X11)=Var(X)=Var(X22)=35/12; E(X)=35/12; E(X11XX22)=E(X)=E(X11)E(X)E(X22)=12.25; )=12.25;

E[(XE[(X11-3.5)(X-3.5)(X22-3.5)]=E(X-3.5)]=E(X11-3.5)E(X-3.5)E(X22-3.5)=0. -3.5)=0.

Y=XY=X11+X+X2 2 →E(Y)= E(X→E(Y)= E(X11)+E(X)+E(X22)=7; )=7;

Var(Y)=E[(XVar(Y)=E[(X11+X+X22-7)-7)22]=Var(X]=Var(X11)+Var(X)+Var(X22)=35/6.)=35/6. The p.m.f. g(y) of Y with S={2,3,4,…,12} is The p.m.f. g(y) of Y with S={2,3,4,…,12} is

55

General Cases General Cases If XIf X11,…,X,…,Xnn are indep., then their joint p.d.f. is f are indep., then their joint p.d.f. is f11(x(x11) …f) …fnn(x(xnn). ).

The expected value of the product uThe expected value of the product u11(x(x11) …u) …unn(x(xnn) is the product of the ) is the product of the

expected values of uexpected values of u11(x(x11),…, u),…, unn(x(xnn). ).

If all these n distributions are the same, the collection of n If all these n distributions are the same, the collection of n indep. and identically distributed (indep. and identically distributed (iidiid) random variables, X) random variables, X11,,

…,X…,Xnn, is a , is a random sample of size n from that common random sample of size n from that common

distributiondistribution.. Ex6.1-3: XEx6.1-3: X11, X, X22, X, X33, are a random sample from a distribution , are a random sample from a distribution

with p.d.f. f(x)=ewith p.d.f. f(x)=e-x-x, 0<x<∞. , 0<x<∞. The joint p.d.f. is The joint p.d.f. is

P(0 < X1 < 1,2 < X2 < 4,3 < X3 < 7)

66

Distributions of Sums of Indep. R.V.sDistributions of Sums of Indep. R.V.s Distributions of the product of indep. R.V.s are Distributions of the product of indep. R.V.s are

straightforward. straightforward. However, distributions of the sum of indep. R.V.s However, distributions of the sum of indep. R.V.s

are fetching: are fetching: First, the joint p.m.f. or p.d.f. is a simple product. First, the joint p.m.f. or p.d.f. is a simple product. However, through summation, these R.V.s interfere with However, through summation, these R.V.s interfere with

each other. each other. Care must be taken to distinguish some sum value happens Care must be taken to distinguish some sum value happens

more frequently than the others. more frequently than the others.

Sampling distribution theory Sampling distribution theory is to derive the is to derive the distributions of the functions of R.V.s (random distributions of the functions of R.V.s (random variables). variables). The sample mean and variance are famous functions. The sample mean and variance are famous functions. The summation of R.V.s is another example. The summation of R.V.s is another example.

77

Example Example Ex6.2-1: XEx6.2-1: X1 1 and Xand X2 2 are two indep. R.V.s from casting a 4-sided are two indep. R.V.s from casting a 4-sided

die twice.die twice. The p.m.f. f(x)=, x=1,2,3,4. The p.m.f. f(x)=, x=1,2,3,4. The p.m.f. of Y=XThe p.m.f. of Y=X11+X+X2 2 with S={2,3,4,5,6,7,8} is g(y): with S={2,3,4,5,6,7,8} is g(y):

((convolution formulaconvolution formula)) g(2)=g(3)=

g(y)=

88

Theorems Theorems Thm6.2-1: XThm6.2-1: X11,…,X,…,Xn n are indep. and have the joint p.m.f. is are indep. and have the joint p.m.f. is

ff11(x(x11) …f) …fnn(x(xnn). Y=u(X). Y=u(X00,…,X,…,Xnn) have the p.m.f. g(y)) have the p.m.f. g(y) ThenThenif the summations exist. if the summations exist. For continuous type, integrals replace the summations.For continuous type, integrals replace the summations.

Thm6.2-2: XThm6.2-2: X11,…,Xn are indep. and their means exist,,…,Xn are indep. and their means exist, Then, Then,

Thm6.2-3: If XThm6.2-3: If X11,…,X,…,Xn n are indep. with means μare indep. with means μ11,…,μ,…,μn n and and variances σvariances σ11

22,…,σ,…,σnn22, then Y=a, then Y=a11XX11+…+a+…+annXXnn, where a, where aii’s are ’s are

real constants, have the mean and variance:real constants, have the mean and variance:

Ex6.2-3: XEx6.2-3: X11 & X & X22 are indep. with μ are indep. with μ11= -4, μ= -4, μ22=3 and σ=3 and σ1122=4, =4,

σσ2222=9. =9.

Y=3XY=3X11-2X-2X2 2 has has

99

Moment-generating FunctionsMoment-generating Functions Ex6.2-4: XEx6.2-4: X11,…,X,…,Xnn are a random sample of size n from a are a random sample of size n from a

distribution with mean μand variance σdistribution with mean μand variance σ22; then ; then The sample mean:The sample mean:

Thm6.2-4: If XThm6.2-4: If X11,…,X,…,Xn n are indep. R.V.s with moment-are indep. R.V.s with moment-

generating functions , i=1..n, then Y=agenerating functions , i=1..n, then Y=a11XX11+…+a+…+annXXnn, has the , has the

moment-generatingmoment-generating

Cly6.2-1: If XCly6.2-1: If X11,…,X,…,Xnn are indep. R.V.s with M(t), are indep. R.V.s with M(t), then Y=Xthen Y=X11+…+X+…+Xnn has M has MYY(t)=[M(t)](t)=[M(t)]nn..

has has

MY(t)

1010

Examples Examples Ex6.2-5: XEx6.2-5: X11,…,X,…,Xnn are the outcomes on n Bernoulli are the outcomes on n Bernoulli

trials.trials. The moment-generating function of XThe moment-generating function of X ii, i=1..n, is M(t)=q+pe, i=1..n, is M(t)=q+pett. .

Then Y=XThen Y=X11+…+X+…+Xnn has M has MYY(t)=[q+pe(t)=[q+pett]]nn, which is b(n,p). , which is b(n,p).

Ex6.2-6: XEx6.2-6: X11,X,X22,X,X3 3 are the outcomes of a random sample of are the outcomes of a random sample of

size n=3 from the exponential distribution with mean θand size n=3 from the exponential distribution with mean θand M(t)=1/(1-θt), t<1/θ.M(t)=1/(1-θt), t<1/θ. Then Y=XThen Y=X11+X+X22+X+X3 3 has Mhas MYY(t)=[1/(1-θt)](t)=[1/(1-θt)]33=(1-θt)=(1-θt)-3-3,,

which is a gamma distribution with α=3, and θ.which is a gamma distribution with α=3, and θ. has has

a gamma distribution with α=3, and θ/3. a gamma distribution with α=3, and θ/3.

1111

Statistics on Normal Statistics on Normal

DistributionsDistributions Thm6.3-1: XThm6.3-1: X11,…,X,…,Xn n are the outcomes on a random sample of are the outcomes on a random sample of size n from the normal distribution N(μ,σsize n from the normal distribution N(μ,σ22). ). The distribution of the sample mean is N(μ,σThe distribution of the sample mean is N(μ,σ22/n)./n). Pf:Pf:

Thm6.3-2: XThm6.3-2: X11,…,X,…,Xn n are independent and have χare independent and have χ22(r(r11),…, χ),…, χ22(r(rnn) ) distributions, respectively ; Then, Y=Xdistributions, respectively ; Then, Y=X11+…+X+…+Xn n is χis χ22(r(r11+…+r+…+rnn). ).

Pf: Pf:

Thm6.3-3: ZThm6.3-3: Z11,…,Z,…,Znn are independent and all have N(0,1); are independent and all have N(0,1); Then, W=ZThen, W=Z11

22+…+Z+…+Znn22is χis χ22(n). (n).

Pf: Pf: Thm4.4-2: If X is N(μ,σThm4.4-2: If X is N(μ,σ22), then V=[(X-μ)/σ]), then V=[(X-μ)/σ]22=Z=Z2 2 is χis χ22(1).(1).

Thm6.3-2: Y=XThm6.3-2: Y=X11+…+X+…+Xn n is χis χ22(r(r11+…+r+…+rnn)= χ)= χ22(n) in this case. (n) in this case.

MX(t)

MY(t)=

1212

Example Example Fig.6.3-1: p.d.f.s of means of samples from N(50,16).Fig.6.3-1: p.d.f.s of means of samples from N(50,16).

is N(50, 16/n).is N(50, 16/n).

1313

Theoretical Mean and Sample MeanTheoretical Mean and Sample Mean Cly6.3-1: ZCly6.3-1: Z11,…,Z,…,Zn n are independent and have N(μare independent and have N(μii,σ,σii

22),i=1..n; ),i=1..n;

Then, W=[(ZThen, W=[(Z11-μ-μ11)/σ)/σii22]]22+…+[(Z+…+[(Znn-μ-μnn)/σ)/σnn

22]]2 2 is χis χ22(n). (n).

Thm6.3-4: XThm6.3-4: X11,…,X,…,Xn n are the outcomes on a random sample are the outcomes on a random sample

of size n from the normal distribution N(μ,σof size n from the normal distribution N(μ,σ22). Then,). Then, The sample mean & variance The sample mean & variance

are indep. are indep.

is is χχ22(n-1)(n-1)Pf: Pf: (a) omitted; (b):(a) omitted; (b):

As theoretical mean is replaced by the sample mean, one degree of freedom As theoretical mean is replaced by the sample mean, one degree of freedom is lost! is lost!

MV(t)

1414

Linear Combinations of N(μ,σLinear Combinations of N(μ,σ22)) Ex6.3-2: XEx6.3-2: X11,X,X22,X,X33,X,X4 4 are a random sample of size 4 from the are a random sample of size 4 from the

normal distribution N(76.4,383). Then normal distribution N(76.4,383). Then

P(0.711P(0.711W W 7.779)=0.9-0.05=0.85, P(0.352 7.779)=0.9-0.05=0.85, P(0.352 V V 6.251)=0.9-0.05=0.856.251)=0.9-0.05=0.85

Thm6.3-5: If XThm6.3-5: If X11,…,X,…,Xn n are n mutually indep. normal variables are n mutually indep. normal variables

with means μwith means μ11,…,μ,…,μn n & variances σ& variances σ1122,…,σ,…,σnn

22, then the linear , then the linear

functionfunction has the normal distribution has the normal distribution Pf: By moment-generating function, … Pf: By moment-generating function, …

Ex6.3-3: XEx6.3-3: X11:N(693.2,22820) and X:N(693.2,22820) and X22:N(631.7,19205) are indep. :N(631.7,19205) are indep.

Find P(XFind P(X11>X>X22) ) Y=XY=X11-X-X2 2 is N(61.5,42025). is N(61.5,42025).

P(XP(X11>X>X22)=P(Y>0)=)=P(Y>0)=

1515

Central Limit Theorem Central Limit Theorem Ex6.2-4: XEx6.2-4: X11,…,X,…,Xn n are a random sample of size n from a are a random sample of size n from a

distribution with mean μand variance σdistribution with mean μand variance σ22; then ; then

The sample mean: The sample mean:

Thm6.4-1: (Central Limit Theorem) If is the mean of a Thm6.4-1: (Central Limit Theorem) If is the mean of a random sample Xrandom sample X11,…,X,…,Xn n of size n from some distribution with of size n from some distribution with

a finite mean μand a finite positive variance σa finite mean μand a finite positive variance σ22, then the , then the distribution ofdistribution of

is N(0, 1) in the limit as n →∞. Even if Xis N(0, 1) in the limit as n →∞. Even if X ii is not N(μ,σ is not N(μ,σ22).).

W= if n is large

W

1616

More ExamplesMore Examples Ex6.4-1: Let denote the mean of a random sample of size Ex6.4-1: Let denote the mean of a random sample of size

n=15 from the distribution whose p.d.f. is f(x)=3xn=15 from the distribution whose p.d.f. is f(x)=3x22/2, -1<x<1./2, -1<x<1. μ=0, σμ=0, σ22=3/5.=3/5.

Ex6.4-2: Let XEx6.4-2: Let X11,…,X,…,X20 20 be a random sample of size 20 from be a random sample of size 20 from

the uniform distribution U(0,1). the uniform distribution U(0,1). μ=½, σμ=½, σ22=1/12; Y=X=1/12; Y=X11+…+X+…+X2020..

Ex6.4-3: Let denote the mean of a random sample of Ex6.4-3: Let denote the mean of a random sample of size n=25 from the distribution whose p.d.f. is f(x)=xsize n=25 from the distribution whose p.d.f. is f(x)=x33/4, /4, 0<x<20<x<2 μ=1.6, σμ=1.6, σ22=8/75.=8/75.

1717

How large of size n is sufficient?How large of size n is sufficient? If n=25, 30 or larger, the approximation is generally goodIf n=25, 30 or larger, the approximation is generally good. .

If the original distribution is symmetric, unimodal and of continuous If the original distribution is symmetric, unimodal and of continuous type, n can be as small as 4 or 5. type, n can be as small as 4 or 5.

If the original is like normal, n can be lowered to 2 or 3. If the original is like normal, n can be lowered to 2 or 3. If it is exactly normal, n=1 or more is just good. If it is exactly normal, n=1 or more is just good.

However, if the original is highly skew, n must be quite large. However, if the original is highly skew, n must be quite large.

Ex6.4-4: Let XEx6.4-4: Let X11,…,X,…,X4 4 be a random sample of size 4 from the be a random sample of size 4 from the

uniform distribution U(0,1) with p.d.f. f(x)=1, 0<x<1uniform distribution U(0,1) with p.d.f. f(x)=1, 0<x<1. . μ=½, σμ=½, σ22=1/12; Y=X=1/12; Y=X11+X+X22..

Y=XY=X11+…+X+…+X44..

1818

Graphic Illustration Graphic Illustration Fig.6.4-1: Sum of n U(0, 1) R.V.s Fig.6.4-1: Sum of n U(0, 1) R.V.s

N( n(1/2), n(1/12) ) p.d.f. N( n(1/2), n(1/12) ) p.d.f.

1919

Skew DistributionsSkew Distributions Suppose f(x) and F(x) are the p.d.f. and distribution function Suppose f(x) and F(x) are the p.d.f. and distribution function

of a random variable X with mean μ and variance σof a random variable X with mean μ and variance σ22. .

Ex6.4-5: Let XEx6.4-5: Let X11,…,X,…,Xn n be a random sample of size n from a be a random sample of size n from a

chi-square distribution χchi-square distribution χ22(1). (1). Y=XY=X11+…+X+…+Xnn is χ is χ22(n), E(Y)=n, Var(Y)=2n. (n), E(Y)=n, Var(Y)=2n.

n=20 or 100 n=20 or 100 →N( , ). →N( , ).

2020

Graphic Illustration Graphic Illustration Fig.6.4-2: The p.d.f.s of sums of χFig.6.4-2: The p.d.f.s of sums of χ22(1), transformed so that (1), transformed so that

their mean is equal to zero and variance is equal to one, their mean is equal to zero and variance is equal to one, becomes closer to the N(0, 1) as the number of degrees of becomes closer to the N(0, 1) as the number of degrees of freedom increases. freedom increases.

2121

Simulation of R.V. X with f(x) & F(x)Simulation of R.V. X with f(x) & F(x) Random number generator will produce values y’s for U(0,1). Random number generator will produce values y’s for U(0,1).

Since F(x)=U(0,1)=Y, x=FSince F(x)=U(0,1)=Y, x=F-1-1(y) is an observed or simulated value of X. (y) is an observed or simulated value of X.

Ex.6.4-6: Let XEx.6.4-6: Let X11,…,X,…,Xn n be a random sample of size n from the be a random sample of size n from the

distribution with f(x), F(x), mean μand variance σdistribution with f(x), F(x), mean μand variance σ22.. 1000 random samples are simulated to compute the values of W.1000 random samples are simulated to compute the values of W. A histogram of these values are grouped into 21 classes of equal A histogram of these values are grouped into 21 classes of equal

width.width.f(x)=(x+1)/2, F(x)=(x+1)f(x)=(x+1)/2, F(x)=(x+1)22/4, f(x)=3x/4, f(x)=3x22/2, F(x)=(x/2, F(x)=(x33+1)/2, +1)/2,

-1<x<1; μ=1/3, σ-1<x<1; μ=1/3, σ22=2/9. -1<x<1; μ=0, σ=2/9. -1<x<1; μ=0, σ22=3/5.=3/5.

N(0,1)N(0,1)

2222

Approximation of Discrete Approximation of Discrete

DistributionsDistributions Let XLet X11,…,X,…,Xn n be a random sample from a be a random sample from a Bernoulli Bernoulli

distribution with μ=p and σdistribution with μ=p and σ22=npq, 0<p<1.=npq, 0<p<1. Thus, Y=XThus, Y=X11+…+X+…+Xn n is is binomial binomial b(n,p). →N(np,npq) as n →∞. b(n,p). →N(np,npq) as n →∞.

Rule: n is “sufficiently large” if npRule: n is “sufficiently large” if np5 and nq5 and nq5. 5. If p deviates from 0.5 (skew!!), n need to be larger. If p deviates from 0.5 (skew!!), n need to be larger.

Ex.6.5-1: Y, b(10,1/2), Ex.6.5-1: Y, b(10,1/2), can be approximated by can be approximated by N(5,2.5). N(5,2.5).

Ex.6.5-2: Y, b(18,1/6), Ex.6.5-2: Y, b(18,1/6), can be hardly approx. by can be hardly approx. by N(3,2.5), 3<5. ∵N(3,2.5), 3<5. ∵

2323

Another ExampleAnother Example Ex6.5-4: Y is b(36,1/2).Ex6.5-4: Y is b(36,1/2).

Correct Probability: Correct Probability:

Approximation of Binomial Distribution b(n,p): Approximation of Binomial Distribution b(n,p):

Good approx.!

2424

Approximation of Poisson Approximation of Poisson

DistributionDistribution Approximation of a Poisson Distribution Y with mean λ: Approximation of a Poisson Distribution Y with mean λ:

Ex6.5-5: X that has a Poisson distribution with mean 20 can Ex6.5-5: X that has a Poisson distribution with mean 20 can be seen as the sum Y of the observations of a random be seen as the sum Y of the observations of a random sample of size 20 from a Poisson distribution with mean 1.sample of size 20 from a Poisson distribution with mean 1.

Correct Probability: Correct Probability:

Fig.6.5-3: Normal approx. of the Poisson Probability Histogram

2525

Student’s T DistributionStudent’s T Distribution Gossett, William Sealy published “t-test” in Biometrika1908 Gossett, William Sealy published “t-test” in Biometrika1908

to measure the confidence interval, the deviation of “small to measure the confidence interval, the deviation of “small samples” from the “real”. samples” from the “real”. Suppose the underlying distribution is normal with unknown σSuppose the underlying distribution is normal with unknown σ22. .

Fig.6.6-1(a): The T p.d.f. Fig.6.6-1(a): The T p.d.f. becomes closer to the becomes closer to the N(0, 1) p.d.f. as the numbeN(0, 1) p.d.f. as the number r of degrees of freedom incrof degrees of freedom increases.eases.

ttαα(r) (r) is the 100(1-α) percentile, or the upper 100αpercent point. [Table VI, p.658] is the 100(1-α) percentile, or the upper 100αpercent point. [Table VI, p.658]

f(t)=

Only depends on r!

2626

Examples Examples Ex6.6-1: Suppose T has a t distribution with r=7. Ex6.6-1: Suppose T has a t distribution with r=7.

From Table VI on p.658, From Table VI on p.658,

Ex6.6-2: Suppose T has a t distribution with r=14.Ex6.6-2: Suppose T has a t distribution with r=14. Find a constant c, such that P(|T|<c)=0.9 Find a constant c, such that P(|T|<c)=0.9 From Table VI on p.658, From Table VI on p.658,

2727

F Distribution: F(rF Distribution: F(r11, r, r22)) From two indep. random samples of size nFrom two indep. random samples of size n1 1 & n& n2 2 from from

N(μN(μ11,σ,σ1122) & N(μ) & N(μ22,σ,σ22

22),some comparisons can be performed, ),some comparisons can be performed,

such as:such as:

FFαα(r(r11,r,r22))is the is the upper upper 100α percent point. [Table VII, p.659~663]100α percent point. [Table VII, p.659~663]

Ex6.6-4: Suppose F has a F(4,9) distribution.Ex6.6-4: Suppose F has a F(4,9) distribution. Find constants c & d, such that P(F Find constants c & d, such that P(F c)=0.01, P(F c)=0.01, P(F d)=0.05d)=0.05

If F is F(6,9):

c=F0.99(4,9)