1G89.2228 Lect 8a

G89.2228 Lecture 8a

• Power for specific tests: Noncentral sampling distributions

• Illustration of Power/Precision Software

• A priori and post hoc power analyses• Power for tests of proportions and

RxC tables• Power analyses for fun and profit

2G89.2228 Lect 8a

Power for specific tests: Noncentral sampling

distributions• The z test is not only easy to study for

calculating power, but it also is useful as a general approximation for a variety of applications. Howell uses this approximation for all the tests for which he discusses power.

• t distributions have a shape like z distributions, but they have somewhat fatter tails. Exact power calculations are possible when this explicit distribution is considered. Like z tests, the noncentral t distribution is shifted.

• Noncentral distributions for statistics such as 2 and F are not simply shifted with the same shape. Special tables or software are needed to study these distributions.

3G89.2228 Lect 8a

Effect Sizes

• Specify the nature of the H1• For t test

Cohen's Small, Med, Large=(.2,.5,.8)

• For Pearson's Chi Square

Cohen's Small, Med, Large=(.1,.3,.5)

12 d

0

201

H

HH

PPP

w

4G89.2228 Lect 8a

Power for t tests• The calculation of power for the two

sample t test follows directly from the analysis of the z test for the one sample test. The effect size is d=(µ1-µ2)/, where µ1 is the mean in population 1, µ2 is the mean in population 2, and is the assumed-common within population standard deviation.

• The noncentral distribution of the t distribution takes into account the sample size in terms of both its effect on the standard error of the mean difference, and its effect on the precision of the estimate of the variance.

• The analysis is simple when group n’s are the same. Different n’s require use of the harmonic mean of the ni’s.

5G89.2228 Lect 8a

Kinds of power calculations

• A priori calculations: planning a study before data are collected.– Speculate about effect sizes, and

determine sample size that yields good power

– If feasible, set power goal to .9 or .95.

– Example:To replicate H-K&N’s effect on expressed antagonism, we note the rating difference of (5-4)=1 is compared to a within group sd of about 1.6, so d=1/1.6=.63. Howell’s method says 90% power comes from a value of =3.25. 2d=53.22 tells us that about 53 subjects per group are needed to detect this large effect in a replication.

– Use of Noncentral t suggests we need 55 subjects per group rather than 53.

6G89.2228 Lect 8a

Another kind of power calculation: Post hoc

• If a result is not significant, one can ask what the power was to detect small, medium or large effects.

• In this case, one has data to consider the magnitude of the within population variance. There is also data on the sample means, but these should not automatically be used to consider the effect sizes. Remember that they are subject to sampling fluctuations.

• Example:H-K&N report in study 3 that the antagonism effect is just significant for Black antagonism (comparing 5.04 to 4.00) but not for White antagonism (comparing 4.16 to 4.71). If the effects had the same magnitude and d=.63, then the sample size of 23 per group would give . Howell’s normal approximation gives about 57% power, but that may be slightly inflated. Cohen’s table gives about 55% power.

14.22/2363.n/2d

7G89.2228 Lect 8a

Power for tests of proportions

• In principle, the analysis of power for a test of proportions follows directly from the z test we considered before.

• A complication arises in that the magnitude of the within population variance changes with the size of the proportions that are being considered.

• E.g., a difference of .10 in proportions may be large when the comparison is .15 vs .05, but the same effect may be small when comparing .45 to .55

• From Fleiss (1981) the n needed for 80% power (two tailed test with =.05) for the former large effect (.05 vs .15) is 160 per group, but 411 for the smaller effect (.45 vs .55).

8G89.2228 Lect 8a

Power for RxC Contingency Table Problem

• Suppose we were interested in whether the distribution of attitudes of males and females differed regarding some policy. Attitudes could be Favor, NotFavor, Undecided

• Suppose we expect proportions for males to be .5, .25, .25, but for females we expect .3, .25, .45.

• Under HO the average is .4, .25, .35• Calculation of Power illustrated

9G89.2228 Lect 8a

Illustration of 2x3 Crosstab

Favor NotFavor UndecidedMales 0.5 0.25 0.25 1Females 0.3 0.25 0.45 1

H1 Probs for equal sample sizeFavor NotFavor Undecided

Males 0.25 0.125 0.125 0.5Females 0.15 0.125 0.225 0.5

0.4 0.25 0.35 1

H0 Probs for equal sample sizeFavor NotFavor Undecided

Males 0.2 0.125 0.175 0.5Females 0.2 0.125 0.175 0.5

0.4 0.25 0.35 1

(o-e)^2/E0.0125 0 0.0142860.0125 0 0.014286

0.053571 SUM0.231455 w

10G89.2228 Lect 8a

A priori sample size planning

• Needed for all grant applications• Alternative approaches:

– Statistical power

– Width of CI

• Estimates often done in the face of ignorance– Cohen's Large, Medium and Small

effects become useful

– Realize that they should not be taken uncritically

• Studies often have multiple goals with different power