Power Calculator Show To

7/30/2019 Power Calculator Show To

1/10

UCDHSC Center for Nursing Research Updated 5/18/06

Page 1 of 10

How to Use the Power Calculators

Disclaimer: This tutorial shows you how to use an online power calculator created by

Dr. Russ Lenth at the University of Iowa for two purposes: (a) to calculate the targetsample size for a planned study using a generic Cohens d effect size, and (b) to

calculate post hoc power after a study has been completed, to determine what thechances were of finding a significant difference if one was really there. It should be noted

that the creator of these calculatorsspecifically disagrees with the use of power analysisfor both of these purposes (see his note on the web page about two very wrong things

that people try to do with his software). Please note that it is always advisable to do a

pilot study and collect data on the actualbetween-group and within-group variability forthe specific measures that you are interested in. The Cohens deffect size essentially

short-cuts this process by using a ratio of the two quantities and labeling that ratio as

small, medium, or large. However, it is sometimes useful to take this shortcut.Therefore, with apologies to the author of the website, this tutorial is an unauthorized

commentary on how to use his very helpful software to accomplish pragmatic goals.

How to Use the Power Calculators

First, access thepower calculators web page.

Next, choose the statistical test that you are planning to perform from this listby clicking

on one of the entries, and then click on the run selection button.


2/10


Page 2 of 10

Your choices are:

Confidence Interval for 1 proportion Test of 1 proportion Test comparing 2 proportions (Fishers exact test)

Confidence Interval for 1 mean One-sample ttest (or dependent t-test) Two-sample ttest (independent t-test) ANOVA (F-test)

Each of these options is described separately, below.

Printing Your ResultsYou will see the output of your power calculation on-screen (the calculators move the

sliding bar for power or sample size immediately as you are making changes, so

there is no run button to push in order to run the analysis). When you have a result that

you are satisfied with, and you want to save it for future reference, do this:

1. Hold down the Control (Ctrl) key on your keyboard.2. While still holding down Control, hit the print screen button (found at the top of

your keyboard, to the right of the function keys but to the left of the number keypad).

Nothing will appear to happen when you hit this key, but whatever is visible on your

monitor will be copied into the clipboard.3. Open a new Word document.

4. Select paste from the edit menu, or use Ctrl-V to paste. The picture of your monitor

screen will appear inside the Word document.5. Save the Word document that shows the image of your power analysis results.

Citing the Power CalculatorsDr. Lenth requests that you cite his website if you use his calculators for any published

work or grant proposals. The requested citation is:

Lenth, R. V. (2006). Java Applets for Power and Sample Size [Computer software].Retrieved month day, year, from http://www.stat.uiowa.edu/~rlenth/Power.

Please do notcite the CNR website as your source for these power calculators, becausewe didnt develop them. This wonderful tool is provided by Dr. Russ Lenth at the

University of Iowa, and we are just providing a link to them.


3/10


Page 3 of 10

If you are reporting on the prevalence of something in a population (e.g., the % of

parolees who recidivate after release on parole), then you should use the confidence

interval for one proportion. (x% plus or minusz% of criminals recidivate).

1) select the confidence level you want (e.g., 0.95 for 95% confidence)2) type in the number of people in the population that your sample came from [e.g.,

all criminals. This is largeN, as opposed to small n, the number of people in your

actual sample]. If you cant quantify exactly how many people are in the largerpopulation that you want to generalize to, you can un-check this check box to

estimate results for an infinite population.

3) type in the proportion that you want a confidence interval around (e.g., if youfound 20% recidivism in your sample, or if youre still just planning the study andthats what you expectto find, type in 0.20 here).

4) use the scroll bars:a. if you are planning a study, select the margin of error that you are willing

to live with, and look at the bottom scroll bar to find out what sample sizen you will need in order to get there.

b. if you have already completed your study and want to know whatconfidence interval you can report, use the bottom scroll bar to select youractual sample size n, and look at the top scroll bar to find out what your

margin of error is.


4/10


Page 4 of 10

If you are testing whether a proportion is equal to some specific value (e.g., you

found that 20% of people in your sex offenders group were abused as children, you

know that only 5% of people in the general population were abused as children, and youwant to know if this difference is statistically significant or might just be due to chance),

then you should use the test of one proportion.

1) Use the top scroll bar to indicate what thepopulation value is for the proportionyou want to test (in the example above, 5% or .05).

2) Use the next scroll bar to indicate what the obtained value in your sample is forthe proportion (in the example above, 20% or .20).

3) Generally, the three drop-down menus can be left at their default settings (the nullhypothesis is that the sample proportion equals the population proportion; the

alpha level is .05; and the method to be used is based on the standard normalcurve).

4) Use the other two scroll bars to get your results:a. If you are planning a study, select the power level you want (usually at

least .80), and see what n is needed for your sample.

b. If you have already completed the study, select the actual sample size nthat you have, and see what level of power you have to avoid making a

Type II error.


5/10


Page 5 of 10

If you are testing whethertwo groups have different rates of something (e.g., whether

teenagers with poor social skills have a higher suicide rate than teenagers with good

social skills), then you should use the test of equality of two proportions (also known asa Fishers Exact Test).

1) Use the first two scroll bars to put in the (actual or expected) proportions for thetwo groups. Youreffect size is determined by how far apart the proportions are.

2) Generally you can leave these drop-down menus on their default settings: the nullhypothesis is that the two proportions are equal, and the alpha level equals .05.

3) Use the other two scroll bars to get your results:a. If you are planning a study, select the power you want (usually at least

.80), and see how large an n you need in each group for your study.b. If you have already completed your study, select the actual n in each group

(uncheck the equal ns checkbox if there are different numbers in the two

groups), and find out how much power you had to avoid a Type II error.


6/10


Page 6 of 10

If you are reporting on the average value of something in a population (e.g., the

average IQ score among people with a first-degree relative who has bipolar disorder isx,

plus or minuszIQ points), you should use the confidence interval for one mean.

1) select the confidence level you want (e.g., 0.95 for 95% confidence)2) type in the number of people in the population that your sample came from [e.g.,

all criminals. This is largeN, as opposed to small n, the number of people in your

actual sample]. If you cant quantify exactly how many people are in the largerpopulation that you want to generalize to, you can un-check this check box to

estimate results for an infinite population.

3) Use the top scroll barto indicate the standard deviation of your sample (if youdont know, take a guessif your average is larger, your standard deviation maybe larger as well).

4) Use the other two scroll bars to get your results:a. if you are planning a study, select the margin of error that you are willing

to live with, and look at the bottom scroll bar to find out what sample sizen you will need in order to get there.

b. if you have already completed your study and want to know whatconfidence interval you can report, use the bottom scroll bar to select youractual sample size n, and look at the top scroll bar to find out what your

margin of error is.


7/10


Page 7 of 10

If you are testing for the differences between average pre- and post-test scores for asingle group of people (e.g., average depression score before vs. after treatment), or if

you are testing differences between two groups where the members are connected witheach other (e.g., scores for husbands vs. wives), you should use the one-sample t test.

1) youralpha level (in the drop-down menu) will usually be set at .05.2) Use the default for a two-tailed test if you arent sure which of the two groups will

have a higher mean than the other, or if you have a specific theory, you can un-

check this box to get a one-tailed test.3) To enter an effect size, use the top two scroll bars. The Cohens deffect size is

defined as the true difference between means divided by the pooled standard

deviationso, if you leave the standard deviation (sigma) set at 1, the Cohens

deffect size will be equal to the true difference between the means. Leave the topscroll bar alone, and use the second scroll bar to select a Cohens deffect size.

a. If you think you will find asmall difference between groups, use .5b. If you think you will find a moderate difference between groups, use .75c. If you think you will find a large difference between groups, use 1.00d. If you think you will find a very large difference between groups, use 1.25

4) Use the other two scroll bars to:a. Enter Power (usually at least .80), and solve for sample size n, or b. Enter your actual sample n, and find out how much Power your study had

to avoid a Type II error.


8/10


Page 8 of 10

If you are testing for differences in scores on some measure between twoindependent groups of people (i.e., the groups are not correlated with each other), you

should use the two-sample t test.

1) youralpha level (in the drop-down menu) will usually be set at .05.2) Use the default for a two-tailed test if you arent sure which of the two groups will

have a higher mean than the other, or if you have a specific theory, you can un-check this box to get a one-tailed test.

3) To enter an effect size, leave the two scroll bars marked sigma set at 1 (alsoleave the equal sigmas checkbox checked), and use the true difference of

means scroll bar to select a Cohens deffect size.

a. If you think you will find asmall difference between groups, use .5b. If you think you will find a moderate difference between groups, use .75c. If you think you will find a large difference between groups, use 1.00d. If you think you will find a very large difference between groups, use 1.25

4) Use the other scroll bars to:a. Enter Power (usually at least .80), and solve for the sample size n that will

be required in each group to get the desired level of power, or b. Enter your actual sample n (which can be different for the two groups),

and find out how much Power your study had to avoid a Type II error.


9/10


Page 9 of 10

If you are testing fordifferences between three or more groups, you should use the

Balanced ANOVA (F-test)power calculator. TheF-test selection takes you through a

preliminary screen before you get to the actual power calculator:

If you are doing a basicF-test (one-way ANOVA), just click the F tests button to go on(you can change the other values later). If you are doing a more complicated test (like a

two-way ANOVA, discussed in the advanced statistics class), select the appropriate testfrom the drop-down menu before clicking the F tests button to go on.

Note that you can use this same screen to examine two-way designs (by specifyingadditional variables in the model line), or to examine ANCOVA designs (by specifying

covariates in the random factors line).

For MANOVA designs, you can test the power of the test on one individual dependent

variable, but the power of the multivariate test will be influenced by the degree of

multicollinearity between the dependent variables, and the direction of the effect is not

always the same: Power for the multivariate test is highest when the pooled within-cellcorrelation among two DVs is high and negative. The multivariate test has much less

power when the correlation is positive, zero, or moderately negative. An interesting thing

happens, however, when one of the two DVs is affected by the treatment and the other isnot. The higher the absolute value of the correlation between the two DVs, the greater

the power of the multivariate test.Tabachnik & Fidell,Multivariate Statistics, 4th ed.

(2001), p. 329. Therefore, additional consideration is needed for MANOVA designs.


10/10


Page 10 of 10

This screen shows the power analysis setup for a basicF-test (one-way ANOVA):

1) Select the number of groups you are comparing, using the levels[treatment]scroll bar. The default is two groups; but you can select three or more. Leave the

radio button set to fixed effects.

2) Select an alpha level for your test (usually .05). Remember that if you areperforming multiple tests (e.g., if you are testing for differences between groupson more than one variable), you should use a more conservative alpha level (such

as .01) due to the Bonferroni correction.

3) To select an effect size, leave the SD[Within] (which represents random error)set at 1, and move the SD[treatment] (which reflects the average difference

between group means) to a level that represents your predicted effect size. When

the SD[Within] is 1, then the SD[treatment] will give you a Cohens d.a. If you think you will find asmall difference between groups, use .5b. If you think you will find a moderate difference between groups, use .75c. If you think you will find a large difference between groups, use 1.00d. If you think you will find a very large difference between groups, use 1.25

4) Use the other scroll bars to:a. Enter Power (usually at least .80), and solve for the sample size n that will

be required in each group to get the desired level of power, or b. Enter your actual sample n (which can be different for the two groups),

and find out how much Power your study had to avoid a Type II error.

Documents

Power Calculator Show To