Why Meta-Analysis? Mata Hari, not Meta-Analysis Slide 15.1 1. When Narrative Literature Reviewers...

Why Meta-Analysis?

Mata Hari, notMeta-Analysis

Slide 15.1

1. When Narrative Literature Reviewers Have Drawn Conflicting Conclusions

2.When There are Many Studies

3.When It Is Desired to Synthesize Study Results Across Different Samples and Materials.

Meta-Analysis Purposes

Slide 15.2

1. Summarizing available data.

2. Explaining the variability among different studies.

Steps in Completing a Meta-Analysis

Slide 15.3A

Sampling Quantitative

Studies

Steps in Completing a Meta-Analysis

Code EssentialStudy Differences

in Studies and ComputeEffect Sizes

from Studies

Slide 15.3B

Sampling Quantitative

Studies

Steps in Completing a Meta-Analysis

Code EssentialStudy Differences

in Studies and ComputeEffect Sizes

from Studies

If fail-safe number < 5NL + 10, collect additional studies

Slide 15.3C

ComputeFail-Safe Number

Sampling Quantitative

Studies

Steps in Completing a Meta-Analysis

Code EssentialStudy Differences

in Studies and ComputeEffect Sizes

from Studies

If fail-safe number < 5NL + 10, collect additional studies

ComputeMean Effect

Size

Slide 15.3D

ComputeFail-Safe Number

Sampling Quantitative

Studies

Steps in Completing a Meta-Analysis

Code EssentialStudy Differences

in Studies and ComputeEffect Sizes

from Studies

If fail-safe number < 5NL + 10, collect additional studies

ComputeMean Effect

Size

ComputeDiffuse

Comparison

Slide 15.3E

ComputeFail-Safe Number

Sampling Quantitative

Studies

Steps in Completing a Meta-Analysis

Code EssentialStudy Differences

in Studies and ComputeEffect Sizes

from Studies

If fail-safe number < 5NL + 10, collect additional studies

ComputeMean Effect

Size

ComputeDiffuse

Comparison

Slide 15.3F

Compute (additional)Focused

Comparisons

ComputeFail-Safe Number

Sampling Quantitative

Studies

If diffuse comparison statistically significant

Steps in Completing a Meta-Analysis

Code EssentialStudy Differences

in Studies and ComputeEffect Sizes

from Studies

If fail-safe number < 5NL + 10, collect additional studies

ComputeMean Effect

Size

ComputeDiffuse

Comparison If diffuse comparison statistically significant

Conclude If diffuse comparison not statistically significant

Slide 15.3

Compute (additional)Focused

Comparisons

ComputeFail-Safe Number

Sampling Quantitative

Studies

Criteria for Including Studies

• Make sure the studies meet the assumptions underlying the use of meta-analysis:– Empirical studies only– Studies must include clear information about:

» final sample sizes» actual effect sizes—or there must be access to all the

coefficients necessary to compute the effect sizes» reliability for measured variables.

• The number of studies drawn from a single article or research report should not be too great. Usually no more than two or three studies from the same research article or report.

• Studies should be excluded if they are radically different from others.

Slide 15.4

Transforming a Fisher’s Z Back into a Correlations

• To transform a ZFisher back to an untransformed correlation, the following formula is used:

Slide 15.6A

1

12

2

FisherZ

FisherZ

e

er

The Diffuse Test

Slide 15.7A

jFisherZ

FisherZ

is the is the Fisher Z for the effect from study j,

is the mean Fisher Z score, andk is the number of effects analyzed.

Applying the data from the study, the following computation is revealed:

221 *3 FisherjFisherjk ZZn , where

n is the number of events in study j,

Focused Comparison 1:The Number of Variables in the Study

Slide 15.10A

3

2

j

j

Fisherj

n

ZZ j

Computing a “Fail-Safe” Number

Slide 15.14A

Lp

j Nz

ZX

2

2

Computing a “Fail-Safe” Number

Slide 15.14B

, where

• Zj is (Zrj * ) [the Fisher’s Z transformation of correlations (r) for each (j) study included in the meta-analysis multiplied by , where Nj is the sample size of each (j) study];

L

p

j Nz

ZX

2

2

3-N j

3-N j

Computing a “Fail-Safe” Number

Slide 15.14C

, where

• Zj is (Zrj * ) [the Fisher’s Z transformation of correlations (r) for each (j) study included in the meta-analysis multiplied by , where Nj is the sample size of each (j) study];• is the square of the z value the corresponds to the one- tailed probability level of the significance tests. Since p is commonly .05, the one-tailed z score including all but the last 5% of the area under the standard normal curve is 1.645

L

p

j Nz

ZX

2

2

3-N j

3-N j

2pz

Computing a “Fail-Safe” Number

Slide 15.14

, where

• Zj is (Zrj * ) [the Fisher’s Z transformation of correlations (r) for each (j) study included in the meta-analysis multiplied by , where Nj is the sample size of each (j) study];• is the square of the z value the corresponds to the one- tailed probability level of the significance tests. Since p is commonly .05, the one-tailed z score including all but the last 5% of the area under the standard normal curve is 1.645 • NL is the number of studies located for use in the meta- analysis.

L

p

j Nz

ZX

2

2

3-N j

3-N j

2pz