1)Test the effects of IV on DV

2)Protects against threats to internal validity

Internal Validity – Control through Experimental Design

Chapter 10 – Lecture 10

Causation

Highest Constraint

Comparisons btw grps

Random sampling

Random assignment

Experimental Design

Infer Causality

1)One or more hypothesis

2)Includes at least 2 “levels” of IV

3)Random assignment

4)Procedures for testing hypothesis

5)Controls for major threats to internal validity

Experimental Design(5 characteristics)

Develop the problem statementDefine IV & DVDevelop research hypothesisIdentify a population of interestRandom sampling & Random

assignmentSpecify procedures (methods)Anticipate threats to validityCreate controlsSpecify Statistical tests•Ethical considerations

Experimental Design

Clear Experimental Design…

1.between groups variance (systematic)

Experimental Design2 sources of variance

2. Within groups variance (nonsystematic) (error variance)

drugno drug

Remember…Sampling error

Significant differences…variability btw means is larger than expected on the basis of sampling error alone (due to chance alone)

Variance Need it!Without it…

No go

Between Group Within GroupExperimental Variance

(Due to your treatment)+

Extraneous Variance(confounds etc.)

VARIANCE

Error Variance(not due to treatment – chance)

CON TX Subs

“Partitioning of the variance”

between groups varianceWithin groups variance

Variance: Important for the statistical analysis

F =

Systematic effects + error varianceerror variance

F =

1.00F = No differences btw groups

Variance

Your experiment should be designed to

• Maximize experimental variance

•Control extraneous variance

•Minimize error variance

Maximize “Experimental” Variance

• At least 2 levels of IV (IVs really vary?)

•Manipulation check: make sure the levels (exp. conditions) differ each other

Ex: anxiety levels (low anxiety/hi anxiety) performance on math task

anxiety scale

Control “Extraneous” Variance

1. Ex. & Con grps are similar to begin with

2. Within subjects design (carryover effects??)

3. If need be, limit population of interest (o vs o )

4. Make the extraneous variable an IV (age, sex, socioeconomic) = factorial design

M F

Lo Anxiety

Hi Anxiety

M-low

M-hi

F-low

F-hi

Factorial design(2 IV’s)

YOUR Proposals

1. Ex Post Facto2. Single-group, posttest only 3. Single-group pretest-posttest4. Pretest-Posttest natural control

group

Group A Naturally Occurring Event Measurement

1. Ex Post Facto – “after the fact”

Control through Design – Don’ts

No manipulation

Control through Design – Don’ts

Single group posttest only

Single group Pretest-posttest

Group A TX Posttest

Pretest Group A TX Posttest

Compare

Control through Design – Don’ts

Pretest-Posttest Naturalistic Control Group

Group A Pretest TX Posttest

Group B Pretest no TX Posttest

Compare

NaturalOccurring

• Manipulate IV • Control Group• Randomization

Control through Design – Do’s – Experimental Design

Testing One IV4 Basic Designs

1. Randomized Posttest only, Control Group2. Randomized Pretest-Posttest, Control Group3. Multilevel Completely Randomized Between

Groups4. Solomon’s Four- Group

Randomized Posttest Only – Control Group(most basic experimental design)

R Group A TX Posttest (Ex)

R Group B no TX Posttest (Con)

Compare

Randomized, Pretest-Posttest, Control Group Design

R Group A Pretest TX Posttest (Ex)

R Group B Pretest no TX Posttest (Con)

Compare

Multilevel, Completely Randomized Between Subjects Design (more than 2 levels of IV)

R Group A Pretest TX1 Posttest

R Group B Pretest TX 2 Posttest

R Group C Pretest TX3 Posttest R Group D Pretest TX4 Posttest

Compare

Solomon’s Four Group Design(extension Multilevel Btw Subs)

R Group A Pretest TX Posttest

R Group B Pretest ---- Posttest

R Group C -------- TX Posttest R Group D -------- ---- Posttest

Compare

Powerful Design!

What stats do you use to analyze experimental designs?

Depends the level of measurement

Test difference between groups

Nominal data chi square (frequency/categorical)

Ordered data Mann-Whitney U test

Interval or ratio t-test / ANOVA (F test)

t-Test Compare 2 groups

IndependentSamples (between Subs)

One sample (Within)

Evaluate differences bwt 2 independent groups

Evaluate differences bwt two conditions in a single groups

Assumptions to use t-Test

1. The test variable (DV) is normally distributed in each of the 2 groups

2. The variances of the normally distributed test variable are equal – Homogeniety of Variance

3. Random assignment to groups

Represents the distribution of t that would be obtained if a value of t were calculated for each sample mean for all possible random samples of a given size from some population

t-distribution

Degrees of freedom (df)

When we use samples we approximate means & SD to represent the true population

Sample variability (SS = squared deviations) tendsto underestimate population variability

Restriction is placed = making up for this mathematically by using n-1 in denominator

Degrees of freedom (df): n-1

The number of values (scores) that are free to vary given mathematical restrictions on a sample of observed values used to estimate some unknown population = price we pay for sampling

S2 = variance ss (sum of squares)

df (degrees of freedom)

(x - )2

n-1x

Degrees of freedom (df): n-1

Number of scores free to vary

Data Set you know the mean (use mean to compute

variance)

n=2 with a mean of 6X 8?6

In order to get a mean of 6 with an n of 2…need a sum of 12…second score must be 4… second score is restricted by sample mean (this score is not free to vary)

=x

Group Statistics

10 7.9000 1.1972 .3786

10 2.6000 1.2649 .4000

DRUGdoped

no dope

ENDURANCN Mean Std. Deviation

Std. ErrorMean

Independent Samples Test

.065 .801 9.623 18 .000 5.3000 .5508 4.1429 6.4571

9.623 17.946 .000 5.3000 .5508 4.1427 6.4573

Equal variancesassumed

Equal variancesnot assumed

ENDURANCF Sig.

Levene's Test forEquality of Variances

t df Sig. (2-tailed)Mean

DifferenceStd. ErrorDifference Lower Upper

95% ConfidenceInterval of the

Difference

t-test for Equality of Means

ANOVA

ENDURANC

140.450 1 140.450 92.604 .000

27.300 18 1.517

167.750 19

Between Groups

Within Groups

Total

Sum ofSquares df Mean Square F Sig.

Analysis of Variance (ANOVA)

Two or more groups ….can use on two groups…t2 = F

Variance is calculated more than oncebecause of varying levels (combo of differences)

Several Sources of VarianceSS – between

SS – WithinSS – Total

Sum of Squares: sum of squared deviations from the mean

Partitioning the variance

Assumptions to use ANOVA

1. The test variable (DV) is normally distributed

2. The variances of the normally distributed test variable is equal – Homogeniety of Variance

3. Random assignment to groups

between groups varianceWithin groups variance

F =

Systematic effects + error varianceerror variance

F =

1.00F = No differences btw groups

F = 21.5022 times as much variance betweenthe groups than we would expect by chance

Planned comparisons & Post Hoc tests

A Priori (spss: contrast)

part of your hypothesis…beforedata are collected…prediction is made

A Posteriori

Not quite sure where differences will occur

After Omnibus F…

2 types of errors that you must consider when doing Post Hoc Analysis

Why not just do t-tests!

1. Per-comparison error (PC)2. Family wise error (FW)

Alpha

Inflate Alpha!!!!

FW = c(

c = # of comparisons made= your PC

Ex: IV ( 5 conditions)

1 vs 21 vs 31 vs 41 vs 52 vs 32 vs 42 vs 5

3 vs 43 vs 54 vs 5

FW = c(

10 (0.05) = .50

HSD