Introduction to ValidityTrue Experiment – searching for

causality What effect does the I.V. have on the

D.V.Correlation Design – searching for an

association between variables No causation, but often very

accurate predictors of results

Introduction to Validity• What is validity?

Are the ideas that are being investigated the same ideas that are being measured?

How appropriate or sound is the methodology that is being employed?

Overview of Today’s Lecture • Topics:

• Statistical Validity

• Construct Validity

• External Validity

• Internal Validity

Statistical Validity• Are the results of the data due to a

systematic factor (I.V.) or are the results due to chance?

• Appropriate statistical test (Chi-square, t-test, ANOVA)

A common threat to statistical validity is the violation of 1 or more assumptions of the test

Statistical Validity• P – value and the null hypothesis

• Psychology and the .05 Alpha shelf

• Significance vs. Meaningfulness

• The final question of statistical validity-

How accurate are the results of a statistical test?

Construct Validity• Research hypotheses must have a

theoretical basis

• Construct validity is concerned with how results support the underlying theory

• Is the theory that is supported the best theoretical explanation?

Construct ValiditySteps to help maintain construct

validity:

1.Operationally define variables with clear definitions

2.Develop hypotheses that are based upon strong, well supported theories

External Validity• Generalizability of findings to other:ParticipantsSubjectsPlacesTimesEnvironmental Conditions

External Validity• To generalize from one sample to a

population requires appropriate representation of the population

• Random selection from a population of interest helps in controlling for possible confounds

External Validity• Ecological Validity – Properly

generalizing from the laboratory to the “real world”

Internal Validity• Is the I.V. responsible for the

observable changes that occur in the D.V.

• Any factor (variable) that varies with the I.V. is a confound

Internal Validity• Nine primary confounding variables:

1. Maturation (normal age change)2. History (9/11) unrelated events3. Testing (test-retest)4. Instrumentation (alteration in

calibration)5. Regression to the mean

Internal Validity6. Selection (non-equivalent groups)7. Attrition (those who drop-out are likely different from the remaining)

8. Diffusion of treatment (talk among participants)

9. Sequence effects (experience during one part of the study influencing another part of the study)

ConclusionValidity concerns accuracy:

Are our statistical results accurate?Are we using an accurate theoretical basis?Are we accurate in implying that our results

can be generalized to a population?Are we measuring what we say that we are

measuring?