psyc 321_06 threats to validity and control

04/10/2023 1

Developmental Research MethodsTopic 5: Testing Hypotheses

CEDP321 Ryan Sain, Ph.D.

VariationRemember, we are interested in

two types of varaiationSystematic and unsystematic

(chance)There are two sources of

systematic variance◦Ones due to the IV◦Ones due to confounds

Types of Error

The Null Hypothesis Is …..

True False

Based on the Test, We

either…

Fail to Reject the Null

RESEARCH OBJECTIVE

TYPE II ERROR

β

Reject the NullTYPE I ERROR

α

RESEARCH OBJECTIVE

04/10/2023 CEDP 596-04 Ryan Sain, Ph.D.

Threats to ValidityInternal:

◦History◦Maturation◦Instrumentation◦Selection◦Testing effects

Pretest sensitization

◦Regression toward the mean

◦Mortality

04/10/2023 CEDP 596-04 Ryan Sain, Ph.D.

More Threats

Reactance◦To experiment –

Hawthorne◦Expectancy Effect◦To testing (sensitized)

Multiple treatment◦Cannot control for prior

treatments

04/10/2023 CEDP 596-04 Ryan Sain, Ph.D.

Even more threatsSubject –

Experimenter Effects◦Hypothesis guessing◦Diffusion of treatment◦Good subject effect◦Evaluation

apprehension◦Obedience

Demand Characteristics◦Acting out the role of

subject

04/10/2023 CEDP 596-04 Ryan Sain, Ph.D.

Control

Controlling for confounds◦ Constancy of

conditions THOUSANDS of

designs Experimental Quasi-experimental Varying levels of

internal validity Do what you can

04/10/2023 CEDP 596-04 Ryan Sain, Ph.D.

Controlling for threats

Random assignment

Pilot studies◦ Quasi-control

subjectsBlindingDistracter

itemsStandardizin

g

04/10/2023 CEDP 596-04 Ryan Sain, Ph.D.

Creditswww.sxc.hu

◦Padlocks and faces – emospada◦Steering Nature – johnnyberg◦Spices – pape2000