Lecture 8: Quasi-experiments

• Aims & Objectives

– To differentiate between true and quasi-experiments

– To discuss the nature of random allocation

– To examine threats to experimental validity

– To examine some basic quasi-experimental designs

Type of general approaches to design

• Descriptive• What, where, when and to whom

• Relational• Co-variaton

• Experimental• Causal analysis via random allocation

• Quasi-experimental• Causal statements when groups are not equivalent – no

random allocation

Random allocation

• Every potential subject has an equal chance of being in any condition

• Simple randomisation

• Block randomisation– Blocks A&B, produce sequences e.g., AABB, ABAB.

Sequences are selected at random and subjects selected at random into that block

• Stratified randomisation– Select on a characteristic that influences the groups and have

block randomisation lists within those blocks

Internal validity: I

• Ruling out a third cause– Randomisation controls for

• History effects• Maturation effects• Mortality• Statistical regression to the mean

– Randomisation does not control for• Effects equalising groups

– Diffusion of treatment effects– Compensatory rivalry– Compensatory equalisation

• Effect separating groups– Resentful demoralisation

Statistical validity

• Risk of making a type 1 error

– Power– Fishing– Reliability of measures, treatments– Random irrelevance– Random heterogeneity of respondents

External validity:generalisation

• Is the effect stable

– Over time– Across individuals– Across IVs & DVs– Across places

Mook

• Research is not always about generalizability of findings

• Conceptualisation of generalizability are base don an agricultural model

• Experiments are about generalizability of theory not findings

Construct validity

• Experimenter effects

– Structural• Mono-operation bias• Mono-method bias• Poor explication of constructs

– Interpersonal• Demand characteristics• Apprehension evaluation• Rosenthal effect

Quasi-experiments

Nomenclature

X = a treatment

O = Observation

… = Not randomly assigned

Uninterrupted designs

O X O

One group pre- post test design

Threats = history, maturation regression

Non-equivalent groups

O X O…………O O

Untreated control group with pre and post test

Reverse treatments

O X+ O…………….O x- O

ITSDs

OOOOXOOOO………………..OOOO OOOO

OO OOOXOOOOOXOOO OOO

With switch replication

ARIMA

OOOXOOO333 456 Upward drift334 444 Upward constant335 466 Gradual upwards333 333 No change

Regression discontinuityDepression

Short LongPoverty

Randomized field trials

• Randomisation by independent group

• Make seek treatment elsewhere– Within condition effects

• Placebo-control

Experiments: the last word

• Experiments are important because they allow us to show what can or ought to happen

– Bio feedback– Milgram– Sherrif’s boys camp study