© 2001 Dr. Laura Snodgrass, Ph.D.1 Basic Experimental Design Common Problems Assigning Participants to Groups Single variable experiments –bivalent –multivalent

© 2001 Dr. Laura Snodgrass, Ph.D. 1

Basic Experimental Design

• Common Problems• Assigning Participants to Groups• Single variable experiments

– bivalent– multivalent– baseline

• Multivariate– factorial– converging series


Common Problems

• Confounds• Lack of control group(s)• Nonequivalent control groups• Why control groups

– history– maturation– testing– instrument decay– statistical regression


Assigning Participants to Groups

• Independent or Random Groups Design– between groups

• Repeated Measures– within groups


Between Groups

• Advantages– generalizable– collect more data at a given level– shorter time for each participant

• Disadvantages– may not be random– unequal N– potential confounds– requires more participants


Between Groups

• Matching to equate groups and decrease error variance

• How– correlated variables– pairs– yoked controls– performance criterion


Matching

• Advantages– equates groups– increase power of experiment– decrease number of participants needed

• Disadvantages– extra work– extra testing– lose individual differences - less generalizable


Repeated Measures

• Advantages– fewer participants needed– impt for special groups– statistically more powerful

• Disadvantages– not naïve after first trials– order effects

• practice and fatigue• non-symmetric or differential transfer


Counterbalancing

• Vary order of treatment to distribute or measure order effects

• Complete counterbalancing– within participants ABBA– between AB for some, BA for others

• Latin Squares– each cond at each ordinal position– precedes and follows each other once


Counterbalancing

• Randomized blocks

• Time interval between trials– mortality


Single Variable Experiments

• Bivalent– one independent variable with two levels

• Multivalent (functional)

– one independent variable with three or more levels

• Baseline


Bivalent

• Two levels of the independent variable– experimental and control groups– two different levels of the variable

• Post-test only vs. pre-test/post-test

• Advantages– easy to interpret and analyze– decide if IV is worth studying


Bivalent

• Disadvantages– limited theoretical value– conclusions may be based on arbitrary choice of

levels– negative findings are not conclusive– does not describe shape of relationship therefore you

may over generalize for non-linear relationships• interpolation and extrapolation• plateau or asymptote


Multivalent (functional)

• Gives more info about the shape of the relationship

• Advantages– better estimate true relationship– individual choice of levels becomes less critical

• Disadvantages– more: time, effort, cost, subjects– more complex statistics and interpretation


Baseline

• Only works with certain types of variables– will not work with variables that cause permanent

change

• Procedure:– establish baseline or steady-state response level– introduce IV until stable transition– allow subject to return to baseline


Baseline

• Advantages– rules out most confounds– easy to interpret (often no statistics)

– flexible and replicable– investigate behavior of an individual

• Disadvantages– does not show small changes– may not generalize


Multivariate Experiments

• Factorial Designs– two or more independent variables, each with two or

more levels– variables can be all between, all within, or mixed in

many combinations

• Converging series– series of small experiments in which a variable

manipulated in an earlier experiment becomes a control variable in a later experiment


Factorial

• Design matrix– produces a family of functions– study main effects and interactions

• Advantages– study interactions– increases precision and generalizability– decrease statistical error and increase power– theoretical value


Factorial

• Disadvantages– increases time, money and number of subjects

increases dramatically as number of cells increases– assumptions of ANOVA may not be met– N-way interactions are very difficult to interpret


Converging series for applied problems

• Optimal designs– e.g. car, medical treatment, office

• Find an optimal level of a variable and turn it into a control variable

– lose higher order interactions


Converging Operations

• Converge on a single hypothesis– start with several possible hypotheses or

explanations– each experiment eliminates one or more until only

one remains (hopefully)

• For example: – perceptual defense against vulgar words– isolation tank


Converging Series

• Advantages– flexible, many choice points– efficient, leave out factors that have no effect– built in replications

• Disadvantages– interactions are lost– almost always between subjects– analyze and interpret prior before next experiment

so can take a long time

Documents

© 2001 Dr. Laura Snodgrass, Ph.D.1 Basic Experimental Design Common Problems Assigning Participants to Groups Single variable experiments –bivalent –multivalent