Factorial Designs

Research Methods & StatisticsSummer 2014Kirstie Hawkey

Example drawn from Ch. 12 of McBurney’s research methods textbook

Good tutorials

• Between subjects factorial design:– http://web.mst.edu/~psyworld/between_subjects.htm#– http://www.experiment-resources.com/factorial-design.html– http://www.experiment-resources.com/factorial-anova.html

• Mixed factorial design: – http://web.mst.edu/~psyworld/mixed_designs.htm

• Interaction effects:– http://www.socialresearchmethods.net/kb/expfact.php

Factorial Designs

• When you are manipulating more than one independent variable– Need to examine the impact of multiple IVs on the

DVs– More efficient than running 2+ experiments– Can also examine interactions between the IVs– Must examine all possible cominations of the

values of the variables

Terminology• #levels of 1st IV x # of levels of 2nd IV– 2 IVs each with 2 levels: 2 x 2– 3 IVs each with 2 levels: 2 x 2 x 2– 2 IVs each with 3 levels: 3 x 3

• Usually identify the levels when writing about it:– A 2 (interface: A, B) x 2 (screen size: small/large)

factorial design – A 2 x 2 factorial design was used. Between subject

factors were interface (A or B) and screen size (small or large)

– Follow the conventions in papers in your domain

Simple 2 x 2 design• Examining characteristics of a person that influence

judgment of guilt– Facial expression (smiling person less guilty)– Attractiveness (attractive person less guilty)

• 2 IV’s each with 2 levels• Need to have a condition for each combination of

factors– Between subjects experiment– 4 groups of participants judging different sets of faces:

unattractive neutral faces; unattractive smiling faces; attractive neutral faces; attractive smiling faces

– Participants judge whether guilty or not

Simple 2 x 2 designFactor A1(neutral face)

Factor A2(smiling face)

Row Means (Effect of B)

Factor B1(unattractive)

A1B188

A2B124

56

Factor B2(attractive

A1B216

A2B252

24

Column means (Effect of A)

52 28

Main effect: The effect of a variable averaged over all values of another variable (or variables)Is there a main effect of Facial expression on the judgment of guilt?Is there a main effect of Attractiveness on the judgment of guilt?

Simple 2 x 2 designFactor A1(neutral face)

Factor A2(smiling face)

Row Means (Effect of B)

Factor B1(unattractive)

A1B188

A2B124

56

Factor B2(attractive

A1B216

A2B252

24

Column means (Effect of A)

52 28

Interaction effect: When the effect of one IV depends on the level of another IVIs there an interaction effect of facial expression and attractiveness on the judgment of guilt?

Graph it!Factor A1(neutral face)

Factor A2(smiling face)

Row Means (Effect of B)

Factor B1(unattractive)

A1B188

A2B124

56

Factor B2(attractive

A1B216

A2B252

24

Column means (Effect of A)

52 28

Interaction effect: When the effect of one IV depends on the level of another IVIs there an interaction effect of facial expression and attractiveness on the judgment of guilt?Easier to see if graph it: guilt on y axis, facial expression on x axis

What if you have an interaction?• Report it• Consider the impact on the main effects that you

are observing• What kind of interaction is it?– Antagonistic (the IV’s reverse each other’s effects)

• Lines cross, main effect can be flat– Synergistic (the IV’s reinforce each other’s effects)

• Steeper slopes– Ceiling-effect (one variable has a smaller effect when

paired with higher levels of a second variable)• Converging lines

Within Subjects Factorial Design

• 2 x 2 (each subject sees each of the 4 possible conditions)– E.g., Investigate whether the size of a handbag

impacts our perception of its weight– Weight: 2 levels (heavy/light)– Size (volume): 2 levels (small/large)– Measure the apparent heaviness– Want to have a measure from each participant for

each possible combination

Mixed Designs• Some factors are between, some are within• “A 2-factor mixed design was used: adaptive

accuracy (Low or High) was a between-subjects factor and menu type (Control, Short-Onset or Long-Onset) was a within factor. Order of presentation was fully counterbalanced and participants were randomly assigned to condititions”– Findlater et al., “Ephemeral Adaptation: The Use of

Gradual Onset to Improve Menu Selection Performance” CHI 2009

Assignment 4: • Restate your research problem and your hypotheses.• Briefly describe the study that you intend to run (an outline with

open questions is fine)• For each of your hypotheses, answer the following:

1. State your research hypothesis’s corresponding alternate and null hypotheses.

2. State whether it is one-tailed or two-tailed3. If you had a type I error for this hypothesis, what would be the conclusion

statement based on the error? 4. If you had a type II error, what would be the conclusion statement based on

the error? 5. Where you will get your data6. How you will rule out alternative explanations for the results (give an

alternative and show why it is not viable

• What are the overall limitations to your study design?

Recap: hypothesis testing & p-values

• http://www.youtube.com/watch?v=0zZYBALbZgg

• http://www.youtube.com/watch?feature=player_detailpage&v=eyknGvncKLw

• Type 1 and Type 2 error:– http://www.youtube.com/watch?

v=k80pME7mWRM– Really good overview – last 10 minutes will help

you talk about Q3 and Q4