Two types of empirical questions

• Descriptive– This kind of empirical question requires a

researcher to describe some aspect of behavior– For example, a researcher might ask, What are

people’s attitudes toward the homeless?

• Causal– This type of empirical question requires a

researcher to determine what causes something to happen

– For example, a researcher might ask, Does stress cause people to have road rage?

Our studies• We want to know whether or not verbal

estimate-based depth perception training benefits subsequent performance on verbal and active tasks?

• In other words, we want to know whether or not such training causes better performance later on

Question• How do you determine that one

thing caused something else to happen?

• For example, how could we determine that a new training simulation improved performance?

A simple logical method1. Collect data about current behavior2. Change the suspected cause3. Do not change anything else4. Collect data about subsequent

behavior5. Compare data collected before and

after the change was made

Example

Example1. Pre-test painting ability2. Provide training via simulator3. Do not change anything else4. Post-test painting ability5. Compare pre and post-test data

Complications• The logical process outlined earlier

is intuitive and straightforward

• When studying behavior, however, several issues could occur that would complicate the interpretation of the data

Potential Complications 1 & 2• Something other than the

suspected cause changes– Something inside the participants

changes• This is known as a maturation problem

– Something outside the participants changes• This is known as a history problem

Maturation1. Pre-test painting ability

– The participant warmed up during the pre-test

2. Provide training via simulator3. Do not change anything else4. Post-test painting ability5. Compare pre and post-test data

– Is the difference due to training or warm-up?

History1. Pre-test painting ability2. Provide training via simulator

– The simulator technician provides some advice

3. Do not change anything else4. Post-test painting ability5. Compare pre and post-test data

– Is the difference due to training or advice?

Training, Maturation or History?

Potential Complication 3• The initial data collection may bias

participants– This is known as a testing problem

Testing1. Pre-test painting ability

– Certain aspects of painting are assessed

2. Provide training via simulator– Participants work hard on aspects of painting that will

be assessed

3. Do not change anything else4. Post-test painting ability5. Compare pre and post-test data

– Is the difference due to training or bias?

Training or Testing?

Potential Complication 4• How one collects the Pre-Test data

may differ from how the Post-Test data are collected– This is known as a instrumentation

problem

Instrumentation1. Pre-test painting ability

– Test involves a door panel

2. Provide training via simulator3. Do not change anything else4. Post-test painting ability

– Test involves a trunk lid

5. Compare pre and post-test data– Is the difference due to training or tasks?

Training or Instrumentation?

Potential Complication 5• Sometimes the Pre-Test scores are

extreme, so it is likely that Post-Test scores will be different, no matter what– This is known as a regression problem

Regression1. Pre-test painting ability

– A number of participants score abnormally low

2. Provide training via simulator3. Do not change anything else4. Post-test painting ability

– Those low scoring participants score more average, while others stay the same

5. Compare pre and post-test data– Is the difference due to training or abnormal scores?

Training or Regression?

Solution• There is a simple way to capture these

issues, if they occur– Include a control group– If Pre and Post-Test scores differ for both

the experimental and control groups, then it is likely that the study was affected by one of these problems• This is known as having a confound in a study

A more complex method1. Collect data about current behavior2a. Experimental group - Change the suspected

cause2b. Control group - Don’t change the suspected

cause3. Don’t change anything else4. Collect data about subsequent behavior5. Compare data collected before and after the

change was made

Example1. Pre-test painting ability2. Provide training

– Via simulator (Experimental group)– Via standard method (Control group)

3. Don’t change anything else4. Post-test painting ability5. Compare pre and post-test data

No Confounds

Confounds

Confounds

Our studies• Experimental group

– Pre-Test, Verbal Training w/ Feedback, Post-Test

• Control group– Pre-Test, Verbal Training w/o Feedback,

Post-Test

Potential Confounds• Maturation• History• Testing• Instrumentation• Regression