PSY 330 Lect 6 1

Research strategies

Research strategy: approach used to answer a research question.

• Non-experimental strategy

• Descriptive research strategy – describe variables

• Correlational research strategy – relate two variables

• Experimental strategy- demonstrate functional relationships between variables

PSY 330 Lect 6 2

Fundamentals Issues for Interpreting Research

In an experiment researchers attempt to determine the effects of changes in one variable (the independent variable, IV) on another (the dependent variable, DV)

Most often in behavioral research, the IV is an environmental event that is manipulated and the DV is behavior or what is measured

PSY 330 Lect 6 3

Fundamentals Issues for Interpreting Research

Experiments always involve two or more levels of the IV (compare across two situations). Often is the presence or absence of the IV.

In single-subject designs, behavior in the same subject is compared across different levels of the IV

In group designs, the different groups receive different levels of the IV

PSY 330 Lect 6 4

Fundamentals Issues for Interpreting Research

In some research designs, quasi-experimental research, investigators try to relate subject variables or non-manipulatable variables to behavior

• Subject variable = existing characteristic of individuals

• Not a true IV because it is not manipulated

• You cannot attribute results to subject variable because other variables were uncontrolled

• Subject variables are often investigated when it is unethical or impossible to manipulate the variable of interest

PSY 330 Lect 6 5

Fundamentals Issues for Interpreting Research

Subject variables are often investigated when it is unethical or impossible to manipulate the variable of interest

• Examples of subject variables:

• Gender

• History of drug use

• Age

• Socio-economic status

• Presence of psychiatric disorder

• Marital status

PSY 330 Lect 6 6

Subject variable or IV

• Two groups of participants, one with depression and one without depression, were given a task to measure their ability to delay gratification

• Two groups of college students were given a task to measure their reaction times. One drank Red Bull, and the other drank juice.

• Married women and single women were given a questionnaire to describe their job satisfaction

• Two groups of rats were trained to press a lever for food. One group was then tested when a running-wheel was present, the other was tested without the running wheel.

• Children in two groups were given a test to measure their problem-solving abilities. One group was made of 2nd graders, one group was made of 3rd graders

Subject

Subject

Subject

IV

IV

PSY 330 Lect 6 7

Validity

Validity = Accuracy of results

Determined by quality of research

Threat to validity is any aspect of the research process that raises doubts about the quality of the research process

As a researcher, you want to minimize threats to validity before the study is conducted.

PSY 330 Lect 6 8

External validity – whether results generalize

Do they apply to

Results ofexperiment

1. Other people?(Population validity)

2. Other situations?(Ecological validity)

Threats to external validity: any characteristic of the study that limits the generality of the results

PSY 330 Lect 6 9

Generalizabilty - do the same results occur under different circumstances?

If you get the same results under different conditions, you say that the results have generality or that the results generalize across situations.

PSY 330 Lect 6 10

1. Population validity – do results generalize from sample to the population of interest? Do the results generalize from the sample to other populations?

1. Subject selection bias

2. College students

3. Volunteer bias

4. Participant characteristics

5. Cross-species generalization

Categories of external validity

PSY 330 Lect 6 11

2. Ecological validity – do results generalize to other

A. Places

B. Conditions

C. Times (Historical contexts)

D. Measures Sensitization – procedures alter how subjects react

─ (Assessment or pretest sensitization) – will you get the same results without the assessment or pre-test?

Generality across measures – will you get the same results if you measure variables differently?

Time of measurement – will you get the same results if you measure behavior at different times?

Categories of external validity

PSY 330 Lect 6 12

Increasing external validity of experiments

Simulations –creating conditions within an experiment that duplicate the natural environment

• E.g., memory in birds

• E.g., economic behavior

Field study – an experiment conducted in a place that the subject perceives as a natural environment

• E.g., observe shoppers’ responses to different advertising displays

• E.g., play “alarm” calls to a primate group

PSY 330 Lect 6 13

Internal validity

Researchers must show that the IV caused the change in behavior and not something else

In an experiment, the researcher tries to eliminate the effects (or control for) extraneous variables - other variables in the study.

If there are extraneous variables, you cannot tell if those variables or the IV (or both) influenced behavior.

Internal validity - results due to the independent variable and not other variables

Extraneous variables

CausesIV or Treatment Behavior Change

Causes?

IV or Treatment Behavior Change

Extraneous variables that may have influenced results are threats to internal validity

PSY 330 Lect 6 15

Confounding variables

The experiment must show that the IV is responsible for changes in the DV and NOT extraneous or confounding variables

An extraneous variable may be a confound if it is unintentionally manipulated along with the IV and affects the results.

Confounds are threats to the internal validity of a study

PSY 330 Lect 6 16

General threat to internal validity

Environmental variables – • Features of the environment that that may

influence results.

• E.g.,• Room condition – bright, cheery vs. dark, small

• Coke (M) vs. Pepsi (Q)

PSY 330 Lect 6 17

Confounding variables

Suppose you are interested in whether massed or spaced studying produces better test performance.

• Group 1 gets massed studying 3 hrs on Monday.

• Group 2 gests spaced studying, 3 hr on Monday, 3 hr on Wednesday

• The test is given on Thursday and Group 2 does better.

• What might be a confound?

PSY 330 Lect 6 18

Confounding variablesSuppose you are interested in whether individuals

prefer to eat foods with or without preservatives.

• Group 1 is given a bright yellow bag of chips labeled “Rippled chips” with no preservatives on the label.

• Group 2 is given a beige bag of chips labeled “Lumpy chips” with preservatives on the label.

• Group 1 eats more chips, suggesting that people prefer chips without preservatives.

• What might be a confound?

PSY 330 Lect 6 19

Ways to eliminate confounds

1. Hold the variable constant

e.g., Everyone tested 1 day after last study session

e.g., Both groups get the same color/labeled bag of chips

2. Match values across conditions

e.g., Equal numbers in each group get yellow bag or beige bag

3. Randomization

e.g., bag color randomized across group

PSY 330 Lect 6 20

Common threats to internal validity:

Threats within single-group experiments.

1. History - extraneous events in environment cause change in behavior, e.g., between the pretest and posttest

2. Maturation – biological events in subjects causes change

3. Statistical Regression – participants chosen due to extreme scores, behavior may return to normal range when tested again

PSY 330 Lect 6 21

Common threats to internal validity:

4. Testing – change in behavior due to previous measurement, i.e., being given a pre-test

Practice effectsFatigue

5. Instrumentation- change in measurement during study

PSY 330 Lect 6 22

Common threats to internal validity:

Threats in multi-group experiments:

6. Assignment bias (subject selection)– treatment and no-treatment groups different before study started

7. Mortality (Attrition) – Differences in groups of subjects in study due to participants dropping out of study

8. Diffusion of treatment – IV unintentionally applied to control group.

9. Compensatory equalization of treatments - Members of control group experience additional compensation from people within or outside the study

PSY 330 Lect 6 23

Internal validity threats These are only some of the most common threats to

internal validity In later units we will talk about how research designs

(e.g., Solomon 4-group design) control for these threats to internal validity.

PSY 330 Lect 6 24

Internal validity threats Some threats to internal validity are also threats to

external validity. How do you decide what to call it?

• A variable threatens internal validity if it threatens interpretation of results

• A variable threatens external validity if it threatens generalizability of results

When designing an experiment, ensuring internal validity is the more important consideration.

PSY 330 Lect 6 25

Other validity threats: Bias

Experimenter bias/effects – changes in behavior are the result of actions of researcher, not IV. Can affect internal and external validity.

• Experimenter expectancies – researcher’s expectations about the outcomes influence the subject’s behavior or how they interpret the results. • Results may not generalize to other experimenters.• How do you reduce experimenter bias?

– Double blind procedures– Automate

• Experimenter characteristics – specific characteristics of experimenter limit generalizability of results.

PSY 330 Lect 6 26

Other validity threats: Bias

Participant bias/effects - changes in behavior are the result of participants knowing they are in a study, rather than IV. Can affect generalizability of results.

Novelty effects – individuals may react differently when they are in a new situation. Results may not generalize to different situations

Hawthorne effect or reactivity - Subjects known they are being observed and they behave differently. Results may not generalize to different conditions.

PSY 330 Lect 6 27

Subject bias cont. Demand characteristics – aspects of a study or cues that indicate to

participants how they should behave. Results may not generalize to situations in which cues are not present.

Subject roles – participants adopt different ways of responding to demand cues.

“Good Subject” participant tries to help the experimenter obtain good results

Negativistic subject – attempts to refute the investigator’s hypothesis. Faithful subject – no biases Apprehensive subject (evaluation apprehension)- participants

behave in ways to make themselves look good.

PSY 330 Lect 6 28

How do you reduce participant bias?

•Deception

•Manipulation check