1

Chapter 4 – Issues in Single-Subject Research

Ps534

Dr. Ken Reeve

Caldwell College

Post-Bac Program in ABA

2

Review…

RULES TO FOLLOW AS AN APPLIED BEHAVIOR ANALYST– 1. Do no harm!– 2. Do some good!– 3. Identify WHAT you did that did some good!

(Focus of this course!)

3

Review…

TARGET BEHAVIOR = DEPENDENT VARIABLE

METHODS TO CHANGE BEHAVIOR (INTERVENTION) = INDEPENDENT VARIABLE

FUNCTIONAL RELATIONSHIP = degree to which the independent variable affects the dependent variable (and can you isolate this effect!!)– Main focus in research is to determine functional

relationship between intervention X and learning outcome Y

4

Features in a Functional Relationship

When a FUNCTIONAL RELATIONSHIP does really exits between IV and DV, then certain other characteristics will be present in the research findings. These include:

Prediction, replication, verification, reliability, and validity (we’ll describe these and other related terms…)

5

PREDICTION

When an intervention really DOES have an effect on behavior, then there is an expected outcome (behavior goes up/down, occurs faster/slower, topography changes, etc.)

When an intervention really has NO effect on behavior, then there is an expected “NO CHANGE” as the behavior outcome

6

VERIFICATION

This refers to CONFIRMING that behavior really changed as expected (behavior goes up/down, occurs faster/slower, topography changes, etc.)

VERIFICATION requires that we have a good operational definition of the behavior in question, a good measurement system, a good way to graphically depict the data, and accurate observers

7

REPLICATION

This refers to REPEATING what you did to see if your prediction about the functional relationship in question holds true more than just once

Why? Sometimes we can get amazing results JUST BY CHANCE and we want to make sure that the results weren’t a fluke

8

REPLICATION: S-S vs. GROUP Research In group studies, researchers usually

replicate the observed functional relationship with NEW participants

In single-subject research, researchers usually replicate the functional relationship within the SAME participant AND then also replicate it with new individuals

9

REPLICATION: DIRECT vs. SYSTEMATIC DIRECT REPLICATION: means to do the study

exactly the same way as before.– Important but boring. – Increases INTERNAL validity but not EXTERNAL

validity (we’ll define these in a minute). SYSTEMATIC REPLICATION: means to do the

study with some variation as compared to before (change population, setting, stimuli, behavior studied, etc.). – Important and more exciting. – Increases INTERNAL AND EXTERNAL validity.

10

RELIABILITY Refers to how dependable our observations of the

functional relationship are It asks: Can we put faith in what we are seeing

happen before our eyes? IF there is a real functional relationship AND we

are using operational definitions, good measurement systems, IOA, THEN we should see the same effect repeating to the same degree

Also means that nothing BREAKS DOWN in the research process (think of a “reliable” car!)

REPLICATION also shows us RELIABILITY of an effect

11

What hurts RELIABILITY?

POOR IOA – observers need to be well trained!

REACTIVITY of participant – when a person changes behavior because he or she finds out an observation is occurring

OBSERVER DRIFT – over time, the observers change their operational definition of the DV

12

VALIDITY Recall, yet again, that our goal is to identify a

functional relationship between IV and DV IF we can say that the IV or intervention was the

ONLY thing that changed the DV, THEN we have high INTERNAL VALIDITY– To be able to do this, we need to “rule out extraneous

variables” by not letting them change in any way while we are manipulating the IV (called “controlling for potential confounds”)

13

INTERNAL VALIDITY Here’s an example you might find in the “results” section

of a research journal: “As can be seen in Figure 1, when treatment was

introduced, there was a 40% increase in rate of responding as compared to baseline levels. When treatment was removed, rate of responding dropped back down to baseline measures. When treatment was re-introduced, rate of responding again increased by 40% as compared to baseline levels. Because no other variables were free to change during the study, it can be inferred that the behavior change was a direct result of the introduction and removal of the treatment procedure.”

14

DREADED THREATS TO INTERNAL VALIDITY We have a “threat to internal validity” when

other extraneous variables ARE free to change during the study

If a threat is confirmed (or we have good reason to think one happened), then we have CONFOUNDED the results– As a result, it CANNOT be inferred that the

behavior change was a direct result of the introduction and removal of the treatment procedure.

15

GOOD EXAMPLE OF THREAT TO INTERNAL VALIDITY “When facilitated communication is used,

participants are able to answer questions that they never could before.”

Sounds like a good functional relationship, right? – “If no FC, then no responding from

participant; If YES FC, then YES responding.”

But…there is a BIG threat to internal validity that, in fact, has been confirmed as a CONFOUND in pro-FC studies

16

TYPES OF INTERNAL VALIDITYTHREATS HISTORY – when some event or action by

another OUTSIDE OF THE “LAB” affects the DV of the participant

If researcher is made aware of these, then we don’t have good internal validity!– EXAMPLES INCLUDE: giving a new diet,

adding another therapy, doing additional training, bad week at home, etc.

17

TYPES OF INTERNAL VALIDITYTHREATS MATURATION – when the DV of the

participant changes as a result of growth and development

S-S research controls for this nicely because we observe DV of individual multiple times to make sure maturation is NOT the cause of any changes observed!

18

TYPES OF INTERNAL VALIDITYTHREATS Can this change in DV be due to

MATURATION?

19

TYPES OF INTERNAL VALIDITYTHREATS ATTRITION – refers to outcome data that is

skewed due to certain participants dropping out of a study– Ex. If only the children with the LEAST severe

autism remain in a study, then it is tough to determine if treatment was really good

20

TYPES OF INTERNAL VALIDITYTHREATS MULTIPLE TREATMENT INTERFERENCE –

refers to problem of selecting out what part(s) of a treatment “package” were responsible for outcome data

This is a HUGE problem in treatment of autism but it is also very pervasive in psychological therapies in general

Problem is that many claims are made about WHAT PART is the cause of the effectiveness– Ex. “Floortime,” “Son-Rise,” and TEEACH use many

ABA techniques. So does “Rational Emotive Behavior Therapy”

21

TYPES OF INTERNAL VALIDITYTHREATS TREATMENT DRIFT - refers to problem of

therapists changing the intervention over time– Ex.) Therapists may inadvertently increase amount

of reinforcement, may make instructions more salient than before, may prompt more effectively

– If this happens, again we have (take your pick!):• A problem inferring WHAT caused the change in DV

• A problem identifying a functional relationship

• A threat to internal validity

• A confound in our results

22

TYPES OF INTERNAL VALIDITYTHREATS EXPERIMENTER BIAS - refers to MANY

possible problems of researcher affecting the results (inadvertently, we hope)– Ex.) failing to acknowledge confounds, incomplete

data collection, selecting participants most likely to do well, etc.

– If this happens, again we have (take your pick!):• A problem inferring WHAT caused the change in DV

• A problem identifying a functional relationship

• A threat to internal validity

• A confound in our results

23

EXTERNAL VALIDITY Will the functional relationship identified hold true

with different participants, settings, materials, variations in instruction, etc. – If so, then we have HIGH external validity.

Do the findings only work under very narrow circumstances? – If so, then we have LOW external validity

BIG PROBLEM: people often confuse poor external validity with poor internal validity in S-S research. How?– “We shouldn’t use that intervention because it has only

been shown to work with one person! We need more studies with groups of people!” (THIS IS POOR LOGIC!)

24

EXTERNAL VALIDITY’S RELATION TO INTERNAL VALIDITY Notice that we need to do SYSTEMATIC

REPLICATIONS to increase external validity But, if each systematic replication continues to

show the functional relationship between IV and DV, then we are ALSO increasing INTERNAL validity

Successful replications show us that the results are likely not due to extraneous variables but are caused by the IV manipulated by the researcher!

This relationship is ONE-SIDED, though: increasing internal validity does NOT increase external validity!

25

ETHICS

What are they? A set of standards for behavior that pertains to

“doing what is right” Behavior analysts must abide by a very stringent

code of ethics since the behavior change techniques used in ABA are extremely powerful

See the BACB’s code for conduct (NOTE: You should download these from the

website to study. There will be some general questions about the BACB code on our class exam!)

26

End of chapter 4