1 PS 526 BEHAVIOR ANALYSIS AND LEARNING CALDWELL COLLEGE GRADUATE PROGRAM IN APPLIED BEHAVIOR ANALYSIS (ABA) DR. KEN REEVE

1

PS 526 BEHAVIOR ANALYSIS AND LEARNING

CALDWELL COLLEGEGRADUATE PROGRAM IN

APPLIED BEHAVIOR ANALYSIS (ABA)DR. KEN REEVE

2

Chapter 1- Introduction

3

SOME QUESTIONS TO PONDER

• Why are we interested in studying how we come to behave a certain way?

• What do we gain from this knowledge?• How important is it to be able to accurately predict

what someone will do in a given situation?• How does what others do affect what you do? • Any answers to the above questions are concerned

with the systematic study of how we learn to behave, act, respond in a certain way

4

MORE QUESTIONS TO PONDER

• How is the ability to learn adaptive?• The ability to learn is adaptive because it allows us to adjust to

where we live so that we can best optimize our ability to keep on existing

• Can you learn and show no change in behavior?• Can you show a change in behavior and yet not have learned to

do it?• Do you need to make a conscious effort to learn?• Is all learning free will (something you chose to do)?• Can you learn to do something and not know why you do it or

how you learned it?

5

TERMS• LEARNING - is enduring, relatively permanent change

in mechanisms of behavior involving specific stimuli and or responses that results from prior experience with those stimuli and responses

• BEHAVIOR - any activity that passes the dead man's test• (if a dead man can do it, it's NOT behavior• - also called a RESPONSE • REPERTOIRE - set of ALL responses you can do

6

ABCs of BEHAVIOR ANALYSIS

• ANTECEDENT (you are in a particular situation, setting, environment or in the presence of a particular cue or signal)

• this situation gives you some information if you've been there before

• it may have given you some clues as to how to act a certain way to get a particular something to happen

• example: how did you know how to sit in these chairs facing the board, take out a notebook and a pencil and NOT take out other stuff?

7


• BEHAVIOR - this is what you do in a particular setting, situation

• as a result of doing something, a change might occur in the setting

8


CONSEQUENCE - this is also called feedback, outcome- when you produce some behavior, some change

will occur in the situation- it may or may not be noticeable to you, or it may

occur immediately or a while afterwards- the consequence is important because it either

makes the likelihood of your doing that particular behavior again stronger or weaker depending on what the consequence is

9

• Our entire life is FULL of these ABCs, and unfortunately many of them CONFLICT

• What does that mean?• certain types of behavior result in good consequences in one

setting BUT they may result in BAD consequences in another setting

• ex) making jokes in a bar might result in people buying you drinks and flirting with you

• BUT making jokes in a classroom might result in the teacher reprimanding you or other students may treat you poorly

• PROBLEM: in a new situation you may or may not have a good idea as to how to act so you may do very little or behave in a way that worked for you before

• only EXPOSURE to these situations allows you to learn about them (others can also tell you how to act but you need to know whether good consequences previously resulted from listening to these people)

10

BEHAVIOR ANALYSIS

• looks to find the principles as to how both antecedents and consequences affect our behavior

• this is done with both human and animal behavior• - is done with both normal and "abnormal" behavior• - is done by changing environments (antecedents

and consequences) in a systematic and objective way

• How do we do this analysis?• use scientific method, specifically experimental

procedures (MORE LATER)

11

BEHAVIOR ANALYSIS

• - examines the CAUSE of behavior change• - assumes it is due to PRIOR experience• - the causes of behavior can only be inferred from

experimental manipulation• - can only be done with experimental research

because you need to look at both subjects without the cause (control) and subjects with the cause (experimental group)

12

EXPERIMENTAL ANALYSIS OF BEHAVIOR

• seeks to find common or general laws of learning• this MUST be verified experimentally not just

observing• need to be careful that learning, and not any other

reason, is the cause behind behavior change• is a great deal of experimental evidence of

generality of these laws

13

APPLIED BEHAVIOR ANALYSIS

• Same goals as plain old experimental analysis of behavior except that it focuses on SOCIALLY SIGNIFICANT BEHAVIOR

• (as opposed to any interesting behavior the researcher wants to look at)

14

TYPES OF CONDITIONING

• CONDITIONING=LEARNING • RESPONDENT LEARNING • Also called CLASSICAL CONDITIONING;

PAVLOVIAN CONDITIONING; S-R LEARNING; ASSOCIATIVE LEARNING

• when a new antecedent stimulus elicits (brings about) the same response as an old antecedent stimulus because it has been paired with the old stimulus

• RESPONDENT=the elicited behavior

15

TYPES OF CONDITIONING

• OPERANT LEARNING• Also called INSTRUMENTAL LEARNING;

SKINNERIAN LEARNING• Particular behavior becomes more or less likely to

occur in the future as a result of the consequences that have followed the particular behavior in the past

• OPERANT=the behavior that is more or less likely to occur as a result of the consequences it produced

16

BEHAVIOR SELECTED BY CONSEQUENCES

• Similar to evolutionary theory

• Genetic trait is to learned behavior as survival advantage is to day-to-day adaptability

• Within each person’s repertoire, what you do is there because it somehow gives you an advantage

17

REMOTE CAUSATION VS. IMMEDIATE CAUSATION

• Why is it hard in our own lives to see what caused us to act the way we do?

• Unlike in a chemical reaction, in which we can see immediate causal events, what we do in a given situation is often remotely caused (long past learning history that is hard to see or remember)

18

GIANTS IN BEHAVIOR ANALYSIS

• Ivan PAVLOV (1849-1936)• Russian physiologist in late 1800, early 1900s• in 1904, won NOBEL for discovering nerves that

stimulate digestive juices of pancreas• normally, animals will begin to salivate when food is

presented in the mouth• BUT Pavlov noticed that dogs began salivating when

they SAW the food, heard the opening of the food, or even saw the technician who prepares the food

• Any idea why? How is this adaptive?

19


• John Watson (1878-1958)

• influential for pushing the study of behavior only (no need to study “psyche”)

• “Psychology as the Behaviorist Views It”

• Little Albert and the White rat

• Advertising

• Child Rearing

20


• B.F. Skinner (1904-1990)• father of modern behavior analysis• developed operant learning chamber• research on reinforcement and how

“schedules” operate on what we do• brought BA to the masses• wrote about uses of BA for the betterment

of the world

21

SKINNER MYTHS

1. we have no mind, feelings, thoughts

2. raised his baby in a skinner box, experimented on her, and she went crazy and killed herself

3. was a communist, fascist, totalitarian

4. was a snappy dresser

5. was a skilled novelist

22

ASSUMPTIONS MADE BY BEHAVIOR ANALYSTS

• PRIVATE EVENTS• - yes, what you are thinking, feeling, etc are

occurring but they do NOT cause what you do• - commonly held that thoughts are “commentary”

about what you are doing or will be doing in a given situation and feelings are elicited by respondent conditioning

• your past learning history (and to some degree your genetics) determines what you are likely to do

23

ASSUMPTIONS MADE BY BEHAVIOR ANALYSTS

• FREE WILL• yes, you have “free will” but not in the

sense that most mean it• Behavior analysts imply that it means we

have CHOICES in what we can learn to do, when to do it, and that we can alter our own environments to make us more or less likely to do certain things

24

CHAPTER 2 – EXPERIMENTAL ANALYSIS

OF BEHAVIOR

25

FUNCTIONAL VS. STRUCTURAL APPROACH

• cognitive psychology (and much of education theory) takes a structural approach

• asks: what does the thought process look like• assumes that thinking determines behavior• major flaw: assuming it is cause of behavior, where

does the thought come from? • behavior analysis takes a functional approach• asks: what is the function (purpose) of the behavior;

what does it “buy” the person; how is it adaptive?

26

ELICITED VS. EMITTED BEHAVIOR (RESPONSES)

• respondent behavior is “elicited” by an antecedent event

• operant behavior is “emitted” in the presence of an antecedent event because, in the past, the behavior produced a particular consequence

27

RESPONSE CLASSES

• CLASS=set, group, category• A response class is a group of responses that serve a

similar, common function (produce a similar outcome or occur in the presence of a particular antecedent)

• a respondent response class could be all the behaviors that you do when someone yells DUCK (are elicited behaviors)

• an operant response class could be all the things you do to get your baby to smile at you (are emitted behaviors in the presence of your baby)

28

STIMULUS CLASS

• is a group of stimuli that have a common effect on behavior (can be antecedents or consequences)

• ex.) LOOKOUT, WATCH IT, DUCK: each would likely elicit a set of respondent flinching behaviors (each is an antecedent)

• ex.) COOKIE, APPLAUSE, BIG HUG: each may be used as an effective reinforcer to increase a child’s eye contact (each is a consequence)

• ex.) what group of antecedent stimuli made it likely that you began to do “student behaviors” when you came in today?

29

ESTABLISHING OPERATIONS

• fancy phrase which says that the functional relationship between what we do and the environment we are in DEPENDS on the context

• That is, reinforcers or punishers can be made more or less powerful as can antecedent stimuli

• (most who criticize behavior analysis as being too simplistic fail to mention these “what ifs”)

- deprivation vs. satiation- time- fatigue

- other available response options (anyone clean a lot when a final paper needs to be written?)

30

TACTICS OF BEHAVIORAL RESEARCH

REVIEW:• SCIENCE is a certain way of knowing about

events, such as the causes of our BEHAVIOR, based on OBJECTIVE observations or experience

• ”objective” means everyone can measure or see what you are seeing because you've AGREED on what you are measuring and how to measure it by spelling out its OPERATIONAL DEFINITION

31


REVIEW:EXPERIMENTAL- means that rather than just observing things, you

MANIPULATE something to see if that “something” CAUSES something else to happen

ex) if we change ANTECEDENTS or CONSEQUENCES, will either or both of these CAUSE a change to occur in behavior?

- this differs from CORRELATIONAL studies in which we are just looking to see if certain events happen together without necessarily seeing if one cause the other to occur

32


What are these events to be measured?- called VARIABLES What is a variable?- is some property of an event that can be measured- this property can change in value or take on different

characteristicsex) how fast you behave or when you behave or how long

you behave or how your behavior changes can all be measured (so these are variables)

ex) aspects of antecedents and consequences can also change

33


How can we tell if manipulating one variable causes a change in another?

- manipulate ONLY that one variable and hold all other variable constant (do not let any other variables change in value)

- if a change occurs in the person’s behavior, AND you’ve ruled out all other variables, THEN we can infer that the manipulated variable caused the change

34


IV=INDEPDENDENT VARIABLE (“CAUSE”)

- what you manipulate as a researcher, teacher, therapist, or parent to bring about a change in some other event (a person’s behavior)

DV= DEPDENDENT VARIABLE (“EFFECT”)

- what changed (a person’s behavior) as a result of that manipulated variable (the IV)

35

INTERNAL VALIDITY

• VALIDITY = TRUTH = ACCURATE DEPICTION

• internal validity has to do with whether the conclusion drawn about CAUSE and EFFECT relationship between two variables (IV and DV) is correct

• that is, does the conclusion correspond to the actual state of events?

36

CONFOUNDS (are THREATS TO INTERNAL VALIDITY)

• A confound means that what really caused the behavior to change is NOT what you manipulated but was something else

• A confound is when the effects of one or more variables interferes with your ability to determine the cause and effect relationship between IV and DV

37

CONFOUND TYPES

• Each of the following can make it appear as if your IV caused a behavior change (but it really didn’t)

• HISTORY – some other variable was applied to the person without your knowledge

• MATURATION – person’s behavior changed because of physical aging

• INSTRUMENT DECAY – how you are measuring the behavior is accidentally getting more strict or more lax

38

EXTERNAL VALIDITY

• Assuming that the IV did cause a change in the DV, are these results generalizeable to other settings, people, situations, reinforcers, antecedents?

• Need to REPLICATE experiments to gain external validity

39

SINGLE-CASE (SINGLE-SUBJECT) EXPERIMENTAL

DESIGNS • types of experimental designs used in

behavior analysis

• useful in applied clinical settings

• Allows us to study how manipulations affects a single person (as opposed to group studies which only give averages over a group of people)

40


DESIGNS • REQUIREMENTS: first measure how often behavior in

question is occurring over a period of time (called a BASELINE) for a SINGLE person

• THEN introduce the IV and measure behavior over a period of time (called a TREATMENT, EXPERIMENTAL CONDITION, or INTERVENTION

• This is called an A-B design http://www.unlv.edu/faculty/pjones/singlecase/ss2.gif (but it is NOT a very good design if you stop here) – you need to do more…

41


DESIGNS • After the behavior changes during

treatment, you can then REVERSE back to baseline over a period of time

• This is called an A-B-A or REVERSAL DESIGN

• Even better is to do more reversals so that you have an ABAB or ABABAB, etc.

42

ABA Reversal Design

43

CHAPTER 3REFLEXIVE BEHAVIOR

AND RESPONDENT

CONDITIONING

44

What’s the difference?• REFLEX = behavior automatically elicited by

some environmental event (antecedent stimulus)- NOT learned; is NOT a behavior you have learned to

do “without thinking”- Is “phylogenetic”

• RESPONDENT = also behavior automatically elicited by some environmental event (antecedent stimulus) BUT…– Is LEARNED through association with some other

stimulus that already elicits a similar response

45

REFLEXES• Believed to be a product of evolution• All species have them but you are more “rigid” as a species

(less adaptable) the more reflexes you have• The antecedent that ELICITS the behavior is called

UNCONDITIONED STIMULUS (translates to “antecedent event that has its eliciting effect on a behavior with no learning needed”)

• The elicited behavior part of the reflex is called the UNCONDIOTNED RESPONSE

• ex) startle, attention, fight or flight, blink, sneeze, shiver, GSR, cry

46

PRIMARY LAWS OF REFLEXES

• Law of Threshold – there is a “trigger point” for each US; below this point and it won’t elicit the UR

• Law of Intensity-Magnitude – the stronger the US the stronger the UR

• Law of Latency – the more intense the US the shorter the latency to the UR

47

OTHER REFLEX CHARACTERISTICS

• HABITUATION – says the more you present the US uninterrupted, the less and less the UR will be

• Believed to occur because of our phylogeny (genetics)

• Nerve impulse of reflex weakens

48

RESPONDENT LEARNING• ONTOGENY – your learning history• In reality, stimuli (events) often predict other stimuli (other

events)• So...you can learn that one event will lead to another and

you can get "ready" for it before it happens • That is, you learn something about one stimulus predicting

another's occurrence through experience and this changes your behavior (def of LEARNING)

• CLASSICAL CONDITIONING is process in which organisms learn about relationships between stimuli and adjust their behavior

49

RESPONDENT LEARNING

50

RESPONDENT LEARNING IN

ACTION IN A BABY• At first baby sees needle and is not upset (sight of

needle is “neutral” or “not meaningful”).• After getting a shot (US), baby gets upset (this UR

occurs naturally without learning—unconditioned response).

• Baby now “associates” the needle with the pain (she learned they “go together”)

• This results in responding to needle as something to be avoided (needle is now a learned or CONDITIONED STIMULUS)

51

RESPONDENT ACQUISITION

• We learn our associations between stimuli bit by bit

• The more pairings, the more we learn they go together

• At some point, you’ve learned this very strongly• The first pairing teaches us the MOST!• Each additional pairing teaches us a little less and

a little less

52

RESPONDENT EXTINCTION

• EXTINCTION – means to “gradually die out” (think of the dinosaurs)

• If we no longer pair the US and CS together, but present them separately, then a person learns that they no longer “go together”

• What is “extinguished”?• The association between the stimuli!

53

A strange phenomenon… SPONTANEOUS RECOVERY

OF RESPONDENT• After EXTINCTION has occurred, if we

wait a while and then present the old CS again, guess what it elicits?

• Yup, the old CR

• Why? Probably because there is some other stimulus present that was originally present when the association was first learned

54

RESPONDENT GENERALIZATION

• To “generalize” means to treat different things the same way

• In this case, if we present stimuli similar to the CS, then we should expect a person to respond similarly to those stimuli

• For example, if a person gets sick eating Cuban food, he may then find himself feeling ill when he tries to eat Mexican food

• This is another example of learning that is very adaptive

55

RESPONDENT DISCRIMINATION

• To “discriminate” means to treat different things differently

• In this case, if we present stimuli different from the CS, then we should expect a person to NOT respond to those stimuli

• For example, if a baby shows excitement at the sound of his father’s voice in the next room, we would not expect this excitement to occur if James Earl Jones is speaking on the television in the nest room

• This is another example of learning that is very adaptive

56

TEMPORAL RELATIONS & RESPONDENT LEARNING

• TEMPORAL = time

• RELATION = how things are arranged with respect to one another

• So…TEMPORAL RELATIONS refers to how CS and US are arranged (presented) with respect to one another as a function of time

57


• SHORT-DELAYED CONDITIONING

• CS comes on just before US and is still on when US comes on (most effective way to learn an association)

CS ___________------_________

US _____________------________

58


• SIMULTANEOUS CONDITIONING

• CS comes on at same time as US (not as effective because CS no longer predicts the US)

CS ______-------________

US ______-------________

59


• TRACE CONDITIONING

• CS goes off before the onset of the US (even LESS effective for learning associations. Why?)

CS ____________------__________

US ___________________------_____

60


• BACKWARD CONDITIONING• US comes on BEFORE the CS (kinda like

yelling DUCK after the ball hits you in the noggin)

• Rarely produces an association CS ____________----___________US ______----_________________

61

Second-Order Respondent Conditioning

• In first order conditioning, a neutral stimulus is paired with an unconditioned stimulus.

• When this occurs the control of the response to the US is transferred to the neutral stimulus, which is now called the CS.

• Second-order conditioning extends this transfer of control to events that have not been directly associated with the unconditioned stimulus. Aha!

• Second-order conditioning involves pairing a neutral stimulus with a CS, rather than pairing a neutral stimulus and US. More…

62

Second-Order Respondent Conditioning

• Why is this important?• Because it means that you can associate a

REMOTE stimulus with some other prior US• This may explain why someone might be afraid of

a dog even though they were never bitten

63

Second Order Conditioning

• CS1 paired with US

• Then CS1 elicits the CR

• IF CS2 then paired with CS1

• Then CS2 ALSO elicits CR– Occurrence in natural environments– Used in advertising: (Warning! Sexist

image to follow)

64

Second Order Conditioning

65

Biological Preparedness

• Humans– Taste aversion– Phobias

66

Applications

• Neuropsychoimmunology

• Systematic Desensitization

• Aversive counterconditioning

67

Systematic Desensitization

• technique based on respondent conditioning• used to treat phobias• Two major components: deep muscle relaxation

and fear hierarchy.• The first step is to teach the client deep muscle

relaxation. The theory was that relaxation was incompatible with fear responses and that the conditioned stimuli that elicited fear would eventually evoke relaxation which is called reciprocal inhibition.

68

The Fear Hierarchy

• After a client learns to relax, a fear hierarchy is constructed.

• Once the situation of maximum fear is identified, a series of verbal stimuli are arranged that describe settings that progressively evoke less and less fear.

• There are usually 10-30 items on the anxiety hierarchy, but some therapists have used over 100 items.

• Typically each item is written on a separate index card and the cards are sorted so that those causing the least fear are at the top and those causing the most fear are on the bottom.

69

The Fear Hierarchy• Once the fear hierarchy has been constructed, systematic

desensitization can begin.• At this point, the therapist reads the least anxiety provoking

description and asks the client to imagine himself or herself in that circumstance.

• The client is also told to lift a finger if the slightest amount of anxiety is felt.

• If the person feels anxious, the same item is repeated until at least three repetitions occur without anxiety.

• Following three or more successful repetitions, the next item on the hierarchy is presented. This continues until the last, and most fearful, item is completed.

70

Conditioning and Compound Stimuli

• To investigate the effects of context in respondent behavior, researchers have used situations involving compound stimuli.

• Compound Stimuli - two (or more) conditioned stimuli (tone and light) are presented together and acquire the capacity to evoke a single conditioned response.

71

Overshadowing

• Overshadowing - when a compound stimulus is used as the CS in a conditioning experiment.

• For example, a light and tone (CS) may be turned on at the same time and paired with an unconditioned stimulus such as food.

• Which PART of the compound will elicit salivation?

72

Overshadowing

• Pavlov found that the most salient property of the compound stimulus came to regulate exclusively the conditioned response.

• For example, if a loud tone and a faint light were used as the compound CS, then the tone would evoke salivation while the light would not. The tone is said to overshadow conditioning to the light. This happens even though the weak light will function as a CS if it is originally presented by itself and paired with a US.

73

Blocking• Blocking- a CS associated with a US blocks a subsequent

association between the US and a new CS. • In blocking, a CS is paired with a US until the conditioned

response reaches maximum strength. Following this conditioning, a second stimulus is presented at the same time as the original CS, and both are paired with the unconditioned stimulus.

• On test trials, the original CS evokes the CR but the second stimulus does not.

• For example, a tone (CS) may be associated with food (US) until the tone reliably evokes salivation. Next, the tone and light are presented together and both are associated with food. On test trials, the tone will elicit salivation but the light will not.

74

• Conditioned Suppression- a previously neutral stimulus (e.g. tone, light) is paired with an aversive US such as an electric shock. After several pairings, the originally neutral stimulus becomes a conditioned aversive stimulus (CS). The CS is said to evoke a conditioned emotional response (CER) that is commonly called anxiety or fear.

Conditioned Suppression (Learned Emotional “Shutdown”)

75

Blocking• Using a conditioned-suppression procedure, Kamin (1969)

discovered the phenomenon of blocking. • Two groups of rats were used: a blocking group and a

control group.• In the blocking group, rats were presented with a tone

(CSave) that was associated with electric shocks for 16 trials.• Following this, the rats received eight trials during which the

compound stimulus tone and light were followed by shock.• The control group did not receive the 16 light-shock

conditioning trials but did have the eight trials during which the compound stimulus tone and light were paired with shock.

76

Kamin (1969)• Both groups were tested for conditioned suppression of

lever pressing in the presence of the light. That is, the light was presented alone and suppression of bar pressing for food indicated the occurrence of the conditioned emotional response (CER).

• Kamin found that the light suppressed bar pressing in the control group but did not affect lever pressing in the blocking group. In other words, prior conditioning with the tone blocked or prevented conditioning to the light.

• Functionally, the light was a CSave in the control group but not in the blocking group.

77

Sensory Preconditioning

• Sensory preconditioning is another example of stimulus control by compound events.

• Sensory Preconditioning- two stimuli such as light and tone are repeatedly presented together without the occurrence of a known US (preconditioning).

• Later, one of these stimuli is paired with an unconditioned stimulus and the other stimulus is tested for conditioning.

• Even though the second stimulus was never directly associated with the US, it comes to evoke a conditioned response.

78

CS-US Correlations

• Contiguity - temporal relationship

• Contingency - correlation

79

Rescorla-Wagner Model of Conditioning

• CS acquires a limited amount of associative strength on each trial.– Associative strength – magnitude of elicited CR

80

The Rescorla-Wagner Model• The basic idea of the Rescorla-Wagner model is

that a conditioned stimulus acquires a limited amount of associative strength on any trial (pairing with US).

• Associative strength describes the relation between the CS and the magnitude of the conditioned response. In general, associative strength increases over conditioning trials and reaches some maximum, asymptotic level.

81

Rescorla-Wagner Model of Conditioning

• CS gains gains certain amount of associative strength on any one trial.

• Strength gained diminishes with each pairing as associative strength is increased.

82

Acquisition

Trials

0 2 4 6 8 10

Ass

oci

ativ

e S

tren

gth

0

2

4

6

8

10VMAX

Si=0.25VMAX=10.00VSUM= 0.00

83

Drug Tolerance and Classical Conditioning

• A psychoactive drug such as heroin is a US when it is in the bloodstream; its effects are a UR (the “high”).

• BUT, body also has an UR in which it FIGHTS the drug and decreases its potency (more…)

84

Drug Tolerance and Classical Conditioning

• OD cases have resulted from drug use outside the user’s typical drug administration environment. A high dose that was tolerated in the CS setting is lethal in a setting which is still a neutral stimulus.– Repeated pairings of a drug and a CS causes the conditioned response to

gain strength and oppose the unconditioned drug effect– Reduction of drug effects (UR) is not due to habituation, but rather to the

counteractive effects (CR) on the setting and injection process (CS)

• What if the normal situation is present (CS) without the drug US?– Elicited respondents are often called “cravings”, a process known as

conditioned withdrawal– CS elicits reactions normally countered by US, but when US is not

delivered when CR reactions occur, subjects experience withdrawal

85

Conditioned Immunosuppression• Another example of environmental influences altering what is

generally thought to be internal and autonomously controlled processes– CS is paired with a US drug that suppresses immune system function

(eg. Production of antibodies)– After several pairings the CS is presented alone, immune system

response is measured

• Can the immune system be conditioned to increase immune reaction?– Buske-Kirschbaum, Kirschbaum, Stierle, Jabaij, and Hellhammer

study (1994)• Flavor CS paired with adrenaline injection US• After several pairings, presentation of CS alone raised cell production normally

caused by adrenaline!

86

87

Chapter 4

REINFORCEMENT

AND EXTINCTION

OF OPERANT BEHAVIOR

88

Learning Objectives

• Respondent vs. Operant Learning

• Reinforcement vs. Punishment

• “Side Effects” of Operant Learning Processes

• Extinction

• Partial Reinforcement

• Forgetting vs. Extinction

89

TYPES OF BEHAVIOR ANALYSIS

• RESPONDENT CONDITIONING - organism does NOT control presentation of stimuli but only responds to them

• OPERANT CONDITIONING – more “active” and goal-directed

90

OPERANT CONDITIONING

• INVOLVES LEARNING WHAT TO DO BASED ON CONSEQUENCES OF PRIOR “DOING”

• LEARNED BY HAVING BEHAVIORS FOLLOWED BY PARTICULAR CONSEQUENCES

• EXAMPLES? (see next slide!)

91

OPERANT CONDITIONING EXAMPLES

• After Ringo makes a sound resembling a word, his mom gives him a big hug. Ringo makes the sound again (wouldn’t you?)

• After Bobby sticks his finger in the dog’s nose, the dog barks loudly and snaps at him. Bobby does NOT do that again!

• After Jonathan fusses, his therapist removes all curriculum materials. Jonathan fusses again the next time work is put in front of him.

• Susan screams in the supermarket and her mother gives her some candy to quiet her. She screams again the next time they enter the supermarket.

92

What is a Contingency?

• A contingency is an IF-THEN relationship

• Usually describes how reinforcement and punishment are arranged

• If behavior X occurs in a particular setting, THEN consequence Y will follow

• The 3-TERM contingency is the ABC relationship of antecedent-behavior consequence

93

OPERANT CONDITIONING TYPES

• REINFORCEMENT- a consequence that increases the chance that the behavior will be made again

• PUNISHMENT- a consequence that decreases the chance that the behavior will be made again

94


• REINFORCEMENT- a consequence that increases the chance that the behavior will be made again

- can be POSITIVE contingency (present something such as hugs, praise, kisses, tokens, access to TV or video games, stars, stickers)

- or NEGATIVE contingency (take something away such as pain, hunger, fear, annoyances, wet diaper)

95

96


• PUNISHMENT- a consequence that decreases the chance that the behavior will be made again

- can be POSITIVE contingency (present something such as yell, loud noise, frown, pain, restraint)

- or NEGATIVE contingency (take something away such as access to TV or video games, privileges, access to dessert) (also called RESPONSE COST or TIME OUT)

97

Summary Table

BEHAVIOR INCREASES

BEHAVIOR DECREASES

CONSEQUENCE INVOLVES

ADDING SOMETHING

POSITIVE

REINFORCEMENT

POSITIVE PUNISHMENT

CONSEQUENCE INVOLVES REMOVING

SOMETHING

NEGATIVE REINFORCEMENT

NEGATIVE PUNISHMENT

(TIME OUT; REPSONSE

COST)

98

ANOTHER OPERANT CONDITIONING

TYPE: EXTINCTION

• Like punishment, likely to decrease behavior but is done by ignoring child’s behavior (assuming this behavior had previously been reinforced)

• NEVER use this if behavior child is doing is dangerous

99

“SIDE EFFECTS” OF OPERANT CONDITIONING

• REINFORCEMENT HAPPY

• PUNISHMENT NOT HAPPY

• EXTINCTION NOT HAPPY

100

Behavioral Effects of Extinction• Extinction produces several behavioral “side

effects” in addition to a decline in a rate of response• Extinction Burst (“tantrum”)

When extinction is started, operant behavior tends to increase in frequency (called extinction burst)

• Response TopographyIn addition to extinction bursts, operants

become increasingly variable in form (“topography”) as extinction proceeds

101

Other Behavioral Effects of Extinction

• Emotional ResponsesA variety of emotional responses occur

under conditions of extinction. One important kind of emotional behavior that occurs during extinction is aggression.

• Why are some behaviors “resistant to extinction”? (see next…)

102

Partial Reinforcement Extinction Effect

• Resistance to extinction substantially increases when intermittent schedule of reinforcement has been used to maintain behavior.

• Extinction occurs more rapidly when behavior had previous CONTINUOUS REINFORCEMENT than on intermittent reinforcement

• 2004 Honda vs. 1971 Chevy

103

Discriminative Stimuli• Antecedent event (setting) that precedes an operant

behavior• What is present in your surroundings when a behavior

is reinforced (or punished)• When learning occurs, the discriminative stimuli alter

the likelihood that the operant behavior will occur• Discriminative stimuli are said to “set the occasion” for

behavior• discriminative stimulus symbol is SD.• Fancy way of saying that we learn a type of rule: “In a

particular situation, performing a certain behavior usually produces a particular consequence.”

104

Discriminative Stimuli• Discriminative stimuli change the probability that

an operant will be emitted.• When an SD is followed by an operant response

that produces reinforcement, the operant is more likely to occur the next time the stimulus is present.

• When an operant does not produce reinforcement, the stimulus that precedes the response is called an S∆ (“S-delta”). In the presence of an S-delta, the probability of emitting an operant declines.

105

Reinforcement, Intrinsic Motivation, and Creativity

• Many educators and social psychologists argue that rewards/reinforcement reduce individual self-determination, motivation, and creativity– Rewards/reinforcement are interpreted as

“controlling” which leads to the reduction in creative performance

106

Rewards and Intrinsic Motivation

• Opponents of rewards often cite experimental data showing rewards as having a negative effect(s)– Results are not consistent, some studies find negative

effects, some find positive effects– Tangible rewards given for meeting a certain criterion

level of performance or exceeding the performance of others actually maintained or enhanced intrinsic interest

– Rewards tied to level or quality of performance increase intrinsic motivation or leave intrinsic interest unaffected

107

Rewards and Creativity

• Commonly accepted as fact, the phrase “rewards lessen creativity” is a broad generalization– Divergent thinking is the most widely studied form of

creativity• Involves varied novel responses to a problem or question that

has multiple possible solutions

• Opponents of rewards report that offering an individual a reward results in reduced divergent thinking

• Robert Eisenberger suggests failures to find increased creativity may occur when the reward is not actually contingent on creative performance

108

Premack’s Principle

• “more probable behaviors will reinforce less probable behaviors”

• “less probable behaviors will punish more probable behaviors”

– Premack suggests it is possible to describe reinforcing events as actions of the organism rather than as discrete stimuli

109

Procedures in Operant Conditioning

• The Method of Successive Approximation, aka Shaping

• Many novel forms of behavior may be shaped by the method of successive approximation. Shaping is key part of acquisition and necessarily involves differential reinforcement.

• Variability is a good thing, it is the “stuff” that novel operants are selected from

• As behavior continues to be reinforced variability tends to decrease.

110

Applying Extinction• Extinction is an important role in behavior

modification• When put to bed by his parents a 20-month-old boy

would “throw” a temper tantrum requiring the parents to stay up with him.

• The parental attention given to the boy served as a reinforcement to the tantrums.

• Extinction was implemented by the parents leaving the room after the child was put to bed.

• The first extinction session, the tantrum lasted 45 min. and on the third session only 10 min. After ten days the boy had no tantrums.

111

Applying Extinction

• The child’s aunt reinforced his crying by staying in the room and his tantrums reoccurred.

• This intermittent reinforcement increases resistance to extinction of the tantrums

• The second extinction procedure took longer due to the intermittent reinforcement.

• The first procedure was more effective since the child was continuously reinforced.

112

Remembering and Recalling

• In much of psychology, people and other organisms are said to store information about events in memory– The use of the noun memory is an example of

reification or treating an action as if it were a thing

– In behavior analysis, the verb remembering (or forgetting) is used to refer to the effect of some event on behavior after the passage of time

113

Remembering and Recalling• For humans, we may say that a person recalls his/her trip to

Costa Rica, when a picture of the trip occasions a verbal description of the vacation– From a behavioral perspective, recalling the trip is behavior

(mostly verbal) emitted now with respect to events that occurred in the past

• Remembering and recalling are treated as behavior processes rather than some mysterious thing (I.e. memory) within us– Behavior analysts assume that the event recalled is one that was

described when it first occurred– Recalling refers to reoccurrence of behavior (mostly verbal) that

has already occurred at least once

114

Chapter 5 - Schedules of Reinforcement

116

REINFORCEMENT INTHE “REAL” WORLD

• So far we've talked about rather simplistic cases in which reinforcers are delivered after each response

• in actuality, how reinforcers are presented is far more complex

• rarely do we see a 1:1 relationship between R and Sr

117

SCHEDULES OF REINFORCEMENT

• these are rules about HOW and WHEN reinforcer follows a R

• usually we study these schedules by first conditioning an animal with CONTINUOUS reinforcement and then seeing how different schedules maintain or change levels of responding

• Why are these schedules important? Who needs to know about them?– teachers, parents, managers, industry, etc.

118

Types of Schedules of Reinforcement:

• Most simple schedule:• CONTINUOUS REINFORCEMENT (CRF) - is

simplest schedule• involves a 1:1 ratio between R and Sr• this ratio stays constant

• INTERMITTENT (PARTIAL) SCHEDULES:• these involve cases where reinforcement is delivered

only part of the time or intermittently

119

RATIO SCHEDULES

• here the reinforcer depends on a certain number of responses to occur with no regard to WHEN the responses are made

• Fixed Ratio - ratio of R/Sr remains constant– is denoted by the number of responses to get one reinforcer

(FR x)

• ex) Paper boy gets paid after delivering 30 papers• ex) factory worker makes 10 televisions to get paid• ex) salesman gets commission after every 5 cars he sells• ASK: what is CRF really? answer: FR 1

120

RATIO SCHEDULES

• What type of responding does FR produce?• CRF (FR 1) produces steady responding• "leaner" FR schedules produce pattern called

BREAK and RUN or STEPPING• that is, there is a RATIO RUN up to the point of

reinforcement, followed by a POST-REINFORCEMENT PAUSE

• the "leaner" the schedule, the greater the run and the longer the pause

• the harder you work to get the reward, the longer you "rest"

121

FIXED RATIO SCHEDULES

122

RATIO SCHEDULES

• What happens if you abruptly and severely lean out a FR schedule?

• you get RATIO STRAIN• person will stop responding before getting the

reinforcer• A lean FR schedule functions as EXTINCTION• person says "Oh, I guess I don't get the reward

anymore so I will stop responding"• THEREFORE, if you plan on leaning out a

schedule you need to do it GRADUALLY to avoid ratio strain

123

RATIO SCHEDULES• In real world, the ratio to get rewards usually varies to some

degree so it is important to study:• VARIABLE RATIO - ratio of R/Sr varies about some

average• is denoted by the AVERAGE number of responses to get one

reinforcer (VR x)• ex) slot machine• ex) number of turns to get dough right, • ex) number of good deeds to get someone to notice you, etc.• how many responses needed to get the reward is now

UNPREDICTABLE• sometimes it can be few responses, sometimes many• for this reason, you do NOT get break and run but instead

get STEADY responding

124

VARIABLE RATIO SCHEDULES

125

INTERVAL SCHEDULES

• here, delivery of Sr depends not only on the response being made but also WHEN the response is made

• reinforcement RESETS the interval clock• 1. FIXED INTERVAL - a certain constant or fixed time period

must elapse since the last reinforcer was presented before a response can result in another reinforcer

• IMPORTANT: the person still needs to make the response after the time has elapsed

• denoted by the time period (FI y)• ex) checking the mailbox before 2:00pm• ex) going in for a paycheck before Thursday• ex) taking out a cake before the 45 minutes are up• ex) checking to see if the water has boiled• ex) weekly scheduled quizzes• ex) weekly meetings• ex) dog checking for master to be home

126

INTERVAL SCHEDULES

• What type of responding does FI produce?

• produces SCALLOPING

• person learns when in time Sr is available so person to respond at higher and higher rates when that time is near

127

FIXED INTERVAL SCHEDULES

128

INTERVAL SCHEDULES

• VARIABLE INTERVAL - a varying time period must elapse since the last reinforcer was presented before a response can result in another reinforcer

• IMPORTANT: again, the person still needs to make the response after the time has elapsed to get reinforcer

• ex) child in car asking "are we there yet?"• ex) pop quizzes• ex) wondering when to make the "first move" during a

date• ex) checking to see if chicken is done

129

VARIABLE INTERVAL SCHEDULES

• What type of responding does VI produce?

• steady responding without pauses because you never know exactly when the reinforcer will be available

130

INTERVAL SCHEDULES

• So far we've implied that you can make the response at any time once the interval has elapsed BUT

• in real world there is often only a short window of time in which the reinforcer is available

• - this is called a LIMITED HOLD INTERVAL SCHEDULE

• ex) checks from payroll are available around the 20th of each month (VI schedule) but once they come they are only available for two weeks or they are forfeited (LIMITED HOLD)

131

INTERVAL vs. RATIO SCHEDULES

• How does the RATE of RESPONDING affect the occurrence of Sr?

• RATIO SCHEDULES - responding at a higher rate ALWAYS results in more Sr

• INTERVAL SCHEDULES - reinforcement depends more on WHEN the R is made as opposed to how many R are made

• So logically...which schedule yields HIGHER RESPONSE RATES?

• the RATIO SCHEDULE• Implications of this?• management wants a ratio schedule (get more work out of

workers) whereas labor does not

132

VARIABLE RATIO vs. VARIABLE INTERVAL

SCHEDULES• Why do VR schedules yield higher response rates

than VI?• may have to do with IRT (inter-response time)• VI - there is a benefit in waiting (conserve energy)

because the longer you wait the more likely it is that reinforcement will be available when you actually respond

• VR - there is no benefit to waiting because the longer you wait the longer you are putting off getting the reinforcer

133

REVIEW

134

Chapter 8:Stimulus Control

135

Stimulus Control• controlling stimuli = antecedent events that

precede operant behavior and affect its occurrence• A controlling stimulus (S) is said to alter the

probability of an operant, in the sense that the response is more (or less) likely to occur when the stimulus is present.

136

Stimulus Control• Discriminative stimulus (SD) - a controlling

stimulus that sets the occasion for reinforcement of an operant response.

• S-delta (SΔ)or extinction stimulus - a stimulus that sets the occasion for extinction of an operant response.

• SP – a stimulus that sets the occasion for punishment of an operant response

137

Emitted versus evoked•Operants can and do occur in the absence of any eliciting stimulus (they are said to be “freely emitted”). •However, when an SD comes to control occurrences of an operant (alters the response’s probability of occurring,) then we say that the “SD evokes the operant.” •The term evoke indicates that operant response is under stimulus control of an antecedent stimulus.•We can also say that “the SD sets the occasion for reinforcement of the particular response.”

138

Differential Reinforcement and Discrimination

• When a person responds in one situation but not in another, we say that the person shows a discrimination between the situations.

• simplest way to teach differential responding (aka discrimination) is to reinforce an operant in one situation and withhold reinforcement in the other.

• Stimulus control refers to a change in behavior that occurs when either an SD or SΔ is presented.

• When SD is presented, probability of response increases; when an SΔ is presented, the probability of response decreases.

139

Generalization vs. Discrimination

• DISCRIMINATION = particular response in one situation but not in another

• We “discriminate among/between settings, people, stimuli”

• GENERALIZATION = particular response occurs across different situations

• We “generalize across settings, people, stimuli”

140

Stimulus Generalization

• Particular response is evoked by new stimuli that presumably share common properties with the original discriminative stimulus used in training.

• For example, child says “monkey” when shown novel pictures of monkeys not used in original teaching situation

• Generalization and discrimination refer to differences in the precision of stimulus control (e.g. under what situation does one behavior “end” and another one “begins”?)

• Discrimination and generalization are inversely related (as one goes up the other goes down)

141

Generalization Gradient

• A generalization gradient shows the relationship between the probability of response and stimulus value.

142


• Another example…

143


• There are many stimulus characteristics that might control behavior (size, color, presence or absence of certain features, location, etc.)

• Generally, the more you vary these values, the LESS GENERALIZATION you will see

144

Studying Stimulus Control

• To study generalization/discrimination, the researcher may arrange the presentation of SD or SΔ so that one follows the other. This is called successive discrimination.

• In alternative procedure, simultaneous discrimination, the SD and the SΔ are presented at the same time and the organism responds to one or the other.

145

Problems with “Regular” DiscriminationTraining Procedures

• When the SD and the SΔ are alternately presented, the organism initially makes many errors.

• That is, person continues to respond in the presence of the SΔ because generalization often occurs unless you teach otherwise.

• As discrimination training progresses, a differential response occurs to the SD and SΔ.

146


• BASIC RESEARCH IN “REGULAR” DISCRIMINATION TRAINING: pigeon taught to peck (R) a green key (SD) for food (Sr).

• Once behavior is well established, color on key is changed to blue (SΔ) and pecking not reinforced

• blue and green colors are alternately presented• During early training sessions, onset of extinction

will generate emotional behavior that interferes with ongoing operant behavior. (Bird is Mad!)

147


• Extinction is an aversive procedure!

• Pigeons flap their wings aggressively and will work for an opportunity to attack another bird during the presentation of SΔ !

• Birds will even peck a different key if pecking turns off the extinction stimulus (SΔ), implying that the stimulus is aversive. (this is an avoidance behavior!)

148


• Because “emotional behavior” is elicited by the frustrating trial and error sessions, discriminative responding takes a long time to develop!

• So…is there a better option?

149

Yes: Errorless Discrimination• Here, the teacher does not allow the child to make mistakes by

responding to the extinction stimulus SΔ.

• Errorless discrimination involves gradually introducing the SΔ initially at a very weak “intensity” so that responding to it is very low in probability. Thus, person is actually practicing NOT responding to it.

• Over repeated trials, intensity of the SΔ gradually increased.

• Eventually SΔ is presented in full intensity and the person will not respond to it.

• So, a discrimination between the (SD) and the SΔ was acquired without the errors of responding to the SΔ.

150

STIMULUS FADING

• When some stimulus ALREADY occasions a response, but we want to have a DIFFERENT stimulus occasion the response, then we might use a STIMULUS FADING PROCEDURE

• PROMPT = an “artificial” SD that we use to make a

behavior likely to occur when the “natural” SD is ineffective

• We then gradually reduce (FADE OUT) the prompt so that

the control of the behavior transfers to the “natural” SD

• Sometimes we FADE IN the “natural” SD

151

More on FADING

• Sometimes we can “morph” an “artificial” SD into the

“natural” SD

• This can be done with morphing software.

• Often done where a picture is morphed into a word over successive trials so that the word will control a verbal label such as CAT.

152

COMPLEX STIMULUS CONTROL

• Refers to situations in which the behavioral function of an antecedent stimulus changes depending on the presence of another stimulus.

• That is, in some situations, a stimulus might be an SD and in other situations the SΔ

• This is called CONDITIONAL DISCRIMINATION LEARNING because the correct response is “conditional on” (dependent on) another antecedent

• A CONDITIONAL DISCRIMINATION is an “IF-THEN” rule

153

EXAMPLE OF CONDITIONAL DISCRIMINATION

Please select the PLUS SIGN by pressing it:

+ &

Note that here the stimulus “+” is an SD for pressing the + sign AND the stimulus “&” is an SΔ for pressing the & sign.

The correct response is conditional on (dependent on) the instructions give.

What instructions would REVERSE the behavioral functions of the stimuli?

154


Please select the animal that says MEOW by pointing to it:

CAT DOG

Note that here the stimulus “CAT” is an SD for pressing CAT AND the stimulus “DOG” is an SΔ for pressing DOG.

The correct response is conditional on (dependent on) the instructions give.


155

CONDITIONAL DISCRIMINATION =

MATCHING TO SAMPLEOften in conditional discrimination training, the

conditional stimulus is referred to as a SAMPLE and the choices we respond to are called COMPARISON STIMULI

This is also called MATCHING TO SAMPLE (MTS) because you are trying to make the correct response in the presence of the sample

Did you know? A multiple-choice exam question is a type of matching to sample or conditional discrimination trial!

156


For a child, the stimulus of MOMMY being present is usually an SD for talking to her

But WHAT IF mommy and the child are in church or temple?

Note that the correct response to mommy (being quiet) is conditional on (dependent on) the setting in this case.


157

Delayed Matching to Sample (DMTS) and Remembering

• This variation of the matching to sample task has been used to investigate behavior said to reflect cognition and memory– Here, the sample stimulus is turned off before the

comparisons are presented

– This time period is known as the retention interval

– Theoretically, organism is covertly doing something that helps to retain the information about the sample

158

• Traditional cognitive psychology talks about the internal rules defining what things “go together” and why

• Behavior analysts refer to concepts as “sets of stimuli that occasion a common response”

• Those “sets of stimuli” can be incredibly complex such as exemplars of “immoral conduct”

• The “common response” can also be complex

Concept Formation

159

• Behavior analysts identify the characteristics of the stimuli in the “concept” that controls the behavior

• They also examine how certain kinds of training (discrimination training, programming for generalization) affect the likelihood of stimulus classes (see work of Reeve, haha!)

Concept = stimulus class

160

• PERCEPTUAL CLASS = stimuli in the set share some physical characteristics

• RELATIONAL CLASS = stimuli in the set share some abstract relationship, such as examples of “bigger than”.

• EQUIVALENCE CLASS = stimuli do NOT share any physical characteristics. Stimuli are instead taught to belong together just because society says so.– Example: 1 = one = spoken word “WUN”

stimulus class types

161

Non-Human Studies on Natural Concept Formation

• Herrnstein - Pigeons– Pigeons can learn to form concepts of a person as

discriminated from non-persons, i.e., statues, mannequins, etc., a specific person from others, etc. The abstract stimulus class of “person” readily learned.

– Pigeons can learn to form natural concepts of persons, trees, fish, etc., with a high degree of accuracy. Artificial concepts such as buildings also learned.

Pigeons appear to respond to two-dimensions objects (photographs) as representations of three dimensional objects

162

Stimulus Control andLife

• Reading

• Studying

• Sleeping (Insomnia)

• Writing

163

Stimulus Control and Multiple Schedules

• Behavior analysts often use multiple schedules of reinforcement to study stimulus control in the laboratory.

• On a multiple schedule, two or more simple schedules are presented one after the other and each schedule is accompanied by a distinctive stimulus.

164

Behavioral Contrast

• Occurs within subjects

• Operant responding is the dependent variable

• Measured on multiple schedules of reinforcement– Two or more simple schedules that alternate– Subject does not get to “choose” on which

schedule to respond

165

Behavioral Contrast (cont.)

• Negative Behavioral Contrast– A decrease in response rate in one component of

a multiple schedule of reinforcement because the conditions of reinforcement in another component have gotten better

• Positive Behavioral Contrast– An increase in response rate in one component of

a multiple schedule because the conditions of reinforcement in another component have gotten worse

166

The Moral Behind Contrast Effects

• Contrast effects are instrumental in demonstrating that reinforcement in other situations can alter present behavior– Even when the other reinforcement is currently

available

167

End of notes for MIDTERM exam

Documents

1 PS 526 BEHAVIOR ANALYSIS AND LEARNING CALDWELL COLLEGE GRADUATE PROGRAM IN APPLIED BEHAVIOR ANALYSIS (ABA) DR. KEN REEVE