View
279
Download
0
Category
Tags:
Preview:
Citation preview
Paradoxical Effects of Reward
• Overtraining extinction effect: more training leads to faster extinction
• Reinforcement magnitude effect: Big rewards lead to faster extinction
• And, of course, the partial reinforcement extinction effect (PREE)
Paradoxical effects of reward: Why?
• Discrimination hypothesis: Nonreinforcement is easier to detect after CRF than PRF.
Discrimination Hypothesis: Test
CRF CRF EXT
vs.
PRF CRF EXT
Paradoxical effects of reward: Why?
• Discrimination hypothesis: Nonreinforcement is easier to detect after CRF than PRF.
• Frustration hypothesis (Amsel): animals learn to make response as a reaction to nonreward.
• Sequential theory (Capaldi): The memory of nonreinforcement becomes a cue that elicits responding.
Stimulus Control
Stimulus Control of Behavior
• Having stimulus control means that the probability of the behavior varies depending upon the stimuli present
• Most of our behavior is under stimulus control– A person that contributes to charity generously while in
church may watch every penny spent while at work
Discrimination
0
0.2
0.4
0.6
0.8
1
1.2
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
Trials
Re
sp
on
se
Str
en
gth
CS+
CS-
Discrimination and Stimulus Control• Discrimination is demonstrated when differential
responding occurs to two or more stimuli.
Train
Test
Reynolds (1961)
Generalization
• Generalization is when responses to one stimulus occur to other, usually similar, stimuli
• Generally, as the training and test stimuli become more different responding will decline, producing what is called a generalization gradient
Generalization GradientGuttman & Kalish (1956)
0
50
100
150
200
250
300
350
510
530
550
570
590
610
630
Wavelength (nanometers)
Res
pons
es
– pigeons reinforced for pecking a 580 nm lit key (orange-yellow) (S+) on a VI schedule
– A test session was then given where many different colored key lights were presented in extinction
S+
Interpreting Generalization Gradients
Pigeons trained to peck a moderately bright light (S+) to get food.
(S- = dim light)
After asymptote is reached, present occasional non-reinforced probe trials at various wavelengths or levels of brightness.
Excitatory andinhibitory gradients
Pigeons trained to peck at a 800 hz tone (S+), with a 500 nm light S-.
1000 Hz Tone always on
1000 Hz Tone S+ / No Tone S-
1000 Hz Tone S+ / 950 Hz Tone S-
Control group: 550 nm Light S+Experimental group: 550 nm Light S+ / 590 nm Light S-
0
100
200
300
400
500
480 500 520 540 560 580 600 620
Wavelength (nanometers)
Res
pons
es
ControlExperimental
Peak Shift Effect – Hanson (1959)
Spence’s Theory to Account for Peak Shift
-60
-40
-20
0
20
40
60
80
490 510 530 550 570 590 610 630 650 670Wavelength (nanometers)
Inh
ibit
ory
or E
xcit
ator
y S
tren
gth Inhibitory
ExcitatoryDifference
S+
S-
Interdimensional discrimination
0
100
200
300
400
500
600
700
501 530 555 576 606
Wavelength
# R
espo
nses
PseudodiscrimDiscrimination
Discrimination: S+ = 555nm Light; S- = Tone
How do we learn discriminations with complex stimuli?
How do we learn discriminations with complex stimuli?
A BA+B
Complex Discrimination: Example
Pre-exposure Devalue Test
-- Saline-LemonLiCL Sucrose-Lemon?
Lemon Saline-LemonLiCL Sucrose-Lemon?
Another example…
Pre-exposure Devalue Test
-- Saline-LemonLiCL Sucrose-Lemon?
Sal-L/Suc-L Saline-LemonLiCL Sucrose-Lemon?
Complex Discriminations: Mechanism # 2
the method of pre-exposure matters…
Mondragon & Hall (2002)
A = lemon B = salt C = sucrose X = quinine
Pre-Exposure: AXBXAXBX | CXCXCXCX
Devalue: AXLiCL
Test: BX? CX?
Question: How much does aversion generalize to BX and CX?
What’s going on?
• Juxtaposition of stimuli clearly matters
• But why? AXAXAX… produces habituation to AX
AXBXAXBX….
A
Remember: expected things are less salient or associable
B
Treating Different Stimuli Alike: Categorization
“Categorization can be viewed as the ability to treat similar, but not identical, things as somehow equivalent, by sorting them into their proper categories and by reacting to them in the same manner” (Huber, 2001)
• Classical view: categories united by a defining feature or features
• But Consider: Oak leaves v. Non-oak leavesChairs v. non chairs
Categorization Experiments
Train Test
Scenes with Trees + New Set tree scenese
Scenes w/o trees - New Set of no-tree scenes
"A pigeon pecks rapidly at a small photograph of Harvard Yard containing trees, buildings, people, sky. After a few seconds, a hopper of grain appears and the pigeon eats. Now the scene changes to a treeless Manhattan street. The bird emits a few desultory pecks, then turns away and paces about. After a minute or so, a picture of a leafy suburban garden appears and the bird begins pecking again." (Shettleworth 1998)
Other categories pigeons can form•Aerial v. non-aerial photos
•Chairs
•Humans
•Cars
•Defective pharmaceutical capsules!
•Oak leaves versus other leaves
Human v. Non-Human
How do they do it?
• Exemplar theory: remember category members and then generalize.– Vaughn & Greene 1984: pigeons can remember
no less than 320 individual slides! Outdoor scenes randomly assigned to + or –
Exemplar theory: more evidence
• Cook (1990)– Birds versus Mammals used in slides
– Real Category Group: Birds v. Mammals
– Pseudocategory Group: Random Bird & Mammals versus Random Birds & Mammals
Feature Theory
• Individual features acquire associative value.
• Response rate to stimulus depends on total expectancy (V) evoked.
Feature Theory: Evidence
Cerella (1980): Train: Charlie Brown +, other characters –
Test: Keep all features intact, but alter whole
Prototype theory
• Abstract the “ideal” (or average) category exemplar.
• To test: train with only extreme exemplars, test with average of extremes.
Prototype Theory in Humans
Posner & Keele 1968
Conclusions:
• Not clear whether birds can extract abstract concepts in categorization experiments
• Birds may use features and exemplars
• Another animals may be capable of more complex feats.
Recommended