Operant Conditioning

• Action results in consequence– Decision

• B.F. Skinner

• Three term contingency– Stimulus - response - outcome

• Outcomes– Positive/negative; reinforcement/punishment

Differences from Classical

• Usually assumed to be under “conscious” control

• Operant conditioned after the behaviour– Outcome feeds back to alter response

Probabilities

• Hierarchy of behaviours (actions)– Probability

• Operant conditioning changes events and/or consequences

• Results in adjustment of probability hierarchy

Shaping

• Directed learning

• Behavioural outcome more certain– Select a specific response to occur in a specific

way

• Gradual process

• Chaining– Forward and backward

Reinforcement Schedules

• Fixed ratio

• Fixed interval

• Variable ratio

• Variable interval

Reinforcement Consistency

• Continuous schedules

• Intermittent schedules

• Response-reinforcer?– Technically, only FR-1 is continuous

• Systematic reinforcer– Any fixed schedule (FR or FI)

Extinction

• Response - outcome pattern disrupted

• Easiest for:– Continuous reinforcement/punishment– Low schedules

• Variable ratio schedules hardest to extinguish

Reinforcers

• Primary

• Secondary

• Speed of learning

• Extinction

• Money

Delay

• Immediate reinforcement

• Delayed reinforcement

• Generally, delayed harder to condition– Difference with well conditioned system

Changing Schedules

• Cost of response

• Contingency– Rate of reinforcement

• Modification– Decrease– Increase

• Delay

Applications• Discriminative stimulus

– “Blue-light special,” coloured sale tags, logos (if previous positive experience with product)

– Christmas music in October

• Positive reinforcement– Give-aways, purchase points, Canadian Tire

money, parking lot barbeques

• Negative reinforcement– Purchase to avoid pushy sales pitch

• VR schedule– Lotteries, door prizes, etc.

• Shaping– Free trial periods, leading signs/displays (get

person into store area)

• Punishment– Unusual in advertising/marketing– Commercials showing customer who suffers

from not using product (observational learning)

Ecological Design

• Structuring the environment

• Facilitation of particular behaviour(s)– Increase/decrease probability of response

• Store layout, purchase locations, noises, odors, lighting

• A type of shaping of a response

Behaviour Modification

• Application of operant theory to change behaviour

• Primary application of operant principles

• Skinner’s behaviour analyst techniques

Behav. Mod. in Marketing

• Role of marketing as influencing, modifying, and controlling consumer behaviour to achieve purchasing objectives

• An applied field– Not aimed at developing theory, but applying

theory

• Observable behaviour– No inferred behavioural constructs

Economic Psychology

• Integration of psychology and economic analysis

• Marketing– Not a discipline– An application area for the social sciences and

other disciplines

• EcPsyc offers detailed analysis of consumer-firm interactions

Behavioural Perspective Model

• Gordon R. Foxall

• Operant behaviourist paradigm

• Modern marketing firms– Embedded in networks of marketing relationships– Extra-firm environment (e.g., consumers) drive

marketing behaviour– Reinforcement/punishment shift firm’s behaviour

• Applies also to behaviour of individuals comprising the firm (e.g., employees, owners, shareholders, etc.)

A Firm’s Purpose

• To make marketing relationships more economic– Production and selling are independent of firms

• Don’t need firms to do these

– Creation and maintenance of marketing is what firms do

Uhm… So?

• Operant conditioning theory and firms• Economic behaviour is instrumentally

conditioned• Behaviour that operates on the environment

to produce consequences changes the future rates of behaviour

• Reinforcement/punishment shifts economic (market) factors

Consumer Behaviour

• Economic purchasing and consumption activities

• Basic three-term contingency applies– Stimulus - response - outcome

• Plus, consumer behavioural setting and learning history

Behav. Persp. Model and Consumer’s Choice

• Consumer choice reduces aversive consequences of facing multiple equivalent options

Model

Consumerbehaviour setting

Consumer’s learning history

Consumerbehaviour

Aversiveconsequences

Utilitarianreinforcement

Informationalreinforcement

Marketing Management in BPM

• Influence two factors

• Consumer behaviour settings– Social, physical, temporal, and regulatory

discriminative stimuli

• Utilitarian and information reinforcers– Actual outcome and knowledge gained

Managing Reinforcers

• Three ways

• Enhancing effectiveness of reinforcers

• Controlling the schedules of reinforcer delivery

• Increasing the quantity or quality of reinforcers

Complexities

• Multiple systems operating simultaneously

• Is operant conditioning separable from classical?

• Do stimuli fulfill role of CS, SD, or both?

Role of Operant Reinforcer in Classical Conditioning

• In classical conditioning– US presented regardless of CR– Defining feature– But, operant reinforcement can slip in

• Operant reinforcement via– 1. Reinforcing CR directly

• e.g., food (US) coming after CR

– 2. CR increases “value” of US• e.g., salivation (CR) makes swallowing food (US)

easier

Omission Control Procedure

• US presentation depends on occurrence of CR– CS presented; if no CR, US follows– CS presented; if CR, no US follows

• Therefore, US can’t operantly reinforce CR

Omission Control

CS

US

CR

Trial with a CR

CS

US

CR

Trial without a CR

Blocks of sessions%

of

CR

s

Omission control

Standard classical conditioning

Conclusion

• Can have classical conditioning without operant reinforcement

• But what about classical conditioning in operant conditioning?

Associative Structure in Operant Conditioning

• Basic form of association– S-R– S-O

• Pavlovian processes

• Can keep instrumental reinforcement out of classical conditioning, but not vise versa

Stimulus Outcome

Instrumental response

S-R, S-O, rg-sg

• Thorndike’s Law of Effect– Focus on S-R association

• Hull and Spence– Law of Effect plus a classical conditioning

process

• rg-sg

– Fractional anticipatory goal response; sensory feedback

Fractional Anticipatory Goal Response

• SD influences rg-sg (expectancy of reward from classical conditioning) through sensory substitution-like process

• Motivation

Stimulus rg sg

Response Outcome

Timeline

Prediction

• According to rg-sg– CR occurs before operant response– But, not always true– e.g., lever pressing and salivation– CR should occur before operant, but it doesn’t

Central Emotional State

• Classical conditioning in operant conditioning

• Not for learning response

• For CES (Central Emotional State)

• CES --> motivation, “mood”

Modern Two-Process Theory

• Classical in operant conditioning

• Neutral stimulus --> elicit motivation (CES)

• CES elicited by CS corresponds to US– CES a characteristic of CNS = “mood”

• CES doesn’t produce only one response– e.g., anger --> multiple responses

• CES conditioned during ordinary operant training– CES conditioned to situational cues or discriminative stimulus

– CES motivates operant behaviour

Prediction

• Rate of instrumental response will be modified by presentation of CS

• Consider– In operant conditioning, CES develops to

motivate operant response– CS from classical conditioning also elicits CES– Therefore, giving CS during operant

conditioning will alter CES that motivates/maintains operant response

Conditioned Emotional Response

• Suppression ratio

• CES elicited by CS --> decrease response

“Explicit” Predictions

• Emotional states

USCS Appetitive Aversive

(e.g., food) (e.g., shock)CS+ Hope FearCS- Disappointment Relief

• Behavioural predictions

Aversive US

Instrumental schedule CS+(fear) CS-(relief)

Positive reinforcement decrease increaseNegative reinforcement increase decrease