If behavior was dominated in the past by Hull’s S-R reinforcement paradigm, what

paradigm is it dominated by today?• There is a causal relationship

between actions and outcomes

• There is some degree of control over actions according to the anticipation of, and desire for, the outcome

• The circuits in the BG allow for control and expression of this paradigm (i.e. via the A-O and S-R systems later discussed)

Edward Thorndike

Clark L. Hull

Explain the two important contributors to behavior: the remembered value of the expected outcome, and the

knowledge of the causal relationship between the action and the outcome.

• “remembered value of the expected outcome”– We can learn the reward of an action and

respond/act accordingly• “knowledge of the causal relationship

between the action and the outcome”– We know that there is a relationship between how

we act and what the outcome will be.– This allows us to place value on certain actions and

the rewards we receive from them

Explain the two important contributors to behavior: the remembered value of the expected outcome, and the

knowledge of the causal relationship between the action and the outcome.

• “remembered value of the expected outcome”– Our sensitivity to manipulations of outcome value

determines whether an action is habitual or goal-directed

• “knowledge of the causal relationship between the action and the outcome”– This acknowledges the intentionality of behavior

itself

Explain the experimental paradigms used to study these two psychological functions.

• The value of the outcome is increased(inflated) or decreased(devalued).– Devaluation is far more common because it is

easier to reduced the value of a reward.– If performance changes based on outcome value,

then the behavior is controlled by the anticipation of the outcome (goal directed action)

Explain the experimental paradigms used to study these two psychological functions.

• The Action-outcome (A-O) contingency is manipulated. (outcome depends on action)– Often done through contingency degradation

• Introduces free rewards that are independent of any action

– Instrumental contingency can be viewed as the probability of reward given a particular action relative to the probability of a reward given no action.• If the probabilities are the same, then the contingency is

said to be completely degraded.

The A-O and S-R Systemsand Habit Formation

• A-O : Action-Outcome• S-R: Stimulus-Response• Both engaged under different conditions• Amount of training or number of rewarded

responses plays significant role in shift between systems-habit formation

• Promoted by overtraining• Schedule of reinforcement also plays a role

14. What are interval and ratio schedules?• Ratio Schedules:

– Response results in a certain probability of a reward (more responses= more rewards)

– Produce goal directed actions• Controlled by A-O contingency• A rat knows that a specific outcome

will produce a reward, so the rat will direct it’s actions to reach that outcome

• Interval Schedules:– A response is rewarded only after

a certain time interval has passed – Produce stimulus response (S-R)

habits

14. How do they relate to research on BG function?

• Consensus that A-O and S-R systems control instrumental behaviors– Engaged in different conditions

• Amount of training determines the shift from A-O to S-R (habit formation)

• Use of interval schedules in these studies explains why they failed to find evidence for A-O learning– Even when reinforcement was equated, interval

schedules produced habits, and ratio schedules did not

14. How do they relate to research on BG function?

• So…– DMS, specifically the pDMS (posterior

dorsomedial striatum) is involved in goal directed behavior (A-O learning)

– DLS is involved in habit formation (S-R learning)