Information in “Associative” Learning

C. R. GallistelRutgers Center for Cognitive Science

Sloan-Swartz 7/22/08 2

Temporal Pairing

• Thought to be essential for the formation of associations

• Assumed to be the critical variable in work on neurobiology of learning (LTP)

• Basis of unsupervised learning in neural net models

Sloan-Swartz 7/22/08 3

But

• It’s never been objectively defined for any paradigm: What is the critical interval?

• Neither necessary nor sufficient for development of a conditioned response to the CS (the warning signal)

Sloan-Swartz 7/22/08 4

Not Necessary

• Subjects develop a conditioned response to a CS that is never paired with the US (the predicted event)--conditioned inhibition

• Pavlov and Hull struggled with this problem

• It has not been solved

Sloan-Swartz 7/22/08 5

Not Sufficient

• The truly random control (Rescorla, 1968)– It is the mutual information between CS & US that is

critical

– Not their temporal pairing

Sloan-Swartz 7/22/08 6

It’s Information!

• People believe in “temporal pairing” because they are intuitively sensitive to the fact that a relatively more proximal warning gives more information

• It’s the information that matters, not the temporal pairing

Sloan-Swartz 7/22/08 7

Information Derives From Temporal Representation

• Information-theoretic analysis explains BOTH cue competition AND the data on the temporal pairing

• Founded on the assumption that animals learn the intervals

• AND, they represent the uncertainty with which they can remember them (about +/- 15%)

Sloan-Swartz 7/22/08 8

Principles I

• Subjects respond only to stimuli (CSs) that provide information about the timing of future events (USs)

• CSs inform to the extent they change the subject’s uncertainty about the time to the next US

Sloan-Swartz 7/22/08 9

Principles II

• Bandwidth maximization by minimizing number of information-carrying CSs attended to

• Information carried by intervals and numbers

• They are what is learned

• Weber’s law: uncertainty scales with delay: =wT

Sloan-Swartz 7/22/08 10

Rate-Change Protocols

€

˙ H = λ log2

e

λΔτ

⎛

⎝ ⎜

⎞

⎠ ⎟

€

H =1

λλ log 2

e

λΔτ

⎛

⎝ ⎜

⎞

⎠ ⎟= k − log 2 λ

Hb −Hcs = k−log2 λb( )− k−log2 λcs( ) =log2 λcs −log2 λb

Information communicated by CS log2

λcs

λb

⎛

⎝⎜⎞

⎠⎟=log2

Ius-usIus-us|cs

⎛

⎝⎜⎞

⎠⎟

Sloan-Swartz 7/22/08 11

Delay Protocols

• They are additive

• Only one depends on protocol parameters

H =log2

λcs

λb

⎛

⎝⎜⎞

⎠⎟+ k λcs =1 T

€

k =1

2log2

e

2π

⎛

⎝ ⎜

⎞

⎠ ⎟− log2 w

• Two sources of information:

1) The rate change 2) The fixed delay

Sloan-Swartz 7/22/08 12

Gibbon & Balsam

• Reinforcements to acquisition, as a function of the

Ius-us/Ics-us ratio

• Slope (log-log) ~ -1

Sloan-Swartz 7/22/08 13

Trials Don’t Matter

• These two protocols are equi-effective!• The number of trials is not in and of itself a

learning-relevant parameter of a training protocol• Gottlieb (2008)

Sloan-Swartz 7/22/08 14

Associability

• where Ncs-us = the number of CS reinforcements required to produce an anticipatory response.

(The onset of conditioned responding is abrupt)

• Definition parallels definition of sensitivity (1/Intensity) in sensory psychophysics

• Purely operational: no implication that associations exist

A =1 / Ncs-us

Sloan-Swartz 7/22/08 15

Informativeness

• We define the ratio of the background rate to the rate in presence of CS to be the informativeness of the CS-US relation in an associative learning protocol

• Thus, the information conveyed is the log of the informativeness

Sloan-Swartz 7/22/08 16

A Simple Quantitative Law

€

Associabilty ∝ Informativeness

A ∝λcs

λb=

IUS-USIUS-US|CS

Sloan-Swartz 7/22/08 17

Why trials don’t matter

• When there are 8 times fewer trials,• the trials are 8 times more informative• Provided one maintains total protocol duration• The only way to speed up learning is to increase

informativeness of the CS-US relation.• Adding trials won’t do it!

Sloan-Swartz 7/22/08 18

Conclusion 1

• Temporal pairing is–Undefinable

–Insufficient

–Unnecessary

• “Trials” are a pernicious fiction. Banish them from your models

Sloan-Swartz 7/22/08 19

Conclusion 2

• What matters is the mutual information (between CS and US), a component of which is the change in US rate when the CS comes on

• The informativeness of the CS-US relation is the factor by which CS onset changes the expected time to the next US

• Associability is proportional to informativeness

• That’s why people believe in in temporal pairing

Sloan-Swartz 7/22/08 20

Conclusions 3

• Focus on mutual information gives an empirically supported quantitative account of the notion of temporal pairing

• And an account of “cue competition:” how the system solves the multivariate prediction problem (aka the assignment-of-credit problem; what is predicting what), the other problem posed by Rescorla’s experiment

Sloan-Swartz 7/22/08 21

Thank You

• Collaborators– The late John Gibbon– Peter Balsam– Stephen Fairhurst– Daniel Gottlieb

• Support– RO1 MH68073 Time and Associative Learning