580.691 Learning Theory

Reza Shadmehr

Neural mechanisms of classification

Generalization in linear classification

Patient H.M.

27 year old assembly line worker who had suffered from untreatable and debilitating temporal lobe seizures for many years. Surgeon removed medial portion of the temporal lobes bilaterally (only right lobe’s removal is shown on the figure on the right).

H.M.’s seizures were improved, but there was a devastating side effect: he could no longer form long-term memories.

R. C

arter (1998) Mapping the M

ind

Kandel et al. P

rinciples of Neural S

cience 2000 (62-1)

Patient H.M.

• After recovery from surgery, he maintained his vocabulary and language skills,

maintained his high IQ, and ability to recall facts about his life that preceded the surgery:

• could remember job that he held, where he had lived, and events of childhood. His

memory of public and personal events extend only to when he was 16 years old

(1942), 11 years before his operation. This is not typical of an amnesic individual,

who generally remember facts and events up to near the date of their brain damage.

• normal immediate memory: he can retain a number for a short period of time. He

can carry on a conversation.

• could not recognize people that he had talked to just the day before at the hospital.

He does not know where he lives, who cares for him, or what he ate at his last meal.

• He rarely complains. There could be something seriously wrong with him, but you

would have to guess. At the nursing home, when H.M. is observed to be acting

differently, the nurses question him by running through a list of possible complaints,

such as toothache, headache, stomachache, until they hit upon the correct one. He

will not spontaneously say that “I have a toothache”.

Corkin, Seminars in Neurology 4:249-259 1984.

Immediate memory is intact in amnesia

• Subjects with medial temporal lobe damage and normal individuals were read a

sequence of digits (for example, 5-7-4-1) and then asked immediately to repeat back the

sequence.

• Each time the subject was successful, the number of digits in the test sequence was

increased by one.

• Digit span: the number of digits that was successfully repeated back before a subject

failed twice at the same sequence.

• The amnesic patients and the control subjects both repeated back an average of 6.8

digits.

Cave and Squire

Delayed paired-comparison task.

Clicks, flashes, tones, or hues were presented and then some seconds later, the same or another cue was presented and the subject was asked to determine whether the two stimuli were the same or different.

Average performance of H.M.

Delayed recall in H.M. became severely impaired within 1 minute

Source: Brenda Milner

Mirror tracing task in H.M.

While viewing hand in mirror, H.M. tries to trace between the two lines. Number of errors refers to times that the border was crossed.

Kandel et al. Principles of Neural Science 2000 (62-2)

• Could learn to do mirror writing: performance would

improve with practice and remain good on next day,

despite no conscious recall of prior practice.

Lesions of the temporal lobe appear to affect forms of

learning and memory that require a conscious record, and

are called declarative memories.

Non-declarative memory is expressed through performance rather than recollection.

Squire (2004) Neurobiology of Learning and Memory

Memory systems of the brain

( ) ( ) ( )1

( ) ( ) ( )

( )

( )

( 1) ( ) ( ) ( )

( ) ( ) ( ) ( )

1

11

1 exp

1

1

Tn n n

f m f

n n n

T n

n

n n n nTn n n n

g g

P y q

y qq q

g x x x

g xw g x

g xw w

g x g x

1

1

( ) ( ) ( )

( 1) ( ) ( ) ( )

( ) ( )

1

0,1

11

1 exp

1

1

T

f f

T

f

n n n

iT

nn n n n

n T nn n

x x

x x

y

P y q

y qq q

x

x

xw x

xw w

x x

Review of online linear classification

Linear classification with linear encoding of feature space

Linear classification with non-linear encoding of feature space

Knowlton et al. (1996) “A neo-striatal habit learning system in humans” Science 273:1399

Task: Individuals learned to predict which of two outcomes would occur on each trial, given the particular cue that appeared.

x p x 1P s x

1

1

4

1

1

1

1

1 1 0,1 1 1 ?

1 0

0 1 1

1 1

0 1

1 1 11

0 0 1 10

ii

ii

ii

ii

i i

i i

i i

xxi ii

xxi ii

xxi ii

ii i

xxi ii

ii

x P x sx P s P s

P x sx

P x s

p x s

p x s

p x s P sP s x

p x p x

p x s P sP s x

p x

x x

1

1

1 1

0 1 1

ii

ii

i

xxi ii

xxi ii

p x

P s x

P s x

Setting up the Knowlton et al. (1996) task in on-line learning

Let’s begin with the simpler problem of observing only one cue. We want to know the probability of sunshine, given that the one cue was observed.

1log log log 1 log 1 log 1

0

log 1 log 1 log 1

1exp

0

1 exp 0

exp 1 1

exp

1 exp

11

1 exp

ii i i i

i

i i i i

i i

ii i

i

i i i i

i i i

i i

i i

ii i

P s xx x

P s x

x x

w x c

P s xw x c

P s x

P s x w x c P s x

w x c P s x

w x c

w x c

P s xw x c

11

11

11

11

, 1 1 1

1 1

, 0 1 1

, 1 11 ,

0 , , 0 1

1 1

1 1 1

1 ,log

0 ,

jjii

jjii

jjii

jjii

i j i j

xxxxi ji j

xxxxi j i ji j

i ji j

i j i j

xxxxi ji j

xxxxi ji j

i j

i j

p x x s p x s p x s

p x x s

p x x s P sP s x x

P s x x p x x s P s

P s x x

P s x x

11 ,

1 exp

11

1 exp

i i j j

i ji i j j

T

w x w x c

P s x xw x w x c

P sc

xw x

Therefore, the weather forecasting task is linear classification in the feature space of the cards.

PD-star represents the PD patients with the most severe symptoms. PD also involves damage to the frontal lobe. They tested frontal patients and found that they were normal in learning the classification problem. When PD patients were tested on an additional 100 trials, their performance was now comparable to control subjects. This was a little puzzling.

Parkinson patients were impaired in learning the classification task, while amnesic patients were normal

Similar to PD patients, Huntington’s disease patients exhibited impaired ability to learn the weather prediction task. (Knowlton et al., Dissociations within nondeclarative memory in Huntington’s disease, Neuropsychology 10 (1996) 538–548.

After completing the task, subjects were given eight multiple-choice questions to determine how well they remembered the testing situation. These questions asked, for example, about the layout of the screen, the number of cards that could appear together on the computer screen, the number of weather prediction trials presented, and the appearance of the cues.

Medial temporal lobe structures damaged in Amnesic patients appear to support acquisition of “declarative” memory of the training episode. In contrast, basal ganglia structures damaged in Parkinson’s disease appear to support acquisition of internal models for classification.

cerebellar damage

Witt et al. (2002) Dissociation of Habit-Learning in Parkinson's and Cerebellar Disease. J. Cognitive Neurosci 14:493

Eldridge et al. (2002) Intact Implicit Habit Learning in Alzheimer's Disease. Behavioral Neurosci 116:735

Alzheimer’s diseasecontrol

Parkinson’s disease

In the post-experiment interview (explicit memory component), recall of AD patients did not differ from chance.

Brief notes on Alzheimer’s disease: In early stages of the disease, there is neurodegeneration in the medial temporal lobes, similar to damage observed in amnesic patients. In later stages, neuronal loss extends to the neocortex.

Poldrack et al. (2001) Interactive memory systems in the human brain. Nature 414:546

A “block” design: one group of subjects performed the FB task (and the baseline task), while another performed the PA task (and the baseline task). Classification ability at end of training was similar for the two groups.

Between subject contrast: PA vs. FB

The FB task requires that you first select the class, and then you are provided with an error signal regarding your choice. In the PA task, there is no explicit error signal because no choices are made.

Poldrack et al. (2001) Interactive memory systems in the human brain. Nature 414:546

Activity in caudate Activity in hippocampus

Plot shows activity (with respect to baseline) in an event related design during the feedback-learning task. Initially, as the task is performed there is increased activity in the hippocampus and decreased activity in the caudate. With further training, the caudate activity increases and the hippocampus activity declines. This suggests there may be a competition between these two memory systems in the brain.

Prototype Low distortion

High distortion Random

Study items Test items

Per

cen

t co

rrec

t40 examples were generated from a prototype and studied. Subjects were instructed that all examples belonged to the same category. Five minutes later, performance was measured on 84 new examples generated from the same prototype. Subjects were asked “does this belong to the same category?”

Proto

type

Low distorti

on

High distorti

on

Random

Control

Amnesic

Generalization properties of classifiers

Knowlton and Squire (1993) The learning of categories: parallel brain systems for item memory and category knowledge. Science 262:1747.

( ) ( ) ( )

( ) ( )

( )

( ) ( ) ( )

( ) ( ) ( )

( )

( 1) ( ) ( )

( ) ( ) ( ) ( )

( )

( )

( ) ( )

( 1)

11 ,

1 exp

1 ,

1 , 1 , 1

0 , 1 1 , exp

1

,

1

exp

T

n n n

n n

n

n n n T

n n n

n

n n nTn n n n

Tn

nTn n

n

P y

q P y

P y P yo

P y P y

y y q

yq q

b

o

x ww g x

x w

x w x wx

x w x w w g x

g xw w

g x g x

g x g xx x

g x g x

x

w

( ) ( ) ( )

( ) ( )

( 1) ( ) ( ) ( )

( ) ( )

,1

exp ,1

n T n n

n n

n n n n

n n

y bq q

o o y bq q

g x x x

x x x x

A generalization function for a linear classifier: system identification

“odds”

Error experienced in trial n

Generalization function

( )

( ) ( )

( )

( 1) ( ) ( ) ( )

( ) ( )

( 1) ( ) ( ) ( )

( ) ( )

1 ,log log

0 ,

log log ,1

,1

n

n n

n

n n n n

n n

n n n n

n n

P yz o

P y

o o y bq q

z z y bq q

x wx x

x w

x x x x

x x x x

A generalization function for a linear classifier: system identification

“State” of the learner: log of the odds

State transition equation

Error in trial n

Generalization function

Input where error was experienced

mean+/-SD

Early in training After 300 trials Catch Trial

Sh

adm

ehr,

Bra

nd

t &

Co

rkin

, J N

euro

ph

ysio

l 199

8

Cerebellar patients Huntington’s Disease patients

Training set (bin=100 trials)

Smith and Shadmehr (2005) Intact ability to learn internal models of arm dynamics in Huntington’s disease but not cerebellar degeneration. J. Neurophysiology