correct. - Stanford Universityjlmcc/papers/McClelland95_MemDist.pdfMemory Distortion Constructive...

Page 1

Mem

ory

Dis

tort

ion

Con

stru

ctiv

e M

emor

y an

d M

emor

yD

isto

rtio

ns: A

Par

alle

l-D

istr

ibut

edPr

oces

sing

App

roac

h

How

Min

ds, B

rain

s, a

nd S

ocie

ties

Rec

onst

ruct

the

Past

Edi

tor

Daniel L. Schacter

Contributing Editors

Joseph T. Coyle

Gerald D. Fischbach

Mar

ek-M

arse

l Mes

ulam

Law

renc

e E

. Sul

livan

Jam

es L

. McC

lella

nd

Bar

tlett

(193

2) in

trod

uced

and

insi

sted

on

the

view

that

mem

ory

isa

cons

truc

tive

proc

ess,

His

vie

w w

as e

ssen

tially

that

rec

all i

s no

t a r

etri

eval

,bu

t a r

econ

stru

ctio

n, in

whi

ch a

spec

ts o

f the

con

tent

of p

revi

ousl

y pr

esen

ted

mat

eria

l are

wov

en in

to a

coh

eren

t who

le, w

ith th

e ai

d of

pre

exis

ting

know

l-ed

ge. D

etai

ls m

ay b

e di

stor

ted

to in

crea

se c

oher

ence

; rat

iona

lizat

ions

not

pres

ent i

n th

e or

igin

al m

ay b

e in

trod

uced

; det

ails

that

are

con

sist

ent w

ithth

e sy

nthe

size

d co

here

nt s

tory

may

be

adde

d; a

nd d

etai

ls th

at a

re in

cons

is-

tent

may

be

drop

ped.

Nei

sser

(19

67)

liken

ed b

oth

perc

eptio

n an

d m

emor

yto

the

cons

truc

tive

activ

ities

of a

pal

eont

olog

ist,

who

use

s a

colle

ctio

n of

bone

fra

gmen

ts, a

s w

ell a

s ev

eryt

hing

she

kno

ws

abou

t din

osau

rs f

rom

pre

-vi

ous

expe

rien

ce, t

o re

cons

truc

t the

ske

leto

n of

a p

artic

ular

din

osau

r, T

hese

idea

s ar

e co

nsis

tent

with

wha

t we

wou

ld r

efer

to to

day

as a

con

stra

int s

atis

-faction process, i

n w

hich

rem

embe

ring

is s

imul

tane

ousl

y co

nstr

aine

d by

trac

es le

ft in

the

min

d by

the

even

t we

are

rem

embe

ring

itse

lf, b

y ba

ckgr

ound

know

ledg

e of

rel

ated

mat

eria

l, a

nd b

y co

nstr

aint

s an

d in

fluen

ces

impo

sed

by th

e si

tuat

ion

surr

ound

ing

the

act o

f rec

olle

ctio

n. O

bvio

usly

if m

emor

yis

con

stru

ctiv

e in

this

way

, thi

s ha

s pr

ofou

nd im

plic

atio

ns f

or th

e qu

estio

nof

the

veri

dica

lity

of m

emor

y an

d th

e ex

tent

to w

hich

it m

ay b

e in

flue

nced

by suggestion, p

reex

istin

g kn

owle

dge

, and

oth

er r

elat

ed e

xper

ienc

es.

My

inte

rest

is in

the

mec

hani

sms

that

may

impl

emen

t thi

s co

nstr

uctiv

eco

nstr

aint

sat

isfa

ctio

n pr

oces

s. R

emem

beri

ng, I

will

arg

ue, t

akes

pla

ce in

apa

ralle

l dis

trib

uted

pro

cess

ing

syst

em-a

.sy

stem

con

sist

ing

of a

larg

e nu

m-

ber

of s

impl

e bu

t mas

sive

ly in

terc

onne

cted

pro

cess

ing

units

, Pro

cess

ing

insu

ch s

yste

ms

take

s pl

ace

thro

ugh

the

prop

agat

ion

of a

ctiv

atio

n am

ong

the

units

, bas

ed o

n ex

cita

tory

and

inhi

bito

ry c

onne

ctio

ns. F

orm

ing

a m

emor

ytr

ace

for

som

ethi

ng-s

ay, a

n ep

isod

e or

eve

nt-b

egin

s w

ith th

e co

nstr

uctio

n

Har

vard

Uni

vers

ity P

ress

Cam

brid

ge, M

assa

chus

etts

Lon

don,

Eng

land

1995

Page 2

COGNITIVE PERSPECTIVES

of a pattern of activity over t~e pr

oces

sing

uni

ts, w

ith th

e ex

peri

ence

itse

lfst

rong

ly in

flue

ncin

g th

e pa

ttern

, But

the

exis

ting

conn

ectio

ns a

mon

g th

eun

its w

ill a

lso

infl

uenc

e th

e pa

ttern

con

stru

cted

, the

reby

intr

oduc

ing

the

poss

ibili

ty o

f ad

ditio

ns, o

mis

sion

s, a

nd d

isto

rtio

ns, S

tora

ge o

f a tr

ace

ofth

e ep

isod

e or

eve

nt th

en o

ccur

s th

roug

h th

e m

odif

icat

ion

of th

e st

reng

ths

of th

e co

nnec

tions

am

ong

the

units

; to

a fi

rst a

ppro

xim

atio

n, t

he c

onne

ctio

nfr

om a

uni

t tha

t is

activ

e in

the

repr

esen

tatio

n to

ano

ther

suc

h ac

tive

unit

will

tend

to in

crea

se in

str

engt

h, w

hile

the

stre

ngth

of

conn

ectio

ns b

etw

een

active and inactive units will tend to decrease,

Rem

embe

ring

may

occ

ur w

hen

som

e as

pect

or

aspe

cts

of a

n ev

ent a

rise

agai

n as

inpu

t. T

his

may

act

ivat

e so

me

of th

e un

its th

at p

revi

ousl

y pa

rtic

i-pa

ted

in th

e re

pres

enta

tion

of th

e ep

isod

e or

eve

nt, a

nd th

ese

may

in tu

rnac

tivat

e ot

her

units

, via

the

wei

ghte

d co

nnec

tions

, The

pat

tern

that

is c

on-

stru

cted

aga

in d

epen

ds o

n th

e co

nnec

tions

am

ong

the

units

, and

sin

ce th

ese

wer

e ad

just

ed p

revi

ousl

y w

hen

the

epis

ode

was

fir

st e

xper

ienc

ed, t

he p

atte

rn -

that

is c

onst

ruct

ed w

ill te

nd to

cor

resp

ond

to th

e pa

ttern

that

was

pre

sent

at th

e tim

e of

'storage. But the units that participate in representing one

epis

ode

or e

vent

als

o pa

rtic

ipat

e in

rep

rese

ntin

g ot

her

epis

odes

, and

so

the

repr

esen

t~tio

n th

at is

con

stru

cted

may

be

affe

cted

by

man

y ot

her

expe

ri-en

ces.

Thi

s m

eans

that

my

mem

ory

of a

nyon

e ep

isod

e or

eve

nt w

ill te

ndto

ref

lect

the

infl

uenc

e of

wha

t I h

ave

lear

ned

from

man

y ot

her

epis

odes

or

even

ts.

I w

ill d

escr

ibe

two

mod

els

that

cap

ture

this

con

stru

ctiv

e pr

oces

s in

dif

fer-

ent w

ays,

Bot

h m

odel

s ha

ve th

eir

orig

ins

in e

arly

con

nect

ioni

st p

aper

s, o

neby

mys

elf

(McC

lella

nd, 1

981)

and

one

by

Hin

ton

(198

1), N

eith

er m

odel

isfu

lly a

dequ

ate

in it

self

, but

I w

ill p

ropo

se a

syn

thes

is o

f th

e tw

o th

at m

ayca

ptur

e so

me

of th

e m

ain

feat

ures

of

hum

an m

emor

y, in

clud

ing

aspe

cts

ofm

emor

y di

stor

tions

. The

syn

thes

is m

ay p

rovi

de a

s w

ell o

ne w

ay o

f th

inki

ngabout amnesia.

A T

race

Syn

thes

is M

odel

The

fir

st m

odel

(M

cCle

lland

, 198

1) f

ocus

es o

n di

stor

tion

proc

esse

s th

at c

anoc

cur

duri

ng a

cts

of r

emem

beri

ng, u

sing

a s

impl

e, lo

calis

t con

nect

ioni

st n

et-

wor

k fo

r th

e st

orag

e an

d re

trie

val o

f in

form

atio

n, I

use

d in

the

exam

ple

the

task

of

rem

embe

ring

fac

ts a

bout

a c

olle

ctio

n of

two

som

ewha

t uns

avor

yin

divi

dual

s, b

elon

ging

to tw

o m

ade-

up g

angs

, the

Jet

s an

d th

e S

hark

s, T

he

Jets tended to be in their twenties, t

o be

sin

gle

, and

to h

ave

only

a J

unio

rH

igh

Scho

ol e

duca

tion,

thou

gh n

o on

e Je

t had

all

thes

e ch

arac

teri

stic

s; th

eSh

arks

tend

ed to

be

olde

r, to

be

mar

ried

, and

to h

ave

atte

nded

Hig

h S

choo

lth

ough

aga

in n

o on

e Sh

ark

had

all t

hese

pro

pert

ies,

Mem

bers

of

both

gan

gsw

ere

equa

lly li

kely

to b

e pu

sher

s, b

ooki

es, o

r bu

rgla

rs,

Con

stru

ctiv

e M

emor

y an

d M

emor

y D

isto

rtio

ns

I: .

Figu

re

1 T

he n

etw

ork

used

to d

emon

stra

te a

spec

ts o

f co

nstr

uctiv

e m

emor

y by

McC

lel-

land

(19

81),

The

uni

ts p

artic

ipat

ing

in th

e re

pres

enta

tion

of a

few

of

the

mem

bers

of

the

Jets

and

Sha

rks

gang

s de

scri

bed

in th

e te

xt a

re s

how

n, U

nits

with

in th

e sa

me

grou

p ar

em

utua

lly in

hibi

tory

; uni

ts c

onne

cted

with

bid

irec

tiona

l arr

ows

are

mut

ually

exc

itato

ry,

From

McC

lella

nd (

1981

), f

ig, 1

, p, 1

71.

In th

e m

odel

, I r

epre

sent

ed e

ach

indi

vidu

al w

ith it

s ow

n co

nnec

tioni

stpr

oces

sing

uni

t tha

t I w

ill c

all a

n in

stan

ce u

nit (

see

Figu

re 2

.1),

The

mod

elal

so c

onta

ined

pro

pert

y un

its, o

ne fo

r ea

ch p

rope

rty

an in

divi

dual

mig

htha

ve, N

ames

wer

e tr

eate

d as

pro

pert

ies ,

so

ther

e w

ere

units

for

nam

es, f

orga

ng m

embe

rshi

p, f

or e

duca

tion,

for

mar

ital s

tatu

s , a

nd fo

r oc

cupa

tion,

Bi-

dire

ctio

nal,

exc

itato

ry c

onne

ctio

ns b

etw

een

units

wer

e us

ed to

link

inst

ance

units to the units representing their properties, s

o th

at if

one

activated a

nam

e un

it, i

t act

ivat

ed th

e co

rres

pond

ing

inst

ance

uni

t , and the instance

unit

activ

ated

the

othe

r pr

oper

ties

of th

e in

stan

ce. T

he in

stan

ce u

nits

for

med

a gr

oup

of u

nits

that

wer

e m

utua

lly in

hibi

tory

, so

that

if o

ne w

as a

ctiv

e it

tend

ed to

sup

pres

s th

e ot

hers

; sim

ilarl

y, th

e un

its f

or e

ach

type

of

prop

erty

wer

e gr

oupe

d in

to c

lust

ers

of m

utua

lly in

hibi

tory

uni

ts. T

he u

se o

f si

ngle

units

to r

epre

sent

who

le it

ems

is a

sim

plif

icat

ion-

I w

ill a

rgue

late

r th

at it

Page 3

CO

GN

ITIV

E P

ER

SPE

CT

IVE

S

is m

ore

corr

ect t

o us

e di

stri

bute

d re

pres

enta

tions

, bot

h fo

r th

e w

hole

and

for

the

part

s, H

owev

er, e

ven

this

sim

ple

mod

el c

aptu

res

cruc

ial a

spec

ts o

fth

e ki

nds

of r

econ

stru

ctiv

e pr

oces

ses

that

take

pla

ce d

urin

g re

mem

beri

ng,

My

focu

s in

the

orig

inal

wor

k re

port

ed in

McC

lella

nd (

1981

) w

as o

n th

epr

oces

s of

con

stru

ctin

g re

pres

enta

tions

of

mat

eria

l not

exp

licitl

y st

ored

inm

emor

y, O

ne s

uch

case

invo

lved

con

stru

ctin

g a

com

posi

te r

ecol

lect

ion

ofth

e ty

pica

l Jet

or

Shar

k. I

n th

e m

odel

this

cou

ld b

e do

ne s

impl

y by

act

ivat

ing

the

unit

for

Jet,

Thi

s un

it th

en s

ent a

ctiv

atio

n to

the

inst

ance

uni

ts f

or a

llof the Jets, a

nd th

ese

in tu

rn s

ent a

ctiv

atio

n to

the

units

for

eac

h in

stan

cepr

oper

ties,

The

inhi

bitio

n am

ong

the

inst

ance

uni

ts p

reve

nted

any

of

thes

eun

its f

rom

bec

omin

g to

o st

rong

ly a

ctiv

ated

, but

they

all

cont

ribu

ted

som

eac

tivat

ion

to th

eir

prop

erty

uni

ts, A

s a

resu

lt th

e pr

oper

ties

of th

e ty

pica

lJe

t bec

ame

activ

e (a

ge in

20'

, sin

gle

, JH

edu

catio

n); a

ll of

the

occu

patio

nsw

ere

part

ially

act

ivat

ed.

Thi

s fi

rst e

xam

ple

show

s a

desi

rabl

e pr

oper

ty o

f th

is s

ort o

f m

emor

y sy

s-te

m-i

t can

spo

ntan

eous

ly g

ener

aliz

e fr

om e

xam

ples

. Ano

ther

pro

pert

y-w

hich

may

ofte

n be

des

irabl

e bu

t whi

ch c

an a

lso

be u

ndes

irabl

e-is

rev

eale

d

whe

n th

e m

odel

is u

sed

to tr

y to

ret

riev

e th

e pr

oper

ties

of a

sin

gle

indi

vidu

alby

act

ivat

ing

the

unit

for

that

indi

vidu

al's

nam

e. T

his

indi

vidu

al's

pro

pert

ies

tend

to b

e m

ore

stro

ngly

act

ivat

ed th

an th

e pr

oper

ties

of a

ny o

ther

indi

vid-

ual,

but

one

fin

ds th

at a

s th

e ac

tivat

ion

proc

ess

goes

on,

oth

er, s

imila

r in

di-

vidu

als

beco

me

part

ially

act

ivat

ed, T

his

happ

ens

beca

use

as th

e pr

oper

ties

of th

e ta

rget

indi

vidu

al b

ecom

e ac

tive,

they

sen

d ac

tivat

ion

to th

e in

stan

ceun

its fo

r ot

her

indi

vidu

als

, and

thes

e, in

turn

, ten

d to

act

ivat

e th

e un

its fo

rth

eir

prop

ertie

s. T

his

effe

ct c

an b

e pa

rtic

ular

ly p

oten

t-an

d ca

n le

ad to

strong distortions-when some piece of

info

rmat

ion

abou

t the

targ

et in

di-

vidu

al is

mis

sing

. To

show

this

, I f

irst

del

eted

the

conn

ectio

n be

twee

n th

ein

stan

ce n

ode

for

Lan

ce a

nd th

e pr

oper

ty n

ode

for

his

occu

patio

n-he

hap

-pe

ned

to b

e a

burg

lar,

The

n I

activ

ated

the

nam

e un

it fo

r L

ance

, Thi

s ca

used

the

inst

ance

nod

e to

bec

ome

activ

e, a

nd th

e in

stan

ce n

ode

then

act

ivat

edth

e pr

oper

ty n

odes

for

Jet

, 20'

, JH

edu

catio

n, a

nd m

arrie

d. N

ow, t

here

happ

ened

to b

e se

vera

l oth

er J

ets

who

had

man

y of

thes

e pr

oper

ties,

and

they

all

happ

ened

to b

e bu

rgla

rs. A

s a

resu

lt, th

e m

odel

fill

ed in

this

occ

upa-

tion

for

Lan

ce. I

n th

is in

stan

ce th

e re

sult

happ

ened

to b

e co

rrec

t, bu

t the

sam

e th

ing

wou

ld o

f co

urse

hav

e ha

ppen

ed w

heth

er L

ance

had

bee

n a

bur-

glar

or

not,

Had

he

been

a p

ushe

r or

a b

ooki

e (o

r, f

or th

at m

atte

r, s

omeo

ne

with

an

entir

ely

inno

cuou

s oc

cupa

tion)

, the

mod

el w

ould

hav

e fil

led

in b

ur-

glar

any

way

. In

that

cas

e th

is w

ould

hav

e be

en a

cle

ar e

xam

ple

of a

mem

ory

distortion: Lance would have been guilty by association.

The

mod

el il

lust

rate

s tw

o ke

y po

ints

cen

tral

to th

e is

sues

rai

sed

in th

isvo

lum

e, F

irst,

it pr

ovid

es a

n ex

plic

it th

ough

sim

ple

mec

hani

sm il

lust

ratin

gho

w m

emor

y di

stor

tions

can

aris

e fr

om th

e w

orki

ngs

of o

rdin

ary

mem

ory

retr

ieva

l pro

cess

es, T

hese

pro

cess

es a

re o

ften

ben

efic

ial-t

hey

allo

w th

e fo

r-

Con

stru

ctiv

e M

emor

y an

d M

emor

y D

isto

rtio

ns

mat

i on

of g

ener

aliz

atio

ns o

ver

sim

ilar

inst

ance

s an

d th

e fi

lling

in o

f m

issi

ngpr

oper

ties

base

d on

the

prop

ertie

s of

oth

er, s

imila

r in

divi

dual

s-but they

can

pote

ntia

lly b

e ha

rmfu

l in

that

the

info

rmat

ion

fille

d in

need not be

corr

ect.

Seco

nd

, , th

e m

odel

has

the

sam

e pr

oper

ty th

at h

uman

mem

ory

has,

of

ofte

n fa

iling

to s

epar

ate

info

rmat

ion

that

ari

ses

from

dif

fere

nt s

ourc

es, S

up-

pose

that

a n

ew in

stan

ce n

ode

is f

orm

ed f

rom

eve

ry e

xper

ienc

e (a

pro

posa

lst

riki

ngly

sim

ilar

to th

e m

emor

y m

odel

of

Hin

tzm

an, 1

988)

, and

sup

pose

one

has

a nu

mbe

r of

sim

ilar

expe

rien

ces.

The

n w

hen

we

try

to r

ecal

l one

pieces of other similar experiences will tend to intrude particularly in those

aspe

cts

of th

e or

igin

al f

or w

hich

the

info

rmat

ion

is w

eak

or missing. In

the

mod

el, it is un

fort

unat

ely

not p

ossi

ble

to in

spec

t eac

h m

emor

y tr

ace

indi

vidu

ally

; the

info

rmat

ion

is n

ot s

tore

d in

the

units

them

selv

es b

ut in

thei

r co

mie

ctio

ns; l

ike

conn

ectio

ns a

mon

g ne

uron

s in

the

brai

n, ~

e on

lykn

ow w

hat i

s st

ored

in th

e co

nnec

tions

thro

ugh

the

effe

cts

thes

e co

nnec

tions

have

on

the

outc

ome

of p

roce

ssin

g. B

ut, s

ince

man

y un

its a

nd c

onne

ctio

nsco

ntri

bute

to th

is o

utco

me,

ful

l dis

enta

nglin

g of

the

spec

ific

cau

se o

f ea

chas

pect

of

the

outc

ome

is im

poss

ible

. It w

ill, t

hen,

in g

ener

al n

ot b

e po

ssib

leto

iden

tify

the

spec

ific

sou

rce

of a

ny a

spec

t of

cons

truc

ted

reco

llect

ion.

Giv

en th

e m

odel

, the

n, t

he m

emor

y di

stor

tions

rep

orte

d in

this

vol

ume

by Loftus, Ceci, M

osco

vitc

h, and others are to be expected, Perhaps the

only

thin

g th

at is

une

xpec

ted

abou

t the

m is

the

resi

stan

ce th

at o

ften

ari

ses

to th

eir

acce

ptan

ce. T

his

resi

stan

ce m

ay c

ome

from

impl

icit

acce

ptan

ce o

fan

alte

rnat

ive

mod

el o

f m

emor

y, in

whi

ch m

emor

y tr

aces

are

not

so

muc

hco

nstr

ucte

d as

ret

riev

ed, l

ike

book

s fr

om a

libr

ary.

The

met

apho

r of

hum

anm

emor

y an

d hu

man

kno

wle

dge

as a

libr

ary

prov

ides

a b

asis

for

acc

ount

ing

for

the

role

of

orga

niza

tion

in m

emor

y, b

ut g

ives

no

basi

s fo

r un

ders

tand

ing

dist

ortio

n. I

bel

ieve

that

as

we

com

e to

und

erst

and

mem

ory

bette

r an

d be

t-te

r, it

will

bec

ome

incr

easi

ngly

cle

ar th

at th

is is

a m

isle

adin

g m

etap

hor.

An

Exp

erim

enta

l Tes

t of

the

Tra

ce S

ynth

esis

Mod

el

To

test

the

mod

el d

escr

ibed

abo

ve, a

nd to

ext

end

the

empi

rica

l dat

a ba

seof

evi

denc

e of

mem

ory

dist

ortio

ns, L

eigh

Nys

trom

and

I d

evel

oped

an

expe

r-im

enta

l par

adig

m d

esig

ned

to e

licit

trac

e sy

nthe

sis

erro

rs in

rem

embe

ring

(Nys

trom

and

McC

lella

nd, 1

992)

. In

this

par

adig

m, s

ubje

cts

stud

y a

list o

fse

nten

ces,

and

then

are

late

r cu

ed fo

r co

mpl

ete

reca

ll of

indi

vidu

al w

hole

sent

ence

s w

hen

a fr

agm

ent i

s pr

esen

ted

as a

pro

be, C

onsi

der

sent

ence

s of

the form: "T

he p

olic

eman

gav

e th

e ac

coun

tant

the

ham

mer

in th

e ba

se-

men

t."

We

imag

ine

that

the

sent

ence

is a

naly

zed

into

a s

et o

f ro

le-f

iller

pai

rsw

hich

are

then

rep

rese

nted

by

the

activ

atio

ns o

f in

put u

nits

in th

e ne

twor

ksh

own

in F

igur

e 2.

2, T

he n

etw

ork

is s

tric

tly a

nalo

gous

to th

e m

odel

pre

vi-

ousl

y de

scri

bed,

The

re is

a p

ool o

f in

stan

ce u

nits

, with

one

uni

t ass

igne

d to

Page 4

COGNITIVE PERSPECTIVES

Con

stru

ctiv

e M

emor

y an

d M

emor

y D

isto

rtio

ns

Prop

ositi

onT

able

E

xam

ple

over

lap

and

cont

rol s

ente

nce

pair

s fr

om N

ystr

om a

ndM

cCle

lland

(19

92),

with

cor

resp

ondi

ng te

st p

robe

s

Ove

rlap

Pai

r:

' , ,

The

pol

icem

an g

ave

the

acco

unta

nt th

e ha

mm

er il

l the

din

ing

room

,T

he f

arm

er g

ave

the

acco

unta

nt th

e ha

mm

er in

the

gara

ge,

Con

trol

Pai

r:T

he d

river

sho

wed

the

rece

ptio

nist

the

toas

ter

in th

e ki

tche

n.T

he s

wim

mer

loan

ed th

e sa

lesm

an th

e en

velo

pe in

the

base

men

t.

Ove

rlap

Pr

obe:

The

, , ,

gave

the

acco

unta

nt th

e ha

mm

er in

the.

.

Con

trol

Pr

obe:

The

, , ,

show

ed th

e re

cept

ioni

st th

e to

aste

r in

the,

,

Age

ntA

ctio

nR

ecip

ient

Obj

ect

Loc

atio

nsentence, One such pair

of

over

lapp

ing

sent

ence

s is

sho

wn

in T

able

2, 1

; apa

ir

of control sentences, w

ith n

o ov

erla

p, is

als

o sh

own,

In

the

mod

el, w

.eas

sign

ed a

dif

fere

nt p

ropo

sitio

n un

it to

eac

h se

nt~n

ce, a

nd c

onne

cted

this

unit to the input units for each

of

the correspondmg words. In the case

overlap sentences

, thr

ee

of

the

five

role

-fill

ers

are

~he

sam

e , S?

the

pr~p

osl-

tion units in these cases are connected to overlappmg sets

of

mpu

t UnI

ts, a

ssh

own

in F

igur

e 2.

3.

Figu

re

2 Sketch

of the network architecture used by

Nys

trom

an

d M

cCle

lland

(19

92),

The units participating in the representation

of

a pa

rtic

ular

pro

posi

tion

are

show

n , a

long

with the bidirectional connections that allow this model to perform cued recall

of

the

who

lesentence when' part

of

the

sent

ence

is r

epre

sent

ed a

s a

cue,

each proposition, The model also contains several pools

of

prop

erty

uni

tsone for each type

of

role that can occur in one

of

the

sent

ence

s. E

ach

pool

cont

ains

a u

nit f

or e

ach

wor

d th

at c

an a

ppea

r in

that

rol

e. W

e re

pres

ent t

hese

nten

ce a

bout

the

polic

eman

as

show

n in

the

figu

re, A

noth

er s

ente

nce , w

itha non-overlapping set

of

wor

ds, w

ould

invo

lve

a di

ffer

ent w

ord-

in-r

ole

unit

in e

ach

prop

erty

uni

t poo

l , a

nd a

diff

eren

t pro

posi

tion

unit.

In a

mod

el s

uch

as th

is, cued remembering occurs

by

sim

ply

turn

ing

onth

e un

its f

or th

e w

ords

con

tain

ed in

the

prob

e , a

nd a

skin

g th

e ne

twor

k to

esse

ntia

lly fi

ll in

the

rest

, Thi

s oc

curs

via

a g

radu

al c

onst

rain

t sat

isfa

ctio

nprocess, Processing begins with the units for the cue words clamped and

continues until a stable pattern

of

activ

atio

n is

ach

ieve

d, W

hen

the

cue

uniq

uely

mat

ches

one

sto

red

sent

ence

, and

ther

e ar

e no

oth

er v

ery

sim

ilar

sent

ence

s , th

e co

rrec

t sen

tenc

e te

nds

to b

e re

calle

d, H

owev

er, e

rror

s ca

noc

cur

whe

n th

ere

are

two

or. m

ore

stor

ed it

ems

that

hav

e th

e sa

me

or a

similar degree

of

mat

ch to

the

prob

e. I

n th

is c

ase

the

" rem

embe

red"

pat

tern

is a constructed synthesis

of two or more stored traces,

Nystrom and I studied the adequacy

of

this

mod

el to

acc

ount

for

mem

ory

and memory distortions in a series

of

four

exp

erim

ents

. Her

e I

will

dis

cuss

only one

of these. The subjects studied lists

of

sent

ence

s se

t up

so th

at s

ome

of

the

sent

ence

s sh

ared

thre

e co

nten

t wor

ds in

com

mon

with

ano

ther

pai

red

Prop

ositi

on

Age

ntA

ctio

nR

ecip

ient

Obj

ect

Loc

atio

n

Figu

re

2.3

The

uni

ts a

nd c

onne

ctio

ns p

artic

ipat

i~g

in t~

e r~

pres

enta

tion

of a s~c

ond

sen-

.te

nce

that

ove

rlaps

with

the

first

one

sho

wn

prev

IOus

ly II

I Fig

ure

2,2, Presentation

of

ambi

guou

s pr

obe

tend

s to

act

ivat

e bo

th s

ente

nces

and

may

pro

duce

a b

lend

err

or,

Page 5

COGNITIVE PERSPECTIVES

Con

stru

ctiv

e M

emor

y an

d M

emor

y D

isto

rtio

ns

Our

foc

us in

this

res

earc

h w

as o

n tr

ace

synt

hesi

s at

the

time

of r

ecal

lan

d w

e th

eref

ore

wen

t to

som

e le

ngth

s to

min

imiz

e th

e po

ssib

ility

that

sub

-je

cts

wou

ld b

e re

min

ded

of th

e fi

rst m

embe

r of

eac

h ov

erla

p pa

ir w

hen

give

nth

e se

cond

mem

ber

duri

ng th

e ex

posu

re p

hase

of

the

expe

rim

ent.

Thi

s w

asdo

ne, f

irst

of

all,

by

deve

lopi

ng a

cov

er ta

sk th

at f

ocus

ed s

ubje

cts

on a

naly

z-in

g ea

ch in

divi

dual

sen

tenc

e se

para

tely

fro

m a

ll of

the

othe

rs w

ithou

t any

men

tion

of a

late

r m

emor

y te

st: s

ubje

cts

wer

e to

ld th

eir

task

was

to r

ate

each

sen

tenc

e fo

r its

ove

rall

plau

sibi

lity

and

to s

ay h

ow w

ell t

hey

thou

ght

each

wor

d fi

t with

the

over

all e

vent

des

crib

ed b

y th

e w

hole

sen

tenc

e. O

ther

prec

autio

ns in

clud

ed v

aryi

ng th

e pl

acem

ent w

ithin

the

sent

ence

s of

the

over

-la

ppin

g w

ords

, and

sep

arat

ing

over

lapp

ing

sent

ence

s as

far

as

poss

ible

inth

e st

udy

list o

f 34

sen

tenc

es, A

lthou

gh s

ubje

cts

did

notic

e th

at s

ome

wor

dsw

ere

occa

sion

ally

rep

eate

d , o

nly

a fe

w s

ubje

cts

repo

rted

that

the

seco

ndm

embe

r of

an

over

lap

pair

eve

r ca

used

them

to r

ecal

l the

pre

viou

s m

embe

rof

the

pair

, and

elim

inat

ing

thes

e su

bjec

ts f

rom

the

anal

yses

did

not

cha

nge

the

resu

lts, T

hus

we

wer

e re

ason

ably

con

fide

nt th

at r

emin

ding

and

trac

esy

nthe

sis

duri

ng th

e st

udy

phas

e w

as n

ot a

maj

or f

acto

r in

det

erm

inin

g th

ere

sults

.A

fter

the

subj

ects

com

plet

ed a

fill

er ta

sk f

or 5

min

utes

, the

cue

d re

call

.ph

ase

of th

e ex

peri

men

t was

adm

inis

tere

d. T

his

invo

lved

pre

sent

ing

16 s

en-

tenc

e fr

agm

ents

, eac

h w

ith b

lank

s fo

r tw

o co

nten

t wor

ds, E

ight

frag

men

tsw

ere

from

ove

rlap

pai

rs a

nd e

ight

wer

e fr

om c

ontr

ol p

airs

. Thu

s a

subj

ect

mig

ht s

ee p

robe

s lik

e th

e on

es s

how

n in

Tab

le 2

,In both cases, t

he s

ubje

cts

wer

e to

ld to

com

plet

e th

e pr

obe

with

the

firs

tst

udie

d se

nten

ce th

at c

ame

to m

ind,

Sub

ject

s w

ere

aler

ted

to th

e fa

ct th

atso

met

imes

the

prob

e m

atch

ed tw

o st

udie

d se

nten

ces

equa

lly w

ell ,

and

they

wer

e to

ld to

rec

all o

nly

one

of th

e tw

o sentences and to take care not to

mix

up

the

two,

Aft

er th

e fi

rst r

ecal

l the

y w

ere

give

n an

opp

ortu

nity

to r

ecal

lth

e se

cond

sen

tenc

e. I

will

be

disc

ussi

ng o

nly

the

resu

lts o

f th

e fi

rst r

ecal

lhe

re. S

uffi

ce it

to s

ay th

at s

econ

d re

calls

wer

e ge

nera

lly le

ss a

ccur

ate

than

firs

t rec

alls

,O

n th

e fi

rst r

ecal

ls, t

he p

roba

bilit

y of

cor

rect

ly r

ecal

ling

a co

mpl

ete

sen-

tenc

e di

d no

t dif

fer

betw

een

the

over

lap

and

cont

rol p

airs

: tw

o w

ords

fro

mth

e sa

me

sent

ence

that

mat

ched

the

prob

e w

ere

reca

lled

42%

of

the

time

inth

e ov

erla

p co

nditi

on a

nd 4

1 %

of

the

time

in th

e co

ntro

l con

ditio

n. H

ow-

ever

, in

the

over

lap

cond

ition

sub

ject

s di

d so

met

imes

mak

e w

hat w

e ca

nca

ll sy

nthe

sis errors-errors in which one w

ord

cam

e fr

om o

ne o

f th

e se

n-te

nces

that

mat

ched

the

prob

e an

d th

e ot

her

cam

e fr

om th

e ot

her

of th

ese

two

sent

ence

s, T

his

occu

rred

on

near

ly 1

0% o

f th

e er

ror

tria

ls (

5.4%

of

all

tria

ls w

ith o

verl

ap p

robe

s , c

ompa

red

to le

ss th

an 1

% o

f tr

ials

with

con

trol

prob

es).

The

rat

e of

syn

thes

is e

rror

s m

ay s

eem

rel

ativ

ely

low

, but

they

-wer

ere

liabl

y m

ore

freq

uent

than

cha

nce,

Con

fiden

ce r

atin

gs w

ere

obta

ined

on

each

rec

all t

rial

; con

fide

nce

was

slig

htly

less

on

aver

age

for

synt

hesi

s er

rors

than for completely correct responses, b

ut o

n 40

% o

f th

e sy

nthe

sis

erro

rssu

bjec

ts g

ave

the

high

est c

onfi

denc

e ra

ting,

cor

resp

ondi

ng to

the

stat

emen

tI a

m s

ure

both

wor

ds r

tx:a

lled

cam

e fr

om a

sin

gle

stud

ied

sent

ence

that

mat

ched

the

prob

e."

We

take

the

expe

rim

ent,

then

, as

dem

onst

ratin

g th

atm

emor

y di

stor

tions

can

ari

se f

rom

trac

e sy

nthe

sis

at th

e tim

e of

rec

all.

We

wou

ld n

ot w

ant t

o cl

aim

, of c

ours

e, t

hat t

race

syn

thes

is d

oes

not o

ften

occ

urea

rlie

r, w

hen

an in

terv

enin

g ev

ent r

emin

ds u

s of

a p

revi

ous

even

t; in

deed

it m

ay b

e th

at th

is is

one

com

mon

sou

rce

of m

emor

y di

stor

tions

. We

wou

ldon

ly s

ugge

st th

at o

ur r

esul

ts s

uppo

rt th

e vi

ew th

at tr

ace

synt

hesi

s ca

n oc

cur

at r

ecal

l as

wel

l as

betw

een

initi

al s

tudy

and

rec

all.

We

mod

eled

the

data

fro

m th

is a

nd th

e ot

her

thre

e st

udie

s w

e co

nduc

ted

usin

g th

e m

odel

dis

cuss

ed p

revi

ousl

y. T

o fi

t the

dat

a it

was

nec

essa

ry to

mak

e tw

o ad

ditio

nal s

tipul

atio

ns: f

irst, that processing has an inherently

rand

om c

ompo

nent

; and

sec

ond

, tha

t sub

ject

s so

met

imes

fai

led

to e

ncod

eea

ch s

ente

nce

com

plet

ely,

We

impl

emen

ted

this

latte

r as

sum

ptio

n by

ran

-do

mly

elim

inat

ing

a fr

actio

n of

the

conn

ectio

ns b

etw

een

the

inpu

t and

pro

pun

its, T

hese

ass

umpt

ions

do

diff

eren

t and

impo

rtan

t wor

k in

acc

ount

ing

for the data,

The

firs

t ass

umpt

ion-

intr

insi

c va

riab

ility

-allo

ws

the

netw

ork

to s

elec

tes

sent

ially

ran

dom

ly b

etw

een

two

equa

lly g

ood

resp

onse

s in

cas

es w

here

two

stud

ied

sent

ence

s fi

t the

pro

be e

qual

ly w

ell.

Intr

insi

c va

riab

ility

is im

ple-

men

ted

sim

ply

by in

corp

orat

ing

norm

ally

dis

trib

uted

ran

dom

noi

se in

to th

ein

put t

o ea

ch u

nit.

Eac

h tim

e th

e un

it's

stat

e is

upd

ated

, thi

s no

ise

affe

cts

the

exac

t deg

ree

of a

ctiv

atio

n. I

f hi

gh levels of noise are used,.-th

e m

odel

beco

mes

tota

lly r

ando

m in

its

beha

vior

; but

with

sm

all a

mou

nts

of n

oise

the

vari

abili

ty e

ffec

tivel

y ca

uses

the

netw

ork

to c

hoos

e ra

ndom

ly a

mon

geq

ually

goo

d al

tern

ativ

es. W

ithou

t any

noi

se th

e ne

twor

k w

ill h

ave

a te

n-de

ncy

to p

artia

lly a

ctiv

ate

both

mat

chin

g sentences most of the time, a

ndw

ill n

ot te

nd to

rec

all o

ne o

r th

e ot

her:

with

the

rand

omne

ss in

pla

ce, o

nth

e ot

her

hand

, the

net

wor

k w

ill te

nd to

set

tle to

one

of

the

two

answ

ers.

The

idea

of

intr

insi

c va

riab

ility

in p

roce

ssin

g w

as in

trod

uced

into

con

nect

ioni

stm

odel

ing

by s

ever

al in

vest

igat

ors

in th

e m

id-1980s(Geman and Geman

1984

; Sm

olen

sky,

198

6; H

into

n an

d Se

jnow

ski,

198

6). I

hav

e su

gges

ted

else

-w

here

that

intr

insi

c va

riab

ility

is a

gen

eral

pro

pert

y of

hum

an c

ogni

tive

func

tion,

and

I th

ink

it is

nec

essa

ry if

we

are

to m

odel

'th

e ki

nds

of r

esul

tsw

e se

e in

a w

ide

rang

e of

task

s, s

uch

as fr

ee a

ssoc

iatio

n, s

tem

com

plet

ion

or p

erce

ptua

l ide

ntifi

catio

n, w

here

sub

ject

s ge

nera

lly e

mit

one

or th

e ot

her

of a

set

of

alte

rnat

ive

cohe

rent

res

pons

es, r

athe

r th

an a

ble

nd o

f man

y al

ter-

nativ

es (

McC

lella

nd, 1

991)

. Oth

ers

have

est

ablis

hed

that

the

outc

ome

ofth

is s

ettli

ng p

roce

ss is

opt

imal

fro

m th

e po

int o

f vi

ew o

f m

axim

izin

g th

epr

obab

ility

of

sele

ctin

g th

e co

rrec

t ans

wer

, giv

en th

at th

e w

eigh

ts a

ccur

atel

yen

code

info

rmat

ion

abou

t the

dom

ain

(Gem

an a

nd G

eman

, 198

4),

The

sec

ond

assu

mpt

ion~

enco

ding

fai

lure

s-al

low

s th

e m

odel

to a

c-

Page 6

CaG

NIT

IVE

PE

RSP

EC

TIV

ES

caun

t sim

ulta

neau

sly

far

the

exis

tenc

e af

syn

thes

is e

rrar

s , ta

geth

er w

ith th

efa

ct th

at th

e pr

abab

ility

af c

orre

ct r

ecal

l did

nat

suf

fer

in th

e av

erla

p ca

ndi-

tian

cam

pare

d to

. the

can

tral

can

ditia

n, S

impl

ifyin

g a

bit ,

with

this

ass

ump-

tian

in p

lace

, car

rect

rec

all a

f a s

ingl

e se

nten

ce d

epen

ds a

n w

heth

er it

has

been

cam

plet

ely

enca

ded ,

and

the

prab

abili

ty a

f ca

mpl

ete

enca

ding

is in

de-

pend

ent a

f w

heth

er th

ere

is a

n av

erla

p se

nten

ce in

the "

stu

dy s

et. I

ncam

plet

een

cadi

ng a

ffer

s an

app

artu

nity

far

syn

thes

is e

rrar

s: w

e ab

tain

ed a

n ex

celle

ntfit

to. t

he d

ata

by a

ssum

ing

that

sub

ject

s fa

iled

to. e

ncad

e 20

% a

f th

e w

ards

.In cases where there are gaps in the encading af ane af the sentences, t

heat

her

can

cant

ribut

e, c

reat

ing

a m

emar

y di

star

tian.

Intr

usia

ns f

rom

the

athe

r se

nten

ce r

ush

in w

hen

the

mas

t act

ive

trac

e pr

avid

es n

o. in

farm

atia

n,

Sum

mar

y af

the

Tra

ce S

ynth

esis

Mod

el

The

mad

el I

have

des

crib

ed h

as c

ansi

dera

ble

appe

al a

s a

sim

ple

desc

riptiv

eac

caun

t af t

he p

roce

ss a

f mem

ary

trac

e sy

nthe

sis

in c

ued

reca

ll an

d ga

essa

me

way

taw

ard

impl

emen

ting

the

cans

truc

tive

mem

ary

retr

ieva

l pra

cess

af w

hich

Bar

tlett

and

Nei

sser

wro

te. I

sha

uld

nate

that

ther

e ar

e at

her

mad

-el

stha

t can

acc

aunt

far

trac

e sy

nthe

sis,

incl

udin

g th

e M

INE

RV

A m

adel

af

Hin

tzm

an (

1988

), a

s w

ell a

s th

e m

adel

s by

Met

calf

e (1

990)

and

by

Hum

-ph

reys

, Bai

n, a

nd P

ike

(198

9) th

at u

se d

istr

ibut

ed r

epre

sent

atia

ns (

see

also

.M

cCle

lland

and

Rum

elha

rt, 1

985)

. The

mad

el a

f M

etca

lfe

(199

0) h

as b

een

appl

ied

to. a

num

ber

af im

part

ant f

indi

ngs

an b

lend

err

ors

and

athe

r m

em-

ary

dist

artia

ns .

and

shaw

s th

at s

uch

mad

els

can

affe

r ve

ry n

ice

acca

unts

af

muc

h af

the

exis

ting

data

an

blen

ding

and

mem

ary

dist

artia

n.In spite af their success, a

ll af

thes

e m

adel

s la

ck s

amet

hing

. The

re a

reat

her ,

dee

per ,

mar

e fu

ndam

enta

l far

ces

at w

ark

shap

ing

mem

ary

perf

ar-

man

ce, T

hese

pra

cess

es, I

bel

ieve

, ape

rate

gra

dual

ly a

ver

the

caur

se a

f ca

g-ni

tive

deve

lapm

ent t

o. s

hape

the

way

we

repr

esen

t the

can

stitu

ents

af

mem

-ar

y tr

aces

- far

exa

mpl

e , th

e ca

ncep

ts th

at c

antr

ibut

e to

. pro

.pas

itian

s,T

hese

repr

esen

tatia

ns, i

n tu

rn, p

ravi

de th

e ba

sis

far

mar

e pa

wer

ful f

arm

s O

f can

est

ruct

ive

mem

ary

effe

cts,

Ele

phan

tC

olor

Gra

y

Mad

els

of R

epre

sent

atia

n F

orm

atio

n vi

a G

radu

al L

earn

ing

An

earl

y m

adel

that

pai

nted

the

way

taw

ard

this

idea

was

pre

sent

ed b

yH

inta

n (1

981)

. Thi

smad

el is

ske

tche

d in

Fig

ure

2.4,

It c

ansi

sts

af s

epar

ate

sets

af

units

far

rep

rese

ntin

g th

e fi

rst n

aun ,

rel

atia

n, a

nd s

ecan

d na

un a

fthree-term propasitians such as "F

ish

can

swim

" " S

amm

y is

a fi

shE

le-

phan

ts a

re g

ray,

" "

Cly

de is

whi

te" and so

. far

th, T

he m

adel

is s

imila

r to

.th

e pr

evia

usm

adel

, but

naw

eac

h w

ard

is a

pat

tern

af a

ctiv

atia

n av

er th

eap

prap

riate

uni

ts r

athe

r th

an a

sin

gle

activ

e un

it, T

he w

hale

pro

pasi

tian

isre

pres

ente

d as

a. p

atte

rn a

f ac

tivity

ave

r th

e th

ree

sets

af

cans

titue

nt u

nits

Figu

re

2.4

A s

light

ly e

labo

rate

d ve

rsio

n of

the

conn

ectio

nist

net

wor

k us

ed b

y H

into

n to

repr

esen

t pro

posi

tions

abo

ut th

e co

ncep

ts, E

ach

rect

angl

e re

pres

ents

a p

ool o

f co

nnec

tioni

stin

form

atio

n pr

oces

sing

uni

ts, a

nd e

ach

arro

w r

epre

sent

s.a

full

set o

f co

nnec

tions

fro

m e

ach

unit

on th

e se

ndin

g si

de o

f the

arr

ow to

eac

h un

it on

the

rece

ivin

g si

de, P

roce

ssin

g oc

curs

by th

e pr

opag

atio

n of

act

ivat

ion

amon

g th

e un

its v

ia th

e co

nnec

tion

wei

ghts

, Eac

h un

it si

m-

ply

sets

its

activ

atio

n to

a v

alue

bet

wee

n 0

and'

1 ba

sed

on th

e su

mm

ed in

put i

t rec

eive

sfr

om o

ther

uni

ts v

ia th

e w

eigh

ted

conn

ectio

ns, F

or e

xam

ple ,

the

conn

ectio

ns f

rom

the

Wor

d 1

inpu

t uni

ts td

the

Nou

n 1

repr

esen

tatio

n un

its a

llow

a p

atte

rn o

f ac

tivat

ion

repr

e-se

ntin

g th

e fi

rst w

ord

of a

pro

posi

tion

to a

ctiv

ate

the

appr

opri

ate

sem

antic

pat

tern

ove

r th

eN

oun

1 ,u

nits

, The

con

nect

ions

bet

wee

n th

e di

ffer

ent p

ools

of

cons

titue

nt u

nits

and

the

Prop units encode the system

s kn

owle

dge

abou

t pro

posi

tions

, and

aft

er th

e w

eigh

ts h

ave

been

acq

uire

d th

roug

h le

arni

ng, t

hese

con

nect

ions

allo

w th

e th

ird

cons

titue

nt o

f a

prop

osi-

tion

to b

e co

mpl

eted

giv

en th

e ot

her

two

cons

titue

nts

as in

puts

, Ret

urn

conn

ectio

ns f

rom

the

cons

titue

nt u

nits

to th

e w

ord

units

then

pro

vide

.for

out

put o

f th

e pa

ttern

fill

ed in

by

the

netw

ork,

The

upp

er p

anel

sho

ws

each

poo

l of

units

with

labe

ls, T

he lo

wer

pan

el il

lus-

trat

es h

ow a

par

ticul

ar p

ropo

sitio

n w

ould

act

ually

be

repr

esen

ted,

Page 7

CO

GN

ITIV

E P

ER

SPE

CT

IVE

S

and

over

an

addi

tiona

l, fo

urth

set

of

units

cal

led

" PR

OP"

uni

ts. T

he n

et-

wor

k co

ntai

ns b

idir

ectio

nal c

onne

ctio

ns f

rom

eac

h se

t of

cons

titue

nt u

nits

to th

e PR

OP

units

. The

re is

als

o an

othe

r se

t of

conn

ectio

ns f

or in

put t

o an

dou

tput

fro

m th

e ne

twor

k; th

ese

allo

w in

puts

sta

ndin

g fo

r sp

ecifi

c w

ords

toac

tivat

e di

strib

uted

sem

antic

pat

tern

s ov

er th

e in

p~t

uni

ts, T

he r

ecur

rent

conn

ectio

ns w

ithin

eac

h po

ol o

f un

its a

llow

loca

l pat

tern

com

plet

ion

with

inea

ch p

ool.

The

eff

ect o

f th

is is

to im

plem

ent a

"cl

ean-

" pr

oces

s in

whi

chth

e pa

ttern

of

activ

atio

n te

nds

to c

onve

rge

to th

e re

pres

enta

tion

of a

spe

cifi

cw

ord,

and

has

muc

h th

e sa

me

effe

ct a

s th

e m

utua

l inh

ibiti

on w

ithin

laye

rsin

the

prev

ious

mod

el.

Onc

e ag

ain,

the

know

ledg

e or

mem

ory

in th

is s

yste

m is

sto

red

in th

e co

nnec

-tio

ns a

mon

g th

e un

its, W

e ca

n th

ink

of th

e in

put/o

utpu

t wei

ghts

as

enco

ding

know

ledg

e ab

out t

he s

eman

tic p

atte

rn c

orre

spon

ding

to e

ach

wor

d of

the

prop

ositi

on, a

nd w

e ca

n th

ink

of th

e co

nnec

tion

wei

ghts

bet

wee

n th

e co

nstit

-ue

nt a

nd P

RO

P po

ols

as e

ncod

ing

know

ledg

e ab

out t

he p

ropo

sitio

ns th

at th

ese

cons

titue

nts

ente

r in

to w

ith o

ther

con

stitu

ents

, Hin

ton

(198

1) s

ugge

sted

that

this

net

wor

k w

ould

be

able

to g

ener

aliz

e w

hat i

t kno

ws

abou

t one

con

cept

toot

her

rela

ted

conc

epts

if s

imila

r co

ncep

ts a

re r

epre

sent

ed b

y si

mila

r pa

ttern

sof

act

ivat

ion,

Thu

s , if

Cly

de is

a p

artic

ular

ele

phan

t , an

d C

lyde

is r

epre

sent

edby

a p

atte

rn th

at is

sim

ilar

to th

e pa

ttern

for

ele

phan

t , th

en w

hat w

e kn

owab

out e

leph

ants

will

tend

to g

ener

aliz

e to

Cly

de. S

uch

effe

cts

do n

ot s

tric

tlyob

ey th

e la

ws

of lo

gic;

inst

ead

they

obe

y th

e la

ws

of a

ssoc

iatio

n,In

Hin

ton

s (1

981)

wor

k , th

e re

pres

enta

tions

of

the

conc

epts

wer

e as

-signed by hand. Connectionist learning algorithms have evolved consider-

ably

sin

ce th

at ti

me ,

how

ever

, and

we

now

hav

e al

gori

thm

s th

at c

an d

isco

ver

how

to r

epre

sent

dif

fere

nt c

once

pts

thro

ugh

repe

ated

exp

osur

e to

info

rma-

tion

abou

t the

ent

ire s

eman

tic d

omai

n in

whi

ch th

e co

ncep

t is

embe

dded

,I

will

con

side

r on

e su

ch domain-the broad domain of

livi

ng th

ings

. I s

how

in Figure 2,5 a representation of a fragment of the knowledge someone

mig

ht h

ave

abou

t liv

ing

thin

gs, T

his

form

at is

typi

cal o

f th

e ap

proa

ch to

know

ledg

e re

pres

enta

tion

used

in c

lass

ical

art

ific

ial i

ntel

ligen

ce a

ppro

ache

sto cognition, b

egin

ning

with

Qui

llian

(19

68),

The

kno

wle

dge

has

seve

ral

char

acte

rist

ics:

it is

str

uctu

red ,

in th

at it

is o

rgan

ized

into

a h

iera

rchy

. Ind

i-vi

dual

type

s or

spe

cies

are

list

ed a

t the

bot

tom

of

the

hier

arch

y, a

nd th

eir

orga

niza

tion

into

bro

ad c

lass

es, a

nd th

e or

gani

zatio

ns o

f the

se in

to la

rger

clas

ses ,

is in

dica

ted

by "

isa"

link

s. W

e ca

n im

agin

e at

tach

ing,

bel

ow th

ele

vel o

f th

e ty

pes ,

spe

cific

inst

ance

s of

the

type

s. F

or e

xam

ple

, we

coul

d ad

dT

wee

ty is

a ca

nary

, " a

nd s

o fo

rth;

or

if w

e ha

d an

Ele

phan

t nod

e , w

e co

uld

add

" Cly

de is

a E

leph

ant,

" T

he n

etw

ork

is p

oten

tially

qui

te e

cono

mic

al, i

nth

at fa

cts

that

are

true

of w

hole

sub

-tre

es o

f th

e hi

erar

chy

can

be a

ttach

edat

the

high

est l

evel

to w

hich

they

app

ly. G

iven

this

, whe

n so

me

info

rmat

ion

is n

ot s

tore

d on

a s

peci

fic c

once

pt, i

t may

be

infe

rred

by

sear

chin

g th

roug

hth

e tr

ee. T

he p

roce

ss is

equ

ival

ent t

o th

e st

anda

rd lo

gica

l syl

logi

stic

rea

son-

Con

stru

ctiv

e M

emor

y an

d M

emor

y D

isto

rtio

ns

livin

g th

ing

grow

Figu

re

5 A

sem

antic

net

wor

k of

the

type

for

mer

ly u

sed

in m

odel

s of

the

orga

niza

tion

ofkn

owle

dge

in m

emor

y, c

onta

inin

g th

e co

ncep

ts a

nd p

ropo

sitio

ns u

sed

in th

e le

arni

ng e

x-pe

rim

ent o

f R

umel

hart

(19

90),

Ada

pted

fro

m f

ig, 1

.8 o

f R

umel

hart

and

Tod

d (1

993)

by

McC

leIla

nd e

t al.

(199

4) a

s th

eir

fig, 3

, p, 1

3,

ing

proc

ess

thro

ugh

whi

ch w

e in

fer

that

Soc

rate

s is

mor

tal.

We

know

that

Socr

ates

is a

man

, and

we

know

that

men

are

mor

tal ,

ther

efor

e w

e ca

n in

fer

that

Soc

rate

s is

mor

tal t

oo.

Whe

n on

e tr

ains

a n

etw

ork

like

the

one

show

n in

the

prev

ious

figu

rew

ith e

xam

ple

prop

ositi

ons

from

this

dom

ain

, it l

earn

s tw

o th

ings

, It l

earn

sco

nnec

tion

wei

ghts

inte

rnal

ly th

at e

ncod

e th

e pr

opos

ition

s , a

nd th

at a

llow

com

plet

ion

of a

pro

posi

tion

from

two

of it

s te

rms,

It a

lso

lear

ns c

onne

ctio

nw

eigh

ts fr

om th

e w

ord

inpu

t uni

ts to

the

cons

titue

nt u

nits

that

ess

entia

llyas

sign

use

ful s

eman

tic r

epre

sent

atio

ns to

eac

h w

ord.

Hin

ton

(198

9) d

emon

-strated this for kinship relationships, R

umel

hart

(19

90; R

umel

hart

and

Tod

d, 1

993)

app

lied

the

sam

e id

ea to

the

dom

ain

of li

ving

thin

gs (

the

actu

alsi

mul

atio

n m

odel

Rum

elha

rt u

sed

was

slig

htly

sim

pler

than

the

one

show

nin

Fig

ure

2,5)

, and

I h

ave

chos

en to

use

this

cas

e as

my

exam

ple,

The

res

ults

on w

hich

the

follo

win

g di

scus

sion

dep

ends

com

e fr

om a

rep

etiti

on o

f th

eR

umel

hart

(19

90)

sim

ulat

ion

repo

rted

in M

cCle

lland

, McN

augh

ton

, and

Rei

lly (

1994

),

Page 8

oak

pine

rose

dais

y

cana

ry

robi

n

salm

on

sunf

ish

COGNITIVE PERSPECTIVES

Con

stru

ctiv

e M

emor

y an

d M

emor

y D

isto

rtio

ns

Epoch 25

pine

robi

nro

seoa

kca

nary salm

onda

isy

sunf

ish

Epo

ch 2

00

pine

dais

yoa

kro

seca

nary

robi

nsa

lmon

sunf

ish

Epoch 25

Epoch 200

Epo

ch 5

00

Figu

re

6 R

epre

sent

atio

ns d

isco

vere

d in

a r

eplic

atio

n of

Rum

elha

rt's

(19

90)

expe

rimen

ttr

aini

ng a

sem

antic

net

wor

k m

uch

like

the

one

show

n in

Fig

ure

2,5,

The

fig

ure

show

s th

eac

tivat

ion

of e

ach

of th

e N

oun-

1 un

its f

or e

ach

of th

e ei

ght s

peci

fic

conc

epts

use

d, T

hehe

ight

of

each

ver

tical

bar

indi

cate

s th

e ac

tivat

ion

of th

e un

it on

a s

cale

fro

m 0

to 1

. One

can

see

that

initi

ally

all

the

conc

eprs

hav

e fa

irly

sim

ilar

repr

esen

tatio

ns, A

fter

200

epo

chs

oftr

aini

ng, t

here

is a

cle

ar d

iffe

rent

iatio

n of

the

repr

esen

tatio

ns o

f th

e pl

ants

and

ani

mal

s,A

fter

500

epo

chs ,

the

furt

her

diff

eren

tiatio

n of

the

plan

ts in

to tr

ees

and

flow

ers

and

of th

ean

imal

s in

to fi

sh a

nd b

irds

is a

ppar

ent,

Fro

m M

cCle

lland

et a

l. (1

994)

, fig

,, p

, 16,

oak

pine

rose

dais

yca

nary

robi

nsa

lmon

sunf

ish

Epo

ch 5

00

Thr

ough

gra

dual

trai

ning

on

exam

ples

fro

m th

e do

mai

n of

pla

nts

and

anim

als ,

the

netw

ork

lear

ned

mor

e th

an ju

st th

e pr

opos

ition

s. I

t als

o le

arne

dto

ass

ign

usef

ul r

epre

sent

atio

ns to

eac

h co

ncep

t, T

he representations the

netw

ork

lear

ned

to u

se f

or th

e fi

rst n

oun

are

illus

trat

ed in

Fig

ures

2, 6

and

7, T

hese

rep

rese

ntat

ions

are

det

erm

ined

by

the

conn

ectio

n w

eigh

ts f

rom

the concept input units to the concept representation un

its. T

hrou

gh th

eco

urse

of

lear

ning

, the

se w

eigh

ts g

radu

ally

cha

nge , s

o th

at th

e re

pres

enta

-tio

ns o

f th

e di

ffer

ent c

once

pts

grad

ually

com

e to

cap

ture

how

sim

ilar

the

conc

epts

are

in te

rms

of th

e pr

opos

ition

s th

ey e

nter

into

. Can

ary

and

Rob

in~t

er in

to h

ighl

y ov

erla

ppin

g se

ts o

f pr

opos

ition

s-fo

r ex

ampl

e , b

oth

are

bIrd

s , b

oth

can

fly,

bot

h ha

ve f

eath

ers,

As

a re

sult

of th

is, t

he n

etw

ork

com

esto

ass

ign

them

rep

rese

ntat

ions

that

are

ver

y si

mila

r; s

imila

r re

pres

enta

tions

lead

to s

imila

r ou

tput

s. M

ost i

mpo

rtan

t , on

ce it

has

lear

ned

to u

se s

uch

Figu

re

7 Si

mila

rity

str

uctu

re d

isco

vere

d In

our

rep

hcat

io'n

o!.

Kum

elha

rts

(199

0) le

arni

ngex

peri

men

t, u

sing

the

repr

esen

tatio

ns s

how

n in

Fig

ure

2,6,

Ini

tially

, the

pat

tern

s ar

e al

lquite similar; and the weak similarity structure that exists is random, The

conc

epts

be

com

epr

ogre

ssiv

ely

diff

eren

tiate

d as

lear

ning

pro

gres

ses,

Fro

m M

cCle

lland

et a

l. (1

994)

, fig

, 617

,

repr

esen

tatio

ns, t

he n

etw

ork

can

use

the

sim

ilari

ty o

f th

e re

pres

enta

tions

as

the

basi

s fo

r in

fere

nces

. Thu

s, o

nce

the

netw

ork

has

lear

ned

how

to r

epre

sent

Can

ary

and

Rob

in a

s si

mila

r to

eac

h ot

her

and

dist

inct

fro

m o

ther

con

cept

sit now adds

to

the

trai

ning

set

ene

prop

ositi

on a

bout

a n

ew ty

pe o

f ani

mal

-le

ts

say

the

prop

ositi

on th

at "

Spar

row

isa

bird

"-th

enet

Wor

k le

arns

toas

sign

"Sp

arro

w"

a re

pres

enta

tion

sim

ilar

to th

e re

pres

enta

tion

it us

es f

orot

her

bird

s. A

fter

this

pro

posi

tion

has

been

lear

ned,

we

can

then

ask

the

Page 9

CO

GN

ITIV

E P

ER

SPE

CT

IVE

SC

onst

ruct

ive

Mem

ory

and

Mem

ory

Dis

tort

ions

netw

ork

if it

kno

ws

wha

t a s

parr

ow c

an d

o. T

his

can

be d

one

by p

atte

rnco

mpl

etio

n-w

e ca

n te

st th

e ne

twor

k to

see

if it

can

com

plet

e th

e pa

ttern

Spa

rrow

can

, , ,

" w

ith "

fly.

" In

deed

, Rum

elha

rt s

how

ed in

his

sim

ulat

ions

that if the network was trained on the full set

of

prop

ositi

ons

conc

erni

ngca

narie

s an

d ro

bins

, he

coul

d te

ach

it on

ly o

ne p

ropo

sitio

n ab

out s

par-

row

s-na

mel

y " S

parr

ow is

a b

ird"

and

it w

as a

ble

to c

orre

ctly

com

plet

eot

her

prop

ositi

ons

abou

t spa

rrow

s, T

he o

utpu

t was

qui

te c

lear

abo

ut th

ose

things that are generally true

of

the

othe

r bi

rds,

For

thos

e pr

oper

ties

that

diffe

red

betw

een

cana

ry a

nd r

obin

, it p

rodu

ced

ambi

guou

s ou

tput

s. T

hus

it ap

plie

d w

hat i

t alr

eady

kno

ws

abou

t can

arie

s an

d ro

bins

to s

parr

ows.

Now

, taking this

mod

el

at least as a sketch

of

a model

of

our

know

ledg

eof

fa

cts

and

even

ts, let us consider the nature

of

mem

ory

as r

econ

stru

ctio

nag

ain.

Ind

ivid

ual e

xper

ienc

es th

emse

lves

are

not

sep

arat

ely

repr

esen

ted;

in-

stea

d, they leave what I would call a structured system

of

know

ledg

e st

ored

in th

e co

nnec

tion

wei

ghts

. Fur

ther

mor

e , th

is k

now

ledg

e is

not

itse

lf d

irec

tlyac

cess

ible

to o

vert

res

pond

ing

or d

irec

t rep

ort,

Inst

ead

the

know

ledg

e pr

o-vides a mechanism that can construct responses to queries presented

to

the

netw

ork ,

whether the actual proposition was ever actually experienced, a

sin

the

case

of

the

actu

al tr

aini

ng e

xam

ples

, or

not ,

as

in th

e ca

se o

f qu

estio

nsw

e m

ay a

sk a

bout

, for

exa

mpl

e , what a sparrow can

do

afte

r th

e tr

aini

ngdescribed above, The outputs of such a network might then be the basis

perf

orm

ance

we

take

as

indi

cativ

e of

rem

embe

ring,

but

man

y tim

es th

eym

ight

ref

lect

gen

eral

izat

ion

base

d on

the

accu

mul

ated

eff

ects

of

prio

r ex

peri

-en

ce, r

athe

r th

an th

e ef

fect

s of

sto

ring

anyt

hing

like

the

spec

ific

item

in m

em-

ory,

Suc

h ge

nera

lizat

ion

is, I

wou

ld s

ugge

st, central to our ability

to

act

intelligently, and the process of learning the sorts

of

repr

esen

tatio

ns o

nwhich such generalizations are based is central

to

cogn

itive

dev

elop

men

t(s

ee M

cCle

lland

, 199

4, f

or d

iscu

ssio

n), B

ut s

uch

gene

raliz

atio

n gi

ves

rise

todi

stor

tions

as

an in

here

nt b

y-pr

oduc

t: it

beco

mes

impo

ssib

le to

dis

tingu

ish

betw

een

wha

t has

act

ually

bee

n ex

peri

ence

d an

d w

hat c

an b

e co

nstr

ucte

dba

sed

on o

ther

rel

ated

thin

gs th

at h

ave

been

exp

erie

nced

.B

ut th

ere

is s

omet

hing

slig

htly

wro

ng h

ere.

Our

abi

lity

to is

olat

e pa

rtic

u-la

r m

emor

ies

is n

ot a

s ba

d as

it w

ould

app

ear

to b

e if

we

assu

med

that

memory consists solely

of

the gradually acquired residue

of

a large body

expe

rien

ce. I

can

tell

you

a ne

w fa

ct-s

uch

as th

e fa

ct th

at "

Sam

my

is a

Sunf

ish"

-and

this

can

aff

ect y

our

sem

antic

mem

ory

righ

t aw

ay. W

e ne

edsome mechanism capable

of relatively rapid learning

of the contents

of

indi

-vi

dual

epi

sode

s an

d ex

perie

nces

, O

ne m

ight

thin

k th

at o

ne c

ould

sim

ply

add

new

mem

orie

s on

e at

a ti

me

into

a n

etw

ork

like

the

Hin

ton

(198

1) n

etw

ork ,

but in fact this is not

so.

lf one attempts to store additional memories all at once in such systems, i

tca

n be

don

e, but at the cost

of

a ph

enom

enon

cal

led

" Cat

astr

ophi

c In

terf

er-

ence" (McCloskey and Cohen, 1989). The addition

of

the

new

mat

eria

l

causes a dramatic loss

of

the

abili

ty to

per

form

cor

rect

ly w

ith o

ther

, sim

ilar

mat

eria

l , p

artic

ular

ly w

hen ,

as

is o

ften

the

case

, the

new

mat

eria

l is

not

com

plet

ely

cons

iste

nt w

ith w

hat i

s al

read

y kn

own.

Thu

s if

I tr

ain

the

net-

wor

k w

ith th

e pr

opos

ition

s " S

parr

ow is

a bi

rd"

and

" Spa

rrow

is b

row

nit will drastically interfere with my ability

to

reco

ver

the

colo

r of

oth

er b

irds

like

cana

ries

and

robi

ns (

McC

lella

nd e

t aI., 1994), The only way

to

add

new information robustly

to a sg;uctured memory system is

to

add

it th

roug

ha

proc

ess

calle

d in

terl

eave

d le

arni

ng, i

n w

hich

lear

ning

occ

urs

very

gra

dual

lyth

roug

h re

peat

ed e

xpos

ure

to th

e ne

w m

ater

ial ,

inte

rlea

ved

with

ong

oing

exposure to other examples,of the same domain

of

know

ledg

e. C

onne

ctio

nw

eigh

t adj

ustm

ents

occ

ur d

urin

g ex

posu

re to

the

new

mat

eria

l and

the

old

ther

eby

grad

ually

allo

win

g th

e ne

w m

ater

ial'to be incorporated into the

mem

ory

syst

em w

ithou

t at t

he s

ame

time

disr

uptin

g w

hat i

s al

read

y kn

own,

A Proposed Synthesis

of

the

Mod

els

A n

atur

al p

ropo

sal t

hat a

rise

s fr

om th

is o

bser

vatio

n, t

hen

, is

to s

ugge

st th

atthe human memory is essentially a synthesis

of

the two types

of

mod

els

Ihave described above, One part

of

the

syst

em g

radu

ally

lear

ns to

rep

rese

ntan

d us

e co

ncep

ts a

s in

the

Hin

ton

mod

el, w

hile

ano

ther

par

t is

give

n th

e ta

skof

rapidly learning the specific content

of

indi

vidu

al e

vent

s an

d ex

peri

ence

sst

orin

g th

em in

a w

ay th

at is

sim

ilar

to th

e m

etho

d us

ed in

the

Tra

ce S

ynth

e-sis model

of McClelland (1981). I have presented a visualization

of

this

idea

in Figure 2,

Thi

s pr

opos

al m

ay s

eem

at f

irst

som

ewha

t unp

arsi

mon

ious

, but in fact

it provides an account

of

the pattern

of

amne

sia

that

res

ults

fro

m b

ilate

ral

lesi

ons

to th

e m

edia

l tem

pora

l lob

es. I

ndiv

idua

ls w

ith e

xten

sive

dam

age

tothese brain regions show a very striking pattern

of

mem

ory

defic

its (

for

over

-vi

ews

see

Squi

re, Chapter 7

of

this

vol

ume)

, The

se p

atie

nts

appe

ar p

ro-

foun

dly

defi

cien

t in

the

abili

ty to

for

m n

ew s

eman

tic o

r ep

isod

ic m

emo

but t

he a

bilit

y to

acq

uire

new

impl

icit

know

ledg

e su

ch a

s ne

w c

ogni

tive

skill

s or

sen

sitiv

ity to

the

sequ

entia

l dep

ende

ncie

s am

ong

stim

uli i

n im

plic

itle

arni

ng ta

sks

rem

ains

inta

ct, a

nd e

xist

ing

sem

antic

kno

wle

dge

such

as

se-

man

tic a

ssoc

iatio

ns c

an b

e pr

imed

. Sem

antic

and

epi

sodi

c kn

owle

dge

ac-

quir

ed lo

ng b

efor

e th

e da

mag

e oc

curr

ed is

spa

red-

that

is, i

t is

as g

ood

insu

ch p

atie

nts

as, i

t is

in a

ge-m

atch

ed c

ontr

ols,

In

fact

, the

re is

a te

mpo

rally

grad

ed r

etro

grad

e am

nesi

a , w

hich

in h

uman

s ca

n ex

tend

ove

r se

vera

l yea

rssu

ch th

at s

eman

tic a

nd e

piso

dic

mem

orie

s th

at w

ere

acqu

ired

sho

rtly

bef

ore

the occurrence

of

the

dam

age

are

prof

ound

ly a

ffect

ed, a

nd m

emor

ies

that

wer

e ac

quire

d at

pro

gres

sive

ly e

arlie

r tim

es a

re p

rogr

essi

vely

less

and

less

affe

cted

. Cru

cial

ly, i

n se

vera

l stu

dies

bot

h in

hum

ans

and

in o

ther

ani

mal

sm

emor

y fo

r th

e m

ost r

ecen

t pre

mor

bid

time

peri

ods

can

actu

ally

be

~uc

hw

orse

than

mem

ory

for

mat

eria

l fro

m s

light

ly m

ore

rem

ote

time

perI

ods,