Embed Size (px)
Citation preview
Mem
ory
Dis
tort
ion
Con
stru
ctiv
e M
emor
y an
d M
emor
yD
isto
rtio
ns: A
Par
alle
l-D
istr
ibut
edPr
oces
sing
App
roac
h
How
Min
ds, B
rain
s, a
nd S
ocie
ties
Rec
onst
ruct
the
Past
Edi
tor
Daniel L. Schacter
Contributing Editors
Joseph T. Coyle
Gerald D. Fischbach
Mar
ek-M
arse
l Mes
ulam
Law
renc
e E
. Sul
livan
Jam
es L
. McC
lella
nd
Bar
tlett
(193
2) in
trod
uced
and
insi
sted
on
the
view
that
mem
ory
isa
cons
truc
tive
proc
ess,
His
vie
w w
as e
ssen
tially
that
rec
all i
s no
t a r
etri
eval
,bu
t a r
econ
stru
ctio
n, in
whi
ch a
spec
ts o
f the
con
tent
of p
revi
ousl
y pr
esen
ted
mat
eria
l are
wov
en in
to a
coh
eren
t who
le, w
ith th
e ai
d of
pre
exis
ting
know
l-ed
ge. D
etai
ls m
ay b
e di
stor
ted
to in
crea
se c
oher
ence
; rat
iona
lizat
ions
not
pres
ent i
n th
e or
igin
al m
ay b
e in
trod
uced
; det
ails
that
are
con
sist
ent w
ithth
e sy
nthe
size
d co
here
nt s
tory
may
be
adde
d; a
nd d
etai
ls th
at a
re in
cons
is-
tent
may
be
drop
ped.
Nei
sser
(19
67)
liken
ed b
oth
perc
eptio
n an
d m
emor
yto
the
cons
truc
tive
activ
ities
of a
pal
eont
olog
ist,
who
use
s a
colle
ctio
n of
bone
fra
gmen
ts, a
s w
ell a
s ev
eryt
hing
she
kno
ws
abou
t din
osau
rs f
rom
pre
-vi
ous
expe
rien
ce, t
o re
cons
truc
t the
ske
leto
n of
a p
artic
ular
din
osau
r, T
hese
idea
s ar
e co
nsis
tent
with
wha
t we
wou
ld r
efer
to to
day
as a
con
stra
int s
atis
-faction process, i
n w
hich
rem
embe
ring
is s
imul
tane
ousl
y co
nstr
aine
d by
trac
es le
ft in
the
min
d by
the
even
t we
are
rem
embe
ring
itse
lf, b
y ba
ckgr
ound
know
ledg
e of
rel
ated
mat
eria
l, a
nd b
y co
nstr
aint
s an
d in
fluen
ces
impo
sed
by th
e si
tuat
ion
surr
ound
ing
the
act o
f rec
olle
ctio
n. O
bvio
usly
if m
emor
yis
con
stru
ctiv
e in
this
way
, thi
s ha
s pr
ofou
nd im
plic
atio
ns f
or th
e qu
estio
nof
the
veri
dica
lity
of m
emor
y an
d th
e ex
tent
to w
hich
it m
ay b
e in
flue
nced
by suggestion, p
reex
istin
g kn
owle
dge
, and
oth
er r
elat
ed e
xper
ienc
es.
My
inte
rest
is in
the
mec
hani
sms
that
may
impl
emen
t thi
s co
nstr
uctiv
eco
nstr
aint
sat
isfa
ctio
n pr
oces
s. R
emem
beri
ng, I
will
arg
ue, t
akes
pla
ce in
apa
ralle
l dis
trib
uted
pro
cess
ing
syst
em-a
.sy
stem
con
sist
ing
of a
larg
e nu
m-
ber
of s
impl
e bu
t mas
sive
ly in
terc
onne
cted
pro
cess
ing
units
, Pro
cess
ing
insu
ch s
yste
ms
take
s pl
ace
thro
ugh
the
prop
agat
ion
of a
ctiv
atio
n am
ong
the
units
, bas
ed o
n ex
cita
tory
and
inhi
bito
ry c
onne
ctio
ns. F
orm
ing
a m
emor
ytr
ace
for
som
ethi
ng-s
ay, a
n ep
isod
e or
eve
nt-b
egin
s w
ith th
e co
nstr
uctio
n
Har
vard
Uni
vers
ity P
ress
Cam
brid
ge, M
assa
chus
etts
Lon
don,
Eng
land
1995
COGNITIVE PERSPECTIVES
of a pattern of activity over t~e pr
oces
sing
uni
ts, w
ith th
e ex
peri
ence
itse
lfst
rong
ly in
flue
ncin
g th
e pa
ttern
, But
the
exis
ting
conn
ectio
ns a
mon
g th
eun
its w
ill a
lso
infl
uenc
e th
e pa
ttern
con
stru
cted
, the
reby
intr
oduc
ing
the
poss
ibili
ty o
f ad
ditio
ns, o
mis
sion
s, a
nd d
isto
rtio
ns, S
tora
ge o
f a tr
ace
ofth
e ep
isod
e or
eve
nt th
en o
ccur
s th
roug
h th
e m
odif
icat
ion
of th
e st
reng
ths
of th
e co
nnec
tions
am
ong
the
units
; to
a fi
rst a
ppro
xim
atio
n, t
he c
onne
ctio
nfr
om a
uni
t tha
t is
activ
e in
the
repr
esen
tatio
n to
ano
ther
suc
h ac
tive
unit
will
tend
to in
crea
se in
str
engt
h, w
hile
the
stre
ngth
of
conn
ectio
ns b
etw
een
active and inactive units will tend to decrease,
Rem
embe
ring
may
occ
ur w
hen
som
e as
pect
or
aspe
cts
of a
n ev
ent a
rise
agai
n as
inpu
t. T
his
may
act
ivat
e so
me
of th
e un
its th
at p
revi
ousl
y pa
rtic
i-pa
ted
in th
e re
pres
enta
tion
of th
e ep
isod
e or
eve
nt, a
nd th
ese
may
in tu
rnac
tivat
e ot
her
units
, via
the
wei
ghte
d co
nnec
tions
, The
pat
tern
that
is c
on-
stru
cted
aga
in d
epen
ds o
n th
e co
nnec
tions
am
ong
the
units
, and
sin
ce th
ese
wer
e ad
just
ed p
revi
ousl
y w
hen
the
epis
ode
was
fir
st e
xper
ienc
ed, t
he p
atte
rn -
that
is c
onst
ruct
ed w
ill te
nd to
cor
resp
ond
to th
e pa
ttern
that
was
pre
sent
at th
e tim
e of
'storage. But the units that participate in representing one
epis
ode
or e
vent
als
o pa
rtic
ipat
e in
rep
rese
ntin
g ot
her
epis
odes
, and
so
the
repr
esen
t~tio
n th
at is
con
stru
cted
may
be
affe
cted
by
man
y ot
her
expe
ri-en
ces.
Thi
s m
eans
that
my
mem
ory
of a
nyon
e ep
isod
e or
eve
nt w
ill te
ndto
ref
lect
the
infl
uenc
e of
wha
t I h
ave
lear
ned
from
man
y ot
her
epis
odes
or
even
ts.
I w
ill d
escr
ibe
two
mod
els
that
cap
ture
this
con
stru
ctiv
e pr
oces
s in
dif
fer-
ent w
ays,
Bot
h m
odel
s ha
ve th
eir
orig
ins
in e
arly
con
nect
ioni
st p
aper
s, o
neby
mys
elf
(McC
lella
nd, 1
981)
and
one
by
Hin
ton
(198
1), N
eith
er m
odel
isfu
lly a
dequ
ate
in it
self
, but
I w
ill p
ropo
se a
syn
thes
is o
f th
e tw
o th
at m
ayca
ptur
e so
me
of th
e m
ain
feat
ures
of
hum
an m
emor
y, in
clud
ing
aspe
cts
ofm
emor
y di
stor
tions
. The
syn
thes
is m
ay p
rovi
de a
s w
ell o
ne w
ay o
f th
inki
ngabout amnesia.
A T
race
Syn
thes
is M
odel
The
fir
st m
odel
(M
cCle
lland
, 198
1) f
ocus
es o
n di
stor
tion
proc
esse
s th
at c
anoc
cur
duri
ng a
cts
of r
emem
beri
ng, u
sing
a s
impl
e, lo
calis
t con
nect
ioni
st n
et-
wor
k fo
r th
e st
orag
e an
d re
trie
val o
f in
form
atio
n, I
use
d in
the
exam
ple
the
task
of
rem
embe
ring
fac
ts a
bout
a c
olle
ctio
n of
two
som
ewha
t uns
avor
yin
divi
dual
s, b
elon
ging
to tw
o m
ade-
up g
angs
, the
Jet
s an
d th
e S
hark
s, T
he
Jets tended to be in their twenties, t
o be
sin
gle
, and
to h
ave
only
a J
unio
rH
igh
Scho
ol e
duca
tion,
thou
gh n
o on
e Je
t had
all
thes
e ch
arac
teri
stic
s; th
eSh
arks
tend
ed to
be
olde
r, to
be
mar
ried
, and
to h
ave
atte
nded
Hig
h S
choo
lth
ough
aga
in n
o on
e Sh
ark
had
all t
hese
pro
pert
ies,
Mem
bers
of
both
gan
gsw
ere
equa
lly li
kely
to b
e pu
sher
s, b
ooki
es, o
r bu
rgla
rs,
Con
stru
ctiv
e M
emor
y an
d M
emor
y D
isto
rtio
ns
I: .
Figu
re
1 T
he n
etw
ork
used
to d
emon
stra
te a
spec
ts o
f co
nstr
uctiv
e m
emor
y by
McC
lel-
land
(19
81),
The
uni
ts p
artic
ipat
ing
in th
e re
pres
enta
tion
of a
few
of
the
mem
bers
of
the
Jets
and
Sha
rks
gang
s de
scri
bed
in th
e te
xt a
re s
how
n, U
nits
with
in th
e sa
me
grou
p ar
em
utua
lly in
hibi
tory
; uni
ts c
onne
cted
with
bid
irec
tiona
l arr
ows
are
mut
ually
exc
itato
ry,
From
McC
lella
nd (
1981
), f
ig, 1
, p, 1
71.
In th
e m
odel
, I r
epre
sent
ed e
ach
indi
vidu
al w
ith it
s ow
n co
nnec
tioni
stpr
oces
sing
uni
t tha
t I w
ill c
all a
n in
stan
ce u
nit (
see
Figu
re 2
.1),
The
mod
elal
so c
onta
ined
pro
pert
y un
its, o
ne fo
r ea
ch p
rope
rty
an in
divi
dual
mig
htha
ve, N
ames
wer
e tr
eate
d as
pro
pert
ies ,
so
ther
e w
ere
units
for
nam
es, f
orga
ng m
embe
rshi
p, f
or e
duca
tion,
for
mar
ital s
tatu
s , a
nd fo
r oc
cupa
tion,
Bi-
dire
ctio
nal,
exc
itato
ry c
onne
ctio
ns b
etw
een
units
wer
e us
ed to
link
inst
ance
units to the units representing their properties, s
o th
at if
one
activated a
nam
e un
it, i
t act
ivat
ed th
e co
rres
pond
ing
inst
ance
uni
t , and the instance
unit
activ
ated
the
othe
r pr
oper
ties
of th
e in
stan
ce. T
he in
stan
ce u
nits
for
med
a gr
oup
of u
nits
that
wer
e m
utua
lly in
hibi
tory
, so
that
if o
ne w
as a
ctiv
e it
tend
ed to
sup
pres
s th
e ot
hers
; sim
ilarl
y, th
e un
its f
or e
ach
type
of
prop
erty
wer
e gr
oupe
d in
to c
lust
ers
of m
utua
lly in
hibi
tory
uni
ts. T
he u
se o
f si
ngle
units
to r
epre
sent
who
le it
ems
is a
sim
plif
icat
ion-
I w
ill a
rgue
late
r th
at it
CO
GN
ITIV
E P
ER
SPE
CT
IVE
S
is m
ore
corr
ect t
o us
e di
stri
bute
d re
pres
enta
tions
, bot
h fo
r th
e w
hole
and
for
the
part
s, H
owev
er, e
ven
this
sim
ple
mod
el c
aptu
res
cruc
ial a
spec
ts o
fth
e ki
nds
of r
econ
stru
ctiv
e pr
oces
ses
that
take
pla
ce d
urin
g re
mem
beri
ng,
My
focu
s in
the
orig
inal
wor
k re
port
ed in
McC
lella
nd (
1981
) w
as o
n th
epr
oces
s of
con
stru
ctin
g re
pres
enta
tions
of
mat
eria
l not
exp
licitl
y st
ored
inm
emor
y, O
ne s
uch
case
invo
lved
con
stru
ctin
g a
com
posi
te r
ecol
lect
ion
ofth
e ty
pica
l Jet
or
Shar
k. I
n th
e m
odel
this
cou
ld b
e do
ne s
impl
y by
act
ivat
ing
the
unit
for
Jet,
Thi
s un
it th
en s
ent a
ctiv
atio
n to
the
inst
ance
uni
ts f
or a
llof the Jets, a
nd th
ese
in tu
rn s
ent a
ctiv
atio
n to
the
units
for
eac
h in
stan
cepr
oper
ties,
The
inhi
bitio
n am
ong
the
inst
ance
uni
ts p
reve
nted
any
of
thes
eun
its f
rom
bec
omin
g to
o st
rong
ly a
ctiv
ated
, but
they
all
cont
ribu
ted
som
eac
tivat
ion
to th
eir
prop
erty
uni
ts, A
s a
resu
lt th
e pr
oper
ties
of th
e ty
pica
lJe
t bec
ame
activ
e (a
ge in
20'
, sin
gle
, JH
edu
catio
n); a
ll of
the
occu
patio
nsw
ere
part
ially
act
ivat
ed.
Thi
s fi
rst e
xam
ple
show
s a
desi
rabl
e pr
oper
ty o
f th
is s
ort o
f m
emor
y sy
s-te
m-i
t can
spo
ntan
eous
ly g
ener
aliz
e fr
om e
xam
ples
. Ano
ther
pro
pert
y-w
hich
may
ofte
n be
des
irabl
e bu
t whi
ch c
an a
lso
be u
ndes
irabl
e-is
rev
eale
d
whe
n th
e m
odel
is u
sed
to tr
y to
ret
riev
e th
e pr
oper
ties
of a
sin
gle
indi
vidu
alby
act
ivat
ing
the
unit
for
that
indi
vidu
al's
nam
e. T
his
indi
vidu
al's
pro
pert
ies
tend
to b
e m
ore
stro
ngly
act
ivat
ed th
an th
e pr
oper
ties
of a
ny o
ther
indi
vid-
ual,
but
one
fin
ds th
at a
s th
e ac
tivat
ion
proc
ess
goes
on,
oth
er, s
imila
r in
di-
vidu
als
beco
me
part
ially
act
ivat
ed, T
his
happ
ens
beca
use
as th
e pr
oper
ties
of th
e ta
rget
indi
vidu
al b
ecom
e ac
tive,
they
sen
d ac
tivat
ion
to th
e in
stan
ceun
its fo
r ot
her
indi
vidu
als
, and
thes
e, in
turn
, ten
d to
act
ivat
e th
e un
its fo
rth
eir
prop
ertie
s. T
his
effe
ct c
an b
e pa
rtic
ular
ly p
oten
t-an
d ca
n le
ad to
strong distortions-when some piece of
info
rmat
ion
abou
t the
targ
et in
di-
vidu
al is
mis
sing
. To
show
this
, I f
irst
del
eted
the
conn
ectio
n be
twee
n th
ein
stan
ce n
ode
for
Lan
ce a
nd th
e pr
oper
ty n
ode
for
his
occu
patio
n-he
hap
-pe
ned
to b
e a
burg
lar,
The
n I
activ
ated
the
nam
e un
it fo
r L
ance
, Thi
s ca
used
the
inst
ance
nod
e to
bec
ome
activ
e, a
nd th
e in
stan
ce n
ode
then
act
ivat
edth
e pr
oper
ty n
odes
for
Jet
, 20'
, JH
edu
catio
n, a
nd m
arrie
d. N
ow, t
here
happ
ened
to b
e se
vera
l oth
er J
ets
who
had
man
y of
thes
e pr
oper
ties,
and
they
all
happ
ened
to b
e bu
rgla
rs. A
s a
resu
lt, th
e m
odel
fill
ed in
this
occ
upa-
tion
for
Lan
ce. I
n th
is in
stan
ce th
e re
sult
happ
ened
to b
e co
rrec
t, bu
t the
sam
e th
ing
wou
ld o
f co
urse
hav
e ha
ppen
ed w
heth
er L
ance
had
bee
n a
bur-
glar
or
not,
Had
he
been
a p
ushe
r or
a b
ooki
e (o
r, f
or th
at m
atte
r, s
omeo
ne
with
an
entir
ely
inno
cuou
s oc
cupa
tion)
, the
mod
el w
ould
hav
e fil
led
in b
ur-
glar
any
way
. In
that
cas
e th
is w
ould
hav
e be
en a
cle
ar e
xam
ple
of a
mem
ory
distortion: Lance would have been guilty by association.
The
mod
el il
lust
rate
s tw
o ke
y po
ints
cen
tral
to th
e is
sues
rai
sed
in th
isvo
lum
e, F
irst,
it pr
ovid
es a
n ex
plic
it th
ough
sim
ple
mec
hani
sm il
lust
ratin
gho
w m
emor
y di
stor
tions
can
aris
e fr
om th
e w
orki
ngs
of o
rdin
ary
mem
ory
retr
ieva
l pro
cess
es, T
hese
pro
cess
es a
re o
ften
ben
efic
ial-t
hey
allo
w th
e fo
r-
Con
stru
ctiv
e M
emor
y an
d M
emor
y D
isto
rtio
ns
mat
i on
of g
ener
aliz
atio
ns o
ver
sim
ilar
inst
ance
s an
d th
e fi
lling
in o
f m
issi
ngpr
oper
ties
base
d on
the
prop
ertie
s of
oth
er, s
imila
r in
divi
dual
s-but they
can
pote
ntia
lly b
e ha
rmfu
l in
that
the
info
rmat
ion
fille
d in
need not be
corr
ect.
Seco
nd
, , th
e m
odel
has
the
sam
e pr
oper
ty th
at h
uman
mem
ory
has,
of
ofte
n fa
iling
to s
epar
ate
info
rmat
ion
that
ari
ses
from
dif
fere
nt s
ourc
es, S
up-
pose
that
a n
ew in
stan
ce n
ode
is f
orm
ed f
rom
eve
ry e
xper
ienc
e (a
pro
posa
lst
riki
ngly
sim
ilar
to th
e m
emor
y m
odel
of
Hin
tzm
an, 1
988)
, and
sup
pose
one
has
a nu
mbe
r of
sim
ilar
expe
rien
ces.
The
n w
hen
we
try
to r
ecal
l one
pieces of other similar experiences will tend to intrude particularly in those
aspe
cts
of th
e or
igin
al f
or w
hich
the
info
rmat
ion
is w
eak
or missing. In
the
mod
el, it is un
fort
unat
ely
not p
ossi
ble
to in
spec
t eac
h m
emor
y tr
ace
indi
vidu
ally
; the
info
rmat
ion
is n
ot s
tore
d in
the
units
them
selv
es b
ut in
thei
r co
mie
ctio
ns; l
ike
conn
ectio
ns a
mon
g ne
uron
s in
the
brai
n, ~
e on
lykn
ow w
hat i
s st
ored
in th
e co
nnec
tions
thro
ugh
the
effe
cts
thes
e co
nnec
tions
have
on
the
outc
ome
of p
roce
ssin
g. B
ut, s
ince
man
y un
its a
nd c
onne
ctio
nsco
ntri
bute
to th
is o
utco
me,
ful
l dis
enta
nglin
g of
the
spec
ific
cau
se o
f ea
chas
pect
of
the
outc
ome
is im
poss
ible
. It w
ill, t
hen,
in g
ener
al n
ot b
e po
ssib
leto
iden
tify
the
spec
ific
sou
rce
of a
ny a
spec
t of
cons
truc
ted
reco
llect
ion.
Giv
en th
e m
odel
, the
n, t
he m
emor
y di
stor
tions
rep
orte
d in
this
vol
ume
by Loftus, Ceci, M
osco
vitc
h, and others are to be expected, Perhaps the
only
thin
g th
at is
une
xpec
ted
abou
t the
m is
the
resi
stan
ce th
at o
ften
ari
ses
to th
eir
acce
ptan
ce. T
his
resi
stan
ce m
ay c
ome
from
impl
icit
acce
ptan
ce o
fan
alte
rnat
ive
mod
el o
f m
emor
y, in
whi
ch m
emor
y tr
aces
are
not
so
muc
hco
nstr
ucte
d as
ret
riev
ed, l
ike
book
s fr
om a
libr
ary.
The
met
apho
r of
hum
anm
emor
y an
d hu
man
kno
wle
dge
as a
libr
ary
prov
ides
a b
asis
for
acc
ount
ing
for
the
role
of
orga
niza
tion
in m
emor
y, b
ut g
ives
no
basi
s fo
r un
ders
tand
ing
dist
ortio
n. I
bel
ieve
that
as
we
com
e to
und
erst
and
mem
ory
bette
r an
d be
t-te
r, it
will
bec
ome
incr
easi
ngly
cle
ar th
at th
is is
a m
isle
adin
g m
etap
hor.
An
Exp
erim
enta
l Tes
t of
the
Tra
ce S
ynth
esis
Mod
el
To
test
the
mod
el d
escr
ibed
abo
ve, a
nd to
ext
end
the
empi
rica
l dat
a ba
seof
evi
denc
e of
mem
ory
dist
ortio
ns, L
eigh
Nys
trom
and
I d
evel
oped
an
expe
r-im
enta
l par
adig
m d
esig
ned
to e
licit
trac
e sy
nthe
sis
erro
rs in
rem
embe
ring
(Nys
trom
and
McC
lella
nd, 1
992)
. In
this
par
adig
m, s
ubje
cts
stud
y a
list o
fse
nten
ces,
and
then
are
late
r cu
ed fo
r co
mpl
ete
reca
ll of
indi
vidu
al w
hole
sent
ence
s w
hen
a fr
agm
ent i
s pr
esen
ted
as a
pro
be, C
onsi
der
sent
ence
s of
the form: "T
he p
olic
eman
gav
e th
e ac
coun
tant
the
ham
mer
in th
e ba
se-
men
t."
We
imag
ine
that
the
sent
ence
is a
naly
zed
into
a s
et o
f ro
le-f
iller
pai
rsw
hich
are
then
rep
rese
nted
by
the
activ
atio
ns o
f in
put u
nits
in th
e ne
twor
ksh
own
in F
igur
e 2.
2, T
he n
etw
ork
is s
tric
tly a
nalo
gous
to th
e m
odel
pre
vi-
ousl
y de
scri
bed,
The
re is
a p
ool o
f in
stan
ce u
nits
, with
one
uni
t ass
igne
d to
COGNITIVE PERSPECTIVES
Con
stru
ctiv
e M
emor
y an
d M
emor
y D
isto
rtio
ns
Prop
ositi
onT
able
E
xam
ple
over
lap
and
cont
rol s
ente
nce
pair
s fr
om N
ystr
om a
ndM
cCle
lland
(19
92),
with
cor
resp
ondi
ng te
st p
robe
s
Ove
rlap
Pai
r:
' , ,
The
pol
icem
an g
ave
the
acco
unta
nt th
e ha
mm
er il
l the
din
ing
room
,T
he f
arm
er g
ave
the
acco
unta
nt th
e ha
mm
er in
the
gara
ge,
Con
trol
Pai
r:T
he d
river
sho
wed
the
rece
ptio
nist
the
toas
ter
in th
e ki
tche
n.T
he s
wim
mer
loan
ed th
e sa
lesm
an th
e en
velo
pe in
the
base
men
t.
Ove
rlap
Pr
obe:
The
, , ,
gave
the
acco
unta
nt th
e ha
mm
er in
the.
.
Con
trol
Pr
obe:
The
, , ,
show
ed th
e re
cept
ioni
st th
e to
aste
r in
the,
,
Age
ntA
ctio
nR
ecip
ient
Obj
ect
Loc
atio
nsentence, One such pair
of
over
lapp
ing
sent
ence
s is
sho
wn
in T
able
2, 1
; apa
ir
of control sentences, w
ith n
o ov
erla
p, is
als
o sh
own,
In
the
mod
el, w
.eas
sign
ed a
dif
fere
nt p
ropo
sitio
n un
it to
eac
h se
nt~n
ce, a
nd c
onne
cted
this
unit to the input units for each
of
the correspondmg words. In the case
overlap sentences
, thr
ee
of
the
five
role
-fill
ers
are
~he
sam
e , S?
the
pr~p
osl-
tion units in these cases are connected to overlappmg sets
of
mpu
t UnI
ts, a
ssh
own
in F
igur
e 2.
3.
Figu
re
2 Sketch
of the network architecture used by
Nys
trom
an
d M
cCle
lland
(19
92),
The units participating in the representation
of
a pa
rtic
ular
pro
posi
tion
are
show
n , a
long
with the bidirectional connections that allow this model to perform cued recall
of
the
who
lesentence when' part
of
the
sent
ence
is r
epre
sent
ed a
s a
cue,
each proposition, The model also contains several pools
of
prop
erty
uni
tsone for each type
of
role that can occur in one
of
the
sent
ence
s. E
ach
pool
cont
ains
a u
nit f
or e
ach
wor
d th
at c
an a
ppea
r in
that
rol
e. W
e re
pres
ent t
hese
nten
ce a
bout
the
polic
eman
as
show
n in
the
figu
re, A
noth
er s
ente
nce , w
itha non-overlapping set
of
wor
ds, w
ould
invo
lve
a di
ffer
ent w
ord-
in-r
ole
unit
in e
ach
prop
erty
uni
t poo
l , a
nd a
diff
eren
t pro
posi
tion
unit.
In a
mod
el s
uch
as th
is, cued remembering occurs
by
sim
ply
turn
ing
onth
e un
its f
or th
e w
ords
con
tain
ed in
the
prob
e , a
nd a
skin
g th
e ne
twor
k to
esse
ntia
lly fi
ll in
the
rest
, Thi
s oc
curs
via
a g
radu
al c
onst
rain
t sat
isfa
ctio
nprocess, Processing begins with the units for the cue words clamped and
continues until a stable pattern
of
activ
atio
n is
ach
ieve
d, W
hen
the
cue
uniq
uely
mat
ches
one
sto
red
sent
ence
, and
ther
e ar
e no
oth
er v
ery
sim
ilar
sent
ence
s , th
e co
rrec
t sen
tenc
e te
nds
to b
e re
calle
d, H
owev
er, e
rror
s ca
noc
cur
whe
n th
ere
are
two
or. m
ore
stor
ed it
ems
that
hav
e th
e sa
me
or a
similar degree
of
mat
ch to
the
prob
e. I
n th
is c
ase
the
" rem
embe
red"
pat
tern
is a constructed synthesis
of two or more stored traces,
Nystrom and I studied the adequacy
of
this
mod
el to
acc
ount
for
mem
ory
and memory distortions in a series
of
four
exp
erim
ents
. Her
e I
will
dis
cuss
only one
of these. The subjects studied lists
of
sent
ence
s se
t up
so th
at s
ome
of
the
sent
ence
s sh
ared
thre
e co
nten
t wor
ds in
com
mon
with
ano
ther
pai
red
Prop
ositi
on
Age
ntA
ctio
nR
ecip
ient
Obj
ect
Loc
atio
n
Figu
re
2.3
The
uni
ts a
nd c
onne
ctio
ns p
artic
ipat
i~g
in t~
e r~
pres
enta
tion
of a s~c
ond
sen-
.te
nce
that
ove
rlaps
with
the
first
one
sho
wn
prev
IOus
ly II
I Fig
ure
2,2, Presentation
of
ambi
guou
s pr
obe
tend
s to
act
ivat
e bo
th s
ente
nces
and
may
pro
duce
a b
lend
err
or,
COGNITIVE PERSPECTIVES
Con
stru
ctiv
e M
emor
y an
d M
emor
y D
isto
rtio
ns
Our
foc
us in
this
res
earc
h w
as o
n tr
ace
synt
hesi
s at
the
time
of r
ecal
lan
d w
e th
eref
ore
wen
t to
som
e le
ngth
s to
min
imiz
e th
e po
ssib
ility
that
sub
-je
cts
wou
ld b
e re
min
ded
of th
e fi
rst m
embe
r of
eac
h ov
erla
p pa
ir w
hen
give
nth
e se
cond
mem
ber
duri
ng th
e ex
posu
re p
hase
of
the
expe
rim
ent.
Thi
s w
asdo
ne, f
irst
of
all,
by
deve
lopi
ng a
cov
er ta
sk th
at f
ocus
ed s
ubje
cts
on a
naly
z-in
g ea
ch in
divi
dual
sen
tenc
e se
para
tely
fro
m a
ll of
the
othe
rs w
ithou
t any
men
tion
of a
late
r m
emor
y te
st: s
ubje
cts
wer
e to
ld th
eir
task
was
to r
ate
each
sen
tenc
e fo
r its
ove
rall
plau
sibi
lity
and
to s
ay h
ow w
ell t
hey
thou
ght
each
wor
d fi
t with
the
over
all e
vent
des
crib
ed b
y th
e w
hole
sen
tenc
e. O
ther
prec
autio
ns in
clud
ed v
aryi
ng th
e pl
acem
ent w
ithin
the
sent
ence
s of
the
over
-la
ppin
g w
ords
, and
sep
arat
ing
over
lapp
ing
sent
ence
s as
far
as
poss
ible
inth
e st
udy
list o
f 34
sen
tenc
es, A
lthou
gh s
ubje
cts
did
notic
e th
at s
ome
wor
dsw
ere
occa
sion
ally
rep
eate
d , o
nly
a fe
w s
ubje
cts
repo
rted
that
the
seco
ndm
embe
r of
an
over
lap
pair
eve
r ca
used
them
to r
ecal
l the
pre
viou
s m
embe
rof
the
pair
, and
elim
inat
ing
thes
e su
bjec
ts f
rom
the
anal
yses
did
not
cha
nge
the
resu
lts, T
hus
we
wer
e re
ason
ably
con
fide
nt th
at r
emin
ding
and
trac
esy
nthe
sis
duri
ng th
e st
udy
phas
e w
as n
ot a
maj
or f
acto
r in
det
erm
inin
g th
ere
sults
.A
fter
the
subj
ects
com
plet
ed a
fill
er ta
sk f
or 5
min
utes
, the
cue
d re
call
.ph
ase
of th
e ex
peri
men
t was
adm
inis
tere
d. T
his
invo
lved
pre
sent
ing
16 s
en-
tenc
e fr
agm
ents
, eac
h w
ith b
lank
s fo
r tw
o co
nten
t wor
ds, E
ight
frag
men
tsw
ere
from
ove
rlap
pai
rs a
nd e
ight
wer
e fr
om c
ontr
ol p
airs
. Thu
s a
subj
ect
mig
ht s
ee p
robe
s lik
e th
e on
es s
how
n in
Tab
le 2
,In both cases, t
he s
ubje
cts
wer
e to
ld to
com
plet
e th
e pr
obe
with
the
firs
tst
udie
d se
nten
ce th
at c
ame
to m
ind,
Sub
ject
s w
ere
aler
ted
to th
e fa
ct th
atso
met
imes
the
prob
e m
atch
ed tw
o st
udie
d se
nten
ces
equa
lly w
ell ,
and
they
wer
e to
ld to
rec
all o
nly
one
of th
e tw
o sentences and to take care not to
mix
up
the
two,
Aft
er th
e fi
rst r
ecal
l the
y w
ere
give
n an
opp
ortu
nity
to r
ecal
lth
e se
cond
sen
tenc
e. I
will
be
disc
ussi
ng o
nly
the
resu
lts o
f th
e fi
rst r
ecal
lhe
re. S
uffi
ce it
to s
ay th
at s
econ
d re
calls
wer
e ge
nera
lly le
ss a
ccur
ate
than
firs
t rec
alls
,O
n th
e fi
rst r
ecal
ls, t
he p
roba
bilit
y of
cor
rect
ly r
ecal
ling
a co
mpl
ete
sen-
tenc
e di
d no
t dif
fer
betw
een
the
over
lap
and
cont
rol p
airs
: tw
o w
ords
fro
mth
e sa
me
sent
ence
that
mat
ched
the
prob
e w
ere
reca
lled
42%
of
the
time
inth
e ov
erla
p co
nditi
on a
nd 4
1 %
of
the
time
in th
e co
ntro
l con
ditio
n. H
ow-
ever
, in
the
over
lap
cond
ition
sub
ject
s di
d so
met
imes
mak
e w
hat w
e ca
nca
ll sy
nthe
sis errors-errors in which one w
ord
cam
e fr
om o
ne o
f th
e se
n-te
nces
that
mat
ched
the
prob
e an
d th
e ot
her
cam
e fr
om th
e ot
her
of th
ese
two
sent
ence
s, T
his
occu
rred
on
near
ly 1
0% o
f th
e er
ror
tria
ls (
5.4%
of
all
tria
ls w
ith o
verl
ap p
robe
s , c
ompa
red
to le
ss th
an 1
% o
f tr
ials
with
con
trol
prob
es).
The
rat
e of
syn
thes
is e
rror
s m
ay s
eem
rel
ativ
ely
low
, but
they
-wer
ere
liabl
y m
ore
freq
uent
than
cha
nce,
Con
fiden
ce r
atin
gs w
ere
obta
ined
on
each
rec
all t
rial
; con
fide
nce
was
slig
htly
less
on
aver
age
for
synt
hesi
s er
rors
than for completely correct responses, b
ut o
n 40
% o
f th
e sy
nthe
sis
erro
rssu
bjec
ts g
ave
the
high
est c
onfi
denc
e ra
ting,
cor
resp
ondi
ng to
the
stat
emen
tI a
m s
ure
both
wor
ds r
tx:a
lled
cam
e fr
om a
sin
gle
stud
ied
sent
ence
that
mat
ched
the
prob
e."
We
take
the
expe
rim
ent,
then
, as
dem
onst
ratin
g th
atm
emor
y di
stor
tions
can
ari
se f
rom
trac
e sy
nthe
sis
at th
e tim
e of
rec
all.
We
wou
ld n
ot w
ant t
o cl
aim
, of c
ours
e, t
hat t
race
syn
thes
is d
oes
not o
ften
occ
urea
rlie
r, w
hen
an in
terv
enin
g ev
ent r
emin
ds u
s of
a p
revi
ous
even
t; in
deed
it m
ay b
e th
at th
is is
one
com
mon
sou
rce
of m
emor
y di
stor
tions
. We
wou
ldon
ly s
ugge
st th
at o
ur r
esul
ts s
uppo
rt th
e vi
ew th
at tr
ace
synt
hesi
s ca
n oc
cur
at r
ecal
l as
wel
l as
betw
een
initi
al s
tudy
and
rec
all.
We
mod
eled
the
data
fro
m th
is a
nd th
e ot
her
thre
e st
udie
s w
e co
nduc
ted
usin
g th
e m
odel
dis
cuss
ed p
revi
ousl
y. T
o fi
t the
dat
a it
was
nec
essa
ry to
mak
e tw
o ad
ditio
nal s
tipul
atio
ns: f
irst, that processing has an inherently
rand
om c
ompo
nent
; and
sec
ond
, tha
t sub
ject
s so
met
imes
fai
led
to e
ncod
eea
ch s
ente
nce
com
plet
ely,
We
impl
emen
ted
this
latte
r as
sum
ptio
n by
ran
-do
mly
elim
inat
ing
a fr
actio
n of
the
conn
ectio
ns b
etw
een
the
inpu
t and
pro
pun
its, T
hese
ass
umpt
ions
do
diff
eren
t and
impo
rtan
t wor
k in
acc
ount
ing
for the data,
The
firs
t ass
umpt
ion-
intr
insi
c va
riab
ility
-allo
ws
the
netw
ork
to s
elec
tes
sent
ially
ran
dom
ly b
etw
een
two
equa
lly g
ood
resp
onse
s in
cas
es w
here
two
stud
ied
sent
ence
s fi
t the
pro
be e
qual
ly w
ell.
Intr
insi
c va
riab
ility
is im
ple-
men
ted
sim
ply
by in
corp
orat
ing
norm
ally
dis
trib
uted
ran
dom
noi
se in
to th
ein
put t
o ea
ch u
nit.
Eac
h tim
e th
e un
it's
stat
e is
upd
ated
, thi
s no
ise
affe
cts
the
exac
t deg
ree
of a
ctiv
atio
n. I
f hi
gh levels of noise are used,.-th
e m
odel
beco
mes
tota
lly r
ando
m in
its
beha
vior
; but
with
sm
all a
mou
nts
of n
oise
the
vari
abili
ty e
ffec
tivel
y ca
uses
the
netw
ork
to c
hoos
e ra
ndom
ly a
mon
geq
ually
goo
d al
tern
ativ
es. W
ithou
t any
noi
se th
e ne
twor
k w
ill h
ave
a te
n-de
ncy
to p
artia
lly a
ctiv
ate
both
mat
chin
g sentences most of the time, a
ndw
ill n
ot te
nd to
rec
all o
ne o
r th
e ot
her:
with
the
rand
omne
ss in
pla
ce, o
nth
e ot
her
hand
, the
net
wor
k w
ill te
nd to
set
tle to
one
of
the
two
answ
ers.
The
idea
of
intr
insi
c va
riab
ility
in p
roce
ssin
g w
as in
trod
uced
into
con
nect
ioni
stm
odel
ing
by s
ever
al in
vest
igat
ors
in th
e m
id-1980s(Geman and Geman
1984
; Sm
olen
sky,
198
6; H
into
n an
d Se
jnow
ski,
198
6). I
hav
e su
gges
ted
else
-w
here
that
intr
insi
c va
riab
ility
is a
gen
eral
pro
pert
y of
hum
an c
ogni
tive
func
tion,
and
I th
ink
it is
nec
essa
ry if
we
are
to m
odel
'th
e ki
nds
of r
esul
tsw
e se
e in
a w
ide
rang
e of
task
s, s
uch
as fr
ee a
ssoc
iatio
n, s
tem
com
plet
ion
or p
erce
ptua
l ide
ntifi
catio
n, w
here
sub
ject
s ge
nera
lly e
mit
one
or th
e ot
her
of a
set
of
alte
rnat
ive
cohe
rent
res
pons
es, r
athe
r th
an a
ble
nd o
f man
y al
ter-
nativ
es (
McC
lella
nd, 1
991)
. Oth
ers
have
est
ablis
hed
that
the
outc
ome
ofth
is s
ettli
ng p
roce
ss is
opt
imal
fro
m th
e po
int o
f vi
ew o
f m
axim
izin
g th
epr
obab
ility
of
sele
ctin
g th
e co
rrec
t ans
wer
, giv
en th
at th
e w
eigh
ts a
ccur
atel
yen
code
info
rmat
ion
abou
t the
dom
ain
(Gem
an a
nd G
eman
, 198
4),
The
sec
ond
assu
mpt
ion~
enco
ding
fai
lure
s-al
low
s th
e m
odel
to a
c-
CaG
NIT
IVE
PE
RSP
EC
TIV
ES
caun
t sim
ulta
neau
sly
far
the
exis
tenc
e af
syn
thes
is e
rrar
s , ta
geth
er w
ith th
efa
ct th
at th
e pr
abab
ility
af c
orre
ct r
ecal
l did
nat
suf
fer
in th
e av
erla
p ca
ndi-
tian
cam
pare
d to
. the
can
tral
can
ditia
n, S
impl
ifyin
g a
bit ,
with
this
ass
ump-
tian
in p
lace
, car
rect
rec
all a
f a s
ingl
e se
nten
ce d
epen
ds a
n w
heth
er it
has
been
cam
plet
ely
enca
ded ,
and
the
prab
abili
ty a
f ca
mpl
ete
enca
ding
is in
de-
pend
ent a
f w
heth
er th
ere
is a
n av
erla
p se
nten
ce in
the "
stu
dy s
et. I
ncam
plet
een
cadi
ng a
ffer
s an
app
artu
nity
far
syn
thes
is e
rrar
s: w
e ab
tain
ed a
n ex
celle
ntfit
to. t
he d
ata
by a
ssum
ing
that
sub
ject
s fa
iled
to. e
ncad
e 20
% a
f th
e w
ards
.In cases where there are gaps in the encading af ane af the sentences, t
heat
her
can
cant
ribut
e, c
reat
ing
a m
emar
y di
star
tian.
Intr
usia
ns f
rom
the
athe
r se
nten
ce r
ush
in w
hen
the
mas
t act
ive
trac
e pr
avid
es n
o. in
farm
atia
n,
Sum
mar
y af
the
Tra
ce S
ynth
esis
Mod
el
The
mad
el I
have
des
crib
ed h
as c
ansi
dera
ble
appe
al a
s a
sim
ple
desc
riptiv
eac
caun
t af t
he p
roce
ss a
f mem
ary
trac
e sy
nthe
sis
in c
ued
reca
ll an
d ga
essa
me
way
taw
ard
impl
emen
ting
the
cans
truc
tive
mem
ary
retr
ieva
l pra
cess
af w
hich
Bar
tlett
and
Nei
sser
wro
te. I
sha
uld
nate
that
ther
e ar
e at
her
mad
-el
stha
t can
acc
aunt
far
trac
e sy
nthe
sis,
incl
udin
g th
e M
INE
RV
A m
adel
af
Hin
tzm
an (
1988
), a
s w
ell a
s th
e m
adel
s by
Met
calf
e (1
990)
and
by
Hum
-ph
reys
, Bai
n, a
nd P
ike
(198
9) th
at u
se d
istr
ibut
ed r
epre
sent
atia
ns (
see
also
.M
cCle
lland
and
Rum
elha
rt, 1
985)
. The
mad
el a
f M
etca
lfe
(199
0) h
as b
een
appl
ied
to. a
num
ber
af im
part
ant f
indi
ngs
an b
lend
err
ors
and
athe
r m
em-
ary
dist
artia
ns .
and
shaw
s th
at s
uch
mad
els
can
affe
r ve
ry n
ice
acca
unts
af
muc
h af
the
exis
ting
data
an
blen
ding
and
mem
ary
dist
artia
n.In spite af their success, a
ll af
thes
e m
adel
s la
ck s
amet
hing
. The
re a
reat
her ,
dee
per ,
mar
e fu
ndam
enta
l far
ces
at w
ark
shap
ing
mem
ary
perf
ar-
man
ce, T
hese
pra
cess
es, I
bel
ieve
, ape
rate
gra
dual
ly a
ver
the
caur
se a
f ca
g-ni
tive
deve
lapm
ent t
o. s
hape
the
way
we
repr
esen
t the
can
stitu
ents
af
mem
-ar
y tr
aces
- far
exa
mpl
e , th
e ca
ncep
ts th
at c
antr
ibut
e to
. pro
.pas
itian
s,T
hese
repr
esen
tatia
ns, i
n tu
rn, p
ravi
de th
e ba
sis
far
mar
e pa
wer
ful f
arm
s O
f can
est
ruct
ive
mem
ary
effe
cts,
Ele
phan
tC
olor
Gra
y
Mad
els
of R
epre
sent
atia
n F
orm
atio
n vi
a G
radu
al L
earn
ing
An
earl
y m
adel
that
pai
nted
the
way
taw
ard
this
idea
was
pre
sent
ed b
yH
inta
n (1
981)
. Thi
smad
el is
ske
tche
d in
Fig
ure
2.4,
It c
ansi
sts
af s
epar
ate
sets
af
units
far
rep
rese
ntin
g th
e fi
rst n
aun ,
rel
atia
n, a
nd s
ecan
d na
un a
fthree-term propasitians such as "F
ish
can
swim
" " S
amm
y is
a fi
shE
le-
phan
ts a
re g
ray,
" "
Cly
de is
whi
te" and so
. far
th, T
he m
adel
is s
imila
r to
.th
e pr
evia
usm
adel
, but
naw
eac
h w
ard
is a
pat
tern
af a
ctiv
atia
n av
er th
eap
prap
riate
uni
ts r
athe
r th
an a
sin
gle
activ
e un
it, T
he w
hale
pro
pasi
tian
isre
pres
ente
d as
a. p
atte
rn a
f ac
tivity
ave
r th
e th
ree
sets
af
cans
titue
nt u
nits
Figu
re
2.4
A s
light
ly e
labo
rate
d ve
rsio
n of
the
conn
ectio
nist
net
wor
k us
ed b
y H
into
n to
repr
esen
t pro
posi
tions
abo
ut th
e co
ncep
ts, E
ach
rect
angl
e re
pres
ents
a p
ool o
f co
nnec
tioni
stin
form
atio
n pr
oces
sing
uni
ts, a
nd e
ach
arro
w r
epre
sent
s.a
full
set o
f co
nnec
tions
fro
m e
ach
unit
on th
e se
ndin
g si
de o
f the
arr
ow to
eac
h un
it on
the
rece
ivin
g si
de, P
roce
ssin
g oc
curs
by th
e pr
opag
atio
n of
act
ivat
ion
amon
g th
e un
its v
ia th
e co
nnec
tion
wei
ghts
, Eac
h un
it si
m-
ply
sets
its
activ
atio
n to
a v
alue
bet
wee
n 0
and'
1 ba
sed
on th
e su
mm
ed in
put i
t rec
eive
sfr
om o
ther
uni
ts v
ia th
e w
eigh
ted
conn
ectio
ns, F
or e
xam
ple ,
the
conn
ectio
ns f
rom
the
Wor
d 1
inpu
t uni
ts td
the
Nou
n 1
repr
esen
tatio
n un
its a
llow
a p
atte
rn o
f ac
tivat
ion
repr
e-se
ntin
g th
e fi
rst w
ord
of a
pro
posi
tion
to a
ctiv
ate
the
appr
opri
ate
sem
antic
pat
tern
ove
r th
eN
oun
1 ,u
nits
, The
con
nect
ions
bet
wee
n th
e di
ffer
ent p
ools
of
cons
titue
nt u
nits
and
the
Prop units encode the system
s kn
owle
dge
abou
t pro
posi
tions
, and
aft
er th
e w
eigh
ts h
ave
been
acq
uire
d th
roug
h le
arni
ng, t
hese
con
nect
ions
allo
w th
e th
ird
cons
titue
nt o
f a
prop
osi-
tion
to b
e co
mpl
eted
giv
en th
e ot
her
two
cons
titue
nts
as in
puts
, Ret
urn
conn
ectio
ns f
rom
the
cons
titue
nt u
nits
to th
e w
ord
units
then
pro
vide
.for
out
put o
f th
e pa
ttern
fill
ed in
by
the
netw
ork,
The
upp
er p
anel
sho
ws
each
poo
l of
units
with
labe
ls, T
he lo
wer
pan
el il
lus-
trat
es h
ow a
par
ticul
ar p
ropo
sitio
n w
ould
act
ually
be
repr
esen
ted,
CO
GN
ITIV
E P
ER
SPE
CT
IVE
S
and
over
an
addi
tiona
l, fo
urth
set
of
units
cal
led
" PR
OP"
uni
ts. T
he n
et-
wor
k co
ntai
ns b
idir
ectio
nal c
onne
ctio
ns f
rom
eac
h se
t of
cons
titue
nt u
nits
to th
e PR
OP
units
. The
re is
als
o an
othe
r se
t of
conn
ectio
ns f
or in
put t
o an
dou
tput
fro
m th
e ne
twor
k; th
ese
allo
w in
puts
sta
ndin
g fo
r sp
ecifi
c w
ords
toac
tivat
e di
strib
uted
sem
antic
pat
tern
s ov
er th
e in
p~t
uni
ts, T
he r
ecur
rent
conn
ectio
ns w
ithin
eac
h po
ol o
f un
its a
llow
loca
l pat
tern
com
plet
ion
with
inea
ch p
ool.
The
eff
ect o
f th
is is
to im
plem
ent a
"cl
ean-
" pr
oces
s in
whi
chth
e pa
ttern
of
activ
atio
n te
nds
to c
onve
rge
to th
e re
pres
enta
tion
of a
spe
cifi
cw
ord,
and
has
muc
h th
e sa
me
effe
ct a
s th
e m
utua
l inh
ibiti
on w
ithin
laye
rsin
the
prev
ious
mod
el.
Onc
e ag
ain,
the
know
ledg
e or
mem
ory
in th
is s
yste
m is
sto
red
in th
e co
nnec
-tio
ns a
mon
g th
e un
its, W
e ca
n th
ink
of th
e in
put/o
utpu
t wei
ghts
as
enco
ding
know
ledg
e ab
out t
he s
eman
tic p
atte
rn c
orre
spon
ding
to e
ach
wor
d of
the
prop
ositi
on, a
nd w
e ca
n th
ink
of th
e co
nnec
tion
wei
ghts
bet
wee
n th
e co
nstit
-ue
nt a
nd P
RO
P po
ols
as e
ncod
ing
know
ledg
e ab
out t
he p
ropo
sitio
ns th
at th
ese
cons
titue
nts
ente
r in
to w
ith o
ther
con
stitu
ents
, Hin
ton
(198
1) s
ugge
sted
that
this
net
wor
k w
ould
be
able
to g
ener
aliz
e w
hat i
t kno
ws
abou
t one
con
cept
toot
her
rela
ted
conc
epts
if s
imila
r co
ncep
ts a
re r
epre
sent
ed b
y si
mila
r pa
ttern
sof
act
ivat
ion,
Thu
s , if
Cly
de is
a p
artic
ular
ele
phan
t , an
d C
lyde
is r
epre
sent
edby
a p
atte
rn th
at is
sim
ilar
to th
e pa
ttern
for
ele
phan
t , th
en w
hat w
e kn
owab
out e
leph
ants
will
tend
to g
ener
aliz
e to
Cly
de. S
uch
effe
cts
do n
ot s
tric
tlyob
ey th
e la
ws
of lo
gic;
inst
ead
they
obe
y th
e la
ws
of a
ssoc
iatio
n,In
Hin
ton
s (1
981)
wor
k , th
e re
pres
enta
tions
of
the
conc
epts
wer
e as
-signed by hand. Connectionist learning algorithms have evolved consider-
ably
sin
ce th
at ti
me ,
how
ever
, and
we
now
hav
e al
gori
thm
s th
at c
an d
isco
ver
how
to r
epre
sent
dif
fere
nt c
once
pts
thro
ugh
repe
ated
exp
osur
e to
info
rma-
tion
abou
t the
ent
ire s
eman
tic d
omai
n in
whi
ch th
e co
ncep
t is
embe
dded
,I
will
con
side
r on
e su
ch domain-the broad domain of
livi
ng th
ings
. I s
how
in Figure 2,5 a representation of a fragment of the knowledge someone
mig
ht h
ave
abou
t liv
ing
thin
gs, T
his
form
at is
typi
cal o
f th
e ap
proa
ch to
know
ledg
e re
pres
enta
tion
used
in c
lass
ical
art
ific
ial i
ntel
ligen
ce a
ppro
ache
sto cognition, b
egin
ning
with
Qui
llian
(19
68),
The
kno
wle
dge
has
seve
ral
char
acte
rist
ics:
it is
str
uctu
red ,
in th
at it
is o
rgan
ized
into
a h
iera
rchy
. Ind
i-vi
dual
type
s or
spe
cies
are
list
ed a
t the
bot
tom
of
the
hier
arch
y, a
nd th
eir
orga
niza
tion
into
bro
ad c
lass
es, a
nd th
e or
gani
zatio
ns o
f the
se in
to la
rger
clas
ses ,
is in
dica
ted
by "
isa"
link
s. W
e ca
n im
agin
e at
tach
ing,
bel
ow th
ele
vel o
f th
e ty
pes ,
spe
cific
inst
ance
s of
the
type
s. F
or e
xam
ple
, we
coul
d ad
dT
wee
ty is
a ca
nary
, " a
nd s
o fo
rth;
or
if w
e ha
d an
Ele
phan
t nod
e , w
e co
uld
add
" Cly
de is
a E
leph
ant,
" T
he n
etw
ork
is p
oten
tially
qui
te e
cono
mic
al, i
nth
at fa
cts
that
are
true
of w
hole
sub
-tre
es o
f th
e hi
erar
chy
can
be a
ttach
edat
the
high
est l
evel
to w
hich
they
app
ly. G
iven
this
, whe
n so
me
info
rmat
ion
is n
ot s
tore
d on
a s
peci
fic c
once
pt, i
t may
be
infe
rred
by
sear
chin
g th
roug
hth
e tr
ee. T
he p
roce
ss is
equ
ival
ent t
o th
e st
anda
rd lo
gica
l syl
logi
stic
rea
son-
Con
stru
ctiv
e M
emor
y an
d M
emor
y D
isto
rtio
ns
livin
g th
ing
grow
Figu
re
5 A
sem
antic
net
wor
k of
the
type
for
mer
ly u
sed
in m
odel
s of
the
orga
niza
tion
ofkn
owle
dge
in m
emor
y, c
onta
inin
g th
e co
ncep
ts a
nd p
ropo
sitio
ns u
sed
in th
e le
arni
ng e
x-pe
rim
ent o
f R
umel
hart
(19
90),
Ada
pted
fro
m f
ig, 1
.8 o
f R
umel
hart
and
Tod
d (1
993)
by
McC
leIla
nd e
t al.
(199
4) a
s th
eir
fig, 3
, p, 1
3,
ing
proc
ess
thro
ugh
whi
ch w
e in
fer
that
Soc
rate
s is
mor
tal.
We
know
that
Socr
ates
is a
man
, and
we
know
that
men
are
mor
tal ,
ther
efor
e w
e ca
n in
fer
that
Soc
rate
s is
mor
tal t
oo.
Whe
n on
e tr
ains
a n
etw
ork
like
the
one
show
n in
the
prev
ious
figu
rew
ith e
xam
ple
prop
ositi
ons
from
this
dom
ain
, it l
earn
s tw
o th
ings
, It l
earn
sco
nnec
tion
wei
ghts
inte
rnal
ly th
at e
ncod
e th
e pr
opos
ition
s , a
nd th
at a
llow
com
plet
ion
of a
pro
posi
tion
from
two
of it
s te
rms,
It a
lso
lear
ns c
onne
ctio
nw
eigh
ts fr
om th
e w
ord
inpu
t uni
ts to
the
cons
titue
nt u
nits
that
ess
entia
llyas
sign
use
ful s
eman
tic r
epre
sent
atio
ns to
eac
h w
ord.
Hin
ton
(198
9) d
emon
-strated this for kinship relationships, R
umel
hart
(19
90; R
umel
hart
and
Tod
d, 1
993)
app
lied
the
sam
e id
ea to
the
dom
ain
of li
ving
thin
gs (
the
actu
alsi
mul
atio
n m
odel
Rum
elha
rt u
sed
was
slig
htly
sim
pler
than
the
one
show
nin
Fig
ure
2,5)
, and
I h
ave
chos
en to
use
this
cas
e as
my
exam
ple,
The
res
ults
on w
hich
the
follo
win
g di
scus
sion
dep
ends
com
e fr
om a
rep
etiti
on o
f th
eR
umel
hart
(19
90)
sim
ulat
ion
repo
rted
in M
cCle
lland
, McN
augh
ton
, and
Rei
lly (
1994
),
oak
pine
rose
dais
y
cana
ry
robi
n
salm
on
sunf
ish
COGNITIVE PERSPECTIVES
Con
stru
ctiv
e M
emor
y an
d M
emor
y D
isto
rtio
ns
Epoch 25
pine
robi
nro
seoa
kca
nary salm
onda
isy
sunf
ish
Epo
ch 2
00
pine
dais
yoa
kro
seca
nary
robi
nsa
lmon
sunf
ish
Epoch 25
Epoch 200
Epo
ch 5
00
Figu
re
6 R
epre
sent
atio
ns d
isco
vere
d in
a r
eplic
atio
n of
Rum
elha
rt's
(19
90)
expe
rimen
ttr
aini
ng a
sem
antic
net
wor
k m
uch
like
the
one
show
n in
Fig
ure
2,5,
The
fig
ure
show
s th
eac
tivat
ion
of e
ach
of th
e N
oun-
1 un
its f
or e
ach
of th
e ei
ght s
peci
fic
conc
epts
use
d, T
hehe
ight
of
each
ver
tical
bar
indi
cate
s th
e ac
tivat
ion
of th
e un
it on
a s
cale
fro
m 0
to 1
. One
can
see
that
initi
ally
all
the
conc
eprs
hav
e fa
irly
sim
ilar
repr
esen
tatio
ns, A
fter
200
epo
chs
oftr
aini
ng, t
here
is a
cle
ar d
iffe
rent
iatio
n of
the
repr
esen
tatio
ns o
f th
e pl
ants
and
ani
mal
s,A
fter
500
epo
chs ,
the
furt
her
diff
eren
tiatio
n of
the
plan
ts in
to tr
ees
and
flow
ers
and
of th
ean
imal
s in
to fi
sh a
nd b
irds
is a
ppar
ent,
Fro
m M
cCle
lland
et a
l. (1
994)
, fig
,, p
, 16,
oak
pine
rose
dais
yca
nary
robi
nsa
lmon
sunf
ish
Epo
ch 5
00
Thr
ough
gra
dual
trai
ning
on
exam
ples
fro
m th
e do
mai
n of
pla
nts
and
anim
als ,
the
netw
ork
lear
ned
mor
e th
an ju
st th
e pr
opos
ition
s. I
t als
o le
arne
dto
ass
ign
usef
ul r
epre
sent
atio
ns to
eac
h co
ncep
t, T
he representations the
netw
ork
lear
ned
to u
se f
or th
e fi
rst n
oun
are
illus
trat
ed in
Fig
ures
2, 6
and
7, T
hese
rep
rese
ntat
ions
are
det
erm
ined
by
the
conn
ectio
n w
eigh
ts f
rom
the concept input units to the concept representation un
its. T
hrou
gh th
eco
urse
of
lear
ning
, the
se w
eigh
ts g
radu
ally
cha
nge , s
o th
at th
e re
pres
enta
-tio
ns o
f th
e di
ffer
ent c
once
pts
grad
ually
com
e to
cap
ture
how
sim
ilar
the
conc
epts
are
in te
rms
of th
e pr
opos
ition
s th
ey e
nter
into
. Can
ary
and
Rob
in~t
er in
to h
ighl
y ov
erla
ppin
g se
ts o
f pr
opos
ition
s-fo
r ex
ampl
e , b
oth
are
bIrd
s , b
oth
can
fly,
bot
h ha
ve f
eath
ers,
As
a re
sult
of th
is, t
he n
etw
ork
com
esto
ass
ign
them
rep
rese
ntat
ions
that
are
ver
y si
mila
r; s
imila
r re
pres
enta
tions
lead
to s
imila
r ou
tput
s. M
ost i
mpo
rtan
t , on
ce it
has
lear
ned
to u
se s
uch
Figu
re
7 Si
mila
rity
str
uctu
re d
isco
vere
d In
our
rep
hcat
io'n
o!.
Kum
elha
rts
(199
0) le
arni
ngex
peri
men
t, u
sing
the
repr
esen
tatio
ns s
how
n in
Fig
ure
2,6,
Ini
tially
, the
pat
tern
s ar
e al
lquite similar; and the weak similarity structure that exists is random, The
conc
epts
be
com
epr
ogre
ssiv
ely
diff
eren
tiate
d as
lear
ning
pro
gres
ses,
Fro
m M
cCle
lland
et a
l. (1
994)
, fig
, 617
,
repr
esen
tatio
ns, t
he n
etw
ork
can
use
the
sim
ilari
ty o
f th
e re
pres
enta
tions
as
the
basi
s fo
r in
fere
nces
. Thu
s, o
nce
the
netw
ork
has
lear
ned
how
to r
epre
sent
Can
ary
and
Rob
in a
s si
mila
r to
eac
h ot
her
and
dist
inct
fro
m o
ther
con
cept
sit now adds
to
the
trai
ning
set
ene
prop
ositi
on a
bout
a n
ew ty
pe o
f ani
mal
-le
ts
say
the
prop
ositi
on th
at "
Spar
row
isa
bird
"-th
enet
Wor
k le
arns
toas
sign
"Sp
arro
w"
a re
pres
enta
tion
sim
ilar
to th
e re
pres
enta
tion
it us
es f
orot
her
bird
s. A
fter
this
pro
posi
tion
has
been
lear
ned,
we
can
then
ask
the
CO
GN
ITIV
E P
ER
SPE
CT
IVE
SC
onst
ruct
ive
Mem
ory
and
Mem
ory
Dis
tort
ions
netw
ork
if it
kno
ws
wha
t a s
parr
ow c
an d
o. T
his
can
be d
one
by p
atte
rnco
mpl
etio
n-w
e ca
n te
st th
e ne
twor
k to
see
if it
can
com
plet
e th
e pa
ttern
Spa
rrow
can
, , ,
" w
ith "
fly.
" In
deed
, Rum
elha
rt s
how
ed in
his
sim
ulat
ions
that if the network was trained on the full set
of
prop
ositi
ons
conc
erni
ngca
narie
s an
d ro
bins
, he
coul
d te
ach
it on
ly o
ne p
ropo
sitio
n ab
out s
par-
row
s-na
mel
y " S
parr
ow is
a b
ird"
and
it w
as a
ble
to c
orre
ctly
com
plet
eot
her
prop
ositi
ons
abou
t spa
rrow
s, T
he o
utpu
t was
qui
te c
lear
abo
ut th
ose
things that are generally true
of
the
othe
r bi
rds,
For
thos
e pr
oper
ties
that
diffe
red
betw
een
cana
ry a
nd r
obin
, it p
rodu
ced
ambi
guou
s ou
tput
s. T
hus
it ap
plie
d w
hat i
t alr
eady
kno
ws
abou
t can
arie
s an
d ro
bins
to s
parr
ows.
Now
, taking this
mod
el
at least as a sketch
of
a model
of
our
know
ledg
eof
fa
cts
and
even
ts, let us consider the nature
of
mem
ory
as r
econ
stru
ctio
nag
ain.
Ind
ivid
ual e
xper
ienc
es th
emse
lves
are
not
sep
arat
ely
repr
esen
ted;
in-
stea
d, they leave what I would call a structured system
of
know
ledg
e st
ored
in th
e co
nnec
tion
wei
ghts
. Fur
ther
mor
e , th
is k
now
ledg
e is
not
itse
lf d
irec
tlyac
cess
ible
to o
vert
res
pond
ing
or d
irec
t rep
ort,
Inst
ead
the
know
ledg
e pr
o-vides a mechanism that can construct responses to queries presented
to
the
netw
ork ,
whether the actual proposition was ever actually experienced, a
sin
the
case
of
the
actu
al tr
aini
ng e
xam
ples
, or
not ,
as
in th
e ca
se o
f qu
estio
nsw
e m
ay a
sk a
bout
, for
exa
mpl
e , what a sparrow can
do
afte
r th
e tr
aini
ngdescribed above, The outputs of such a network might then be the basis
perf
orm
ance
we
take
as
indi
cativ
e of
rem
embe
ring,
but
man
y tim
es th
eym
ight
ref
lect
gen
eral
izat
ion
base
d on
the
accu
mul
ated
eff
ects
of
prio
r ex
peri
-en
ce, r
athe
r th
an th
e ef
fect
s of
sto
ring
anyt
hing
like
the
spec
ific
item
in m
em-
ory,
Suc
h ge
nera
lizat
ion
is, I
wou
ld s
ugge
st, central to our ability
to
act
intelligently, and the process of learning the sorts
of
repr
esen
tatio
ns o
nwhich such generalizations are based is central
to
cogn
itive
dev
elop
men
t(s
ee M
cCle
lland
, 199
4, f
or d
iscu
ssio
n), B
ut s
uch
gene
raliz
atio
n gi
ves
rise
todi
stor
tions
as
an in
here
nt b
y-pr
oduc
t: it
beco
mes
impo
ssib
le to
dis
tingu
ish
betw
een
wha
t has
act
ually
bee
n ex
peri
ence
d an
d w
hat c
an b
e co
nstr
ucte
dba
sed
on o
ther
rel
ated
thin
gs th
at h
ave
been
exp
erie
nced
.B
ut th
ere
is s
omet
hing
slig
htly
wro
ng h
ere.
Our
abi
lity
to is
olat
e pa
rtic
u-la
r m
emor
ies
is n
ot a
s ba
d as
it w
ould
app
ear
to b
e if
we
assu
med
that
memory consists solely
of
the gradually acquired residue
of
a large body
expe
rien
ce. I
can
tell
you
a ne
w fa
ct-s
uch
as th
e fa
ct th
at "
Sam
my
is a
Sunf
ish"
-and
this
can
aff
ect y
our
sem
antic
mem
ory
righ
t aw
ay. W
e ne
edsome mechanism capable
of relatively rapid learning
of the contents
of
indi
-vi
dual
epi
sode
s an
d ex
perie
nces
, O
ne m
ight
thin
k th
at o
ne c
ould
sim
ply
add
new
mem
orie
s on
e at
a ti
me
into
a n
etw
ork
like
the
Hin
ton
(198
1) n
etw
ork ,
but in fact this is not
so.
lf one attempts to store additional memories all at once in such systems, i
tca
n be
don
e, but at the cost
of
a ph
enom
enon
cal
led
" Cat
astr
ophi
c In
terf
er-
ence" (McCloskey and Cohen, 1989). The addition
of
the
new
mat
eria
l
causes a dramatic loss
of
the
abili
ty to
per
form
cor
rect
ly w
ith o
ther
, sim
ilar
mat
eria
l , p
artic
ular
ly w
hen ,
as
is o
ften
the
case
, the
new
mat
eria
l is
not
com
plet
ely
cons
iste
nt w
ith w
hat i
s al
read
y kn
own.
Thu
s if
I tr
ain
the
net-
wor
k w
ith th
e pr
opos
ition
s " S
parr
ow is
a bi
rd"
and
" Spa
rrow
is b
row
nit will drastically interfere with my ability
to
reco
ver
the
colo
r of
oth
er b
irds
like
cana
ries
and
robi
ns (
McC
lella
nd e
t aI., 1994), The only way
to
add
new information robustly
to a sg;uctured memory system is
to
add
it th
roug
ha
proc
ess
calle
d in
terl
eave
d le
arni
ng, i
n w
hich
lear
ning
occ
urs
very
gra
dual
lyth
roug
h re
peat
ed e
xpos
ure
to th
e ne
w m
ater
ial ,
inte
rlea
ved
with
ong
oing
exposure to other examples,of the same domain
of
know
ledg
e. C
onne
ctio
nw
eigh
t adj
ustm
ents
occ
ur d
urin
g ex
posu
re to
the
new
mat
eria
l and
the
old
ther
eby
grad
ually
allo
win
g th
e ne
w m
ater
ial'to be incorporated into the
mem
ory
syst
em w
ithou
t at t
he s
ame
time
disr
uptin
g w
hat i
s al
read
y kn
own,
A Proposed Synthesis
of
the
Mod
els
A n
atur
al p
ropo
sal t
hat a
rise
s fr
om th
is o
bser
vatio
n, t
hen
, is
to s
ugge
st th
atthe human memory is essentially a synthesis
of
the two types
of
mod
els
Ihave described above, One part
of
the
syst
em g
radu
ally
lear
ns to
rep
rese
ntan
d us
e co
ncep
ts a
s in
the
Hin
ton
mod
el, w
hile
ano
ther
par
t is
give
n th
e ta
skof
rapidly learning the specific content
of
indi
vidu
al e
vent
s an
d ex
peri
ence
sst
orin
g th
em in
a w
ay th
at is
sim
ilar
to th
e m
etho
d us
ed in
the
Tra
ce S
ynth
e-sis model
of McClelland (1981). I have presented a visualization
of
this
idea
in Figure 2,
Thi
s pr
opos
al m
ay s
eem
at f
irst
som
ewha
t unp
arsi
mon
ious
, but in fact
it provides an account
of
the pattern
of
amne
sia
that
res
ults
fro
m b
ilate
ral
lesi
ons
to th
e m
edia
l tem
pora
l lob
es. I
ndiv
idua
ls w
ith e
xten
sive
dam
age
tothese brain regions show a very striking pattern
of
mem
ory
defic
its (
for
over
-vi
ews
see
Squi
re, Chapter 7
of
this
vol
ume)
, The
se p
atie
nts
appe
ar p
ro-
foun
dly
defi
cien
t in
the
abili
ty to
for
m n
ew s
eman
tic o
r ep
isod
ic m
emo
but t
he a
bilit
y to
acq
uire
new
impl
icit
know
ledg
e su
ch a
s ne
w c
ogni
tive
skill
s or
sen
sitiv
ity to
the
sequ
entia
l dep
ende
ncie
s am
ong
stim
uli i
n im
plic
itle
arni
ng ta
sks
rem
ains
inta
ct, a
nd e
xist
ing
sem
antic
kno
wle
dge
such
as
se-
man
tic a
ssoc
iatio
ns c
an b
e pr
imed
. Sem
antic
and
epi
sodi
c kn
owle
dge
ac-
quir
ed lo
ng b
efor
e th
e da
mag
e oc
curr
ed is
spa
red-
that
is, i
t is
as g
ood
insu
ch p
atie
nts
as, i
t is
in a
ge-m
atch
ed c
ontr
ols,
In
fact
, the
re is
a te
mpo
rally
grad
ed r
etro
grad
e am
nesi
a , w
hich
in h
uman
s ca
n ex
tend
ove
r se
vera
l yea
rssu
ch th
at s
eman
tic a
nd e
piso
dic
mem
orie
s th
at w
ere
acqu
ired
sho
rtly
bef
ore
the occurrence
of
the
dam
age
are
prof
ound
ly a
ffect
ed, a
nd m
emor
ies
that
wer
e ac
quire
d at
pro
gres
sive
ly e
arlie
r tim
es a
re p
rogr
essi
vely
less
and
less
affe
cted
. Cru
cial
ly, i
n se
vera
l stu
dies
bot
h in
hum
ans
and
in o
ther
ani
mal
sm
emor
y fo
r th
e m
ost r
ecen
t pre
mor
bid
time
peri
ods
can
actu
ally
be
~uc
hw
orse
than
mem
ory
for
mat
eria
l fro
m s
light
ly m
ore
rem
ote
time
perI
ods,
CO
GN
ITIV
E P
ER
SPE
CT
IVE
S
Neo
cort
ical
Lea
rnin
g Sy
stem
Figu
re
8 A
syn
thes
is o
f th
e M
cCle
lland
(19
81)
mod
el a
nd th
e H
into
n (1
981)
mod
el. O
nepa
rt, b
ased
on
the
McC
lella
nd (
1981
) m
odel
, allo
ws
the
stor
age
and
retr
ieva
l of
indi
vidu
altr
aces
, sub
ject
to tr
ace
synt
hesi
s , w
hile
the
othe
r pa
rt, m
ore
sim
ilar
to th
e m
odel
of
Hin
ton
(198
1), m
akes
use
of
grad
ual ,
inte
rlea
ved
lear
ning
to a
cqui
re a
str
uctu
red
syst
em o
f kn
owl-
edge
gra
dual
ly f
rom
exp
osur
es to
ens
embl
es o
f ev
ents
and
exp
erie
nces
, The
par
t of
the
mod
el b
ased
on
McC
lella
nd (
1981
) pl
ays
a ro
le a
kin
to th
at p
laye
d by
med
ial t
empo
ral
lobe
str
uctu
res
in th
e br
ain ,
whi
le th
e pa
rt b
ased
on
Hin
ton
(198
1) p
lays
a r
ole
sim
ilar
toth
at p
laye
d by
oth
er le
arni
ng s
yste
ms
in th
e hu
man
neo
cort
ex,
We
can
acco
unt f
or th
ese
find
ings
by
assu
min
g th
at o
lder
, con
solid
ated
mem
orie
s , a
s w
ell a
s co
gniti
ve s
kills
and
oth
er "
impl
icit
" fo
rms
of k
now
l-ed
ge, a
re s
ubse
rved
by
info
rmat
ion
proc
essi
ng s
yste
ms
loca
ted
in th
e la
rge
neoc
ortic
al in
form
atio
n pr
oces
sing
sys
tem
situ
ated
out
side
the
med
ial t
em-
pora
l reg
ion,
I id
entif
y th
ese
syst
ems-
here
afte
r la
bele
d co
llect
ivel
y th
e ne
o-co
rtic
al s
yste
m- w
ith th
e sy
stem
s th
at a
cqui
re k
now
ledg
e ve
ry g
radu
ally
,th
roug
h sm
all a
djus
tmen
ts to
con
nect
ion
wei
ghts
, rep
rese
nted
in F
igur
e 2,
sche
mat
ical
ly b
y th
e ne
twor
k of
the
type
intr
oduc
ed b
y H
into
n (1
981)
. The
conn
ectio
n ad
just
men
ts in
this
sys
tem
, as
we
have
see
n , le
ad to
the
grad
ual
emer
genc
e of
str
uctu
red
know
ledg
e sy
stem
s su
ch a
s th
ose
that
are
req
uire
dfo
r ad
equa
te g
ener
aliz
atio
n in
dom
ains
that
oth
ers
have
tend
ed to
trea
t as
impl
icit-
dom
ains
suc
h as
syntax-and domains th
at o
ther
s ha
ve te
nded
'----
-,_,
- --
..-..-
--..,
Con
stru
ctiv
e M
emor
y an
d M
emor
y D
isto
rtio
ns
to tr
eat a
s ex
plic
it-do
mai
ns s
uch
as s
eman
tic m
emor
y, A
t the
sam
e tim
ew
e as
sum
e th
at th
e ab
ility
to p
erfo
rm c
orre
ctly
in e
xplic
it m
emor
y ta
sks
base
don
rap
idly
for
med
mem
ory
trac
es o
f re
cent
eve
nts
and
expe
rien
ces
aris
es f
rom
lear
ning
that
take
s pl
ace
with
in th
e m
edia
l tem
pora
l lob
es, h
ere-
afte
r ca
lled
the
med
ial t
empo
ral l
obe
syst
em. R
ecen
tly, B
ruce
McN
augh
ton
Ran
dy O
'Rei
lly, a
nd I
hav
e pr
opos
ed a
n ac
coun
t of
the
amne
sic
synd
rom
eba
sed
on th
ese
idea
s (M
cCle
lland
et a
I., 1
994)
. On
this
vie
w, a
n ex
peri
ence
such
as
hear
ing
som
eone
say
"S
amm
y is
a s
unfis
h"
prod
uces
a p
atte
rn o
fac
tivat
ion
wid
ely
dist
ribut
ed th
roug
hout
the
neoc
ortic
al s
yste
m; c
onne
c-tio
ns fr
om' t
his
syst
em in
to th
e m
edia
l tem
pora
l reg
ion
prod
uce
a co
rre-
spon
ding
pat
tern
of
activ
atio
n ov
er th
e ne
uron
s th
ere.
The
med
ial t
empo
ral
lobe
sys
tem
then
pla
ys th
e ro
le th
at th
e pr
opos
ition
uni
ts p
lay
in th
e M
cCle
l-la
nd (
1981
) m
odel
, lin
king
all
of th
e co
nstit
uent
s of
the
even
t tog
ethe
r in
toa
sing
le tr
ace,
We
do n
ot th
ink
this
is d
one
by a
ssig
ning
an
indi
vidu
al n
euro
nto
eac
h ep
isod
ic m
emor
y, a
s or
igin
ally
pro
pose
d by
McC
lella
nd (
1981
).H
owev
er, a
n ex
plic
atio
n of
the
deta
ils o
f ou
r vi
ew o
f th
e na
ture
of
med
ial
tem
pora
l lob
e re
pres
enta
tion
is b
eyon
d th
e sc
ope
of th
is c
hapt
er. S
uffic
e it
to say that we think of the representation as sharing m
any
char
acte
rist
ics
with
the
.rep
rese
ntat
ions
use
d in
the
trac
e sy
nthe
sis
mod
el: t
he r
epre
sent
a-tio
ns, t
houg
h di
strib
uted
, are
rel
ativ
ely
spar
se (
few
uni
ts a
ctiv
e), a
nd e
ach
unit
that
par
ticip
ates
in th
e re
pres
enta
tion
is a
ctiv
ated
onl
y w
hen
a co
njun
c-tio
n of
ele
men
ts o
ccur
s in
the
inpu
t (fo
r fu
ller
disc
ussi
on, s
ee O
'Rei
lly a
ndM
cCle
lland
, in
pres
s).
Onc
e a
repr
esen
tatio
n ha
s be
en s
et u
p in
the
med
ial t
empo
ral l
obe
syst
emm
emor
y ca
n be
pro
bed
by p
rese
ntin
g an
inco
mpl
ete
frag
men
t , ju
st a
s in
the
trac
e sy
nthe
sis
mod
el, and reconstruction occurs, via return connections.
Eac
h tim
e a
trac
e is
syn
thes
ized
, a s
mal
l am
ount
of
conn
ectio
n ad
just
men
tta
kes
plac
e w
ithin
the
neoc
ortic
al s
yste
m a
s w
ell.
Con
solid
atio
n is
thou
ght
to b
e th
e re
sult
of th
is g
radu
al n
eoco
rtic
al le
arni
ng th
at o
ccur
s ev
ery
time
a m
emor
y tr
ace
is r
econ
stru
cted
. The
pro
cess
is g
radu
al, s
o th
at th
e ne
win
form
atio
n in
itial
ly s
tore
d vi
a th
e m
edia
l tem
pora
l sys
tem
, its
elf
can
begr
adua
lly in
tegr
ated
into
the
syst
em o
f re
pres
enta
tions
use
d in
the
neoc
orti-
cal s
yste
m w
ithou
t dis
rupt
ing
exis
ting
know
ledg
e st
ored
ther
ein,
Thi
s so
rtof
dua
l mem
ory
syst
em th
en a
llow
s ne
w in
form
atio
n to
be
rapi
dly
stor
edin
the
med
ial t
empo
ral s
yste
m w
ithou
t pro
duci
ng c
atas
trop
hic
inte
rfer
ence
with
wha
t is
know
n in
the
neoc
orte
x, I
nfor
mat
ion
that
is r
epea
tedl
y re
in-
stat
ed, i
nter
leav
ed w
ith o
ngoi
ng e
xpos
ure
to o
ther
info
rmat
ion,
gra
dual
lybe
com
es in
corp
orat
ed in
to th
e re
pres
enta
tions
in th
e ne
ocor
tex.
Con
clus
ion
I ha
ve c
oncu
rred
with
thos
e w
ho h
old
that
mem
ory
is a
con
stru
ctiv
e pr
oces
san
d I
have
pro
pose
d tw
o ra
ther
dif
fere
ntty
pes
of c
onne
ctio
nist
mec
hani
sms
that
can
con
trib
ute
to th
e sy
nthe
sis
of m
emor
y tr
aces
; and
I h
ave
sugg
este
d
COGNITIVE PERSPECTIVES
that
bot
h m
odel
s ca
ptur
e as
pect
s of
the
cons
truc
tive
proc
esse
s th
at o
ccur
inm
emor
y. I
ndee
d, I
wou
ld s
ugge
st th
at b
oth
sort
s of
con
stru
ctiv
e pr
oces
ses
may
con
trib
ute
to e
very
act
of
rem
embe
ring
. One
con
trib
utio
n to
rem
embe
r-in
g m
ay c
ome
from
the
med
ial t
empo
ral l
obe
trac
e of
the
epis
ode
or e
vent
itsel
f; if
this
wer
e th
e on
ly c
ontr
ibut
ion ,
we
coul
d sp
eak
sim
ply
of r
ecal
ling
that previous experience, p
erha
ps w
ith g
aps
aris
ing
from
for
getti
ng o
r en
-co
ding
fai
lure
. But
rem
embe
ring
invo
lves
an
activ
atio
n an
d sy
nthe
sis
pro-
cess
, in
whi
ch th
e re
pres
enta
tions
of
othe
r ev
ents
and
exp
erie
nces
sto
red
inth
e m
edia
l tem
pora
l lob
e ca
n co
ntri
bute
to th
e re
cons
truc
ted
trac
e , a
s th
eyso
met
imes
do
in th
e tr
ace
synt
hesi
s m
odel
. It a
lso
invo
lves
con
trib
utio
nsfr
om b
ackg
roun
d kn
owle
dge
base
d on
info
rmat
ion
acqu
ired
ver
y gr
adua
llyover the course of a lifetime of experience directly within the neocortical
syst
em, I
f th
ese
idea
s ha
ve a
ny v
alid
ity, w
e ca
nnot
see
rem
embe
ring
as
re-
call, but as a synthesis of contributions fr
om m
any
diff
eren
t sources of
info
rmat
ion,
The
not
ion
that
muc
h of
mem
ory
may
be
base
d on
kno
wle
dge
built
up
grad
ually
thro
ugh
expo
sure
to la
rge
ense
mbl
es o
f ex
peri
ence
s ha
s m
any
im-
plic
atio
ns f
or th
e lik
elih
ood
of v
erid
ical
mem
ory
and
for
our
abili
ty to
un-
cove
r th
e so
urce
of
the
outp
uts
gene
rate
d by
our
mem
ory
syst
ems,
an
issu
ece
ntra
l to
man
y of
the
othe
r ch
apte
rs in
this
vol
ume.
If
mem
ory
wer
e a
prop
ositi
onal
net
wor
k of
the
Qui
llian
type
, the
n se
para
ting
real
fro
m il
lu-
sory
mem
orie
s w
ould
just
be
a m
atte
r of
sep
arat
ing
prop
ositi
ons
deri
ved
from
exp
erie
nce
from
thos
e th
at w
ere
infe
rred
; in
prin
cipl
e w
e m
ight
imag
-in
e w
e co
uld
trai
n ou
rsel
ves ,
like
She
rloc
k H
olm
es, t
o av
oid
mak
ing
inap
-pr
opria
te in
fere
nces
and
ther
efor
e be
fudd
ling
our
mem
orie
s. If
mem
ory
isal
way
s an
act
ive
synt
hesi
s of
trac
es a
s in
the
McC
lella
nd (
1981
) m
odel
, the
reis
stil
l hop
e to
som
ehow
min
imiz
e th
e in
trus
ion
of in
appr
opri
ate
trac
es b
ypr
obin
g ou
r m
emor
y ju
st r
ight
, so
that
act
ivat
ion
of in
appr
opri
ate
trac
es is
min
imiz
ed. B
ut if
mem
orie
s ar
e al
way
s co
nstr
ucte
d w
ith a
t lea
st a
par
tial
relia
nce
on a
sys
tem
of
conn
ectio
n w
eigh
ts a
cqui
red
grad
ually
thro
ugh
ex-
tens
ive
expe
rien
ce w
ith a
dom
ain
of k
now
ledg
e , o
ur h
ope
of d
isen
tang
ling
the
indi
vidu
al tr
aces
dis
appe
ars
com
plet
ely,
and
the
likel
ihoo
d th
at w
e ca
nis
olat
e th
e co
ntri
butio
n of
a s
peci
fic
expe
rien
ce b
ecom
es in
crea
sing
ly r
emot
e.
Ref
eren
ces
Bar
tlett, F, C. (1932),
Rem
embe
ring
. C
ambr
idge
, Mass,
: Cam
brid
ge U
nive
rsity
Pres
s,
Gem
an, S
, and
Gem
an, D
, (19
84).
Sto
chas
tic r
elax
atio
n , G
ibbs
dis
trib
utio
ns, a
ndthe Bayesian restoration of images,
IEE
E: T
rans
actio
ns o
f Pa
ttern
Ana
lysi
s an
dM
achi
ne In
telli
genc
ePA
MI-
, 721
- 741
.H
into
n , G
. E. (
1981
), I
mpl
emen
ting
sem
antic
net
wor
ks in
par
alle
l har
dwar
e, I
n
Con
stru
ctiv
e M
emor
y an
d M
emor
y D
isto
rtio
ns
G, E
, Hin
ton
and
J, A
, And
erso
n (E
ds,),
Para
llel m
odel
s of
ass
ocia
tive
mem
ory,
Hill
sdal
e , N
, J.:
Erlb
aum
, Cha
p. 6
, 161
- 187
,H
into
n , G
, E. (
1989
), L
earn
ing
dist
ribu
ted
repr
esen
tatio
ns o
f co
ncep
ts, I
n R
, G, M
,M
orri
s (E
d,
),
Para
llel d
istr
ibut
ed p
roce
ssin
g: I
mpl
icat
ions
for
psy
chol
ogy
and
neur
obio
logy
, O
xfor
d: C
lare
ndon
Pre
ss, C
hap,
3, 4
6-61
.H
into
n , G
, E, a
nd S
ejno
wsk
i , T
, J. (
1986
), L
earn
ing
and
rele
arni
ng in
bol
tzm
ann
mac
hine
s, I
n D
, E. R
umel
hart
and
J. L
. McC
lella
nd (
Eds
.),
Para
llel d
istr
ibut
edpr
oces
sing
: Exp
lora
tions
in th
e m
icro
stru
ctur
e of
cog
nitio
nV
ol. 1
. Cam
brid
geM
ass,
: MIT
Pre
ss, C
hap,
728
2-31
7.H
intz
man
, D, L. (1988), Ju
dgem
ents
of
freq
uenc
y an
d re
cogn
ition
mem
ory
in a
multiple-trace model.
Psy
chol
ogic
al R
evie
w, 5
28-5
51.
Hum
phre
ys, M
, S"
Bai
n , J
. D"
and
Pike
, R, (
1989
), D
iffe
rent
way
s to
cue
a c
oher
ent
mem
ory
syst
em: A
theo
ry f
or e
piso
dic ,
sem
antic
, and procedural tasks,
Psyc
ho-
logical Review,
, 208
- 233
,M
cCle
lland
, J. L
. (19
81),
Ret
riev
ing
gene
ral a
nd s
peci
fic
info
rmat
ion
from
sto
red
knowledge of specifics, In
Proc
eedi
ngs
of th
e T
hird
Ann
ual C
onfe
renc
e of
the
Cognitive Science Society,
Ber
kele
y: 1
70- 1
72,
McC
lella
nd, J
. L. (
1991
), S
toch
astic
inte
ract
ive
activ
atio
n an
d th
e ef
fect
of
cont
ext
on perception,
Cognitive Psychology,
, 1-4
4,M
cCle
lland
, J. L
. (19
94).
The
inte
ract
ion
of n
atur
e an
d nu
rtur
e in
dev
elop
men
t:A parallel distributed processing perspective, In p, Bertelson, P
. Eel
en, a
nd G
,Y
dew
alle
(E
ds,),
Cur
rent
adv
ance
s in
psy
chol
ogic
al s
cien
ce: O
ngoi
ng r
e-se
arch
, H
illsd
ale ,
N, J
.: E
rlbau
m,
McC
lella
nd, J
. L.,
McN
augh
ton
, B, L
., a
nd O
' Rei
lly, R
, C. (
1994
), W
hy th
ere
are
com
plem
enta
ry le
arni
ng s
yste
ms
in h
ippo
cam
pus
and
neoc
orte
x: I
nsig
hts
from
the
succ
esse
s an
d fa
ilure
s of
con
nect
ioni
st m
odel
s of
lear
ning
and
mem
ory,
Pitt
s-bu
rgh:
Dep
artm
ent o
f Psy
chol
ogy,
Car
negi
e M
ello
n U
nive
rsity
, Tec
hnic
al R
e-po
rt P
DP,
CN
S,94
.M
cCle
lland
, J. L
. and
Rum
elha
rt, D
, E, (
1985
), D
istr
ibut
ed m
emor
y an
d th
e re
pre-
sentation of general and specific information,
Jour
nal o
f Exp
erim
enta
l Psy
chol
-og
y, G
ener
al11
415
9-18
8,M
cClo
skey
, M, a
nd C
ohen
, N, J
. (19
89),
Cat
astr
ophi
c in
terf
eren
ce in
con
nect
ioni
stne
twor
ks: T
he s
eque
ntia
l lea
rnin
g pr
oble
m, I
n G
, H, B
ower
(E
d,
),
The
psy
chol
-og
y of learning and motivation,
New
Yor
k: A
cade
mic
Pre
ss,
Met
calf
e , J
. (19
90),
Com
posi
te h
olog
raph
ic a
ssoc
iativ
e re
call
mod
el (
CH
AR
M)
and
blended memories in eyewitness testimony,
Psy
chol
ogic
al R
evie
w11
9, 1
45-
160,
Nei
sser
, U, (1967),
Cognitive psychology,
New
Yor
k: A
pple
ton-
Cen
tury
- Cro
fts,
Nys
trom
, L. E
, and
McC
lella
nd, J. L. (1992), Trace synthesis in cued recall.
Jour
nal
of Memory and Language,
, 591
- 614
,R
eilly
, R, C
. and
McC
lella
nd, J
. L. (
in p
ress
), H
ippo
cam
pal c
onju
nctiv
e en
codi
ng,
stor
age,
and recall: Avoiding a tradeoff,
Hip
poca
mpu
s,Q
uilli
an, M
, R, (
1968
), S
eman
tic m
emor
y, I
n M
. Min
sky
(Ed,
),
Semantic informa-
tion processing,
Cam
brid
ge, M
ass,
: MIT
Pre
ss,
Rum
elha
rt, D
, E, (
1990
), B
rain
sty
le c
ompu
tatio
n: L
earn
ing
and
gene
raliz
atio
n, I
nS,
F, Z
orne
tzer
, J. L
. Dav
is, a
nd C
. Lau
(E
ds,),
An
intr
oduc
tion
to n
eura
l and
electronic networks,
San
Die
go: A
cade
mic
Pre
ss, C
hap,
21.
-"--
'-~-
-"--
"
CO
GN
ITIV
E P
ER
SPE
CT
IVE
S
Rum
elha
rtD
, E. a
nd T
odd ,
P, M
, (19
93).
Lea
rnin
g an
d co
nnec
tioni
st r
epre
sent
a-tio
ns, I
n D
. E, M
eyer
and
S. K
ornb
lum
(E
ds,),
Atte
ntio
n an
d pe
rfor
man
ce X
IV:
Syne
rgie
s in
ex
peri
men
tal p
sych
olog
y, a
rtif
icia
l int
ellig
ence
, and
cog
nitiv
e ne
u-ro
scie
nce.
C
ambr
idge
, Mas
s,: M
IT P
ress
, 3- 3
0,Sm
olen
sky,
P. (
1986
), I
nfor
mat
ion
proc
essi
ng in
dyn
amic
al s
yste
ms:
Fou
ndat
ions
of
hann
ony
theo
ry, I
n D
. E. R
umel
hart
and
J. L
. McC
lella
nd (
Eds
.),
Para
llel
distributed processing: Explorations
in
the
miC
rost
ruct
ure
of c
ogni
tion
Vol
. 1.
Cam
brid
ge, M
ass.
: MIT
Pre
ss, C
hap,
619
4-28
1.
Squi
re, L
. R. (
1992
). M
emor
y an
d th
e hi
ppoc
ampu
s: A
syn
thes
is f
rom
fin
ding
s w
ithra
ts, monkeys and humans.
Psy
chol
ogic
al R
evie
w, 1
95- 2
31.
Squi
re, L. R. (1995). Biological foundations
of
accu
raC
y an
d in
accu
racy
in m
emor
y,Chapter 7, t
his
volu
me.
Ack
now
ledg
men
tsT
he w
ork
repo
rted
her
ein
was
sup
port
ed b
y G
rant
s M
HO
0385
and
MH
4756
6, I
wou
ld li
ke to
than
k C
raig
Sta
rk f
or c
ontr
ibut
ing
to th
e th
inki
ng th
at le
d to
this
pape
r,
, --.
.....
__
h___
- -
