Embed Size (px)
Citation preview
Intro
duct
ion
Supe
rvis
ed C
oref
eren
ce O
verv
iew
Mod
els
Feat
ures
Adva
nced
Tec
hniq
ues
and
Tren
dsC
lust
erin
gIn
corp
orat
ing
Wor
ld K
now
ledg
eIn
corp
orat
ing
Synt
actic
Fea
ture
sN
on-re
fere
ntia
l Pro
noun
sR
ule-
base
d sy
stem
•Bu
t th
e lit
tle
prin
ceco
uld
not
rest
rain
adm
irat
ion:
•"O
h! H
ow b
eaut
iful
you
are!
"
•"A
m I
not?
" th
e fl
ower
resp
onde
d, s
weet
ly. "
And
Iwa
s bo
rn
at t
he s
ame
mom
ent
as t
he s
un .
. ."
•Th
e lit
tle
prin
ceco
uld
gues
s ea
sily
eno
ugh
that
she
was
not
any
too
mod
est-
-but
how
mov
ing-
-and
exc
itin
g--s
hewa
s!
•"I
thin
k it
is t
ime
for
brea
kfas
t,"
she
adde
d an
inst
ant
late
r.
"If
you
woul
d ha
ve t
he k
indn
ess
to t
hink
of
my
need
s--"
•A
nd t
he li
ttle
pri
nce,
com
plet
ely
abas
hed,
wen
t to
look
for
a
spri
nklin
g-ca
n of
fre
sh w
ater
. So,
he
tend
ed t
he f
lowe
r.
Cor
efer
ence
Res
olut
ion:
Fr
om M
entio
ns to
Ent
ities
Mod
elin
g C
oref
eren
ce R
esol
utio
n•
Det
aile
d Su
rvey
and
Com
paris
on in
(Ng,
201
0)M
etho
dR
efer
ence
sAd
vant
ages
Dis
adva
ntag
esM
entio
n-P
air
mod
el
Cla
ssify
whe
ther
two
men
tions
are
cor
efer
entia
l or
not +
clu
ster
ing
(Soo
n et
al.
2001
; Ng
and
Car
die
2002
; Ji e
t al.,
20
05; M
cCal
lum
& W
elln
er,
2004
; Nic
olae
& N
icol
ae,
2006
)
easy
to e
ncod
e fe
atur
esgr
eedy
clu
ster
ing
algo
rithm
; Eac
h ca
ndid
ate
ante
cede
nts
is c
onsi
dere
d in
depe
nden
tly o
f the
oth
er
Entit
y-M
entio
nM
odel
Cla
ssify
whe
ther
a m
entio
n an
d a
prec
edin
g, p
ossi
bly
parti
ally
form
ed c
lust
er a
re
core
fere
ntia
l or n
ot
Pasu
la e
t al.
2003
; Lu
o et
al
. 200
4; Y
ang
et a
l. 20
04,
2008
; Dau
me
& M
arcu
, 20
05; C
ulot
ta e
t al.,
200
7
Impr
oved
expr
essi
vene
ss,
allo
ws
clus
ter l
evel
fe
atur
es
Each
can
dida
te c
lust
er is
co
nsid
ered
inde
pend
ently
of
the
othe
rs
Men
tion
- Ran
king
Mod
el
Impo
ses
a ra
nkin
gon
a s
et
of c
andi
date
ant
eced
ents
Den
is &
Bal
drid
ge 2
007,
20
08C
onsi
ders
all
the
cand
idat
ean
tece
dent
ssi
mul
tane
ousl
y
Insu
ffici
ent i
nfor
mat
ion
to
mak
e an
info
rmed
co
refe
renc
e de
cisi
on; s
till
need
to d
o cl
uste
ring
Clu
ster
Ran
king
mod
el
Ran
ks a
ll th
e pr
eced
ing
clus
ters
for e
ach
men
tion;
cr
eate
inst
ance
s w
ith e
ntity
-m
entio
n m
odel
, ran
k in
stan
ces
with
men
tion-
rank
ing
mod
el
Rah
man
and
Ng,
200
9C
ombi
nes
the
stre
ngth
of p
revi
ous
mod
els;
Ach
ieve
d th
e be
st
perfo
rman
ce
-
Sing
le-li
nk c
lust
erin
g (S
oon
et a
l., 2
001)
For e
ach
NP j
, sel
ect a
s its
ant
eced
ent t
he c
lose
stpr
eced
ing
NP
that
is d
eter
min
ed a
s co
refe
rent
with
itPo
sit N
P jas
non
-ana
phor
ic if
no
prec
edin
g N
P is
cor
efer
ent
with
it
Best
-firs
t clu
ster
ing
(Ng
& C
ardi
e, 2
002)
Sam
e as
sin
gle-
link
clus
terin
g, e
xcep
t tha
t we
sele
ct a
s th
e an
tece
dent
the
NP
that
has
the
high
est c
oref
eren
celik
elih
ood
Clu
ster
s ar
e fo
rmed
bas
ed o
n a
smal
l sub
set o
f the
pai
rwis
eco
refe
renc
ede
cisi
ons
Man
y pa
irwis
ede
cisi
ons
are
not u
sed
in th
e cl
uste
ring
proc
ess
Mr.
Clin
ton
Clin
ton
she
Clu
ster
s ar
e fo
rmed
bas
ed o
n a
smal
l sub
set o
f the
pai
rwis
eco
refe
renc
ede
cisi
ons
Man
y pa
irwis
ede
cisi
ons
are
not u
sed
in th
e cl
uste
ring
proc
ess
Mr.
Clin
ton
Clin
ton
she
Clu
ster
s ar
e fo
rmed
bas
ed o
n a
smal
l sub
set o
f the
pai
rwis
eco
refe
renc
ede
cisi
ons
Man
y pa
irwis
ede
cisi
ons
are
not u
sed
in th
e cl
uste
ring
proc
ess
Mr.
Clin
ton
Clin
ton
she
Use
all
the
pairw
ise
core
fere
nce
deci
sion
s
Gra
ph p
artit
ioni
ng a
lgor
ithm
sea
ch te
xt is
repr
esen
ted
as a
gra
phea
ch v
erte
x co
rresp
onds
to a
NP;
edg
e w
eigh
t is
core
flik
elih
ood
Goa
l: pa
rtitio
n th
e gr
aph
node
s to
form
cor
efer
ence
clus
ters
Use
all
the
pairw
ise
core
fere
nce
deci
sion
s
Gra
ph p
artit
ioni
ng a
lgor
ithm
sea
ch te
xt is
repr
esen
ted
as a
gra
phea
ch v
erte
x co
rresp
onds
to a
NP;
wei
ght o
f an
edge
indi
cate
s th
elik
elih
ood
that
the
two
NPs
are
cor
efer
ent
Goa
l: pa
rtitio
n th
e gr
aph
node
s to
form
cor
efer
ence
clus
ters
Cor
rela
tion
clus
terin
g(e
.g.,
McC
allu
m &
Wel
lner
(200
4))
clus
ter t
hat r
espe
cts
as m
any
pairw
ise
deci
sion
s as
pos
sibl
e
Min
imum
-cut
-bas
ed c
lust
erin
g(N
icol
ae&
Nic
olae
, 200
6)Fi
nd th
e m
incu
tof t
he g
raph
and
par
titio
n th
e gr
aph
node
s;
re
peat
unt
il so
me
stop
ping
crit
erio
n is
reac
hed
Bell
Tree
: Fro
m M
entio
ns to
Ent
ities
The
Amer
ican
Med
ical
As
soci
atio
nvo
ted
yest
erda
y to
inst
all t
he
heir
appa
rent
as
itspr
esid
ent-e
lect
, rej
ectin
g a
stro
ng, u
psta
rt ch
alle
nge
…
[AM
A]
heir
its
[AM
A,h
eir]
its
[AM
A]
[hei
r]its
[AM
A,h
eir,i
ts]
[AM
A,h
eir]
[its]
[AM
A, i
ts]
[hei
r]
[AM
A]
[hei
r,its
]
[AM
A]
[hei
r][it
s]
Men
tions
:AM
Ahe
irits
….
Bell
Tree
=Sea
rch
Spac
e#L
eave
s=Be
ll N
umbe
r B(m
) B(
20)=
5172
4158
2353
72
Rec
ast a
s a
sear
ch p
robl
emEx
pand
s th
e m
ost p
rom
isin
g pa
ths
Scor
es a
pat
h ba
sed
on p
airw
ise
prob
abilit
ies
Few
em
piric
al c
ompa
rison
s
Luo
et a
l. (2
004)
did
n’t c
ompa
re th
eir B
ell-t
ree
appr
oach
ag
ains
t the
real
ly g
reed
y al
gorit
hms
Few
em
piric
al c
ompa
rison
s
Luo
et a
l. (2
004)
did
n’t c
ompa
re th
eir B
ell-t
ree
appr
oach
ag
ains
t the
real
ly g
reed
y al
gorit
hms
Klei
n (2
005,
pc)
: sea
rch
spac
e is
too
larg
e, n
eed
to a
pply
a lo
tof
heu
ristic
s to
pru
ne th
e sp
ace,
mak
ing
it a
gree
dy a
lgor
ithm
Few
em
piric
al c
ompa
rison
s
Luo
et a
l. (2
004)
did
n’t c
ompa
re th
eir B
ell-t
ree
appr
oach
ag
ains
t the
real
ly g
reed
y al
gorit
hms
Klei
n (2
005,
pc)
: sea
rch
spac
e is
too
larg
e, n
eed
to a
pply
a lo
tof
heu
ristic
s to
pru
ne th
e sp
ace,
mak
ing
it a
gree
dy a
lgor
ithm
Nic
olae
& N
icol
ae(2
006)
: not
muc
h di
ffere
nce
in p
erfo
rman
ce
betw
een
Bell
tree
clus
terin
g an
d th
e re
ally
gre
edy
algo
rithm
s
Step
1: L
earn
a c
oref
eren
cem
odel
Step
2: A
pply
a c
lust
erin
g al
gorit
hm
Step
1: L
earn
a c
oref
eren
cem
odel
Men
tion-
pair
mod
el
Step
2: A
pply
a c
lust
erin
g al
gorit
hmR
eally
gre
edy
algo
rithm
sLe
ss g
reed
y al
gorit
hms
Tim
e-aw
are
algo
rithm
s
Lim
ited
expr
essi
vene
ssin
form
atio
n ex
tract
ed fr
om tw
o N
Ps m
ay n
ot b
e su
ffici
ent f
or
mak
ing
an in
form
ed c
oref
eren
cede
cisi
on
Can
’t de
term
ine
whi
ch c
andi
date
ant
eced
ent i
s th
e be
ston
ly d
eter
min
e ho
w g
ood
a ca
ndid
ate
is re
lativ
e to
NP
to b
e re
solv
ed, n
ot h
ow g
ood
it is
rela
tive
to th
e ot
hers
Lim
ited
expr
essi
vene
ssin
form
atio
n ex
tract
ed fr
om tw
o N
Ps m
ay n
ot b
e su
ffici
ent f
or
mak
ing
an in
form
ed c
oref
eren
cede
cisi
on
Can
’t de
term
ine
whi
ch c
andi
date
ant
eced
ent i
s th
e be
ston
ly d
eter
min
e ho
w g
ood
a ca
ndid
ate
is re
lativ
e to
NP
to b
e re
solv
ed, n
ot h
ow g
ood
it is
rela
tive
to th
e ot
hers
Men
tion-
rank
ing
mod
el
Entit
y-m
entio
n m
odel
Wan
t a c
oref
eren
cem
odel
that
can
tell
usho
w li
kely
“she
”an
d a
prec
edin
g cl
uste
r of “
she”
are
core
fere
nt
Mr.
Clin
ton
Clin
ton
she
?
?
a cl
assi
fier t
hat d
eter
min
es w
heth
er (o
r how
like
ly) a
n N
P be
long
s to
a p
rece
ding
cor
efer
ence
clus
ter
mor
e ex
pres
sive
than
the
men
tion-
pair
mod
elca
n em
ploy
clu
ster
-leve
lfea
ture
s de
fined
ove
r any
sub
set o
f N
Ps in
a p
rece
ding
clu
ster
addr
esse
s th
e ex
pres
sive
ness
pro
blem
Pasu
laet
al.
(200
3), L
uoet
al.
(200
4), Y
ang
et a
l. (2
004,
200
8),
Dau
me
& M
arcu
(200
5), C
ulot
taet
al.
(200
7), …
Lim
ited
expr
essi
vene
ssin
form
atio
n ex
tract
ed fr
om tw
o N
Ps m
ay n
ot b
e su
ffici
ent f
or
mak
ing
an in
form
ed c
oref
eren
cede
cisi
on
Can
’t de
term
ine
whi
ch c
andi
date
ant
eced
ent i
s th
e be
ston
ly d
eter
min
e ho
w g
ood
a ca
ndid
ate
is re
lativ
e to
NP
to b
e re
solv
ed, n
ot h
ow g
ood
it is
rela
tive
to th
e ot
hers
Idea
: tra
in a
mod
el th
at im
pose
s a
rank
ing
on th
e ca
ndid
ate
ante
cede
nts
for a
n N
P to
be
reso
lved
so th
at it
ass
igns
the
high
est r
ank
to th
e co
rrect
ant
eced
ent
Idea
: tra
in a
mod
el th
at im
pose
s a
rank
ing
on th
e ca
ndid
ate
ante
cede
nts
for a
n N
P to
be
reso
lved
so th
at it
ass
igns
the
high
est r
ank
to th
e co
rrect
ant
eced
ent
A ra
nker
allo
ws
all c
andi
date
ant
eced
ents
to b
e co
nsid
ered
si
mul
tane
ousl
y an
d ca
ptur
es c
ompe
titio
n am
ong
them
allo
ws
us fi
nd th
e be
st c
andi
date
ant
eced
ent f
or a
n N
P
Ther
e is
a n
atur
al re
solu
tion
stra
tegy
for a
rank
ing
mod
elAn
NP
is re
solv
ed to
the
high
est-r
anke
d ca
ndid
ate
ante
cede
nt
Con
vert
the
prob
lem
of r
anki
ng m
NPs
into
the
a se
t of
pairw
ise
rank
ing
prob
lem
sEa
ch p
airw
ise
rank
ing
prob
lem
invo
lves
det
erm
inin
g w
hich
of
two
cand
idat
e an
tece
dent
s is
bet
ter f
or a
n N
P to
be
reso
lved
Each
one
is e
ssen
tially
a c
lass
ifica
tion
prob
lem
Con
vert
the
prob
lem
of r
anki
ng m
NPs
into
the
a se
t of
pairw
ise
rank
ing
prob
lem
sEa
ch p
airw
ise
rank
ing
prob
lem
invo
lves
det
erm
inin
g w
hich
of
two
cand
idat
e an
tece
dent
s is
bet
ter f
or a
n N
P to
be
reso
lved
Each
one
is e
ssen
tially
a c
lass
ifica
tion
prob
lem
Firs
t sup
ervi
sed
core
fere
nce
mod
el: C
onno
lly e
t al.
(199
4)Tr
ain
a de
cisi
on tr
ee to
det
erm
ine
whi
ch o
f the
two
cand
idat
e an
tece
dent
s of
an
NP
is m
ore
likel
y to
be
its a
ntec
eden
t
Dur
ing
test
ing,
nee
d to
heu
ristic
ally
com
bine
the
pairw
ise
rank
ing
resu
lts to
sel
ect a
n an
tece
dent
for e
ach
NP
The
rank
ing
mod
el is
theo
retic
ally
bet
ter b
ut fa
r les
s po
pula
r th
an th
e m
entio
n-pa
ir m
odel
in th
e de
cade
follo
win
g its
pr
opos
al
Red
isco
vere
d al
mos
t ten
yea
rs la
ter i
ndep
ende
ntly
by
Yang
et a
l. (2
003)
: tw
in-c
andi
date
mod
elIid
a et
al.
(200
3): t
ourn
amen
t mod
el
Den
is &
Bal
drid
ge(2
007,
200
8): t
rain
the
rank
er u
sing
m
axim
um e
ntro
py
mod
el o
utpu
ts a
rank
val
ue fo
r eac
h ca
ndid
ate
ante
cede
ntob
viat
es n
eed
to h
euris
tical
ly c
ombi
ne p
airw
ise
rank
ing
resu
lts
Den
is &
Bal
drid
ge(2
007,
200
8): t
rain
the
rank
er u
sing
m
axim
um e
ntro
py
mod
el o
utpu
ts a
rank
val
ue fo
r eac
h ca
ndid
ate
ante
cede
ntob
viat
es n
eed
to h
euris
tical
ly c
ombi
ne p
airw
ise
rank
ing
resu
lts
Sinc
e a
rank
er o
nly
impo
ses
a ra
nkin
g on
the
cand
idat
es, i
t ca
nnot
det
erm
ine
whe
ther
an
NP
is a
naph
oric
Nee
d to
trai
n a
clas
sifie
r to
dete
rmin
e if
an N
P is
ana
phor
ic
Can
not d
eter
min
e be
st c
andi
date
Lim
ited
expr
essi
vene
ss
Men
tion
Ran
king
Entit
y M
entio
nPr
oble
m
Can
we
com
bine
the
stre
ngth
s of
thes
e tw
o m
odel
?
Con
side
r pre
cedi
ng c
lust
ers,
no
t can
dida
te a
ntec
eden
tsR
ank
cand
idat
e an
tece
dent
s
Men
tion-
rank
ing
mod
elEn
tity-
men
tion
mod
el
Con
side
r pre
cedi
ng c
lust
ers,
no
t can
dida
te a
ntec
eden
tsR
ank
cand
idat
e an
tece
dent
s
Men
tion-
rank
ing
mod
elEn
tity-
men
tion
mod
el
Ran
k pr
eced
ing
clus
ters
Con
side
r pre
cedi
ng c
lust
ers,
no
t can
dida
te a
ntec
eden
tsR
ank
cand
idat
e an
tece
dent
s
Men
tion-
rank
ing
mod
elEn
tity-
men
tion
mod
el
Ran
k pr
eced
ing
clus
ters
Trai
ning
train
a ra
nker
to ra
nk p
rece
ding
clu
ster
s
Test
ing
reso
lve
each
NP
to th
e hi
ghes
t-ran
ked
prec
edin
g cl
uste
r
Trai
ning
train
a ra
nker
to ra
nk p
rece
ding
clu
ster
s
Test
ing
reso
lve
each
NP
to th
e hi
ghes
t-ran
ked
prec
edin
g cl
uste
r
Lapp
in&
Leas
s’s
(199
4) h
euris
tic p
rono
un re
solv
er
As a
rank
er, t
he c
lust
er-ra
nkin
g m
odel
can
not d
eter
min
e w
heth
er a
n N
P is
ana
phor
icBe
fore
reso
lvin
g an
NP,
we
still
need
to u
se a
n an
apho
ricity
clas
sifie
r to
dete
rmin
e if
it is
ana
phor
icyi
elds
a p
ipel
ine
arch
itect
ure
Pote
ntia
l pro
blem
erro
rs m
ade
by th
e an
apho
ricity
clas
sifie
r will
be p
ropa
gate
d to
th
e co
refe
renc
ere
solv
er
Solu
tion
join
t lea
rnin
gfo
r ana
phor
icity
and
core
fere
nce
reso
lutio
n
B3
CEA
F R
P F
R P
F M
entio
n-Pa
ir Ba
selin
e 50
.8
57.9
54
.1
56.1
51
.0
53.4
Entit
y-M
entio
n Ba
selin
e 51
.2
57.8
54
.3
56.3
50
.2
53.1
Men
tion-
Ran
king
Bas
elin
e (P
ipel
ine)
52.3
61
.8
56.6
51
.6
56.7
54
.1
Men
tion-
Ran
king
Bas
elin
e (J
oint
) 50
.4
65.5
56
.9
53.0
58
.5
55.6
Clu
ster
-Ran
king
Mod
el (P
ipel
ine)
55
.3
63.7
59
.2
54.1
59
.3
56.6
Clu
ster
-Ran
king
Mod
el (J
oint
) 54
.4
70.5
61
.4
56.7
62
.6
59.5
B3
CEA
F R
P F
R P
F M
entio
n-Pa
ir Ba
selin
e 50
.8
57.9
54
.1
56.1
51
.0
53.4
Entit
y-M
entio
n Ba
selin
e 51
.2
57.8
54
.3
56.3
50
.2
53.1
Men
tion-
Ran
king
Bas
elin
e (P
ipel
ine)
52.3
61
.8
56.6
51
.6
56.7
54
.1
Men
tion-
Ran
king
Bas
elin
e (J
oint
) 50
.4
65.5
56
.9
53.0
58
.5
55.6
Clu
ster
-Ran
king
Mod
el (P
ipel
ine)
55
.3
63.7
59
.2
54.1
59
.3
56.6
Clu
ster
-Ran
king
Mod
el (J
oint
) 54
.4
70.5
61
.4
56.7
62
.6
59.5
Clu
ster
rank
ing
is b
ette
r tha
n m
entio
n ra
nkin
g, w
hich
in tu
rn is
bette
r tha
n th
e en
tity-
men
tion
mod
el a
nd th
e m
entio
n-pa
ir m
odel
Join
t mod
els
perfo
rm b
ette
r tha
n pi
pelin
e m
odel
s
1.O
nlin
e en
cycl
oped
ia a
nd le
xica
l kno
wle
dge
base
sYA
GO
Fram
eNet
2.C
oref
eren
ce-a
nnot
ated
dat
a
3.U
nann
otat
edda
ta
1.O
nlin
e en
cycl
oped
ia a
nd le
xica
l kno
wle
dge
base
sYA
GO
Fram
eNet
2.C
oref
eren
ce-a
nnot
ated
dat
a
3.U
nann
otat
edda
ta
cont
ains
5 m
illion
fact
s de
rived
from
Wik
iped
ia a
nd W
ordN
et
each
fact
is a
trip
le d
escr
ibin
g a
rela
tion
betw
een
two
NPs
<N
P1, r
el, N
P2>,
relc
an b
e on
e of
90
YAG
O re
latio
n ty
pes
cont
ains
5 m
illion
fact
s de
rived
from
Wik
iped
ia a
nd W
ordN
et
each
fact
is a
trip
le d
escr
ibin
g a
rela
tion
betw
een
two
NPs
<N
P1, r
el, N
P2>,
relc
an b
e on
e of
90
YAG
O re
latio
n ty
pes
focu
ses
on tw
o ty
pes
of Y
AGO
rela
tions
: TYP
Ean
d M
EAN
S(B
ryle
t al.,
201
0, U
ryup
ina
et a
l., 2
011)
cont
ains
5 m
illion
fact
s de
rived
from
Wik
iped
ia a
nd W
ordN
et
each
fact
is a
trip
le d
escr
ibin
g a
rela
tion
betw
een
two
NPs
<N
P1, r
el, N
P2>,
relc
an b
e on
e of
90
YAG
O re
latio
n ty
pes
focu
ses
on tw
o ty
pes
of Y
AGO
rela
tions
: TYP
Ean
d M
EAN
S(B
ryle
t al.,
201
0, U
ryup
ina
et a
l., 2
011)
TYPE
: the
IS-A
rela
tion
<Alb
ertE
inst
ein,
TYP
E, p
hysi
cist
>
<Bar
ackO
bam
a, T
YPE,
US
pres
iden
t>
cont
ains
5 m
illion
fact
s de
rived
from
Wik
iped
ia a
nd W
ordN
et
each
fact
is a
trip
le d
escr
ibin
g a
rela
tion
betw
een
two
NPs
<N
P1, r
el, N
P2>,
relc
an b
e on
e of
90
YAG
O re
latio
n ty
pes
focu
ses
on tw
o ty
pes
of Y
AGO
rela
tions
: TYP
Ean
d M
EAN
S(B
ryle
t al.,
201
0, U
ryup
ina
et a
l., 2
011)
TYPE
: the
IS-A
rela
tion
<Alb
ertE
inst
ein,
TYP
E, p
hysi
cist
>
<Bar
ackO
bam
a, T
YPE,
US
pres
iden
t>M
EAN
S: a
ddre
sses
syn
onym
y an
d am
bigu
ity<E
inst
ein,
MEA
NS,
Alb
ertE
inst
ein>
,
<E
inst
ein,
MEA
NS,
Alfr
edEi
nste
in>
cont
ains
5 m
illion
fact
s de
rived
from
Wik
iped
ia a
nd W
ordN
et
each
fact
is a
trip
le d
escr
ibin
g a
rela
tion
betw
een
two
NPs
<N
P1, r
el, N
P2>,
relc
an b
e on
e of
90
YAG
O re
latio
n ty
pes
focu
ses
on tw
o ty
pes
of Y
AGO
rela
tions
: TYP
Ean
d M
EAN
S(B
ryle
t al.,
201
0, U
ryup
ina
et a
l., 2
011)
TYPE
: the
IS-A
rela
tion
<Alb
ertE
inst
ein,
TYP
E, p
hysi
cist
>
<Bar
ackO
bam
a, T
YPE,
US
pres
iden
t>M
EAN
S: a
ddre
sses
syn
onym
y an
d am
bigu
ity<E
inst
ein,
MEA
NS,
Alb
ertE
inst
ein>
,
<E
inst
ein,
MEA
NS,
Alfr
edEi
nste
in>
prov
ide
evid
ence
that
the
two
NPs
invo
lved
are
cor
efer
ent
com
bine
s th
e in
form
atio
n in
Wik
iped
ia a
nd W
ordN
et
can
reso
lve
the
cele
brity
to M
arth
a St
ewar
tne
ither
Wik
iped
ia n
or W
ordN
etal
one
can
crea
te a
bin
ary-
valu
ed Y
AGO
feat
ure
Men
tion-
pair
mod
el
Clu
ster
-rank
ing
mod
el
crea
te a
bin
ary-
valu
ed Y
AGO
feat
ure
Men
tion-
pair
mod
elde
term
ines
whe
ther
two
NPs
are
cor
efer
ent
each
inst
ance
cor
resp
onds
to tw
o N
Ps1
if th
e tw
o N
Ps a
re in
a T
YPE
or M
EAN
S re
latio
n0
oth
erw
ise
Clu
ster
-rank
ing
mod
el
crea
te a
bin
ary-
valu
ed Y
AGO
feat
ure
Men
tion-
pair
mod
elde
term
ines
whe
ther
two
NPs
are
cor
efer
ent
each
inst
ance
cor
resp
onds
to tw
o N
Ps1
if th
e tw
o N
Ps a
re in
a T
YPE
or M
EAN
S re
latio
n0
oth
erw
ise
Clu
ster
-rank
ing
mod
elra
nks
core
fere
nce
clus
ters
pre
cedi
ng e
ach
NP
to b
e re
solv
edea
ch in
stan
ce c
orre
spon
ds to
NP
kan
d a
prec
edin
g cl
uste
r cfe
atur
es a
re d
efin
ed b
etw
een
NP
kan
d c
1 i
f NP
kan
d at
leas
t 1 N
P in
c a
re in
a T
YPE
or M
EAN
S re
latio
n0
oth
erw
ise
1.O
nlin
e en
cycl
oped
ia a
nd le
xica
l kno
wle
dge
base
sYA
GO
Fram
eNet
2.C
oref
eren
ce-a
nnot
ated
dat
a
3.U
nann
otat
edda
ta
Pete
r Ant
hony
dec
riesp
rogr
am tr
adin
gas
“lim
iting
the
gam
e to
a fe
w,”
but h
e is
not s
ure
whe
ther
he
wan
ts to
deno
unce
itbe
caus
e …
To re
solv
e it
to p
rogr
am tr
adin
g, it
may
be
help
ful t
o kn
ow1.
itan
d pr
ogra
m tr
adin
g ha
ve th
e sa
me
sem
antic
role
2.de
cry
and
deco
unce
are
“sem
antic
ally
rela
ted”
Pete
r Ant
hony
dec
riesp
rogr
am tr
adin
gas
“lim
iting
the
gam
e to
a fe
w,”
but h
e is
not s
ure
whe
ther
he
wan
ts to
deno
unce
itbe
caus
e …
Feat
ures
enc
odin
g th
e se
man
tic ro
les
of th
e tw
o N
Ps u
nder
con
side
ratio
nw
heth
er th
e as
soci
ated
pre
dica
tes
are
“sem
antic
ally
rela
ted”
coul
d be
use
ful f
or id
entif
ying
cor
efer
ence
rela
tions
.
Feat
ures
enc
odin
g th
e se
man
tic ro
les
of th
e tw
o N
Ps u
nder
con
side
ratio
nw
heth
er th
e as
soci
ated
pre
dica
tes
are
“sem
antic
ally
rela
ted”
coul
d be
use
ful f
or id
entif
ying
cor
efer
ence
rela
tions
.
Use
ASS
ERT
Prov
ides
Pro
pBan
k-st
yle
role
s (A
rg0,
Arg
1, …
)
Feat
ures
enc
odin
g th
e se
man
tic ro
les
of th
e tw
o N
Ps u
nder
con
side
ratio
nw
heth
er th
e as
soci
ated
pre
dica
tes
are
“sem
antic
ally
rela
ted”
coul
d be
use
ful f
or id
entif
ying
cor
efer
ence
rela
tions
.
Use
ASS
ERT
Prov
ides
Pro
pBan
k-st
yle
role
s (A
rg0,
Arg
1, …
)U
se F
ram
eNet
Che
cks
whe
ther
the
two
pred
icat
es a
ppea
r in
the
sam
e fra
me
Feat
ures
enc
odin
g th
e se
man
tic ro
les
of th
e tw
o N
Ps u
nder
con
side
ratio
nw
heth
er th
e as
soci
ated
pre
dica
tes
are
“sem
antic
ally
rela
ted”
coul
d be
use
ful f
or id
entif
ying
cor
efer
ence
rela
tions
.
Use
ASS
ERT
Prov
ides
Pro
pBan
k-st
yle
role
s (A
rg0,
Arg
1, …
)U
se F
ram
eNet
Che
cks
whe
ther
the
two
pred
icat
es a
ppea
r in
the
sam
e fra
me
Con
side
r tw
o ve
rbs
rela
ted
as lo
ng a
s th
ere
exis
ts a
fram
e th
at
cont
ains
bot
h of
them
Assu
me
NP
jand
NP
kar
e th
e ar
gum
ents
of t
wo
pred
icat
es
1.En
code
kno
wle
dge
from
Fra
meN
etas
one
of t
hree
val
ues
The
two
pred
icat
es a
ppea
r in
the
sam
e fra
me
Both
app
ear i
n Fr
ameN
etbu
t nev
er in
the
sam
e fra
me
One
or b
oth
of th
em d
o no
t app
ear i
n Fr
ameN
et
2.En
code
sem
antic
role
s of
NP
jand
NP
kas
one
of f
ive
valu
esAr
g0-A
rg0,
Arg
1-Ar
g1, A
rg0-
Arg1
, Arg
1-Ar
g0, O
THER
S
3.C
reat
e 15
bin
ary-
valu
ed fe
atur
es b
y pa
iring
the
3 po
ssib
le
valu
es fr
om F
ram
eNet
and
5 po
ssib
le v
alue
s fro
m A
SSER
T
Assu
me
NP
jand
NP
kar
e th
e ar
gum
ents
of t
wo
pred
icat
es
1.En
code
kno
wle
dge
from
Fra
meN
etas
one
of t
hree
val
ues
The
two
pred
icat
es a
ppea
r in
the
sam
e fra
me
Both
app
ear i
n Fr
ameN
etbu
t nev
er in
the
sam
e fra
me
One
or b
oth
of th
em d
o no
t app
ear i
n Fr
ameN
et
2.En
code
sem
antic
role
s of
NP
jand
NP
kas
one
of f
ive
valu
esAr
g0-A
rg0,
Arg
1-Ar
g1, A
rg0-
Arg1
, Arg
1-Ar
g0, O
THER
S
3.C
reat
e 15
bin
ary-
valu
ed fe
atur
es b
y pa
iring
the
3 po
ssib
le
valu
es fr
om F
ram
eNet
and
5 po
ssib
le v
alue
s fro
m A
SSER
T
Assu
me
NP
jand
NP
kar
e th
e ar
gum
ents
of t
wo
pred
icat
es
1.En
code
kno
wle
dge
from
Fra
meN
etas
one
of t
hree
val
ues
The
two
pred
icat
es a
ppea
r in
the
sam
e fra
me
Both
app
ear i
n Fr
ameN
etbu
t nev
er in
the
sam
e fra
me
One
or b
oth
of th
em d
o no
t app
ear i
n Fr
ameN
et
2.En
code
sem
antic
role
s of
NP
jand
NP
kas
one
of f
ive
valu
esAr
g0-A
rg0,
Arg
1-Ar
g1, A
rg0-
Arg1
, Arg
1-Ar
g0, O
THER
S
3.C
reat
e 15
bin
ary-
valu
ed fe
atur
es b
y pa
iring
the
3 po
ssib
le
valu
es fr
om F
ram
eNet
and
5 po
ssib
le v
alue
s fro
m A
SSER
T
Assu
me
NP
jand
NP
kar
e th
e ar
gum
ents
of t
wo
pred
icat
es
1.En
code
kno
wle
dge
from
Fra
meN
etas
one
of t
hree
val
ues
The
two
pred
icat
es a
ppea
r in
the
sam
e fra
me
Both
app
ear i
n Fr
ameN
etbu
t nev
er in
the
sam
e fra
me
One
or b
oth
of th
em d
o no
t app
ear i
n Fr
ameN
et
2.En
code
sem
antic
role
s of
NP
jand
NP
kas
one
of f
ive
valu
esAr
g0-A
rg0,
Arg
1-Ar
g1, A
rg0-
Arg1
, Arg
1-Ar
g0, O
THER
S
3.C
reat
e 15
bin
ary-
valu
ed fe
atur
es b
y pa
iring
the
3 po
ssib
le
valu
es fr
om F
ram
eNet
and
5 po
ssib
le v
alue
s fro
m A
SSER
T
Men
tion-
pair
mod
elth
e 15
feat
ures
can
be
empl
oyed
dire
ctly
by
the
men
tion-
pair
mod
el, s
ince
they
are
def
ined
on
two
NPs
Clu
ster
-rank
ing
mod
elex
tend
thei
r def
initi
ons
so th
at th
ey c
an b
e co
mpu
ted
betw
een
an N
P an
d a
prec
edin
g cl
uste
r
No
core
fere
nce
wor
k th
at e
mpl
oys
Fram
eNet
But …
rela
ted
toBe
an &
Rilo
ff’s
(200
4)us
e of
pat
tern
s fo
r ind
ucin
g do
mai
n-sp
ecifi
c co
ntex
tual
role
kno
wle
dge
Ponz
etto
& St
rube
’s(2
006)
use
of s
eman
tic ro
les
for i
nduc
ing
feat
ures
1.O
nlin
e en
cycl
oped
ia a
nd le
xica
l kno
wle
dge
base
sYA
GO
Fram
eNet
2.C
oref
eren
ce-a
nnot
ated
dat
a
3.U
nann
otat
edda
ta
Obs
erva
tion
Sinc
e w
orld
kno
wle
dge
is n
eede
d fo
r cor
efer
ence
reso
lutio
n, a
hu
man
ann
otat
or m
ust h
ave
empl
oyed
wor
ld k
now
ledg
e w
hen
core
fere
nce-
anno
tatin
g a
docu
men
t
Goa
lD
esig
n fe
atur
es th
at c
an “r
ecov
er”s
uch
wor
ld k
now
ledg
e
Obs
erva
tion
Sinc
e w
orld
kno
wle
dge
is n
eede
d fo
r cor
efer
ence
reso
lutio
n, a
hu
man
ann
otat
or m
ust h
ave
empl
oyed
wor
ld k
now
ledg
e w
hen
core
fere
nce-
anno
tatin
g a
docu
men
t
Goa
lD
esig
n fe
atur
es th
at c
an “r
ecov
er”s
uch
wor
ld k
now
ledg
e
Wha
t kin
d of
wor
ld k
now
ledg
e ca
n w
e ex
tract
from
ann
otat
ed d
ata?
1.w
orld
kno
wle
dge
for i
dent
ifyin
g co
refe
renc
ere
latio
nsif
Bara
ck O
bam
aan
d U
.S. p
resi
dent
appe
ar in
the
sam
e co
refe
renc
ech
ain
in a
trai
ning
text
, we
can
gath
er th
e w
orld
kn
owle
dge
that
Bar
ack
Oba
ma
is a
U.S
. pre
side
nt
1.w
orld
kno
wle
dge
for i
dent
ifyin
g co
refe
renc
ere
latio
nsif
Bara
ck O
bam
aan
d U
.S. p
resi
dent
appe
ar in
the
sam
e co
refe
renc
ech
ain
in a
trai
ning
text
, we
can
gath
er th
e w
orld
kn
owle
dge
that
Bar
ack
Oba
ma
is a
U.S
. pre
side
nt
2.w
orld
kno
wle
dge
for d
eter
min
ing
non-
core
fere
nce
infe
r tha
t a li
onan
d a
tiger
are
unlik
ely
to re
fer t
o th
e sa
me
entit
y af
ter r
ealiz
ing
that
they
nev
er a
ppea
r in
the
sam
e co
refe
renc
ech
ain
in th
e tra
inin
g da
ta
1.w
orld
kno
wle
dge
for i
dent
ifyin
g co
refe
renc
ere
latio
nsif
Bara
ck O
bam
aan
d U
.S. p
resi
dent
appe
ar in
the
sam
e co
refe
renc
ech
ain
ina
train
ing
text
, we
can
gath
er th
e w
orld
know
ledg
e th
at B
arac
k O
bam
ais
a U
.S. p
resi
dent
2.w
orld
kno
wle
dge
for d
eter
min
ing
non-
core
fere
nce
infe
r tha
t a li
onan
d a
tiger
are
unlik
ely
to re
fer t
o th
e sa
me
entit
y af
ter r
ealiz
ing
that
they
nev
er a
ppea
r in
the
sam
e co
refe
renc
ech
ain
in th
e tra
inin
g da
tafe
atur
es c
ompu
ted
base
d on
Wor
dNet
dist
ance
or d
istri
butio
nal
sim
ilarit
ym
ay in
corre
ct s
ugge
st th
at th
e tw
o ar
e co
refe
rent
Obs
erva
tion
The
NP
pairs
col
lect
ed fr
om c
oref
eren
ce-a
nnot
ated
trai
ning
da
ta c
ould
be
usef
ul fe
atur
es (e
.g.,
<Oba
ma,
U.S
. pre
side
nt>)
Obs
erva
tion
The
NP
pairs
col
lect
ed fr
om c
oref
eren
ce-a
nnot
ated
trai
ning
da
ta c
ould
be
usef
ul fe
atur
es (e
.g.,
<Oba
ma,
U.S
. pre
side
nt>)
How
to c
ompu
te v
alue
s fo
r the
se fe
atur
es?
Men
tion-
pair
mod
el: f
eatu
re v
alue
is1
if th
e fe
atur
e is
com
pose
d of
the
two
NPs
und
er c
onsi
dera
tion
0 o
ther
wis
e
Clu
ster
-rank
ing
mod
elEx
tend
this
feat
ure
defin
ition
so
that
the
feat
ure
can
be a
pplie
d to
an
NP
and
a pr
eced
ing
clus
ter
Pote
ntia
l pro
blem
Dat
a sp
arsi
ty: m
any
NP
pairs
in tr
aini
ng d
ata
may
not
app
ear
in te
st d
ata
Pote
ntia
l pro
blem
Dat
a sp
arsi
ty: m
any
NP
pairs
in tr
aini
ng d
ata
may
not
app
ear
in te
st d
ata
Solu
tion
Empl
oy n
ot o
nly
the
NP
pairs
as
feat
ures
but
als
o ge
nera
lized
ve
rsio
ns o
f the
se fe
atur
es. E
.g.,
repl
ace
a na
med
ent
ity b
y its
nam
ed e
ntity
tag
repl
ace
a co
mm
on N
P by
its
head
nou
n…
Rec
all t
hat …
feat
ures
enc
odin
gth
e se
man
tic ro
les
of tw
o N
Ps
whe
ther
the
asso
ciat
ed v
erbs
are
“sem
antic
ally
rela
ted”
coul
d be
use
ful f
eatu
res
for c
oref
eren
cere
solu
tion
Goa
l: cr
eate
var
iant
sof
thes
e fe
atur
es
Rec
all t
hat …
feat
ures
enc
odin
gth
e se
man
tic ro
les
of tw
o N
Ps
whe
ther
the
asso
ciat
ed v
erbs
are
“sem
antic
ally
rela
ted”
coul
d be
use
ful f
eatu
res
for c
oref
eren
cere
solu
tion
Goa
l: cr
eate
var
iant
sof
thes
e fe
atur
es
Rec
all t
hat …
feat
ures
enc
odin
gth
e se
man
tic ro
les
of tw
o N
Ps
the
asso
ciat
ed v
erbs
coul
d be
use
ful f
eatu
res
for c
oref
eren
cere
solu
tion
Goa
l: cr
eate
var
iant
sof
thes
e fe
atur
es
Rec
all t
hat …
feat
ures
enc
odin
gth
e se
man
tic ro
les
of tw
o N
Ps
the
asso
ciat
ed v
erbs
coul
d be
use
ful f
eatu
res
for c
oref
eren
cere
solu
tion
Goa
l: cr
eate
var
iant
sof
thes
e fe
atur
es
Each
feat
ure
is re
pres
ente
d by
two
verb
s an
d th
e se
man
tic ro
les
e.g.
, <de
cry,
den
ounc
e, A
rg1-
Arg1
>
They
allo
w a
lear
ner t
o le
arn
from
ann
otat
ed d
ata
whe
ther
tw
o N
Ps s
ervi
ng a
s th
e ob
ject
s of
dec
ryan
d de
noun
cear
e lik
ely
to b
e co
refe
rent
, for
inst
ance
1.O
nlin
e en
cycl
oped
ia a
nd le
xica
l kno
wle
dge
base
sYA
GO
Fram
eNet
2.C
oref
eren
ce-a
nnot
ated
dat
a
3.U
nann
otat
edda
ta
can
extra
ct s
ynta
ctic
app
ositi
ons
heur
istic
ally
show
n to
be
usef
ul fo
r cor
efer
ence
reso
lutio
n
(e
.g.,
Dau
me
& M
arcu
, 200
5, N
g, 2
007,
Hag
high
i& K
lein
, 200
9)
Each
ext
ract
ion
is a
n N
P pa
ir. E
.g.,
<Bar
ack
Oba
ma,
the
pres
iden
t>, …
can
extra
ct s
ynta
ctic
app
ositi
ons
heur
istic
ally
show
n to
be
usef
ul fo
r cor
efer
ence
reso
lutio
n
(e
.g.,
Dau
me
& M
arcu
, 200
5, N
g, 2
007,
Hag
high
i& K
lein
, 200
9)
Each
ext
ract
ion
is a
n N
P pa
ir. E
.g.,
<Bar
ack
Oba
ma,
the
pres
iden
t>, <
Del
ta A
irlin
es, t
he c
arrie
r>
Cre
ate
a da
taba
se c
onsi
stin
g of
the
synt
actic
app
ositi
ons
extra
cted
from
an
unan
nota
ted
corp
us1.
057
milli
on N
P pa
irs
Cre
ate
a bi
nary
-val
ued
feat
ure
Men
tion-
pair
mod
el: f
eatu
re v
alue
is1
if th
e tw
o N
Ps a
ppea
r as
a pa
ir in
the
data
base
0 ot
herw
ise
Clu
ster
-rank
ing
mod
elex
tend
the
defin
ition
abo
ve s
o th
at th
e fe
atur
e ca
n be
app
lied
to a
n N
P an
d a
prec
edin
g cl
uste
r
