Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
New Algorithms for Seman3cs-‐Based Machine Transla3on
USC/ISI
2012 November 2
Why Seman3cs?
• Explicit meaning representa0on will improve meaning-‐preserving transla0on
• Modeling meaning instead of surface realiza0ons → learn more from less data
2
sourcestring
targetstring
NLU NLG
WANT
BOY
GO
instance
instance
instanceARG0
ARG1
ARG0
Rooted, edge-‐labeled,leaf-‐labeled graph
The boy wants to go. 男孩子想去。
∃w, b, g : instance(w, WANT) ^ instance(g, GO) ^ instance(b, BOY) ^ agent(w, b) ^ pa0ent(w, g) ^ agent(g, b)
Five Equivalent Meaning Representa3on Formats
(w / WANT :agent (b / BOY) :pa0ent (g / GO :agent b)))
WANT
BOY
GO
instance
instance
instanceagent
pa0ent
agent
((x0 instance) = WANT((x1 instance) = BOY((x2 instance) = GO((x0 agent) = x1((x0 patent) = x2((x2 agent) = x1
instance: WANTagent:pa0ent: instance: GO
agent:
instance: BOY1
1
LOGICAL FORM PATH EQUATIONS
FEATURE STRUCTURE
DIRECTED ACYCLIC GRAPH
PENMAN
“The boy wantsto go.”
Mul3ple Roles Graph Structure
“Pascale was charged with public intoxica3on and resis3ng arrest.”
instance
charge-‐05
ARG1
ARG2
instance name
person Pascale
...
Mul3ple Roles Graph Structure
“Pascale was charged with public intoxica3on and resis3ng arrest.”
instance
charge-‐05
ARG1
ARG2
instance name
person Pascale
op2
op1
instanceand
instance
intoxicate-‐01 resist-‐01
instance
Mul3ple Roles Graph Structure
“Pascale was charged with public intoxica3on and resis3ng arrest.”
instance
charge-‐05
ARG1
ARG2
op2
op1
instanceand
instance
intoxicate-‐01
public
instance
loca3on
resist-‐01
instance name
instance
person Pascale
Mul3ple Roles Graph Structure
“Pascale was charged with public intoxica3on and resis3ng arrest.”
instance
charge-‐05
ARG1
ARG2
op2
op1
instanceand
instance
intoxicate-‐01
public
instance
loca3on
resist-‐01ARG1
instance name
instance
person Pascale
Mul3ple Roles Graph Structure
“Pascale was charged with public intoxica3on and resis3ng arrest.”
instance
charge-‐05
ARG1
ARG2
op2
op1
instanceand
instance
intoxicate-‐01
public
instance
loca3on
resist-‐01ARG1
ARG0
instance name
instance
person Pascale
Mul3ple Roles Graph Structure
“Pascale was charged with public intoxica3on and resis3ng arrest.”
instance
charge-‐05
ARG1
ARG2
op2
op1
instanceand
instance
intoxicate-‐01
public
instance
loca3on
resist-‐01ARG1
ARG0
arrest-‐01
instance
instance name
ARG1instance
person Pascale
Mul3ple Roles Graph Structure
“Pascale was charged with public intoxica3on and resis3ng arrest.”
instance
charge-‐05
ARG1
ARG2
op2
op1
instanceand
instance
intoxicate-‐01
public
instance
loca3on
resist-‐01ARG1
ARG1ARG0
arrest-‐01
instance
instance name
ARG1instance
person Pascale
Meaning-‐based MT
• Too big for just this MURI:–What content goes into the meaning representa3on?• linguis3cs, annota3on (Nathan Schneider, CMU)
–How are meaning representa3ons probabilis3cally generated, transformed, scored, ranked?• automata theory, efficient algorithms
–How can a full MT system be built and tested?• engineering, language modeling, features, training
Abstract Meaning Representa0on• 35-‐page guidelines.• Extensive use of PropBank predicates, but cover all words in sentence.• AMR Editor with logins and worksets.• 7 minutes per sentence, enabling very large-‐scale Sembanking.
• Too big for just this MURI:–What content goes into the meaning representa3on?• linguis0cs, annota0on
–How are meaning representa3ons probabilis3cally generated, transformed, scored, ranked?• automata theory, efficient algorithms
–How can a full MT system be built and tested?• engineering, language modeling, features, training
Meaning-‐based MT
General-‐Purpose Algorithms for Feature Structures (Graphs)
String World
(words)
Tree World
(syntax)
Graph World?
(seman0cs)Acceptor Finite-‐state acceptors Tree automata
Transducer Finite-‐state transducers Tree transducers
Membership checking O(n) O(n) for trees
O(n3) for strings
N-‐best … … paths through an WFSA
(Viterbi, 1967; Eppstein, 1998)
… trees in a weighted forest (Jiménez & Marzal, 2000; Huang & Chiang, 2005)
EM training Forward-‐backward EM (Baum/Welch, 1971; Eisner 2002)
Tree transducer EM training
(Graehl & Knight, 2004)
Intersec;on WFSA intersec0on Tree acceptor intersec0on
Transducer composi;on WFST composi0on
(Pereira & Riley, 1996)
Many tree transducers not closed under composi0on
(Malep et al 09)
General tools Carmel, OpenFST Tiburon (May & Knight 10)
Hyperedge Replacement Grammars
–Survey: Drewes et al., 1997
–Several NLP-‐related publica3ons from USC/ISI forthcoming
–Key idea: context-‐free rewri3ng
16
HRG Deriva3on
instance
ARG0
WANT
B
ARG1
instance
ARG0
BELIEVE
ARG1
G
instance
WANT
ARG1
= boy wants girl to believe that he is wanted
LET’S DERIVE THIS:
HRG Deriva3on
instance
ARG0
WANT
B
ARG1
instance
ARG0
BELIEVE
ARG1
G
instance
WANT
ARG1
“the boy wantssomethinginvolving himself”
LET’S DERIVE THIS:
= boy wants girl to believe that he is wanted
instance
WANT
B
X
ARG0
ARG1
HRG Deriva3on
instance
ARG0
WANT
B
ARG1
instance
ARG0
BELIEVE
ARG1
G
instance
WANT
ARG1
“the boy wantssomethinginvolving himself”
instance
WANT
B
LET’S DERIVE THIS:
= boy wants girl to believe that he is wanted
ARG0
ARG1
X
HRG Deriva3on
instance
ARG0
WANT
B
ARG1
instance
ARG0
BELIEVE
ARG1
G
instance
WANT
ARG1
instanceARG0
WANT
B
Xinstance
ARG0
BELIEVE
G
“the boy wants the girl to believesomething involving him”
ARG1
LET’S DERIVE THIS:
= boy wants girl to believe that he is wanted
HRG Deriva3on
instance
ARG0
WANT
B
ARG1
instance
ARG0
BELIEVE
ARG1
G
instance
WANT
ARG1
instanceARG0
WANT
B
Xinstance
ARG0
BELIEVE
G
“something involving B”
ARG1
LET’S DERIVE THIS:
= boy wants girl to believe that he is wanted
HRG Deriva3on
instance
ARG0
WANT
B
ARG1
instance
ARG0
BELIEVE
ARG1
G
instance
WANT
ARG1
instanceARG0
WANT
B
instance
ARG0
BELIEVE
G
instance
WANT
ARG1
ARG1
ARG1
FINISHED!
LET’S DERIVE THIS:
= boy wants girl to believe that he is wanted
General-‐Purpose Algorithms for Feature Structures (Graphs)
String World
(words)
Tree World
(syntax)
Graph World
(seman0cs)Acceptor Finite-‐state acceptors Tree automata HRG
Transducer Finite-‐state transducers Tree transducers Synchronous HRG
Membership checking O(n) O(n) for trees
O(n3) for strings
O(nk+1) for graphs
N-‐best … … paths through an WFSA
(Viterbi, 1967; Eppstein, 1998)
… trees in a weighted forest (Jiménez & Marzal, 2000; Huang & Chiang, 2005)
… graphs in a weighted forest
EM training Forward-‐backward EM (Baum/Welch, 1971; Eisner 2003)
Tree transducer EM training
(Graehl & Knight, 2004)
EM on forests of graphs
Intersec;on WFSA intersec0on Tree acceptor intersec0on Not closed
Transducer composi;on WFST composi0on
(Pereira & Riley, 1996)
Many tree transducers not closed under composi0on
(Malep et al 09)
Not closed
General tools Carmel, OpenFST Tiburon (May & Knight 10) Bolinas
SHRG Deriva3on
S
wantsB INF
instanceARG0
WANT
B
X
“the boy wantssomethinginvolving himself”
SHRG Deriva3on
S
wantsB INF
to believeG S
instanceARG0
WANT
B
X
ARG1
instance
ARG0
BELIEVE
G
“something involving B”
SHRG Deriva3on
instanceARG0
WANT
B
ARG1
instance
ARG0
BELIEVE
G
instance
WANT
ARG1
S
wantsB INF
to believeG S
is wantedhe
FINISHED!
ARG1
sourcestring
targetstring
NLU NLG
WANT
BOY
GO
instance
instance
instanceARG0
ARG1
ARG0
Rooted, edge-‐labeled,leaf-‐labeled graph
The boy wants to go. 男孩子想去。
SHRGSHRG
• Task: Given a graph H, find the best
• derivation of H
• transduction of H into a syntactic tree
Recognizing (hyper)graphs
arg1
wantʹ
believeʹ
boyʹ
girlʹ
arg1S
NP VP
the boy V S
NP VPwants
the girl V
to believe
NP
him
arg0
Recognizing (hyper)graphs
• Previous algorithms (Drewes 1997) fairly theoretical
• What we want:
• Extract a bunch of HRG rules
• Throw out the troublemakers
• Guarantee recognition in O(nk) time
Pathwidth
Pathwidth
Pathwidth
Pathwidth
Pathwidth
Treewidth
• Pathwidth k = a single boundary with at most k nodes
• Treewidth k = multiple independent boundaries, each with at most k nodes
Recognizing (hyper)graphs
• If rules have treewidth at most k, we can recognize in time O(nk+1)
want′ arg1
X
E
arg0
treewidth 2
Recognizing (hyper)graphs
treewidth runtime continents strings(words)
trees(syntax)
graphs(semantics)
1 n2 CFG
HRG in practice
2 n3 AustraliaS America
CFG TAGHRG in practice
3 n4 Africa
HRG in practice
4 n5 N AmericaEurope
HRG in practice
5 n6 Asia TAG