Embed Size (px)
Citation preview
Modeling morphology in translation
Inflectional morphologyWord form changes to reflect case, person,
number, gender, animacy, tense, mood, definiteness, social status, negation, etc.
Inflectional morphologyWord form changes to reflect case, person,
number, gender, animacy, tense, mood, definiteness, social status, negation, etc.
Politeness to addressee
Referent honorific
Both
Inflectional morphologyInflection is not just concatenation
Reduplication
Infixation
Vowel harmony
Inflectional morphologyInflection systematically increases the number of word forms that a system must understand and/ or generate. So it affects data sparsity (if we only model word forms).
Inflectional morphologyIt also affects meaning
cf. “Kim loves Pat” vs. “Pat loves Kim” (syntax also affects meaning)
Agglutinationostoskeskuksessa
ostos#keskus+N+Sg+Loc:inshopping#center+N+Sg+Loc:in
‘in the shopping center’
ᖃᖓᑕᓲᒃᑯᕕᒻᒨᕆᐊᖃᓛᖅᑐᖓ qangatasuukkuvimmuuriaqalaaqtunga
“I'll have to go to the airport”
A basic oracle experiment (Goldwater & mcClosky 2005)
Translating from rich inflections
Translating from rich inflectionsA basic oracle experiment (Goldwater & mcClosky 2005)
Translating from rich inflectionsA basic oracle experiment (Goldwater & mcClosky 2005)
Translating from rich inflectionsA basic oracle experiment (Goldwater & mcClosky 2005)
Translating from rich inflectionsA basic oracle experiment (Goldwater & McClosky 2005)
Translating from rich inflectionsA basic oracle experiment (Goldwater & mcClosky 2005)
Encouraging, but in most data, we don’t have the morphological analysis (and it is ambiguous)
Translating from rich inflections
Rather than decide on a particular representation of words, allow translation system to choose.
Translating from rich inflections
Rather than decide on a particular representation of words, allow translation system to choose.
Translating from compounding
Translating from compoundingByte pair encoding:
iteratively replace most common bigram with new
symbol (stop using heuristic) aaabdaaabac ZabdZabac
ZYdZYac XdXac
When applied to words:
Morphological analysis
прочий +Adj +Sg +Neut +Instrпрочий +Adj +Sg +Masc +Instrпрочий +Adj +Pl +Datпрочить +Verb +Pl +1Pпрочее +Pro +Sg +Ins
прочим
… is ambiguous
word
possible analyses
But… generation is generally unambiguous
Translating to rich inflections
Translating to rich inflections
genitive case: marks noun modifying another noun
Translating to rich inflections
verb and its argument must agree on these features
Translating to rich inflections
p(y|x) =nY
t=1
p(yt|yt�1...yt�k, xt)
Translating to rich inflections
p(y|x) =nY
t=1
p(yt|yt�1...yt�k, xt)
Translating to rich inflections
p(y|x) =nY
t=1
p(yt|yt�1...yt�k, xt)
Translating to rich inflections
Translating to rich inflections
Handling unknown input words
Word representations composed from characters can be applied to any word, even when unseen.
Generating novel output words
Character-based language models can generate any word
Character-based neural MT
Character-based neural MT
English & German
English & German
English & German
English & German
English & Arabic
English & Arabic
English & Arabic
English & Arabic
English & Arabic
English & Turkish
English & Turkish
Word Order: Syntax
The brown dog on the mat saw the striped cat through the window. The brown cat saw the striped dog through the window on the mat.
i.e. the difference between a sentence and a bag of words.
Q: Does IBM Model 1 give these translations different probabilities (given a fixed conditioning sentence)?
Garcia and associates .
Garcia y asociados .Carlos Garcia has three associates .
Carlos Garcia tiene tres asociados .his associates are not strong .
sus asociados no son fuertes .Garcia has a company also .
Garcia tambien tiene una empresa .its clients are angry .
sus clientes estan enfadados .the associates are also angry .
los asociados tambien estan enfadados .
la empresa tiene enemigos fuertes en Europa .
the company has strong enemies in Europe .the clients and the associates are enemies .
los clientes y los asociados son enemigos .the company has three groups .
la empresa tiene tres grupos .its groups are in Europe .
sus grupos estan en Europa .the modern groups sell strong pharmaceuticals .
los grupos modernos venden medicinas fuertes .the groups do not sell zanzanine .
los grupos no venden zanzanina .the small groups are not modern .
los grupos pequenos no son modernos .
Garcia and associates .
Garcia y asociados .Carlos Garcia has three associates .
Carlos Garcia tiene tres asociados .his associates are not strong .
sus asociados no son fuertes .Garcia has a company also .
Garcia tambien tiene una empresa .its clients are angry .
sus clientes estan enfadados .the associates are also angry .
los asociados tambien estan enfadados .
la empresa tiene enemigos fuertes en Europa .
the company has strong enemies in Europe .the clients and the associates are enemies .
los clientes y los asociados son enemigos .the company has three groups .
la empresa tiene tres grupos .its groups are in Europe .
sus grupos estan en Europa .the modern groups sell strong pharmaceuticals .
los grupos modernos venden medicinas fuertes .the groups do not sell zanzanine .
los grupos no venden zanzanina .the small groups are not modern .
los grupos pequenos no son modernos .
Garcia and associates .
Garcia y asociados .Carlos Garcia has three associates .
Carlos Garcia tiene tres asociados .his associates are not strong .
sus asociados no son fuertes .Garcia has a company also .
Garcia tambien tiene una empresa .its clients are angry .
sus clientes estan enfadados .the associates are also angry .
los asociados tambien estan enfadados .
la empresa tiene enemigos fuertes en Europa .
the company has strong enemies in Europe .the clients and the associates are enemies .
los clientes y los asociados son enemigos .the company has three groups .
la empresa tiene tres grupos .its groups are in Europe .
sus grupos estan en Europa .the modern groups sell strong pharmaceuticals .
los grupos modernos venden medicinas fuertes .the groups do not sell zanzanine .
los grupos no venden zanzanina .the small groups are not modern .
los grupos pequenos no son modernos .
Same pattern:NN JJ → JJ NN
Garcia and associates .
Garcia y asociados .Carlos Garcia has three associates .
Carlos Garcia tiene tres asociados .his associates are not strong .
sus asociados no son fuertes .Garcia has a company also .
Garcia tambien tiene una empresa .its clients are angry .
sus clientes estan enfadados .the associates are also angry .
los asociados tambien estan enfadados .
la empresa tiene enemigos fuertes en Europa .
the company has strong enemies in Europe .the clients and the associates are enemies .
los clientes y los asociados son enemigos .the company has three groups .
la empresa tiene tres grupos .its groups are in Europe .
sus grupos estan en Europa .the modern groups sell strong pharmaceuticals .
los grupos modernos venden medicinas fuertes .the groups do not sell zanzanine .
los grupos no venden zanzanina .the small groups are not modern .
los grupos pequenos no son modernos .
Same pattern:NN JJ → JJ NN
Phrase-based models do not capture this generalization.
