ParaMor & Morpho Challenge 2008 Christian Monson Jaime Carbonell, Alon Lavie, Lori Levin

ParaMor&

Morpho Challenge 2008Christian Monson

Jaime Carbonell, Alon Lavie, Lori Levin

2

You are not being taken

Turkish Morphology – Beads on a String

götür ül m sunüyor

take passive negativepresent

progressive2nd person singular

One Turkish Word

3

Computational Morphology Improves:

Machine TranslationTurkish-English (Oflazer, 2007)

Czech-English (Goldwater and McClosky, 2005)

Information RetrievalEnglish, German, Finnish (Kurimo et al., 2008)

Speech RecognitionFinnish (Creutz, 2006)

Grapheme-to-Phoneme ConversionGerman (Demberg, 2007)

4

Morphology is Complex – Operations

Prefixation

Suffixation

5


Prefixation

Reduplication

Suffixation

6


Prefixation

Reduplication

Infixation

Suffixation

7


Prefixation

Reduplication

Infixation

Suffixation

8


Prefixation

Reduplication

Infixation

Suffixation

9

götür ül m sunüyor


progressive

You are not being taken

2nd person singular

Morphology is Complex – Morphophonology

10

sunyecek

future2nd person singular


götür ül m

take passive negative

You will not be taken

11

sunyecek



götür ül m



12

sunyecek



götür ül me



13

sinyecek



götür ül me



14

sinyecek



götür ül me



15

Morphology is Complex – Ambiguity

Hungarianmentek

men +tekgo +Present.2nd.Plural‘yinz go’

16

Morphology is Complex – Ambiguity

Hungarianmentek

men +tekgo +Present.2nd.Plural‘yinz go’

men +t +ekgo +PastParticiple

+Plural‘those who have gone’

17

In Morphology Systems for New Languages

Complexity Time + Expertise

18

In Morphology Systems for New Languages

Complexity Time + Expertise

Kemal OflazerExpert on

Turkish

Computational morphology

Time3 - 4 Months to manually build a basic Turkish analyzer

Plus lexicon development and maintenance

19

The SolutionRaw Text

Unsupervised Morphology

Induction

20


?

21


Language Structure

22

Techniques for Unsupervised Morphology Induction

Transition Likelihood

Harris (1955) – Finite State Automata

Bernhard (2007)

23

Transition Likelihood

Harris (1955) – Finite State Automata

Bernhard (2007)

Minimum Description LengthGoldsmith (2001, 2006)

Creutz’s Morfessor (2006)


24

Contextual Similarity

Wicentowski (2002)

Schone (2002)


25

Contextual Similarity

Wicentowski (2002)

Schone (2002)

The ParadigmSnover (2002)

ParaMor (2007)


26

What is a Paradigm?

ül m sunüyor



götür

27

ül m sunüyor



götür

Person & Number

Paradigms Structure Inflectional Morphology

28

um

Person & Number

1st person singular

umül m üyor


progressive

götür


29

um

Person & Number

3rd person singular

umØ

ül m üyor


progressive

götür


30

umumØuz

ül m üyor


progressive

götür

Person & Number


31

umumØuz

ül m üyor


progressive

götür

ParadigmMutually substitutable morphological operations

Paradigm


32

ül m um

Voice PolarityTense & Aspect

Person & Number

umØuz

üyoryecek


33

Paradigms

ParadigmMutually substitutable morphological operations

ül m umumØuz

üyoryecek


34

Paradigm

ül m umumØuz

üyoryecek

ParadigmMutually substitutable strings

The ParaMor Algorithm

35

Paradigm

ül m umumØuz

üyoryecek

Candidate Stems

1 Morpheme Boundary


36


Simplifying Assumptions

Suffixes only70% of the World’s Languages are Suffixing (Dryer, 2005)

Strict Concatenation

37


Simplifying Assumptions

Suffixes only70% of the World’s Languages are Suffixing (Dryer, 2005)

Strict Concatenation

Only a High-Level Overview

38


Identify Paradigms in 3 Steps

ParaMorIdentify

39


Identify Paradigms in 3 Steps1. Search for candidate paradigms

ParaMorIdentify

Search

40



2. Cluster candidates modeling the same paradigm

ParaMorIdentify

SearchCluster

41




3. Filter least likely candidates

ParaMorIdentify

SearchClusterFilter

42





Segment Words Using the discovered paradigms

ParaMorIdentify

SearchClusterFilter

Segment

43




3. Filter


ParaMorIdentify

SearchClusterFilter

SegmentEvaluationResults

Today

44




3. Filter


ParaMorIdentify

SearchClusterFilter


45

s10697

autorizacionesbuscabamos

costasimportadoras

vallas…

Search for Candidate Paradigms

Propose a morpheme boundary at every character boundary in every word

Consolidate identical candidate suffixes into paradigm seeds

Word List50,000 Types

ParaMorIdentify

SearchClusterFilter


Spanish Example

46

s10697

autorizacionesbuscabamos

costaØ costasimportadoraØ importadoras

vallaØ vallas…

Ø s5513

Identify the most frequent mutually replaceable candidate suffix

Stems that occur with one suffix in a paradigm will likely occur with other suffixes in that paradigm

Search for Candidate ParadigmsParaMor

IdentifySearchClusterFilter


Spanish Example

47

s10697

A Parameter halts the introduction of suffixes When the most frequent

mutually replaceable candidate suffix severely decreases the stem count

Ø s5513

Ø r s

281autorizaciones

buscabamos costar costaØ

costasimportadoraØ importadoras

vallaØ vallas…




48

s10697

Ø s5513

Ø r s

281autorizaciones

buscabamos costar costaØ

costasimportadoraØ importadoras

vallaØ vallas…




Parameters set to produce High-recall

Spanish paradigms

And then frozen

49

Move on to the next most frequent paradigm seed

a9020

s10697

Ø s5513

Ø r s

281




50

a9020

a o2325

a o os

1418

a as o os899

s10697

Ø s5513

Ø r s

281




51

n6039

Ø n1863

Ø n r

512

Ø do n r357

Ø da das do dos n ndo r ron

115

a9020

a o2325

a o os

1418

a as o os899

s10697

Ø s5513

Ø r s

281




52

es2750

Ø es845

n6039

Ø n1863

Ø n r

512

Ø do n r357


115

a9020

a o2325

a o os

1418

a as o os899

s10697

Ø s5513

Ø r s

281

ParaMorIdentify

SearchClusterFilter



53

an1784

a an1045

a an ar

417

a an ar ó355

a ada adas ado ados an

ar aron ó148

es2750

Ø es845

n6039

Ø n1863

Ø n r

512

Ø do n r357


115

a9020

a o2325

a o os

1418

a as o os899

s10697

Ø s5513

Ø r s

281

ParaMorIdentify

SearchClusterFilter



54

strado15

rado167

rada radas rado rados

53

rada radorados

67

rada rado89

ra rada radasrado rados ran

rar raron ró23

strada strado12

strada strado stró

9

strada strado strar stró

8

strada stradas strado strar stró

7

...an

1784

a an1045

a an ar

417

a an ar ó355

a ada adas ado ados an

ar aron ó148

es2750

Ø es845

n6039

Ø n1863

Ø n r

512

Ø do n r357


115

a9020

a o2325

a o os

1418

a as o os899

s10697

Ø s5513

Ø r s

281

ParaMorIdentify

SearchClusterFilter



55

strado15

rado167

rada rado89

strada strado12

...an

1784

a an1045

es2750

Ø es845

n6039

Ø n1863

a9020

a o2325

s10697

Ø s5513

Ø r s

281

ParaMorIdentify

SearchClusterFilter



Size of Search Space

Huge: 2|candidate suffixes|

Most candidate suffixes have no common stems

Still Exponential

Greedily searched space: O(|candidate suffixes|)

This example is just 0.1% of the searched space

56

Step 2: Clustering



3. Filter


Bottom-up Agglomerative Clustering

ParaMorIdentify

SearchClusterFilter


57

Step 3: Filtering





Adapted from Harris (1955) and Goldsmith (2006)

Improved over 2007 Challenge

ParaMorIdentify

SearchClusterFilter


58

A Few of the 42 Final Paradigms4 SuffixesØ menente mente s

11 Suffixes a amente as illa illas o or ora oras ores os

41 Suffixes a aba aban acion aciones ación ada adas ado ador adora adoras adores ados amos an ando ante antes ar ara aran aremos arla arlas arlo arlos arme aron arse ará arán aré aría arían ase e en ándose é ó

29 Suffixes e edor edora edoras edores en er erlo erlos erse erá erán ería erían ida idas ido idos iendo iera ieran ieron imiento imientos iéndose ió í ía ían

20 Suffixes ida idas ido idor idores idos imos ir iremos irle irlo irlos irse irá irán iré iría irían ía ían

29 Suffixes ce cedores cemos cen cer cerlo cerlos cerse cerá cerán cería cida cidas cido cidos ciendo ciera cieran cieron cimiento cimientos cimos ció cí cía cían zca zcan zco

6 SuffixesØ es idad idades mente ísima

59

4 SuffixesØ menente mente s







A Few of the 42 Final Paradigms

Number on Nouns

60








Number & Gender on Adjectives

61








Verbal Suffixes

62




3. Filter


ParaMorIdentify

SearchClusterFilter


Improved over 2007 Challenge

63

Segment Words Using the Paradigms4 SuffixesØ menente mente s







administradas‘Feminine gender nouns under administration’

ParaMorIdentify

SearchClusterFilter


64








administr + ad + a + s

Past Participle

FemininePlural

ParaMorIdentify

SearchClusterFilter


65








administradas

Segment Words Using the ParadigmsParaMor



66








administradas administrada

Also in corpus




67












Morpheme Boundary

68












Morpheme Boundary

69








administradas administradaØ




Morpheme Boundary

70








administr + ad + a + s

Recovers multiple morpheme boundaries from candidate paradigms which each propose single morpheme boundaries

ParaMorIdentify

SearchClusterFilter


71

Linguistic EvaluationParaMor



F1

50

30

10English German Finnish Turkish Arabic

52.8

Par

aMor

72

MorfessorBaseline system for Challenge

Freely available

Minimum Description Length




F1

50

30


47.2

52.8

Mor

fess

or

Par

aMor

73

MorfessorBaseline system for Challenge

Freely available

Minimum Description Length




F1

50

30


47.2

52.8

Mor

fess

or

Par

aMor

74

Join ParaMor and MorfessorFor each word, submit 2 analyses:

a ParaMor analysis and

a Morfessor analysis

The EffectOracle Recall

Averaged Precision




F1

50

30


47.2

52.8

Mor

fess

or

Par

aMor

75




F1

50

30


47.2

52.8

56.3

Mor

fess

or

Par

aMor

Par

aMor

& M

orfe

ssor

76




F1

50

30


60.8

47.2

52.8

56.3

Mor

fess

or

Par

aMor

Par

aMor

& M

orfe

ssor

Ber

nhar

d

77




F1

50

30


60.8

52.9

47.2 47.8

52.8

44.5

56.3

Mor

fess

or

Mor

fess

or

Par

aMor

Par

aMor

Par

aMor

& M

orfe

ssor

Ber

nhar

d

Ber

nhar

d

78




F1

50

30


60.8

52.9

47.2 47.8

52.8

44.5

56.354.1

Mor

fess

or

Mor

fess

or

Par

aMor

Par

aMor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Ber

nhar

d

Ber

nhar

d

79




F1

50

30


60.8

52.9

48.247.2 47.8

40.6

52.8

44.5

39.5

56.354.1

Mor

fess

or

Mor

fess

or

Mor

fess

or

Par

aMor

Par

aMor

Par

aMor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Ber

nhar

d

Ber

nhar

d

Ber

nhar

d

48.5

80




F1

50

30


60.8

52.9

48.2

24.7

47.2 47.8

40.6

52.8

44.5

39.5

56.354.1

Mor

fess

or

Mor

fess

or

Mor

fess

or

Par

aMor

Par

aMor

Par

aMor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Ber

nhar

d

Ber

nhar

d

Ber

nhar

d

Ber

nhar

d

48.5

81




F1

50

30


60.8

52.9

48.2

24.7

47.2 47.8

40.6

37.1

52.8

44.5

39.5

56.354.1

Mor

fess

or

Mor

fess

or

Mor

fess

or

Mor

fess

or

Par

aMor

Par

aMor

Par

aMor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Ber

nhar

d

Ber

nhar

d

Ber

nhar

d

Ber

nhar

d

48.5

82




F1

50

30


60.8

52.9

48.2

24.7

47.2 47.8

40.6

37.1

52.8

44.5

39.5

46.5

56.354.1

Mor

fess

or

Mor

fess

or

Mor

fess

or

Mor

fess

or

Par

aMor

Par

aMor

Par

aMor

Par

aMor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Ber

nhar

d

Ber

nhar

d

Ber

nhar

d

Ber

nhar

d

48.5

83




F1

50

30


60.8

52.9

48.2

24.7

47.2 47.8

40.6

37.1

52.8

44.5

39.5

46.5

56.354.1

52.0

Mor

fess

or

Mor

fess

or

Mor

fess

or

Mor

fess

or

Par

aMor

Par

aMor

Par

aMor

Par

aMor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Ber

nhar

d

Ber

nhar

d

Ber

nhar

d

Ber

nhar

d

48.5

84




F1

50

30


60.8

52.9

48.2

24.7

21.9

47.2 47.8

40.6

37.1

34.0

52.8

44.5

39.5

46.5

15.4

56.354.1

52.0

Mor

fess

or

Mor

fess

or

Mor

fess

or

Mor

fess

or

Mor

fess

or

Par

aMor

Par

aMor

Par

aMor

Par

aMor

P.

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Ber

nhar

d

Ber

nhar

d

Ber

nhar

d

Ber

nhar

d

Zem

an

48.5

85




F1

50

30


60.8

52.9

48.2

24.7

21.9

47.2 47.8

40.6

37.1

34.0

52.8

44.5

39.5

46.5

15.4

56.354.1

52.0

40.9

Mor

fess

or

Mor

fess

or

Mor

fess

or

Mor

fess

or

Mor

fess

or

Par

aMor

Par

aMor

Par

aMor

Par

aMor

P.

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Ber

nhar

d

Ber

nhar

d

Ber

nhar

d

Ber

nhar

d

Zem

an

48.5

86




F1

50

30


60.8

52.9

48.2

24.7

21.9

47.2 47.8

40.6

37.1

34.0

52.8

44.5

39.5

46.5

15.4

56.354.1

52.0

40.9

Mor

fess

or

Mor

fess

or

Mor

fess

or

Mor

fess

or

Mor

fess

or

Par

aMor

Par

aMor

Par

aMor

Par

aMor

P.

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Ber

nhar

d

Ber

nhar

d

Ber

nhar

d

Ber

nhar

d

Zem

an

48.5

Sometimes Morfessor wins

87




F1

50

30


60.8

52.9

48.2

24.7

21.9

47.2 47.8

40.6

37.1

34.0

52.8

44.5

39.5

46.5

15.4

56.354.1

52.0

40.9

Mor

fess

or

Mor

fess

or

Mor

fess

or

Mor

fess

or

Mor

fess

or

Par

aMor

Par

aMor

Par

aMor

Par

aMor

P.

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Ber

nhar

d

Ber

nhar

d

Ber

nhar

d

Ber

nhar

d

Zem

an

48.5

Sometimes ParaMor wins

88




F1

50

30


60.8

52.9

48.2

24.7

21.9

47.2 47.8

40.6

37.1

34.0

52.8

44.5

39.5

46.5

15.4

56.354.1

52.0

40.9

Mor

fess

or

Mor

fess

or

Mor

fess

or

Mor

fess

or

Mor

fess

or

Par

aMor

Par

aMor

Par

aMor

Par

aMor

P.

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Ber

nhar

d

Ber

nhar

d

Ber

nhar

d

Ber

nhar

d

Zem

an

48.5

ParaMor and Morfessor are Complementary

89

IR EvaluationParaMor



F1

45

35


39.4

36.4

39.339.9

Mor

fess

or

Par

aMor

Par

aMor

& M

orfe

ssor

Ber

nhar

d

90




F1

45

35


39.4

47.3

36.4

46.7

39.3

36.3

39.9

47.3

Mor

fess

or

Mor

fess

or

Par

aMor

Par

aMor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Ber

nhar

d

Ber

nhar

d

91




F1

45

35


39.4

47.3

49.2

36.4

46.7 46.8

39.3

36.3

39.739.9

47.3

Mor

fess

or

Mor

fess

or

Mor

fess

or

Par

aMor

Par

aMor

Par

aMor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Ber

nhar

d

Ber

nhar

d

Ber

nhar

d

46.7

92




F1

45

35


39.4

47.3

49.2

36.4

46.7 46.8

39.3

36.3

39.739.9

47.3

Mor

fess

or

Mor

fess

or

Mor

fess

or

Par

aMor

Par

aMor

Par

aMor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Par

aMor

& M

orfe

ssor

Ber

nhar

d

Ber

nhar

d

Ber

nhar

d

46.7

93

ParaMor: State-of-the-Art Unsupervised Morphology Induction System

ParaMorIdentifies paradigms

The organizing structure of inflectional morphology

Segments words As discovered paradigms suggest

Combined with MorfessorAmong the best in Morpho Challenge

Consistent across languages

94

The Next Steps for ParaMor

Beyond suffixesStraightforward extension to ParaMor for

Prefixes

More ChallengingReduplication, Infixation, etc.

Morphophonology

Incorporate contextual information when clustering

Improve system combinationTrue merging of analysesCombine more systems

95

96

Documents

ParaMor & Morpho Challenge 2008 Christian Monson Jaime Carbonell, Alon Lavie, Lori Levin