Proc. of ICON-2017, Kolkata, India. December 2017 c©2017 NLPAI, pages 485–494

Neural Morphological Disambiguation Using Surface andContextual Morphological Awareness

Akhilesh SudhakarIIT (BHU), Varanasi, Indiaakhileshs.s4@gmail.com

Anil Kumar SinghIIT (BHU), Varanasi, India

nlprnd@gmail.com

Abstract

Morphological disambiguation, partic-ularly for morphologically rich lan-guages, is a crucial step in many NLPtasks. Morphological analyzers providemultiple analyses of a word, only one ofwhich is true in context. We presenta language-agnostic deep neural sys-tem for morphological disambiguation,with experiments on Hindi. We achieveaccuracies of around 95.22% withoutthe use of any language-specific fea-tures or heuristics, which outperformsthe existing state of the art. One con-tribution through this work is build-ing the first morphological disambigua-tion system for Hindi. We also showthat using phonological features canimprove performance. On using phono-logical features and pre-trained wordvectors, we report an accuracy of97.02% for Hindi.

1 IntroductionMorphologically inflected words are derivedfrom a root by modifying it (e.g., by apply-ing prefixes, suffixes and infixes) based on lin-guistic features (manifested as the inflectiontagset). Morphological analysis involves ex-tracting this root word and the set of featuresthat describe the inflected form. These anal-yses contain syntactic and semantic informa-tion about inflected words. Table 1 shows anexample for the Hindi word ‘पूरे’ [pUre]1. Ex-isting morphological analyzers typically workin isolation, meaning that they generate multi-ple analyses of a word, purely based on surfacestructure. For many NLP tasks like machine

1We use the Roman notation popularly known asWX for representing Hindi words

translation and topic modeling, however, it isessential to know which morphological analy-sis is correct in the context of the sentence.Morphological disambiguation aims to solvethis problem. The task of disambiguation isnon-trivial and is complicated by the depen-dencies of the correct analysis on the surfacestructure of the inflected word, on the surfacestructures of the neighboring words, and onthe analyses of neighboring words.

We present a deep neural morphological dis-ambiguation system that leverages context in-formation as well as surface structure. Whilewe have experimented on Hindi in our work,we report accuracies without employing anylanguage-specific features to show that oursystem can generalize across different lan-guages. We also show performance boost byusing phonological features and pre-training ofword vectors. To the best of our knowledge,this is the first ever non-naive morphologicaldisambiguation system to be built for Hindi.

Like other Indo-Aryan languages, Hindi ismorphologically rich and a word form mayhave over 40 morphological analyses (Goyaland Lehal, 2008). Though the inflectionalmorphology of Hindi is not agglutinative, thederivational suffixes are. This leads to an ex-plosion in the number of inflectional root forms(Singh et al., 2013). One of the reasons forour focus on Hindi is that it has a wide cov-erage of speaking population, with over 260million speakers across 5 countries2 and is thefifth most spoken language in the world3. Wepresent four neural architectures for this task,each different from the others by the natureof context information used as disambiguating

2https://www.ethnologue.com/statistics/size3The exact rank may be a matter of debate due to

the socio-linguistic scenario in South Asia, with somesurveys claiming it to be even more popularly spoken.485

Root Category Gender Number Person Case TAM SuffixpUrA adj m sg - o - -pUrA adj m pl - d - -pUrA adj m pl - o - -pUrA n m pl 3 d 0 0pUrA n m sg 3 o 0 0pUra v any sg 2 - ए epUra v any sg 3 - ए epUra v m pl any - या yA

Table 1: Morphological analyses of the word `पूरे' [pUre] (A ‘-’ indicates that the feature is notapplicable and an ‘any’ indicates that it can take any value in the domain for that feature)

evidence. We assess our results by implement-ing an existing state-of-the-art system (Shenet al., 2016) on our Hindi dataset. Our systemoutperforms this state-of-the-art system.

2 Related Work

There is very little directly corresponding pre-vious work on morphological disambiguationand it cannot be formulated in the same wayas POS tagging. This is because the numberof classes is fixed in POS tagging, whereas itis variable in our problem. Still, since partof speech (POS) tagging is a closely relatedtask, the work on POS tagging can also pro-vide useful insights. However, morphologicaldisambiguation is a harder task to performthan POS tagging. The earliest approachesto POS tagging were rule-based (Karlsson etal. (1995), Brill (1992)) and required a set ofhand-crafted rules learnt from a tagged cor-pus. More recently, Kessikbayeva and Ci-cekli (2016) present a morphological disam-biguation system using rules based on disam-biguations of context words.

Statistical approaches are also used for mor-phological disambiguation. Hakkani-Tür etal. (2000) propose a model based on joint con-ditional probabilities of the root and tags. Saket al. (2007) use a perceptron model, whileother statistical models use decision trees asby Görgün and Yildiz (2011). Hybrid ap-proaches have also been tried, with Orosz andNovák (2013) using an approach combiningrule-based and statistical approaches, to prunegrammar-violating parses.

The use of deep learning for morphologi-cal disambiguation, has been explored. Strakaand Straková (2017) build a neural system fortasks such as sentence segmentation, tokeniza-tion and POS tagging. Plank et al. (2016)

build a multilingual neural POS tagger. Whilewe draw insights from works on tasks suchas POS tagging, we bear in mind that POStagging and morphological disambiguation aresignificantly different. Morphological disam-biguation is more complex because it workswith multiple categories and not just part-of-speech. This introduces sparseness in themodel, as well as considerations of whetherthe different categories are dependent on eachother, on how to combine classifiers for eachcategory, etc. The number of analyses for aword also varies.

Yildiz et al. (2016) propose a convolu-tional neural net architecture, which takescontext disambiguation into account. Shenet al. (2016) use a deep neural model withcharacter-level and well as tag-level LSTMs toembed analyses. Our work shares certain as-pects in common with theirs but is different inmany ways. We experiment on Hindi (whichhas significantly different morphological prop-erties from the three languages they explore),use different neural structures, show the effectof language specific phonological features andstudy the impact of unsupervised pre-trainingof embeddings under different settings. Fur-ther, as mentioned earlier, we consider theirresults to be state-of-the-art because theirs isa language-agnostic system which gives state-of-the-art results on all 3 languages they haveexperimented on. We show that our modelgives better performance than an implemen-tation of their best model on Hindi.

3 Neural Models

We present four models for morphologicaldisambiguation. Some aspects are commonamong them. They all use a deep neural net-work, which, given the current word in con-

2

486

sideration and one of the candidate morpho-logical analyses of the word, acts as a binarytrue/false classifier. A final softmax layer out-puts probabilities for correct and incorrect,based on whether the candidate analysis is cor-rect or not. An ideal classifier would predictthe probability of correct as 1 and incorrectas 0 for the correct morphological analysis ofthe word. As is usual in word sense disam-biguation, we make the ‘one sense per colloca-tion’ assumption (a word in a particular con-text has only one correct morphological anal-ysis), with which our dataset is in accordance.The choices of neural architectures used by usare influenced by the findings in the work ofHeigold et al. (2016), in which the authors con-clude that on morphological tagging tasks, dif-ferent neural architectures (CNNs, RNNs etc.)give comparable results, and careful tuning ofmodel structure and hyperparameters can givesubstantial gains. We also draw insights fromtheir work on augmenting character and word-level embeddings.

3.1 Terminology UsedFor each of ‘category’, ‘gender’, ‘number’,‘person’, ‘case’, ‘TAM’ and ‘suffix’, we use theterm ‘feature’. We call each of the values of afeature for a particular word, a ‘tag’. For in-stance the feature ‘gender’ can have tags ‘M(male)’, ‘F (female)’ and ‘N (neuter)’. Theroot and the tagset together make up a mor-phological analysis for a word. We use theterm ‘candidate analysis’ to refer to eachof the morphological analyses generated by theanalyzer for a given word.

3.2 Broad Basis for the ArchitecturesWe first establish an intuitive and statisticalfoundation to justify our decision choices inbuilding the deep neural network. We extractdependencies between roots and features fromthe work by Hakkani-Tür et al. (2000), notingthat the assumptions used for Turkish by theauthors hold good for Hindi too. We also ob-tain surface-information-related dependenciesfrom the work by Faruqui et al. (2016). Thefollowing is the full set of extended dependen-cies:

• Dependency #1: The root of a worddepends on the roots as well as the fea-

tures of all previous and following wordsin the window

• Dependency #2: Each feature of aword depends on the roots and the fea-tures of all previous and following wordsin the window

• Dependency #3: Each feature of aword depends on the root of the currentword, as well as all other features of thecurrent word

• Dependency #4: The root and eachfeature of a word depend on the surfaceform of the word

• Dependency #5: The root and eachfeature of a word depend on the surfaceforms of the word as well as those of allprevious and following words in the win-dow.

In all these four models, the following networkcomponents are also consistent.

Figure 1: Architecture for the word surfacevector. ‘i’ indicates the ith input word.

3.3 Word InputWord inputs to the network are embedded attwo levels. A word embedding vector is gener-ated using the word as a whole. Each charac-ter in the word is also embedded in a characterembedding vector and these character embed-dings are fed, in sequence, to a bidirectionalGRU. The output vector of the GRU and the

3

487

word embedding vector are concatenated to-gether to form the ‘word surface vector’that takes into account surface features of theword. The part of the network that generatesthe word surface vector is shown in Figure 1.

The character-level GRU allows for captur-ing of surface properties of a word, and takesinto account Dependency #4 (section 3.2).We obtain marginal accuracy gains (of around0.1%) by using a GRU instead of an LSTM atthe character-level.

3.4 Candidate Analysis InputCandidate analysis inputs to the network aretreated as two inputs: the root word and restof the tags. The root is treated in the sameexact fashion as the word inputs (for the samereasons mentioned in the above section) withthe only difference being that all words share acommon embedding layer, while all roots sharea separate common embedding layer. Conse-quently, a corresponding ‘root surface vec-tor’ will be generated as described for eachroot input. Tagsets are represented as binaryvectors, with positional encoding. The rootsurface vector and all the tag encodings aretreated as a sequence and fed as input to abidirectional LSTM. This design choice, in-cluding the bidirectionality, has been made toaddress Dependency #3 (section 3.2). We callthe output vector of this LSTM, the ‘root fea-tures sequence vector’. The part of thenetwork architecture that generates the rootfeatures sequence vector is shown in Figure 2.

3.5 Hyperparameters and TrainingAll GRUs and LSTMs have a hidden layer sizeof 256, and deep GRUs and LSTMs have adepth of 2 layers. Deep convolutional net-works have a filter width of 3, hidden layersize of 64 and depth of 3 layers. The modelcan run for 10,000 epochs but we make use ofearly stopping with a patience of 10 epochsin order to keep the generalization error incheck. This is a validation-based early stop-ping on the development set. The word androot embedding layers have a dimension of100, while the character embedding layer hasa dimension of 64. All models use the categor-ical cross-entropy loss function and the Adamoptimization method as proposed by Kingmaand Ba (2014). The sequence of words in each

Figure 2: Network architecture for generatingthe root feature sequence vector. The sub-script i indicates the morphological featuresof the ith input word. These might be candi-date analyses or correct analyses, dependingon where (training or testing) they are used inother figures.

sentence act as a mini-batch during training.The best model is saved for predictions on thetest data.

3.6 Baseline ModelAs mentioned earlier, since there does not ex-ist a non-naive state of the art system forHindi, we use a low baseline model that picksone candidate analysis at random and predictsthis to be the correct morphological analy-sis for the given word. This4 is the defaultfor building several machine translation (MT)systems, such as the Sampark5 system, for In-dian languages. The most that is currentlydone for these MT systems is to apply someagreement based rules6. Proper evaluation ofthese modules may be needed, but it is beyondthe scope of this paper. It must be mentionedhere that the baseline we have picked is a weakbaseline. However, we have done so for a cou-ple of reasons. Firstly, we compare our resultsto the state-of-the-art system mentioned ear-lier and show performance gain. Therefore itis not a case of inflation of results using a weak

4The so called ‘pick one morph’ module.5http://sampark.org.in6The ‘guess morph’ module.

4

488

baseline. Secondly, we have presented 4 mod-els which show a gradation of performance onthis task (as shown in Table 5). Thirdly, thebaseline presents the case when absolutely nocharacter, word or context-level information isavailable to the model. We believe that sincewe study the impact of each of these kindsof knowledge on the models’ performance, wemust also study the case when none of thisknowledge is available.

3.7 Model 1This model solely relies on the surface infor-mation of the current word to make predic-tions about its morphological analysis. Givena current word and the root and tags of thecandidate analysis, the model uses only theword surface vector (section 4.2) with the rootfeatures sequence vector (section 4.3) of thecandidate to make predictions. Figure 3 showsthe exact structure of the network used.

Figure 3: Structure of Model 1. The subscript‘0’ indicates the current word. P(T) and P(F)indicate probability of True and False respec-tively.

3.8 Model 2This model makes use of not only the currentword’s surface information but also the surfaceinformation of all words in a window which has4 words to the left and 4 words to the right ofthe current word in the sentence. (We exper-iment with values from 2 to 6). Building themodel this way accounts for Dependency #5(section 3.2). This model uses out-of-sentencetokens too, to ensure that words towards the

beginning and end of the sentence also have afull window. The word surface vectors (section4.2) of all the words in this window, along withthe root features sequence vector (section 4.3)of the candidate are fed into a bidirectionalLSTM in this model. Figure 4 shows the ex-act network structure used.

Figure 4: Model 2. The subscript ‘0’ indi-cates the current word, the subscript ‘-m’ rep-resents any word in the left context of the cur-rent word and the subscript ‘+m’ representsany word in the right context of the currentword. P(T) and P(F) indicate probability ofTrue and False respectively.

3.9 Model 3This model makes use of not only the currentword’s surface information and the surface in-formation of all words in a window which has5 words to the left of the current word inthe sentence, but also the correct morpho-logical annotations of the words in this left-context. This model partially accounts forDependency #1 and Dependency #2 (section3.2). The word surface vector (section 4.2) ofeach word is concatenated with the root fea-tures sequence vector (section 4.3) of the wordto give a ‘complete vector’. Complete vectors,each concatenated with their own convolutionsare fed as inputs to a deep LSTM. Model 3uses the network structure shown in Figure 5,except that Figure 5 shows the model using

5

489

the current word’s left and right context, whileModel 3 uses only its left context.

Figure 5: Model 4. Model 3 also has a sim-ilar configuration except for the context ofthe current word. The subscript ‘0’ indicatesthe current word, the subscript ‘-m’ representsany word in the left context of the currentword and the subscript ‘+m’ represents anyword in the right context of the current word.P(T) and P(F) indicate probability of Trueand False respectively.

3.10 Model 4Model 4 is similar to Model 3, except that thismodel makes use of surface information andcorrect morphological annotations of not onlywords to the left but also to the right of thecurrent word. The complete window for thecurrent word has 4 words to the left and 4words to the right of it. (We experiment withvalues from 2 to 6). This model takes into ac-count all the dependencies mentioned in sec-tion 3.2. Figure 5 shows the network structurefor Model 4.

It must be mentioned here that Model 3and Model 4 are slightly complex due to thepresence of a CNN and further concatenationof the convolved inputs with the original in-puts. We have conducted experiments on sim-pler versions of Model 3 and Model 4 but withpoorer results (an average drop in accuracy of

1.2%). An elaborate discussion of these re-sults is not possible due to space constraints.Specifically, we used the same context as usedin these two models but did not use the inter-mediary CNN in these simpler experiments.The reasons for the performance improvementupon using a CNN could be that CNNs haveproved to be particularly useful for classifica-tion tasks on data that has the property of lo-cal consistency. This is evident from previouswork on using CNNs for similar classificationtasks such as those by Collobert et al. (2011),Yildiz et al. (2016) and Heigold et al. (2016).Well-formed sentences of any language (Hindi,in our case) display local consistency becausethey have a natural order, with context wordsforming abstractive concepts/features. Hence,a CNN was an intuitive choice for our task.

4 Experimental setup

4.1 DatasetWe use a manually annotated Hindi Depen-dency TreeBank7, which is part of the Hindi-Urdu Dependency TreeBank (HUTB)8 as thesource of the correct morphological analysisof words in the context of their sentences.The treebank annotates words from sentencestaken from news articles and textual conversa-tions. Each word in every sentence of the tree-bank is annotated with the correct morpholog-ical analysis. We use a morphological analyzerfor Hindi9, which was developed earlier, butis now used for the Sampark MT system andother purposes. We use it to generate the dif-ferent possible morphological analyses for eachword (in isolation) in the treebank. For eachword in the treebank, the candidate analysismatching the word’s treebank-annotated mor-phological analysis is labeled as true while allother candidate analyses are labeled as false.In the entire dataset, we ensure that out ofall the candidate analyses of a word, there isone that matches the treebank annotation forthat word. Table 2 provides specific statisticsabout the dataset used. Table 3 describes thefeatures of a morphological analysis as well asprovides the domain of possible tags (values)for each feature.

7http://ltrc.iiit.ac.in/treebank_H20148http://verbs.colorado.edu/hindiurdu9http://sampark.iiit.ac.in/hindimorph

6

490

Attribute CountTotal words 298,285Unique words 17,315Manual additions of 115432

treebank annotationAmbiguous words 179,453Unambiguous words 118,742Sentences in treebank 13,933Mean sentence length 21.40Mean morphological

analyses per word 2.534Mean morphological analyses

per ambiguous word 3.550Standard deviation of morpho-

-logical analyses per word 1.620Maximum morphological

analyses for a word 10

Table 2: Dataset statistics

Feature List of possible tagsname

Root Not fixedCategory Noun(n), Pronoun(pn), Adjective(adj)

verb(v), adverb(adv)post-position(psp), avvya(avy)

Gender Masculine(m), Feminine(f),Neuter(n)

Number Singular(sg), Plural(pl), Dual(d)Person 1st Person(1), 2nd Person(2),

3rd Person(3)Case Direct(d), Oblique(o)TAM है,का,ना,में,या,या1,से,ए,

कर,ता,0,था,को,गा,नेSuffix kA,e,wA,WA,yA,nA,ko,ne,

0,kara,gA,yA1,meM,se,hE

Table 3: Domain of tags (values) for each fea-ture. The tag ‘TAM’ denotes the tense, aspectand modality marker.

We would like to mention here that sinceour system is built to pick the correct analysisfrom the morphological analyses generated bythe analyzer, it assumes that every word hasa set of candidate analyses. In the context ofour task, it is not relevant to discuss the casewhen the morphological analyzer itself fails toprovide candidate analyses.

Table 4 shows the number of ambiguouswords and the total number of words usedfor training, development and testing. Thetest set is held-out and is used solely for re-porting final results. The development set isused to validate and tune model hyperparam-eters. During testing, the model is providedwith only the different candidate morpholog-ical analysis outputs from the morphologicalanalyzer, for each word. The correct analyses(also referred to as annotations in the follow-

Phase Ambiguous Words Total WordsTraining 149,540 248,572Development 11,963 19,885Testing 17,950 29,828

Table 4: Word counts in training, developmentand test splits

ing section) provided by the treebank for eachword are used to calculate the reported accu-racies.

4.2 Methods of TestingModels that use morphological analyses of thecontext words (Models 3 and 4) have access tocorrect annotations of these contexts duringtraining and validation but not during test-ing. During testing, in order to provide ‘cor-rect annotations’ of words in the context ofa word, to these models, we use Model 2 orModel 3. This is because Model 2 does notitself use context annotations and Model 3 it-self uses only left context annotations (it canhence, predict the correct morphological anal-ysis for each word in the test data from left toright, treating the predictions of the previouswords as the correct annotations of the left-context of the words that occur next). How-ever, in the case where Model 3 is being usedas the test set annotator, a strategic choice hasto be made for annotation. A greedy strategywould pick the morphological analysis with thehighest softmax probability of being the cor-rect annotation and annotate the word withthis annotation. However, the greedy strat-egy fails if the model makes mistakes towardsthe start of a sentence or performs poorly ononly some types of words, because these errorspropagate to every consecutive word and getcompounded. In order to avoid these kinds oferrors, we use a beam search with width 10for pre-annotating the test set in the case ofcontext-based models.

5 Results and Analysis

Table 5 presents performance accuracies of dif-ferent models and with different methods usedto annotate test data in the cases where aninitial pre-annotation of test data is needed,as discussed in the previous section. As men-tioned before, model accuracy is calculated bycomparing each trained model’s predictions on

7

491

Model Pre-testing Accuracy on Accuracy onTest Data Ambiguous All Words

Annotation WordsModel

Baseline NA 29.40 38.43S-O-T-A S-O-T-A 90.13 92.06Model 1 NA 80.21 83.96Model 2 NA 87.35 89.18Model 3 Treebank 93.82 96.17Model 3 Model 2 89.40 93.35Model 3 Model 3 91.23 94.90Model 4 Treebank 94.77 97.59Model 4 Model 2 90.41 94.74Model 4 Model 3 92.65 95.22

Table 5: Performance of different models (allaccuracies are percentages). S-O-T-A standsfor state-of-the-art, which is the full-contextmodel of Shen et al. (2016). The accuracygain we have achieved over S-O-T-A is 2.8%on ambiguous words and 3.43% on all words.

the test data with the correct analyses fromthe treebank data, regardless of which modelwas used for the initial test annotation (if any).The standard measure for accuracy is used:

# of correct disambiguationstotal # of words in test set

For practical purposes, the best performingsystem is the last row in Table 5, i.e., in whichModel 4 uses Model 3 for pre-annotating thetest set (though the 7th row has the highestaccuracy, we cannot assume treebank anno-tations on the test data as well). Table 5also presents the results of using the exist-ing state of the art (S-O-T-A) model on ourHindi dataset. We have used the best perform-ing model (on our Hindi dataset) proposed byShen et al. (2016), the full-context model, asthe state of the art. The accuracy gain wehave achieved over state of the art is 2.8% onambiguous words and 3.43% on all words.

We suggest possible reasons for the observedperformance behavior in table 5. Typologi-cally, Hindi is a Subject-Object-Verb, head-final language and uses post-positional casemarking. This means that on an average,words show disambiguation dependencies onthe words following them. However, there isalso disambiguation evidence for a word to begained from its left context. For instance, ad-verbs usually occur before (to the left of) theverb or object they refer to. Similarly, relativeclauses, adjectives and articles are written be-fore the noun they refer to. Model 4 uses the

morphological analyses of the right-context ofa word as well as the left context and henceis able to leverage information from both pre-ceding and following words. Hence it is ableto achieve better performance than Model 3.Models 2 and 1 do not leverage evidence aboutthe morphological analysis of the words in thewindow and perform worse than the other twomodels. This shows (as is also quite intuitive)that the morphological analysis of the contextis far stronger evidence in disambiguating aword, than just the surface forms of the wordsand its context. Model 2 performs better thanModel 1 as it has access to the surface forms ofthe surrounding words, which in turn providesome level of knowledge about their inflectedproperties.

From control experiments, we conclude thatour gain over the state of the art is due tofactors that include careful tuning of hyper-parameters, increasing model complexity andleveraging the strength of combining models.At the end of section 3.10, we have alreadydescribed the advantage of using a CNN. Theexisting state of the art does not leverage thisadvantage. Further, in allowing Model 3 topre-annotate the test data, we have allowedour full-context model to take advantage of thestrengths of a left-to-right model, which is alsosomething that the existing state of the artdoes not explore.

5.1 Language-specific EnhancementsWhile the reported results in Table 5 areobtained without using pre-training of wordvectors or phonological features, we also ex-perimented with using these enhancements.We present results on the experimental setupwhere we train using Model 4 and pre-annotate the test set using Model 3. All per-formance improvements are reported as thoseobtained over and above the performance ofthis particular experiment setup.

5.1.1 Pre-training of Word VectorsWe pre-trained word embeddings using theword vector representation methods proposedby Bojanowski et al. (2016). This methodmakes use of an unsupervised skip-gram modelto generate word vectors of dimension 100.We used an augmented corpus comprising ofWikipedia text dump for Hindi, as well as

8

492

Model Accuracy Accuracygain over

Baseline (%)Baseline 38.43 0Model 4 95.22 147.74Model 4 +

Pre-training 96.64 151.47Model 4 +

Phonological Features 96.04 149.91Model 4 +

Pre-training +Phonological Features 97.02 152.46

Table 6: Performance with language-specificenhancements

the collection of news articles and conversa-tions that the Hindi treebank annotated wordscome from. Using vector pre-training gave usan accuracy improvement of 1.42%. One ofthe main reasons for the performance boostobtained during pre-training could perhapsbe that the pre-trained word vectors capturesyntactic and morphological information fromshort neighbouring windows.

5.1.2 Use of Phonological FeaturesMorphology interacts closely with phonologyand there is ample work on the phonology-morphology interface (Booij, 2007). It is quiteintuitive, therefore, to use phonological fea-tures (Chomsky and Halle, 1968) for a mor-phological problem. Besides, Hindi is writtenin the Devanagari script, in which the map-ping from letters to phonemes is almost one toone. Each letter can therefore be representedas a set of feature-value pairs, where the fea-tures are phonological features such as type(whether consonant or vowel), place, manneretc. (Singh, 2006). This is true for almostall languages that use Brahmi-derived scripts.Phonological features are incorporated intothe model by concatenating them with thecharacter-level embeddings for words. We ob-serve a performance enhancement of 0.82%upon using these phonological features.

Employing pre-training as well as phonolog-ical features boosted our model’s performancefrom 95.22 % to 97.02%. These enhanced re-sults are summarized in Table 6.

6 Future WorkWe plan to test all our models on different lan-guages and analyze which models perform beston each language and hope to be able to cor-

relate these results with the linguistic phono-morphological properties of the languages. Wewill also try out this model in the Sampark10

machine translation system to evaluate the ef-fect it has on translation.

Recently, an attention-based machine trans-lation model was proposed by Bahdanau etal. (2014) that defines a selective contextaround a word rather than a fixed window forall words. Models 3 and 4 can be modifiedto use an attentional mechanism based on thecontext words’ positional and morphologicalproperties. This would allow these models toincrease their range of information-capturingacross words in the sentence, without losinginformation due to propagation in a recurrentunit running across a large window. Experi-ments have been done in the past for morpho-logical disambiguation using Conditional Ran-dom Fields (CRFs). It might be interesting tosee the hybrid use of CRF models with themodels we propose.

7 Conclusion

We propose multiple deep learning models formorphological disambiguation. We show thatthe model that makes use of morphologicalinformation in both the left and right con-text of a word performs best on this task, atleast in the case of Hindi. We also study theeffect of different context settings on modelperformance. The differences in performanceobtained using these different context set-tings, we believe, follows from the typologicaland morphological properties of the language.Hence, we also believe that different languagesmay work better with different models thatwe propose. The use of phonological featuresenhances the quality of predictions by thesemodels, at least in the case of Hindi.

ReferencesDzmitry Bahdanau, Kyunghyun Cho, and Yoshua

Bengio. 2014. Neural machine translation byjointly learning to align and translate. arXivpreprint arXiv:1409.0473.

Piotr Bojanowski, Edouard Grave, Armand Joulin,and Tomas Mikolov. 2016. Enriching word vec-

10https://sampark.iiit.ac.in/sampark/web/index.php

9

493

tors with subword information. arXiv preprintarXiv:1607.04606.

Geert Booij. 2007. The interface between morphol-ogy and phonology. Oxford University Press.

Eric Brill. 1992. A simple rule-based part of speechtagger. In Proceedings of the Third Conferenceon Applied Natural Language Processing, ANLC’92, pages 152–155, Stroudsburg, PA, USA. As-sociation for Computational Linguistics.

Noam Chomsky and Morris Halle. 1968. TheSound Pattern of English. Harper & Row, NewYork.

Ronan Collobert, Jason Weston, Léon Bottou,Michael Karlen, Koray Kavukcuoglu, and PavelKuksa. 2011. Natural language processing (al-most) from scratch. Journal of Machine Learn-ing Research, 12(Aug):2493–2537.

Manaal Faruqui, Yulia Tsvetkov, Graham Neubig,and Chris Dyer. 2016. Morphological inflectiongeneration using character sequence to sequencelearning. In Proc. of NAACL.

Onur Görgün and Olcay Taner Yildiz. 2011. Anovel approach to morphological disambiguationfor turkish. In Computer and Information Sci-ences II, pages 77–83. Springer.

Vishal Goyal and Gurpreet Singh Lehal. 2008.Hindi morphological analyzer and generator. InEmerging Trends in Engineering and Technol-ogy, 2008. ICETET’08. First International Con-ference on, pages 1156–1159. IEEE.

Dilek Z Hakkani-Tür, Kemal Oflazer, and GökhanTür. 2000. Statistical morphological disam-biguation for agglutinative languages. In Pro-ceedings of the 18th conference on Computa-tional linguistics-Volume 1, pages 285–291. As-sociation for Computational Linguistics.

Georg Heigold, Guenter Neumann, and Josef vanGenabith. 2016. Neural morphological taggingfrom characters for morphologically rich lan-guages. arXiv preprint arXiv:1606.06640.

Fred Karlsson, Atro Voutilainen, Juha Heikkila,and Arto Anttila, editors. 1995. ConstraintGrammar: A Language-Independent System forParsing Unrestricted Text. Walter de Gruyter& Co., Hawthorne, NJ, USA.

Gulshat Kessikbayeva and Ilyas Cicekli. 2016. Arule based morphological analyzer and a mor-phological disambiguator for kazakh language.Linguistics and Literature Studies, 4(1):96–104.

Diederik P. Kingma and Jimmy Ba. 2014. Adam:A method for stochastic optimization. In Pro-ceedings of the 3rd International Conference onLearning Representations (ICLR).

György Orosz and Attila Novák. 2013. Purepos2.0: a hybrid tool for morphological disambigua-tion. In RANLP, volume 13, pages 539–545.

Barbara Plank, Anders Søgaard, and Yoav Gold-berg. 2016. Multilingual part-of-speech tag-ging with bidirectional long short-term mem-ory models and auxiliary loss. arXiv preprintarXiv:1604.05529.

Haşim Sak, Tunga Güngör, and Murat Saraçlar.2007. Morphological disambiguation of turk-ish text with perceptron algorithm. Computa-tional Linguistics and Intelligent Text Process-ing, pages 107–118.

Qinlan Shen, Daniel Clothiaux, Emily Tagtow,Patrick Littell, and Chris Dyer. 2016. The roleof context in neural morphological disambigua-tion. In COLING, pages 181–191.

Pawan Deep Singh, Archana Kore, RekhaSugandhi, Gaurav Arya, and Sneha Jadhav.2013. Hindi morphological analysis and inflec-tion generator for english to hindi translation.International Journal of Engineering and Inno-vative Technology (IJEIT), pages 256–259.

Anil Kumar Singh. 2006. A computational pho-netic model for indian language scripts. In Con-straints on Spelling Changes: Fifth Interna-tional Workshop on Writing Systems. Nijmegen,Nijmegen, The Netherlands.

Milan Straka and Jana Straková. 2017. Tokeniz-ing, pos tagging, lemmatizing and parsing ud 2.0with udpipe. Proceedings of the CoNLL 2017Shared Task: Multilingual Parsing from RawText to Universal Dependencies, pages 88–99.

Eray Yildiz, Caglar Tirkaz, H. Bahadir Sahin,Mustafa Tolga Eren, and Ozan Sonmez. 2016.A morphology-aware network for morphologicaldisambiguation. In Proceedings of the Thirti-eth AAAI Conference on Artificial Intelligence,AAAI’16, pages 2863–2869. AAAI Press.

10

494