Natural Language Processing for Amazigh Language

Natural Language Processing for Amazigh Language:

Challenges and Future Directions

Fadoua Ataa Allah Siham BoulaknadelCEISIC, IRCAM

{ataaallah, boulaknadel}@ircam.ma

LREC-2012: SALTMIL-AfLaT Workshop 2

Outline

Amazigh Language

Amazigh Complexity in NLP

State of the Technology on Amazigh

Future Directions


North African autochthonous language

Spoken by millions of people as dialects

Sociolinguistic ContextAmazigh language

LREC-2012: SALTMIL-AfLaT Workshop 410/07/2012

Amazigh language

Languages of Morocco Classical Arabic as an official language.

Amazigh, since 2011 it becomes an officiallanguage.

Moroccan Arabic or Darija is the diglossia ofClassical Arabic.

French as the first foreign language.

Spanish is used in the north of Morocco.

English is becoming the second foreign language.

Sociolinguistic Context


Tinzouline Inscriptions (Zagora, Morocco)

Amazigh language

Amazigh abjed Tifinagh is attested from

25 centuries.

Its writing form hascontinued to changefrom the traditionalTuareg writing to theTifinaghe-IRCAM .

History


Plate 9

Anou Elias, Mammanet Valley (Niger).

Henri Lhote, Oued Mammanet gravures.

Les Nouvelles Editions Africaines. 1979

Direction

Amazigh languageHistory


Amazigh writing system Direction: horizontal from left to right. Alphabet:

27 consonants: ⴱ, ⴳ, ⴳⵯ, ⴷ, ⴹ, ⴼ, ⴽ, ⴽⵯ, ⵀ, ⵃ, ⵄ, ⵅ, ⵇ, ⵊ, ⵍ, ⵎ, ⵏ, ⵔ, ⵕ ,ⵖ, ⵙ, ⵚ, ⵛ, ⵜ, ⵟ, ⵣ, ⵥ;

2 semi-consonants: ⵢ and ⵡ; 4 vowels: ⴰ, ⵉ, ⵓ, ⴻ.

Punctuation marks: conventional signs including: “ ”(space), “.”, “,”, “;”, “:”, “?”, “!”, “…” , etc.

Numerals: Hindu-Arabic numerals [0-9].

Moroccan Amazigh characteristics


Different writing forms Complex phonology and phonetic

systems Rich morphology

Amazigh Complexity in NLP


Writing prescriptions’ conversion into‘Tifinaghe – Unicode’ is confronted with: Spelling variation related to regional

varieties ([tfucht] [tafukt] (sun)), Spelling variation based on the use or the

elimination of spaces within or betweenwords ([tadartino] [tadart ino] (my house)).

Arabic or Latin transcription systems.

Amazigh Complexity in NLPAmazigh script


The main problem of Amazigh phonologyand phonetic consists on allophones:

/ll/ that is realized as [dj] in the North.

Amazigh Complexity in NLPPhonology & phonetic


Amazigh Complexity in NLPMorphology

High inflected language.

Word structure:

Affixes set: Prefixes, Infixes, and Suffixes. Base form varies with paradigms:

(qqim svim (make sit)).

Prefix Stem Suffix


State of the Amazigh technology

Tifinaghe Encoding

Optical character recognition

Fundamental processing tools

Language resources

ANSI Unicode

State of the Amazigh technology Tifinaghe Encoding

13



Amazigh OCR systems: System focused on isolated printed characters

based on a syntactic approach using finiteautomata.

Global approach based on Hidden MarkovModels for recognizing handwritten characters.

Method using invariant moments for recognizingprinted script.

System based on artificial neural network torecognize printed characters.

OCR



Transliterator

Tagging assistance tool

Light stemmer

Search engine

Concordancer

Fundamental processing



Transliterator


Tifinaghe Unicode

Arabic script

Latin script

Tifinaghe Latin

Convertisor

Transliterator



Tagging assistance tool


Amazigh raw

corpora

Tag set

Tokenization

Manual Stemming Manual POS

Tagged corpus

Stem list

Validation

Standard output



Light stemmer


Find the largest suffix

Find the largest prefix

Prefix + Stem + Suffix

Begin

Stem + Suffix

Stem

End



Search engineFundamental processing

User Interface

Query EngineNatural Language Processing Tools

Data Searching Indexer

Natural Language Processing Tools

Index

Data Indexing

Crawler Repository

Data Crawling

Web



Concordancer


input field

.txt,.doc

.pdf, .zip

Tokenization

Word / expression

Context display

List of the text words

and their frequency



Corpora

Dictionary

Terminology database

Language resources



Corpora:

General corpus,

POS corpus.

Language resources



Dictionary Definition, Arabic equivalent words, French equivalent words, English equivalent words, Synonyms, Classification by domains, Derivational families.

Language resources



Terminology database

Media vocabulary

Grammatical vocabulary

Language resources


Future Directions

Building a large and representativeAmazigh corpora.

Developing a machine translationsystem.

Creating a pool of competent humanresources.


Thank you for

your attention

ⵜⴰⵏⵎⵉⵔⵜ