SALA October 2001

Design of an Electronic Sanskrit Reader

SALA XXI,SALA XXI,

Konstanz, October 2001Konstanz, October 2001

GérardGérard Huet Huet

INRIAINRIA

SALA October 2001

History

• 1994 - personal lexicon in TeX

• 1996 - available on Internet

• 1998 - 10000 entries - invariants design

• 1999 - reverse engineering

• 2000 - Hypertext version on the Web, sandhi processor, grammatical engine

• 2001 - Segmenter, tagger

SALA October 2001

SALA October 2001

SALA October 2001

SALA October 2001

Processing chains

Data base source Abstract structure

devnag tex dvi ps pdf

Internet

Index a h...

Web site (html)

CGI bin (Ocaml)

SALA October 2001

Close-up view of functionalities

Index Engine

Grammar Engine

Dico.ps Dico.pdf

Web site (html)

Index

a h...

Flex

DicoDB Grind FlexedEntries

CGI

SALA October 2001

Tools used

Printable document

• Knuth TeX, Metafont, LaTeX2e

• Velthuis devnag font & ligature comp

• Adobe Postscript, Pdf, Acrobat

Hypertext document

• W3C HTTP, HTML, CSS

• Unicode UTF-8

• Chris Fynn Indic Times Font

Processing & Search

• INRIA Objective Caml

SALA October 2001

Each entry is a (typed) tree

Entry

syntax usage opt cogs

Type entry = [ | Crossref ]

....

N.B. Syntax is really morphology, usage is part of speech roles plus meanings

SALA October 2001

Grammatical information

type gender = [ Mas | Neu | Fem | Any ];

type number = [ Singular | Dual | Plural ];

type case = [ Nom | Acc | Ins | Dat | Abl | Gen | Loc | Voc ];

SALA October 2001

The verb system

type voice = [ Active | Reflexive ]

and mode = [ Indicative | Imperative | Causative | Intensive | Desiderative ]

and tense = [ Present of mode | Perfect | Imperfect | Aorist | Future ]

and nominal = [ Pp | Ppr of voice | Ppft | Ger | Infi | Peri ]

and verbal = [ Conjug of (tense * voice)

| Passive | Absolutive | Conditional | Precative

| Optative of voice

| Nominal of nominal

| Derived of (verbal * verbal) ];

SALA October 2001

Governance templates(Grammatical valence)

\word{ga.n} ... \sem{imputer qqc. <acc.> à qqn. <loc.>}

\ca{chandayati} … \sem{gratifier qqn. <acc.> de <i.>}

\word{niyuj} ... \sem{confier qqc. <acc.> à qqn. <loc.>}

\root{krii} ... \sem{acheter (qqc. <acc.> à qqn. <g. abl.>)}

Other specific notations for synonyms, antonyms, cross-references.

SALA October 2001

Key points

• Each entry is a structured piece of data on which one may compute

• Consistency and completeness checks : • every reference is well defined once, there is no dangling reference

• etymological origins, when known, are systematically listed

• lexicographic ordering at every level is mechanically enforced

• Specialised views are easily extracted

• Search engines are easily programmable

• Maintenance and transfer to new technologies is ensured

• Independence from input format, diacritics conventions, etc.

• The technology is scalable to much bigger corpus

SALA October 2001

Generic reuse of the technology

The structure of the dictionary makes separate as much as possible 3 layers :

• sanskrit

• french

• generic dictionary structure

Thus the french meanings, at the leaves, could be replaced by e.g. english definitions or glosses.

SALA October 2001

Morphological analysis, sandhi

• Sanskrit is pronounced as written

• … and thus is written as pronounced

• Phonetic alliteration is rendered by morphology junction (sandhi)

• The sentence is formed of words joined by external sandhi

• Compound words are also formed by external sandhi

• Whereas flexion, prefixing and suffixing use internal sandhi

• External sandhi is local, internal sandhi is less

• Sandhi analysis is non-deterministic and sometimes involves sem

SALA October 2001

Grammatical engine

• In sanskrit, declension is determined by stem and gender

• Sanskrit is very regular, since the classical language was frozen by Pânini (4th century BC) who invented context-free notation

• But it spans about 35 centuries, and thus there are many exceptions

• Substantive (adjectives, pronouns, numerals) declension may be arranged in 84 tables of 24 endings (3 numbers * 8 cases)

• Then internal sandhi is applied to a stem and an ending

• Two applications :• online declension of words given with gender (cgi-bin)

• offline computation of flexed forms (2000 pages of double-column fineprint)

SALA October 2001

Interactions lexicon-grammar

• The index engine, when given a string which is not a stem defined in one of the entries of the lexicon, attempts to find it within the flexed forms persistent database, and if found there will propose the corresponding lexicon entry or entries

• From within the lexicon, the grammatical engine may be called online as a cgi which lists the declensions of a given stem. It is directly accessible from the gender declarations, because of an important scoping invariant:

• every substantive stem is within the scope of one or more genders

• every gender declaration is within the scope of a unique substantive stem

SALA October 2001

Inverting external sandhi

• External sandhi rules are of a finite-state nature

• The flexed forms lexicon index may be seen as the graph of a deterministic finite automaton recognizing its words

• This tree may be uniformly decorated by relevant sandhi rules seen as non-deterministic choice points

• This structure may be evaluated as a finite-state transducer graph segmenting an input text as words joined by sandhi

SALA October 2001

Examples of segmentationChunk: o.mnama.h"sivaaya

may be segmented as:

• [ om with sandhi m|n -> .mn ]

• [ namas with sandhi s|"s -> .h"s ]

• [ "sivaaya with no sandhi ]

Chunk: kusuma.mgopiibhya.hk.r.s.nodadati

may be segmented as:

• [ kusumam with sandhi m|g -> .mg]

• [ gopiibhyas with sandhi s|k -> .hk]

• [ k.r.s.nas with sandhi as|d -> od]

• [ dadati with no sandhi]

SALA October 2001

From segments to tagged lemmas

Chunk: kusuma.mgopiibhya.hk.r.s.nodadati

may be lemmatized with tags as:

• [ kusumam < acc. sg. n. of kusuma

| nom. sg. n. of kusuma

| voc. sg. n. of kusuma > with sandhi m|g -> .mg ]

• [ gopiibhyas < abl. pl. f. of gopa

| dat. pl. f. of gopa > with sandhi s|k -> .hk ]

• [ k.r.s.nas < nom. sg. m. of k.r.s.na > with sandhi as|d -> od ]

• [ dadati <…> with no sandhi]

SALA October 2001

Future work• Verb conjugation tables preparation - full flexed forms database

• Fixing sandhi analysis for bahuvrihi compounds

• Choice of taggings from concord and valency constraints

• Semantic guidance from ontology classification

• and we shall then able to semi-automatically index corpuses towards• computer-aided concordance of corpus

• computer-aided preparation of critical editions

• statistical analysis of corpus (co-occurrence, style, etc)

• computer-aided accretion of lexicon

• fully indexed citations

• extraction of corpus-specific lexicons

• diachnony control of lexical information