Embed Size (px)
Citation preview
Proc. of the 3rd Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature, pp. 122–132Minneapolis, MN, USA, June 7, 2019. c©2019 Association for Computational Linguistics
122
Sign Clustering and Topic Extraction in Proto-Elamite∗
Logan Born1
Kate Kelley2
Nishant Kambhatla1
Carolyn Chen1
Anoop [email protected]
1Simon Fraser UniversitySchool of Computing Science
2University of British ColumbiaDepartment of Classical, Near Eastern,
and Religious Studies
Abstract
We describe a first attempt at using techniquesfrom computational linguistics to analyze theundeciphered proto-Elamite script. Using hi-erarchical clustering, n-gram frequencies, andLDA topic models, we both replicate resultsobtained by manual decipherment and revealpreviously-unobserved relationships betweensigns. This demonstrates the utility of thesetechniques as an aid to manual decipherment.
1 Introduction
In the late 19th century, excavations at the an-cient city of Susa in southwestern Iran began touncover clay tablets written in an unknown scriptlater dubbed ‘proto-Elamite’. Over 1,500 tabletshave since been found at Susa, and a few hun-dred more at sites across Iran, making it the mostwidespread writing system of the late 4th and early3rd millennia BC (circa 3100–2900 BC) and thelargest corpus of ancient material in an undeci-phered script.1
Proto-Elamite (PE) is the conventional designa-tion of this script, whose language remains un-known but was presumed by early researchers aslikely to be an early form of Elamite. A number offeatures of the PE writing system are understood.These include tablet format and direction of writ-ing, the numeric systems, and the ideographic as-sociations of some non-numeric signs, predomi-nantly those for livestock accounting, agriculturalproduction, and possibly labor administration. Yet
∗We would like to thank Jacob Dahl and the anony-mous reviewers for their helpful remarks. This researchwas partially supported by the Natural Sciences and Engi-neering Research Council of Canada grants NSERC RGPIN-2018-06437 and RGPAS-2018-522574 and a Department ofNational Defence (DND) and NSERC grant DGDND-2018-00025 to the last author.
1New PE texts have been found as recently as 2006–2007,when excavations at Tepe Sofalin near Tehran uncovered tentablets (Dahl et al., 2012).
the significance of the majority of PE signs, thenature of those signs (syllabic, logographic, ideo-graphic, or other) and the linguistic context(s) ofthe texts remain unknown. It was recognized fromthe outset, due to the features of the script, thatall the proto-Elamite tablets were administrativerecords, rather than historical or literary composi-tions (Scheil, 1905).
Texts are written in lines from right to left,but are rotated in publication to be read from topto bottom (then left to right) following academicpractice for publishing the contemporary proto-cuneiform tablets. The content of a text is dividedinto entries, logical units which may span morethan one physical line. The entry itself is a stringof non-numeric signs whose meanings are for themost part undeciphered. Each entry is followed bya numeric notation in one of several different nu-meric systems, which quantifies something in re-lation to the preceding entry. This serves to markthe division between entries. An important ex-ception exists in what are currently understood tobe ‘header’ entries: these can present informationthat appears to pertain to the text as a whole, andare followed directly by the text’s first content en-try with no intervening numeric notation. A digitalimage and line drawing of a simple PE text alongwith transliteration are shown in Figure 1.
Although a complete digital corpus of PE textsexists (Section 2), it has not been studied usingthe standard toolkit of data exploration techniquesfrom computational linguistics. The goals of thispaper are threefold. By applying a variety of com-putational tools, we hope to
i. promote interest in and awareness of the prob-lems surrounding PE decipherment
ii. demonstrate the effectiveness of computa-tional approaches by reproducing results pre-viously obtained by manual decipherment
123
CDLI transliteration convention:obverse1. |M218+M218| ,header 2. M056~f M288 , 1(N14) 3(N01)3. |M054+M384~i+M054~i| M365 , 5(N01)4. M111~e , 4(N14) 1(N01) 3(N39B)5. M365 , 1(N14) 3(N01)6. M075~g , 1(N14) 3(N01)7. M387~l M348 , 1(N14) 3(N01)
reverse
column 11. M056~f M288 , 1(N45) 1(N14)# 3(N39B)
Direction of writingline: 1 2 3
reverse line: 1
numerical notation
obverse
Figure 1: PE tablet Mémoires de la Délégation en Perse (MDP) 6, no. 217 (P008016; Scheil 1905). Digitalimage, line art, and transcription (called transliteration by the CDLI) from the Cuneiform Digital Library Initiative.Explanatory annotation added by current authors.
iii. highlight novel patterns in the data which mayinform future decipherment attempts
We hope to show that interesting data may be ex-tracted from the corpus even in the absence of acomplete linguistic decipherment. To encouragefurther study in this vein, we are also releasingall data and code used in this work as part of anonline suite of data exploration tools for PE.2 Ad-ditional figures and interactive visualizations arealso available as part of this toolkit.
2 Conventional Decipherment Efforts
Studies towards the decipherment of PE can besummarised by a relatively short bibliographyof serious efforts (Englund, 1996).3 Stumblingblocks to decipherment have included inaccura-cies in published hand-copies of the texts, a lackof access to high-quality original images, andthe associated difficulty in drawing up an accu-rate signlist and producing a consistently-renderedfull transcription of the corpus. Members ofthe Cuneiform Digital Library Initiative (CDLI)have been remedying these deficiencies over the
2https://github.com/sfu-natlang/pe-decipher-toolkit
3For the most complete and up-to-date bibliography, seeDahl 2019.
past two decades, and the PE script can nowboast (i) a working signlist with a consistent man-ner of transcribing signs in ASCII, and (ii) anopen-access, searchable database hosting the en-tire corpus in transcription, alongside digital im-ages and/or hand-copies of almost every text.4
Historically, specialists of PE have operated ona working hypothesis that it may be, like laterSumerian cuneiform, to some extent a mixed sys-tem of ideographic or logographic signs alongsidesigns that may represent syllables. However, thelevel of linguistic content represented in both PEand proto-cuneiform has been called into question(Damerow, 2006), and the presence of a set of syl-labic signs in PE is yet to be proven.
The strict linear organisation of signs in PEis the earliest such known to a writing system:proto-cuneiform arranged signs in various wayswithin cases (and sometimes subcases), and onlyin cuneiform from several hundred years later didscribes begin to consistently write in lines with onesign following the next. However, it is not clear towhat extent the linear sign organization of PE re-flects the flow of spoken language as in later writ-ing systems.5
4http://cdli.ox.ac.uk/wiki/doku.php?id=proto-elamite
5Dahl (2019:83): “proto-Elamite texts are organized in an
124
Analysis of sign and entry ordering in the textshas also revealed some tabular-like organisingprinciples familiar from proto-cuneiform. Longersequences of signs can often be broken down intoconstituent parts appearing to follow hierarchicalordering patterns apparently based upon adminis-trative (rather than phonetic/linguistic) principles,and hierarchies can be seen across entries as well(Hawkins, 2015; Dahl et al., 2018).
Traditional linguistic decipherment efforts havenot yet succeeded in identifying a linguistic con-text for PE, though progress has been made, forexample in positing sets of syllabo-logographicsigns thought to be used to write personal names(PNs). We refer to Meriggi’s (1971:173–174) syl-labary as shorthand for these signs, as he wasthe first to identify such a set and his work hassince been closely imitated (Desset 2016; Dahl2019:85). Although he called it a syllabary,Meriggi was aware that the signs might not proveto be syllabic and that object or other signs mightremain mixed in.
Continued efforts to establish the organizationalprinciples of the PE script and to isolate possiblesyllable sequences or PNs may be advanced bycomputational techniques, which can be used toevaluate hypotheses much faster than purely man-ual approaches. In this endeavour it is necessaryto remember that although early writing encodesmeaningful information, that information may ormay not be linguistic (Damerow, 2006). Althoughit is not known why PE disappeared after a rel-atively short period of use, one of several possi-bilities is that this relates to the way it representsinformation, perhaps providing a poorer, less ver-satile encoding compared to later cuneiform withits mixed syllabo-logography.
3 Data
All data in this work are based on the PE cor-pus provided by the CDLI. After removing tabletswhich only bear unreadable or numeric signs, thisdataset comprises 1399 distinct texts. Most ofthese are very short: the mean text length is 27readable signs, of which only 10 are non-numericon average. Long texts do exist, however, up to amaximum length of 724 readable signs of which198 are non-numeric.
Our working signlist (extracted from the tran-
in-line structure that is more prone to language coding thanproto-cuneiform...”
scribed texts) contains 49 numeric signs and 1623non-numeric signs. Of these, 287 are ‘basic’signs, and 1087 are labeled as variants due to mi-nor graphical differences. Sign variants are de-noted by ∼, as in M006∼b, a variant of the ba-sic sign M006. In an on-going process, analy-sis of the corpus aims to confirm whether signvariants are semantically distinct, or reflect purelygraphical variation. Where the latter case is un-derstood, the sign is given a numeric rather thanalphabetic subscript, as in M269∼1. The remain-ing 249 non-numeric signs are compounds calledcomplex graphemes which are made up of two ormore signs in combination, as in |M136+M365|.
Future work is required to establish which signvariants are meaningfully distinct from their basesigns; in the absence of such work, we have chosento treat all variants as distinct until proven other-wise. Our models give interpretable results underthis assumption, suggesting this is a reasonable ap-proach. There are, however, cases where collaps-ing sign variants together would seem to affect ourresults, and we highlight these where relevant.
4 Analysis of Signs
4.1 Hierarchical Sign Clustering
Manual decipherment of PE has proceeded in partby identifying that some signs occur in largely thesame contexts as other signs. This has producedgroupings of signs into “owners”, “objects”, andother functionally related sets (Dahl, 2009). Forexample, M388 and M124 are known to be par-allel “overseer” signs which appear in alternationwith one another (Dahl et al., 2018:25).
In the same vein, we have investigated tech-niques for clustering signs hierarchically basedon the way they occur and co-occur within texts.Our work considers three approaches to sign clus-tering: a neighbor-based clustering groups signsbased on the number of times each other sign oc-curs immediately before or after that sign in thecorpus; an HMM clustering groups signs basedon the emission probabilities of a 10 state hiddenMarkov model (HMM) trained on the corpus; anda generalized Brown clustering groups signs as de-scribed in Derczynski and Chester 2016. By us-ing three different clustering techniques, we cansearch for clusters which recur across all threemethods to maximize the likelihood of findingthose that are meaningful. This reduces the im-pact of noise in the data, which is especially useful
125
given the small size of the PE corpus.
4.1.1 Clustering EvaluationWe identified commonalities between our threeclusterings using the following heuristic. Givena set of signs S, we found for each clusteringthe height of the smallest subtree containing ev-ery sign in S. If all of these subtrees were short(which we took to mean not larger than 2|S|) thenwe called S a stable cluster.
In many cases, the stable clusters comprise vari-ants of the same sign. This is the case for M157and M157∼a, which cluster together across alltechniques and are already believed to functionsimilarly to each other, if not identically.
One very large stable cluster consists of thesigns M057, M066, M096, M218, and M371.This cluster is shown as it appears in eachclustering in Figure 2. These signs belong toMeriggi’s proposed syllabary (Meriggi 1971, esp.pp. 173–4) and are hypothesized to representnames syllabically (or logographic-syllabically;Desset 2016:83). Desset (2016:83) likewise iden-tified “approximately 200 different signs” frompossible anthroponyms, “among which M4, M9,M66, M96, M218 and M371 must be noticed fortheir high frequency.” Desset’s list differs from ourcluster by only two signs, replacing M057 withM004 and M009. M004 and M009 group withother members of the putative syllabary in eachclustering, but their position is much more variableacross the three techniques. For M009 at least, thismay indicate multivalent use: besides its inclusionin hypothesised PNs (e.g. Meriggi 1971:173; Dahl2019:85), it appears in various different adminis-trative contexts that don’t appear to include PNs(e.g P008206) and as an account postscript (seebelow here and 5.3).
All three methods group the five signs in ourcluster close to other suspected syllabic signs;however, since each technique groups them witha different subset of the syllabary, only these fiveform a stable group across all three methods. Thismay be due simply to their frequency, or theycould in fact form a distinct subgroup within theproposed syllabary; future work may yield a betterunderstanding of possible anthroponyms by tryingto identify other such subgroups.
While this discussion has focused on the sta-ble clusters for which we can provide some in-terpretation, others represent groups of signs withno previously recognised relationship, such as
(a) (b) (c)
Figure 2: Detail of the (a) neighbor-based, (b) HMM,and (c) Brown clusterings showing signs possibly usedin anthroponyms. M057, M066, M096, M218, andM371 are considered a stable cluster due to their prox-imity in all three clusterings.
M003∼b and M263∼a (Figure 3). M003(∼a/b)are “stick” signs ( , ) understood in some PEcontexts to denote worker categories (Dahl et al.,2018); they are graphically comparable to proto-cuneiform PAP∼a-c ( ) and PA ( ), the lat-ter of which can, in later Sumerian, indicate ugula,a work group foreman/administrator.
Figure 3: M003∼b clusters identically with M263∼ain all three techniques.
M263∼a is one of a series of depictions of “ves-sels” ( ), this particular variant appearing in 27texts; notably the base sign M263 appears as apossible element in PNs (Dahl, 2019:85). Inter-estingly, M003∼b and M263∼a only appear to-gether in a single text (P008727), one of a closely-related group of short texts6 that each end in theadministrative postscript M009 M003∼b or M009M003∼c. It can also be noted that M263∼1 oc-curs in another text belonging to this small group.
It thus remains for future work to interpret thisand the many other stable clusters resulting fromour work. These additional groupings are detailedin our data exploration toolkit, along with com-plete dendrograms for each clustering which aretoo large to include in this publication.
Although we have not performed a full studyof the clusterings produced when sign variantsare collapsed together, a preliminary comparison
6 Available online at https://cdli.ucla.edu/search/search_results.php?SearchMode=Text&requestFrom=Search&TextSearch=M009+M003
126
suggests this is worth pursuing. For instance,a new cluster of small livestock signs arises inthe neighbor-based clustering, comprising M367(“billy-goat”), M346 (“sheep”), M006 (“ram”),and M309 (possible animal byproduct). Existingclusters, such as the stable cluster of syllabic signs,appear to remain intact, but a complete compari-son of the techniques in this setting is warranted.
4.2 Sign Frequency and n-Gram Counts
Sign frequency is another useful datapoint for un-derstanding the overall content of the corpus andfor building a more nuanced understanding of signuse (Dahl, 2002; Kelley, 2018). Figure 4 showsthe most common PE unigrams, bigrams, and tri-grams. These counts exclude n-grams contain-ing numeric signs or broken or unreadable signs(transcribed as X or [...]); n-grams which span theboundary between entries are also excluded. Notethe sharp drop-off in frequency from the most fre-quent signs to the rest of the signary; in fact nearlyhalf the attested signs (745 out of 1623) occur onlyonce. Similar results were presented in Dahl 2002.
The most common unigrams include “object”signs and signs belonging to Meriggi’s syllabary.The object signs are M288 (a grain container),M388 (“person/man”), M124 (a person/workercategory paralleling M388), M054 (a yoke, usu-ally indicating a person/worker category or ani-mal), M297 (“bread”), and M346 (“ewe”). Thesyllabary signs are M218, M371 (which may dou-ble as an object sign/worker category), M387 (alsoa numeral meaning “100”), and M066.
The n-gram counts reveal the scale at whichcomplex sequences of information are repeatedacross tablets. Over 1600 strings contain at least3 non-numeric signs. Of these, only 11 trigramsare repeated at least 5 times across the corpus; twoof these end in the “grain container” sign M288and are therefore best parsed as undeciphered bi-grams followed by an object sign. Followingthis, 52 other trigrams are repeated three or fourtimes across the corpus, leaving the great majority(98%) of trigrams to appear only once or twice.7
The most frequent trigram, M377∼e M347 M371(found 17 times per Figure 4), appears in no morethan about 1.5% of the texts. Even among bi-
7This assumes that sign variants are meaningfully distinct,as is the working hypothesis among PE specialists. Collaps-ing variants together does not appreciably change these re-sults, however, as it only increases most trigram counts by 1or 2 instances. A similar result holds for bigram counts.
0 100 200 300 400 500 600 700 800Sign Frequency
M066M387M346M054M297M124M371M218M388M288
243249249258265
294308
525620
829
0 10 20 30 40 50Bigram Frequency (with constituent unigram counts)
M371 M288M259 M218
M377~e M347M096 M288M009 M371M347 M371M388 M218M218 M288M305 M388M004 M218
22 (308, 829)22 (69, 525)
23 (56, 84)26 (212, 829)
30 (223, 308)31 (84, 308)
32 (620, 525)36 (525, 829)
40 (127, 620)45 (115, 525)
0 5 10 15 20Trigram Frequency (with constituent bigram counts)
M347 M219 M101M219 M218 M288M371 M009 M371M259 M218 M288
|M131+M388| M101 M066M386~a M240 M096
M004 M263 M218M340 M054 M388
M097~h M004 M218M377~e M347 M371
5 (6, 8)5 (16, 36)5 (6, 30)5 (22, 36)5 (5, 8)
6 (12, 16)7 (9, 18)7 (14, 17)
11 (13, 45)17 (23, 31)
Figure 4: The 10 most frequent PE unigrams, bigrams,and trigrams (top to bottom). In parentheses are giventhe frequencies of the two unigrams comprising eachbigram, and the two bigrams comprising each trigram:note that some frequent n grams are comprised of rela-tively infrequent n− 1-grams.
grams, the most common can only occur in up to3.2% of texts.
External comparisons may help determinewhether this is a meaningful degree of repeti-tion, but such comparisons are not straightforward.Third millennium Sumerian or Akkadian account-ing tablets are reasonable corpora to compareagainst, but these are available only in transliter-ation (using sign readings) while PE is transcribed(using sign names). This distinction makes n-gram counts from the two corpora incomparablewithout further work to transform the data.
Despite this, an impressionistic assessment ofUr III Sumerian administrative texts suggeststhat they are highly repetitious: information ofwide importance to the administration (e.g. ba-sic nouns, phrases describing administrative func-tions, month names, ruler names, etc.) occurs fre-quently. If one expects a similar pattern in the PEadministrative record, our initial analysis suggeststhat trigrams (and perhaps bigrams) may not be asignificant tactic for encoding these types of infor-mation, although unigrams might.
An n-gram analysis can also be used to be-
127
gin exploring the frequency of suspected anthro-ponyms within the PE corpus. Dahl (2019:85)lists frequently-attested signs (10 instances ormore) with “proposed syllabic values” obtainedthrough traditional graphotactical analysis; Fig-ure 5 presents the frequency of the most com-mon bigrams and trigrams limited to this signset.This list fails to include what is thought to bethe most commonly attested PN, M377∼e M347M371 mentioned above, since the middle sign,M347, is uncommon. Nonetheless the strings inthis figure are more representative of possible PNs,since object signs which are understood to encodeseparate units of information have been weededout. Overall we see that a small handful of 3-signPNs are repeated at least 4 times across the corpus,but the majority appear 3 times or less. 2-sign PNsmight be more frequent,8 although some of thebigrams in the figure simply represent substringsfrom the trigrams. The ten most common bigramsall appear 13 or more times across the corpus, andthe most frequent alone appears 45 times (M004M218, including as part of a common trigram inFigure 5, accounting for 11 of its uses).
0 10 20 30 40 50Bigram Frequency (with constituent unigram counts)
M242~b M096M097~h M004
M240 M096M219 M218
M066 M352~oM263 M218M387 M218M259 M218M009 M371M004 M218
13 (25, 212)13 (46, 115)
16 (49, 212)16 (88, 525)
18 (243, 40)18 (174, 525)
21 (249, 525)22 (69, 525)
30 (223, 308)45 (115, 525)
0 2 4 6 8 10 12Trigram Frequency (with constituent bigram counts)
M032 M387 M218M262 M259 M218
|M131+M388| M004 M263M101 M066 M263
M066 M352~o M218M371 M009 M371
|M131+M388| M101 M066M386~a M240 M096
M004 M263 M218M097~h M004 M218
3 (3, 21)3 (4, 22)3 (3, 9)3 (8, 5)
4 (18, 4)5 (6, 30)5 (5, 8)
6 (12, 16)7 (9, 18)
11 (13, 45)
Figure 5: The 10 most frequent PE bigrams and tri-grams (top to bottom), limited to signs in Dahl’s (2019)syllabary. In parentheses are given the frequencies ofthe two unigrams comprising each bigram, and the twobigrams comprising each trigram.
Repeated n-grams, anthroponymic or other-wise, become increasingly rare for n > 3. No4-gram or 5-gram appears more than 3 times; no
8However, according to Desset’s (2016) traditional analy-sis of 515 hypothetical anthroponymic sequences,“250 (48.5%) were made of 3 signs, 118 (22.9 %) of 4 signs, 83 (16.1%) of 2 signs, 38 (7.3 %) of 5 signs, 15 (2.9 %) of 6 signs, 8(1.5 %) of 7 signs and 3 (0.5 %) of 8 signs.”
6-gram appears more than twice; and no 7-gramappears more than once. This low level of repe-tition indicates that common frequency-based lin-guistic decipherment methods may be ineffectiveon this corpus. We can, however, identify repeatedstrings which are similar to one another, if not ex-act copies, which may lead to insights about thefunction of certain PE signs and sign sequences.For example, the only two 6-grams which occurmultiple times in the corpus differ from one an-other by only a single sign:
M305 M388 M240 M097∼h M004 M218M305 M388 M146 M097∼h M004 M218
A further variant appears once in the corpus:
M305 M388 M347 M097∼h M004 M218
Traditional graphotactical analysis parses the firstof these strings as follows:
• Institution, household, or person class: M305• Person class: M388• Further designations of the individual: M240
M097∼h M004 M218Side-by-side comparison of these 6-grams
raises the question of whether the third sign ineach sequence (M240, M146, and M347 respec-tively) is yet another classifier preceding a stablePN M097∼h M004 M218, or may reflect a PNpattern in which the first element (perhaps a lo-gogram?) can alternate.
Although there are no repeated 7-grams or 8-grams, there are three pairs of 7-grams which dif-fer by only a single sign, and one such pair of8-grams. We hope that by exploring sign usagewithin such strings, future work will be able toidentify new sign ordering principles and possiblyreach a more controlled set of signs that may rep-resent anthroponyms. Such a list would offer abetter (if still slim) chance at linguistic decipher-ment. Our data exploration toolkit provides an in-terface for fuzzy string matching to facilitate fur-ther investigation of strings like these.
5 LDA Topic Model
Latent Dirichlet Allocation (LDA; Blei et al. 2003)is a topic modeling algorithm which attempts togroup related words into topics and determinewhich topics are discussed in a given set of doc-uments. Notably, LDA infers topical relationshipssolely based on rates of term co-occurrence, mean-ing it can run on undeciphered texts to yield infor-mation on which terms may be related. Note, how-ever, that topics may be semantically broad, and
128
one must be careful not to infer too much abouta sign’s meaning simply from its appearance in agiven topic. LDA differs from the other clusteringtechniques we have considered in that it also pro-vides a means for grouping tablets based on thetopics they discuss, which may reveal genres orother meaningful divisions of the corpus.
We induced a 10-topic LDA model over the PEcorpus. We chose a small number of topics tomake the task of interpreting the model more man-ageable; fewer topics make for fewer sets of rep-resentative signs to analyze. Furthermore, with 10topics the model learns topics which are mostlynon-overlapping (Figure 6), meaning there are fewredundant topics to sort through. We note, how-ever, that model perplexity drops sharply above 80topics, and topic coherence peaks around 110 top-ics; future work may therefore do well to investi-gate larger models.
PC1
PC2
1
2
3
4
5
6
78
9
10
Figure 6: Intertopic distance (measured as Jensen-Shannon divergence) visualized with LDAVis (Siev-ert and Shirley, 2014) using two principal components(PC1 and PC2). Larger circles represent more commontopics.
The following sections begin to elaborate onthe topics which we can most easily interpret, al-though space constraints prohibit full analysis ofeach individual topic. Our data exploration toolkitprovides additional details including informationabout topic stability using the stability measure in-troduced by Mäntylä et al. (2018).
5.1 Topic 1
The most representative signs for this topic areM376 and M056∼f. M376 has been speculated
to represent either a human worker category orcattle; M056∼f is a depiction of a plow ( ,comparable to the proto-cuneiform sign for plow,APIN ). This is an intriguing connection asa sign-set for bovines has not yet been identifiedin PE, despite the clear cultural importance of cat-tle suggested by PE cylinder seal depictions (Dahl,2016). More interesting still is the fact that M376and M056∼f never occur in the same text. Theirinclusion in the same topic implies that they sim-ply occur in the presence of similar signs (thoughnot as direct neighbors of those signs, since theydo not group together in the neighbor-based clus-tering). Topic modelling in this case has broughtto light tendencies in the writing system that mayhave been intuitively grasped but would be diffi-cult to quantify manually.
5.2 Topic 3The signs M297∼b and M297 are both highly rep-resentative of this topic. This is interesting as therelationship between these two signs has been un-certain (Meriggi, 1971:74). M297∼b was hypoth-esised to indicate a “keg” by Friberg (1978). It isan “object” sign that almost always appears in theultimate or penultimate position of sign strings;it sometimes appears in the summary line of ac-counts followed by numerical notations that quan-tify amounts of grain or liquids. Friberg sus-pected such texts referred to ale distributions. Aleis thought to have been a staple of the PE dietat Susa. Meriggi suggested M297 may indicate“bread”, but he also included it in his syllabary; itis the 6th most common sign in PE, appearing in145 texts, and M297∼b is the 31st most commonappearing in 66 texts. Yet topic 3 is the dominanttopic in only 85 texts, suggesting that the LDAmodel has identified a particular subset of the ac-counts that refer to M297 or M297∼b. Also ofnote is the fact that M297∼b occurs in topic 3 ata significantly higher rate than M297, despite be-ing rarer in general—a much higher percentage ofthe overall uses of M297∼b appear in this topic(around 75%) than do the overall uses of M297(less than 15%).
5.3 Topics 4 and 7The texts included in topics 4 and 7 success-fully reproduce aspects of Dahl 2005 with ref-erence to the genres of PE livestock husbandryand slaughter texts. Dahl was able to decipherthe ideographic meaning (if not phonetic realiza-
129
tion) of signs for female, male, young, and ma-ture sheep and goats and some of their products,beginning with the key observation that proto-cunieform UDU ( , “mixed sheep and goats”) isgraphically comparable to M346 ( ). The mostrepresentative signs in topic 4 are M346 (“ewe”)and M367 (“billy-goat”).
While almost every instance of M346 is repre-sentative of topic 4, it is assigned to topic 5 in theatypical text P272825 (see 5.4). Several other typ-ical livestock context uses of M346 belong to topic7. Topic 7 was the most stable topic across 30 re-peated runs in our topic stability evaluation. Themost predictive sign for this topic is M009 ( ),which is also representative of topic 4 (and ap-peared in Section 4.1.1). The most representativetexts in this topic include a few nanny-goat herd-ing texts; many more texts in this topic have noknown association with livestock or animal prod-ucts, though a few (e.g. P009141 and P008407) dobear seal impressions depicting livestock.
5.4 Topic 5The reason that the LDA model groups these 144texts is not immediately apparent to the traditionalPE specialist. An odd feature of the topic is thatM388 (“person/man”) is considered the most rep-resentative sign, but the most representative text isa simple tally of equids that never uses M388, andin fact uses few non-numerical signs overall. Thismay be due simply to noise in the model: M388may be a kind of “stopword” which crops up in un-related topics due to its high frequency. That said,an intriguing feature is that a significantly largerproportion of the texts in this topic bear a seal im-pression than do texts in the other topics. Sealimpressions are unknown to the LDA model, andtheir presence suggests that it is at least possiblethe model has identified similarities in tablet con-tent not easily observed through traditional anal-ysis. The atypical “elite redistributive account”(Kelley, 2018:163) P272825, which is also sealed,is associated with this topic. This text has around116 entries using complex sign-strings, fifteen ofwhich include M388.
5.5 Topic 6The ten most representative signs for topic 6 in-clude the five of Meriggi’s possible syllabic signsthat grouped most stably in our clustering eval-uation (see 4.1.1). Nine of the ten are alsoincluded in Meriggi’s syllabary, excluding only
M388, the second most representative sign in thetopic. M388 has been key to the identification ofpossible PNs, since it tends to appear just beforelonger sign strings and, through a series of argu-ments drawing on cuneiform parallels, may func-tion as a Personenkeil (a marker for human names;Damerow and Englund 1989; Kelley 2018:222ff.). The texts of topic 6 are of diverse size andstructure, but do tend to include many tradition-ally identifiable PNs.
5.6 Topic 10
This topic also confirms existing understanding ofa PE administrative genre, namely that of “laboradministration” (Damerow and Englund, 1989;Nissen et al., 1994). The most representative signsare the characteristic “worker category signs” de-scribed in the very long ration texts discussed byDahl et al. (2018:24–23), and indeed all of thosetexts appear in this topic, in addition to a varietyof other identifiable labor texts of somewhat dif-ferent (but partially overlapping) content.
5.7 Remaining Topics (2, 8, and 9)
Initial assessments also suggest promising av-enues of analysis for topics 2, 8, and 9. Topic2 is heavily skewed towards M288 (“grain con-tainer”), the most common PE sign;9 its thirdmost representative sign (M391, possibly mean-ing “field”) may suggest an agricultural manage-ment context for some texts in this topic. Topic8 is strongly represented by |M195+M057|. Thisis an undeciphered complex grapheme, frequentlyoccurring as a text’s second sign after the “header”M157. In topic 9, the two most representativesigns are M387 and M036 (possibly associatedwith rationing). Since the LDA model is not awareof the numeric notation between entries, it is inter-esting that the bisexagesimal numeric systems B#and B appear prominently in this topic, whether ornot M036 (associated with those systems) appears:see particularly P009048 (the text most stronglyassociated with this topic) and P008619.
5.8 LDA Summary
The preceding sections confirm that the LDAmodel largely learns topics which traditional PE
9A remarkable 37.3% of the topic’s probability mass isallocated to this sign, compared to just 2.5% for the secondmost predictive sign (M157, the “household” header sign).No other topic is so skewed: only topic 4 comes close, with20.3% of its mass assigned to M346 (“sheep”).
130
specialists recognise as meaningful. Our brief in-terpretations of the topics serve only to highlightthe amount of potentially fruitful analysis that stillremains to be done. It also remains to see whattopics arise when sign variants are collapsed to-gether: preliminary results suggest that topics re-sembling our topic 6 and topic 10 are still found,but new topics also appear which have no clearcorrelates in the model discussed in this paper.
6 Related Work
Meriggi (1971:173–174) conducted manualgraphotactic analysis of PE (and later linearElamite) texts, for example by noting the po-sitions in which certain signs could appear insign-strings. Dahl (2002) was the first to usebasic computer-assisted data sorting to presentinformation on sign frequencies, and Englund(2004:129–138) concluded his discussion of “thestate of decipherment” by suggesting that thenewly transliterated corpus would benefit frommore intensive study of sign ordering phenomena.Apart from the use of Rapidminer10 to performsimple data sorting in Kelley 2018, no publi-cations have yet described any effort to applycomputational approaches to the dataset.
Computational approaches to decipherment(Knight and Yamada, 1999; Knight et al., 2006),which resemble the setup typically followedby human archaeological-decipherment experts(Robinson, 2009), have been useful in several realworld tasks. Snyder et al. (2010) propose an au-tomatic decipherment technique that further im-proves existing methods by incorporating cognateidentification and lexicon induction. When ap-plied to Ugaritic, the model is able to correctlymap 29 of 30 letters to their Hebrew counter-parts. Reddy and Knight (2011) study the Voynichmanuscript for its linguistic properties, and showthat the letter sequences are generally more pre-dictable than in natural languages. Following this,Hauer and Kondrak (2016) treat the text in theVoynich manuscript as anagrammed substitutionciphers, and their experiments suggest, arguably,that Hebrew is the language of the document. Hi-erarchical clustering has previously been used byKnight et al. (2011) to aid in the decipherment ofthe Copiale cipher, where it was able to identifymeaningful groups such as word boundary mark-ers as well as signs which correspond to the same
10https://www.rapidminer.com/
plaintext symbol.
Homburg and Chiarcos (2016) report prelim-inary results on automatic word segmentationfor Akkadian cuneiform using rule-based, dictio-nary based, and data-driven statistical techniques.Pagé-Perron et al. (2017) furnish an analysis ofSumerian text including morphology, parts-of-speech (POS) tagging, syntactic parsing, and ma-chine translation using a parallel corpus. AlthoughSumerian and Akkadian are both geographicallyand chronologically close to PE, these corporaare very large (e.g. 1.5 million lines for Sume-rian), and are presented in word level translitera-tions rather than sign-by-sign transcriptions. Thismakes most of these techniques inapplicable toPE. Our study is more similar in spirit to Reddyand Knight (2011), as the Voynich manuscript andPE are both undeciphered and resource-poor, mak-ing analysis especially difficult.
7 Conclusions
We have shown that methods from computationallinguistics can offer valuable insights into theproto-Elamite script, and can substantially im-prove the toolkit available to the PE specialist. Hi-erarchical sign clustering replicates previous workby rediscovering meaningful groups of signs, andsuggests avenues for future work by revealing sim-ilarities between yet-undeciphered signs. Analysisof n-gram frequencies highlights the level of rep-etition of sign strings across the corpus as a pointof further research interest, and also reveals setsof similar strings worth examining in detail. LDAtopic modelling has replicated previous work inidentifying known text genres, but has also sug-gested new relationships between tablets whichcan be explored using more traditional analysis.The methods we have used are by no means ex-haustive, and there remain many more approachesto consider in future work. Particularly in a fieldpopulated by a small handful of researchers, thefaster data processing and ease of visualization of-fered by computational methods may significantlyaid progress towards understanding this writingsystem. We hope that our data exploration toolswill help facilitate future discoveries, which mayeventually lead to a more complete deciphermentof the largest undeciphered corpus from the an-cient world.
131
ReferencesDavid M. Blei, Andrew Y. Ng, and Michael I. Jordan.
2003. Latent Dirichlet allocation. Journal of Ma-chine Learning Research, 3:993–1022.
Jacob L. Dahl. 2002. Proto-Elamite sign frequencies.Cuneiform Digital Library Bulletin, 2002/1.
Jacob L. Dahl. 2005. Animal husbandry in Susa duringthe proto-Elamite period. Studi Micenei ed Egeo-Anatolici, 47:81–134.
Jacob L. Dahl. 2009. Early writing in Iran, a reap-praisal. Iran, 47(1):23–31.
Jacob L. Dahl. 2016. The production and storage offood in early Iran. In M.B. D’Anna, C. Jauß, andJ.C. Johnson, editors, Food and Urbanisation. Ma-terial and Textual Perspectives on Alimentary Prac-tice in Early Mesopotamia, volume 37, pages 45–50.Gangemi Editore.
Jacob L. Dahl. 2019. Tablettes et fragments proto-élamites / proto-Elamite tablets and fragments.Textes Cunéiform Tomes XXXII Musée de Louvre.
Jacob L. Dahl, Laura Hawkins, and Kate Kelley. 2018.Labor administration in proto-Elamite Iran. In Ag-nès Garcia-Ventura, editor, What’s in a Name? Ter-minology related to the Work Force and Job Cat-egories in the Ancient Near East, pages 15–44.Alt Orient und Altes Testament 440. Ugarit Verlag:Münster.
Jacob L. Dahl, M. Hessari, and R. Yousefi Zoshk. 2012.The proto-Elamite tablets from Tape Sofalin. Ira-nian Journal of Archaeological Studies, 2(1):57–73.
Peter Damerow. 2006. The origins of writing as a prob-lem of historical epistemology. Cuneiform DigitalLibrary Journal, 2006/1.
Peter Damerow and Robert K. Englund. 1989. TheProto-Elamite Texts from Tepe Yahya. Bul-letin (American School of Prehistoric Research).Peabody Museum of Archaeology and Ethnology,Harvard University.
Leon Derczynski and Sean Chester. 2016. Gener-alised Brown clustering and roll-up feature gener-ation. In Proceedings of the Thirtieth AAAI Con-ference on Artificial Intelligence, AAAI 2016, pages1533–1539. AAAI Press.
François Desset. 2016. Proto-Elamite writing in Iran.Archéo-nil. Revue de la société pour l’étude des cul-tures prépharaoniques de la valée du Nil, 26:67–104.
Robert K. Englund. 1996. The proto-elamite script.In Peter Daniels and William Bright, editors, Theworld’s writing systems. Oxford University Press,Oxford, UK.
Robert K. Englund. 2004. The state of decipherment ofproto-Elamite. The First Writing: Script Inventionas History and Process, pages 100–149.
Jöran Friberg. 1978. The Third Millennium Roots ofBabylonian Mathematics I-II. Göteborg Dept. ofMathematics, Chalmers University of Technology,Göteborg, Sweden.
Bradley Hauer and Grzegorz Kondrak. 2016. Decod-ing anagrammed texts written in an unknown lan-guage and script. Transactions of the Associationfor Computational Linguistics, 4:75–86.
Laura F. Hawkins. 2015. A new edition of the Proto-Elamite text MDP 17, 112. Cuneiform Digital Li-brary Journal, 1.
Timo Homburg and Christian Chiarcos. 2016. Akka-dian word segmentation. In Tenth Interna-tional Conference on Language Resource Evalua-tion (LREC 2016), pages 4067–4074.
Kate Kelley. 2018. Gender, Age, and Labour Organi-zation in the Earliest Texts from Mesopotamia andIran (c. 3300–2900 BC). Doctoral dissertation, Uni-versity of Oxford.
Kevin Knight, Beáta Megyesi, and Christiane Schaefer.2011. The Copiale cipher. In Proceedings of the4th Workshop on Building and Using ComparableCorpora: Comparable Corpora and the Web, pages2–9. Association for Computational Linguistics.
Kevin Knight, Anish Nair, Nishit Rathod, and KenjiYamada. 2006. Unsupervised analysis for deci-pherment problems. In Proceedings of the COL-ING/ACL on Main Conference Poster Sessions,COLING-ACL ’06, pages 499–506, Stroudsburg,PA, USA. Association for Computational Linguis-tics.
Kevin Knight and Kenji Yamada. 1999. A computa-tional approach to deciphering unknown scripts. InProceedings of the ACL Workshop on UnsupervisedLearning in Natural Language Processing.
Mika Mäntylä, Maelick Claes, and Umar Farooq.2018. Measuring LDA topic stability from clus-ters of replicated runs. In Proceedings of the 12thACM/IEEE International Symposium on Empiri-cal Software Engineering and Measurement, ESEM’18, New York, NY, USA. ACM.
Piero Meriggi. 1971. La scrittura proto-elamica. ParteIa: La scrittura e il contenuto dei testi. AccademiaNazionale dei Lincei, Rome.
Hans J. Nissen, Peter Damerow, and Robert K. En-glund. 1994. Archaic Bookkeeping: Writing andTechniques of Economic Administration in the An-cient Near East. University of Chicago Press.
Émilie Pagé-Perron, Maria Sukhareva, Ilya Khait, andChristian Chiarcos. 2017. Machine translation andautomated analysis of the Sumerian language. In
132
Joint SIGHUM Workshop on Computational Lin-guistics for Cultural Heritage, Social Sciences, Hu-manities and Literature, ACL, pages 10–16. Associ-ation for Computational Linguistics.
Sravana Reddy and Kevin Knight. 2011. What weknow about the Voynich manuscript. In Proceedingsof the 5th ACL-HLT Workshop on Language Tech-nology for Cultural Heritage, Social Sciences, andHumanities, pages 78–86.
A. Robinson. 2009. Lost Languages: The Enigma ofthe World’s Undeciphered Scripts. Thames & Hud-son.
Jean-Vincent Scheil. 1905. Documents archaïques enécriture proto-élamite. Mémoires de la Délégationen Perse, 6:57–128.
Carson Sievert and Kenneth Shirley. 2014. LDAvis: Amethod for visualizing and interpreting topics. InProceedings of the Workshop on Interactive Lan-guage Learning, Visualization, and Interfaces, pages63–70. Association for Computational Linguistics.
Benjamin Snyder, Regina Barzilay, and Kevin Knight.2010. A statistical model for lost language deci-pherment. In Proceedings of the 48th Annual Meet-ing of the Association for Computational Linguis-tics, pages 1048–1057. Association for Computa-tional Linguistics.
