Towards a Universal Wordnet by Learning from Combined Evidence

IntroductionExisting Lexical Knowledge Bases

Building a Multilingual WordnetResults and Experiments

Summary and Future Work

Towards a Universal Wordnetby Learning from Combined Evidence

Gerard de Melo and Gerhard Weikum

Max Planck Institute for InformaticsSaarbrucken, Germany

2009-11-03

Gerard de Melo and Gerhard Weikum Towards a Universal Wordnet 1/29

Lexical KnowledgeMultilingualityVision

Introduction

Lexical Knowledge

What meanings doesa word have?

How do those meaningsrelate to the meaningsof other words?

Many Applications

examples:NLP, AIquestion answeringquery expansionhuman consultation

person whogives a talk

“speaker”

device that produces

sounds

Introduction

Lexical Knowledge

Many Applications

flat piece of wood

“board”

committee

panel for writingwith chalk

to enter a transportation

vehicle

Introduction

Lexical Knowledge

Many Applications

someone who studies

“student”

“pupil”

Introduction

Lexical Knowledge

Many Applications

faculty

professor

memberpart

Introduction

Lexical Knowledge

Many Applications

entity

institution

educationalinstitution

university

Introduction

Lexical Knowledge

Many Applications

entity

institution

university

Introduction

Multilinguality

the world ismultilingual

the Internet is alsoincreasinglymultilingual

Top 10 Languages byApprox. No. of Speakers

Source: Ethnologue 2005

Introduction

Multilinguality

the world ismultilingual

the Internet is alsoincreasinglymultilingual

Internet users by Region

Source:

Internet World Stats

Introduction

person who gives a talk

eng: “speaker”

jpn: “ ”話者

rus: “докладчик”

ces: “řečník”

... ......

Vision

universal index of wordmeanings

large-scale semantic networkwith class hierarchy

look up any wordin any language,get a list of its meanings

Introduction

entitypor: “entidade”

cmn: “ ”制度 institution

university

heb: “ישות.”

deu: “Bildungs-einrichtung”

cym: “prifysgol”

Vision

universal index of wordmeanings

large-scale semantic networkwith class hierarchy

meanings should be connectedvia semantic relations

Outline

1 Existing Lexical Knowledge Bases

2 Building a Multilingual Wordnet

3 Results and Experiments

4 Summary and Future Work

WordNetNon-English WordnetsOther Resources

Outline

Existing Lexical Knowledge Bases

WordNet

lexical database created at Princeton

enumerates meanings of Englishwords

meaning-to-meaning links

Miller, Fellbaum et al. (1990)among most-cited papersin computer science(source: CiteseerX)

WordNet

hypernym hierarchymeronymy (part of)etc.

Non-English Wordnets

EuroWordNet, BalkaNet, Global WordNet Association

problem: many are small, incomplete

problem: different identifiers, formats, etc.

problem: only ∼10 languages with freely available wordnets

not a single, coherent resource

Other Resources

PANGLOSS Ontology: Knight & Luk (1994)

TransGraph system: Etzioni et al. (2007)

DBPedia, YAGO, OpenCyc

2 languages, around 70 000 entities

Other Resources

large translation graphlimited structuree.g. no semantic hierarchy

Other Resources

class hierarchy not multilingual

StrategyInput GraphApproachFeatures

Outline

Building a Multilingual Wordnet

Strategy

use existing wordnets as backbone

add new terms, link to meaning nodes

spa: “trayectoria”

academic course

part of a meal

route of travel

series of events

eng: “course”

eng: “class”

Existing Wordnets

−→

deu: “Reihe”

academic course

part of a meal

route of travel

series of events

ita: “piatto”

fra: “suite”

eng: “course”

deu: “Kurs”

eng: “class”

Desired Output

Strategy

add new terms, link to meaning nodes

academic course

part of a meal

route of travel

series of events

eng: “course”

eng: “class”

Existing Wordnets

−→

deu: “Reihe”

academic course

part of a meal

route of travel

series of events

ita: “piatto”

fra: “suite”

eng: “course”

deu: “Kurs”

eng: “class”

Desired OutputGerard de Melo and Gerhard Weikum Towards a Universal Wordnet 11/29

Input Graph

add translations to graph

mainly English, Spanish, Catalan

academic course

part of a meal

route of travel

series of events

eng: “course”

eng: “class”

Input Graph G0

Input Graph

add translations to graph

dictionaries (e.g. Wiktionary)thesauri and ontologiesparallel corpora (word alignment)

also: predict new translations

deu: “Reihe”

academic course

part of a meal

route of travel

series of events

ita: “piatto”

fra: “suite”

eng: “course”

deu: “Kurs”

eng: “class”

Input Graph G0

Approach: Link new words to meanings of their translations

Huge Challenge: Disambiguation!

academic course

part of a meal

route of travel

series of events

ita: “piatto”

eng: “course”

trans-lation

Approach: Link new words to meanings of their translations

Huge Challenge: Disambiguation!

academic course

part of a meal

route of travel

series of events

ita: “piatto”

eng: “course”

trans-lation

academic course

part of a meal

route of travel

series of events

ita: “piatto”

eng: “course”

trans-lation

Approach

variety of features that analyseprevious graph Gi−1,incorporate neighbourhoodinformation into anedge’s feature vector

supervised learning: new edgeweights determined usingRBF-kernel SVM with posteriorprobability estimation

academic course

part of a meal

route of travel

series of events

ita: “piatto”

eng: “course”

trans-lation

Approach

variety of features that analyseprevious graph Gi−1,incorporate neighbourhoodinformation into anedge’s feature vector

supervised learning: new edgeweights determined usingRBF-kernel SVM with posteriorprobability estimation

Example Feature:

fra: “suite” academic course?

Given term tand meaning m

Question: Should they be linked?

Example Feature:

fra: “suite”

eng: “course”

part of a meal

academic course

route of travel

series of eventst'

Given term tand meaning m

Question: Should they be linked?

Look at neighbours t ′ ∈ Γt

Example Feature:

fra: “suite”

eng: “course”

part of a meal

academic course

route of travel

series of eventst'

∑t′∈Γ(t)

sim∗(t ′,m)

sim∗(t ′,m) + dissim(t ′,m)

sim∗(t′,m)= maxm′∈Γ(t′)

sim(m′,m)

dissim(t′,m)=P

m′∈Γ(t′)(1−sim(m′,m))

Example Feature:

fra: “suite”

eng: “course”

part of a meal

academic course

route of travel

series of eventst'

∑t′∈Γ(t)

sim∗(t ′,m)

sim(m′,m)

dissim(t′,m)=P

m′∈Γ(t′)(1−sim(m′,m))

Example Feature:

fra: “suite”

eng: “course”

part of a meal

academic course

route of travel

series of eventst'

∑t′∈Γ(t)

φ1(t, t ′) sim∗(t ′,m)

φ2(t′,m′)sim(m′,m)

dissim(t′,m)=P

m′∈Γ(t′)φ2(t′,m′)(1−sim(m′,m))

weighting based on:part-of-speechcorpus frequency...

deu: “Reihe”

academic course

part of a meal

route of travel

series of events

ita: “piatto”

fra: “suite”

eng: “course”

deu: “Kurs”

eng: “class”

Other Features

cosine similarity oftranslations with gloss

scores assessing polysemy bylooking at back-translations

many more(see paper for details)

deu: “Reihe”

academic course

part of a meal

route of travel

series of events

ita: “piatto”

fra: “suite”

eng: “course”

deu: “Kurs”

eng: “class”

Approach

use scores as features forRBF-kernel SVM

multiple iterations:each graphs Gi based on theprevious Gi−1

stop when F1 score plateauis reached on a validation set

deu: “Reihe”

academic course

part of a meal

route of travel

series of events

ita: “piatto”

fra: “suite”

eng: “course”

deu: “Kurs”

eng: “class”

Approach

deu: “Reihe”

academic course

part of a meal

route of travel

series of events

ita: “piatto”

fra: “suite”

eng: “course”

deu: “Kurs”

eng: “class”

Approach

SetupOutputEvaluationApplication: Semantic RelatednessApplication: Cross-Lingual Text Classification

Outline

Results

input graph G0:448,069 pre-existing term-meaning links10,805,400 translation edges1.3 million term nodes with candidates7.7 candidate meanings per new term

2,445 term-meaning links for training (French/German)

2,901 term-meaning links as validation set

Results

deu: “Schulgebäude”

school (group of fish)

school(institution)

school(building)

deu: “Schulhaus”

deu: “Fischschwarm”

ces: “hejno”

fra: “banc”

ind: “sekolah”

jpn: “ ”学校

kor: “ ”학교

lao: “ໂຮງຮຽນ”

kat: “ ”სკოლა

Excerpt from final UWN graph G3 after 3 iterationsretaining only edges with sufficiently high weights (0.5 / 0.6)

Evaluation

Relation Precision1

Term-Meaning Links (French) 89.2% ± 3.4%Term-Meaning Links (German) 85.9% ± 3.8%Term-Meaning Links (Mandarin Chinese) 90.5% ± 3.3%

Generalization (Hypernymy) 87.1% ± 4.8%Instance 89.3% ± 4.4%Similarity 92.0% ± 3.8%Category 93.3% ± 4.5%Part (Meronymy) 94.4% ± 4.1%Member (Meronymy) 92.7% ± 4.0%Substance (Meronymy) 95.6% ± 3.5%Opposite 94.3% ± 3.9%

1: Wilson score intervals for random samples

Coverage

Language Term-Meaning Links Distinct Terms

Overall 1,595,763 822,212

German 132,523 67,087French 75,544 33,423Esperanto 71,247 33,664Dutch 68,792 30,154Spanish 68,445 32,143Turkish 67,641 31,553Czech 59,268 33,067Russian 57,929 26,293Portuguese 55,569 23,499Italian 52,008 24,974Hungarian 46,492 28,324Thai 44,523 30,815

Application: Semantic Relatedness

Experimental Setup

Example: “curriculum” considered closely related to“school”, but not to “water”

compute term relatedness using UWNsim(t1, t2) = max

s1∈σ(t1)max

s2∈σ(t2)sim(s1, s2) sim(s1, s2):

combined graph-/gloss-based method

compare with assessments of relatedness made by humanjudges

Experimental Setup

s1∈σ(t1)max

Experimental Setup

s1∈σ(t1)max

Results for 3 German DatasetsDataset GUR65 GUR350 ZG222

r Cov. r Cov. r Cov.

Inter-Annot. Agreement 0.81 (65) 0.69 (350) 0.49 (222)

Wikipedia (ESA*) 0.56 65 0.52 333 0.32 205GermaNet (Lin*) 0.73 60 0.50 208 0.08 88

UWN 0.80 60 0.68 242 0.51 106r : Pearson product-moment correlation coefficient

Cov.: absolute coverage

∗: scores by Gurevych et al. (2007)

Application: Cross-Lingual Text Classification

cross-lingual TC: train using documents in one language,classify documents in another language

used bag-of-words/meanings TF-IDF vectors

Dataset: Reuters corpora (RCV1/2)for each language pair:105 binary classification tasks, each using200 training documents, 600 test documents

SVMlight

Language Pair Terms only Terms + Meanings

English-Italian 68.3% 76.3%

English-Russian 51.7% 71.2%

Italian-English 74.4% 78.1%

Italian-Russian 58.4% 73.2%

Russian-English 67.3% 76.8%

Russian-Italian 62.2% 71.8%

(all values are F1 scores)

SummaryFuture Work

Outline

SummaryFuture Work

Summary

large-scale multilingual wordnet:85% accuracy, 800,000 terms, over 1.5 million links fromterms to meanings,

built by learning edge weights using graph-based evidence

useful for monolingual and cross-lingual tasks

SummaryFuture Work

Summary

SummaryFuture Work

Summary

SummaryFuture Work

Future Work

ongoing work: user interface incl. user contributions

techniques to automatically discover new word meanings

word sense disambiguation, query expansion using UWN

SummaryFuture Work

Future Work

SummaryFuture Work

Future Work

SummaryFuture Work

Thanks!

expression of gratitude

eng: “thank you”

yue: “ ”唔該

cmn: “ ”谢谢

jap: “ ”ありがとう

spa: “gracias”

ara: “شكرا.”

Towards a Universal Wordnet by Learning from Combined Evidence

Data & Analytics

Wordnet, EuroWordNet, Global Wordnet

Consortium Project on Development of Dravidian WordNet: An Integrated WordNet for Telugu, Tamil, Kannada and Malayalam

The Global Wordnet Grid: anchoring languages to universal meaning Piek Vossen Irion Technologies/Vrije Universiteit Amsterdam 6 th International Plain

Combined Charging the universal charging systemopi2020.com/images/4_opi2020_2012_11_Combined_Charging... · 2015. 12. 9. · Combined Charging System Combined Charging Survey, V1.3

Wordnet eBook

Introduction to WordNet

1 From WordNet, to EuroWordNet, to the Global Wordnet Grid: anchoring languages to universal meaning Piek Vossen VU University Amsterdam

Wordnet affect-071116

Amharic WSD using WordNet

Urdu WordNet 1.0 Wordlist

WordNet ® and its Java API ♦ Introduction to WordNet ♦ WordNet API for Java Name: Hao Li Uni: hl2489

Indo WordNet A WordNet for Hindi

Urdu WordNet Wordlist 2 - cle.org.pkcle.org.pk/Downloads/ling_resources/wordlists/Urdu WordNet 1.0 Wordlist.pdf · Center for Language Engineering 2013 Urdu WordNet 1.0 Wordlist Page

Wordnet Introduction

Punjabi WordNet

Random Walk on WordNet to Measure Lexical Semantic …tpederse/Pubs/yanbo-report.pdf · 2 WordNet WordNet is a lexical database of English with nouns, verbs, adjectives and adverbs

Nouns In WordNet

Wordnet and Semantick Similarity

Urdu WordNet Wordlist 2 - Center for Language Engineering WordNet 1.… · Center for Language Engineering 2013 Urdu WordNet 1.0 Wordlist Page 1 Urdu WordNet 1.0 Wordlist This wordlist

WordNet - iaoa.org · • WordNet is organized by meaning: words in close proximity are semantically similar • Human users and computers can browse WordNet and find words that are