Relational Inference for Wikification Xiao Cheng and Dan Roth University of Illinois at...

Relational Inference for Wikification

Xiao Cheng and Dan RothUniversity of Illinois at Urbana-Champaign

Wikification

Blumenthal (D) is a candidate for the U.S. Senate seat now held by Christopher Dodd (D), and he has held a commanding lead in the race since he entered it. But the Times report has the potential to fundamentally reshape the contest in the Nutmeg State.

Applications

Knowledge Acquisition via Grounding Coreference Resolution

Learning-based multi-sieve co-reference resolution with knowledge (Ratinov et al. 2012)

Information Extraction Unsupervised relation discovery with sense disambiguation (Yao et al.

2012) Automatic Event Extraction with Structured Preference Modeling (Lu

and Roth, 2012 ) Text Classification

Gabrilovich and Markovitch, 2007; Chang et al., 2008 Entity Linking

Ambiguity

Concepts outside of Wikipedia (NIL) Blumenthal ?

Variability

Scale Millions of labels

Challenges

ConnecticutCT

The Nutmeg StateTimesThe New York TimesThe Times

State-of-the-art systems (Ratinov et al. 2011) can achieve the above with local and global statistical features Reaches bottleneck around 70%~ 85% F1 on non-wiki datasets What is missing?

Challenges

, the of deposed , …

Motivating Example

Mubarak wife Egyptian President Hosni Mubarak

What are we missing with Bag of Words (BOW) models? Who is Mubarak?

Constraining interaction between concepts (Mubarak, wife, Hosni Mubarak)

Mubarak, the wife of deposed Egyptian President Hosni Mubarak, …

Relational Inference for Wikification

Our contribution Identify key textual relations for Wikification A global inference framework to incorporate relational knowledge

Significant improvement over state-of-the-art systems

(Mubarak, wife, Hosni Mubarak)

Talk Outline

Why Wikification? Introduction

Motivation Approach

Wikification Pipeline Formulation Relational Analysis

Evaluation Result Entity Linking

Mention Segmentation

Candidate Generation

Candidate Ranking

NIL Linking

Wikification

Mention Segmentation

Candidate Generation

Candidate Ranking

Determine NILs

Wikification Pipeline 1 - Mention Segmentation

...ousted long time Yugoslav President Slobodan Milošević in October. Mr. Milošević's Socialist Party…

sub-NP (Noun Phrase) chunks NER Regular expressions

Wikification Pipeline 2 - Candidate Generation

1 Socialist_Party_(France)

2 Socialist_Party_(Portugal)

3 Socialist_Party_of_America

4 Socialist_Party_(Argentina)

1 Slobodan_Milošević

2 Milošević_(surname)

3 Boki_Milošević

4 Alexander_Milošević

Wikification Pipeline 3 - Candidate Ranking

k ek4 sk

1 Socialist_Party_(France) 0.23

2 Socialist_Party_(Portugal) 0.16

3 Socialist_Party_of_America 0.07

4 Socialist_Party_(Argentina) 0.06

k ek3 sk

1 Slobodan_Milošević 0.7

2 Milošević_(surname) 0.1

3 Boki_Milošević 0.1

4 Alexander_Milošević 0.05

Local and global statistical features

Wikification Pipeline 4 – Determine NILs

k ek4 sk

k ek3 sk

Is the top candidate really what the text referred to?

Talk Outline

Motivation Approach

Formulation (0)

Intuition Promote pairs of concepts coherent with textual relations

(Mubarak, wife, Hosni Mubarak)

Formulate as an Integer Linear Program (ILP):

If no relation exists, collapse to the non-structured decision

Formulation (1)

Whether a relation exists between and

weight of a relation

Whether to output th candidate of the th mentionweight to output

Formulation (2)

k ek4 sk

k ek3 sk

r(1,2)34

eki: whether a concept is chosen

ski : score of a concept

r(k,l)ij: whether a relation is present

w(k,l)ij : score of a relation

r(4,3)34

Overall Approach

Wikification

Candidate

Generation

Relation Analysis

Relation Identification

ACE style in-document coreference Extract named entity-only coreference relations with high precision

Syntactico-Semantic relations (Chan & Roth ‘10)

Easy to extract with high precision Aim for high recall, as false-positives will be verified and discarded Sparse, but covers ~80% relation instances in ACE2004

Type Example

Premodifier Iranian Ministry of Defense

Possessive NYC’s stock exchange

Formulaic Chicago, Illinois

Preposition President of the US

Argument 1 Relation Type Argument 2

Yugoslav President apposition Slobodan Milošević

Slobodan Milošević coreference Milošević

Milošević possessive Socialist Party

Overall Approach

Wikification

Candidate

Generation

Relation Analysis

Relation Retrieval

Current approach Collect known mappings from Wikipedia page titles, hyperlinks… Limit to top-K candidates based on frequency of links (Ratinov et al.

2011) What concepts can “Socialist Party” refer to?

Uninformative Mentions

Relation Retrieval

What concepts can “Socialist Party” refer to? More robust candidate generation

Identified relations are verified against a knowledge base (DBPedia) Retrieve relation arguments matching “(Milošević ,?,Socialist Party)”

as our new candidates

Query Pruning Only 2 queries per pair necessary due to strong baseline.

Relation Retrieval

q1=(Socialist Party of France,?, *Milošević*)q2=(Slobodan Milošević,?,*Socialist Party*)

k ek4 sk

k ek3 sk

Relation Retrieval

Argument 1 Relation Type Argument 2

Milošević possessive Socialist Party

Relation Retrieval

k ek4 sk

21 Socialist_Party_of_Serbia 0.0

k ek3 sk

𝑟34(1,21)=1

Overall Approach

Wikification

Candidate

Generation

Relation Analysis

Relational Inference

k ek4 sk

k ek3 sk

𝑟34(1,21)=1

𝑤34(1,21)=?

Relation scoring

Query scoring as a tie-breaker between multiple relations Explicit relations are stronger than a hyperlink relations Normalize score for each pair of mention to

𝑤=1𝑍𝑓 (𝑞 ,𝜎 )

Retrieved relation tupleRelation query

k ek4 sk

k ek3 sk

𝑟34(1,21)=1

Relational Inference - coreference

k ek2 sk

k ek3 sk

𝑟23(1,1)=1

Overall Approach

Wikification

Candidate

Generation

Relation Analysis

Determine unknown concepts (NILs)

How to capture the fact: “Dorothy Byrne” does not refer to any concept in Wikipedia

Identify coreferent nominal mention relations Generate better features for NIL classifier

Dorothy Byrne, a state coordinator for the Florida Green Party,…

k ek2 sk

1 Green_Party_of_Florida 1.0

k ek1 sk

1 Dorothy_Byrne_(British_Journalist)

2 Dorothy_Byrne_(mezzo-soprano)

nominal mention

Determine unknown concepts (NILs)

Create NIL candidate for propagation

Dorothy Byrne, a state coordinator for the Florida Green Party,…

k ek2 sk

1 Green_Party_of_Florida 1.0

k ek1 sk

0 NIL 1.0

1 Dorothy_Byrne_(British_Journalist)

2 Dorothy_Byrne_(mezzo-soprano)

nominal mention

Talk Outline

Motivation Approach

Wikification Performance Result

ACE MSNBC AQUAINT Wikipedia60

F1 Performance on Wikification datasets

Milne&WittenRatinov&RothRelational Inference

Evaluation – TAC KBP Entity Linking

Wikipedia Articles

500000

1000000

1500000

2000000

2500000

3000000

3500000

4000000

4500000

5000000

TAC KBNon-TAC KB

Task Definition Links a named entity in a document to

either a TAC Knowledge Base (TAC KB) node or NIL

Cluster NIL entities Relevant Tasks

Wikification Cross-document coreference

Evaluation – TAC KBP Entity Linking

Run Relational Inference (RI) Wikifier “as-is”: No retraining using TAC data

CogCompLC

MS_MLI

NUShime

*Median

TAC KBP 2011 Entity Linking Performance

Micro AverageB³F1

System Names*Median of top 14 systems

Conclusion

Importance of linguistic and world knowledge Identification of relational information benefits Wikification Introduced an inference framework to leverage better

language understandings Future work

Accumulate what we know about NIL concepts Joint entity typing, coreference and disambiguation Incorporate more relations

*Demo will be updated in a week at: http://cogcomp.cs.illinois.edu/demo/wikify Download at: http://cogcomp.cs.illinois.edu/page/download_view/Wikifier

Thank you!

BACK UP SLIDESBack up slides

Massive Textual Information

How can we know more from large volumes of possibly unfamiliar raw texts?

Massive Textual Information

We naturally “look up” concepts and accumulate knowledge.

Challenges

It’s a version of Chicago – the standard classic Macintosh menu font…

Tuesday marks the 142nd anniversary of an event that forever altered the course of Chicago's development as a then-young American city

By the time the New Orleans Saints kicked off at Soldier Field on Sunday afternoon, their woeful history in the Windy City was fully understood.

Ambiguity

Variety

Concepts outside of Wikipedia (NIL)

Challenges

It’s a version of Chicago – the standard classic Macintosh menu font…

Tuesday marks the 142nd anniversary of an event that forever altered the course of Chicago's development as a then-young American city

By the time the New Orleans Saints kicked off at Soldier Field on Sunday afternoon, their woeful history in the Windy City was fully understood.

?Chicago Chicago

Chicago font

ChicagoChicago

The Windy City

Organizing knowledge

Wikipedia as a source of “common sense” knowledge Naturally bridges rich structured knowledge and text data Comprehensive for most purposes

~4.3 million English articles as of today Cross-lingual

Estimated size of a printed version of Wikipedia as of August 2010. *Picture courtesy of Wikipedia

Motivating Example

Relatively sparse, but act as hard constraints

Intuitively, we need to “de-coreference” this pair of mention Opens a new dimension in text understanding

helps all stages of Wikification

Mubarak

Hosni Mubarak

Egyptian President

Wikification Approach

Mention Segmentation Use Shallow Parsing, NER and regular expression to generate likely

mentions of concepts. Match nested mentions using dictionary. Discard unknown mentions.

Ranking

Features Local features from Bag of Words (BOW) representation, such as

various TFIDF windows. Global features from Bag of Concepts (BOC) representation.

What are we losing?

Mubarak

Hosni Mubarak

Egyptian President

Relation Extraction

In document coreference Extract entity-only coreference relations with high precision

Syntactico-Semantic relations (Chan & Roth ‘10)

Easy to extract with high precision Deep parsing too slow

Premodifier Iranian Ministry of Defense

Possessive NYC’s stock exchange

Formulaic Chicago, Illinois

Preposition President of the US

...ousted long time Yugoslav President Slobodan Milošević in October. Mr. Milošević's Socialist Party

Coreference

Possessive

...ousted long time Yugoslav President Slobodan Milošević in October. Mr. Milošević's Socialist Party

How to look up knowledge and use these relations?

Wikification Approach

Recall concepts Rank concepts Determine unknown concepts (NILs)

Determine NILs

Dorothy Bryne, a state coordinator for the Florida Green Party

Leverage nominal mention relations to construct limited context

Relational Inference Example

...like Chicago O'Hare and [La Guardia] in [New York],...

Wikipedia Page Ski

LaGuardia_Airport 0.36La_Guardia 0.22Fiorello_La_Guardia 0.18La_Guardia,_Spain 0.11

Wikipedia Page Ski

New_York 0.53New_York_City 0.44New_York_(magazine) 0.01New_York_Knicks 0.01

Relational Inference Scenario 1

Wikipedia Page ScoreLaGuardia_Airport 0.36La_Guardia 0.22Fiorello_La_Guardia 0.18La_Guardia,_Spain 0.11

Wikipedia Page ScoreNew_York 0.53New_York_City 0.44New_York_(magazine) 0.01New_York_Knicks 0.01

Γ = 0.18+0.53 = 0.71

Relational Inference Scenario 2

Wikipedia Page ScoreLaGuardia_Airport 0.36La_Guardia 0.22Fiorello_La_Guardia 0.18La_Guardia,_Spain 0.11

Wikipedia Page ScoreNew_York 0.53New_York_City 0.44New_York_(magazine) 0.01New_York_Knicks 0.01

Γ = 0.36+0.44 = 0.8

Relational Inference Conclusion

Pr(Γ= ) > Pr(Γ= )

Constraint scaling

Lexical similarity as a tie-breaker between multiple relations Explicit relations are stronger than a hyperlink relations Normalize score for each pair of mention to [0,1]

Relation scaling factor

Query lexical similarityNormalization factor

: query: a retrieved relation triple

Sub noun phrase chunks and NER segments (Ratinov et al. 2011)

Regular expression capturing longer composite mentions President of the United States of America

Relational Inference for Wikification Xiao Cheng and Dan Roth University of Illinois at...

Documents

Xiao zhang portfolio

2009 Project Profiles for Champaign County, Illinois...Thank you, on behalf of Champaign County, for your consideration. Participating Agencies Champaign County Champaign Park District

The wikification-of-knowledge-5

Guang Xiao Thesis

Yingcai Xiao

Xiao Proceedings

1 Wikification CSE 6339 (Section 002) Abhijit Tendulkar

Zhu Xiao-Mei

11.3.2013.1020 xiao

Diabetes ( Xiao Ke)

Xiao-Yao-SanFormulaImprovesCognitiveAbilitybyProtecting

Assyria Abigail, Cassidy, Xiao Xiao, Fannie Period 1

LASER THERAPY IN GLAUCOMA Sun Xiao Dong Sun Xiao Dong

Eight Views of the Xiao and Xiang (Xiao Xiang bajing

Champaign Delivers

Xiao Xin 2002

World Paper_Mimi Xiao

Xiao Pan Tutorial

Anomaly Detection in Data Docent Xiao-Zhi Gao xiao-zhi.gao@aalto.fi

By Xiao Ou