Trains of Thought: Generating Information Maps Dafna Shahaf, Carlos Guestrin and Eric Horvitz

Trains of Thought: Generating Information Maps

Dafna Shahaf, Carlos Guestrin and Eric Horvitz

The abundance of books is a distraction‘‘

,,Lucius Annaeus Seneca

4 BC – 65 AD

So, you want to understand a complex topic…

Now what?

Search Engines are Great

• But do not show how it all fits together

Timeline Systems

Real Stories are not Linear

Metro Map

• A set of lines• Each line follows a coherent narrative thread• Structure + multiple aspects

austerity

bailout

junk status

Germany

protests

strike

labor unionsMerkel

Map Definition• A map M is a pair (G, P) where – G=(V,E) is a directed graph– P is a set of paths in G (metro lines)– Each e Î E must belong to at least one metro line

austerity

bailout

junk status

protests

strike

Germany

labor unionsMerkel

Game Plan

Objective Algorithm Does itwork?

Properties of a Good Map

1. Coherence

???

1 2 3 4 5

Greece

Europe

ItalyRepublican

Protest

Coherence: Main IdeaConnecting the Dots [S, Guestrin, KDD’10]

Debt default

Coherence is not a property of local interactions:

Incoherent: Each pair shares different words

1 2 3 4 5

Greece

Austerity

ItalyRepublican

Protest

Coherence: Main IdeaConnecting the Dots [S, Guestrin, KDD’10]

Debt default

A more-coherent chain:

Coherent: a small number of words captures the story


1. Coherence

Is it enough?

Max-coherence MapQuery: Clinton

Clinton visitsBelfast

Clinton setfor Dublin

High hopes for Clinton visit

Clinton, Religious Leaders Share

Thoughts

Church Leaders Praise Clinton's

'Spirituality'

Religion Leaders Divided on Clinton

Moral Issue

Clinton Should Resign, 2 Religious

Leaders Say


1. Coherence

2. Coverage

Should cover diverse topics important to

the user

Coverage• Select a small set of diverse articles that

covers the most important stories

January 17, 2009

Turning Down the Noise [El-Arini, Veda, S, Guestrin, KDD’09]

Coverage: The Idea• Documents cover concepts:

CorpusCoverage

High-coverage, Coherent Map

Greek Civil ServantsStrike over

Austerity MeasuresGreece Paralyzed

by New Strike

Greek Take to theStreets, but Lacing

Earlier Zeal

Infighting Adds to Merkel’s Woes

It’s Germany that Matters

UK Backs Germany’s Effort

Germany says the IMF should Rescue

Greece

IMF more Likely to Lead Efforts

IMF is Urged to Move Forward


1. Coherence

2. Coverage

3. Connectivity

Definition: Connectivity

• Experimented with formulations• Users do not care about connection type• Encourage connections between pairs of lines

Tying it all Together:Map Objective

• Coherence– Either coherent or not: Constraint

• Coverage– Must have!

• Connectivity– Nice to have

Consider all coherent maps with maximum possible coverage.

Find the most connected one.

Game Plan


Approach Overview

Documents D

…

1. Coherence graph G 2. Coverage function f

f( ) = ?

3. Increase Connectivity

Coherence Graph: Main Idea

• Vertices correspond to short coherent chains• Directed edges between chains which can be

conjoined and remain coherent

1 2 3

4 5 6 5 8 9

1 2 3 5 8 9

Finding Vertices

• Vertices are short, coherent chains• Can use [KDD’10]– Expensive– Solving many LPs

• Take advantage of simplicity of short stories– No topic drift– Sampling-based (fast) algorithm

Finding Edges

• Problem: Combining several strong chains may result in a much-weaker chain

Discontinuity:Change of focus

A chain is m-coherent if each sub-chain (di, …, di+m) is coherent.

m-Coherence• Control discontinuity points:

• m: size of user's ‘history window‘– m=length(chain) : standard coherence– m=1: optimize transitions without context

Observation

• If two chains are m-Coherent and have m-1 overlap, the conjoined chain is m-coherent:

Using the Observation

1 2 3

2 3 4 2 3 5

1 2 3 5

• If two chains are m-Coherent and have m-1 overlap, the conjoined chain is m-coherent:

• Useful for divide and conquer:– Add edge if m-1 overlap

Approach Overview

Documents D

…


f( ) = ?


Finding High-Coverage Chains• Paths correspond to coherent chains.• Problem: find a path of length K maximizing

coverage of underlying articles

1 2 3

2 3 4 2 3 5

1 2 3 51 2 3 4Cover( ) > Cover( ) ?

Reformulation• Paths correspond to coherent chains.• Problem: find a path of length K maximizing

coverage of underlying articles

• Submodular orienteering– [Chekuri and Pal, 2005]– Quasipolynomial time recursive greedy– O(log OPT) approximation

Orienteering

a function of the nodes visited

Approach Overview: Recap

Documents D

…


f( ) = ?


Encodes all m-coherent

chains as graph paths

Submodular orienteering [Chekuri & Pal, 2005]

Quasipoly time recursive greedy

O(log OPT) approximation

Example Map: Greece Debt

Game Plan


Evaluation

• User study– Document selection: capturing important content?– Micro-knowledge: question-answering– Macro-knowledge: high-level summaries– Effect of structure

• New York Times (2008-2010)– 18K+ articles– Chile, Haiti, Greece

Document Selection• Experts compose a list of important events• Subtopic recall (% of events in the map):

# lines

Subtopicrecall

Micro-Knowledge (Question Answering)

• Mechanical Turk

• Competitors:– Google News– Event threading (TDT) [Nallapati et al, 04]

– Structureless maps• Results: minor gains– map structure helps

Question 2: How many miners were trapped?

Macro-Knowledge(High-Level Summaries)

• Summarize complex story in a paragraph– Maps vs. Google News– ~15 paragraphs per task

• Mturk to evaluate paragraphs:– Which paragraph provided a more complete and

coherent picture of the story?– Justification: Paragraph A is more… – ~300 evaluations per task

Macro-Knowledge: Results

• Greece: 72% prefer maps– Justifications:

• Haiti: 59% prefer maps– Map users mostly summarized one story line

MapsGoogle News

Bottom line: maps are more useful as high-level tools for stories without a single dominant storyline

Conclusions• Formulated metrics characterizing good maps• Efficient methods with theoretical guarantees• User studies highlight the promise of the method• Website on the way!• Personalization

Thank you!

Finding Coherent Chains

• Goal: represent all coherent chains• Problem: intractable

• Divide and conquer:– Find short coherent chains– Concatenate to form longer coherent chains

Website