Linking Born-Digital News and Social Media Collections via Automated Entity Detection and Authority Matching

@peterbroadwell and @mart1nkle1nLinking Born-Digital News and Social Media Collections via

Automated Entity Detection and Authority MatchingDLF Forum, Vancouver, 28 October 2015

Linking Born-Digital News and Social Media Collections via Automated Entity Detection and Authority Matching

DLF Forum, Vancouver, 28 October 2015

Linking Born-Digital News and Social Media Collections

via Automated Entity Detection and

Authority Matching

Peter Broadwell@peterbroadwell

Martin Klein@mart1nkle1n

University of California Los AngelesResearch Library



2

• Collected by researchers• Donated by activists• Diverse in format:

• Images, audio, video, scanned documents, social media, web server logs

Collections related to news events



3

• 829 digitally recorded Iranian dissident news programs• 9,166 other videos from the Iranian Green Movement• 29,441 digital photographs from the Green Movement• 543 documents from Tahrir Square

International Digitizing Ephemera

http://digital.library.ucla.edu/dep/



4

• Tahrir Square Egypt & Libya unrest, 2011• Tōhoku earthquake and tsunami, Japan, 2011• AirAsia 8501 crash, December 2014• Charlie Hebdo shooting, January 2015

International Digitizing Ephemera – Tweets



5

• Tahrir Square Egypt & Libya unrest, 2011• Tōhoku earthquake and tsunami, Japan, 2011• AirAsia 8501 crash, December 2014• Charlie Hebdo shooting, January 2015

International Digitizing Ephemera – Tweets

Social Feed Managerhttp://social-feed-manager.readthedocs.org/



6

NewsScape

• 264,000 hours of TV news archived digitally• Recorded 2005-present, ca. 100 shows/day• 13 countries, 9 languages• 38 networks• Searchable by captions, on-screen text, official transcripts



7

NewsScape

• 264,000 hours of TV news archived digitally• Recorded 2005-present, ca. 100 shows/day• 13 countries, 9 languages• 38 networks• Searchable by captions, on-screen text, official transcripts



8

NewsScape



9

Social Local Global



10

Linking social media, TV news, and web news



11

Linking social media, TV news, and web newsCollection on AirAsia QZ8501 crash on 12/28/2014, recorded TV and social media through 1/17/2015• 7.3 million tweets containing #AirAsia or #QZ8501• 1.3 million distinct users• 262 distinct television recordings• 1,535 on-air mentions of AirAsia or [QZ]8501• ~3,000 on-screen appearances of AirAsia or [QZ]8501



12

Linking social media, TV news, and web newsCollection on AirAsia QZ8501 crash on 12/28/2014, recorded TV and social media through 1/17/2015• 7.3 million tweets containing #AirAsia or #QZ8501• 1.3 million distinct users• 262 distinct television recordings• 1,535 on-air mentions of AirAsia or [QZ]8501• ~3,000 on-screen appearances of AirAsia or [QZ]8501



13

Linking via Automated Entity Detection

• Discover and highlight commonalities and relationships between disjoint collections on related news events• Link to authorities• Address problem of disambiguation

• Establish workflow for automatic linking • Integration with search and discovery interfaces• Exposure via APIs



14

CNN09/16/201505:22pm



15

CNN09/16/201505:22pm

Twitter09/16/2015

06:22pm



16



17



18

Experiment 1/3

• Apply DBpedia Spotlight Named Entity Recognition (NER) software to collections on second GOP presidential primary debate on 09/16/2015• Twitter: 800,000 tweets• TV: CNN coverage of debate• Minute granularity• Persons, Organizations, Places

Results:• Linked entities with URIs to DBpedia resources• Visualization of correlations between entities



19

Experiment 1/3 - Persons



20

Experiment 1/3 - Places



21

Experiment 1/3 - Organizations



22

Experiment 2/3

• Expanded range of TV news coverage up to 4 days after the debate on 17 local, U.S., and international channels

Results:• Discovery of related news shows by matching terms

and entities from Twitter• Visualization highlighting degree of relationships



23

Experiment 2/3 – Terms matched



24

Experiment 2/3 – Persons matched



25

Experiment 3/3

• Automatic geocoding of extracted place names from Twitter and CNN coverage

Results:• Using geographical proximity to explore potentially

relevant correlations • Visualization of places/regions and their frequency of

reference in each collection



26

Experiment 3/3 – Twitter



27

Experiment 3/3 – NewsScape



28

Next steps

• Apply techniques to other DL collections• Explore special domain customization of NER tools and

authority matching• Investigate methods to quantify collection overlap• Incorporate more linked open data ontologies• Improve support of other languages



Linking Born-Digital News and Social Media Collections via Automated Entity Detection and Authority Matching

DLF Forum, Vancouver, 28 October 2015

Linking Born-Digital News and Social Media Collections

via Automated Entity Detection and

Authority Matching

Peter Broadwell@peterbroadwell

Martin Klein@mart1nkle1n

University of California Los AngelesResearch Library

Data & Analytics

Linking Born-Digital News and Social Media Collections via Automated Entity Detection and Authority Matching