Crisis Computing

Crisis ComputingFinding relevant and credible information on social media during disasters

Big Data Analytics ConferenceDelhi, India, December 2014

January 2010How/when did it start for me?

Humanitarian ComputingAt least 775 publications:

● Crisis Analysis (55)

● Crisis Management (309)

● Situational Awareness (67)

● Social Media (231)

● Mobile Phones (74)

● Crowdsourcing (116)

● Software and Tools (97)

● Human-Computer Interaction (28)

● Natural Language Processing (33)

● Trust and Security (33)

● Geographical Analysis (53)

Source: http://humanitariancomp.referata.com/

Humanitarian Computing Topics

http://www.youtube.com/watch?v=0UFsJhYBxzY

Carlos Castillo – chato@acm.orghttp://www.chato.cl/research/

An earthquake hits a Twitter user

• When an earthquake strikes, the first tweets are posted 20-30 seconds later

• Damaging seismic waves travel at 3-5 km/s, while network communications are light speed on fiber/copper + latency

• After ~100km seismic waves may be overtaken by tweets about them

http://xkcd.com/723/

Examples of crisis tweets

Alexandra Olteanu, Sarah Vieweg and Carlos Castillo: What to Expect When the Unexpected Happens: Social Media Communications Across Crises.To appear in CSCW 2015.

Examples of crisis tweets (cont.)

Fertile grounds for applied research✔ Problems of global significance✔ Solved with labor-intensive methods✔ Better solution provides a public good✔ Large and noisy data sets available✔ Engage volunteer communities

Fertile grounds for applied research✔ Problems of global significance✔ Solved with labor-intensive methods✔ Better solution provides a public good✔ Large and noisy data sets available✔ Engage volunteer communities• Relevance to practitioners?

Current collaboratorsPatrick Meier– QCRI

Sarah Vieweg– QCRI

Muhammad Imran– QCRI

Irina Temnikova– QCRI

Alexandra Olteanu– EPFL

Aditi Gupta– IIIT Delhi

“P.K.” Kumaraguru– IIIT Delhi

Fernando Diaz– Microsoft

Outline

Crisis MapsExtractionMatching

VerificationCredibility

Crisis maps from social mediaCarlos Castillo, Fernando Diaz, and Hemant Purohit:Leveraging Social Media and Web of Data to Assist Crisis Response CoordinationTutorial at SDM, Philadelphia, PA, USA. April 2014.

Hemant Purohit, Carlos Castillo, Patrick Meier and Amit Sheth: Crisis Mapping, Citizen Sensing and Social Media AnalyticsTutorial at ICWSM, May 2013.

Patrick Meier, Social Innovation Director @ QCRI – http://irevolution.net/

“What can speed humanitarian

response to tsunami-ravaged

coasts? Expose human rights

atrocities? Launch helicopters to

rescue earthquake victims?

Outwit corrupt regimes?

A map.”

Crisis mapping goes mainstream (2011)

http://newsbeatsocial.com/watch/0_s6xxcr3p

Understanding Crisis TweetsAlexandra Olteanu, Sarah Vieweg and Carlos Castillo: What to Expect When the Unexpected Happens: Social Media Communications Across Crises.To appear in CSCW 2015.

Types of Disaster

Extraction

Our approach

Classification1.

Filtering

Is disaster-related?

Contributes tosituational

awareness?

Yes Yes

ClassificationCaution &

AdviceInformation

SourcesDamage &Casualties Donations

Eyewitness

Outsider

Filteredtweets

A large-scale study of crisis tweets

• Collect tweets from 26 disasters• Classify according to:

● Informative / Not informative● Information provided● Information source

Advice on labeling

• Your instructions will never be correct the first time you try– e.g. personal / eyewitness– Instructions must be re-written reactively– Perform small-scale labeling first

• Instructions must be concrete and brief– If you can't do it, the task has to be divided

Information Provided in Crisis Tweets

N=26; Data available at http://crisislex.org/

What do people tweet about?• Affected individuals

– 20% on average (min. 5%, max. 57%)– most prevalent in human-induced, focalized & instantaneous events

• Sympathy and emotional support– 20% on average (min. 3%, max. 52%)– most prevalent in instantaneous events

• Other useful information– 32% on average (min. 7%, max. 59%)– least prevalent in diffused events

What do people tweet about? (cont.)• Infrastructure and utilities

– 7% on average (min. 0%, max. 22%)– most prevalent in diffused events, in particular floods

• Caution and advice– 10% on average (min. 0%, max. 34%)– least prevalent in instantaneous & human-induced events

• Donations and volunteering– 10% on average (min. 0%, max. 44%)– most prevalent in natural hazards

Distribution over information sources

Distribution over time

Extracting information and matching emergency-related resourcesMuhammad Imran, Shady Elbassuoni, Carlos Castillo, Fernando Diaz and Patrick Meier: Extracting Information Nuggets from Disaster-Related Messages in Social MediaIn ISCRAM. Baden-Baden, Germany, 2013. Best paper award.

Hemant Purohit, Amit Sheth, Carlos Castillo, Patrick Meier, Fernando Diaz:Emergency-Relief Coord. on Social Media: Auto. Matching Resource Requests and OffersFirst Monday 19 (1), January 2014

Muhammad Imran, Shady Elbassuoni, Carlos Castillo, Fernando Diaz and Patrick Meier: Practical Extraction of Disaster-Relevant Information from Social MediaIn SWDM. Rio de Janeiro, Brazil, 2013

Information Extraction

Classifiedtweets

@JimFreund: Apparently we have no choice.

There is a tornado watch in effect

tonight.

Extraction

• #hashtags, @user mentions, URLs, etc.– Regular expressions– Text library from Twitter

• Temporal expressions– Part-of-speech tagger + heuristics– Natty library

• Supervised learning

Labels for extraction• Type-dependent instruction• Ask evaluators to copy-paste a word/phrase from

each tweet

Learning: Conditional Random Fields

• Used extensively in NLP for part-of-speech tagging and information extraction

• Representation of observations is important (capitalization, position, etc.)

HMM Linear-chain CRF

hidden

observed

• CMU ARK Twitter NLP– Tokenization– Feature extraction– CRF learning

• Very easy to use: simply change the training set (part-of-speech tags) into anything, and re-train

Output examplesRT @weatherchannel: .@NYGovCuomo orders closing of NYC bridges. Only Staten Island bridges unaffected at this time. Bridges must close by 7pm. #Sandy #NYC

Wow what a mess #Sandy has made. Be sure to check on the elderly and homeless please! Thoughts and prayers to all affected

RT @twc_hurricane: Wind gusts over 60 mph are being reported at Central Park and JFK airport in #NYC this hour. #Sandy

RT @mitchellreports: Red Cross tells us grateful for Romney donation but prefer people send money or donate blood dont collect goods NOT best way to help #Sandy

Extractor evaluationSetting Rec Prec

Train 2/3 Joplin, Test 1/3 Joplin 78% 90%

Train 2/3 Sandy, Test 1/3 Sandy 41% 79%

Train Joplin, Test Sandy 11% 78%

Train Joplin + 10% Sandy, Test 90% Sandy 21% 81%

• Precision is: one word or more in common with what humans extracted

Donations matching• Identify and match requests/offers for donations

– Money, clothing, food, shelter, volunteers, blood

Average precision = 0.21 (0.16 if only text similarity is used)

Crowdsourced stream processing systemsMuhammad Imran, Ioanna Lykourentzou and Carlos Castillo: Engineering Crowdsourced Stream Processing Systemshttp://arxiv.org/abs/1310.5463

Design objectives and principlesDesign principles

Design objective Example metric Automatic components

Crowdsourced components

Low latency End-to-end time Keep-items moving Trivial tasks

High throughput Output items per unit of time

High-performance processing

Task automation

Load adaptability Rate response function

Load shedding, load queueing

Task prioritization

Cost effectiveness Cost vs. quality, throughput, etc.

N/A Task frugality

High quality Application-dependent

Redudancy, aggregation and quality control

Design patterns● QA loop

● Task assignment

● Process/verify

● Supervised learning

● Crowdwork sub-task chaining

● Humans are not a bottleneck

● Humans review every output element

http://aidr.qcri.org/

Self-service for crisis-related classification

Unstructuredtext reports

Categorizedinformation

Automaticclassifier

ModelBuilder

Crowdsourcedground-truth

Library of training data

Credibility and verificationAditi Gupta, Ponnurangam Kumaraguru, Carlos Castillo and Patrick Meier:TweetCred: A Real-time Web-based System for Credibility of Content on TwitterIn SocInfo 2014. Runner-up for best paper award.

Carlos Castillo, Marcelo Mendoza, Barbara Poblete:Predicting Information Credibility in Time-Sensitive Social MediaIn Internet Research, Vol. 23, Issue 5. October 2013.

A. Popoola, D. Krasnoshtan, A. Toth, V. Naroditskiy, C. Castillo, P. Meier and I. Rahwan:Information Verification during Natural DisastersSocial Web and Disaster Management (SWDM) workshop, 2013.

http://www.youtube.com/watch?v=pAHoEO-K0Ek

Crowdsourced verification: Veri.ly

• Frame crowdwork correctly• Not upvoting/downvoting a claim• Instead, providing evidence for/against

@VeriDotLy — http://veri.ly/

Examples of evidence provided

Automatic credibility evaluation: TweetCred

• Real-time web-based service• Used as a Chrome extension• Annotates Twitter's timeline with credibility

scores

http://twitdigest.iiitd.edu.in/TweetCred/

Next steps

• Credibility facets– Factually written– Detailed– Author on the ground– ...

• Respond to searches about an event

Closing remarks

Computationally feasible

Supported bydata

Useful

Good projects in this space

Computationally feasible

Supported bydata

Useful

Good projects in this space

Temptation! Danger!

Poorly planned projects :-(

AI-complete problems

Some venues

• SWDM – Workshop on Social Webfor Disaster Management– Deadline: January 24th

• ISCRAM – International Conference on Information Systems for Crisis Response and Management

+ the usual suspects, depending on your area ;-)

Possibility of large impact by using computer science to support humanitarian work

=Applied computing at its best

Thank you!Carlos Castillo · chato@acm.org

http://www.chato.cl/research/With thanks to Patrick Meier for several slides

Crisis Computing

Social Media

The Current Crisis in Computing: What are the Real Issues?

Pakistan’s Energy Sector: From Crisis to Crisis – Breaking ...tcpbckup1.yolasite.com/resources/Pakistan Energy Sector From Crisis... · From Crisis to Crisis – Breaking the

CRISIS (Seri Management Crisis)

The Opioid Crisis as Health Crisis, Not Criminal Crisis

Crisis: what Crisis?

Supertramp - Crisis What Crisis (Book)

Crisis management - Simulation Crisis - 4ward

The Pipeline Crisis in Computing Eric Roberts Professor of Computer Science, Stanford University Co-chair of the ACM Education Board Taking the Initiative

USA Crisis & Euro Crisis

Crisis and Crisis Intervention

Tourism Crisis CommuniCaTion ToolkiT - Impartmedia · Crisis communication toolkit purpose 4 RTO crisis role 5 Pre-crisis preparation steps6 Crisis response steps15 Post-crisis recovery

Stages of Crisis & Crisis Management

The Pipeline Crisis in Computing

Social Computing and Crisis in the New Information Age - ISC Intelligence in Science · 2013-11-14 · Social Computing and Crisis in the New Information Age 10 ISC ǀ Brussels, October

Pakistan’s Energy Sector: From Crisis to Crisis – Breaking ... · PDF fileFrom Crisis to Crisis – Breaking the Chain ... PAKISTAN’S ENERGY SECTOR: FROM CRISIS TO CRISIS—

Crisis in Crisis

Creativity Crisis, Crisis Creativity

Handbook crisis management crisis communication terrorist ......crisis management, crisis communication, using Crisis Management Team (CMT) structures effectively, and providing sufficient

Crisis Management Crisis Management Goals -

Crisis? What Crisis?_v.Eng