Upload
alice-oh
View
3.060
Download
0
Tags:
Embed Size (px)
Citation preview
Topic Models & Computational Social Science
October 17, 2013Alice [email protected]@seas.harvard.eduhttp://uilab.kaist.ac.kr/members/aliceoh/
Thursday, October 17, 2013
What is topic modeling?
Thursday, October 17, 2013
Blei, Communications of the ACM, 2012
Thursday, October 17, 2013
Motivation
Thursday, October 17, 2013
Motivation
• What are the topics discussed in the article?
• Is the article related to
• household finances?
• price of gasoline?
• price of Apple stock?
• How would you build an automatic system for answering these questions?
Thursday, October 17, 2013
http://www.nytimes.com/2010/08/09/sports/autoracing/09nascar.html?hp
nascar, races, track, raceway, race, cars, fuel, auto, racing
economic, slowdown, sales, recession, costs, spending, save
fans, spectators, sports, leagues, teams, competition6
Thursday, October 17, 2013
nascar, races, track, raceway, race, cars, fuel, auto, racing
economic, slowdown, sales, recession, costs, spending, save
fans, spectators, sports, leagues, teams, competition
Topics: multinomial over wordsThursday, October 17, 2013
nascar, races, track, raceway, race, cars, fuel, auto, racing
economic, slowdown, sales, recession, costs, spending, save
fans, spectators, sports, leagues, teams, competition
Topics: multinomial over wordsTopic DistributionsThursday, October 17, 2013
http://www.nytimes.com/2010/08/09/sports/autoracing/09nascar.html?
nascar, races, track, raceway, race, cars, fuel, auto, racing
economic, slowdown, sales, recession, costs, spending, save
fans, spectators, sports, leagues, teams, competition
Topics: multinomial over wordsTopic DistributionsThursday, October 17, 2013
http://www.nytimes.com/2010/08/09/sports/autoracing/09nascar.html?
nascar, races, track, raceway, race, cars, fuel, auto, racing
economic, slowdown, sales, recession, costs, spending, save
fans, spectators, sports, leagues, teams, competition
Topics: multinomial over wordsTopic DistributionsThursday, October 17, 2013
http://www.nytimes.com/2010/08/09/sports/autoracing/09nascar.html?
nascar, races, track, raceway, race, cars, fuel, auto, racing
economic, slowdown, sales, recession, costs, spending, save
fans, spectators, sports, leagues, teams, competition
Topics: multinomial over wordsTopic DistributionsThursday, October 17, 2013
Input to LDA
8
Thursday, October 17, 2013
Input to LDA
8
http://www.nytimes.com/2010/08/09/sports/autoracing/09nascar.html?
Thursday, October 17, 2013
Topics Discovered by LDA
nascar 0.12 spending 0.09 sports 0.12
races 0.10 economic 0.07 team 0.11
cars 0.10 recession 0.06 game 0.10
racing 0.09 save 0.05 player 0.10
track 0.08 money 0.05 athlete 0.09
speed 0.06 cut 0.04 win 0.07
... ... ...
money 0.002 speed 0.003 nascar 0.001
Topics: multinomial over vocabulary9
Thursday, October 17, 2013
Graphical View
10
Thursday, October 17, 2013
Graphical View
sales xxx slowdown recession cars races spending xxx save costs fuel
10
Observed
Thursday, October 17, 2013
Graphical View
Topics
sales xxx slowdown recession cars races spending xxx save costs fuel
10
Observed
nascar, races, track, raceway, race, cars, fuel, auto, racing
economic, slowdown, sales, recession, costs, spending, save
fans, spectators, sports, leagues, teams, competition
Topics: multinomial over words
Discovered
Topic Distributions
Discovered
Thursday, October 17, 2013
Do you feel what I feel?Social Aspects of Emotions in Twitter Conversations
Suin Kim, JinYeong Bak, Alice OhICWSM 2012
11
Thursday, October 17, 2013
Twitter conversation data
• Twitter conversation data: approx 220k dyads who “reply” to each other, 1,670k conversational chains (We now have about 5x this amount)
!"!
#!
$!
%!
Thursday, October 17, 2013
Asking Research Questions
13
Thursday, October 17, 2013
Asking Research Questions
13
Thursday, October 17, 2013
Asking Research Questions
Human emotion is typically studied as a within-person, one-direction, non-repetitive phenomenon; focus has traditionally been on how one individual feels in reaction to various stimuli at a certain point of time. But people recognize and inevitably react emotionally and otherwise to expressions of emotion of other people. We propose that organizational dyads and groups inhabit emotion cycles: Emotions of an individual influence the emotions, thoughts and behaviors of others; others’ reactions can then influence their future interactions with the individual expressing the original emotion, as well as that individual’s future emotions and behaviors. People can mimic the emotions of others, thereby extending the social presence of a specific emotion, but can also respond to others’ emotions, extending the range of emotions present.
14
Thursday, October 17, 2013
Topic model with a twist
• Dirichlet forest prior (Andrzejewski et al.)
• Mixture of Dirichlet tree distribution
• Dirichlet tree: Generalization of Dirichlet distribution
• Knowledge is expressed using Must-link and Cannot-link primitives
• Must-link(love, sweetheart)
• Cannot-link(exciting, bored)
15DF-LDA
Thursday, October 17, 2013
Topic model with a twist
• Dirichlet forest prior (Andrzejewski et al.)
• Mixture of Dirichlet tree distribution
• Dirichlet tree: Generalization of Dirichlet distribution
• Knowledge is expressed using Must-link and Cannot-link primitives
• Must-link(love, sweetheart)
• Cannot-link(exciting, bored)
15
qβ
η
DF-LDA
Thursday, October 17, 2013
Domain knowledge in Dirichlet forest prior
16
Seed Words
anticipationhopewaitawaitinspirexcitborereadiexpectnervoucalmmotivpreparcertainanxiouoptimistforese
joyawesomamazwonderexcitgladfinebeautihighluckisuperperfectcompletspecialblesssafeproud
angershitbitchassmeandamnmadjealoupissannoiangriupsetmoronragescrewstuckirrit
surpriseamazwowwonderweirdluckidiffer
awkwardconfusholistrangshockodd
embarrassoverwhelmastoundastonish
fearscarestresshorrornervouterroralarmbehindpanicfearafraiddesperthreatentensterrififrightanxiou
sadnesssorribadawsadwronghurtbluedeadlostcrushweakdepressworslowterribllone
disgustsickwrongevilfatuglihorriblgrossterriblselfishmiserpathetdisgustworthlessaw
ashamfuck
acceptanceokaioksamealrightsafelazirelaxpeaccontentnormalsecurcompletnumbfulfil
comfortdefeat
Must-link within a class Cannot-link between classes
Thursday, October 17, 2013
Emotion Topics How do we express emotions?
JoyAnticipation AngerTopic 114omglovehahathankreallyTopic 107lovethankfollowwow
Topic 159gooddayhopemorningthankTopic 158lovethankmisshug
Topic 125hopebetterfeelthanksoonTopic 26goodthankhopemiss
Topic 146comewaitweekdayjuneTopic 146gooddaytimework
Topic 131lmaofuckassbitchshitTopic 4assyolmaonigga
Topic 19lmaoshitdamnfuckohTopic 13shitniggasmhyea
FearTopic 48omgohlmaoshitscareTopic 78happenheartattackhospital
Topic 27don’tcomenightsleepoutsideTopic 140timegotworkday
SurpriseTopic 172yeagknowthinktruefunnyTopic 89knowdon’tthinklook
Topic 15thinkdon’tknowmakereallyTopic 94hahadontthinkreally
29 70 21 14 5
Sadness DisgustTopic 6ohsorryhahaknowdidntTopic 59hurtgotgoodbad
Topic 106tweetreplydidn’treadsorryTopic 155ohreallymakefeel
Topic 116ohfuckdon’tyeewTopic 116lookhahaohknow
Topic 22don’tohthinkyeahlmaoTopic 174don’tthinksaypeople
AcceptanceTopic 43okohthankcoolokayTopic 102knowtryletok
Topic 199xxthankgoodokayfollowTopic 8nightlovegoodsleep
17 7 18 NeutralTopic 180comwwwhttpcheckyoutubeTopic 156twitterfacebookpeopleaccount
Topic 184accountgoogleappworkemailTopic 67foodchickencookrt
19
17
Thursday, October 17, 2013
Emotion Topics How do we express emotions?
JoyAnticipationTopic 114omglovehahathankreallyTopic 107lovethankfollowwow
Topic 125hopebetterfeelthanksoonTopic 26goodthankhopemiss
SadnessTopic 6ohsorryhahaknowdidntTopic 59hurtgotgoodbad
NeutralTopic 180comwwwhttpcheckyoutubeTopic 156twitterfacebookpeopleaccount
GreetingCaring Sympathy IT/Tech
18
Thursday, October 17, 2013
Emotion-tagged conversations
19
A (Love): @amithpr @dhempe @OperaIndia - Would you have any update on @mrunmaiy's health - hope she is recovering well?B (neut): @labnol @dhempe she is recovering but slow. The injury is on the spine therefore worrisome. Still in icu.A (Sadness): @amithpr thanks for the update.. extremely said to hear that news..B (neut): @labnol #prayformrun She is a fighter and will come out of this
B (neut): @AyeItsMeiMei just tell ur followers to report her for spam. then she'll be kicked off twitterA (Anger): @Jakeosaurous dude I didn't even do shit to her I'm just here tweeting & she calls me a ugly bitch? I was like oh wow thanks?B (neut): @AyeItsMeiMei yeah clearly shes so ugly she cant even use her real pic:P so dont feel badA (Love): @Jakeosaurous haha. I don't care. She's getting spammed with hate. Hahaha. (": thanks though.B (neut): @AyeItsMeiMei np
Thursday, October 17, 2013
Emotion Transitions Plutchik’s Wheel of Emotions
Joy39.7%
0.51
Acceptance10.4%
0.23
Fear2.6%
0.11
Surprise7.4%
0.17
Anticipation15.1%
0.26
Disgust2.9%
0.11
Sadness9.1%
0.19
0.31Anger12.8%
0.37
0.33
0.32
0.31
0.33
0.21
0.34
0.15
0.140.13
0.15
20
Thursday, October 17, 2013
Defining “Influence”
User A
User B
Having a tough day today. RIP Harrison. I’ll
miss you a ton :/
Just pray about it. God will help you.
Not really religious, but thanks man. :)
If you need talk you know I’m here.
Time
(Sadness) (Acceptance)
(Anticipation)
21
Thursday, October 17, 2013
Defining “Influence”
emotion influencing tweet
User A
User B
Having a tough day today. RIP Harrison. I’ll
miss you a ton :/
Just pray about it. God will help you.
Not really religious, but thanks man. :)
If you need talk you know I’m here.
Time
(Sadness) (Acceptance)
(Anticipation)
21
Thursday, October 17, 2013
Topic 117tweetpeopledon’treadpostTopic 59hurtgotbadpainfeel
Emotion Influences What can you say to make your partner feel better?
Joy → SadnessSadness → Joy
Topic 18wearlookthinkloveblackTopic 24lovethankgreatnewlook
Anticipation → Surprise
Topic 96musiclistenplaysonggoodTopic 178followtweetpeopletwitterthank
Acceptance → Anger
Topic 31i’mgotlmaxshitdaTopic 13lmaoshitniggasmhyea
Disgust → Joy
Topic 61watchnewlivetvtonightTopic 63watchgoodthinkknowlook
Suggesting GreetingSympathy
Swear words Complaining
22
Thursday, October 17, 2013
Self-disclosure and relationship strength in online conversations
JinYeong Bak, Suin Kim, and Alice OhACL 2012
23
Thursday, October 17, 2013
2012-07-11
Methodology} Twitter Data} 131K users } 2M conversations
} Relationship Strength} Chain frequency (CF)} Chain length (CL)
} Self-Disclosure} Personal information} Open communication} Profanity
} Analysis with Topic Models} Latent Dirichlet allocation (LDA, [Blei, JMLR 2003])} Aspect and sentiment unification model (ASUM, [Jo, WSDM 2011])
24
Thursday, October 17, 2013
2012-07-11
Relationship Strength} Social psychology literature states relationship strength can be
measured by communication frequency and length [Granovetter, 1973;
Levin and Cross, 2004]} CF: chain frequency} The number of conversational chains between the dyad
averaged per month} CL: chain length} The length of conversational chains between the dyad
averaged per month} Relationship strength} A high CF or CL for a dyad means the relationship is strong} A low CF or CL for a dyad means the relationship is weak
25
Thursday, October 17, 2013
2012-07-11
Self-Disclosure} Open communication - Openness} Negative openness} Nonverbal openness} Emotional openness} Receptive openness – difficult to find in tweets} General-style openness – not clearly defined in the literature
} Personal Information} Personally Identifiable Information (PII)} Personally Embarrassing Information (PEI)
} Profanity} nigga, ass, wtf, lmao
26
Thursday, October 17, 2013
2012-07-11
Negative openness
} Method} We use ASUM with emoticons as seed words
[ “Aspect and sentiment unification model for online review analysis”, Jo, WSDM’11]} ASUM is LDA-based joint model of topic and sentiment} ASUM takes unannotated data and classifies each sentence (tweet) as
positive/negative/neutral
Self-Disclosure - Openness
27
Thursday, October 17, 2013
2012-07-11
Self-Disclosure - OpennessNonverbal openness
} Method} We look for emoticons, ‘lol’, ‘xxx’} Emoticons are like facial expressions -- :) :( :P} ‘lol’ (laughing out loud) and ‘xxx’ (kisses) are very frequently used in a
similar manner to nonverbal openness
28
Thursday, October 17, 2013
2012-07-11
Self-Disclosure - OpennessEmotional openness
} Method} Look for tweets that contain common expressions of feeling words
[We feel fine (Harris, J, 2009)]
29
Thursday, October 17, 2013
2012-07-11
Self-Disclosure – Personal InformationPersonally Identifiable Information (PII)
Personally Embarrassing Information (PEI)
30
Ex) name, location, email address, job,social security number
Ex) clinical history,sexual life,job loss, family problem
Thursday, October 17, 2013
2012-07-11
Self-Disclosure – Personal Information}
31
Thursday, October 17, 2013
2012-07-11
Self-Disclosure – Personal InformationExample of PII, PEI and Profanity topics } Shown by high probability words in each topic
PII 1 PII 2 PEI 1 PEI 2 PEI 3 Profanity
san tonight pants teeth family nigga
live time wear doctor brother lmao
state tomorrow boobs dr sister shit
texas good naked dentist uncle ass
south ill wearing tooth cousin bitch
32
Thursday, October 17, 2013
2012-07-11
Results
Thursday, October 17, 2013
2012-07-1134
weak ßà strong weak ßà strong
weak ßà strong weak ßà strong
sentiment nonverbal emotional profanity PII & PEI
Thursday, October 17, 2013
2012-07-1135
weak ßà strong
weak ßà strong
emotional PII & PEI
weak ßà strong
weak ßà strong
Thursday, October 17, 2013
2012-07-11
Results: Interpretation} Emotional openness} When they are not very close, they express frequent encouragements,
or polite reactions to baby or pets
36
Thursday, October 17, 2013
2012-07-11
Results: Interpretation} PII} When they meet new acquaintances, they use PII to introduce
themselves
37
Thursday, October 17, 2013
2012-07-11
ResultsAnalyzing outliers: a dyad linked weakly but shows high self-disclosure
38
Thursday, October 17, 2013
Computational Analysis of Agenda Setting Theory
Yeooul Kim and Alice [email protected]
Thursday, October 17, 2013
Agenda Setting Theory How does media affect the thoughts of the audience?
Thursday, October 17, 2013
Agenda Setting Theory (McCombs & Shaw, 1972)
• Media affects audiences by having an influence on
• What to think about
• How to think about it
• Examples of traditional media studies
• Media affects the outcome of presidential elections (Perloff and Krauss, 1985)
• Media coverage influences the control of infectious diseases (Cui et al., 2008)
• Tone of news articles affects the number of visitors to museums (Zyglidopoulos et al., 2012)
Thursday, October 17, 2013
1.Use of traditional off-line newspapers and TV as target media
• Analysis is limited to a small volume over a short duration
• Issues are arbitrarily chosen
2.Use of off-line MIP (Most Important Problems) surveys
• Self-reports are not reliable
• Only a small subset of the population can be surveyed
3.Use of manual coding for content analysis
• You need experts
• It is difficult to replicate and generalize to other domains
Limitation of Traditional Media Studies
Thursday, October 17, 2013
Computational Analysis of Agenda Setting Theory
1.Use of traditional off-line newspapers and TV as target media
• Crawl online news to get several years’ data
• Use machine learning to automatically discover the important issues
2.Use of off-line MIP (Most Important Problems) surveys
• Look at counts of social media shares
• Look at counts of user comments
3.Use of manual coding for content analysis
• Use unsupervised machine learning to analyze content for tone (polarity) of articles and comments
• Try it for different issues to see whether ML approach can generalize over many domains
Thursday, October 17, 2013
44
Gay marriageCOMMENT
SHARE
AUDIENCE’S BEHAVIOR
Thursday, October 17, 2013
44
Gay marriageCOMMENT
SHARE
AUDIENCE’S BEHAVIOR
Thursday, October 17, 2013
45
Section #Articles #Comments #Commenters #Shares
Politics 1,863 174,680 14,106 2,080,889
Business 2,043 130,921 17,791 3,657,544
Opinion 4,820 149,618 30,556 6,620,489
Sports 814 17,282 5,484 712,507
Technology 456 13,571 4,993 570,732
Science 945 50,113 11,114 4,709,041
World 3,673 134,572 14,882 3,534,637
Health 3,060 92,964 18,185 6,001,082
Total 17,674 763,721 117,111 27,886,921
From http://www.npr.org/
2011.01 – 2013.04
DATA STATISTICS
Thursday, October 17, 2013
46
Section Issue (Labeled by using Mturk) #Articles
Politics presidential electioninfringement of human rightsrace for Washingtongovernment economics presidential campaigns and money candidate-marriage & immigration political viewpoints
575195167274163261157
Business economic decline under Obamaemployment and paid slavery agriculturebanks and loan stock market and business housing markettax and businessenergy and finance new business and running
514218131198166170180222138
Health health care reform laws vaccinationHIV and treatment medication healthcare and costs food and obesitysleep study and children food and safety health tech and new treatment mental health in families
349189496197224245210223125117
Issue Detection using HDP
Detected Issue list and the number of articles of each issue for three sections out of eight sections.
Thursday, October 17, 2013
47
▶ Effects from media exposure CORRELATION IN ISSUE
Thursday, October 17, 2013
Contentious Issues
48
Thursday, October 17, 2013
Contentious Issues
49
Thursday, October 17, 2013
INFLUENTIAL FACTOR Tone (Polarity) of article
GOALIdentify the effects of article tone, positive and negative, on the commenting and sharing behaviors of the audience
50
Content Polarity & Audience Behavior
Thursday, October 17, 2013
51
ARTICLE POLARITY
Thursday, October 17, 2013
52
DETECTED POS./NEG. WORDS
The sets of positive and negative words obtained from model analysis for news articles. Words depending on sections differentiate positive and negative traits of each section.
BUSINESS HEALTH OPINION POLITICS Positive joined viral smoothly better balance respect forward empower fair moderate
Negative cutthroat axed lawsuit beating lose opposite battle unjust fuming sequester
Positive care respect admit clarify essential healthy repair benign hope repaired
Negative tough severe emergency affected risk dying war spitting tricks abnormal
Positive spectacular useful created prize confirm love sublime win confident mellow
Negative weird fog distressing slam doubted fail wrong fears slippery peril
Positive expert forward proud consent carol rights great worth integrity truth
Negative ironic heinous arguing dick undo grinding outlaw meaningless theft lost
SCIENCE SPORTS TECHNOLOGY WORLD Positive fortunate cleanup essential credit safety comforting milestone learn gang dim
Negative spill crude busted upset concern problems dark smash prize creating
Positive victory won grace fun champion passion ace belief luck balance
Negative chase shock busted beating defeat thwart lost alleged assault cockeyed
Positive best fancy easy help intelligence strong improve fit trust fame
Negative blocks shabby shy wicked rash shaky mortal grave pity unfinished
Positive free respected support moderate consistent prompt afford gratitude joined affluent
Negative tension protest heavy raging slam war crime oppress poverty poor
Thursday, October 17, 2013
53
Positive and Negative Articles
Thursday, October 17, 2013
For more information
David Blei’s homepage:h2p://www.cs.princeton.edu/~blei/
David Mimno’s bibliography:h2p://www.cs.princeton.edu/~mimno/topics.html
videolectures.net – David Blei, Yee-‐Whye Teh, Michael JordanConferences: NIPS, ICML, UAI, ECML, KDD, EMNLP
Tools: Mallet, GenSym, various LDA libraries
Email me: [email protected]
Thursday, October 17, 2013