Upload
emerce
View
1.217
Download
1
Embed Size (px)
Citation preview
Introduction Methods Results Ongoing research: Mood contagion Conclusions
Twitter sentiment voorspelt aandelenkoersen
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot
Smith, Peter Todd, and Ishani Banerji
[email protected] of Informatics and Computing
Center for Complex Networks and Systems ResearchIndiana University
February 9, 2011
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion Conclusions
Objective
Public mood states and the markets
Do societies experience varying mood states like individuals?If so, can we assess such mood states from online materials anddetermine its socio-economic correlates?
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion Conclusions
Outline
1 IntroductionCollective Intelligence: from mobs to crowdsSentiment analysis: from mood to behavior
2 MethodsDataSentiment tracking instrument
3 ResultsCase-studiesCross-validation
4 Ongoing research: Mood contagion
5 ConclusionsDiscussionLiterature
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCollective Intelligence: from mobs to crowds Sentiment analysis: from mood to behavior
Outline
1 IntroductionCollective Intelligence: from mobs to crowdsSentiment analysis: from mood to behavior
2 MethodsDataSentiment tracking instrument
3 ResultsCase-studiesCross-validation
4 Ongoing research: Mood contagion
5 ConclusionsDiscussionLiterature
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCollective Intelligence: from mobs to crowds Sentiment analysis: from mood to behavior
Crowds and mobs: reasons to be pessimistic
H. L. Mencken
“Democracy is the art of running the circusfrom the monkey cage.”
Assumption: large groups = random = bad decisions
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCollective Intelligence: from mobs to crowds Sentiment analysis: from mood to behavior
Condorcet jury theorem: large groups can make betterdecisions
1785, jury votes, by majority rule, on binary decision:
Definition (Condorcet theorem)
PN =�
N
i=m
�N!
(N−i)!i!
�pi (1− p)N−i
1 where
N = the number of jurors
p = the probability of an individual juror being right
m = the number of jurors required for a majority
1First and last equation.Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCollective Intelligence: from mobs to crowds Sentiment analysis: from mood to behavior
Condorcet theorem: bigger crowds do better!
Conclusion: large groups makebetter decisions!BUT:
Same individualprobability {right,wrong}Optimization problems
Independent judgements:often not true!
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCollective Intelligence: from mobs to crowds Sentiment analysis: from mood to behavior
From collective intelligence to crowd sourcing
Emergence
Complex systems and behavior can emerge from multiplicity ofinteractions between simple units
Large groups of networkedindividuals can exhibitcollective intelligence: anthills, fish schooling, bees,routing, humans...
Open Source software
Mechanical Turk
Folksonomies
Wikipedia
and... web optimization(Bollen, 1996)
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCollective Intelligence: from mobs to crowds Sentiment analysis: from mood to behavior
Not even trying?
The Naughts: Rise of social networking and micro-blogging
Stigmergy: medium coordinates collective intelligence
Facebook: +500M users
Twitter: +150M users
BUT!
Not trying to solve particular problems.Objective: recreation, socializing, sharing
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCollective Intelligence: from mobs to crowds Sentiment analysis: from mood to behavior
Twitter!
tweets and updates
users broadcast brief text updates to the public or to a limitedgroup of contacts: 140 characters or lessTwitter, Facebook, Myspace
Examples
“Our Rights from Creator(h/t @JLocke). Life,Liberty, PoH FTW! Yourtransgressions = FAIL.GTFO, @GeorgeIII.-HANCOCK et al.”
“at work feeling lousy”
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCollective Intelligence: from mobs to crowds Sentiment analysis: from mood to behavior
Analyzing the chatter
Predicting the present from side-effects
Mapping online traffic to real-world outcomes
+70M tweets per day +20GB of text per day the EEG of theglobal brain
Box office receipts from Twitter chatter: Asur (2010)
Google trends: flu (verbal autopsies)
Predicting consumer behavior from search query volume(Goel, 2010)
Contagion of “Loneliness” and happiness in social networks(Cacioppo, 2010 - Bollen, 2011)
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCollective Intelligence: from mobs to crowds Sentiment analysis: from mood to behavior
Link between sentiment, mood and behavior
Behavior is shaped not just by rational, conscious considerations
Emotion plays a significant role in human decision-making(behavioral economics, behavioral finance, social psychology).Emotion plays a tremendously important role online andconsequently in the “real” world, cf. Tunesia, Egypt.
Extract indicators of individual and collective sentiment fromonline media feeds?
Predict not just the present, but the future?
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCollective Intelligence: from mobs to crowds Sentiment analysis: from mood to behavior
Extracting sentiment indicators from text
Happy tweets.
So...nothing quite feels like a good shower, shave and haircut...love itMy beautiful friend. i love you sweet smile and your amazing souli am very happy. People in Chicago loved my conference. Love you, my sweetfriends@anonymous thanks for your follow I am following you back, great group amazingpeopleUnhappy tweets.
She doesn’t deserve the tears but i cry them anywayI’m sick and my body decides to attack my face and make me break out!! WTF:(I think my headphones are electrocuting me.My mom almost killed me this morning. I don’t know how much longer i can behere.
Different Approaches: Natural Language processing (n-grams) for reviews
(Nasukawa, 2003), topics (Yi, 2003), Support Vector Machines: text
classification (positive vs. negative) using pre-classified learning sets: Gamon
(2004), Pang (2008), Blogs, web sites: mixed approaches. Mishne (2006),
Balog (2006), Gruhl (2005),...Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCollective Intelligence: from mobs to crowds Sentiment analysis: from mood to behavior
Sentiment and mood analysis is difficult for tweets
Individual tweets
Length: 140 characters, lack of text content
Diversity:no standardized training sets, dimensions of mood?
Lack of topic specificity
Public mood from tweet collections and other microblog contents?
We Feel Fine http://www.wefeelfine.org/
Moodviews http://moodviews.com
Myspace: Thelwall (2009), FB: United States Gross NationalHappiness http://apps.facebook.com/usa_gnh/, MichaelJackson (Kim, 2009)
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCollective Intelligence: from mobs to crowds Sentiment analysis: from mood to behavior
What we did:
Trends in general public mood from a large-scale collection oftweets
Each tweet= patient taking psychometric instrument (POMS)
Large-scale collection of tweets: 10M, 2006-2008
Daily public mood assessment: Time series depictingfluctuations of public mood
Correlations to socio-economic indicators?
Contagion of mood in social network, cf. Haiti disaster?
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsData Sentiment tracking instrument
Outline
1 IntroductionCollective Intelligence: from mobs to crowdsSentiment analysis: from mood to behavior
2 MethodsDataSentiment tracking instrument
3 ResultsCase-studiesCross-validation
4 Ongoing research: Mood contagion
5 ConclusionsDiscussionLiterature
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsData Sentiment tracking instrument
Data sets
Collection of tweets:
April 29, 2006 to December 20, 2008
2.7M users
Subset: August 1, 2008 to December 2008 - 9,664,952 tweets
2008
log
(n t
we
ets
)
Aug 1 Sep 1 Oct 1 Nov 1 Dec 1 Dec 20
2e
+0
21
e+
03
5e
+0
32
e+
04
1e
+0
5
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsData Sentiment tracking instrument
Each tweet:ID date-time type text1 2008-
11-2802:35:48
web Getting ready for Black Friday. Sleep-ing out at Circuit City or Walmart notsure which. So cold out.
2 2008-11-2802:35:48
web @dane I didn’t know I had an un-cle named Bob :-P I am going to bechecking out the new Flip sometimesoon
· · ·
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsData Sentiment tracking instrument
Collective mood: Profile of Mood States (POMS-bi)
Definition
72 term questionnaire (McNair, 1971) to assess individual moodstates. Each mood term maps into one of 6 mood dimensions:
composed/anxious composed (+1), tense (-1), untroubled (+1), ...
clearheaded/confused confused (-1), clearheaded (+1), mixed-up (-1), ...
confident/unsure weak (-1), strong (+1), timid (-1), ...
energetic/tired vigorous (+1), fatigued (-1), energetic (+1), ...
agreeable/hostile sympathetic (+1), bad tempered (-1), agreeable (+1), ...
elated/depressed mad (-1), good-natured (+1), annoyed (-1), ...
About 6 minutes to complete by individual.Designed for repeated within-subjects testing.
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsData Sentiment tracking instrument
Tweet:
I am so not bored. way too busy! I feel really great!
composed/anxiousclearheaded/confusedconfident/unsureenergetic/tiredagreeable/hostileelated/depressed
0.017250.051250.7256250.666625
0.3610.53175
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsData Sentiment tracking instrument
Aggregating daily tweets into a mood time series
Twitterfeed
~
Calm
Mood indicators (daily)
textanalysis
Happy
Confident
...
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCase-studies Cross-validation
Outline
1 IntroductionCollective Intelligence: from mobs to crowdsSentiment analysis: from mood to behavior
2 MethodsDataSentiment tracking instrument
3 ResultsCase-studiesCross-validation
4 Ongoing research: Mood contagion
5 ConclusionsDiscussionLiterature
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCase-studies Cross-validation
Ratio of emotional tweets, over time.
23
45
67
89
ratio of # tweets with mood expressions over all tweets%
moo
d ex
pres
sion
s
Aug 08 Sep 08 Oct 08 Nov 08 Dec 08
−1.5
0.0
1.0
resi
dual
(%)
Aug 08 Oct 08 Dec 08 −1.5 −0.5 0.5
0.0
0.4
0.8
residual (%)
prob
abilit
y
Ratio of tweets containing mood expressions vs. all tweets on agiven day, including residuals from trendline.
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCase-studies Cross-validation
Public mood trends: overview
composed/anxious
clearheaded/confused
confident/unsure
energetic/tired
agreeable/hostile
elated/depressed
+2sd
+2sd
+2sd
+2sd
+2sd
+2sd
−2sd
−2sd
−2sd
−2sd
−2sd
−2sd
Election08 Thanksgiving08
08/01 09/01 10/01 11/01 12/01 12/20
Figure: Sparklines for G-POMS measured public mood states in August2008 to December 2008 period highlight long-term changes.
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCase-studies Cross-validation
Case study 1: November 4th, 2008 - the presidentialelection
composed/anxious
clearheaded/confused
confident/unsure
energetic/tired
agreeable/hostile
elated/depressed
+2sd
+2sd
+2sd
+2sd
+2sd
+2sd
−2sd
−2sd
−2sd
−2sd
−2sd
−2sd
Election08
10/20 11/04 11/19
Figure: Sparklines for public mood before, during and after the USpresidential election on November 4th, 2008.
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCase-studies Cross-validation
TFIDF scoring of tweet terms
2008 U.S. Presidential Election
Nov 03 Nov 04 Nov 05robocal poll historibusiness plumber wonvoter result barackcleanser absente propgrandmoth ballot speechrussert turnout resultsocialist barack president-electhalloween citizen hologramacknowledg joe victorirace thoughtfulli ecstat
Table: Top 10 TF-IDF ranking terms 1 day before, on and 1 day afterelection day.
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCase-studies Cross-validation
Case study 2: November 27th, 2008 - Thanksgiving
composed/anxious
clearheaded/confused
confident/unsure
energetic/tired
agreeable/hostile
elated/depressed
+2sd
+2sd
+2sd
+2sd
+2sd
+2sd
−2sd
−2sd
−2sd
−2sd
−2sd
−2sd
Thanksgiving08
11/12 11/27 12/12
Figure: Sparklines for public mood before, during and after Thanksgivingon November 27th, 2008.
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCase-studies Cross-validation
Long-term changes in public mood: statistical significance
Mood dimension Period 1 Period 2 p-valueAgreeable/Hostile 08/01-20 12/01-20 0.0001338
Mean 1= Mean 2= Difference-0.007sd 1.286sd 1.292sd
Confident/Unsure 08/01-20 12/01-20 0.002381Mean 1= Mean 2= Difference-0.120sd 0.785sd 0.905sd
Composed/Anxious 08/01-20 12/01-20 0.0272Mean 1= Mean 2= Difference
0.162 0.897 0.736
Table: T-tests to compare mood levels in two 20-day periods (August1-20 and December 1-20, 2008) show statistically significant elevatedz-scores for Agreeable, Confident and Composed mood.
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCase-studies Cross-validation
Outline
1 IntroductionCollective Intelligence: from mobs to crowdsSentiment analysis: from mood to behavior
2 MethodsDataSentiment tracking instrument
3 ResultsCase-studiesCross-validation
4 Ongoing research: Mood contagion
5 ConclusionsDiscussionLiterature
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCase-studies Cross-validation
Comparison to existing sentiment tracking tools:OpinionFinder
1.2
51.7
5
OpinionFinder day afterelection
Thanksgiving
!1
1
pre!electionanxiety
CALM
!1
1
ALERT
!1
1
electionresults
SURE
!1
1
pre!electionenergy
VITAL
!1
1 KIND
!1
1
Thanksgivinghappiness
HAPPY
Oct 22 Oct 29 Nov 05 Nov 12 Nov 19 Nov 26
http://www.cs.pitt.edu/mpqa/Theresa Wilson, Janyce Wiebe, and PaulHoffmann (2005). Recognizing ContextualPolarity in Phrase-Level SentimentAnalysis. Proc. of HLT-EMNLP-2005.
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCase-studies Cross-validation
Table: Multiple Regression Results for OpinionFinder vs. GPOMSdimensions.
Parameters Coeff. Std.Err. t pCalm (X1) 1.731 1.348 1.284 0.20460Alert (X2) 0.199 2.319 0.086 0.932Sure (X3) 3.897 0.613 6.356 4.25e-08 ��Vital (X4) 1.763 0.595 2.965 0.004�Kind (X5) 1.687 1.377 1.226 0.226
Happy (X6) 2.770 0.578 4.790 1.30e-05 ��Summary Residual Std.Err Adj.R2 F6,55 p
0.078 0.683 22.93 2.382e-13
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCase-studies Cross-validation
Comparison to DJIA
DJIA daily closing value (March 2008−December 2008
Mar Apr May Jun Jul Aug Sep Oct Nov Dec 2008
8000
9000
10000
11000
12000
13000
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCase-studies Cross-validation
Comparison to DJIA
Twitterfeed ~
(1) OpinionFinder
(2) G-POMS (6 dim.)
Mood indicators (daily)
DJIA ~
Stock market (daily)
(3) DJIA
Grangercausality
-n (lag)
F-statisticp-value
textanalysis
normalization
SOFNN
predictedvalue MAPE
Direction %
1
2
t-1t-2t-3
3
t=0value
Figure: Methodological diagram outlining use of Granger causalityanalysis and Self-Organizing Fuzzy Neural Network to predict daily DJIAvalues from (1) past DJIA values at t − 1, t − 2, t − 3, and variouspermutations of Twitter mood values (OpinionFinder and GPOMS).
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCase-studies Cross-validation
bivariate-causal analysis: DJIA vs. public mood
Table: Calm (X1), Alert (X2),Sure (X3), Vital (X4), Kind (X5), Happy(X6)
lag XOF X1 X2 X3 X4 X5 X6
1 0.703 0.080� 0.521 0.422 0.679 0.712 0.300
2 0.633 0.004�� 0.777 0.828 0.996 0.935 0.697
3 0.928 0.009�� 0.920 0.563 0.897 0.995 0.652
4 0.657 0.03�� 0.54 0.61 0.87 0.78 0.68
5 0.235 0.053� 0.753 0.703 0.246 0.837 0.05
�
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCase-studies Cross-validation
Calm vs. DJIA
-2
-1
0
1
2D
JIA
z-s
core
Aug 09 Aug 29 Sep 18 Oct 08 Oct 28
-2
-1
0
1
2
-2
-1
0
1
2
-2
-1
0
1
2
DJIA
z-s
core
Calm
z-s
core
Calm
z-s
core
bank
bail-out
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCase-studies Cross-validation
Table: DJIA Daily Prediction Using SOFNN
Evaluation IOF I0 I1 I1,2 I1,3 I1,4 I1,5 I1,6
MAPE (%) 1.95 1.94 1.83 2.03 2.13 2.05 1.85 1.79�
Direction (%) 73.3 73.3 86.7� 60.0 46.7 60.0 73.3 80.0
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsCase-studies Cross-validation
Citation:
Johan Bollen, Huina Mao, and Xiao-Jun Zeng. Twitter moodpredicts the stock market. Journal of Computational Science,2010, http://arxiv.org/abs/1010.3003.
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion Conclusions
Outline
1 IntroductionCollective Intelligence: from mobs to crowdsSentiment analysis: from mood to behavior
2 MethodsDataSentiment tracking instrument
3 ResultsCase-studiesCross-validation
4 Ongoing research: Mood contagion
5 ConclusionsDiscussionLiterature
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion Conclusions
Data set to study mood contagion in Twitter socialnetwork
Nov. 20, 2008 until May 29, 2009
4,844,430 users
129M tweets
Each tweet:
ID, date:time, user ID, text, type, referred
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion Conclusions
Turning followers into friends
Twitter 2: Nov08-May09, 129M tweets
1 undirected network: Friend= A follows B, B follows A
2 sufficient tweet coverage: More than 1 tweet per day onaverage.
edge weight
Jaccard index of node’s friend list:
wij =Ci ∩ Cj
Ci ∪ Cj
where Ci is k=1 neighborhood of i
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion Conclusions
Twitter social network: users=130,636, edges=7,586,895
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion Conclusions
Preliminary sentiment tracking via emoticons
Emoticon ratio
En =tnT
Ep =tpT
where tn and tp number of negative or positive tweets, and T totalnumber of tweets submitted in 188 days
user 1
user 2
;-) ;-) ;-) ;-{ :->
;-/ ;-( ;-( ;-{ :-\ :-\ :-\
N=189
N=210
http://en.wikipedia.org/wiki/Emoticon
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion Conclusions
Identifying sour and happy users
On the way home. Yay. Having pizza tonight. I think I got a cold.Poof! Just like that! :-/ - http://bkite.com/02yfl Morning so far:woke up to awful smell - 2 doggie ’presents’ in living room. :-/Then a nearly flat rear tire. :-/ (squared) - http://... Well, Ishould be in bed ... so I guess I should try that. Work at 10 in themorning. :-/ - http://bkite.com/02Q2L Chance of Snow thisweekend. :-/ - http://bkite.com/02V98
I’ve been hearing the words ”last” and ”final” a lot lately. :-)@HipMamaB what? I don’t understand that. why not?? ;-)ScarfWatch’09 - the theater lost & found has it!! Things arelooking up. ;-) It’s Crafty Mama’s night! Yay for CM night. :-)cast-iron skillets are the best. And having a pre-loved one is evenbetter - already seasoned. :-)
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion Conclusions
Assortativity vs. disassortativity
From: Sid Redner (2008) Networks: Teasing out the missing links, Nature 453, 47-48
See also: M. E. J. Newman (2003) Mixing patterns in networks, Phys. Rev. E 67, 026126
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion Conclusions
Assortativity vs. disassortativity
PS Bearman (2004). ”Chains of affection”, American Journal of Sociology, 100(1)
Not me ③ ✍✌✎☞
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion Conclusions
Example twitter mood assortativity
low SWB
high SWB
Red=happy, blue=sad white=neutralHuina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion Conclusions
Assortativity of positive and negative sentiment
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion Conclusions
ego network of world’s most depressed user
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsDiscussion Literature
Outline
1 IntroductionCollective Intelligence: from mobs to crowdsSentiment analysis: from mood to behavior
2 MethodsDataSentiment tracking instrument
3 ResultsCase-studiesCross-validation
4 Ongoing research: Mood contagion
5 ConclusionsDiscussionLiterature
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsDiscussion Literature
Discussion
Power of collective intelligence:Wisdom of crowds extends to their mood state?Predictive power?Research front: growing support
BusinessCan present models be translated to business applications?Prediction → control?
Issues:User privacy, rightsDependencies on proprietary social networking systemsBusiness vs. science
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsDiscussion Literature
References
Johan Bollen, Huina Mao, and Xiao-Jun Zeng. Twitter moodpredicts the stock market. Journal of Computational Science,2010, http://arxiv.org/abs/1010.3003. (featured on CNBC,CNN International and Bloomberg News!)
Johan Bollen, Huina Mao, and Alberto Pepe. Determining thepublic mood state by analysis of microblogging posts.Proceedings of the Proc. of the Alife XII Conference, Odense,Denmark, MIT Press, August 2010.
Huina Mao, Alberto Pepe, and Johan Bollen. Structure andevolution of mood contagion in the Twitter social network.Proceedings of the International Sunbelt Social NetworkConference XXX, Riva del Garda, Italy, July 2010
Johan Bollen, Alberto Pepe, and Huina Mao. Modeling publicmood and emotion: Twitter sentiment and socio-economicphenomena. arXiv:0911.1583 (November 2009)
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen
Introduction Methods Results Ongoing research: Mood contagion ConclusionsDiscussion Literature
THANK [email protected]
Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen