Emotion Rating From Short Blog Texts

Alastair J. Gill, Darren Gergle, Robert M. French, Jon Oberlander

CHI 2008 April 5-10, 2008 · Florence, Italy

1

computer-mediated communication (CMC)

Abstract• automatically perceive a variety of emotions from text alone

– important applications in CMC and HCI– from identifying mood from online posts ( 動機 1)– to enabling dynamically adaptive interfaces ( 動機 2)

• such ability has not been proven in human raters or computational systems– naive raters to detect one of 8 emotional categories from 50 and 200

word samples of real blog text– expert raters as a ‘gold standard’

• naive-expert rater agreement increased with longer texts– high for ratings of joy, disgust, anger and anticipation– low for acceptance and ‘neutral’ texts.

2

INTRODUCTION

• Face-to-face or on the phone– people can often guess a speaker’s emotion ‘just

from their tone of voice’– without being able to identify the words being

used

• ever want to rely on words alone?– without using information from the speech signal? – computer-mediated communication (CMC), email,

textchat and websites

3

Blog• business intelligence– Blogs can discuss a company’s performance one day, and

perhaps influence its share price the next• tools exist (Mishne 2006)– look at usage of mood terms in blog posts, analyzing large

amounts of text to capture national responses to news or sporting events (too coarse)

4

[2007-04-18] Candlelight vigil for victims of Virginia Tech shootings

[2007-04-16] Shooting at Virginia Tech; at least 31 dead

[2007-04-17] 33 dead, 15 injured in Virginia Tech shootings

[2007-04-17] Shooting at Virginia Tech college in USA; at least 33 dead

[2007-04-17] Virginia Tech shooter identified, witness reports emerge

Blog posts:

Smaller Text

• Greater specificity• Smaller text segments reflecting particular

opinions may need – to be extracted – and classified for opinion or emotion

• 訓練出來的分類 tool 可滿足 ( 動機 1) ( 動機 2)– detect emotion in email clients or in a friend’s recent

blog posts– user interfaces which can automatically detect and

adapt to user emotion may be possible

5

Social Presence Theory (1976)

• Rich-Environment– Face to Face– a full range of emotional and interpersonal

information due to greater social presence

• Less Rich-Environment– text-based CMC environments, inhibit

communicating emotional expression– 所以有需要加以強化 , 但是到底欠什麼

6

Social Information Processing (1992)

• interpersonal cues,– such as emotional information– are present in computermediated environments– but it just takes longer to derive the same information

• to derive the same perceptions as is possible in face-to-face

• communication, either by placing greater emphasis on existing cues (linguistic features), or by developing new strategies such as emoticons

7

Hancock, Landrigan, & Silver (2007)

• 請參予者用文字模式交談，並表達– positive (happy) or negative (unhappy) emotions

• 發現 naïve judges 都可以判斷，而且不想跟有壞情緒的人交談

• 語言學分析– 表達正面情緒的人用比較多驚嘆號，也用比較

多自– 負面情緒的人用比較多情緒字眼、負面字眼、

否定詞

8

這篇 Paper

• 比 Hancock 多做一些事情1. expand their classification of emotion from

positive and negative into the eight main categories

2. rather than focus on extended interactions, we examine whether emotion can be accurately classified on the basis of asynchronous short blog text extracts of 50 and 200 words

3. real emotional blogs (not actors)

9

METHOD

• 65 judges (Scottish university)– 23 males, 42 females, mean age = 22.24– all were frequent email users (mean score 6.30 on a

0-7 Likert scale)– few used blogs frequently (mean score 1.38), with 33

participants never using blogs• an ‘expert’ research assistant selected the first

200 words of each post if they contained some emotional content or were ‘neutral’– 135 text extracts totaling 27,000 words

10

Gold Standard

• 5 expert raters rated these texts as expressing one of eight emotions (anticipation, acceptance, sadness, disgust, anger, fear, surprise, joy)– After all experts had assigned an emotional category

rating to each of the 135 texts– 20 were selected as expressing strong and clear

emotional content• 2*8 emotions + 4 neutral• short version we extract the middle 50 words of

the 200 word text

11

Rating Procedure• All 20 texts were presented to the naive raters of emotion in random order

– One group saw 10 long versions, then 10 short versions of the texts– The second group saw 10 short, then 10 long text versions

12

the judges were asked to rate ‘how they perceive the author’s emotions’ but ‘not to spend too long thinking about their answer

as we are particularly interested in [their] initial response’

All ratings took less than 30 minutes,

Analysis• Nominal logistic regression• simply coding expert-naive rater agreement as a binary value

(agreement=1; disagreement=0)– analysis of emotional intensity to future work

13

補充 : 卡方檢定

14

• 虛無假設 H0: 專家和工讀生標記無差異• 對立假設 H1: H0 錯誤 ( 即標記有差異 )

• 檢定統計量• 否定域 C={X2| X2>X2

0.05(9)=16.91}

• 不否定 , 因此接受 H0, 專家和工讀生標記無差異

4.65

)55(...

5

)56(

5

)54(

5

)54(

5

)56( 22222

q

Results 卡方檢定

– Acceptance (χ2=13.6, p<.001) Sadness (χ2=5.85, p=.015)– Fear (χ2=4.11, p=.043) Disgust (χ2=5.50, p=.019)

1. 長的文章比較好2. 不論長短都可以通過檢定的情緒

Joy, Disgust, Anger, Anticipation3. 最差很難一致的 – Acceptance, Neutral

15

DISCUSSION

• greatest naive-expert rater agreement is related to strongly positive and negative emotions (anger, disgust, joy, anticipation)– 強烈的正負面情緒較容易取得一致

• 有些情緒不用很長的 sample 就可以判斷了• 短文章也有用 -> 那 Social Presence Theory…• 文章越長越有用 ->

Social Information Processing 提到過的 cue

16

Limitations

• Our ‘experts’ were very familiar with personal blogs, but they were not psychologically trained for emotion rating

• the experts may have only selected blog texts which express emotion very saliently

• Future studies– ideally draw upon self reports or even physiological

measures of emotion from the authors during writing ( 最好跟作者取樣，看他寫文章時是什麼情緒 )

– 各情緒強度

17

Contributions

• 正反面 ->8+1 類• Joy, Disgust, Anger, Anticipation 最一致

歡喜，噁心，生氣，希望• 這四個在短文章也做很好• Future– apply machine learning classification to the blog

emotions• emotion monitoring of blog posts• dynamic interfaces which adapt to user state

based on linguistic features of the texts.

18