NUS-ISS Learning Day 2017 - Sense the Sentiments in Social Media Data

SENSE THE SENTIMENTS IN SOCIAL MEDIA DATAISS LEARNING DAY 2017

Dr Leong Mun Kew

isslmk@nus.edu.sg

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 1

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 2

Dr Leong Mun Kew is currently Deputy Director of ISS

responsible for ISS’ graduate programmes, and also running the

Analytics and Intelligent Systems practice.

Before joining ISS, he was with the National Library Board as

Chief Technology Officer and Deputy CIO. Before joining NLB,

he was Principal Scientist and Programme Director for Services

Research at I2R. Mun Kew’s personal research was in

information retrieval, digital libraries, distributed multilingual

search systems and media semantics, and he also looked after

research groups in natural language processing, security,

distributed systems, data mining and analytics. From 1999 to

2001, Mun Kew was VP and CTO of an IT start-up delivering

custom distributed deep search technologies and services.

MunKew received his PhD from Stanford University. He has

more than 30 years of R&D and commercial experience in IT. He

was Chair and Editor of the ISO/JTC1/WG2/24800 JPSearch

international standard, editor or past editor of a bunch of

journals and conference series, and a member of the Steering

Committee of the Singapore Bio-Imaging Consortium. MunKew

is a Senior Member of the Singapore Computer Society and was

a council member of the ITMA for many years.

Leong Mun Kew

Deputy Director,

NUS-ISS

Yuhao is currently Assistant Lecturer and Consultant

in NUS, Institute of Systems Science. Before joining

ISS, Yuhao was a Master Student of Mathematics in

NUS.

He currently teaches fundamental data analytics as

well as text analytics. More recently, he’s started to

work on deep learning for image and text analysis.

His research interest is in statistical data analysis

and data visualization. He is also a level 2 Candidate

in the CFA Program.

He has also completed several research projects :

PICO project studying financial incentives on the public's

travel behaviour, in partnership with the Land Transport

Authority (LTA) of Singapore

Competed a project in social network analysis which focus

on visualizing Facebook and Twitter networks with NodeXl

and Gephi

Analysed farecard data and visualized the data by using

the Microstrategy cloud platform

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 3

Cai YuhaoAssistant Lecturer

& Consultant,

Analytics &

Intelligent Systems

WHAT IS SENTIMENT ANALYSIS?

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 4

Wikipedia…

• Sentiment analysis (also known as opinion

mining) refers to the use of natural language

processing, text analysis and computational

linguistics to identify and extract subjective

information in source materials.

• Generally speaking, sentiment analysis aims

to determine the attitude of a speaker or a

writer with respect to some topic or the

overall contextual polarity of a document.

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 5

Wikipedia…

• Sentiment analysis (also known as opinion

mining) refers to the use of natural language

processing, text analysis and computational

linguistics to identify and extract subjective

information in source materials.

• Generally speaking, sentiment analysis aims

to determine the attitude of a speaker or a

writer with respect to some topic or the

overall contextual polarity of a document.

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 6

Let’s analyze an example to see…

(1) AhSeng123 2015:09:25:16:50:21 (2) Just lined up for

3 hours to get my iPhone 6S! (3) Score! (4) Got the pink

one for my gf. (5) She’s going to owe me big for that :-)

(6) Mine’s a grey one. (7) Screen is good but maybe not

as clear as my Samsung. (8) 3D touch is fun, but not

sure what I’m going to do with it (yet!) (9) Really slim

though. (10) Kinda expensive but I really like it.

Numbers in parentheses added to simplify discussion.

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 7

Let’s analyze an example to see…

(1) AhSeng123 2015:09:25:16:50:21 (2) Just lined up for

3 hours to get my iPhone 6S! (3) Score! (4) Got the pink

one for my gf. (5) She’s going to owe me big for that :-)

(6) Mine’s a grey one. (7) Screen is good but maybe not

as clear as my Samsung. (8) 3D touch is fun, but not

sure what I’m going to do with it (yet!) (9) Really slim

though. (10) Kinda expensive but I really like it.

• What can you say about the entire paragraph?

• How about individual sentences?

• How about the product(s)?

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 8

Consider each sentence…

(1) AhSeng123 2015:09:25:16:50:21

(2) Just lined up for 3 hours to get my

iPhone 6S!

(3) Score!

(4) Got the pink one for my gf.

(5) She’s going to owe me big for that :-)

(6) Mine’s a grey one.

(7) Screen is good but maybe not as

clear as my Samsung.

(8) 3D touch is fun, but not sure what I’m

going to do with it (yet!)

(9) Really slim though.

(10) Kinda expensive but I really like it.

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 9

Consider each sentence…

(1) AhSeng123 2015:09:25:16:50:21 States an opinion holder and a time

(2) Just lined up for 3 hours to get my

iPhone 6S!

(3) Score!

(4) Got the pink one for my gf.

(5) She’s going to owe me big for that :-)

(6) Mine’s a grey one.

(7) Screen is good but maybe not as

clear as my Samsung.

(8) 3D touch is fun, but not sure what I’m

going to do with it (yet!)

(9) Really slim though.

(10) Kinda expensive but I really like it.

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 10

Consider each sentence…

(1) AhSeng123 2015:09:25:16:50:21 States an opinion holder and a time

(2) Just lined up for 3 hours to get my

iPhone 6S!

You find out the product – iPhone 6S

(3) Score!

(4) Got the pink one for my gf.

(5) She’s going to owe me big for that :-)

(6) Mine’s a grey one.

(7) Screen is good but maybe not as

clear as my Samsung.

(8) 3D touch is fun, but not sure what I’m

going to do with it (yet!)

(9) Really slim though.

(10) Kinda expensive but I really like it.

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 11

Consider each sentence…

(1) AhSeng123 2015:09:25:16:50:21 States an opinion holder and a time

(2) Just lined up for 3 hours to get my

iPhone 6S!

You find out the product – iPhone 6S

(3) Score! Positive sentiment

(4) Got the pink one for my gf.

(5) She’s going to owe me big for that :-)

(6) Mine’s a grey one.

(7) Screen is good but maybe not as

clear as my Samsung.

(8) 3D touch is fun, but not sure what I’m

going to do with it (yet!)

(9) Really slim though.

(10) Kinda expensive but I really like it.

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 12

Consider each sentence…

(1) AhSeng123 2015:09:25:16:50:21 States an opinion holder and a time

(2) Just lined up for 3 hours to get my

iPhone 6S!

You find out the product – iPhone 6S

(3) Score! Positive sentiment

(4) Got the pink one for my gf. Colour feature of the product

(5) She’s going to owe me big for that :-)

(6) Mine’s a grey one.

(7) Screen is good but maybe not as

clear as my Samsung.

(8) 3D touch is fun, but not sure what I’m

going to do with it (yet!)

(9) Really slim though.

(10) Kinda expensive but I really like it.

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 13

Consider each sentence…

(1) AhSeng123 2015:09:25:16:50:21 States an opinion holder and a time

(2) Just lined up for 3 hours to get my

iPhone 6S!

You find out the product – iPhone 6S

(3) Score! Positive sentiment

(4) Got the pink one for my gf. Colour feature of the product

(5) She’s going to owe me big for that :-) Positive sentiment

(6) Mine’s a grey one.

(7) Screen is good but maybe not as

clear as my Samsung.

(8) 3D touch is fun, but not sure what I’m

going to do with it (yet!)

(9) Really slim though.

(10) Kinda expensive but I really like it.

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 14

Consider each sentence…

(1) AhSeng123 2015:09:25:16:50:21 States an opinion holder and a time

(2) Just lined up for 3 hours to get my

iPhone 6S!

You find out the product – iPhone 6S

(3) Score! Positive sentiment

(4) Got the pink one for my gf. Colour feature of the product

(5) She’s going to owe me big for that :-) Positive sentiment

(6) Mine’s a grey one. Another product (now it’s two)

(7) Screen is good but maybe not as

clear as my Samsung.

(8) 3D touch is fun, but not sure what I’m

going to do with it (yet!)

(9) Really slim though.

(10) Kinda expensive but I really like it.

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 15

Consider each sentence…

(1) AhSeng123 2015:09:25:16:50:21 States an opinion holder and a time

(2) Just lined up for 3 hours to get my

iPhone 6S!

You find out the product – iPhone 6S

(3) Score! Positive sentiment

(4) Got the pink one for my gf. Colour feature of the product

(5) She’s going to owe me big for that :-) Positive sentiment

(6) Mine’s a grey one. Another product (now it’s two)

(7) Screen is good but maybe not as

clear as my Samsung.

Screen: positive or not? Comparison?

(8) 3D touch is fun, but not sure what I’m

going to do with it (yet!)

(9) Really slim though.

(10) Kinda expensive but I really like it.

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 16

Consider each sentence…

(1) AhSeng123 2015:09:25:16:50:21 States an opinion holder and a time

(2) Just lined up for 3 hours to get my

iPhone 6S!

You find out the product – iPhone 6S

(3) Score! Positive sentiment

(4) Got the pink one for my gf. Colour feature of the product

(5) She’s going to owe me big for that :-) Positive sentiment

(6) Mine’s a grey one. Another product (now it’s two)

(7) Screen is good but maybe not as

clear as my Samsung.

Screen: positive or not? Comparison?

(8) 3D touch is fun, but not sure what I’m

going to do with it (yet!)

Another feature: 3D Touch – was it

positive?

(9) Really slim though.

(10) Kinda expensive but I really like it.

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 17

Consider each sentence…

(1) AhSeng123 2015:09:25:16:50:21 States an opinion holder and a time

(2) Just lined up for 3 hours to get my

iPhone 6S!

You find out the product – iPhone 6S

(3) Score! Positive sentiment

(4) Got the pink one for my gf. Colour feature of the product

(5) She’s going to owe me big for that :-) Positive sentiment

(6) Mine’s a grey one. Another product (now it’s two)

(7) Screen is good but maybe not as

clear as my Samsung.

Screen: positive or not? Comparison?

(8) 3D touch is fun, but not sure what I’m

going to do with it (yet!)

Another feature: 3D Touch – was it

positive?

(9) Really slim though. Another feature: slim – positive

(10) Kinda expensive but I really like it.

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 18

Consider each sentence…

(1) AhSeng123 2015:09:25:16:50:21 States an opinion holder and a time

(2) Just lined up for 3 hours to get my

iPhone 6S!

You find out the product – iPhone 6S

(3) Score! Positive sentiment

(4) Got the pink one for my gf. Colour feature of the product

(5) She’s going to owe me big for that :-) Positive sentiment

(6) Mine’s a grey one. Another product (now it’s two)

(7) Screen is good but maybe not as

clear as my Samsung.

Screen: positive or not? Comparison?

(8) 3D touch is fun, but not sure what I’m

going to do with it (yet!)

Another feature: 3D Touch – was it

positive?

(9) Really slim though. Another feature: slim – positive

(10) Kinda expensive but I really like it. Another feature: price – negative? But

“I really like it”?

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 19

So, what’s sentiment analysis?

• What’s the sentiment of the entire document?• What’s the polarity: +ve, -ve, neutral?

• How about each sentence individually?• Can this be subjective?

• What’s the opinion holder’s sentiment about the product?

• Are there positive aspects and negative aspects about the product?

• Can you conclude (business wise) anything definitive from this single document?

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 20

So, what’s sentiment analysis?

• What’s the sentiment of the entire document?• What’s the polarity: +ve, -ve, neutral?

• How about each sentence individually?• Can this be subjective?

• What’s the opinion holder’s sentiment about the product?

• Are there positive aspects and negative aspects about the product?

• Can you conclude (business wise) anything definitive from this single document?

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 21

Well, it depends on what you want to do!

Ultimately…

What you want to do depends on your business question

A business question is one where, if you knew the answer, you would act

differently

Sentiment mining/analysis is just another tool to help you get the

answers you need

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 22

WHY IS UNDERSTANDING SENTIMENT IMPORTANT?

(WHY DO WE CARE?)

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 23

Opinions Matter to People

• Human beings are social creatures

• When we make decisions,• We want to know what others think (and do)

• E.g., TripAdvisor; HungryGoWhere, etc.

• We want to understand the product better• E.g., Amazon book reviews; product reviews in general

• Forums to exchange knowledge and experience

• We want reassurance• “My friends think I look fabulous in this”

• “500 people have bought this, they all can’t be wrong!”

• We have conviction• “I don’t care what they think; I know what I want”

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 24

Opinions Matter to Organizations

• Customers (citizens) are people after all

• Companies:• Use market studies, opinion polls, user studies, etc.

• Employ consultants, marketers, etc.

• Are concerned about publicity, brand and image• Reflects the organization core values

• Reflects the trust of the customers

• Really really worried about the effect of poor image on sales, stock prices, etc.

• Governments:• Similar concerns as above

• Need to win elections by the people

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 25

New Media’s changing landscape

• Social Media: new channels for consumers• Instead of friends, we are trusting strangers

• Instead of few “experts”, we rely on large numbers

• Social Media: omnichannel for organizations• No longer soliciting opinions; freely available

• Companies more responsive to social media postings than to traditional feedback/complaints

• Social Media: opportunities for startups• Many startups in sentiment space

• Driven from solving real problems

• Diverse domains from politics to stock market predictions

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 26

Obama’s 2012 Presidential Campaign Sentiment Analysis and Analytics

Two sentiments: • Support Obama

• Cast a Ballot (voting is optional)

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 27

Support Obama

Cas

t a

Ball

ot

NO

NO

YES

YESNo need to

do anything

No need to

do anything

Targeted ads

to change

their mind!

Initiatives to

get them out

to vote!

Adapted from: http://www.technologyreview.com/featuredstory/509026/how-obamas-team-used-big-data-to-rally-voters/

Business Outcome?

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 28

From: http://www.bloomberg.com/bw/articles/2013-05-31/obamas-data-team-totally-schooled-gallup

SOME EXAMPLES OF THE APPLICATIONS OF SENTIMENT MINING

FROM: HTTP://SENTIMENT.CHRISTOPHERPOTTS.NET/OVERVIEW.HTML#APPLICATIONS

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 29

Unless otherwise stated, all images and examples in this sub-section are from the above

This is an Amazon review of a book, but the target of the sentiment is the publisher.

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 30

2. Brand Analysis, Reputation Management and Social Mentions

Twitter discussion after Netflix announced a price increase.

Netflix stock price…

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 31

3. The Emotions of a Nation

Sentiment “mobs” from wefeelfine

Facebook’sGross Happiness Index

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 32

4. Monitoring Real-World Events

Twitter sentiment in tweets about Libya, from the project Modeling Discourse and Social Dynamics in Authoritarian Regimes. The vertical line marks the timing of the announcement that Gaddafi had been killed.

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 33

5. Media Studies

Sentiment Mining used to study media coverage and media bias:

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 34

From:

http://www.onthemedia.org/story/165916-

media-president-and-horse-race/transcript/

6. Financial Prediction

Rises and falls in the number of instances of words related to a calm mood could be used to predict the same moves in the Dow's closing price between two and six days later.

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 35

From:

http://www.thewire.com/business/2011/08/h

ow-twitter-based-hedge-fund-beat-stock-

market/41389/

Thank you!

munkew@nus.edu.sg

(Total Slides=5) <Project code, file name, version> © 2015 National University of Singapore. All Rights Reserved 36