Leveraging Big Data to Derive Actionable People Insights

PowerPoint Presentation

Leveraging Big Data to Derive
Actionable People Insights

Huahai YangUSER Group, Computer Science

About Us

Aka. IBM Almaden Research Center (ARC)

On top of a hill in the southern tip of Silicon Valley

About Us

ARCScience & Technology

Storage Systems

Service Science Research

Computer ScienceTheory

Database Management

Intelligent Information System

Healthcare Information Technology

User System and Experience Research (USER)Currently led by Michelle X. Zhou

Join us, we are always hiring!Interns, Postdocs, Software Engineers and Research Staff Members

Big Data Opportunities

IndustriesFinance

Retail

Product Manufacturer

Tel-communication

Entertainment

PotentialsCustomer acquisition/retention

Market segmentation

Brand management

Risk assessment

300+ million tweets daily

1+ million blog posts daily

Our Focus: Insights about People

Perceptions, Sentiments, Personalities and other profiles

crowdsourcing2

480px-Twitter_2010_logoblogger-logo54px-Digg-new266px-FacebookTumblr-logoWordpress-logoFlixsterlogo

Outline

Consumer usersOpinionBlocksVisually summarizing product reviews

Business usersBrandy Understanding user perception and personality for direct marketing

qCrowdActively engaging individuals on social media

OpinionBlocks Motivation

Online product reviews Significant in consumer decision making

Large volume, uncurated

Limited search tools

Users have different priorities

Help consumer makes better use of reviews

Difficulties with Review Text

A lot of variations in terms of: length

clarity of language

to the point vs vague

emotional vs subjective

Incoherent with the rating

Redundant

Prior Work

Chen et al. Visualizing Analysis of Conflicting Opinions, 2006

Oelka et al. Visual Opinion Analysis of Customer Feedback Data, 2009

Analyze first, and visualize the analysis results only

Can Consumer Trust the Analysis?

Sentiment analysis is not a solved problem in NLPOften less than 80% accuracy

Aspect oriented sentiment is even less accurate

Low resolutionOften polarity only: positive, negative, neutral

Not very actionable

it was not clean, but I am not expecting a better performance from any vacuum.

Our Approach

Support an interactive reading experience where users can search for relevant information.

Visualize the text itself while highlighting analysis results.

Show the categorized text in context so that users can judge fairness of the sentiment analysis.

Progressively disclose textual information while continuously providing visual graphical summaries

Overview First

Provide summary of overall opinion

Identify important features and key issues in each

Interactivity reveals correlations among features

Filter on Demand

Polarity of feature

Keywords

Snippets

Zoom across LODs

Zoom across LODs

Work in Progress

Formal user studiesDoes the system help consumers?Learn better about the product domain

Find information faster

Make better decisions

Resign of the UICompare products

Scalable with larger number of reviews

Brandy

360 understanding of a business brand: evidence-based brand management from social media

Brand associations (e.g., key aspects)

Competitive brands

Brand evolution

User modeling of those who voiced brand perception

Demographics, personality traits, locations, brand association and sentiments

Active brand management for effective marketing

Craft/adjust marketing messages based on brand analysis

Associations and customer needs

Deliver marketing messages to target customers

Individual customers (e.g., customer retention)

Customer segments (e.g., customer acquisition)

Social Data

Enriched/newCustomer Profile

Segment-based Direct Marketing(Customer Acquisition)

BrandManagement(Marketing Research)

Existing Customers

New Customers

Social Data of Known Customers

PerceptualMap

Social Data of Unknown Customers

(A) Profiling Fusion

(C) Brand perception from social data

(D) Overlay

Unica (Today)

(B) User profiling from social data

CustomerProfiles

Individual-basedDirect Marketing(Customer retention)

Data

Key Technology

Applications

Project Map

(E)

(F)

(G)

G is a must, and E&F are bonus

Features

#

Examples

Computation

LIWC

(dictionary-based measurement of aspects of word usage)68

First personNegationFeelingCommunicationLeisureDeath

Let g be a LIWC category, Ng denotes the number of occurrences of words in that category in ones tweets and N denotes the total number of words in his/her tweets. A score for category g is then: Ng/N.

Big Five(personality types, OCEAN)

5

Openness

Using correlations with LIWC features as reported by previous researchers (e.g., Yarkoni et al.)

Big Five Facets

30

Liberalism Imagination

Using correlations with LIWC features as reported by previous researchers (e.g., Yarkoni et al.)

Modeling Personality

Research in psychology have shown that word usage in ones writings such as blogs and essays is related with ones personality

Our research has correlated Big 5 facet features with willingness and readiness to respond to questions on social networks

Industry Solutions Joint Program

Ongoing: Correlating Brands with Personalities

Data

3000 Twitter users discussing Walmart, Costco, and Sears

Analytics

Measured personality traits from twits

Openness scores per brand shown to right.

All Brands

Costco

Sears

Walmart

Brand Perceptions from Twitter Data

Twitter Data

Collected from Twitter4J Stream-fashion API

Queries: Walmart, Costco, Sears

Total ~1M tweets , from May 31st- June 21 (3 weeks)

Data Filtering & Enhancements

Started with 6 categories from Consumer Report, selection, quality, layout, service, checkout, value

Category expansion- using synonyms, Wordnet and related terms. E.g.,

Added new categories that POP out of the data, jobs, people, visiting, experience...

E.g., I don"t know why I even try anymore. Walmart always disappoints. #walmartsucks

Walmart stay crowded!

checkout

line,wait,cashier,counter,self checkout,cash,register

layout

Isle,passage,lane,shelf,door,gate,lost,spacious,narrow,row

Classification- Inputs

Training Data

Manually categorized ~3500 tweets

Walmart (1341), Costco (1420), Sears (1109)

Run WEKA with 3 different models

SVM

Multinomial

KNN

Run ICM with some tuning

Nave based

Knowledge based and rules based

Cross validation with 2-folds.

Data Expansion

Run algorithm that finds similar tweets for those 3500 categorized tweets, ended up with ~75,000 tweets.

Classification- Results

Walmart

Costco

WEKA

ICM

WEKA

ICM

Category

#tweets

Precision

Recall

Precision

Recall

#tweets

Precision

Recall

Precision

Recall

Visiting

112

0.313

0.357

0.412

0.5

244

0.457

0.496

0.86

0.39

Jobs

187

0.907

0.834

1

0.782

32

0.773

0.531

1

0.538

Experience

99

0.253

0.222

0.272

0.312

114

0.199

0.254

0.177

0.544

Checkout

135

0.927

0.748

0.963

0.791

16

0.333

0.188

0.2

0.12

Service

17

0

0

0

0

3

0

0

0

0

Value

19

0.5

0.053

0

0

53

0.1

0.057

0.125

0.08

Layout

6

0

0

0

0

4

0

0

0

0

Quality

18

0.25

0.056

0

0

53

0.16

0.075

0.2

0.318

People

81

0.294

0.185

0.357

0.256

43

0.053

0.023

0.2

0.3889

Pharmacy

63

0.925

0.778

0.92

0.741

95

0.926

0.916

0.905

0.941

Selection

50

0.389

0.14

0.112

0.409

132

0.284

0.189

0.407

0.159

Other Category

574

0.598

0.763

0.673

0.111

712

0.625

0.704

0.702

0.5938

Visiting,112OtherCategory,574Jobs,187Experience,99Checkout,135Value,19Service,17Layout,6Quality,18People,81Pharmacy,63Selection,50

Classification- Insights

Insufficient evidence: some categories are rarely discussed in the tweets. E.g., layout, service.

Complexity of category: Even for the same number of training data, the classification for some categories have much lower accuracy

Infrastructure- UI

Work in progress

Improve classification quality of brand perceptionsUnsupervised methods to uncover unknown perceptions

Relating brand perceptions with consumer personalities

User studies on marketing professionalsHow well the tools support marketing tasks?

Qcrowd: asking targeted strangers questions

Engagement Continuum

System Architecture

Research Questions

Where might this be helpful?Questions about an event that are best answeredsoon after the event

Questions for which there might be a diversity of opinions

More?

How feasible is this approach?Will people answer questions from strangers?

Will use of incentives increase responses?

What is the quality of the answers?

Test Scenarios: TSA Tracker & Camera Review

Crowdsourcing airport security wait time via twitter

Crowdsourcing product reviews via twitterAsk follow-up questions if responded

Questions Asked

TSA TrackerWithout incentive

With incentive

Camera Reviews

Results

Follow-up Questions

Observations

Thank You! Questions?

User System and Experience Research (USER)IBM Research Almadenhttp://www.almaden.ibm.com/cs/disciplines/user/

Jilin Chen

Allen Cypher

Eben Haber

Eser Kandogan

Tessa Lau

Jalal Mahmud

Jeffrey Nichols

Barton Smith

Huahai Yang

Michelle X. Zhou

Tara Mathews

IBM Research - Almaden

2012 IBM Corporation




IBM Research Haifa

