Sentiment: The Art and Science Of Measuring Emotion

Karina Taylor, U.S. Bank@karinataylor

Sentiment:The Art and Science Of Measuring Emotion

• Heavy focus on marketing as an ‘art’

• Attempt to copy success without quantifying goals or using data

How Information Becomes Knowledge

• Heavy focus on marketing as an ‘art’

• Attempt to copy success without quantifying goals or using data

• Desire to be more data driven

• Focus on high-level metrics

• Lacks cadence

• Lacks specific goals

How Information Becomes Knowledge

• Heavy focus on marketing as an ‘art’

• Attempt to copy success without quantifying goals or using data

• Desire to be more data driven

• Focus on high-level metrics

• Lacks cadence

• Lacks specific goals

• Periodic reporting is standard, though often over longer periods of time

• Patterns of successful tactics are identified but not translated to case study

How Information Becomes Knowledge

• Heavy focus on marketing as an ‘art’

• Attempt to copy success without quantifying goals or using data

• Desire to be more data driven

• Focus on high-level metrics

• Lacks cadence

• Lacks specific goals

• Clear, regular cadence to measurement.

• Knows key metrics across channels and has specific goals for improvement

• Understands how to measure marketing ROI

• Benchmarking is ongoing and pervasive

• Periodic reporting is standard, though often over longer periods of time

• Patterns of successful tactics are identified but not translated to case study

How Information Becomes Knowledge

The Problems We Have

Problems We Have

Strategy• Segmentation• Targeting• Positioning• Product

Tactics• Social Channel• Content Format• Ad Format

Creative• Copy/Voice• Images• Video

Real problems that we need to test

Problems We Have

Strategy• Segmentation• Targeting• Positioning• Product

Tactics• Social Channel• Content Format• Ad Format

Creative• Copy/Voice• Images• Video

Statistics• Not Enough Data• Bad Experiment

Design• Video• Correlation vs Cause

OR

Real problems that we need to test Data problems

Problems We Have

Strategy• Segmentation• Targeting• Positioning• Product

Tactics• Social Channel• Content Format• Ad Format

Creative• Copy/Voice• Images• Video

Statistics• Not Enough Data• Bad Experiment

Design• Video• Correlation vs Cause

When you’re wrong, you don’t know it.

OR

Real problems that we need to test Data problems

Sentiment & Share of Voice

Challenges• Neutral mentions have

the most volume

• Variance is common

• One time events can affect sentiment significantly

• Share of Voice is too general to be actionable

Brand Negative Neutral Positive Total Volume

Bank of America 6% 88% 6% 253,713

Chase 7% 84% 9% 123,950

Citibank 5% 88% 7% 200,936

Fifth Third 6% 83% 12% 10,069

PNC 5% 86% 9% 42,476

U.S. Bank 3% 87% 10% 51,288

Wells Fargo 5% 87% 8% 327,957

Sentiment & Share of Voice

How do you use it?

Sentiment• Do you use it for planning, goal setting or strategy?• Can you understand your own data?• Is the data accurate enough for reporting?

Share of Voice• Does it help you monitor trends and adjust your strategy?• Is the data relevant?• Is there a place for share of voice in your reporting?

Sentiment: Don’t Work Without a Net

Using Net Sentiment

Calculate sentiment on a scale of -5 to +5, excluding mentions with neutral sentiment.

Improves readability and comparability over time.

Calculations

difference = positive - negative sum = positive + negative

net sentiment = (difference / sum)*5

Brand Negative Positive Total Volume Share of Voice NetSentiment

Bank of America 6% 6% 253,713 25% -0.1

Chase 7% 9% 123,950 12% 0.9

Citibank 5% 7% 200,936 20% 1.1

Fifth Third 6% 12% 10,069 1% 1.7

PNC 5% 9% 42,476 4% 1.3

U.S. Bank 3% 10% 51,288 5% 2.6

Wells Fargo 5% 8% 327,957 32% 1.1

Optimize Listening: AuditingAuditing Considerations

Auditing = Accuracy. You need to audit (verify and adjust) sentiment for all brand mentions.

Unadjusted, totals were: 10% positive and 3% negative (net sentiment +2.6)

Social data to the left has an adjusted net sentiment of -0.7

A difference of 3.1 points

It is getting SO much easier, but…

Sentiment Queue’s Abound• Emoji's• Facebook Reactions• Twitter GIFs• Bitmoji

Reporting & Sentiment

1 Awareness

2 Engagement

3 Acquisition

AwarenessAwareness: How many people do we reach?

Measurements include:• Impressions• Organic Reach• Promoted Reach• Ad Reach• Buzz• Share of Voice• Shifts in Sentiment

EngagementConsideration: How is the campaign being received and are we leveraging any influencers?

Measurements:• Sentiment• Clicks• Comments/Replies• Shares/Retweets• Completions [Video/Quiz/activity]

AcquisitionConversion: How many people completed the action we predetermined during the objective phase?

Measurements include:• Application Starts/Submits• New Accounts• View a video segment in its entirety or

for :30+?• Lead Form/Inquiry Completion• Referral to MLO or Wealth Manager

Challenges in Reporting Sentiment

Issues In Reporting Sentiment:

• Explaining sentiment• Accounting for variation• Differentiating from click and

impression data

Scorecards with Sentiment

Working Around The Challenges:

• Visual representation of complex topics

• Separate from more common digital metrics

• Educate within reporting

Thank You!