Introduction to social media research print version

Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research

Introduction to Social Media Research 16 October 2013

Annie Pettit, Peanut Labs

All copyright owned by The Future Place and the presenters of the material For more information about NewMR events visit http://newmr.org

This webinar was broadcast on 16 October, 2013, as part of the NewMR Lecture Series. You can access the other lectures, and the materials from other events via NewMR’s Play Again http://newmr.org/play-again/

http://newmr.org/

http://newmr.org/

http://newmr.org/play-again/




An Introduction to

Social Media Research

By Annie Pettit

Chief Research Officer, Peanut Labs

@LoveStats @PeanutLabsMR


MROCs

Surveys

Ethnography

Mobile qualitative

Online focus groups

Social media research

More tools in our toolbox


• Online surveys

• Online focus groups

• Online communities

• Facebook polls

• Sample from social networks

Social media listening research is NOT


• Market research using naturally occurring social media conversations as the data source

• Distinctive because:

• NOT permission based

• NOT elicited

• Massive potential for individual harm

What is social media listening research?


What is individual harm?

@UserName If my rabbi finds out I just ate pork, I’m in huge trouble @UserName I’m doing

crap all at work today #FreeMoney @UserName If anyone at

school says anything about me today, I will finally kill myself @UserName I hate pretending

to be happy. &^%& %#&@ $*@&


• Separate guidelines developed by MRA, ESOMAR, CASRO

• Essence is the same as traditional guidelines

• Respect for privacy

• Be aware of implications re contacting/sharing without permission

• http://www.mra-net.org/rq/documents/MRA_IMRO_SMR16.pdf

• http://www.casro.org/pdfs/1011/Social_Media_Research_Guidelines.pdf

• http://www.esomar.org/uploads/public/knowledge-and-standards/codes-and-guidelines/ESOMAR_Guideline-on-Social-Media.pdf

Be knowledgeable about ethics guidelines

http://www.mra-net.org/rq/documents/MRA_IMRO_SMR16.pdf




http://www.casro.org/pdfs/1011/Social_Media_Research_Guidelines.pdf



http://www.esomar.org/uploads/public/knowledge-and-standards/codes-and-guidelines/ESOMAR_Guideline-on-Social-Media.pdf


















Create search

Clean Search

Crawl Internet

Variable Cleaning

Content Analysis

Sentiment Scoring

Sample

Weight

Analyze

Results

Social Media Research Method


• All research methods have limitations • Surveys are difficult to write and may include poorly worded

questions and answers

• Focus groups employ small sample sizes of carefully selected

people whose opinions might not generalize outside the group

• Mobile surveys need to be short and simple

• Social media research also has limitations

Limitations


Cannot identify geo/demographics

Female, White, Not Hispanic, Canadian

Male, White, Not Hispanic, Belgium

Female, White, Not Hispanic, American


• Gazprom

• McKesson

• Sinopec

• AXA

• Paribas

• ThyssenKrupp

• State Grid

• Dexia Group

• Sysco

• Repsol YPF

• BNP

• Fortis

Hard for unknown companies to research their own brand


• Procter & Gamble

• Dow Chemical

• Pfizer

• Unilever

• Siemens

Hard for untalked about companies to research their own brand

What exactly do they make?

I prefer to chat about Lipton

I prefer to chat about Neosporin

I prefer to chat about Olay

I prefer NOT to chat about Dow


• Annie’s Bakery

• Joe’s Corner Store

• Little Rock Tackle Shop

• Lexington Spa and Nails

• Jacksonville Coffee Stop

• Rockland Best Ice Cream Shoppe

Harder for local/small businesses to research their own brand via quant


• Burger King

• WalMart

• Fruit Loops

• Pepsi

• Kroger

• Nike

• Microsoft

• Toyota

• iPad

• Pringles

Large consumer facing companies can measure anything


• People may not be aware of the brand

• People who ARE aware aren’t chatting in social media

• People may not feel like talking about the brand

• People may not feel like talking about ANY brands

SMR cannot measure incidence nor awareness


The Good Stuff: Time travel!


The Good Stuff: Daily Tracking

1.00

1.50

2.00

2.50

3.00

3.50

4.00

4.50

5.00

01

-Jan

-12

03

-Jan

-12

05

-Jan

-12

07

-Jan

-12

09

-Jan

-12

11

-Jan

-12

13

-Jan

-12

15

-Jan

-12

17

-Jan

-12

19

-Jan

-12

21

-Jan

-12

23

-Jan

-12

25

-Jan

-12

27

-Jan

-12

29

-Jan

-12

31

-Jan

-12

02

-Feb

-12

04

-Feb

-12

06

-Feb

-12

08

-Feb

-12

10

-Feb

-12

12

-Feb

-12

14

-Feb

-12

16

-Feb

-12


The Good Stuff: Thousands of Variables

Strongly Disagree

Somewhat Disagree

Neutral Somewhat Agree

Strongly Agree

Honest

Reliable

Creative

Determined

Fashionable

Kind

Funny

Classy

Confident

Sociable

Energetic

Intelligent


Finding the needle in the haystack


This is the internet

Spam

Sales Pitches

Business Discussions

Real People


This is spam

• Adderall 20/30mg IR/XR online without prescription.Contact now if interested at [email protected].

• ambien without prescription ambien online paypal buy ambien no rx online

• buy ambien without prescription buy ambien bulk buy ambien online no rx

• buy ambien without prescription buy ambien in uk buy ambien tijuana

• buy ambien without prescription buy ambien online fast shipping generic ambien safe

• http://t.co/nQ3E8MAt Buy Eprex online - Generic Eprex legally - Purchase Eprex without script

• zithromax cost buy canadian zithromax prescription online and antibiotics that treat stds


This isn’t Spam

22

RT @AriesTerms: #Aries are the sexual energizer bunnies. They keep going, & going, & going...

http://twitter.com/personsname1/status/217775548674486273








http://twitter.com/cpersonsname5/status/217775684737695744




These are sales pitches

23

• This is an excellent movie, I saw it online for free in HD at: watchmovies4free {.} org

• This is a very cool film, I saw it online for free in HD here: viewfilmsfree {.} com

• This is a great movie, I downloaded it online for free here: VIEWFILMSFREE {.} COM

• This is an excellent movie, I watched it online for free here: WATCHMOVIES4FREE {.} ORG

• "Xbox 360 Messenger Kit (Chatpad & headset), Only 10 @ HMV http://t.co/vSJK2VUH #deal"

• "New comedy DVDs and Blu-rays at HMV http://t.co/XjsH130e #deal" • "#Offers #Deals Amazing Christmas deals at HMV Blackadder Remastered:

Ultimate Edition: 6 DVD 17.99 and fr... http://bit.ly/eS2GZd"


These are not sales pitches

24

• I unlocked the The Glee Project: Sexuality sticker on @GetGlue! http://t.co/65a7aB6E

• I unlocked the The Glee Project: Sexuality sticker on @GetGlue! http://t.co/756xG3mR

• I unlocked the The Glee Project: Sexuality sticker on @GetGlue! http://t.co/8FFS3Xzj

• I unlocked the The Glee Project: Sexuality sticker on @GetGlue! http://t.co/9i9OZG1i

• I unlocked the The Glee Project: Sexuality sticker on @GetGlue! http://t.co/aa1Xbb1F

• I unlocked the The Glee Project: Sexuality sticker on @GetGlue! http://t.co/AmAgrCL1

• I unlocked the The Glee Project: Sexuality sticker on @GetGlue! http://t.co/BKCM29b6

• I unlocked the The Glee Project: Sexuality sticker on @GetGlue! http://t.co/D8wLvPzR

• "Be grateful if you used my referral link for this deal http://share.livingsocial.com/deals/...4&rui=95087725"

• "http://www.hotukdeals.com/deals/batt.../817097?page=4 stock on order but hukd seems to think its a very hot deal"

• "its part of the 5 for 10 pounds universal 100 deal on hmv http://hmv.com/hmvweb/displayProduct...RODUCT-620866#"

• "Picked this up at HMV though it was a steal. http://yfrog.com/h22plaj Still can't find a Kinect Sports deal tho"


• "HMV: what the analysts say - # Telegraph ow.ly/84WHA # HMV"

• "UK: HMVs New Loan Facilities Have Retailer Over A Barrel, Says Analyst http://goo.gl/fb/pzKi9 2:33 AM Jun 8th via Google"

• "#HMV announces another profit warning, #VerdictRetail analyst comment http://bit.ly/fD301z @ hmvtweets @ retailfocus 4:14 AM Apr 6th via bitly"

• "HMV warns on covenants as debt climbs: Embattled music and entertainment retailer says it will miss analysts... http://on.ft.com/efK5cu 3:56 AM Mar 1st via twitterfeed"

• "HMV: What the analysts say: Today's grim profits warning from HMV Group has reignited debate over the comp... http://tinyurl.com/2cdozec 2:20 AM Jan 5th via twitterfeed"

• "HMV struggles to survive amid tumbling sales: With more than 400 stores across the country, industry analysts sa... http://bit.ly/dVdgHY 12:27 PM Dec 9th via twitterfeed"

• "Jan. 10 (Bloomberg) -- Natalie Berg, an analyst at Planet Retail Ltd., discusses holiday sales reports by Debenhams Plc, Marks & Spencer Group Plc and HMV Group Plc, and the outlook for retailers. She speaks with Linzie Janis and Mark Barton on..."

These are business discussions

25


• CaDbury ;p tehehe !!! whch one z ur fav ?eish i lyk them all

• On a fyn weathr lyk ths......smbdy z cravin 4 a Cadbury chocolate

• Neva eva lyk neva chew cadbury swt infrnt of ur man or any otha cute fellow! Nt cute at al!

• My favorite player Randy Moss mite be makin ah cum back to tha NFL

• wtf would i say sheesh? "o sry bout ur balls dude“

• I shlda jus taken the credit smh ;)

• dnt giv a faak boy...they will do what ever to get a follower; twar, steal tweets, diss a country, screenmunch, dey r relentles

• Slick talkn , sneek diss n, all that ole under the table @!! Ish,, bring it 2 a brother face!!

This is hidden gold

26


These are always consumers

27


• Every research method has data quality processes

• Seek out and fix leading questions

• Seek out and recruit the right target not the most convenient target

• Seek out and shorten overly long surveys

• SMR begins with quality extraction

Data quality is top priority


What does HFCS stand for?


What does HFCS stand for?


What does BP Stand For?


What does BP Stand For?


Acronyms need special care

http://www.abbreviations.com/BP


• Traditional research sample data sources

• Are you interested in opinions from women

• Are you interested in opinions from younger people

• Are you interested in opinions from lower income people

• SMR samples data sources

Quality in sample sources


Social Media Sampling Matrix


Same brand, different result

0%

25%

50%

75%

100%

Walmart Adidas BP iPod KindleFire

Kroger Poptarts Sam'sClub

Tablet -iPad 2

TigerWoods

Social Networks - facebook.com Video - YouTube.com

Top 2 Box (Positive)


• Traditional research organizes content

• Q1: Are you a man or woman?

• Q12: How likely are you to purchase coffee?

• Q15: Which of the following brands do you use?

• SMR organizes content

Quality in Coding


in my college, people voted against this strike... wonder if their tuition fees gonna be

paid in cash or cards.

Strike

Schooling Voting

SMR Variables


Content Analysis Validity

Beefy

Heavy

Weighs a ton

Pounds sterling

Pound puppy

Tons of stuff


Content Analysis Validity

Blue ray

Blue tooth

Bobbi Brown

Charlie Brown

Fifty Shades of Grey

Grey Goose


• Ethnographies are scored by researchers

• SM data are scored by researchers

• “I would totally buy that”

• “No way in hell am I buying that crap”

Quality in Sentiment Analysis

• How long can you run a country by holding all the poor provinces hostage and saying "were gonna cut out your equalizations unless you do what we say". cuts are gonna be made, people will be unhappy when provinces cut jobs education and you're gonna have a hard time getting re-elected because you were too busy implementing your nonsense.


• This fabulous movie delicately portrayed the horrible atrocities of war, including terrifying images of torture and rape

• This fabulous movie delicately portrayed the horrible atrocities of war, including terrifying images of torture and rape

Automated sentiment analysis requires care


Sentiment analysis is never perfect

New emoticons

New acronyms

New slang

Sarcasm

Strange typos

Strange grammar

Descriptions vs Sentiment

Human scoring is only 85% accurate.

Machine scoring strives for 75% accuracy.


: )

: (

:o)

=]

=)

:^)

Use of emoticons: How smart are you?


: )

: (

:o)

=]

=)

:^)

Use of emoticons: How smart are you?

;)

:|

>;]

>:X

\,,/

\m/

d^_^b

(¬_¬)

d(>w<)b

(,_,)

( °٢° )

(V)!_!(V)

m9(^Д^)

(`･ω･´)

＿|￣|○

(｀-´)>

(´；ω；`)

ヽ(`Д´)ﾉ


LOL

FTW

WFT

ROFL

SMH

Use of acronyms

Laugh out loud

For the win/ F*** the world

What the f***

Rolling on the floor laughing

Rolling on the floor laughing my ass off

Shaking my head

To you too

Too late

Tomorrow

Forever

Too cool for you


LOL

FTW

WFT

ROFL

SMH

Use of acronyms: Still feeling smart?

2U2

2l8

2mr

4ever

2C4U

AWHFY

IMHO

BION

CUL

FCFS

JK

KMA

OMG

POS

RTF

Laugh out loud

For the win/ F*** the world

What the f***

Rolling on the floor laughing

Rolling on the floor laughing my ass off

Shaking my head

To you too

Too late

Tomorrow

Forever

Too cool for you


Sick

Whack

Dope

Mafan

Crush

Use of Slang

Good Crazy bad Good Pain in the ass Good Good Best one Slut/slob Inferior fool


Sick

Whack

Dope

Mafan

Crush

Use of Slang: Feeling inadequate now?

Clutch

Koota

Slub

Scrub

Wassock

Bomb

Burned

Dough

Aiight

Krunk

Sup

Pwn

Kicks

Dat

Lekker

Good Crazy bad Good Pain in the ass Good Good Best one Slut/slob Inferior fool


How non-researchers validate scoring

Verbatim Step 1: Auto Score

Step 2: Human Validation

What a horrible thing to happen. Can’t believe people died on a costa cruise.

Negative Agree

What a joke. Costa is giving people a big discount after the whole sinking ship thing.

Positive Disagree

We had our honeymoon on the costa concordia. It was wonderful.

Positive Agree

I hope the costa concordia captain goes to hell. Negative Agree

Plus 996 other evaluations…

The “yeah, yeah” method


How researchers validate scoring

Verbatim Step 1: Blind Score

Step 2: Blind Score

Step 3: Concurrence

What a horrible thing to happen. Can’t believe people died on a costa cruise.

Negative Negative Yes

What a joke. Costa is giving people a big discount after the whole sinking ship thing.

Negative Positive No

We had our honeymoon on the costa concordia. It was wonderful.

Positive Positive Yes

I hope the costa concordia captain goes to hell.

Negative Negative Yes

Plus 996 other evaluations…

The Correct Blind Method


1. I hate hate HATE grocery shopping at Walmart!

2. beautiful. loved it. amazing, keep up the excellent work!

3. Im eating a big mac meal as i type , and it's fkin lovely

4. just ate mcds...i feel ten pounds heavier..its a love hate

5. I love Rebecca Black

6. Yeah right. Keep arguing, at least you look smarter

7. Bouta crush this big mac...

8. dirk is just too clutch its scary

9. the black eyeliner is the bomb

10. expect 400% from you dude--no excuses. >;p

Words in context are much easier?


Is social media data normally distributed?

53

0

500

1000

1500

1 1.5 2 2.5 3 3.5 4 4.5 5

0

2000

4000

6000

1 1.5 2 2.5 3 3.5 4 4.5 5

0

1000

2000

3000

1 1.6 2.1 2.6 3.1 3.6 4.1 4.6

0

1000

2000

3000

1 1.5 2 2.5 3 3.5 4 4.5 5

0

2000

4000

6000

8000

1 1.5 2 2.5 3 3.5 4 4.5 5

0

400

800

1200

1 1.5 2 2.5 3 3.5 4 4.5 5

0

2000

4000

6000

1 1.5 2 2.5 3 3.5 4 4.5 5

0

400

800

1200

1 1.5 2 2.5 3 3.5 4 4.5 5

0

750

1500

2250

1 1.5 2 2.5 3 3.5 4 4.5 5

Junior Seau

Nissan Rogue

Neiman Marcus

Fuel Prices

Toronto Transit Commission

Sephora

Bullying

Toilet Paper

Cake Boss


Branded Social Media Data is (Mostly) Normally Distributed (n=60 million)

54

0

2000

4000

6000

8000

10000

12000

-100 -89 -79 -69 -59 -49 -39 -29 -19 -9 1 11 21 31 41 51 61 71 81 91

Extremely Negative

Extremely Positive

Neutral


• Measure census rep(representative)

• Measure frequencies and quantities precisely

• Measure geographics

• Ask follow up questions

• But we can do so much…

What CAN’T we do with the data?

55


What other brands should you ask about?

Identify the other top of mind brands that come up during conversations of your target topic. Make sure you do follow up research with them. Make sure they are included on your survey


Which celebrity should endorse the Blackberry?

57

These are the celebrities YOUR consumers want to talk about. These celebrities are already meaningful. Are they the RIGHT celebrities?


Find the positives

58

Do more of these. Don’t get rid of these. Brag about these.


Find the negatives

59

Solve these problems. Make sure consumers know you’re working on this.


Find the most common topic

60

Which issue is top of mind for your consumers? Which issue should you address first? Which great idea should you brag about the most?


Find the least common topic

61

Is a problem about to explode? Are your consumers unaware of or uninterested in your bragging points?


http://ca.linkedin.com/in/anniepettit https://www.facebook.com/anniepettit http://lovestats.wordpress.com/ http://twitter.com/lovestats

Thank you!

62

http://ca.linkedin.com/in/anniepettit

https://www.facebook.com/anniepettit

https://www.facebook.com/anniepettit

http://lovestats.wordpress.com/

http://twitter.com/lovestats


Thank you

Annie Pettit

Peanut Labs

Marketing

Introduction to social media research print version