Market Research Social Media Listening for Challenges and ... · "business models" OR "content management" OR "crowd sourcing" OR "customer service" OR "disruptive technologies" OR

Mike Moran

Challenges and Solutions for Social Media Listening for Market Research

Text Analytics WorldChicago

June 22. 2016

© 2013 Converseon Inc. Proprietary and Confidential2

How accurate do we need to be?


It depends on what you are doing

▪ Market Research

▪ Product Development

▪ Reputation Management

▪ Brand Awareness/Image

▪ Crisis Management

▪ Sales Leads

▪ Recruiting

▪ Customer Service


What do you need to use listening for?

▪ Market Research





▪ Sales Leads

▪ Recruiting


Need High Accuracy Less Accuracy OK

Aggregated data OR Individual postings


A familiar example

● Most of us consider Google the most accurate search engine in the world

● But this search produces the first accurate result here

● And it doesn’t bother us...but what about tougher problems?


There’s never been more noise and less signal

• How does Sprint analyze what is being said about their company?

• They can’t just search for the word “sprint”

• It’s too common a word


• Suppose 60% of theconversations containingthe word “sprint” havepositive sentiment

• What does that mean?

• What decision can you make?

• Isolating the right conversation makes aggregated statistics possible, just as the right sample enables correct survey results

Without the right conversations, aggregation fails

1


• Some names will work fine: Jetta, Touareg…

• But what about these models?

• Beetle

• Up!

• Fox

• Polo

• Golf

“Word spotting” doesn’t find the right conversation

1


1

• “sprint” AND “phone”• “sprint” AND “cell”• “sprint” AND NOT “car”• “sprint” AND NOT “race”• “sprint” AND NOT “run”

• Sprint sucks• Sprint’s phones are awful• Sprint drops calls in the car• Sprint is losing the 4G race• My Sprint app won’t run

Can Boolean searches solve the problem?

If you use this Boolean You lose this tweet

Boolean searches simply are not up to this challenge


1

• Few can do it without getting hurt

• And there are many things you just can’t do that way:("enterprise 2.0" OR "Social Biz" OR "social business" OR "#SocBiz" OR "#SocialBiz" OR "#socialbusiness" OR SocBiz OR SocialBiz OR socialbusiness OR "enterprise social business" OR "enterprise social network" OR "enterprise social grid" OR "social CRM" OR "social BPM") AND ("activity stream" OR "activity streams" OR "application development" OR "best practices" OR "business model" OR "business models" OR "content management" OR "crowd sourcing" OR "customer service" OR "disruptive technologies" OR "disruptive technology" OR "human resources" OR "information technology" OR "next generation social" OR "open social" OR "organizational culture" OR "predictive analytics" OR "product lifecycle management" OR "social analytics" OR "social mail" OR "social portal" OR "social portals" OR "social commerce" OR "customer experience management" OR "innovation management" OR "social app" OR "social application" OR "social applications" OR "social apps" OR "social business app" OR "social business application" OR "social business applications" OR "social business apps" OR "social collaboration" OR "social computing" OR "social learning" OR "social media" OR "social network" OR "social networks" OR "social platform" OR "social platforms" OR "social software" OR "social softwares" OR "social technologies" OR "social technology" OR "soft ware" OR "tool kit" OR "unified telephony" OR "analytics" OR "app" OR "application" OR "applications" OR "apps" OR "benefit" OR "benefits" OR "blog" OR "blogs" OR "bookmark" OR "bookmarks" OR "BPM" OR "collaboration" OR "collaborative" OR "commerce" OR "communication" OR "communities" OR "community" OR "compliance" OR "innovation" OR "connection" OR "connections" OR "CRM" OR "crowdsourcing" OR "engagement" OR "ERM" OR "information" OR "intelligence" OR "intelligent" OR "interaction" OR "interactive" OR "internet" OR "interoperability" OR "learning" OR "mail" OR "marketing" OR "mobile" OR "network" OR "nimble" OR "optimization" OR "optimized" OR "organization" OR "organizational" OR "organizations" OR "platform" OR "platforms" OR "portal" OR "portals" OR "productive" OR "productivity" OR "risk" OR "risks" OR "sales" OR "software" OR "softwares" OR "technologies" OR "technology" OR "tool" OR "toolkit" OR "tools" OR "transparency" OR "transparent" OR "value" OR "voIP" OR "wiki" OR "collective intelligence" OR "customer self service" OR "social grid" OR "content" OR "social content" OR "insights" OR "social selling" OR "social communications" OR "collaborative communications" OR "human capital management" OR "business process management" OR "enterprise resource management" OR "enterprise resource planning")

• 15% accuracy, despite great effort

Boolean search queries carve with a chainsaw


• Relevance feedback allows you to be in control

• “Social business”with relevancefeedback was85% accuratewith less thanone hour’s work

How can you isolate the right conversations?

1


2“This car handles like I am steering an aircraft carrier.”

• You know that these are negative, but there is no word to tell the algorithm

• These would be marked neutral by most algorithms

Word-spotting fails for sentiment, too

“Volkswagen never told me I had no spare tire.” “Amazon wouldn’t

refund my money.”


2

• Humananalystsdecide thecorrect answersfor the training data

• Machine learning algorithms identify distinguishing patterns that are tested for accuracy

• Those patterns are used to classify new data with a level of confidence

Machine learning relies on human knowledge

Knowledge-based

Resources

Machine Learning

System Test Data

Training Data


The same words mean different things

• An “unpredictable” movie is good, but “unpredictable” braking, not so much

• We like “small” cell phones but not “small” legroom

• “Faded” jeans are good, butnot “faded” interiors

Word-spotting fails because words are ambiguous

Your industry has its own lexicon

3


• Unpredictable braking

• Small legroom

• Faded interiors

• What if you had a modeljust for the automotiveindustry?

• Or your automotivecompany?

Industry and client-specific models raise accuracy

3


•30% ofdocumentswere rated as “mixed” sentiment

•Some platforms classify this entire document as neutral

4

Whole documents aren’t usually positive or negative

Duct-Taped Piglets and Other

Atrocities at Walmart Pork

Supplier: According to the

Sustainability and Stewardship report

on Seaboard Farms website, the

company is committed to proper

animal care and has a moral and

ethical obligation to the humane

treatment of animals.

This makes me sick!! Walmart I will

never buy your meat again!!

Example: A Facebook post about Walmart


• Negative

• Neutral

• Negative

Analyzing individual mentions yields insights

Duct-Taped Piglets and Other

Atrocities at Walmart Pork

Supplier: According to the

Sustainability and Stewardship report

on Seaboard Farms website, the

company is committed to proper

animal care and has a moral and

ethical obligation to the humane

treatment of animals.

This makes me sick!! Walmart I will

never buy your meat again!!

4

Maybe the whole

document is neutral, but is that the

insight here?


• Is this tweet positive or negative?

• It depends on whether you work for Verizon or Sprint

• Often, your viewpoint is what makes something positive or negative, but most text analytics systems are not up to the challenge

Sentiment depends on viewpoint

5


5

• Instead of vanilla sentiment…

• Can you specify your point of view?

• Can you have the entities automatically recognized?

You must be able to share your viewpoint


• Consider the following two tweets:

• “I hate Sprint”

• “I am so disappointed in Sprint”

• It’s the difference between anger and sadness

• Consider the difference in your approach between an angry customer and someone who is rooting for you but was let down

Sentiment is not enough for every decision

6


• With onlysentimentanalysis, youcan’t knowthe real story

• Emotion analysis gives you the deeper insights that drive better decision making

Emotion goes beyond simple sentiment analysis

6


• It had a slightly buttery off-note

• I disliked the aftertaste

• The taste was decidedly foul

• I would never eat this again

• The taste of it turned my stomach

• This the worst tasting #@&*% ever

Not all negative conversations are created equal

7


• Knowing the difference between “negative”and “intensely negative” helps you identifythe most critical conversations

• Your customer serviceteam can go after thebiggest problems

Intensity lets you focus on the biggest issues

7


What do you need to use listening for?

▪ Market Research





▪ Sales Leads

▪ Recruiting


Need High Accuracy Less Accuracy OK

Aggregated data OR Individual postings


Training systems always have a human in the loop

● Better systems emphasize inter-coder agreement

● Crowdsourcing can lower costs


But we often totally automate the run-time system


Semi-supervision keeps moderation to a minimum

Can run-time systems use humans in the loop?


● Study looked at two different companies in two different industries

● One company showed social media sentiment predicted stock price changes

● The other company showed social media sentiment predicted brand tracking study results

Source: http://journals.ama.org/doi/abs/10.1509/jmr.12.0424

Can these methods be predictive?

Documents

Market Research Social Media Listening for Challenges and ... · "business models" OR "content management" OR "crowd sourcing" OR "customer service" OR "disruptive technologies" OR