Exploring and Understanding Social Medialih307/slides/socialmedia.pdfWhat is Social Media So, before 2.0 ... Static web pages Minimum user interactions ... Sampled approximately 500

Exploring and Understanding Social Media

Liangjie Hong

Dept. of Computer Science & EngineeringLehigh University

November 6, 2009

Liangjie Hong (Lehigh Univ.) Social Media November 6, 2009 1 / 59

Outline

1 What is Social Media

2 Question Answering in Social Media

3 Enhancing Web Search with Social Media


What is Social Media

So, before 2.0 ...

• Static web pages

• Minimum user interactions

• Information creators vs. Information consumers

• Just an extension of libraries?



So, before 2.0 ...







So, before 2.0 ...







So, before 2.0 ...







So, before 2.0 ...







We are still in Web 1.0 ¨̂






Weare still in Web 1.0 ¨̂









A Typical Search Engine for Web 1.0

• Full-text indexing

• Query → Term Weighting + Link Analysis

• Ranked results






• Ranked results






• Ranked results






• Ranked results



Static Web → Dynamic Web

• Mailing lists

• Discussion Boards, Forums

• Wikipedia, Question Answering Portals

• MySpace, Facebook, Linkedin

• Social Bookmarking

• Twitter

• ...




• Mailing lists





• Twitter

• ...




• Mailing lists





• Twitter

• ...




• Mailing lists





• Twitter

• ...




• Mailing lists





• Twitter

• ...





So What?

• Changed the way search engine works?

• Changed users’ expectation?

• Changed business

• Changed marketing

• ...



So What?





• ...



So What?





• ...



So What?





• ...



The Challenges

• Document boundary

• Independent assumption ? → Information Networks !

• Presentation

• Natural language queries

• Need answers!

• Need to communicate

• ...



The Challenges



• Presentation


• Need answers!


• ...



The Challenges



• Presentation


• Need answers!


• ...



The Challenges



• Presentation


• Need answers!


• ...



The Challenges



• Presentation


• Need answers!


• ...



The Challenges



• Presentation


• Need answers!


• ...



The Challenges



• Presentation


• Need answers!


• ...


Question Answering in Social Media






How do we find answers on the Web?



How do we find answers on the Web?







Two Sources


• Question Answering Portals (Community Question Answering)



Two Sources





Two Sources





Question Answering on Discussion Boards

Difficult?

• Mixed content: comments, news, tutorials, personal experiences

• Quality: no punctuation, spell errors ...




Difficult?






Difficult?






Difficult?






What is a question / answer? (document boundary problem!)

• a sentence

• a paragraph

• several paragraphs

• a post

• ...

Simplified version: One question → One answer





• a sentence

• a paragraph


• a post

• ...






• a sentence

• a paragraph


• a post

• ...






• a sentence

• a paragraph


• a post

• ...





Two sub-tasks

• Question detection

• Answer detection

• a classification problem

• simple features (combination) vs. NLP




Two sub-tasks








Two sub-tasks








For Questions

• Question mark (1 feature)

• 5W1H words (6)why, what, where, which, when, how

• Thread length (1)total number of posts

• Authorship (1)#first post/#total posts

• N-gram (1000-3000)




For Questions

• Question mark (1 feature)

• 5W1H words (6)why, what, where, which, when, how

• Thread length (1)total number of posts

• Authorship (1)#first post/#total posts

• N-gram (1000-3000)




For Answers

• Post position (2)

• Authorship (1)

• N-gram (1000-3000)

• Stopwords (571)

• Query Likelihood Model (Language Model) (1)




For Answers

• Post position (2)

• Authorship (1)

• N-gram (1000-3000)

• Stopwords (571)

• Query Likelihood Model (Language Model) (1)




Datasets

• PhotographyOnTheNet (http://www.photography-on-the.net/)

721, 442 threads

• UbuntuForums (http://www.ubuntuforums.org)

555, 954 threads

• Sampled approximately 500 threads for each sub-task and dataset




Classification Methods

• Manually labeledquestions vs. non-questions

one best answer per thread

• 10-fold cross validation

• libSVM for classification

• Measured performance by Precision, Recall, F-Measure, Accuracy




Comparisons to existing methods

• For questionsPart-Of-Speech tagging + Sequential Pattern Mining

• For answersGraph-based model incorporated with inter-posts relevance, authorship and similarity



Question Answering on Discussion BoardsQuestion Detection

QM 5W1H Length Author SPM N−gram0

0.2

0.4

0.6

0.8

1

F−

Mea

sure

0.6410.679 0.707 0.712

0.7540.833

Single Feature on UbuntuForums




5W1H Length QM SPM Author N−gram0

0.2

0.4

0.6

0.8

1

F−

Mea

sure

0.500

0.636 0.664 0.6710.755 0.775

Single Feature on PhotographyOnNet




A+LEN Q+5+LEN A+Q+5+LEN SPM N−gram0

0.2

0.4

0.6

0.8

1

F−

Mea

sure

0.716 0.722 0.7540.746

0.833

Combined Features on UbuntuForums




SPM Q+5+LEN N−gram A+Len A+Q+5+Len0

0.2

0.4

0.6

0.8

1

F−

Mea

sure

0.6710.711

0.7750.843

0.876

Combined Features on PhotographyOnNet



Question Answering on Discussion BoardsAnswer Detection

GQL Stop NG LM POSI AUTH0

0.2

0.4

0.6

0.8

1

F−

Mea

sure

0.620 0.640 0.663 0.6820.737

0.765

Single Feature on UbuntuForums




GQL LM NG Stop AUTH POSI0

0.2

0.4

0.6

0.8

1

F−

Mea

sure

0.5910.659

0.706 0.712 0.735

0.827

Single Feature on PhotographyOnNet




Stop+NG LM+Stop LM+POSI LM+A POSI+Stop LM+POSI+A POSI+A0

0.2

0.4

0.6

0.8

1

F−

Mea

sure

0.760 0.761 0.770 0.786 0.798

0.946 0.952

Combined Features on UbuntuForums




Stop+NG LM+A LM+Stop LM+POSI POSI+Stop LM+Stop+A POSI+A0

0.2

0.4

0.6

0.8

1

F−

Mea

sure

0.712 0.734 0.740

0.8270.873

0.970 0.975

Combined Features on PhotographyOnNet



Summary

• Question DetectionN-gramAuthorship+Question Mark+5W1H+Length

• Answer DetectionPosition, AuthorshipPosition+AuthorshipLanguage Model+Position+Authorship



Two Sources





Two Sources







Question Answering on CQA

Problems

• same or similar questions

• different category

• different time

• Read them all?

• Trust them all?




• content

• user reputation




• content

• user reputation




HITS-like Scheme

questioner - hubanswerer - authority

User

User

User

User

User

User

User

Answerer

Questioner




HITS-like Scheme

questioner - hubanswerer - authority User

User

User

User

User

User

User

Answerer

Questioner




PageRank-like Scheme

User User

User

User

User




PageRank-like Scheme

User User

User

User

User




Incorporating topics

• Combine content for users

• Unsupervised topic model (Probabilistic Latent SemanticAnalysis)

• Topical PageRank




Incorporating topics

• Combine content for users

• Unsupervised topic model (Probabilistic Latent SemanticAnalysis)

• Topical PageRank




Other heuristics

• Z = a−n/2√n/2

= a−q√a+q

• SimpleRank = θ ∗ a + (1− θ) ∗ q




Dataset

• Yahoo Answers!14,122 questions + 45,814 answers12,731 users

• Strict & RelaxedPrecision@1, MRR, Precision@10, MAP

• BM25 + User Reputation




EvaluationP@1(S) MRR P@1(R) P@10 MAP0.0857 0.1414 0.3410 0.3170 0.3081












Summary

• user reputation helps retrieval

• but may not help question answering


Enhancing Web Search with Social Media







Integrating Search Results with Twitter

• When

• How

• What




Integrating Search Results with Twitter: When

P(S = s|Q,T ) =P(Q,T |S = s)P(S = s)

P(Q,T )




Integrating Search Results with Twitter: When

Assumption 1: P(Q,T |S = s) = P(Q|T , S = s)P(T |S = s)Assumption 2: P(Q,T ) = P(Q)P(T )

P(S = s|Q,T ) ∝ P(Q|T , S = s)P(S = s|T )

P(Q)




Integrating Search Results with Twitter: How

Document Boundary Problem!

• “Virtual Document”, an extension of inverted index

• “cat” → 10, 23

• “cat” → “dog”, “food”, ...




Integrating Search Results with Twitter: What

Presentation Problem!

• “Tag Cloud”

• “Top Messages”




Dataset

• 300 queries from Google Insights

• MSN query log

• 29,762,170 individual messages and 2,349,723 distinct terms




When to integrate

Top 10 Bottom 10

iddaa Rising 1 (Sports) tv Top 4 (Entertainment)mynet Rising 4 (Internet) real Top 2 (Real Estate)emlak Rising 3 (Real Estate) free Top 3 (Entertainment)kariyer Rising 3 (Business) us Top 2 (Society)milliyet Rising 3 (News) blog Top 9 (Social Network)myvideo Rising 4 (Photo Video) video Top 8 (Entertainment)bitesize Rising 6 (Science) loveminiclip Rising 8 (Game) semappy Rising 5 (Reference) as

meb Rising 4 (Society) you tube Top 2 (Entertainment)




When to integrate

0 50 100 150 200 250 300 35010

0

102

104

106

−lo

gP(s

|QT

)

10−6

10−4

10−2

100

P(Q

)




What to present

Query Methods Tag Cloud

cakeTopic Model eat eating cheese white tea pizza w bar apple yummy

Retrieval Model sugarery evaarrs heavencarrot mayyyyy bannanana desset booop spelndid

googleTopic Model twitter via blog tv facebook web iphone internet video de

Retrieval Model appenginge lookskindawierd billenkoek twotiny fillhoo fluidapp

educationTopic Model public state power law american world daily president united america

Retrieval Model parentale iguides #edu estroke animalsays sombiz tewksbury globalhighered

boatTopic Model looking state heavy cross happy square office garden board lake

Retrieval Model nmma politicising maxum tuffin sevylor boatshoes lawrencetown

hondaTopic Model motor ford audi posted mini toyota sport cars vw diesel

Retrieval Model warungmobil ayutthya #honda gresini sohc dohc honda catback crosstour crx




What to present

Query Methods Tag Cloud

cakeTopic Model #wine #ff #followfriday #recipe #sports #music #money #mac #gfree #food

Retrieval Model #baking #hw #mustfollow #chocolate #sales #recipe #fun

googleTopic Model #twitter #followfriday #internet #google #facebook #myspace #wikipedia #marketing

Retrieval Model #groovy #latitude #tweefind #gps #horticulture #downloads #googlereader

educationTopic Model #tweetmyjobs #followfriday #marketing #job #tcot #jobs #twitter #freedom

Retrieval Model #edu #scholarship #massteacher #daleyforum #jobmatcha #advice #dc #zimbabwe

boatTopic Model #followfriday #job #photography #tweetmyjobs #music #ff #sgp #art #camera #san

Retrieval Model #sailing #dhl #finalfour #gadgets #tech #news #fb

hondaTopic Model #vw #golf #followfriday #tweetmyjobs #thinkofnick #ebay #firefox #bmwland

Retrieval Model #honda #bumpers #decals #mazda #car #oregon #toyota #automotive




Sample “Top Twitter Messages” for Query “bikes”Community Discovery Top

#Nelson #eBay bikes http://bit.ly/fjOUi@Rebecca Black Okay. The bikes are in the back. You can use @Accident Prones bike. Don’t for get the helmet.Leave it to kristen we were getting the bikes from the car she manages to hit me in the face with the handleand give me a swollen eye#fbhttp://tinyurl.com/lq9qnmYBP - Bikesusing the bikes and the skateboard as cross training for the up-coming ski season.

Community Discovery Bottom

@dPain119 makesa my dreams come trueWhat a fabulous Saturday this has been! It’s great to be out and see so many bikes and hot rods!!oh it sunday , that means my lap top go off all night as it gone off 6 times in last 10 mins,and bikes will be speeding on bye passDecided to give my kids bikes away so they wont try to ride them on my 2 story roof!Looks like a beautiful day. Coffee: check! Bikes and all gear: check! Heading to N.Shore for a day up above Dillingham. Long climb: Master!

Retrieval Model

@ohjp bikes bikes bikes#Nelson #eBay bikes http://bit.ly/fjOUihttp://tinyurl.com/lq9qnmYBP - BikesBikes http://yfrog.com/67pu9sjhttp://bit.ly/DwfS8 .....nice bikes





Social Media

• interesting

• challenging

• emerging research area

Thank you!



Social Media

• interesting

• challenging

• emerging research area

Thank you!


Documents

Exploring and Understanding Social Medialih307/slides/socialmedia.pdfWhat is Social Media So, before 2.0 ... Static web pages Minimum user interactions ... Sampled approximately 500