Upload
kavita-ganesan
View
450
Download
3
Tags:
Embed Size (px)
DESCRIPTION
This talk was given at the Sentiment Analysis Innovation Summit on how to leverage large amounts of opinions to help users make decisions. Topics include methods to abstract out opinions, opinion-driven search engine and how FindiLike Hotel Search uses some of the state-of-the-art opinion-driven decision making tools.
Citation preview
Enabling Opinion-Driven Decision MakingKavita GanesanFounder, FindiLike LLC.
State of the Web… Unstructured opinions:-e-commerce sites-online directories-blog articles-review sites-social networking sites-travel sites, more…
State of the Web…
Abundance of opinions…but rarely used to help users make better & faster decisions!
Unstructured opinions:-e-commerce sites-online directories-blog articles-review sites-social networking sites-travel sites, more…
Classic example: Looking for a hotel…
Shortlist hotels by date availability
Shortlist hotels by price
Find hotels with specific amenities
• pool & spa
Find hotels that fulfill specific
opinion criteria
• “Safe neighborhood”• “Clean”• “Comfortable beds”
• Have to read 100s of reviews of different hotels
• May visit different websitesHotels.com TripAdvisor Yelp Hotels.com
Users: Highly tedious process!• Users perform own data mining• Personally done it…not fun!• No draw for users to come back to your site
decide to use other sites next time
Companies: Users are straying away from your site• Taking too long to get information needed
use some other method of booking
• Worse: Found deals elsewhere while reading reviews especially true with modern day contextual advertising
Problematic to users & organizations!
Need to use available opinions in a more intelligent & efficient way to enable decision making!
In the research world … Many methods to leverage opinions for decision making Not really used by industry!...YET
Opinion-driven decision making tools
Opinion-driven decision making tools
Opinion-Driven Search
Opinion Summarization
Opinion Trend Visualization
• Help users find entities (e.g. products, people, businesses) based on opinion preferences
• Limits set of choices in consideration from a large number options
Opinion-driven decision making tools
Opinion-Driven Search
Opinion Summarization
Opinion Trend Visualization
• Help users understand underlying opinions in large amounts of unstructured text
• Whole range of formats to abstract out opinions
Opinion-driven decision making tools
Opinion-Driven Search
Opinion Summarization
Opinion Trend Visualization
• Helps users understand change in opinion over a period of time
• E.g. Smartphone:• started out with positive sentiments• sentiments changed after
recent firmware upgrade
Opinion-driven decision making tools
Opinion-Driven Search
Opinion Summarization
Opinion Trend Visualization
Goal of task: • Rank/recommend entities • Based on how well a users “opinion requirements”
match the unstructured opinions about a set of entities
Opinion-driven search
“lightweight, responsive screen”
opinion requirements• User provided• System suggested• Various interface options
Smartphones
Opinions
New task in literature [Ganesan & Zhai IRJ 2012] [Ganesan PhD Thesis, 2013] [Choi et. al WWW’12] [Choi et. al CIKM’12]
Referred to as “opinion-based entity ranking” Practical Implementation
1. Put search over results of opinion summarization Fairly involved requires work on search & summarization Can be more accurate than other methods
2. Treat as information retrieval problem [Ganesan & Zhai 2012] [Ganesan 2013] [Choi et. al 2012a] [Choi et. al 2012b]
Extend robust IR models for this task Easily scales up & can be used with different domains
Opinion-driven search
Preference based search: find entities based on preferences• Buy LCD TV: user preferences - “rich picture” ,“no glare”• System takes in preferences and lists TV’s matching criteria
Search filters: users filter results by specific opinions• Laptop search: limit to “lightweight” laptops
Restaurant search: limit to “authentic” restaurants• Can be used in conjunction with your current search engine
Opinions can be “suggested” or “user enters”• Can also think about this as another form of faceted navigation
How can we use opinion-driven search?
Search filters: users filter results by specific opinions
How can we use opinion-driven search?
Babies R Us
Car Seat
Contextual advertising: recommend entities with similar or different opinionsShopping for Laptop:
• Recommend similar laptops to one being viewed• Viewing laptop with negative opinions: recommend
laptops with positive opinions• Brings users closer to the preferred type of laptop
How can we use opinion-driven search?
Big area of study in the research world• Sentiment classification• Aspect-based summarization• Text summarization• Co-reference resolution
Many different formats [Kim et al 2011]
• Can range from simple sentiment summary to unstructured textual summaries
Opinion Summarization
Sentiment Summary
----+
+++
source2
source1
source3
source4
source5
source6
source7
Final sentiment summary: (+ve/–ve) or (score 3/5)
-whole documents-passages-sentences-phrases
--
Gives users high level overview of underlying sentiments• Similar to overall ratings see in e-commerce sites
Widely studied in literature [Pang et al. 2002] [Pang and Lee 2004; 2005] [Dave et al. 2003] [Turney 2002] [Turney and Littman 2003]…
• Heuristic based approaches to supervised methods
Fairly easy to implement • With supervised machine learning models + training data
Issues to consider:• Lacks sufficient details
E.g. “hotel may get overall positive rating with extremely bad service”
• To make it useful: pair with other summarization methods
Sentiment Summary
Aspect-Based Summaries
Aspects Ratings
Design
Screen
Sound
Battery
4.0/5
2.0/5
3.0/5
4.5/5
smartphone xyz
Aspect-Based Summaries
Feature/Aspect IdentificationDesign, batterysound, screen
Sentiment PredictionBattery life is great… +veLong battery life… +veHorrible sound quality -ve
Star rating?
Score based approach? 4/5, 2/5
Percentage? 60% 40%
Presentation
Each step can be implemented in different ways with different levels of sophistication
aggregate
Gives more details than overall sentiment ratings Widely studied in literature – hot topic for several
years! [Lu et al 09 ] [Titov & McDonald 2008] [Hu & Liu 2004a] [Hu & Liu 2006] [Ku et al. 2006] [Popescu & Etzioni 2005] [Zhuang et al. 2006]
• Different options for implementation• Tweaked as needed to fit domain needs
Aspect-Based Summaries
Issues to consider:• Finding aspects or features in each domain
Varies from domain to domain E.g. Electronics: features for television vs. smartphones How to find features in a general and scalable way?
• Lacks detailed reasons E.g. Smartphone screen received score of 2/5? Why? Screen too small? Screen non-responsive? no way of
knowing! To give more info: complement this textual summaries
Aspect-Based Summaries
Gives more details than structured summaries Easiest implementation method:
• Traditional text summarization • Select few sentences from text to make up summary
Textual Opinion Summaries
The xyz smartphone is easy to use and has a very bright screen. I mostly love that screen is so responsive.
Not recommended for opinions! [ Ganesan et al COLING’10 ]
• Can introduce bias when selecting sentences Can select sentences that don’t represent major opinions
“Battery life is great (10), but this phone is too small for me (1)” Miss out key opinions when not selecting enough sentences
• Not suitable for smaller screens verbose if you keep adding sentences
Textual Opinion Summaries
Over last few years, people have been looking into micro-summarization approaches [Yatani et al CHI’11] [Ganesan et al COLING’10] [Ganesan et al WWW’12] [Khabiri 2013, PhD Thesis] [Potthast & Becker ECIR’10]
Concise summaries that represent key opinions in large amounts of text
Micropinion Summaries
Micropinion Summaries [Ganesan et al COLING’10, Ganesan et al WWW’12] http://www.findilike.com/demo.jsp
Summary generated on reviews of Acura 2007 from Edmunds.com
-Concise and readable-Picks up on aspects naturally-Contains important details -Displays how many people said it / how many times appeared
Micropinion Summaries [Ganesan et al COLING’10, Ganesan et al WWW’12] http://www.findilike.com/demo.jsp
-display snippets-display whole passages-up to application
Highly flexible• Adjust summaries to screen size
longer summaries for larger screens shorter summaries for smaller screens
• Limiting duplicates, increasing diversity
• Don’t need to know aspects/features in advance Generate summaries for arbitrary aspects You can limit to certain aspects if needed (e.g. battery, sound)
• Recent user study showed with micro-summaries:[Yatani et al CHI’11] Users took significantly less time to decide on a restaurant
compared to reading full reviews Micro-summaries shown to be effective in decision making
Micropinion Summaries
Practical Implementation• Not too hard to implement with n-gram methods • More sophisticated approaches – keyphrase extraction,
multi-sentence compression [Fillipova COLING’10] [Ganesan et al COLING’10] [Ganesan et al WWW’12] [Boudin & Morin 2013]
Issues to consider:• Simple methods: chances of generating junk• E.g. iPhone 5s:
“the battery life is”, “the iPhone 5s” meaningless to user• You want: “battery life is short”, “iPhone 5s is reliable”
Micropinion Summaries
Comparative summaries• compare contradicting opinions
Entity based summarization
Other summarization formats
Decision making tools shown:• Are powerful, but have their limitations • Just as in any new state-of-the art methods
Pair techniques wisely + take edge cases into consideration powerful solution!
How FindiLike Hotel Search uses opinion-driven decision making tools
Example Implementation
Search system: Helps users find hotels by preferences• Unstructured opinion preferences
“friendly service”, “clean”, “good views” • Common structured preferences
price [$0-$100], distance [5 miles from campus]
Beyond search: Support for analysis of hotels• Micropinion summaries • Multi-word buzz phrases of reviews
Developed to showcase published ideas: related to enabling opinion-driven decision making [Ganesan et al COLING’10, Ganesan & Zhai IRJ, Ganesan & Zhai WWW’12, Ganesan 2013 PhD. Thesis]
FindiLike Hotel Search
Let us say…
• Visiting: Los Angeles• Main opinion criteria: hotels said to be clean & safe
Input requirements as natural keywords
Hotels ranked by how well preferences are matched in opinions
• Preferences entered• How well preferences
matched (stars)• Snippets to show why hotel
was selected
More reviews on why the hotel matched ‘clean’
• Can add more preferences: ‘cheap’, ‘good room service’
• Remove preferences • Combine with structured
preferences – price, distance
Benefits of opinion-driven search: • Limits number of hotels in consideration • No need to read reviews to find hotels matching
preferences
Micropinion summaries:• Concise opinion summaries • Highlights key opinions about
hotel
Benefit of micropinion summaries: • Helps understand underlying opinions within reviews• Further refine choices by knowing key opinions
Benefit of multi-word buzz phrase: • Helps users explore the opinion space. Learning
through exploration• Extremely suitable when opinions are sparse
Multi-word buzz phrases:• Highlights common phrases • Weighted by frequency and
readability
snippets related to “parking lot”
Benefit of multi-word buzz phrase: • Helps users explore the opinion space. Learning
through exploration• Extremely suitable when opinions are sparse
Improves user productivity • Eliminates need to read large number of opinions
Don’t need to perform own data mining
Helps user retention• Users can make better, faster and more informed
decisions on your site Likely to come back
Improves conversion rates• Users can make faster decisions
More likely to complete transaction on your site
The need for opinion-driven applications
Are users asking for this? Yes!
QUORA: Are there any service to summarize yelp reviews? I want to check the reviews of a roofing company but there are hundreds of them. Can I get some kind of summary (and maybe bar graph) without reading all of them?
Contact: Kavita Ganesan Email: [email protected] web: kavita-ganesan.comCompany web: findilike.com
Thank you! Questions?