ABSAAspect-basedsentimentanalysisof
customerreviewsOrphée DeClercq
Séminaires duCENTAL28October2016
UCL:greatbeeratthe“cercles”,butverydirrrrty!!
GOOD TEACHERSSHOPPING
Sentimentanalysis˚Early2000s:Wiebe(2000)Pangetal.(2002)…è newswiretext
RiseofWeb2.0applications2010-2016:• 20,000GoogleScholar• 731papersinWoSè user-generatedcontent
Sentimentanalysis• Opinionpolls,surveys• SentimentanalysisonUGC:
Ø Totrackhowabrandisperceivedbyconsumers(Zabin &Jefferies,2008)
Ø Formarket(Sprenger etal.,2014),electionprediction(Bermingham &Smeaton,2011)
Ø Todeterminethesentimentoffinancialbloggerstowardscompaniesandtheirstocks(O’Hareetal.,2009)
Ø Byindividualswhoneedadviceonpurchasingtherightproductorservice(Dabrowski etal.,2010)
Ø Bynonprofitorganizations,e.g., forthedetectionofsuicidalmessages(Desmet,2014)
Ø …
SentimentanalysisCoarse-grained:documentorsentence=POS |NEG |NEUTRAL
àDoesnotallowtodiscoverwhatpeople likeanddislikeexactly.
àNotonlyinterested ingeneralsentimentaboutacertainproduct,butalsointheiropinionsaboutspecificfeatures,partsorattributesofthatproduct.
Fine-grained: “almostallreal-lifesentimentanalysissystemsinindustryarebasedonthislevelofanalysis”(Liu,2015,p.10).
ABSAAspect-based(orfeature-based)sentimentanalysissystems
focusonthedetectionofallsentimentexpressionswithinagivendocumentandtheconceptsandaspects (orfeatures)towhich
theyrefer.
• VanHee etal.(2014):Coarse-grainedSAonTwitter• DeClercq etal.(2015):ABSA(Englishresto)• DeClercq (2015):SemEval ABSA(Dutchresto)• DeClercq andHoste (2016):ABSA(Dutchresto,smartphones)• Pontiki etal.(2016):SemEval ABSA8languages,4domains• 2016-2017:valorisation project(variousdomains,languages)
ABSAThebestresearch=teamresearch
Overview① Introduction② TaskDefinition③ DatasetsandAnnotation④ Subtasks
ØAspectTermExtractionØAspectTermCategorizationØAspectTermPolarityClassification
⑤ Challenges⑥ Conclusion
Overview① Introduction② TaskDefinition③ DatasetsandAnnotation④ Subtasks
ØAspectTermExtractionØAspectTermCategorizationØAspectTermPolarityClassification
⑤ Challenges⑥ Conclusion
TaskdefinitionThereexistmanyreferenceworks(Pang&Lee,2008,Liu2012,Liu2015):
DefinitionofanopinionbyLiu(2012):
“Anopinionisaquintuple,(ei;aij ;sijkl;hk;tl),whereei isthenameofanentity,aij isanaspect ofei,sijkl isthesentimentonaspectaij ofentityei,hk istheopinionholder,andtl isthetimewhentheopinionisexpressedbyhk.Thesentimentsijk ispositive,
negative,orneutral,orexpressedwithdifferentstrength/intensitylevels.”(pp.19-20)
è Automaticallyderivingquintuples=fivedifferenttasks
Taskdefinition
1.Entityextraction+ categorization
Extractallentityexpressionsinadocumentcollection,andcategorizeorgroupsynonymousentityexpressionsintoentityclusters.
ENTITYCATEGORY
2.Aspectextraction+categorizationExtractallaspectexpressionsoftheentities,andcategorizetheseaspectexpressionsintoclusters.Theseaspectscanbebothexplicitandimplicit.
AmbienceFood Service
Restaurant
Extractopinionholdersforopinionsfromtextorstructureddataandcategorizethem.
3.Opinionholderextraction+categorization
Extractthetimeswhenopinionsaregivenandstandardizedifferenttimeformats
4.Timeextraction+standardization
4.AspectsentimentclassificationDeterminewhetheranopiniononanaspectispositive,negativeorneutral,orassignanumericsentimentratingtotheaspect.
AmbienceFood Service
Restaurant
J J
J
J
Taskdefinition
Derivedquintuples:• (Uma,Food,positive,ReviewerX,May-31-2016)• (Uma,Ambience, positive,ReviewerX,May-31-2016)• (Uma,Service,positive,ReviewerX,May-31-2016)• (Uma,Restaurant,positive,ReviewerX,May-31-2016)
• (Uma,Food,positive,ReviewerX,May-31-2016)• (Uma,Ambience, positive,ReviewerX,May-31-2016)• (Uma,Service,positive,ReviewerX,May-31-2016)• (Uma,Restaurant,positive,ReviewerX,May-31-2016)
èABSAofcustomerreviews:Ø AspectExtractionØ Apsect CategorizationØ Apect sentimentclassification
Taskdefinition:customerreviews
SemEval taskDescription
(Pontiki etal.,2014,2015,2016)
META-DATA
Overview① Introduction② TaskDefinition③ DatasetsandAnnotation④ Subtasks
ØAspectTermExtractionØAspectTermCategorizationØAspectTermPolarityClassification
⑤ Challenges⑥ Conclusion
CustomerreviewsPreviousresearchMoviereviews(Thet etal.2010),electronicproducts(HuandLiu2004,BrodyandElhadad 2010),restaurants(Ganu etal.2009).
è Difficulttocompare
SemEval sharedtaskOnlinedatacompetition:everyoneworksonthesamedata.
èBettertocompareèStateoftheart
SemEval benchmarkdataèThreerunsofthetask(2014,2015&2016)è Lotsofdataindifferentdomains&languages
AnnotationGuidelinesareavailableonline:http://goo.gl/wOf1dX
Threesteps:
I.Allexplicitandimplicittargets-thewordorwordsreferringtoaspecificentityoraspect- areannotated.II.Thesetargetsareassignedtodomain-specificpredefinedclustersofaspectcategories.III.Sentimentexpressedtowardseveryaspectisindicated.
Annotation
ExperimentaldataTrainandtestsplithavebeencreatedforallSemEvaldatasets
èFocusonDutch(restaurantreviews)300reviewsfortraining(development)100reviewsfortesting(held-out)
èExplainthepipelinewedevelopedèStateoftheartapproachesandresultsonEnglish(restaurantreviews)
Overview① Introduction② TaskDefinition③ DatasetsandAnnotation④ Subtasks
ØAspectTermExtractionØAspectTermCategorizationØAspectTermPolarityClassification
⑤ Challenges⑥ Conclusion
ASPECT TERM EXTRACTION
ASPECT CATEGORY CLASSIFICATION
Term Extraction with TExSIS
Lexical• Bag-of-words
Features for category classification
Preprocessing(LeTs)
TermhoodUnithood
AdditionalFiltering
Features for polarity classification
Subjectivity Heuristic
Semantic• Cornetto• DBpedia• Semantic roles
ASPECT POLARITY CLASSIFICATION
Lexical• Token and character n-grams• Sentiment lexicons• Word-shape
PipelineforDutch:overviewTastypizza,butrudewaiter.
pizza à FOOD_qualitywaiterà SERVICE_general
FOOD_qualitySERVICE_general
AspectTermExtractionExtract allaspectexpressionsoftheentities.
Subjectivity Heuristic
Term Extraction with TExSISPreprocessing
(LeTs)TermhoodUnithood
AdditionalFiltering
Onlywhensubjective!Lexicons
• Pattern(ref)• Duoman (ref)
TExSIS =hybrid system combining linguistic and statistical information (Macken et al. 2013)
Linguistic =whichwords?• PreprocessingusingLeTs (VandeKauter etal.2013)• PoS patterns(i.e.nouns,nounphrases)
Statistical =aretheyterms?• Termhood,unithood measures (LL,c-value)
Additionalfiltering…
AspectTermExtractionTExSISoutput:
Aftera[goodappetizer]our[mother]ordereda[pizzamargherita],whichwasdivine!
…Additionalfiltering• Subjectivity(basedonsamelexicons)• Semantic
– Cornetto(Vossen etal.2013):synsets lookforhypernym-synonymlinks.
– DBPedia (Mendesetal.2011):tagtermswithDBPediaSpotlightandlookforcategories.
AspectTermExtraction
Additionalfilteringoutput:Afteragood[appetizer]ourmotherordereda[pizza
margherita],whichwasdivine!
AspectTermExtractionResultsTrainingdatasplitindevtrain (250)anddevtest (50)Bestsettingonheld-outtestset(100).Evaluationmetrics:precision,recallandF-1
AspectTermExtractionStateoftheartEnglishSupervisedmachinelearningapproachesmostsuccessfulSequential labelingtask(IOB2annotation~NER)
Toh andSu(2016)=topsystem• CRFclassifier• NEfeatures• Additional featuresfromRNN(Liu,Joty &Meng,2015)• 72.34F-1
AspectTermCategorizationCategorizeallextractedaspectexpressions.
Classificationtask• Predefinedcategories• Multiclassproblem:
Ø MaincategoriesØ Subcategories
Lexical• Typicalbag-of-words: tokenunigram
Lexico-semantic• Cornetto(insynset orhypernym/hyponymofmaincats)• DBPedia (belongtouniquecategories)
Semanticroles• Termevokessemanticrole,whichrole(Thefoodtasted goodvsThefoodjustcosttoomuch)
ASPECT CATEGORY CLASSIFICATION
Features for category classification
Lexical• Bag-of-words
Semantic• Cornetto• DBpedia• Semantic roles
AspectTermCategorizationResultsTen-foldcrossvalidationontrainingdata.LibSVMRound1:graduallyaddingmorefeaturesRound2:jointoptimization,featuregroupsvsindividualfeaturesBestresultsonheld-outtestAccuracy
AspectTermCategorizationStateoftheartEnglishSupervisedmachinelearningapproachesmostsuccesful
Toh andSu(2016)=topsystem• Individualbinaryclassifierstrainedoneachcategory
(combined)• Lexicalbagofwords(unigram,bigram)• Lexical-semantic:clusterslearnedfromlargereferencecorpus• Additional featuresfromCNN(Severyn &Moschitti,2015)• 73.031F-1
AspectPolarityClassificationDeterminewhetheropinionisPOS |NEG |NEUTRAL
Three-wayclassification
Tokenandcharactern-gramfeaturesunigram,bigramandtrigram(tok)&trigram,fourgram (char)
Sentimentlexicon• DuoMan andPatternlexicon,matchespos,neg,neut
Word-shape• UGCcharacteristics,character ofpunctuationflooding
(cooooool!!!!!),lasttokenhaspunct,capitalizedtokens
Features for polarity classification
Lexical• Token and character n-grams• Sentiment lexicons• Word-shape
AspectPolarityClassificationResultsTen-foldcrossvalidationontrainingdata.LibSVMDefault:allfeaturesJointoptimization:individualfeatureselectionBestresultsonheld-outtestsetAccuracy
AspectPolarityClassificationStateoftheartEnglishSupervisedmachinelearningapproachesmostsuccesful
Brun,Perez&Roux(2016)=topsystem• Ensembleclassifiers• Syntacticparser=basicfeatures (prepro +NER+syntax)• Semanticcomponentadded(basedondesignatedpolarity&
semanticlexicons)• 88.126accuracy
ABSAè AcceptableresultsforEnglishonallthreesubtasks.èDutch:subtasks1and2stillquitechallengingèSametrueforother languagesorotherdomains!!
Note:Inreality,thesearenotseparatetasksè errorpercolation
e.g.forDutchpolarityclassification,accuracydropsto39.70
Overview① Introduction② TaskDefinition③ DatasetsandAnnotation④ Subtasks
ØAspectTermExtractionØAspectTermCategorizationØAspectTermPolarityClassification
⑤ Challenges⑥ Conclusion
DomainadaptationFocusonconsumerreviews• Product-oriented• Aspectexpressions:nounsornounsphrases• Willalmostalwaysincludeanopinion
Inreality• Non-opinionatedtextco-occurswithopinionatedtext(skewed)
• Verbalexpressionsoravarietyofwordscanbeusedtorefertocertainaspects.E.g.politicaltweets,discussionforums,…
User-generatedcontent
• Differentfromstandardtext.• Highlyexpressive:emoticons,flooding(cooool!!)BUT• Fullofmisspellings,grammaticalerrors,abbreviations,…è hinderstandardNLPtools.
Itssokeeeeewl, lol
è polarityclassification:importanceoflexicalfeatures
User-generatedcontent• Normalization(VanHee etal.,underreview)
è Helps,especiallyforunseendata
CreativelanguageuseItwassoniceofmydadtocometomygraduationparty#notGoingtothedentistforarootcanal.Yay,can’twait!!!!
• Sarcasm,irony,humour andmetaphor.• NLP=difficulttointerpretthis
è Interestingresearchemerging.SemEval 2015taskonirony(Ghoshetal.,2015),howevertoomuchfocusonhashtags.VanHee etal.(2016)proposealternativeà alsopapertoappearatCOLING2016.
Requiresdeepunderstanding“Sentiment analysisrequiresadeepunderstandingoftheexplicitandimplicit,regularandirregular,andsyntacticandsemantic
languagerules.”(Cambriaetal.,2013)
• Explicitsentiment:seemseasybutwordsareneverusedinisolation– Negation,modifiers(intensifiers,diminishers,…)è crucial!
• Implicitsentiment:morecomplex,readbetweenthelines.Evenfactualstatementscanevokedifferentopinions.
• Coreference:crucialbutnotmuchresearch.
Overview① Introduction② TaskDefinition③ DatasetsandAnnotation④ Subtasks
ØAspectTermExtractionØAspectTermCategorizationØAspectTermPolarityClassification
⑤ Challenges⑥ Conclusion
ConclusionWhatisaspect-basedsentimentanalysis?
• Taskdefinition• Benchmarkdatasets(SemEval)• Stateoftheartapproaches(customerreviews)• Challenges
(AB)SAisfarfromsolved
MuchmoretoberesearchedLet’scooperate
J
Orphée [email protected]
https://www.lt3.ugent.be/people/orphee-de-clercq/@OrfeeDC
ReferencesBermingham,A.,&Smeaton,A.F.(2011).Onusingtwittertomonitorpoliticalsentimentandpredictelectionresults.Psychology,2-10.Brody,S.&Elhadad,N.(2010).Anunsupervisedaspect-sentiment model foronlinereviews,ProceedingsofNAACL-2010,pp.804-812.Brun,C.,Perez,J.& Roux,C.(2016).XRCEatSemEval-2016Task5:Feedbacked EnsembleModelingonSyntactico-SemanticKnowledgeforAspectBasedSentimentAnalysis,ProceedingsofSemEval-2016,pp.277–281.Cambria,E.,Schuller,B.,Xia,Y.& Havasi,C.(2013).Newavenuesinopinionminingandsentiment analysis, IEEEIntelligentSystems,vol.28,no.2,2013,pp.15–21.Dabrowski,M.,Acton,T.,Jarzebowski,P.&O’Riain,S.(2010).Improvingcustomerdecisionsusingproductreviews- CROM- CarReviewOpinionMiner, inProceedingsofWEBIST-2010,pp.354–357.DeClercq,O.,VandeKauter,M.,Lefever,E.,&Hoste,V.(2015).Applying hybridterminology extraction toaspect-based sentiment analysis. Proceedings of SemEval2015,pp.719–724.DeClercq,O.(2015). Tippingthe scales:exploring the added value of deep semanticprocessing on readability prediction andsentiment analysis.PhD,GhentUniversity.
DeClercq,O.,&Hoste,V.(2016).Rude waiter but mouthwatering pastries!Anexploratory study into Dutchaspect-based sentimesnt analysis, Proceedings of LREC2016, pp.2910–2917.Desmet,B.(2014)“Findingtheonlinecryforhelp:automatictextclassificationforsuicideprevention,”PhD,GhentUniversity.Ganu,G.,Elhadad,N.& Marian,A.(2009).Beyondthestars:improvingratingpredictionsusingreviewtextcontent,ProceedingsofWebDB-2009,pp.1-6.Ghosh,A.,Li,G.,Veale,T.,Rosso,P,Shutova,E.,Barnden,J.&Reyes,A.(2015).Semeval-2015task11:Sentimentanalysisoffigurativelanguageintwitter,inProceedingsofSemEval 2015,pp.470–478.Hu,M.andLiu,B.(2004).Miningandsummarizingcustomerreviews,ProceedingsofKDD-2004,pp.168-177.Liu,B.(2012).Sentimentanalysisandopinionmining,SynthesisLecturesonHumanLanguageTechnologies, vol.5,no.1,pp.1–167.Liu,B.(2015)SentimentAnalysis - MiningOpinions,Sentiments,andEmotions.CambridgeUniversityPress.Liu,P.,Joty,S.&Meng,H.(2015).Fine-grainedopinionminingwithrecurrentneuralnetworksandwordembeddings,ProceedingsEMNLP-2015,pp.1433–1443.
O’Hare,N.,Davy,M.,Bermingham,A.,Ferguson,P.Sheridan,P.Gurrin,C.&Smeaton,A.F.(2009).Topic-dependentsentiment analysisoffinancialblogs,ProceedingsofTSA-2009,pp.9–16.Macken,L.,Lefever,E.&Hoste,V.(2013).TExSIS:BilingualTerminologyExtractionfromParallelCorporaUsingChunk-basedAlignment,Terminology 19(1),pp.1-30.Mendes,P.N.,Jakob,M.,Garca-Silva,A.&Bizer,C.(2011).DBpedia Spotlight:Sheddinglightonthewebofdocuments,Proceedingsofthe7thInternationalConferenceonSemanticSystems(I-Semantics-2011),pp.1-8.Pang,B.,Lee,L.&Vaithyanathan,S(2002).Thumbsup?:Sentimentclassificationusingmachine learningtechniques,ProceedingsofEMNLP-2002,pp.79-86.Pang,B.&Lee,L.(2008).Opinionminingandsentiment analysis,FoundationsandTrendsinInformationRetrieval,vol.2,no.1-2,pp.1–135.Pontiki,M.,Galanis,D.,Pavlopoulos,J.,Papageorgiou,H.,Androutsopoulos I.&Manandhar,S.(2014).Semeval-2014task4:Aspectbasedsentimentanalysis,ProceedingsofSemEval-2014,pp.27–35.Pontiki,M.,Galanis,D.,Papageorgiou,H.,Manandhar,S.&Androutsopoulos,I.(2015).Semeval-2015task12:Aspectbasedsentimentanalysis,ProceedingsofSemEval-2015,pp.486–495.
Pontiki,M.,Galanis,D.,Papageorgiou,H.,Androutsopoulos,I.,Manandhar,S.,AL-Smadi,M.,Al-Ayyoub,M.,Zhao,Y.,Qin,B.,DeClercq,O.,Hoste,V.,Apidianaki,M.,Tannier,X.,Loukachevitch,N.,Kotelnikov,E.,Bel,N.,Jiménez-Zafra,S.M.&Eryiğit,G.(2016).Semeval-2016task5:Aspectbasedsentiment analysis,ProceedingsofSemEval-2016,pp.19–30.Severyn,A.&Moschitti,A.(2015)UNITN:TrainingDeepConvolutionalNeuralNetworkforTwitterSentimentClassification, ProceedingsofSemEval-2015,pp.464–469.Sprenger,T.O.,Tumasjan,A.,Sandner P.G.&Welpe, I.M.(2014).Tweetsandtrades:Theinformationcontentofstockmicroblogs,EuropeanFinancialManagement,vol.20,no.5,pp.926–957.Thet,T.T.,Na,J.-C.andKhoo,C.S.(2010).Aspect-basedsentiment analysisofmoviereviewsondiscussionboards,JournalofInformationScience36(6),823-848.Toh,Z.&Su,J.(2016).NLANGPatSemEval-2016Task5:ImprovingAspectBasedSentimentAnalysisusingNeuralNetworkFeatures,ProceedingsofSemEval2016,pp.282–288.VandeKauter,M.,Coorman,G.,Lefever,E.,Desmet,B.,Macken,L.&Hoste,V.(2013),LeTs Preprocess:Themultilingual LT3linguisticpreprocessingtoolkit,ComputationalLinguistics intheNetherlandsJournal3,103- 120.
VanHee,C.,VandeKauter,M.,DeClercq,O.,Lefever,E.,&Hoste,V.(2014).LT3:sentiment classification inuser-generated content using arich feature set,Proceedingsof SemEval 2014, pp.406–410.VanHee,C.,Lefever,E.,&Hoste,V.(2016).Exploring the realization of irony inTwitterdata,Proceedings LREC 2016,pp.1795–1799.Vossen,P.(1998)EuroWordNet:amultilingual databasewithlexicalsemanticnetworksforEuropeanLanguages,Kluwer,Dordrecht.Zabin,J.&Jefferies,A.(2008).Socialmediamonitoringandanalysis:Generatingconsumerinsightsfromonlineconversation,AberdeenGroupBenchmarkReport,AberdeenGroup,Tech.Rep.
SurveypapertoappearatHUSOmidNovember2016:DeClercq,O.(2016).TheManyAspectsofFine-GrainedSentimentAnalysis:AnOverviewoftheTaskanditsMainChallenges,ProceedingsoftheInternationalConferenceonHumanandSocialAnalytics (HUSO).