Embed Size (px)
Citation preview
ContextOur Summarizing SystemResultsCon lusions and Future WorkClustering to Improve Mi roblog StreamSummarizationAndrei OlariuUniversity of Bu harestFa ulty of Mathemati s and Computer S ien eSYNASC 2012
Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future WorkOutline1 ContextMi robloggingPrevious WorkMotivation2 Our Summarizing SystemApproa h OutlineEvent Dete tionMessage ClusteringSummarization3 ResultsCorpusMetri sSummarization Results4 Con lusions and Future WorkAndrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Mi robloggingPrevious WorkMotivationOutline1 ContextMi robloggingPrevious WorkMotivation2 Our Summarizing SystemApproa h OutlineEvent Dete tionMessage ClusteringSummarization3 ResultsCorpusMetri sSummarization Results4 Con lusions and Future WorkAndrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Mi robloggingPrevious WorkMotivationWhat is Mi robloggingmi roblogging form of blogging hara terized by very short postsmi roblogging_platforms Twitter, Tumblr, Fa ebookTwitter's main highlights:hundreds of millions of posts per daydata is publi ly a essible (unlike Fa ebook)posts are mainly text (unlike Tumblr - mostly images)posts are limited to 140 hara tersspe i� vo abulary (internet slang)abbreviations, misspelled wordsAndrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Mi robloggingPrevious WorkMotivationWhat is Mi robloggingData on Twitter is organized as a stream (sequen e of posts)
Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Mi robloggingPrevious WorkMotivationSummarizing Mi roblogging Streamssummarize generate a brief statement highlighting the mainpoints of a larger text (in our ase, a stream ofmessages)for end users - �ght information overloadfor organizations - business / so ial / politi al insights
Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Mi robloggingPrevious WorkMotivationSummarizing Mi roblogging Streamssummarize generate a brief statement highlighting the mainpoints of a larger text (in our ase, a stream ofmessages)for end users - �ght information overloadfor organizations - business / so ial / politi al insights
Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Mi robloggingPrevious WorkMotivationOutline1 ContextMi robloggingPrevious WorkMotivation2 Our Summarizing SystemApproa h OutlineEvent Dete tionMessage ClusteringSummarization3 ResultsCorpusMetri sSummarization Results4 Con lusions and Future WorkAndrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Mi robloggingPrevious WorkMotivationMi roblog Event Dete tiondete t the main topi s in a stream
Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Mi robloggingPrevious WorkMotivationMi roblog Event Dete tionmodel an event based on a stream of related posts luster similar messagesdete t words that experien e an in reased frequen y
Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Mi robloggingPrevious WorkMotivationMulti-senten e Compressionmulti-senten e_ ompression generate a short senten e thatsummarizes a group of related senten esExampleThe wife of a former U.S. president Bill Clinton Hillary Clintonvisited China last Monday.Hillary Clinton wanted to visit China last month but postponed herplans till Monday last week.Hillary Clinton paid a visit to the People Republi of China onMonday.Last week the Se retary of State Ms. Clinton visited Chineseo� ials. Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Mi robloggingPrevious WorkMotivationMulti-senten e Compressionmulti-senten e_ ompression generate a short senten e thatsummarizes a group of related senten esExampleThe wife of a former U.S. president Bill Clinton Hillary Clintonvisited China last Monday.Hillary Clinton wanted to visit China last month but postponed herplans till Monday last week.Hillary Clinton paid a visit to the People Republi of China onMonday.Last week the Se retary of State Ms. Clinton visited Chineseo� ials. Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Mi robloggingPrevious WorkMotivationMulti-senten e CompressionMulti-senten e Compression �nds a path minimizing a ost fun tionin a word graph:
Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Mi robloggingPrevious WorkMotivationSummarizing Mi roblogging Streamsapproa hed in two ways: hoose a post that best des ribes the input streamgenerate a short senten e based on the stream - �PhraseReinfor ement� algorithmboth approa hes have been developed for streams of messagesrelated to a given event
Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Mi robloggingPrevious WorkMotivationSummarizing Mi roblogging Streamsapproa hed in two ways: hoose a post that best des ribes the input streamgenerate a short senten e based on the stream - �PhraseReinfor ement� algorithmboth approa hes have been developed for streams of messagesrelated to a given event
Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Mi robloggingPrevious WorkMotivationSummarizing Mi roblogging Streamsapproa hed in two ways: hoose a post that best des ribes the input streamgenerate a short senten e based on the stream - �PhraseReinfor ement� algorithmboth approa hes have been developed for streams of messagesrelated to a given event
Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Mi robloggingPrevious WorkMotivationPhrase Reinfor ementPhrase_Reinfor ement algorithm that generates a summarystarting from a given keyphrase and a stream of postsrelated to that keyphraseExampleA tragedy: Ted Kennedy died today of an erTed Kennedy died todayTed Kennedy was a leaderTed Kennedy died at age 77Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Mi robloggingPrevious WorkMotivationPhrase Reinfor ementPhrase_Reinfor ement algorithm that generates a summarystarting from a given keyphrase and a stream of postsrelated to that keyphraseExampleA tragedy: Ted Kennedy died today of an erTed Kennedy died todayTed Kennedy was a leaderTed Kennedy died at age 77Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Mi robloggingPrevious WorkMotivationPhrase Reinfor ementThe graph built starting from the keyphrase �Ted Kennedy�:,
Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Mi robloggingPrevious WorkMotivationOutline1 ContextMi robloggingPrevious WorkMotivation2 Our Summarizing SystemApproa h OutlineEvent Dete tionMessage ClusteringSummarization3 ResultsCorpusMetri sSummarization Results4 Con lusions and Future WorkAndrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Mi robloggingPrevious WorkMotivationMotivationAll previous summarizing te hniques require as input a stream ofrelated posts:posts are �ltered based on a given set of keywordskeywords are manually sele ted to mat h a spe i� event/topi Yet, most streams are not about a spe i� event/topi and su�erfrom a large amount of noise.How an we approa h summarizing any kind of stream?Contributions:developed a system for summarizing un�ltered streamsadapted the Phrase Reinfor ement algorithm in order tointegrate it into our systemAndrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Mi robloggingPrevious WorkMotivationMotivationAll previous summarizing te hniques require as input a stream ofrelated posts:posts are �ltered based on a given set of keywordskeywords are manually sele ted to mat h a spe i� event/topi Yet, most streams are not about a spe i� event/topi and su�erfrom a large amount of noise.How an we approa h summarizing any kind of stream?Contributions:developed a system for summarizing un�ltered streamsadapted the Phrase Reinfor ement algorithm in order tointegrate it into our systemAndrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Mi robloggingPrevious WorkMotivationMotivationAll previous summarizing te hniques require as input a stream ofrelated posts:posts are �ltered based on a given set of keywordskeywords are manually sele ted to mat h a spe i� event/topi Yet, most streams are not about a spe i� event/topi and su�erfrom a large amount of noise.How an we approa h summarizing any kind of stream?Contributions:developed a system for summarizing un�ltered streamsadapted the Phrase Reinfor ement algorithm in order tointegrate it into our systemAndrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Approa h OutlineEvent Dete tionMessage ClusteringSummarizationOutline1 ContextMi robloggingPrevious WorkMotivation2 Our Summarizing SystemApproa h OutlineEvent Dete tionMessage ClusteringSummarization3 ResultsCorpusMetri sSummarization Results4 Con lusions and Future WorkAndrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Approa h OutlineEvent Dete tionMessage ClusteringSummarizationApproa h Outline
Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Approa h OutlineEvent Dete tionMessage ClusteringSummarizationOutline1 ContextMi robloggingPrevious WorkMotivation2 Our Summarizing SystemApproa h OutlineEvent Dete tionMessage ClusteringSummarization3 ResultsCorpusMetri sSummarization Results4 Con lusions and Future WorkAndrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Approa h OutlineEvent Dete tionMessage ClusteringSummarizationEvent Dete tiondete t words that show an unusual in rease in frequen y luster words based on how often they appear together in postsea h luster of words represents an event
Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Approa h OutlineEvent Dete tionMessage ClusteringSummarizationEvent Dete tiondete t words that show an unusual in rease in frequen y luster words based on how often they appear together in postsea h luster of words represents an event
Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Approa h OutlineEvent Dete tionMessage ClusteringSummarizationEvent Dete tiondete t words that show an unusual in rease in frequen y luster words based on how often they appear together in postsea h luster of words represents an event
Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Approa h OutlineEvent Dete tionMessage ClusteringSummarizationOutline1 ContextMi robloggingPrevious WorkMotivation2 Our Summarizing SystemApproa h OutlineEvent Dete tionMessage ClusteringSummarization3 ResultsCorpusMetri sSummarization Results4 Con lusions and Future WorkAndrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Approa h OutlineEvent Dete tionMessage ClusteringSummarizationMessage Clusteringfor ea h message, determine the word luster most similar to itif the similarity is above a threshold, assign it to the event,otherwise onsider it noise
Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Approa h OutlineEvent Dete tionMessage ClusteringSummarizationMessage Clusteringfor ea h message, determine the word luster most similar to itif the similarity is above a threshold, assign it to the event,otherwise onsider it noise
Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Approa h OutlineEvent Dete tionMessage ClusteringSummarizationOutline1 ContextMi robloggingPrevious WorkMotivation2 Our Summarizing SystemApproa h OutlineEvent Dete tionMessage ClusteringSummarization3 ResultsCorpusMetri sSummarization Results4 Con lusions and Future WorkAndrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Approa h OutlineEvent Dete tionMessage ClusteringSummarizationSummarization Approa hesWe test two di�erent approa hes:Multi-senten e Compression (MSC)Frequent Phrase Summarization (FPS)an adaptation of Phrase Reinfor ement that does not require astarting keyphrasethe algorithm retrieves a popular sequen e of words from theinput streamone of our ontributionsAndrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work Approa h OutlineEvent Dete tionMessage ClusteringSummarizationSummarization Approa hesWe test two di�erent approa hes:Multi-senten e Compression (MSC)Frequent Phrase Summarization (FPS)an adaptation of Phrase Reinfor ement that does not require astarting keyphrasethe algorithm retrieves a popular sequen e of words from theinput streamone of our ontributionsAndrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work CorpusMetri sSummarization ResultsOutline1 ContextMi robloggingPrevious WorkMotivation2 Our Summarizing SystemApproa h OutlineEvent Dete tionMessage ClusteringSummarization3 ResultsCorpusMetri sSummarization Results4 Con lusions and Future WorkAndrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work CorpusMetri sSummarization ResultsCorpuswe used the Twitter API to retrieve re ent tweetswe olle ted around 140000 tweets per day, between the 22ndof April and the 2nd of Maytweets before the 28th were used as ba kground data
Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work CorpusMetri sSummarization ResultsOutline1 ContextMi robloggingPrevious WorkMotivation2 Our Summarizing SystemApproa h OutlineEvent Dete tionMessage ClusteringSummarization3 ResultsCorpusMetri sSummarization Results4 Con lusions and Future WorkAndrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work CorpusMetri sSummarization ResultsMetri swe manually assessed the summaries regarding: ompleteness - how mu h information the summary expressesrelative to the dete ted eventgrammati ality - the degree of grammati al and synta ti al orre tnesswe assigned ratings on a s ale of 1 to 5
Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work CorpusMetri sSummarization ResultsMetri sExamples of ratings:Metri Rating ExamplesGramm. 1 i have monday to on a good morningthe elti s2 i have to go to hur h was goodthe fa mad- no o�en e to roy hodgson3 president obama in afghanistan to the bin ladenhere are the new nets logo is this4 papiss isse take a goal of the season ity vs man hester united to the man hester derby5 rays have signed hideki matsui to a minor league ontra tAndrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work CorpusMetri sSummarization ResultsMetri sExamples of ratings:Metri Rating ExamplesCompl. 1 more for a month of the may daywhere have you been2 we all follow the derbyfor me this is the most beautiful moment3 the beta period is extended for 1 weekat the mar h on ows mayday m1gs m1ny 4 i fan y ity to beat united today tevez and rooney5 fa ebook adds organ donation initiative onmembers to timelineAndrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work CorpusMetri sSummarization ResultsOutline1 ContextMi robloggingPrevious WorkMotivation2 Our Summarizing SystemApproa h OutlineEvent Dete tionMessage ClusteringSummarization3 ResultsCorpusMetri sSummarization Results4 Con lusions and Future WorkAndrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work CorpusMetri sSummarization ResultsSummarization without Clustering29th AprilMSC: �if you have to me and i love�FPS: �you an now play ube a webgl game on hromeexperiments about google maps at a webgl enabled browser isrequired�30th AprilMSC: �you have to me and i just a�FPS: �ba kinelementarys hool i had some weird obsession withdrawing this 's' never knew what it meant rt if you did too�Summaries generated by MSC re eive a grammati ality rating of 1and a ompleteness rating of 1.Summaries generated by FPS re eive a grammati ality rating of 5and a ompleteness rating of 1.Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work CorpusMetri sSummarization ResultsSummarization with ClusteringThe Event Dete tion module dis overed a total of 111 eventsduring the four day time span we analyzed.The ratings given for summaries generated by Multi-senten eCompression (MSC) and Frequent Phrase Summarization (FPS) onthe event lusters are distributed as follows:Algorithm \ Rating 1 2 3 4 5 Average ratingMSC Grammati ality 18 11 18 21 43 3.54FPS Grammati ality 23 4 8 21 55 3.73MSC Completeness 31 11 22 18 29 3.03FPS Completeness 30 15 25 15 26 2.93Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work CorpusMetri sSummarization ResultsIssues and ExamplesWhen onfronted with very di�erent senten es:MSC generates a meaningless summary� ongratulations to for the title of this out�FPS pi ks a long and frequent phrase (usually the one thatwas retweeted the most)�real & ajax win titles �orentina's manager is sa ked after atta kinghis own player & messi gets his 68th goal�Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work CorpusMetri sSummarization ResultsIssues and ExamplesWhen onfronted with very di�erent senten es:MSC generates a meaningless summary� ongratulations to for the title of this out�FPS pi ks a long and frequent phrase (usually the one thatwas retweeted the most)�real & ajax win titles �orentina's manager is sa ked after atta kinghis own player & messi gets his 68th goal�Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work CorpusMetri sSummarization ResultsIssues and ExamplesWhen onfronted with similar, but slightly di�erent senten es:MSC manages to determine a ommon phrasing�bla k ops 2 trailer o� ial all of duty�FPS an only generate a short phrase�bla k ops 2�Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work CorpusMetri sSummarization ResultsIssues and ExamplesWhen onfronted with similar, but slightly di�erent senten es:MSC manages to determine a ommon phrasing�bla k ops 2 trailer o� ial all of duty�FPS an only generate a short phrase�bla k ops 2�Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future Work CorpusMetri sSummarization ResultsIssues and ExamplesMSC is easily in�uen ed by noise:on the 1st of May, a luster was generated around the word�anniversary� after a lot of people remembered Ayrton Senna'sdeath (using �anniversary� instead of � ommemoration�)other posts using the word �anniversary� were also added tothe event lusterthe summary: �happy anniversary of the tragi death ofayrtonsenna�Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future WorkCon lusions and Future Workwe showed that summarizing a stream of posts an beapproa hed by:dete ting the events people are talking about lustering posts related to those eventsapplying lassi al summarizing algorithms to ea h luster ofpostsfuture workimprove event dete tion and message lusteringuse hierar hi al event analysis �> hierar hi al summariestry topi dete tion instead of event dete tionAndrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future WorkCon lusions and Future Workwe showed that summarizing a stream of posts an beapproa hed by:dete ting the events people are talking about lustering posts related to those eventsapplying lassi al summarizing algorithms to ea h luster ofpostsfuture workimprove event dete tion and message lusteringuse hierar hi al event analysis �> hierar hi al summariestry topi dete tion instead of event dete tionAndrei Olariu Clustering to Improve Mi roblog Stream Summarization
ContextOur Summarizing SystemResultsCon lusions and Future WorkThank YouDo you have any questions?
Andrei Olariu Clustering to Improve Mi roblog Stream Summarization
