Event-Based Extractive SummarizationEvent-Based Extractive Summarization

E. Filatova and V. Hatzivassiloglou Department of Computer Science

Columbia University

(ACL 2004)

2/23

Abstract Most approaches to extractive summarization define

a set of features upon which selection of sentences is based, using algorithms independent of the features

We proposed a new set of features based on low-level atomic events that describe relationships between important actors

Our experimental results indicate that not only the event-based features offer an improvement in summary quality over words as features, but that this effect is more pronounced for more sophisticated summarization methods

3/23

Introduction

Identify what information is important and should

be included into the summary

Break the input text into textual units

(sentences, clauses, etc.)

Score every textual unitaccording to what information

is covered in it

textual unitsinformation features

Choose the textual unitthat should be added to the

summary

repeat until wereach the desired length

rescore thetextual unitsbased on whatinformationis alreadycovered bythe summary

4/23

General Summarization Model

Ti: text unit I Ci: concept i* Each concept ci has an associated weight wi Indicating the importance

5/23

General Summarization Model Using the above matrix it is possible to

formulate the extractive summarization problem as extracting the minimal amount of text units which cover all the concepts that are interesting or important

To account for the cost of long summaries, we can constrain the total length of the summary or balance it against the total weight of covered concepts

6/23

Associating Concepts with Features Lexical features:

Words: tf*idf weights show what words are important Words used in titles and section headings (Luhn’59) Presence of cue phrases in the textual unit: in conclusion, significant (Kupie

c et al’95) Co-occurrence of some particular terms: lexical chains (Barzilay & Elhadad’

97), topic signatures (Lin & Hovy’2000)

Non-lexical features: Textual unit’s position in the input text: head-line, first sentence in the para

graph (Baxedale’58) Rhetorical representation of the source text (Marcu’97)

We suggest to use atomic events as features signaling out the important sentences

7/23

Atomic EventsAtomic events = Relation + Connector (potential label for

the relation)

Relation is a pair of Named Entities or significant nouns For the input text, get all possible pairs of named entities

within one sentence For every relation analyze all the verbs and action defining

nouns in-between the named entities in the relation, these verbs/nouns can be used as labels for the extracted relations

Some important words are not marked by name entities but are highly likely to be among the most frequently used nouns

Top ten most frequent nouns are added to the relation list

8/23

Atomic Events Algorithm for atomic event extraction

Analyze each input sentence one at a time; ignore sentences that do not contain at least two name entities or frequent nouns

Extract all the possible pairs (relations) of named entities / frequent nouns in the sentence and words in-between the name entities (connectors)

For each relation, count how many times this relation is used in the input texts

Keep only connectors that are content verbs or action nouns, according to Word-Net’s noun hierarchy. For each connector calculate how many times it is used for the extracted relations

9/23

Atomic Events Calculate normalized frequencies for all relations

Normalized frequency of a relation = n/N, where n – frequency of the current relation in a topic N – overall frequency of all relations in a topic

Calculate normalized frequencies for all connectors

Normalized frequency of a connector = c/S, where

c – frequency of the current connector in a relationS – overall frequency of all connectors for a relation

10/23

Atomic EventsNorm. Rel.Frequency

First Element

Second Element

0.0212 China Airlines Taiwan

0.0191 China Airlines Taipei

0.0170 China Airlines Monday

0.0170 Taiwan Monday

0.0170 Bali Taipei

0.0148 Taipei Taiwan

0.0148 Bali Taiwan

0.0148 Taipei Monday

0.0127 International Airport Taiwan

11/23

Atomic EventsRelation Connecto

rNorm. Conn. Freq.

China Airlines - Taiwan crashed/VBDtrying/VBGburst/VBPland/VB

0.03120.03120.02670.0267

China Airlines - Taipei burst/VBPcrashed/VBDcrashed/VBN

0.03110.03110.0198

12/23

Atomic Events Atomic event score:

The score of the atomic event predicts how well the important this atomic event for the collection of texts is

Atomic Event Score = Normalized freq.

of the relation( ) * ( Normalized freq. of the connector)

13/23

Textual Unit Selection Static Greedy Algorithm

For every textual units, calculate the weight of this text unit as the sum of the weights of all the concepts covered by this textual unit

Choose the text unit with maximum weight and add it to the final output

Continue extracting other textual units in order of total weight till we get the summary of the desired length

14/23

Textual Unit Selection Adaptive Greedy Algorithm

For each textual calculate its weight as the sum of weights of all concepts it covers

Choose the textual unit with the maximum weight and add it to the output. Add the concepts covered by this textual unit to the list of concepts covered in the final output.

Recalculate the weights of text units: subtract from each unit’s weight the weight of all concepts in it that are already covered in the output

Continue extracting text units in order of their total weight (back to step 2) until the summary is of the desired length

15/23

Textual Unit Selection Modified Adaptive Greedy Algorithm

For every textual unit calculate its weight as the sum of weights of all concepts it covers

Consider only those textual units that contain the concept with the highest weight that has not yet been covered. Out of these, choose the one with highest total weight and add it to the final output. Add the concepts which are covered by this textual unit to the list of concepts covered in the final output.

Recalculate the weights of text units: subtract from each unit’s weight the weight of all concepts in it that are already covered in the output

Continue extracting text units in order of their total weight (back to step 2) until the summary is of the desired length

16/23

Experiment Input Data

The document sets used in the evaluation of multi-document summarization during the first Document Understanding Conference (DUC2001)

30 test document sets, each with approximately 10 news stories on different events

For each document set three human-constructed summaries are provided for each of the target lengths 50, 100, 200 and 400 words

17/23

Experiment Evaluation Metric

ROUGE (Lin and Hovy, 2003) Recall-based measure

Summary Length For each document set, four summaries of length 50, 100,

200 and 400 are created Rouge evaluation has not yet been tested extensivel

y , and ROUGE scores are difficult to interpret as they are not absolute and not comparable across source document sets

18/23

Experiment Results: Static Greedy Algorithm

19/23

Experiment Results: Adaptive Greedy Algorithm

20/23

Experiment

21/23

Experiment Results: Modified Greedy Algorithm

22/23

Experiment Results: Comparison with DUC systems

In DUC2003 the task was to create summaries only of length 100

Events as features and adaptive greedy algorithm In 14 out of 30 cases our system outperforms the median

of the ROUGE scores of all the 15 participating systems over that specific document set

The suitability of the event-based summarizer varies according to the type of documents being summarized

23/23

Conclusion Our experimental results indicate that events are indeed an

effective features, at least in comparison with words With all three of our summarization algorithms, we achieved

a gain in performance when using events Our approach to defining and extracting events can be

improved in many ways Matching connectors that are similar in meaning Representing paraphrases of the same event Methods for detecting and prioritizing special event components such

as time and location phrases Merging information across related atomic events Partial matches between atomic events and input sentences