Text Summarization usingText Summarization using

Lexical ChainsLexical Chains

Summarization using Lexical Chains

Summarization?Summarization?

• What is Summarization?

• Advantages…

• Challenges…

Summarization using Lexical Chains

Project GoalsProject Goals

• Aim: Summary of an original text without requiring full semantic interpretation

• Tools: WordNet thesaurus, shallow parser, POS & Brill’s tagger, Segmentation algorithm.

Summarization using Lexical Chains

DescriptionDescription

• Input Domain: Technical_Article.txt

• Processing: Algorithm by Regina Barzilay and Michael Elhadad

• Output: Lexical Chains & Extract

Summarization using Lexical Chains

Design steps…Design steps…

• Segment the original text • Construct lexical chains • Identify strong chains • Extract significant sentences

Summarization using Lexical Chains

Step One…Step One…

• Segment the original text • Construct lexical chains • Identify strong chains • Extract significant sentences

Summarization using Lexical Chains

Seg-ment-ation AlgorithmSeg-ment-ation Algorithm

• Form a Token Sequence•Parameter: w

• Form a Block•Parameter: b

• Computation of Similarity •Parameter: sim(b1,b2)

Summarization using Lexical Chains

……..

• Plot Graph.•Parameter: depth score

• Sort

• Segment boundary

Summarization using Lexical Chains

Step Two…Step Two…

• Segment the original text

• Construct lexical chains • Identify strong chains • Extract significant sentences

Summarization using Lexical Chains

Construction of Lexical ChainsConstruction of Lexical Chains

Procedure:

1. Select a set of candidate words.

2. Find appropriate chain.

3. Insert the word in the chain.

Summarization using Lexical Chains

Step Three…Step Three…

• Segment the original text • Construct lexical chains

• Identify strong chains • Extract significant sentences

Summarization using Lexical Chains

Strong chains?Strong chains?

Good predictors of strength of a chain • Length = number of occurrences of

members in a chain

• Homogeneity index = 1 – (number of distinct occurrences / length)

Summarization using Lexical Chains

Chain ScoreChain Score

Score(Chain) =Length * Homogeneity_Index

Strength Criterion :Score(Chain) > Average(Scores) +

2* StandardDeviation(Scores)

Summarization using Lexical Chains

Step Four…Step Four…

• Segment the original text • Construct lexical chains • Identify strong chains

• Extract significant sentences

Summarization using Lexical Chains

ExtractionExtraction

A Heuristic

• Select representative words

• Extract sentence with first appearance of representative Word.

Thank YouThank You