View
212
Download
0
Category
Tags:
Preview:
Citation preview
Advantages of Query Biased Summaries in Information
Retrieval
by A. Tombros and M. Sanderson
Presenters:Omer Erdil Albayrak
Bilge Koroglu
OutlineBrief Introduction to Typical Information Retrieval System
Why query biased summaries needed?
How a query biased summary generated?
Experimental EnvironmentExperimental ResultsConclusion and Future Works
2/13
Introduction
3/13
Typical IR System: input: information needoutput: ranked document list
Figure 1. Screenshot from an IR System
Introduction (con’t...)Deciding whether the retrieved document
is worth to further investigationExamining the static summariesChecking the whole content
Weak content indicators: static summaries
Time consuming job: refering full text nearly all times
Aiming to minimise reaching whole content Generating query biased summaries
4/13
Generating Query Biased SummariesQuery-specific summaries for each retrieved
documentClassical approach to summarization
Extracting sentencesAssigning scores to sentencesSelection of best-scoring sentences
Some modification on score assignmentExtra importance to titles and subtitlesMore weights to sentences with clusters of terms
Additional points to sentences includes query terms5/13
Experimental Environment
6/13
TREC test collection: Wall Street Journal news50 queries of which relevant documents are known
2 groups of 10 postgraduate students to find relevant docs for 50 queriesOne group with static summariesAnother is to use query biased summaries
50 retrieved docs per each query5 queries per student5 minutes allocated per queryIdentical computers in hardware/software
aspects
Experimental Environment (con’t...)
7/13 Figure 2. The subjects performing on query biased & static summaries in the experiments
Experimental Results
8/13
Recall: the ratio of total number of relevant documents for query to the number of retrieved relevant documents
Precision: the ratio of total number of retrieved documents for query to the number of retrieved relevant documents
Figure 4. Recall values of 2 groups
Figure 5. Precision values of 2 groups
Experimental Results (con’t...)
9/13
Speed of evaluators’ judgments of relevancy2.62% on 20 documents corresponds 13%
increase of average number of examined documents
Figure 6. The number of doc percentage for 2 group
Experimental Results (con’t...)
10/13
The need for checking full textAverage number of full text reference per
query: ‘query biased summary group’ : 0.3 ‘static summary group’: 4.74
Figure 7. Average number of references to the full text of documents per query
Conclusion and Future Works
11/13
Effective method: employing query biased summaries in IR SystemsEasily identifiable more relevant documentsDecreasing the need to check whole
content of documentApplicable to web search engines
Expensive to retrieve the documenst from slow, not reliable, and remote servers
Requiring to manage an index fileMore experiments on different summarization
methods with different datasets
12/13
Thank you...
Recommended