Predictive Caching and Prefetching of Query Results in Search Engines

Based on a Paper by:

Ronny Lempel and Shlomo Moran

Presentation:

Omid Fatemieh

CS598CXZ

Spring 2005

2

Outline

Introduction Some analysis on the query log Existing and the proposed

approaches for caching and prefetching

Results Conclusion

3

Introduction

Search engines receive millions of queries per day From millions of unrelated people Studies show that a small set of popular

queries account for a significant fraction of the query stream

Answering some queries from cache instead of through index can: Lower the response time Reduce hardware requirements

4

Caching vs Prefetching

Caching: storing the results that were requested by the users in the a cache in case other user requested same pages. E.g. many people would search for information

about Illinois and NCA on the match day. Prefetching: storing the results that we

predict they would be requested shortly. E.g. prefetching the second page of the results

whenever a new query is submitted by a user.

5

The Query Log

Over 7 million keyword driven search queries submitted to Alta Vista in Summer 2001

Size of the results: A multiple of 10 For r>=1, the results whose rank is 10(r-1)+1,

…, 10r : r th page. Queries asking for a batch of 10k results: asking for k

logical result pages. Each Query: q=(z, t, f, l)

t: timestamp z: topic of the query f and l: the range of result pages requested.

6

Analysis on the Query Log

97.7 percent requested 10 results (one logical page)

This implies over 7,500,000 logical pages (also called page views) were requested.

63.5% of which were for the first pages of the results

11.7% of the views were of second pages…

No. Result Pages

1 2 3 4 5 6 7,8,9 10

No. Queries 6998473

31477

84795

939 42851

125 0 1530

7

Views of Result Pages

The sequential browsing behavior allows for predictive prefetching of result pages.

8

Population of Topics

The distribution of distinct result pages viewed per topic = population of the topic.

In 78% of topics, only a single page was viewed (usually the first page).

Next slide shows the rest.

9

Population of Topics (Number of Pages Requested per Topic)

Note the unusual strength of topics with population s of 10, 15, and 20

10

Topic Popularities

The log contained over 2,600,00 distinct topics

Over 67% were only requested once, in a single query.

The most popular topic was queried 31546 times.

The plot for this, conforms to the power-law except for the most popular topics.

Power Law: the probability of a topic being requested x times is proportional to x-c. c ~ 2.4 for topic popularities.

11

Page popularities

Similar phenomenon is observed when counting the number of requests for individual result pages. 48% of the result pages are only requested

once. The 50 most requested pages account for

almost 2% of the total number of page views. Again, distribution follows a power law for all

but the most frequently requested pages. Power law in this log:

c ~ 2.8 for page popularities.

12

Page and Topic Popularities

13

Behaviour of the Most Popular Result Pages

The log contained 200 result pages that were requested more than 512 times

These pages do not obey the power law The number of the requests for the rth

most popular page is proportional to r-c.

Here, r is 0.67.

14

Behaviour of the Most Popular Result Pages

15

Fetch Units and Result Prefetching

Fetching more results than requested may be relatively cheap.

The dilemma is whether storing the extra results in cache is worthwhile? This would be at the expense of evicting

previously stored results. All the caching schemes fetch results in

bulks whose size is a multiple of k, the basic fetch unit.

16

Fetch Units and Result Prefetching

q: a query requesting result pages f through l for some topic.

Let a and b be the first and last uncached pages in that range.

A k-fetch policy effectively fetches a, a+1, …a+mk-1, such that a+mk-1>=b.

K=1: fetching on demand

17

Upper Bounds on Hit Ratios

Cache of infinite size For each topic t, Pt is the subset of 32

potential result pages that were actually requested in the log.

Cover Pt with minimal number of fetch units possible : f k (t)

Σt fk(t) is a close approximation to minimal number of faults any policy whose fetch unit is k will have on this query log.

18

Upper Bounds on Hit Ratios

19

Cache Replacement Policies

Page Least Recently Used (PLRU) Page Segmented LRU (PSLRU) Topic LRU (TLRU) Topic SLRU (SLRU)

All the above approaches require O(1) for treating each query.

Probability Driven Cache (PDC) The novel approach presented in the paper. Requires O(log(size-of-cache)) operation per

query.

20

Terminology

C(q) is the subset of the requested pages P(q) that are cached when q is submitted.

Let G(q) denote the set of pages that are fetched as a consequence of q. F(q) is the subset of the uncached pages of G(q).

21

PLRU

PLRU: The pages of C(q), merged with pages of F(q) are moved back to the tail of the queue.

Once the queue is full, cached pages are evicted from the head of the queue. Tail: the most recently requested (and

prefetched) pages Head: The least recently requested pages.

22

SLRU

Two LRU segments A protected segment A probationary segment.

F(q): inserted into the probationary segment (tail). C(q): transferred to the protected segment (tail). Evicted from the protected segment:

Remain cached Moved to the tail of the probationary segment.

Pages are removed from the cache only when they are evicted from the probationary segment.

The Pages in the protected segment were requested at least twice since they were last fetched.

23

TLRU

The Topic LRU (TLRU) policy is a variation on the PLRU scheme.

Let t(q) denote the topic of the query q. TLRU performs two actions for every query q:

The pages of F(q) are inserted into the page queue.

Any cached result page of t(q) is moved to the tail of the queue.

Each topic's pages will always reside contiguously in the queue.

blocks of different topics ordered by the LRU policy.

24

TSLRU

The Topic SLRU (TSLRU) policy is a variation on the PSLRU scheme. It performs two actions for every query q (whose topic is t(q)):

The pages of F(q) are inserted into the probationary queue.

Any cached result page of t(q) is moved to the tail of the protected queue.

25

Probability Driven Cache (PDC)

qiu=( zi

u, tiu, fi

u, liu) } i >=1 the sequence of queries that user u issues

Consider qiu and qi+1

u two successive queries issued by user u.

qi+1u may be a folllow-up, or start of a new session.

If qi+1u is a follow-up on qi

u, then qi+1u is submitted no

more that W time units after qiu (that is, zi+1

u <= ziu

+W) and fi+1u = liu +1. Obviously, in this case ti+1

u = tiu.

qi+1u starts a new search session whenever fi+1

u =1.

We also assume that the first query of every user u, q1

u, requests the top result page of some topic.

26

Probability Driven Cache (PDC)

W=attention span: users do not submit follow-up queries after being inactive for W time units.

The result pages viewed in every search session are pages (t,1),..,(t,m) for some topic t and m>=1.

At any given moment z, every user has at most one query that will potentially be followed upon.

27

Probability Driven Cache (PDC)

The set of queries that will potentially be followed upon is defined by

Q =  {  qzu, qz

u was submitted after z-W }    The model assumes that there are topic

and user independent probabilities sm, m>=1, such that: sm is the probability of a search session

requesting exactly m result pages. sm values are known

28

Probability Driven Cache (PDC)

For a query q: t(q): query's topic l(q): the last result page requested in q.

For every result page (t,m), we can now calculate PQ(t,m), the probability that (t,m) will be requested as a follow-up to at least one of the queries in Q:

P[m|l] is the probability that a session will request result page m, given that the last result page requested so far was page l.

29

Implementation

PQ(t,1) cannot be determined by this model. It cannot predict the topics that will be the

focus of future search sessions. PDC prioritizes cached (t,1) pages by a

different mechanism. PDC maintains two buffers:

A priority Queue PQ for prioritizing the cached pages.

And SLRU buffer for caching (t,1) pages. The relative size of these buffers: subject

to optimization

30

Implementation

probabilities sm, m>=1 are determined based on the characteristics of the log.

PDC tracks the set Q by maintaining a query window QW, QW holds a subset of the queries

submitted during the last W time units. For every kept query q=(z,t,f,l), its time z

and last requested page (t,l) are saved.

31

Steps for a Query (z,t,f,l)1. q is inserted into QW,

queries submitted before z-W are removed from QW. If there is a query q' in QW such that the last page

requested by q' is (t,f-1), the least recent such query is also removed from QW..

2. T: The set of topics whose corresponding set of QW queries has changed.

The priorities of all T-pages in PQ are updated.3. If f=1 and page (t,1) is not cached, (t,1) is

inserted at the tail of the probationary segment of the SLRU.

If (t,1) is already cached, it is moved to the tail of the protected segment of the SLRU.

4. Let (t,m), 1<m<=l be a page requested by q that is not cached.

Its priority is calculated and if it merits so, it is kept in PQ (causing perhaps an eviction of a lower priority page).

32

Results

Hits and Faults Query counts as a cache hit only if it can be fully

answered from the cache, otherwise it is a fault.

Cold and Warm caches The reported hit ratios are for warm caches. The definition of a warm cache PLRU and TLRU:

Once the page queue is full. PSLRU and TSLRU:

Once the probationary segment of the SLRU becomes full for the first time.

PDC cache becomes warm Once either the probationary segment of the SLRU or

the PQ component reach full capacity.

33

Results (PLRU and TLRU)

LRU(s,3) is always higher than LRU(4s,1). In fact, LRU(16000,3) is higher than LRU(512000,1), despite the latter cache being 32 times larger.

For s=4000, the optimal fetch unit for both schemes (PLRU, TLRU) is 3.

The optimal fetch unit increases as the size of the cache increases.

The increase in performance that is gained by doubling the cache size is large for small caches

34

Results for PSLRU and TSLRU

The Ratio:The ratio of probationary segment to the whole cache.

For larger cache sizes, increased fetch unit helps more. Lower ratios help more influential when the cache size is

large.

35

Results for PDC

Two new degrees of freedom: Length of the Query Window (QW) The ratio between the capacity of the

SLRU, which holds the (t,1) pages, and the capacity of the priority queue PQ, that holds all other result pages.

Again the most significant degree of freedom: fetch unit.

36

Results for PDC

37

Results for PDC

Again the optimal fetch unit increases as the cache size increases.

The results indicate that the optimal window length grows as the cache size grows.

With small cache sizes, it is best to consider only the most recent requests. As the cache grows, it pays to consider growing request histories when replacing cached pages.

The optimal PQ size shrinks as the cache grows. The probationary segment of the SLRU should

be dominant for small caches, but both SLRU segments should be roughly of equal size in large caches.

38

Comparing all Policies

39

Conclusion

Five replacement policies for cached search result pages.

PDC is superior to the other tested caching schemes.

For large cache sizes, PDC outperforms LRU-based caches that are twice as large.

It achieves hit ratios of 0.53 on a query log whose theoretic hit ratio is bounded by 0.629.

The impact of fetch unit is much greater than any other internal partitioning of the cache.

The optimal fetch unit depends only on the total size of the cache.

40

Discussion

Is the computational complexity of O(log(cache size) acceptable?

Considering the memory getting cheaper, does the slight improvement worth these computations?

They don’t show how they used their initial analysis on the logs for presenting their method.