W

Web Caching

based on:

✗ Web Caching, Geoff Huston

✗ Web Caching and Zipf-like Distributions:Evidence and Implications, L. Breslau, P.Cao, L. Fan, G. Phillips, S. Shenker

✗ On the scale and performance ofcooperative Web proxy caching, A.Wolman, G. Voelker, N. Sharma, N.Cardwell, A. Karlin, H. Levy

Page 1 of 24eb Caching

W

h

Hypertext Transfer Protocol (HTTP)

based on an end-to-end model (the network isa passive element)

Pros:

✗ The server can modify the content;

✗ The server can track all content requests;

✗ The server can differentiate betweendifferent clients;

✗ The server can authenticate the client.

Cons:

✗ Popular servers are under continuous higstress

✗ high number of simultaneously openedconnections

✗ high load on the surrounding network

✗ The network may not be efficiently utilized

Page 2 of 24eb Caching

W

n

Why Caching in the Network?

✔ reduce latency by avoiding slow links betweeclient and origin server

✗ low bandwidth links

✗ congested links

✔ reduce traffic on links

✔ spread load of overloaded origin servers tocaches

Page 3 of 24eb Caching

W

d;

ffic;

on

Caching points

✔ content provider (server caching)

✔ ISP

✗ more than 70% of ISP traffic is Web base

✗ repeated requests at about 50% of the tra

✗ ISPs interested in reducing the transmissicosts;

✗ provide high quality of service to the endusers.

✔ end client

Page 4 of 24eb Caching

W

Caching Challenges

✔ Cache consistency

✗ identify if an object is stale or fresh

✔ Dynamic content

✗ caches should not store outputs of CGIscripts (Active Cache?)

✔ Hit counts and personalization

✔ Privacy concerned user

✗ how do you get a user to point to a cache

✔ Access control

✗ legal and security restrictions

✔ Large multimedia files

Page 5 of 24eb Caching

W

st

e

Web cache access pattern -(commonthoughts)

✔ the distribution of page requests follows aZipf-like distribution: the relative probabilityof a request for the i-th most popular page isproportional to with

✔ for an infinite sized cache, the hit-ratio growin a log-like fashion as a function of the clienpopulation and of the number of requests

✔ the hit-ratio of a web cache grows in a log-likfashion as a function of cache size

✔ the probability that a document will bereferenced requests after it was lastreferenced is roughly proportional to(temporal locality)

1 iα⁄ α 1≤

k1 k⁄

Page 6 of 24eb Caching

W

es

its

t’sns

w

5)

Web cache access pattern - (P. CaoObservations)

✔ distribution of web requests follow a a Zipf-like distribution (fig. 1)

✔ the 10/90 rule does not apply to web access(70% of accesses to about 25%-40% of thedocuments) (fig. 2)

✔ low statistical corellation between thefrequency that a document is accessed andsize (fig. 3)

✔ low statistic correlation between a documenaccess frequency and its average modificatioper request.

✔ request distribution to servers does not folloa Zipf law.; there is no single servercontributing to the most popular pages (fig.

α 0.64 0.83,∈

Page 7 of 24eb Caching

W

ts

f

Implications of Zipf-like behavior

✔ Model:

✗ A cache that receives a stream of reques

✗ N = total number of pages in the universe

✗ = probability that given the arrival oa page request, it is made for page

✗ Pages ranked in order of their popularity,page i is thei-th most popular

✗ Zipf-like distribution for the arrivals:

,

✔

✗ no pages are invalidated in the cache.

PN i( )

i i 1 N,=( ),

PN i( ) Ω iα⁄=

Ω 1 iα⁄i 1=

N

∑ 1–

=

Page 8 of 24eb Caching

W

or

Implications of Zipf-like behavior

infinite cache, finite request stream:

finite cache, infinite request stream:

page request interarrival time:

H(R) = hit ratio

d(k) = probability distribution that the nextrequest for page iis followed by k-1 requests fpages other than i

H R( )Ω 1 α–( )⁄( ) RΩ( )1 α⁄ 1–= 0 α 1<

W

Cache Replacement Algorithms(fig. 9)

✔ GD-Size:

✗ performs best for small sizes of cache.

✔ Perfect-LFU

✗ performs comparably with GD-Size in hit-ratio and much better in byte hit-ratio forlarge cache sizes.

✗ drawback: increased complexity

✔ In-Cache-LFU

✗ performs the worst

Page 10 of 24eb Caching

W

to

ve

Web document sharing and proxycaching

✔ What is the best performance one couldachieve with “perfect” cooperative caching?

✔ For what range of client populations cancooperative caching work effectively?

✔ Does the way in which clients are assigned caches matter?

✔ What cache hit rates are necessary to achieworthwhile decreases in document accesslatency?

Page 11 of 24eb Caching

W

al

For what range of client populations cancooperative caching work effectively?

✔ for smaller populations, hit rate increasesrapidly with population (cooperative cachingcan be used effectively)

✔ for larger population cooperative caching isunlikely to bring benefits

Conclusion:

✔ use cooperative caching to adapt to proxyassignments made for political or geographicreasons.

0 5000 10000 15000 20000 25000 30000

Population

0

10

20

30

40

50

60

70

80

90

Req

ues

t H

it R

ate

(%)

Ideal (UW)Cacheable (UW)Ideal (MS)Cacheable (MS)

Page 12 of 24eb Caching

W

Object Latency vs. Population

✔ Latency stays unchanged when populationincreases

✔ Caching will have little effect on mean andmedian latency beyond very small clientpopulation (?!)

0 5000 10000 15000 20000

Population

0

500

1000

1500

2000

Ob

ject

Lat

ency

(m

s)Mean No CacheMean CacheableMean IdealMedian No CacheMedian CacheableMedian Ideal

Page 13 of 24eb Caching

W

Byte Hit Rate vs. Population

✔ same knee at about 2500 clients

✔ shared documents are smaller

0 5000 10000 15000 20000

Population

0

10

20

30

40

50

60B

yte

Hit

Rat

e (%

)

IdealCacheable

Page 14 of 24eb Caching

W

on

Bandwith vs. population

✔ while caching reduces bandwidth consumptithere is no benefit to increased clientpopulation.

0 5000 10000 15000 20000

Population

0

1

2

3

4

5M

ean

Ban

dw

idth

(M

bit

s/s) No Cache

CacheableIdeal

Page 15 of 24eb Caching

W

ct is

Proxies and organizations

✔ there is some locality in organizational membership, but the impanot significant

978

735

480

425

392

337

251

246

239

228

222

219

213

200

192

15 Largest Organizations

0

10

20

30

40

50

60

70

80

90

100

Hit

Rat

e (%

)

Ideal CooperativeIdeal LocalCacheable CooperativeCacheable Local

978

735

480

425

392

337

251

246

239

228

222

219

213

200

192

15 Largest Organizations

0

10

20

30

40

50

60

70

Hit

Rat

e (%

)

UWRandom

Page 16 of 24eb Caching

W

e

Impact of larger population size

✔ unpopular documents are universallyunpopular => unlikely that a miss in one larggroup of population to get a hit in anothergroup.

UW

Idea

l

UW

Cac

heab

le

MS

Idea

l

MS

Cac

heab

le

0

20

40

60

80

100

Hit

Rat

e (%

)

CooperativeLocal

Page 17 of 24eb Caching

W

h

t

Performance of large scale proxycaching - model

✔ N = clients in population

✔ n = total number of documents ((aprox. 3.2billions)

✔ pi = fraction of all requests that are for the i-tmost popular document ( , )

✔ exponential distribution of the requests doneby client with parameter , where = 590reqs/day = average client request rate

✔ exponential distribution of time for documenchange, with parameter (unpopulardocument days, populardocuments days)

✔ pc = 0.6 = probability that the documents arecacheable

✔ E(S) = 7.7KB = avg. document size

✔ E(L) = 1.9 sec = last-byte latency to server

pi1

iα----≈ α 0.8=

λN λ

µµu 1 186⁄=

µp 1 14⁄=

Page 18 of 24eb Caching

W

te

of

ith

Hit rate, latency and bandwidth

Hit rate = f(Population) has three areas:

✔ 1. the request rate too low to dominate the raof change for unpopular documents

✔ 2. marks a significant increase in the hit ratethe unpopular documents

✔ 3. the request rate is high enough to cope wthe document modifications

Latency

Bandwidth

10^2 10^3 10^4 10^5 10^6 10^7 10^8

Population (log)

0

20

40

60

80

100

Cac

hea

ble

Hit

Rat

e (%

)

Slow (14 days, 186 days)Mid Slow (1 day, 186 days)Mid Fast (1 hour, 85 days)Fast (5 mins, 85 days)

latreq 1 HN–( ) E L( )⋅ HN lathit⋅+=

BN HN E S( )⋅=

Page 19 of 24eb Caching

W

etsitn

Document rate of change

Proxy cache hit is very sensitive to the changrates of both popular and unpopular documenwith a decrease on the time scales at which hrate is sensitive once the size of the populatioincreases.

1min

2 5 10 30 1hour

2 6 12 1day

2 7 14 30 180

Change Interval (log)

0

20

40

60

80

100

Cac

hea

ble

Hit

Rat

e (%

)

mu_pmu_u

A

B

C

1min

2 5 10 30 1hour

2 6 12 1day

2 7 14 30 180

Change Interval (log)

0

20

40

60

80

100

Cac

hea

ble

Hit

Rat

e (%

)

mu_pmu_u

AB

C

Page 20 of 24eb Caching

W

ash

ree

ve

Comparing cooperative cachingscheme

Three types of cache schemes: hierarchical, hbased and summary caches;

Conclusions:

✔ the achievable benefits in terms of latency aachieved at the level of city-level cooperativcaching;

✔ a flat cooperative scheme is no longer effectiif the served area increases after a limit

✔ in a hierarchical caching scheme the higherlevel might be a bottleneck in serving therequests.

Page 21 of 24eb Caching

W

in

Hierarchical Caching (Squid):

✔ a hierarchy of k levels

✔ a client request is forwarded up in thehierarchy until a cache hit occurs.

✔ a copy of the requested object is then storedall caches of the request path

Page 22 of 24eb Caching

W

s

d

Hash-based caching system:

✔ assumes a total of m caches cumulativelyserving a population of size N

✔ upon a request for an URL the clientdetermines the cache in charge and forwardthe request to it.

✔ if the cache has the URL the page is returneto the client, otherwise the request isforwarded to the server

Page 23 of 24eb Caching

W

f

a

ter

Directory based system(Summary cache)

✔ a total of m caches serving cumulatively apopulation of size N

✔ the population is partitioned intosubpopulations of size N/m and eachpopulation is served by a single cache

✔ each cache maintains a directory thatsummarizes the documents stored at each othe other cache in the system

✔ when a client request can not be served by local cache, it is randomly forwarded to acache which is registered to have thedocument. If no document has the documenthe request is forwarded to the original serv

Page 24 of 24eb Caching

Web Cachingbased on:Web Caching, Geoff HustonWeb Caching and Zipf-like Distributions: Evidence and Implications, L. Breslau, P. Cao, L. Fan, G...On the scale and performance of cooperative Web proxy caching, A. Wolman, G. Voelker, N. Sharma, ...

Hypertext Transfer Protocol (HTTP)based on an end-to-end model (the network is a passive element)Pros:The server can modify the content;The server can track all content requests;The server can differentiate between different clients;The server can authenticate the client.

Cons:Popular servers are under continuous high stresshigh number of simultaneously opened connectionshigh load on the surrounding networkThe network may not be efficiently utilized

Why Caching in the Network?reduce latency by avoiding slow links between client and origin serverlow bandwidth linkscongested links

reduce traffic on linksspread load of overloaded origin servers to caches

Caching pointscontent provider (server caching)ISPmore than 70% of ISP traffic is Web based;repeated requests at about 50% of the traffic;ISPs interested in reducing the transmission costs;provide high quality of service to the end users.

end client

Caching ChallengesCache consistencyidentify if an object is stale or fresh

Dynamic contentcaches should not store outputs of CGI scripts (Active Cache?)

Hit counts and personalizationPrivacy concerned userhow do you get a user to point to a cache

Access controllegal and security restrictions

Large multimedia files

Web cache access pattern - (common thoughts)the distribution of page requests follows a Zipf-like distribution: the relative probability of a...for an infinite sized cache, the hit-ratio grows in a log-like fashion as a function of the clien...the hit-ratio of a web cache grows in a log-like fashion as a function of cache sizethe probability that a document will be referenced requests after it was last referenced is rough...

Web cache access pattern - (P. Cao Observations)distribution of web requests follow a a Zipf- like distribution (fig. 1)the 10/90 rule does not apply to web accesses (70% of accesses to about 25%-40% of the documents)...low statistical corellation between the frequency that a document is accessed and its size (fig. 3)low statistic correlation between a document’s access frequency and its average modifications per...request distribution to servers does not follow a Zipf law.; there is no single server contributi...

Implications of Zipf-like behaviorModel:A cache that receives a stream of requestsN = total number of pages in the universe= probability that given the arrival of a page request, it is made for pagePages ranked in order of their popularity, page i is the i-th most popularZipf-like distribution for the arrivals:

,no pages are invalidated in the cache.

Implications of Zipf-like behaviorinfinite cache, finite request stream:finite cache, infinite request stream:page request interarrival time:H(R) = hit ratiod(k) = probability distribution that the next request for page iis followed by k-1 requests for p...

Cache Replacement Algorithms (fig. 9)GD-Size:performs best for small sizes of cache.

Perfect-LFUperforms comparably with GD-Size in hit- ratio and much better in byte hit-ratio for large cache ...drawback: increased complexity

In-Cache-LFUperforms the worst

Web document sharing and proxy cachingWhat is the best performance one could achieve with “perfect” cooperative caching?For what range of client populations can cooperative caching work effectively?Does the way in which clients are assigned to caches matter?What cache hit rates are necessary to achieve worthwhile decreases in document access latency?

For what range of client populations can cooperative caching work effectively?for smaller populations, hit rate increases rapidly with population (cooperative caching can be u...for larger population cooperative caching is unlikely to bring benefitsConclusion:use cooperative caching to adapt to proxy assignments made for political or geographical reasons.

Object Latency vs. PopulationLatency stays unchanged when population increasesCaching will have little effect on mean and median latency beyond very small client population (?!)

Byte Hit Rate vs. Populationsame knee at about 2500 clientsshared documents are smaller

Bandwith vs. populationwhile caching reduces bandwidth consumption there is no benefit to increased client population.

Proxies and organizationsthere is some locality in organizational membership, but the impact is not significant

Impact of larger population sizeunpopular documents are universally unpopular => unlikely that a miss in one large group of popul...

Performance of large scale proxy caching - modelN = clients in populationn = total number of documents ((aprox. 3.2 billions)pi = fraction of all requests that are for the i-th most popular document (, )exponential distribution of the requests done by client with parameter , where = 590 reqs/day = a...exponential distribution of time for document change, with parameter (unpopular document days, po...pc = 0.6 = probability that the documents are cacheableE(S) = 7.7KB = avg. document sizeE(L) = 1.9 sec = last-byte latency to server

Hit rate, latency and bandwidthHit rate = f(Population) has three areas:1. the request rate too low to dominate the rate of change for unpopular documents2. marks a significant increase in the hit rate of the unpopular documents3. the request rate is high enough to cope with the document modifications

LatencyBandwidth

Document rate of changeProxy cache hit is very sensitive to the change rates of both popular and unpopular documents wit...

Comparing cooperative caching schemeThree types of cache schemes: hierarchical, hash based and summary caches;Conclusions:the achievable benefits in terms of latency are achieved at the level of city-level cooperative c...a flat cooperative scheme is no longer effective if the served area increases after a limitin a hierarchical caching scheme the higher level might be a bottleneck in serving the requests.

Hierarchical Caching (Squid):a hierarchy of k levelsa client request is forwarded up in the hierarchy until a cache hit occurs.a copy of the requested object is then stored in all caches of the request path

Hash-based caching system:assumes a total of m caches cumulatively serving a population of size Nupon a request for an URL the client determines the cache in charge and forwards the request to it.if the cache has the URL the page is returned to the client, otherwise the request is forwarded t...

Directory based system (Summary cache)a total of m caches serving cumulatively a population of size Nthe population is partitioned into subpopulations of size N/m and each population is served by a ...each cache maintains a directory that summarizes the documents stored at each of the other cache ...when a client request can not be served by a local cache, it is randomly forwarded to a cache whi...