1 Quality of Service Guarantees for Multimedia Digital Libraries and Beyond Gerhard Weikum...

1

Quality of Service Guarantees for Multimedia Digital Libraries

and Beyond

Gerhard Weikum

weikum@cs.uni-sb.de

http://www-dbs.cs.uni-sb.de

2

Vannevar Bush’s Memex (1945)Collect all human knowledge into computer storage

Size of today‘s and tomorrow‘s applications:

Everything you see or hear: 1 MB/s * 50 years 2 PB

Library of Congress: 20 TB books + 200 TB maps + 500 TB video + 2 PB audio

Challenges: size of data• performance & QoS• intelligent search

3

Multimedia Data Management

Discrete Data Index Data Continuous Data

ParallelDiskSystem

ServerMemoryBuffer

Clients

High-speed Networkwith QoS Guarantees

. . .

QoSGuarantees

byData Server

4

Internal Server Error.Our system administrator has been notified. Please try later again.

Check Availability(Look-Up Will Take 8-25 Seconds)

The Need for Performance and QoS Guarantees

• Service performance is best-effort only• Response time is unacceptable during peak load because of queueing delays• Performance is mostly unpredictable !

Observations:

5

From Best Effort To Performance & QoSGuarantees

”Our ability to analyze and predict the performance of the enormously complex software systems ...are painfully inadequate”

(Report of the US President’s Technology Advisory Committee)

• Very slow servers are like unavailable servers• Tuning for peak load requires predictability of workload config performance function• Self-tuning requires mathematical models• Stochastic guarantees for huge #clients

6

Outline

The Need for Performance Guarantees

Towards a Science of QoS Guarantees

QoS for Continuous-Data Streams

Caching and Prefetching for Discrete Data

Self-tuning Servers using Stochastic Predictions

7

Performance and Service Qualityof Continuous-Data Streams

Quality of service (QoS): (almost) no "glitches"

High throughput (= concurrently active streams)

admission control

8

Data Placement and SchedulingPartitioning of C-data Objects with VBR (Variable Bit Rate)into CTL Fragments (of Constant Time Length)Coarse-grained Striping with Round-robin AllocationPeriodic, Variable-order SchedulingOrganized in Rounds of Duration T (= Fragment Time Length)

0 T 3T2T

...

0 T 3T2T

0 T 3T2T

2 2 2

1

2

3

1

2

1

2

3

1

2

1

2

3

1 1 1

3 3 3

1

2

3

1

1

2

Admission control:

No way!Now go ahead!

1

Yes, go ahead!

1 1

9

Admission Control

Stochastic QoS: Admit at most N streams such thatP [ total service time > T ]

- tolerable by most multimedia applications- appropriate with many workload and system parameters being random variables- allows much better resource utilization compared to worst-case modeling

threshold

Worst-case QoS: Admit at most N streams such thatN * Tmax T

with Stochastic QoS Guarantees

10

Mathematical Tools

X, Y, ...: continuous random variables with non-negative, real values

(cumulative) distribution function of X :][)( xXPxFX

probability density function of X :)(')( xFxf XX

:][)()(*0 sX

Xsx

X eEdxxfesf Laplace-Stieltjes transform(LST) of X

z

YXYX dxxzFxfzF0

)()()(Convolution

)(*)(*)(* sfsfsf YXYX

0|)(*inf][ X

t fetXP Chernoff bound

11

Total Service Time Per Round(With N Streams Per Disk)

T T T Tserv seek rot i trans ii

N

i

N

, ,

11f f f fserv seek rot

Ntrans

N* * * *

T N tseekZ

NSEEKseek

( ) :1

1 f s eseek

s SEEK* ( )

f se

s ROTrot

s ROT

* ( ) 1

f t

ROTrot ( ) 1

f sC ROT

C ROT strans* ( )/

/

F t F

tROT

Ctrans size( )

with

f x x esizex( ) ( ) / ( ) 1

P T t e fservt

serv[ ] inf * ( )

0

12

Total Service Time Per Round(With N Streams)

T T Tserv seek rot ii

N

i

N

,

11

T N tseekZ

NSEEKseek

( ) :1

1

f tROTrot ( )

1

F t Ft

ROTCtrans size( )

with

f x x esizex( ) ( ) / ( ) 1

P T t e fservt

serv[ ] inf * ( )

0

f f f fserv seek rotN

transN* * * *

f s eseeks SEEK* ( )

f se

s ROTrot

s ROT

* ( ) 1

f sC ROT

C ROT strans* ( )/

/

Ttrans,i

13

Stochastic versus Worst-Case QoS Guarantees

0

0,2

0,4

0,6

0,8

1

1,2

12 14 16 18 20 22 24 26 28 30 32 34 36 38 40

analyticreal

N

p late

14

Stochastic versus Worst-Case QoS Guarantees

0

0,05

0,1

0,15

0,2

0,25

0,3

12 14 16 18 20 22 24 26 28 30

analyticreal

N

p late

analytic realworst-case

15

Generalization to Mixed-Workload Servers

0 T 3T2T

arrivals ofdiscrete-datarequests

departures of completed discrete-datarequests

4T

response time

response time

response time

Additional performance guarantee for discrete-data requests: ][ ttimeresponseP (e.g., with t = 2s and = 0.95)

Needs clever scheduling and sophisticated stochastic model,to provide both continuous-data and discrete-data guarantees

16

QoS & Performance Guarantees for Mixed Workload Servers

P [ glitch frequency of a stream > tolerance ] threshold

P [ admission/startup delay of a stream > tolerance ] threshold

forContinuousData

forDiscreteData

P [ response time > tolerance t ] threshold (e.g., t = 2 seconds, = 5 percent)

Detailed analytic model can derive minimum-cost server configurationfor specified QoS & performance requirementsincl. differentiated QoS for multiple user/request classes

Auto-Configuration of Data Server:

17

Outline

The Need for Performance Guarantees

Towards a Science of QoS Guarantees

QoS for Continuous-Data Streams

Caching and Prefetching for Discrete Data

Self-tuning Servers using Stochastic Predictions

18

The Need for Caching in Storage Hierarchies

Searchengine

Internet

Proxy

Clients

...

DL server

Ontologies,XML etc.

Very high access latency ! CachingPrefetching

100 TB

5 TB

50 GB

19

Basic Caching PoliciesLRU: Drop page that has been least recently used

Example:

time

AB

C DX Y X Y

AB

C DX Y X Y

AB

C DX Y X Y

1 2 3 4 5 10 15 20 24 now

LRU-k: Drop page with the oldest k-th last reference

estimates heat (p) =

optimal for IRM)( ptnow

k

k

20

LRU-k OptimalityIRM: pages 1 ... n with ref. probabilities 1 ... n (i i+1)

and backward distances b1 ... bn

timenow

3 2 13221323

b2

]|..[ dbprobrefhasxP xi

n

hhhx

iix

hasxPhasxdbP

hasxPhasxdbP

1][]|[

][]|[

]|..[ dbxofprobrefE x

n

hxhh dbhasxP

1]|[

Theorem: ...][......][... yExEbb yx

n

h

kdh

khk

d

kdi

kik

d

n

n

111

11

/1)1(

/1)1(

21

LRU-k as Maximum Likelihood Estimator

n

iibPL

1]|[

1 ii

for observation b1, ..., bn with bi < b i+1

maximize

n

i

kbi

kik

ib i

111 )1(

0)ln(

i

L 1/

/1

kb

bk

i

ii

ii b

k for k << bi

IRM: pages 1 ... n with ref. probabilities 1 ... n (i i+1)and backward distances b1 ... bn

timenow

3 2 13221323

b2

22

Cache Size Configuration

170

$50064

$5032

sMBKB 1min21.0

Keep page in cache if diskcache CC

Cost / throughput consideration:

yh

sMB

KB 1$

10164

$5032 19 y

Keep page in cache if waitcache CC

Cost / response-time consideration:

Minimum cache size M such that

goalpercentile RTMgfratiohitfRT ...)),((...),(

Response-time guarantee:

23

LRU-k Cache Hit Rate (for Cache Size M)][:)( WwindowintimeskleastatreferencedpagesdistinctEWP

jWi

ji

n

i

W

kjj

W

)1(1

)(:~ 1 MPW

][: cacheinresidesipagePpi jWi

ji

W

kj jW

~~ ~)1(

][:)( hitcacheisreferencePMH

n

iii p

1

20

40

60

80 LRU

LRU-2

LRU-3

A0

cache size M

hit rate H(M) [%]

24

Stochastic Response Time Guaranteewith cache size M, block size S, and multi-zone disk with known seek-time function, Z tracks of capacity Cmin Ci Cmax, rotation time T

n

iRdiskiiRcacheiiR tfptfptf

1)()1()()(

n

iRdiskiiR sfpsf

1

** )()1()(

0

* )()(t

Xst

X dttfesfwith LST

)(

)1()(

***

sfs

ssff

servdiskdiskservRdisk

n

iiidisk p

1)1(

][ servdisk tE

with

)()()()( **** sfsfsfsf transrotseekserv

M/G/1 queue:

)( inf ][

0

*RfetRP tChernoff bound:

25

Extended LRU-k-based Policies

Generalization to variable-size documents:

Generalization to non-uniform / hierarchical storage:

temperature (d) =

benefit (d) =

drop documents with lowest

drop documents with lowest

)()(

psizepheat

)(cos)( dtdetemperatur fetch

Generalization to cooperative cachingin computer cluster

singletisdifdtreplicaisdifdt

fetch diskcacheremotedt )(cos

)(cos)(cos

Speculative PrefetchingArchive

Cache

Mask high access latency

Speculative prefetching

Keep long-term beneficial data in cache

Throttling of prefetching

Prefetch x iff benefit(x,T) > {benefit(y,T) | y victims}

with benefit (x,T) =

))()(()(

][#xRTxRT

xsizeTtimeinxtoaccessesE

cachearchive

with time horizon T = „max“ (RTarchive)

Context-aware Prefetching and Caching

Session 1 Session 2 ...

accessdoc. i

accessdoc. k

doc. f doc. g doc. h

... ...

Pif=0.80.1

0.9

0.30.1 ...

P[time in i t]Hi=E[...]=10s

Hk=10s

Hf=30s

Sessionarrival rate

newsessions

...

...

HN+1=c/

...

0.1

with continuous state-residence timesModel session behavior as Markov chain

Superimpose CTMCs of all active sessions

Incorporate arrivals of new sessions

CTMC-based Access PredictionGiven: states di (i=1, ..., N+c) with transition probabilites pij and mean residence times Hi (departure rates i=1/Hi)

Uniformization:

ijfor

ijforpij

i

iijp

/1with = max{i}

N(x,T) = E[#accesses to x in T] = s j

jxjsstate p

TE

/1

)(),(

where

1

1

0

)(

!)(1

)(n

n

m

mij

nt

ij pnt

etE

cN

kkj

mik

mij ppp

1

)1()( jiifjiif

ijp 0

1)0(

Transient analysis for time horizon T:

MCMin Prefetching and Caching Algorithm

access tracking and online bookkeeping for statistics

periodic evaluation of N(state(s),T) for active sessionsbased on approximative CTMC transient analysis

prefetching candidates

Prefetch x iff benefit(x,T) > {benefit(y,T) | y victims}

with benefit (x,T) =

))()()(()(),(

xxRTxRTxsizeTxN

penaltycachearchive

)()()( xtxtx servservpenalty with

+ appropriate device scheduling at server

Overhead: • size of bookkeeping data < 0.02%• compute time per access 1 ms• both dynamically adjustable

Performance ExperimentsSimulations based on WWW-server access patterns

05

101520253035404550

0,20% 1% 2%

LazyTempMCMin

Mean response time [s]

Cache size / archive size

Applicability of LRU-k and MCMin Familyfor Internet and intranet proxies and clients

for data hoarding in mobile clients

for (stochastically) guaranteed response time

for caching of (partial) search results

w.r.t. heterogeneous data servers,as opposed to best-effort caching

when client goes on low (or zero) connectivity,prefetch near-future relevant data and programs

with careful management of access statistics

in data warehouses, digital libraries, etc.

for adaptive broadcast of data feedsin networks with asymmetric bandwidth

Interesting Research Problems

Response-time guarantee for MCMin

Optimal (online) decisions about amoung of bookkeeping

?

?

Caching and prefetching fordifferentiated QoS (multiple user/request classes)

?

Caching of (partial) search results andprefetching for (speculative) query evaluationin ranked (XML) retrieval

?

????

33

Outline

The Need for Performance Guarantees

Towards a Science of QoS Guarantees

QoS for Continuous-Data Streams

Caching and Prefetching for Discrete Data

Self-tuning Servers using Stochastic Predictions

34

Advancing the State of the Art on QoS

+ Substantially Better Cost/Performance

+ Major Building Blocks for Configuration Tool for Specified QoS Guarantees and Self-tuning, Zero-admin Operation

Benefit of stochastic models and derived algorithms/systemsover commercial state-of-the-art systems(e.g., Oracle Media Server, MS NetShow Theater Server, etc.):

+ Predictable Performance

35

QoS in (Web) Query Processing

Credibility

Timeliness

Responsiveness & Cost-effectivity (Performance)

AccuracyExample: Select ... ).( AmountOSum

ComprehensivenessExample: association rules of the kind

Software Engineering & Y2K Astrology

Combined with IR & MultimediaExamples: Where ... P About {„Mining“, „19th Century“} ...

Where ... P.Category =„CDs“ And P Sounds Like

36

The End

„low-hanging fruit“ engineering: 90% solution with 10% intellectual effort

self-tuningservers withguaranteedperformance

„Web engineering“ for end-to-end QoSwill rediscover stochastic modeling or will fail

need libraries of composable building blocks withpredictable behavior and (customizable) QoS guarantees

Conceivable killer argument:Infinite RAM & network bandwidth and zero latency (for free)

But:• An engineer is someone who can do for a dime what any fool can do for a dollar.• Predictions are very difficult, especially about the future.