Mor Harchol-Balter Carnegie Mellon University Computer Science

1

Mor Harchol-BalterCarnegie Mellon

UniversityComputer Science

Heavy Tails: Performance Models & Scheduling Disciplines

2

“size” = service requirement

load < 1

jobs SRPT

jobs

jobs PS

FCFS

Q: Which minimizes mean response time?

3

“size” = service requirement

jobs SRPT

jobs

load < 1

jobs PS

FCFS

Q: Which best represents scheduling in web servers ?

4

IDEA: Use SRPT instead of PS in Web servers

client 1

client 2

client 3

“Get File 1”

“Get File 2”

“Get File 3” Request for file

Requested file

Linux 0.S.

1231000

APACHEWEB

SERVERInternet

5

Objections to SRPT:

Need to know “size” of request

Unfairness to requests for big files

6

Outline of Talk

THEORY

IMPLEMENT

Papers are joint with Adam Wierman & Bianca Schroeder

I: Investigating Unfairness in SRPT (M/G/1, c.f.m.f.v.) [Sigmetrics 01]

II: Unfairness in All Scheduling Policies (M/G/1, c.f.m.f.v.) [Performance 02], [Sigmetrics 03]

III: Implementation of SRPT in Web servers [Transactions on Computer Systems 03], [ITC03]

THEORY

7

Outline of Talk

THEORY

IMPLEMENT

www.cs.cmu.edu/~harchol/

I: Investigating Unfairness in SRPT (M/G/1, c.f.m.f.v.) [Sigmetrics 01]

II: Unfairness in All Scheduling Policies (M/G/1, c.f.m.f.v.) [Performance 02], [Sigmetrics 03]

III: Implementation of SRPT in Web servers [Transactions on Computer Systems 03], [ITC03]

THEORY

8

SRPT has a long history ...

1966 Schrage & Miller derive M/G/1/SRPT response time:

1968 Schrage proves optimality

1979 Pechinkin & Solovyev & Yashkov generalize

1990 Schassberger derives distribution on queue length

BUT WHAT DOES IT ALL MEAN?

9

SRPT has a long history (cont.)

1990 - 97 7-year long study at Univ. of Aachen under Schreiber SRPT WINS BIG ON MEAN!

1998, 1999 Slowdown for SRPT under adversary: Rajmohan, Gehrke, Muthukrishnan, Rajaraman, Shaheen, Bender, Chakrabarti, etc. SRPT STARVES BIG JOBS!

Various o.s. books: Silberschatz, Stallings, Tannenbaum: Warn about starvation of big jobs ...

Kleinrock’s Conservation Law: “Preferential treatment given to one class of customers is afforded at the expense of other customers.”

10

Real-world job sizes are Heavy Tailed

Job size (x seconds)

log-log plot

xxsizejob 1} Pr{

[Sigmetrics 96]

Heavy-tailedProperty:

“Largest 1% of jobs comprise half the load.”

11

Unfairness Question

SRPT

PS?

?

Let =0.9. Let G: Bounded Pareto(= 1.1, max=1010)

Question: Which queue does biggest job prefer?

M/G/1

M/G/1

12

Results on UnfairnessLet =0.9. Let G: Bounded Pareto(= 1.1, max=1010)

SRPT

PSI SRPT

13

Results on UnfairnessLet G: Bounded Pareto(= 1.1, max=1010)

PS

SRPT

14

Unfairness – General Distribution

All-can-win-theorem:

For all distributions, if ½,

E[T(x)]SRPT E[T(x)]PS for all x.

15

All-can-win-theorem:

For all distributions, if ½,

E[T(x)]SRPT E[T(x)]PS for all x.

Proof:

x

tdt

0 )1 1x

)

x

xFx

2

2

))1(2

(

0x

dttft 2 )(

E[Wait(x)]SRPT

E[Res(x)]SRPT E[T(x)]PS

16

All-can-win-theorem:

For all distributions, if ½,

E[T(x)]SRPT E[T(x)]PS for all x.

Proof cont.

x

tdt

0 )1 1x

Need sufficient condition s.t.

)

x

xFx

2

2

))1(2

(

0x

dttft 2 )(

17

All-can-win-theorem:

For all distributions, if ½,

E[T(x)]SRPT E[T(x)]PS for all x.

Proof cont.

Need sufficient condition s.t.

x t

0 1)

t

yf(y)dy) t

)

x

xFx

2

2

))1(2

(

0x

dttft 2 )(

18

All-can-win-theorem:

For all distributions, if ½,

E[T(x)]SRPT E[T(x)]PS for all x.

Proof cont.

Need sufficient condition s.t.

1

)())( 2x

dttftxx

)())( 2 xFxxx Observe:

)

x

xFx

2

2

))1(2

(

0x

dttft 2 )(

19

Proof cont.

All-can-win-theorem:

For all distributions, if ½,

E[T(x)]SRPT E[T(x)]PS for all x.

1 x 2))1(2

) xFx2 (0x

dttft 2 )( ) xFx2 (0x

dttft 2 )(

Suffices that: 2(1 - (x))2 > 1 - .

Suffices that (x) < 1/2

20

Intuition

E[Wait(x)] E[Res(x)]: small

small

1) All-can-win-theorem: For all distributions, if ½, E[T(x)]SRPT E[T(x)]PS for all x.

2) Under Bounded Pareto distribution ( = 1.1), , E[T(x)]SRPT < E[T(x)]PS for all x. *HT Property*

21

Outline of Talk

THEORY

IMPLEMENT

www.cs.cmu.edu/~harchol/

I: Investigating Unfairness in SRPT (M/G/1, c.f.m.f.v.) [Sigmetrics 01]

II: Unfairness in All Scheduling Policies (M/G/1, c.f.m.f.v.) [Performance 02], [Sigmetrics 03]

III: Implementation of SRPT in Web servers [Transactions on Computer Systems 03], [ITC03]

THEORY

22

What is fair?

xxTxS

1

1PSxSE

Slowdown forjob of size x

Response time forjob of size x

PS does not bias towards any particular job size. Slowdown isindependentof size

Definition: A policy P is fair if E[S(x)]P ≤ E[S(x)]PS for all x. Otherwise, P is unfair.

23

Always Fair Always Unfair

Fair for all loads and distributions

Unfair for all loads and distributions

Sometimes Unfair

Fair for some loads, and unfair for other loads*

Classification of Scheduling Policies

* and distributions

24

Testing your intuition:

* and distributions

Always Fair Always Unfair

Fair for all loads and distributions

Unfair for all loads and distributions

Sometimes Unfair

Fair for some loads, and unfair for other loads*

• Where does SRPT lie? __________• What about other policies that prioritize based on remaining size? ____• What about preemptive policies that prioritize based on age? _____• What about preemptive policies that prioritize based on size? _____

25

Classification of Scheduling Policies

Lots of open problems…

FSPAlwaysFAIR

AlwaysUnfair

Sometimes Unfair

Age-BasedPolicies

Preemptive Remaining-size based Policies

Preemptive Size-basedPolicies

Non-preemptive

PS

PLCFS

FB

PSJFLRPT

FCFS LJFSJF

SRPT

26

Always Unfair

Unfair for all loads and distributions

Theorem: Any preemptive, size based policy, P, is Always Unfair.

Always Unfair

Case1: A finite size, y, receives lowest priority

Case 2: No finite size receives lowest priority (2a) Priorities decrease monotonically -- PSJF (2b) Priorities decrease non-monotonically.

27

Always Unfair

Unfair for all loads and distributions

Theorem: Any preemptive, size based policy, P, is Always Unfair.

Always Unfair

Case1: A finite size, y, receives lowest priority

y

V = Work in System

yVyBEySE P

1

11

1yVBE

28

Always Unfair

Unfair for all loads and distributions

Theorem: Any preemptive, size based policy, P, is Always Unfair.

Always Unfair

Case2a: Priorities decrease monotonically (PSJF)

1

1lim PSJF

xxSE

Infinite sized job has lowest priority ...

… but that job is treated fairly?

29

Always Unfair

Unfair for all loads and distributions

Theorem: Any preemptive, size based policy, P, is Always Unfair.

Always Unfair

Case2a: Priorities decrease monotonically (PSJF)

E[S(x)]

0

PS

PSJF

x

Finding a hump shows PSJF is Always Unfair

30

Always Unfair

Unfair for all loads and distributions

Theorem: Any preemptive, size based policy, P, is Always Unfair.

Always Unfair

Case2b: Priorities decrease non-monotonically

E[S(x)]

0

PS

PSJF

x

Find y beyond which PSJF treats all jobs unfairly.Find x > y, where x has lower priority than y.=> x is treated unfairly.

31

The mysterious hump

0 x

0

PS

PSJF

E[S(x)]

x

This hump appears in many common policies, 1. PSJF2. FB3. SRPT 4. SJF

32

Outline of Talk

THEORY

IMPLEMENT

www.cs.cmu.edu/~harchol/

I: Investigating Unfairness in SRPT (M/G/1, c.f.m.f.v.) [Sigmetrics 01]

II: Unfairness in All Scheduling Policies (M/G/1, c.f.m.f.v.) [Performance 02], [Sigmetrics 03]

III: Implementation of SRPT in Web servers [Transactions on Computer Systems 03], [ITC03]

THEORY

33

What does SRPT mean within a Web server?

• Many devices: Where to do the scheduling?

• Many jobs at once.

From theory to practice:

34

Server’s Performance Bottleneck

5

Linux 0.S.

WEBSERVER(Apache)

client 1

client 2

client 3

“Get File 1”

“Get File 2”

“Get File 3”

Rest ofInternet ISP

Site buyslimited fractionof ISP’s bandwidth

We schedule bandwidth at server’s uplink.

35

Network/O.S. insides of traditional Web server

Sockets take turnsdraining --- FAIR = PS.

WebServer

Socket 1

Socket 3

Socket 2Network Card

Client1

Client3

Client2BOTTLENECK

36

Network/O.S. insides of our improved Web server

Socket corresponding to filewith smallest remaining datagets to feed first.

WebServer

Socket 1

Socket 3

Socket 2Network Card

Client1

Client3

Client2

priorityqueues.

1st

2nd3rd

S

M

L

BOTTLENECK

37

Experimental Setup

Implementation SRPT-based scheduling: 1) Modifications to Linux O.S.: 6 priority Levels 2) Modifications to Apache Web server 3) Priority algorithm design.

Linux 0.S.

123

APACHEWEB

SERVER

Linux

123

200

Linux

123

200

Linux

123

200

switch

WAN

EM

UW

AN E

MU

WAN

EM

U

38

Experimental Setup

APACHEWEB

SERVER

Linux 0.S.

123

Linux

123

200

Linux

123

200

Linux

123

200

switch

WAN

EM

UW

AN E

MU

WAN

EM

U

Trace-based workload: Number requests made: 1,000,000Size of file requested: 41B -- 2 MBDistribution of file sizes requested has HT property.

FlashApache

WAN EMUGeographically-dispersed clients

10Mbps uplink100Mbps uplinkSurgeTrace-basedOpen systemPartly-open

Load < 1Transient overload

+ Other effects: initial RTO; user abort/reload; persistent connections, etc.

39

Load

FAIR

SRPTMea

n R

espo

nse

Tim

e (s

ec)

Results: Mean Response Time

.

.

.

.

.

.

40

Percentile of Request Size

Mea

n R

espo

nse

time

(s)

FAIR

SRPT

Load =0.8

Mean Response Time vs. Size Percentile

41

Transient Overload --Mean response time

SRPTFAIR

42

Transient overloadResponse time as function of job

size

small jobswin big!

big jobsaren’t hurt!

FAIR

SRPT

43

New project:Scheduling dynamic web requests

“$$$buy$$$”

“buy”

“buy”

Web Server(eg: Apache/Linux)

Internet

Internet

Database

Disks

Low-PriorityClient

Locks

High-PriorityClient

CPU(s)

Need to schedule the database ...

44

Conclusion

Misconceptionsabout

unfairness

Discrimination againsthigh-performancescheduling policies

Good news: Many high-performing policies are also fair in practice!

Classifying policies with respect to unfairnessis counter-intuitive.