1

Mor Harchol-Balter

Taka Osogami, Adam Wierman, Alan Scheller-Wolf, Mark Squillante

Dimensionality Reductionfor Analyzing Cycle Stealing,

Task Assignment,Priority Queueing,

and Threshold Policies

Carnegie Mellon University

2

Many multiserver scheduling problems

H

H

LHL

FIFO

FIFO

Goal: Mean response time per job type

3

Many multiserver scheduling problems

Common Problem: 2D-infinite Markov chain (or nD-infinite)

H

H

MHL

FIFO

FIFO

4

Dimensionality Reduction (2D)Recursive Dimensionality

Reduction(nD)— Approximate technique

— Problem-dependent. Doesn’t always apply.

+ Provides first accurate performance numbers for many common problems.

+ Very fast (less than 1 sec).

+ < 1% error typically, and can improve.

+ Can analyze any load (non-limiting).

+ Allows PH service time distributions.

+ Adding thresholds and switching costs

is often easy.

5

Cycle Stealing Problem

B jobs/sec D jobs/sec

Load B Load D

Betty the beneficiary

Dan the donor

6

Cycle Stealing Problem

B jobs/sec D jobs/sec

Load B Load D

When donor is idle, donorhelps beneficiary.

switch

7

Cycle Stealing Problem

B jobs/sec D jobs/sec

Load B Load D

When new donor job arrives,donor switches back to donorqueue.

switch back

8

Generalized Cycle Stealing [Sigmetrics 03]

D jobs/sec

Load B Load D

time KSW

time KBA

B jobs/sec

NBthresh

Only switchif Betty hasNB

thresh jobs queued.

NDthresh

Only switch backif Dan hasND

thresh jobs queued.

PH job size PH job size

9

Cycle Stealing Problem

B jobs/sec D jobs/sec

Load B Load D

What is Betty’s/Dan’s mean response time?

10

What’s so hard?

Even simplest-case chain grows infinitely in 2D.

0B,0D 1B,0D 2B,0D

2B,1D

2B,2D

1B,1D

1B,2D

0B,1D

0B,2D

B B B

B B B

B B B

B B B

B B B

B B B

D

D

D

D

D

D

D

D

D

D

D

D

D

D

D

D

D

D

11

Prior work:cycle stealing & coupled-processor

Fayolle, Iasnogorodski,Konheim, Meilijson,Melkman ...

Convert to Riemann-

Hilbertproblem

Very complexintegrals.

Cohen, Boxma, Borst,Uitert, Jelenkovic ...

Convert to Wiener-Hopf

boundary problem

Very complexintegrals forWORKLOAD.No numbers.

Exponentialjob sizes

(80’s)

GeneralJob sizes

(80,90,00’s)

Truncatethe

chain

Tail Asymp or

Heavy traff.

Bob FoleyMcDonaldMike HarrisonBoxmaBorstvan UitertWilliams

12

Dimensionality Reduction Key idea:

2D-infinitechain

1D-infinitechain

VERY HARD EASY

13

What’s so hard?

Even simplest-case chain grows infinitely in 2D.

0B,0D 1B,0D 2B,0D

2B,1D

2B,2D

1B,1D

1B,2D

0B,1D

0B,2D

B B B

B B B

B B B

B B B

B B B

B B B

D

D

D

D

D

D

D

D

D

D

D

D

D

D

D

D

D

D

14

Solution

1D-infinite chain!

New type of transition. Donor Busy period: BD

0B,0D 1B,0D 2B,0D

2B,1+D1B,1+D0B,1+D

B B B

B B B

B B B

B B B

D D D DD D

Cooperatingservers

Workingindependently

15

Solution

0B,0D 1B,0D 2B,0D

2B,1+D1B,1+D0B,1+D

B B B

B B B

B B B

B B BD

=

D

B B B

B B B

D D

Approximation,but can be madeas close to exactas desired.[Tools03a,Tools03b]

Cooperatingservers

Workingindependently

16

Markov chain is easy to generalize

B jobs/sec D jobs/sec

Load B Load D

time KSW

time KBA

NBthresh

Genl. job size Genl. job size

NDthresh

• Generalize service distribution• Generalize to switching times• Generalize to include thresholds

17

MODIFIED 1D-chain:cycle stealing with switching costs and NB

th = 3

18

Some interesting questionsQ: When does cycle stealing pay?

Q: How does donor job size variability affect benef. resp. time?

A: For B>1, always pays. For B<1, sometimes pays.

E[TB]

B

E[TD]

B

orig cs

orig

cs

A: Hardly, for B < 1 ! ExpC2=8

C2=50

E[TB]

B

19

Some interesting questions

Q: How should we set NDth?

Q: How should we set NBth?

Increasing NBth

doesn’t hurt benef!

A: Want NBth high.

Increasing NBth

helps donor

E[TB] E[TD]

B B

A: Want NDth low if B<1.

Want NDth high if B>1.

Increasing NDth

helps benefonly in overload!

Increasing NDth

hurts donor

E[TB] E[TD]

BB

20

Dimensionality Reduction (DR)

applied to Cycle stealing with switching costs and thresholds.

2D-infinite chain 1D-infinite chain

So far

Recursive Dimensionality Reduction (RDR)

applied to Priority scheduling in multiserver systems.

nD-infinite chain 1D-infinite chain

Next

21

Goal: Mean response time per job type.

Priority Scheduling

M/PH/k with priority classes

L

H

H

M HL

22

Scaling as Single-server:

Buzen and Bondi

Aggregation intoTwo classes:

Mitrani and KingNishida

Multi-classsimpleapprox.

Two jobclasses,

exponential

Cidon and SidiFeng, Kawada, Adachi, Gail, Hantler, TaylorMiller

Matrix Analyticor

Gen. Functions

Aggregationor

Truncation

Kao and Narayanan,Kao and Wilson,Kapadia, Kazumi,Mitchell, NishidaNgo and Lee

Little known for > 2 classes or non-exponential.

Prior Work:Multi-Server Priority Queues

23

What’s so Hard?Low

Hi

Med

Now chain grows infinitely in 3 dimensions!

24

2 Priority classesis easy via Dimensionality

Reduction

0L,1H 1L,1H 2L,1H

2L,2+H1L,2+H0L,2+H

L L L

L L L

L L L

H H H HH H

0L,0H 1L,0H 2L,0H

L L L

L L L

H H HH H H

2 22

25

What about 3 classes H,M,L?

• Can get H and M response times from this chain:

1,00,0 2,0

1,10,1 2,1

1,2+0,2+ 2,2+

• Obvious Idea: Combine H and M into single class HM. L’s are second class.

• IMPRECISE: Duration of HM busy period depends on who started it: H&H, H&M, or M&M. Must differentiate.

26

Busy Periods of HM class

Start of HM busyperiod

H

H

L L

End of HM busyperiod

orH

M

L LM

M

L Lor

H

L

L L orM

L

L L

B1

B2B3

B4B5

B6

27

1D-infinite Markov Chain abstraction

1

(i-1)L,0M,0H iL,0M,0H (i+1)L,0M,0H

(i-1)L,1M,0H iL,1M,0H (i+1)L,1M,0H

(i-1)L,0M,1H iL,0M,1H (i+1)L,0M,1H

(i-1)L,2M,0H iL,2M,0H (i+1)L,2M,0H

(i-1)L,1M,1H iL,1M,1H (i+1)L,1M,1H

(i-1)L,0M,2H iL,0M,2H (i+1)L,0M,2H

2

3

4

5

6

28

How to get busy period durations?

Start of HM busyperiod

H

H

L L

End of HM busyperiod

orH

M

L LM

M

L Lor

H

L

L L orM

L

L L

B1

B2B3

B4B5

B6

29

Q: How to get busy period durations?

1,00,0 2,0

1,10,1 2,1

1,2+0,2+ 2,2+

Start

of H

M

Busy

perio

ds

A: Passage times in this H & M chain.

End o

f HM

Busy

perio

ds

Fortunately [Neuts 78] givesus passage times in chain.

30

Extensions

RDR extends to any number of classes and servers.

Generalize service distribution.

Simple representation allows us to derive not just mean response time, but all moments.

Rest of talk: Lessons learned about priorities in multiserver systems.

31

Results: 4 priority classes

% error in delay

Mean

resp

on

se t

ime

M/M/2 with 4 classes

32

Effect of Variability

M/G/2 M/G/1

H

L

C2H

Mean

resp

on

se t

ime

Mean

resp

on

se t

ime

C2H

L

H

33

Effect of more servers

M/G/k with 2 priority classes

Mean

resp

on

se t

ime

C2H

1 server

2 servers

4 servers8 servers

34

Effect of Number servers

L

H

C2H

Low-priority perspective

Mean

resp

on

se t

ime

Mean

resp

on

se t

ime

High-priority perspective

C2H

1

2

4

884

2

1

35

Optimal number of servers(2 classes with same mean)

C2H

36

Optimal number of servers

Stupid Prioritization(Low priority jobs SMALLER)

Smart Prioritization(Low priority jobs BIGGER)

C2H

C2H

37

Lessons learned on priorities in multiserver

Multiserver priority queue performance isvery different from single server.

Performance depends on many factors:

Variability of job size Load Number of servers Stupid vs. Smart prioritization

38

Dimensionality Reduction (DR)

applied to Cycle stealing with switching costs and thresholds.

2D-infinite chain 1D-infinite chain

So far

Recursive Dimensionality Reduction (RDR)

applied to Priority scheduling in multiserver systems.

nD-infinite chain 1D-infinite chain

Next Other applications of DR and RDR

39

Other ProblemsSolved by Dimensionality Reduction

1. Threshold Scheduling with Affinities

BeneficiarySide Control

Help me when I have > TB

jobs, or you’re free.

TBTD

I’ll helpyou whenI have < TD jobs.

DonorSide Control

40

Other ProblemsSolved by Dimensionality Reduction

1. Threshold Scheduling with Affinities

2. Server Placement Questions

3. Smart task assignment2D-

infinite chain

1D-infinite chain

41

Dimensionality Reduction (2D)Recursive Dimensionality

Reduction(nD)— Approximate Technique

— Problem-dependent. Doesn’t always apply.

+ Provides first accurate performance numbers for many common problems.

+ Very fast (less than 1 sec).

+ < 1% error typically, and can improve.

+ Can analyze any load (non-limiting).

+ Allows PH service time distributions.

+ Adding thresholds and switching costs

is often easy.

Solves problems where one class depends on the other,but dependencies can be solved sequentially (H,M,L).Open Question: Systems that do not decouple?