Effect of higher moments of job size distribution on the performance of an M/G/k system VARUN GUPTA...

Effect of higher moments of job size distribution on the performance of

an M/G/k system

VARUN GUPTA

Joint work with:

Mor Harchol-Balter

Carnegie Mellon University

Jim Dai, Bert Zwart

Georgia Institute of Technology

2

Multi-server/resource sharing systems are the norm today

Multicore chips

Call centers

ServerFarms

3

M/G/k: the classical multi-server model

Poisson arrivals (rate )

J1Ji+1JiJ2Ji+2

GOAL : Analysis of mean delay (time spent in buffer)

4

M/G/k model assumptions and notation

• Poisson arrivals

• Service requirements (job sizes) are i.i.d.• S ≡ random variable for job sizes

• Define

• Define

Per server utilization or load:0 < < 1

Squared coefficient of variability (SCV) of job sizes:

C2 0

5

M/G/k mean delay analysis

• Lets take a step back: M/G/1

6

M/G/k mean delay analysis

• Lets take a step back: M/G/1

7

M/G/k mean delay analysis

• Lee and Longton (1959)

– Simple and closed-form– Involves only first two moments of S– Exact for k=1– Asymptotically exact in heavy traffic [Köllerström[74]]

• No exact analysis exists• All closed-form approximations involve only the

first two moments of S – Takahashi[77], Hokstad[78], Nozaki Ross[78], Boxma

Cohen Huffels[79], Whitt [93], Kimura[94]

8

But…

Q1: Are 2 moments of S enough to reasonably approximate E[Delay]?

• Does the third moment have no/negligible effect?

Q2: How inaccurate can a 2-moment approximation be?

Q3: Are 3 moments enough? 4 moments?

9

Outline

Q1: Are 2 moments of S enough to reasonably approximate E[Delay]?

• Does the third moment have no/negligible effect?

Q2: How inaccurate can a 2-moment approximation be?

Q3: Are 3 moments enough? 4 moments?

10

H2

• The H2 class has three degrees of freedom

• Can vary E[S3] while keeping first two moments constant

• Can numerically evaluate M/H2/k using the matrix analytic method

11

E[Delay] vs. E[S3]k=10, E[S]=1, C2=19, =0.9

0

1

2

3

4

5

6

7

0 1 2 3 4 5 6

E[Delay]

E[S3] X104

2-moment approx

E[Delay]M/H2/k

12

0

1

2

3

4

5

6

7

0 1 2 3 4 5 6X104

E[S3]

E[Delay]

E[Delay]M/H2/k

2-moment approx

E[S3] can have a huge impact on mean delay!

The mean delay decreases as E[S3] increases!

13

0

0.2

0.4

0.6

0.8

1

0 5 10 15 20

x)

Intuition for the effect of E[S3]

x) = load due to jobs smaller than x E[S]=1 C2=19

x

14

0

0.2

0.4

0.6

0.8

1

0 5 10 15 20

x)

E[S3]=600

Intuition for the effect of E[S3]

x) = load due to jobs smaller than x E[S]=1 C2=19

x

15

0

0.2

0.4

0.6

0.8

1

0 5 10 15 20

x)

E[S3]=600

E[S3]=700

Intuition for the effect of E[S3]

x) = load due to jobs smaller than x E[S]=1 C2=19

x

16

0

0.2

0.4

0.6

0.8

1

0 5 10 15 20

x)

E[S3]=600

E[S3]=700

E[S3]=1200

Intuition for the effect of E[S3]

x) = load due to jobs smaller than x E[S]=1 C2=19

x

17

0

0.2

0.4

0.6

0.8

1

0 5 10 15 20

E[S3]=600

E[S3]=700

E[S3]=1200

E[S3]=15000

x) = load due to jobs smaller than x E[S]=1 C2=19

Intuition for the effect of E[S3]

x

x)

18

0

0.2

0.4

0.6

0.8

1

0 5 10 15 20

As E[S3] increases (with fixed E[S] and E[S2]):• Load gets ‘concentrated’ on small jobs• Load due to ‘big’ jobs vanishes• Bigs become rarer, usually see small jobs only• Causes drop in E[Delay]M/H2/k

x

x)

Increasing E[S3]

Intuition for the effect of E[S3]

19

0

1

2

3

4

5

6

7

0 1 2 3 4 5 6X104

E[S3]

E[Delay]

E[Delay] vs. E[S3]k=10, E[S]=1, C2=19, =0.9

E[Delay]M/H2/k

2-moment approx

20

Outline

Q1: Are 2 moments of S enough to reasonably approximate E[Delay]?

• Does the third moment have no/negligible effect?

Q2: How inaccurate can a 2-moment approximation be?

Q3: Are 3 moments enough? 4 moments?

21

Outline

Q1: Are 2 moments of S enough to reasonably approximate E[Delay]?

• Does the third moment have no/negligible effect?

Q2: How inaccurate can a 2-moment approximation be?

Q3: Are 3 moments enough? 4 moments?

22

{G|C2} ≡ positive distributions with mean 1 and SCV C2

E[Delay]

G1

G2

GAP Error of2-moment approx

23

Our Theorems• Upper bound

• Lower bound– <1-1/k

– 1-1/k

24

E[D

elay

]

GAP

25

E[D

elay

]

0

D* has the smallest third

moment in {G|C2}

third moment as

0

26

E[D

elay

]

Conjecture

27

Outline

Q1: Are 2 moments of S enough to reasonably approximate E[Delay]?

• Does the third moment have no/negligible effect?

Q2: How inaccurate can a 2-moment approximation be?

Q3: Are 3 moments enough? 4 moments?

28

Outline

Q1: Are 2 moments of S enough to reasonably approximate E[Delay]?

• Does the third moment have no/negligible effect?

Q2: How inaccurate can a 2-moment approximation be?

Q3: Are 3 moments enough? 4 moments?

29

What about higher moments?

{G|C2}

H2

H*3

• H*3 class has four

degrees of freedom

• Can vary E[S4] while keeping first three moments constant

30

0

1

2

3

4

5

6

7

0 1 2 3 4 5 6X104

E[S3]

E[Delay]Increasing fourth moment

E[Delay] vs. E[S4]

k=10, E[S]=1, C2=19, =0.9

E[Delay]M/H2/k

2-moment approx

31

0

1

2

3

4

5

6

7

0 1 2 3 4 5 6X104

E[S3]

E[Delay]Increasing fourth moment

E[Delay]M/H2/k

2-moment approx

• Even E[S4] can have a significant impact on mean delay!• High E[S4] can nullify the effect of E[S3]!

32

E[D

elay

]

The BIG picture

LB1=E[Delay]M/D/k

UB1,2=(C2+1)E[Delay]M/D/k

LB1,2,3

UB1,2,3,4

LB1,2,3,4,5

Odd/Even moments refine the Lower/Upper bounds on mean delay

33

Outline

Q1: Are 2 moments of S enough to reasonably approximate E[Delay]?

• Does the third moment have no/negligible effect?

Q2: How inaccurate can a 2-moment approximation be?

Q3: Are 3 moments enough? 4 moments?

34

Conclusions

• Shown that 2-moment approximations for M/G/k are insufficient

• Shown bounds on inaccuracy of 2-moment only approximations– (C2+1) inaccuracy factor

• Observed alternating effects of odd and even moments

35

Thank you!

36

Open Questions

• Proof (or counter-example) of conjectures on bounds

• Are there other attributes of service distribution that characterize it better than moments?– For example, mean and variability of small and big

jobs

• Where do real world service distributions sit with respect to these attributes?

37

{G|C2} ≡ positive distributions with mean 1 and SCV C2

H2

• The H2 class has three degrees of freedom (s, p, ps)

• Can vary E[S3] while holding first two moments constant

38

Look at the moments of H2 … • Load due to big jobs vanishes as E[S3] increases

• When k>1, a big job does not block small jobs

• This reduces the effect of variability (C2) as third moment increases

39

Observations

• < 1-1/k UB/LB (C2+1)– No 2-moment approximation can be accurate in this

case

• [Kiefer Wolfowitz] [Scheller-Wolf]: When > 1-1/k, E[Delay] is finite iff C2 is finite.– Matches with the conjectured lower bound– Also popular as the “0 spare server” case

40

Proof outline: Upper bound

• THEOREM:

• PROOF: Consider the following service distribution

• Intuition for conjecture: k>1 should mitigate the effect of variability; D* exposes it completely

• Note: D* has the smallest third moment in {G|C2}

41

Proof outline: Lower bound

• THEOREM:– < 1-1/k

– 1-1/k

• PROOF: Consider the following sequence of service distributions in {G|C2} as 0

42

What about higher moments?

{G|C2}

H2

H*3

H*3

• H*3 allows control over fourth

moment while holding first three moments fixed

• The fourth moment is minimized when p0=0 (H2 distribution)