8/3/2019 Queueings

1/50

1

Queueing Theory

Frank Y. S. Lin

Information Management Dept.

National Taiwan University

yslin@im.ntu.edu.tw

8/3/2019 Queueings

2/50

2

References

Leonard Kleinrock, Queueing SystemsVolume I: Theory, New York: Wiley, 1975-1976.

D. Gross and C. M. Harris, Fundamentals of

Queueing Theory, New York: Wiley, 1998.

8/3/2019 Queueings

3/50

3

Agenda

Introduction Stochastic Process

General Concepts

M/M/1 Model

M/M/1/KModel

Discouraged Arrivals M/M/ andM/M/m Models

M/M/m/m Model

8/3/2019 Queueings

4/50

4

Introduction

8/3/2019 Queueings

5/50

5

Queueing System

A queueing system can be described as customersarriving for service, waiting for service if it is not

immediate, and if having waited forservice, leaving

the system after being served.

8/3/2019 Queueings

6/50

6

Why Queueing Theory

Performance Measurement Average waiting time of customer / distribution of

waiting time.

Average number of customers in the system /distribution of queue length / current work backlog.

Measurement of the idle time of server / length of an

idle period.

Measurement of the busy time of server / length of a

busy period.

System utilization.

8/3/2019 Queueings

7/50

7

Why Queueing Theory (contd)

Delay AnalysisNetwork Delay =

Queueing Delay

+ Propagation Delay (depends on the distance)+ Node Delay Processing Delay

(independent of packet length,

e.g. header CRC check)Adapter Delay (constant)

8/3/2019 Queueings

8/50

8

Characteristics of Queueing Process

Arrival Pattern of Customers Probability distribution

Patient / impatient (balked) arrival

Stationary / nonstationary

Service Patterns

Probability distribution

State dependent / independent service

Stationary / nonstationary

8/3/2019 Queueings

9/50

9

Characteristics of Queueing Process(contd)

Queueing Disciplines First come, first served (FCFS)

Last come, first served (LCFS)

Random selection for service (RSS) Priority queue

Preemptive / nonpreemptive

System Capacity Finite / infinite waiting room.

8/3/2019 Queueings

10/50

10

Characteristics of Queueing Process(contd)

Number of Service Channels Single channel / multiple channels

Single queue / multiple queues

Stages of Service Single stage (e.g. hair-styling salon)

Multiple stages (e.g. manufacturing process)

Process recycling or feedback

8/3/2019 Queueings

11/50

11

Notation

A queueing process is described byA/B/X/Y/Z

8/3/2019 Queueings

12/50

12

Notation (contd)

For example,M/D/2//FCFS indicates aqueueing process with exponential inter-arrival

time, deterministic service times, two parallel

servers, infinite capacity, and first-come, first-

served queueing discipline.

YandZcan be omitted ifY= andZ= FCFS.

8/3/2019 Queueings

13/50

13

Stochastic Process

8/3/2019 Queueings

14/50

14

Stochastic Process

Stochastic process: any collection of randomvariables(t), t T, on a common probability

space where tis a subset of time.

Continuous / discrete time stochastic process Example:(t) denotes the temperature in the class on t

= 7:00, 8:00, 9:00, 10:00, (discrete time)

We can regard a stochastic process as a family ofrandom variables which are indexed by time.

For a random processX(t), the PDF is denoted by

FX(x;t) = P[X(t)

8/3/2019 Queueings

15/50

15

Some Classifications of StochasticProcess

Stationary Processes: independent of timeFX(x; t+) = FX(x; t)

Independent Processes: independent variables

FX(x; t) = FX1,,Xn(x1,, xn ; t1,,tn)

= FX1(x1; t1) FXn(xn; tn)

Markov Processes: the probability of the next state

depends only upon the current state and not upon anyprevious states.

P[X(tn+1) =xn+1 |X(tn) =xn, .,X(t1) =x1]

= P[X(tn+1) =xn+1 |X(tn) =xn]

8/3/2019 Queueings

16/50

16

Some Classifications of StochasticProcess (contd)

Birth-death Processes: state transitions take placebetween neighboring states only.

Random Walks: the next position the process occupies

is equal to the previous position plus a random variablewhose value is drawn independently from an arbitrary

distribution.

8/3/2019 Queueings

17/50

17

General Concepts

8/3/2019 Queueings

18/50

18

Continuous-time MemorylessProperty

IfX~Exp(), for any a,b > 0,PP[[X > a + b | X > aX > a + b | X > a]] = P= P[[X > bX > b]]

Proof:

P[X> a + b |X> a]

( ) ( )( ) ( ) ( )[ ]P X a b X a X a b X a

P X a> + >= > + >>

I

( )

( )

( )

( )

( )1(

1

)a b

x b

a

x

P X a b F a b ee

P X a F

P

e

X

a

b

+

> + += = = = = >

>

8/3/2019 Queueings

19/50

19

Global Balance Equation

Define Pi = P[system is in state i]Pij = P[get into statej right after leaving state i]

rate out of statej = rate into state j

0 0( ) ( )

j ij i ij

i ii j i j

P P P P

= =

=

8/3/2019 Queueings

20/50

20

General Balance Equation

Define S = a subset of the state space

0 0 0 0( ) ( ) ( ) ( )

j ij i ij

j i i j j s i s i s j s

P P P P

= = = =

=

S

j

rate in = rate out

8/3/2019 Queueings

21/50

21

General Equilibrium Solution

Notation: Pk= the probability that the system contains k

customers (in state k)

k= the arrival rate of customers when the system is in

state k. k= the service rate when the system is in state k.

0

1kk

P

==

8/3/2019 Queueings

22/50

22

General Equilibrium Solution (contd)

Consider state k:

k-1 k k+1

k-1

k k+1

k1 1k k k k P P + + =

rate in rate out=

1

1

kk k

k

P P

+

+

=

11

kk k

k

P P

=

.

.

.1

1 2 0

0 001 1 1

kk k i

kik k iP P P

= +

= =

#

8/3/2019 Queueings

23/50

23

General Equilibrium Solution (contd)

01k

kP

= =1

0

0 0 1

1k

i

k i i

P

= = +

=

0 1

0 0 1

1

1k

i

k i i

P

= =+

=

+ 0,k k

k

NP T

=

= =

1waiting time w T

=

8/3/2019 Queueings

24/50

24

Littles Result

= average number of customers in the system T= system time (service time + queueing time)

= arrival rate

N

N T=

Black box

N

System time T

Queueing time

Service time

8/3/2019 Queueings

25/50

25

M/M/1 Model

Single Server, Single Queue

(The Classical Queueing System)

8/3/2019 Queueings

26/50

26

M/M/1 Queue

Single server, single queue, infinite population:

Interarrival time distribution:

Service time distribution

Stability condition

8/3/2019 Queueings

27/50

27

M/M/1 Queue (contd)

System utilization

Define state Sn = n customers in the system(n-1 in the queue and 1 in service)

S0 = empty system

= P[system is busy], 1- P[system is idel]

= =

rate out

rate in

S

8/3/2019 Queueings

28/50

28

M/M/1 Queue (contd)

Definepn = P[n customers in the system](rate in = rate out)

Since

1n np p + =

1n n n p p p

+ = =

1

1 0

n

np p ++ =

0 1i

i p

= =0

0 1

n

i p

= =0

0 1

n

ip

= =0 1 , (1 )

n

np p = =

#

8/3/2019 Queueings

29/50

29

M/M/1 Queue (contd)

Average number of customers in the system(1 ) kN k = (1 ) kk =

(1 ) / kd d = (1 ) kd

d

=

1(1 )

1

d

d

=

1

=

1

N

=

#

8/3/2019 Queueings

30/50

30

M/M/1 Queue (contd)

Average system time

P[ kcustomers in the system]

(Littles Result)N

T

=

(1 ) (1 )1

ki k

i k

=

= = =

#

1/ 11

1

= = =

8/3/2019 Queueings

31/50

31

M/M/1/KModel

Single Server, Finite Storage

8/3/2019 Queueings

32/50

32

M/M/1/KModel

The system can hold at most a total ofKcustomers (including the customer in service)

k= ifk< K

0 ifk Kk=

8/3/2019 Queueings

33/50

33

M/M/1/KModel (contd)1

0 0

0

kk

k

iP P P k K

=

= = 0kP k K= >

1

0 11

1 /1 ( / ) 0

1 ( / )

Kk

Kk

P k K

+=

= + =

0 otherwise

8/3/2019 Queueings

34/50

34

Discouraged Arrivals

8/3/2019 Queueings

35/50

35

Discouraged Arrivals

Arrivals tend to get discouraged when more andmore people are present in the system.

1k

k

=

+k =

8/3/2019 Queueings

36/50

36

Discouraged Arrivals (contd)

( )

1

0 0

0

/( 1) 1/

!

kk

k

i

iP P P

k

=

+= =

( )0

1

1

11 /!

k

k

P e

k

=

= =

+ ( )/

!

k

kP ek

= N

=

8/3/2019 Queueings

37/50

37

Discouraged Arrivals (contd)

( / )

0 0

( / )

1 !k k

k k

P ek k

= == = +

( / )

01 ( , 1 )e P = = = Q

/

/

(1 )

NT

e

= =

8/3/2019 Queueings

38/50

38

M/M/ andM/M/m

M/M/ - Infinite Servers, Single Queue

(Responsive Servers)

M/M/m - Multiple Servers, Single Queue(The m-Server Case)

8/3/2019 Queueings

39/50

39

M/M/ Queue

k =

k k =

There is always a new server available for eacharriving customer.

8/3/2019 Queueings

40/50

40

M/M/ Queue (contd)1

/

00

( / )

( 1) !

kk

ki

P P ei k

== =+

N

=

1T

= (Littles Result)

8/3/2019 Queueings

41/50

41

M/M/m Queue

TheM/M/m queue AnM/M/m queue is shorthand for a single queue served

by multiple servers.

Suppose there are m servers waiting for a single line.

For each server, the waiting time for a queue is a

system with service rate and arrival rate/m.

TheM/M/1 analysis has been done, at risk conclusion:

delay =

throughput

1

/n

/n

n

= =

8/3/2019 Queueings

42/50

42

M/M/m Queue (contd)

k

=

k= k ifk m

m ifk> m

For k m

For k> m

0 01( )

2 !

k

k p p pk k

= =

0 0

1 1( ) ( )

2 !

k k n

k p p p

n n n n

= =

8/3/2019 Queueings

43/50

43

M/M/n Queue (contd)

P[queueing] =

Total system time =

01i

ip

= =0 1

0

1

( ) ( ) 1

! ! (1 )

k nn

i

k

p wherenp np n

p

k n

=

= =

+

k

k m

p

=

02

1 (/ )

( 1)!( )

n

pn n

+

8/3/2019 Queueings

44/50

44

Comparisons (contd)

M/M/1 v.sM/M/4If we have 4M/M/1 systems: 4 parallel communication

links that can each handle 50 pps (), arrival rate = 25

pps per queue.

average delay = 40 ms.

Whereas for anM/M/4 system,

average delay = 21.7 ms.

8/3/2019 Queueings

45/50

45

Comparisons (contd)

Fast Server v.s A Set of Slow Servers #1If we have anM/M/4 system with service rate=50 pps

for each server, and anotherM/M/1 system with service

rate 4 = 200 pps. Both arrival rate is = 100 pps

delay forM/M/4 = 21.7 ms

delay forM/M/1 = 10 ms

8/3/2019 Queueings

46/50

46

Comparisons (contd)

Fast Server v.s A Set of Slow Servers #2If we have n M/M/1 system with service rate pps for

each server, and anotherM/M/1 system with service rate

n pps. Both arrival rate is n pps

n n

1

1/

1T

=

S1 S2

2

1/ 1/

11

n n

T n

n

= =

1

2

T

T n =

8/3/2019 Queueings

47/50

47

M/M/m/m

Multiple Servers, No Storage

(m-Server Loss Systems)

8/3/2019 Queueings

48/50

48

M/M/m/m

There are available m servers, each newly arrivingcustomers is given a server, if a customers arrives

when all servers are occupied, that customer is lost

e.g. telephony system.

k =

kk =

if k m