1

Queueing Theory

Frank Y. S. LinInformation Management Dept.

National Taiwan University

yslin@im.ntu.edu.tw

2

References Leonard Kleinrock, “Queueing Systems

Volume I: Theory”, New York: Wiley, 1975-1976.

D. Gross and C. M. Harris, “Fundamentals of Queueing Theory”, New York: Wiley, 1998.

3

Agenda Introduction Stochastic Process General Concepts M/M/1 Model M/M/1/K Model Discouraged Arrivals M/M/∞ and M/M/m Models M/M/m/m Model

4

Introduction

5

Queueing System A queueing system can be described as customers

arriving for service, waiting for service if it is not immediate, and if having waited for service, leaving the system after being served.

6

Why Queueing Theory

Performance Measurement Average waiting time of customer / distribution of

waiting time. Average number of customers in the system /

distribution of queue length / current work backlog. Measurement of the idle time of server / length of an

idle period. Measurement of the busy time of server / length of a

busy period. System utilization.

7

Why Queueing Theory (cont’d) Delay Analysis Network Delay =

Queueing Delay + Propagation Delay (depends on the distance) + Node Delay Processing Delay

(independent of packet length, e.g. header CRC check)

Adapter Delay (constant)

8

Characteristics of Queueing Process

Arrival Pattern of Customers Probability distribution Patient / impatient (balked) arrival Stationary / nonstationary

Service Patterns Probability distribution State dependent / independent service Stationary / nonstationary

9

Characteristics of Queueing Process (cont’d)

Queueing Disciplines First come, first served (FCFS) Last come, first served (LCFS) Random selection for service (RSS) Priority queue Preemptive / nonpreemptive

System Capacity Finite / infinite waiting room.

10

Characteristics of Queueing Process (cont’d)

Number of Service Channels Single channel / multiple channels Single queue / multiple queues

Stages of Service Single stage (e.g. hair-styling salon) Multiple stages (e.g. manufacturing process) Process recycling or feedback

11

Notation A queueing process is described by A/B/X/Y/Z

12

Notation (cont’d) For example, M/D/2/∞/FCFS indicates a

queueing process with exponential inter-arrival time, deterministic service times, two parallel servers, infinite capacity, and first-come, first-served queueing discipline.

Y and Z can be omitted if Y = ∞ and Z = FCFS.

13

Stochastic Process

14

Stochastic Process Stochastic process: any collection of random

variables Χ(t), t T, on a common probability space where t is a subset of time.

Continuous / discrete time stochastic process Example: Χ(t) denotes the temperature in the class on t

= 7:00, 8:00, 9:00, 10:00, … (discrete time) We can regard a stochastic process as a family of

random variables which are “indexed” by time. For a random process X(t), the PDF is denoted by

FX(x;t) = P[X(t) <= x]

15

Some Classifications of Stochastic Process

Stationary Processes: independent of time

FX (x; t + τ) = FX (x; t)

Independent Processes: independent variables

FX (x; t) = FX1,…, Xn(x1,…, xn ; t1,…,tn)

= FX1(x1; t1) …FXn(xn; tn)

Markov Processes: the probability of the next state depends only upon the current state and not upon any previous states.

P[X(tn+1) = xn+1 | X(tn) = xn, …., X(t1) = x1]

= P[X(tn+1) = xn+1 | X(tn) = xn]

16

Some Classifications of Stochastic Process (cont’d)

Birth-death Processes: state transitions take place

between neighboring states only. Random Walks: the next position the process occupies

is equal to the previous position plus a random variable whose value is drawn independently from an arbitrary distribution.

17

General Concepts

18

Continuous-time Memoryless Property

If X ~Exp(λ), for any a,b > 0,

PP[[X > a + b | X > aX > a + b | X > a] ] = P= P[[X > bX > b]]

Proof:

P[X > a + b | X > a]

[ ]P X a b X a

X a b X aP X a

( )1(

1)

a bx b

ax

P X a b F a b ee

P X a FP

eX

ab

19

Global Balance Equation Define Pi = P[system is in state i]

Pij = P[get into state j right after leaving state i]

rate out of state j = rate into state j

0 0( ) ( )

j ij i iji ii j i j

P P P P

20

General Balance Equation Define S = a subset of the state space

0 0 0 0( ) ( ) ( ) ( )

j ij i ijj i i jj s i s i s j s

P P P P

S

j

rate in = rate out

21

General Equilibrium Solution Notation:

Pk = the probability that the system contains k customer

s (in state k)

λk= the arrival rate of customers when the system is in state k.

μk= the service rate when the system is in state k.

0

1kk

P

22

General Equilibrium Solution (cont’d)

Consider state k:

k-1 k k+1

λk-1

μk μk+1

λk

1 1k k k kP P

rate in rate out

11

kk k

k

P P

11

kk k

k

P P

.

.

.

11 2 0

0 001 1 1

kk k i

kik k i

P P P

#

23

General Equilibrium Solution (cont’d)

0

1kk

P

1

00 0 1

1k

i

k i i

P

0 1

0 0 1

1

1k

i

k i i

P

0

,k kk

NP T

1waiting time w T

24

Little’s Result = average number of customers in the system T = system time (service time + queueing time) λ= arrival rate

N

N T

Black box

N

System time T

…

Queueing time

Service time

25

M/M/1 Model

Single Server, Single Queue

(The Classical Queueing System)

26

M/M/1 Queue Single server, single queue, infinite population:

Interarrival time distribution:

Service time distribution

Stability condition λ < μλ < μ

( ) tp t e

00

0 0( ) 1

t ttp t t e dt e

k

k

27

M/M/1 Queue (cont’d) System utilization

Define state Sn = n customers in the system (n-1 in the queue and 1 in service)

S0 = empty system

= P[system is busy], 1- P[system is idel]

rate out

rate in

S

28

M/M/1 Queue (cont’d) Define pn = P[n customers in the system]

(rate in = rate out)

Since

1n np p

1n n np p p

11 0

nnp p

0

1ii

p

00

1n

i

p

00

1n

i

p

0 1 , (1 )nnp p

#

29

M/M/1 Queue (cont’d) Average number of customers in the system

(1 ) kN k (1 ) kk (1 ) /kd d (1 ) kd

d

1(1 )

1

d

d

1

1N

#

30

M/M/1 Queue (cont’d) Average system time

P[ ≧ k customers in the system]

(Little’s Result)

NT

(1 ) (1 )1

ki k

i k

#

1/ 111

31

M/M/1/K Model

Single Server, Finite Storage

32

M/M/1/K Model The system can hold at most a total of K

customers (including the customer in service)λk = λ if k < K

0 if k K

μk = μ

33

M/M/1/K Model (cont’d)1

0 00

kk

ki

P P P k K

0kP k K

1

0 11

1 /1 ( / ) 0

1 ( / )

Kk

Kk

P k K

0 otherwise

34

Discouraged Arrivals

35

Discouraged Arrivals Arrivals tend to get discouraged when more and

more people are present in the system.

1k k

k

36

Discouraged Arrivals (cont’d)

1

0 00

/( 1) 1/

!

kk

ki

iP P P

k

0

1

11

1 /!

k

k

P e

k

/

!

k

kP ek

N

37

Discouraged Arrivals (cont’d)

( / )

0 0

( / )

1 !k kk k

P ek k

( / )01 ( , 1 )e P

/

/

(1 )

NT

e

38

M/M/∞ and M/M/m

M/M/∞ - Infinite Servers, Single Queue

(Responsive Servers)

M/M/m - Multiple Servers, Single Queue

(The m-Server Case)

39

M/M/∞ Queue

k

k k

There is always a new server available for each arriving customer.

40

M/M/∞ Queue (cont’d)1

/0

0

( / )

( 1) !

kk

ki

P P ei k

N

1T

(Little’s Result)

41

M/M/m Queue The M/M/m queue

An M/M/m queue is shorthand for a single queue served by multiple servers.

Suppose there are m servers waiting for a single line. For each server, the waiting time for a queue is a system with service rateμ and arrival rate λ/m.

The M/M/1 analysis has been done, at risk conclusion:

delay =

throughput

1

/ n

/ n

n

42

M/M/m Queue (cont’d)

λk = λ

μk = kμ if k m

mμ if k > m

For k m

For k > m

0 0

1( )

2 !k

kp p pk k

0 0

1 1( ) ( )

2 !k k n

kp p pn n n n

43

M/M/n Queue (cont’d)

∴

P[queueing] =

Total system time =

0

1ii

p

0 1

0

1

( ) ( ) 1! ! (1 )

k nn

ik

p wherenp np n

pk n

kk m

p

02

1 (/ )

( 1)!( )

n

pn n

44

Comparisons (cont’d) M/M/1 v.s M/M/4

If we have 4 M/M/1 systems: 4 parallel communication links that can each handle 50 pps (μ), arrival rate λ = 25 pps per queue.

average delay = 40 ms.

Whereas for an M/M/4 system,

average delay = 21.7 ms.

45

Comparisons (cont’d) Fast Server v.s A Set of Slow Servers #1

If we have an M/M/4 system with service rate μ=50 pps for each server, and another M/M/1 system with service rate 4μ = 200 pps. Both arrival rate is λ = 100 pps

delay for M/M/4 = 21.7 ms

delay for M/M/1 = 10 ms

46

Comparisons (cont’d) Fast Server v.s A Set of Slow Servers #2

If we have n M/M/1 system with service rate μ pps for each server, and another M/M/1 system with service rate nμ pps. Both arrival rate is nλ pps

μλ

μλ

μλ

…

nλ nμ

1

1/

1T

S1 S2

2

1/ 1/

11

n nT

nn

12

TT

n

47

M/M/m/m

Multiple Servers, No Storage

(m-Server Loss Systems)

48

M/M/m/m There are available m servers, each newly arriving

customers is given a server, if a customers arrives when all servers are occupied, that customer is lost

e.g. telephony system.

k

k k

if k m 0 if k m

49

M/M/m/m (cont’d)

kP 0

1( / ) if k m

!kP

k

0 if k m

1

00

1( / )

!k

k

Pk

50

M/M/m/m (cont’d) Let pm describes the fraction of time that all m serv

ers are busy. The name given to this probability expression is Erlang’s loss formula and is given by

This equation is also referred to as Erlang’s B formula and is commonly denoted by B(m,λ/μ)

http://www.erlang.com

0

( / ) / !

( / ) / !

m

m mk

k

mp

k