Introduction to Queuing Theory

2

Queuing theory definitions

(Kleinrock) “We study the phenomena of standing, waiting, and serving, and we call this study Queueing Theory."

(Mathworld) “The study of the waiting times, lengths, and other properties of queues.”

http://www2.uwindsor.ca/~hlynka/queue.html

3

Queuing theory for studying networks View network as collections of queues

FIFO data-structures Queuing theory provides probabilistic

analysis of these queues Examples:

Average length Average waiting time Probability queue is at a certain length Probability a packet will be lost

4

Model Queuing System

Server System Queuing System

Queue Server

Queuing System

Use Queuing models to Describe the behavior of queuing systems Evaluate system performance

5

Characteristics of queuing systems Arrival Process

The distribution that determines how the tasks arrives in the system.

Service Process The distribution that determines the task

processing time Number of Servers

Total number of servers available to process the tasks

6

Kendall Notation 1/2/3(/4/5/6)

Six parameters in shorthand• First three typically used, unless specified

1. Arrival Distribution2. Service Distribution3. Number of servers 4. Total Capacity (infinite if not specified) 5. Population Size (infinite) 6. Service Discipline (FCFS/FIFO)

7

Arrival time distributions

M: stands for "Markovian“, Poisson arrivials

D: Deterministic (e.g. fixed constant) G: General (anything)

8

Kendall Notation Examples

M/M/1: Poisson arrivals and exponential service, 1 server,

infinite capacity and population, FCFS (FIFO) the simplest ‘realistic’ queue

M/M/m Same, but M servers

G/G/3/20/1500/SPF General arrival and service distributions, 3 servers,

17 queue slots (20-3), 1500 total jobs, Shortest Packet First

9

Analysis of M/M/1 queue

Given: • : Arrival rate of jobs (packets on input link) • : Service rate of the server (output link)

Solve: L: average number in queuing system Lq average number in the queue W: average waiting time in whole system Wq average waiting time in the queue

10

M/M/1 queue model

Wq

W

L

Lq

11

Little’s Law

Little’s Law: Average number of jobs in system = average arrival rate of jobs × average service time Observed before, Little was first to prove

Applies to any system in equilibrium, as long as nothing in black box is creating or destroying jobs

Arrivals Departures

System

12

Formally, Little’s Law is

L = W

: Average arrival rate W: Average time job is in the system (average service

time) L: Average number of jobs in the system

Average number of jobs in system = average arrival rate × average service time for each job

13

Proof of Little’s Law:Let J be the sum of all service time, that is, the sum of time spent

on each job. E.g., J = j1+j2+j3…., where j1 is the service time of job 1, and so on.

Let N be the number of jobs (packets).Let T be the total service time of the system on the jobs. Note that

T ≤ J. We know is the average arrival rate, W is the average service

time of jobs, and L is the average number of jobs in the system.

On the right-hand side, by definitions, in a stable system we have

W)( WTN )( ))(( N

JTN T

JBy definition, J is the sum of service time for all jobs, T is the total service time of

the system. Thus, J/T is the average number of jobs in the system, which is L. Therefore, the Little’s Law holds.

14

An example of J

J = Shaded area = 9

Same in all cases!

1 2 3 4 5 6 7 8

Packet #

Time

123

1 2 3 4 5 6 7 8

# in System

123

Time

1 2 3

Time inSystem

Packet #

123

Arrivals

Departures

15

M/M/1 queue model

Wq

W

L

Lq

L=λWLq=λWq

• : Arrival rate of jobs (packets on input link)

• : Service rate of the server (output link)

L: average number in queuing system

Lq average number in the queue

W: average waiting time in whole system

Wq average waiting time in the queue

16

Poisson process

Used to model random events in time that are largely independent of one another

E.g., a event may be that a customer arrives at the ice cream shop

E.g., or you see a roadkill when driving E.g., or a packet arrives at the router

17

Poisson Process

For a poisson process with average arrival rate , the probability of seeing n arrivals in time interval Δt

0...)2Pr(

)1Pr()(...]!2

)(1[)1Pr(

1)0Pr()(1...!2

)(1)0Pr(

)(!

)()Pr(

2

2

ttott

ttte

ttott

te

tnEn

ten

t

t

nt

18

Poisson process & exponential distribution Inter-arrival time t (time between

arrivals) in a Poisson process follows exponential distribution with parameter

1)(

)Pr(

tE

et t

19

M/M/1 queue model

1

Wq

W

L

Lq

L=λWLq=λWq

W = Wq + (1/μ)

• : Arrival rate of jobs (packets on input link)

• : Service rate of the server (output link)

L: average number in queuing system

Lq average number in the queue

W: average waiting time in whole system

Wq average waiting time in the queue

20

Solving queuing systems

4 unknowns: L, Lq W, Wq

Relationships: L=W Lq=Wq W = Wq + (1/)

If we know any 1, can find the others

0

n

nnPL

21

Analysis of M/M/1 queue

Goal: A closed form expression of the probability of the number of jobs in the queue (Pi) given only and

Online M/M/1 animation http://www.dcs.ed.ac.uk/home/jeh/Simjava/

queueing/mm1_q/mm1_q.html

22

Equilibrium conditions

10

11

10

)(

nnn PPP

PP

Pn is the probability that in equilibrium the system has n number of jobs

n+1nn-1

23

Solving for P0 and Pn at equilibrium Step 1

Step 2

0,0

2

201 , PPPPPPn

n

0

0

000

1,1,1

n

n

n

n

nn PPthenP

Verify this after class

24

Step 3

Step 4

, then

n

n0

n 1

1 n0

1

1 1

P0 1

nn0

1 and Pn n 1

Solving for P0 and Pn at equilibrium

Pn: probability of n jobs in the system

25

Solving for L

0

n

nnPL )1(0

n

nn )1(1

1

n

nn

(1 ) dd

11

0

)1(n

ndd

(1 ) 1(1 )2

)1(

26

Solving W, Wq and Lq

W L

1 1

Wq W 1

1

( )

Lq Wq ( )

2

( )

L

27

Response time vs. utilization (ρ)

1W

Waiting vs. Utilization

0

0.05

0.1

0.15

0.2

0.25

0 0.2 0.4 0.6 0.8 1 1.2

W(s

ec)

28

Stable Region

Waiting vs. Utilization

0

0.005

0.01

0.015

0.02

0.025

0 0.2 0.4 0.6 0.8 1

W(s

ec)

linear region

29

An example

On a network gateway, measurements show that the packets arrive at a mean rate of 125 packets per second (pps) and the gateway takes about 2 millisecs to forward them. Assuming an M/M/1 model.

Measurement of a network gateway: mean arrival rate (l): 125 Packets/s mean response time (m): 2 ms

Assuming exponential arrivals: What is the gateway’s utilization? What is the probability of n packets in the gateway? mean number of packets in the gateway?

30

Example cont’d Arrival rate λ = 125 pps Service rate μ = 1/0.002 = 500 pps Gateway utilization ρ = λ/μ = 0.25 Prob. of n packets in gateway = Pn =

Mean number of packets in gateway = L =

nn )25.0(75.0ρ)ρ1(

33.057.0

25.0

ρ1

ρ