1

Chapter 07

QUEUEING THEORY

1

Goals for Today

Understand the Queuing Model and its

applications

Understand how to describe a Queue Model

Lean the most important queuing models (Part 02)

Single Queue

Multiple Queues

Multiple Servers

2

Course Outline

The Queuing Model and Definitions

Application of Queuing Theory

Little’s Law

Queuing System Notation

Stationary Analysis of Elementary Queueing Systems

M/M/1

M/M/m

M/M/1/K

…

The Queuing Model

Use Queuing models to

Describe the behavior of queuing systems

Evaluate system performance

A Queue System is characterized by

Queue (Buffer): with a finite or infinite size

The state of the system is described by the Queue Size

Server: with a given processing speed

Events: Arrival (birth) or Departure (death) with given rates

Queue Server

Queuing System

Click for Queue Simulator

3

Queuing theory definitions 5

(Bose) “the basic phenomenon of queueing arises whenever a shared facility

needs to be accessed for service by a large number of jobs or customers.”

(Wolff) “The primary tool for studying these problems [of congestions] is known

as queueing theory.”

(Kleinrock) “We study the phenomena of standing, waiting, and serving, and we

call this study Queueing Theory." "Any system in which arrivals place demands

upon a finite capacity resource may be termed a queueing system.”

(Mathworld) “The study of the waiting times, lengths, and other properties of

queues.”

http://www2.uwindsor.ca/~hlynka/queue.html

6 Applications of Queuing Theory

6

4

Applications of Queuing Theory 7

Telecommunications

Computer Networks

Predicting computer performance

Health services (eg. control of hospital bed assignments)

Airport traffic, airline ticket sales

Layout of manufacturing systems.

Example application of queuing theory 8

In many stores and banks, we can find:

multiple line/multiple checkout system → a queuing

system where customers wait for the next available

cashier

We can prove using queuing theory that : throughput

improves/increases when queues are used instead of

separate lines

http://www.andrews.edu/~calkins/math/webtexts/prod10.htm#PD

5

Example application of queuing theory 9

http://www.bsbpa.umkc.edu/classes/ashley/Chaptr14/sld006.htm

Queuing theory for studying networks

10

View network as collections of queues

FIFO data-structures

Queuing theory provides probabilistic analysis of these queues

Examples:

Average length

Average waiting time

Probability queue is at a certain length

Probability a packet will be lost

6

QNAP/Modline

Example of a Queue Simulator 11

12 The Little’s Law

12

The long-term average number of customers in a stable system N, is equal to the long-term

average arrival rate, λ, multiplied by the long-term average time a customer spends in the system, T.

7

The Queuing Times

Queue Server

Queuing System

Queuing Time Service Time

Response Time (or Delay)

Little’s Law

a (t): the process that counts the number of arrivals up to time t.

d (t): the process that counts the number of departures up to time t.

N(t)= a(t)- d(t)

N(t)

a(t)

Time t

Area J(t)

Average arrival rate (up to t) λt= a(t)/t

Average time each customer spends in the system (Response Time) Tt= J(t)/a(t)

Average number in the system Nt= J(t)/t

d(t)

# A

rriv

als

8

Little’s Law

t t tN T

Taking the limit as t goes to infinity

E EN T

Expected number of

customers in the system

Expected time in the system

Arrival rate IN the system

N(t)

a(t)

Time t

Area J(t)

d(t)

Generality of Little’s Law

Little’s Law is a pretty general result

It does not depend on the arrival process distribution

It does not depend on the service process distribution

It does not depend on the number of servers and buffers in the system.

Applies to any system in equilibrium, as long as nothing in black box is

creating or destroying tasks

E EN T

Queueing

Network

λ

Aggregate

Arrival rate

Mean number tasks in system = mean arrival rate x mean response time

9

17 Specification of Queuing Systems

17

Characteristics of queuing systems 18

Arrival Process

The distribution that determines how the tasks arrives in

the system.

Service Process

The distribution that determines the task processing time

Number of Servers

Total number of servers available to process the tasks

10

Specification of Queueing Systems

Arrival/Departure

Customer arrival and service stochastic models

Structural Parameters

Number of servers: What is the number of servers?

Storage capacity: are buffer finite or infinite?

Operating policies

Customer class differentiation

are all customers treated the same or do some have priority over others?

Scheduling/Queueing policies

which customer is served next

Admission policies

which/when customers are admitted

Kendall Notation A/B/m(/K/N/X) 20

To specify a queue, we use the Kendall Notation.

The First three parameters are typically used,

unless specified

1. A: Arrival Distribution

2. B: Service Distribution

3. m: Number of servers

4. K: Storage Capacity (infinite if not specified)

5. N: Population Size (infinite)

6. X: Service Discipline (FCFS/FIFO)

http://en.wikipedia.org/wiki/Kendall's_notation

11

Kendall Notation of Queueing System

A/B/m/K/N/X

Arrival Process

• M: Markovian

• D: Deterministic

• Er: Erlang

• G: General

Service Process

• M: Markovian

• D: Deterministic

• Er: Erlang

• G: General

Number of servers

m=1,2,…

Storage Capacity

K= 1,2,…

(if ∞ then it is omitted)

Number of customers

N= 1,2,…

(for closed networks, otherwise

it is omitted)

Service Discipline

FIFO, LIFO, Round Robin, …

Distributions

CS352 Fall,2005

22

M: stands for "Markovian", implying exponential

distribution for service times or inter-arrival times.

D: Deterministic (e.g. fixed constant)

Ek: Erlang with parameter k http://en.wikipedia.org/wiki/Erlang_distribution

Hk: Hyper-exponential with parameter k

G: General (anything)

12

Kendall Notation Examples 23

M/M/1 Queue

Poisson arrivals (exponential inter-arrival), and exponential

service, 1 server, infinite capacity and population, FCFS (FIFO)

the simplest ‘realistic’ queue

M/M/m Queue

Same, but m servers

M/D/1 Queue

Poisson arrivals and CONSTANT service times, 1 server, infinite

capacity and population, FIFO.

G/G/3/20/1500/SPF

General arrival and service distributions, 3 servers, 17 queues

(20-3), 1500 total jobs, Shortest Packet First

24 Performance Measures

24

13

Performance Measures of Interest

We are interested in steady state behavior

Even though it is possible to pursue transient results, it is a significantly more difficult task.

E[S]: average system (response) time (average time spent in the system)

E[W]: average waiting time (average time spent waiting in queue(s))

E[X]: average queue length

E[U]: average utilization (fraction of time that the resources are being used)

E[R]: average throughput (rate that customers leave the system)

E[L]: average customer loss (rate that customers are lost or probability that a customer is lost)

Recall the Birth-Death Chain Example

At steady state, we obtain

λ0

0 1 μ1

λ1

2 μ2

λj-2

j-1 μj-1

λj-1

j μj

μ3

λ2 λj

μj+

1

0 0 1 1 0 0

1 0

1

In general

1 1 1 1 0j jj j j j j

0

1 0

1 1

...

...

j

j

j

Making the sum equal to 1

0 1

0

1 1

...1 1

...

j

j j

Solution exists if

0 1

1 1

...1

...

j

j j

S

14

27 End of Part 01

27