Introduction to Queuing Theory

1

2

Queueing SystemsQueue

a line of waiting customers who require service from one or more service providers.

Queueing systemwaiting room +

customers + service provider?

ArrivalsCustomers

Queue Server(s) Departures

3

Queuing ModelsWidely used to estimate desired performance

measures of the systemProvide rough estimate of a performance measureTypical measures

Server utilizationLength of waiting linesDelays of customers

Applications o Determine the minimum number of servers needed

at a service centero Detection of performance bottleneck or congestiono Evaluate alternative system designs

Introduction

4

Examples are:– waiting to pay in the supermarket– waiting at the telephone for information– planes circle before they can land

Example questions:– what is the average waiting time of a customer?– how many customers are waiting on average?– how long is the average service time?– what is the chance that one of the servers has

nothing to do?

5

5. Customer Population

Queue

3. Number of Servers

4. Server Capacity=# of seats for customer waiting for service +# of servers

1. Arrival Process

2. Service time distribution

Queueing Notation

Parameters of a queuing system

6

The behavior of a queuing system is dependent on:

1. Arrival process (l and distribution of interarrival times)

2. Service process (m and distribution of service times)

3. Number of servers4. Capacity of the system5. Size of the population for this system

Kendall notationA/S/m/B/K/SDA: arrival processS: service time distributionm: number of serversB: number of buffers (system capacity)K: population sizeSD: service discipline

7

8

Number of ServersNumber of servers available

Single Server Queue

Multiple Server Queue

9

Service DisciplinesFirst-come-first-served(FCFS)Last-come-first-served(LCFS)Shortest processing time first(SPT)Shortest remaining processing time first(SRPT)Shortest expected processing time first(SEPT)Shortest expected remaining processing time

first(SERPT)Biggest-in-first-served(BIFS)Loudest-voice-first-served(LVFS)

10

Common DistributionsM : ExponentialEk : Erlang with parameter kHk : Hyperexponential with parameter

k(mixture of k exponentials)D : Deterministic(constant)G : General(all)

f(t)1/m

tm

f(t)

tm

sExponential

General

ExampleM/M/3/20/1500/FCFSTime between successive arrival is

exponentially distributed.Service times are exponentially

distributed.Three servers.20 buffers = 3 service + 17 waitingAfter 20, all arriving jobs are lost.Total of 1500 jobs that can be serviced.Service disciple is fist-come-first served.

11

DefaultsInfinite buffer capacityInfinite population sizeFCFS service disciple=> The first three of the six parameter are

sufficientG/G/1=G/G/1/∞/∞/FCFSAssume only individual arrivals (no bulk

arrivals)

12

Arrival processArrival times: Interarrival times: tj form a sequence of Independent and

Identically Distributed (IID) random variablesThe most common arrival process: Poisson

arrivalsInter-arrival times are exponential + IID

Poisson arrivals

13

Service time distributionAmount of time each customer spends

at the serverAgain, usually assume IIDMost commonly used distribution is the

exponential distribution. Distribution: M, E, H, or G

14

Key Variables

t = Inter-arrival time = time between two successive arrivals

l = Mean arrival rate = 1/E[t]May be a function of the state of the system, e.g., number of jobs already in the system

s = Service time per jobm = Mean service rate per server = 1/E[s]Total service rate for m servers is mm

Service

ArrivalRate (l

Average Number

in Queue (Nq )

Average Waitin Queue (w )

Rate (m Departure

15

Key Variables (cont’d)nq = Number of jobs waitingns = Number of jobs receiving servicen = Number of jobs in the system.

This is also called queue length Note: Queue length includes jobs currently

receiving service as well as those waiting in the queue

r = Response time or the time in the system = time waiting + time receiving service

w = Waiting time = Time between arrival and beginning of service

16

Key Variables1

2

m

PreviousArrival Arrival

BeginService

EndService

t w sr

nnq ns

l m

Time

Note that all of these variables are random variablesexcept for l and m .

17

18

1. Stability ConditionIf the number of jobs becomes infinite, the

system is unstable. For stability, the mean arrival rate less than the mean service rate.

l < mmDoes not apply to finite buffer system or the

finite population systems They are always stable. Finite population: queue length is always finite. Finite buffer system: arrivals are lost when the

number of jobs in the system exceed the system capacity.

Rules for All Queues

Rules for All Queues2. Number in System versus Number in

Queue: n = nq+ ns

Notice that n, nq, and ns are random variables.

E[n]=E[nq]+E[ns]

If the service rate is independent of the number in the

queue, Cov(nq,ns) = 0

19

Rules for All Queues3. Number versus Time:

If jobs are not lost due to insufficient buffers, Mean number of jobs in the system

= Arrival rate Mean response time4. Similarly,

Mean number of jobs in the queue = Arrival rate Mean waiting time

This is known as Little's law.

20

Rules for All Queues5. Time in System versus Time in Queue

r = w + sr, w, and s are random variables

E[r] = E[w] + E[s]

If the service rate is independent of the number

of jobs in the queue, Cov(w,s)=0

21

Little's LawMean number in the system

= Arrival rate Mean response time This relationship applies to all systems or parts of

systems in which the number of jobs entering the system is equal to those completing service

Based on a black-box view of the system:

In systems in which some jobs are lost due to finite buffers, the law can be applied to the part of the system consisting of the waiting and serving positions

BlackBox

Arrivals Departures

22

Application of Little's Law

Applying to just the waiting facility of a service center Mean number in the queue = Arrival rate Mean

waiting timeSimilarly, for those currently receiving the service,

we have: Mean number in service = Arrival rate Mean service

time

Arrivals Departures

23

ExampleA monitor on a disk server showed that the

average time to satisfy an I/O request was 100 milliseconds. The I/O rate was about 100 requests per second. What was the mean number of requests at the disk server?Using Little's law:

Mean number in the disk server = Arrival rate Response time = 100 (requests/second) (0.1 seconds) = 10 requests

24

Stochastic Processes

25

A stochastic process is a sequence X1,X2, . . . of random variables indexed by the parameter t, such as time.

For example, number of jobs at CPU of computer system at time t is a random variable X(t)

Time and state can be discrete or continuous.A queuing system is characterized by three

elements: A stochastic input process A stochastic service mechanism or process A queuing discipline

26

PoissonProcesses

Birth-deathProcesses

MarkovProcesses

Types of Stochastic Processes

27

Markov ProcessIf future states depend only on the present and

are independent of the past (memoryless) then called markov process

A discrete state Markov Process is a Markov chain

M/M/m queues can be modeled using Markov process

The time spent by a job in such a queue is a Markov process and the number of jobs in the queue is a Markov chain

Types of Stochastic Processes

28

Birth-Death ProcessThe discrete space Markov processes in which

the transitions are restricted to neighboring states

Process in state n can change only to state n+1 or n-1

ExampleThe number of jobs in a queue with a single

server and individual arrivals(not bulk arrivals)

Types of Stochastic Processes

0 1 2 j-1 j j+1…l0 l1 l2 lj-1 lj lj+1

m1 m2 m3 mj mj+1 m j+2

Analysis of A Single Queue

M/M/1 QueueM/M/1 queue is the most commonly used type of

queue Used to model single processor systems or to model

individual devices in a computer system Assumes that the interarrival times and the service

times are exponentially distributed and there is only one server

No buffer or population size limitations and the service discipline is FCFS

Need to know only the mean arrival rate l and the mean service rate m

State = number of jobs in the system 0 1 2 J-1 J J+1…

l l l l l l

m m m m m m30

Results for M/M/1 QueueBirth-death processes with

Probability of n jobs in the system:

31