Probability, Statistics, and Traffic Theoriesmonet.skku.edu/.../2019/08/W07.-Probability_Theory.pdf · 2019-10-17 · Mobile Computing Networking Laboratory 2/54 Contents Introduction

Copyright 2000-2019 Networking Laboratory

Sungkyunkwan University

Probability, Statistics, and Traffic Theories

Mobile Computing

Sungkyunkwan University

Hyunseung Choo

[email protected]

mailto:[email protected]

Mobile Computing Networking Laboratory 2/54

Contents

Introduction

Probability Theory and Statistics

Random variables

Probability mass function: pmf

Probability density function: pdf

Cumulative distribution function: cdf

Expected value, nth moment, nth central moment, and variance

Some important distributions

Summary of Random Distributions

Random number generation

Traffic Theory

Poisson arrival model, etc.

Basic Queuing Systems

Little’s law

Basic queuing models


Introduction

Several factors influence the performance of wireless

systems

Density of mobile users

Cell size

Moving direction and speed of users: Mobility models

Call rate, call duration

Interference, etc.

Probability theory, statistics, and traffic patterns help make

these factors tractable


Probability Theory and Statistics Theory

Random Variables (RVs)

Let S be a sample space associated with an experiment E

X is a function that associates a real number to each sS

RVs can be of two types: Discrete or Continuous

Discrete random variable: probability mass function (pmf)

Continuous random variable: probability density function (pdf)

s X(s)X

S R


Discrete Random Variables

X(s) contains a finite or infinite number of values

The possible values of X can be enumerated

e.g., Throw a 6-sided dice and calculate the probability

of a particular number appearing

1 2 3 4 6

0.1

0.3

0.1 0.1

0.2 0.2

5

Probability

RV



The probability mass function (pmf) p(k) of X is defined as:

p(k) = p(X = k), for k = 0, 1, 2, ...

where,

Probability of each state occurring

0 p(k) 1, for every k

Sum of all states

p(k) = 1, for all k


Continuous Random Variables

In this case, X contains an infinite number of values

Mathematically, X is a continuous random variable if there

is a function f, called probability density function (pdf) of X

that satisfies the following criteria:

f(x) 0, for all x

f(x) dx = 1


Cumulative Distribution Function

Applies to all random variables

A cumulative distribution function (cdf) is defined as:

For discrete random variables:

F(k) = P(X k) = P(X = k)

For continuous random variables:

F(k) = P(X k) = f(x) dx

all k

-

k


Probability Density Function

The pdf f(x) of a continuous random variable X is the

derivative of the cdf F(x), i.e.,

( )( )

dx

xdFxf =

?

)()( dxxfdXcP

d

c=

c dx

f(x) Probability is area under curve!


Expected Value, nth Moment,

nth Central Moment, and Variance


Expected value represented by E or average of random variable

E[X] = kP(X = k)

nth moment

E[Xn] = knP(X = k)

nth central moment

E[(X – E[X])n] = (k – E[X])nP(X = k)

Variance or the second central moment

2 = Var(X) = E[(X – E[X])2] = E[X2] - (E[X])2

all k

all k

all k




5.32.061.052.041.033.021.01)( =+++++=XE

1 2 3 4 6

0.1

0.3

0.1 0.1

0.2 0.2

5

Probability

X




Continuous Random Variables

Expected value or mean value

E[X] = x f(x) dx

nth moment

E[Xn] = xn f(x) dx

nth central moment

E[(X – E[X])n] = (x – E[X])n f(x) dx

Variance or the second central moment

2 = Var(X) = E[(X – E[X])2] = E[X2] - (E[X])2

-

+

-

+

-

+


Some Important

Discrete Random Distributions

Binomial distribution

Event : success(1) with p prob., fail(0) with (1-p)

r.v. : No. of successes out of n trials

Out of n die, exactly k die have the same value

probability pk

And (n-k) die have different values

probability (1-p)n-k

For any k die out of n

k)!k!(nn!

knndability, aucess probp is the s...,,,nn...,,,,k

where

,knp)(kpknk)P(X

−===

−−==

; 2 1 0 ; 2 1 0

,

1


Some Important


Coefficient of binomial expansion

Given a random variable X along with Binomial distribution. What is the

E[X] and Var[X] ?

nnnnnn yxyxn

yxn

yx 2)11(n

n

10)( 0110 =+=

++

+

=+ −


Some Important


Poisson distribution

Event : success at the rate of

r.v. : No. of successes in time T

As n is large and p is small, the Binomial probability can be approximated by

the Poisson probability function

The theorem by Simeon Denis Poisson(1781-1840)

P(X=x) = e-l lx / x! , where e =2.71828 and parameter l= np

Where the parameter l > 0. What is the mean and variance of the

Poisson distribution ?

l

pmf


Some Important


Geometric distribution

Event : success(1) with p prob., fail(0) with (1-p)

r.v. : No. of trials up to the first success

P(X = k) = p(1-p)k-1 where, p is success probability

Modified geometric distribution

r.v. : No. of trials before the first success

P(X=k) = p(1-p)k for k = 0, 1, 2,…..


Some Important


Given a random variable X along with Geometric distribution.

What is the E[X] and Var[X] ?


Some Important

Continuous Random Distributions

Uniform distribution

pdf

cdf

Given a random variable X along with Uniform distribution.

What is the E[X] and Var[X] ?

−=

otherwise ,0

for ,1

)(bxa

abxfX

−

−

=

bfor x ,1

for ,

for ,0

)( bxaab

ax

ax

xFX

ab −

1

0b

X

fX

a


Some Important


Normal distribution (or Gaussian distribution)

Describes Many Random Processes or Continuous Phenomena

Can Be Used to Approximate Discrete Probability Distributions

Example: Binomial

Basis for Classical Statistical Inference

pdf

f(x) = Frequency of Random Variable X

= Population Standard Deviation

= 3.14159; e = 2.71828

x = Value of Random Variable (- < X < )

= Population Mean

2

2

1

e 2

1)(

−

−

=

x

xf

X

f(x)


Some Important


Exponential distribution

Continuous case of geometric distribution

Event : success at the rate of

r.v. : time to the first success

X has an exponential distribution with parameter l > 0

pdf

cdf

Given a random variable X along with Exponential distribution. What is

the E[X] and Var[X] ?

( ), 0

0, otherwise

xe xf x

ll − =

( )0, 0

1 , 0x

xF x

e xl−

=

−

l


Some Important


Erlang Distribution

r sequential phases have independent identical exponential

distributions

,.....2,1,0,0,)!1(

)(

,....2,1,0,0 ,!

)(1)(

1

1

0

=−

=

=−=

−−

−

=

−

rtr

ettf

rtek

ttF

trr

r

k

tk

ll

ll

l

l


Some Important


Hypoexponential Distribution

Time in each sequential phase is independent and exponentially

distributed

2-stage : X˜HYPO

0 t,1)(

0 t),()(

21

21

12

1

12

2

12

21

−

−−

−=

−−

=

−−

−−

tt

tt

eetF

eetf

ll

ll

ll

l

ll

l

ll

ll

2121 ),,( llll


Some Important


Hyperexponential Distribution

A process consisting of alternate phases, while experiencing one

and only one of the independent exponentially distributed phases

=

−

==

−

−=

==

k

i

t

i

k

i

ii

k

i

i

t

ii

i

i

etF

tetf

1

11

)1()(

1,0,0,0 ,)(

l

l

ll


Summary of Random Distributions (1/4)

Modified Geometric Distribution

qz)-p/(1(z)F

p)/p(1σ

p)/p(1E[K]

p)p-(1f(k)

p)-(11pp)(1F(k)

p)p-(1 flip]1)th (kon headfirst theP[flip

event)between time(ie.

heads aobtain torequired flips additional ofnumber the: (K) Variable Random

*

2

k

k to0n

1kn

k

2

=

−=

−=

=

−=−=

=+

=

+



Binomial Distribution

N*

2

2

mmN

k to0m

kk-N

p))(1(pz(z)F

Np)pNp(1]E[K

p)Np(1σ

NpE[K]

pp)m)(1(N,F(k)

)k!k)!/((NN!k), where(N

pp)-k)(1(N, heads]P[k

flipscoin N e thos

in appearing successes)or (failures heads ofnumber the: (K) Variable Random

−+=

+−=

−=

=

−=

−=

=

−

=



Exponential Distribution

s)λ/(λ(s)F

1/λσ

2/λ]E[T

1/λE[T]

λef(t)

e1F(t)

(events) successes obetween tw time: (T) Variable Random

*

22

22

λt

λt

+=

=

=

=

=

−=

−

−



Poisson Distribution

1)λt(z*

2

λt-k

λt-n

k to0n

e(z)F

λtσ

λtE[X]

/k!eλt)(f(k)

/n!eλt)(F(k)

tin time (events) successes ofnumber the: (K) Variable Random

−

=

=

=

=

=

=


Bayes Theorem

A theorem concerning conditional probabilities of the form

P(X|Y) (read: the probability of X, given Y) is

where, P(X) and P(Y) are the unconditional probabilities of X and Y

respectively.

Theorem of total probability, if event Xi is mutually exclusive

and probability sum to 1

)(

)()|()|(

YP

XPXYPYXP =

=

=n

i

ii XPXYPYP1

)()|()(


Bayes’ Theorem: example

Consider a group of PGs who submit jobs to a system. For

any job, it may run or fail to run. If we know the prob. that a

job submitted by a particular PG will fail, and the relative

frequency with which each PG submits jobs, we can

calculate, using Bayes’ theorem, the prob. that a PG

submitted a particular job, given that the job failed.

Consider the case of 3 PGs 1, 2, and 3. Then define the

following events:

Event A1: job was submitted by PG 1



Event B: job failed


Bayes’ Theorem: example (con’t)

Given the following probabilities:

P[A1] = 0.2, P[A2] = 0.3, P[A3] = 0.5

P[B|A1] = 0.1, P[B|A2] = 0.7, P[B|A3] = 0.1

We can calculate the desired probability using Bayes’

theorem.

P[A1|B] = (P[A1]*P[B|A1] )/(SUMi P[Ai]*P[B|Ai] )

= (0.2 * 0.1)/(0.2 * 0.1 + 0.3 * 0.7 + 0.5 * 0.1)

= 0.07144


Central Limit Theorem

The Central Limit Theorem (CLT) states that

whenever a random sample (X1, X2,.. Xn) of size n is taken

from any distribution with expected value E[Xi] = and

variance Var[Xi] = 2, where i =1,2,..,n, then their arithmetic

mean is defined by

=

=n

i

in Xn

S1

1


Central Limit Theorem

The sample mean is approximated to a normal distribution

with

E[Sn] =

Var[Sn] = 2 / n

The larger the value of the sample size n, the better the

approximation to the normal

This is very useful when inference between signals needs

to be considered


Poisson Arrival Model

A Poisson process is a sequence of events “randomly

spaced in time”

For example, customers arriving at a bank and Geiger

counter clicks are similar to packets arriving at a buffer

The rate l of a Poisson process is the average number of

events per unit time (over a long time)


Properties of a Poisson Process

Properties of a Poisson process

For a time interval [0, t), the probability of n arrivals in t units of

time is

For two disjoint (non overlapping ) intervals (t1, t2) and (t3, t4), (i.e.,

t1 < t2 < t3 < t4), the number of arrivals in (t1, t2) is independent of

arrivals in (t3, t4)

tn

n en

ttP ll −=

!

)()(


Interarrival Times of Poisson Process

Interarrival times of a Poisson process

We pick an arbitrary starting point t0 in time.

Let T1 be the time until the next arrival.

We have P( T1 > t ) = P0(t) = e-lt

Thus the cumulative distribution function of T1 is given by

FT1(t) = P( T1≤ t ) = 1 - e -lt

The pdf of T1 is given by

fT1(t) = le-lt

Therefore, T1 has an exponential distribution with mean arrival rate

l



Similarly T2 is the time between first and second arrivals,

we define T3 as the time between the second and third

arrivals, T4 as the time between the third and fourth arrivals

and so on

The random variables T1, T2, T3… are called the interarrival

times of the Poisson process

T1, T2, T3,… are independent of each other and each has

the same exponential distribution with mean arrival rate l


Memoryless and Merging Properties

Memoryless property

A random variable X has the property that “the future is

independent of the past” i.e., the fact that it hasn’t happened yet,

tells us nothing about how much longer it will take before it does

happen

Merging property

If we merge n Poisson processes with distributions for the inter

arrival times

1 - where i = 1, 2, …, n

into one single process, then the result is a Poisson process for

which the inter arrival times have the distribution 1- e-lt with mean

l = l1 + l2 + … + ln

tiel−


Random Number Generation

It is assumed that a source of uniform [0,1] random

numbers exists

Random numbers R1, R2, R3, … with

pdf

cdf

=otherwise 0

10for 1)(

xxfR

1

10

0

1

0

)(

=

x

x

x

xxFR



Inverse-transform Technique

The concept

For cdf function: r = F(x)

Generate r from uniform [0,1]

Find x = F-1(r)

This technique can be used in principle for any distribution

Most useful when the cdf F(x) has an inverse F-1(x) which is easy

to compute

Required steps

Compute the cdf of the desired random variable X

Set F(X) = R on the range of X

Solve the equation F(X) = R for X in terms of R

Generation uniform random numbers R1, R2, R3, … and compute

the desired random variable by Xi = F-1(Ri)





Basic Queuing Systems

What is queuing theory?

Queuing theory is the study of queues

(sometimes called waiting lines)

Can be used to describe real world queues, or more abstract

queues, found in many branches of computer science, such as

operating systems

Basic queuing theory

Queuing theory is divided into 3 main sections

Traffic flow

Scheduling

Facility design and employee allocation


Kendall’s Notation

D. G. Kendall in 1951 proposed a standard notation for

classifying queuing systems into different types

Accordingly the systems were described by the notation A/B/C/D/E

where

A and B can take any of the following distributions types

A Distribution of inter arrival times of customers

B Distribution of service times

C Number of servers

D Maximum number of customers in the system

E Calling population size

M Exponential distribution (Markovian)

D Degenerate (or deterministic) distribution

Ek Erlang distribution (k = shape parameter)

Hk Hyper exponential with parameter k


Little’s Law

Assuming a queuing environment to be operating in a

stable steady state where all initial transients have

vanished, the key parameters characterizing the system

are as follows:

l - the mean steady state consumer arrival

N - the average number of customers in the system

T - the mean time spent by each customer in the system

which gives N = lT


Markov Process

A Markov process is one in which the next state of the

process depends only on the present state, irrespective of

any previous states taken by the process

The knowledge of the current state and the transition

probabilities from this state allows us to predict the next

state


Birth-Death Process

Special type of Markov process

Often used to model a population (or, number of jobs in a

queue)

If, at some time, the population has n entities (n jobs in a

queue), then birth of another entity (arrival of another job)

causes the state to change to n+1

On the other hand, a death (a job removed from the queue

for service) would cause the state to change to n-1

Any state transitions can be made only to one of the two

neighboring states


State Transition Diagram

The state transition diagram of the continuous birth-death

process

0 1 2 n-1 n n+1……

l0

1

l1

2

l2

3

ln-2

n-1

ln-1

n

ln

n+1

ln+1

n+2

…


M/M/1/ or M/M/1 Queuing System

When a customer arrives in this system it will be served if

the server is free

Otherwise the customer is queued

In this system customers arrive according to a Poisson

distribution and compete for the service in a FIFO (First In

First Out) manner

Service times are Independent and Identically Distributed

(IID) random variables, the common distribution being

exponential


Queuing Model and State Transition

Diagram

The M/M/1/ queuing model

The state transition diagram of the M/M/1/ queuing system

l

Queue Server

0 1 2 i-1 i i+1……

l

l

l

l

l

l

l

…


Equilibrium State Equations

If mean arrival rate is l and mean service rate is , i = 0, 1,

2.. be the number of customers in the system and P(i) be

the state probability of the system having i customers

From the state transition diagram, the equilibrium state

equations are given by

1 )1()1()()(

0 )1()0(

++−=+

==

iiPiPiP

iPP

ll

l


Traffic Intensity

We know that the P(0) is the probability of server being

free. Since P(0) > 0, the necessary condition for a system

being in steady state is,

This means that the arrival rate cannot be more than the

service rate, otherwise an infinite queue will form and jobs

will experience infinite service time

1=

l


Queuing System Metrics

= 1 - P(0), is the probability of the server being busy.

Therefore, we have

P(i) = i (1- )

The average number of customers in the system is

Ls =

The average dwell time of customers is

Ws =

The average queuing length is

Lq =

The average waiting time of customers is

Wq =

l

l

−

l −

1

)(

2

1 1

2)()1(

l

l

−

=

=−

=−

i

iPi

)()1(

2

l

l

l

l −=

−=

qL


M/M/S/ Queuing Model

State Transition Diagram

The M/M/S/ queuing model

State transition diagram

0 1 2 S-1 S S+1……

l

l

2

l

3

l

(S-1)

l

S

l

S

l

S

…

l

Queue

Servers

S

S

.

.

2

1


Queuing System Metrics

The average number of customers in the system is

The average dwell time of a customer in the system is

given by

The average queue length is

The average waiting time of customers is

= −+==

02)1(!

)0()(

i

S

s

S

PiiPL

2)1(!

)0(1

l −+==

SS

PLW

SS

S

=

+

−−=−=

si

S

q

SS

PiPSiL

2

1

)()!1(

)0()()(

2)1(!

)0(

l −==

SS

PLW

Sq

q


M/G/1 Queuing Model

We consider a single server queuing system whose arrival

process is Poisson with mean arrival rate l

Service times are Independent and Identically Distributed

with distribution function FG and pdf fG

Jobs are scheduled for service as FIFO

Documents

Probability, Statistics, and Traffic Theoriesmonet.skku.edu/.../2019/08/W07.-Probability_Theory.pdf · 2019-10-17 · Mobile Computing Networking Laboratory 2/54 Contents Introduction