Computer Simulation

2WB05 SimulationLecture 8: Generatingrandom variables

Marko Boonhttp://www.win.tue.nl/courses/2WB05

January 7, 2013

2/36

Department of Mathematics and Computer Science

1. How do we generate random variables?

2. Fitting distributions

Outline

3/36


How do we generate random variables? Sampling from continuous distributions Sampling from discrete distributions

Generating random variables

4/36


Inverse Transform MethodLet the random variable X have a continuous and increasing distribution function F . Denote the inverse of Fby F1. Then X can be generated as follows: Generate U from U (0, 1); Return X = F1(U ).

If F is not continuous or increasing, then we have to use the generalized inverse function

F1(u) = min{x : F(x) u}.

Continuous distributions

5/36


Examples X = a + (b a)U is uniform on (a, b); X = ln(U )/ is exponential with parameter ; X = ( ln(U ))1/a/ is Weibull, parameters a and .

Unfortunately, for many distribution functions we do not have an easy-to-use (closed-form) expression for theinverse of F .


6/36


Composition methodThis method applies when the distribution function F can be expressed as a mixture of other distribution func-tions F1, F2, . . .,

F(x) =i=1

piFi(x),

wherepi 0,

i=1

pi = 1

The method is useful if it is easier to sample from the Fi s than from F .

First generate an index I such that P(I = i) = pi , i = 1, 2, . . . Generate a random variable X with distribution function FI .


7/36


Examples Hyper-exponential distribution:

F(x) = p1F1(x)+ p2F2(x)+ + pkFk(x), x 0,where Fi(x) is the exponential distribution with parameter i , i = 1, . . . , k. Double-exponential (or Laplace) distribution:

f (x) =

12e

x, x < 0;12ex, x 0,

where f denotes the density of F .


8/36


Convolution methodIn some case X can be expressed as a sum of independent random variables Y1, . . . , Yn, so

X = Y1 + Y2 + + Yn.where the Yi s can be generated more easily than X .

Algorithm:

Generate independent Y1, . . . , Yn, each with distribution function G; Return X = Y1 + + Yn.


9/36


ExampleIf X is Erlang distributed with parameters n and , then X can be expressed as a sum of n independentexponentials Yi , each with mean 1/.

Algorithm:

Generate n exponentials Y1, . . . , Yn, each withmean ;

Set X = Y1 + + Yn.More efficient algorithm:

Generate n uniform (0, 1) random variablesU1, . . . ,Un;

Set X = ln(U1U2 Un)/.


10/36


Acceptance-Rejection methodDenote the density of X by f . This method requires a function g that majorizes f ,

g(x) f (x)for all x . Now g will not be a density, since

c =

g(x)dx 1.

Assume that c

11/36


Validity of the Acceptance-Rejection methodNote

P(X x) = P(Y x |Yaccepted).Now,

P(Y x, Yaccepted) = x

f (y)g(y)

h(y)dy = 1c

x

f (y)dy,

and thus, letting x givesP(Yaccepted) = 1

c.

Hence,

P(X x) = P(Y x, Yaccepted)P(Yaccepted)

= x

f (y)dy.


12/36


Note that the number of iterations is geometrically distributed with mean c.

How to choose g?

Try to choose g such that the random variable Y can be generated rapidly; The probability of rejection in step 3 should be small; so try to bring c close to 1, which mean that g should

be close to f .


13/36


ExampleThe Beta(4,3) distribution has density

f (x) = 60x3(1 x)2, 0 x 1.The maximal value of f occurs at x = 0.6, where f (0.6) = 2.0736. Thus, if we define

g(x) = 2.0736, 0 x 1,then g majorizes f . Algorithm:

1. Generate Y and U from U (0, 1);

2. If U 60Y3(1 Y )22.0736

, then set X = Y ; else reject Y and return to step 1.


14/36


Normal distributionMethods:

Acceptance-Rejection method Box-Muller method


15/36


Acceptance-Rejection methodIf X is N (0, 1), then the density of |X | is given by

f (x) = 22pi

ex2/2, x > 0.

Now the functiong(x) = 2e/piex

majorizes f . This leads to the following algorithm:

1. Generate an exponential Y with mean 1;

2. Generate U from U (0, 1), independent of Y ;

3. If U e(Y1)2/2 , then accept Y ; else reject Y and return to step 1.4. Return X = Y or X = Y , both with probability 1/2.


16/36


Box-Muller methodIf U1 and U2 are independent U (0, 1) random variables, then

X1 =2 lnU1 cos(2piU2)

X2 =2 lnU1 sin(2piU2)

are independent standard normal random variables.

This method is implemented in the function nextGaussian() in java.util.Random


17/36


Discrete version of Inverse Transform MethodLet X be a discrete random variable with probabilities

P(X = xi) = pi , i = 0, 1, . . . ,i=0

pi = 1.

To generate a realization of X , we first generate U from U (0, 1) and then set X = xi ifi1j=0

p j U ;

Set X = I .

Discrete distributions

27/36


Poisson (alternative) Generate U(0,1) random variables U1,U2, . . .;

let I be the smallest index such thatI+1i=1

Ui < e;

Set X = I .

Discrete distributions

28/36


Input of a simulationSpecifying distributions of random variables (e.g., interarrival times, processing times) and assigning parame-ter values can be based on:

Historical numerical data Expert opinion

In practice, there is sometimes real data available, but often the only information of random variables that isavailable is their mean and standard deviation.

Fitting distributions

29/36


Empirical data can be used to:

construct empirical distribution functions and generate samples from them during the simulation; fit theoretical distributions and then generate samples from the fitted distributions.


30/36


Moment-fittingObtain an approximating distribution by fitting a phase-type distribution on the mean, E(X), and the coefficientof variation,

cX = XE(X),of a given positive random variable X , by using the following simple approach.


31/36


Coefficient of variation less than 1: Mixed ErlangIf 0 < cX < 1, then fit an Ek1,k distribution as follows. If

1k c2X

1k 1,

for certain k = 2, 3, . . ., then the approximating distribution is with probability p (resp. 1 p) the sum of k 1(resp. k) independent exponentials with common mean 1/. By choosing

p = 11+ c2X

(kc2X

k(1+ c2X) k2c2X

), = k p

E(X),

the Ek1,k distribution matches E(X) and cX .

Moment-fitting

32/36


Coefficient of variation greater than 1: HyperexponentialIn case cX 1, fit a H2(p1, p2;1, 2) distribution.

Phase diagram for the Hk(p1, . . . , pk;1, . . . , k) distribution:

1

k

1

k

p 1

pk

Moment-fitting

33/36


But the H2 distribution is not uniquely determined by its first two moments. In applications, the H2 distributionwith balanced means is often used. This means that the normalization

p11= p22

is used. The parameters of the H2 distribution with balanced means and fitting E(X) and cX ( 1) are given by

p1 = 12

(1+

c2X 1c2X + 1

), p2 = 1 p1,

1 = 2p1E(X), 2 =2p2E(X)

.

Moment-fitting

34/36


In case c2X 0.5 one can also use a Coxian-2 distribution for a two-moment fit.

Phase diagram for the Coxian-k distribution:

1 2 k

1 2 kp 1 p 2 pk 1

1 p 1 1 p 2 1 pk 1

The following parameter set for the Coxian-2 is suggested:

1 = 2/E(X), p1 = 0.5/c2X , 2 = 1p1.

Moment-fitting

35/36


Fitting nonnegative discrete distributionsLet X be a random variable on the non-negative integers with mean E(X) and coefficient of variation cX . Thenit is possible to fit a discrete distribution on E(X) and cX using the following families of distributions:

Mixtures of Binomial distributions; Poisson distribution; Mixtures of Negative-Binomial distributions; Mixtures of geometric distributions.

This fitting procedure is described in Adan, van Eenige and Resing (see Probability in the Engineering andInformational Sciences, 9, 1995, pp 623-632).

Moment-fitting

36/36


Adequacy of fit

Graphical comparison of fitted and empirical curves; Statistical tests (goodness-of-fit tests).


Documents

Computer Simulation