Random Number Generators. Why do we need random variables? random components in simulation → need for a method which generates numbers that are random

Random Number Generators

2

Why do we need random variables?

• random components in simulation

→ need for a method which generates numbers that are random

• examples

– interarrival times

– service times

–demand sizes

040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I

3

terminology

• Random Numbers

– stochastic variable that meets certain conditions

–numbers randomly drawn from some known distribution

– It is impossible to “generate” them with the help of a digital computer.

• Pseudo-random Numbers

–numbers which seem to be randomly drawn from some known

distribution

–may be generated with the help of a digital computer


4

techniques for generating random variables

– ten-sided die (each throw generates a decimal)

– throwing a coin n times • get a binary number between 0 and 2^n-1

–other physical devices

–observe substances undergoing atomic decay• e.g., Caesium-137, Krypton-85,. . .


5

techniques for generating pseudo-random variables

• numbers are generated using a recursive formula

– ri is a function of ri-1, ri-2, …

– relation is deterministic – no random numbers

– if the mathematical relation not known and well chosen it is practically

impossible to predict the next number• In most cases we want the generated random numbers to simulate a

uniform distribution over (0, 1), that is a U(0, 1)-distribution

– there’re many simple techniques to transform uniform (U(a, b)) samples

into samples from other well-known distributions


1

0 1 0 1

6

terminology

• seed

– random number generators are usually initialized with a starting

number which is called the “seed”

–different seeds generates different streams of pseudorandom numbers

– the same seed results in the same stream

• cycle length

– length of the stream of pseudorandom numbers without repetition


7

methods for generating pseudo-random variables

• all methods usually generate uniformly distributed pseudorandom numbers in the interval [0,1]

–midsquare method (outdated)

– congruential method (popular)

– shift register (Tausworthe generator)


8

midsquare method

• arithmetic generator

–proposed by von Neumann and Metropolis in 1940s

• algorithm

– start with an initial number Z0 with m digits (seed) [m even]

–square it and get a number with 2m digits (add a zero at the beginning

if the number has only 2m-1 digits)

–obtain Z1 by taking the middle m digits

– to be in the interval [0,1) divide the Zi by 10m

–etc….


9

midsquare method: example (Z0 = 7182, m = 4)


i01234567…

Zi

7182 5811 7677 9363 6657 3156 9603

2176 …

Ui

0.71820.58110.76770.93630.66570.31560.96030.2176

…

Zi2

51581124 33767721 58936329 87665769 44315649

9960336 92217609

4734976 …

10

midsquare method (cont.)

• advantages

– fairly simple• disadvantage

– tends to degenerate fairly rapidly to zero• example: try Z0 = 1009

–not random as predictable


11

Linear Congruential Generator (LCG)

• LCGs

– first proposed by Lehmer (1951).

–produce pseudorandom numbers such that each single number

determines its successor by means of a linear function followed by a

modular reduction

– to be in the interval [0,1) divide the Zi by m

–Z0 seed a multiplier (integer)

– c increment (integer) m modulus (integer)• Variations are possible

– combinations of previous numbers instead of using only the last value040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I

12

LCG: example


i01234567…

Zi

7 6

1 8

11 10

5 12 …

Ui

0.43750.3750.06250.5 0.6875 0.625 0.31250.75

…

a = 5c = 3m = 16Z0 = 7

Z1 = (5*Z0 + 3) mod 16 = 38 mod 16 = 6Z2 = (5*Z1 + 3) mod 16 = 33 mod 16 = 1Z3 = (5*Z2 + 3) mod 16 = 8 mod 16 = 8…

13

properties of LCGs

• parameters need to be chosen carefully

–nonnegative

–0 < m a < m c < m Z0 < m

• disadvantages

–not random

– if a and m are properly chosen, the Uis will “look like” they are randomly

and uniformly distributed between 0 and 1.

– can only take rational values 0, 1/m, 2/m, … (m-1)/m

– looping


14

LCG (cont.)

• looping

–whenever Zi takes on a value it has had previously, exactly the same

sequence of values is generated, and this cycle repeats itself endlessly.

– length of sequence = period of generator

–period can be at most m • if so: LCG has full period

–LCG has full period iff• the only positive integer that divides both m and c is 1• if q is a prime number that divides m then q divides a-1• if 4 divides m, then 4 divides a-1


15

Linear Feedback Shift Register Generators (LFSR)

• developed by Tausworthe (1965)

– related to cryptographic methods

–operate directly on bits to form random numbers• linear combination of the last q bits

– ci constants (equal to 0 or 1, cq = 1)

– in most applications only two of the cj coefficients are nonzero

–execution can be sped up • modulo 2 equivalent to exclusive-or


16

LFSR (cont.)

• form a sequence of binary integers W1, W2, …

– string together l consecutive bits

– consider them as numbers in base 2

• transform them into Uis

• maximum period 2q -1

– if l is relatively prime to 2q-1 LFSR has full period


17

LFSR : example

• r = 3 q = 5 b1 = b2 = b3 = b4 = b5 = 1 l = 4


I 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 15 17 18

bi 1 1 1 1 1 0 0 0 1 1 0 1 1 1 0 1 0 1

i 1 2 3

Wi (base 2) 1111 1000 1101

Wi (base 10) 15 8 13

Ui 0.9375 0.5 0.8125

Testing RNGsEmpirical Tests

040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 18

19

random number generators

• methods presented so far…

– completely deterministic

–Uis appear as if they were U(0,1)

• test their actual quality

–how well do the generated Uis resemble values of true IID U(0,1)

• different kinds of tests

–empirical tests• based on actual Uis produced by RNG

– theoretical tests040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I

20

Chi-Square test (with all parameters known)

• checks whether the Uis appear to be IID U(0,1)

–divide [0,1] into k subintervals of equal length (k > 100)

–generate n random variables: U1, U2, .. Un

– calculate fj (number of Uis that fall into jth subinterval)

– calculate test statistic

– for large n χ2 will have an approximate chi-square distribution with k-1

df under the null hypothesis that the Uis are IID U(0,1)

– reject null hypothesis at level ® if


Generating Random Variates

22

Generating Random Variates

• General Approach

– Inverse Transformation

–Composition

–Convolution• Generating Continuous Random Variates

–Uniform U(a,b)

–Exponential

–m-Erlang

–Gamma, Weibull

–Normal

–etc…040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I

23

Inverse Transformation

• generate continuous random variate X

–distribution function F (continuous, strictly increasing)• 0 < F(x) < 1• if x1 < x2 then 0 < F(x1) · F(x2) < 1

– inverse of F: F-1

• algorithm

–generate U ~ U(0,1)

– return X = F-1(U)


24

Inverse Transformation (cont.)


x

F(x)

0

1

U1

X1X2

U2

• returned value X has desired distribution F

25

Inverse Transformation (cont.)

• create X according to exponential distribution with mean ¯

• in order to find F-1 set


take natural logarithm (base e)

26

Inverse Transformation for discrete variates

• distribution function

• probability mass function

• algorithm

–generate U ~ U(0,1)

–determine smallest possible integer i such that U · F(xi)

– return X = xi


27

Composition

• applies then the distribution function F can be expressed as a convex combination of other distribution functions F1, F2, ..

–we hope to be able to sample from the Fj’s more easily than from the

original F

– if X has a density it can be written as

• algorithm

–generate positive random integer J such that P(J = j) = pj

–Return X with distribution function FJ


28

Composition (example)

• double-exponential distribution (laplace distribution)

– can be rewritten as

–where IA(x) is the indicator function of set A

• f(x) is a convex combination of

– f1(x) = ex I(-1, 0) and f2(x) = e-x I[0, 1)

–p1 = p2 = 0.5


29

Convolution

• desired random variable X can be expressed as sum of other random variables that are IID and can be generated more readily then X directly

• algorithm

–generate Y1, Y2, … Ym IID each with distribution function G

– return X = Y1 + Y2 + + Ym

• example: m-Erlang random variable X with mean ¯

– sum of m IID exponential random variables with common mean ¯/m


30

Generate Continuous Variates

• Uniform distribution X(a,b)

X = a + (b-a)U• Exponential (mean ¯)

X = - ¯ ln (1-U) or X = -¯ ln U• m-Erlang


Documents

Random Number Generators. Why do we need random variables? random components in simulation → need for a method which generates numbers that are random