Ch2. Random Numbers - Home - Department of Statistics …stazjt/teaching/ST4231/lectures/... · Introduction Pseudo-random Number Generation Application: Monte Carlo Integration Ch2

IntroductionPseudo-random Number GenerationApplication: Monte Carlo Integration

Ch2. Random Numbers

Zhang Jin-Ting

Department of Statistics and Applied Probability

August 2, 2012

Zhang J.T. Ch2. Random Numbers


Outline

Introduction

Pseudo-random Number Generation

Application: Monte Carlo Integration



AimsI The building block of a simulation study is the ability to

generate random numbers, where a Random Numberrepresents the value of a random variable UniformlyDistributed on (0,1).

I In this chapter we will explain how such numbers aregenerated using computer and illustrate the use of themthrough some examples.



AimsI The building block of a simulation study is the ability to

generate random numbers, where a Random Numberrepresents the value of a random variable UniformlyDistributed on (0,1).

I In this chapter we will explain how such numbers aregenerated using computer and illustrate the use of themthrough some examples.



ModulusI Our aim is to construct one system using Number Theory

which can generate a sequence of values, which, althoughthey are deterministically generated, have all theappearances of being independent uniform (0,1) randomvariables.

I Let a mod m denote the remainder after the integer a isdivided by the integer m. Here “mod” is a shorthand of“modulus”.

I ExamplesI 10 mod 8=2.I 10453 mod 10000 =453.



























Multiplicative Congruential MethodI Starting with an initial value x0, called the seed, and then

recursively computes successive values xn, n ≥ 1, byletting

xn = axn−1 mod m (1)

where a and m are given positive integers. Thus, each xnis either 0,1,· · · ,m-1.

I The quantity xn/m—called a pseudorandom number—istaken as an approximation to the value of a uniform (0,1)random variable.



Multiplicative Congruential MethodI Starting with an initial value x0, called the seed, and then

recursively computes successive values xn, n ≥ 1, byletting

xn = axn−1 mod m (1)

where a and m are given positive integers. Thus, each xnis either 0,1,· · · ,m-1.

I The quantity xn/m—called a pseudorandom number—istaken as an approximation to the value of a uniform (0,1)random variable.



Three Criteria for Pseudo-random NumberI In general the constants a and m should be chosen to

satisfy three criteria:

1. For any initial seed, the resultant sequence has the“appearance” of being a sequence of independent uniform(0,1) random variables.

2. For any initial seed, the number of variables that can begenerated before repetition begins is large.

3. The values can be computed efficiently on a digitalcomputer.
























GuidelineI A guideline that appears to be of help in satisfying the

above three conditions is that m should be chosen to be alarge prime number that can be fitted to the computer wordsize.

I For a 32-bit word machine, it has been shown that thechoice of m = 231 − 1 and a = 75 = 16807 result indesirable properties.

I For a 36-bit computer, m = 235 − 31 and a = 55.















Three Main GeneratorsI Mixed Congruential

xn = (axn−1 + c) mod m

I Inversive Congruential

xn = (ax−1n−1 + c) mod m

I Nonlinear Congruential

xn = (dx2n−1 + axn−1 + c) mod m



















Other GeneratorsI Other random number generators include Feedback Shift

Register methods, Coupled Generators, PortableGenerators, and the methods based on Chaotic Systems.

I Most computer languages already have a Built-in (standarduniform) Random Number Generator which can be calledto generate other Non-uniform random numbers.



Other GeneratorsI Other random number generators include Feedback Shift

Register methods, Coupled Generators, PortableGenerators, and the methods based on Chaotic Systems.

I Most computer languages already have a Built-in (standarduniform) Random Number Generator which can be calledto generate other Non-uniform random numbers.



Monte Carlo IntegrationI This is one of the earliest applications of random numbers.I Suppose we want to compute

θ =

∫ 1

0g(x)dx .

I If U is uniformly distributed over (0,1), then we can write θas

θ = E [g(U)]

I If U1, U2, · · · , Uk are independent uniform (0,1) randomvariables, then we have that g(U1), · · · , g(Uk ) are i.i.d.random variables.




θ =

∫ 1

0g(x)dx .


θ = E [g(U)]





θ =

∫ 1

0g(x)dx .


θ = E [g(U)]





θ =

∫ 1

0g(x)dx .


θ = E [g(U)]




I By strong law of large numbers, it follows that, withprobability 1,

1k

k∑

i=1

g(Ui) → E [g(U)] = θ (k →∞)

I Hence, we can generate a large number of randomnumbers ui and approximate θ by the average value ofg(ui), i.e.,

θ̂ =1k

k∑

i=1

g(ui)

I This approach to approximating integrals is called theMonte Carlo Integration.




1k

k∑

i=1

g(Ui) → E [g(U)] = θ (k →∞)


θ̂ =1k

k∑

i=1

g(ui)





1k

k∑

i=1

g(Ui) → E [g(U)] = θ (k →∞)


θ̂ =1k

k∑

i=1

g(ui)




Example 1. Integration Transformation:[a, b] → [0, 1]

θ =

∫ b

ag(x)dx

Make the substitution y = (x − a)/(b − a), dy = dx/(b − a), weget

θ =

∫ 1

0g(a + [b − a]y)(b − a)dy

=

∫ 1

0h(y)dy



Example 2. Integration Transformation:[0,∞] → [0, 1]

θ =

∫ ∞

0g(x)dx

Make the substitution y = 1/(x + 1),dy = −dx/(x + 1)2 = −y2dx , we get

θ =

∫ 1

0h(y)dy

whereh(y) =

g(1/y − 1)

y2



Example 3. Multi-dimensional Integral

θ =

∫ 1

0

∫ 1

0· · ·

∫ 1

0g(x1, x2, · · · , xn)dx1dx2 · · ·dxn

Generate k independent sets, each consisting of n independentuniform (0,1) random variables

u(1)1 · · · u(1)

n

u(2)1 · · · u(2)

n...

u(k)1 · · · u(k)

n

Since g(U(i)1 , · · · , U(i)

n ), i = 1, 2, · · · , k are i.i.d, we have

θ̂ =1k

k∑

i=1

g(U(i)1 , · · · , U(i)

n ) → θ



Example 4. Estimation of π

I Suppose the random vector (X , Y ) is uniformly distributedin the square of area 4 centered at the origin (0,0).

I We have the following relationship

P(X 2 + Y 2 ≤ 1) =π

4

I Hence, if we generate a large number of random points inthe square, the proportion of points that fall within the circlewill be approximated π/4.






P(X 2 + Y 2 ≤ 1) =π

4







P(X 2 + Y 2 ≤ 1) =π

4



Documents

Ch2. Random Numbers - Home - Department of Statistics …stazjt/teaching/ST4231/lectures/... · Introduction Pseudo-random Number Generation Application: Monte Carlo Integration Ch2