View
218
Download
0
Embed Size (px)
Citation preview
Random Number GeneratorsRandom Number Generators
Based upon specific mathematical Based upon specific mathematical algorithmsalgorithms
Which are repeatable and sequentialWhich are repeatable and sequential
RandomRandom
Truly RandomTruly Random– Exhibiting true randomnessExhibiting true randomness
PseudorandomPseudorandom– Appearance of randomness but having a Appearance of randomness but having a
specific repeatable patternspecific repeatable pattern Quasi-randomQuasi-random
– Having a set of non-random numbers in Having a set of non-random numbers in a randomized ordera randomized order
ProblemsProblems
Difficult to isolateDifficult to isolate– Often need to replace current generatorOften need to replace current generator– RequireRequire
Knowledge of current generatorKnowledge of current generatorSometimes in-depth understanding of Sometimes in-depth understanding of
random number generators themselvesrandom number generators themselves Large scale tests cause most Large scale tests cause most
problemsproblems– Needing sometimes millions or billions Needing sometimes millions or billions
of random numbersof random numbers
Desirable PropertiesDesirable Properties
When performing Monte Carlo When performing Monte Carlo SimulationsSimulations– Attributes of each particle should be Attributes of each particle should be
independent of those attributes of any independent of those attributes of any other particleother particle
– Fill the entire attribute space in a Fill the entire attribute space in a manner which is consistent with the manner which is consistent with the physicsphysics
Random Number CycleRandom Number Cycle
BasisBasis– sequence of pseudorandom integerssequence of pseudorandom integers
Some exceptionsSome exceptions
Integers (“Fixed”)Integers (“Fixed”)– Manipulated arithmetically to yield Manipulated arithmetically to yield
floating point (“real”)floating point (“real”) Can be presented in either Integer or Can be presented in either Integer or
Real numbersReal numbers
What Does This Show Us?What Does This Show Us?
Properties of pseudorandom Properties of pseudorandom sequences of integers sequences of integers – The sequence has a finite number of The sequence has a finite number of
integers integers – The sequence gets traversed in a The sequence gets traversed in a
particular orderparticular order– The sequence repeats if the period of The sequence repeats if the period of
the generator is exceeded the generator is exceeded
LCGLCG
Most commonly used RNGMost commonly used RNG– Linear Congruential GeneratorLinear Congruential Generator
Requires initial “seed” denoted as X0Requires initial “seed” denoted as X0Appears random because of Modulo functionAppears random because of Modulo functionNext “random” number depends heavily on Next “random” number depends heavily on
previous Xprevious X– Typical of linear, congruential generatorsTypical of linear, congruential generators– Restricts periodRestricts period
Using LCGUsing LCG
Choosing Correct Input is KeyChoosing Correct Input is Key LCG (a,c,m,X0)LCG (a,c,m,X0)
– LCG (5, 1, 16, 1)LCG (5, 1, 16, 1) Yields – Yields –
1,6,15,12,13,2,11,8,9,14,7,4,5,10,3,0,1,6,15,12,13,2,11,8,9,14,7,4,5,10,3,0, 1,6,15,12,13,2,11,8,9,14,…1,6,15,12,13,2,11,8,9,14,…
When the next result depends upon only the When the next result depends upon only the previous integer, the longest period possible is P=Mprevious integer, the longest period possible is P=M
Odd/Even patternOdd/Even pattern lack of randomness results from using a power of two lack of randomness results from using a power of two
for Mfor M
Example #2Example #2
LCG(5,0,16,1)LCG(5,0,16,1)– Yields - 1,5,9,13,1,5,9,13,…Yields - 1,5,9,13,1,5,9,13,…– M is a power of 2 (here: 2^4)M is a power of 2 (here: 2^4)
C=0C=0Maximum period is going to be 2^(m-2)Maximum period is going to be 2^(m-2)
– Correlation (each differ by 4)Correlation (each differ by 4)
SeedSeed
Using the date and timeUsing the date and time– Enter the date and time into an Enter the date and time into an
equations and return an integer then equations and return an integer then make sure it is oddmake sure it is odd
– Standard seed for these equationsStandard seed for these equations
Overflow & Negative NumbersOverflow & Negative Numbers
Using large values of a and large Using large values of a and large values of M are neededvalues of M are needed– Often 31 bits longOften 31 bits long
On 32 bit machinesOn 32 bit machines
– A*M results in 62 bit numberA*M results in 62 bit number– OverflowOverflow
Can result in 32Can result in 32ndnd bit being a negative bit being a negative
N-Tuple GeneralizationN-Tuple Generalization
Choose R1 and R2Choose R1 and R2– Choose Rn and R(n+1)Choose Rn and R(n+1)– Then plot this point of interest in a Then plot this point of interest in a
surrounding area.surrounding area.Plot these points in successionPlot these points in successionThe area will be uniformly covered by the The area will be uniformly covered by the
LCG in a “random” orderLCG in a “random” order– Covering of only part of the unit or certain areas Covering of only part of the unit or certain areas
of the unit would prove to be not useful for Monte of the unit would prove to be not useful for Monte Carlo MethodsCarlo Methods
Embarrassingly Parallel'Embarrassingly Parallel'
Little or no interprocessor Little or no interprocessor communicationcommunication
Easy to codeEasy to code
N StreamsN Streams
N StreamsN Streams– N independent random numbersN independent random numbers– N independent processesN independent processes
Need to find N seeds far away from Need to find N seeds far away from each other on the cycleeach other on the cycle
Find SeedsFind Seeds
Find SeedsFind Seeds– LCG rule successively applied: LCG rule successively applied:
Lagged Fibonacci Generators Lagged Fibonacci Generators
Increasingly popularIncreasingly popular– Lags are k and lLags are k and l– M is power of 2M is power of 2
With proper choice of k and LWith proper choice of k and L Period of Generator can bePeriod of Generator can be
– [(2^L)-1] * [2^(m-1)][(2^L)-1] * [2^(m-1)]
LFGLFG
Computationally simpleComputationally simple– Integer addInteger add– Logical ANDLogical AND– Decrement of 2 array pointersDecrement of 2 array pointers
– Must keep L words current in memoryMust keep L words current in memory– LCG needs only oneLCG needs only one
LFG (cont)LFG (cont) LFG are an attempt to improve LCGLFG are an attempt to improve LCG Similar to Combined LCGSimilar to Combined LCG
– Take 2 previous numbers in the sequence to Take 2 previous numbers in the sequence to produce a new numberproduce a new number
– Where p and q are the “lags”Where p and q are the “lags”– Some arithmetic computation is performedSome arithmetic computation is performed– Then mod that answer for the next numberThen mod that answer for the next number
Overview Overview
IntroductionIntroduction HistoryHistory ExamplesExamples ApplicationsApplications Real Life practices Real Life practices
IntroductionIntroduction
Define Monte Carlo MethodDefine Monte Carlo Method– The Monte Carlo method is a numerical The Monte Carlo method is a numerical
method for solving mathematical method for solving mathematical problems using stochastic sampling. problems using stochastic sampling.
– It performs simulation of any process It performs simulation of any process whose development is influenced by whose development is influenced by random factors, but also if the given random factors, but also if the given problem involves no chance, the method problem involves no chance, the method enables artificial construction of a enables artificial construction of a probabilistic model. probabilistic model.
Introduction cont…Introduction cont…
Similarly, Monte Carlo methods randomly Similarly, Monte Carlo methods randomly select values to create scenarios of a select values to create scenarios of a problem. These values are taken from problem. These values are taken from within a fixed range and selected to fit a within a fixed range and selected to fit a probability distribution [e.g. bell curve, probability distribution [e.g. bell curve, linear distribution, etc.]. This is like rolling a linear distribution, etc.]. This is like rolling a dice. The outcome is always within the dice. The outcome is always within the range of 1 to 6 and it follows a linear range of 1 to 6 and it follows a linear distribution - there is an equal opportunity distribution - there is an equal opportunity for any number to be the outcome.for any number to be the outcome.
Introduction cont…Introduction cont…
MC method is often referred to as the MC method is often referred to as the “method of last resort”, as it is apt to “method of last resort”, as it is apt to consume large computing resources;consume large computing resources;
Characteristics:Characteristics:– consuming vast computing resources consuming vast computing resources – have historically had to be executed upon have historically had to be executed upon
the fastest computers available at the time the fastest computers available at the time – and employ the most advanced algorithmsand employ the most advanced algorithms– implemented with substantial implemented with substantial
programming acumen. programming acumen.
Introduction cont…Introduction cont…
Major components of Monte Carlo Major components of Monte Carlo methods:methods:– Probability distribution functionsProbability distribution functions – Random number generatorRandom number generator – ScoringScoring– Error estimationError estimation – Variance reduction techniquesVariance reduction techniques – Parallelization and vectorizationParallelization and vectorization
HistoryHistory
Where does Monte Carlo method come Where does Monte Carlo method come from? When? Who?from? When? Who?– The name "Monte Carlo" comes from the city of The name "Monte Carlo" comes from the city of
Monte Carlo in the principality of Monaco, Monte Carlo in the principality of Monaco, famous for its gambling housefamous for its gambling house
– Birth date of the Monte Carlo method is 1949, Birth date of the Monte Carlo method is 1949, when an articale entitled "The Monte Carlo when an articale entitled "The Monte Carlo Method"( by N. Metropolis and S. Ulam ) Method"( by N. Metropolis and S. Ulam ) appeared. appeared.
– The American mathematicians J. Neyman and The American mathematicians J. Neyman and S. Ulam are considered its originators. S. Ulam are considered its originators.
History cont…History cont… The theoretical foundation of the method The theoretical foundation of the method
had been known long before first articles had been known long before first articles were published. were published.
Well before 1949 certain problems in Well before 1949 certain problems in statistics were sometimes solved by means statistics were sometimes solved by means of random sampling of random sampling
However, simulation of random variables However, simulation of random variables by hand is a laborious process by hand is a laborious process
Use of the Monte Carlo method as a Use of the Monte Carlo method as a universal numerical technique became universal numerical technique became practical only with the advent of computers practical only with the advent of computers and high-quality pseudorandom number and high-quality pseudorandom number generators generators
History cont…History cont… Buffon's needle problemBuffon's needle problem
– In 1768 Buffon, a French mathematician, In 1768 Buffon, a French mathematician, experimentally determined a value of experimentally determined a value of ππ by by casting a needle on a ruled gridcasting a needle on a ruled grid
Lord Rayleigh even delved into this field Lord Rayleigh even delved into this field near the turn of the century.near the turn of the century.
Fredericks and Levy in 1928 showed how Fredericks and Levy in 1928 showed how the method could be used to solve the method could be used to solve boundary value problemsboundary value problems
Enrico Fermi in the 1930's used Monte Enrico Fermi in the 1930's used Monte Carlo in the calculation of neutron Carlo in the calculation of neutron diffusion (involving nuclear reactors )diffusion (involving nuclear reactors )
History cont…History cont…
In the 1940's, a formal foundation for In the 1940's, a formal foundation for the Monte Carlo method was the Monte Carlo method was developed by von Neumann (PDE)developed by von Neumann (PDE)
Stanislaw Ulam realized the Stanislaw Ulam realized the importance of the digital computer in importance of the digital computer in the implementation of the approach the implementation of the approach from collaboration results of the work from collaboration results of the work on the Manhattan project during on the Manhattan project during World War IIWorld War II
ExamplesExamples
Simple Example to Understand: Simple Example to Understand: computing the area of a plane figure computing the area of a plane figure S. S. – completely arbitary figure with a completely arbitary figure with a
curvilinear boundary, given graphically curvilinear boundary, given graphically or analytically, connected or consisting or analytically, connected or consisting of several pieces of several pieces
– assume that it is contained completely assume that it is contained completely within the unit square. within the unit square.
Examples cont…Examples cont…
Figure S in the unit square, being covered with sampling points randomly
Examples cont…Examples cont…
Applying Randomness to the Applying Randomness to the example:example:– Choose at random N points in the Choose at random N points in the
square and designate the number of square and designate the number of points lying inside S by N'. It is points lying inside S by N'. It is geometrically obvious that the area of S geometrically obvious that the area of S is approximately equal to the ratio N'/N. is approximately equal to the ratio N'/N. The greater the N, the greater the The greater the N, the greater the accuracy of this estimate. accuracy of this estimate.
Examples cont…Examples cont…
Buffon's Needle:Buffon's Needle:– A simple Monte Carlo method for the A simple Monte Carlo method for the
estimation of the value of estimation of the value of ππ, 3.1415926, 3.1415926– Assumptions:Assumptions:
Suppose you have a tabletop with a number Suppose you have a tabletop with a number of parallel lines drawn on it, which are of parallel lines drawn on it, which are equally spaced (say the spacing is 1 inch, for equally spaced (say the spacing is 1 inch, for example). example).
Suppose you also have a pin or needle, Suppose you also have a pin or needle, which is also an inch long. which is also an inch long.
Examples cont…Examples cont…
Dropping needles on the tablet:Dropping needles on the tablet:– The needle crosses or touches one of the lines The needle crosses or touches one of the lines – The needle crosses no linesThe needle crosses no lines
Keep dropping this needle over and over Keep dropping this needle over and over on the tableon the table
Record the statistics.Record the statistics.– Keep track of both the total number of times Keep track of both the total number of times
that the needle is randomly dropped on the that the needle is randomly dropped on the table N, and the number of times that it table N, and the number of times that it crosses a line N’. crosses a line N’.
Examples cont…Examples cont…
Findings:Findings:– 2N/N’=2N/N’=ππ– Because, Because, the probability on any given the probability on any given
drop of the needle that it should cross a drop of the needle that it should cross a line is given by 2/pi line is given by 2/pi
– After many tries, N/N’ will approach the After many tries, N/N’ will approach the probability number.probability number.
Applications Applications
Monte Carlo methods can help in Monte Carlo methods can help in design and prediction of behavior of design and prediction of behavior of systems in nuclear applications and systems in nuclear applications and radiation physicsradiation physics
The use of MC in the area of nuclear The use of MC in the area of nuclear power has undergone an important power has undergone an important evolution. Notable are the extensions evolution. Notable are the extensions to compute burnup in reactor cores, to compute burnup in reactor cores, and full core neutronic simulations. and full core neutronic simulations.
Applications cont…Applications cont…
help researchers understand the help researchers understand the probability of the occurrence of an probability of the occurrence of an adverse effect associated with adverse effect associated with exposures to chemicals. Monte Carlo exposures to chemicals. Monte Carlo sampling simulates the distribution of sampling simulates the distribution of total exposures, by simulating random total exposures, by simulating random samples of factors associated with each samples of factors associated with each exposure route and accumulating them exposure route and accumulating them to arrive at an individual total exposure. to arrive at an individual total exposure.
Applications cont…Applications cont…
The use of MC methods to model physical The use of MC methods to model physical problems allows us to examine more problems allows us to examine more complex systems than we otherwise can. complex systems than we otherwise can. Solving equations which describe the Solving equations which describe the interactions between two atoms is fairly interactions between two atoms is fairly simple; solving the same equations for simple; solving the same equations for hundreds or thousands of atoms is hundreds or thousands of atoms is impossible. With MC methods, a large system impossible. With MC methods, a large system can be sampled in a number of random can be sampled in a number of random configurations, and that data can be used to configurations, and that data can be used to describe the system as a whole. describe the system as a whole.
Applications cont…Applications cont…
Random numbers generated by the Random numbers generated by the computer are used to simulate computer are used to simulate naturally random processesnaturally random processes
many previously intractable many previously intractable thermodynamic and quantum thermodynamic and quantum mechanics problems have been mechanics problems have been solved using Monte Carlo techniquessolved using Monte Carlo techniques
Real Life Practice Real Life Practice
Quantum Monte CarloQuantum Monte Carlo– The microscopic world is described by The microscopic world is described by
quantum mechanics. We need to use quantum mechanics. We need to use simulation techniques to “solve” many-simulation techniques to “solve” many-body quantum problems.body quantum problems.
– Both the wavefunction and expectation Both the wavefunction and expectation values are determined by the values are determined by the simulations.simulations.
– QMC gives most accurate method for QMC gives most accurate method for general quantum many-body systems.general quantum many-body systems.
Real Life Practice cont…Real Life Practice cont…
• WeatherWeather Equipment ProductivityEquipment Productivity Soil ConditionsSoil Conditions
Projects are often associated with a high Projects are often associated with a high degree of uncertainty resulting from degree of uncertainty resulting from the unpredictable nature of eventsthe unpredictable nature of events
Real Life Practice cont…Real Life Practice cont…
Risk Analysis and Risk Management Risk Analysis and Risk Management – Monte Carlo Simulation is a valuable modeling Monte Carlo Simulation is a valuable modeling
tool that generates multiple scenarios tool that generates multiple scenarios depending upon the data and the assumptions depending upon the data and the assumptions fed into the model. fed into the model.
– Simulation calculates multiple scenarios by Simulation calculates multiple scenarios by repeatedly inserting different sampling values repeatedly inserting different sampling values from probability distribution for the uncertain from probability distribution for the uncertain variables into the computerized spread-sheet.variables into the computerized spread-sheet.
– probability or percentage chance that a probability or percentage chance that a particular forecast value will fall within a certain particular forecast value will fall within a certain specified range. specified range.
Why has the Monte Carlo method Why has the Monte Carlo method become so popular? become so popular?
Analytic methods tend to be Analytic methods tend to be prohibitive prohibitive (but some very difficult (but some very difficult problems have finally been solved problems have finally been solved using MC)using MC)
Monte Carlo is somewhat Monte Carlo is somewhat intuitive intuitive (and several good books have now (and several good books have now been written on the subject)been written on the subject)
Computers continue to get Computers continue to get faster faster and and cheapercheaper