M03-13 Muhammad Khan Muneer (Presentation on Random Numbers)

7/27/2019 M03-13 Muhammad Khan Muneer (Presentation on Random Numbers)

http://slidepdf.com/reader/full/m03-13-muhammad-khan-muneer-presentation-on-random-numbers 1/57

By

Jarry Jafery (M15-13)

Muhammad Khan Muneer (M03-13)



Lacking a definite plan, purpose, or pattern

A set where each of the elements has equalprobability of occurrence



A sequence in which each term isunpredictable

D. H. Lehmer (1951)When discussing a sequence of random numberseach number drawn must be statisticallyindependent of the others.

Examples between 1 and 100 29, 95, 11, 60, 22



The Chinese were perhaps the earliest people toformalize odds and chance 3,000 years ago.

The Greek philosophers discussed randomness atlength, but only in non-quantitative forms.

It was only in the sixteenth century that Italianmathematicians began to formalize the odds

associated with various games of chance.



In 1937 Kendall and smith obtained 100,000 random

numbers grouped in two’s and four’s from machine. Also

F isher and Yates generated 15,000 random numbers

arranged in two’s which were taken from A.J Thompson’s table of logarithm.

In 1951, Derr ick Hennery Lehmer invented the linear

congruential generator, used in most pseudorandomnumber generator today.

In 1955 Rand Corporation built a special machine to

generate pseudorandom binary bits of 0 and1 that werethen used to produce a table of one million random

decimal digits.



John Von Neumann was a pioneer in

computer-based random numbergenerators.

With the spread of the use of computersalgorithmic pseudorandom number

generators replaced random number tables,

and “true” random number generators

(Hardware random number generators) are

only used in a few cases.



Almost all network security protocols rely on the

randomness of certain parameters

Nonce - used to avoid replay

Session Key

Unique parameters in digital signatures

Monte Carlo Simulations -

is a mathematical technique for numerically solving differential

equations. Randomly generates scenarios for collecting statistics.



Simulation

Computer ProgrammingDecision Making

Recreation



Simulate natural phenomena on a computer

Used for experiments in sterile conditions tomake them more realistic

Useful in all of the Applied Disciplines



Test program effectiveness

Test algorithm correctness

Instead of all possible inputs use a few random

numbers Microsoft has used this logic in testing their software



When an “unbiased” decision is needed

Fixed decision can cause some algorithms to runmore slowly

Good way of choosing who goes first Sporting events



Lottery Equal odds

The KY Lottery uses Microsoft Excel’s RNG for“various second chance drawings“

Casinos Provides a chance for “luck”



Video Games Random events keep games entertaining

Q-bert



True random numbersPseudo-random numbers



Truly random - is defined as exhibiting ``true''randomness, such as the time between ``tics''from a Geiger counter exposed to a radioactiveelement

Pseudorandom - is defined as having theappearance of randomness, but neverthelessexhibiting a specific, repeatable pattern.

numbers calculated by a computer through a

deterministic process, cannot, by definition, berandom



Examples of random numbers:

Noise in electrical circuits

Radioactive decay Flow of water from a vessel

Session keys

Numbers to be hashed with passwords

Prepaid card numbers

Nonce



Anyone who considers arithmetical methods ofproducing random digits is, of course, in a state of sin.

John Von Neumann (1951)



Properties of pseudo-random numbers

• Continuous numbers between 0 and 1

• Probability of selecting a number in interval (a,b)~ (b-a) – i.e. Uniformly distributed

• Numbers are statistically independent

• Can’t really generate random numbers

• Also, want fast and repeatable



Muhammad Khan Muneer

M03-13



How to generate random numbers

Table look-up

Computer generation: these values cannot betruly random and a computer cannot express anumber to an infinite number of decimal places.



Measurements of electric current passing through awire at some time point.

Inflow and outflow of water at a reservoir at sometime point.

Timing of keystrokes when a user enters a password.

Measurement of air turbulence due to the movementof hard drive heads.

Precise measurement of current leakage from a CPU

or any other system component. Measurement of timing skew between two systems’

timers: A hardware timer

A software timer



Selection of random numbers from random numbertables:

A practical method of selecting a random sample is tochoose units one-by-one with the help of a Table ofrandom numbers. By considering two-digits numbers,we can obtain numbers from 00 to 99, all havingthe same frequency. Similarly, three or moredigit numbers may be obtained by combining three ormore rows or columns of these Tables. The simplestway of selecting a sample of the required size

is by selecting a random number from 1 to N andthen taking the unit bearing that number. Thisprocedure involves a number of rejections since allnumbers greater than N appearing in the Tableare not considered for selection.



So, the selected numbers are, therefore,modified and some of the modificationprocedures are:

Remainder ApproachQuotient Approach

Independent Choice of Digits



Let N be an r-digit number and let its r-digithighest multiple be N'. A random number k is chosen from 1 to N' and the unit with theserial number equal to the remainderobtained on dividing k by N, is selected. Ifthe remainder is zero, the last unit isselected.



Example:

Let N = 127, the highest three-digit multiple of127 is 889. For selecting a unit, one random

number from 001 to 889 has to be selected.Let the random number selected be 503.Dividing 503 by 127, the remainder is122. Hence, the unit with serial number 122

is selected in the sample.



Let N be an r-digit number and let its r-digithighest multiple be N' such that N' / N = d. Arandom number k is chosen from 0 to (N'-1).Dividing k by d, the quotient q is obtainedand the unit bearing the serial number(q - 1) is selected in the sample.



Let N = 122 and hence N' = 976 and d = 976 /122 = 8. Let the three-digit random numberchosen be 503 which lies between 0 and975. Dividing 503 by 8, the quotient is 62and hence the unit bearing serial number(62-1) = 61 is selected in the sample.



This method, suggested by Mathai (1954),consists of the selection of two randomnumbers which are combined to form onerandom number. One random number ischosen according to the first digit and otheraccording to the remaining digits of thepopulation size. If the number chosen is 0,the last unit is chosen. But if the numbermade up is greater than or equal to N, thenumber is rejected and the operation isrepeated.



Example:

Select a random sample of 11 households froma list of 112 households in a village by usingthe 3-digit random numbers given in columns 1

to 3, 4 to 6 and so on of the random numbertable and rejecting numbers greater than 112(also the number 000), we have for the samplebearing serial numbers 033, 051, 052, 099, 102,081, 092, 013, 017, 076 and 079. In the

above procedure, a large number ofrandom numbers is rejected. Hence, acommonly used device i.e., remainderapproach, is employed to avoid the rejection ofsuch large numbers.



The greatest three-digit multiple of 112 is 896.By using three digit random numbers as above,the sample will comprise of households withserial numbers 086, 033, 049, 097, 051, 052,

066, 107, 015, 106 and 020. In case the quotientapproach is applied, the 3-digit multiple of 112is 896 and 896/112 = 8. Using the same randomnumber and dividing them by 8, we have thesample of households with list numbers 025,

004, 020, 026, 006, 006, 092, 041, 085, 027 and086 with the replacement method and with listnumbers 025, 004, 020, 026, 006, 092, 041, 085,027, 086 and 042 without the replacementmethod.



There are many methods and algorithms havebeen developed for generation of randomnumbers. The most common ones are thefollowing:

Midsquare Method

Linear Congruential Generator (LCG)



Random Number Seed:

Virtually all computer methods of randomnumber generation start with an initial

random number seed. This seed is used togenerate the next random number and thenis transformed into a new seed value.



Midsquare method consists of following steps:

1. Start with an initial seed (e.g. a 4-digitinteger).

2. Square the number.

3. Take the middle 4 digits.

4. This value becomes the new seed. Dividethe number by 10,000. This becomes therandom number. Go to step 2.



Example:

x 0 = 5497

x 1: 54972 = 30217009 x 1 = 2170, R1 = 0.2170

x 2: 21702

= 04708900 x 2 = 7089, R2 = 0.7089 x 3: 70892 = 50253921 x 3 = 2539, R3 = 0.2539

Drawback: It’s hard to state conditions for picking initial

seed that will generate a “good” sequence.



Example:

“Bad” sequences:

x 0

= 5197 x 1: 51972 = 27008809 x 1 = 0088, R1 = 0.0088 x 2: 00882 = 00007744 x 2 = 0077, R2 = 0.0077 x 3: 00772 = 00005929 x 3 = 0059, R3 = 0.0059

x i = 6500

x i+1: 65002=42250000 x i+1=2500, Ri+1= 0.0088 x i+2: 25002=06250000 x i+2=2500, Ri+1= 0.0088



Proposed by Lehmer in 1951 Let Zi be the ith number (integer) in the sequence

Zi = (aZi-1+c)mod(m)

Zi

{0,1,2,…,m-1}

Where Z0 = initial seed

a = multiplier

c = incrementm = modulus

DefineRi = Zi /m (to obtain U[0,1) value)



Start with random seed Z0 Where Z0 < m = largest possible integer on machine

Recursively generate integers between 0 and M

Zi = (a Zi-1 + c) mod m

Use R = Z/m to get pseudo-random number(avoid 0 and 1)

When c = 0 Called Multiplicative CongruentialGenerator

When c > 0 Mixed LCG



Example:

16-bit machine

a = 1217

c = 0Z0 = 23

m = 215-1 = 32767

Z1 = (1217*23) mod 32767 = 27991

U1 = 27991/32767 = 0.85424

Z2 = (1217*27991) mod 32767 = 20134

U2 = 20134/32767 = 0.61446



What makes one LCG better than

another?

A full period (full cycle) LCGgenerates all m values before it cycles.



The period of an LCG is m (full period or fullcycle) if and only if

If q is prime that divides m, then q divides a-1c and m should be relatively prime, i.e., g.c.d.

= 1. (The only positive integer that divides both mand c is 1)

If 4 divides m, then 4 divides a-1. (e.g., a = 1,5, 9, 13,…)



m = 2B where B is the no. of bits in the machineis often a good choice to maximize the period.

If c = 0, we have a power residue ormultiplicative generator.

When c > 0 Mixed LCG Note that Zn = (aZn-1) mod(m) Zn = (anZ0)

mod(m).

If m = 2B, where B is the no. of bits in the

machine, the longest period is m/4 (best one can do) if and only if Z0 is odd

a = 8k + 3, k Z+ (5,11,13,19,21,27,…)



Never “invent” your own LCG. It will probably

not be “good.”

All simulation languages and many software

packages have their own PRN generator. Most usesome variation of a linear congruential generator.



Theoretical tests

Prove sample moments over entire cycle arecorrect

Lattice (plotted in 2 or 3 dimensions) structure ofLCGs “random numbers fall mainly in the planes”

(Marsaglia)

Spacing hyperplanes: the smaller, the better



Empirical tests Uniformity

Compute sample moments

Goodness of fit

Independence Gap Test

Runs Test

Poker Test

Autocorrelation Test



This test consists of following steps:

Divide “n” observations into “k ” equal intervals.

Do a frequency count f i, i=1,2,…,k

Compute

2

2 1

k

i

i

n f

k n

k



2

2

1

k i i

i i

f np

np

1 1, 2, 3, ,iwhere p and i k

k



Data Classification

ei = expected number of observations ininterval i = n pi = n / k, i = 1, 2, …, k

• • •

f 1 f 2 f k-1 f k

01

k

2

k

k

2

k

k

1

k

e1 e2 ek-1 ek

1



Repeat test “m” times with independent

samples of size “n”.

Do Not

Reject HOReject HO

k 1,2



Important things to note here are:

Choose the intervals evenly

Choose the intervals such that you would expect

each class to contain at least 5 or 10 observations pi should (ideally) be small (<.05)



Example

n = 1000

k = 10

pi = 1/k = 0.1 ei = npi = 100

2

2 1 6.28

k

i i

i

i

f np

np



i Intervals i f i i f np

2

i i f np

2

i i

i

f np

np

1 [0, .1) 87 -13 169 1.69

2 [.1, .2) 93 -7 49 0.49

3 [.2, .3) 113 13 169 1.694 [.3, .4) 106 6 36 0.36

5 [.4, .5) 108 8 64 0.64

6 [.5, .6) 99 -1 1 0.01

7 [.6, .7) 91 -9 81 0.818 [.7, .8) 95 -5 25 0.25

9 [.8, .9) 103 3 9 0.09

10 [.9, 1.] 105 5 25 0.25

1000 6.28



2

2 1 6.28

k

i i

i

i

f np

np

2

0.05 (9) 16.919

Do not reject H0 : U(0,1)



Runs of increasing and decreasing numbers

This test consists of following steps:

Assign + if xi < xi+1, assign - if xi>xi+1

Test Statistic: S = number of runs up AND down(sequence of + and -)

E(S) = (2N-1)/3

Var(S) = (16N-29)/90

Use Normal approximation for N>30



Example:

N = 15

S = 8 ~ N(µ = 29/3, 2 = 211/90)

Maximum value for S:N-1; negative dependency

Minimum value for S: 1; positive dependency

0.87 0.15 0.23 0.45 0.69 0.32 0.3 0.19 0.24 0.18 0.65 0.82 0.93 0.22 0.81

- + + + - - - + - + + + - +



Normal Curve Rejection Regions

REJECT (-ve)REJECT

Do Not

REJECT

H0 : Independence

HA : Dependence

Z/2-Z

/2

Reject H0 in favor of HA if

Z = (S - (2N-1)/3) / (16N-29/90)1/2 Z/2 or Z Z

/2





Documents

M03-13 Muhammad Khan Muneer (Presentation on Random Numbers)