RandomizedAlgorithms Fundamentals of Algorithmics.Gilles Brassard and Paul Bratley.Chapter 10. Randomiized Algorithms SC440/CSC401/CPE4011

Randomized Algorithms

Fundamentals of Algorithmics. Gilles Brassard and PaulBratley. Chapter 10.

RandomiizedAlgorithms

SC440/CSC401/CPE401 1

Introduction (‘Traditional’ algorithms)

Traditional algorithms have the following properties:••

They are always correctThey are deterministic --- there may be more than one correct output, but the same instance of a problem always produces the same outputThey are always precise --- the answer is not given as a range

•

• Each of them operates at the same efficiency for thesame instance of a problem

→ 224 → Prime factor


SC440/CSC401/CPE401 2

Introduction (‘Randomized’ algorithms)

Randomized (probabilistic) algorithms can be• Nondeterministic --- they can make random but

correct decisions : the same algorithm may behave differently when it is applied twice to the sameinstanceNot very is given,

of a problem.precise sometimes --- usually the more the better precision can be obtained

• time

• operating at different efficiencies for theruns.

sameinstance of a problem in different

→24 → Prime factor


SC440/CSC401/CPE401 3

2 or 3

• Incorrect sometimes: hope we have a high, knownprobability of being correct.

Nonterminating sometimes : do not produce an answer at all (hope fore a low probability for that)

•

because

• A randonmized algorithm is one that makes randomchoices during the execution


SC440/CSC401/CPE401 4

Why do we need randomized (probabilistic algorithms)

• When an algorithm is confronted by a choice, it issometimes preferable to choose a course of action at random, rather than spending time to work out which alternative is the best.

Sometimes we do not have a better method than making random choices

•


SC440/CSC401/CPE401 5

Classes of randomized (probabilistic) algorithms

1. Numerical probabilistic algorithms --- give anapproximation to the correct answer.

Monte Carlo Algorithms --- always give an answer, but there is a probability of being completely wrong.

2.

3. Las Vegas Algorithms --- sometimes fail to give ananswer, but if an answer is given, it is correct.

These are what we cover in this chapter.


SC440/CSC401/CPE401 6

Expected versus average time

• The average time of a deterministic algorithm refers tothe average time taken by the algorithm when eachpossible instance of a given size is considered equallylikely. E.g sorting 3 integers

T123

T132

T213

T231

T312

T321

1,

1,

2,

2,

3,

3,

2,

3,

1,

3,

1,

2,

3

2

3

1

2

1

A sorting algorithm

+

6


SC440/CSC401/CPE401 7

• The expected time of a probabilistic algorithm isdefined on each individual instance. It is the meantimeover

thatand

it would takeover again.

to solve the same instance

T1

T2

T3

T4

T5

T6

1, 2, 3 A sorting algorithm Computethe mean


SC440/CSC401/CPE401 8

• The worst case expected time of a probabilisticalgorithm refers to the expected time taken by the worst possible instance of a given size, not the time incurred if the worst possible probabilistic choices are unfortunately taken.


SC440/CSC401/CPE401 9

Numerical probabilistic algorithms

• For certain real life problems, computation of anexact solution is not possible, maybe– because of uncertainties in the experimental data

to be usedBecause a digital computer cannot represent an irrational number exactlyBecause a precise answer will take too long to compute

–

–

• A numerical probabilistic algorithm will give anapproximate answer (a confidence interval can be provided).The precision of the answer increases when more time is given to the algorithm to work on

•


SC440/CSC401/CPE401 10

Buffon’s needle

18thIn the century, Georges de Buffon, proved that ifyou throw a needle at random (in a random positionand at a random angle, with uniform distribution) on a floor made of planks of constant width, if the needleis exactly half as long as the planks in the floor are wide and if the width of the cracks between the planksis zero, the probability that the needle will fall acrossa crack is 1/π.


SC440/CSC401/CPE401 11

Monte Carlo Algorithms

• There are problems for which no efficient algorithm isknown, and approximate solutions do not make sense.A Monte Carlo algorithm always gives an answer but occasionally makes a mistake.But it finds a correct solution with high probability whatever the instance is processed, i.e. there is no instance on which the probability of error is high.However, no warning is usually given when the algorithm gives a wrong solution.

•

•

•

Example: to find the mode (most common element) in abag of values. Pick an element at random: theprobability that it is the mode is high, but there is a


SC440/CSC401/CPE401 17

non-zero probability that itis completely wrong. Making several random choices and combining the results improves theprobability of correctness.


SC440/CSC401/CPE401 18

Primality testingThe problem is to decide whether a given odd integer is prime or composite.The problem is important for public-key cryptography.

•

•

A Naive methodGiven an input number N, we check whether it isdivisible by any integer greater than 1 and less than or equal to the square root of N. If the answer is NO, then N is a prime, otherwise not. This relies on the not so trivial fact that if there are any factors other than 1 or N, theycannot all be greater than this square root.

Composite

Divisible byN →prime


SC440/CSC401/CPE401 19

Probabilistic tests

Most popular primality tests are probabilistic tests. In a certain sense, those tests are not really primality

•

tests -- they do not determine with certainty whethernumber is prime or not.The basic idea is as follows:

a

•1.2.

Randomly pick a number x called a witness.Check some formula involving x and the given number N. If the formula is no fit, then N is a composite number and the test stops.Repeat step 1 unless the required certainty is achieved.

3.

After several iterations, if N is notcomposite number, then it can beprime.

found to be adeclared probably


SC440/CSC401/CPE401 20

• The simplest probabilistic primality test is the Fermatprimality test.It is sometimes used if a rapid screening of numbers•is needed, for instance in the key generation phase ofthe RSA public key cryptographical algorithm.

Fermat’s little Theorem

If n is a prime, then

an-1 mod n = 1

For any integer a such that 1 ≤ a ≤ n - 1.

E.g. n

n n

===

7, a =8, a =

1 or 2 or 3 or … 6272, we have mod 8 = 0 ≠ 1

2910, a = 2, we have mod 10 = 2 ≠ 1RandomiizedAlgorithms

SC440/CSC401/CPE401 21

The contrapositive:If there is a value a such that 1 ≤ a ≤ n – 1 and an-1 mod n ≠

1, then n is not a prime.

Fermat (n)a = uniform(1, n-1);

(an-1if mod n ≠ 1) then return “n is composite”

else return “n is prime”

an-1• However, there are composite numbers such that414mod n = 1. For example, n = 15, mod 15 = 1.

• The Miller-Rabin test is a more sophisticated variant ofthe Fermat test but with a lower error rate; it is often the method of choice.


SC440/CSC401/CPE401 22

• Suppose n > 4 is an odd integer which we want to testfor primality.

Choose a random integer a with 2 ≤ a ≤ n - 2.

Find s and t such that t is odd and n - 1 = 2st .••• 2rtIf (at mod n = 1) or (a mod n = n - 1) for at least one r

= 0, ..., s-1, then n is declared "prime";If not, then n is definitely composite.•

MillerRabin(n, k)for (i = 1 to k do)

a = uniform(2, n – 2);if (Btest(a, n) == false) then return

return “prime”“composite”


SC440/CSC401/CPE401 23

Btest(a, n)s = 0; t = n – 1;Repeat

s = s + 1; t = t/2;Until t is odd;

atx = mod n;If (x == 1 or x == n – 1) then returnFor (i = 1 to s – 1)

true

x2x = mod n;if (x == n – 1) then return true

Return false

The Miller-Rabin algorithm always answer when n > 4 is a prime.

• returns the correct

• When n > 4 is an odd composite, each call on Btesthas probability at most ¼ of making a mistake.


SC440/CSC401/CPE401 24

Las Vegas Algorithms

• Las Vegas algorithms make probabilistic choices to help guide them more quickly to a correct solution.

• Unlike Monte Carlo algorithms, they never return wrong answer.Two main categories of Las Vegas algorithms.

a

•

First category• The first kind is often used when a known

deterministic algorithm to solve a problem runs much faster on the average than in the worst case, e.g. quicksort. [See previous lecture]


SC440/CSC401/CPE401 26

Second category

• The second kind of Las Vegas algorithms now and then make choices that bring the problem solving process to a dead end. For example,

Eight-queen problem

1st• When we use backtracking, we got the solutionafter building 114 nodes of the 2057 nodes in the state-space tree.

• A greedy Las Vegas algorithm places the queens randomly on successive rows, taking care not to put a queen in an illegal position.Being a greedy algorithm, we do not backtrack. We either succeed if we manage to place all thequeens, or we run into a dead end --- failure.

••


SC440/CSC401/CPE401 33

Using a computer, it is calculated that the probability ofsuccess, p, is approximately 0.1293.


SC440/CSC401/CPE401 34

QQ

Q

Q

QQ

Q

After failing, try, try again

• If we fail to place all the queens in one attempt, werestart from the beginning.

• The expected number of attempts∞

until success is

∑ i * P[ X = i]i =1

E[X] =

= ∞

∑ i *(1− p)i -1 p

i =1

1 p *

1 − p∞

p ∑ i * (1− p)i= = =2

1 − p p1 − p pi =1

So for the randomized 8-queen,attempts to reach success.

we expect 8


SC440/CSC401/CPE401 35

Finding Hamiltonian paths

• Given a graph, is there a path, any path, that passesthrough all the vertices of the graph exactly once. Such a path is called Hamiltonian path.

There are n vertices in graph G.

Try to build the path vertex by vertex.

•

•

• Start from an arbitrary vertex, v0 in G and select a neighbour of v0 to get a path v0v1

randomly.

• Continue to randomly choose the next vertexstop when all vertices are chosen

and


SC440/CSC401/CPE401 36

• choose the next vertex randomly from the neighboursof the current vertex which are not already on the path


SC440/CSC401/CPE401 37

• If at a certain vi, all its neighbours are already on thepath, we pick one neighbour, vj, change the path fromv0 v1 ,…,vjvj+1,…, vi-1vi to v0 v1 ,…,vjvivi-1,…,vj+1 and

continue to choose the next vertex for the path

v5v0The path built:v0 v1 v2v3v4 v5

v3v => v v v v v v (v = v )4 0 1 2 5 4 3 j 2

v1 v2What is the new path ifvj = v0?


SC440/CSC401/CPE401 38

Randomized_hamiltonian_path(G)v0 = Random vertex in G;

i = 0;

{

do { N =if N

N(vi) – {v0, …,

is not empty {

i = i + 1;

vi-1}

vi = Random vertex in N; }≤ j < i - 1)else if (there is vj is in N(vi) for 0 {

}(v0, …, vi) = (v0, …,

return no_solution;

vjvivi-1, …, vj+1);

else} while (i < |V|return (v0, …,


- 1);vi); }

SC440/CSC401/CPE401 39

• If randomized_hamiltonian_path returns true, aHamiltonian path for graph G has indeed been found.

It is possible that the algorithm runs into a dead end and fails to find a path, although such a path exists.

The algorithm is good at finding a Hamiltonian path in large graphs.

The algorithm is almost always successful for

•

•

•graphs of sufficiently large minimum degree


SC440/CSC401/CPE401 40

Documents

RandomizedAlgorithms Fundamentals of Algorithmics.Gilles Brassard and Paul Bratley.Chapter 10. Randomiized Algorithms SC440/CSC401/CPE4011