Randomized Algorithms-2013

7/27/2019 Randomized Algorithms-2013

1/30

Introduction to Randomized

Algorithms

Srikrishnan Divakaran

DA-IICT


2/30

Lecture Outline

Preliminaries and Motivation

Analysis of the Randomized hiring problem

Analysis of Randomized Quick Sort

References


3/30

Preliminaries and Motivation


4/30

Quick Sort

Select: pick an arbitrary element xin S to be the pivot.

Partition: rearrange elements so

that elements with value less than xgo to List L to the left of x andelements with value greater than xgo to the List R to the right of x.

Recursion: recursively sort the listsL and R.


5/30

Worst Case Partitioning of

Quick Sort


6/30

Best Case Partitioning of Quick

Sort


7/30

Average Case of Quick Sort


8/30

Randomized Quick Sort

Randomized-Partition(A , p, r)1. iRandom(p, r)

2. exchangeA[r]A[i]

3. returnPartition(A,p, r)

Randomized-Quicksort(A , p, r)1. ifp < r

2. thenqRandomized-Partition(A,p, r)

3. Randomized-Quicksort(A,p , q-1)

4. Randomized-Quicksort(A, q+1, r)


9/30

Randomized Quick Sort

Exchange A[r] with an element chosen at random from A[pr] inPartition.

The pivot element is equally likely to be any of input elements.

For any given input, the behavior of Randomized Quick Sort isdetermined not only by the input but also by the random choices ofthe pivot.

We add randomization to Quick Sort to obtain for any input theexpected performance of the algorithm to be good.


10/30

Deterministic Algorithms

Goal: Prove for all input instances the algorithm solves theproblem correctly and the number of steps is bounded by a

polynomial in the size of the input.

ALGORITHMINPUT OUTPUT


11/30

Randomized Algorithms

In addition to input, algorithm takes a source of random numbers

and makes random choices during execution;

Behavior can vary even on a fixed input;


RANDOM NUMBERS


12/30

Las Vegas Randomized

Algorithms

Goal: Prove that for all input instances the algorithm solves theproblem correctly and the expected number of steps is bounded by

a polynomial in the input size.

Note: The expectation is over the random choices made by the

algorithm.


RANDOM NUMBERS


13/30

Probabilistic Analysis of

Algorithms

Input is assumed to be from a probability distribution.

Goal:Show that for all inputs the algorithm works correctly and formost inputs the number of steps is bounded by a polynomial in thesize of the input.

ALGORITHMRANDOM

INPUT

OUTPUT

DISTRIBUTION


14/30

Monte Carlo Randomized

Algorithms

Goal:Prove that the algorithm with high probability solves the problem correctly;

for every input the expected number of steps is bounded by apolynomial in the input size.

Note: The expectation is over the random choices made by the

algorithm.


RANDOM NUMBERS


15/30

Monte Carlo versus Las Vegas

A Monte Carlo algorithm runs produces an answer that is correct

with non-zero probability, whereas a Las Vegas algorithm always

produces the correct answer.

The running time of both types of randomized algorithms is arandom variable whose expectation is bounded say by a polynomial

in terms of input size.

These expectations are only over the random choices made by the

algorithm independent of the input. Thus independent repetitions ofMonte Carlo algorithms drive down the failure probability

exponentially.


16/30

Motivation for Randomized

Algorithms

Simplicity;

Performance;

Reflects reality better (Online Algorithms);

For many hard problems helps obtain better complexity bounds

when compared to deterministic approaches;


17/30

Linearity of Expectation

If X1, X2, , Xn are random variables, then

n

i

XiEn

i

XiE

1

][

1


18/30

Hiring Problem


19/30

Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

A Deterministic Algorithm for

hiring


20/30

Analysis of the Deterministic Hiring

Algorithm

Ci cost for interviewing a candidate

(typically low)

Ch

cost for hiring a candidate (typically

high)

M is the number of people hired

O(ci * h + ch * m) is the total costassociated with the deterministic hiring

algorithm.


21/30

Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

A Randomized Algorithm for

hiring


22/30

Analysis of the Randomized

Hiring AlgorithmLet X denote the number of times we hire an office assistant.

E[X] = x = 1 to n x * Pr { X = x}

Let Xi

= I {candidate i is hired}

= 1 if the candidate i is hired and 0 otherwise

Notice X = X1 + X2+ + Xn

E[Xi] = 1/i

By linearity of expectation, we get

E[X] = E[X1 + X2+ + Xn ] = i = 1 to n E [Xi] = i = 1 to n 1/I = ln n.


23/30

Analysis of Randomized QuickSort


24/30

Notation

Rename the elements of A as z1, z2, . . . , zn, with zi being the ith

smallest element (Rank i).

Define the set Zij = {zi , zi+1, . . . , zj } be the set of elements between

zi and zj, inclusive.

106145389 72

z1z2 z9 z8 z5z3 z4 z6 z10 z7


25/30

Expected Number of Total

Comparisons in PARTITION

Let Xij = I {zi is compared to zj }

Let X be the total number of comparisons performed by the

algorithm. Then

][XE

by linearity

of expectation

1

1 1

n

i

n

ijijXE

1

1 1

n

i

n

ijijXE

1

1 1

}Pr{n

i

n

ij

ji ztocomparedisz

indicatorrandom variable

1

1 1

n

i

n

ij

ijXX

The expected number of comparisons performed by the algorithm is


26/30


Observation 1: Each pair of elements is compared at most once

during the entire execution of the algorithm

Elements are compared only to the pivot point!

Pivot point is excluded from future calls to PARTITION

Observation 2: Only the pivot is compared with elements in bothpartitions

z1z2 z9 z8 z5z3 z4 z6 z10 z7

106145389 72

Z1,6= {1, 2, 3, 4, 5, 6} Z8,9 = {8, 9, 10}{7}

pivot

Elements between different partitions are never compared


27/30


Case 1: pivot chosen such as: zi < x < zj

ziand zj will never be compared

Case 2: zi orzj is the pivot

zi and zjwill be compared

only if one of them is chosen as pivot before any other element

in range zito zj

106145389 72

z1z2 z9 z8 z5z3 z4 z6 z10 z7

Z1,6= {1, 2, 3, 4, 5, 6} Z8,9 = {8, 9, 10}{7}

Pr{ }?i jz is compared to z


28/30

Expected Number of Comparisons

in PARTITION

1

1 1

}Pr{][n

i

n

ij

ji ztocomparediszXE

Pr {Zi is compared with Zj}

= Pr{Zi or Zj is chosen as pivot before other elements in Zi,j} = 2 / (j-i+1)

1 1 1 1

1 1 1 1 1 1 1

2 2 2[ ] (lg )

1 1

n n n n i n n n

i j i i k i k i

E X O nj i k k

= O(nlgn)


29/30

1. Amihood Amir, Kargers Min-cut Algorithm, Bar-Ilan University, 2009.

2. George Bebis, Randomizing Quick Sort, Lecture Notes of CS 477/677:

Analysis of Algorithms, University of Nevada.

3. Avrim Blum and Amit Gupta, Lecture Notes on Randomized Algorithms, CMU,

2011.

4. Rada Mihalcea, Quick Sort, Lecture Notes of CSCE3110: Data Structures,

University of North Texas, http://www.cs.unt.edu/~rada/CSCE3110.

5. Rajeev Motwani and Prabhakar Raghavan, Randomized Algorithms,

Cambridge University Press, 1995.

6. Prabhakar Raghavan, AMS talk on Randomized Algorithms, Stanford

University.

References

1000 Wh t Wh t N t
http://www.cs.unt.edu/~rada/CSCE3110http://www.cs.unt.edu/~rada/CSCE3110http://www.cs.unt.edu/~rada/CSCE3110http://www.cs.unt.edu/~rada/CSCE3110http://www.cs.unt.edu/~rada/CSCE3110http://www.cs.unt.edu/~rada/CSCE3110http://www.cs.unt.edu/~rada/CSCE3110http://www.cs.unt.edu/~rada/CSCE3110http://www.cs.unt.edu/~rada/CSCE3110http://www.cs.unt.edu/~rada/CSCE3110http://www.cs.unt.edu/~rada/CSCE3110


30/30

1000 Whats, What Nuts

&

Wall (Face Book) Nuts?DA-IICTs Wall Broken (i.e.,

No Boundaries)

Documents

Randomized Algorithms-2013