Analysis & Design of Algorithms (CSCE 321)

Prof. Amr Goneid, AUC 1

Analysis & Design of Analysis & Design of AlgorithmsAlgorithms(CSCE 321)(CSCE 321)

Prof. Amr GoneidDepartment of Computer Science, AUC

Part 7. Divide & Conquer Algorithms

Divide & Conquer AlgorithmsDivide & Conquer Algorithms

The General Method Time Complexity Common Recurrence relations A General Divide & Conquer Recurrence Merge Sort Algorithm Quicksort Algorithm An Application of Quicksort: Convex Hull Selection by Partitioning Closest Pair Problem

1. The General Method1. The General MethodTo solve a problem using Divide and

Conquer:

• Divide it into smaller problems

• Solve the smaller problems recursively

• Combine their solutions into a solution

for the big problem

The General MethodThe General Method

The base case for recursion is a sub-problem of constant (and small) size.

Analysis of D&Q algorithms is done by constructing a recurrence relation

We can obtain T(n) in a closed form by solving the recurrence relation

The General MethodThe General Method For problem p, small(p) = true if p is simple. In this

case S(p) is the solution. Otherwise, Divide, Conquer, then Combine.

Divide & Conquer Algorithm

{ if small(p) return S(p);

else {

1. Divide p into smaller instances p1,p2,..,pk

2. Apply DQ to each pi

3. Return Combine(DQ(p1),DQ(p2),…,DQ(pk))}

2. Time Complexity2. Time Complexity

Let g(n) = time for S(p) when p is small

f(n) = time to divide p and combine

solutions

for p small T(n) = g(n)

otherwise

)()()(1

nfnTnTk

3. Common Recurrence Relations3. Common Recurrence Relations

A recursive algorithm that halves the input in one step then processes the two halves:

T(n) = 2T(n/2) + 1 for n >1 with T(1) given

A recursive algorithm that halves the input in one step then processes one half and ignores the other:

T(n) = T(n/2) + 1 for n >1 with T(1) given

Common Recurrence RelationsCommon Recurrence Relations A recursive algorithm that examines every element

before dividing and then ignores one half:

T(n) = T(n/2) + n for n >1 with T(1) given

A recursive algorithm that makes a linear pass through the input, before, during or after it is split into two halves then processes the two halves:

T(n) = 2 T(n/2) + n for n >1 with T(1) given

Example: Recursive Binary Search Example: Recursive Binary Search AlgorithmAlgorithmBinSearch (a , s , e , x)

{ if (s == e) // if small (p), i.e. one element{ if (x == as) return s ; else return -1 ;} // return S(p)

else { // two or more elements mid = (s + e)/2 ; // Divide if (x == amid) return mid ; else if (x < amid) // Conquer return BinSearch (a , s , mid-1 , x) ; // n/2 or else return BinSearch (a , mid+1 , e , x) ; } // n/2

}// No Combine

http://www.cosc.canterbury.ac.nz/people/mukundan/dsal/BSearch.html

Binary Search DemoBinary Search Demo

Worst Case of Binary SearchWorst Case of Binary Search

Let T(n) be the worst case number of comparisons

used on an array of n items. Hence

Now we seek a closed form solution for T(n)

11122 )(Twithnfor)/n(T)n(T

Worst Case of Binary SearchWorst Case of Binary Search

)n(logOnlog)n(Tand

m)m(THence

)(Twithmfor)m(T)m(T

mn,mn,nlogm

/n,.....,,,m,n

)(Twithnfor)/n(T)n(T

recurrencesecondary aobtain We

0101 Hence,

222102 Assume

4. A General D&Q Recurrence4. A General D&Q Recurrence

given. is F(1) that meansit given is T(c) if

Similarly, given. is F(0) that meansit given is T(1) If

RecurrenceSecondary

2,3,...c constants,areca,where

given.with or given 1

recurrence a have algorithms Q&DMany

)c(f)m(aF)m(F

)c(f)c(aT)c(THence

)m()cn(,)m()n(andcc/n

nlogmHence,...,m,cnLet

)c(Tcn)(Twithcnfor

,)n(f)c/n(aT)n(T

A General SolutionA General Solution

The secondary recurrence is solved as a first order linear recurrence. For T(1) i.e. F(0) given, we have as shown before:

given. T(c)for Also

given. T(1)for 1

)c/n(faa)c(T)n(T

)c/n(faa)(T)n(THence

)c(fa)(Fa

)c(fa)c(fa)(F)m(F

Very Common: Very Common: f(n) = b nf(n) = b nxx

and :arise cases Two Now,

given. T(c)for

given. T(1)for 1

:become solutions The

a(bn)c(Ta)n(T

a(bn)(Ta)n(T

n(fbn)n(f

Case(a): Case(a): f(n) = b nf(n) = b nxx, a = c, a = cxx

)1(log)()(

log)1()1()(

nbncTanT

nbnTabmnTanT

:given T(c)For

:given T(1)For

SummarySummary

given T(c)For

given T(1)For

:solutions the has Case Special

given. T(c)for

given. T(1)for

:become solutions The

:Recurrence Q&D Common

)1(log)()(

log)1()1()(

)()()(

)()1()(

nbncTanT

nbnTabmnTanT

abncTanT

abnTanT

bncnaTnT

Case(a): Case(a): f(n) = b nf(n) = b nxx, a = c, a = cxx

: andgiven T(1)For

:Search Binary :exampleFor

nlog)n(T

nlogbn)(Tabmn)(Ta)n(T

cahence,x,c,b,aHere

)(Twith)/n(T)n(T

Other examples forOther examples for a = c a = cxx

Recursive Power FunctionRecursive Power Function

The following function computes xn where x is of type double:

Pow (x , n)

if (n == 0) return 1.0; else if (n == 1) return x;

else if (n mod 2) return Pow (x*x , n / 2) * x;

else return Pow (x*x , n / 2);}

Show that the number of double arithmetic operations performed by this algorithm in the best case is T(n) = log n

SolutionSolution

In the best case n = 2m , so that n is always even and we

obtain the recurrence relation:

T(n) = T(n/2) + 1 , for n > 1 with T(1) = 0

Here, a = 1, b = 1, c = 2, x = 0

with the solution :

T(n) = bnx log n = log n

Recursive log FunctionRecursive log Function

A recursive function to compute the largest integer less than or equal

to log2 n. (Hint: for n > = 2 , the value of this function for n is one

greater than for n/2 ).

int Log2(int n)

{ if (n == 1) return 0;

else return (1 + Log2(n/2)); }

The number of integer divisions has the recurrence relation:

T(n) = T(n/2) + 1 , for n > 1 with T(1) = b = 0

with the solution :

T(n) = log n = O(log n)

Find the Number of Binary Digits in the Binary Representation of a Positive Decimal Integer using a D&Q recursive algorithm.

int BinDig (int n)

{ if (n == 1) return 1;

else return (1 + BinDig(n/2)); }

T(n) = number of arithmetic operations.

T(n) = T(n/2) + 2 for n > 1 with T(1) = 0

a = 1, b = 2, c = 2, x = 0

T(n) = 2 log n = O(log n)

Other Recurrences with a = cx

T(n) = 4 T(n/2) + n2 for n > 1 with T(2) = 1

Hence, T(n) = n2(log n – ¾) = O(n2 log n)

T(n) = 9T(n/3) + n2 for n > 1 with T(1) = 0

Hence, T(n) = n2 log3 n

T(n) = 16 T(n/4) + n2 for n > 1 with T(1) = 0

Hence, T(n) = n2 log4 n

Case(b): Case(b): f(n) = b nf(n) = b nxx, a , a c cxx

Example: a Fractal AlgorithmThe following function draws recursive squares (called a fractal star).

The drawing primitive is Box (x , y , n) which draws a square of side (n)

pixels centered at (x,y) :

void STAR( int x, int y, int n)

{ if (n > 1) { STAR(x-n , y+n , n/2); STAR(x+n , y+n , n/2);

STAR(x-n , y-n , n/2); STAR(x+n , y-n , n/2);

Box(x , y , n); }

Fractal AlgorithmFractal Algorithm

The recurrence relation is:

T(n) = 4 T(n/2) + 4 n for n > 1 with T(1) = 0

Here, a = 4, b = 4, c = 2, x =1 so that a cx

The general solution is:

)n(O)n(n)(nn)n(T

a(bn)(Ta)n(T

1412424 Hence,

given. T(1)for 1

Example: MaxMin Example: MaxMin Straightforward AlgorithmStraightforward Algorithm

// Return max and min values in locations s to e in array a[ ]MaxMin (a, s , e , mx, mn){ mx = mn = as; for i = s+1 to e

{if (ai > mx) mx = ai ;if (ai < mn) mn = ai ;

Invoke as MaxMin(a , 1 , n , mx , mn) ;Let T(n) be the number of comparisons of array elements.

Hence,T(n) = 2 ( n – 1) independent of the data in a[ ].

MaxMin D&Q AlgorithmMaxMin D&Q Algorithm

MaxMin (a, s , e , mx , mn){

if (s e-1) // one or two elementsif (as > ae) {mx = as; mn = ae;} else {mx = ae; mn = as;}

else {

m = (s+e)/2;MaxMin (a,s,m,mx,mn);Maxmin (a,m+1,e,mx1,mn1);mx = max(mx,mx1); mn = min(mn,mn1);

}}// The functions max/min(a,b) find the larger/smaller of (a,b). Each will

cost one comparison.

MaxMin AnalysisMaxMin Analysis

The recurrence relation is:

T(n) = 2 T(n/2) + 2 for n > 2 with T(2) = 1

Here, a = 2, b = 2, c = 2, x =0 so that a cx

The general solution is:

This is only 75% of the cost of the straightforward algorithm.

25112250 Hence,

given. T(2)for 222

n.)(n.)n(T

)(Ta)n(T

Other Examples withOther Examples with a a c cxx

Copy Binary Tree Algorithm

The following function receives a pointer (t) to a binary tree and

returns a pointer to an exact copy of the tree :

treeNode *CopyTree ( treeNode *t )

{ treeNode *p;

if (t)

{ p = new treeNode ;

p->left = CopyTree(t->left) ;

p->right = CopyTree(t->right);

p->info = t->info ; return p ; };

else return NULL ;

Copy Binary Tree Algorithm

In the worst case, the tree is full and the number of visits is:

T(n) = 2 T(n/2) + 1 for n > 1 with T(1) = 1Hence

T(n) = 2n – 1 = O(n)

Pre-Order Traversal of a Binary Tree

void preorder (tree t)

{ if (t)

{ visit (t); preorder (t -> left); preorder (t -> right); }

In the worst case, the tree is full and the number of visits is:

T(n) = 2 T(n/2) + 1 for n > 1 with T(1) = 1

T(n) = 2n – 1 = O(n)

Other Algorithms T(n) = T(n/2) + n2 for n > 1 with T(1) = 0

Hence T(n) = (4/3) (n2 – 1) T(n) = 2 T(n/2) + n2 for n > 1 with T(1) = 0

Hence T(n) = 2n(n – 1) T(n) = 8 T(n/2) + n2 for n > 1 with T(1) = 0

Hence T(n) = n2 (n – 1)

5. MergeSort5. MergeSort (a) Merging(a) Merging Definition:

Combine two or more sorted sequences of data into a single sorted sequence.

Formal Definition: The input is two sorted sequences,

A={a1, ..., an} and B={b1, ..., bm} The output is a single sequence, merge(A,B), which is a sorted permutation of

{a1, ..., an, b1, ..., bm}.

MergingMerging

merge(A, B) is

A, if B is empty,

B, if A is empty,

{a1}.merge({a2, ..., an}, B) if a1 <= b1, and

{b1}.merge(A, {b2, ..., bm}) otherwise.

The symbol "." stands for concatenation, for example,

{a1, ..., an}.{b1, ..., bm} = {a1, ..., an, b1, ..., bm}.

Merge AlgorithmMerge Algorithm

Merge(A,p,q,r)

copy Ap..r to Bp..r and set Ap..r to empty

while (neither L1 nor L2 empty)

{ compare first items of L1 & L2

remove smaller of the two from its list

add to end of A

concatenate remaining list to end of A

return A

array B

q+1L1 L2

ExampleExample

2 9 11 20 1 15 17 30

p q rB

ExampleExample

2 9 11 20 1 15 17 30

p q rB

ExampleExample

2 9 11 20 1 15 17 30

p q rB

ExampleExample

2 9 11 20 1 15 17 30

1 2 9 11

p q rB

ExampleExample

2 9 11 20 1 15 17 30

1 2 9 11 15

p q rB

ExampleExample

2 9 11 20 1 15 17 30

1 2 9 11 15 17

p q rB

ExampleExample

2 9 11 20 1 15 17 30

1 2 9 11 15 17 20

p q rB

ExampleExample

2 9 11 20 1 15 17 30

1 2 9 11 15 17 20 30

p q rB

Worst Case AnalysisWorst Case Analysis |L1| = size of L1, |L2| = size of L2

In the worst case |L1| = |L2| = n/2

Both lists empty at about same time, so everything has to be compared.

Each comparison adds one item to A so the worst case is

T(n) = |A|-1 = |L1|+|L2|-1 = n-1 = O(n)

comparisons.

(b) MergeSort Methodology(b) MergeSort Methodology

Invented by Von Neumann in 1945 Recursive Divide-And-Conquer

MergeSort MethodologyMergeSort Methodology

Divides the sequence into two subsequences of equal size, sorts the subsequences and then merges them into one sorted sequence

Fast, but uses an extra space

Methodology (continued)Methodology (continued)

Divide:

Divide n element sequence into two subsequences each of n/2 elements

Conquer:

Sort the two sub-arrays recursively Combine:

Merge the two sorted subsequences to produce a sorted sequence of n elements

AlgorithmAlgorithm

MergeSort (A, p, r) // Mergesort array A[ ] locations p..r {

if (p < r) // if there are 2 or more elements { q = (p+r)/2; // Divide in the middle // Conquer both MergeSort (A , p , q); MergeSort(A , q+1 , r); Merge(A , p , q , r); // Combine solutions}

Invoke as MergeSort(a , 1 , n)

Merge Sort ExampleMerge Sort Example8 3 2 9 7 1 5 4

8 3 2 9 7 1 5 4

3 8 2 9 1 7 4 5

2 3 8 9 1 4 5 7

1 2 3 4 5 7 8 9

http://www.cosc.canterbury.ac.nz/people/mukundan/dsal/MSort.html

http://coderaptors.com/?MergeSort

Merge Sort DemosMerge Sort Demos

Worst case Analysis of MergeSortWorst case Analysis of MergeSort

Merge will use n-1 comparisons operations so that:

)log(1log)(

2)2()1(2)(

2 n for giveson substituti Successive

)1()2()4/(2

)1(}1)2/()4/(2{2)(

0)1(1)2/(2)(

nnOnnnnTHence

nmnTnT

nnnTnT

TwithnnTnT

Performance of MergeSortPerformance of MergeSort

MergeSort divides the array to two halves at each level. So, the number of levels is O(log n)

At each level, merge will cost O(n) Therefore, the complexity of MergeSort is

O(n log n) In-Place Sort No (uses extra array) Stable Algorithm Yes

This technique is satisfactory for large data sets

6. QuickSort6. QuickSort(a) Methodology(a) Methodology

Invented by Sir Tony Hoare in 1962 Recursive Divide-And-Conquer algorithm Partitions array around a Pivot , then sorts parts

independently Fast, in-place sorting

(a) Methodology(a) Methodology

Left Right

Divide:Array a[p..r] is rearranged into a sub-array a[p..q-1], a pivot a[q] and a second sub-array a[q+1..r] such that each element of the first is <= pivot and each element of the second is >= pivot ( the pivot index (q) is computed as part of this process)

Methodology (continued)Methodology (continued)

Conquer:Sort the two sub-arrays recursively

Combine: The sub-arrays are already sorted in place, i.e., a[p..r] is sorted.

(b) Algorithm(b) AlgorithmQuickSort (a, p , r)// Sorts the elements a[p],..., a[r] which reside in// the global array a[1:n] into ascending order; // a[n+1] is considered to be defined and must be >= all the// elements in a[1:n].{

if (p < r ) { // If there are more than one elementq = partition (a , p , r+1); // partition around Pivot// q is the position of the pivotQuickSort (a , p ,q-1); // Sort left sub-arrayQuickSort (a , q+1 , r); // Sort Right sub-array

PartitioningPartitioning

int partition(a, p, r){

pivot = ap; // pivot is initially the first element i = p; j = r;do{

do {i++; } while (ai < pivot);do {j--; } while (aj > pivot);if(i < j) swap (ai , aj);

} while (i < j);ap = aj; aj = pivot;return (j);// j is the final location of the pivot

ExampleExample

5 3 1 9 8 2 4 7

2 3 1 4 5 8 9 7

1 2 3 4 5 7 8 9

http://www.cosc.canterbury.ac.nz/people/mukundan/dsal/QSort.html

http://coderaptors.com/?QuickSort

Quick Sort DemoQuick Sort Demo

(c)Analysis of QuickSort(c)Analysis of QuickSort

Partitioning will cost (n) comparisons. Sorting costs T(i) for left

sub-array + pivot, T(n-i) for right sub-array. Hence:

T(n) = n + T(i) + T(n-i) for n > 1 with T(1) = 0

i elements n-i elements

partition cost = n

Best CaseBest Case

Best Case when pivot has the middle value. In this case, the array is partitioned into two almost equal halves.

T(n) = 2T(n/2) + n for n > 1 with T(1) = 0

hence, T(n) = O(n log n)

n/2 elements n/2 elements

partition cost = n

Worst CaseWorst Case

Worst Case when pivot is the smallest element so that i = 1

T(n) = T(n-1) + n for n > 1

with T(1) = 0

Hence, T(n) = n(n+1)/2 -1= O(n2)

In this case, Quicksort is not really quick!

1 n-1 elements

partition cost = n

Summary ofSummary ofPerformance of QuickSortPerformance of QuickSort

Partitioning will cost O(n) at each level Best Case: Pivot has the middle value

Quicksort divides the array to two equal halves at each level. So, the number of levels is O(log n)Therefore, the complexity of QuickSort is O(n log n)

Worst Case: Pivot is the minimum (maximum) valueThe number of levels is O(n)Therefore, the complexity of QuickSort is O(n2)

In-Place Sort Yes Stable Algorithm No This technique is satisfactory for large data sets

Median-of-Three PartitioningMedian-of-Three Partitioning

If we take the elements in the first, last and middle locations in the array and then take the element middle in value between these three, this will be the median of three. In this case, there is always at least one element below and one element above. The worst case is therefore unlikely

A Better Choice: Random PivotA Better Choice: Random Pivot

A better solution that avoids the worst case of O(n2) is to use a randomizer to select the pivot element.

Pick a random element from the sub-array (p..r) as the partition element.

Do this only if (r-p) > 5 to avoid cost of the randomizer

A Better Choice: Random PivotA Better Choice: Random Pivot

void RQuickSort (a , p , r){

if (p < r ) { if ((r-p) > 5) { int m = rand()%(r-p+1) + p;

swap (ap , am); }q = partition(a , p , r+1);RQuickSort(a , p ,q-1);RQuickSort(a , q+1 , r);

Average Case AnalysisAverage Case Analysis

If the pivot is randomly chosen using a random number generator, we have (n-1) equally probable configurations:

{1,n-1},{2,n-2},..., {i, n-i},..{n-2,2},{n-1,1}

The probability of each configuration is 1/(n-1)

We obtain an average case cost given by:

})()({1

ninTiTn

nniTnTnAlso

nniTnTnAnd

nTHence

)1()(get 1st we from 2ndSubtract

)2()1()(2)1()2(

)1()(2)()1(

: thatrecall weequation, thissolve To

)log()( still is case average theSo,

log386.1log/log2ln2)(

equations, usefull from Also

0T(1) F(1) andT(n)/n F(n) that Recall

12)1()(

:solution with the

)/2()1()(

:as written becan This

nnennnnnTHence

See Internet Animation Sites:See Internet Animation Sites:

For Example: http://www.cs.ubc.ca/spider/harrison/Java/sorting-

demo.html

http://coderaptors.com/?All_sorting_algorithms

7. An Application of Quicksort:7. An Application of Quicksort:Convex HullConvex HullThe Problem:Given a set S of n points in 2-dimensional space, find

the smallest convex polygon containing all the points of

Convex HullConvex Hull The convex hull of a set of points is an elementary

problem in computational geometry. The hull gives a rough idea of the shape or extent

of a data set. It is an important step for various geometric

algorithms. For example, finding the diameter of a set of points.

A famous algorithm is the Graham Scan

Graham ScanGraham Scan Select lowest-y point (p) Compute angle i between

lines p-pi and the x-axis

Sort points according to Scan through the sorted set

starting at p: select 3 successive points if they form a left turn, move one point ahead if not, delete the 2nd point and move back repeat until we reach point p again

Graham ScanGraham Scan Triplet = (p,1,2) Left, move to (1)

Graham ScanGraham Scan Triple = (1,2,3)Right, Delete (2), back to (p)

Graham ScanGraham Scan Triplet = (p,1,3) left, Move to (1)

Graham ScanGraham Scan Triplet = (1,3,4) left, Move to (3)

Graham ScanGraham Scan Triplet = (3,4,5) left, move to (4)

Graham ScanGraham Scan Triplet = (4,5,6) Right, Delete (5), back to (3)

Graham ScanGraham Scan Triplet = (3,4,6) left, move to (4)

Graham ScanGraham Scan Triplet = (4,6,7) left, Move to (6)

Graham ScanGraham Scan Triplet = (6,7,p) left, Move to (7), 3rd point is (p)

Graham ScanGraham Scan Stop, Final Hull

Graham ScanGraham Scan

After the scan, all points remaining will lie on the hull To distinguish a left turn from a right turn, we compute the

signed area of the triangle formed by the point triplet:signed area = (1/2) det (A)

det (A) = the determinant of the matrix:

If the area is (+)ve, we have a left turn, if (-)ve , we have a right turn. If zero, the 3 points are co-linear.

111321

Graham Scan AlgorithmGraham Scan Algorithm

void GScan (pointList S) {

// p is the start point in S//(p1 , p2 , p3) is a point triplet// A is the signed area of the triangle// formed by the point tripletset p1 = p;

do { p2 = succ(p1);

if (p2 is not the last point) then p3 = succ(p2); else p3 = p;

A = Area(p1 , p2 , p3); // signed area// if p1,p2,p3 form a left turn

// move one point ahead; // if not, delete p2 and move back.

if (A >= 0.0) p1 = succ(p1); // left turn else { // right turn delete p2; p1 = pred(p1); } } while (!((p3 == p) && (A >= 0.0))); }

An implementation of the Graham scan algorithm is given in the Book, p. 187

It uses a doubly linked list to represent the sorted point list.

The scan algorithm runs in O(n) time. Since sorting takes O(n log n) time, the total

time of Graham scan algorithm is

O(n log n)

Graham Scan DemoGraham Scan Demo

http://www.cosc.canterbury.ac.nz/people/mukundan/cgeo/Gscan.html

http://www.cs.princeton.edu/courses/archive/fall08/cos226/demo/ah/GrahamScan.html

8. Selection by Partitioning8. Selection by Partitioning

The Selection Problem

Let A be an array of n distinct elements. Find

the kth smallest element.

Some Applications

Finding nearest neighbor

Finding the Median

Partition around median for closest pair problem

Traveling salesman problem

Selection by PartitioningSelection by Partitioning

Algorithms: Several algorithms exist. Sort array (e.g. by Mergesort), kth smallest element

will be located at kth location. Complexity is

O(n logn) Insert all elements in a minimum Heap, then remove

k elements from Heap. Last removed is the kth

smallest. Complexity is O(n logn) A more efficient algorithm uses partitioning around a

pivot like Quicksort. It is a Divide and Conquer algorithm with an average cost of O(n).

•Selection by PartitioningSelection by PartitioningLet A [1..n] be an array of n distinct elements. Find the kth smallest element

from location p through location r. Invoke as select (A, 1, n, k). A[n+1] is

considered to be defined and must be >= all the elements in A[1:n].

Type select (Type A[ ], p, r, k){

if (p == r) return A[p]; else{ //Choose a pivot (v) from A and partition A around (v) // Let A1, A2 be the elements of A which are respectively <,> v

q = partition (A, p, r+1); // new pivot location// A1 = elements from p to q-1, A2 = elements from q+1 to rL = q – p + 1;if (k = = L) return A[q];

else if (k < L) return select (A, p, q-1, k);else return select (A, q+1, r, k − L);

A1 v A21 j j+2 nq

cost =T(j)whenk <= j

i.e.j = k .. n-1

cost =T(n-j-1)when

k > j+1i.e.

j = 0..k-2

partitioncost = n

Analysis: The cost of partitioning is (n) Let |A1| = j and |A2| = n-j-1 The pivot position is q = j+1 with j = 0..n-1 We call select on A1 when k <= j, i.e. in the cases of j = k...n-1 We call select on A2 when k > j+1, i.e. in the cases of j = 0 .. k-2 Each case happens with equal probability (1/n) Hence the average cost will be:

2,)}1()({1

nnjnTjTn

)()1(2)(solution thehas recurrence This

0)1( with 2)1()(get 1st we from 2ndSubtract

)1()(2)1()1(

:nby sidesboth Multiply .2/ when is maximum The

k with decreases )(

k with increases )()1(

: thatrecall weequation, thissolve To

njTnTnAnd

njTnnTHence

Selection using a BSTSelection using a BST

Type select (Type A[ ], int n, int k){

Type Amin;binaryTree<Type, int> BST; for (int i = 0; i<n; i++) BST.insert(A[i],0);for (i = 1; i < k; i++) { Amin = BST.min(); BST.remove(Amin);} return BST.min();}

invoke as select (A, n, k)AnalysisInsert n elements O(n log n)Find and delete (k-1) minima O((k-1) log n)Find kth min O(log n)Total O((k+n) log n)

Other Selection AlgorithmsOther Selection Algorithms

Explore other selection algorithms at:http://en.wikipedia.org/wiki/Selection_algorithm

9. Closest Pair Problem9. Closest Pair Problem

The Closest Pair Problem in 1-DGiven a set S = {p1, p2, ..., pN} of N points on a line,

find the two points of S whose distance is smallest.

Closest Pair ProblemClosest Pair Problem

Simple solution

for each pi in S

for each pj in S and (i != j)

if (Dij < Dmin) { Dmin= Dij ; K = i; L = j;}

return K , L, Dmin

Time Complexity = O(N2) ,

Space Complexity = O(N)

Can we do better?

Solution with Sorting Sort the points according to their distance from the

origin. After the points are sorted, the closest pair will lie

next to each other somewhere in sorted order. Thus a linear-time scan through them completes the job.

Sorting will cost O(N log N) and the scan will cost O(N) so that the total cost is O(N log N) + O(N) = O(N log N).

Obviously, the use of sorting is much more efficient than the first method.

D&Q Algorithm in 1-D

Partition S, a set of points on a line, into two sets S1 and S2 at some point m such that for every point p S1 and q S2, p < q. Point m could be the median

Solve Closest Pair recursively on S1 and S2 Let {p1, p2} be the closest pair in S1 with 1 =

|p2 - p1| and {q1, q2} the closest pair in S2 with 2 =|q2 - q1|

Let be the smallest distance found so far:

= min (1, 2) The closest pair in S is either {p1,

p2} or {q1, q2} or some {p3, q3} with p3 S1 and q3 S2.Which one?

Let be the smallest separation found so

= min(|p2 - p1|; |q2 - q1|)

How many points of S1 or S2 can lie within of m?

Since is the distance between the closest pair in either S1 or S2, there can only be at most 1 point in each side.

Therefore, the number of distance computations required to check for a closest pair {p3, q3} with p3 S1 and q3 S2 is O(1).

Closest Pair Problem:Closest Pair Problem:D&Q Algorithm for 1-DD&Q Algorithm for 1-DClosest-Pair (S) If |S| = 1, = else If |S| = 2, = |p2 - p1|; else do the following steps:

1. Let m = median(S).2. Divide S into S1; S2 at m.

3. 1 = Closest-Pair(S1).

4. 2 = Closest-Pair(S2).

5. 12 is minimum distance across the cut = |p3 – q3|

6. Return = min(1 , 2 ,,12).

Closest Pair Problem:Closest Pair Problem:D&Q Algorithm for 1-DD&Q Algorithm for 1-DAnalysis: Finding the median point (selection problem) is O(N) Partitioning around the median is O(N) Computing in each recursive call is O(1) Hence, T(N) = 2T(N/2)+O(N) so that

T(N) = O(N log N)Source Material for 1-D and 2-D cases:http://www.eecs.tufts.edu/~tklein/closestpair.htmhttp://www.cs.mcgill.ca/~cs251/ClosestPair/http://www.cs.ucsb.edu/~suri/cs235/http://www.cs.ucf.edu/courses/cot5520/lectures.htmlAnimationhttp://www.cs.mcgill.ca/~cs251/ClosestPair/ClosestPairApplet/

ClosestPairApplet.html

Analysis & Design of Algorithms (CSCE 321)

Documents

CSCE 210 Data Structures and Algorithms

Prof. Amr Goneid, AUC1 Analysis & Design of Algorithms (CSCE 321) Prof. Amr Goneid Department of Computer Science, AUC Part R4. Disjoint Sets

CSCE 411 Design and Analysis of Algorithms - Texas A&M ...faculty.cs.tamu.edu/klappi/csce411-f11/csce411-set1.pdf · CSCE 411 Design and Analysis of Algorithms Andreas Klappenecker

CSCE 310J: Data Structures & Algorithms

1 Dr. Scott Schaefer Data Structures and Algorithms CSCE 221H

Analysis & Design of Algorithms (CSCE 321)

CSCE 411H Design and Analysis of Algorithms Set 8: Greedy Algorithms Prof. Evdokia Nikolova* Spring 2013 CSCE 411H, Spring 2013: Set 8 1 * Slides adapted

CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1

CSCE 411H Design and Analysis of Algorithms

Analysis & Design of Algorithms (CSCE 321)

CSCE 411 Design and Analysis of Algorithms Set 9: Randomized Algorithms Prof. Jennifer Welch Fall 2014 CSCE 411, Fall 2014: Set 9 1

CSCE 668 DISTRIBUTED ALGORITHMS AND SYSTEMS Fall 2011 Prof. Jennifer Welch CSCE 668 Self Stabilization 1

CSCE 3110 Data Structures & Algorithms

CSCE 668 DISTRIBUTED ALGORITHMS AND SYSTEMS Fall 2011 Prof. Jennifer Welch CSCE 668 Set 14: Simulations 1

CSCE 411 Design and Analysis of Algorithms

CSCE 668 DISTRIBUTED ALGORITHMS AND SYSTEMS