Divide-and-Conquer
Outline
Introduction Merge Sort Quick Sort Closest pair of points Large integer multiplication
Steps Divide the problem into a
number of subproblems. Conquer the subproblems by
solving them recursively. If the subproblem sizes are small
enough, however, just solve the subproblems in a straightforward manner.
Combine the solutions to the subproblems into the solution for the original problem.
Merge Sort
Steps in Merge Sort
Divide Divide the sequence to be sorted
into two halves. Conquer
Sort the two subsequences recursively using merge sort.
Combine Merge the two sorted subsequences
to produce the sorted answer.
Merge Sort: Example
36 86 45 12 84 77 51 64
36 86 45 12 84 77 51 64
36 86 45 12 84 77 51 64
12 45 77 84 51 64
36 8612 45 77 8451 64
36 8612 45 77 8451 64
divide
dividedivide
divide
36 86conquer
36 86combine
divide
45 12 conquer
combine
conquer
combine
84 77 51 64
Merge AlgorithmMERGE(A, p, q, r)
► initialize left and right arrays
n1 = q − p + 1
n2 = r − q
create array L[1 . . n1 + 1]
create array R[1 . . n2 + 1]
for i = 1 to n1
do L[i ] = A[p + i − 1]
for j = 1 to n2
do R[ j ] = A[q + j ]
L[n1 + 1]=∞
R[n2 + 1]=∞
i = 1
j = 1
► merge two arraysfor k = p to r
do if L[i ] ≤ R[ j ]
then A[k] = L[i ]
i = i + 1
else A[k] = R[ j ]
j = j + 1
Merge: ComplexityMERGE(A, p, q, r)
► initialize left and right arrays
n1 = q − p + 1
n2 = r − q
create array L[1 . . n1 + 1]
create array R[1 . . n2 + 1]
for i = 1 to n1
do L[i ] = A[p + i − 1]
for j = 1 to n2
do R[ j ] = A[q + j ]
L[n1 + 1]=∞
R[n2 + 1]=∞
i = 1
j = 1
► merge two arraysfor k = p to r
do if L[i ] ≤ R[ j ]
then A[k] = L[i ]
i = i + 1
else A[k] = R[ j ]
j = j + 1
(n) (1)
(n)
(n)
Merge Sort Algorithm
MERGE-SORT(A, p, r)
if p < r
then q = (p + r)/2MERGE-SORT(A, p, q)
MERGE-SORT(A, q + 1, r)
MERGE(A, p, q, r)
Merge Sort: Complexity
MERGE-SORT(A, p, r)if p < r
then q = (p + r)/2MERGE-SORT(A, p, q)MERGE-SORT(A, q + 1, r)MERGE(A, p, q, r)
If n1, T(n) = 2 T(n/2) + (n)If n=1, T(n) = (1)T(n) = (n lg n)
T(n/2) T(n/2)
(n)
(1)
Master TheoremLet a ≥ 1 and b > 1 be constants,
T(n) = aT(n/b) + f(n) be defined on the nonnegative integers, where f(n) = (nd)
Then T (n) can be bounded asymptotically as follows.
If a < bd, then T(n) = (nd). If a = bd, then T(n) = (nd lg n). If a > bd, then T(n) = (n logba ).
Solving Recurrence Equation
T(n) = 2 T(n/2) + (n) when n > 1 T(n) = (1) when n = 1
That is, T(n) = (n lg n)
… cn cn cn cn cn cn cn cn
… 2c 2c 2c2c2c
… 4c 4c
cn/2 cn/2
cn
lg n
Quicksort
Steps in Quicksort Divide
Partition (rearrange) the array A[p . . r] into two subarrays A[p . . q −1] and A[q +1 . . r] such that
each element of A[p . . q −1] A[q], A[q] each element of A[q + 1 . . r]
Compute the index q. Conquer
Sort A[p . . q−1] and A[q +1 . . r] by recursive calls.
Combine Since the subarrays are sorted in place,
nothing else need to be done.
Quicksort: Example
36 5572 61 77 4451 64
51 7236 44 77 6155 64
51 7236 44 77 6155 6444 7736 51 61 7255 6444 7736 51 61 7255 64
44 7736 51 64 7255 61
44 7736 51 64 7255 61
44 7736 51 64 7255 61
Quicksort Algorithm
QUICKSORT(A, p, r)
if p < r
then q = PARTITION(A, p, r)
QUICKSORT(A, p, q − 1)
QUICKSORT(A, q + 1, r)
Initial call:
QUICKSORT(A, 1, length[A]).
Partition: Example
36 5572 61 77 4451 64
36 5572 61 77 4451 64
72 5536 61 77 4451 6472 5536 61 77 4451 6451 5536 61 77 4472 64
51 5536 61 77 4472 64
51 5536 61 77 4472 64
51 5536 44 77 6172 64
51 7236 44 77 6155 64
Partition Algorithm
PARTITION(A, p, r)
x = A[r]
i = p − 1
for j = p to r − 1
do if A[ j ] ≤ x
then i = i + 1
exchange A[i] and A[ j]
exchange A[i + 1] and A[r]
return i + 1
PARTITION(A, p, r)
x = A[r]
i = p − 1
for j = p to r − 1
do if A[ j ] ≤ x
then i = i + 1
exchange A[i] and A[ j]
exchange A[i + 1] and A[r]
return i + 1
Partition: Complexity
O(1)
O(1)
O(n)
O(1)
Quicksort: Complexity
QUICKSORT(A, p, r)
if p < r
then q = PARTITION(A, p, r)
QUICKSORT(A, p, q − 1)
QUICKSORT(A, q + 1, r)
O(n)
T(n/?)
T(n/?)
Quicksort: Analysis
Worst case:
T(n) = T(n-1) + T(0) + (n)
= T(n-1) + (n)
T(n) (n2)
Best case:
T(n) = 2 T(n/2) + (n)
T(n) (n lg n)
Average-case Analysis
T(n) (n lg n)
Closest Pairs of Points
Problem
Given a set P of points in 2-dimensional space, find a closest pair of points.
Distance between two points are measured by Euclidean distance --(x2+y2)
Brute force method takes O (n2).
Closest Pair of Points: Example
divideagain
conquer
combine
Steps in Finding Closest Pair of Points Divide
Vertically divide P into PL and PR. PL is to the left and PR is to the right of the dividing line.
Conquer Recursively find closest pair of points among
PL and PR. dL and dR are the closest-pair distances
returned for PL and PR, and d = min(dL, dR). Combine
The closest pair is either the pair with distance d found by one of the recursive calls, or it is a pair of points with one point in PL and the other in PR.
Find a pair whose distance < d if there exists.
P
Closest point on the other side
PRPL
S
d d
Only 8 points need to be considered
P
Why 8 Points
PRPL
S
d/2
d/2
d/2
d/2
d/2
d/2
d d
Combine
Let PRY be the array of points in PR sorted by the value of Y.
For each point s in the d-width strip of PL, find a point t in PR such that dist(s, t) < d. To find t, look in the array PRY for 8
such points. If there is such a point, let d be the
dist(s, t).
Closest Pair of Points: Algorithm
Let px and py be the points in P, sorted by the value of x and y, respectively.
ClosestPair(px, py)if (|px|<=3) thenreturn(BruteForceCP(px))Using px, find the line L which vertically divides P into 2 halves.Using the dividing line L, split px into pxL and pxR. Using the dividing line L, split py into pyL and pyR.dL = ClosestPair(pxL, pyL)dR = ClosestPair(pxR, pyR)d = min(dL, dR)return ( combineCP(pyL, pyR, d) )
Closest Pair of Points: ComplexityLet px and py be the points in P, sorted by the value of x
and y, respectively.ClosestPair(px, py)
if (|px|<=3) thenreturn(BruteForceCP(px))Using px, find the line L which vertically divides P into 2 halves.Using the dividing line L, split px into pxL and pxR. Using the dividing line L, split py into pyL and pyR.dL = ClosestPair(pxL, pyL)dR = ClosestPair(pxR, pyR)d = min(dL, dR)return ( combineCP(pyL, pyR, d) )
O(1)
O(n)
2T(n/2)
O(1)
O(1)
O(n)
Solving the recurrence equation
T(n) = 2 T(n/2) + O(n)
T(n) Ο(n log n)
Large Integer Multiplication
Dividing Multiplication
x = (x1*2n/2 + x0) y = (y1*2n/2 + y0)
x y = (x1*2n/2 + x0) (y1*2n/2 + y0)
= x1 y1*2n + (x0 y1+ x1 y0)*2n/2 + x0 y0
(x1+ x0) ( y1+ y0) = x1 y1+ x0 y1+ x1 y0 + x0 y0
= (x0 y1+x1 y0)+(x1 y1+ x0 y0)
(x0 y1+x1 y0) = (x1+ x0)( y1+ y0) - (x1 y1+ x0 y0)
Algorithm
recMul(x, y)
n = maxbit(x, y)
if (n<k) then return (x*y)
x1 = x div 2n/2 x0 = x mod 2n/2
y1 = y div 2n/2 y0 = y mod 2n/2
p = recMul(x1+x0, y1+y0)
q = recMul(x1, y1)
r = recMul(x0, y0)
return(q*2n+(p-q-r)*2n/2+r)
Complexity
recMul(x, y)
n = maxbit(x, y)
if (n<k) then return (x*y)
x1 = x div 2n/2 x0 = x mod 2n/2
y1 = y div 2n/2 y0 = y mod 2n/2
p = recMul(x1+x0, y1+y0)
q = recMul(x1, y1)
r = recMul(x0, y0)
return(q*2n+(p-q-r)*2n/2+r)
O(1)
3T(n/2)
O(1)
Solving the recurrence equation
T(n) = 3 T(n/2) + O(n)
T(n) Ο(n log2 3)
T(n) Ο(n 1.59)