Algorithms: Design and Analysis

Summer School 2013 at VIASM: Random Structures and Algorithms

Lecture 2: Divide and Conquer algorithms

Phan Thị Hà Dương1

Lecture 2: Divide and Conquer algorithms

0. Introduction1. Mergesort2. Quicksort3. *Strassen’s algorithm for matrix

multiplication

0. Introduction

• Divide: Divide the problem to subproblems.• Conquer: Solve recursively subproblems.• Combine: Use results of subproblems and

combine them to obtain result of initial problem.

• Determine Threshold: for which problem, the algorithm return directly result without dividing to smaller problems.

1. Merge sort algorithm

• Divide: Divide the n-element sequence to be sorted into two subsequences of n/2 elements each.

• Conquer: Sort the two subsequences recursively using merge sort.

• Combine: Merge the two sorted subsequences to produce the sorted answer.

MERGE-SORT(A, p, r) 1. if p < r2. then q← (p+r)/2 ⌊ ⌋3. MERGE-SORT(A, p, q)4. MERGE-SORT(A, q + 1, r) 5. MERGE(A, p, q, r)

Example

Analyzing Merge sort algorithms

• Divide: D(n) = Θ(1).• Conquer: solve two subproblems, each of size n/2, which

contributes 2T (n/2) to the running time.• Combine: the MERGE procedure on an n-element subarray

takes time Θ(n), so C(n) = Θ(n).

T(n) = θ(1) if n = 12 T(n/2) + θ(n) if n >1

And then T(n) = O (n lg n)

2. Quick Sort algorithms

• Quick Sort is a sorting algorithm whose worst-case running time is Θ(n2).

• In spite of this slow worst-case running time, Quick Sort is often the best practical choice for sorting because it is remarkably efficient on the average: its expected running time is Θ(n lg n), and the constant factors hidden in the Θ(n lg n) notation are quite small.

• It also has the advantage of sorting in place, and it works well even in virtual memory environments.

• Randomized Quick Sort has an expected running time of 0(n lg n): really efficient.

Idea of Quick Sort Algorithm

• Divide: Partition (rearrange) the array A[p.. r] into two (possibly empty) subarrays A[p.. q - 1] and A[q + 1..r] such that each element of A[p.. q - 1] is less than or equal to A[q], which is smaller than each element of A[q + 1.. r]. Compute the index q as part of this partitioning procedure.

• Conquer: Sort the two subarrays A[p.. q -1] and A[q +1.. r] by recursive calls to quicksort.

• Combine: Since the subarrays are sorted in place, no work is needed to combine them: the entire array A[p.. r] is now sorted.

Quick Sort

QUICKSORT(A, p, r) 1 if p < r2 then q ← PARTITION(A, p, r) 3 QUICKSORT(A, p, q - 1) 4 QUICKSORT(A, q + 1, r)

Partition algorithm

PARTITION(A, p, r) 1 x←A[r]2 i←p-1 3 For j←p to r-14 do if A[j] ≤ x 5 then i ← i + 16 exchange A[i] ↔ A[j] 7 exchange A[i + 1] ↔ A[r]8 return i+1The running time of PARTITION on the subarray A[p.. r] is Θ(n),

where n = r - p + 1

Analyzing Quicksort: worst-case partitioning

• T(n) = T(n-1) + T(0) + Θ(n) = T(n-1) + Θ(n) = Θ(n^2)

Question: In each case we have the worst-case ?

Best-case partitioning

• T(n) ≤ 2T(n/2) + Θ(n) = Θ(n lg n)

Balanced partitioning

T(n) ≤ T(9n/10) + T(n/10) + cn

3. Strassen’s algorithm for matrix multiplication

Presents Strassen's remarkable recursive algorithm for multiplying n × n matrices, which runs in Θ(n^lg 7) = O(n2.81) time. For sufficiently large values of n, therefore, it outperforms the naive Θ(n3) matrix-multiplication algorithm MATRIX-MULTIPLY from

Idea of Strassen’s algorithm

We wish to compute the product C = AB, where each of A, B, and C are n × n matrices. Assuming that n is an exact power of 2, we divide each of A, B, and C into four n/2 × n/2 matrices, rewriting the equation C = AB as follows

r = ae + bg s = af + bht = ce + dg u = cf + dh

Analyzing of Strassen’s algorithm

• Naïve Algorithm T(n) = 8 T(n/2) + Θ(n^2)

• Strassen’s algorithm:

T(n) = 8 T(n/2) + Θ(n^2)

= Θ(n^lg 7)

= O(n ^2,81)

Home Works

1. Implement on C (or C++, or Pascal, or Java) the algorithms of sorting: Insertion Sort, Heap Sort, Merge Sort and Quick Sort. Run your program on inputs of arrays of 8 elements, of 16 elements, of 12 elements.

2. Implement the Strassen’s algorithm, and run your program on:

1. an input of two matrix of size 42. an input of two matrix of size 8