Lecture 5

Lecture 5

2

Quiz 2Input: A nonnegative integer nS:=0for i:=0 to n do

S:=S + i*ireturn S• What Doest this Algorithm Compute• What is its basic operation• How many times is the basic operation executed?• What is the efficiency class of this algorithm• Suggest an improvement or a better algorithm and

its efficiency class

3

• Divide and Conquer

• Merge Sort

• Quick Sort

4

Divide and Conquer1. Base Case, solve the problem directly if

it is small enough

2. Divide the problem into two or more similar and smaller subproblems

3. Recursively solve the subproblems

4. Combine solutions to the subproblems

5

Divide and Conquer - SortProblem: • Input: A[left..right] – unsorted array of integers • Output: A[left..right] – sorted in non-decreasing

order

6

Divide and Conquer - Sort1. Base case

at most one element (left ≥ right), return

2. Divide A into two subarrays: FirstPart, SecondPart Two Subproblems:

sort the FirstPart sort the SecondPart

3. Recursively sort FirstPart sort SecondPart

4. Combine sorted FirstPart and sorted SecondPart

7

Overview

• Divide and Conquer

• Merge Sort

• Quick Sort

8

Merge Sort: Idea

Merge

Recursively sort

Divide intotwo halves

FirstPart SecondPart

FirstPart SecondPart

A

A is sorted!

9

Merge Sort: Algorithm

Merge-Sort (A, left, right)

if left ≥ right return

else

middle ← (left+right)/2

Merge-Sort(A, left, middle)

Merge-Sort(A, middle+1, right)

Merge(A, left, middle, right)

Recursive Call

10A[middle]A[left]

Sorted FirstPart

Sorted SecondPart

Merge-Sort: Merge

A[right]

merge

A:

A:

Sorted

11

6 10 14 223 5 15 28L: R:

Temporary Arrays

5 15 28 30 6 10 145

Merge-Sort: Merge Example

2 3 7 8 1 4 5 6A:

12

Merge-Sort: Merge Example

3 5 15 28 30 6 10 14

L:

A:

3 15 28 30 6 10 14 22

R:

i=0 j=0

k=0

2 3 7 8 1 4 5 6

1

13

Merge-Sort: Merge Example

1 5 15 28 30 6 10 14

L:

A:

3 5 15 28 6 10 14 22

R:

k=1

2 3 7 8 1 4 5 6

2

i=0 j=1

14

Merge-Sort: Merge Example

1 2 15 28 30 6 10 14

L:

A:

6 10 14 22

R:

i=1

k=2

2 3 7 8 1 4 5 6

3

j=1

15

Merge-Sort: Merge Example

1 2 3 6 10 14

L:

A:

6 10 14 22

R:

i=2 j=1

k=3

2 3 7 8 1 4 5 6

4

16

Merge-Sort: Merge Example

1 2 3 4 6 10 14

L:

A:

6 10 14 22

R:

j=2

k=4

2 3 7 8 1 4 5 6

i=2

5

17

Merge-Sort: Merge Example

1 2 3 4 5 6 10 14

L:

A:

6 10 14 22

R:

i=2 j=3

k=5

2 3 7 8 1 4 5 6

6

18

Merge-Sort: Merge Example

1 2 3 4 5 6 14

L:

A:

6 10 14 22

R:

k=6

2 3 7 8 1 4 5 6

7

i=2 j=4

19

Merge-Sort: Merge Example

1 2 3 4 5 6 7 14

L:

A:

3 5 15 28 6 10 14 22

R:2 3 7 8 1 4 5 6

8

i=3 j=4

k=7

20

Merge-Sort: Merge Example

1 2 3 4 5 6 7 8

L:

A:

3 5 15 28 6 10 14 22

R:2 3 7 8 1 4 5 6

i=4 j=4

k=8

21

Merge(A, left, middle, right)1. n1 ← middle – left + 12. n2 ← right – middle3. create array L[n1], R[n2]4. for i ← 0 to n1-1 do L[i] ← A[left +i]5. for j ← 0 to n2-1 do R[j] ← A[middle+j]6. k ← i ← j ← 07. while i < n1 & j < n2 8. if L[i] < R[j] 9. A[k++] ← L[i++]10. else11. A[k++] ← R[j++]12. while i < n1

13. A[k++] ← L[i++]14. while j < n2

15. A[k++] ← R[j++] n = n1+n2

Space: nTime : cn for some constant c

22

6 2 8 4 3 7 5 16 2 8 4 3 7 5 1

Merge-Sort(A, 0, 7)

DivideA:

23

6 2 8 4

3 7 5 1

6 2 8 4

Merge-Sort(A, 0, 3) , divideA:

Merge-Sort(A, 0, 7)

24

3 7 5 1

8 4

6 26 2

Merge-Sort(A, 0, 1) , divideA:

Merge-Sort(A, 0, 7)

25

3 7 5 1

8 4

6

2

Merge-Sort(A, 0, 0) , base caseA:

Merge-Sort(A, 0, 7)

26

3 7 5 1

8 4

6 2

Merge-Sort(A, 0, 0), returnA:

Merge-Sort(A, 0, 7)

27

3 7 5 1

8 4

6

2

Merge-Sort(A, 1, 1) , base caseA:

Merge-Sort(A, 0, 7)

28

3 7 5 1

8 4

6 2

Merge-Sort(A, 1, 1), returnA:

Merge-Sort(A, 0, 7)

29

3 7 5 1

8 4

2 6

Merge(A, 0, 0, 1)A:

Merge-Sort(A, 0, 7)

30

3 7 5 1

8 42 6

Merge-Sort(A, 0, 1), returnA:

Merge-Sort(A, 0, 7)

31

3 7 5 1

8 4

2 6

Merge-Sort(A, 2, 3)

48

, divideA:

Merge-Sort(A, 0, 7)

32

3 7 5 1

4

2 6

8

Merge-Sort(A, 2, 2), base caseA:

Merge-Sort(A, 0, 7)

33

3 7 5 1

4

2 6

8

Merge-Sort(A, 2, 2), returnA:

Merge-Sort(A, 0, 7)

34

4

2 6

8

Merge-Sort(A, 3, 3), base caseA:

Merge-Sort(A, 0, 7)

35

3 7 5 1

4

2 6

8

Merge-Sort(A, 3, 3), returnA:

Merge-Sort(A, 0, 7)

36

3 7 5 1

2 6

4 8

Merge(A, 2, 2, 3)A:

Merge-Sort(A, 0, 7)

37

3 7 5 1

2 6 4 8

Merge-Sort(A, 2, 3), returnA:

Merge-Sort(A, 0, 7)

38

3 7 5 1

2 4 6 8

Merge(A, 0, 1, 3)A:

Merge-Sort(A, 0, 7)

39

3 7 5 12 4 6 8Merge-Sort(A, 0, 3), return

A:

Merge-Sort(A, 0, 7)

40

3 7 5 1

2 4 6 8Merge-Sort(A, 4, 7)

A:

Merge-Sort(A, 0, 7)

41

1 3 5 7

2 4 6 8A:

Merge (A, 4, 5, 7)

Merge-Sort(A, 0, 7)

42

1 3 5 72 4 6 8Merge-Sort(A, 4, 7), return

A:

Merge-Sort(A, 0, 7)

43

1 2 3 4 5 6 7 8Merge(A, 0, 3, 7)

A:

Merge-Sort(A, 0, 7)Merge-Sort(A, 0, 7), done!

44

Merge-Sort Analysis cn

2 × cn/2 = cn

4 × cn/4 = cn

n/2 × 2c = cn

log n levels

• Total running time: (nlogn)• Total Space: (n)

Total: cn log n

n

n/2 n/2

n/4 n/4 n/4 n/4

2 2 2

45

Merge-Sort SummaryApproach: divide and conquerTime

– Most of the work is in the merging– Total time: (n log n)

Space: – (n), more space than other sorts.

46

Overview

• Divide and Conquer

• Merge Sort

• Quick Sort

47

Quick Sort• Divide:

• Pick any element p as the pivot, e.g, the first element

• Partition the remaining elements into FirstPart, which contains all elements < pSecondPart, which contains all elements ≥ p

• Recursively sort the FirstPart and SecondPart

• Combine: no work is necessary since sorting

is done in place

48

Quick Sort

x < p p p ≤ x

PartitionFirstPart SecondPart

ppivot

A:

Recursive call

x < p p p ≤ x

SortedFirstPart

SortedSecondPart

Sorted

49

Quick SortQuick-Sort(A, left, right)if left ≥ right return

else middle ← Partition(A, left,

right) Quick-Sort(A, left, middle–1 ) Quick-Sort(A, middle+1, right)

end if

50

Partitionp

p x < p p ≤ x

p p ≤ xx < p

A:

A:

A:p

51

Partition Example

A: 4 8 6 3 5 1 7 2

52

Partition Example

A: 4 8 6 3 5 1 7 2

i=0

j=1

53

Partition Example

A:

j=1

4 8 6 3 5 1 7 2

i=0

8

54

Partition Example

A: 4 8 6 3 5 1 7 26

i=0

j=2

55

Partition Example

A: 4 8 6 3 5 1 7 2

i=0

383

j=3

i=1

56

Partition Example

A: 4 3 6 8 5 1 7 2

i=1

5

j=4

57

Partition Example

A: 4 3 6 8 5 1 7 2

i=1

1

j=5

58

Partition Example

A: 4 3 6 8 5 1 7 2

i=2

1 6

j=5

59

Partition Example

A: 4 3 8 5 7 2

i=2

1 6 7

j=6

60

Partition Example

A: 4 3 8 5 7 2

i=2

1 6 22 8

i=3

j=7

61

Partition Example

A: 4 3 2 6 7 8

i=3

1 5

j=8

62

Partition Example

A: 4 1 6 7 8

i=3

2 542 3

63

A: 3 6 7 81 542

x < 4 4 ≤ x

pivot incorrect position

Partition Example

64

Partition(A, left, right)1. x ← A[left]2. i ← left3. for j ← left+1 to right4. if A[j] < x then 5. i ← i + 16. swap(A[i], A[j])7. end if8. end for j9. swap(A[i], A[left])10. return i

n = right – left +1 Time: cn for some constant c Space: constant

65

4 8 6 3 5 1 7 22 3 1 5 6 7 84

Quick-Sort(A, 0, 7)Partition

A:

66

2 3 1

5 6 7 84

2 1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 0, 2)

A:

, partition

67

2

5 6 7 84

1

1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 0, 0) , base case, return

68

2

5 6 7 84

1

33

Quick-Sort(A, 0, 7)Quick-Sort(A, 1, 1) , base case

69

5 6 7 842 1 3

2 1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 2, 2), returnQuick-Sort(A, 0, 2), return

70

42 1 3

5 6 7 86 7 85

Quick-Sort(A, 0, 7)Quick-Sort(A, 2, 2), returnQuick-Sort(A, 4, 7) , partition

71

4

5

6 7 87 866

2 1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 5, 7) , partition

72

4

5

6

7 887

2 1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 6, 7) , partition

73

4

5

6

7

2 1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 7, 7)

8

, return, base case

8

74

4

5

6 87

2 1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 6, 7) , return

75

4

5

2 1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 5, 7) , return

6 87

76

42 1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 4, 7) , return

5 6 87

77

42 1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 0, 7) , done!

5 6 87

78

Quick-Sort: Best Case Even Partition

Total time: (nlogn)

cn

2 × cn/2 = cn

4 × c/4 = cn

n/3 × 3c = cn

log n levels

n

n/2 n/2

n/4

3 3 3

n/4n/4n/4

79

cn

c(n-1)

3c

2c

n

n-1

n-2

3

2

c(n-2)

Happens only if input is sortd input is reversely sorted

Quick-Sort: Worst Case Unbalanced Partition

Total time: (n2)

80

Quick-Sort: an Average Case• Suppose the split is 1/10 : 9/10

Quick-Sort: an Average Case

cn

cn

cn

≤cn

n

0.1n 0.9n

0.01n 0.09n 0.09n

Total time: (nlogn)

0.81n

2

2

log10n

log10/9n

≤cn

81

Quick-Sort Summary• Time

– Most of the work done in partitioning.– Average case takes (n log(n)) time.– Worst case takes (n2) time

Space– Sorts in-place, i.e., does not require additional

space

82

Summary• Divide and Conquer

• Merge-Sort – Most of the work done in Merging– (n log(n)) time– (n) space

• Quick-Sort– Most of the work done in partitioning– Average case takes (n log(n)) time– Worst case takes (n2) time– (1) space