Tucker, Applied Combinatorics, Sec. 3.5, Jo E-M

3/10/03 Tucker, Sec.3-5 1

Tucker, Applied Combinatorics, Sec. 3.5, Jo E-M

“Big O” Notation

We say that a function is if for some constant c, when n is large.

For example, is since for n > 3.

This is described by saying that is ‘on the order of’ .

Big O notation calls attention to the part of a function that grows the fastest, so gives a simple estimate of how many steps are required for an algorithm to run.

( )g n O f n ( )g n cf n

33 2 1n n 3O n 3 33 2 1 3n n n

33 2 1n n 3n

3/10/03 Tucker, Sec.3-5 2

Binary Testing TreeCompare things (objects, subsets, lists, whatever) in some way with two outcomes.

Keep comparing…Eventually arrive at one of the n! possible orders

Since there are n! leaves on this tree, by Theorem 3 in section 3.1, the tree has height . Thus, whatever binary comparison technique is used, the worst case will require at least comparisons.

2 2 2log ! log lognn n n n

2logO n n

3/10/03 Tucker, Sec.3-5 3

Bubble SortFor m from 2 to n, do

For j from n (step –1) to m do

If Aj < Aj-1, then interchange Aj and Aj-1.

3

4

6

1

5

2

m = 2, j = 6

3

4

6

1

2

5

m = 2, j = 5

3

4

6

1

2

5

m = 2, j = 4

3

4

1

6

2

5

m = 2, j = 3

3

1

4

6

2

5

m = 2, j = 2

1

3

4

6

2

5

m = 3, j = 6

1

3

4

6

2

5

m = 3, j = 5

1

3

4

2

6

5

m = 3, j = 4

Moves the smallest number to the top Moves the next smallest number to the top

3/10/03 Tucker, Sec.3-5 4

Complexity of the Bubble SortWhen m = 2, j goes from n to 2, so you have to do n –1 comparisons.

When m = 3, j goes from n to 3, so you have to do n –2 comparisons, and so on.

Thus, the total number of comparisons is:

Thus, the Bubble sort takes comparisons, which is a lot more than the

theoretical bound of .

1 2 1n n

121 1

2 21

1

2

n

i

n ni n n

2O n

2logO n n

3/10/03 Tucker, Sec.3-5 5

Merge Sort—Subdivision Tree

First divide the set repeatedly, roughly in half, until only single elements are left at the leaves….

5 0 9 2 6 47 1 3 8

5 0 9 2 6 4 7 1 3 8

2 65 0 9 4 7 1 3 8

5 0 9 62 14 7 83

0 5 4 7

3/10/03 Tucker, Sec.3-5 6

Merge Sort—Merging Tree

1 2 3 4 5 6 7 8 9

0 2 5 6 9 1 3 4 7 8

2 60 5 9 1 4 7 3 8

0 5 9 62 14 7 83

0 5 4 7

…then merge the sequences in order.

Note: This tree is drawn upside down, so that the root is at the bottom. Thus, the set {0, 5} is at level 3

3/10/03 Tucker, Sec.3-5 7

Complexity of MergingA Simplifying Assumption

• Assume our set has n = 2r elements. This means that:

• We always divide exactly in half in the subdivision tree.

• There are r levels

• All the leaves in both the subdivision and the merging tree are on level r.

• There are 2k vertices on level k

• The sets on level k all have 2r-k elements in them

n = 23

* * * * * * *

* * * * * * * *

* * * * * * * *

* * * * * * * *

3/10/03 Tucker, Sec.3-5 8

Complexity of MergingCounting the Comparisons

• At each vertex on level k of the merging tree we merge the two children sets each of size 2r-k-1.Merging two lists of length L into a single list of length 2L requires 2L-1 comparisons (class exercise).

• For each vertex on level k, this merging takes 2 (2r-k-1) – 1 = 2r-k – 1 comparisons

• There are 2k vertices on level k, for a total of 2k (2r-k – 1 ) = 2r - 2k comparisons on each level.

• Since we must do this on each level, the total number of comparisons for a Merge Sort is

• Since n = 2r, this becomes . (Still need to account for the subdivision preprocessing, but this isn’t bad)

• Thus Merge Sort is and achieves the theoretical bound of a binary search.

1 1 1

0 0 0

2 2 2 2 2 2 1r r r

r k r k r r

k k k

r

2log 1n n n

2logO n n

3/10/03 Tucker, Sec.3-5 9

Use the first element in the list to partition the list, then put it at the end of the left hand child list. Worst case, , on average

5 0 9 2 6 47 1 3 8

0 2 4 1 3 5 9 6 7 8

0 6 7 8 9

3 4

9

6

2 1 83

5 4

7

2 4 1 3 5

1 2 4 3 5

7 8 9

8 9

2logO n n 2O n

QUIK Sort

3/10/03 Tucker, Sec.3-5 10

Heap Sort• A heap is a (nearly) binary tree so that a parent is always bigger than its

children (root is biggest of all).

• Put the root at the beginning of a list, then move up the biggest grandchildren.

start of list

start of list start of list

start of list

Preprocessing: Need to first construct the heap (homework…)

3/10/03 Tucker, Sec.3-5 11

• Prove that it takes at most n – 1 comparisons to merge two sorted lists into a single sorted list of length n.

• Sort {4, 5, 2, 3, 0, 1} using each of a Bubble, Merge, QUIK, and Heap Sort.

Class Exercises

Note: if , and if , then don’t have to do the last

comparison so get rl instead of rl + 1

Merge and where k + l = n. 1, , kb b 1, la a

1 * * *( , , , )i ia b b a b

ri of the b’s

Each ai must be compared to ri + 1 of the b’s for a total of

1 1 1

1 1 1 1l l l

i ii i i

r r k l n

1

l

ii

r k

l ka b l ka b