Heap Sort

1

Heap Sort

A Heap is a Binary TreeHeight of tree = longest path from root to leaf= (lgn)

length[A]heap-size[A] A

10

908

703 897

275 503 512 061

170 087 154

1

2 3

4 5 6 7

8 9 10

1 2 3 4 5 6 7 8 9 10

908 703 897 275 503 512 061 170 087 154

A heap is a binary tree satisfying the heap condition: At every node in a heap, the node value is >= all the values in its subtrees.

A heap with heap_size elements can be represented as an array segment: A[1..heap_size]

3

Ordering of Nodes16

14 10

8 7 9 3

2 4 1

1

2 3

4 5 6 7

8 9 10

Each node is filled in order from left to right.

All rows are filled except, possibly, the last row.

Address of left child of node i => 2iAddress of right child of node i => 2i+1Address of parent of node i =>

i /2 Example: Left(4) = 8 Right(4) = 9

Parent(4) = 2

4

Heap properties

1. At every node in the heap, that node's value is greater than or equal to all the values in the subtrees of that node.

2. Heaps can be represented as an array. The elements in the array occur in the order of their heap addresses.

A= 16 14 10 8 7 9 3 2 4 1

Note that the array is not sorted.

Can guarantee that A[1] is the maximum value in the array.

5

Building a Heap To perform a heap sort, we must first build a heap out of our unsorted array.

To build a heap, we will use the function heapify, which creates a heap from a node i and two subtrees that are already heaps.

6

Heapify16

3 14

15 6 9 11

7 8 2

1

i=2 3

4 5 6 7

8 9 10

Example: i = 2Want to move the value, 3, to a position consistent with the heap property.

In English:1. Compare the value at node i with the values at each child node.

Swap the value at node i with the largest of those values.2. Repeat with the subtree that ith value was swapped into.Note: If the value at i is in the correct place, it will be swapped

with itself and the algorithm is done.

7

Pseudocode for Heapify

Heapify (A, i)lft Left(i)rt Right(i)if lft <= heap-size[A] and A[lft] > A[i]

then largest lftelse largest i

if rt <= heap-size[A] and A[rt] > A[largest]then largest rt

if largest != i then Swap( A, i, largest );Heapify (A, largest);

8

Final Tree from example

16

15 14

8 6 9 11

7 3 2

1

i=2 3

4 5 6 7

8 9 10

Can show that heapify runs in O(lgn) time. (Why?)

9

Building a Heap

1. Each leaf of a binary tree is already a heap of size 1.2. Build-Heap will start with the highest addressed, non-leaf

node and heapify it. It then repeats with each node addressed 1 less than the previous node.

3. In a binary tree, the leaf nodes have addresses:

We want to heapify starting with node:

n

2

1...n

n

2

Build-Heap(A)heap-size length[A]for i length[A] / 2 downto 1

do Heapify(A, i)

10

Example

A= 4 7 16 3 27 9 10 34 1 25

4

7 16

3 27 9 10

34 1 25

1

2 3

4 5 6 7

8 9 10

We will work this out in class.

11

Heap SortIn English:

We know that the maximum value is at A[1] in a heap. We put that value at the end of the array, by swapping it with the last element of the heap. We then reduce the size of the heap by 1, so that the sorted element is no longer part of the heap. Heapify the new, smaller heap (A[1] is not necessarily in the correct position for a heap). This will put a new maximum at the top. Repeat this process until all the nodes have been sorted.

12

Pseudocode for Heapsort

Heapsort(A)Build-Heap(A)for i length[A] downto 2

do Swap(A, 1, i) heap-size[A] heap-size[A] - 1

Heapify(A, 1)

What is the running time of Heapsort?Must find running times of Build-Heap and Heapify first.

13

Recall Heapify

Heapify( ) is a function that creates a heap from a node i and two subtrees that are already heaps.

16

3 14

15 6 9 11

7 8 2

1

i=2 3

4 5 6 7

8 9 10

Example: i = 2

Swap the value at node i with the largest of the values at child nodes.If value at node i is larger than those of children, stop.Otherwise, continue with the next level.

14

Running time of Heapify

Maximum number of swaps in Heapify is the height of the heap.1

3

4 5 6 7

8 9 10

2

Height of n element heap is:

h lg(n)

Therefore, the running time of heapify is O(lgn)

15

Running time of Build heap

Recall that Build-Heap starts with the last element and repeatedly calls Heapify to create heaps from the bottom up.

If the height of a node is given by the longest distance from a leaf to that node, then the number of nodes at a given height h is at most:

n

2h1

Why?

Build-Heap calls heapify for each node at a given height:

T(n) n

2h1

h0

lgn

O(h) O(n)

number of nodes at height h

cost of Heapify at height h

T(n) = ?

16

Cost of Heap-Sort

Heapsort(A)Build-Heap(A)for i length[A] downto 2 do

Swap(A, 1, i)heap-size[A] heap-size[A] - 1

Heapify(A, 1)

O(n)nn*constn*constn*O(lgn)

T(n) = O(n) + n*O(lgn) = O(nlgn)

17

Using a Heap in Priority Queues

A priority queue is a queue in which the element with the highest value is retrieved first. WIPO: Whatever in, Priority out.

Priority Queue Operations:Insert(S, x)

Inserts element x into set SMaximum(S)

Returns the element of S with the largest value (or key)Extract-Max(S)

Removes and returns the element of S with the largest value.

Heaps are useful data structures for priority queue functions.

18

Extracting the Maximum

Idea: Extract the maximum element, reduce the heap size by 1, then heapify.

Heap-Extract-Max(A)if heap_size[A] < 1 then error "heap underflow" max A[1] { root value is max } A[1] A[heap_size[A]] { put A[heap_size[A]] in root} heap_size[A] heap_size[A] - 1 { decrease heap size } Heapify(A, 1) { enforce heap property } return max

19

Example

16

12 8

3 2

1

3

4 5

We will work this through in class.

20

Heap-InsertIdea: To insert an element at the proper place in the heap, traverse the tree from leaf to root and find the correct place.

HeapInsert(A, key)heap_size[A] heap_size[A] + 1 { increase size of heap }i heap-size[A]while i > 1 and A[Parent(i)] < key do

A[i] A[Parent(i)]i Parent(i)

A[i] key

21

Example

16

14 10

8 7 9 3

2 4 1

1

3

4 5 6 7

8 9 10

Insert the number, 15.

We will work through this in class.

22

Binary Heap: Definition

_ Binary heap._ Almost complete binary tree.

– filled on all levels, except last, where filled from left to right

_ Min-heap ordered.– every child greater than (or equal to) parent

06

14

78 18

81 7791

45

5347

6484 9983

23

Binary Heap: Properties

_ Properties._ Min element is in root.

_ Heap with N elements has height = log2 N.

06

14

78 18

81 7791

45

5347

6484 9983

N = 14Height = 3

24

Binary Heaps: Array Implementation

_ Implementing binary heaps._ Use an array: no need for explicit parent or child

pointers.– Parent(i) = i/2 – Left(i) = 2i – Right(i) = 2i + 1

06

14

78 18

81 7791

45

5347

6484 9983

1

2 3

4 5 6 7

8 9 10 11 12 13 14

25

Binary Heap: Insertion_ Insert element x into heap.

_ Insert into next available slot._ Bubble up until it's heap ordered.

– Peter principle: nodes rise to level of incompetence

06

14

78 18

81 7791

45

5347

6484 9983 42 next free slot

26




06

14

78 18

81 7791

45

5347

6484 9983 4242

swap with parent

27

Binary Heap: Insertion

_ Insert element x into heap._ Insert into next available slot._ Bubble up until it's heap ordered.


06

14

78 18

81 7791

45

4247

6484 9983 4253

swap with parent

28




_ O(log N) operations.06

14

78 18

81 7791

42

4547

6484 9983 53

stop: heap ordered

29

Binary Heap: Decrease Key

_ Decrease key of element x to k._ Bubble up until it's heap ordered._ O(log N) operations.

06

14

78 18

81 7791

42

4547

6484 9983 53

30

Binary Heap: Delete Min_ Delete minimum element from heap.

_ Exchange root with rightmost leaf._ Bubble root down until it's heap ordered.

– power struggle principle: better subordinate is promoted

06

14

78 18

81 7791

42

4547

6484 9983 53

31

Binary Heap: Delete Min

_ Delete minimum element from heap._ Exchange root with rightmost leaf._ Bubble root down until it's heap ordered.


53

14

78 18

81 7791

42

4547

6484 9983 06

32

Binary Heap: Delete Min

_ Delete minimum element from heap._ Exchange root with rightmost leaf._ Bubble root down until it's heap ordered.


53

14

78 18

81 7791

42

4547

6484 9983

exchange with left child

33




14

53

78 18

81 7791

42

4547

6484 9983

exchange with right child

34




_ O(log N) operations.14

18

78 53

81 7791

42

4547

6484 9983

stop: heap ordered

35

Binary Heap: Heapsort

_ Heapsort._ Insert N items into binary heap._ Perform N delete-min operations._ O(N log N) sort._ No extra storage.

Documents

Heap Sort