Priority Queues and Heaps

RHS – SOC 2

Priority Queues

• We already know a ”standard” queue:– Elements can only be added to

the start of the queue– Elements can only be removed

from the end of the queue– Order of removal is FIFO (first

in, first out)

RHS – SOC 3

Priority Queues

• In a Priority Queue, each element is assigned a priority as well

• A priority is a numeric value – the lower the number, the higher the priority

• Elements are removed in order of their priority – no longer FIFO

• Can still add elements in any order

RHS – SOC 4

Priority Queues

public interface PriorityQueue<T>

{

void add(T element);

T remove();

T peek();

}

RHS – SOC 5

Priority Queues

• The interface to a priority queue is almost the same as to a regular queue

• However, elements in the queue must now be of the type Comparable (i.e. implement the interface)

• The remove method always returns the element with highest priority (lowest numeric value…)

RHS – SOC 6

Priority Queues

• A Priority Queue is an abstract data structure, which can be implemented in various ways– Linked list, removal will be O(n)– Binary search tree, removal will usually be

O(log(n)), but not always– Heap, removal will always be O(log(n))

RHS – SOC 7

Heaps

• A Heap is a Binary Tree, but not a Binary Search Tree

• In order for a Binary Tree to be a Heap, two conditions must be fulfilled:– A heap is almost complete; only nodes

missing in bottom layer– All nodes store values which are at most as

large as the values stored in any descendant

RHS – SOC 8

Heaps

RHS – SOC 9

Heaps

• On a heap, only two operations are of interest:– Insert a new element– Remove the element with lowest value, i.e.

the root element

• However, both of these operations must preserve the heap property of the tree

RHS – SOC 10

Heaps - insertion

23

29 42

5637

32

Find an empty slot in the tree

RHS – SOC 11

Heaps - insertion

23

29 42

5637

32

If the value in the parent is larger than the new value, swap the parent and the new slot (repeat this)

RHS – SOC 12

Heaps - insertion

23

29

5637 42

32

Now insert the new value

32

RHS – SOC 13

Heaps - insertion

• Since the heap is ”almost complete”, the number of layers in a tree with n nodes is at most (log(n) + 1)

• Each swap operation can be done in constant time, so insertion of an element has the run-time complexity O(log(n))

RHS – SOC 14

Heaps - removal

23

29

5637 42

Remove the root node (always smallest)

32

RHS – SOC 15

Heaps - removal

29

5637 42Move the last element into the root position

32

RHS – SOC 16

Heaps - removal

29

5637

42If any child has a lower value, swap position with child with lowest value(repeat this)

32

RHS – SOC 17

Heaps - removal

42

5637

29If any child has a lower value, swap position with child with lowest value(repeat this)

32

RHS – SOC 18

Heaps - removal

37

5642

29The heap property has now been reestablishedThis is known as ”fixing the heap”

32

RHS – SOC 19

Heaps - removal

• Since the heap is ”almost complete”, the number of layers in a tree with n nodes is at most (log(n) + 1)

• Each swap operation can be done in constant time, so deletion of an element has the run-time complexity O(log(n))

RHS – SOC 20

Heaps - removal

• The important point is that for a heap, we are guaranteed O(log(n)) run time for insertion and deletion

• This cannot be guaranteed for a binary search tree

• A binary search tree can ”degenerate” into a linked list; a heap cannot

RHS – SOC 21

Heaps - representation

• Due to the regularity of a heap, it can efficiently be stored in an array– Root node is stored in position 1 (not 0)– A node in position i has its children stored in

position 2i and (2i + 1)– A node in position i has its parent stored in

position i/2 (integer division)– When running out of space, double the size

RHS – SOC 22

Heapsort

• A heap can be used for a quite efficient way of sorting an array of n objects– Run-time complexity of O(nlog(n))– Does not use extra space

• Main steps– Turn the array into a heap– Repeatedly remove the root element (which

has the smallest value)

RHS – SOC 23

Heapsort

• In order to turn the array into a heap, we could just insert all the elements into a new heap

• However, we can do this without using an extra heap!

• We use the ”fix heap” procedure from the bottom in the tree and upwards

RHS – SOC 24

Heapsort

• Why will this work…?

• Remember that the ”fix heap” procedure takes two ”subheaps” as input, plus a root node with a ”wrong” value

• If we work from the bottom and up, the input will always be like above

RHS – SOC 25

Heapsort

Trivially a heap

Fix heaps here

Fix heaps here

Fix heap here

RHS – SOC 26

Heapsort

• Now the tree is a heap

• Repeatedly remove the root from the heap, and fix the remaining heap

• We ”remove” the root by placing it at the end of the array, beyond the last element in the remaining heap

RHS – SOC 27

Heapsort

23

29

5637 42

32

23 29 32 37 56 42

RHS – SOC 28

Heapsort

29

5637 42

32

29 32 37 56 42 23

RHS – SOC 29

Heapsort

37

5642

29

32

29 37 32 42 56 23

RHS – SOC 30

Heapsort

37

5642

32

37 32 42 56 29 23

RHS – SOC 31

Heapsort

37

32

42

56

32 37 56 42 29 23

RHS – SOC 32

Heapsort

37

42

56

37 56 42 32 29 23

RHS – SOC 33

Heapsort

42

37

56

37 42 56 32 29 23

RHS – SOC 34

Heapsort

42 56

42 56 37 32 29 23

RHS – SOC 35

Heapsort

42

56

42 56 37 32 29 23

RHS – SOC 36

Heapsort

56

56 42 37 32 29 23

RHS – SOC 37

Heapsort

56

56 42 37 32 29 23

RHS – SOC 38

Heapsort

56 42 37 32 29 23

RHS – SOC 39

Heapsort

• One minor issue – numbers are sorted in wrong order

• Could just reverse order, takes O(n)

• Or use max-heap– Min-heap: All nodes store values at most as

large as the values stored in any descendant– Max-heap: All nodes store values at least as

large as the values stored in any descendant

RHS – SOC 40

Exercises

• Review: R16.20

• Programming: P16.21

• Tip: You can find a Java implementation of Heapsort on my website, under classes, week 40 (HeapSortInJava)