14.1 Introduction to trees14 14.1 Introduction to trees[Theorem] An undirected graph is a tree iff there is a unique path between any two of its vertices. Proof: 1) ( → ) assume

1

14.1 Introduction to trees

A tree is an undirected graph that is connected and has no cycles.

Trees can be used to:● construct efficient algorithms for location items in a

list

● study games (checkers, chess) and determine winning strategies

● model procedures carried our using a sequence of decisions, which helps to determine the computational complexity of the algorithm

● in data compression (Huffman coding)...

2



3



a tree a tree not a tree

4


A forest is an undirected graph that has no cycles.Each of its connected components is a tree.

a forest

5



alternative definition: a tree is an undirected graph such that there is a unique path between every pair of its vertices.

6



alternative definition: a tree is an undirected graph such that there is a unique path between every pair of its vertices.

[Theorem]An undirected graph is a tree iff there is a unique path between any two of its vertices.

7



Proof:1) ( → ) assume that T is a tree

8



Proof:1) ( → ) assume that T is a tree, then T is connected and has no cycles.

9



Proof:1) ( → ) assume that T is a tree, then T is connected and has no cycles. Let x and y be any two vertices of T.

10



Proof:1) ( → ) assume that T is a tree, then T is connected and has no cycles. Let x and y be any two vertices of T.Then there is a path between x and y because T is connected.

11



Proof:1) ( → ) assume that T is a tree, then T is connected and has no cycles. Let x and y be any two vertices of T.Then there is a path between x and y because T is connected.Assume that there exists another path between x and y.

12



Proof:1) ( → ) assume that T is a tree, then T is connected and has no cycles. Let x and y be any two vertices of T.Then there is a path between x and y because T is connected.Assume that there exists another path between x and y. In this case we will be able to form a closed walk by combining two paths.

13



Proof:1) ( → ) assume that T is a tree, then T is connected and has no cycles. Let x and y be any two vertices of T.Then there is a path between x and y because T is connected.Assume that there exists another path between x and y. In this case we will be able to form a closed walk by combining two paths. From it we can build cycle.

14



Proof:1) ( → ) assume that T is a tree, then T is connected and has no cycles. Let x and y be any two vertices of T.Then there is a path between x and y because T is connected.Assume that there exists another path between x and y. In this case we will be able to form a closed walk by combining two paths. From it we can build cycle.This contradicts to the statement that T is a tree.

15



Proof:1) ( → ) assume that T is a tree, then T is connected and has no cycles. Let x and y be any two vertices of T.Then there is a path between x and y because T is connected.Assume that there exists another path between x and y. In this case we will be able to form a closed walk by combining two paths. From it we can build cycle.This contradicts to the statement that T is a tree.Therefore the path is unique.

16



Proof:2) ( ← ) assume that there is a unique path between any two vertices of T.

17



Proof:2) ( ← ) assume that there is a unique path between any two vertices of T. Then T is connected.

18



Proof:2) ( ← ) assume that there is a unique path between any two vertices of T.Then T is connected.Assume that T has a cycle that contains vertices x and y.

19



Proof:2) ( ← ) assume that there is a unique path between any two vertices of T.Then T is connected.Assume that T has a cycle that contains vertices x and y. Then we can form two paths between these vertices. This contradicts to the assumption about uniqueness of a path.

20



Proof:2) ( ← ) assume that there is a unique path between any two vertices of T.Then T is connected.Assume that T has a cycle that contains vertices x and y. Then we can form two paths between these vertices. This contradicts to the assumption about uniqueness of a path. Therefore, T has no cycles.

21



Proof:2) ( ← ) assume that there is a unique path between any two vertices of T.Then T is connected.Assume that T has a cycle that contains vertices x and y. Then we can form two paths between these vertices. This contradicts to the assumption about uniqueness of a path. Therefore, T has no cycles.By definition, T is a tree.

q.e.d.

22


[Def] A free tree is a tree with no particular organization of the vertices and edges

[Def] A rooted tree is a tree in which one vertex has been designated as the root and every edge is directed away from the root.

root

a rooted treea free tree

23


[Def] A rooted tree is a tree in which one vertex has been designated as the root and every edge is directed away from the root.

[Def] The level of a vertex is its distance from to the root.

[Def] The height of a tree is the highest level of any vertex.

root

a rooted tree

height of a tree = 4

level 4 verticeslevel 3 vertices

24


a rooted tree

a

b c de f g

h i j k lm n

root

[Corollary] There is a unique path from the root of the tree to each vertex of the tree.

This follows from Theorem we just proved.

25


root

a rooted tree

a

b c de f g

h i j k lm n

vertex g is the parent of vertices j, k, and l.

● Every vertex in a rooted tree T has a unique parent, except for the root which does not have a parent. The parent of vertex v is the first vertex after v encountered along the path from v to the root.

26


root (and ancestor of vertex n)

a rooted tree

a

b c de f g

h i j k lm n

ancestors of vertex n

● Every vertex along the path from v to the root (except for the vertex v itself, but including the root) is an ancestor of vertex v.

ancestor of vertex n

27


root

a rooted tree

a

b c de f g

h i j k lm n

descendants of vertex d are vertices g, j, k, l and n

● Every vertex along the path from v to the root (except for the vertex v itself, but including the root) is an ancestor of vertex v.

● If u is an ancestor of v, then v is a descendant of u.

descendant of vertex k is n

28


root

a rooted tree

a

b c de f g

h i j k lm n

vertex g is the parent of vertices j, k, and l.

vertices j, k, and l are siblings and are children of vertex g.

● If v is the parent of vertex u, then u is a child of vertex v.

● Two or more vertices are siblings if they have the same parent.

29


root a

b c de f g

h i j k lm n

● a leaf is a vertex which has no children.

● vertices that have children are called internal vertices

b, f, I, k, l, m and n are leaves

internal vertices

...

30


● a leaf is a vertex which has no children.

● vertices that have children are called internal vertices

● a subtree rooted at vertex v is the tree consisting of v and all v's descendants and all the edges incident to the descendants.

root a

b c de f g

h i j k lm n

a subtree rooted at vertex g

root a

b c de f g

h i j k lm n

b, f, I, k, l, m and n are leaves

internal vertices

...

31


a

b c

d e f g

h i j k l

m n

● the root of the tree T● all leaves● all internal vertices● the parent of g● the ancestors of h● the descendants of b● the height of the tree● the level of vertex i

Practice: for the given tree T find

T

32


a

b c

d e f g

h i j k l

m n

● the root of the tree T : a● all leaves : m, I, j, f, n, l● all internal vertices :

a, b, c, d, e, g, h, k● the parent of g : c● the ancestors of h : d, b, a● the descendants of b :

d, e, h, i, j, m● the height of the tree : 4● the level of vertex i : 3

Practice: for the given tree T find

T

33


a

b c

d e f g

h i j k l

m n

[Def] a rooted tree is called m-ary tree if every internal vertex has no more than m children.

3-ary tree

34


a

b c

d e f g

h i j k l

m n


3-ary tree

[Def] The tree is call a full m-ary tree if every internal vertex has exactly m children.

a

b c

d e

a

b c

d e f g

a

b c

d e

f g

35


a

b c

d e f g

h i j k l

m n


3-ary tree


[Def] An m-ary tree with m = 2 is called a binary tree.

a

b c

d e f

36


a

b c

d e f g

h i j k l

m n


3-ary tree


[Def] A complete m-ary tree is a full m-ary tree in which each leaf is at the same level. a

b c

d e f g

37


a

b c

d e f g

h i j k l

m n

We will be “ordering rooted trees” so that the children of each internal vertex are shown in order from left to right.

This is a binary tree.

Vertex b has the left child d and the right child e.

The subtree rooted at the left child is called left subtree.

The subtree rooted at the right child is called right subtree.

38


Trees are used as models in Computer Science, Geology, Biology, Chemistry, Botany and Psycology.

39

14.2 Tree application examples


Example 1: File trees

Files can are organized into directories/folders.

A directory/folder can Contain both files and subdirectories/subfolders

The root directory/folder contains the entire file system.

rooted tree

40



Example 1: File trees

Files can are organized into directories/folders.

A directory/folder can Contain both files and subdirectories/subfolders

The root directory/folder contains the entire file system.

41


Example 2: the structure of a large organization can be modeled using a rooted tree

42


Example 3: tree-connected parallel processorsA tree-connected network is one of the ways to interconnect processors.

Consider a complete binary tree of height 2: 7 processors are interconnected with each other.Each edge is a two-way connection.

P1

P2

P3

P4

P5 P

6P

7

Let's add 8 numbers using 3 steps:

43




P1

P2

P3

P4

P5 P

6P

7

x1+x

2

x3+x

4

x5+x

6x

7+x

8


44




P1

P2

P3

P4

P5 P

6P

7

x1+x

2

x3+x

4

x5+x

6x

7+x

8

x1,2

+x3,4

x5,6

+x7,8


45




P1

P2

P3

P4

P5 P

6P

7


x1+x

2

x3+x

4

x5+x

6x

7+x

8

x1,2

+x3,4

x5,6

+x7,8

x1,2,3,4

+x5,6,7,8

46


Binary Search Trees

Binary search tree, or BST, is a binary tree with keys assigned to vertices/nodes, where every vertex/node v has the following property:

● each key in the left subtree is less than the key at the vertex v,

● each key in the right subtree is greater than the key at the vertex v. 9

6 15

11 27

47


Binary Search Trees

Inserting keys into a BST (one by one), starting with an empty tree: 9, 6, 15, 11, 27, 13, 1, 7, and 2.

9 9

6 15

11

9

6

9

156

9

6 15

11 27

9

6 15

11 271

2

7

13

9

6 15

11 27

13

48


Binary Search Trees

Inserting keys into a BST (one by one), starting with an empty tree: 9, 6, 15, 11, 27, 13, 1, 7, and 2.

9

6 15

11 271

2

7

13

No duplicates can be inserted!

49


Binary Search Trees

Inserting word keys into a BST (one by one), starting with an empty tree: instance, node, root, child, left, right, and parent.

instance

node

leftroot

child

instance

node

left

parent

right

child

root

50


Binary Search Trees9

36

15

27

42

To determine the computational complexity of insertion operation let's see how many comparisons are performed in a worst-case scenario: the item is added to the longest path's leaf.

length of the longest path = height of the tree = n

51


Binary Search Trees9

36

15

27

42

To determine the computational complexity of insertion operation let's see how many comparisons are performed in a worst-case scenario: the item is added to the longest path's leaf.

length of the longest path = height of the tree = n

but when the tree is balanced, the timing is improved to log(n).

52


Decision Trees

Rooted trees can be used to model problems in which a series of decisions leads to a solution.

Example: BST can be used to locate items based on a series of comparisons.

Let's check if 11 is in the BST(searching for 11)

9

6 15

11 271

2

7

13

53


Decision Trees




9

6 15

11 271

2

7

13

start at the root11 = 9? Nogo right

54


Decision Trees




9

6 15

11 271

2

7

13

start at the root

11 = 15? Nogo left

55


Decision Trees




9

6 15

11 271

2

7

13

start at the root

11 = 11? Yesreturn True

56


Decision Trees

A rooted tree in which each internal vertex corresponds to a decision, with a subtree at these vertices for each possible outcome of the decision, is called a decision tree.

The possible solutions of the problem correspond to the paths to the leaves of this rooted tree.

57


Decision Trees

Example: suppose there are seven coins of the same weight, and a counterfeit coin that weighs less than the others.How many weighings are necessary using a balance scale to determine which coin is the counterfeit one?Give an algorithm.

58


Decision Trees

Example: suppose there are seven coins of the same weight, and a counterfeit coin that weighs less than the others. How many weighings are necessary using a balance scale to determine which coin is the counterfeit one? Give an algorithm.

Solution: 8 coins, and balance scale.

59


Decision Trees


Solution: 8 coins, and balance scale. 3 outcomes for weighing two coins: equal, left pan is heavier, right pan is heavier.Let's use 3-ary decision tree.

60


Decision Trees


Solution: 8 coins, and balance scale. 3 outcomes for weighing two coins: equal, left pan is heavier, right pan is heavier.Let's use 3-ary decision tree.There are 8 possible outcomes, hence at least 8 leaves.

61


Decision Trees


Solution: 8 coins, and balance scale. 3 outcomes for weighing two coins: equal, left pan is heavier, right pan is heavier.Let's use 3-ary decision tree.There are 8 possible outcomes, hence at least 8 leaves.Height of the decision tree = largest number of weighings

62


Decision Trees


Solution:

lighter lighter

{1,2,3} vs {4,5,6}

{1} vs {2} {4} vs {5}{7} vs {8}

balance

63


Decision Trees


Solution: at least 2 weighing are needed.

lighter lighter

{1,2,3} vs {4,5,6}

{1} vs {2} {4} vs {5}

{1} {2}{3} {4} {5}{6}

{7} vs {8}

{7} {8}

balance

64


Decision Trees

Many different sorting algorithms have been developed. To decide whether a particular algorithm is efficient, its complexity is determined.

Using decision trees as models, a lower bound for the worst-case complexity of sorting algorithms that are based on binary comparisons (two elements at a time) can be found.

65


Decision Trees

Example: let's draw a decision tree that orders elements x,y and z.

x : yx > y x < y

66


Decision Trees


x : y

x : z

x > y x < y

x > z x < z

y : z

y > z y < z

67


Decision Trees


x : y

x : z

x > y x < y

x > z x < z

y : z

y : z

y > z y < z

x : zz > x > y z > y > xy > z y < z

x > z x < z

68


Decision Trees


x : y

x : z

x > y x < y

x > z x < z

y : z

y > z y < z

y : z

y > z y < z

x : z

x > z x < z

x > y > z x > z > y

z > x > y

y > x > z y > z > x

z > y > x

69


Prefix Codes

Consider a problem of using bit strings to encode the letters of the English alphabet (not case sensitive)

We can use bit strings of length 5 : 25 = 32 different bit strings of length 5 (26 letters in the alphabet)___ ___ ___ ___ ___ 2 2 2 2 2 = 25 = 32

70


Prefix Codes

Consider a problem of using bit strings to encode the letters of the English alphabet (not case sensitive)

We can use bit strings of length 5 : 25 = 32 different bit strings of length 5 (26 letters in the alphabet)___ ___ ___ ___ ___ 2 2 2 2 2 = 25 = 32

Of course we would like to use fewer bits if possible, i.e. not every letter should get a bit-string of length 5.So we will use bit strings of different lengths to encode letters. More frequently occurring letters will have shorter bit strings encoding.

71


Prefix Codes

Notice, that we have to be careful!

For example:If a is encoded with 0, b with 1, and c with 01,

The bit string 10101 can correspond to:bcc (1 01 01),babc (1 0 1 01), bcab (1 01 0 1), and so forth.

72


Prefix Codes

Let's consider prefix codes: letters are encoded in such a way that the bit string for a letter never occurs as the first part of the bit string for another letter.

73


Prefix Codes


Prefix codes can be represented by a binary tree, whereleaves' labels are characters.“left edge” is assigned 0, and“right edge” is assigned 1.

f

a

b

e

0

0

0

1

1

1

74


Prefix Codes



codesa: 0e: 10f: 110b: 111

f

a

b

e

0

0

0

1

1

1

75


Prefix Codes



The string 10101111100 corresponds to eebfa

10 10 111 110 0f

a

b

e

0

0

0

1

1

1

codesa: 0e: 10f: 110b: 111

76


Huffman Coding

An algorithm, known as Huffman coding, was developed by David Albert Huffman in a term paper he wrote in 1951 while a graduate student at MIT.

It allows to produce prefix code for a string using fewest possible bits, based on frequencies (which are the probabili-ties of occurrences) of symbols in a string.

This algorithm is a fundamental algorithm in data compression (including audio and image compressions).

77


Huffman Coding

procedure Huffman(C: symbols ai with freq. w

i,i = 1,...,n)

F := forest of n rooted trees, each contains a single vertex a

i and assigned weight w

i.

while F is not a treeTake two rooted trees, T and T', with the least weights and w(T) w(T').Replace them with a tree having a new root that has T as its left subtree and T' as its right subtree.Label the edge to T with 0, and the edge to T' with 1.Assign w(T)+w(T') as the weight of the new tree.

Output: Huffman code for ai: the concatenation of labels of

the edges in the unique path from the root to the vertex vi.

78


Huffman Coding

Example: Use Huffman coding to encode the following symbols with the frequencies:A: 0.10B: 0.25C: 0.05D: 0.15E: 0.30F: 0.07G: 0.08What is the average number of bits required to encode a symbol?

79


Huffman Coding

0.05

C F

0.07

G

0.08

A

0.10

D

0.15

B

0.25

E

0.30 Initial step

80


Huffman Coding

0.05

C F

0.07

G

0.08

A

0.10

D

0.15

B

0.25

E

0.30 Initial step

0.12

CF

G

0.08

A

0.10

D

0.15

B

0.25

E

0.300 1

1st iteration

81


Huffman Coding

0.05

C F

0.07

G

0.08

A

0.10

D

0.15

B

0.25

E

0.30 Initial step

0.12

CF

G

0.08

A

0.10

D

0.15

B

0.25

E

0.300 1

1st iteration

0.12

CF A

0.18

G

D

0.15

B

0.25

E

0.300 1

2nd iteration0 1

82


Huffman Coding

0.12

CF A

0.18

G

D

0.15

B

0.25

E

0.300 1

2nd iteration0 1

83


Huffman Coding

0.12

CF A

0.18

G

D

0.15

B

0.25

E

0.300 1

2nd iteration0 1

0.27

CF

A

0.18

G D

B

0.25

E

0.30

0 1

3rd iteration0 1 0 1

84


Huffman Coding

0.27

CF

A

0.18

G D

B

0.25

E

0.30

0 1

3rd iteration0 1 0 1

0.27

CF A

0.43

G

D B

E

0.30

0 1

4th iteration

0 1

0 1 0 1

85


Huffman Coding

0.27

CF A

0.43

G

D B

E

0.30

0 1

4th iteration

0 1

0 1 0 1

0.57

CF

D

E0 1

0 1

0 1

A

0.43

G

B0 1

0 1

5th iteration

CF

D

E0 1

0 1

0 1

A G

B 0 1

0 1

0 1

6th iteration

86


Huffman Coding

Example: Use Huffman coding to encode the following symbols with the frequencies:A: 0.10 110 B: 0.25 10C: 0.05 0111D: 0.15 010E: 0.30 00F: 0.07 0110G: 0.08 111

CF

D

E0 1

0 1

0 1

A G

B 0 1

0 1

0 1

87


Huffman Coding

Example: Use Huffman coding to encode the following symbols with the frequencies:A: 0.10 110 B: 0.25 10C: 0.05 0111D: 0.15 010E: 0.30 00F: 0.07 0110G: 0.08 111What is the average number of bits required to encode a symbol?3 * 0.10 + 2 * 0.25 + 4 * 0.05 + 3 * 0.15 + 2 * 0.3 + 4 * 0.07 + 3 * 0.08 = 2.57

CF

D

E0 1

0 1

0 1

A G

B 0 1

0 1

0 1

88


Huffman Coding

Note on Huffman coding algorithm:

Huffman coding is a greedy algorithm. Replacing two subtrees with the smallest weight at each step leads to an optimal code in the sense that no binary prefix code for these symbols can encode these symbols using fewer bits (needs to be proved)

Note: there are many variations of Huffman coding.

Documents

14.1 Introduction to trees14 14.1 Introduction to trees[Theorem] An undirected graph is a tree iff there is a unique path between any two of its vertices. Proof: 1) ( → ) assume