Algorithm Design & Analysis Full

7/21/2019 Algorithm Design & Analysis Full

http://slidepdf.com/reader/full/algorithm-design-analysis-full 1/714

Lecture 01 Analysis of Algorithms

AlgorithmInput Output

An algorithm is a step-by-step procedure forsolving a problem in a finite amount of time.



[Lect01] Analysis of Algorithms 2

Running Time (1.1)

Most algorithms transforminput objects into outputobjects.

The running time of an

algorithm typically growswith the input size.

Average case time is oftendifficult to determine.

We focus on the worst caserunning time. Easier to analyze

Crucial to applications such asgames, finance and robotics

0

20

40

60

80

100

120

RunningTime

1000 2000 3000 4000

Input Size

best case

average case

worst case




Experimental Studies (1.6)

Write a programimplementing thealgorithm

Run the program withinputs of varying size andcomposition

Use a method likeSystem.currentTimeMillis() to

get an accurate measureof the actual running time

Plot the results 0

1000

2000

3000

4000

5000

6000

7000

8000

9000

0 50 100

Input Size

Time(ms)




Limitations of Experiments

It is necessary to implement thealgorithm, which may be difficult

Results may not be indicative of therunning time on other inputs not includedin the experiment.

In order to compare two algorithms, thesame hardware and softwareenvironments must be used




Theoretical Analysis

Uses a high-level description of the algorithm(pseudocode) instead of an implementation

Characterizes running time as a function ofthe input size, n .

Takes into account all possible inputs

Allows us to evaluate the speed of an

algorithm

independent of thehardware/software environment




Pseudocode (1.1)

High-level descriptionof an algorithm

More structured thanEnglish prose

Less detailed than aprogram

Preferred notation fordescribing algorithms

Hides program designissues

Algorithm arrayMax (A, n )

Input array A of n integersOutput maximum element of A

currentMax A[0]

for i 1 to n 1 do

if A[i] currentMax thencurrentMax A[i ]

return currentMax

Example: find maxelement of an array




Primitive Operations

Basic computationsperformed by an algorithm

Identifiable in pseudocode

Largely independent from theprogramming language

Exact definition not important(we will see why later)

Assumed to take a constantamount of time

Examples:

Performing anarithmetic ops

Assigning a valueto a variable

Indexing into anarray

Calling a method

Returning from amethod

Comparing twonumbers




Counting PrimitiveOperations (1.1)

By inspecting the pseudocode, we can determine themaximum number of primitive operations executed byan algorithm, as a function of the input size

Algorithm arrayMax (A, n ) # operationscurrentMax A[0] 2

for i 1 to n 1 do 1 + n

if A[i] currentMax then 2(n 1)

currentMax A[i ] 2(n 1){ increment counter i } 2(n 1)

return currentMax 1

Total 7n 2




Estimating Running Time

Algorithm arrayMax executes 7n 2 primitive

operations in the worst case. Define:

a = Time taken by the fastest primitive operation

b = Time taken by the slowest primitive operation

Let T (n ) be worst-case time of arrayMax. Thena (7n 2) T (n ) b (7n 2)

Hence, the running time T (n ) is bounded by twolinear functions




Growth Rate of Running Time

Changing the hardware/ softwareenvironment

Affects T (n ) by a constant factor, but Does not alter the growth rate of T (n )

The linear growth rate of the running

time T (n ) is an intrinsic property ofalgorithm arrayMax




Growth Rates

Growth rates offunctions: Constant 1

Logarithmic log n Linear n

N-Log-N n log n

Quadratic n 2

Cubic n 3

Exponential 2n

In a log-log chart,the slope of the linecorresponds to thegrowth rate of the

function

1E-1

1E+1

1E+31E+5

1E+7

1E+9

1E+11

1E+13

1E+15

1E+171E+19

1E+21

1E+23

1E+25

1E+27

1E+29

1E-1 1E+1 1E+3 1E+5 1E+7 1E+9

T ( n )

n

Cubic

Quadratic

Linear




Constant Factors

The growth rate isnot affected by

constant factors or

lower-order terms

Examples 102n

105 is a linearfunction

105

n 2

108

n is aquadratic function1E-1

1E+11E+3

1E+5

1E+7

1E+9

1E+11

1E+131E+15

1E+17

1E+19

1E+21

1E+23

1E+25

1E-1 1E+2 1E+5 1E+8

T ( n )

n

Quadratic

Quadratic

Linear

Linear




Big-Oh Notation (§1.2)Given functions f (n ) andg (n ), we say that f (n ) isO (g (n )) if there are

positive constants

c and n 0 such that

f (n ) cg (n ) for n n 0

Example: 2n + 10 is O (n )

2n + 10 cn

(c 2) n 10

n 10/(c 2)

Pick c = 3 and n 0 = 10

1

10

100

1,000

10,000

1 10 100 1,000

n

3n

2n+10

n




Big-Oh Example

Example: the functionn 2 is not O (n )

n 2 cn

n c

The above inequalitycannot be satisfiedsince c must be a

constant

1

10

100

1,000

10,000

100,000

1,000,000

1 10 100 1,000n

n^2

100n

10n

n




More Big-Oh Examples

7n-27n-2 is O(n)

need c > 0 and n0 1 such that 7n-2 c•n for n n0

this is true for c = 7 and n0 = 1

3n3 + 20n2 + 53n3 + 20n2 + 5 is O(n3)

need c > 0 and n0 1 such that 3n3 + 20n2 + 5 c•n3 for n n0


3 log n + log log n3 log n + log log n is O(log n)

need c > 0 and n0 1 such that 3 log n + log log n c•log n for n n0





Big-Oh and Growth Rate

The big-Oh notation gives an upper bound on thegrowth rate of a function

The statement “f (n ) is O (g (n ))” means that the growth

rate of f (n ) is no more than the growth rate of g (n )We can use the big-Oh notation to rank functionsaccording to their growth rate

f (n ) is O (g (n )) g (n ) is O (f (n ))

g (n ) grows more Yes No

f (n ) grows more No Yes

Same growth Yes Yes




Big-Oh Rules

If is f (n ) a polynomial of degree d , then f (n ) isO (n d ), i.e.,

1. Drop lower-order terms

2. Drop constant factors

Use the smallest possible class of functions

Say “2n is O (n )” instead of “2n is O (n 2

)” Use the simplest expression of the class

Say “3n + 5 is O (n )” instead of “3n + 5 is O (3n )”




Asymptotic Algorithm AnalysisThe asymptotic analysis of an algorithm determinesthe running time in big-Oh notation

To perform the asymptotic analysis We find the worst-case number of primitive operations

executed as a function of the input size We express this function with big-Oh notation

Example: We determine that algorithm arrayMax executes at most

7n 2 primitive operations

We say that algorithm arrayMax “runs in O (n ) time” Since constant factors and lower-order terms areeventually dropped anyhow, we can disregard themwhen counting primitive operations




The Importance of Asymptotics

An algorithm with an asymptotically slow runningtime is beaten in the long run by an algorithm withan asymptotically faster running time

Running Time 1 second 1 minute 1 hour

O(n) 2,500 150,000 9,000,000

O(n[log n]) 4,096 166,666 7,826,087

O(n 2) 707 5,477 42,426

O(n 4) 31 88 244

O(2 n) 19 25 31

Maximum Problem Size (n)




Computing Prefix Averages

We further illustrateasymptotic analysis withtwo algorithms for prefixaverages

The i -th prefix average ofan array X is average of thefirst (i + 1) elements of X :

A[i ] = (X [0] + X [1] + … + X [i ])/(i+1)

Computing the array A ofprefix averages of anotherarray X has applications tofinancial analysis

0

5

10

15

20

25

30

35

1 2 3 4 5 6 7

X

A




Prefix Averages (Quadratic)The following algorithm computes prefix averages inquadratic time by applying the definition

Algorithm pref ixAverages1 (X, n )

Input array X of n integers

Output array A of prefix averages of X #operations

A new array of n integers n

for i 0 to n 1 do n

s X [0] n

for j 1 to i do 1 + 2 + …+ (n 1)s s + X [ j ] 1 + 2 + …+ (n 1)

A[i ] s / (i + 1) n

return A 1




Arithmetic Progression

The running time ofprefixAverages1 isO (1 + 2 + …+ n )

The sum of the first n integers is n (n + 1)

/

2

There is a simple visualproof of this fact

Thus, algorithmprefixAverages1 runs inO (n 2) time

0

1

2

3

4

5

6

7

1 2 3 4 5 6




Prefix Averages (Linear)The following algorithm computes prefix averages inlinear time by keeping a running sum

Algorithm pref ixAverages2 (X, n )

Input array X of n integers

Output array A of prefix averages of X #operations

A new array of n integers n

s 0 1

for i 0 to n 1 do n

s s + X [i ] n A[i ] s / (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O (n ) time




properties of logarithms:

logb(xy) = logbx + logby

logb (x/y) = logbx - logbylogbxa = alogbx

logba = logxa/logxb

properties of exponentials:a(b+c) = aba c

abc = (ab)c

ab /ac = a(b-c)

b = a logab

bc = a c*logab

Summations (Sec. 1.3.1)

Logarithms and Exponents (Sec. 1.3.2)

Proof techniques (Sec. 1.3.3)

Basic probability (Sec. 1.3.4)

Math you need to Review




Intuition for AsymptoticNotation

Big-Oh

f(n) is O(g(n)) if f(n) is asymptotically less than or equal to g(n)

big-Omega

f(n) is (g(n)) if f(n) is asymptotically greater than or equal to g(n)

big-Theta

f(n) is (g(n)) if f(n) is asymptotically equal to g(n)

little-oh

f(n) is o(g(n)) if f(n) is asymptotically strictly less than g(n)little-omega

f(n) is (g(n)) if is asymptotically strictly greater than g(n)




Example Uses of theRelatives of Big-Oh

f(n) is (g(n)) if, for any constant c > 0, there is an integer constant n0 0 such that f(n) c•g(n) for n n0

need 5n02 c•n0 given c, the n0 that satisfies this is n0 c/5 0

5n2 is (n)

f(n) is (g(n)) if there is a constant c > 0 and an integer constant n0 1such that f(n) c•g(n) for n n0

let c = 1 and n0 = 1

5n2 is

(n)

f(n) is (g(n)) if there is a constant c > 0 and an integer constant n0 1such that f(n) c•g(n) for n n0

let c = 5 and n0 = 1

5n2 is (n2)




Time ComplexityTime complexity refers to the use ofasymptotic notation (O, , , o, ) indenoting running time

If two algorithms accomplishing the same taskbelong to two different time complexities: One will be faster than the other

As n is increased further, more benefit will begained from the faster algorithm

Faster algorithm is generally preferred




Time Complexity ComparisonSpeed comparison (fastest to slowest): Constant 1 (fastest) Logarithmic log n

Linear n

N-Log-N n log n

Quadratic n 2

Cubic n 3

Exponential 2n (slowest)

The speed here refers to the speed in solving theproblem, not the growth rate of time as mentionedearlier. A fast algorithm has lower growth rate than aslow algorithm



TCP2101 ADA 1

Review ofBasic Data Structures

Lecture 02a

S U f l STL C t i

http://fit.mmu.edu.my/




TCP2101 ADA 2

Some Useful STL Containers

Container Description

vector "Array" that grows automatically,

Best for rapid insertion and deletion at back.

Support direct access to any element via operator "[]".

set No duplicate key/element allowed.

Keys are automatically sorted.

Best for rapid lookup (searching) of key.

multiset set that allows duplicate keys.

map Collection of (key, value) pairs with non-duplicate key.

Pairs are automatically sorted by key.

Best for rapid lookup of key.

multimap map that allows duplicate keys.stack Last-in, first-out (LIFO) data structure.

queue First-in, first-out (FIFO) data structure.

STL

Cl





TCP2101 ADA 33

#include <iostream> #include <vector> using namespace std;int main() {

vector<int> v;v.push_back(4);v.push_back(2);v.push_back(7);v.push_back(6);

for (int i = 0; i < v.size(); i++)

cout << v[i] << " ";cout << endl;

// Same result as 'for' loop.for (vector<int>::iterator it = v.begin();

it != v.end();it++)

cout << *it << " ";

cout << endl;}

Output:4 2 7 64 2 7 6

Iterator type must

match container type.

Initialize iterator it to

the first element ofcontainer.

Move to the next

element.

Use iterator it like a

pointer.

STL vector Class

STL

t

Cl





TCP2101 ADA 4

A set is a collection of non-duplicate sorted elements called keys.

key_type is the data types of the key/element.

Use set when you want to fast search a sorted collection and you do

not need random access to its elements.

Use insert() method to insert an element into a set:

Duplicates are ignored when inserted.

iterator is required to iterate/visit the elements in set. Operator[] is

not supported.

Use find() method to look up a specified key in a set .

STL set Class

set <key_type> s;

set <int> s;s.insert (321);

STL

t

Cl





TCP2101 ADA 5

#include <iostream> #include <set> using namespace std;

int main() {set<int> s;s.insert (321);s.insert (-999);s.insert (18);s.insert (-999); // duplicate is ignored set<int>::iterator it = s.begin();

while (it != s.end())cout << *it++ << endl; // -999 18 321

int target;cout << "Enter an integer: ";cin >> target;it = s.find (target);if (it == s.end()) // not found

cout << target << " is NOT in set.";else

cout << target << " is IN set.";}

Output1:-99918

321Enter an integer:55 is NOT in set.

Use iterator toiterate the set.

Output2:-99918

321Enter an integer:321321 is IN set.

STL set Class

STL

lti t

Cl





TCP2101 ADA 6

#include <iostream> #include <set> using namespace std;

int main() { multiset<int> s;s.insert (321);s.insert (-999);s.insert (18);s.insert (-999); // duplicateset<int>::iterator it = s.begin();

while (it != s.end())cout << *it++ << endl;

}

Output:-999-999

18321

STL multiset Class

multiset allowsduplicate keys

STL

Cl





TCP2101 ADA 7

A map is a collection of (key,value) pairs sorted by the keys.

key_type and value_type are the data types of the key and

the value respectively.

In array the index is always int starting from 0, whereas in

map the key can be of other data type.

map cannot contain duplicate key ( multimap can).

STL map Class

map <key_type, value_type> m;

map <char, string> m;

m[' A '] = "Apple"; m[' A '] = "Angel"; // key 'A' already in the// map, new 'A' is ignored .// m['A'] is still "Apple".

STL

Class





TCP2101 ADA 8

#include <iostream> #include <string> #include < map> // map, multimap

using namespace std;int main() { map <char, string> m; m['C'] = "Cat"; // insert m['A'] = "Apple"; m['B'] = "Boy";cout << m['A'] << " " // retrieve

<< m['B'] << " " << m['C'] << endl;

map <char, string>::iterator it;it = m.begin(); while (it != m.end()) {

cout << it-> first << " " << it-> second << endl;

it++;}

char key;cout << "Enter a char: ";cin >> key;

it = m.find (key);if (it == m.end())

cout << key << " is NOT in map.";

elsecout << key << " is IN map.";

}

STL map Class

first refers to the key of

current element whereassecond refers to the value

of of current element

STL

Class





TCP2101 ADA 9


using namespace std;int main() { map <char, string> m; m['C'] = "Cat"; // insert m['A'] = "Apple"; m['B'] = "Boy";cout << m['A'] << " " // retrieve

<< m['B'] << " " << m['C'] << endl;

map <char, string>::iterator it;it = m.begin(); while (it != m.end()) {


it++;}

Output 1: Apple Boy Cat A AppleB Boy

C CatEnter a char: ZZ is NOT in map


it = m.find (key);if (it == m.end())



}

STL map Class

Output 2: Apple Boy Cat A AppleB Boy

C CatEnter a char: CC is IN map

STL

Class





TCP2101 ADA 10

Another way of inserting a new (key,value) pair into a map is to

use insert method and pair class.

The pair object and the map must have the same key type and

value type.

STL map Class

map <char,string> m; m.insert ( pair <char,string>('A',"Apple")); m.insert (pair<char,string>('A',"Angel"));

STL

m ltimap

Class





TCP2101 ADA 11

A multimap is similar to map but it allows duplicate keys.

However, insert method and a pair object must be usedwhen inserting a (key,value) pair into multimap.

The pair object and the multimap must have the same key

type and value type.

Operator [ ] is not supported. Iterator must be used to locate a

element.

STL multimap Class

multimap <char,string> mm; mm.insert ( pair <char,string>('A',"Apple"));

mm.insert (pair<char,string>('A',"Angel"));// mm has 2 elements with 'A' as key.

STL

multimap

Class





TCP2101 ADA 12


using namespace std;int main() { multimap <char,string> mm; mm.insert ( pair <char,string>('C',"Cat")); mm.insert ( pair<char,string>('A',"Apple")); mm.insert ( pair<char,string>('B',"Boy")); mm.insert ( pair<char,string>('A',"Angle"));

map <char, string>::iterator it;

it = mm.begin(); while (it != mm.end()) {


it++;}

Output 1: A Apple A AngleB Boy

C CatEnter a char: ZZ is NOT in map


it = mm.find (key);if (it == mm.end())



}

STL multimap Class

Output 2: A Apple A AngleB Boy

C CatEnter a char: CC is IN map

STL

stack

Class






TCP2101 ADA 13

STL stack ClassA stack is a Last-In-First-Out (LIFO) data structures, meaning

that the last item to push (insert) into the top of the stack will

be the first item to pop out (remove) from the top of the stack.

Sample applications:

1. Page-visited history in a Web browser

2. Undo sequence in a text editor

3. Chain of method calls in the Java Virtual Machine or C++

runtime environment

STL

stack

Class





TCP2101 ADA 14

#include <iostream> #include <stack> // STL stackusing namespace std;

int main() {stack <int> st;cout << "Push result: ";for (int i=0; i<5; i++) {

st. push(i); // push into stackcout << st.top() << " "; // check top item

}cout << "\nPop result : "; while (!st.empty()) {

cout << st.top() << " ";st. pop(); // remove the top item.

}}

Output:Push result: 0 1 2 3 4Pop result : 4 3 2 1 0

STL stack Class

The last item to

insert is the first

item to remove

STL

queue

Class





TCP2101 ADA 15

STL queue ClassA stack is a First-In-First-Out (FIFO) data structures, meaning

that the first item to push (insert/enqueue) into the back of the

queue will be the first item to pop out (remove/dequeue).

Sample applications:

1. Waiting lines

2. Access to shared resources (e.g., printer)

STL

queue

Class





TCP2101 ADA 16

#include <iostream>

#include <queue> // STL queueusing namespace std;

int main() {queue <int> q;cout << "Push result:\nfront,back\n";

for (int i=0; i<5; i++) {q. push(i); // push into queue// check front and back of the queuecout << q.front() << "," << q. back() << "\n";

}cout << "Pop result:\nfront,back\n"; while (!q.empty()) {

cout << q.front() << "," << q.back() << "\n";q. pop(); // pop from queue

}}

Output:Push result:front,back0,00,10,2

0,30,4Pop result:front,back0,41,42,43,44,4

STL queue Class

STL Container Efficiency





TCP2101 ADA 17

STL Container Efficiency

Contain

er

[] Insert Remove Find

vector (1) –

cango to any

valid position

directly.

O(n) –

insert atbeginning requires

shifting of all

elements to right

by one position.

O(n) –

remove atbeginning requires

shifting of all

elements to left by

one position.

O(n) –

if thetarget is the

last item.

set/

multiset

n/a O(lg n) O(lg n) O(lg n)

map O(lg n) O(lg n) O(lg n) O(lg n)

multim ap

n/a O(lg n) O(lg n) O(lg n)

stack n/a (1) – happen at

top.

(1) – happen at

top.

n/a

queue n/a (1) – happen at

back.

(1) – happen at

front.

n/a





1

Lecture 02b

Hash Tables



2

Review of Linked List

start

Each blue node is divided into two

sections, for the two members of

the Node struct.



3

Review of Linked List (cont.)

start

The right section is the

pointer called “next”.

The left section is

the info member.

A N d St t



4

A Node Struct

Template

template <typename T>

struct Node {

T info;

Node<T> *next;

};

The info member isfor the data. It can

anything (T), but it is

often the object of

another struct, usedas a record of

information.The next pointer stores

the address of a Node

of the same type! Thismeans that each node

can point to another

node.



5

Review of Linked List (cont.)

start

The last node doesn’t

point to another node, so

its pointer (called next) isset to NULL (indicated by

slash).

The start pointer would

be saved in the private

section of a datastructure class.

Li k d Li t



6

Linked List

Advantages

• Linked lists has 2 main advantages over

arrays.

• 1. Linked lists waste less memory for large

number of elements.

• In arrays, the wasted memory is the part of

the array not being utilized.

• In linked lists, the wasted memory is the

pointer in each node.

Li k d Li t



7

Linked List

Advantages (cont.)

• 2. Linked lists are faster than arrays on the

following 2 operations:

– insert new element at start or middle of link

lists.

– remove existing element from start or middle

of linked lists.

Li k d Li t



8

Linked List

Advantages (cont.)

… …5 3 7 2 1

Removing an element or inserting an

element at the middle of a linked list

is fast.

… …5 3 2 1



9

Inserting a Node at Front

element

start

All new nodes must be made in the heap, SO…



10

element

start

Node<T> *newNode = new Node<T>;

newNode

Inserting a Node at Front (cont.)



11

element

start

Node<T> *newNode = new Node<T>;newNode->info = element;


Now we have to store element into the node

newNode



12

element

start


newNode->next = start;


newNod

e



13

element

start


newNode->next = start;

start = newNode;


newNod

e



Linked List Implementation

• Study LinkedList.cpp

• The implementation is incomplete but

sufficient to implement a hash table.

14

Time Complexities



15

Time Complexities

for Linked List

• insertFront – we’ll insert at the head of the linked

list – ( 1 )

• Find/Delete – in the worst case, all nodes in the

linked list are checked, so it is ( n ) unorderedlist. E.g find the max/min must search the whole

list

• isEmpty – is ( 1 ), because we just test the

linked list to see if it is empty

• makeEmpty – is ( n ), because we need to

delete all nodes



16

Hash Table ADT

• The hash table is a table of elements that

have keys

• A hash funct ion is used for locating a

position in the table

• The table of elements is the set of data

acted upon by the hash table operations

Selected Hash Table ADT



17

Selected Hash Table ADT

Operations

• insert , to insert an element into a table

• retr ieve , to retrieve an element from the

table

• an operation to empty out the hash table



18

Fast Search

• A hash table uses a function of the

key value of an element to identify its

location in an array.• A search for an element can be done

in ( 1 ) time.

• The function of the key value is calleda hash funct ion .



19

Hash Functions

• The input into a hash function is a key

value

• The output from a hash function is an

index of an array (hash table) where theobject containing the key is located

• Example of a hash function:

h( k ) = k % 100

Example Using a



20

Example Using a

Hash Function

• Suppose our hash function is:

h( k ) = k % 100

• We wish to search for the object containing key

value 214• k is set to 214 in the hash function

• The result is 14

• The object containing key value 214 is stored atindex 14 of the array (hash table)

• The search is done in ( 1 ) time



21

Inserting an Element

• An element is inserted into a hash table

using the same hash function

h( k ) = k % 100

• To find where an element is to be inserted,use the hash function on its key

• If the key value is 214, the object is to be

stored in index 14 of the array

• Insertion is done in ( 1 ) time

Consider the Big Picture …



22

Consider the Big Picture …• If we have millions of key values, it may

take a long time to search a regular arrayor a linked list for a specific part number(on average, we might compare 500,000key values)

• Best search algorithm gives O(lg n)• lg 500000 ≈ 19

• 1 vs. 19

• Using a hash table, we simply have afunction which provides us with the indexof the array where the object containingthe key is located



23

Collisions

• Consider the hash function

– h( k ) = k % 100

• A key value of 393 is used for an object, and the

object is stored at index 93• Then a key value of 193 is used for a second

object; the result of the hash function is 93, but

index 93 is already occupied

• This is called a col l is ion

Birthday paradox



Birthday paradox

• Probability of having 2 people with samebirthday

• As you can see once Prob() > 0.5, it goes

up very quickly24

Birthday paradox



Birthday paradox

• p = probability of collision

• N = max no of entry in the hash table

• n is the min no of entry to cause a collision• Let N= 500,000 and p = 0.5

• n = (2 x 500,000 x 0.693)^0.5

= 833 entries• Thus, you must have way to handle

collisions unless you have infinite memory25

)1

1ln(2),(

p N N pn

How are Collisions



26

Resolved?

• The most popular way to resolve collisions is bychain ing, Linear prob ing and other method s

• Instead of having an array of objects, we have an arrayof linked lists, each node of which contains an object

• An element is still inserted by using the hash function --the hash function provides an index of a linked list, andthe element is inserted at the front of that (usually short)linked list

• When searching for an element, the hash function is

used to get the correct linked list, then the linked list issearched for the key (still much faster than comparing500,000 keys)



27

0

1

2

3

4

5

6

A hash table which is initiallyempty.

Every element is a LinkedList

object. Only the start pointer

of the LinkedList object isshown, which is set to NULL.

The hash function is:

h( k ) = k % 7

Example Using Chaining




28

0

1

2

3

4

5

6


h( k ) = k % 7

INSERT objectwith key 31

31 % 7 is 3


(cont.)




29

0

1

2

3

4

5

6

Note: The whole object is stored

but only the key value is shown


h( k ) = k % 7


31 % 7 is 3


(cont.)

31




30

0

1

2

3

4

5

6

9


h( k ) = k % 7


9 % 7 is 2


(cont.)

31




31

0

1

2

3

4

5

6

36


h( k ) = k % 7


36 % 7 is 1


(cont.)

9

31




32


(cont.)

0

1

2

3

4

5

6

42


h( k ) = k % 7


42 % 7 is 0

36

9

31




33

0

1

2

3

4

5

6

46


h( k ) = k % 7


46 % 7 is 4


(cont.)

42

36

9

31




34

0

1

2

3

4

5

6 20The hash function is:

h( k ) = k % 7


20 % 7 is 6


(cont.)

46

42

36

9

31




35

0

1

2

3

4

5

6

COLLISION occurs…


h( k ) = k % 7


2 % 7 is 2


(cont.)

20

46

42

36

9

31




36

0

1

2

3

4

5

6

But key 2 is just inserted in

the linked list


h( k ) = k % 7


2 % 7 is 2


(cont.)

20

46

42

36

9

31




37

0

1

2

3

4

5

6

The insert function of LinkedListinserts a new element at the

BEGINNING of the list


h( k ) = k % 7

INSERT object

with key 2

2 % 7 is 2


(cont.)

20

46

42

36

9

31




38

0

1

2

3

4

5

6

9


h( k ) = k % 7

INSERT object

with key 2

2 % 7 is 2


(cont.)

20

46

42

36

2

31




39

0

1

2

3

4

5

6

31


h( k ) = k % 7

INSERT object

with key 24

24 % 7 is 3


(cont.)

9

20

46

42

36

2

24




40

0

1

2

3

4

5

6


h( k ) = k % 7

**FIND** the

object with key 9

9 % 7 is 2


(cont.)

31

9

20

46

42

36

2

24




41

0

1

2

3

4

5

6

We search this linked list forthe object with key 9


h( k ) = k % 7

**FIND** the

object with key 9

9 % 7 is 2


(cont.)

31

9

20

46

42

36

2

24




42

0

1

2

3

4

5

6

Remember…the whole object isstored, only the key is shown


h( k ) = k % 7

**FIND** the

object with key 9

9 % 7 is 2


(cont.)

31

9

20

46

42

36

2

24




43

0

1

2

3

4

5

6

Does this object contain key 9?


h( k ) = k % 7

**FIND** the

object with key 9

9 % 7 is 2

p g g

(cont.)

31

9

20

46

42

36

2

24




44

0

1

2

3

4

5

6


h( k ) = k % 7

**FIND** the

object with key 9

9 % 7 is 2

p g g

(cont.)

Does this object contain key 9?No, so go on to the next object.

31

9

20

46

42

36

2

24




45


h( k ) = k % 7

**FIND** the

object with key 9

9 % 7 is 2

p g g

(cont.)

Does this object contain key 9?

31

9

20

46

42

36

2

24

0

1

2

3

4

5

6




46

0

1

2

3

4

5

6


h( k ) = k % 7

**FIND** the

object with key 9

9 % 7 is 2

p g g

(cont.)

Does this object contain key 9? YES, found it! Return the object.

31

9

20

46

42

36

2

24

Uniform Hashing



47

Uniform Hashing

• When the elements are spread evenly (or nearevenly) among the indexes of a hash table, it is

called uni form hashing

• If elements are spread evenly, such that thenumber of elements at an index is less than

some small constant, uniform hashing allows a

search to be done in ( 1 ) time

• The hash function largely determines whether ornot we will have uniform hashing

Ideal Hash Function



48

for Uniform Hashing

• The hash table size should be a prime number

that is not too close to a power of 2

• 31 is a prime number but is too close to a power

of 2

• 97 is a prime number not too close to a power of

2

• A good hash function might be:h( k ) = k % 97

Chaining Problem



Chaining Problem

• In nature there areClustering phenomena

• Some place are crowded

and most place are empty

• Thus, most place no entry

• Some entry has long chains

49

• Worst case = O(n) where n is the maxlength of the chain

• Additional memory is required during run

time

Collision Resolution: Linear Probing



g

• If collision, put the entry on the next free entry

• In the above case, collision at index 5, index 6,7 is filled, thus put the item atindex 8.

• Knuth’s parking problem.

• Let assume there are M indexes and the hash table is 50% full.

• Average search time for one item that exist in the hash table is 3/2 (searchhit). Either hash and found the item is there or found the item next to thehash value.

• Average search time for one item that not exist in the hash table is 5/2(search miss). Found the hash, the item does not match, find the next item,item no match, then the next item (It’s empty).

Hashing Linear Probing Problem



• When inserting 56, we hit index 12, weneed to probe till index 19 (7x probe)

before we can insert 56.

Linear Probing improvement



g p

Quadratic Probing

• Another open addressing strategy isknown as quadratic probing

• Rather than always moving one cell (linear

probing) when encounters collision, thisstrategy moves j 2 cells from the point of

collision, j is the number of attempts to

resolve the collision• The limitation of this strategy: It may not

find an empty cell, if the bucket array is at

least half full52

Chaining vs Linear Probing







• Linear Probing use fix amount of memory,Chaining require extra memory for link list

• Linear Probing is better if the % load is lowerthan 0.85

• Linear probing is better suited for caching,when you load an item the next item isalways loaded

• Clustering does happen with for all data type,thus, linear probing search time can be verylong if there are many collision


Speed vs.



55

Memory Conservation

• Speed comes from reducing the number ofcollisions

• In a search, if there are no collisions, the

first element in the linked list in the one wewant to find (fast)

• Therefore, the greatest speed comesabout by making a hash table much larger

than the number of keys (but there will stillbe an occasional collision)

Speed vs.



56

Memory Conservation

(cont.)• Each empty LinkedList object in a hash table

wastes 4 bytes of memory (4 bytes for the start

pointer)• The best memory conservation comes from

trying to reduce the number of empty LinkedListobjects

• The hash table size would be made muchsmaller than the number of keys (there wouldstill be an occasional empty linked list)

Hash Table Design



57

Hash Table Design

• Decide whether speed or memoryconservation is more important (and how

much more important) for the application

• Come up with a good table size which – Allows for the use of a good hash function

– Strikes the appropriate balance between

speed and memory conservation

Ideal Hash Tables



58

Ideal Hash Tables

• Can we have a hash function which guaranteesthat there will be no collisions?

• Yes:h( k ) = k

• Each key k is unique; therefore, each indexproduced from h( k ) is unique

• Consider 300 employees that have a 4 digit id

• A hash table size of 10000 with the hash

function above guarantees the best possiblespeed

Ideal Hash Tables



59

(cont.)

• Should we use LinkedList objects if there are nocollisions?

• Suppose each Employee object takes up 100 bytes

• An array size of 10000 Employee objects with only 300

used indexes will have 9700 unused indexes, eachtaking up 100 bytes

• Best to use LinkedList objects (in this case) – the 9700unused indexes will only use 4 bytes each

Ideal Hash Tables



60

(cont.)

• Can we have a hash table without any collisionsand without any empty linked lists?

• Sometimes. Consider 300 employees with id’s

from 0 to 299. We can make a hash table sizeof 300, and use h( k ) = k

• LinkedList objects wouldn’t be necessary and in

fact, would waste space

• Array is the best for this ideal case

Time Complexities



61

for Hash Table

• insert – we’ll insert at the head of the linked list –( 1 )

• retrieve – element is found by hashing, so it is

( 1 ) for uniform hashing (the hash function andhash table are designed so that the length of the

collision list is bounded by some small constant)

Hash table issuesHash table are not cache friendly & memory



• Hash table are not cache friendly & memory

inefficient for very large data

• E.g. Router needs to have approximately500,000 prefixes in the router. Each prefixes

takes 8 bytes.

• As you know if hash table > 50% full thenperformance may drop dramatically. So you

keep it < 50%

• You need a memory of 500,000 x 2 (50%) x 8

bytes = 8,000,000 bytes• = 8 Mbytes just to store the whole table in the

router memory for fast lookup

• Acceptable for routers 62

Hash table issue 2E L t h 50 billi URL t t i



• E.g. Let say you have 50 billion URL to store in

your Internet Cache Server using hashing

• Each URL takes about 1 KBytes

• Assume 50% hash table capacity

• Total storage required is

• 50 billion x 1 KB x 2 (50%) = 10 billion KB

• = 10 Terabytes (Hardisk technology)

• You cannot have 10 TB RAM but can have 10

TB hardisk, thus caching is any issue• Each time you hash you need to fetch it from the

disk since it random= Slow.

63

Reference



Reference

• Childs, J. S. (2008). Methods for MakingData Structures. C++ Classes and Data

Structures. Prentice Hall.

64



1

Lec 03aBinary Search Tree

Definition of Tree



2

Definition of Tree

• A tree is a set of linked nodes, such thatthere is one and only one path from a

unique node (called the roo t node) to

every other node in the tree.• A path exists from node A to node B if one

can follow a chain of pointers to travel

from node A to node B.

Paths



3

A

E

F

B

C

D

G

A set of linked nodes

Paths

There is one path from A to B

There is a path from D to B

There is also a second path

from D to B.

Paths (cont )



4

A

E

F

B

C

D

G

There is no path from C to any

other node.

Paths (cont.)

Cycles



5

Cycles

• There is no cyc le (circle of pointers) in atree.

• Any linked structure that has a cycle would

have more than one path from the rootnode to another node.

Example of a Cycle



6

Example of a Cycle

A

C

D

B

E

C → D → B → E → C

Tree Cannot Have a

C l



7

Cycle

A

C

D

B

E2 paths exist from A to C:

1. A → C

2. A → C → D → B → E → C

Example of a Tree



8

Example of a Tree

A

C B D

E F

G

root

In a tree, every

pair of linked

nodes have a

parent-child

relationship (the

parent is closerto the root)

Example of a Tree

( t )



9

(cont.)

A

C B D

E F

G

root For example, C is a

parent of G

Example of a Tree

( t )



10

(cont.)

A

C B D

E F

G

rootE and F are

children of D

Example of a Tree

( t )



11

(cont.)

A

C B D

E F

G

root The root node is theonly node that has no

parent.

Example of a Tree

( t )



12

(cont.)

A

C B D

E F

G

root Leaf nodes (orleaves for short)

have no children.

Binary Trees



13

Binary Trees

• A binary tree is a tree in which each node

can only have up to two children…

NOT a Binary Tree



14

NOT a Binary Tree

A

B C

I K D E F

JG H

root C has 3 child nodes.

Example of a Binary Tree



15

A

B C

I K E

J G H

rootThe links in a tree

are often called

edges

Levelst



16

A

B C

I K E

J G H

rootlevel 0

level 1

level 2

level 3

The level of a node is the number of edges in the path

from the root node to this node

Full Binary Tree



17

B

D

H

root

I

A

E

J K

C

F

L M

G

N O

In a ful l binary tree , each node has two children except for

the nodes on the last level, which are leaf nodes

Complete Binary Trees



18

p y

• A complete binary tree is a binary treethat is either

– a full binary tree

– OR – a tree that would be a full binary tree but it is

missing the rightmost nodes on the last level

NOT a Complete Binary Trees



19

B

D

H

root A

E

C

F G

I Missing non-rightmost

nodes on the last level

Complete Binary Trees

(cont )



20

(cont.)

B

D

H

root

I

A

E

J K

C

F

L

G

Missing rightmost

nodes on the last

level

Complete Binary Trees(cont.)



21

( )

B

D

H

root

I

A

E

J K

C

F

L M

G

N O

A full binary tree is

also a complete binary

tree.

Binary Search Trees



22

y

• A binary search tree is a binary tree thatallows us to search for values that can beanywhere in the tree.

• Usually, we search for a certain key value,and once we find the node that contains it,we retrieve the rest of the info at thatnode.

Properties of

Binary Search Trees



23

Binary Search Trees

• A binary search tree does not have to be acomplete binary tree.

• For any particular node,

– the key in its left child (if any) is less than itskey.

– the key in its right child (if any) is greater than

or equal to its key.

• Left < Parent <= Right.

Binary Search Tree

Node



24

Node

template <typename T>

BSTNode {

T info;

BSTNode<T> *left;BSTNode<T> *right;

};

The implementation

of a binary search

tree usually just

maintains a single

pointer in the private

section called roo t ,

to point to the root

node.

Inserting Nodes

Into a BST



25

Into a BST

37, 2, 45, 48, 41, 29, 20, 30, 49, 7

Objects that need to be inserted (only key values areshown):

root:

NULL

BST starts off empty

Inserting NodesInto a BST (cont.)



26

o a S (co )

37, 2, 45, 48, 41, 29, 20, 30, 49, 7

root

37




27

( )

2, 45, 48, 41, 29, 20, 30, 49, 7

root

37

2 < 37, so insert 2 on the

left side of 37




28

( )

2, 45, 48, 41, 29, 20, 30, 49, 7

root

37

2




29

( )

45, 48, 41, 29, 20, 30, 49, 7

root

37

2

45 > 37, so insert it at the right of 37




30

( )

45, 48, 41, 29, 20, 30, 49, 7

root

37

245




31

( )

48, 41, 29, 20, 30, 49, 7

root

37

245

When comparing, we always

start at the root node

48 > 37, so look to the right




32

( )

48, 41, 29, 20, 30, 49, 7

root

37

245

This time, there is a node alreadyto the right of the root node. We

then compare 48 to this node

48 > 45, and 45 has no right child,

so we insert 48 on the right of 45




33

( )

48, 41, 29, 20, 30, 49, 7

root

37

245

48




34

( )

41, 29, 20, 30, 49, 7

root

37

245

4841 > 37, so look tothe right

41 < 45, so look to

the left – there is no

left child, so insert




35

( )

41, 29, 20, 30, 49, 7

root

37

245

4841




36

( )

29, 20, 30, 49, 7

root

37

245

4841

29 < 37, left

29 > 2, right




37

( )

29, 20, 30, 49, 7

root

37

245

484129




38

20, 30, 49, 7

root

37

245

48412920 < 37, left

20 > 2, right

20 < 29, left




39

20, 30, 49, 7

root

37

245

484129

20




40

30, 49, 7

root

37

245

484129

2030 < 37

30 > 2

30 > 29




41

30, 49, 7

root

37

245

484129

20 30




42

49, 7

root

37

245

484129

20 30

49 > 37

49 > 4549 > 48




43

49, 7

root

37

245

484129

20 30 49




44

7

root

37

245

484129

20 30 49

7 < 37

7 > 2

7 < 29

7 < 20




45

7

root

37

245

484129

20 30 49

7




46

root

37

245

484129

20 30 49All elements havebeen inserted

7

Searching for aKey in a BST



47

root

37

245

484129

20 30 49

Searching for a

key in a BST usesthe same logic

7Key to search for: 29

Searching for aKey in a BST (cont.)



48

root

37

245

484129

20 30 49

Key to search for: 29

29 < 37

7




49

root

37

245

484129

20 30 49


29 > 2

7




50

root

37

245

484129

20 30 49


29 == 29

FOUND IT!

7




51

root

37

245

484129

20 30 49





52

root

37

245

484129

20 30 49


3 < 7

7

3 < 20

3 < 293 > 2

3 < 37




53

root

37

245

484129

20 30 49


When the child pointer

you want to follow isset to NULL, the key

you are looking for is

not in the BST

7

Time Complexities



54

• If the binary search tree happens to be acomplete binary tree:

– the time for insertion is ( lg n )

– the time for the search is O( lg n )• Search in array is O( n )

• However, we could run into some bad

luck…

Bad Luckroot



55

2, 7, 20, 29, 30, 37, 41, 45, 48, 49

root 2

7

20

29

30

37

41

45

48

49

Exactly the same keys wereinserted into this BST – but

they were inserted in a

different order (the order

shown below)

Bad Luck (cont.)root 2



56

2, 7, 20, 29, 30, 37, 41, 45, 48, 49

root 2

7

20

29

30

37

41

45

48

49

This is some bad luck, but aBST can be formed this way

Bad Luck (cont.)root 2



57

2, 7, 20, 29, 30, 37, 41, 45, 48, 49

root 2

7

20

29

30

37

41

45

48

49

Using the “tightest” possiblebig-oh notation, the insertion

and search time is O( n )

Balanced vs. Unbalanced



58

• If a BST takes ( lg n ) time for insertion,and O( lg n ) time for a search, we say it is

a balanced binary search tree

• If a BST take O( n ) time for insertion andsearching, we say it is an unbalanced

binary search tree

Deleting a BST Node

D l ti d i BST i littl t i k



59

• Deleting a node in a BST is a little tricky –

it has to be deleted so that the resultingstructure is still a BST with each nodegreater than its left child and less than itsright child.

• Deleting a node is handled differentlydepending on whether the node:

– has no children

– has one child

– has two children

Deletion Case 1:No Children



60

root

37

245

484129

20 30 49

Node 49 has no

children – todelete it, we just

remove it

Deletion Case 1:No Children (cont.)



61

root

37

245

484129

20 30

Deletion Case 2:One Child



62

root

37

245

484129

20 30 49

Node 48 has one

child – to delete

it, we just splice

it out

Deletion Case 2:One Child (cont.)



63

root

37

245

484129

20 30 49

Node 48 has one

child – to delete

it, we just splice

it out




64

root

37

245

4129

20 30 49




65

root

37

245

484129

20 30 49

Another example:

node 2 has one child

– to delete it we also

splice it out




66

root

37

245

484129

20 30 49

Another example:

node 2 has one child

– to delete it we also

splice it out

Deletion Case 2:One Child (cont.)t



67

root

37

45

484129

20 30 49

Deletion Case 3:Two Children

t



68

root

37

245

484129

20 30 49

Node 37 has two

children…

Deletion Case 3:Two Children (cont.)

t



69

root

37

245

484129

20 30 49

to delete it, first we

find the greatest

node in its leftsubtree


t



70

root

37

245

484129

20 30 49

First, we goto the left

once, then

follow the

right pointers

as far as wecan


t



71

root

37

245

484129

20 30 49

30 is the greatest

node in the leftsubtree of node 37


t



72

root

37

245

484129

20 30 49

Next, we copy the

object at node 30into node 37

Deletion Case 3:Two Children (cont.)root



73

root

30

245

484129

20 49

Finally, we delete

the lower red node

using case 1 or

case 2 deletion




74

root

30

245

484129

20 49Let’s delete node 30

now




75

root

30

245

484129

20 49

29 is the greatest node

in the left subtree ofnode 30




76

root

30

245

484129

20 49

Copy the object at node

29 into node 30




77

root

29

245

484129

20 49

This time, the lower red

node has a child – to deleteit we use case 2 deletion




78

root

29

245

484129

20 49

This time, the lower red

node has a child – to deleteit we use case 2 deletion




79

root

29

245

4841

20 49

Deletion Time Complexity



80

• In all cases, we must find the node we wish todelete first, using the standard search method.

• Finding the greatest node in the left subtree is

just a continuation of a path down the BST

• For balanced BST’s, the time complexity for

deletion is O( lg n ) in all 3 cases

• For unbalanced BST’s, the time complexity is

O( n ) in all 3 cases

Traversing a BST



81

• There are 3 ways to traversal a BST (visitevery node in BST):

• 1. Preorder (parent → left → right)

– Root is output first• 2. Inorder (left → parent → right)

– Output is sorted

• 3. Postorder (left → right → parent) – Root is output last

Traversing a BST (cont.)



82

• Based on the BST on page 47, the resultof traversal:

• 1. Preorder (parent → left → right)

– 37 (root first) 2 29 20 7 30 45 41 48 49• 2. Inorder (left → parent → right)

– 2 7 20 29 30 37 41 45 46 49 (sorted)

• 3. Postorder (left → right → parent) – 7 20 30 29 2 41 49 48 45 37 (root last)

Time Complexities for BST



83

• Balanced BST – Insertion is ( lg n )

– Search is O( lg n )

– Deletion is O( lg n )

• Unbalanced BST – Insertion is O( n )

– Search is O( n )

– Deletion is O( n )

References



• Childs, J. S. (2008). Trees. C++ Classesand Data Structures. Prentice Hall.

84



1

Lecture 03b

Priority Queues, and Heaps

Priority Queue ADT

• The data in a pr ior i ty queue is



2

p y q

(conceptually) a queue of elements• The “queue” can be thought of as sorted

with the largest in front, and the smallest

at the end – Its physical form, however, may differ from

this conceptual view considerably

Priority Queue ADT

Operations



3

• enqueue , an operation to add an elementto the queue

• dequeue , an operation to take the largest

element from the queue

• an operation to determine whether or not

the queue is empty

• an operation to empty out the queue

Another Priority Queue ADT



4

• A priority queue might also be designed todequeue the element with the minimum value

• Priority queues are generally not designed to

arbitrarily dequeue both minimum and maximum

values, whichever the client wants at anyparticular time

• We will only consider priority queues that

dequeue maximum values (can be easilymodified for dequeuing minimum values)

Priority QueueImplementation



5

• To implement a priority queue, an arraysorted in descending order comes to mind

• Dequeuing from a sorted array is easy – just get the value at the current front and

increment a front index – this is ( 1 ) time• However, enqueuing into a sorted array

would take some time – the element would

have to be inserted into its proper positionin the array…

Enqueuing an Element



6

211 204 201 81 79 70 69 67 7 5… …

0 1 2 48 49 50 51 52 98 99 100

In this array of elements, each element might be

an object, but only the data member considered

for maximum value is shown.

Enqueuing an Element(cont.)



7

0 1 2 48 49 50 51 52 98 99 100

Suppose that element 71 needs to be enqueued.

We could enqueue 71 by using a loop.

item: 71

211 204 201 81 79 70 69 67 7 5… …




8

0 1 2 48 49 50 51 52 98 99 100

item: 71

i

for ( i = 100; i > 0 && arr[ i - 1 ] < item; i-- )

arr[ i ] = arr[ i – 1 ];

211 204 201 81 79 70 69 67 7 5… …




9

0 1 2 48 49 50 51 52 98 99 100

item: 71

i

for ( i = 100; i > 0 && arr[ i - 1 ] < item; i-- )

arr[ i ] = arr[ i – 1 ];

211 204 201 81 79 70 69 67 7 5… … 5




10

0 1 2 48 49 50 51 52 98 99 100

item: 71

i

for ( i = 100; i > 0 && arr[ i - 1 ] < item; i-- )

arr[ i ] = arr[ i – 1 ];

211 204 201 81 79 70 69 67 7 5… … 5




11

0 1 2 48 49 50 51 52 98 99 100

item: 71

i

for ( i = 100; i > 0 && arr[ i - 1 ] < item; i-- )

arr[ i ] = arr[ i – 1 ];

211 204 201 81 79 70 69 67 7 7… … 5




12

0 1 2 48 49 50 51 52 98 99 100

item: 71

i

for ( i = 100; i > 0 && arr[ i - 1 ] < item; i-- )

arr[ i ] = arr[ i – 1 ];

211 204 201 81 79 70 69 67 7 7… … 5

This process

continues and i

eventuallybecomes 51




13

0 1 2 48 49 50 51 52 98 99 100

item: 71

i

for ( i = 100; i > 0 && arr[ i - 1 ] < item; i-- )

arr[ i ] = arr[ i – 1 ];

211 204 201 81 79 70 69 69 20 7… … 5

This process

continues and i

eventuallybecomes 51




14

0 1 2 48 49 50 51 52 98 99 100

item: 71

i

for ( i = 100; i > 0 && arr[ i - 1 ] < item; i-- )

arr[ i ] = arr[ i – 1 ];

211 204 201 81 79 70 70 69 20 7… … 5




15

0 1 2 48 49 50 51 52 98 99 100

item: 71

i

for ( i = 100; i > 0 && arr[ i - 1 ] < item; i-- )

arr[ i ] = arr[ i – 1 ];

211 204 201 81 79 70 70 69 20 7… … 5




16

0 1 2 48 49 50 51 52 98 99 100

item: 71

i

for ( i = 100; i > 0 && arr[ i - 1 ] < item; i-- )

arr[ i ] = arr[ i – 1 ];

FALSE

211 204 201 81 79 70 70 69 20 7… … 5




17

0 1 2 48 49 50 51 52 98 99 100

item: 71

i

211 204 201 81 79 71 70 69 20 7… … 5

Now we can use:

arr[ i ] = item;

to enqueue the element



Using a Heap to Implement aPriority Queue



19

• An alternative to using a sorted array for apriority queue is to use a heap

• Here, heap does not mean an area of

memory used for dynamic allocation –rather, it is a data structure

• Enqueue in a heap is a O( lg n ) operation

• Dequeue in a heap is a O( lg n ) operation



Comparing Operations(cont.)



21

• In the heap, each enqueue-dequeue pairof operations takes O( lg n ) + O( lg n )time, giving us an overall time complexityof O( lg n ) per pair of operations

• The heap is usually better, although thearray can be good in situations where agroup of initial elements are supplied and

sorted, then only dequeue operations areperformed (no enqueue operations)

Heaps

• A heap is a complete binary tree in which the



22

• A heap is a complete binary tree in which thevalue of each node is greater than or equal tothe values of its children (if any) – Parent >= Children

• Technically, this is called a maxheap • In a minheap , the value of each node is less

than or equal to the values of its children

• A maxheap can be easily modified to make a

minheap• In our discussion, the word “heap” will refer to a

maxheap

Example of a Heap

root



23

39

16

14 3

46

28

232

29

24

Only the data member is shownthat we want to prioritize

Example of a Heap(cont.)

46root



24

39

16

14 3

46

28

232

29

24

Where is the greatest valuein a heap always found?

Root

Dequeue



25

• Dequeuing the object with the greatestvalue appears to be a ( 1 ) operation

• However, after removing the object, wemust turn the resultant structure into a

heap again, for the next dequeue• Fortunately, it only takes O( lg n ) time to

turn the structure back into a heap again

(which is why dequeue in a heap is aO( lg n ) operation

Dequeue (cont.)

46root



26

39

16

14 3

28

232

29 15

24

5

Dequeue (cont.)46

rootremElement: 46



27

39

16

14 3

28

232

29 15

24

5

Save the root object

in remElement

5root

remElement: 46

Dequeue (cont.)



28

39

16

14 3

28

232

29 15

24

5

Copy object in last

node into root object

5root

remElement: 46

Dequeue (cont.)



29

39

16

14 3

28

232

29 15

24

Remove last node

5root

remElement: 46

Dequeue (cont.)



30

39

16

14 3

28

232

29 15

24

If the value of the greatest

child of 5 is greater than 5,

then swap with the greatestchild

Greatest Child

39 > 5, so

swap

rootremElement: 46

Dequeue (cont.)39



31

16

14 3

5 28

232

29 15

24

Notice that 39 is correctly

placed – it is guaranteed to

be greater than or equal toits two children

rootremElement: 46

Dequeue (cont.)39



32

16

14 3

5 28

232

29 15

24

It was the greatest child, so

it is greater than the other

child (28)

rootremElement: 46

Dequeue (cont.)39



33

16

14 3

5 28

232

29 15

24

It is greater than the

element it was swapped

with (5), or else we wouldn’thave swapped.



rootremElement: 46

Dequeue (cont.)39



35

16

14 3

5 28

232

29 15

24




rootremElement: 46

Dequeue (cont.)39



36

16

14 3

5 28

232

29 15

24




Greatest

Child

rootremElement: 46

Dequeue (cont.)39



37

16

14 3

5 28

232

29 15

24




32 > 5, so

swap

rootremElement: 46

Dequeue (cont.)39



38

16

14 3

28

2

32

29 15

24




5

rootremElement: 46

Dequeue (cont.)39



39

16

14 3

28

2

32

29 15

245



then swap with the greatestchildGreatest Child

rootremElement: 46

Dequeue (cont.)39



40

16

14 3

28

2

32

29 15

245



then swap with the greatestchildGreatest Child

29 > 5, so

swap

rootremElement: 46

Dequeue (cont.)39



41

16

14 3

28

2

32

29

15

24

5




rootremElement: 46

Dequeue (cont.)39



42

16

14 3

28

2

32

29

15

24

5The final result is a heap!

rootremElement: 46

Dequeue (cont.)39



43

16

14 3

28

2

32

29

15

24

5

Sometimes, it is not

necessary to swap all the

way down through theheap

rootremElement: 46

Dequeue (cont.)39



44

16

14 3

28

2

32

29

15

24

5

If 5 would have been

greater than or equal to

both of its children, wewould stop there

Heapify• The process of swapping downwards to form a

new heap is called heapify ing



45

• When, we heapify, it is important that the rest ofthe structure is a heap, except for the root nodethat we are starting off with; otherwise, a newheap won’t be formed

• A loop is used for heapifying; the number oftimes through the loop is always lg n or less,which gives the O( lg n ) complexity

• Each time we swap downwards, the number of

nodes we can travel to is reduced byapproximately half

39root

value to enqueue: 37

Enqueue



46

32

16

14 3

28

229

5 15

24

39root


Enqueue (cont.)



47

32

16

14 3

28

229

5 15

24

Create a new node

in the last position

39root


Enqueue (cont.)



48

32

16

14 3

28

229

5 15

24

Place the value to

enqueue in the last

node

37

39root

Enqueue (cont.)



49

32

16

14 3

28

229

5 15

24

37If 37 is larger than

its parent, swap

39root

Enqueue (cont.)



50

32

16

14 3

28

229

5 15

24


its parent, swap

37 > 24,

so swap

39root

Enqueue (cont.)



51

32

16

14 3

28

229

5 15

37


its parent, swap

39root

Enqueue (cont.)



52

32

16

14 3

28

229

5 15

37


its parent, swap

37 > 28,

so swap

39root

Enqueue (cont.)



53

32

16

14 3

229

5 15

37

24

28

Notice that 37 is

guaranteed to be

greater than orequal to its children

39root

Enqueue (cont.)



54

32

16

14 3

229

5 15

37

24

28

It was greater than the

value swapped with

(28), or we wouldn’thave swapped…

39root

Enqueue (cont.)



55

32

16

14 3

229

5 15

37

24

28

and 28 was greater

than the other node

(2) or it wouldn’t havebeen a heap…

39root

Enqueue (cont.)



56

32

16

14 3

229

5 15

37

24

28

so 37 must be greater

than the other node

(2) as well.

39root

Enqueue (cont.)



57

32

16

14 3

229

5 15

37

24

28

If 37 is larger than

its parent, swap

39root

Enqueue (cont.)37 < 39, so

don’t swap



58

32

16

14 3

229

5 15

37

24

28

If 37 is larger than

its parent, swap

39root

Enqueue (cont.)



59

32

16

14 3

229

5 15

37

24

28

The result is a heap!

39root

Enqueue (cont.)



60

32

16

14 3

229

5 15

37

24

28

Enqueue uses a loop,

and it is a O( lg n )

operation (swappingin reverse)

Implementing a Heap

• Although it is helpful to think of a heap as a

li k d t t h i li i th



61

linked structure when visualizing the enqueue

and dequeue operations, it is often implemented

with an array

• Don’t get mixed up with implementing a priorityqueue (PQ) and implementing a heap

• We have discussed earlier that to implement a

PQ, heap is better than array overall

• We are discussing implementing a heap here

• Let’s number the nodes of a heap, starting with

0, going top to bottom and left to right…

Implementing a Heap with Array

39

root46

280



62

39

16

14 3

28

25

29 15

24

5 7 18 17

0

1 2

3 4 5 6

7 8 9 10 11 12 13 14

11 9

1615

32

Heap Properties

39

root46

280



63

39

16

14 3

28

25

29 15

24

5 7 18 17

0

1 2

3 4 5 6

7 8 9 10 11 12 13 14

11 9

1615

32

(left child #) = 2 * (parent #) + 1

Heap Properties(cont.)

39

root46

280



64

39

16

14 3

28

25

29 15

24

5 7 18 17

0

1 2

3 4 5 6

7 8 9 10 11 12 13 14

11 9

1615

32


39

root46

280




65

39

16

14 3

28

25

29 15

24

5 7 18 17

0

1 2

3 4 5 6

7 8 9 10 11 12 13 14

11 9

1615

32


39

root46

280




66

39

16

14 3

28

25

29 15

24

5 7 18 17

0

1 2

3 4 5 6

7 8 9 10 11 12 13 14

11 9

1615

32


39

root46

280




67

39

16

14 3

28

25

29 15

24

5 7 18 17

0

1 2

3 4 5 6

7 8 9 10 11 12 13 14

11 9

1615

32

(right child #) = 2 * (parent #) + 2

39

root46

280




68

39

16

14 3

28

25

29 15

24

5 7 18 17

0

1 2

3 4 5 6

7 8 9 10 11 12 13 14

11 9

1615

32


39

root46

280




69

39

16

14 3

28

25

29 15

24

5 7 18 17

0

1 2

3 4 5 6

7 8 9 10 11 12 13 14

11 9

1615

32


39

root46

280




70

39

16

14 3

28

25

29 15

24

5 7 18 17

0

1 2

3 4 5 6

7 8 9 10 11 12 13 14

11 9

1615

32

(parent #) = (child # - 1) / 2(using integer division)

39

root46

280




71

39

16

14 3

28

25

29 15

24

5 7 18 17

0

1 2

3 4 5 6

7 8 9 10 11 12 13 14

11 9

1615

32


39

root46

280




72

39

16

14 3

28

25

29 15

24

5 7 18 17

0

1 2

3 4 5 6

7 8 9 10 11 12 13 14

11 9

1615

32


Array Implementation of Heap

• These remarkable properties of the heap

ll t l th l t i t



73

allow us to place the elements into an

array

• The red numbers on the previous slide

correspond to the array indexes

Array Implementation of Heap(cont.)



74

46 39 28 16 32 24 25 14 3 29 15 5 7 18 17 11 9

So now, this is our heap. It has no linkednodes, so it is much easier to work with.

Let’s dequeue an element.

The highest value is stored in the root node(index 0).

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16




75

46 39 28 16 32 24 25 14 3 29 15 5 7 18 17 11 9

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

remElement: 46Now we need to move the object atthe last node to the root node

We need to keep track of the object

in the last position of the heap

using a heapsize variable




76

46 39 28 16 32 24 25 14 3 29 15 5 7 18 17 11 9

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

remElement: 46Now we need to move the object atthe last node to the root node

We need to keep track of the object

in the last position of the heap

using a heapsize variable

heapsize: 17




77

46 39 28 16 32 24 25 14 3 29 15 5 7 18 17 11 9

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

remElement: 46

heapsize: 17

Now we can access the object

in the last node using

elements[ heapsize – 1 ] and

assign it to elements[ 0 ]




78

9 39 28 16 32 24 25 14 3 29 15 5 7 18 17 11 9

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

remElement: 46

heapsize: 17

Now we can access the object

in the last node using

elements[ heapsize – 1 ] and

assign it to elements[ 0 ]

9 39 28 16 32 24 2 3 29 1 1 9




79

9 39 28 16 32 24 25 14 3 29 15 5 7 18 17 11 9

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

remElement: 46

heapsize: 17

Next, decrement the heap size

9 39 28 16 32 24 25 14 3 29 15 5 7 18 17 11 9




80

9 39 28 16 32 24 25 14 3 29 15 5 7 18 17 11 9

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

remElement: 46

heapsize: 16

Next, decrement the heap size

9 39 28 16 32 24 25 14 3 29 15 5 7 18 17 11 9




81

9 39 28 16 32 24 25 14 3 29 15 5 7 18 17 11 9

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

remElement: 46

heapsize: 16

The value at index 16 can’t

be used anymore; it will be

overwritten on the next

enqueue



9 39 28 16 32 24 25 14 3 29 15 5 7 18 17 11 9




83

9 39 28 16 32 24 25 14 3 29 15 5 7 18 17 11 9

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

remElement: 46

heapsize: 16By using the formulas we noted earlier

(this is why an array can be used)…

9 39 28 16 32 24 25 14 3 29 15 5 7 18 17 11 9


Greatest Child 39 > 9, so swap



84

9 39 28 16 32 24 25 14 3 29 15 5 7 18 17 11 9

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

remElement: 46

heapsize: 16

(left child #) =

2*(parent #) + 1 =

2 * 0 + 1 =

1

(right child #) =

2*(parent #) + 2 =

2 * 0 + 2 =

2

39 28 16 32 24 25 14 3 29 15 5 7 18 17 11 9


9



85

39 28 16 32 24 25 14 3 29 15 5 7 18 17 11 9

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

remElement: 46

heapsize: 16

9

If the greatest child of 9 is

greater than 9, then swap

39 28 16 32 24 25 14 3 29 15 5 7 18 17 11 9


9




86

39 28 16 32 24 25 14 3 29 15 5 7 18 17 11 9

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

remElement: 46

heapsize: 16

9

(left child #) =

2*(parent #) + 1 =

2 * 1 + 1 =

3

(right child #) =

2*(parent #) + 2 =

2 * 1 + 2 =

4

39 28 1632 24 25 14 3 29 15 5 7 18 17 11 9


9



87

39 28 1632 24 25 14 3 29 15 5 7 18 17 11 9

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

remElement: 46

heapsize: 16

9



39 28 1632 24 25 14 3 29 15 5 7 18 17 11 9


9




88

39 28 1632 24 25 14 3 29 15 5 7 18 17 11 9

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

remElement: 46

heapsize: 16

9

(left child #) =

2*(parent #) + 1 =

2 * 4 + 1 =

9

(right child #) =

2*(parent #) + 2 =

2 * 4 + 2 =

10

39 28 1632 24 25 14 329 15 5 7 18 17 11 9


9



89

39 28 1632 24 25 14 329 15 5 7 18 17 11 9

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

remElement: 46

heapsize: 16

9



39 28 1632 24 25 14 329 15 5 7 18 17 11 9


9



90

39 28 1632 24 25 14 329 15 5 7 18 17 11 9

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

remElement: 46

heapsize: 16

9

(left child #) =

2*(parent #) + 1 =

2 * 9 + 1 =

1919 > heapsize

39 28 1632 24 25 14 329 15 5 7 18 17 11 9


9



91

39 28 1632 24 25 14 329 15 5 7 18 17 11 9

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

remElement: 46

heapsize: 16

9

(left child #) =

2*(parent #) + 1 =

2 * 9 + 1 =

19

so 9 must be a leaf node

(we can stop)



39 28 1632 24 25 14 329 15 5 7 18 17 11 9


9



93

39 28 1632 24 25 14 329 15 5 7 18 17 11 9

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

heapsize: 16

9

When enqueuing, the parent is

always found by using the parent

formula:

(parent #) = (child # - 1 ) / 2

Reducing the Workin a Swap

• A swap (using simple assignments) would

normally involve three statements:



94

normally involve three statements:

temp = elements[ i ];

elements[ i ] = elements[ j ];

elements[ j ] = temp;

Reducing the Workin a Swap (cont.)

• In an enqueue or a dequeue for a heap,

we do not really have to use this three-



95

we do not really have to use this three

assignment swap (although it helped to

visualize how the enqueue and dequeue

process worked)

• We can save the value we are swapping

upwards or downwards

• Let’s look at an enqueue…

39root


Enqueue



96

32

16

14 3

28

229

5 15

24

39root


Enqueue (cont.)



97

32

16

14 3

28

229

5 15

24

Create a new node

in the last position

39root


Enqueue (cont.)



98

32

16

14 3

28

229

5 15

24

We just “pretend”

that 37 is here

39root


Enqueue (cont.)



99

32

16

14 3

28

229

5 15

24

37 > 24, so we

pretend to swap

(we just copy 24)

39root

Enqueue (cont.)value to enqueue: 37



100

32

16

14 3

28

229

5 15

We now “pretend” 37

has been placed here24

24

39root




101

32

16

14 3

28

229

5 15 24

24

and we compare 37

to 28 to see if we

should “swap” again

39root




102

32

16

14 3

28

229

5 15 24

24

37 > 28, so we do a

one-assignmentswap again

39root




103

32

16

14 3

229

5 15 24

28

28

We “pretend” 37

is here

39root




104

32

16

14 3

229

5 15 24

28

28

and compare 37 to

39 to see if weshould “swap” again

39root




105

32

16

14 3

229

5 15 24

28

28

This time we

shouldn’t swap…

39root




106

32

16

14 3

229

5 15 24

37

28

but we have one

final assignment

39root




107

32

16

14 3

229

5 15 24

37

28

(we can stop

pretending)

39root

Dequeue



108

32

16

14 3

229

5 15 24

37

28

The same technique

can be used whenswapping downward

Parent-Child Formulas

• Parent-child formulas can also be sped up:



109

(left child #) = (parent #) << 1 + 1

(right child #) = (left child #) + 1

• when finding the greatest child, the left and right

children are always found together

(parent #) = (child # - 1) >> 1

• using the shift operator is the same as integerdivision

Forming an Initial Heap

• A heap of size n can be made by enqueuing nelements one at a time – but it would takeO( n lg n ) time (even though n starts off assmall)



110

small)

• A faster method can be used to make a heap outof an initial array in ( n ) time

• The heap will be shown as a linked tree,because it is easier to visualize how the methodworks – but in actuality, when we use themethod, we use it on the array

5 34 11 7 7 12 38 5 32 1 34 27 166 31 3934

Forming an Initial Heap(cont.)



111

Suppose the initial array looks like this.

It is not initially a heap, but by rearranging the

elements, it will be.

We will look at the tree form of this to see why

the method works.

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

31

root6

50




112

34

7 12

7

39 38

11

5 32 1 34

1 2

3 4 5 6

7 8 9 10 11 12 13 14

27 16

1615

34

Drawing it this way, we can easily seethat the initial array is not a heap

31

root6

50




113

34

7 12

7

39 38

11

5 32 1 34

1 2

3 4 5 6

7 8 9 10 11 12 13 14

27 16

1615

34

We realize that nodes 8 through 16 aresubheaps of only one node.

31

root6

50

1 2




114

34

7 12

7

39 38

11

5 32 1 34

1 2

3 4 5 6

7 8 9 10 11 12 13 14

27 16

1615

34

We want to heapify starting with theparent of the last leaf node.

31

root6

50

1 2




115

34

7 12

7

39 38

11

5 32 1 34

1 2

3 4 5 6

7 8 9 10 11 12 13 14

27 16

1615

34

To find the last leaf node, we useheapsize – 1 (17 – 1 = 16)

31

root6

50

1 2




116

34

7 12

7

39 38

11

5 32 1 34

1 2

3 4 5 6

7 8 9 10 11 12 13 14

27 16

1615

34

Then the parent of the last node is(16 – 1) / 2 = 7

31

root6

50

1 2




117

34

7 12

7

39 38

11

5 32 1 34

1 2

3 4 5 6

7 8 9 10 11 12 13 14

27 16

1615

34

Heapifying works only if the structure is aheap except for possibly the root.

31

root6

50

1 2




118

34

7 12

7

39 38

11

5 32 1 34

1 2

3 4 5 6

7 8 9 10 11 12 13 14

27 16

1615

34

This is true for the structure rooted atindex 7, so we can use heapify on it…

31

root6

50

1 2




119

34

27 12

7

39 38

11

5 32 1 34

1 2

3 4 5 6

7 8 9 10 11 12 13 14

7 16

1615

34

This is true for the structure rooted atindex 7, so we can use heapify on it…

31

root6

50

1 2




120

34

27 12

7

39 38

11

5 32 1 34

1 2

3 4 5 6

7 8 9 10 11 12 13 14

7 16

1615

34

Then we decrement index 7 and useheapify again…

31

root6

50

1 2




121

34

27 12

34

39 38

11

5 32 1 7

1 2

3 4 5 6

7 8 9 10 11 12 13 14

7 16

1615

34

Then we decrement index 7 and useheapify again…

31

root6

50

1 2




122

34

27 12

34

39 38

11

5 32 1 7

1 2

3 4 5 6

7 8 9 10 11 12 13 14

7 16

1615

34

This process continues until we heapify atthe root



31

root6

50

1 2




124

34

27 12

34

34 38

32

5 11 1 7

1 2

3 4 5 6

7 8 9 10 11 12 13 14

7 16

1615

39

31

root6

50

1 2




125

34

27 12

34

34 38

32

5 11 1 7

1 2

3 4 5 6

7 8 9 10 11 12 13 14

7 16

1615

39

already a heap

31

root6

50

1 2


temp



126

34

27 12

34

34 38

32

5 11 1 7

1 2

3 4 5 6

7 8 9 10 11 12 13 14

7 16

1615

39

one-assignment swaps

31

root6

340

1 2




127

34

27 12

7

34 38

32

5 11 1 5

1 2

3 4 5 6

7 8 9 10 11 12 13 14

7 16

1615

39

31

root6

340

1 2




128

34

27 12

7

34 38

32

5 11 1 5

1 2

3 4 5 6

7 8 9 10 11 12 13 14

7 16

1615

39

temp

39

root6

340

1 2




129

34

27 12

7

34 31

32

5 11 1 5

1 2

3 4 5 6

7 8 9 10 11 12 13 14

7 16

1615

38

39

root6

340

1 2




130

34

27 12

7

34 31

32

5 11 1 5

1 2

3 4 5 6

7 8 9 10 11 12 13 14

7 16

1615

38

temp

38

root39

340

1 2




131

34

27 12

7

6 31

32

5 11 1 5

1 2

3 4 5 6

7 8 9 10 11 12 13 14

7 161615

34

The result is a heap

Linked Heap

• If we have a large element size and we

don’t know the max size, we might want toid i d idi



132

consider conserving memory and avoiding

resizing array with a linked heap

• Should have the same time complexitiesas the array-based heap

Reference

• Childs, J. S. (2008). Methods for Making

Data Structures. C++ Classes and Data

Structures Prentice Hall



Structures. Prentice Hall.

133



1

Lecture 04a

Graphs

Graphs

• Graphs are a very general data structure

•

A graph consists of a set of nodes calledvert ices



2

• Any vertex can point to any number of other

vertices

• It is possible that no vertex in the graph points to

any other vertex

• Each vertex may point to every other vertex,

even if there are thousands of vertices

A Directed Graph(Digraph)

A

BC



3

D

E

F

Each pointer is calledan edge

An Undirected GraphA

BC



4

D

E

F

Each edge

points in

bothdirections

– ex: A

points to D

and Dpoints to A

Another DigraphA

B

F



5

D

C

E

Nodes A and F point

to each other – it is

consideredimproper to draw a

single undirected

edge between them

Undirected Weighted GraphGraphville

Node Town

3

8

4



6

Vertex City

Pointerburgh

Builder’s

Paradise BinaryTree Valley

8

410

4

2

Graph Implementation

• A vertex A is said to be adjacent to a

vertex B if there is an edge pointing from Ato B



7

to B

• Graphs can be implemented in a couple of

popular ways: – adjacency matrix

– adjacency list

Adjacency Matrix

• Nodes are given a key from 0 to n – 1

• If weight is not important, the adjacency

t i i 2 di i l f b l



8

matrix is a 2-dimensional array of bool

(True or False) type variables

• If weight is important, the adjacency matrix

is a 2-dimensional array of int type

variables

Undirected Weighted GraphNode 0

Node 2

3

8

4



9

Node 1

Node 4

Node 3 Node 5

8

410

4

2

T F F T F T

0

1

Adjacency Matrix (cont.)0 1 2 3 4 5

F T T F F F

Undirectedweighted graph



10

T F F F T F

F T F F F F

F F T F F T

F T F F T F

2

3

4

5

The row

numbers give

the vertexnumber of the

vertex that an

edge is pointing

from

Adjacency Matrix (cont.)

Example:T F F T F T

0

1

0 1 2 3 4 5

F T T F F F



11

Example:

Node 1 points

to Node 2 (setto T)

Node 2 also

points to Node

1 for undirectedgraph

T F F F T F

F T F F F F

F F T F F T

F T F F T F

2

3

4

5


Weighted graph4 ∞ ∞ 4 ∞ 10

0

1

0 1 2 3 4 5

∞ 4 3 ∞ 8 ∞



12

Weighted graph

∞ - Unconnected3 ∞ ∞ ∞ 4 ∞

∞ 4 ∞ ∞ ∞ ∞

8 ∞ 4 ∞ ∞ 2

∞ 10 ∞ ∞ 2 ∞

2

3

4

5

Directed Weighted GraphNode 0

Node 2

3

8

4



13

Node 1

Node 4

Node 3 Node 5

8

410

4

2


Directed graph

Example:F F F T F T

0

1

0 1 2 3 4 5

F T T F T F



14

p

Node 0 points

to Node 1 (set

to T)

But Node 1

doesn’t point to

Node 0 (set to

F)

F F F F T F

F F F F F F

F F F F F T

F F F F F F

2

3

4

5


Directed∞ ∞ ∞ 4 ∞ 10

0

1

0 1 2 3 4 5

∞ 4 3 ∞ 8 ∞



15

ected

weighted graph

∞ - Unconnected

∞ ∞ ∞ ∞ 4 ∞

∞ ∞ ∞ ∞ ∞ ∞

∞ ∞ ∞ ∞ ∞ 2

∞ ∞ ∞ ∞ ∞ ∞

2

3

4

5


∞ ∞ ∞ 4 ∞ 10

0

1

0 1 2 3 4 5

∞ 4 3 ∞ 8 ∞ Note that we can

construct a graph

from the adjacency

matrix – the



16

∞ ∞ ∞ ∞ 4 ∞

∞ ∞ ∞ ∞ ∞ ∞

∞ ∞ ∞ ∞ ∞ 2

∞ ∞ ∞ ∞ ∞ ∞

2

3

4

5

vertices may not

be drawn in the

same locations asthe original graph,

but the

connections will

be the same

Adjacency List

• An adjacency list is an array of linked lists

• The vertices are numbered 0 through

n 1



17

n – 1

• Each index in the array corresponds to the

number of a vertex

• The vertex (with an index number) is

adjacent to every node in the linked list at

that index

Adjacency List (cont.)1

3 5

4

0

1

2 4

Directed weighted



18

4

5

2

3

4

5

graph


3 5

4

0

1

2 4

Vertex 1 has a link



19

4

5

2

3

4

5

to vertex 3, and

vertex 1 also has a

link to vertex 5


3 5

4

0

1

2 4

Vertex 3 has an

t li k d li t it



20

4

5

2

3

4

5

empty linked list – it

has no link to any

other vertices

Adjacency List for

Directed Weighted Graph1 40

1

2

2 3 4 8

3 4 5 10



21

2

3

4

5

4 4

5 2 Red font is for vertex

numbers, black font

is for weights

Adjacency Matrix vs.

Adjacency List

• The speed of the adjacency matrix or the

adjacency list depends on the algorithm– some algorithms need to know if there is a



22

some algorithms need to know if there is a

direct connection between two specific

vertices –

the adjacency matrix would befaster

– some algorithms are written to process the

linked list of an adjacency list, node by node –

the adjacency list would be faster

Adjacency Matrix vs. Adjacency List (cont.)

• When both seem equally fast for a certain

algorithm, then we consider memoryusage



23

• We will consider the space complexi ty of

each implementation• Space complexities are like time

complexities, but they tell us the effects onmemory space usage as the problem size

is varied

•

An array of vertex info is used for eachimplementation, which is ( n )




24

• In the adjacency matrix, each dimension is

n, so the spatial complexity is

( n2

)

•In the adjacency list, there are n elements in thearray, giving a spatial complexity of ( n ) for the




25

array

• Note that each edge corresponds to a linked list

node in the adjacency list

• There can be no more than n( n – 1 ) edges in a

graph, so the spatial complexity for the linked list

nodes is O( n2

)• the total spatial complexity for the adjacency list

is O( n2 )

•

Both of the spatial complexities for theadjacency matrix and the adjacency list

b b h ( ) i l l i d




26

absorb the ( n ) spatial complexity used

in the vertex info array

• The spatial complexity of the adjacency list

really depends on whether the graph is

sparse or dense

• A sparse graph does not have that manyedges relative to the number of vertices –

if the number of edges is less than or




27

gequal to n, then the spatial complexity of

the adjacency list is

( n )• In a dense graph, there may be close to n2

edges, and then the spatial complexity ofthe adjacency list is ( n2 ), the same as

that for the adjacency matrix

Lecture 04b Graph Algorithms



Graphs 1

ORD

DFW

SFO

LAX

• Depth-first search

• Breadth-First Search

• Topological Sorting

Graph A graph is a pair (V, E ), where

V is a set of nodes, called vertices

E is a collection of pairs of vertices, called edges

Vertices and edges are positions and store elements

Example:



Graphs 2

Example: A vertex represents an airport and stores the three-letter airport code

An edge represents a flight route between two airports and stores the

mileage of the route

ORDPVD

MIA DFW

SFO

LAX

LGA

HNL

Edge TypesDirected edge ordered pair of vertices (u ,v )

first vertex u is the origin second vertex v is the destination

e.g., a flight

ORD PVDflight AA 1206



Graphs 3

e.g., a flight

Undirected edge unordered pair of vertices (u ,v )

e.g., a flight routeDirected graph all the edges are directed

e.g., route network

Undirected graph

all the edges are undirected e.g., flight network

ORD PVD

849

miles

cs.brown.edu

math.brown.edu

cslab1bcslab1a

ApplicationsElectronic circuits

Printed circuit board Integrated circuit

Transportation networks



Graphs 4

John

DavidPaul

brown.edu

cox.net

att.netqwest.net

Transportation networks

Highway network

Flight network

Computer networks

Local area network

Internet

Web

Databases

Entity-relationship diagram

TerminologyEnd vertices (or endpoints) ofan edge U and V are the endpoints of a

Edges incident on a vertex a d and b are incident on V

Va b

h j



Graphs 5

a, d, and b are incident on V

Adjacent vertices U and V are adjacent

Degree of a vertex X has degree 5

Parallel edges h and i are parallel edges

Self-loop j is a self-loop

XU

W

Z

Y

c e

d

f

gi

P

Terminology (cont.)

Path

sequence of alternatingvertices and edges

begins with a vertex

ends with a vertex

Va b



Graphs 6

P1 ends with a vertex

each edge is preceded andfollowed by its endpoints

Simple path path such that all its vertices

and edges are distinct

Examples P1=(V,b,X,h,Z) is a simple path

P2=(U,c,W,e,X,g,Y,f,W,d,V) is apath that is not simple

XU

W

Z

Y

c e

d

f

g

hP2

Terminology (cont.)Cycle

circular sequence of alternatingvertices and edges

each edge is preceded andfollowed by its endpoints

Va b



Graphs 7

followed by its endpoints

Simple cycle

cycle such that all its vertices

and edges are distinct

Examples

C1=(V,b,X,g,Y,f,W,c,U,a,) is asimple cycle

C2=(U,c,W,e,X,g,Y,f,W,d,V,a,)

is a cycle that is not simple

C1

XU

W

Z

Y

c e

d

f

g

hC2

PropertiesNotation

n number of verticesm number of edges

deg(v ) degree of vertex v

Property 1

S

v deg(v ) = 2m Proof: each edge is

counted twice



Graphs 8

Property 2In an undirected graph

with no self-loops andno multiple edges

m n (n 1)/2

Proof: each vertex hasdegree at most (n 1)

What is the bound for adirected graph?

Example

n=

4

m =

6

deg(v ) = 3

Asymptotic Performancen vertices, m edges

no parallel edges

no self-loops

Bounds are “big-Oh”

AdjacencyList

AdjacencyMatrix

S 2



Graphs 9

Space n + m n 2

incidentEdges(v ) deg(v ) n

areAdjacent (v, w ) min(deg(v ), deg(w )) 1

insertVertex(o ) 1 n 2

insertEdge(v, w, o ) 1 1

removeVertex(v ) deg(v ) n 2

removeEdge(e ) 1 1

Depth-First Search:Outline and Reading

Depth-first search (6.3.1)

Algorithm Example

Properties



Graphs 10

Properties

Analysis

DB

A

C

E

Subgraphs

A subgraph S of a graph

G is a graph such that The vertices of S are a

subset of the vertices of G



Graphs 11

The edges of S are asubset of the edges of G

A spanning subgraph of Gis a subgraph thatcontains all the verticesof G

Subgraph

Spanning subgraph

Connectivity

A graph isconnected if there isa path between

f



Graphs 12

every pair ofvertices

A connectedcomponent of agraph G is amaximal connectedsubgraph of G

Connected graph

Non connected graph with twoconnected components

Trees and Forests

A (free) tree is an

undirected graph T suchthat T is connected

T h l



Graphs 13

T has no cycles

This definition of tree is

different from the one ofa rooted tree

A forest is an undirectedgraph without cycles

The connected

components of a forestare trees

Tree

Forest

Spanning Trees and Forests

A spanning tree of a

connected graph is aspanning subgraph that isa tree

A spanning tree is not



Graphs 14

A spanning tree is notunique unless the graph isa tree

Spanning trees haveapplications to the designof communicationnetworks

A spanning forest of a

graph is a spanningsubgraph that is a forest

Graph

Spanning tree

Depth-First Search

Depth-first search (DFS)

is a general techniquefor traversing a graph

A DFS traversal of ah G

DFS on a graph with n

vertices and m edgestakes O (n + m ) time

DFS can be furthert d d t l th



Graphs 15

graph G Visits all the vertices and

edges of G Determines whether G is

connected

Computes the connectedcomponents of G

Computes a spanningforest of G

extended to solve othergraph problems Find and report a path

between two givenvertices

Find a cycle in the graph

Depth-first search is to

graphs what Euler touris to binary trees

DFS AlgorithmThe algorithm uses a mechanismfor setting and getting “labels” of

vertices and edges

Algorithm DFS (G, v )

Input graph G and a start vertex v of G

Output labeling of the edges of G in the connected component of v

di d d b k d

Algorithm DFS (G )

Input graph G



Graphs 16

as discovery edges and back edges

setL abel (v, VISI TED )

for all e

G.incidentEdges (v )if getL abel (e ) = UNEXPLORED

w opposite (v,e )

if getL abel (w ) = UNEXPLORED

setLabel (e, DISCOVERY )

DFS (G, w )

else

setLabel (e, BACK )

g

Output labeling of the edges of G as discovery edges and

back edgesfor all u G.vertices ()

setL abel (u, UNEXPLORED )

for all e G.edges ()

setL abel (e, UNEXPLORED )

for all v G.vertices ()

if getLabel (v ) = UNEXPLORED

DFS (G, v )

Example

DB

A

C

E

discovery edge

A visited vertex

A unexplored vertex

unexplored edge



Graphs 17

DB

A

C

E

C

DB

A

C

E

discovery edge

back edge

Example (cont.)

DB

A

EDB

A

E



Graphs 18

DB

A

C

E

C

DB

A

C

E

C

DFS and Maze Traversal

The DFS algorithm is

similar to a classicstrategy for exploringa maze

We mark each



Graphs 19

We mark eachintersection, cornerand dead end (vertex)visited

We mark each corridor(edge ) traversed

We keep track of thepath back to the

entrance (start vertex)by means of a rope(recursion stack)

Properties of DFS

Property 1

DFS (G, v ) visits all thevertices and edges inthe connected A



Graphs 20

component of v

Property 2The discovery edgeslabeled by DFS (G, v )form a spanning tree ofthe connected

component of v

DB

C

E

Analysis of DFS

Setting/getting a vertex/edge label takes O (1) time

Each vertex is labeled twice once as UNEXPLORED

once as VISITED



Graphs 21

Each edge is labeled twice once as UNEXPLORED

once as DISCOVERY or BACK

Method incidentEdges is called once for each vertex

DFS runs in O (n + m ) time provided the graph isrepresented by the adjacency list structure

Recall thatS

v deg(v ) = 2m

Breadth-First Search



Graphs 22

CB

A

E

D

L 0

L 1

FL 2

Breadth-First Search

Breadth-first search

(BFS) is a generaltechnique for traversinga graph

A BFS traversal of a

BFS on a graph with n

vertices and m edgestakes O (n + m ) time

BFS can be further



Graphs 23

A BFS traversal of agraph G Visits all the vertices and

edges of G

Determines whether G isconnected

Computes the connected

components of G Computes a spanning

forest of G

extended to solve other

graph problems Find and report a path

with the minimumnumber of edgesbetween two given

vertices Find a simple cycle, if

there is one

BFS AlgorithmThe algorithm uses amechanism for setting and

getting “labels” of verticesand edges

Algorithm BFS (G, s )L 0 new empty sequenceL

0.insertLast (s )

setL abel (s, VISITED )i 0

while L i .isEmpty ()L new empty sequence

Algorithm BFS (G )

Input graph G



Graphs 24

L i +1 new empty sequence

for all v L i .elements ()for all e G.incidentEdges (v )

if getLabel (e ) = UNEXPLORED

w opposite (v,e )if getL abel (w ) = UNEXPLORED

setL abel (e, DISCOVERY )setL abel (w, VISITED )L i +1.insertLast (w )

elsesetL abel (e, CROSS )

i i +1

Input graph G

Output labeling of the edgesand partition of the

vertices of Gfor all u G.vertices ()

setL abel (u, UNEXPLORED )

for all e G.edges ()


for all v G.vertices ()if getL abel (v ) = UNEXPLORED

BFS (G, v )

Example

discovery edge

A visited vertex

A unexplored vertex

unexplored edgeCB

A

E

D

L 0

L1

F



Graphs 25

CB

A

E

D

discovery edge

cross edge

L 0

L 1

F

E F

CB

A

E

D

L 0

L 1

F

Example (cont.)

CB

A

D

L 0

L 1CB

A

D

L 0

L 1

L 2



Graphs 26

E F

CB

A

E

D

L 0

L 1

FL 2

E F2

CB

A

E

D

L 0

L 1

FL 2

Example (cont.)

CB

A

E

D

L 0

L 1

FL 2

CB

A

E

D

L 0

L 1

FL 2



Graphs 27

E F

CB

A

E

D

L 0

L 1

FL 2

E F

PropertiesNotation

G s : connected component of s

Property 1BFS (G, s ) visits all the vertices andedges of G s

CB

A

E

D

F



Graphs 28

Property 2The discovery edges labeled byBFS (G, s ) form a spanning tree T s of G s

Property 3For each vertex v in L i

The path of T s from s to v has i

edges Every path from s to v in G s has at

least i edges

CB

A

E

D

L 0

L 1

FL 2

E F

Analysis

Setting/getting a vertex/edge label takes O (1) time

Each vertex is labeled twice once as UNEXPLORED

once as VISITED

E h d i l b l d t i



Graphs 29

Each edge is labeled twice once as UNEXPLORED

once as DISCOVERY or CROSS

Each vertex is inserted once into a sequence L i

Method incidentEdges is called once for each vertex

BFS runs in O (n + m ) time provided the graph is

represented by the adjacency list structure Recall that Sv deg(v ) = 2m

Applications

Using the template method pattern, we can

specialize the BFS traversal of a graph G tosolve the following problems in O (n + m ) time

Compute the connected components of G



Graphs 30

Compute the connected components of G

Compute a spanning forest of G

Find a simple cycle in G , or report that G is a

forest

Given two vertices of G , find a path in G between

them with the minimum number of edges, orreport that no such path exists

DFS vs. BFS

Applications DFS BFS

Spanning forest, connectedcomponents, paths, cycles

Shortest paths



Graphs 31

CB

A

E

D

L 0

L 1

FL 2

CB

A

E

D

F

DFS BFS

Biconnected components

DFS vs. BFS (cont.)

Back edge (v,w )

w is an ancestor of v inthe tree of discoveryedges

Cross edge (v,w )

w is in the same level asv or in the next level in

the tree of discoveryedges



Graphs 32

edges

CB

A

E

D

L 0

L 1

F

L 2

CB

A

E

D

F

DFS BFS

DAGs and Topological Sorting

A directed acyclic graph (DAG) is adigraph that has no directed cycles

A topological sorting/ordering of adigraph is a numbering

v 1 , …, vn

B

D

C

E

DAG



33

of the vertices such that for everyedge (v i , v j ), we have i < j

Example: in a task schedulingdigraph, a topological sorting atask sequence that satisfies theprecedence constraints

Theorem

A digraph admits a topologicalsorting if and only if it is a DAG

A DAG G

B

A

D

C

E

Topologicalordering of G

v 1

v 2

v 3

v 4 v 5

Topological Sorting

Number vertices, so that (u,v) in E implies u < v

wake up

eatstudy computer sci.

A typical student day 1

2 3

5



34

write c.s. program

play

nap more c.s.

work out

sleep

dream about graphs

4 5

6

7

8

9

1011

make cookies

for professors

Note: This algorithm is different than the

one in Goodrich-Tamassia (next slide)

Algorithm for Topological Sorting

Method TopologicalSort(G )

H G // T f G



35

Running time: O(n + m).

H

G // Temporary copy of G

n G.numVertices ()

while H is not empty doLet v be a vertex with no outgoing edges

Label v n

n

n - 1

Remove v from H

Topological Sorting Algorithm using DFS

Simulate the algorithm by usingdepth-first search

Algorithm topologicalDFS (G, v )

Input graph G and a start vertex v of G

Output labeling of the vertices of G in the connected component of v

setL abel (v, VISI TED )

for all e G.incidentEdges (v )

Algorithm topologicalDFS (G )

Input dag G

Output topological ordering of G n G numVertices()



36

O(n+m) time.

if getL abel (e ) = UNEXPLORED

w opposite (v,e )

if getL abel (w ) = UNEXPLORED

setLabel (e, DISCOVERY )

topologicalDFS (G, w )

else

{e is a forward or cross edge}

Label v with topological number n

n n - 1

n G.numVertices ()

for all u G.vertices ()

setL abel (u, UNEXPLORED )for all e G.edges ()


for all v G.vertices ()

if getL abel (v ) = UNEXPLORED

topologicalDFS (G, v )

Topological Sorting Example



37




38

9




39

8

9




40

7

8

9




41

7

8

6

9




42

7

8

56

9


4



43

7

4

8

56

9


4

3



44

7

4

8

56

9


2

4

3



45

7

4

8

56

9


2

4

1

3



46

7

4

8

56

9

Lecture 05a



1

Selectionsort, Heapsort &Quicksort

Sorting

• Sort ing is the process of placing elements inorder

• If elements are objects, they are ordered by aparticular data member

Th l ith f ti h



2

• There are many algorithms for sorting, each

having its own advantages• No sorting algorithm is better than every other

sorting algorithm all the time

• Choosing the right sorting algorithm for a

problem requires careful consideration

Selectionsort

• Idea: Find the largest number from the unsorted

numbers, place it at the correct location.• Array: 3 6 5 2 4

• n = 5



3

•

i = 4: 3 4 5 2 6• i = 3: 3 4 2 5 6

• i = 2: 3 2 4 5 6

• i = 1: 2 3 4 5 6 (sorted)

Selectionsort (cont.)void selectionSort (int a[], int n) {

for (int i = n-1; i > 0; i--) {

// find the max element in the unsorted a[i .. n-1]

int maxIndex = i; // assume the max is the last element// test against elements before i to find the largest

for (int j = 0; j < i; j++) {

// if this element is larger, then it is the new max

if (a[j] > a[maxIndex])



4

if (a[j] > a[maxIndex])

// found new max; remember its index

maxIndex = j;}

// maxIndex is the index of the max element,

// swap it with the current position

if (maxIndex != i)

swap (a[i], a[maxIndex]);

}}

Selectionsort TimeComplexity

• By the same analysis as the

prefixAverages1 algorithm in Lecture 1 (the "for" loops of the 2 algorithms are

similar), Selectionsort runs in ( n2 ) time



5

similar), Selectionsort runs in ( n ) time

Heapsort

Algorithm Heapsort (S, n)

Input sequence S of n elementsOutput sorted sequence S

pq priority queue (S)

f 1 0



6

for i n-1 to 0

pq.dequeue (S[i ])

Create a heap-based priority

queue from S

Heapsort (cont.)




f i 1 t 0



7

for i n-1 to 0

pq.dequeue (S[i ])

In the loop, we start with the last sequence

index

Heapsort (cont.)




f i 1 t 0



8

for i n-1 to 0

pq.dequeue (S[i ])

On the first dequeue, the largest element from

the heap will be placed into the last positionof the sequence

Heapsort (cont.)




f i 1 t 0



9

for i n-1 to 0

pq.dequeue (S[i ])

On the next dequeue (i is decremented), the

remaining highest value from the heap will beplaced in the next left sequence position

Heapsort (cont.)




f i 1 t 0



10

for i n-1 to 0

pq.dequeue (S[i ])

The loop stops when the index becomes less

than 0

Heapsort (cont.)

• S = 3 6 5 2 4: pq = 6 4 5 2 3

• i = 4: pq = 5 4 3 2: S = 3 6 5 2 6

• i = 3: pq = 4 2 3: S = 3 6 5 5 6

i 2 3 2 S 3 6 4 5 6



11

• i = 2: pq = 3 2: S = 3 6 4 5 6

• i = 1: pq = 2: S = 3 3 4 5 6

• i = 0: pq = : S = 2 3 4 5 6 (sorted)

Heapsort (cont.)

• The previous Heapsort algorithm requires

2 arrays to work. The second array is usedto create a heap (priority queue)

• Instead of create a heap separately we



12

Instead of create a heap separately, we

can actually create a heap on the originalarray

– Lec03b, section Forming an Initial Heap

Heapsort (cont.)

• S = 3 6 5 2 4

• Forming an Initial Heap

• S/pq = 6 4 5 2 3 (S and pq are the array)

i 4 S 5 4 3 2 6 ( 5 4 3 2)



13

• i = 4: S = 5 4 3 2 6 ( pq = 5 4 3 2)

• i = 3: S = 4 2 3 5 6 ( pq = 4 3 2)

• i = 2: S = 3 6 4 5 6 ( pq = 3 2)

• i = 1: S = 3 3 4 5 6 ( pq = 2)

• i = 0: S = 2 3 4 5 6 (sorted)

Heapsort TimeComplexity

• Heapsort runs in ( n lg n ) time on

average• It has a best-case time of ( n ) if all

elements have the same value



14

elements have the same value

– This is an unusual case, but we may want toconsider it if many of the elements have the

same value

Quicksort• Time complexities of Quicksort

– best: ( n lg n )

– average: ( n lg n )

– worst: ( n2 )

• The average case usually dominates over the



15

• The average case usually dominates over the

worst case in sorting algorithms, and inQuicksort, the coefficient of the n lg n term is

small

– makes Quicksort one of the most popular general-

purpose sorting algorithms

Functions of Quicksort

• A recursive function, called quicksor t

A i f ti ll ll d



16

• A nonrecursive function, usually called

part i t ion

• quicksort calls partition

Partition

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 11474635 6 4 28

I th titi f ti th l t l t i

pivot



17

In the partition function, the last element is

chosen as a pivot – a special element usedfor comparison purposes

Each element is compared to the pivot.

When partition is done, it produces theresult shown next…

Partition (cont.)

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 11474635 6 4 28

pivot

0 1 2 3 4 5 6 7 8 9 10 11



18

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 28 38 471136 4 35 46

All elements less than or equal to the pivot are on its left

Partition (cont.)

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 11474635 6 4 28

pivot

0 1 2 3 4 5 6 7 8 9 10 11



19

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 28 38 471136 4 35 46

All elements greater than the pivot are on its right side

Partition (cont.)

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 11474635 6 4 28

pivot

0 1 2 3 4 5 6 7 8 9 10 11



20

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 28 38 471136 4 35 46

The pivot is where it will be in the final sorted array

Partition (cont.)

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 11474635 6 4 28

pivot

0 1 2 3 4 5 6 7 8 9 10 11



21

0 3 5 6 8 9 0

14 1028 28 38 471136 4 35 46

Partition is called from quicksort more than once, but

when called again, it will work with a smaller section of

the array.

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 28 38 471136 4 35 46

Partition (cont.)



22

The next time partition is called, for

example, it will only work with this

section to the left of the previous pivot.

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 28 38 471136 4 35 46

Partition (cont.)

i t



23

pivot

0 1 2 3 4 5 6 7 8 9 10 11

3104

28 38 4711146

28 35 46

Partition (cont.)

i t



24

pivot

0 1 2 3 4 5 6 7 8 9 10 11

3104

28 38 4711146

28 35 46

Partition (cont.)

Each section of the array separated by a



25

previous pivot will eventually have partition

called for it

0 1 2 3 4 5 6 7 8 9 10 11

3104

28 38 4711146

28 35 46

Partition (cont.)



26

Except that partition is not called for a section of

just one element – it is already a pivot and it is

where it belongs

0 1 2 3 4 5 6 7 8 9 10 11

3104

28 38 4711146

28 35 46

Partition (cont.)



27

Partition is called

for this section

0 1 2 3 4 5 6 7 8 9 10 11

3104

28 38 4711146

28 35 46

Partition (cont.)

i t



28

pivot

0 1 2 3 4 5 6 7 8 9 10 11

3104

28 38 4711146

28 35 46

Partition (cont.)

pivotAll elements are smaller



29

pivotAll elements are smaller

and stay to the left of thepivot

0 1 2 3 4 5 6 7 8 9 10 11

3104

28 38 4711146

28 35 46

Partition (cont.)

Partition is called for



30

Partition is called for

this section

0 1 2 3 4 5 6 7 8 9 10 11

3104

28 38 4711146

28 35 46

Partition (cont.)

pivot



31

pivot

0 1 2 3 4 5 6 7 8 9 10 11

3104

28 38 4714116

28 35 46

Partition (cont.)

pivot



32

pivot

0 1 2 3 4 5 6 7 8 9 10 11

3 104 28 38 4714116 28 35 46

Partition (cont.)



33

partition

0 1 2 3 4 5 6 7 8 9 10 11

3 104 28 38 4714116 28 35 46

Partition (cont.)

pivot



34

pivot

0 1 2 3 4 5 6 7 8 9 10 11

3 64 28 38 47141110 28 35 46

Partition (cont.)

pivot



35

pivot

0 1 2 3 4 5 6 7 8 9 10 11

3 64 28 38 47141110 28 35 46

Partition (cont.)



36

partition notcalled for this

0 1 2 3 4 5 6 7 8 9 10 11

3 64 28 38 47141110 28 35 46

Partition (cont.)



37

partition notcalled for this

0 1 2 3 4 5 6 7 8 9 10 11

3 64 28 38 47141110 28 35 46

Partition (cont.)



38

partition calledfor this



0 1 2 3 4 5 6 7 8 9 10 11

3 64 28 38 35141110 28 46 47

Partition (cont.)

pivot



40

p

0 1 2 3 4 5 6 7 8 9 10 11

3 64 28 38 35141110 28 46 47

Partition (cont.)



41

partition calledfor this

0 1 2 3 4 5 6 7 8 9 10 11

3 64 28 38 35141110 28 46 47

Partition (cont.)

pivot



42

p

0 1 2 3 4 5 6 7 8 9 10 11

3 64 28 35 38141110 28 46 47

Partition (cont.)

pivot



43

p

0 1 2 3 4 5 6 7 8 9 10 11

3 64 28 35 38141110 28 46 47

Partition (cont.)

Partition not



44

called for this

0 1 2 3 4 5 6 7 8 9 10 11

3 64 28 35 38141110 28 46 47

Partition (cont.)

Partition not



45

called for this

0 1 2 3 4 5 6 7 8 9 10 11

3 64 28 35 38141110 28 46 47

Partition (cont.)

At this point, the



46

array is sorted

0 1 2 3 4 5 6 7 8 9 10 11

3 64 28 35 38141110 28 46 47

Partition (cont.)

But to achieve this, what steps does the

algorithm for partition go through?



47

algorithm for partition go through?

Partition (cont.)

0 1 2 3 4 5 6 7 8 9 10 11

14 102 3 38 11474635 6 4 28

pivot



48

Partition has a loop which iterates through eachelement, comparing it to the pivot

Partition (cont.)

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 11474635 6 4 28

pivot



49

When partition is in progress, there is a partitioned

section on the left, and an unpartitioned section on

the right

partitioned section unpartitioned section

Partition (cont.)

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 11474635 6 4 28

pivot



50

The dividing line in the partitioned section means that

all elements to the left are less than or equal to the

pivot; all elements to the right are greater than the

pivot


Partition (cont.)

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 11474635 6 4 28

pivot



51

On each iteration, the partitioned section grows by

one and the unpartitioned section shrinks by one,

until the array is fully partitioned


Partition (cont.)

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 11474635 6 4 28

pivot



52

i is an index used to mark the last value in the small

side of the partition, while j is used to mark the first

value in the unpartitioned section.


Partition (cont.)

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 11474635 6 4 28

pivoti j



53

i is an index used to mark the last value in the small

side of the partition, while j is used to mark the first

value in the unpartitioned section.


Partition (cont.)

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 11474635 6 4 28

pivoti j



54

On each iteration of partition, the value at j is

compared to the pivot.


Partition (cont.)

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 11474635 6 4 28

pivoti j



55

On each iteration of partition, the value at j is

compared to the pivot.


Partition (cont.)

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 11474635 6 4 28

pivoti j



56

If the value at j is greater, j is just incremented (the

partitioned section grows by one)


Partition (cont.)

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 11474635 6 4 28

pivoti j



57

If the value at j is greater, j is just incremented (the

partitioned section grows by one)


Partition (cont.)

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 11474635 6 4 28

pivoti j



58

If, on an iteration, the value at j is less than the

pivot…


Partition (cont.)

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 11474635 6 4 28

pivoti j



59

then i is incremented…


Partition (cont.)

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 11474635 6 4 28

pivoti j



60

then i is incremented…


Partition (cont.)

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 11474635 6 4 28

pivoti j



61

and the value at i is swapped with the value at j…


Partition (cont.)

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 114746 356 4 28

pivoti j



62

and the value at i is swapped with the value at j…


Partition (cont.)

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 114746 356 4 28

pivoti j



63

then j is incremented


Partition (cont.)

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 114746 356 4 28

pivoti j



64

then j is incremented


Partition (cont.)1 partition( arr , p, r )

2 i = p – 1

3 for all j from p to r –

14 if arr[ j ] <= arr[ r ]

5 i++

6 swap( arr[ i ], arr[ j ] )

7 swap ( arr[ i + 1 ], arr[ r ] )

Partition starts off by

passing in the array

arr …



65

p ( [ ], [ ] )

8 return i + 1

Partition (cont.)1 partition( arr, p, r )

2 i = p – 1


14 if arr[ j ] <= arr[ r ]

5 i++


7 swap ( arr[ i + 1 ], arr[ r ] )

the beginning index of

the array section it is

working with…



66

p ( [ ], [ ] )

8 return i + 1


2 i = p – 1


14 if arr[ j ] <= arr[ r ]

5 i++


7 swap ( arr[ i + 1 ], arr[ r ] )

and the ending index

of the array section it

is working with



67

p ( [ ], [ ] )

8 return i + 1


2 i = p – 1


14 if arr[ j ] <= arr[ r ]

5 i++


7 swap ( arr[ i + 1 ], arr[ r ] )

i is used to mark the

end of the small-side

part of the partition



68

p ( [ ] [ ] )

8 return i + 1


2 i = p – 1


14 if arr[ j ] <= arr[ r ]

5 i++


7 swap ( arr[ i + 1 ], arr[ r ] )

initially, there isn’t

one, so i is set to p – 1

(-1 if p is 0)



69

p ( [ ] [ ] )

8 return i + 1


2 i = p – 1


14 if arr[ j ] <= arr[ r ]

5 i++


7 swap ( arr[ i + 1 ], arr[ r ] )

this loop iterates

through theelements…



70

( )

8 return i + 1


2 i = p – 1


14 if arr[ j ] <= arr[ r ]

5 i++


7 swap ( arr[ i + 1 ], arr[ r ] )

comparing them to thepivot arr

[ r ]



71

8 return i + 1

Partition (cont.)

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 114746 356 4 28

i j

1 partition( arr, p, r )

2 i = p – 1 j is currently 7,

p r



72

3 for all j from p to r – 14 if arr[ j ] <= arr[ r ]

5 i++


7 swap ( arr[ i + 1 ], arr[ r ] )

8 return i + 1

with partitionhaving already

iterated through

elements 0

through 6

Partition (cont.)

0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 114746 356 4 28

i j


2 i = p – 1

p r



73

3 for all j from p to r – 14 if arr[ j ] <= arr[ r ]5 i++


7 swap ( arr[ i + 1 ], arr[ r ] )

8 return i + 1

Partition (cont.)0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 114746 356 4 28

i j


2 i = p – 1

p r



74


5 i++6 swap( arr[ i ], arr[ j ] )

7 swap ( arr[ i + 1 ], arr[ r ] )

8 return i + 1

Partition (cont.)0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 1147 46356 4 28

i j


2 i = p – 1

p r



75


5 i++

6 swap( arr[ i ], arr[ j ] )7 swap ( arr[ i + 1 ], arr[ r ] )

8 return i + 1

Partition (cont.)0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 1147 46356 4 28

i j


2 i = p – 1

p r



76


5 i++


7 swap ( arr[ i + 1 ], arr[ r ] )

8 return i + 1

Partition (cont.)0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 1147 46356 4 28

i j


2 i = p – 1

p r



77



7 swap ( arr[ i + 1 ], arr[ r ] )

8 return i + 1

Partition (cont.)0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 1147 46356 4 28

i j


2 i = p – 1

p r



78


5 i++


7 swap ( arr[ i + 1 ], arr[ r ] )

8 return i + 1

Partition (cont.)0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 1147 46356 4 28

i j


2 i = p – 1

p r



79



7 swap ( arr[ i + 1 ], arr[ r ] )

8 return i + 1

Partition (cont.)0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 38 1147 46356 4 28

i j


2 i = p – 1

p r



80


5 i++6 swap( arr[ i ], arr[ j ] )

7 swap ( arr[ i + 1 ], arr[ r ] )

8 return i + 1

Partition (cont.)0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 3811 4746356 4 28

i j


2 i = p – 1

p r



81


5 i++


8 return i + 1

Partition (cont.)0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 3811 4746356 4 28

i j


2 i = p – 1

p r



82


5 i++


7 swap ( arr[ i + 1 ], arr[ r ] )

8 return i + 1

Partition (cont.)0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 3811 4746356 4 28

i j


2 i = p – 1

p r



83



7 swap ( arr[ i + 1 ], arr[ r ] )

8 return i + 1

Partition (cont.)0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 3811 4746356 4 28

i j


2 i = p – 1

p r



84


5 i++6 swap( arr[ i ], arr[ j ] )

7 swap ( arr[ i + 1 ], arr[ r ] )

8 return i + 1

Partition (cont.)0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 3811 47466 4 28

i j


2 i = p – 1

p r

35



85


5 i++


8 return i + 1

Partition (cont.)0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 3811 47466 4 28

i j


2 i = p – 1

p r

35

j isn’t incremented



86


5 i++


7 swap ( arr[ i + 1 ], arr[ r ] )

8 return i + 1

past r - 1

Partition (cont.)0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 3811 476 4 28

i j


2 i = p – 1

3 f ll j f 1

p r

35 46



87


5 i++


7 swap ( arr[ i + 1 ], arr[ r ] )8 return i + 1

Partition (cont.)0 1 2 3 4 5 6 7 8 9 10 11

14 1028 3 3811 476 4 28

i j


2 i = p – 1

3 f ll j f t 1

p r

35 46

Th fi l t f



88


5 i++


7 swap ( arr[ i + 1 ], arr[ r ] )

8 return i + 1

The final step ofpartition returns the

INDEX of the pivot

back to the quicksort

function that called it

The Quicksort

Function

1 quicksort( arr, p, r )

2 if p < r 3 pi = partition( arr, p, r )

4 quicksort( arr, p, pi – 1 )

5 quicksort( arr, pi + 1, r )

Since quicksort is

called recursively, it

also passes in thearray arr, the

beginning index

p of

the section it is

ki ith d th



89

working with, and theending index r of the

section it is working

with

The Quicksort

Function (cont.)If p < r, those same

indexes are used in

the call to partition (insuch a case, a call to

quicksort always

produces a matching

ll t titi )







90

call to partition)

The Quicksort

Function (cont.)

The pivot index

pi is

returned from partition







91

p r

p i

The Quicksort

Function (cont.)

i k t i ll d







92

quicksort is calledrecursively here,

working with the

section on the left of

the pivotp r

p i - 1

p i

The Quicksort

Function (cont.)

d i k t i ll d







93

and quicksort is calledrecursively here,

working with the

section on the right of

the pivotp r

p i +

1

p i - 1

p i

The Quicksort

Function (cont.)

if the array sections

are at least 2 elementsin size, these quicksort

functions will call

partition for the same

array section it is

ki ith







94

working with

The Quicksort

Function (cont.)

and partition will break

th ti i t







95

the sections into evensmaller sections

The Quicksort

Function (cont.)

The recursive case

occurs if p < r

(this means the array

section has at least

two elements, the one

at p and the one at r)







96

at p and the one at r)

The Quicksort

Function (cont.)

If the recursive case

occurs, a call toquicksort will match a

call to partition







97

The Quicksort

Function (cont.)

If p == r, the base case

occurs – there is onlyone element in the

array section (recall

that

partition is not

called for a section







98

called for a sectionthat just has one

element)

Lecture 05b Sorting and

Selection Algorithms7 2 9 4 2 4 7 9

7 2 2 7 9 4 4 9



1

7

2 2 7 9

4 4 9

7 7 2 2 9 9 4 4

Learning Outcome

Merge-sort (4.1.1)

Summary of Sorting AlgorithmsRadix-Sort (4.5.2)

Quick-Select (§ 4.7)



2

Merge-SortMerge-sort is a sorting algorithm based on thedivide-and-conquer paradigm

Like heap-sort It has O (n log n ) running time

Unlike heap-sort It does not use a priority queue

It accesses data in a sequential manner (suitable to sortd t di k)



3

It accesses data in a sequential manner (suitable to sortdata on a disk)

Merge-Sort (cont.)

Merge-sort on an input sequence S with n elementsconsists of three steps: Divide: partition S into two sequences S 1 and S 2 of about n /2

elements each

Recur: recursively sort S 1 and S 2

Conquer: merge S 1 and S 2 into a unique sorted sequence



4

Algorithm Merge-Sort

Input Parameters: array a , start index p , end index r.

Output Parameter: array a.

Mergesort (a , p , r ) {

// if only one element, just return.

if (p == r )

return

// Divide: divide a into two nearly equal parts.

( ) / 2



m = (p + r ) / 2

// Recur: sort each half.

Mergesort (a , p , m )

Mergesort (a , m + 1, r )

// Conquer: merge the two sorted halves.Merge (a , p , m , r )

}5

Merging Two Sorted SequencesThe conquer step ofmerge-sort consistsof merging two

sorted sequences Aand B into a sortedsequence S

containing the unionof the elements of Aand B

Merging two sorted

Algorithm Merge (a, p, m, r )

Input sequences A = a[ p]…a[m] and

B = a[m+1]…a[r ] with

Output sorted sequence a[ p]…a[r ]

S empty sequence

while A.isEmpty () B.isEmpty ()

if A.first ().element () < B.first ().element ()

S.insertLast (A.remove (A.first ()))

else



6

Merging two sortedsequences, eachwith n /2 elementsand implemented bymeans of a doublylinked list, takesO (n ) time

else

S.insertLast (B.remove (B.first ()))

while A.isEmpty ()S.insertLast (A.remove (A.first ()))

while B.isEmpty ()S.insertLast (B.remove (B.first ()))

return S

Merge-Sort Tree An execution of merge-sort is depicted by a binary tree

each node represents a recursive call of merge-sort and stores unsorted sequence before the execution and its partition

sorted sequence at the end of the execution

the root is the initial call

the leaves are calls on subsequences of size 0 or 1

7 2 9 4 2 4 7 9



7

7 2 9 4 2 4 7 9

7 2 2 7 9 4 4 9

7 7 2 2 9 9 4 4

Execution ExamplePartition

7 2 9 4 2 4 7 9 3 8 6 1 1 3 8 6

7 2 9 4 3 8 6 1 1 2 3 4 6 7 8 9



8

7 2 2 7 9 4 4 9 3 8 3 8 6 1 1 6

7 7 2 2 9 9 4 4 3 3 8 8 6 6 1 1

Execution Example (cont.)Recursive call, partition

7 2 9 4 2 4 7 9 3 8 6 1 1 3 8 6

7 2 9 4 3 8 6 1 1 2 3 4 6 7 8 9



9

7 2 2 7 9 4 4 9 3 8 3 8 6 1 1 6

7 7 2 2 9 9 4 4 3 3 8 8 6 6 1 1

Execution Example (cont.)Recursive call, partition

7 2 9 4 2 4 7 9 3 8 6 1 1 3 8 6

7 2 9 4 3 8 6 1 1 2 3 4 6 7 8 9



10

7 2 2 7 9 4 4 9 3 8 3 8 6 1 1 6

7 7 2 2 9 9 4 4 3 3 8 8 6 6 1 1

Execution Example (cont.)Recursive call, base case

7 2 9 4 2 4 7 9 3 8 6 1 1 3 8 6

7 2 9 4 3 8 6 1 1 2 3 4 6 7 8 9



11

7 2 2 7 9 4 4 9 3 8 3 8 6 1 1 6

7

7 2 2 9 9 4 4 3 3 8 8 6 6 1 1

Execution Example (cont.)

Recursive call, base case

7 2 9 4 2 4 7 9 3 8 6 1 1 3 8 6

7 2 9 4 3 8 6 1 1 2 3 4 6 7 8 9



12

7 2 2 7 9 4 4 9 3 8 3 8 6 1 1 6

7

7 2

2 9 9 4 4 3 3 8 8 6 6 1 1

Execution Example (cont.)Merge

7 2 9 4 2 4 7 9 3 8 6 1 1 3 8 6

7 2 9 4 3 8 6 1 1 2 3 4 6 7 8 9



13

7 2 2 7 9 4 4 9 3 8 3 8 6 1 1 6

7

7 2

2 9 9 4 4 3 3 8 8 6 6 1 1

Execution Example (cont.)Recursive call, …, base case, merge

7 2 9 4 2 4 7 9 3 8 6 1 1 3 8 6

7 2 9 4 3 8 6 1 1 2 3 4 6 7 8 9



14

7 2 2 7 9 4 4 9 3 8 3 8 6 1 1 6

7

7 2

2 3 3 8 8 6 6 1 19 9 4

4


7 2 9 4 2 4 7 9 3 8 6 1 1 3 8 6

7 2 9 4 3 8 6 1 1 2 3 4 6 7 8 9



15

7 2 2 7 9 4 4 9 3 8 3 8 6 1 1 6

7

7 2

2 9 9 4

4 3 3 8 8 6 6 1 1

Execution Example (cont.)

Recursive call, …, merge, merge

7 2 9 4 2 4 7 9 3 8 6 1 1 3 6 8

7 2 9 4 3 8 6 1 1 2 3 4 6 7 8 9



16

7 2 2 7 9 4 4 9 3 8 3 8 6 1 1 6

7

7 2

2 9 9 4

4 3

3 8

8 6

6 1

1


7 2 9 4 2 4 7 9 3 8 6 1 1 3 6 8

7 2 9 4 3 8 6 1 1 2 3 4 6 7 8 9



17

7 2 2 7 9 4 4 9 3 8 3 8 6 1 1 6

7

7 2

2 9 9 4

4 3

3 8

8 6

6 1

1

Analysis of Merge-SortThe height h of the merge-sort tree is O (log n )

at each recursive call we divide in half the sequence,

The overall amount or work done at the nodes of depth i is O (n )

we partition and merge 2i sequences of size n 2i

we make 2i +1 recursive calls

Thus, the total running time of merge-sort is O (n log n )

depth #seqs size

0 1



18

0 1 n

1 2 n 2

i 2i n 2i

… … …

Summary of Sorting Algorithms Algorithm Time Notes

selection-sort O (n 2)

in-place

slow (good for smallinputs)

quick-sort

O (n log n )

O (n 2) for worst case if

not randomized

in-place, randomized

fastest (good for largeinputs)



19

not randomized

heap-sort O (n log n )in-place

fast (good for large inputs)

merge-sort O (n log n )sequential data access

fast (good for huge inputs)

Bucket-Sort and Radix-Sort

1, c 7, d 7, g 3, b 3, a 7, e



20

0 1 2 3 4 5 6 7 8 9

B

Bucket-Sort (4.5.1)Let be S be a sequence of n (key, element) items with keysin the range [0, N - 1]

Bucket-sort uses the keys asindices into an auxiliary array B

of sequences (buckets)Phase 1: Empty sequence S by

moving each item (k , o ) into itsbucket B [k ]

Phase 2: For i = 0, … , N - 1, moveh f b k h

Algorithm bucketSort (S, N )

Input sequence S of (key, element)items with keys in the range

[0, N - 1]Output sequence S sorted by

increasing keys

B array of N empty sequences

while S.isEmpty ()

f S.first ()

(k , o ) S.remove (f )



21

the items of bucket B [i ] to theend of sequence S

Analysis: Phase 1 takes O (n ) time

Phase 2 takes O (n +N ) time

Bucket-sort takes O (n + N ) time

B [k ].insertLast ((k , o ))

for i 0 to N - 1

while B [i ].isEmpty ()

f B [i ].first ()

(k , o ) B [i ].remove (f )S.insertLast ((k , o ))

ExampleKey range [0, 9]

7, d 1, c 3, a 7, g 3, b 7, e

Phase 1

B

1, c 7, d 7, g 3, b 3, a 7, e



22

1, c 3, a 3, b 7, d 7, g 7, e

Phase 2

0 1 2 3 4 5 6 7 8 9

B

Radix-Sort (4.5.2)Radix-sort is aspecialization oflexicographic-sort that

uses bucket-sort as thestable sorting algorithmin each dimension

Radix-sort is applicableto tuples where the

keys in each dimension i

Algorithm radixSort (S, N )

Input sequence S of d -tuples such

that (0 , …, 0) (x 1 , …, x d ) and

(x 1 , …, x d ) (N - 1 , …, N - 1)for each t ple (x x ) in S



23

yare integers in therange [0, N - 1]

Radix-sort runs in timeO (d ( n + N ))

( ) ( )for each tuple (x 1 , …, x d ) in S

Output sequence S sorted inlexicographic order

for i d downto 1

bucketSort (S , N )

Radix-Sort for

Binary NumbersConsider a sequence of n b -bit integers

x =x b - 1 … x 1x 0

We represent each elementas a b -tuple of integers inthe range [0, 1] and applyradix-sort with N = 2

This application of the

Algorithm binaryRadixSort (S )

Input sequence S of b -bitintegers

Output sequence S sorted

replace each element x

of S with the item (0, x )



24

s app ca o o eradix-sort algorithm runs inO (bn ) time

For example, we can sort a

sequence of 32-bit integersin linear time

for i 0 to b - 1

replace the key k ofeach item (k, x ) of S with bit x i of x

bucketSort (S, 2)

ExampleSorting a sequence of 4-bit integers

1001

0010

1101

0010

1110

1001

1001

1101

0001

1001

0001

0010

0001

0010

1001



25

0001

1110

1101

0001

0010

1110

1101

1110

1101

1110

Selection: The Selection Problem

Given an integer k and n elements x1, x2, …, xn,taken from a total order, find the k-th smallestelement in this set.

Of course, we can sort the set in O(n log n) timeand then index the k-th element.

7 4 9 6 2 2 4 6 7 9k=3



26

Can we solve the selection problem faster? Let’s

say O(n).

7 4 9 6 2 2 4 6 7 9k 3

Quick-Select (§ 4.7)Quick-select is a randomized

selection algorithm based on

the prune-and-searchparadigm: Prune: pick a random element x

(called pivot) and partition S into

L elements less than x

E elements equal x

x

x

L GE



27

G elements greater than x

Search: depending on k, eitheranswer is in E , or we need torecurse in either L or G

L G E

k < |L |

|L | < k < |L |+|E| (done)

k > |L |+|E|

k’ = k - |L | - |E|

Algorithm Quick-SelectInput Parameters: array a , start index p , end index r ,

target index k.

Output Parameter: a [k ] at the correct position.

QuickSelect (a , p , r , k ) {

if (p < r ) {

pi = Partition (a , p , r ) // pivot index.

if (k == pi )

return



if (k < pi )

QuickSelect (a , p , pi - 1, k )

else

QuickSelect (a , pi + 1, r , k )

}

}28

Partition

The partition step of Quick-Select is the same

partition in Quick-Sort which takes O (n ) time.Based on Probabilistic Facts, Quick-Select runs intime O(n).



29

Quick-Select Visualization An execution of quick-select can be visualized by arecursion path

Each node represents a recursive call of quick-select, andstores k and the remaining sequence

k=5, S=(7 4 9 3 2 6 5 1 8)

k=2, S=(7 4 9 6 5 8)



30

5

k=2, S=(7 4 6 5)

k=1, S=(7 6 5)

Lecture 06 Divide & ConquerLearning Outcome

Divide-and-conquer paradigm (5.2)

Review Merge-sort (4.1.1)

Recurrence Equations (5.2.1)

Iterative substitution

Recursion trees



1

The master method

Integer Multiplication (5.2.2)

Review Quick-sort (§4.3)

Divide-and-ConquerDivide-and conquer is ageneral algorithm designparadigm: Divide: divide the input data S in

two or more disjoint subsets S 1,

S 2, …

Recur: solve the subproblemsrecursively

Conquer: combine the solutionsfor S1 S2, , into a solution for S



2

for S 1, S 2, …, into a solution for S

The base case for therecursion are subproblems ofconstant size

Analysis can be done usingrecurrence equations

Merge-Sort Review

Merge-sort on an input sequence S with n elementsconsists of three steps: Divide: partition S into two sequences S 1 and S 2 of about n /2

elements each

Recur: recursively sort S 1 and S 2

Conquer: merge S 1 and S 2 into a unique sorted sequence



3

Merge-Sort ReviewInput Parameters: array a , start index p , end index r.

Output Parameter: array a.

Mergesort (a , p , r ) {

// if only one element, just return.

if (p == r )

return

// Divide: divide a into two nearly equal parts.

m = (p + r ) / 2



// Recur: sort each half.

Mergesort (a , p , m )

Mergesort (a , m + 1, r )

// Conquer: merge the two sorted halves.

Merge (a , p , m , r )

}4

Recurrence Equation

AnalysisThe conquer step of merge-sort consists of merging two sortedsequences, each with n /2 elements and implemented by means ofa doubly linked list, takes at most bn steps, for some constant b .

Likewise, the basis case (n < 2) will take at b most steps.Therefore, if we let T (n ) denote the running time of merge-sort:

2if )2/(2

2if )(

nbnnT

nbnT



5

We can therefore analyze the running time of merge-sort byfinding a closed form solution to the above equation. That is, a solution that has T (n ) only on the left-hand side.

)(

Iterative SubstitutionIn the iterative substitution, or “plug-and-chug,” technique, weiteratively apply the recurrence equation to itself and see if we canfind a pattern:

bnnT

bnnT

bnnT

bnnbnT

bnnT nT

...

4)2/(2

3)2/(2

2)2/(2

))2/())2/(2(2

)2/(2)(

44

33

22

2



6

Note that base, T(n)=b, case occurs when 2i=n. That is, i = log n.

So,

Thus, T(n) is O(n log n).

ibnnT ii )2/(2

nbnbnnT log)(

The Recursion TreeDraw the recursion tree for the recurrence relation and look for apattern:

depth T’s size

0 1 n

1 2 n 2

2if )2/(2

2if )(

nbnnT

nbnT

time

bn

bn



7

i 2i n 2i

… … …

bn

…

Total time = bn + bn log n

(last level plus all previous levels)

Master MethodMany divide-and-conquer recurrence equations havethe form:

The Master Theorem:

d nn f bnaT

d ncnT

if )()/(

if )(

)l(i)(h)l(i)(if2)(is)(then),(is)(if 1.

1loglog

loglog

TfnnT nOn f

kaka

aa bb



8

.1somefor)()/( provided

)),((is)(then),(is)(if 3.

)log(is)(then),log(is)(if 2.

log

1loglog

n f bnaf

n f nT nn f

nnnT nnn f

a

k ak a

b

bb

Master Method, Example 1The form:

The Master Theorem:

Example:

d nn f bnaT

d ncnT

if )()/(

if )(




)(is)(then),(is)(if 1.

log

1loglog

loglog

n f bnaf

n f nT nn f

nnnT nnn f

nnT nOn f

a

k ak a

aa

b

bb

bb



9

Example:nnT nT )2/(4)(

Solution: logba=2, so case 1 says T(n) is O(n2).


The Master Theorem:

Example:

d nn f bnaT

d ncnT

if )()/(

if )(





log

1loglog

loglog

n f bnaf

n f nT nn f

nnnT nnn f

nnT nOn f

a

k ak a

aa

b

bb

bb



10

Example:

nnnT nT log)2/(2)(

Solution: logba=1, so case 2 says T(n) is O(n log2 n).


The Master Theorem:

Example:

d nn f bnaT

d ncnT

if )()/(

if )(





log

1loglog

loglog

n f bnaf

n f nT nn f

nnnT nnn f

nnT nOn f

a

k ak a

aa

b

bb

bb



11

Example:

nnnT nT log)3/()(

Solution: logba=0, so case 3 says T(n) is O(n log n).


The Master Theorem:

Example:

d nn f bnaT

d ncnT

if )()/(

if )(





log

1loglog

loglog

n f bnaf

n f nT nn f

nnnT nnn f

nnT nOn f

a

k ak a

aa

b

bb

bb



12

Example:2)2/(8)( nnT nT



The Master Theorem:

Example:

d nn f bnaT

d ncnT

if )()/(

if )(





log

1loglog

loglog

n f bnaf

n f nT nn f

nnnT nnn f

nnT nOn f

a

k ak a

aa

b

bb

bb



13

Example:3)3/(9)( nnT nT



The Master Theorem:

Example:

d nn f bnaT

d ncnT

if )()/(

if )(





log

1loglog

loglog

n f bnaf

n f nT nn f

nnnT nnn f

nnT nOn f

a

k ak a

aa

b

bb

bb



14

Example:

1)2/()( nT nT

Solution: logba=0, so case 2 says T(n) is O(log n).

(binary search)


The Master Theorem:

Example:

d nn f bnaT

d ncnT

if )()/(

if )(





log

1loglog

loglog

n f bnaf

n f nT nn f

nnnT nnn f

nnT nOn f

a

k ak a

aa

b

bb

bb



15

Example:

nnT nT log)2/(2)( Solution: logba=1, so case 1 says T(n) is O(n).

(heap construction)

Integer Multiplication Algorithm: Multiply two n-bit integers I and J.

Divide step: Split I and J into high-order and low-order bits

We can then define I*J by multiplying the parts and adding:

l

n

h

l n

h

J J J

I I I

2/

2/

2

2

nnn

l

n

hl

n

h

JIJIJIJI J J I I J I

2/2/

2/2/

222)2(*)2(*



16

So, T(n) = 4T(n/2) + n, which implies T(n) is O(n2).

But that is no better than the algorithm we learned in grade

school.

l l

n

hl

n

l h

n

hh J I J I J I J I 2/2/ 222

An Improved Integer

Multiplication Algorithm Algorithm: Multiply two n-bit integers I and J.

Divide step: Split I and J into high-order and low-order bits

Observe that there is a different way to multiply parts:l

n

h

l

n

h

J J J

I I I

2/

2/

2

2

ll

n

llhhhlhhlllh

n

hh

l l

n

l l hhhl l h

n

hh

J I J I J I J I J I J I J I J I

J I J I J I J J I I J I J I

2/

2/

2])[(2

2]))([(2*



17

So, T(n) = 3T(n/2) + n, which implies T(n) is O(nlog23), by

the Master Theorem. Thus, T(n) is O(n1.585).

l l

n

hl l h

n

hh

l l l l hhhl hhl l l hhh

J I J I J I J I 2/2)(2

])[(

Integer multiplication

(1234 x 5678)

= (12x1000 + 34) (56x1000 + 78)

=1,000,000(12x56) + 1000[(12x78) +(34x56)] + (34 x 78)

4 calculation needed



12x56 , 12x78, 34x56, 34x78

AA[Lec05] The Greedy Method 18

Improved Integer multiplication

[(12)(78) + (34)(56)]= [(12-34)(78-56) + 12(56)+34(78)]

=[(12)(78)+(34)(56)-34(78) -12(56)

+12(56) + 34(78)] (underline canceleach other.

Thus, (1234 x 5678)

= 12(56) + [(12-34)(78-56) + 12(56) +3 ( 8)] 3 ( 8)




34(78)] + 34(78)

3 calculations needed

12x56 , 34x78, (12-34)x(78-56)

Quick-Sort7 4 9 6 2 2 4 6 7 9

4 2 2 4 7 9 7 9



20

2 2 9 9

Quick-SortQuick-sort is a sortingalgorithm based on the

divide-and-conquerparadigm:

Divide: pick a randomelement x (called pivot) andpartition S into

L elements less than x

E elements equal x

x

x

L G E



21

E elements equal x

G elements greater than x

Recur: sort L and G

Conquer: join L , E and G

Recall that Lec05a picks thelast element as pivot

x

Algorithm QuickSort


2 if p < r

3 pi = partition( arr, p, r )





q ( , p , )

22

PartitionWe partition an inputsequence as follows:

We remove, in turn, each

element y from S and We insert y into L , E or G ,

depending on the result ofthe comparison with thepivot x

Each insertion and removalis at the beginning or at thed f d

Algorithm partition (S, p )

Input sequence S , position p of pivot

Output subsequences L , E , G of the

elements of S less than, equal to,or greater than the pivot, resp.

L, E, G empty sequences

x S.remove (p )

while S.isEmpty ()

y S.remove (S.first ())

if y < x

L.insertLast (y )



23

end of a sequence, andhence takes O (1) time

Thus, the partition step of

quick-sort takes O (n ) time

(y)

else if y = x

E.insertLast (y )

else { y > x }

G.insertLast (y )return L , E , G

Quick-Sort Tree An execution of quick-sort is depicted by a binary tree

Each node represents a recursive call of quick-sort and stores

Unsorted sequence before the execution and its pivot

Sorted sequence at the end of the execution

The root is the initial call

The leaves are calls on subsequences of size 0 or 1

7 4 9 6 2 2 4 6 7 9



24

4 2 2 4 7 9 7 9

2 2 9 9

Execution ExamplePivot selection

7 2 9 4 2 4 7 9

7 2 9 4 3 7 6 1 1 2 3 4 6 7 8 9

3 8 6 1 1 3 8 6



25

2 2 3 3 8 89 4 4 9

9 9 4 4

Execution Example (cont.)Partition, recursive call, pivot selection

2 4 3 1 2 4 7 9

9 9

7 2 9 4 3 7 6 1 1 2 3 4 6 7 8 9

3 8 6 1 1 3 8 6



26

9 4 4 9

9 9 4 4

3 3 8 82 2

Execution Example (cont.)Partition, recursive call, base case

2 4 3 1 2 4 7

1 1 9 4 4 9

7 2 9 4 3 7 6 1 1 2 3 4 6 7 8 9

3 8 6 1 1 3 8 6

3 3 8 8



27

1 1 9 4 4 9

9 9 4 4

3 3 8 8

Execution Example (cont.)Recursive call, …, base case, join

3 8 6 1 1 3 8 6

3 3 8 8

7 2 9 4 3 7 6 1 1 2 3 4 6 7 8 9

2 4 3 1 1 2 3 4

1 1 4 3 3 4



28

3 3 8 81 1 4 3 3 4

9 9 4 4

Execution Example (cont.)Recursive call, pivot selection

7 9 7 1 1 3 8 6

8 8

7 2 9 4 3 7 6 1 1 2 3 4 6 7 8 9

2 4 3 1 1 2 3 4

1 1 4 3 3 4 9 9



29

8 81 1 4 3 3 4

9 9 4 4

9 9

Execution Example (cont.)Partition, …, recursive call, base case

7 9 7 1 1 3 8 6

8 8

7 2 9 4 3 7 6 1 1 2 3 4 6 7 8 9

2 4 3 1 1 2 3 4

1 1 4 3 3 4 9 9



30

8 81 1 4 3 3 4

9 9 4 4

9 9

Execution Example (cont.)Join, join

7 9 7 17 7 9

8 8

7 2 9 4 3 7 6 1 1 2 3 4 6 7 7 9

2 4 3 1 1 2 3 4

1 1 4 3 3 4 9 9



31

8 81 1 4 3 3 4

9 9 4 4

9 9

Worst-case Running TimeThe worst case for quick-sort occurs when the pivot is the uniqueminimum or maximum element (already sorted)

One of L and G has size n 1 and the other has size 0

The running time is proportional to the sum

n (n 1) … 2 1

Thus, the worst-case running time of quick-sort is O

(n 2)

depth time

0 n



32

1 n 1

… …

n 1 1

Expected Running TimeQuick-sort performs the best when L and G are ofequal size. In this case, a single quick-sort call involvesbn work of partition plus two recursive calls on lists of

size n/2, hence the recurrence relation is (same asmerge-sort)

T(n) = T(n/2) + bn

From the Master Theorem, the best case running time

of quick-sort is O(n log n

).For average case, if a random pivot is selected, it will



33

reduce the probability of worst case, the expectedrunning time is also O(n log n).

Lecture 07

Greedy Algorithms



1

Outline and Reading

The Greedy Method Technique (5.1)

Fractional Knapsack Problem (5.1.1)

Dijkstra’s algorithm (7.1.1)Th P i J ik Al ith (7 3 2)



2

The Prim-Jarnik Algorithm (7.3.2)

Kruskal's Algorithm (7.3.1)

The Greedy Method

TechniqueThe greedy method is a general algorithmdesign paradigm, built on the following

elements: configurations: different choices, collections, or

values to find

objective function: a score assigned toconfigurations, which we want to either maximize or

minimize

It works best when applied to problems with the



3

It works best when applied to problems with thegreedy-choice property: a globally-optimal solution can always be found by a

series of local improvements from a startingconfiguration.

Making ChangeProblem: A dollar amount to reach and a collection ofcoin amounts to use to get there.

Configuration: A dollar amount yet to return to acustomer plus the coins already returned

Objective function: Minimize number of coins returned.

Greedy solution: Always return the largest coin you can

Example 1: Coins are valued $.32, $.08, $.01 Has the greedy-choice property, since no amount over $.32 can

be made with a minimum number of coins by omitting a $.32



4

be made with a minimum number of coins by omitting a $.32coin (similarly for amounts over $.08, but under $.32).

Example 2: Coins are valued $.30, $.20, $.05, $.01 Does not have greedy-choice property, since $.40 is best made

with two $.20’s, but the greedy solution will pick three coins(which ones?)

The Fractional Knapsack

ProblemGiven: A set S of n items, with each item i having bi - a positive benefit

wi - a positive weight

Goal: Choose items with maximum total benefit but withweight at most W.

If we are allowed to take fractional amounts, then this isthe fractional knapsack problem.

In this case, we let xi denote the amount we take of item i

Obj i i i b )/(



5

Objective: maximize

Constraint:

S i

iii w xb )/(

S i

i W x

ExampleGiven: A set S of n items, with each item i having bi - a positive benefit



1 2 3 4 5Items:

Solution:• 1 ml of 5• 2 ml of 3

“knapsack”



6

Weight:

Benefit:

1 2 3 4 5

4 ml 8 ml 2 ml 6 ml 1 ml

$12 $32 $40 $30 $50

Value: 3($ per ml)

4 20 5 50 10 ml

• 2 ml of 3• 6 ml of 4• 1 ml of 2

The Fractional Knapsack

AlgorithmGreedy choice: Keep takingitem with highest value(benefit to weight ratio) Since Run time: O(n log n). Why?

Correctness: Suppose thereis a better solution there is an item i with higher

value than a chosen item j,but xi<wi, x j>0 and vi >v j

Algorithm fractionalKnapsack (S, W )

Input: set S of items w/ benefit bi

and weightw

i; max. weightW

Output: amount xiof each item i

to maximize benefit w/ weightat most W

for each i tem i in S

x i 0

v i b i / w i {value}w 0 {total weight}

while w < W

S i

iii

S i

iii xwbw xb )/()/(



7

If we substitute some j with i,we get a better solution

How much of i: min{wi-xi, x j}

Thus, there is no bettersolution than the greedy one

while w < W

remove item i w/ highest v i

x i min{w i , W - w }

w w + min{w i

, W - w }

Shortest Paths

CB

A

D

0

328

5 8

48

7 1

2

3 9



Lec09 Shortest Paths 8

E F

5 82 5

Outline and ReadingWeighted graphs (7.1)

Shortest path problem

Shortest path properties

Dijkstra’s algorithm (7.1.1)

Algorithm

Edge relaxation



9

Weighted GraphsIn a weighted graph, each edge has an associated numericalvalue, called the weight of the edge

Edge weights may represent, distances, costs, etc.

Example: In a flight route graph, the weight of an edge represents the

distance in miles between the endpoint airports

ORD

PVD

SFOLGA



10

MIA DFWLAX

LGA

HNL

Shortest Path ProblemGiven a weighted graph and two vertices u and v , we want tofind a path of minimum total weight between u and v.

Length of a path is the sum of the weights of its edges.

Example: Shortest path between Providence and Honolulu

Applications Internet packet routing

Flight reservations

Driving directions

ORD PVDSFO



11

MIA DFWLAX

LGA

HNL

Shortest Path PropertiesProperty 1:

A subpath of a shortest path is itself a shortest path

Property 2:

There is a tree of shortest paths from a start vertex to all the othervertices

Example:Tree of shortest paths from Providence

ORD PVDSFO

LGA



12

MIA DFWLAX

LGA

HNL

Dijkstra’s AlgorithmThe distance of a vertexv from a vertex s is the

length of a shortest pathbetween s and v

Dijkstra’s algorithmcomputes the distancesof all the vertices from agiven start vertex s

Assumptions:

the graph is connected

We grow a “cloud” of vertices,beginning with s and eventually

covering all the vertices

We store with each vertex v alabel d (v ) representing thedistance of v from s in thesubgraph consisting of the cloudand its adjacent vertices

At each step

We add to the cloud the vertex



13

g p

the edges areundirected

the edge weights arenonnegative

u outside the cloud with thesmallest distance label, d (u )

We update the labels of the

vertices adjacent to u

Edge RelaxationConsider an edge e (u,z )

such that

u is the vertex most recentlyadded to the cloud

z is not in the cloud

The relaxation of edge e

updates distanced (z ) asfollows:

d(z) min{d(z) d(u) + weight(e)}

d (z ) 75d (u ) 50

z s u

d(z) 60d (u ) 50

e



14

d (z ) min{d (z ),d (u ) + weight (e )} d (z ) 60

z s u e

Example

CB

A

E

D

F

0

428

48

7 1

2 5

2

3 9

A 0 48

2

CB

A

E

D

F

0

328

5 8

48

7 1

2 5

2

3 9

A 0 48

2



15

CB

E

D

F

328

5 11

7 1

2 5

2

3 9

CB

E

D

F

327

5 8

7 1

2 5

2

3 9

Example (cont.)

CB

A

E

D

F

0

327

5 8

48

7 1

2 5

2

3 9

CB

A

D

0

327

48

7 1

2



16

CB

E

D

F

5 8

7 1

2 5

3 9

Dijkstra’s Algorithm A priority queue storesthe vertices outside thecloud

Key: distance Element: vertex

Locator-based methods insert (k,e ) returns a

locator

replaceKey (l ,k ) changesthe key of an item

We store two labelsith h t

Algorithm DijkstraDistances (G, s )Q new heap-based priority queue

for all v G.vertices ()if v s

setDistance (v, 0)else

setDistance (v,)l Q.insert (getDistance (v ), v )setL ocator (v,l )

while Q.isEmpty ()

u Q.removeM in ()for all e G.incidentEdges (u )

{ relax edge e }



17

with each vertex: Distance (d(v) label)

locator in priority

queue

{ g }z G.opposite (u,e )r getDistance (u ) + weight (e )if r < getDistance (z )

setDistance (z,r )Q.replaceKey (getLocator (z ),r )

AnalysisGraph operations Method incidentEdges is called once for each vertex

Label operations

We set/get the distance and locator labels of vertex z O (deg(z )) times Setting/getting a label takes O (1) time

Priority queue operations Each vertex is inserted once into and removed once from the priority

queue, where each insertion or removal takes O (log n ) time

The key of a vertex in the priority queue is modified at most deg(w )times, where each key change takes O (log n ) time

Dijkstra’s algorithm runs in O ((n + m ) log n ) time provided theh i t d b th dj li t t t



18

graph is represented by the adjacency list structure

Recall that Sv deg(v ) 2m

The running time can also be expressed as O (m log n ) since thegraph is connected

Why Dijkstra’s Algorithm

WorksDijkstra’s algorithm is based on the greedymethod. It adds vertices by increasing distance.

CB

A

E

D

F

0

327

5 8

48

7 1

2 5

2

3 9

Suppose it didn’t find all shortestdistances. Let F be the first wrongvertex the algorithm processed.

When the previous node, D, on thetrue shortest path was considered,its distance was correct.

But the edge (D,F) was relaxed atthat time!



19

Thus, so long as d(F)>d(D), F’sdistance cannot be wrong. That is,there is no wrong vertex.

Why It Doesn’t Work for

Negative-Weight Edges

If a node with a negativeincident edge were to be addedlate to the cloud, it could mess

up distances for vertices alreadyin the cloud.

CB

A

E

D

F

0

457

5 9

48

7 1

2 5

6

0 -8

Dijkstra’s algorithm is based on the greedymethod. It adds vertices by increasing distance.



20

C’s true distance is 1, but

it is already in the cloudwith d(C)=5!

Minimum Spanning Trees (MST)

JFK

BOS

MIA

ORD

LAX

DFW

SFOBWI

PVD

867

2704

187

1258

849

144740

1391

184

946

1090

1121

1846 621

802

1464

1235

337



21

MIA

2342

Outline and Reading

Minimum Spanning Trees (7.3)

Definitions A crucial fact

The Prim-Jarnik Algorithm (7.3.2)

Kruskal's Algorithm (7.3.1)



22

Minimum Spanning TreeSpanning subgraph

Subgraph of a graph G

containing all the vertices of G

Spanning tree Spanning subgraph that is

itself a (free) tree

Minimum spanning tree (MST)

Spanning tree of a weighted

graph with minimum totaledge weight

Applications

ORD

PIT

STL

DEN

DCA

101

9

8

6

3

25

7

4



23

Applications

Communications networks

Transportation networks ATLDFW

Cycle PropertyCycle Property:

Let T be a minimumspanning tree of a

weighted graphG

Let e be an edge of G that is not in T and C letbe the cycle formed by e

with T

For every edge f of C,weight (f ) weight (e )

Proof:

By contradiction

8

4

23

6

7

7

9

8 e

C

f

8

4C

f

Replacing f with e yieldsa better spanning tree



24

If weight (f ) > weight (e ) wecan get a spanning treeof smaller weight by

replacing e with f

23

6

7

7

9

8

C

e

U V

Partition PropertyPartition Property:

Consider a partition of the vertices ofG into subsets U and V

Let e be an edge of minimum weightacross the partition

There is a minimum spanning tree ofG containing edge e

Proof:

Let T be an MST of G

If T does not contain e, consider thecycle C formed by e with T and let f

be an edge of C across the partition

B h l

7

4

28

5

7

3

9

8 e

f

7

4

9

f

Replacing f with e yieldsanother MST

U V



25

By the cycle property,weight (f ) weight (e )

Thus, weight (f ) weight (e )

We obtain another MST by replacingf with e

28

5

7

3

9

8e

Prim AlgorithmInput: A non-empty connected weighted graph withvertices V and edges E (the weights can be negative).

Initialize: V new = {x }, where x is an arbitrary node (startingpoint) from V , E new = {}

Repeat until V new = V :

Choose an edge {u , v } with minimal weight such that u isin V new and v is not (if there are multiple edges with the

same weight, any of them may be picked) Add v to V new, and {u , v } to E new




Prim-Jarnik’s AlgorithmSimilar to Dijkstra’s algorithm (for a connected graph)

We pick an arbitrary vertex s and we grow the MST as acloud of vertices, starting from s

We store with each vertex v a label d (v ) = the smallestweight of an edge connecting v to a vertex in the cloud

At each step: We add to the cloud the

vertex u outside the cloudwith the smallest distancelabel

W d t th l b l f th



27

We update the labels of thevertices adjacent to u

Prim-Jarnik’s Algorithm (cont.) A priority queue storesthe vertices outside thecloud

Key: distance Element: vertex

Locator-based methods insert (k,e ) returns a

locator

replaceKey (l ,k ) changes

the key of an itemWe store three labelswith each vertex:

Algorithm PrimJarnikMST (G )Q new heap-based priority queues a vertex of G for all v G.vertices ()

if v s setDistance (v, 0)

elsesetDistance (v,)

setParent (v,)l Q.insert (getDistance (v ), v )setL ocator (v,l )

while Q.isEmpty ()


z G.opposite (u,e )r weight(e)



28

with each vertex: Distance

Parent edge in MST

Locator in priority queue

r weight (e )if r < getDistance (z )

setDistance (z,r )setParent (z,e )

Q.replaceKey (getLocator (z ),r )

ExampleB

D

C

A

F

E

7

4

28

5

7

3

9

8

07

2

8

B D

F

74

2 59

2

5

7

B

D

C

A

F

E

7

4

28

5

7

3

9

8

07

2

5

7

BD

F

74

59

2

5 4

7



29

C

A

F

E

28

5

7

38

0 7

C

A

F

E

28

5

7

38

0 7

Example (contd.)

B

D

C

A

F

E

7

4

28

5

7

3

9

8

03

2

5 4

7

B

D

CF

7

4

28

59

2

5 4

7



30

A E

8

7

38

03

Dijkstra vs Prim

Algorithm PrimJarnikMST (G )Q new heap-based priority queues a vertex of G for all v G.vertices ()



setParent (v,

)l Q.insert (getDistance (v ), v )setL ocator (v,l )

while Q.isEmpty ()u Q.removeM in ()for all e G.incidentEdges (u )

z G.opposite (u,e )

r weight (e )if r < getDistance (z )

Algorithm DijkstraDistances (G, s )Q new heap-based priority queuefor all v G.vertices ()

if v s

setDistance (v, 0)

else

setDistance (v,)l Q.insert (getDistance (v ), v )setL ocator (v,l )

while Q.isEmpty ()


{ relax edge e }z G.opposite (u,e )

r getDistance (u ) +i h ( )




setDistance (z,r )

setParent (z,e )Q.replaceKey (getLocator (z ),r )

weight (e )if r < getDistance (z )

setDistance (z,r )

Q.replaceKey (getLocator (z ),r )

Dijkstra vs Prim

Min Spanning Tree :- How to connect all nodes with the least cost?

Graph must be connected.

E.g. Minimize the TNB power line transmission cost, Railway line cost

Save material/construction cost

You can start at any node, the Min Spanning Tree is the same.

Dijkstra: Shortest Path to all nodes from an individual node

a b C5 5

9

a b C5 5

9

Minimum Spanning Tree

Shortest Path to all nodesfrom node A

a b C5 5

9

Shortest Path to all nodesfrom node C




Dijkstra: Shortest Path to all nodes from an individual node

Specific to each node. Graph is different for node A and node C

Graph differs depending on which node is the root (starting point)

How to reach all nodes with the least cost. Save time and fuel

AnalysisGraph operations Method incidentEdges is called once for each vertex

Label operations

We set/get the distance, parent and locator labels of vertex z O (deg(z ))times

Setting/getting a label takes O (1) time

Priority queue operations Each vertex is inserted once into and removed once from the priority

queue, where each insertion or removal takes O (log n ) time

The key of a vertex w in the priority queue is modified at most deg(w )times, where each key change takes O (log n ) time

Prim-Jarnik’s algorithm runs in O ((n + m ) log n ) time provided the



33

g (( ) g ) pgraph is represented by the adjacency list structure

Recall that Sv deg(v ) 2m

The running time is O (m log n ) since the graph is connected

Kruskal’s Algorithm A priority queue storesthe edges outside thecloud Key: weight

Element: edge

At the end of thealgorithm We are left with one

cloud that encompasses

the MST A tree T which is our

MST

Algorithm KruskalMST (G )for each vertex V in G do

define a Cloud(v) of {v }let Q be a priority queue.Insert all edges into Q using theirweights as the keyT

while T has fewer than n -1 edges doedge e = Q.removeM in() Let u , v be the endpoints of eif Cloud(v) Cloud(u) then

Add edge e to T

Merge Cloud(v) and Cloud(u) return T



34

Kruskal vs PrimAlgorithm KruskalMST (G )

for each vertex V in G dodefine a Cloud(v) of {v }

let Q be a priority queue.Insert all edges into Q using theirweights as the keyT

while T has fewer than n -1 edges do

// remove edge by edgeedge e = Q.removeM

in() Let u , v be the endpoints of eif Cloud(v) Cloud(u) then

Add edge e to T Merge Cloud(v) and Cloud(u)

return T

Algorithm PrimJarnikMST (G )

Q new heap-based priority queues a vertex of G for all v G.vertices ()



setParent (v,

)l Q.insert (getDistance (v ), v )setL ocator (v,l )

while Q.isEmpty ()u Q.removeM in ()

//Remove vertex/node by node and//find the incident edge

for all e G.incidentEdges (u )z G.opposite (u,e )r weight (e )if r < getDistance (z )

setDistance(z r)




setDistance (z,r )setParent (z,e )Q.replaceKey (getLocator (z ),r )

Kruskal vs Prim

If algorithm is stopped before completion

Prim :- Always one connected tree

Kruskal : 1 connected tree or a forest with multiple

trees.



36

Data Structure for

Kruskal AlgortihmThe algorithm maintains a forest of trees

An edge is accepted it if connects distinct trees

We need a data structure that maintains a partition,

i.e., a collection of disjoint sets, with the operations:-find(u): return the set storing u

-union(u,v): replace the sets storing u and v withtheir union



37

Representation of a

PartitionEach set is stored in a sequence

Each element has a reference back to the set

operation find(u) takes O(1) time, and returns the set ofwhich u is a member.

in operation union(u,v), we move the elements of thesmaller set to the sequence of the larger set and updatetheir references

the time for operation union(u,v) is min(nu,nv), where nuand nv are the sizes of the sets storing u and v

Whenever an element is processed, it goes into a



38

p , gset of size at least double, hence each element isprocessed at most log n times

Partition-Based

Implementation A partition-based version of Kruskal’s Algorithmperforms cloud merges as unions and tests as finds.

Algorithm Kruskal(G ):

Input: A weighted graph G .

Output: An MST T for G .

Let P be a partition of the vertices of G , where each vertex forms a separate set.

Let Q be a priority queue storing the edges of G , sorted by their weights

Let T be an initially-empty tree

while Q is not empty do

(u,v ) Q .removeMinElement()

if P find(u) != P find(v) then R i ti



39

if P.find(u ) != P.find(v ) then

Add (u,v ) to T

P.union(u,v )

return T

Running time:O((n+m)log n)

Kruskal

Example

JFK

BOS

ORD

LAX

DFW

SFOBWI

PVD

8672704

187

1258

849

144740

1391

184

946

1090

1846 621

802

1464

1235

337



40

MIA

1121

2342

JFK

BOS

ORD

LAX

DFW

SFOBWI

PVD

8672704

187

1258

849

144740

1391

184

946

1090

1846 621

802

1464

1235

337

Example



MIA

1121

2342

41

JFK

BOS

ORD

LAX

DFW

SFOBWI

PVD

8672704

187

1258

849

144740

1391

184

946

1090

1846 621

802

1464

1235

337

Example



MIA

1121

2342

42

JFK

BOS

ORD

LAX

DFW

SFOBWI

PVD

8672704

187

1258

849

144740

1391

184

946

1090

1846 621

802

1464

1235

337

Example



MIA

1121

2342

43

JFK

BOS

ORD

LAX

DFW

SFOBWI

PVD

8672704

187

1258

849

144740

1391

184

946

1090

1846 621

802

1464

1235

337

Example



MIA

1121

2342

44

JFK

BOS

ORD

LAX

DFW

SFOBWI

PVD

8672704

187

1258

849

144740

1391

184

946

1090

1846 621

802

1464

1235

337

Example



MIA

1121

2342

45

JFK

BOS

ORD

LAX

DFW

SFOBWI

PVD

8672704

187

1258

849

144740

1391

184

946

1090

1121

1846 621

802

1464

1235

337

Example



MIA

1121

2342

46

JFK

BOS

ORD

LAX

DFW

SFOBWI

PVD

8672704

187

1258

849

144740

1391

184

946

1090

1121

1846 621

802

1464

1235

337

Example



MIA

1121

2342

47

JFK

BOS

ORD

LAX

DFW

SFOBWI

PVD

8672704

187

1258

849

144740

1391

184

946

1090

1121

1846 621

802

1464

1235

337

Example



MIA

1121

2342

48

JFK

BOS

ORD

LAX

DFW

SFOBWI

PVD

8672704

187

1258

849

144740

1391

184

946

1090

1121

1846 621

802

1464

1235

337

Example



MIA

1121

2342

49

JFK

BOS

ORD

LAX

DFW

SFOBWI

PVD

8672704

187

1258

849

144740

1391

184

946

1090

1121

1846 621

802

1464

1235

337

Example



MIA

1121

2342

50

JFK

BOS

ORD

LAX

DFW

SFOBWI

PVD

8672704

187

1258

849

144740

1391

184

946

1090

1121

1846 621

802

1464

1235

337

Example



MIA

1121

2342

51

JFK

BOS

ORD

LAX

DFW

SFOBWI

PVD

8672704

187

1258

849

144740

1391

184

946

1090

1121

1846 621

802

1464

1235

337

Example



MIA

1121

2342

52

Example

JFK

BOS

ORD

LAX

DFW

SFOBWI

PVD

8672704

187

1258

849

144740

1391

184

946

1090

1846 621

802

1464

1235

337



53

MIA

1121

2342

Dijkstra, Prim vs Kruskal

Minimum edge weight datastructure

Time complexity (total)

Prim adjacency matrix, searching O(|V|2)

Prim binary heap and adjacency list O((|V| + |E|) log |V|) = O(|E| log |V|)

Prim Fibonacci heap and ad

jacency list

O(|E| + |V| log |V|)

Kruskal O( |E| log |V|)

Dijk t O((|V| |E|) l |V|) O(|E| l |V|)

http://en.wikipedia.org/wiki/Adjacency_matrix

http://en.wikipedia.org/wiki/Binary_heap

http://en.wikipedia.org/wiki/Adjacency_list








http://en.wikipedia.org/wiki/Fibonacci_heap


http://en.wikipedia.org/wiki/Binary_heap

http://en.wikipedia.org/wiki/Adjacency_matrix




Dijkstra O((|V| + |E|) log |V|) = O(|E| log |V|)

Lecture 08Dynamic Programming



1

Learning OutcomeMatrix Chain-Product (5.3.1)

The General Technique (5.3.2)

0-1 Knapsack Problem (5.3.3)

Transitive closure (6.4.2)

The Floyd-Warshall Algorithm



2

Matrix Chain-ProductsDynamic Programming is a generalalgorithm design paradigm. Rather than give the general structure, let us

first give a motivating example:

Matrix Chain-Products

Review: Matrix Multiplication. C = A*B

A is d × e and B is e × f

A C

B

f

e

e j

1

0

],[*],[],[e

k

jk Bk i A jiC



3

O (def ) timeA C

d d

f

i i,j

Matrix Chain-ProductsMatrix Chain-Product: Compute A=A 0*A 1*…*A n-1

A i is di × di+1

Problem: How to parenthesize?

Example B is 3 × 100

C is 100 × 5

D is 5 × 5

(B*C)*D takes (3 x 100 x 5) + (5 x 5 x5) =1500 + 75 1575 ops



4

1500 + 75 = 1575 ops

B*(C*D) takes (3 x 100 x 5) + (100 x 5

x5) = 1500 + 2500 = 4000 ops

An Enumeration ApproachMatrix Chain-Product Alg.: Try all possible ways to parenthesize

A=A 0*A 1*…*A n-1

Calculate number of ops for each one Pick the one that is best

Running time: The number of paranethesizations is equal

to the number of binary trees with n nodes This is exponential!

It is called the Catalan number and it is



5

It is called the Catalan number, and it isalmost 4n.

This is a terrible algorithm!

A Greedy ApproachIdea #1: repeatedly select the product thatuses (up) the most operations.

Counter-example: A is 10 × 5

B is 5 × 10

C is 10 × 5

D is 5 × 10 Greedy idea #1 gives (A*B)*(C*D), which takes

(10x5x10)+(10x10x10)+(10x5x10) =500 1000 500 2000



6

500+1000+500 = 2000 ops

A*((B*C)*D) takes (10x5x10)+(5x10x5)+(5x5x10)= 500+250+250 = 1000 ops

Another Greedy ApproachIdea #2: repeatedly select the product that usesthe fewest operations.

Counter-example: A is 101 × 11

B is 11 × 9

C is 9 × 100

D is 100 × 99 Greedy idea #2 gives A*((B*C)*D)), which takes

109989+9900+108900=228789 ops



7

(A*B)*(C*D) takes 9999+89991+89100=189090 ops

The greedy approach is not giving us theoptimal value.

A “Recursive” ApproachDefine subproblems: Find the best parenthesization of A i*A i+1*…*A j.

Let Ni,j denote the number of operations done by this

subproblem (i to j). The optimal solution for the whole

problem (0 to n-1) is N0,n-1.

Subproblem optimality: The optimal solution can bedefined in terms of optimal subproblems There has to be a final multiplication (root of the expression

tree) for the optimal solution. Say, the final multiply is at index i: (A 0*…*A i)*(A i+1*…*A n-1).

Then the optimal solution N0,n-1 is the sum of two optimalsubproblems N and N plus the time for the last multiply



8

subproblems, N0,i and Ni+1,n-1 plus the time for the last multiply.

If the global optimum did not have these optimal subproblems,

we could define an even better “optimal” solution.

A Dynamic Programming

AlgorithmSince subproblemsoverlap, we don’tuse recursion.

Instead, weconstruct optimalsubproblems “bottom-

up.”

Ni,i’s are easy, sostart with them

Then do length2,3,… subproblems,and so on.

Algorithm matrixChain (S ):

Input: sequence S of n matrices to be multiplied

Output: number of operations in an optimal paranethization of S

for i 1 to n-1 do

N i ,i 0

for b 1 to n-1 do

for i 0 to n-b-1 do

j i+b

N i ,j + infinity

for k i to j-1 do



9

Running time: O(n3)

o to j do

N i ,j min{ N i ,j , N i ,k + N k+ 1,j +d i d k+ 1 d j+ 1}

answerN 0 1

0

1

2 …

n-1

…

n-1 j

i

A Dynamic Programming

Algorithm VisualizationThe bottom-upconstruction fills in theN array by diagonals

Ni,j gets values fromprevious entries in i-throw and j-th column

Filling in each entry inthe N table takes O(n)

time.Total run time: O(n3)

Getting actual

}{min 11,1,,

jk i jk k i jk i

ji d d d N N N



10

gparenthesization can bedone by remembering

“k” for each N entry

Dynamic Programming Algorithm

Visualization A 0: 30 X 35; A 1: 35 X15; A 2: 15X5;

A 3: 5X10; A 4: 10X20; A 5: 20 X 25

7125

}

1137520*10*3504375,712520*5*3510002625

,1300020*15*3525000

min{

5414,43,1

5314,32,1

5214,21,1

4,1

d d d N N d d d N N

d d d N N

N

}{min 11,1,, jk i jk k i jk i ji d d d N N N



7125

The General Dynamic

Programming Technique Applies to a problem that at first seems torequire a lot of time (possibly exponential),provided we have: Simple subproblems: the subproblems can be

defined in terms of a few variables, such as j, k, l,m, and so on.

Subproblem optimality: the global optimum value

can be defined in terms of optimal subproblems Subproblem overlap: the subproblems are not

independent, but instead they overlap (hence,h ld b t t d b tt )



12

should be constructed bottom-up).

The 0/1 Knapsack ProblemGiven: A set S of n items, with each item i having bi - a positive benefit



If we are not allowed to take fractional amounts, thenthis is the 0/1 knapsack problem. In this case, we let T denote the set of items we take

Objective: maximize T i

ib



13

Constraint:

T ii W w

Given: A set S of n items, with each item i having bi - a positive benefit



Example

Weight:

1 2 3 4 5

4 in 2 in 2 in 6 in 2 in

Items:

Solution:• 5 (2 in)• 3 (2 in)• 1 (4 in)

“knapsack”

http://loki.cs.brown.edu:8081/webae/images/cover-large.jpg

http://loki.cs.brown.edu:8081/webae/images/cover-large.jpg



14

Weight:

Benefit:

4 in 2 in 2 in 6 in 2 in

$20 $3 $6 $25 $80W = 9 in w = 8 in

B = $126

A 0/1 Knapsack Algorithm,

First AttemptGreedy: repeatedly add item with maximum ratio,benefit/ weight

Problem: greedy method does not have subproblem

optimality:{5, 2, 1} has benefit = 35, greedy not optimal

{3, 4} has optimum benefit of 40!

W = 11 kg

Weight (kg):

Benefit:

1 2 5 6 7

1 6 18 22 28

Items: 1 2 3 4 5



15

Benefit: 1 6 18 22 28

A 0/1 Knapsack Algorithm,

Second AttemptSk : Set of items numbered 1 to k.

Define B[k,w] = best selection from Sk withweight at most w

Good news: this does have subproblemoptimality:

i.e., best subset of Sk with weight exactly w is

else}],1[],,1[max{

if ],1[

],[ k k

k

bwwk Bwk B

wwwk B

wk B



16

i.e., best subset of Sk with weight exactly w iseither the best subset of Sk-1 w/ weight w or the

best subset of Sk-1 w/ weight w-wk plus item k.

The 0/1 Knapsack

AlgorithmRecall definition of B[k,w]:

Since B[k,w] is defined interms of B[k-1,*], we canreuse the same array

Running time: O(nW).

Not a polynomial-timealgorithm if W is large

This is a pseudo-polynomialtime algorithm

Algorithm 01Knapsack (S, W ):

Input: set S of items w/ benefit b i and weight w i ; max. weight W

Output: benefit of best subset withweight at most W

for w 0 to W do

B [w ] 0for k 1 to n do

for w W downto w k do

if B[ ] b B[ ] th

else}],1[],,1[max{

if ],1[],[

k k

k

bwwk Bwk B

wwwk Bwk B



17

time algorithm if B [w-w k ]+b k > B [w ] then

B [w ] B [w-w k ]+b k

The 0/1 Knapsack Algorithm0 1 2 3 4 5 6 7 8 9 10 11

ø 0 0 0 0 0 0 0 0 0 0 0 0

{1} 0 1 1 1 1 1 1 1 1 1 1 1

{1,2} 0 1 6 7 7 7 7 7 7 7 7 7

{1,2,3} 0 1 6 7 7 18 19 24 25 25 25 25

{1,2,3,4} 0 1 6 7 7 18 22 24 28 29 29 40

{1,2,3,4,5} 0 1 6 7 7 18 22 28 29 34 35 40



18

Transitive Closure ProblemGiven a digraph G , thetransitive closure of G is thedigraph G* such that

G* has the same verticesas G

if G has a directed pathfrom u to v (u v ), G*

has a directed edge fromu to v

The transitive closureprovides reachability

B

A

D

C

E

B

D

C

E

G



19

provides reachabilityinformation about a digraph

A

C

G*

Computing the

Transitive ClosureWe can performDFS starting ateach vertex

O(n(n+m))

If there's a way to get

from A to B and from

B to C, then there's a

way to get from A to C.

Alternatively ... Usedynamic programming:The Floyd-WarshallAl ith



20

Algorithm

Floyd-Warshall

Transitive ClosureIdea #1: Number the vertices 1, 2, …, n.

Idea #2: Consider paths that use only

vertices numbered 1, 2, …, k, asintermediate vertices:

j

i

Uses only verticesb d 1 k 1

Uses only vertices numbered 1,…,k (add this edge if it’s not already in)



21

k

numbered 1,…,k -1Uses only vertices

numbered 1,…,k -1

Floyd-Warshall’s AlgorithmFloyd-Warshall’s algorithmnumbers the vertices of G asv 1 , …, vn and computes aseries of digraphs G 0 , …, G n G 0=G

G k has a directed edge (v i , v j )if G has a directed path fromv i to v j with intermediatevertices in the set {v 1 , …, vk }

We have that G n = G* In phase k , digraph G k iscomputed from G k 1

Running time: O(n3

),

Algorithm FloydWarshall (G )

Input digraph G

Output transitive closure G* of G

i 1

for all v G.vertices ()denote v as v i i i + 1

G 0 G

for k 1 to n do

G k G k 1

for i 1 to n (i k ) dofor j 1 to n ( j i , k ) do

if G k 1.areAdjacent (v i , v k )

G k 1.areAdjacent (v k , v j )



22

Running time: O(n ),assuming areAdjacent is O(1)

(e.g., adjacency matrix)

if G k .areAdjacent (v i , v j )

G k .insertDirectedEdge (v i , v j , k )

return G n

Floyd-Warshall Example

JFK

BOS

ORD

LAXDFW

SFO

v2

v1

v3

v4

v6

v7



23

MIA1

v 5

Floyd-Warshall, Iteration 1

JFK

BOS

ORD

LAXDFW

SFO

v2

v1

v3

v4

v6

v7



24

MIA1

v 5


JFK

BOS

ORD

LAXDFW

SFO

v2

v1

v3

v4

v6

v7



25

MIA

v 5


JFK

BOS

ORD

LAXDFW

SFO

v2

v1

v3

v4

v6

v7



26

MIA

v 5


JFK

BOS

ORD

LAXDFW

SFO

v2

v1

v3

v4

v6

v7



27

MIA

v 5


JFK

ORD

LAXDFW

SFO

v2

v1

v3

v4

v6

v7

BOS



28

MIA

v 5


JFK

ORD

LAXDFW

SFO

v2

v1

v3

v4

v6

v7

BOS



29

MIA

v 5

Floyd-Warshall, Conclusion

JFK

ORD

LAXDFW

SFO

v2

v1

v3

v4

v6

v7

BOS



30

MIA

v 5

Lecture 09Text Processing

1

a b a c a a b

234

a b a c a b

a b a c a b



1

Outline and ReadingPattern matching algorithms

Brute-force algorithm (9.1.2)

Boyer-Moore algorithm (9.1.3)

Knuth-Morris-Pratt algorithm (9.1.4)

Trie

Huffman encoding



2

Strings A string is a sequence ofcharacters

Examples of strings: Java program

HTML document

DNA sequence

Digitized image

An alphabet is the set ofpossible characters for a

family of stringsExample of alphabets: ASCII

Unicode

Let P be a string of size m

A substring P [i .. j ] of P is thesubsequence of P consisting ofthe characters with ranks

between i and j A prefix of P is a substring of

the type P [0 .. i ]

A suffix of P is a substring ofthe type P [i ..m - 1]

Given strings T (text) and P

(pattern), the pattern matchingproblem consists of finding asubstring of T equal to P

Applications:



3

{0, 1}

{A, C, G, T}

Text editors

Search engines Biological research

Brute-Force AlgorithmThe brute-force patternmatching algorithm comparesthe pattern P with the text T for each possible shift of P

relative to T , until either a match is found, or

all placements of the patternhave been tried

Brute-force pattern matchingruns in time O (nm )

Example of worst case: T = aaa … ah

P = aaah

may occur in images and

Algorithm BruteForceMatch (T, P )

Input text T of size n and patternP of size m

Output starting index of a

substring ofT

equal toP

or-

1if no such substring exists

for i 0 to n - m

{ test shift i of the pattern }

j 0

while j < m T [i + j ] = P [ j ]

j j + 1

if j = m

return i {match at i }

else



4

may occur in images andDNA sequences

unlikely in English text

else

break while loop {mismatch}

return -1 {no match anywhere}

Brute-Force AlgorithmMatching:

Brute-Force Matching:

Comparing from left to right

ABABC ABABC ABABC

ABABABCCA ABABABCCA ABABABCCA

Successful match!

Text to find

Source TextError Error



5

Brute Force AlgorithmBrute-Force Matching II

Worst case looks like:Scan_Text: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaab

Pattern: aaaab

Best case looks like:Scan_Text: aaaabjhgasjhgasjhagsjhagsjhgahgaaaaaaaaabPattern: aaaab

Typical average case:Scan_Text: This is an algorithm difficult to beatPattern: algorithm

Analysis: Worst Case: O(M * N)



6

Analysis: Worst Case: O(M N)Best Case: O(M)

Average Case: O(M+N)

Boyer-Moore HeuristicsThe Boyer-Moore’s pattern matching algorithm is based on twoheuristics

Looking-glass heuristic: Compare P with a subsequence of T moving backwards

Character-jump heuristic: When a mismatch occurs at T [i ] = c

If P contains c , shift P to align the last occurrence of c in P with T [i ]

Else, shift P to align P [0] with T [i + 1]

Example

1

a p a t t e r n m a t c h i n g a l g o r i t h m

r i t h m r i t h m r i t h m r i t h m

3 5 7891011



7

r i t h m r i t h m r i t h m

2 4 6

Last-Occurrence FunctionBoyer-Moore’s algorithm preprocesses the pattern P and thealphabet to build the last-occurrence function L mapping tointegers, where L (c ) is defined as

the largest index i such that P [i ] = c or

-1 if no such index exists

Example:

= {a, b, c, d }

P = abacab

The last-occurrence function can be represented by an arrayindexed by the numeric codes of the characters

The last-occurrence function can be computed in time O (m + s ),where m is the size of P and s is the size of

c a b c d

L (c ) 4 5 3 -1



8

m - j

i

j l

. . . . . . a . . . . . .

. . . . b a

. . . . b a

j

Case 1: j 1 + l The Boyer-Moore AlgorithmAlgorithm BoyerMooreMatch (T, P, )

L lastOccurenceFunction (P, )i m - 1 j m - 1

repeatif T [i ] = P [ j ]if j = 0

return i { match at i }else

i i - 1 j j - 1

else{ character-jump }l L [T [i ]]i i + m – min( j , 1 + l ) j m - 1 m - (1 + l )

i

j l

. . . . . .

a . . . . . .

. a . . b .

Case 2: 1 + l j



9

until i > n - 1

return-

1 { no match }

( )

. a . . b .

1 + l

Example

1

a b a c a a b a d c a b a c a b a a b b

234

5

6

7

891012

a b a c a b

a b a c a b

a b a c a b

a b a c a b

a b a c a b

a b a c a b

1113



10

Another Boyer-Moore Example must

|

If you wish to understand others you must intensify ..

No Match and y is not in must, so move the whole word.

must

|


No Match and w is not in must, so move the whole word.

must

|

If you wish to understand others you must intensify ..No Match and is not in must, so move the whole word.



11

No Match and _ is not in must, so move the whole word.

Another Boyer-Moore Example must

|


No Match, but u is in must, so move the whole word only toright until u fits.

must

|If you wish to understand others you must intensify ..

Again: No Match and d is not in must, so move the whole word.

must

|||


From the right: No Match and r is not in must, so move the whole word.

must

|


From the right: No Match and n is not in must, so move the



12

From the right: No Match and n is not in must, so move the whole word.

Another Boyer-Moore ExampleThe rest goes this way:

must

must

must

must

must

|| | | |


Success!!

Try the Boyer-Moore algorithm with:

scan_Text: THISISASTRINGSEARCHEXAMPLE

pattern: HINGE



13

AnalysisBoyer-Moore’s algorithmruns in time O (nm + s )

Example of worst case:

T = aaa … a

P = baaa

The worst case may occur inimages and DNA sequencesbut is unlikely in English text

Boyer-Moore’s algorithm issignificantly faster than thebrute-force algorithm onEnglish text

11

1

a a a a a a a a a

23456

b a a a a a

b a a a a a

b a a a a a

b a a a a a

7891012

131415161718

192021222324



14

The KMP Algorithm - MotivationKnuth-Morris-Pratt’s algorithmcompares the pattern to thetext in left-to-right, but shifts

the pattern more intelligentlythan the brute-force algorithm.

When a mismatch occurs,what is the most we can shiftthe pattern so as to avoid

redundant comparisons? Answer: the largest prefix ofP [0.. j ] that is a suffix of P [1.. j ]

x

j

. . a b a a b . . . . .

a b a a b a

a b a a b a

No need tot th

Resume



15

repeat these

comparisons

comparing

here

KMP Failure FunctionKnuth-Morris-Pratt’salgorithm preprocesses thepattern to find matches ofprefixes of the pattern with

the pattern itself

The failure function F ( j ) is

defined as the size of thelargest prefix of P [0.. j ] that isalso a suffix of P [1.. j ]

Knuth-Morris-Pratt’salgorithm modifies the brute-force algorithm so that if amismatch occurs at P [ j ] T [i ]

j 0 1 2 3 4 5

P [ j ] a b a a b a

F ( j ) 0 0 1 1 2 3

x

j

. . a b a a b . . . . .

a b a a b a

a b a a b a



16

[j] [ ]

we set j F ( j - 1)F

( j -

1)

The KMP AlgorithmThe failure function can berepresented by an array andcan be computed in O (m ) time

At each iteration of the while-loop, either

i increases by one, or

the shift amount i - j

increases by at least one(observe that F ( j - 1) < j )

Hence, there are no morethan 2n iterations of the

while-loop

Thus, KMP’s algorithm runs in

Algorithm KMPMatch (T, P )

F failureFunction (P )i 0 j 0while i < m

if T [i ] = P [ j ]if j = m - 1

return i - j { match }else

i i + 1 j j + 1

elseif j > 0

j F [ j - 1]else

i i + 1t 1 { t h }



17

Thus, KMP s algorithm runs inoptimal time O (m + n )

return -1 { no match }

Computing the Failure

FunctionThe failure function can berepresented by an array andcan be computed in O (m ) time

The construction is similar tothe KMP algorithm itself

At each iteration of the while-loop, either

i increases by one, or

the shift amount i- j increases by at least one

(observe that F ( j - 1) < j )

Hence, there are no morethan 2m iterations of the

Algorithm failureFunction (P )

F [0] 0i 1 j 0while i < m

if P [i ] = P [ j ]{we have matched j + 1 chars}F [i ] j + 1i i + 1

j j + 1else if j > 0 then

{use failure function to shift P } j F [ j - 1]

elseF [i] 0 { no match }



18

a a o o

while-loopF [i ] 0 { no match }i i + 1

Example

1

a b a c a a b a c a b a c a b a a b b

7

8

19181715

a b a c a b

1614

13

2 3 4 5 6

9

a b a c a b

a b a c a b

a b a c a b

a b a c a b

10 11 12

c

j 0 1 2 3 4 5

P [ j ] a b a c a b



19

a b a c a b F ( j ) 0 0 1 0 1 2

TriePreprocessing the pattern speeds up pattern matchingqueries

After preprocessing the pattern, KMP’s algorithm performs

pattern matching in time proportional to the text sizeIf the text is large, immutable and searched for often(e.g., works by Shakespeare), we may want topreprocess the text instead of the pattern

A trie is a compact data structure for representing aset of strings, such as all the words in a text

A tries supports pattern matching queries in timeproportional to the pattern size



20

Trie (2)Given a string X, efficiently encode X into a smallerstring Y Each uncompressed character in X is 7-bit for ASCII and 16-

bit for UNICODE

A compressed character in Y has less bit

Saves memory and/or bandwidth

A good approach: Huffman encoding Compute frequency f(c) for each character c.

Encode high-frequency characters with short code words(Greedy method)

No code word is a prefix for another code

Use an optimal encoding tree to determine the code words



21

Why Trie?

N is no of string

L = length of string Above is the performance for Red-Black BST andHashing.

Can we do better than the above for string?



22

Yes.

Trie Implementation

Root = store nothing

Each node can have R children/keys :- 26 foralphabets

If node = last character (link it a value)

Else



23

Else

link it to null.

Trie: Search

Get word (“Shells”) . 3 is return.

Get word “she”. 0 is return.

Get word “shelter” there is no match after “shel” so

Null pointer(key word Shelter)



24

Get word “shelter”, there is no match after “shel”, so

a null is returned

Trie: Insertion

Put(“shore”, 7)



25

Trie: Deletion

delete(“shells”, 3)

All the character l, l, s and its pointer will have to be removed.



26

The removal stop when there is a non-null value character orwhen the is another sub-tree

Trie: Deletion

delete(“shore”, 7)Stop at (h).



27

Trie: Cost

R-way trie

Good when R is small

When R is large, too much memory is required.E.g. R-Way for english words, R = 26 , N = 10

Space = (26 +1) x 10 = 270 (still OK)

E.g. R-Way for UTF-16, R = 65536-way, N = 10



28

Space = (65536 + 1) x 10 = 655370 (too big).

Trie ConclusionTrade memory with speed

When you do a string search, you don’thave to compare the whole word

Quick search hit worst case is thelength of character

Quicker search miss, just check the 1st

few character, if miss then it’s notthere.

Note: Just one implementation is givenin this lecture other implementation



in this lecture, other implementation

may varies[Lec11] Pattern Matching 29

Other implementation. Words at Leaf

M

A

C

A

D Q R WCB

C

MACCBOY

MACAWMACACO

A

N O

OMACAQUE

M

MACABRE

MACARONI

MACADAMI

C

MACAROONI

MACADAMI

.

.

.

. .

.

.

.

.

MACCBOY MACAWMACARONMACARONIMACARONICMACAQUEMACADAMIAMACADAMMACACACO

MACABRE



[Lec11] Pattern Matching 30

MACARONIC

26 character implementation



[Lec11] Pattern Matching 31

Encoding Trie (1) A code is a mapping of each character of an alphabet to a binarycode-word

A prefix code is a binary code such that no code-word is the prefixof another code-word

An encoding trie represents a prefix code

Each leaf stores a character

The code word of a character is given by the path from the root tothe leaf storing the character (0 for a left child and 1 for a right child

a d e

00 010 011 10 11

a b c d e



32b c

Encoding Trie (2)Given a text string X , we want to find a prefix code for the charactersof X that yields a small encoding for X

Frequent characters should have short code-words

Rare characters should have long code-words

Example X = abracadabra

T 1 encodes X into 29 bits

T 2 encodes X into 24 bits

c d b a b r

T 1 T 2



33a r c d

Huffman’s AlgorithmGiven a string X ,Huffman’s algorithmconstruct a prefixcode the minimizes

the size of theencoding of X

It runs in timeO (n + d log d ), wheren is the size of X and d is the numberof distinct charactersof X

A heap-basedpriority queue isused as an auxiliary

Algorithm HuffmanEncoding (X )

Input string X of size n

Output optimal encoding trie for X

C distinctCharacters (X )

computeFrequencies (C, X )

Q new empty heap

for all c C

T new single-node tree storing c

Q.insert (getFrequency (c ), T )

while Q.size () > 1

f 1

Q.minKey ()T 1 Q.removeM in ()

f 2 Q.minKey ()

T 2 Q.removeM in ()

T join (T 1 , T 2 )

Q i t(f + f T)



34

used as an auxiliary

structure

Q.insert (f 1 + f 2 , T )

return Q.removeM in ()

Example

a b c d r

5 2 1 1 2

X = abracadabra

Frequencies

ca rdb

5 2 1 1 2

db

2

bd

2 4

ca bd r

2

5

4

6

c

a

bd r

2 4

6

11



35

ca rdb

5 2 2

ca bd r

5

Huffman CodingString

Frequency

A= 20

B = 18

C = 15D = 12

E = 8

F -= 6

G = 4

H = 2I = 1

J = 1



36

Lecture 10: NP-Completeness

x1 x3 x2 x1 x4 x3 x2 x4

11

12

13 21

22

23 31

32

33



1

Learning OutcomeP and NP (13.1) Definition of P

Definition of NP

Alternate definition of NP

NP-completeness (13.2)

Definition of NP-hard and NP-complete The Cook-Levin Theorem



2

Running Time Revisited

Input size, n To be exact, let n denote the number of bits in a nonunary

(binary) encoding of the input A time bound of the form O(nk ), for some fixed k, is called

polynomial time. All the polynomial-time algorithms studied so far in this course runin polynomial time using this definition of input size. Exception: any pseudo-polynomial time algorithm

ORD PVDSFO

LAX

LGA

HNL



3

MIA DFW

LAX

Dealing with Hard ProblemsWhat to do when we find a problemthat looks hard…

I couldn’t find a polynomial-time algorithm;



4

I couldn t find a polynomial time algorithm;

I guess I’m too dumb.(cartoon inspired by [Garey-Johnson, 79])

Dealing with Hard ProblemsSometimes we can prove a strong lowerbound… (but not usually)

I couldn’t find a polynomial-time algorithm,



5

cou d t d a po y o a t e a go t ,

because no such algorithm exists!(cartoon inspired by [Garey-Johnson, 79])

Dealing with Hard ProblemsNP-completeness let’s us showcollectively that a problem is hard.

I couldn’t find a polynomial-time algorithm



6

I couldn t find a polynomial-time algorithm,

but neither could all these other smart people.(cartoon inspired by [Garey-Johnson, 79])

Polynomial-Time

Decision ProblemsTo simplify the notion of “hardness,” we willfocus on the following:

Polynomial-time as the cut-off for efficiency

Decision problems: output is 1 or 0 (“yes” or “no”)

Examples:

Does a given graph G have an Euler tour?

Does a text T contain a pattern P? Does an instance of 0/1 Knapsack have a solution with

benefit at least K?

Does a graph G have an MST with weight at most K?



7

Problems and Languages A language L is a set of strings defined over somealphabet Σ

Every decision algorithm A defines a language L L is the set consisting of every string x such that A outputs

“yes” on input x.

We say “A accepts x’’ in this case

Example:

If A determines whether or not a given graph G has anEuler tour, then the language L for A is all graphs withEuler tours.



8

The Complexity Class P

A complexity class is a collection of languages

P is the complexity class consisting of all languagesthat are accepted by polynomial-time algorithms

For each language L in P there is a polynomial-timedecision algorithm A for L.

If n=|x|, for x in L, then A runs in p(n) time on input x. The function p(n) is some polynomial



9

The Complexity Class NP

We say that an algorithm is non-deterministic if ituses the following operation:

Choose(b): chooses a bit b

Can be used to choose an entire string y (with |y| choices)We say that a non-deterministic algorithm A acceptsa string x if there exists some sequence of chooseoperations that causes A to output “yes” on input x.

NP is the complexity class consisting of all languagesaccepted by polynomial-time non-deterministicalgorithms.



10

NP example

Problem: Decide if a graph has an MST of weight K

Algorithm:

1. Non-deterministically choose a set T of n-1 edges

2. Test that T forms a spanning tree

3. Test that T has weight at most K

Analysis: Testing takes O(n+m) time, so thisalgorithm runs in polynomial time.



11

The Complexity Class NP

Alternate DefinitionWe say that an algorithm B verifies the acceptanceof a language L if and only if, for any x in L, there

exists a certificate y such that B outputs “yes” oninput (x,y).

NP is the complexity class consisting of all languagesverified by polynomial-time algorithms.

We know: P is a subset of NP.

Major open question: P=NP?

Most researchers believe that P and NP are different.



12

NP example (2)

Problem: Decide if a graph has an MST of weight K

Verification Algorithm:

1. Use as a certificate, y, a set T of n-1 edges

2. Test that T forms a spanning tree

3. Test that T has weight at most K

Analysis: Verification takes O(n+m) time, so thisalgorithm runs in polynomial time.



13

Equivalence of the

Two DefinitionsSuppose A is a non-deterministic algorithm

Let y be a certificate consisting of all the outcomes of thechoose steps that A uses

We can create a verification algorithm that uses y instead of A’s choose steps

If A accepts on x, then there is a certificate y that allows us toverify this (namely, the choose steps A made)

If A runs in polynomial-time, so does this verification

algorithmSuppose B is a verification algorithm

Non-deterministically choose a certificate y

Run B on y

If B runs in polynomial time so does this non deterministic



14

If B runs in polynomial-time, so does this non-deterministic

algorithm

An Interesting Problem

NOT

OR

AND

Logic Gates:

Inputs:

0

1

0

1

1

1

1

1

Output:

0

1

00 1

A Boolean circuit is a circuit of AND, OR, and NOTgates; the CIRCUIT-SAT problem is to determine ifthere is an assignment of 0’s and 1’s to a circuit’s

inputs so that the circuit outputs 1.



15

CIRCUIT-SAT is in NP

NOT

OR

AND

Logic Gates:

Inputs:

0

1

0

1

11

1

1

Output:

0

1

00 1

Non-deterministically choose a set of inputs and theoutcome of every gate, then test each gate’s I/O.



16

NP-Completeness A problem (language) L is NP-hard if everyproblem in NP can be reduced to L inpolynomial time.

That is, for each language M in NP, we cantake an input x for M, transform it inpolynomial time to an input x’ for L such thatx is in M if and only if x’ is in L.

L is NP-complete if it’s in NP and is NP-hard.

NP poly-time L



17

p y

Cook-Levin TheoremCIRCUIT-SAT is NP-complete.

We already showed it is in NP.

To prove it is NP-hard, we have to show that every

language in NP can be reduced to it. Let M be in NP, and let x be an input for M.

Let y be a certificate that allows us to verify membership in M inpolynomial time, p(n), by some algorithm D.

Let S be a circuit of size at most O(p(n)2) that simulates a

computer (details omitted…)

NP poly-time CIRCUIT-SATM



18

p y

Some Thoughts

about P and NP

Belief: P is a proper subset of NP.

Implication: the NP-complete problems are the hardest in NP.

Why: Because if we could solve an NP-complete problem inpolynomial time, we could solve every problem in NP in polynomialtime.

That is, if an NP-complete problem is solvable in polynomial time,

then P=NP.Since so many people have attempted without success to findpolynomial-time solutions to NP-complete problems, showing yourproblem is NP-complete is equivalent to showing that a lot of smartpeople have worked on your problem and found no polynomial-

NP P

CIRCUIT-SAT

NP-completeproblems live here



19time algorithm.

NP-Completeness (2)

x1 x3 x2 x1 x4 x3 x2 x4

11

12

13 21

22

23 31

32

33



20

Outline and Reading

Definitions (13.1-2)

NP is the set of all problems (languages) that can be

accepted non-deterministically (using “choose”operations) in polynomial time.

verified in polynomial-time given a certificate y.

Some NP-complete problems (13.3)

Problem reduction SAT (and CNF-SAT and 3SAT)

Vertex Cover

Hamiltonian Cycle



21

Problem Reduction A language M is polynomial-time reducible to alanguage L if an instance x for M can be transformed inpolynomial time to an instance x’ for L such that x is in Mif and only if x’ is in L. Denote this by ML.

A problem (language) L is NP-hard if every problem inNP is polynomial-time reducible to L.

A problem (language) is NP-complete if it is in NP and

it is NP-hard.CIRCUIT-SAT is NP-complete: CIRCUIT-SAT is in NP

For every M in NP, M CIRCUIT-SAT.

Inputs:0

1

0

1

11

1

1

Output:

0

1

00 1



22

1 0 1

Transitivity of ReducibilityIf A B and B C, then A C.

An input x for A can be converted to x’ for B, such that x is in Aif and only if x’ is in B. Likewise, for B to C.

Convert x’ into x’’ for C such that x’ is in B if x’’ is in C.

Hence, if x is in A, x’ is in B, and x’’ is in C.

Likewise, if x’’ is in C, x’ is in B, and x is in A.

Thus, A C, since polynomials are closed under composition.

Types of reductions:

Local replacement: Show A B by dividing an input to Ainto components and show how each component can beconverted to a component for B.

Component design: Show A B by building specialcomponents for an input of B that enforce properties needed



23

components for an input of B that enforce properties needed

for A, such as “choice” or “evaluate.”

SAT

A Boolean formula is a formula where thevariables and operations are Boolean (0/1):

(a+b+¬d+e)(¬a+¬c)(¬b+c+d+e)(a+¬c+¬e) OR: +, AND: (times), NOT: ¬

SAT: Given a Boolean formula S, is Ssatisfiable, that is, can we assign 0’s and 1’s

to the variables so that S is 1 (“true”)? Easy to see that CNF-SAT is in NP:

Non-deterministically choose an assignment of 0’s and1’s to the variables and then evaluate each clause. If



24they are all 1 (“true”), then the formula is satisfiable.

SAT is NP-completeReduce CIRCUIT-SAT to SAT.

Given a Boolean circuit, make a variable for every inputand gate.

Create a sub-formula for each gate, characterizing itseffect. Form the formula as the output variable AND-edwith all these sub-formulas:

Example:m((a+b)↔e)(c↔¬f)(d↔¬g)(e↔¬h)(ef ↔i)…Inputs:

a b

c

e

f i

m

Output:

h

k

g

The formula is satisfiableif and only if theBoolean circuitis satisfiable.



25

d

g

j n

3SAT

The SAT problem is still NP-complete even if the formula is aconjunction of disjuncts, that is, it is in conjunctive normal form(CNF).The SAT problem is still NP-complete even if it is in CNF andevery clause has just 3 literals (a variable or its negation): (a+b+¬d)(¬a+¬c+e)(¬b+d+e)(a+¬c+¬e)

Reduction from SATDNF (disjunctive normal form):

_ _ _ _ _

B=a.b.c+a.b.c+a.b.c+a.b.cCNF (conjunctive normal form):

_ _ _ _ _ _ _ B=(a+b+c).(a+b+c).(a+b+c).(a+b+c)



26

Illustration of Reductions

Every problem in NP

CIRCUIT-SAT

CNF-SAT

comp. design

local rep.

3SAT

local rep.

VERTEX-COVER

comp. design

CLIQUE SET-COVER SUBSET-SUM HAMILTONIAN-

CYCLE

local rep.

local rep. comp. design comp. design

restriction restriction

Cook-Levin TheoremCIRCUIT-SAT is NP-complete



27

KNAPSACK TSP

Vertex Cover

A vertex cover of graph G=(V,E) is a subset W of V, suchthat, for every edge (a,b) in E, a is in W or b is in W.

VERTEX-COVER: Given an graph G and an integer K,does G have a vertex cover of size at most K?

VERTEX-COVER is in NP: Non-deterministically choose a



28subset W of size K and check that every edge is coveredby W.

Vertex Cover

Example

Suppose we are given a graph G representing acomputer network where the vertices represent routersand edges represent physical connections. Supposefurther that we wish to upgrade some of the routers inour network with special new, but expensive, routersthat can perform sophisticated monitoring operations for

incident connections. If we would like to determine if knew routers are sufficient to monitor every connection inour network, then we have an instance of VERTEX-COVER problem.



29

Vertex-Cover is NP-completeReduce 3SAT to VERTEX-COVER.

Let S be a Boolean formula in CNF with each clausehaving 3 literals.

For each variable x, create a node for x and ¬x,and connect these two:

For each clause (a+b+c), create a triangle and

connect these three nodes.

x ¬x

a



30c b

Vertex-Cover is NP-completeCompleting the construction

Connect each literal in a clause triangle to its copyin a variable pair.

E.g., a clause (¬x+y+z)

Let n=# of variables

Let m=# of clauses

Set K=n+2m

y ¬y

a

¬xx z ¬z



31c b

Vertex-Cover is NP-complete

¬d d

12 22 32

Example: (a+b+c)(¬a+b+¬c)(¬b+¬c+¬d)

Graph has vertex cover of size K=4+6=10 if formula issatisfiable.

¬cc¬aa ¬bb



3211 13 21 23 31 33

Some Other

NP-Complete ProblemsSUBSET-SUM: Given a set of integers and adistinguished integer K, is there a subset of

the integers that sums to K? NP-complete by reduction from VERTEX-COVER



33

SUBSET-SUM

SUBSET-SUM: Given a set S of n integers and a distinguished

integer k, is there a subset of the integers in S that

sum to k?

Example: Suppose we have an internet web server, and we arepresented with a collection of download requests. For each downloadrequest we can easily determine the size of the requested file. Thus,we can abstract each web request simply as an integer - the size ofthe requested file. Given this set of integers, we might be interested

in determining a subset of them that exactly sums to the bandwidthour server can accommodate in one minute. Unfortunately, thisproblem is an instance of SUBSET-SUM. Moreover, because it is NP-Complete, this problem will actually harder to solve as our webserver’s bandwidth and request-handling ability improves.



34

Some Other

Documents

Algorithm Design & Analysis Full