CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1

CSCE 411Design and Analysis

of Algorithms

Background ReviewProf. Jennifer Welch

CSCE 411, Background Review 1

Outline Algorithm Analysis Tools Sorting Graph Algorithms


Algorithm Analysis Tools Big-Oh notation Basic probability


How to Measure Efficiency Machine-independent way:

analyze "pseudocode" version of algorithm assume idealized machine model

one instruction takes one time unit "Big-Oh" notation

order of magnitude as problem size increases Worst-case analyses

safe, often occurs most often, average case often just as bad


Faster Algorithm vs. Faster CPU A faster algorithm running on a slower machine will

always win for large enough instances Suppose algorithm S1 sorts n keys in 2n2 instructions Suppose computer C1 executes 1 billion instruc/sec

When n = 1 million, takes 2000 sec Suppose algorithm S2 sorts n keys in 50nlog2n

instructions Suppose computer C2 executes 10 million instruc/sec

When n = 1 million, takes 100 sec


Caveat No point in finding fastest algorithm

for part of the program that is not the bottleneck

If program will only be run a few times, or time is not an issue (e.g., run overnight), then no point in finding fastest algorithm


Review of Big-Oh Notation Measure time as a function of input size

and ignore multiplicative constants lower order terms

f(n) = O(g(n)) : f is “≤” g f(n) = Ω(g(n)) : f is “≥” g f(n) = (g(n)) : f is “=” g f(n) = o(g(n)) : f is “<” g f(n) = (g(n)) : f is “>” g


Big-Oh f(n) = O(g(n)) means, informally, that g is an

upper bound on f Formally: there exist c > 0, n0 > 0 s.t.

f(n) ≤ cg(n) for all n ≥ n0. To show that f(n) = O(g(n)), either find c and

n0 that satisfy the definition, or use this fact: if limn->∞ f(n)/g(n) = some nonnegative constant,

then f(n) = O(g(n)).


Big-Omega f(n) = Ω(g(n)) means, informally, that g is an

lower bound on f Formally: there exist c > 0, n0 > 0 s.t.

f(n) ≥ cg(n) for all n ≥ n0. To show that f(n) = Ω(g(n)), either find c and

n0 that satisfy the definition, or use this fact: if limn->∞ f(n)/g(n) = some positive constant or ∞,

then f(n) = Ω(g(n)).


Big-Theta f(n) = Θ(g(n)) means, informally, that g is a

tight bound on f Formally: there exist c1 > 0, c2 > 0, n0 > 0 s.t.

c1g(n) ≤ f(n) ≤ c2g(n) for all n ≥ n0.

To show that f(n) = Θ(g(n)), either find c1, c2, n0 that satisfy the definition, or use this fact: if limn->∞ f(n)/g(n) = some positive constant,

then f(n) = Θ(g(n)).


Tying Back to Algorithm Analysis Algorithm has worst case running time O(g(n))

if there exist c > 0 and n0 > 0 s.t. for all n ≥ n0, every execution (including the slowest) on an input of size n takes at most cg(n) time.

Algorithm has worst case running time Ω(g(n)) if there exist c > 0 and n0 > 0 s.t. for all n ≥ n0, at least one execution (including the slowest) on an input of size n takes at least cg(n) time.


More Definitions f(n) = o(g(n)) means, informally, g is a

non-tight upper bound on f limn->∞ f(n)/g(n) = 0 pronounced “little-oh”

f(n) = ω(g(n)) means, informally, g is a non-tight lower bound on f limn->∞ f(n)/g(n) = ∞ pronounced “little-omega”


CSCE 411, Spring 2012: Set 3 13

How To Solve Recurrences Ad hoc method:

expand several times guess the pattern can verify with proof by induction

Master theorem general formula that works if recurrence has the form

T(n) = aT(n/b) + f(n) a is number of subproblems n/b is size of each subproblem f(n) is cost of non-recursive part


Probability Every probabilistic claim ultimately refers to

some sample space, which is a set of elementary events

Think of each elementary event as the outcome of some experiment Ex: flipping two coins gives sample space

HH, HT, TH, TT

An event is a subset of the sample space Ex: event "both coins flipped the same" is HH, TT


Sample Spaces and Events

S

A

HH THTT

HT


Probability Distribution A probability distribution Pr on a

sample space S is a function from events of S to real numbers s.t. Pr[A] ≥ 0 for every event A Pr[S] = 1 Pr[A U B] = Pr[A] + Pr[B] for every two

non- intersecting ("mutually exclusive") events A and B

Pr[A] is the probability of event A


Probability DistributionUseful facts: Pr[Ø] = 0 If A B, then Pr[A] ≤ Pr[B] Pr[S — A] = 1 — Pr[A] // complement Pr[A U B] = Pr[A] + Pr[B] – Pr[A B] ≤ Pr[A] + Pr[B]


Probability Distribution

A

B

Pr[A U B] = Pr[A] + Pr[B] – Pr[A B]


Example Suppose Pr[HH] = Pr[HT] = Pr[TH] =

Pr[TT] = 1/4. Pr["at least one head"]

= Pr[HH U HT U TH] = Pr[HH] + Pr[HT] + Pr[TH] = 3/4. Pr["less than one head"]

= 1 — Pr["at least one head"] = 1 — 3/4 = 1/4

HH THTT

HT1/4

1/4

1/4

1/4


Specific Probability Distribution discrete probability distribution: sample

space is finite or countably infinite Ex: flipping two coins once; flipping one coin

infinitely often uniform probability distribution: sample

space S is finite and every elementary event has the same probability, 1/|S| Ex: flipping two fair coins once


Flipping a Fair Coin Suppose we flip a fair coin n times Each elementary event in the sample space is

one sequence of n heads and tails, describing the outcome of one "experiment"

The size of the sample space is 2n. Let A be the event "k heads and nk tails occur". Pr[A] = C(n,k)/2n.

There are C(n,k) sequences of length n in which k heads and n–k tails occur, and each has probability 1/2n.


Example n = 5, k = 3 Event A is

HHHTT, HHTTH, HTTHH, TTHHH,HHTHT, HTHTH, THTHH,HTHHT, THHTH, THHHT

Pr[3 heads and 2 tails] = C(5,3)/25

= 10/32


Flipping Unfair Coins Suppose we flip two coins, each of

which gives heads two-thirds of the time

What is the probability distribution on the sample space?

HH THTT

HT4/9

2/9

2/9

1/9

Pr[at least one head] = 8/9


Independent Events Two events A and B are independent if

Pr[A B] = Pr[A]·Pr[B] I.e., probability that both A and B

occur is the product of the separate probabilities that A occurs and that B occurs.


Independent Events ExampleIn two-coin-flip example with fair coins: A = "first coin is heads" B = "coins are different"

HH THTT

HT1/4

1/4

1/4

1/4

A B Pr[A] = 1/2Pr[B] = 1/2Pr[A B] = 1/4 = (1/2)(1/2)so A and B are independent


Discrete Random Variables A discrete random variable X is a function

from a finite or countably infinite sample space to the real numbers.

Associates a real number with each possible outcome of an experiment

Define the event "X = v" to be the set of all the elementary events s in the sample space with X(s) = v.

Pr["X = v"] is the sum of Pr[s] over all s with X(s) = v.


Discrete Random Variable

X=v

X=v

X=v

X=vX=v

Add up the probabilities of all the elementary events inthe orange event to get the probability that X = v


Random Variable Example Roll two fair 6-sided dice. Sample space contains 36 elementary

events (1:1, 1:2, 1:3, 1:4, 1:5, 1:6, 2:1,…) Probability of each elementary event is 1/36 Define random variable X to be the

maximum of the two values rolled What is Pr["X = 3"]? It is 5/36, since there are 5 elementary

events with max value 3 (1:3, 2:3, 3:3, 3:2, and 3:1)


Independent Random Variables It is common for more than one random

variable to be defined on the same sample space. E.g.: X is maximum value rolled Y is sum of the two values rolled

Two random variables X and Y are independent if for all v and w, the events "X = v" and "Y = w" are independent.


Expected Value of a Random Variable Most common summary of a random

variable is its "average", weighted by the probabilities called expected value, or expectation, or

mean

Definition: E[X] = ∑ v Pr[X = v]

v


Expected Value Example Consider a game in which you flip two fair coins. You get $3 for each head but lose $2 for each tail. What are your expected earnings? I.e., what is the expected value of the random

variable X, where X(HH) = 6, X(HT) = X(TH) = 1, and X(TT) = —4?

Note that no value other than 6, 1, and —4 can be taken on by X (e.g., Pr[X = 5] = 0).

E[X] = 6(1/4) + 1(1/4) + 1(1/4) + (—4)(1/4) = 1


Properties of Expected Values E[X+Y] = E[X] + E[Y], for any two

random variables X and Y, even if they are not independent!

E[a·X] = a·E[X], for any random variable X and any constant a.

E[X·Y] = E[X]·E[Y], for any two independent random variables X and Y


Conditional Probability Formalizes having partial knowledge about

the outcome of an experiment Example: flip two fair coins.

Probability of two heads is 1/4 Probability of two heads when you already know

that the first coin is a head is 1/2

Conditional probability of A given that B occurs, denoted Pr[A|B], is defined to be

Pr[AB]/Pr[B]


Conditional Probability

A

B

Pr[A] = 5/12Pr[B] = 7/12Pr[AB] = 2/12Pr[A|B] = (2/12)/(7/12) = 2/7


Conditional Probability

Definition is Pr[A|B] = Pr[AB]/Pr[B]

Equivalently, Pr[AB] = Pr[A|B]·Pr[B]

Sorting Insertion Sort Heapsort Mergesort Quicksort



Insertion Sort Review How it works:

incrementally build up longer and longer prefix of the array of keys that is in sorted order

take the current key, find correct place in sorted prefix, and shift to make room to insert it

Finding the correct place relies on comparing current key to keys in sorted prefix

Worst-case running time is (n2)


Heapsort Review How it works:

put the keys in a heap data structure repeatedly remove the min from the heap

Manipulating the heap involves comparing keys to each other

Worst-case running time is (n log n)


Mergesort Review How it works:

split the array of keys in half recursively sort the two halves merge the two sorted halves

Merging the two sorted halves involves comparing keys to each other

Worst-case running time is (n log n)


Mergesort Example

1 32 42 5 66

2 64 5 1 2 63

5 2 64 1 3 62

5 2 64

2 5 64

1 3

1 3

62

62

5 62 4

5 62 14 3 62

1 3 62


Recurrence Relation for Mergesort Let T(n) be worst case time on a sequence

of n keys If n = 1, then T(n) = (1) (constant) If n > 1, then T(n) = 2 T(n/2) + (n)

two subproblems of size n/2 each that are solved recursively

(n) time to do the merge Solving recurrence gives T(n) = (n log

n)


Quicksort Review How it works:

choose one key to be the pivot partition the array of keys into those keys

< the pivot and those ≥ the pivot recursively sort the two partitions

Partitioning the array involves comparing keys to the pivot

Worst-case running time is (n2)

Graphs Mathematical definitions Representations

adjacency list, adjacency matrix

Traversals breadth-first search, depth-first search

Minimum spanning trees Kruskal’s algorithm, Prim’s algorithm

Single-source shortest paths Dijkstra’s algorithm, Bellman-Ford algorithm


Graphs Directed graph G = (V,E)

V is a finite set of vertices E is the edge set, a binary relation on E (ordered pairs of

vertice) In an undirected graph, edges are unordered pairs of

vertices If (u,v) is in E, then v is adjacent to u and (u,v) is

incident from u and incident to v; v is neighbor of u in an undirected graph, u is also adjacent to v, and (u,v) is

incident on u and v Degree of a vertex is number of edges incident from

or to (or on) the vertex


More on Graphs A path of length k from vertex u to vertex u’ is

a sequence of k+1 vertices s.t. there is an edge from each vertex to the next in the sequence In this case, u’ is reachable from u

A path is simple if no vertices are repeated A path forms a cycle if last vertex = first vertex A cycle is simple if no vertices are repeated

other than first and last


Graph Representations So far, we have discussed graphs purely as

mathematical concepts How can we represent a graph in a

computer program? We need some data structures.

Two most common representations are: adjacency list – best for sparse graphs (few edges) adjacency matrix – best for dense graphs (lots of

edges)


Adjacency List Representation

Given graph G = (V,E) array Adj of |V| linked lists, one for each vertex in V The adjacency list for vertex u, Adj[u], contains all

vertices v s.t. (u,v) is in E each vertex in the list might be just a string,

representing its name, or it might be some other data structure representing the vertex, or it might be a pointer to the data structure for that vertex

Requires Θ(V+E) space (memory) Requires Θ(degree(u)) time to check if (u,v) is in E



Adjacency List Representation

a

c d

b

e

a

b

c

d

e

b c

a d e

a d

b c e

b d

Space-efficient for sparse graphs

Adjacency Matrix Representation

Given graph G = (V,E) Number the vertices 1, 2, ..., |V| Use a |V| x |V| matrix A with (i,j)-entry of A

being 1 if (i,j) is in E 0 if (i,j) is not in E

Requires Θ(V2) space (memory) Requires Θ(1) time to check if (u,v) is in E



Adjacency Matrix Representation

a

c d

b

e

a

b

c

d

e

Check for an edge in constant time

a b c d e0 1 1 0 0

1 0 0 1 1

1 0 0 1 0

0 1 1 0 1

0 1 0 1 0

More on Graph Representations Read Ch 22, Sec 1 for

more about directed vs. undirected graphs how to represented weighted graphs

(some value is associated with each edge) pros and cons of adjacency list vs.

adjacency matrix



Graph Traversals Ways to traverse/search a graph

Visit every vertex exactly once

Breadth-First Search Depth-First Search


Breadth First Search (BFS) Input: G = (V,E) and source s in V for each vertex v in V do

mark v as unvisited mark s as visited enq(Q,s) // FIFO queue Q while Q is not empty do

u := deq(Q) for each unvisited neighbor v of u do

mark v as visited enq(Q,v)


BFS Example start at s and consider in alphabetical

order

a

c

d

b

s


BFS Tree We can make a spanning tree rooted

at s by remembering the "parent" of each vertex


Breadth First Search #2 Input: G = (V,E) and source s in V for each vertex v in V do

mark v as unvisited parent[v] := nil

mark s as visited parent[s] := s enq(Q,s) // FIFO queue Q


Breadth First Search #2 while Q is not empty do


mark v as visited parent[v] := u enq(Q,v)


BFS Tree Example

a

c

d

b

s


BFS Trees BFS tree is not necessarily unique for

a given graph Depends on the order in which

neighboring vertices are processed


BFS Numbering During the breadth-first search, assign

an integer to each vertex Indicate the distance of each vertex

from the source s


Breadth First Search #3 Input: G = (V,E) and source s in V for each vertex v in V do

mark v as unvisited parent[v] := nil d[v] := infinity

mark s as visited parent[s] := s d[s] := 0 enq(Q,s) // FIFO queue Q


Breadth First Search #3 while Q is not empty do


mark v as visited parent[v] := u d[v] := d[u] + 1 enq(Q,v)


BFS Numbering Example

a

c

d

b

sd = 0

d = 1

d = 1

d = 2

d = 2


Shortest Path Tree Theorem: BFS algorithm

visits all and only vertices reachable from s sets d[v] equal to the shortest path

distance from s to v, for all vertices v, and sets parent variables to form a shortest

path tree


BFS Running Time Initialization of each vertex takes O(V) time Every vertex is enqueued once and

dequeued once, taking O(V) time When a vertex is dequeued, all its

neighbors are checked to see if they are unvisited, taking time proportional to number of neighbors of the vertex, and summing to O(E) over all iterations

Total time is O(V+E)


Depth-First Search Input: G = (V,E) for each vertex u in V do

mark u as unvisited

for each unvisited vertex u in V do call recursiveDFS(u)

recursiveDFS(u): mark u as visited for each unvisited neighbor v of u do

call recursiveDFS(v)


DFS Example start at a and consider in alphabetical

order a

c

d

b

e

f g h


Disconnected Graphs What if graph is disconnected or is

directed? call DFS on several vertices to visit all

vertices purpose of second for-loop in non-recursive

wrapper a

cb

d

e


DFS Tree Actually might be a DFS forest

(collection of trees) Keep track of parents


Depth-First Search #2 Input: G = (V,E) for each vertex u in

V do mark u as unvisited parent[u] := nil

for each unvisited vertex u in V do parent[u] := u // a

root call recursive DFS(u)

recursiveDFS(u): mark u as visited for each unvisited

neighbor v of u do parent[v] := u call recursiveDFS(v)


More Properties of DFS Need to keep track of more

information for each vertex: discovery time: when its recursive

call starts finish time: when its recursive call

ends


Depth-First Search #3 Input: G = (V,E) for each vertex u in V do

mark u as unvisited parent[u] := nil

time := 0 for each unvisited

vertex u in V do parent[u] := u // a root call recursive DFS(u)

recursiveDFS(u): mark u as visited time++ disc[u] := time for each unvisited

neighbor v of u do parent[v] := u call recursiveDFS(v)

time++ fin[u] := time


Running Time of DFS initialization takes O(V) time second for loop in non-recursive wrapper

considers each vertex, so O(V) iterations one recursive call is made for each vertex in recursive call for vertex u, all its

neighbors are checked; total time in all recursive calls is O(E)

Total time is O(V+E)


Minimum Spanning Tree

7

1645

6 8

11

15

14

17

10

13

3

12

29

18

Given a connected undirected graph with edge weights,find subset of edges that spans all the nodes,creates no cycle, and minimizes sum of weights


Facts About MSTs There can be many spanning trees of

a graph In fact, there can be many minimum

spanning trees of a graph But if every edge has a unique

weight, then there is a unique MST


Generic MST Algorithm input: weighted undirected graph

G = (V,E,w) T := empty set while T is not yet a spanning tree of G

find an edge e in E s.t. T U e is a subgraph of some MST of G

add e to T return T (as MST of G)


Kruskal's MST algorithm

7

1645

6 8

11

15

14

17

10

13

3

12

29

18

consider the edges in increasing order of weight,add in an edge iff it does not cause a cycle


Implementing Kruskal's Alg. Sort edges by weight

efficient algorithms known How to test quickly if adding in the

next edge would cause a cycle? use disjoint set data structure, later


Kruskal's Algorithm as a Special Case of Generic Algorithm Consider edges in increasing order of

weight Add the next edge iff it doesn't cause

a cycle At any point, T is a forest (set of

trees); eventually T is a single tree


Another MST Algorithm Kruskal's algorithm maintains a forest

that grows until it forms a spanning tree Alternative idea is keep just one tree and

grow it until it spans all the nodes Prim's algorithm

At each iteration, choose the minimum weight outgoing edge to add greedy!


Idea of Prim's Algorithm Instead of growing the MST as possibly

multiple trees that eventually all merge, grow the MST from a single vertex, so that there is only one tree at any point.

Also a special case of the generic algorithm: at each step, add the minimum weight edge that goes out from the tree constructed so far.


Prim's Algorithm input: weighted undirected graph G = (V,E,w) T := empty set S := any vertex in V while |T| < |V| - 1 do

let (u,v) be a min wt. outgoing edge (u in S, v not in S) add (u,v) to T add v to S

return (S,T) (as MST of G)


Prim's Algorithm Example

a

b

h

i

c

g

d

f

e

4

8 7

8

11

2

7 6

1 2

4 14

9

10


Implementing Prim's Algorithm How do we find minimum weight

outgoing edge? First cut: scan all adjacency lists at

each iteration. Results in O(VE) time. Try to do better.


Implementing Prim's Algorithm Idea: have each vertex not yet in the

tree keep track of its best (cheapest) edge to the tree constructed so far.

To find min wt. outgoing edge, find minimum among these values use a priority queue to store the best edge

info (insert and extract-min operations)


Implementing Prim's Algorithm When a vertex v is added to T, some

other vertices might have their best edges affected, but only neighbors of v add decrease-key operation to the priority

queueu

vTi

Ti+1 w

w's best edge to Ti

check if this edge is cheaper for w

x

x's best edge to Ti

v's best edge to Ti


Details on Prim's AlgorithmAssociate with each vertex v two fields: best-wt[v] : if v is not yet in the tree, then

it holds the min. wt. of all edges from v to a vertex in the tree. Initially infinity.

best-node[v] : if v is not yet in the tree, then it holds the name of the vertex (node) u in the tree s.t. w(v,u) is v's best-wt. Initially nil.


Details on Prim's Algorithm input: G = (V,E,w)// initialization initialize priority queue Q to contain all

vertices, using best-wt values as keys let v0 be any vertex in V decrease-key(Q,v0,0)

// last line means change best-wt[v0] to 0 and adjust Q accordingly


Details on Prim's Algorithm while Q is not empty do

u := extract-min(Q) // vertex w/ smallest best-wt

if u is not v0 then add (u,best-node[u]) to T for each neighbor v of u do

if v is in Q and w(u,v) < best-wt[v] then best-node[v] := u decrease-key(Q,v,w(u,v))

return (V,T) // as MST of G


Running Time of Prim's AlgorithmDepends on priority queue implementation. Let Tins be time for insert Tdec be time for decrease-key Tex be time for extract-minThen we have |V| inserts and one decrease-key in the

initialization: O(VTins+Tdec) |V| iterations of while

one extract-min per iteration: O(VTex) total


Running Time of Prim's Algorithm Each iteration of while includes a for

loop. Number of iterations of for loop varies,

depending on how many neighbors the current vertex has

Total number of iterations of for loop is O(E).

Each iteration of for loop: one decrease key, so O(ETdec) total


Running Time of Prim's Algorithm O(V(Tins + Tex) + ETdec) If priority queue is implemented with

a binary heap, then Tins = Tex = Tdec = O(log V) total time is O(E log V)

(Think about how to implement decrease-key in O(log V) time.)


Shortest Paths in a Graph Let's review two important single-

source shortest path algorithms: Dijkstra's algorithm Bellman-Ford algorithm


Single Source Shortest Path Problem Given: directed or undirected graph G

= (V,E,w) and source vertex s in V Find: For each t in V, a path in G from

s to t with minimum weight Warning! Negative weights are a

problem:

s t4

5


Shortest Path Tree Result of a SSSP algorithm can be

viewed as a tree rooted at the source Why not use breadth-first search? Works fine if all weights are the same:

weight of each path is (a multiple of) the number of edges in the path

Doesn't work when weights are different


Dijkstra's SSSP Algorithm Assumes all edge weights are nonnegative Similar to Prim's MST algorithm Start with source vertex s and iteratively

construct a tree rooted at s Each vertex keeps track of tree vertex that

provides cheapest path from s (not just cheapest path from any tree vertex)

At each iteration, include the vertex whose cheapest path from s is the overall cheapest


Prim's vs. Dijkstra's

s

5

4

1

6

Prim's MST

s

5

4

1

6

Dijkstra's SSSP


Implementing Dijkstra's Alg. How can each vertex u keep track of its

best path from s? Keep an estimate, d[u], of shortest path

distance from s to u Use d as a key in a priority queue When u is added to the tree, check each of

u's neighbors v to see if u provides v with a cheaper path from s: compare d[v] to d[u] + w(u,v)


Dijkstra's Algorithm input: G = (V,E,w) and source vertex

s// initialization d[s] := 0 d[v] := infinity for all other vertices v initialize priority queue Q to contain

all vertices using d values as keys


Dijkstra's Algorithm while Q is not empty do

u := extract-min(Q) for each neighbor v of u do

if d[u] + w(u,v) < d[v] thend[v] := d[u] + w(u,v)decrease-key(Q,v,d[v])parent(v) := u


Dijkstra's Algorithm Example

a b

d e

c

2

84 9

2

4

12

10 6 3

source is vertex a

0 1 2 3 4 5Q abcd

ebcde cde de d Ø

d[a]

0 0 0 0 0 0

d[b]

∞ 2 2 2 2 2

d[c] ∞ 12 10 10 10 10

d[d]

∞ ∞ ∞ 16 13 13

d[e]

∞ ∞ 11 11 11 11

iteration


Running Time of Dijkstra's Alg. initialization: insert each vertex once

O(V Tins)

O(V) iterations of while loop one extract-min per iteration => O(V Tex) for loop inside while loop has variable number of

iterations…

For loop has O(E) iterations total one decrease-key per iteration => O(E Tdec)

Total is O(V (Tins + Tex) + E Tdec)


Using Different Heap Implementations O(V(Tins + Tex) + ETdec) If priority queue is implemented with a

binary heap, then Tins = Tex = Tdec = O(log V) total time is O(E log V)

There are fancier implementations of the priority queue, such as Fibonacci heap: Tins = O(1), Tex = O(log V), Tdec = O(1) (amortized) total time is O(V log V + E)


Using Simpler Heap Implementations O(V(Tins + Tex) + ETdec) If graph is dense, so that |E| = (V2), then it

doesn't help to make Tins and Tex to be at most O(V).

Instead, focus on making Tdec be small, say constant.

Implement priority queue with an unsorted array: Tins = O(1), Tex = O(V), Tdec = O(1) total is O(V2)


What About Negative Edge Weights? Dijkstra's SSSP algorithm requires all

edge weights to be nonnegative even more restrictive than outlawing

negative weight cycles Bellman-Ford SSSP algorithm can

handle negative edge weights even "handles" negative weight cycles by

reporting they exist


Bellman-Ford Idea Consider each edge (u,v) and see if u

offers v a cheaper path from s compare d[v] to d[u] + w(u,v)

Repeat this process |V| - 1 times to ensure that accurate information propgates from s, no matter what order the edges are considered in


Bellman-Ford SSSP Algorithm input: directed or undirected graph G = (V,E,w)//initialization initialize d[v] to infinity and parent[v] to nil for all v in V

other than the source initialize d[s] to 0 and parent[s] to s// main body for i := 1 to |V| - 1 do

for each (u,v) in E do // consider in arbitrary order if d[u] + w(u,v) < d[v] then

d[v] := d[u] + w(u,v) parent[v] := u


Bellman-Ford SSSP Algorithm// check for negative weight cycles for each (u,v) in E do

if d[u] + w(u,v) < d[v] then output "negative weight cycle exists"


Running Time of Bellman-Ford O(V) iterations of outer for loop O(E) iterations of inner for loop O(VE) time total


Bellman-Ford Example

s

c

a

b3

—44

2

1

process edges in order(c,b)(a,b)(c,a)(s,a)(s,c)

<board work>


Speeding Up Bellman-Ford The previous example would have

converged faster if we had considered the edges in a different order in the for loop move outward from s

If the graph is a DAG (no cycles), we can fully exploit this idea to speed up the running time

Documents

CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1