2004 SDU

Lecture11- All-pairs shortest paths

Dynamic programming

Comparing to divide-and-conquer1. Both partition the problem into sub-problems

2. Divide-and-conquer partition problems into independent sub-problems, dynamic programming is applicable when the sub-problems are dependent.

2002 SDU 2

Dynamic programming

• Comparing to greedy method:• Both are applicable when a problem exhibits optimal

substructure

• In greedy algorithms, we use optimal substructure in a top-down fashion, that is, without finding optimal solutions to sub-problems, we first make a choice that looks best at the time-and then solve a resulting sub-problem. While in dynamic programming, we use optimal substructure in a bottom-up fashion, that is, we first find optimal solutions to sub-problems and, having solved the sub-problems, we find an optimal solution to the problem.

2002 SDU 3

Dynamic programming

The development of a dynamic programming algorithm can be broken into four steps:1. Characterize the structure of an optimal solution.

2. Recursively define the value of an optimal solution.

3. Compute the value of an optimal solution in a bottom-up fashion.

4. Construct an optimal solution from computed information.

Steps 1-3 form the basis of a dynamic-programming solution to a problem. Step 4 can be omitted if only the value of an optimal solution is required. When we do perform step 4, we sometimes maintain additional information during the computation in step 3 to ease the construction of an optimal solution.

2002 SDU 4

5

0-1 version

v/w:

1,

3,

3.6,

3.66,

4.

23/4/186

23/4/187

23/4/18 8

23/4/189

23/4/1810

35

2002 SDU 11

All –pairs shortest-paths problem

Problem: Given a directed graph G=(V, E), and a weight function w: ER, for each pair of vertices u, v, compute the shortest path weight (u, v), and a shortest path if exists.

Output: A VV matrix D = (dij), where, dij contains the shortest path

weight from vertex i to vertex j. //Important! A VV matrix =(ij), where, ij is NIL if either i=j or there is

no path from i to j, otherwise ij is the predecessor of j on some shortest path from i. // Not covered in class, but in Exercises!

2002 SDU 12

Methods

1) Application of single-source shortest-path algorithms

2) Direct methods to solve the problem:1) Matrix multiplication

2) Floyd-Warshall algorithm

3) Johnson’s algorithm for sparse graphs

3) Transitive closure (Floyd-Warshall algorithm)

2002 SDU 13

Matrix multiplication--suppose there are no negative cycles.

A dynamic programming method: Study structure of an optimal solution Solve the problem recursively Compute the value of an optimal solution in a bottom-

up manner

The operation of each loop is like matrix multiplication.

2002 SDU 14

Matrix multiplication—structure of a shortest path

Suppose W = (wij) is the adjacency matrix such that

),( and if

),( and if ),( edge of weight the

if ,0

Ejiji

Ejijiji

ji

wij

Consider a shortest path P from i to j, and suppose that P has at most m edges. Then, if i = j, P has weight 0 and no edges. If i j, we can decompose P into i kj, P’ is a shortest path from i to k.

P’

2002 SDU 15

Matrix multiplication—recursive solution

Let lij(m) be the minimum weight of any path from i

to j that contains at most m edges.

lij(0) = 0, if i = j, and otherwise.

For m 1,

lij(m) = min{lik

(m-1)+wkj}, 1 k n

The solution is lij(n-1)

2002 SDU 16

Matrix Multiplication

Solve the problem stage by stage (dynamic programming)

L(1) = W

L(2)

…

L(n-1)

where L(m), contains the shortest path weight with path length m.

2002 SDU 17

Matrix multiplication (pseudo-code)

2002 SDU 18

Matrix multiplication (pseudo-code)

2002 SDU 19

Matrix multiplication (running time)

O(n4)

Improving the running time:No need to compute all the L(m) matrices for 1 m n-1.We are interested only in L(n-1), which is equal to L(m) for all integers m ≥ n-1, with assuming that there are no negative cycles.

2002 SDU 20

Improving the running time

Compute the sequenceL(1) = W,L(2) = W2 = WW ,L(4) = W4 = W2 W2,L(8) = W8 = W4 W4

...

We need only lg(n-1) matrix products Time complexity: 　 O(n3 lg n)

1)1lg(1)1lg()1lg()1lg( 22)2()2(

nnnn

WWWL

2002 SDU 21

Improving running time

2002 SDU 22

Floyd-Warshall algorithm--suppose there are no negative cycles.

-structure of shortest path

2002 SDU 23

Floyd-Warshall algorithm(idea)

dij(k): shortest path weight from i to j with intermediate

vertices (excluding i, j) from the set {1,2,…,k}

Intermediate vertex of a simple path p = <v1, v2, …, vl> is any vertex of p other than v1 or vl.

dij(0) = wij

(no intermediate vertices at all)

How to compute dij(k) from D(r), for r < k?

2002 SDU 24

Floyd-Warshall algorithm-recursive solution

dij(0) = wij

(no intermediate vertices at all)

dij(k) = min(dij

(k-1), dik(k-1) + dkj

(k-1)) if k ≥ 1

Result: D(n) = (dij(n))

(because all intermediate vertices are in the set {1, 2, …, n})

2002 SDU 25

Floyd-Warshall algorithm-compute shortest-path weights

Solve the problem stage by stage:D(0)D(1)D(2)…D(n)where D(k) contains the shortest path weight with all the intermediate vertices from set {1,2…,k}.

2002 SDU 26

Floyd-Warshall algorithm

5

1

2

3

4

3 4

-5

6

-4 2 1

8

7

k=1: d43(1) ＝ -5; d42

(1)=5; d45(1)=-2

2002 SDU 27

Floyd-Warshall algorithm

5

1

2

3

4

3 4

-5

6

-4 2 1

8

7

k=2: d43(2) =-5; d42

(2)=5; d45

(2)= 2

2002 SDU 28

Floyd-Warshall algorithm

5

1

2

3

4

34

-5

6

-4 2 1

8

7

k=3: d42(3)=-1;

d43(3)=-5; d45

(3)=-2

2002 SDU 29

Floyd-Warhsall algorithm (pseudo-code)

Time complexity: O(n3)Space: O(n3)

2002 SDU 30

Floyd-Warshall algorithm(less space)

Notice that:

we can write dij(k) directly on D(k-1)

)1()1()1()1()(

)1()1()1()1()(

},min{

},min{

k

kjk

kjk

kkk

kjk

kj

kik

kkk

kik

kik

kik

ddddd

ddddd

k-th row

k-th column

dij(k) dik

(k-1)

dkj(k-1)

dij(k)

dkj(k)

dik(k)

2002 SDU 31

Floyd-Warhsall algorithm(less space)

2002 SDU 32

Constructing a shortest path

For k = 0

For k 1

ij

ij

ij wjii

wjiifNIL

and if

or )0(

, if

if )1()1()1()1(

)1()1()1()1(

)(

kkj

kik

kij

kkj

kkj

kik

kij

kijk

ijddd

ddd

2002 SDU 33

Print all-pairs shortest paths

2002 SDU 34

Example:

1

5 4

3

2

3 4

7-4

8

1 -52

6

2002 SDU 35

06

052

04

710

4830

D(0)=

NILNILNILNIL

NILNILNIL

NILNILNILNIL

NILNILNIL

NILNIL

5

44

3

22

111

(0)=

D(1)= (1)=

06

20552

04

710

4830

NILNILNILNIL

NIL

NILNILNILNIL

NILNILNIL

NILNIL

5

1414

3

22

111

2002 SDU 36

06

20552

11504

710

44830

D(2)=

NILNILNILNIL

NIL

NILNIL

NILNILNIL

NIL

5

1414

223

22

1211

(2)=

D(3)= (3)=

06

20512

11504

710

44830

NILNILNILNIL

NIL

NILNIL

NILNILNIL

NIL

5

1434

223

22

1211

2002 SDU 37

06158

20512

35047

11403

44130

D(4)=

NIL

NIL

NIL

NIL

NIL

5434

1434

1234

1244

1241

(4)=

D(5)= (5)=

06158

20512

35047

11403

42310

NIL

NIL

NIL

NIL

NIL

5434

1434

1234

1244

1543

2002 SDU 38

1

5 4

3

2

3 4

7-4

8

1 -52

6

)5(Shortest path from 1 to 2 in

2002 SDU 39

Transitive closure (the problem)

Given a graph G=(V, E), compute the transitive closure G* = (V, E*),

where E* = {(i, j): There is a path from i to j in G}

i j

ab

c

2002 SDU 40

Transitive closure

One way:

set wij = 1 and

run the Floyd-Warshall algorithm, if dij(n)

<, then (i, j) E*

Running time O(n3)

2002 SDU 41

transitive closure

Another way:

Substitute “+” and “min” by AND and OR

in Floyd’s algorithm

Running time O(n3)

2002 SDU 42

Transitive closure (pseudo-code)

2002 SDU 43

Transitive closure