1

Chapter 9

Dynamic Programming

• General Problem-Solving strategy

• Was born in the 1950s (Bellman)

• Based on successive decomposition

• Leads to functional equations

• More “Art” than “Science” (?)

• Difficult to teach/learn

• No general purpose software

• Usually taught/learned by example

2

programme

• Shortest Path problem

• Knapsack problem

• Tower of Hanoi Puzzle

• Travelling Salesman Problem (??)

• Critical Path Problem (Chapter 10)

• And

• Dijkstra’s Algorithm (BYO) (details in due course)

3

9.1 Shortest Path Problem

• Graphs:

–Nodes (vertices)

–Arcs (edges)

• Networks

–Nodes

–Arcs

–Additional information (eg. length)

4

• The significance of the “drawing” is the precedence relationships it represents, not the visual picture.

• On the other hand, the picture can tell us a lot about the nature of the relationships...

• Make sure that you distinguish between the drawing and the relationships!!!!!

5

5

1

4 3

2

5

1 432

6

Terminology

• Path

• Predecessor

• Successor

• Immediate Predecessor

• immediate Successor

• Origin (no predecessor)

• Destination (no successors)

• Cycle

• Acyclic graph

7

Notation

• S(n) := Set of immediate predecessors of node n

• P(n) := Set of Immediate successors of node n.

• := empty set

8

Lemma 9.1.1

• If a directed graph with a finite number of nodes N is acyclic, then the nodes can be numbered in such a way that the following condition is satisfied:

m < n for all m in P(n), n=1,2,...,N

In words, an ordering of nodes exists such that all arcs are directed from a lower numbered node to a higher numbered node.

9

Labeling Procedure

• Find a node without a predecessor

• Label this node “1”

• Delete all arcs incident from “1”

• Set n=1

• Repeat the process for n=n+1 until all nodes are labeled.

10

Example 9.1.2

11

1

1

2

12

1

2

1

2

3

4

5

6

7

13

Problem Statement

• Given a directed graph where each arc is assigned a numerical length, we want to find the shortest path between a specified origin s and a specified destination t, where the length of a path is equal to the sum of the arc lengths on that path.

• There are “generalised” shortest path problems where the length of a path is not “additive” (Eg. ??????)

14

DP Strategy

• We embed the problem of interest (shortest path from origin to destination) in a family of other related problems.

• We establish some relationaships between the optimal solutions to these problems.

• We solve this relationship (functional equation)

• From the solution of the functional equation we obtain the solution to the problem of interest.

15

Convention

aij :,

,=

∞⎧⎨⎩

length of ( ,arc i)j if such an arc exists

otherwise

Origin = node 1destination = node N

16

•Let• f(j) := length of the shortest path

from the origin to node j, j=1,2,...,N

• Note that we are interested “only” in f(N).

17

Observation #1

f(j) = f(k) + ak,j

for some k in P(j) (why?)

j

k

1

N

(NILN)

18

But, .... ?

• What is the value of k ?????????

f(j) = f(k) + ak,j

• (k := “Best” immediate predecessor)

• We identify k by “doing the best we can”, ie. we find the value of k for which

f(j) = f(k) + ak,j

is the smallest possible among all immediate predecessor of node j.

19

result

• DP functional equation

{ }f j f k a j Nk j

kj( ) min ( ) , , . . . ,= + =≠

1

f ( )1 0=

20

Remark

• Since aij= infinity if k is not in P(j), the functional equation can be rewritten as follows:

{ }f j f k a j Nk P j

kj( ) min ( ) , , .. . ,( )

= + =∈

2

f ( )1 0=

21

Notation

• Let

{ }K j k f j f k a k P jk j( ): *: ( ) ( *) , * ( )*= = + ∈

(Constructed as we solve the functional equation)

22

Procedure

• We start by setting f(1)=0

• Then, we determine the values of f(2),f(3),f(4),....... in this order.

• This is possible because our networks are acyclic.

• For small problems we can do it on the picture itself.

23

Example 9.1.4

• P(2)={1}, P(3){1}; P(4)={2,3}; P(5)={2,4}; P(6)={3,4}; P(7)={5,6}.

1

2

3

4

5

6

7

3

2

5

4

10

10

3

4 5

6

24

1

2

3

4

5

6

7

3

2

5

4

10

10

3

4 5

6

(0)

25

1

2

3

4

5

6

7

3

2

5

4

10

10

3

4 5

6

(0)

1

2

3

4

5

6

7

3

2

5

4

10

10

3

4 5

6

(0)

(3)

(2)

x

x

26

1

2

3

4

5

6

7

3

2

5

4

10

10

3

4 5

6

(0)

(3)

(2)

1

2

3

4

5

6

7

3

2

5

4

10

10

3

4 5

6

(0)

(3)

(2)

(6)

x

x x

x

x

27

1

2

3

4

5

6

7

3

2

5

4

10

10

3

4 5

6

(0)

(3)

(2)

(6)

x

x x

1

2

3

4

5

6

7

3

2

5

4

10

10

3

4 5

6

(0)

(3)

(2)

(6)

x

x x

(10)

x

28

1

2

3

4

5

6

7

3

2

5

4

10

10

3

4 5

6

(0)

(3)

(2)

(6)

x

x x

(10)

x

1

2

3

4

5

6

7

3

2

5

4

10

10

3

4 5

6

(0)

(3)

(2)

(6)

x

x x

(10)

x

(9)

x

29

1

2

3

4

5

6

7

3

2

5

4

10

10

3

4 5

6

(0)

(3)

(2)

(6)

x

x x

(10)

x

(9)

x

1

2

3

4

5

6

7

3

2

5

4

10

10

3

4 5

6

(0)

(3)

(2)

(6)

x

x x

(10)

x

(9)

x

(15)

x

x

30

1

2

3

4

5

6

7

3

2

5

4

10

10

3

4 5

6

(0)

(3)

(2)

(6)

x

x x

(10)

x

(9)

x

(15)

x

x

• Summary:

f(1)=0; f(2)=3; f(3)=2;f(4)=6;f(5)=10;f(6)=9; f(7)=15.

• Question:

How do we recover optimal paths?

31

• We go backward along marked arcs.

1

2

3

4

5

6

7

3

2

5

4

10

10

3

4 5

6

(0)

(3)

(2)

(6)

x

x x

(10)

x

(9)

x

(15)

x

x

32

1

2

3

4

5

6

7

3

2

5

4

10

10

3

4 5

6

(0)

(3)

(2)

(6)

x

x x

(10)

x

(9)

x

(15)

x

x

1

2

3

4

5

6

7

3

2

5

4

10

10

3

4 5

6

(0)

(3)

(2)

(6)

x

x x

(10)

x

(9)

x

(15)

O

O

33

1

2

3

4

5

6

7

3

2

5

4

10

10

3

4 5

6

(0)

(3)

(2)

(6)

x

x x

(10)

x

(9)

x

(15)

O

O

1

2

3

4

5

6

7

3

2

5

4

10

10

3

4 5

6

(0)

(3)

(2)

(6)

x

x x

(10)

O

(9)

O

(15)

O

O

34

1

2

3

4

5

6

7

3

2

5

4

10

10

3

4 5

6

(0)

(3)

(2)

(6)

x

x O

(10)

O

(9)

O

(15)

O

O

1

2

3

4

5

6

7

3

2

5

4

10

10

3

4 5

6

(0)

(3)

(2)

(6)

x

x x

(10)

O

(9)

O

(15)

O

O

35

1

2

3

4

5

6

7

3

2

5

4

10

10

3

4 5

6

(0)

(3)

(2)

(6)

x

x O

(10)

O

(9)

O

(15)

O

O

1

2

3

4

5

6

7

3

2

5

4

10

10

3

4 5

6

(0)

(3)

(2)

(6)

x

O O

(10)

O

(9)

O

(15)

O

O

36

• There are two optimal paths:

p1=(1,3,4,5,7) ; p2=(1,3,4,6,7)

1

2

3

4

5

6

7

3

2

5

4

10

10

3

4 5

6

(0)

(3)

(2)

(6)

x

O O

(10)

O

(9)

O

(15)

O

O

37

Remark

• It is OK (in the exam, assignment, etc) to do the calculation directly on the picture, but ........ write down the details for one nontrivial node, eg:

{ } { } { }f f k a f k a f a f a

K

k Pk

kk( ) min ( ) min ( ) min ( ) , ( )

min{ , } min{ , }

( ) { }.

( ),

{ , }, , ,4 2 3

3 5 2 4 8 6 6

4 3

44

2 34 2 4 3 4= + = + = + +

= + + = =

=

∈ ∈

38

Comment

• The are many generalisations ..........