Supplementary:Feasible Labels and Linear Programming
Consider the LP formulation of the shortest s-t path problem:
(P) Minimize (i, j)A cijxij
subject to {j: (i, j)A} xij - {j: (j, i)A} xji = +1, for i = s
= -1, for i = t
= 0, for i N\{s, t}
xij 0 for all (i, j)A
It is guaranteed that the optimal solution to (P), if exists, is integer-valued (proof later when we study MCF). Hence we can solve the LP instead of IP.
Multiply (-1) on both sides of the constraints, then get
{j: (j, i)A} xji - {j: (i, j)A} xij = -1, for i = s
= +1, for i = t
= 0, for i N\{s, t}
i.e consider (inflow to i) – (outflow from i)Network Theory and Applications 2010 1
(P) Minimize (i, j)A cijxij (S-1)
subject to {j: (j, i)A} xji - {j: (i, j)A} xij = -1, for i = s
= +1, for i = t
= 0, for i N\{s, t}
xij 0 for all (i, j)A
The dual of (P) is
(D) maximize yt – ys (S-2)
subject to yj - yi cij for all (i, j) A
Note that, for a dual feasible solution y, we may subtract any constant from each dual variable and we still have a dual feasible solution with the same objective value. we may set ys = 0.
Network Theory and Applications 2010 2
We assumed there exists a s-t path in G, hence (P) is feasible.
If (P) has a finite optimal value (with directed s-t walk) , then by strong duality, (D) is also feasible and its optimal value is the same as (P) (shortest s-t distance = yt )
Conversely, if (D) has a feasible solution, then (P) and (D) have the same optimal value (since (P) is feasible too).
(Compare with Thm 5.1) If the network contains a negative cycle, (P) is unbounded, hence (D) is
infeasible, i.e. no y (d()) satisfies (5.2)
Network Theory and Applications 2010 3
The constraint matrix in (P) has special structure; each column has exactly one 1 and one –1 (except nonnegativity constraints). Adding the rows of the matrix A, we get 0 vector, which implies the rows are linearly dependent (rank is n-1).
Prop : Let G be a connected directed graph and D = {ae : e A} be its node-
arc incidence matrix ( ae is the column vector for arc e in the constraint
matrix) . A set {ae : e T} is a column basis of D iff T is the arc-set of a
spanning tree of G (not considering the directions of the arcs).
( G has n nodes, m arcs. D is n m matrix. Spanning tree has n-1 arcs.)
Network Theory and Applications 2010 4
So the rank of D is n-1. To solve the problem by the simplex method, we need nonsingular basis B.
Two approaches:Drop one of the constraints and solve the full row rank problemAdd an artificial variable to any one constraint (objective coefficient is 0). It
makes D full row rank (the artificial variable always remains in the basis and has value 0 in any b.f.s).
b.f.s. and dual vector satisfies
xe = 0 for all eT
yj – yi = cij for all (i, j)T
( From y’B = cB’ for current basis B)
This y vector gives the path length to each node j in the current tree T (If T is directed out-tree).
Note that if we add artificial variable for node s (root), then we have ys = 0 in
any b.f.s. ( from y’B = cB’)
Network Theory and Applications 2010 5
A b.f.s. for (P) does not necessarily correspond to an directed out-tree. It may be just a spanning tree.
However, the simplex iteration can be modified so that we always have directed out-tree, once we started the algorithm with a directed out-tree.
Each step of the simplex method may correspond to multiple Ford’s step;
(1) Change of directed out-tree
(2) update the node labels so that it gives the actual path length in the directed out-tree.
Network Theory and Applications 2010 6
Suppose we solve s-t shortest path problem by LP (with simplex method) and have a feasible directed out-tree (bfs);
entering nonbasic arc selection: choose arc (i, j) with cij – (yj – yi) < 0, i.e. cij + yi
< yj. ( from cj – y’Aj < 0 since we solve min problem )
leaving basic variable: select the entering arc to node j in the current directed out-tree as leaving basic arc. (updating tree in Ford algorithm)
Update flow (x-value) so that we have value 1 for each arc on the s-t path.
Note that we may solve the shortest s-t path problem by LP. But, in an optimal basis (directed out-tree), we have shortest s-j path, for all jN. ( If you use the above modified simplex method)
( We have feasible potential (label) y and the s-j paths in the directed out-tree for all jN gives actual paths with length yj for all jV.)
Network Theory and Applications 2010 7
The identification of the basis of the LP as a spanning tree (not necessarily directed out-tree) is used in the network simplex algorithms for the minimum cost network flow problem. (Chapter 11)
Network Theory and Applications 2010 8
Other Interpretation of Ford-Bellman Algorithm Reference: Combinatorial Optimization: Networks and Matroids, E. Lawler
Bellman’s equations: u1 = 0
uj = min kj { uk +akj }, j=2, 3, …, n
(necessary and sufficient conditions for shortest paths when G does not contain a nonpositive cycle.)
Solve Bellman’s eq. iteratively. Still effective if no negative cycle in G. Initial values
u1(1) = 0
uj(1) = a1j, j 1
Iteration
uj(m+1) = min { uj
(m) , min kj { uk(m) + akj }}
Interpretation
uj(m) = the length of a shortest path from the origin to j, subject to the
condition that the path contains no more than m arcs.
(compare it with the presentation given in the text)Network Theory and Applications 2010 9
5.6 All-Pairs Shortest Path Probelm
Assume G is strongly connected. (add (s, i) and (i, s) for iN\{s} with large costs if not exist)
Two approaches:Repeated shortest path algorithm (suitable for sparse graph)All-pairs label-correcting algorithm (suitable for dense graph)
Repeated Shortest Path Algorithm:
Let S(n, m, C) denote the time needed to solve a shortest path problem with nonnegative arc lengths.
Select a node s as a source node and use FIFO label-correcting algorithm.
Then, have cijd = cij + d(i) – d(j) 0 for all arcs (i, j).
Using cijd 0, solve single-source shortest path problem (n-1) times.
From Property 5.2.(b) ((i, j)P cijd = (i, j)P cij + d(k) – d(l)), actual shortest path
length from k to l is ((i, j)P cijd + (d(l) – d(k) )
Running time: O(nm + nS(n, m, C)) = O(nS(n, m, C)).Network Theory and Applications 2010 10
All-Pairs Shortest Path Optimality Conditions: Thm 5.5. Let d[i, j] represent the length of some directed path from node i to
node j. Then d[i, j] optimal d[i, j] d[i, k] + d[k, j] for all i, j, and k. (5.3)
Pf) ) If d[i, k] + d[k, j] < d[i, j], use path from i → k → j walk. Contradiction to the optimality of d[i, j].
) d[i, j] length of some path upper bound on the shortest length
Let i = i1 – i2 – i3 - … - ik = j be any i-j path P
d[i, j] = d[i1, ik] d[i1, i2] + d[i2, ik] ci(1)i(2) + d[i2, ik],
d[i2, ik] ci(2)i(3) + d[i3, ik],
…
d[ik-1, ik] ci(k-1)i(k).
d[i, j] ci(1)i(2) + ci(2)i(3) + … + ci(k-1)i(k) = (i, j)P cij.
Hence d[i, j] is a lower bound on any i-j path. d[i, j] also upper bound, so optimal.Network Theory and Applications 2010 11
All-Pairs Generic Label Correcting Algorithm:
algorithm all-pairs label-correcting;
begin
set d[i, j] := for all [i, j]NN;
set d[i, i] := 0 for all iN;
for each (i, j)A do d[i, j] := cij;
while the network contains three nodes i, j, and k
satisfying d[i, j] > d[i, k] + d[k, j] do d[i, j] := d[i, k] + d[k, j];
end;
Network Theory and Applications 2010 12
Correctness: Whenever d[i, j] < , walk of length d[i, j] from i to j.
The walk may decompose into a path P and some directed cycles. At termination of the algorithm, all of the directed cycles must have length 0, hence the path length is d[i, j]. Also labels d[i, j] satisfy the optimality condition (5.3).
Finiteness: Distance label is bounded from above by nC, from below by –nC. Each iteration decreases some d[i, j]. Hence, total O(n3C) iterations.
O(n3) implementation: Floyd-Warshall Algorithm
Network Theory and Applications 2010 13
Floyd-Warshall Algorithm:
dk[i, j]: i-j shortest path length using only the nodes 1, 2, … , k-1 as internal nodes.
Then, dn+1[i, j] represents shortest path length from i to j.
Starting from d1[i, j], update the labels recursively.
Property 5.6. dk+1[i, j] = min {dk[i, j], dk[i,k] + dk[k, i]}.
Two cases:1. Do not use node k. Then dk+1[i, j] = dk[i, j]
2. Use node k. Then dk+1[i, j] = dk[i,k] + dk[k, i]
Network Theory and Applications 2010 14
Algorithm Floyd-Warshall;
begin
for all node pairs [i, j]NN
d[i, j] := and pred[i, j] := 0;
for all nodes iN do d[i, i] := 0;
for each arc (i, j)A do d[i, j] := cij and pred[i, j] := i;
for each k := 1 to n do
for each [i, j]NN do
if d[i, j] > d[i, k] + d[k, j] then
begin
d[i, j] := d[i, k] + d[k, j];
pred[i, j] := pred[k, j];
end;
end;
Network Theory and Applications 2010 15
View as operation on matrix:
Network Theory and Applications 2010 16
d[i, j]
d[k, j]
d[i, k]
k
k
If there exists no negative cycle, dk[i, i] = 0 for all i, k
Hence row k, column k do not change.
maintain only one copy of the matrix.
If there exists a negative cycle d[i, i] < 0 for some i.
Retrieving actual paths:
pred[i, j] denotes the last node prior to node j in the tentative shortest path from node i to node j.
Consider a path P from node k to node l. Starting from node l, let g=pred[k, l].
Then g is the last node prior to node l in P. Similarly, h = pred[k, g] is the node prior to node g in P, and so on. Repeat until we reach node k.
Thm 5.7. The Floyd-Warshall algorithm runs in O(n3).
Network Theory and Applications 2010 17
Detection of negative cycles:
Two tests for negative cycle:If i = j, check whether d[i, i] < 0.If i j, check whether d[i, j] < -nC.
Use predecessor graph to retrieve negative cycle.
Network Theory and Applications 2010 18
Other algebraic view
Reference: Combinatorial Optimization: Networks and Matroids, E. Lawler Let d[i, j] = length of a shortest path from i to j.
dk[i, j] = length of a shortest path from i to j using k arcs.
d0[i, i] = 0, d0[i, j] = + (i j)
dk+1[i, j] = minl { dk[i, l] + clj }
Have d(n-1)[i, j] = d[i, j]
Define the new matrix multiplication as P = A B, Pij = mink {aik + bkj }
We have Dk+1 = Dk C
D(0) = is identity matrix D(0)C = C
multiplication is associative.
Dn-1 = Dn-2 C = … = ((D0 C) C) … C) = Cn-1
Also Cm = Cn-1, m n-1.Network Theory and Applications 2010 19
0...
...0...
...0
Can compute Cn-1 as C → C2 → C4 → … → C2(k), 2k n-1.
log(n) steps O(n3 log n)
Floyd-Warshall is a clever implementation of matrix multiplcation.
Network Theory and Applications 2010 20
5.7 Minimum Cost-To-Time Ratio Cycle Problem
Tramp steamer problem: Identify a directed cycle W of G with the maximum ratio ((i, j)W pij)/((i, j)W ij)
Let cost cij of arc (i, j) as cij = -pij, and identify W with minimum ratio.
Assume data are integral and ij 0 for (i, j)A and (i, j)W ij > 0 for every
directed cycle W in G.
Let (W) = ((i, j)W cij)/((i, j)W ij) and * is optimal value .
Starting from arbitrary value of , define the length of each arc as lij = cij -
ij. Then use negative cycle detection algorithm.
Network Theory and Applications 2010 21
Three cases:(1) G contains a negative length cycle W.
have (i, j)W (cij - ij ) < 0. Alternatively,
> ((i, j)W cij)/((i, j)W ij) *.
Therefore, is a strict upper bound on *.
(2) G contains no negative cycle, but does contain a zero-length cycle W*. (see Exercise 5.19 on how to identify a zero-length cycle)
no negative cycle implies (i, j)W (cij - ij ) 0 for every directed cycle W.
Alternatively, ((i, j)W cij)/((i, j)W ij) for every directed cycle W.
Also, existence of zero-length cycle W* implies = ((i, j)W* cij)/((i, j)W* ij).
Hence = *, and W* is a minimum cost-to-time ratio cycle.
(3) Every directed cycle W in G has a positive length.
have (i, j)W (cij - ij) > 0 for every directed cycle W. Alternatively,
< ((i, j)W cij)/((i, j)W ij) for every directed cycle W.
Hence, is a strict lower bound on *.
Network Theory and Applications 2010 22
Search procedures:
Guess a value and use shortest path algorithm
If negative cycle, *. Use smaller value of .
If zero-length cycle, then optimal.
If positive length cycle, < *. Use larger value of .
Sequential search algorithm:
Let 0 be a known upper bound on *.
Solve the shortest path problem with (cij - 0ij ) as arc lengths.
Either find a zero-length cycle W or a negative cycle W.
In the former case, optimal found, stop. In the latter case, let 1 = ((i, j)W
cij)/((i, j)W ij) as our next guess. Have 0 > 1 *.
Obtain sequence of values 0 > 1 … > k = *.
Network Theory and Applications 2010 23
Binary search algorithm:
Let [L, U] be the interval that contains *. (e.g. [-C, C])
Use 0 = (L + U)/2, and check for negative cycle.
If a negative cycle exists, 0 > *. Reset U = 0.
Otherwise, 0 *. Reset L = 0.
After sufficiently large number of iterations, the search interval has a unique solution.
Let c(W) and (W) denote the cost and travel time of any directed cycle W, and let 0 = max{ij : (i, j)A}.
Claim: The interval of size at most 1/02 contains at most one value from the
set {c(W)/(W): W is a directed cycle}
Let W1 and W2 be two directed cycles with distinct ratios.
Then |c(W1)/(W1) - c(W2)/(W2)| 0, or
| {c(W1)/(W2) - c(W2)/(W1)} / {(W1)(W2)} | 0.
Smallest value of the left-hand side is 1/02.
Hence if the interval becomes smaller than 1/02, using U as the value of
identifies the optimal cycle W. Network Theory and Applications 2010 24
Network Theory and Applications 2010 25
Handout: Combinatorial Optimization: Networks and Matroids, E. Lawler
Enumerate simple paths in nondecreasing order with respect to the weights.
Let P be the set of all 1-n paths.
Find shortest path P1 initially. Modify the lengths using cdij 0.
Partition P – {P1} into subsets P1, P2, … , Pk, where k n-1, so that we can determine the shortest path in each subset.
Let P2 be the shortest of these shortest paths identified. Suppose P2 Pj.
We then partition Pj - {P2} into subsets in the same way, and so on.
The procedure can be described as a tree.
How to partition the subset so that we do not lose any path and we can find a shortest path in each subset easily?
M Shortest Paths without Repeated Nodes
Suppose we have shortest paths P1, P2, …, Pm . Let P be the set of all 1-n paths. We divide the set P – {P1, P2, …, Pm} into P1, P2, … , Pk, where k n-1. Each Pj includes paths with the condition that (a) they include the arcs in a certain specified path from node 1 to another node p, and (b) certain arcs from node p are excluded.
Then we can find a shortest path in Pj by finding a shortest path from node p to n after deleting nodes in the path from 1 to p, and deleting the arcs excluded from node p.
In general, suppose that Pm is the shortest path (1, 2), (2, 3), … , (q-1, q), (q, n) in Pj. Paths in Pj have restrictions that they should include arcs (1, 2), (2, 3), … , (p-1, p), where p q, and that certain arcs from node p are excluded.
Partition Pj – {Pm} as follows:If p = q, apply Dijkstra’s to obtain 1-n shortest path, subject to the conditions that
arcs (1, 2), (2, 3), … , (p-1, p) are included, and that (p, n) is excluded, in addition to the arcs from p excluded for Pm.
Network Theory and Applications 2010 26
If p < q, apply Dijkstra’s to find the shortest path from 1 to n, subject to the following sets of conditions:
(1) Arcs (1, 2), (2, 3), … , (p-1, p) are included and arc (p, p+1) is excluded, in addition to the arcs from p excluded for Pm.
(2) Arcs (1, 2), (2, 3), … , (p, p+1) are included and arc (p+1, p+2) is excluded.
…
(q-p-2) Arcs (1, 2), (2, 3), … , (q-2, q-1) are included and arc (q-1, q) is excluded.
(q-p-1) Arcs (1, 2), (2, 3), … , (q-1, q) are included and arc (q, n) is excluded.
Network Theory and Applications 2010 27