Introduction to Algorithms 6.006

Lecture 16 Prof. Piotr Indyk

Introduction to Algorithms 4/5/11 2

Menu

•  Last week: Bellman-Ford –  O(VE) time –  general weights

•  Today: Dijkstra – O( (V+E)logV ) time –  non-negative weights

Introduction to Algorithms 4/5/11 3

Single source shortest path problem

•  Problem: Given a digraph G = (V, E) with non-negative edge-weight function w, and a node s, find δ(s, v)* for all v in V •  Want a fast algorithm… •  Question: what if all edge weights are equal to 1 ?

*Paths can be found as well

a

c vxz

s d f

BFS

a

c v x z

s d f 1

12

2

2 3

3 0

Running time: O(V+E)

Weighted graphs

Introduction to Algorithms 4/5/11 5

7

s

b

d

c

g

f 3

8

5

5

2

5

4

Question II: what if all edge weights are integers in the range 1…W ?

Weighted graphs

Introduction to Algorithms 4/5/11 6

7

s

b

d

c

g

f 3

8

5

5

2

5

4

Algorithm: •  Create an unweighted graph by splitting each edge with weight w into w pieces •  Run BFS

Running time: O(V+WE)

3

5

Introduction to Algorithms 4/5/11 7

Greedy approach

IDEA: Greedy. 1.  Maintain a set S of vertices whose shortest-

path distances from s are known. 2.  At each step add to S the vertex v ∈ V – S

whose distance estimate from s is minimal. 3.  Update the distance estimates of vertices

adjacent to v.

Introduction to Algorithms 4/5/11 8

Dijkstra’s algorithm d[s] ← 0 for each v ∈ V – {s}

do d[v] ← ∞ S ← ∅ Q ← V ⊳ Q is a priority queue maintaining V – S while Q ≠ ∅

do u ← EXTRACT-MIN(Q) S ← S ∪ {u} for each v ∈ Adj[u]

do if d[v] > d[u] + w(u, v) then d[v] ← d[u] + w(u, v)

relaxation step

Implicit DECREASE-KEY

Dijkstra: Example 7 (∝,-)

a

b

d

c

g

f

e h

i

j 3

8

5

5

2

5

8

4

5

6

6

1

4

1 6

2

(0,*)+

(∝,-)

(∝,-)

(∝,-)

(∝,-)

(∝,-)

(∝,-)

(∝,-)

initialization (∝,-)

Dijkstra: Example 7 (∝,-)

a

b

d

c

g

f

e h

i

j 3

8

5

5

2

5

8

4

5

6

6

1

4

1 6

2

(0,*)+

(5,a)

(8,a)

(∝,-)

(∝,-)

(∝,-)

(∝,-)

(∝,-)

1st iteration (3,a)+

Dijkstra: Example

Introduction to Algorithms 4/5/11 11

(10,b)

a

b

d

c

g

f

e h

i

j 3

8

5

5

2

7

5

8

4

5

6

6

1

4

1 6

2

(0,*)+

(3,a)+

(8,a)

(∝,-)

(∝,-)

(∝,-)

(∝,-)

(∝,-)

2nd iteration

(5,a)+

Dijkstra: Example

a

b

d

c

g

f

e h

i

j 3

8

5

5

2

7

5

8

4

5

6

6

1

4

1 6

2

(0,*)+

(3,a)+ (10,b)

(9,d)

(∝,-)

(∝,-)

(∝,-)

(∝,-)

3rd iteration

(5,a)+

(7,d)+

Dijkstra: Example

Introduction to Algorithms 4/5/11 13

a

b

d

c

g

f

e h

i

j 3

8

5

5

2

7

5

8

4

5

6

6

1

4

1 6

2

(0,*)+

(5,a)+

(3,a)+

(7,d)+

(10,b)

(15,c)

(∝,-)

(∝,-)

(∝,-)

4th iteration

(9,d)+

Dijkstra: Example

a

b

d

c

g

f

e h

i

j 3

8

5

5

2

7

5

8

4

5

6

6

1

4

1 6

2

(0,*)+

(3,a)+

(9,d)+

(15,c)

(∝,-)

(13,g)

(∝,-)

5th iteration

(5,a)+

(7,d)+

(10,b)+

Dijkstra: Example

Introduction to Algorithms 4/5/11 15

a

b

d

c

g

f

e h

i

j 3

8

5

5

2

7

5

8

4

5

6

6

1

4

1 6

2

(0,*)+

(5,a)+

(3,a)+

(7,d)+

(10,b)+

(15,c)

(16,f)

(∝,-)

6th iteration

(9,d)+ (13,g)+

Dijkstra: Example

a

b

d

c

g

f

e h

i

j 3

8

5

5

2

7

5

8

4

5

6

6

1

4

1 6

2

(0,*)+

(3,a)+

(9,d)+

(16,f)

(13,g)+

(19,i)

7th iteration

(5,a)+

(7,d)+

(10,b)+

(14,i)+

Dijkstra: Example

Introduction to Algorithms 4/5/11 17

a

b

d

c

g

f

e h

i

j 3

8

5

5

2

7

5

8

4

5

6

6

1

4

1 6

2

(0,*)+

(5,a)+

(3,a)+

(7,d)+

(10,b)+

(14,i)+ (19,i)

8th iteration

(9,d)+ (13,g)+

(15,e)+

Dijkstra: Example

a

b

d

c

g

f

e h

i

j 3

8

5

5

2

7

5

8

4

5

6

6

1

4

1 6

2

(0,*)+

(3,a)+

(9,d)+

(15,e)+

(13,g)+

9th iteration

(17,h)+

(5,a)+

(7,d)+

(10,b)+

(14,i)+

(10,b)+

a

b

d

c

g

f

e h

i

j 3

5 2

7

4

1

4

1

2

(0,*)+

(5,a)+

(3,a)+

(7,d)+

(9,d)+

(14,i)+

(15,e)+

(13,g)+

(17,h)+

Shortest-path tree

Introduction to Algorithms 4/5/11 19

Correctness — Part I Lemma. Initializing d[s] ← 0 and d[v] ← ∞ for all v ∈ V – {s} establishes d[v] ≥ δ(s, v) for all v ∈ V, and this invariant is maintained over any sequence of relaxation steps.

s u

v

d[u] w(u, v)

d[v]

Proof. Recall relaxation step:

if d[v] > d[u] + w(u, v) set d[v] ← d[u] + w(u, v)

Introduction to Algorithms 4/5/11 20

Correctness — Part II Theorem. Dijkstra’s algorithm terminates with d[v] = δ(s, v) for all v ∈ V. Proof. •  It suffices to show that d[v] = δ(s, v) for every v ∈ V when v is added to S •  Suppose u is the first vertex added to S for which d[u] ≠ δ(s, u) . Let y be the first vertex in V – S along a shortest path from s to u, and let x be its predecessor:

s x y

u

S, just before adding u.

Introduction to Algorithms 4/5/11 21

Correctness — Part II (continued)

•  Since u is the first vertex violating the claimed invariant, we have d[x] = δ(s, x) •  Since subpaths of shortest paths are shortest paths, it follows that d[y] was set to δ(s, x) + w(x, y) = δ(s, y) just after x was added to S •  Consequently, we have d[y] = δ(s, y) ≤ δ(s, u) ≤ d[u] •  But, d[y] ≥ d[u] since the algorithm chose u first •  Hence d[y] = δ(s, y) = δ(s, u) = d[u] - contradiction

s x y

uS

Introduction to Algorithms 4/5/11 22

Analysis of Dijkstra

degree(u) times

|V | times

DECREASE-KEY

Time = Θ(V)·TEXTRACT-MIN + Θ(E)·TDECREASE-KEY

while Q ≠ ∅ do u ← EXTRACT-MIN(Q)

S ← S ∪ {u} for each v ∈ Adj[u]

do if d[v] > d[u] + w(u, v) then d[v] ← d[u] + w(u, v)

Introduction to Algorithms 4/5/11 23

Analysis of Dijkstra (continued)

Time = Θ(V)·TEXTRACT-MIN + Θ(E)·TDECREASE-KEY

Q TEXTRACT-MIN TDECREASE-KEY Total

array O(V) O(1) O(V2) binary heap O(lg V) O(lg V) O(E lg V)

Fibonacci heap

O(lg V) amortized

O(1) amortized

O(E + V lg V) worst case

Introduction to Algorithms 4/5/11 24

Tuesday: generic algorithm

d[s] ← 0 for each v ∈ V – {s}

do d[v] ← ∞ initialization

while there is an edge (u, v) ∈ E s. t. d[v] > d[u] + w(u, v) do select one such edge “somehow” set d[v] ← d[u] + w(u, v) endwhile

relaxation step

s u

v

d[u] w(u, v)

d[v] How to do it in O( (V+E)logV ) time ?