Transcript PPT

CS 332: Algorithms
Minimum Spanning Tree
Shortest Paths
David Luebke
1
7/27/2016
Review: Getting Dressed
Underwear
Socks
Watch
Pants
Shoes
Shirt
Belt
Tie
Jacket
Socks
Underwear
David Luebke
Pants
Shoes
Watch
2
Shirt
Belt
Tie
Jacket
7/27/2016
Review: Topological Sort Algorithm
Topological-Sort()
{
Run DFS
When a vertex is finished, output it
Vertices are output in reverse
topological order
}

Time: O(V+E)
David Luebke
3
7/27/2016
Review: Minimum Spanning Tree

Problem: given a connected, undirected,
weighted graph, find a spanning tree using
edges that minimize the total weight
6
4
5
9
14
2
10
15
3
David Luebke
8
4
7/27/2016
Review: Minimum Spanning Tree

Problem: given a connected, undirected,
weighted graph, find a spanning tree using
edges that minimize the total weight
6
4
5
9
14
2
10
15
3
David Luebke
8
5
7/27/2016
Review: Minimum Spanning Tree


MSTs satisfy the optimal substructure property: an
optimal tree is composed of optimal subtrees
If T is MST of G, and A  T is a subtree of T, and
(u,v) is the min-weight edge connecting A to V-A,
then (u,v)  T
David Luebke
6
7/27/2016
Prim’s Algorithm
MST-Prim(G, w, r)
6
4
9
Q = V[G];
5
for each u  Q
14
2
key[u] = ;
10
15
key[r] = 0;
p[r] = NULL;
3
8
while (Q not empty)
Run on example graph
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
7
7/27/2016
Prim’s Algorithm

MST-Prim(G, w, r)
6
4
9
Q = V[G];
5


for each u  Q
14
2
key[u] = ;
10
15
key[r] = 0;


p[r] = NULL;
3
8

while (Q not empty)
Run on example graph
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
8


7/27/2016
Prim’s Algorithm

MST-Prim(G, w, r)
6
4
9
Q = V[G];
5


for each u  Q
14
2
key[u] = ;
10
15
key[r] = 0;
0
r

p[r] = NULL;
3
8

while (Q not empty)
Pick a start vertex r
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
9


7/27/2016
Prim’s Algorithm

MST-Prim(G, w, r)
6
4
9
Q = V[G];
5



for each u  Q
14
2
key[u] = ;
10
15
key[r] = 0;
0
u


p[r] = NULL;
3
8

while (Q not empty)
u = ExtractMin(Q); Red vertices have been removed from Q
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
10
7/27/2016
Prim’s Algorithm

MST-Prim(G, w, r)
6
4
9
Q = V[G];
5



for each u  Q
14
2
key[u] = ;
10
15
key[r] = 0;
0
u


p[r] = NULL;
3
8
3
while (Q not empty)
u = ExtractMin(Q); Red arrows indicate parent pointers
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
11
7/27/2016
Prim’s Algorithm

MST-Prim(G, w, r)
6
4
Q = V[G];
5
14

for each u  Q
14
2
key[u] = ;
10
key[r] = 0;
0
u

p[r] = NULL;
3
8
3
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
12
9

15

7/27/2016
Prim’s Algorithm

MST-Prim(G, w, r)
6
4
Q = V[G];
5
14

for each u  Q
14
2
key[u] = ;
10
key[r] = 0;
0

p[r] = NULL;
3
8
3
while (Q not empty)
u
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
13
9

15

7/27/2016
Prim’s Algorithm

MST-Prim(G, w, r)
6
4
Q = V[G];
5
14

for each u  Q
14
2
key[u] = ;
10
key[r] = 0;
0
8
p[r] = NULL;
3
8
3
while (Q not empty)
u
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
14
9

15

7/27/2016
Prim’s Algorithm

MST-Prim(G, w, r)
6
4
Q = V[G];
5
10

for each u  Q
14
2
key[u] = ;
10
key[r] = 0;
0
8
p[r] = NULL;
3
8
3
while (Q not empty)
u
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
15
9

15

7/27/2016
Prim’s Algorithm

MST-Prim(G, w, r)
6
4
Q = V[G];
5
10

for each u  Q
14
2
key[u] = ;
10
key[r] = 0;
0
8
p[r] = NULL;
3
8
3
while (Q not empty)
u
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
16
9

15

7/27/2016
Prim’s Algorithm

MST-Prim(G, w, r)
6
4
Q = V[G];
5
10
2
for each u  Q
14
2
key[u] = ;
10
key[r] = 0;
0
8
p[r] = NULL;
3
8
3
while (Q not empty)
u
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
17
9

15

7/27/2016
Prim’s Algorithm

MST-Prim(G, w, r)
6
4
Q = V[G];
5
10
2
for each u  Q
14
2
key[u] = ;
10
key[r] = 0;
0
8
p[r] = NULL;
3
8
3
while (Q not empty)
u
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
18
9
15

15
7/27/2016
Prim’s Algorithm
u

MST-Prim(G, w, r)
6
4
9
Q = V[G];
5
10
2
for each u  Q
14
2
key[u] = ;
10
15
key[r] = 0;
0
8
p[r] = NULL;
3
8
3
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
19

15
7/27/2016
Prim’s Algorithm
u

MST-Prim(G, w, r)
6
4
9
Q = V[G];
5
10
2
for each u  Q
14
2
key[u] = ;
10
15
key[r] = 0;
0
8
p[r] = NULL;
3
8
3
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
20
9
15
7/27/2016
Prim’s Algorithm
u
4
MST-Prim(G, w, r)
6
4
9
Q = V[G];
5
10
2
for each u  Q
14
2
key[u] = ;
10
15
key[r] = 0;
0
8
p[r] = NULL;
3
8
3
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
21
9
15
7/27/2016
Prim’s Algorithm
u
4
MST-Prim(G, w, r)
6
4
9
Q = V[G];
5
5
2
for each u  Q
14
2
key[u] = ;
10
15
key[r] = 0;
0
8
p[r] = NULL;
3
8
3
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
22
9
15
7/27/2016
Prim’s Algorithm
u
4
MST-Prim(G, w, r)
6
4
Q = V[G];
5
5
2
for each u  Q
14
2
key[u] = ;
10
key[r] = 0;
0
8
p[r] = NULL;
3
8
3
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
23
9
15
9
15
7/27/2016
Prim’s Algorithm
u
4
MST-Prim(G, w, r)
6
4
Q = V[G];
5
5
2
for each u  Q
14
2
key[u] = ;
10
key[r] = 0;
0
8
p[r] = NULL;
3
8
3
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
24
9
15
9
15
7/27/2016
Prim’s Algorithm
4
MST-Prim(G, w, r)
6
4
Q = V[G];
5
5
2
for each u  Q
14
2
key[u] = ;
10
key[r] = 0;
0
8
p[r] = NULL;
3
8
3
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
25
u
9
15
9
15
7/27/2016
Prim’s Algorithm
4
MST-Prim(G, w, r)
6
4
Q = V[G];
5
5
2
for each u  Q
14
2
key[u] = ;
10
key[r] = 0;
0
8
p[r] = NULL;
3
8
3
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
26
9
9
u
15
15
7/27/2016
Review: Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
What is the hidden cost
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
27
in this code?
7/27/2016
Review: Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
DecreaseKey(v, w(u,v));
David Luebke
28
7/27/2016
Review: Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u  Q
key[u] = ; How often is ExtractMin() called?
key[r] = 0;
How
often
is
DecreaseKey()
called?
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
DecreaseKey(v, w(u,v));
David Luebke
29
7/27/2016
Review: Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
What will be the running time?
for each u  Q
key[u] = ;
A: Depends on queue
key[r] = 0;
binary heap: O(E lg V)
p[r] = NULL;
Fibonacci heap: O(V lg V + E)
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
30
7/27/2016
Single-Source Shortest Path

Problem: given a weighted directed graph G,
find the minimum-weight path from a given
source vertex s to another vertex v



“Shortest-path” = minimum weight
Weight of path is sum of edges
E.g., a road map: what is the shortest path from
Chapel Hill to Charlottesville?
David Luebke
31
7/27/2016
Shortest Path Properties

Again, we have optimal substructure: the
shortest path consists of shortest subpaths:

Proof: suppose some subpath is not a shortest path
 There
must then exist a shorter subpath
 Could substitute the shorter subpath for a shorter path
 But then overall path is not shortest path. Contradiction
David Luebke
32
7/27/2016
Shortest Path Properties
Define (u,v) to be the weight of the shortest
path from u to v
 Shortest paths satisfy the triangle inequality:
(u,v)  (u,x) + (x,v)
 “Proof”:
x

u
v
This path is no longer than any other path
David Luebke
33
7/27/2016
Shortest Path Properties

In graphs with negative weight cycles, some
shortest paths will not exist (Why?):
<0
David Luebke
34
7/27/2016
Relaxation

A key technique in shortest path algorithms is
relaxation

Idea: for all v, maintain upper bound d[v] on (s,v)
Relax(u,v,w) {
if (d[v] > d[u]+w) then d[v]=d[u]+w;
}
5
2
9
5
2
Relax
5
David Luebke
2
6
Relax
7
5
35
2
6
7/27/2016
Bellman-Ford Algorithm
BellmanFord()
for each v  V
d[v] = ;
d[s] = 0;
for i=1 to |V|-1
for each edge (u,v)  E
Relax(u,v, w(u,v));
for each edge (u,v)  E
if (d[v] > d[u] + w(u,v))
return “no solution”;
Initialize d[], which
will converge to
shortest-path value 
Relaxation:
Make |V|-1 passes,
relaxing each edge
Test for solution
Under what condition
do we get a solution?
Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w
David Luebke
36
7/27/2016
Bellman-Ford Algorithm
BellmanFord()
for each v  V
d[v] = ;
d[s] = 0;
for i=1 to |V|-1
for each edge (u,v)  E
Relax(u,v, w(u,v));
for each edge (u,v)  E
if (d[v] > d[u] + w(u,v))
return “no solution”;
What will be the
running time?
Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w
David Luebke
37
7/27/2016
Bellman-Ford Algorithm
BellmanFord()
for each v  V
d[v] = ;
d[s] = 0;
for i=1 to |V|-1
for each edge (u,v)  E
Relax(u,v, w(u,v));
for each edge (u,v)  E
if (d[v] > d[u] + w(u,v))
return “no solution”;
What will be the
running time?
A: O(VE)
Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w
David Luebke
38
7/27/2016
Bellman-Ford Algorithm
BellmanFord()
for each v  V
s
d[v] = ;
d[s] = 0;
for i=1 to |V|-1
for each edge (u,v)  E
Relax(u,v, w(u,v));
for each edge (u,v)  E
if (d[v] > d[u] + w(u,v))
return “no solution”;
B
-1
A
2
E
2
3
1
4
C
5
-3
D
Ex: work on board
Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w
David Luebke
39
7/27/2016
Bellman-Ford


Note that order in which edges are processed affects
how quickly it converges
Correctness: show d[v] = (s,v) after |V|-1 passes

Lemma: d[v]  (s,v) always
Initially true
 Let v be first vertex for which d[v] < (s,v)
 Let u be the vertex that caused d[v] to change:
d[v] = d[u] + w(u,v)
 Then d[v] < (s,v)
(s,v)  (s,u) + w(u,v) (Why?)
(s,u) + w(u,v)  d[u] + w(u,v) (Why?)
 So d[v] < d[u] + w(u,v). Contradiction.

David Luebke
40
7/27/2016
Bellman-Ford

Prove: after |V|-1 passes, all d values correct

Consider shortest path from s to v:
s  v1  v 2  v3  v4  v
 Initially,
d[s] = 0 is correct, and doesn’t change (Why?)
 After 1 pass through edges, d[v1] is correct (Why?) and
doesn’t change
 After 2 passes, d[v2] is correct and doesn’t change
…
 Terminates in |V| - 1 passes: (Why?)
 What if it doesn’t?
David Luebke
41
7/27/2016
DAG Shortest Paths

Problem: finding shortest paths in DAG



Bellman-Ford takes O(VE) time.
How can we do better?
Idea: use topological sort
 If
were lucky and processes vertices on each shortest
path from left to right, would be done in one pass
 Every path in a dag is subsequence of topologically
sorted vertex order, so processing verts in that order, we
will do each path in forward order (will never relax
edges out of vert before doing all edges into vert).
 Thus: just one pass. What will be the running time?
David Luebke
42
7/27/2016
Dijkstra’s Algorithm
If no negative edge weights, we can beat BF
 Similar to breadth-first search



Grow a tree gradually, advancing from vertices
taken from a queue
Also similar to Prim’s algorithm for MST

Use a priority queue keyed on d[v]
David Luebke
43
7/27/2016
Dijkstra’s Algorithm
Dijkstra(G)
B
2
10
for each v  V
4
3
A
D
d[v] = ;
d[s] = 0; S = ; Q = V;
5
1
C
while (Q  )
u = ExtractMin(Q); Ex: run the algorithm
S = S  {u};
for each v  u->Adj[]
if (d[v] > d[u]+w(u,v))
Relaxation
Note: this
d[v] = d[u]+w(u,v);
Step
is really a
call to Q->DecreaseKey()
David Luebke
44
7/27/2016
Dijkstra’s Algorithm
Dijkstra(G)
How many times is
for each v  V
ExtractMin() called?
d[v] = ;
d[s] = 0; S = ; Q = V;
How many times is
while (Q  )
u = ExtractMin(Q); DecraseKey() called?
S = S  {u};
for each v  u->Adj[]
if (d[v] > d[u]+w(u,v))
d[v] = d[u]+w(u,v);
What will be the total running time?
David Luebke
45
7/27/2016
Dijkstra’s Algorithm
Dijkstra(G)
How many times is
for each v  V
ExtractMin() called?
d[v] = ;
d[s] = 0; S = ; Q = V;
How many times is
while (Q  )
u = ExtractMin(Q); DecraseKey() called?
S = S  {u};
for each v  u->Adj[]
if (d[v] > d[u]+w(u,v))
d[v] = d[u]+w(u,v);
A: O(E lg V) using binary heap for Q
Can acheive O(V lg V + E) with Fibonacci heaps
David Luebke
46
7/27/2016
Dijkstra’s Algorithm
Dijkstra(G)
for each v  V
d[v] = ;
d[s] = 0; S = ; Q = V;
while (Q  )
u = ExtractMin(Q);
S = S  {u};
for each v  u->Adj[]
if (d[v] > d[u]+w(u,v))
d[v] = d[u]+w(u,v);
Correctness: we must show that when u is
removed from Q, it has already converged
David Luebke
47
7/27/2016
Correctness Of Dijkstra's Algorithm
p2
u
s
p2



x
y
Note that d[v]  (s,v) v
Let u be first vertex picked s.t.  shorter path than d[u]
Let y be first vertex V-S on actual shortest path from su


d[u] > (s,u)
 d[y] = (s,y)
Because d[x] is set correctly for y's predecessor x  S on the shortest path, and
When we put x into S, we relaxed (x,y), giving d[y] the correct value
David Luebke
48
7/27/2016
Correctness Of Dijkstra's Algorithm
p2
u
s
p2
x
y
Note that d[v]  (s,v) v
 Let u be first vertex picked s.t.  shorter path than d[u]
d[u] > (s,u)
 Let y be first vertex V-S on actual shortest path from su
 d[y] = (s,y)
 d[u] > (s,u)
= (s,y) + (y,u) (Why?)
= d[y] + (y,u)
 d[y]
But if d[u] > d[y],
wouldn't have chosen u. Contradiction.
David Luebke
49
7/27/2016

The End
David Luebke
50
7/27/2016
Exercise 1 Feedback

First, my apologies…



Harder than I thought
Too late to help with midterm
Proof by substitution:


T(n) = T(n/2 + n) + n
Most people assumed it was O(n lg n)…why?
 Resembled
proof from class: T(n) = 2T(n/2 + 17) + n
 The correct intuition: n/2 dominates n term, so it
resembles T(n) = T(n/2) + n, which is O(n) by m.t.

Still, if it’s O(n) it’s O(n lg n), right?
David Luebke
51
7/27/2016
Exercise 1: Feedback
So, prove by substitution that
T(n) = T(n/2 + n) + n = O(n lg n)
 Assume T(n)  cn lg n
 Then T(n)  c(n/2 + n) lg (n/2 + n)
 c(n/2 + n) lg (n/2 + n)  c(n/2 + n) lg (3n/2)
…

David Luebke
52
7/27/2016