Transcript Algorithms and Data Structures

Algorithms and Data
Structures
Lecture XII
Simonas Šaltenis
Aalborg University
[email protected]
November 13, 2003
1
This Lecture


Weighted Graphs
Minimum Spanning Trees



Greedy Choice Theorem
Kruskal’s Algorithm
Prim’s Algorithm
November 13, 2003
2
Spanning Tree

A spanning tree of G is a subgraph which



is a tree
contains all vertices of G
How many edges
are there in a
spanning tree, if
there are V
vertices?
November 13, 2003
3
Minimum Spanning Trees




Undirected, connected graph
G = (V,E)
Weight function W: E  R
(assigning cost or length or
other values to edges)
Spanning tree: tree that connects all the vertices
Optimization problem – Minimum spanning
tree (MST): tree T that connects all the vertices
and minimizes w(T )   w(u, v)
( u ,v )T
November 13, 2003
4
Optimal Substructure
MST(G’) = T – (u,v)
MST(G) = T
u

v
”u+v”
”Cut and paste” argument

If G’ would have a cheaper ST T’, then we would get
a cheaper ST of G: T’ + (u, v)
November 13, 2003
5
Idea for an Algorithm


We have to make V – 1 choices (edges of
the MST) to arrive at the optimization goal
After each choice we have a sub-problem
one vertex smaller than the original


Dynamic programming algorithm, at each
stage, would consider all possible choices
(edges)
If only we could always guess the correct
choice – an edge that definitely belongs to an
MST
November 13, 2003
6
Greedy Choice


Greedy choice property: locally optimal
(greedy) choice yields a globally optimal
solution
Theorem



Let G=(V, E), and let S V and
let (u,v) be min-weight edge in G connecting S
to V – S : a light edge crossing a cut
Then (u,v)  T – some MST of G
November 13, 2003
7
Greedy Choice (2)

Proof




Suppose (u,v) is light, but (u,v)  any MST
look at path from u to v in some MST T
Let (x, y) – the first edge on path from u to v in T that
crosses from S to V – S. Swap (x, y) with (u,v) in T.
this improves T – contradiction (T supposed to be an
MST)
V-S
S
x
u
November 13, 2003
y
v
8
Generic MST Algorithm
Generic-MST(G, w)
1 A// Contains edges that belong to a MST
2 while A does not form a spanning tree do
3
Find an edge (u,v) that is safe for A
4
AA{(u,v)}
5 return A
Safe edge – edge that does not destroy A’s property
MoreSpecific-MST(G, w)
1
A// Contains edges that belong to a MST
2
while A does not form a spanning tree do
3.1
Make a cut (S, V-S) of G that respects A
3.2
Take the min-weight edge (u,v) connecting S to V-S
4
AA{(u,v)}
5 return A
November 13, 2003
9
Pseudocode assumptions

Graph ADT with an operation





V():VertexSet
E(): EdgeSet
w(u:Vertex, v:Vertex):int – returns a weight of (u,v)
A looping construct “for each v V ”, where V is
of a type VertexSet, and v is of a type Vertex
Vertex ADT with operations:



adjacent():VertexSet
key():int and setkey(k:int)
parent():Vertex and setparent(p:Vertex)
November 13, 2003
10
Prim-Jarnik Algorithm




Vertex based algorithm
Grows one tree T, one vertex at a time
A set A covering the portion of T already
computed
Label the vertices v outside of the set A
with v.key() – the minimum weigth of an
edge connecting v to a vertex in A, v.key()
= , if no such edge exists
November 13, 2003
11
Prim-Jarnik Algorithm (2)
MST-Prim(G,r)
01
02
03
04
05
06
07
08
09
10
11
12
for each vertex u  G.V()
u.setkey()
u.setparent(NIL)
r.setkey(0)
Q.init(G.V())
// Q is a priority queue ADT
while not Q.isEmpty()
u  Q.extractMin()
// making u part of T
for each v  u.adjacent() do
if v  Q and G.w(u,v) < v.key() then
v.setkey(G.w(u,v))
Q.modifyKey(v)
v.setparent(u)
November 13, 2003
updating
keys
12
Prim-Jarnik’s Example

Let’s do an example:
B
4
A
12
8
8
C
3
6
H
4
D
5
3
9
13
I
1

6
F
10
E
G
Let’s keep track:

What is set A, which vertices are in Q, what
cuts are we making, what are the key values
November 13, 2003
13
Implementation issues
MST-Prim(G,r)
01
02
03
04
05
06
07
08
09
10
11
12
for each vertex u  G.V()
u.setkey()
u.setparent(NIL)
r.setkey(0)
Q.init(G.V())
// Q is a priority queue ADT
while not Q.isEmpty()
u  Q.extractMin()
// making u part of T
for each v  u.adjacent() do
if v  Q and G.w(u,v) < v.key() then
v.setkey(G.w(u,v))
Q.modifyKey(v)
v.setparent(u)
November 13, 2003
14
Priority Queues

A priority queue is an ADT for maintaining a set S
of elements, each with an associated value called
key

We need PQ to support the following operations




init(S:VertexSet)
extractMin():Vertex
modifyKey(v:Vertex)
To choose how to implement a PQ, we need to
count how many times the operations are
performed:

init is performed just once and runs in O(n)
November 13, 2003
15
Prim-Jarnik’s Running Time


Time = |V|T(extractMin) + O(E)T(modifyKey)
Binary heap implementation:

Time = O(V lgV + E lgV) = O(E lgV)
Q
T(extractMin) T(modifyKey)
array
O(V)
binary heap O(lg V)
Fibonacci
O(lg V)
heap
November 13, 2003
Total
O(1)
O ( V 2)
O(lg V)
O(E lgV )
O(1) amortized O(V lgV +E )
16
Greedy Algorithms


One more algorithm design technique
Greedy algorithms can be used to solve
optimization problems, if:


There is an optimal substructure
We can prove that a greedy choice at each
iteration leads to an optimal solution.
November 13, 2003
17
Kruskal's Algorithm



Edge based algorithm
Add the edges one at a time, in increasing
weight order
The algorithm maintains A – a forest of
trees. An edge is accepted it if connects
vertices of distinct trees (the cut respects A)
November 13, 2003
18
Disjoint-Set ADT

We need a Disjoint-Set ADT that maintains a
partition, i.e., a collection S of disjoint sets.
Operations:

makeSet(x:Vertex): SetID


union(x:Vertex, y:Vertex) –




S  S  {{x}}
X  S.findSet(x)
Y  S.findSet(y)
S  S – {X, Y}  {X Y}
findSet(x:Vertex): SetID

returns unique X  S, where x  X
November 13, 2003
19
Kruskal's Algorithm

The algorithm keeps adding the cheapest
edge that connects two trees of the forest
MST-Kruskal(G)
01
02
03
04
05
06
07
08
09
10
A  
S.init()
// Init disjoint-set ADT
for each vertex v  G.V()
S.makeSet(v)
sort the edges of G.E() by non-decreasing G.w(u,v)
for each edge (u,v)  E, in sorted order do
if S.findSet(u)  S.findSet(v) then
A  A  {(u,v)}
S.union(u,v)
return A
November 13, 2003
20
Kruskal’s Example

Let’s do an example:
B
4
A
12
8
8
C
3
6
H
4
D
5
3
9
13
I
1

6
F
10
E
G
Let’s keep track:

What is set A, what is a collection S , what
cuts are we making
November 13, 2003
21
Disjoint Sets as Lists



Each set – a list of elements identified by the first
element, all elements in the list point to the first
element
Union – add a smaller list to a larger one
FindSet: O(1), Union(u,v): O(min{|C(u)|, |C(v)|})
1
2
3
4

1
2
November 13, 2003
3
4
A
B
C
A
B
C


22
Kruskal Running Time




Initialization O(V) time
Sorting the edges Q(E lg E) = Q(E lg V) (why?)
O(E) calls to FindSet
Union costs




Let t(v) be the number of times v is moved to a new
cluster
Each time a vertex is moved to a new cluster the size
of the cluster containing the vertex at least doubles:
t(v) log V
Total time spent doing Union  t (v)  V log V
Total time: O(E lg V)
November 13, 2003
vV
23
Next Lecture

Shortest Paths in Weighted Graphs
November 13, 2003
24