Transcript Belief Propagation algorithm in Markov Random Fields

Oct 15 2009
CS774. Markov Random Field :
Theory and Application
Lecture 13
Kyomin Jung
KAIST
The Labeling Problem in Vision
Common idea behind many Computer Vision problems
In the presence of uncertainties, find the best Labeling !
Computing MAP of a corresponding MRF (it is called an
Energy minimization problem in the vision community)
(Stereo, 3D Reconstruction, Segmentation, Image Restoration)
Image Segmentation and Graph Cuts



An image segmentation problem can be interpreted
as partitioning the image elements (pixels) into
different categories.
A Cut of a graph is a partition of the vertices in the
graph into two disjoint subsets.
By constructing a graph with an image, we can
solve the segmentation problem using techniques
for graph cuts in graph theory.
Energy minimization via Graph cut
Simple Example : 2-label case
If an pairwise Energy ftn is submodular, i.e. for all edge potentials,
Eij(0,0) + Eij(1,1) ≤ Eij(0,1) + Eij(1,0)
can compute Minimum energy and a corresponding labeling via Min Cut.
Min Cut problem

2–label minimization can be computed by solving
max-flow (s-t cut) exactly in polynomial time.

Multi-way cut is NP-Hard for ≥ 3 labels.
 approximation algorithm for multi-label
minimization.
Graph Cuts
Partition the graph into two parts separating red and blue
nodes
s-t graph cut
“source”
S
“sink”
T
A graph with two terminals S and T
• Cut cost is a sum of edge weights connecting two colors
Flow Network


A flow network is defined as a directed graph G where
an edge has a nonnegative capacity.
A flow in G is a real-valued (often integer) function that
satisfies the following three properties:
 Capacity Constraint:
For all u, v  V , f (u, v)  c(u, v)
 Skew Symmetry
For all u, v  V , f (u, v)   f (v, u)
 Flow Conservation
For all u  (V \ {s, t}),  f (u, v)  0
vV
where s is the source, t is the sink node.
Graph Cuts & Network Flow
The Max-flow Min-Cut Theorem
If f is a maximum flow,
| f | = c (S,T ) for some cut (S,T ) of G
The cost of the minimum s-t cut = the maximum flow.
Thus, we will find minimum s-t cuts in graphs by
solving for max-flow.
Graph Cut based
Segmentation
User Guided Segmentation:
hard
constraint
t
n-links
s
a cut
hard
constraint
Ford-Fulkerson Algorithm

Main Operation
Starting from zero flow, increase the flow gradually
by finding a path from s to t along which more flow
can be sent, until a max-flow is achieved.
The path for flow to be pushed through is called an
augmenting path.
Ford-Fulkerson Algorithm



The Ford-Fulkerson algorithm uses a residual network of
flow in order to find the solution.
The residual network is defined as the network of edges
containing flow that remains after a fixed flow.
For example, in the graph shown below, there is an initial
path from the source to the sink, and the middle edge has a
total capacity of 3, and a residual capacity of 3-1=2.
0/2
0/1
s
1/2
t
1/3
1/2
0/2
0/2
Algorithm Framework
The basic Ford-Fulkerson algorithm
for each edge
do
(u, v)  E
f (u, v)  0
f (v, u )  0
while there exists a path P from s to t
in the residual network Gf
do cf (P) ← min{cf (u, v ): (u, v) is on P}
for each edge (u, v) in P
do
f (u, v)  f (u, v)  c f ( P)
f (v, u)   f (u, v)
Algorithm Execution Example
f=0
0/2
0/2
0/1
s
0/2
1
1
t
0/3
s
2
t
2
1
0/2
(a)
2
(d)
f=0
2
1
1
1
t
2
f=3
1
2
3
s
2
2
2
2
(b)
2
1
t
1
1
1
1
(e)
1
1
2
1
1
0/2
s
f=2
1
f=1
1/2
1
1
f=3
2/2
1/1
1
s
t
3
2
2
2
(c)
s
t
1/3
2/2
1/2
1/2
(f)
Finding the Min-Cut

After the max-flow is found, the minimum cut is
determined by
S = {all vertices reachable from s in the residual network}
T = V-S
1
1
1
s
2
2
2
1
1
1
t
1
1
Ford-Fulkerson Algorithm Analysis



The running time of the algorithm depends on how the aug
menting path is determined.
If the searching for augmenting path is realized by a breadth
-first search, the algorithm runs in O ( E |fmax| ).
Under some extreme cases the efficiency of the algorithm can
be reduced drastically. One example is shown in the figure be
low, applying Ford-Fulkerson algorithm needs 400 iterations
to get the max flow of 400.
200
v1
200
1
s
200
v2
t
200