Transcript Belief Propagation algorithm in Markov Random Fields

Nov 05 2009
CS774. Markov Random Field :
Theory and Application
Lecture 17
Kyomin Jung
KAIST
Remind: MRF for
Maximum Weight Independent Set (MWIS)



Given a graph G=(V,E), a subset I of V is called an Independent Set, if for
all e  E , the two end points of e does not belong to I simultaneously.
When the vertices are weighted, an independent set I is called MWIS if the
sum of the weights of v  I is maximum.
Finding a MWIS is equivalent to finding a MAP in the following
MRF on X {0,1}|V |


P[ X  x]  expWv  xv    ( xu , xv ),
 vV
 (u ,v )E
where ( x1 , x2 ) 

0
1
if x1  x2  1
otherwise
,
and
Wv is the weight at node v.
MRF for Combinatorial Optimization
Example: Maximum Independent Set (MIS).
Input: A graph.
Feasible solution: A set S of vertices.
Value of solution: |S|.
Objective: Maximize.
Similar example: Max-cut, vertex coloring problem
NP-hardness


Finding the optimal solution is NP-hard.
Practical implication: no polynomial time algorithm
always finds optimum solution.

Approximation algorithms: polynomial time,
guaranteed to find “near optimal” solutions for
every input.

Heuristics: useful algorithmic ideas that often work,
but fail on some inputs. (ex, Belief Propagation)
Approximation Ratio
For maximization problems (ex maxcut):
Approximation Algorithms

Many approximation algorithms are designed.

A large variation between approximation ratios of
different problems.
 FPTAS (Fully Polynomial Time Approximation
Scheme)
 PTAS (Polynomial Time Approximation Scheme)
 Constant ratio, super-constant…
K-Cliques
A K-clique is a set of K nodes with all K(K-1)/2
possible edges between them
This graph contains a 4-clique
K-Cliques
Given: (G, k)
Question: Does G contain a k-clique?
Clique / Independent Set
Two problems are essentially the same
Complement of G
Given a graph G, let G*, the complement of G, be
the graph such that
two nodes are connected in G* if and only if the
corresponding nodes are not connected in G
G
G*
Vertex Coloring Problem




The assignment of labels or colors to the vertices of a
graph.
Each edge has different color at its end nodes.
This problem can be expressed by MRF so that # of total
coloring is the same as the partition function.
The smallest number of colors needed to color a graph G
is called its chromatic number χ(G).
Hardness of MIS and Coloring Problem

For any constant ε > 0 there is no polynomial-time
n1−ε -approximation algorithm for the maximum
independent set problem unless P = NP.

For any constant ε > 0 there is no polynomial-time
n1−ε -approximation algorithm for computing the
chromatic number of G unless P = NP.

Hence we consider some practical and restricted class
of graphs, like planar graph and unit disk graph.
Definition – Planar Graph
Grid Minors for Planar Graphs

r
r  r grid:
 r2
vertices
 Treewidth = r

r
r  r grid is the canonical planar graph of
treewidth Θ(r) :
 every
planar graph of treewidth w has an
Ω(w)  Ω(w) grid minor [Robertson, Seymour,
Thomas, 1994]
PTAS for Independent set on planar
graphs

Method originated by Baker (1994)

Extended by several authors to more
general/other classes of graphs
e.g. SODA 2005: Demaine & Hajigitani – more
problems and minor closed classes of graphs

k-outerplanar graphs





Label vertices of a plane graph by
level.
All vertices on exterior face level 1.
All vertices on exterior face when
vertices of levels 1 … i are removed,
are on level i+1.
Graph is k-outerplanar when at
most k levels.
Theorem: k-outerplanar graphs
have treewidth at most 3k – 1.
3-outerplanar
Independent set on k-outerplanar graphs

For fixed k, finding a maximum independent set
in a k-outerplanar graph can be solved in linear
time (approximately 8kn time).
 By dynamic programming using treedecomposition
Baker’s scheme

For each i in {1,2, …, k} do
 Remove all vertices in levels i, i+k, i+2k, i+3k, …
 Each connected component of the remaining
graph is (k-1)-outerplanar.
 Solve independent set exactly on the
remaining graph.

Output the best of the k obtained independent
sets.
Approximation Ratio

Look at a maximum independent set S.

Each of the k runs deletes a different subset of S.

So, there is a run that deletes at most |S|/k
vertices from S
 one
of the runs gives an answer that is at least (k-1)/k
times the size of the optimum.

This gives a PTAS.
Unit Disk Graph
A
unit disk graph is the intersection graph of a
set of unit disks in the Euclidean plane.

Two disks have edge when they intersect.
 There
exists a PTAS for the MIS (selecting disjoint
disks).
 The
idea is similar to that of Baker.
Independent Set of Disk Graph
Independent set (Greedy)
0
1
2
3
4
5
6
7
8
Independent set (Greedy)
0
1
2
3
4
5
6
7
8
Independent set (Greedy)
0
1
2
3
4
5
6
7
8
Independent set
0
1
K=2
2
3
4
5
6
7
8
Independent set
0
1
2
3
4
5
6
7
8