Transcript Multiplicative Bounds for Metric Labeling

Multiplicative Bounds
for Metric Labeling
M. Pawan Kumar
École Centrale Paris
Joint work with Phil Torr, Daphne Koller
Metric Labeling
Variables V = { V1, V2, …, Vn}
Metric Labeling
Variables V = { V1, V2, …, Vn}
Metric Labeling
wabd(f(a),f(b))
θb(f(b))
wab ≥ 0
θa(f(a))
d is metric
Va
minf E(f)
Vb
= Σa θa(f(a)) + Σ(a,b) wabd(f(a),f(b))
Variables V = { V1, V2, …, Vn}
Labels L = { l1, l2, …, lh}
Labeling f: { 1, 2, …, n}  {1, 2, …, h}
Metric Labeling
Va
minf E(f)
Vb
= Σa θa(f(a)) + Σ(a,b) wabd(f(a),f(b))
NP hard
Low-level vision applications
Approximate Algorithms















Minka. Expectation Propagation for Approximate Bayesian Inference, UAI, 2001
Murphy et al. Loopy Belief Propagation: An Empirical Study, UAI, 1999
Winn et al. Variational Message Passing, JMLR, 2005
Yedidia et al. Generalized Belief Propagation, NIPS, 2001
Besag. On the Statistical Analysis of Dirty Pictures, JRSS, 1986
Boykov et al. Fast Approximate Energy Minimization via Graph Cuts, PAMI, 2001
Komodakis et al. Fast, Approximately Optimal Solutions for Single and Dynamic
MRFs, CVPR, 2007
Lempitsky et al. Fusion Moves for Markov Random Field Optimization, PAMI, 2010
Chekuri et al. Approximation Algorithms for Metric Labeling, SODA, 2001
Goemans et al. Improved Approximate Algorithms for Maximum-Cut, JACM, 1995
Muramatsu et al. A New SOCP Relaxation for Max-Cut, JORJ, 2003
Ravikumar et al. QP Relaxations for Metric Labeling, ICML, 2006
Alahari et al. Dynamic Hybrid Algorithms for MAP Inference, PAMI 2010
Kohli et al. On Partial Optimality in Multilabel MRFs, ICML, 2008
Rother et al. Optimizing Binary MRFs via Extended Roof Duality, CVPR, 2007
.
.
.
Outline
• Linear Programming Relaxation
• Move-Making Algorithms
• Comparison
• Rounding-based Moves
Integer Linear Program
Minimize a linear function over a set of feasible solutions
Indicator xa(i)  {0,1} for each variable Va and label li
Number of facets grows exponentially in problem size
Linear Programming Relaxation
Indicator xa(i)  {0,1} for each variable Va and label li
Schlesinger, 1976; Chekuri et al., 2001; Wainwright et al., 2003
Linear Programming Relaxation
Indicator xa(i)  [0,1] for each variable Va and label li
Schlesinger, 1976; Chekuri et al., 2001; Wainwright et al., 2003
Outline
• Linear Programming Relaxation
• Move-Making Algorithms
• Comparison
• Rounding-based Moves
Move-Making Algorithms
Space of All Labelings
f
Expansion Algorithm
Variables take label lα or retain current label
Slide courtesy Pushmeet Kohli
Boykov, Veksler and Zabih, 2001
Expansion Algorithm
Variables take label lα or retain current label
Status:
InitializeSky
Expand
Ground
House
with Tree
Slide courtesy Pushmeet Kohli
Tree
Ground
House
Sky
Boykov, Veksler and Zabih, 2001
Outline
• Linear Programming Relaxation
• Move-Making Algorithms
• Comparison
• Rounding-based Moves
Multiplicative Bounds
f*: Optimal Labeling
f: Estimated Labeling
Σa θa(f(a)) + Σ(a,b) sabd(f(a),f(b))
≥
Σa θa(f*(a)) +
Σ(a,b) sabd(f*(a),f*(b))
Multiplicative Bounds
f*: Optimal Labeling
f: Estimated Labeling
Σa θa(f(a)) + Σ(a,b) sabd(f(a),f(b))
≤
Σa θa(f*(a)) +
B Σ(a,b) sabd(f*(a),f*(b))
Multiplicative Bounds
Expansion
LP
Potts
2
2
Truncated
Linear
2M
2 + √2
Truncated
Quadratic
2M
O(√M)
Metric
2M
O(log h)
M = ratio of maximum and minimum distance
Outline
• Linear Programming Relaxation
• Move-Making Algorithms
• Comparison
• Rounding-based Moves
– Complete Rounding
– Interval Rounding
– Hierarchical Rounding
Complete Rounding
Treat xa(i)  [0,1] as probability that f(a) = i
Cumulative probability ya(i) = Σj≤i xa(j)
r
0
ya(1)
ya(2)
ya(i)
Generate a random number r  (0,1]
Assign the label next to r
ya(k)
ya(h) = 1
Complete Move
Va
Vb
θab(i,k) = sabd(i,k)
NP-hard
Complete Move
Va
Vb
d’(i,k) ≥ d(i,k)
θab(i,k) = sabd’(i,k)
d’ is submodular
Complete Move
Va
Vb
d’(i,k) ≥ d(i,k)
θab(i,k) = sabd’(i,k)
d’ is submodular
Complete Move
New problem can be solved using minimum cut
Same multiplicative bound as complete rounding
Multiplicative bound is tight
Outline
• Linear Programming Relaxation
• Move-Making Algorithms
• Comparison
• Rounding-based Moves
– Complete Rounding
– Interval Rounding
– Hierarchical Rounding
Interval Rounding
Treat xa(i)  [0,1] as probability that f(a) = i
Cumulative probability ya(i) = Σj≤i xa(j)
0
ya(1)
ya(2)
ya(i)
Choose an interval of length h’
ya(k)
ya(h) = 1
Interval Rounding
Treat xa(i)  [0,1] as probability that f(a) = i
Cumulative probability ya(i) = Σj≤i xa(j)
r
ya(i)
Choose an interval of length h’
ya(k)
REPEAT
Generate a random number r  (0,1]
Assign the label next to r if it is within the interval
Interval Move
Choose an interval of length h’
Va
Vb
θab(i,k) = sabd(i,k)
Interval Move
Choose an interval of length h’
Add the current labels
Va
Vb
θab(i,k) = sabd(i,k)
Interval Move
Choose an interval of length h’
Add the current labels
d’(i,k) ≥ d(i,k)
d’ is submodular
Solve to update labels
Va
Vb
Repeat until convergence
θab(i,k) = sabd’(i,k)
Interval Move
Each problem can be solved using minimum cut
Same multiplicative bound as interval rounding
Multiplicative bound is tight
Kumar and Torr, NIPS 2008
Outline
• Linear Programming Relaxation
• Move-Making Algorithms
• Comparison
• Rounding-based Moves
– Complete Rounding
– Interval Rounding
– Hierarchical Rounding
Hierarchical Rounding
L1
l1
l2
L2
l3
l4
l5
L3
l6
l7
l8
Hierarchical clustering of labels (e.g. r-HST metrics)
l9
Hierarchical Rounding
L1
l1
l2
L2
l3
l4
l5
L3
l6
l7
Assign variables to labels L1, L2 or L3
Move down the hierarchy until the leaf level
l8
l9
Hierarchical Rounding
L1
l1
l2
L2
l3
l4
l5
Assign variables to labels l1, l2 or l3
L3
l6
l7
l8
l9
Hierarchical Rounding
L1
l1
l2
L2
l3
l4
l5
Assign variables to labels l4, l5 or l6
L3
l6
l7
l8
l9
Hierarchical Rounding
L1
l1
l2
L2
l3
l4
l5
Assign variables to labels l7, l8 or l9
L3
l6
l7
l8
l9
Hierarchical Move
L1
l1
l2
L2
l3
l4
l5
L3
l6
l7
l8
Hierarchical clustering of labels (e.g. r-HST metrics)
l9
Hierarchical Move
L1
l1
l2
L2
l3
l4
l5
L3
l6
l7
Obtain labeling f1 restricted to labels {l1,l2,l3}
l8
l9
Hierarchical Move
L1
l1
l2
L2
l3
l4
l5
L3
l6
l7
Obtain labeling f2 restricted to labels {l4,l5,l6}
l8
l9
Hierarchical Move
L1
l1
l2
L2
l3
l4
l5
L3
l6
l7
Obtain labeling f3 restricted to labels {l7,l8,l9}
l8
l9
Hierarchical Move
L1
L2
L3
f3(a)
f3(b)
f2(a)
f2(b)
f1(a)
f1(b)
Va
Vb
Move up the hierarchy until we reach the root
Hierarchical Move
Each problem can be solved using minimum cut
Same multiplicative bound as hierarchical rounding
Multiplicative bound is tight
Kumar and Koller, UAI 2009
Conclusion
MoveMaking
LP
Potts
2
2
Truncated
Linear
2 + √2
2 + √2
Truncated
Quadratic
O(√M)
O(√M)
Metric
O(log h)
O(log h)
M = ratio of maximum and minimum distance
Open Problems
• Moves for general rounding schemes
• Higher-order energy functions
• Better comparison criterion
Questions?
http://www.centrale-ponts.fr/personnel/pawan