Transcript slides
Lena Gorelick
joint work with
Y. Boykov
O. Veksler
I. Ben Ayed
A. Delong
E (x)
p
fp xp
1
p , q Ν
x 0 ,1
[ x p xq ]
f
Potts Model
2
E (x)
u
p
p
xp
v
pq
x p xq
x 0 ,1
p , q
v pq 0
v pq
Potts Model
Submodular Energy
global optimum with graphcut (Boros & Hammer, 2002)
3
E (x)
u
p
p
v
xp
pq
x p x q const.
p , q
pq ,
v pq 0
Middlebury
Non-Submodular Energy
NP-hard
Image credit: Carlos Hernandes
4
General energy - NP-hard
Approximate methods:
Global Linearization:
QPBO, TRWS, SRMP (Kolmogorov et al. 2006, 2014)
Local Linearization:
parallel ICM, IPFP (Leordeanu, 2009)
Message Passing: BP (Pearl 1989)
5
QPBO, TRWS, SRMP (Kolmogorov et al. 2006, 2014)
E (x )
Linearize introducing
large number of
variables and constraints
~
min E ( y )
s .t . y C
Solve relaxed LP
or its dual
relaxed y
*
Rounding
integer
x
*
Integrality Gap
6
~
parallel ICM (Leordeanu, 2009)
Et(x)
large steps weak min
IPFP (Leordeanu, 2009)
controls step size by relaxation
E (x )
x t 1
xt
Integrality Gap
{0 ,1}
N
x
Bounded domain of
discrete configurations
7
Local
Submodular
Approximation model
Non-linear
Two ways to control step
size
~
Et(x)
E (x )
xt
x t 1
{0 ,1}
N
x
Bounded domain of
discrete configurations
8
Trust Region
Local submodular approximation
LSA-TR
Auxiliary Functions = Surrogate Functions =
Upper Bounds = Majorize-Minimize
Local submodular upper bound
LSA-AUX
Never leave the discrete domain
9
Trust Region:
Discrete High Order Energies
Gorelick et al. 2012,2013
Relaxed Quadratic Binary Energies
Levenberg Marquardt
Olsson et al. 2008
Hartley & Zisserman 2004
Auxiliary Functions=Surrogate Functions
=Upper Bounds = Majorize-Minimize
Discrete High Order Energies
Narasimhan & Bilmes 2005
Rother et al. 2006
Ben Ayed et al. 2013
10
E (x )
u
p
xp
p
v
E (x)
pq
x p xq
xt
p , q
x
+
E (x) E
sub
(x) E
sup
(x)
11
E (x)
xt
x
E (x) E
sub
(x) E
sup
(x)
12
E (x)
E (x) E
sub
(x) E
sup
(x)
~
E (x)
t
xt
x
Approximate E (x ) around x t
~
sub
E t (x) E ( x )
13
E (x)
E (x) E
sub
(x) E
sup
~
E (x)
t
xt
(x)
x
Approximate E (x ) around x t
Linear
Approximation
~
sub
approx
E t (x) E ( x ) E t
(x)
Submodular function
LSA
14
E
sup
( x ) v pq x p x q ,
pq
v pq 0
15
v pq x p x q
16
xy
0
17
xy
0
0,1
0,0
1
y
0
1,1
1,0
1
x
18
xy
0
1
y
0
1,0
1
x
19
0 x y const
Linear (Unary)
approximation
0,0
1
y
0
1,1
1,0
1
x
20
u x v y const
1
y
0
1
x
21
E (x) E
sub
(x) E
sup
(x)
E (x )
xt
x
22
~
sub
approx
E t (x) E ( x ) E t
(x)
E (x )
~
Et(x)
x t 1
xt
Newton Step
x
23
~
sub
approx
E t (x) E ( x ) E t
(x)
E (x )
~
s.t.
|| x x t || d t
Et(x)
xt
Trust Region
x
Trust Region
Sub-Problem
Constrained
Submodular
NP-hard!
Optimization
24
L t (x) E
sub
(x) E
approx
t
t || x x t ||
Submodular
(x)
Gorelick et al. 2013
Unary Terms
Boykov et al. 2006
t
fixed in each iteration
inversely related to trust region size
adjusted based on quality of approximation
25
26
Binary De-convolution
All pairwise terms supermodular
?
Original Img
Convolved
Convolved+Noise
27
QPBOI
LBP
Noise:
N(0,0.05)
QPBO
(0.1 sec.)
TRWS
TRWS:
5000 iter.
E=65.07
QPBO-I
(0.2 sec.)
E=66.44
SRMP
SRMP:
5000 iter.
E=39.06
LBP
5000 iter.
E=40.15
LSA-AUX
(0.04 sec)
E=34.70
IPFP
(0.4 sec.)
E=32.90
FTR-L
LSA-TR
(0.3 sec.)
E=21.13
28
Image
QPBO
QPBO-I
E= -77.08
LBP
E= -84.54
IPFP
E= 163.25
Repulsion = Reward different
across high
contrast LSA-TR
edges
TRWS labels
SRMP
LSA-AUX
E= -67.21
Potts, v<0
(submodular)
E= -101.61
SRMP
E= -120.03
with edge repulsion, v>0
(non-submodular)
E= -175.05
29
dtf-chinesechar database
Kappes et al., 2013
Input Img Ground Truth
LSA-TR
30
31
Efficient Squared Curvature model –
(Nieuwenhuis et al. 2014, poster on Friday)
Potts Model
Elastica
Heber et al. 2012
90-degree curvature
El-Zehiry&Grady, 2010
Our curvature
Using LSA-TR 32
Two novel discrete optimization methods
Simple, efficient, state-of-art results
The code is available online -
http://vision.csd.uwo.ca/code/
Extensions:
Find new applications
▪ Convexity Shape Prior (in ECCV14)
Alternative optimization framework with LSA
▪ Pseudo-Bounds (in ECCV14)
Please come by our poster
33