Transcript Graph Cut Algorithms for Binocular Stereo with Occlusions

Graph Cut Algorithms for
Binocular Stereo with
Occlusions
Vladimir Kolmogorov, Ramin Zabih
Overview:






Traditional Stereo Methods
Energy Minimization via Graph Cuts
Stereo with Occlusions
Voxel Labeling Algorithm
Pixel Labeling Algorithm
Results and Conclusions
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007
2/21
Traditional Stereo Methods
Traditional Stereo Problem
pixel correspondences  labeling (disparity)
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007
3/21
Traditional Stereo Methods
Disparity
disparity
depth
ground truth disparity
Evelyn Gutschier, Markus Sareika
 disparity ~ depth
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007
4/21
Traditional Stereo Methods
Binocular Stereo





goal is to compute pixels correspondences
traditional stereo problem  pixel labeling problem
advantage: can be solved by graph cuts
problem is formulated as energy term
new goal: find the minimizing labeling
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007
5/21
Traditional Stereo Methods
Energy Function
find labeling
f
f 1 , ... , f P
E f
that minimizes
Dp f p
p P
cost for assigning labels
V p ,q f p , f q
p,q N
smoothness term
we assign the label f p to pixel p when p
of image I corresponds to p + f p in I‘
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007
6/21
Traditional Stereo Methods
Energy Function
• data cost – gives penalty for different intensities
Dp f p
I p
I' p
fp ²
• smoothness term – gives penalty for discontinuities (Potts model)
V f p, fq
other models:
absolute distance
quadratic
Evelyn Gutschier, Markus Sareika
T fp
fq
V min K , f p f q
V min K , f p f q ²
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007
7/21
Energy Minimization via Graph Cuts
Max-flow / Min-Cut
(Ford and Fulkerson Algorithm, Push-Relabel Method)
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007
8/21
Energy Minimization via Graph Cuts



V ,
0
convex V
V ,
vs. metric / semimetric V ,
V ,
V ,
V
α-β-swap move
α-expansion move: assigning label α to an
arbitrary set of pixels
Initial Labeling
Evelyn Gutschier, Markus Sareika
α-β-swap
,
α-expansion
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007
9/21
Stereo with Occlusions
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007
10/21
Stereo with Occlusions

treat input symmetrically

scene elements only visible in single view

physically correct scenes
 geometric constraints  occlusions 
physically possible labelings

introduce constraints in the problem formulation

graph cuts perform unconstrained energy minimization
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007
11/21
Voxel Labeling Algorithm

discrete scene of voxels

voxel v is active when
visible from both cameras

uniqueness constraint –
1:1 correspondence of
pixels
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007
12/21
Voxel Labeling Algorithm
Energy Function
E g
Edata g
matching penalty
(only active voxels)
Cocc Pocc g
occlusion penalty
Evelyn Gutschier, Markus Sareika
Eocc g
Esmooth g
smoothness term
(Potts model)
set of occluded pixels
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007
13/21
Pixel Labeling Algorithm
Energy Function
E f
Edata
f p
v
p ,q
N
Evelyn Gutschier, Markus Sareika
Edata f
f q
active ?
d v
Esmooth
D v
f
like traditional stereo
but for both images
e.g. Potts model
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007
14/21
Minimizing the Energy
E g
Edata g
Esmooth g
Evalid g
(0=valid, else ∞)
uniqueness

convert constrained into unconstrained minimization problem




write as sum over pairs
form of energy function = standard stereo problem
minimization with α-expansion algorithm
modified definition of α-expansion move for voxel labeling
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007
15/21
Results and Conclusions
ground truth
Tsukuba ref. image
traditional s.p.
Evelyn Gutschier, Markus Sareika
voxel labeling
pixel labeling
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007
16/21
Results and Conclusions




efficient energy minimization
polynominal time instead of exponential time
traditional stereo algorithm is faster
pixel labeling better than voxel labeling:
 prohibits ‚holes‘ in the scene
 allows to use other effective smoothness terms
algorithms can be extended for multiple cameras
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007
17/21
Multi-view Stereo via Volumetric
Graph Cuts
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007
18/21
Recent Work
Graph-cut-based stereo matching using image segmentation with
symmetrical treatment of occlusions, 2006 TUW
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007
19/21
Questions?
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007
20/21
References








M. Bleyer, M. Gelautz, „Graph-cut-based stereo matching using image segmentation
with symmetrical treatment of occlusions“, 2007
Y. Boykov, O. Veksler, R. Zabih, „Fast Approximate Energy Minimization via Graph
Cuts“, 2001
V. Kolmogorov, R. Zabih, „Graph Cut Algorithms for Binocular Stereo with
Occlusions“, 2005
V. Kolmogorov, R. Zabih, „What energy functions can be minimized via graph cuts“,
2004
V. Kolmogorov, R. Zabih, „Generalized multi-camera scene reconstruction using
graph cuts“, July 2003
V. Kolmogorov, R. Zabih, „Multi-camera Scene Reconstruction via Graph Cuts“, 2002
S. Seits, C. Dyer, „Photorealistic Scene Reconstruction by Voxel Coloring“, 1997
R.Szeliski, R. Zabih, „An Experimental Comparison of Stereo Algorithms“, 1999
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007
21/21