Transcript Stereo Vision using PatchMatch Algorithm

Stereo Vision using
PatchMatch Algorithm
Junkyung Kim
Class of 2014
1. Updates
Updates
• Source code written in C
• Re-implemented PatchMatch in MATLAB
• With some additional features (3)
Updates
• Additional Features
• 1. Ability to use weighted distance measures
– Uniform by default
– RBF, Gabor, Vertical Bar
– (hopefully) Reduce ambiguity / being caught in local
minimum
Updates
• Additional Features
• 2. Ability to choose sampling distribution
– Exponential distribution by default
– U, G, chi-squared, etc
– (hopefully) accelerate convergence to optimal NNF
Updates
• Additional Features
• 3. Ability to further restrict search space
– ‘Disparity range’ is smaller than the entire epipolar line.
– (Surely) accelerate convergence & avoid false matches
– User – defined parameter. Hybrid System.
2. PatchMatch on Stereo
PatchMatch on Stereo
• = Constrained correspondence search
algorithm
– Entire image -> Epipolar line
– Epipolar line -> User-defined disparity band
(additional feature)
PatchMatch on Stereo
• Pipeline
– 1. Random Initialization
– 2. Propagation
– 3. Random Search
• Repeat stages 2. and 3. for fixed number of
iterations
– Or until threshold error is reached
PatchMatch on Stereo
• 1. Random Initialization
– Within the search space (epipolar line by default),
– randomly assign offset between two images
PatchMatch on Stereo
• 1. Random Initialization
– Within the search space (epipolar line by default)
– randomly assign offset between two images
PatchMatch on Stereo
• 2. Propagation
– Compare current offset and those of previous
pixels
• One pixel above & One pixel to the left
PatchMatch on Stereo
• 2. Propagation
– Compare current offset and those of previous
pixels
• One pixel above & One pixel to the left
PatchMatch on Stereo
• 2. Propagation
– Compare current offset and those of previous
pixels
• One pixel above & One pixel to the left
PatchMatch on Stereo
• 2. Propagation
– Compare current offset and those of previous
pixels
• One pixel above & One pixel to the left
PatchMatch on Stereo
• 2. Propagation
– Choose offset that minimizes distance measure
PatchMatch on Stereo
• 3. Random Search
– Sample candidate offsets in exponential
distribution, centered at the current offset
PatchMatch on Stereo
• 3. Random Search
– Sample candidate offsets in exponential
distribution, centered at the current offset
PatchMatch on Stereo
• 3. Random Search
– Sample candidate offsets in exponential
distribution, centered at the current offset
PatchMatch on Stereo
• 3. Random Search
– Choose offset that minimizes distance measure
PatchMatch on Stereo
• The algorithm is bound to reach the global
minimum
– As defined by global minimum of distance
measure (L2 – distance by default)
PatchMatch on Stereo
• Exhaustive search achieves the same solution
• Why use random sampling instead of brute-
force?
– Expected to quickly converge the near-optimum
in small number of iterations.
PatchMatch on Stereo
• How quickly does this algorithm approach
global minimum?
3. Results
PatchMatch on Stereo
PatchMatch on Stereo
PatchMatch on Stereo
PatchMatch on Stereo
PatchMatch on Stereo
PatchMatch on Stereo
PatchMatch on Stereo
PatchMatch on Stereo
PatchMatch on Stereo
Black : Baseline / Green : Constrained disp, propagation off / Blue :
Constrained disp, propagation on / Red : Exhaustive Search
4. Comments &Future Work
Comments &Future Work
• Propagation is certainly helping with speed
– Smoothness constraint holds
• Practical advantage in constraining disparity
– Stereo cameras (bumblebee, etc.) usually have
fixed vergence point (+inf).
– No need to consider uncrossed disparity
Comments &Future Work
• PatchMatch achieves global optimum with
enough iterations
– Minimum of 5 with constraints
• How good is the global optimum compared to
ground truth?
– There must be good amount of error
Comments &Future Work
• Orientation disparity
• Half occlusion
• Distortion
• Other tricks can deal with those issues better
Comments &Future Work
• For project final
– Try feature map for representation
• Oriented edges / bars will handle orientation disparity
better, more robust to distortion
– Try different sampling distributions
• Giving more weight at the center will increase accuracy
along the physical contours
PatchMatch on Stereo
• Pipeline
– 1. Random Initialization
– 2. Propagation
– 3. Random Search
• Repeat stages 2. and 3. for fixed number of
iterations
– Or until threshold error is reached
End