Transcript PowerPoint 프레젠테이션

Fast Census Transform-based Stereo
Algorithm using SSE2
Young Ki Baik*
Kyoung Mu Lee
Computer Vision Lab.
School of Electrical Engineering and Computer Science
Seoul National University
Fast Census Transform-based Stereo Algorithm using SSE2
Contents
Stereo Vision
Census Transform Stereo Vision
Fast approaches
Experimental result
Conclusion and Future work
FCV 2006
Fast Census Transform-based Stereo Algorithm using SSE2
Introduction
What is the stereo vision?
The stereo vision is the method to extract 3D information
using image from different view points.
Topographical survey
Obstacle detection
Object tracking
Face recognition
FCV 2006
Fast Census Transform-based Stereo Algorithm using SSE2
Introduction
Trade off of algorithms
Algorithm for accurate results
Complex computation and iteration
Slow processing time
Unable to realize real-time system
Algorithm for fast processing time
Simple computation and no iteration
Fast processing time
Unable to realize accurate system
FCV 2006
Fast Census Transform-based Stereo Algorithm using SSE2
Introduction
Fast stereo vision algorithm
Window size invariant method
Box filtering method

“Box-filtering techniques”, M.J.McDonnell (CGIP-81’)

“Real time correlation-based stereo : algorithm, implementations
and applications”, Olivier Faugeras , Zhengyou Zhang , …
(Tech.Rep.RR-2013, INRIA,1993)
Disparity range invariant method
Rectangular subregioning method

“Rectangular Subregioning and 3-D Maximum-Surface Techniques
for Fast Stereo Matching”, Changming Sun (CVPR-2001)
Parallel processing technique
FCV 2006
Fast Census Transform-based Stereo Algorithm using SSE2
Introduction
Problem
Real images from grabbers can not assure of brightness
consistency in corresponding region.
Intensity correlation method is not proper for real images.
Census transform
Census transform has been evaluated as the method robust to
radiometric distortion.
J. Banks and P. Corke, "Quantitative evaluation of matching methods
and validity measures for stereo vision," Int. J. Robotics Research, vol.
20, pp. 512-532, July 2001.
Heiko Hirschmller, "Improvements in Real-Time Correlation Based
Stereo Vision", Proceedings IEEE Workshop on Stereo and MultiBaseline Vision, pp. 141-148, Kauai, Hawaii, December 2001.
FCV 2006
Fast Census Transform-based Stereo Algorithm using SSE2
Census Transform Stereo Vision
Census transform
Census transform converts relative intensity difference to
0 or 1 and deforms 1 dimensional vector as much as
window size of census transform.
121
130
26
31
39
1
1
0
0
0
109
115
33
40
30
1
1
0
0
0
98
102
78
67
45
1
1
X
0
0
47
67
32
170 198
0
0
0
1
1
39
86
99
159 210
1
1
1
1
1
Census transform window (CTW)
1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 0 0 1 1 1 1 1 1 1
FCV 2006
Fast Census Transform-based Stereo Algorithm using SSE2
Census Transform Stereo Vision
Result of census transform
Census transform makes data of (image size * vector size).
(Square size of CTW)-1
Height
Height
Width
Width
FCV 2006
Fast Census Transform-based Stereo Algorithm using SSE2
Census Transform Stereo Vision
Sum of Hamming distance
The Hamming distance of two transformed vectors with
correlation windows is used to find corresponding region.
Sum of Hamming distance
Disparity
range
Right census
Left census
transformed vector
transformed vector
3D disparity space
FCV 2006
Fast Census Transform-based Stereo Algorithm using SSE2
Census Transform Stereo Vision
Complexity of algorithm
Census transform-based stereo vision (CTSV) has high
complexity.
Method
Conventional
stereo vision
Census transform
stereo vision
Complexity
ND
CND
N : Searching window size
D : Disparity range
C : Census transform window size
FCV 2006
Fast Census Transform-based Stereo Algorithm using SSE2
Fast Census Transform Stereo Vision
Fast Census Transform Stereo Vision
Census transform
Parallel processing
Hamming distance
- SSE2
8bit look-up table
Correlation
Parallel processing
- SSE2
Moving window
technique
Parallel processing
- SSE2
FCV 2006
Fast Census Transform-based Stereo Algorithm using SSE2
Fast Census Transform Stereo Vision
SSE2 (Streaming SIMD Extension 2)
128-bit SIMD packed integer & floating point arithmetic
operation
Cache and memory management operation
Continuous memory structure is required
No advantages in separate data
2 x D o ub le
XM M 7
XM M 6
16 x B Y TE
8 x W O RD
XM M 5
XM M 4
XM M 3
4 x D W O RD
XM M 2
XM M 1
2 x Q W RO D
XM M 0
FCV 2006
Fast Census Transform-based Stereo Algorithm using SSE2
Fast Census Transform Stereo Vision
Fast approaches (census transform)
Usage of parallel processing (SSE2)
16 pixels are loaded to XMM(SSE2 memory) and computed at once.
16
16
16
16
FCV 2006
Fast Census Transform-based Stereo Algorithm using SSE2
Fast Census Transform Stereo Vision
Fast approaches (sum of Hamming distance)
Usage of 8bit look-up table (LUT)
Parallel processing : SSE2
Parallel processing is faster than 8 bit LUT
1 1 0 1 0 0 1 0
1 1 0 1 0 0 1 0
8-bit Look-up
Table
Shift & Bitwise
AND Operation
8 times
4
Sum Up
Number or 1
FCV 2006
Fast Census Transform-based Stereo Algorithm using SSE2
Fast Census Transform Stereo Vision
Fast approaches (correlation)
Moving window technique
x
y
(a)
(b)
Addition
Addition and
Subtraction
Total sum of 2D
window value
Initial value
(c)
(d)
FCV 2006
Fast Census Transform-based Stereo Algorithm using SSE2
Fast Census Transform Stereo Vision
Fast approaches (correlation)
Combination of moving window and SIMD
Moving window technique for x, y-axis
SSE2 for d-axis
d
y
x
Mo
Wi ving
nd
ow
SIM
D
FCV 2006
Fast Census Transform-based Stereo Algorithm using SSE2
Experimental result
Environment
System : Pentium-IV 2.4GHz
Cache memory : 512Kbyte
Camera : Stereo Mega-D (Videre design)
Condition
Image size : 320 x 240 gray stereo images
Census transform window size :5x5, 7x7, 9x9
Disparity searching range : 32
Correlation window size : 11x11
FCV 2006
Fast Census Transform-based Stereo Algorithm using SSE2
Experimental result
Detail processing time
32 disparity searching range
Time (msec)
Census
Method
Census
with
SSE2
Census
without
SSE2
Frame
Transform
Window
CT
Cal HD
Corr
Total
Rate
5
5.9
17.5
5.8
29.2
34.2
7
11.6
29.6
5.6
46.8
21.4
9
35.4
52.6
4.3
92.3
10.8
5
34.3
24.1
14.2
72.5
13.8
7
66.2
35.1
14.1
115.4
8.7
9
208.3
66.5
14.2
289.0
3.5
FCV 2006
Fast Census Transform-based Stereo Algorithm using SSE2
Experimental result
Performance of census transform stereo vision
35
30
25
Frame Rate
No SSE2
SSE2
20
15
10
5
0
SSE2
5
7
No SSE2
9
Census Window Size
FCV 2006
Fast Census Transform-based Stereo Algorithm using SSE2
Conclusion and Future work
Conclusion
Moving window technique reduces processing time to constant
except in transforming stage
SSE2 instructions reduces running time by 2.5 to 3 times
Possibility for faster result
Specialized Instruction
16 bit look-up table
Fixed window size of census transform
Future work
Applying real-time approach to another stereo algorithm
Combine stereo system to other applications
FCV 2006