Transcript ppt slides
Semi-automatic Range to Range
Registration: A Feature-based Method
Chao Chen & Ioannis Stamos
Computer Science Department
Graduate Center, Hunter College
The City University of New York
Motivation
Goal: highly accurate photo-realistic description of 3D world
Applications: urban planning, historical preservation, virtual reality
Texture
Mapping
Registration
3D model
Range scans
Photo-realistic 3D Model
Highly accurate
Computational efficient
Minimum human interaction
2D Pictures
2
Contribution
Related methods on range image registration
Iterative Closest Point algorithm
[Johnson & Hebert]
More suitable for curved surfaces; needs accurate normal
Previous feature-based algorithm
Require close initial registration
Spin images
[Besl & McKay, Chen & Medioni, Rusinkiewicz]
[Stamos & Leordeanu]
Exhaustively searching line pairs; high complexity
Symmetric structures require manual registration
Our semi-automatic 3D registration system
No rough pre-registration required
Automated registration procedure:
Utilize global information to compute transformation
ICP algorithm to optimize registration
Context-sensitive user interface to:
Display registration result at each step
Conveniently adjust translation and rotation
3
Outline
Automated registration procedures
Previous exhaustive search approach
Improved automated registration
Global stitching process to register all images
User interface
Experimental results
Conclusions and future work
4
Exhaustive Search Approach
Range image segmentation
Intersection
line
Each Segmented
Planar Area is
shown with different
color for clarity
Range sensing
direction
Interior
border
Segmented Planar Area
Exterior and Interior borders shown
5
Exhaustive Search Approach
One pair of correctly matched lines provides
rotation
Line in left range image
Line in right range image
Plane normal
in right image
Plane normal
in left image
xleft
xright
zleft
yleft
Left range image
coordinate system
yright
zright
Right range image
coordinate system
6
Exhaustive Search Approach
Two pairs of correctly matched lines
provide exact translation
xleft
zleft
yleft
Rotated left range image
coordinate system
xright
zright
yright
Right range image
coordinate system
7
Exhaustive Search Approach
Find the two pairs of
corresponding lines that
maximizes the total number of
line matches
Consider two corresponding
line pairs
Compute transformation
Grade of computed transform:
total number of line matches
Keep the transform with the
highest grade
At the end refine best
transform using all matched
lines
White lines
(left scan)
Blue lines
(right scan)
Red/Green lines (matches)
8
Exhaustive Search Approach
No initial registration needed
High computational complexity
Symmetry problem unsolved
Improvements
Extract object-based coordinate system
Context-sensitive user interface
9
Framework of New Solution
Lines and planes from segmentation
Image1
line
clustering
Image2
line
clustering
Display registered pair
Rotation estimation
Translation estimation
Transform refinement by ICP
Automated Registration
Wrong
registration
due to
symmetry
Correct
registration
Next pair
of scans
Rotation adjustment
Translation adjustment
Global
stitching
User Interactions
10
Line Clustering
Line clustering
Line directions
Plane normals
Building’s local coordinate system
11
Rotation Estimation
Rotation estimation
y2
y1
x2
x1
z1
R = [x2 y2 z2]
T
* [x1 y1 z1]
z2
24 possible R’s?
12
Rotation Estimation
Heuristic: eliminate candidates based on observations
Scanner moves on the ground plane: y axis not change much
Overlapping images from close by viewpoints: smallest rotation
candidate is chosen
y1
x1 * R = x2
y1 * R = y2
z1 * R = z 2
Building
-z1
x1
Scanner position 1
-z2
y2
x2
Scanner position 2
(0,1,0) * R = R11 : projection of y1 on y2
R11> cos(45°) = 0.7
Sort by R00+R11+R22
2 to 5 R’s
Return the R
with the largest
diagonal sum
13
Translation Estimation
Translation estimation
y1
y2
y1
R
x1
x1
z1
z1
x2
z2
Left and right axes parallel accordingly after rotation
Pick robust line pairs to estimate translation
14
Translation Estimation
One pair of matched lines provides an estimated translation
Two pairs with similar estimated translations provide translation candidate
y1
y2
z1
z2
x1
T
Line pair 1
x2
Line pair 2
15
Translation Estimation
Two types of translation candidates
y1
y2
z1
x1
T = (d1 + d2) / 2
z2
d1
d2
x2
T linear system
16
Translation Estimation
Find the translation that maximizes the total number
of line matches
Cluster all estimated T’s, pick 10
most frequently appeared T’s
For each T:
Find all matches, solve linear
system to update R&T
Count matched line pairs again
Choose the R&T with the most
number of matched line pairs
Refine transformation with ICP
17
Registration System Flowchart
start
Read in all image pairs, form
the transformation graph
Read in one image pair
Last pair?
Y
Find pivot image, compute path
N
Automated registration
Compute transform from
each image to pivot image
Display to user
1
correct
save
2
Rotation
wrong by 90°
3
Need
manual
adjustment
Display other rotations
Display draggers
Choose a new
R, compute T
Adjust R/T,
optimize
save
Global optimization
Global stitching
Display and save
exit
18
User Interface
Display window: Points and lines of registered two scans
Correct Rotation & Translation
Rotation Symmetry by 90deg.
Need Manual Adjustment on R/T
19
User Interface
start
Read in all image pairs, form
the transformation graph
Read in one image pair
Last pair?
Y
Find pivot image, compute path
N
Automated registration
Compute transform from
each image to pivot image
Display to user
1
correct
save
2
Rotation
wrong by 90°
3
Need
manual
adjustment
Display other rotations
Display draggers
Choose a new
R, compute T
Adjust R/T,
optimize
save
Global optimization
Global stitching
Display and save
exit
20
User Interface
Rotation wrong by 90 degrees: choose from
other candidate rotations computed previously
21
User Interface
start
Read in all image pairs, form
the transformation graph
Read in one image pair
Last pair?
Y
Find pivot image, compute path
N
Automated registration
Compute transform from
each image to pivot image
Display to user
1
correct
save
2
Rotation
wrong by 90°
3
Need
manual
adjustment
Display other rotations
Display draggers
Choose a new
R, compute T
Adjust R/T,
optimize
save
Global optimization
Global stitching
Display and save
exit
22
User Interface
Adjusting rotation and translation based on
the building’s coordinate system
White draggers
for translation
Blue spheres
for rotation
23
User Interface
Display window: Points and lines of registered two scans
Correct Rotation & Translation
Exhaustive Search Approach
Rotation Symmetry by 90deg.
Need Manual Adjustment on R/T
ICP Optimization
24
Global Stitching
start
Read in all image pairs, form
the transformation graph
Read in one image pair
Last pair?
Y
Find pivot image, compute path
N
Automated registration
Compute transform from
each image to pivot image
Display to user
1
correct
save
2
Rotation
wrong by 90°
3
Need
manual
adjustment
Display other rotations
Display draggers
Choose a new
R, compute T
Adjust R/T,
optimize
save
Global optimization
Global stitching
Display and save
exit
25
Global Stitching
Global stitching for all images
Pivot image – the image with the most number of neighbors
Transform composition – along the strongest path from each
image to the pivot image
Further improvement to consider
Global optimization to minimize registration error
26
Experimental Results
Thomas Hunter building, Hunter College
14 scans, 15 pairs (13 automated, 2 manually adjusted)
10~20 seconds per pair; a few minutes for entire registration
27
Registration Results
Shepard Hall, City College
20 scans, 24 pairs (9 automated, 8 R symmetry, 7 adjust R/T)
20~90 seconds per pair; 1 hour for entire registration
video
28
Registration Results
Great Hall (interior of Shepard Hall)
21 scans, 44 pairs (12 automated, 18 R symmetry, 13
adjust R/T )
20~90 seconds per pair; 1.5 hour for entire registration
29
Registration Results
Great Hall interior scene
30
Registration Results
Great Hall interior scene
31
Registration Results
Great Hall interior scene
32
Algorithm Performance
Time for automated
registration
Lines in two scans
Average error of
matching planes
Number of
matching lines
33
Algorithm Performance
Thomas Hunter building
(rectangular)
Shepard Hall
Great Hall
(intricate) (much symmetry)
Number of scans
14
20
21
Number of pairs
15
24
44
13 / 2 / 0
9 / 8 / 7
12 / 18 / 13
200
600
600
10-20s
20-90s
20-90s
a few minutes
1 hour
1.5 hour
Before ICP
Optimization
21.77mm
51.72mm
17.59mm
After ICP
Optimization
1.77mm
3.23mm
7.26mm
Automated / Rotation only / Manual
Approximate number of lines per scan
Average time to register a pair
Approximate time for entire procedure
Average distance between
matching planes
34
Conclusions
Semi-automatic registration system
Automated 3D registration routines
Context-sensitive user interface
Fast computation, accurate registration
Future work
Global optimization
Extract higher ordered curvatures from range
data for faster and more accurate feature-based
registration
36
Spin Image Representation
Histogram of surface points about
a rotation around surface normal at the sample
Point at varying radii from the sample point
A method of measuring shape and curvature with
local support
Acknowledgement
NSF CAREER IIS-01-21239
NSF MRI/RUI EIA-0215962
Conference committee and all audiences
Contact us:
http://www.cs.hunter.cuny.edu/~ioannis/Vision.htm
Ioannis Stamos, [email protected]
Cecilia Chao Chen, [email protected]
38