Transcript ppt slides

Semi-automatic Range to Range
Registration: A Feature-based Method
Chao Chen & Ioannis Stamos
Computer Science Department
Graduate Center, Hunter College
The City University of New York
Motivation


Goal: highly accurate photo-realistic description of 3D world
Applications: urban planning, historical preservation, virtual reality
Texture
Mapping
Registration
3D model
Range scans
Photo-realistic 3D Model
Highly accurate
Computational efficient
Minimum human interaction
2D Pictures
2
Contribution

Related methods on range image registration

Iterative Closest Point algorithm




[Johnson & Hebert]
More suitable for curved surfaces; needs accurate normal
Previous feature-based algorithm


Require close initial registration
Spin images

[Besl & McKay, Chen & Medioni, Rusinkiewicz]
[Stamos & Leordeanu]
Exhaustively searching line pairs; high complexity
Symmetric structures require manual registration
Our semi-automatic 3D registration system


No rough pre-registration required
Automated registration procedure:



Utilize global information to compute transformation
ICP algorithm to optimize registration
Context-sensitive user interface to:


Display registration result at each step
Conveniently adjust translation and rotation
3
Outline

Automated registration procedures



Previous exhaustive search approach
Improved automated registration
Global stitching process to register all images

User interface

Experimental results

Conclusions and future work
4
Exhaustive Search Approach

Range image segmentation
Intersection
line
Each Segmented
Planar Area is
shown with different
color for clarity
Range sensing
direction
Interior
border
Segmented Planar Area
Exterior and Interior borders shown
5
Exhaustive Search Approach

One pair of correctly matched lines provides
rotation
Line in left range image
Line in right range image
Plane normal
in right image
Plane normal
in left image
xleft
xright
zleft
yleft
Left range image
coordinate system
yright
zright
Right range image
coordinate system
6
Exhaustive Search Approach

Two pairs of correctly matched lines
provide exact translation
xleft
zleft
yleft
Rotated left range image
coordinate system
xright
zright
yright
Right range image
coordinate system
7
Exhaustive Search Approach

Find the two pairs of
corresponding lines that
maximizes the total number of
line matches

Consider two corresponding
line pairs

Compute transformation

Grade of computed transform:
total number of line matches

Keep the transform with the
highest grade

At the end refine best
transform using all matched
lines
White lines
(left scan)
Blue lines
(right scan)
Red/Green lines (matches)
8
Exhaustive Search Approach

No initial registration needed

High computational complexity

Symmetry problem unsolved

Improvements


Extract object-based coordinate system
Context-sensitive user interface
9
Framework of New Solution
Lines and planes from segmentation
Image1
line
clustering
Image2
line
clustering
Display registered pair
Rotation estimation
Translation estimation
Transform refinement by ICP
Automated Registration
Wrong
registration
due to
symmetry
Correct
registration
Next pair
of scans
Rotation adjustment
Translation adjustment
Global
stitching
User Interactions
10
Line Clustering

Line clustering


Line directions
Plane normals
Building’s local coordinate system
11
Rotation Estimation

Rotation estimation
y2
y1
x2
x1
z1
R = [x2 y2 z2]
T
* [x1 y1 z1]
z2
24 possible R’s?
12
Rotation Estimation

Heuristic: eliminate candidates based on observations


Scanner moves on the ground plane: y axis not change much
Overlapping images from close by viewpoints: smallest rotation
candidate is chosen
y1
x1 * R = x2
y1 * R = y2
z1 * R = z 2
Building
-z1
x1
Scanner position 1
-z2
y2
x2
Scanner position 2
(0,1,0) * R = R11 : projection of y1 on y2
R11> cos(45°) = 0.7
Sort by R00+R11+R22
2 to 5 R’s
Return the R
with the largest
diagonal sum
13
Translation Estimation

Translation estimation
y1
y2
y1
R
x1
x1


z1
z1
x2
z2
Left and right axes parallel accordingly after rotation
Pick robust line pairs to estimate translation
14
Translation Estimation


One pair of matched lines provides an estimated translation
Two pairs with similar estimated translations provide translation candidate
y1
y2
z1
z2
x1
T
Line pair 1
x2
Line pair 2
15
Translation Estimation

Two types of translation candidates
y1
y2
z1
x1
T = (d1 + d2) / 2
z2
d1
d2
x2
T  linear system
16
Translation Estimation

Find the translation that maximizes the total number
of line matches


Cluster all estimated T’s, pick 10
most frequently appeared T’s
For each T:




Find all matches, solve linear
system to update R&T
Count matched line pairs again
Choose the R&T with the most
number of matched line pairs
Refine transformation with ICP
17
Registration System Flowchart
start
Read in all image pairs, form
the transformation graph
Read in one image pair
Last pair?
Y
Find pivot image, compute path
N
Automated registration
Compute transform from
each image to pivot image
Display to user
1
correct
save
2
Rotation
wrong by 90°
3
Need
manual
adjustment
Display other rotations
Display draggers
Choose a new
R, compute T
Adjust R/T,
optimize
save
Global optimization
Global stitching
Display and save
exit
18
User Interface
Display window: Points and lines of registered two scans
Correct Rotation & Translation
Rotation Symmetry by 90deg.
Need Manual Adjustment on R/T
19
User Interface
start
Read in all image pairs, form
the transformation graph
Read in one image pair
Last pair?
Y
Find pivot image, compute path
N
Automated registration
Compute transform from
each image to pivot image
Display to user
1
correct
save
2
Rotation
wrong by 90°
3
Need
manual
adjustment
Display other rotations
Display draggers
Choose a new
R, compute T
Adjust R/T,
optimize
save
Global optimization
Global stitching
Display and save
exit
20
User Interface

Rotation wrong by 90 degrees: choose from
other candidate rotations computed previously
21
User Interface
start
Read in all image pairs, form
the transformation graph
Read in one image pair
Last pair?
Y
Find pivot image, compute path
N
Automated registration
Compute transform from
each image to pivot image
Display to user
1
correct
save
2
Rotation
wrong by 90°
3
Need
manual
adjustment
Display other rotations
Display draggers
Choose a new
R, compute T
Adjust R/T,
optimize
save
Global optimization
Global stitching
Display and save
exit
22
User Interface

Adjusting rotation and translation based on
the building’s coordinate system
White draggers
for translation
Blue spheres
for rotation
23
User Interface
Display window: Points and lines of registered two scans
Correct Rotation & Translation
Exhaustive Search Approach
Rotation Symmetry by 90deg.
Need Manual Adjustment on R/T
ICP Optimization
24
Global Stitching
start
Read in all image pairs, form
the transformation graph
Read in one image pair
Last pair?
Y
Find pivot image, compute path
N
Automated registration
Compute transform from
each image to pivot image
Display to user
1
correct
save
2
Rotation
wrong by 90°
3
Need
manual
adjustment
Display other rotations
Display draggers
Choose a new
R, compute T
Adjust R/T,
optimize
save
Global optimization
Global stitching
Display and save
exit
25
Global Stitching

Global stitching for all images



Pivot image – the image with the most number of neighbors
Transform composition – along the strongest path from each
image to the pivot image
Further improvement to consider

Global optimization to minimize registration error
26
Experimental Results

Thomas Hunter building, Hunter College


14 scans, 15 pairs (13 automated, 2 manually adjusted)
10~20 seconds per pair; a few minutes for entire registration
27
Registration Results

Shepard Hall, City College


20 scans, 24 pairs (9 automated, 8 R symmetry, 7 adjust R/T)
20~90 seconds per pair; 1 hour for entire registration
video
28
Registration Results

Great Hall (interior of Shepard Hall)


21 scans, 44 pairs (12 automated, 18 R symmetry, 13
adjust R/T )
20~90 seconds per pair; 1.5 hour for entire registration
29
Registration Results

Great Hall interior scene
30
Registration Results

Great Hall interior scene
31
Registration Results

Great Hall interior scene
32
Algorithm Performance
Time for automated
registration
Lines in two scans
Average error of
matching planes
Number of
matching lines
33
Algorithm Performance
Thomas Hunter building
(rectangular)
Shepard Hall
Great Hall
(intricate) (much symmetry)
Number of scans
14
20
21
Number of pairs
15
24
44
13 / 2 / 0
9 / 8 / 7
12 / 18 / 13
200
600
600
10-20s
20-90s
20-90s
a few minutes
1 hour
1.5 hour
Before ICP
Optimization
21.77mm
51.72mm
17.59mm
After ICP
Optimization
1.77mm
3.23mm
7.26mm
Automated / Rotation only / Manual
Approximate number of lines per scan
Average time to register a pair
Approximate time for entire procedure
Average distance between
matching planes
34
Conclusions

Semi-automatic registration system




Automated 3D registration routines
Context-sensitive user interface
Fast computation, accurate registration
Future work


Global optimization
Extract higher ordered curvatures from range
data for faster and more accurate feature-based
registration
36
Spin Image Representation
Histogram of surface points about
a rotation around surface normal at the sample
Point at varying radii from the sample point
A method of measuring shape and curvature with
local support
Acknowledgement

NSF CAREER IIS-01-21239

NSF MRI/RUI EIA-0215962

Conference committee and all audiences
Contact us:



http://www.cs.hunter.cuny.edu/~ioannis/Vision.htm
Ioannis Stamos, [email protected]
Cecilia Chao Chen, [email protected]
38