Transcript Slides PPT 1,680KB

Computer Graphics International 2009
Automatic Registration of
Color Images to 3D Geometry
Yunzhen Li
and
Kok-Lim Low
School of Computing
National University of Singapore
* Presented by Binh-Son Hua
Problem Statement
Range images
Color images from
untracked camera
...
Automatically
register color
images to
3D model
2
3D model
Colored 3D model
Motivations
 Applications of active range sensing
 Manufacturing, cultural heritage modeling, etc.
 Photometric properties needed for visually-realistic
models
 Only some range scanners can capture color
 Color may not have required resolution
 E.g. for close-up or zoomed-in views of paintings
 View-dependent reflection requires many color images
from different directions
 Therefore, better to capture color separately
 However, impractical to manually register color images
to 3D geometry
3
Previous Work
 Feature-based approaches
 Match corresponding features in both color images and 3D model
 Can be fully automated
 Restricted to certain types of objects
 [Stamos & Allen, ICCV 2001], [Liu & Stamos, CVPR 2005]
 Statistics-based approaches
 Used only if reflected intensities of range sensing light were
recorded with range data
 Sensing light often not in visible light spectrum
 Compute statistical dependence between color images and
sensing light intensities
 Mutual information, chi-square, cross-correlation
 Camera calibrated & tracked, or co-locate with scanner
 [Williams et al, 2004], [Hantak & Lastra, 3DPVT 2006]
4
Our Approach
Color images
Sparse 3D model
Multiview
geometry
reconstruction
...
Registration
Color mapping
5
Colored 3D model
Detailed scanned 3D model
Steps
1. Data acquisition
2. Multiview geometry reconstruction
3. Approximate registration of
sparse model to detailed model
4. Registration refinement
5. Color mapping
6
1. Data Acquisition
 Range data
 Laser range scanner
 Color images
 Uncalibrated and untracked
digital camera
 Project special light pattern
on large textureless
surfaces
 Improve image feature
detection and MVG
reconstruction
7
2. MVG Reconstruction
 Detect and match features in color images
 Use SIFT
 Compute MVG
 Structure-from-motion
 Incrementally add a new image and apply sparse
bundle adjustment (SBA)
 Result is a sparse 3D model
 3D point cloud
 Camera parameters
8
2. MVG Reconstruction
 Example sparse 3D model
9
3. Approximate Registration
 To align sparse model with detailed model
 Unknown relative scale and pose
 Register one image in MVG to 3D model
 User input 6 point correspondences
 Estimated transformation propagated to other views
and 3D points in MVG
 Sparse model only approximately aligned to detailed
model
 Error in user inputs
 Error in MVG
 Geometric distortion in detailed model
10
4. Registration Refinement
 Need non-rigid alignment of MVG with detailed model
 To overcome geometric distortion in range images
 Registration refinement
 Automatically detect planes in detailed model
 Identify 3D points in MVG near the planes
 Refine MVG to minimize distance
between 3D points and planes
 Easily incorporated into
sparse bundle adjustment
 Better than using ICP algorithm
 Two models are treated as rigid shapes
 Cannot refine MVG
11
4. Registration Refinement
 Example result
Before
registration
refinement
12
After
registration
refinement
5. Color Mapping
 Colors from different views can be used for view-
dependent rendering
 View-dependent texture mapping
 Surface light field
 We simply want to assign a single color to each
surface point, but





13
Simple averaging blurs out details
Different exposures
Occlusions
Depth boundaries
Vignetting and view-dependent reflection
5. Color Mapping
 Use weighted blending
 Use lower weights near image and depth boundaries
 Preserve fine details
Without
details preservation
14
With
details preservation
5. Color Mapping
 Smooth color and intensity transitions
Without
weighted
blending
With
weighted
blending
15
Result
 Office scene
 30 color images (7 with projected pattern)
16
Conclusion
 Achieve accuracies within 3–5 pixels everywhere on




17
each image
Not reliant on detection of any specific type of
features in both color images and geometric model
Project light pattern to improve robustness of MVG
Better registration accuracy in face of geometric
distortion
Effective color mapping method
Acknowledgements
 The Photo Tourism team
 For sharing part of their code on MVG
 Prashast Khandelwal
 For contribution to preliminary work
 Singapore Ministry of Education
 For the funding
18