Transcript Hardware Acelerated Voxel Coloring

Hardware Accelerated Voxel Coloring
Anselmo A. Montenegro†, Luiz Velho†, Paulo Carvalho† and Marcelo Gattass‡
†[anselmo,lvelho,pcezar]@visgraf.impa.br, ‡ [email protected]
Space Carving
3D object reconstruction is one of the most investigated topics
in computer graphics and vision. Among different techniques,
image based reconstruction is considered one of the most
promising as high quality digital cameras are becoming a
commodity hardware.
Adaptive Space Carving
Fixed pre-calibrated cameras setup
•Works on an octree representation of the scene space.
•The reconstruction is obtained by a refinement process based
on photo-consistency tests.
Input image
•Undefined cells are subdivided and classified in later stages.
H
K1Rtm1
K2Rtm2
Registered backgrounds
Background
estimation
Camera
calibration
Image
capture
Calibration by model
recongnition
Object
segmentation
Reconstruction
by Adaptive
Space Carving
Segmentation based on
intervals of confidence
Segmentation problem at the pattern
lines due to alignement errors (a).
Z=0
H =K1Rtm1(Rtm2)-1 (K2)-1
 f 0 0
 R11 R12 R13  R1 , T  




K   0 f 0 , Rt   R21 R22 R23  R2 , T  
 0 0 1
 R R R  R ,T  
3


 31 31 32

Solution: the interval of confidence for
a pixel p(i,j) in the target image is
calculated by sampling the pixels from
the
registered
images
at
a
neighborhood of (i,j) whose color is the
closest to p(i,j) (b).
=0
Algorithm
Registration based on projective texture mapping.
Photo-consitency evaluation done by GPU programming.
Still some problems:
Too much elements
Memory waste
Solution:
Hierarchical representation
of scene space
Refinement approach
Adaptive Carving
Test the consistency of the nonclassified cells intersected by
the current registration plane
Subdivide undefined cells and
colorize photo-consistent cells.
Update visibility maps.
Last registration
plane of the level?
Process next octree refinement level
Solution:
Project images on the current
registration plane with resolution
compatible to the octree level
Process next registration plane
Determine the registration
planes at the current level
Registration and evaluation of thousands of individual
elements.
Level 5
Level 6
Level7
Level 8
NO
R22
R31
 R1 , T  

 R2 , T  
 R3 , T  
wrong (a)
correct(a)
Fixed cameras reconstruction results
Hand-held camera reconstruction results
Hand-held camera setup
Final considerations
Calibration
Background estimation
Solution:
NO
YES
Finish
R12
Problems:
YES
No cell
subdivided ?
 R11

Rtm   R21
R
 31
Levels of refinement
Initialize the octree root cell with
the bounding box of the scene
Problems with photometric approaches:
Visibility and noise maps
Background image
Homography
•Classification of the cells: CONSISTENT, INCONSISTENT and UNDEFINED.
Images and
segmentation
Background estimation
and segmentation
Adaptive space carving:
•Uses photometric and silhouette information in multiresolution to
detect coarse empty regions as soon as possible.
Volumetric carving is a very
common technique use for
image based reconstruction. It
may use silhouette and/or
photometric
information.
Silhouette based methods
were successfully used in realtime reconstructions. This is
not the case when we consider
photometric approaches.
Occlusion tolerant
Calibration
Zoom
Insert model in the scene
Background
estimation by
warping images of the scene
without the object
In this work we only explored convencional GPU hardware
accelerated operations, as in the registration step by projective
texture mapping . The mechanism of copying framebuffer
information to main memory introduces significant overhead to the
overall processing time. We believe that by combining our adaptive
approach with photo-consistency test done by GPU programming
we can obtain considerable gains in efficiency.