Transcript Slide 1

DEVELOPMENT OF A COMPUTER PLATFORM
FOR OBJECT 3D RECONSTRUCTION USING
COMPUTER VISION TECHNIQUES
Teresa C. S. Azevedo
João Manuel R. S. Tavares
Mário A. P. Vaz
Contents
I. Introduction to Computer Vision;
II. Computer Platform presentation;
III. Experimental results;
IV. Conclusions;
V. Future work.
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz
2
Computer Vision
Introduction
Platform
Results
Conclusions
Future Work
 Computer Vision is continuously trying to develop theories and
methods for automatic extraction of useful information from
images, as similar as possible to the complex human visual system.
 Some applications:
 Medicine - 3D reconstruction / modelling, surgery planning;
 Identification and navigation systems;
 Virtual reality;
…
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz
3
Goals and Methodology
Introduction
Platform
 Contactless techniques to recover the 3D geometry of an object are
usually divided in two classes:
• active techniques - require some kind of energy projection or the camera’s
(or object’s) movement to obtain 3D information about the shape;
Results
• passive techniques - only use ambient light and so, usually, the extraction of
Conclusions
3D information becomes more difficult.
Future Work
 Our goal was to obtain 3D
models of objects using an
active vision technique called
Structure From Motion (SFM).
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz
4
Computer Platform
Introduction
Platform
Results
Conclusions
Future Work
 Integration of functions for 3D reconstruction, available from five
software programs and one computational library, all open source:
• OpenCV;
Ported to C using
• Peter’s Matlab Functions;
MATLAB Compiler toolbox
• Torr’s Matlab Toolkit;
• KLT;
• Projective Rectification without Epipolar Geometry;
• Depth Discontinuities by Pixel-to-Pixel Stereo.
 Modular structure;
 User’s graphical interface;
 Computer language: C++;
 Developing tool: Microsoft Visual Studio, using MFC libraries
(Microsoft Foundation Classes);
 Operational system: Microsoft Windows.
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz
5
Computer Platform
 The functions integrated enclose several Computer Vision techniques:
Introduction
Platform
Results
• feature points detection;
• feature points matching
between two images;
• epipolar geometry
Conclusions
determination;
Future Work
• rectification;
• dense matching.
 For each technique, the
user can easily choose the
algorithm to use, as well as
conveniently define its
parameters.
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz
6
Feature Points detection
Introduction
Platform
available algorithms
for feature points
detection
Results
Conclusions
Future Work
 Reflect the relevant discrepancies between their intensity values and those
of their neighbours;
 Usually represent vertices of objects, and their detection allows posterior
matching between the images of the sequences.
OpenCV
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz
KLT
7
Feature Points matching
Introduction
Platform
available
algorithms for
feature points
matching
Results
Conclusions
Future Work
1st image
feature points
coordinates
matching points
coordinates on
2nd image
fundamental
matrix
 Image 2D points association between sequential images, which are the
projection of the same 3D object point;
 A short set of matching points is enough to determine the epipolar geometry
between two images (the fundamental matrix).
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz
8
Feature Points matching
 Some results:
Introduction
Platform
Results
Conclusions
Future Work
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz
9
Epipolar Geometry determination
Introduction
Platform
algorithms for
epipolar geometry
determination
Results
Conclusions
Future Work
algorithms for
epipolar lines
determination
 Corresponds to the geometrical structure between two stereo images and its
expressed mathematically by the fundamental matrix;
 Also allows the elimination of some previous wrong matches (outliers), as
well as make easier the determination of new matching points (dense matching).
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz
10
Epipolar Geometry determination
Introduction
 Some results:
Platform
Results
Conclusions
Future Work
Epipolar line
Inlier
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz
11
Rectification
Introduction
Platform
Results
Conclusions
Future Work
available
algorithm for
rectification
 Method that changes two stereo images, in order to make them coplanar;
 Performing this step makes dense matching easier to obtain;
 The quality of the results is proportional to the quality of the epipolar geometry
determination.
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz
12
Dense matching
Introduction
available
algorithms for
dense matching
Platform
Results
Conclusions
Future Work
 Disparity map - codifies the distance between
the object and the camera(s):
closer points will have maximal disparity and farther points will get minimum
disparity;
 A disparity map gives some perception of discontinuity in terms of depth;
 One of the algorithms also returns a discontinuity map – defines the pixels who
border the changing between at least two levels of disparity.
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz
13
Dense matching
 Some results:
Introduction
Original images
Platform
Results
Conclusions
Future Work
Disparity map
Discontinuity map
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz
14
Conclusions
Introduction
Platform
Results
 The functions, already integrated in the computer platform, give
good results when applied to objects with strong characteristics;
Conclusions
 From the experimental results, it is possible to conclude that low
Future Work
quality results are strongly correlated with few (strong) feature points
detection and wrong matching;
 This weakness is higher as the object shape variation is smooth.
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz
15
Future work
Introduction
Platform
Results
 The next steps of this work will focus on improving the results
obtained when the objects have smooth and continuous surfaces:
• inclusion of space carving techniques for object reconstruction;
• the feature points to use in the 3D space object definition will be detected
Conclusions
with the use of a reduced number of markers added on the object;
Future Work
estimation algorithms;
• inclusion of a camera calibration technique, as well as pose and motion
 Finally, the computer platform will be used in 3D reconstruction
and characterization of 3D external human shapes.
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz
16
DEVELOPMENT OF A COMPUTER PLATFORM
FOR OBJECT 3D RECONSTRUCTION USING
COMPUTER VISION TECHNIQUES
Acknowledgments
This work was partially done in the scope of the project “Segmentation, Tracking and Motion
Analysis of Deformable (2D/3D) Objects using Physical Principles”, reference POSC/EEASRI/55386/2004, financially supported by FCT - Fundação para a Ciência e a Tecnologia in
Portugal.
Teresa C. S. Azevedo
João Manuel R. S. Tavares
Mário A. P. Vaz