Transcript Superquadrics

The research project: FP7-People-Marie CurieCOFUND-Trentino-post-doc 2009 - Incoming
Laser-Camera systems and algorithms
to sense the environment for robotic
applications
Presenter: Dr. Ilya Afanasyev
Lab. Mechatronics, University of Trento, Italy
[email protected], [email protected]
PAT mid-term review, Trento, 2011
18/10/2011
1/15
About the grant holder
AREAS OF EXPERTISE
1. Robotics. Computer vision. Data Treatment and Analysis.
2. Experimental Physics. Optical sensors (UV, Visible, and IR).
QUALIFICATIONS AND EDUCATION
2006
Ph.D. (in Technical Sciences), Vavilov State Optical
Institute, St. Petersburg, Russia (www.npkgoi.ru)
Specialization: Optical-electronic instruments
1994-2000 full-time student at the Baltic State Technical
University, Russia (www.voenmeh.ru)
Qualification: Engineer
Specialization: Automatic Control Systems for flying apparatus
JOB EXPERIENCE
2010 – present
Marie Curie-COFUND post-doc, Mechatronics Dep., Faculty
of Engineering, University of Trento, Italy
2007 – 2010
Senior engineer, Sensor Department, Development Center
Russia, Giesecke & Devrient GmbH (G&D, www.gi-de.com)
2006 – 2007
Senior engineer, Aerospace Optical-Electronic Systems
Dep., Vavilov State Optical Institute, Russia (www.npkgoi.ru)
2000 – 2006
Researcher/PhD student, Aerospace Physical Optics Lab.,
Vavilov State Optical Institute, Russia (www.npkgoi.ru)
18/10/2011
2/15
Status of the Project
The 3-years project in Mechatronics has been
doing since May 2010.
Description of the Project
The current project proposes using a Laser-Camera
System (LCS) as an instrument to get 3D information of
the world environment. LCS includes a scanning laser
rangefinder (LRF) and RGB camera. The rotated LRF
captures an object as unstructured “clouds of points” of
3D data, whereas a camera gives a 2D image of the
object. The fusion of 2D and 3D sensor’s data gives full
and detailed information about the object and its
environment.
18/10/2011
3/15
The calibration of laser-camera system
2D and 3D sensor fusion should be a result of a mutual
sensor calibration. The calibration is important for:
 a compatibility of sensors measurements, and
 having the metrical coordinate system with good
accuracy.
We are using the standard camera calibration technique
(Jean-Yves Bouget method).
18/10/2011
http://www.vision.caltech.edu/bouguetj/calib_doc/
4/15
3D data processing
Then joint 3D and 2D data processing is used:
 to recognize an object,
 to estimate its orientation in 3D space, and
 to improve LCS calibration by mathematical modeling.
3D object visualization / reconstruction can be executed by
math. modeling with SuperQuadrics (SQ).
The explicit equation of superquadrics is
a4
a5
 x  a1  signum(cos )  cos  signum(cos )  cos 
 y    a  signum(cos )  cos a4  signum(sin  )  sin  a5 ,

   2
a

 z  
a3  signum(sin )  sin  4


  / 2     / 2;
     .
The implicit equation of
superquadrics is
where x,y,z - superquadric system coordinates; η, ω – spherical coordinates, where
a1, a2, a3 – parameters of object scaling; a4, a5 – parameters of object shape.
18/10/2011
http://lrv.fri.uni-lj.si/~franc/SRSbook/SRS.html
5/15
Object recognition algorithm
The object recognition algorithm is
based on using RANSAC (RANdom
SAmple
Consensus)
method
of
SuperQuadrics model fitting to objects
presented in 3D “cloud of points”. The
solution is verified by evaluating the
matching score between the SQ object
model and 3D real data by a robust
least square fitting.
18/10/2011
http://www.ing.unitn.it/~afanasye/
6/15
Mid-term results & achievements
- Building a prototype of a robotic vehicle with installed
laser-camera system
The laser and the camera oriented and calibrated
together have been formed the Laser-Camera system
18/10/2011
7/15
Mid-term results & achievements
- Testing the sensitivity (laser remission effectiveness)
of the Laser-Camera System to registration of the
objects from different materials (metals, plastic, wood,
paper, polyethylene, etc.) at the different illumination
conditions.
18/10/2011
The polyethylene is transparent for IR rangefinder, but
multilayer polyethylene decreases the depth of recognition
significantly.
8/15
Mid-term results & achievements
- Capturing 3D data and development of 3D object
reconstruction / recognition algorithms (for human
body, etc. achieved at www.ing.unitn.it/~afanasye/)
18/10/2011
3D Human Body Pose Estimation: photo, “cloud of points”,
fitting a math. model to 3D data, final math. model of pose.
9/15
Future project work
According to the work plan, the future project work
should be dealt with:
 installation of the developed Laser-Camera
System on a robotic vehicle.
 research & control of robot vehicle motion.
 improving Laser-Camera System calibration
technique.
 improving object recognition algorithms.
 implementation algorithms to a robotic vehicle.
 developing algorithms of relative position
estimation (robot - object).
 verification of the algorithms with hardware
facilities.
18/10/2011
10/15
Possible impacts
 3D Laser-Camera System can provide 2D/3D
measurements capturing 2D/3D data of an object (with
distance and color information) for indoor applications.
 Object recognition and pose estimation algorithms
can be applied for:
 detection and reconstruction by math. models of
different real objects (including complex objects, like
human body).
 detection, classification and selection a definite
thing in a heap of different objects for robotic
manufacturing (such as a search an object in
mechanical details laying in mess).
 different scientific projects (with a possibility of a
collaboration).
18/10/2011
11/15
Training experiences
18/10/2011
During the project I participated:
 at 21th Summer School in Jyväskylä University,
Finland, 8-19 August 2011 (www.jyu.fi/summerschool/).
 in preparation of 3 conference papers:
 to 21th Int. Conference: GraphiCon'2011 (Moscow,
2011, http://gc2011.graphicon.ru/);
 to RGB-D Workshop on 3D Perception in Robotics
(Sweden, http://ias.cs.tum.edu/events/rgbd2011);
 to Int. Conference VISAPP-2012 (Rome, 2012,
February 2012, http://visapp.visigrapp.org).
 Intensive course of Italian as a second language
(Corso intensivo di lingua italiana per stranieri), CIAL,
University of Trento, test (Livello A2), February 2011.
 3D Modelling in 3ds Max. Training at St. Petersburg
State Polytechnical University, 2011. Certificate
#1KQYHQDCK1.
12/15
Training experiences
During the project I have been engaging in:
- Leading a Mechatronics group (2 Master students) on
measurements of remission from different materials (wood &
aluminum foil & polyethylene) by IR rangefinder.
- Giving a lecture on “Object Detection with Superquadrics”
in “Robotics and Sensor Fusion for Mechatronics Systems”
(course of Assoc. Professor Mariolino De Cecco, University
of Trento, Italy). Povo, 05 April 2011.
- Close collaboration with Italian colleagues for experimental
and theoretical work, making seminars with presentation:
www.ing.unitn.it/~afanasye/10_UniTN_reports/20_Presentations/.
Creating, discussing and sharing some algorithms in
MATLAB: www.ing.unitn.it/~afanasye/.
-
18/10/2011
13/15
Impacts on career development
The project helps in:
 developing the professional skills in
computer vision and robotics,
 getting an experience in resource, time
and money managing, and
 strengthening the researcher reputation,
that must be valuable for future work at a
research center or an industrial company.
18/10/2011
14/15
Acknowledgements
This work of Ilya Afanasyev on laser-camera
system creation and algorithms of 3D object
recognition, localization and reconstruction
has been supported by the grant of EU\FP7Marie Curie-COFUND - Trentino post-doc
program, 2010-2013.
He is very grateful to colleagues from
Mechatronics Dep. of UniTN, especially
supervisor - Professor Mariolino De Cecco.
Grazie!!
18/10/2011
15/15