Transcript Slides

Toward humanoid manipulation in
human-centered environments
T. Asfour, P. Azad, N. Vahrenkamp,
K. Regenstein, A. Bierbaum, K. Welke,
J. Schroder, R. Dillmann
Presentation by Yixing Chen
OUTLINE
•
•
•
•
•
•
Introduction
The humanoid robot ARMAR-III
Robot control architecture
Collision-free motion planning
Object recognition and localization
Programing of grasping and manipulation
tasks
Introduction
• Why do we create the humanoid robot?
• What’s the requirements for humanoid robot
in human-centered environment?
• How can humanoid robot work in humancentered environment?
The humanoid robot ARMAR-III
Video:
https://www.youtube.com/watch?v=SHMSyYLRQPM
Ability:
• Deal with the household environment
• Deal with a wide variety of objects
• Deal with the different activities
Configuration:
• Head – eyes, vision system
• Upper body – arm and hand, help to grasp
• Mobile platform – maintain stability, move
Seven subsystem: head, left arm, right arm, left hand,
right hand, torso and a mobile platform.
All configuration should be similar to human body.
Robot control architecture
• The task planning level
Specify the subtasks for the multiple subsystems of the robot, this
level represents the highest level with functions of task representation
and is responsible for the scheduling of tasks and management of
resources and skills.
• The task coordination level
Activates sequential/parallel actions for the execution level in order to
achieve the given task goal.
• The task execution level
Characterized by control theory to execute specified sensory-motor
control commands.
Collision-free motion planning
• Multiresolutional planning system
Low resolution – path planning algorithm for mobile platform or rough hand
movement. Faster.
High resolution – path planning algorithm for complex hand movement, such
as dexterous manipulation and grasping. Slower.
• Guarantee collision-free path
Using rapidly exploring tree (RRT).
• Using enlarged robot models
The enlarged models are constructed by
slightly scaling up the convex 3D models of
the robot so that the minimum distance
between the surfaces of the original and the
enlarged model reaches a lower bounding
dfreespace. So we can apply this method without
any distance computation and speed up the
algorithm.
Collision-free motion planning
• Lazy collision checking
Robot is very complex in shape. so even though we find the collision-free path and
path points, the path segment between the points may result in collision.
So we use the lazy collision checking to decouple the collision checks for C-space
samples and path-segments, this method can speed up the process of finding path.
• First step:
the normal sampling-based RRT algorithm searches a solution path in the C-space
• Second step:
we use the enlarged model approach to check
the collision status of the path segments of the
solution path. If a path segment between two
configurations ci and ci+1 fails during the
collision test, we try to create a local detour by
starting a subplanner which searches a way
around the Cspace obstacle
Result
Object recognition and localization
• To work in a household environment, the robot must be able
to recognize the objects and localized them with a high
enough accuracy for grasping. In this part, we will introduce
the recognition and localization based on shape and on
texture.
• recognition and localization based on shape:
The objects are colored objects. Simplify the problem of
segmentation, let robot concentrate on complicated tasks such
as the filling and empty of the dishwasher. Combine the
appearance-based method, model-based method and stereo
vision.
• recognition and localization based on texture
The objects are textured objects such as box of food, more
complex in recognition.
recognition and localization based on shape
• Segmentation
Perform color segmentation in HSV color space.
Use stereo vison, the properties of result blob are represented by
bounding box, centroid and number of pixels.
• Region processing pipeline
Use Principle Component Analysis, normalized the region in size. Resize
the region to a squared window of 64*64 pixels.
• 6D localization
Six dimensional space – varying position and orientation.
Calculate the position and orientation independently:
- Estimate of position is calculated by triangulating the centroid of the
color blob.
- Estimate of orientation is retrieved from the database for the
matched view.
recognition and localization based on shape
Typical result of a scene analysis. Input image of the left camera (left) and 3D
visualization of the recognition and localization result (right).
recognition and localization based on texture
• Feature calculation
- Different views of the same image patch around a feature
point vary, so we cannot simply correlate the image.
- The descriptors are computed on the base of a local planar
assumption.
- SIFT (Scale-invariant feature transform) is the best way to
find the feature of images.
- The feature information is about the position(u, v), rotation
angle φ, and feature vector{xj} .
• 2D localization
Get the correspondences between the view and training
image.
recognition and localization based on texture
Correspondences between view of the scene and training image. The left picture is the
view, and the right picture is the training image. The blue box illustrates the result of 2D
localization.
recognition and localization based on texture
• 6D localization
Calculate the pose base on the correspondences between 3D
model coordinates and image coordinates. And to improve
the accuracy, make use of the calibrated stereo system to
calculate depth information with maximum accuracy.
- Determine highly textured points with the calculated 2D
contour of the object in the left camera image
- Determine correspondences with subpixel-accuracy in the
right camera image
- Calculate a 3D point for each correspondence
- Fit a 3D plane into the calculated 3D point cloud
- Calculate the intersections of the four 3D lines through the
3D corners in the left image with the 3D plane.
recognition and localization based on texture
Recognition and localization for boxes and cups. The left picture is the view (input image),
and the right picture is the 3D visualization pf the result.
Programming of
Grasping and manipulation work
• The central idea:
The existence of a database with 3D models of all the objects encountered in
the robot workspace and a 3D model of the robot hand.
• Integrated grasp planning system
- The global model database: Contains the CAD models of all objects and a
set of feasible grasps for each object
- The offline grasp analyser: use the models of objects and hand to
compute a set of stable grasps.
- A online visual procedure to identify objects in stereo images: match the
features of images with the prebuilt model of objects, then determines
the location and pose.
After localizing the object, the robot select the grasp type for the object from
the set of stable grasps.
Integrated grasp planning system
Offline grasp analysis
To ensure the accuracy of the grasp, in our approach,
given an object, the grasp will be described by these
features.
• Grasp type
• Grasp starting point (GSP)
• Grasp center point (GCP)
• Approaching direction
• Hand orientation
Offline grasp analysis
• grasp type
It determines the grasp execution control such as
the hand preshape posture, the control strategy,
which fingers are used in the grasp and so on.
Grasp video
Let’s watch a grasp video.
https://www.youtube.com/
watch?v=QYEJJA52wG8
And the whole work
process video.
https://www.youtube.com/
watch?v=87cbivmjfe8
Summary
This paper introduced:
• A humanoid robot consisting of an active head for
vision system, two arms with five-fingered hands, a
torso and a holonomic platform.
• An integrated system of grasping and manipulation
tasks in humanoid robot. The system incorporates a
vision system for the recognition and localization of
objects, a path planner for the generation of collisionfree trajectories and an offline grasp analyser that
provides the most feasible grasp configuration for each
object.
Thank You!