http://ai.stanford.edu/~asaxena/rccar/ High Speed Obstacle Avoidance using Monocular Vision and Reinforcement Learning Jeff Michels Ashutosh Saxena Andrew Y.
Download
Report
Transcript http://ai.stanford.edu/~asaxena/rccar/ High Speed Obstacle Avoidance using Monocular Vision and Reinforcement Learning Jeff Michels Ashutosh Saxena Andrew Y.
http://ai.stanford.edu/~asaxena/rccar/
High Speed Obstacle Avoidance
using Monocular Vision and
Reinforcement Learning
Jeff Michels
Ashutosh Saxena
Andrew Y. Ng
Stanford University
ICML 2005.
Problem
Drive a remote control car
at high speeds
Unstructured outdoor
environments
Off the shelf hardware,
inexpensive cameras and
little processing power
Vision and Driving Control
ICML 2005.
Jeff Michels, Ashutosh Saxena & Andrew Y. Ng
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Prior Work: Vision
Estimating depth from multiple
images:
Stereovision (e.g., Scharstein &
Szeliski, 2002)
Depth from Defocus (e.g.,
Klarquist et al., 1995)
Optical Flow/Structure from
motion (e.g., Barron et al., 1994)
Motivation #1: Monocular vision.
Stereo vision has limits
baseline distance between cameras
vibration and blur
We would like to explore the use of
monocular cues.
ICML 2005.
Jeff Michels, Ashutosh Saxena & Andrew Y. Ng
Prior Work: Driving Control
Driving
Stereo-vision for driving (LeCun, 2003)
Highways with clear lane markings (Pomerleau, 1989)
Single camera for indoor robot, but known color and texture of ground
(Gini & Marchi, 2002)
Motivation #2: Reinforcement learning
Many past successes used model-based RL.
Does model-based RL still make sense even for tasks requiring
complex perception?
(To simulate vision input, we need to use computer graphics!)
ICML 2005.
Jeff Michels, Ashutosh Saxena & Andrew Y. Ng
Approach
Vision System
Estimate distance to nearest obstacle in each
possible steering direction.
Driving Control
Map from the output of the vision system into
steering commands for the car.
Use reinforcement learning to learn the policy.
ICML 2005.
Jeff Michels, Ashutosh Saxena & Andrew Y. Ng
Vision System:
Training Data
Image divided into vertical
columns corresponding to
possible steering
directions.
Image labeled with depth
for each vertical column
Laser range finder -ground truth distances
ICML 2005.
Jeff Michels, Ashutosh Saxena & Andrew Y. Ng
Vision System: Monocular Cues
Monocular Cues used by
humans for depth perception
Texture Variations - Laws’
Texture Gradient (Linear
Perspective) - Radon, Harris
Haze - Color
Occlusion
Known Object Size
(Loomis, Nature 2001)
ICML 2005.
Jeff Michels, Ashutosh Saxena & Andrew Y. Ng
Feature Vector: Monocular Cues
Texture Variation
Texture Gradient
Occlusion, Object Size,
Global structure
Overlapping windows
Appending adjacent stripe’s
vectors
The feature vector size is
858 dimension
ICML 2005.
Jeff Michels, Ashutosh Saxena & Andrew Y. Ng
Learning Algorithm
Supervised learning to estimate the distance d in each
column of the image.
Learn weights w via ordinary least squares with
quadratic cost.
depth
weights
arg minw ∑i (di - wT xi )2
i = columns, images
features
Other regression methods (SVR, robust regression)
gave similar results
ICML 2005.
Jeff Michels, Ashutosh Saxena & Andrew Y. Ng
Results: Learning Depth Estimates
0.4
Errors on a log scale
E = ∑ | log10(d) – log10(destimated) |
0.35
0.3
0.25
Able to predict depth with
a average error of 0.26
orders of magnitude.
0.2
Radon
(Texture
Gradient)
ICML 2005.
Harris
(Texture
Gradient)
Laws
(Texture
Variations)
All
Jeff Michels, Ashutosh Saxena & Andrew Y. Ng
Synthetic Graphics Data
Graphics images for
training the vision
system.
Variable degree of
graphical realism
Can a system trained
on synthetic images
predict distances on
real images?
ICML 2005.
Jeff Michels, Ashutosh Saxena & Andrew Y. Ng
Results: Combined Vision System
Hazard Rate (%)
25
20
15
10
When the distance to
nearest obstacle in the
chosen direction is less than
5 m, then it is a hazard.
Hazard rate improves by
combining the real and
synthetic trained system.
5
0
Random
ICML 2005.
Graphics
Real
Combined
24% hazard rate reduction
over using only real images.
Jeff Michels, Ashutosh Saxena & Andrew Y. Ng
Control: Reinforcement Learning
Model based RL -- hard perception problem
Randomly generated environment in
graphics simulator
Pegasus (Ng & Jordan, 2000) to learn
control policy
Car initialized at (0,0) and ran for fixed time
horizon.
Learning algorithm converged after 1674
iterations of policy search.
ICML 2005.
Jeff Michels, Ashutosh Saxena & Andrew Y. Ng
Reinforcement Learning: Parameters
1: spatial smoothing of predicted distances
2: threshold distance for evasive action
3: steering angle parameter
4, 5: evasive action parameters
6: throttle parameter
ICML 2005.
Jeff Michels, Ashutosh Saxena & Andrew Y. Ng
QuickTime™ and a
decompressor
are needed to see t his pict ure.
ICML 2005.
Jeff Michels, Ashutosh Saxena & Andrew Y. Ng
Results: Actual Driving Experiments
ICML 2005.
Jeff Michels, Ashutosh Saxena & Andrew Y. Ng
QuickTime™ and a
Cinepak decompressor
are needed to see this picture.
ICML 2005.
Jeff Michels, Ashutosh Saxena & Andrew Y. Ng
QuickTime™ and a
decompressor
are needed to see this picture.
ICML 2005.
Jeff Michels, Ashutosh Saxena & Andrew Y. Ng
Results: Driving Times
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
ICML 2005.
Jeff Michels, Ashutosh Saxena & Andrew Y. Ng
Summary
Monocular depth estimation is an interesting
and important problem.
Supervised learning for depth estimation.
Model-based RL, using computer graphics
simulator, to learn controller.
ICML 2005.
Jeff Michels, Ashutosh Saxena & Andrew Y. Ng
Extensions/Future Work
Learn complete
depth maps
Markov Random
Field (MRF) to
estimate depths.
Learning depth from single
monocular images,
Ashutosh Saxena, Sung H. Chung,
Andrew Y. Ng.
In NIPS 2005.
Image
ICML 2005.
Ground Truth
Jeff Michels, Ashutosh Saxena & Andrew Y. Ng
Predicted
[also with Sung Chung.]
Contact:
Ashutosh Saxena, [email protected]
http://ai.stanford.edu/~asaxena/rccar/
http://ai.stanford.edu/~asaxena/learningdepth/
ICML 2005.
Jeff Michels, Ashutosh Saxena & Andrew Y. Ng