Transcript Improving Human-Robot Interaction

Street Crossing
marked crosswalk
mobile robot
curb cut
• Tracking from a moving
platform
• Need to look left and right to
find a safe time to cross
• Need to look ahead to drive
to other side of road
• Must stay in crosswalk
Algorithm for Tracking Cars
1. Use image differencing method to extract
motion regions
2. Noise filter using 3x3 median filter; effective for
typical CCD sensor noise
3. Compute edges of motion regions using Canny
edge detection
4. Use Mori’s “sign pattern” to find bottoms of
cars [Mori 1994]
5. Find bounding boxes of moving objects
6. Use knowledge from prior frames to mark
direction of travel of each bounding box
Mori Sign Pattern
 Tracking algorithm uses “Mori Scan” to reliably
detect undersides of cars
 The Mori “sign pattern” for vehicle detection
says: the shadow underneath a vehicle is
darker than any other spot on the paved road
 The Mori result is invariant to lighting and
holds for wet and dry roads
 Use of the Mori result obviates the need for
explicit shadow detection and/or removal;
previously, prominent shadow edges caused
oversize bounding boxes
Mori Sign Pattern
Mori Sign Pattern
Street Crossing
Six frames of a tracking sequence
System Validation
• When it is safe to cross, a person monitoring the traffic
scene presses a button
• A second button press means it is no longer safe to
cross
• The time between button presses specifies a safe
crossing window
• Use more than one person to compensate for individual
risk tolerance
• Button press data is synchronized with the video data
• Compare system safety estimates to human safety
judgments
System Validation
• When it is safe to cross, a person monitoring the traffic
scene presses a button
• A second button press means it is no longer safe to
cross
• The time between button presses specifies a safe
crossing window
• Use more than one person to compensate for individual
risk tolerance
• Button press data is synchronized with the video data
• Compare system safety estimates to human safety
judgments
Crosswalk Traversal
• While crossing, devotes more processing
to the right-looking video stream
• Uses a forward-looking camera to detect
and stay on the marked (zebra striped)
crosswalk
• Uses sonar to avoid pedestrians and
stopped cars on the crosswalk
• Uses a laser range finder to detect the
curb cut; driving over curbs is possible, but
undesirable
Crosswalk Traversal
• While crossing, devotes more processing
to the right-looking video stream
• Uses a forward-looking camera to detect
and stay on the marked (zebra striped)
crosswalk
• Uses sonar to avoid pedestrians and
stopped cars on the crosswalk
• Uses a laser range finder to detect the
curb cut; driving over curbs is possible, but
undesirable
Related Work
• Automated driving systems: CMU’s Navlab
project, Dickmanns’ autonomous Autobahn
vehicle, DARPA Challenge
• Traffic scene monitoring systems that analyze
traffic conditions
• Camera orientation and assumptions of existing
vision-based, car-tracking systems do not apply
to street crossing
• Robotic street crossing has not been done
previously
Reasoning about Bounding Boxes
• Larger, lower (in the image plane) bounding
boxes correspond to close cars
• Smaller, higher bounding boxes denote distant
cars
• Tracking in real time using Phission so cars
move very little from frame to frame
• Track individual cars over time to determine
speed and travel direction
• Need to smooth results over time since CCD
cameras produce noisy data
Research conducted under the auspices
of Dr. Holly A. Yanco, the Robotics Lab,
and the Computer Science Department.
Contact [email protected] for additional
information.