Transcript [slides]

Image-Based Segmentation of Indoor
Corridor Floors for a Mobile Robot
Yinxiao Li and Stanley T. Birchfield
Department of Electrical and Computer Engineering
Clemson University
{ yinxial, stb }@clemson.edu
Motivation
Goal: Segment the floor in a single corridor image
Why?
• obstacle avoidance
• mapping
• autonomous exploration and navigation
Motivation
Goal: Segment the floor in a single corridor image
Why?
• obstacle avoidance
• mapping
• autonomous exploration and navigation
Outline
• Previous Work
• Detecting Line Segments
• Score Model for Evaluating Line Segments
– Structure Score
– Bottom Score
– Homogeneous Score
• Experimental Results
• Conclusion
Outline
• Previous Work
• Detecting Line Segments
• Score Model for Evaluating Line Segments
– Structure Score
– Bottom Score
– Homogeneous Score
• Experimental Results
• Conclusion
Previous Work
Free space detection
•
•
Combination of color and histogram
•
(Lorigo 1997)
Apply planar homographies to
optical flow vectors
Optical flow
(Kim 2009, Zhou 2006)
(Stoffler 2000, Santos-Victor 1995)
•
Floor detection
Stereo matching (Sabe 2004)
•
•
Stereo homographies (Fazl-Ersi 2009)
Geometric reasoning (Lee 2009)
Limitations
• Multiple images for motion
• Multiple cameras for stereo
• Require different colors for floor and wall
• Computational efficiency
• Calibrated cameras
• Assume ceiling visible
Our contribution: Segment the floor in real time using a
single image captured by a low-height mobile robot
Challenging problem: Reflections
Also:
• variety of poses (vanishing point, ceiling not always visible)
• sometimes wall and floor color are nearly the same
Outline
• Previous Work
• Detecting Line Segments
• Score Model for Evaluating Line Segments
– Structure Score
– Bottom Score
– Homogeneous Score
• Experimental Results
• Conclusion
Douglas-Peucker Algorithm
Purpose: Reduce the number of
points in a curve (polyline)
Algorithm:
1. Connect farthest endpoints
2. Repeat
1. Find maximum distance
between the original curve
and the simplified curve
2. Split curve at this point
Until max distance is less than
threshold
David Douglas & Thomas Peucker, "Algorithms for the reduction of the number of points required to
represent a digitized line or its caricature", The Canadian Cartographer 10(2), 112–122 (1973)
Douglas-Peucker Algorithm
Purpose: Reduce the number of
points in a curve (polyline)
Algorithm:
1. Connect farthest endpoints
2. Repeat
1. Find maximum distance
between the original curve
and the simplified curve
2. Split curve at this point
Until max distance is less than
threshold
Douglas-Peucker Algorithm
Purpose: Reduce the number of
points in a curve (polyline)
Algorithm:
1. Connect farthest endpoints
2. Repeat
1. Find maximum distance
between the original curve
and the simplified curve
2. Split curve at this point
Until max distance is less than
threshold
Douglas-Peucker Algorithm
Purpose: Reduce the number of
points in a curve (polyline)
Algorithm:
1. Connect farthest endpoints
2. Repeat
1. Find maximum distance
between the original curve
and the simplified curve
2. Split curve at this point
Until max distance is less than
threshold
Douglas-Peucker Algorithm
Purpose: Reduce the number of
points in a curve (polyline)
Algorithm:
1. Connect farthest endpoints
2. Repeat
1. Find maximum distance
between the original curve
and the simplified curve
2. Split curve at this point
Until max distance is less than
threshold
Douglas-Peucker Algorithm
Purpose: Reduce the number of
points in a curve (polyline)
Algorithm:
1. Connect farthest endpoints
2. Repeat
1. Find maximum distance
between the original curve
and the simplified curve
2. Split curve at this point
Until max distance is less than
threshold
Modified Douglas-Peucker Algorithm
dallowed
original algorithm: dallowed is constant
modified algorithm: dallowed is given by half-sigmoid function
original
modified
Detecting Line Segments (LS)
Detecting Line Segments (LS)
1. Compute Canny edges
Detecting Line Segments (LS)
1. Compute Canny edges
2. Modified Douglas-Peucker algorithm to
detect line segments
dallowed
Vertical LS: slope within ±5° of vertical direction
Horizontal LS: Slope within ±45° of horizontal direction
Detecting Line Segments (LS)
1. Compute Canny edges
2. Modified Douglas-Peucker algorithm to
detect line segments
dallowed
Vertical LS: slope within ±5° of vertical direction
Horizontal LS: Slope within ±45° of horizontal direction
3. Pruning line segments (320x240)
Vertical LS: minimum length 60 pixels
Horizontal LS: minimum length 15 pixels
Vanishing point (height selection): The vanishing point is
computed as the mean of the intersection of pairs of nonvertical lines. It is used for throwing away horizontal line
segments coming from windows, ceiling lights, etc.
Outline
• Previous Work
• Detecting Line Segments
• Score Model for Evaluating Line Segments
– Structure Score
– Bottom Score
– Homogeneous Score
• Experimental Results
• Conclusion
Score Model
Weights
horizontal line
Structure Score
Bottom Score
Homogeneous Score
is assigned to each horizontal line
Score Model – Structure
Given a typical corridor image
1. |I(x,y)| > T1
(How to set threshold T1?)
|I(x,y)| > T1
Score Model – Structure
Given a typical corridor image
1. |I(x,y)| > T1
(gradient magnitude)
(How to set threshold T1?)
2. I(x,y) < T2
(image intensity)
(How to set threshold T2?)
|I(x,y)| > T1 and I(x,y) < T2
Score Model – Structure
Given a typical corridor image
1. |I(x,y)| > T1
(gradient magnitude)
(How to set threshold T1?)
2. I(x,y) < T2
(image intensity)
(How to set threshold T2?)
We applied SVM to 800 points in 200 images:
|I(x,y)| > T1 and I(x,y) < T2
Score Model – Structure
Given a typical corridor image
1. |I(x,y)| > T1
(gradient magnitude)
(How to set threshold T1?)
2. I(x,y) < T2
(image intensity)
(How to set threshold T2?)
Approximate using two thresholds:
|I(x,y)| > T1 and I(x,y) < T2
Score Model – Structure
Given a typical corridor image
1. |I(x,y)| > T1
(gradient magnitude)
2. I(x,y) < T2
(image intensity)
3. Now threshold original image using
TLC (average gray level of pixels
satisfying #1 and #2)
I(x,y) < TLC
Score Model – Structure
Given a typical corridor image
1. |I(x,y)| > T1
(gradient magnitude)
2. I(x,y) < T2
(image intensity)
3. Now threshold original image using
TLC (average gray level of pixels
satisfying #1 and #2)
4. Compute the chamfer distance
between the line segments and
structure blocks
Score Model – Structure
Comparison of different threshold methods
Ridler-Calvard
Otsu
Our method is able to remove the spurious pixels
on the floor caused by reflections or shadows
Our method
Method
R-C
Otsu
Ours
Correctness
62%
66%
82%
Outline
• Previous Work
• Detecting Line Segments
• Score Model for Evaluating Line Segments
– Structure Score
– Bottom Score
– Homogeneous Score
• Experimental Results
• Conclusion
Score Model – Bottom
1. Connect the bottom points of
consecutive vertical line segments
to create “bottom” wall-floor
boundary
Score Model – Bottom
1. Connect the bottom points of
consecutive vertical line segments
to create “bottom” wall-floor
boundary
Score Model – Bottom
1. Connect the bottom points of
consecutive vertical line segments
to create “bottom” wall-floor
boundary
2. Compute the distance of each
horizontal line segment to the
“bottom” wall-floor boundary
(red circled horizontal line segments
indicates a positive contribution)
Outline
• Previous Work
• Detecting Line Segments
• Score Model for Evaluating Line Segments
– Structure Score
– Bottom Score
– Homogeneous Score
• Experimental Results
• Conclusion
Score Model – Homogeneous
Idea: The floor tends to have larger
regions (due to decorations,
posters, windows on the wall).
Algorithm:
1. Color-based segmentation of the
image (using FelzenszwalbHuttenlocher’s minimum spanning
tree algorithm)
2. For each horizontal line segment,
compute
size of region
just below
segment
size of largest
segment
Score Model – Homogeneous
Idea: The floor tends to have larger
regions (due to decorations,
posters, windows on the wall).
Algorithm:
1. Color-based segmentation of the
image (using FelzenszwalbHuttenlocher’s minimum spanning
tree algorithm)
2. For each horizontal line segment,
compute
size of region
just below
segment
size of largest
segment
Segmenting the floor
• Normalize the scores and sum
• Threshold the final score:
• Connect the remaining line segments and extend the
endpoints to the edges of the image
Outline
• Previous Work
• Detecting Line Segments
• Score Model for Evaluating Line Segments
– Structure Score
– Bottom Score
– Homogenous Score
• Experimental Results
• Conclusion
Evaluation Criterion
• Wall- floor Boundary
– Green line: Detected wall-floor
boundary
– Red line: Ground truth
Evaluation Criterion
• Wall- floor Boundary
– Green line: Detected wall-floor
boundary
– Red line: Ground truth
• Area
– Blue shade area: misclassified area
– Red shade area: ground truth floor
area
• Error Rate
blue
rerr 
%
red
• Segmentation is considered
successful if rerr < 10%
Sample Results
89.1% success on database of 426 images:
Sample Results
Images downloaded from the internet:
Sample Results
Some successful results on failure examples from Lee et al. 2008
Original Image
Lee’s
D. C. Lee, M. Hebert, and T. Kanade. “Geometric Reasoning for Single Image Structure Recovery”. IEEE
Conference on Computer Vision and Pattern Recognition (CVPR), 2009.
Ours
Sample Results
• Failure examples
Checkered
floor
Dark
image
Textured
wall
Bright
lights
Sample Results
• Video clip 1
Sample Results
• Video clip 2
Outline
• Previous Work
• Detecting Line Segments
• Score Model for Evaluating Line Segments
– Structure Score
– Bottom Score
– Homogeneous Score
• Experimental Results
• Conclusion
Conclusion
• Summary
-
-
-
Edge-based approach to segmenting floors in
corridors
Correctly handles specular reflections on floor
Nearly 90% of the corridor images in our database
can be correctly detected.
Speed: approximately 7 frames / sec
• Future work
-
Speed up the current algorithm
Improving score model by add more visual cues
(highly textured floors, low resolution, or dark environment)
-
Use floor segmentation for mapping
Acknowledgement
 Clemson Computer Vision Group reviewers
• Zhichao Chen
• Vidya Murali
 Anonymous reviewers
Thanks!
Questions?
Image-Based Segmentation of Indoor Corridor
Floors for a Mobile Robot