Transcript ECE 495 – Integrated System Design I Introduction to Image Processing ECE 495, Spring 2013

ECE 495 – Integrated System Design I
1
Introduction to Image Processing
ECE 495, Spring 2013
ECE 495 – Integrated System Design I
2
Overview
•
•
•
•
•
What is image processing?
What is an image?
Introduction to image processing techniques
Experimental setup
Basic MATLAB commands for image processing
ECE 495 – Integrated System Design I
3
What is Image Processing?
Photo stitching
Color boost
Image processing is the collective name for techniques used to
extract information from digital images or to manipulate them to
render variations of the input image.
Vehicle detection and
tracking
ECE 495 – Integrated System Design I
4
What is Image Processing?
Popular technologies which make use of the camera as a sensor
The Wii Remote uses an
IR camera to sense its
location relative to the
Wii Sensor Bar.
The Kinect uses image processing
techniques on depth images to detect
and track locations of multiple persons
in the field of view.
ECE 495 – Integrated System Design I
5
Pixels
Pixel
• A pixel (abbr. for picture element) is the smallest unit of an
image.
• Therefore, a 640x480 image is a matrix of 640 columns and
480 rows, each element of this matrix is called an image pixel.
ECE 495 – Integrated System Design I
6
MATLAB Image Coordinates
• MATLAB stores images as matrices.
• In MATLAB, image pixels are referenced using (row, col)
values.
• Origin of the coordinate system (1,1) is the top left corner of
the image
(1,1)
img
Thus, img(4,3) refers to
the pixel at the 4th row
and 3rd column.
ECE 495 – Integrated System Design I
7
RGB and Grayscale
• In RGB format, each Pixel has 3 color components: Red,
Green, and Blue.
• Other color representations, e.g. HSV, YUV, CMYK, are also
used. Transformations from RGB to these color spaces and
back are defined in MATLAB.
• If only intensity (bright/dark) variations are considered, the
resultant image is called a grayscale image. Each pixel has
only 1 component: intensity.
RGB
Gray
ECE 495 – Integrated System Design I
8
Basic Image Processing techniques
•
•
•
•
•
Background Subtraction
Intensity Thresholding
Morphological Operations
Color-based Processing
Shape-based Processing
A combination of the above methods is usually used in basic
image processing applications. There is no ‘perfect solution’, so
keep discussing (and trying) what suits your project best.
ECE 495 – Integrated System Design I
9
Background Subtraction
• Background subtraction is a technique used to isolate useful
information in an image (foreground) from the rest of the
image (background).
• A reference image is selected as the background.
• Each successive image in a video stream is compared against
this image.
• If the difference between the images is significant, the areas
which are different are considered to be the foreground for
that image.
ECE 495 – Integrated System Design I
10
Background Subtraction
Reference image or
background
Incoming image contains a new
object. This is our foreground,
i.e. useful information for
detecting the system state.
The new object is
identified as the foreground
after the background is
subtracted.
ECE 495 – Integrated System Design I
11
Intensity Thresholding
Bin
• Intensity thresholding is another
foreground separation technique. It
uses histograms.
• A histogram is an image statistic
which usually operates on an
intensity image, i.e. pixels having a
single value between 0-255
• The range 0-255 is divided into bins,
e.g. each intensity value may be
given its own bin
• The Y axis shows the count of
number of pixels in an image which
lie within limits for a bin.
ECE 495 – Integrated System Design I
12
Intensity Thresholding
Threshold
Foreground
• Assumption: Foreground is sufficiently bright/dark compared to
the background, i.e. the histogram has a clear separation.
• Assumption: The threshold value has been estimated using prior
observations or can be computed automatically.
– This method is very sensitive to lighting conditions
• Given these assumptions, set all pixels on the foreground side to
255 (bright), and all other pixels to 0 (dark).
ECE 495 – Integrated System Design I
13
Morphological Operations
Original
After dilation
After erosion
• Morphological operations are used to either remove noise from
an image or recover some details lost in thresholding or
background subtraction.
• Dilation adds to the foreground areas in an image, but can also
add to the noise.
• Erosion removes stray foreground pixels from it, but also
reduces genuine foreground areas.
• Usually, a combination of morphological operators is used to
improve the output.
ECE 495 – Integrated System Design I
14
Color Processing
img
• Used in identifying well color for the Minesweeper problem.
• In MATLAB, a color (RGB) image is stored as a 3D matrix with
the third dimension being color.
• Each pixel has a Red, Green, and Blue value. The values range
(for each color) from 0 to 255, e.g. a purely red pixel at (25,30)
will have the RGB value (255,0,0).
• To access the Red value for this pixel in the 3D image matrix, we
reference img(25,30,1). Similarly, Green will be img(25,30,2)
and Blue, img(25,30,3).
ECE 495 – Integrated System Design I
15
Color Processing
Is there a red sticker in
this well?
RGB (173,58,86)
• Color processing algorithms need to have a
tolerance to color, e.g. a RED sticker won’t
have RGB (255,0,0).
• This tolerance can be built into the algorithm
by
– Testing under varied ambient lighting
conditions, e.g. morning, evening, and
setting color thresholds accordingly.
– Experimenting with more variationtolerant color spaces, e.g. HSV.
– Experimenting with logical operators, e.g.
a RED sticker = (Red enough) AND (NOT
Green enough) AND (NOT Blue enough).
– Take average of pixel values in area
ECE 495 – Integrated System Design I
16
Shape-based Processing
Region 3
Region 2
Region 1
• Once the foreground objects have been
detected, it is important to know their
properties such as area, shape, location,
etc.
• MATLAB has a function called
regionprops which takes a binary image
as input and identifies properties of
contiguous (connected) pixels in
foreground regions.
• The output of regionprops is an array of
structures. Each element of the array
contains information about one
foreground region.
• MATLAB help file is very useful
ECE 495 – Integrated System Design I
17
Shape-based Processing
Regionprops can be used to filter out poor foreground detection
results. For example:
All detected
foreground regions
[1 2 3 4 5 6]
Eliminate those which
are too small [5]
[1 2 3 4 6]
3
6
2
1
5
4
[1 2 3 4]
3
2
1
Eliminate those which
are not circular [6]
Eliminate those which
are not at permitted
locations [4]
[1 2 3]
Genuine foreground
detections
ECE 495 – Integrated System Design I
18
Experimental setup
When the camera is set up in a known configuration relative to the
rest of the system, we can estimate physical dimensions using
image data. For example:
640 pixels
Camera
field of view
Fixed
height
480
pixels
12”
This implies
• 40 pixels/inch
along image
width
• 40 pixels/inch
along image
height
16”
A circle 1.25” diameter will be bounded by an area of:
1.25” * 40 pix/inch * 1.25” * 40 pix/inch = 2500 pix
Width
Height
Use information to
improve game state
detection , criteria for
algorithms
ECE 495 – Integrated System Design I
19
Some MATLAB functions
Name
Functionality
imread
Read input image
imshow
Show the image in a figure window
rgb2gray
Convert a RGB image to grayscale
rgb2hsv
Convert a RGB image to HSV
imdilate
Dilation
imerode
Erosion
im2bw
Convert image to binary using thresholding
regionprops
Get properties of connected components in an image
imhist
Show image histogram