Transcript Image Processing : Basic Concept 1

Image Processing :
Basic Concept
1
Imaging Systems Overview

Consists of two primary components:

Hardware – Image acquisition system,
computer, and display devices

Software – Image manipulation(操作),
analysis, and processing
2
3

Image is accessed (存取) as a 2-D array
(陣列) of data, where each data point is
referred to as a pixel (像素)
Notation:
I(r,c) = Brightness (亮度) of image at the pt
(r,c)
where
r = row(列), and c = column(行)
4
Visible Light Imaging

Reflectance (反射) function determines manner in
which objects (物體) reflect light
5

Sensors : Converts (轉換) light energy into
electrical energy
a) Single imaging sensor ; b) Linear ( line) sensor ; c) 2-D or array sensor
 CCD: 4kx4k
CMOS: less power, cheaper, image quality not as good as CCD
6
Image Representation
Optical (光學) image : Collection of spatially
distributed (空間分佈) light energy
measured by an image sensor to generate
I(r,c)
 Matrix: 2-D array like the image model,
I(r,c)
 Vector: One row or column in a matrix

7
Image Types
Binary (二元化) images: Simplest type of
images, which can take two values,
typically black or white, or “0” or “1”
 Gray scale (灰階) images: One-color or
monochrome images that contains only
brightness information and no color
information
 Color images: 3 band monochrome
images, where each band corresponds to
a different color, typically red, blue and
green or RGB

8

Color pixel vector: Single pixel’s values for
a color image, (R,G,B)

Multispectral(多光譜) Images: Images of
many bands containing information
outside of the visible spectrum(可見光譜)
9
Color Transform/Color Model
Mathematical model or algorithm to
map(映射) RGB data into another color
space (色彩空間)
 Decouples (分開) brightness and color
information
 Hue(色調)/Saturation(飽和度)/Lightness(亮
度) (HSL) Color Transform:
 Describes colors in terms that we can
more readily understand

10
Hue corresponds to color, saturation
corresponds to the amount of white in
color, and lightness is the brightness
 For example: a deep, bright orange color
would have a large intensity (bright), a hue
of “orange” , and a high value of saturation
(“deep(深度)”)
But in terms of RGB components, this
color would have the values as R =245,
G= 110, and B=20

11
12

Equations for mapping RGB to HSL are:
where
13
Digital Image File Formats

Bitmap images (raster images): Images
that can be represented by our image
model, I (r,c)
14
Image file header (檔案標頭): A set of
parameters (參數) found at the start of the
file image and contains information
regarding:

Number of rows (列數)(height, 高)

Number of columns (行數)(width, 寬)

Number of bands (頻帶數)

Number of bits per pixel (每個像數之位
元數)(bpp)

File type (檔案形式)

15

Look-up table (LUT): Used for storing RGB
values for 8-bit color images
16













Common image file formats are:
BIN, RAW
PPM,PBM,PGM
BMP
JPEG
TIFF
GIF
RAS
SGI
PNG
PICT, FPX
EPS
VIP
17
Matlab 影像處理I/O相關指令簡介



在Matlab中，每一個像素(pixel)之顏色係以
0到1之數值來表示。1代表最亮，0代表最
暗。
彩色的組成係由RGB，也就是 red(紅) 、
green(綠) 、blue(籃)的色階來構成。
影像之處理係基於矩陣的運算，矩陣內的
數值，代表了影像的強弱。
18
Matlab 影像處理I/O相關指令簡介

黑白影像用二維的矩陣來表示，彩色的影
像則需藉助三維的矩陣來表示RGB 在該一
位置的色濃度。
19
Matlab 影像處理I/O相關指令簡介

Show 圖指令imshow( )
其指令格式為：imshow(影像矩陣 A,色階 N) ，
代表影像矩陣 A以N個色階來顯示黑白影像。
 若N未設定，則對24位元之影像，內定為256色
階。
 影像矩陣A以三維空間A( : , : , 3)來表示彩色影
像，A(: , : , 1)表示紅色之色階； A(: , : , 2)表示
綠色之色階； A(: , : , 3)表示籃色之色階。

20
Matlab 影像處理I/O相關指令簡介


另外一種設定為設定界限值，以便讓圖形
在範圍以下顯示黑色，在範圍以上顯示白
色。指令格式為：imshow(影像矩陣 A,
[ lim_l lim_h])
代表影像矩陣 A 中的值若小於lim_l，則顯
示黑色；若大於lim_h，則顯示白色。
21
Matlab 影像處理I/O相關指令簡介


若已經有了現成的影像檔，希望讀入
Matlab的workspace中，可用imread(‘檔案
名稱’)讀入，再用imshow( )顯示出來即可。
imwrite( )可用來將影像寫入影像檔中，其
指令格式為：imwrite(影像來源, 寫入之影
像檔名稱, ‘影像格式’)
22
Ex2_1.m













clear; close all;
A=imread('1.bmp');
figure % 開繪圖視窗
imshow(A);
size(A)
figure
imshow(A(:,:,2)); %僅show綠色部份
imwrite(A(:,:,2),'ex2_1.tif','tif');
B=A(100:150,150:200,1); %取紅色之部份影像
figure
imshow(B);
figure
imshow(B,[100 200]);
23
Convolution Mask (迴旋遮罩)


Mask Operation (遮罩運算)
遮罩通常為 3 x 3 (有時使用 5 x 5, 7 x 7)
24
Convolution: Consist of following process
 Overlay (覆蓋) the mask (遮罩) on the
image
 Multiply the coincident terms (乘上一致的
項次)
 Sum all the results (將結果加總)
 Move to the next pixel, across the entire
image
Convolution mask for first order hold (一階保持之迴旋遮罩)
25
26
27
 Spatial
Filters (空間濾波器)

Operate on raw (原始的) image data in the
(r,c) space, by considering small
neighborhoods (鄰居), 3x3, 5x5, 7x7, and
moving sequentially (依序地)across and
down the image

Returns a result based on a linear (線性) or
nonlinear (非線性) operation
28

Consists of three types of filters:
Mean filters (均值濾波器)
 Median filters (中值濾波器)
 Enhancement filters (增強濾波器)


Many spatial filters are linear filters
implemented with a convolution mask (以迴
旋遮罩來實現); the result is a weighted sum
(加權和) of a pixel and its neighbors
29

Mask coefficients (係數) tend to effect (影
響) the image in the following general
ways:

Coefficients are positive (正數): blurs (模糊化)
the image
 Coefficients are alternating positive and
negative (正負交替): sharpens (銳化) the
image
 Coefficients sum to 1 (係數總和為1):
brightness retained (保留)
 Coefficients sum to 0 (係數總和為1): dark (暗)
image
30

Mean filters (均值濾波器) :

Averaging filters (平均濾波器)
Tend to blur (模糊化) the image
Adds a softer look to the image
Example 3x3 convolution mask (迴旋遮罩):



31
Mean filter
Original image
Mean filtered image
32

Median filters (中值濾波器) :

Nonlinear filter

Sorts (排序) the pixel values in a small
neighborhood and replaces the center (中央的)
pixel with the middle value (中間值) in the sorted list
(已排序串列)

Output image (輸出影像) needs to be written to a
separate image (a buffer (緩衝器)), so that results
are not corrupted (破壞)

Neighborhood (鄰居) can be of any size but 3x3,
5x5 and 7x7 are typical (典型)
33
Median filter
Original image with
salt and pepper noise (胡椒鹽雜訊)
Median filtered image (3x3)
34
Ex3_2.m (mean filter 與 median filter
之練習)















%Image: 原影像
%Image_noisy: 受污染的影像%
%Image_low: 執行低通濾波器後的影像%
%Image_med: 執行中值濾波器後的影像%
Image=imread('ex2_1.tif');
%讀取影像
%將胡椒鹽式的雜訊加入原影像中，其中0.06 是雜訊密度%
Image_noisy= imnoise (Image ,'salt & pepper',0.06);
Image2_noisy=double(Image_noisy)/255;
%變為 double格式
h=[1/9 1/9 1/9;1/9 1/9 1/9 ; 1/9 1/9 1/9]; %濾波器的子遮罩
Image_low=filter2(h, Image2_noisy);
%執行低通濾波
Image_med=medfilt2(Image_noisy,[3 3]);
%中值濾波採3×3矩陣
imshow(Image)
figure,imshow(Image_noisy)
figure,imshow(Image_low)
figure,imshow(Image_med)
35

Enhancement filters (增強濾波器) :

Implemented with convolution masks
having alternating positive and negative
(正負交替) coefficients

Enhance the image by sharpening (銳化)

Two types considered here:
1. Laplacian-type (拉普拉斯型) filters
2. Difference filters (差分型)
36
1. Laplacian-type (拉普拉斯型) filters :
 Are rotationally invariant (與旋轉無關),
that is they enhance the details (增強細
節) in all directions equally (所有方向皆相
等)
 Example convolution masks of
Laplacian-type filters are:
Filter 1
Filter 2
Filter 3
37
Laplacian filter
Original image
Laplacian filtered image
Contrast enhanced (對比增強)
Version of Laplacian filtered image
38
2. Difference (差分) filters:
 Also called as emboss (浮雕) filters
 Enhances the details in the direction
specific (特定方向) to the mask selected
 Four primary difference filter convolution
masks, corresponding to the edges (邊) in
the vertical (垂直), horizontal (水平), and
two diagonal directions (對角方向) are:
Vertical
Horizontal
Diagonal 1
Diagonal 2
39
Difference filter
Original image
Difference filtered image
Difference filtered image
added to the original image,
with contrast enhanced
(對比增強)
40
Ex3_3.m (增強濾波器之練習)










close all; clear;
A=imread('cat.bmp');
[m,n]=size(A); %原影像之size為mxn
imshow(A);
CM_lapa=[0 -1 0; -1 5 -1; 0 -1 0]; %Laplacian迴旋遮罩
CM_diff=[1 0 0; 0 1 0; 0 0 -1]; % difference 迴旋遮罩
B=filter2(CM_lapa,A); %執行Laplacian 遮罩運算
C=filter2(CM_diff,A); %執行difference 遮罩運算
figure; imshow(B/256);
figure; imshow(C/256);
41

Binary Image Analysis (二元影像分析)

Binary images are useful in many computer
vision applications which require simple
object shape; such as positioning a robot to
grasp (抓取) an object, to check a
manufactured object for defects (瑕疵), FAX,
OCR(光學字元辨識)
42

Binary Image Analysis (二元影像分析)
Most cameras provide us color or gray
level images, thus we need to convert
those images into binary images
 Next, we extract (擷取) simple binary
features and use them to classify (分辨)
binary objects

43

Thresholding via Histogram (經由直方圖
做臨界化)
•
Thresholding is required to create a binary
image (二元影像) from a gray level image
(灰階影像)
This is done by specifying a threshold
value (指定一臨界值) which will set all
values above the specified gray level to ‘1’
and everything below the specified value
to ‘0’
Typically 255 is used for ‘1’ and 0 is used
for the ‘0’ value
•
•
44
•
In many applications the threshold value is
determined experimentally (由實驗求得)
and is highly dependent on lighting
conditions and object to background
contrast (與光線條件,目標物與背景之對比
度有密切關係)
•
It is much easier to find a good threshold
value with proper lighting, and good
background to object contrast
45
Figure 3.3-1: Effects of Lighting and Object to
Background Contrast on Thresholding
a) An image of a bowl with high object to
background contrast (目標物與背景之
對比度高) and good lighting (照明良好)
b) Result of thresholding (臨界化)
image (a)
46
Figure 3.3-1: Effects of Lighting and Object to
Background Contrast on Thresholding (contd)
c) An image of a bowl with poor object to
background contrast (目標物與背景之
對比度低) and poor lighting(照明不佳)
d) Result of thresholding image (c)
47
•
The histogram (直方圖) is a plot of gray
level versus the number of pixels (灰階對
像素個數之作圖) in the image at each gray
level
•
Histogram of an image is examined (檢視)
to select the proper (適當的) threshold
value
•
The peaks (尖峰) and valleys (深谷) in the
histogram are examined and a threshold is
experimentally selected (經由實驗選取)
that will best separate (最佳的分割) the
object from the background (背景)
48
Figure 3.3-2: Histograms (直方圖)
Threshold 
b) The histogram of image (a), showing
the threshold that separates object
and background (一能將目標物與背景
分開之臨界點)
a) An image of a bowl with high object to
background contrast and good lighting
Image after threshold
(經臨界化後之影像)
49
Figure 3.3-2: Histograms (contd)
Threshold 
d) The histogram of image (c), showing
what appears to be a good threshold,
but it does not successfully separate
object and background
c) An image of a bowl with poor object to
background contrast and poor lighting
Image after threshold
50
Ex3_5.m (臨界化之練習)














A=imread('car_number.bmp'); imshow(A); X=A(:,:,3);
[m,n]=size(X);
figure, imshow(X);
figure, imhist(X); %繪出此影像之直方圖
th=70; %設定臨界值
C=zeros(m,n);
for i=1:m
for j=1:n
if X(i,j)>th;
C(i,j)=1;
end;
end;
end;
figure,imshow(C);
51

•
•
•
Histogram equalization (柱狀圖等化)
A technique where the histogram of the
resultant image is as flat as possible (產生一
柱狀圖很平之影像)
The theoretical basis for histogram
equalization involves probability theory,
where we treat the histogram as the
probability distribution of the gray levels (將柱
狀圖看成灰階之機率分怖)
Its function is similar to that of a histogram
stretch but often provides more visually
pleasing results across a wider range of
images (與柱狀圖伸展類似但常能產生視覺效
果更佳之影像)
52
•
Consists of 4 steps:
1. Find the running sum of the histogram
values (求出柱狀圖值之移動和)
2. Normalize the values from step (1) by
dividing by the total number of pixels (將(1)所
得之值除以像素之總數)
3. Multiply the values from step (2) by the
maximum gray level value and round (將(2)所
得之值乘以最大灰階值並捨去小數部份)
4. Map the gray level values to the results
from step (3) using a one-to-one
correspondence (將灰階值以一對一對應之方
式映射至步驟(3)所得之結果)
53
Example:
3-bits per pixel image – range is 0 to 7.
Given the following histogram:
Gray Level Value
0
1
2
3
4
5
6
7
Number of Pixels
(Histogram values)
10
8
9
2
14
1
5
2
54
1) Create a running sum of the histogram values.
(求出柱狀圖值之移動和) This means the first
value is 10, the second is 10+8=18, next
10+8+9=27, and so on. Here we get 10, 18, 27,
29, 43, 44, 49, 51
2) Normalize by dividing by the total number of
pixels. (將(1)所得之值除以像素之總數)
The total number of pixels is:
10+8+9+2+14+1+5+0 = 51 (note this is the last
number from step 1), so we get: 10/51, 18/51,
27/51, 29/51, 43/51, 44/51, 49/51, 51/51
3) Multiply these values by the maximum gray
level values, in this case 7, and then round the
result to the closest integer (將(2)所得之值乘以
最大灰階值並捨去小數部份). After this is done
55
we obtain: 1, 2, 4, 4, 6, 6, 7, 7
4) Map the original values to the results from step
3 by a one-to-one correspondence (將灰階值以
一對一對應之方式映射至步驟(3)所得之結果).
This is done as follows:
Original Gray
Level Value
0
1
2
3
4
5
6
7
Histogram
Equalized Values
1
2
4
4
6
6
7
7
56


All pixels in the original image with gray level 0
are set to 1, values of 1 are set to 2, 2 set to 4, 3
set to 4, and so on. After the histogram
equalization values are calculated and can be
implemented efficiently with a look-up-table
(LUT)(柱狀圖等化值算出後，可以查表法實現),
as discussed in Chapter 2
We can see the original histogram and the
resulting histogram equalized histogram in Fig.
8.2.14. Although the result is not flat, it is closer
to being flat than the original histogram(雖然結果
並非平坦，它比原本之柱狀圖要平坦多了)
57
58
Histogram Equalization Examples
(柱狀圖等化範例)
1.
Input image
Resultant image after histogram equalization
59
Histogram Equalization Examples (contd)
2.
Input image
Resultant image after histogram equalization
Note: As can be seen histogram equalization provides similar results
regardless of the input image (可看出不管輸入影像之亮度如何，
柱狀圖等化提供類似的結果)
60
•
•
Histogram equalization of a digital image
will not typically provide a histogram that is
perfectly flat, but it will make it as flat as
possible (通常並不一定產生一平坦之柱狀
圖，但卻會使柱狀圖儘可能的平坦)
Histogram equalization may not always
provide the desired effect, since its goal is
fixed – to distribute the gray level values
as evenly as possible. (將灰階值儘可能地
平均分佈)To allow for interactive histogram
manipulation, the ability to specify the
histogram is necessary
61
Ex8_3.m (柱狀圖等化法)








%柱狀圖等化法%
%image_1 原影像；image_2 執行柱狀圖等化後的影像
image_1=imread('L5_1.bmp'); %讀取影像
image_2=histeq(image_1);%對上述影像執行柱狀圖等化
imshow(image_1)
%顯示原影像
figure,imshow(image_2)
%顯示處理後的影像
figure,imhist(image_1)
%顯示原影像的柱狀圖
figure,imhist (image_2)
%顯示處理後影像的柱狀圖
62

Morphological Filtering (形態濾波)

Morphology(形態學) relates to form and
structure of objects (物件之形勢與結構)

Morphological filtering simplifies a segmented
image to facilitate the search for objects of
interest(形態濾波簡化一分割影像,有助於搜尋感
興趣之物件)

This is done by smoothing out object outlines (將
物件之外形平滑化), filling small holes (將小洞填
滿), eliminating small projections (消除小投影),
and with other similar techniques
63

The two principal morphological
operations(形態運算子) are dilation (膨
脹)and erosion (侵蝕)

Dilation allows objects to expand (延伸),
thus potentially filling in small holes (將小
洞填滿), and connecting disjoint objects
(連接不相連之物件)

Erosion shrinks (萎縮) objects by etching
away (eroding) (侵蝕) their boundaries (邊
界)
64

These operations can be customized for
an application by the proper selection of
the structuring element (正確地選取結構元
素), which determines exactly how the
objects will be dilated or eroded

Basically, the structuring element is used
to probe (探測) the image to find how it will
fit, or not fit, into the image object(s)
65

Dilation(膨脹): It is performed by laying the
structuring element on the image and sliding it
across the image in a manner similar to
convolution (將結構元素以類似迴旋的方式在影
像中滑動)
1. If the origin of the structuring element coincides
with a ‘0’ in the image, there is no change;
move to the next pixel (若結構元素之原點對應至
影像中之‘0’ ,沒有改變,移至下一像素)
2. If the origin of the structuring element coincides
with a ‘1’ in the image, perform the OR logic
operation on all pixels within the structuring
element (若結構元素之原點對應至影像中之‘1’ ,
將結構元素內之所有像素執行OR邏輯運算)
66
 With a dilation operation, all the ‘1’ pixels in the original image will be retained
(保留), any boundaries will be expanded (邊界將被延展), and small holes will be
filled (小洞將被填滿)
67
Note: In 1st printing
of book, the structuring
element is incorrect

68

Erosion (侵蝕) : It is similar to dilation, but we
turn pixels to '0', not '1'
1.
If the origin of the structuring element coincides
with a '0' in the image, there is no change; move to
the next pixel (若結構元素之原點對應至影像中之‘0’ ,
沒有改變,移至下一像素)
2.
If the origin of the structuring element coincides
with a ‘1’ in the image, and any of the ‘1’ pixels in
the structuring element extend beyond the object
(‘1’ pixels) in the image, then change the ‘1’ pixel
in the image, whose location corresponds to the
origin of the structuring element, to a ‘0‘(若結構元素
之原點對應至影像中之‘1’,且結構元素內之‘1’ 超越了影
像中之‘1’ ,則將結構元素之原點所對應之像素變成‘0‘).
69
 With an erosion operation, the only remaining pixels are those that coincide to
the origin of the structuring element where it is all contained in the object
70
1

1
1

1

1

1
1

1
1
1
1
1
0
0
1
0
0
1
1
1
1
1
0
0
1
0
0
1
1
1
1
1
1
1
1
1
1
1
0
0
1
1
1

1
1

1

1

1
1

Note: In first printing of book,
the 4th row, 4th col ‘0’, should be a ‘1’,
in the IMAGE
71



Opening (斷開): It consists of an erosion
followed by a dilation (先侵蝕再膨脹)
It can be used to eliminate all pixels in
regions that are too small to contain the
structuring element (可去除小到無法包含
結構元素之區域內之所有像素)
In this case the structuring element is
often called a probe, as it is probing the
image looking for small objects to filter out
of the image
72
 The output image tends to take a shape similar to the structuring element itself
73




Closing (閉合): It consists of a dilation
followed by erosion(先膨脹再侵蝕)
It can be used to fill in holes and small
gaps (填補洞與空隙)
It will connect small, adjacent objects (連
接小且相鄰之物件)
Closing tends to “close up” or “fill in”
objects
74
 Note that holes and gaps are filled, but, unlike dilation, more of the original
boundary is retained
75

Closing and opening will have different
results(斷開與閉合結果不同)even though both
consist of an erosion and a dilation

Therefore, order of operation is important (運算
之順序很重要) for morphological operations

Different structuring elements will also provide
different results (不同之結構元素將產生不同之結
果). As noted before, objects in the output image
will tend to take the shape of the structuring
element (輸出影像中之物件傾向於保有結構元素
之形狀)
76
Figure 4.3-18: Binary Dilation with Various Shape
Structuring Elements
a) Original image, a microscope cell image that
b) Dilation with a circular (圓形的)structuring
has undergone a threshold operation
element
(original image courtesy of Sara Sawyer, SIUE)
77
Figure 4.3-18: Binary Dilation with Various Shape
Structuring Elements (contd)
c) Dilation with a square (方形) structuring
element
d) Dilation with a cross shape (十字形)structuring
element
78
Figure 4.3-19: Dilation with Different Size
Structuring Elements
a) Original image
b) Dilation with a circular
structuring element of
size 3
79
Figure 4.3-19: Dilation with Different Size
Structuring Elements (contd)
c) Dilation with a circular
structuring element of
size 7
d) Dilation with a circular
structuring element of
size 11
80
Figure 4.3-20: Binary Erosion with Various
Shape Structuring Elements
a) Original image, a microscope cell image
that has undergone a threshold operation
(original image courtesy of Sara Sawyer,
SIUE),
b) Erosion with a circular structuring element
81
Figure 4.3-20: Binary Erosion with Various
Shape Structuring Elements (contd)
c) Erosion with a square structuring element
d) Erosion with a cross shape
structuring element
82
Figure 4.3-21: Binary Opening with Various
Shape Structuring Elements
a) Original image, a microscope cell image
that has undergone a threshold operation
(original image courtesy of Sara Sawyer,
SIUE),
b) Opening with a circular structuring element
83
Figure 4.3-21: Binary Opening with Various
Shape Structuring Elements (contd)
c) Opening with a square structuring element
d) Opening with a cross shape
structuring element
84
Figure 4.3-22: Binary Closing with Various
Shape Structuring Elements
a) Original image, a microscope cell image
that has undergone a threshold operation
(original image courtesy of Sara Sawyer,
SIUE),
b) Closing with a circular structuring element
85
Figure 4.3-22: Binary Closing with Various
Shape Structuring Elements (contd)
c) Closing with a square structuring element
d) Closing with a cross shape
structuring element
86
Figure 4.3-23: Opening and Closing with Different
Size Structuring Elements
a) Original microscopic cell image
(courtesy of Sara Sawyer, SIUE)
b) Image after undergoing a
threshold operation
87
Figure 4.3-23: Opening and Closing with Different
Size Structuring Elements (contd)
c) Opening with a circular structuring
element of size 5
d) Closing with a circular structuring
element of size 5
88