Transcript 슬라이드 1 - Vision & Media Research LAB

Computer Vision Application
Image Features
2015. 3. 30
Hyunki Hong
School of Integrative Engneering
Contents
• Image Features
• Edge Detection
What are Image Feature ?
• local, meaningful, detectable parts of the image
• a global property of an image or part
ex) the average grey level, the area in pixel
• a local feature: a part of the image with some properties
ex) a circle, a line, a textured region in an intensity image, a planar
surface in a range image
• meaningful: features are associated to interesting scene elements via
the image formation process.
ex) sharp intensity variations by object contours, image regions with
uniform gray levels (planar surfaces)
• Detectable means location algorithms must exist.
- Feature descriptors specify the position & other essential properties
of the features found in the image.
cf. In 3-D CV, feature extraction is an intermediate step, (not the goal
of the system)
Origin of edges
• Edges are caused by a variety of factors:
surface normal discontinuity
depth discontinuity
surface color discontinuity
illumination discontinuity
Characterizing edges
• An edge is a place of rapid change in the image intensity
function
image
intensity function
(along horizontal scanline)
first derivative
edges correspond to
extrema of derivative
Image gradient
• The gradient of an image:
• The gradient points in the direction of most rapid increase in
intensity
: How does this direction relate to the direction of the edge?
• The gradient direction is given by
• The edge strength is given by the gradient magnitude.
Differentiation and convolution
• Recall, for 2D function, f(x, y):
f
 f x   , y f x, y 
 lim



0

x

 
• This is linear and shift
invariant, so must be the result
of a convolution.
• We could approximate this as
f f xn1 , y  f xn , y

x
x
(which is obviously a
convolution)
-1
1
Finite difference filters
• Other approximations of derivative filters exist:
Finite differences: example
• Which one is the gradient in the x-direction (resp. y-direction)?
Effects of noise
• Consider a single row or column of the image
: Plotting intensity as a function of position gives a signal
Where is the edge?
Effects of noise
• Finite difference filters respond strongly to noise
– Image noise results in pixels that look very different from their
neighbors
– Generally, the larger the noise the stronger the response
• What is to be done?
– Smoothing the image should help, by forcing pixels different
from their neighbors (=noise pixels?) to look more like
neighbors
Solution: smooth first
f
g
f*g
d
( f  g)
dx
• To find edges, look for peaks in
d
( f  g)
dx
Derivative theorem of convolution
• Differentiation is convolution, and convolution is associative:
d
d
( f  g)  f  g
dx
dx
• This saves us one operation:
f
= box filter
d
g
dx
f
d
g
dx
Derivative of Gaussian filter
* [1 -1] =
• Is this filter separable?
Gaussian distribution and its derivatives from order 0 to 4.
Plots of 1D Gaussian derivative funtion for order 0 to 7.
Derivative of Gaussian filter
x-direction
y-direction
• Which one finds horizontal/vertical edges?
Implementation issues
• The gradient magnitude is large along a thick “trail” or “ridge,”
so how do we identify the actual edge points?
• How do we link the edge points to form curves?
Designing an edge detector
• Criteria for an “optimal” edge detector:
– Good detection: the optimal detector must minimize the probability of
false positives (detecting spurious edges caused by noise), as well as that
of false negatives (missing real edges)
– Good localization: the edges detected must be as close as possible to the
true edges
– Single response: the detector must return one point only for each true
edge point; that is, minimize the number of local maxima around the true
edge
Edge Detection
edge: connected chains of edge points, contour fragments
• edges (edge pixels): pixels at or around which the image values
undergo a sharp variation, called edgels (edge elements)
• edge detection problem: Given an image corrupted by acquisition
noise, locate the edges most likely to be generated by scene elements,
not by noise.
• three steps of edge detection
1. noise smoothing: Suppress as much of the image noise as possible,
without destroying the true edges. In the absence of specific information,
assume the noise white and Gaussian.
2. edge enhancement: Design a filter responding to edges.
 edges can be located as the local maxima in the filter’s output
3. edge localization: Decide which local maxima in the filter’s output are
edges & which are just caused by noise.
a. thinning wide edges to 1-pixel width (nonmaximum suppression)
b. establishing the minimum value to declare a local maxima an edge
(thresholding)
Canny Edge Detector
• Modelling Edges & Noise
- Edges of intensity images can be modelled according to their
intensity profiles.
a. step edges : at contours of image regions of different intensities
(the most common type in intensity images)
b. ridge edges : generated by thin lines
c. roof edges : appear along the intersection of surfaces
Various edge types
An image designed to test corner and edge detectors
Canny Edge Detector
• The essential edge descriptor
1. edge normal
a. the direction (unit vector) of the maximum intensity variation at
the edge point
b. the direction perpendicular to the edge
2. edge direction : the direction tangent to the edge
3. edge strength : a measure of the local image contrast
• Modeling edges and noise
0 , x  0
1. Ideal 1- D step edge :
G( x)

A
,
x

0


2. assumptions
a. The edge enhancement filter is linear.
b. The filter must be optimal for noisy step edges.
c. The image noise is additive, white and Gaussian.
• Criteria for optimal edge detection
1. Good detection: minimise the possibility of false positives
(detecting spurious edges caused by noise), as well as that of
missing real edges
a. maximizing the SNR (th ratio of the root-mean-squared responss of
the filter to the ideal step edge and the noise)
b. impulse response of the 1-D linear filter (its width = 2W) : f
c. Consider 1D signal centered in x =0. The response of a linear filter f
0
to this edge:
A
f (t )dt
W

W
2
G (t ) f (t )dt  A
0
W

W
f (t )dt, SNR 
n0 : RMS noise amplitude per unit length
n0
W

W
f 2 (t )dt
2. Good localization
a. the edges detected must be as close as possible to the true edges
 the reciprocal of the RMS distance of the detected edge
A f (0)
from the center of the true edge
LOC 
'
W
n0
 f'
2
(t )dt
W
b. two measures depend only on the filter:
SNR 
A
( f ) ,
n0
LOC 
A
( f ) .
n0
c. product ΣΛ : a measure of how well the filter f satifies both
criteria simultaneously.  look for the f maximizing ΣΛ.
3. single response constraint
: The detector return one point only for each true edge point.
 minimise the number of local maxima around the true edge
created by noise
• Optimal step edge enhancement
1. Given edge and noise models, we can obtain the optimal
enhancing filter f as the function maximing Σ(f)Λ(f).
 turn edge detection into a constrained optimization problem.
2. A good approximation of the ideal step edge is the first derivative
Edge normal 방향에 따라 1D 필터링 진행해야 함.
of a Gaussian.
3. Use circular Gaussian filter for simplification.
= applying Gaussian smoothing to the image followed by
gradient estimation, and use the gradient to estimate all the
directional derivatives required.
a. edge strength: s(i.j)  (G * I)
b. edge normal: n  (G * I )
(G * I )
Optimal step edge enhancement
• I : an intensity image corrupted by noise,
G : Gaussian with zero mean and standard deviation 
1. Apply Gaussian smoothing to I , obtaining J = I * G
2. For each pixel (i, j),
a. the gadient components, Jx and Jy : J  J/x, J/y
b. estimate the edge strengh : es (i, j )  J x2 (i, y)  J y2 (i, y)
c. estimate the orientation of the edge normal : e (i, j )  arctan J y
0
Jx
3. the output: a strength image Es formed by the values es(i, j), and
an orientation image Eo formed by the values eo(i, j).
• Nonmaximum suppression
: Es may contain wide ridges around the local maxima.
 produce 1-pixel width edges
1. the input: the edge strength & orientation images, Es and Eo
2. Consider directions d1, …, d4 (0◦, 45◦, 90◦ and 135◦ orientations)
3. For each pixel,
a. Find the direction dˆk , which best approximates the direction E0(i, j)
: the normal to the edge
b. if Es(i, j) is smaller than at least one of its two neighbors along dˆk ,
assign IN(i, j) = 0. (suppression),
otherwise assign IN(i, j) = Es(i, j)
4. the output image IN(i, j), of the thinned edge points Es(i, j).
Strength images output after nonmaximum
suppression, showing the effect of varying
the filter’s size: the standard deviation of Gaussian
σf = 1, 2, 3 pixel
• Thresholding
: How do we get rid of the local maxima created by noise?
try to discard all pixels of value less than a threshold.
1. Problems
a. in the attempt of capturing true but weak edges, some noisy maxima will
be accepted too: false contours
b. the true maxima along a connected contours may fluctuate above and
below the threshold, fragmenting the resulting edge (streaking)
2. hysteresis threshoding algorithm
a. Locate the next unvisited edge pixel, IN(i, j) the output of non-max supp.,
such that IN(i, j) > h
b. Starting from IN(i, j), follow the chains of connected local maxima, in
both directions perpendicular to the edge normal, as long as IN > l .
c. Mark all visited points, and save a list of the locations of all points in the
connected contour found.
Canny edge detector
• This is probably the most widely used edge detector in
computer vision
• Theoretical model: step-edges corrupted by additive
Gaussian noise
• Canny has shown that the first derivative of the Gaussian
closely approximates the operator that optimizes the
product of signal-to-noise ratio and localization
J. Canny, A Computational Approach To Edge Detection, IEEE Trans. Pattern
Analysis and Machine Intelligence, 8:679-714, 1986.
Canny edge detector
1. Filter image with derivative of Gaussian
2. Find magnitude and orientation of gradient
3. Non-maximum suppression:
: Thin multi-pixel wide “ridges” down to single pixel width
Non-maximum suppression
At q, we have a
maximum if the value
is larger than those at
both p and at r.
Interpolate to get
these values.
Example
original image
(Lena)
norm of
the gradient
thresholding
Non-maximum
suppression
Canny edge detector
1. Filter image with derivative of Gaussian
2. Find magnitude and orientation of gradient
3. Non-maximum suppression
– Thin multi-pixel wide “ridges” down to single pixel width
4.
Linking of edge points
Edge linking
Assume the marked point is
an edge point. Then we
construct the tangent to the
edge curve (which is normal
to the gradient at that point)
and use this to predict the
next points (here either r or s).
Canny edge detector
1. Filter image with derivative of Gaussian
2. Find magnitude and orientation of gradient
3. Non-maximum suppression
– Thin multi-pixel wide “ridges” down to single pixel width
4.
Linking of edge points
– Hysteresis thresholding: use a higher threshold to start edge curves
and a lower threshold to continue them
Hysteresis thresholding
original image
high threshold
(strong edges)
low threshold
(weak edges)
hysteresis threshold
Effect of  (Gaussian kernel spread/size)
original
Canny with
Canny with
• The choice of  depends on desired behavior
1. large  detects large scale edges
2. small  detects fine features
Other Edge Detectors
• Roberts edge detector
: the input image I and a threshold 
1. apply noise smoothing as appropriate (ex. Gaussian smoothing),
obtaining a new image Is
2. filter Is with the masks  1  1  1 1   obtaining two images I1, I2
 1
1 1
 1
3. estimate the gradient magnitude at each pixel (i, j) as
G (i, j )  I12 (i, j )  I 22 (i, j )
 obtain an image of magnitude gradients, G
4. mark as edges all pixels (i, j) such that G (i, j) > 
5. output: the location of edge points
Other Edge Detectors
• Sobel edge detector
: Same as Roberts detector, but replace step 2:
2. filter Is with the masks
 1  2  1   1 0 1 
0
  2 0 2
0
0

 

 1
2
1    1 0 1 
 obtaining two images I1 and I2
• Evaluating edge dectectors & subpixel-precision edge detection
: Experimentally, estimating various performance indices in a large
number of experiments with synthetic images.
1. The number of spurious edges 2. The number of true edges missed
3. The average and RMS errors of estimates of edge position & orientation