Transcript CS 223-B Lecture 1 - Stanford University
Stanford CS223B Computer Vision, Winter 2005
Lecture 3
Filters and Features (with Matlab)
Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp, Stanford [with slides by D Forsyth, D. Lowe, M. Polleyfeys, C. Rasmussen, G. Loy, D. Jacobs, J. Rehg, A, Hanson, G. Bradski,…] Sebastian Thrun CS223B Computer Vision, Winter 2005 1
Assignment 1 FAQ
• Compiling projects – cxcored.dll is the debug version of cxcore.dll, can be compiled from cxcore.dsp
– Use template cvsample.dsp to get paths right • Taking the images – Assignment change: out-of-focus images no longer needed – Don’t print a border around the chessboard Sebastian Thrun CS223B Computer Vision, Winter 2005 2
Assignment 1 FAQ
• Corner finding – Supply correct parameters e.g. corner_count<>0 – Visualize corner ordering • How to verify results – Backproject scene corners into image
x
width/2,
k
1,2 Sebastian Thrun CS223B Computer Vision, Winter 2005 3
Stanford CS223B Computer Vision, Winter 2005
Lecture 3
Filters and Features (with Matlab)
Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp, Stanford [with slides by D Forsyth, D. Lowe, M. Polleyfeys, C. Rasmussen, G. Loy, D. Jacobs, J. Rehg, A, Hanson, G. Bradski,…] Sebastian Thrun CS223B Computer Vision, Winter 2005 4
Today’s Goals
• Features 101 • Linear Filters and Edge Detection • Canny Edge Detector Sebastian Thrun CS223B Computer Vision, Winter 2005 5
Today’s Question
• What is a feature?
• What is an image filter?
• How can we find corners?
• How can we find edges?
Sebastian Thrun CS223B Computer Vision, Winter 2005 6
What is a Feature?
• Local, meaningful, detectable parts of the image Sebastian Thrun CS223B Computer Vision, Winter 2005 7
Features in Computer Vision
• What is a feature?
– Location of sudden change • Why use features?
– Information content high – Invariant to change of view point, illumination – Reduces computational burden Sebastian Thrun CS223B Computer Vision, Winter 2005 8
(One Type of) Computer Vision
Image 1 Feature 1 Feature 2 : Feature N Computer Vision Algorithm Image 2 Feature 1 Feature 2 : Feature N
Sebastian Thrun CS223B Computer Vision, Winter 2005 9
Where Features Are Used
• Calibration • Image Segmentation • Correspondence in multiple images (stereo, structure from motion) • Object detection, classification Sebastian Thrun CS223B Computer Vision, Winter 2005 10
What Makes For
Good
Features?
• Invariance – View point (scale, orientation, translation) – Lighting condition – Object deformations – Partial occlusion • Other Characteristics – Uniqueness – Sufficiently many – Tuned to the task Sebastian Thrun CS223B Computer Vision, Winter 2005 11
Today’s Goals
• Features 101 • Linear Filters and Edge Detection • Canny Edge Detector Sebastian Thrun CS223B Computer Vision, Winter 2005 12
What Causes an Edge?
• Depth discontinuity • Surface orientation discontinuity • Reflectance discontinuity (i.e., change in surface material properties) • Illumination discontinuity (e.g., shadow) Slide credit: Christopher Rasmussen Sebastian Thrun CS223B Computer Vision, Winter 2005 13
Quiz: How Can We Find Edges?
Sebastian Thrun CS223B Computer Vision, Winter 2005 14
Edge Finding 101
im = imread('bridge.jpg'); image(im); figure(2); bw = double(rgb2gray(im)); image(bw); gradkernel = [-1 1]; dx = abs(conv2(bw, gradkernel, 'same')); image(dx); colorbar; colormap gray matlab [dx,dy] = gradient(bw); gradmag = sqrt(dx.^2 + dy.^2); image(gradmag);
colorbar colormap(gray(255)) colormap(default)
Sebastian Thrun CS223B Computer Vision, Winter 2005 15
Edge Finding 101
• Example of a linear Filter Sebastian Thrun CS223B Computer Vision, Winter 2005 16
What is Image Filtering?
Modify the pixels in an image based on some function of a local neighborhood of the pixels 10 5 4 5 1 1 3 1 7 Some function 7 Sebastian Thrun CS223B Computer Vision, Winter 2005 17
Linear Filtering
• Linear case is simplest and most useful – Replace each pixel with a linear combination of its neighbors.
• The prescription for the linear combination is called the convolution kernel. 10 4 1 5 5 1 3 1 7 0 0 0 0 0.5
0 0 1.0 0.5
kernel 7 Sebastian Thrun CS223B Computer Vision, Winter 2005 18
Linear Filter = Convolution
I(.) I(.) I(.) I(.) I(.) I(.) I(.) I(.) I(.) g 11 g 12 g 13 g 21 g 22 g 23 g 31 g 32 g 33
f (i,j) = g 11 I(i-1,j-1) g 21 I(i,j-1) g 31 I(i+1,j-1) + g 12 I(i-1,j) + g 13 I(i-1,j+1) + + g 22 I(i,j) + g 23 I(i,j+1) + + g 32 I(i+1,j) + g 33 I(i+1,j+1) Sebastian Thrun CS223B Computer Vision, Winter 2005 19
Linear Filter = Convolution
f
[
m
,
n
]
I
g
k
,
l I
[
m
k
,
n
l
]
g
[
k
,
l
] with
k
l
,
g
[
k
,
l
] 1 Sebastian Thrun CS223B Computer Vision, Winter 2005 20
Filtering Examples
Sebastian Thrun CS223B Computer Vision, Winter 2005 21
Filtering Examples
Sebastian Thrun CS223B Computer Vision, Winter 2005 22
Filtering Examples
Sebastian Thrun CS223B Computer Vision, Winter 2005 23
Image Smoothing With Gaussian
figure(3); sigma = 3; width = 3 * sigma; support = -width : width; gauss2D = exp( - (support / sigma).^2 / 2); gauss2D = gauss2D / sum(gauss2D); smooth = conv2(conv2(bw, gauss2D, 'same'), gauss2D', 'same'); image(smooth); colormap(gray(255)); gauss3D = gauss2D ' * gauss2D; tic ; smooth = conv2(bw,gauss3D, ' same ' ); toc Sebastian Thrun CS223B Computer Vision, Winter 2005 24
Smoothing With Gaussian
Gaussian Averaging Slide credit: Marc Pollefeys Sebastian Thrun CS223B Computer Vision, Winter 2005 25
Smoothing Reduces Noise
The effects of smoothing
Each row shows smoothing with gaussians of different width; each column shows different realizations of an image of gaussian noise.
Slide credit: Marc Pollefeys Sebastian Thrun CS223B Computer Vision, Winter 2005 26
Example of Blurring
Image Blurred Image = Sebastian Thrun CS223B Computer Vision, Winter 2005 27
Edge Detection With Smoothed Images figure(4); [dx,dy] = gradient(smooth); gradmag = sqrt(dx.^2 + dy.^2); gmax = max(max(gradmag)); imshow(gradmag); colormap(gray(gmax)); Sebastian Thrun CS223B Computer Vision, Winter 2005 28
Scale
Increased smoothing: • Eliminates noise edges.
• Makes edges smoother and thicker.
• Removes fine detail.
Sebastian Thrun CS223B Computer Vision, Winter 2005 29
The Edge Normal
S
dx
2
dy
2 arctan
dy dx
Sebastian Thrun CS223B Computer Vision, Winter 2005 30
Displaying the Edge Normal
figure(5); hold on; image(smooth); colormap(gray(255)); [m,n] = size(gradmag); edges = (gradmag > 0.3 * gmax); inds = find(edges); [posx,posy] = meshgrid(1:n,1:m); posx2=posx(inds); posy2=posy(inds); gm2= gradmag(inds); sintheta = dx(inds) ./ gm2; costheta = - dy(inds) ./ gm2; quiver(posx2,posy2, gm2 .* sintheta / 10, -gm2 .* costheta / 10,0); hold off; Sebastian Thrun CS223B Computer Vision, Winter 2005 31
Separable Kernels
f
[
m
,
n
]
I
g
I
g X
g Y
0.04
0.03
0.02
0.01
0.09
0.08
0.07
0.06
0.05
0 0 5 10 15 20 25 30 35 40 45 Sebastian Thrun CS223B Computer Vision, Winter 2005 32
Combining Kernels / Convolutions
(
I
g
)
h
I
(
g
h
)
0.0030 0.0133 0.0219 0.0133 0.0030
0.0133 0.0596 0.0983 0.0596 0.0133
0.0219 0.0983 0.1621 0.0983 0.0219
0.0133 0.0596 0.0983 0.0596 0.0133
0.0030 0.0133 0.0219 0.0133 0.0030
1 1 Sebastian Thrun CS223B Computer Vision, Winter 2005 33
Effect of Smoothing Radius
1 pixel 3 pixels Sebastian Thrun CS223B Computer Vision, Winter 2005 7 pixels 34
Robert’s Cross Operator
1 0 0 -1 + 0 1 -1 0 S = [ I(x, y) - I(x+1, y+1) ] 2 + [ I(x, y+1) - I(x+1, y) ] 2 or S = | I(x, y) - I(x+1, y+1) | + | I(x, y+1) - I(x+1, y) | Sebastian Thrun CS223B Computer Vision, Winter 2005 35
Sobel Operator
S
1 = -1 -2 -1 0 0 0 1 2 1
S
2 = -1 0 1 -2 0 2 -1 0 1 Edge Magnitude =
S
1 2 +
S
1 2 Edge Direction = tan -1
S
1
S
2 Sebastian Thrun CS223B Computer Vision, Winter 2005 36
The Sobel Kernel, Explained
-1 -2 -1 0 0 0 1 2 1 = 1/4 * [-1 0 -1] 1 2 1 Sobel kernel is separable!
1 0 -1 2 0 -2 -1 1 0 = 1/4 * [ 1 2 1] 1 0 -1 1 -2 -1 1 2 1 Averaging done parallel to edge Sebastian Thrun CS223B Computer Vision, Winter 2005 37
Sobel Edge Detector
figure(6) edge(bw, 'sobel') Sebastian Thrun CS223B Computer Vision, Winter 2005 38
Robinson Compass Masks
-1 0 1 -2 0 2 -1 0 1 0 1 2 -1 0 1 -2 -1 0 1 2 1 0 0 0 -1 -2 -1 2 1 0 1 0 -1 0 -1 -2 1 0 -1 2 0 -2 1 1 -1 0 -1 -2 -1 0 -1 2 1 0 -1 -2 -1 0 0 0 1 2 1 -2 -1 0 -1 0 1 0 1 2 Sebastian Thrun CS223B Computer Vision, Winter 2005 39
Claim Your Own Kernel!
1 1 1 1 -2 1 -1 -1 Prewitt 1 -1 1 1 1 0 0 0 -1 -1 Prewitt 2 -1 5 5 5 -3 0 -3 -3 -3 Kirsch -3 -1 - 2 -1 0 0 1 2 Frei & Chen 1 0 1 2 1 0 0 0 -1 -2 Sobel -1 Sebastian Thrun CS223B Computer Vision, Winter 2005 40
Comparison (by Allan Hanson)
• Analysis based on a step edge inclined at an angle q (relative to y axis) through center of window.
• Robinson/Sobel: true edge contrast less than 1.6% different from that computed by the operator.
• Error in edge direction – Robinson/Sobel: less than 1.5 degrees error – Prewitt: less than 7.5 degrees error • Summary – Typically, 3 x 3 gradient operators perform better than 2 x 2.
– Prewitt2 and Sobel perform better than any of the other 3x3 gradient estimation operators.
– In low signal to noise ratio situations, gradient estimation operators of size larger than 3 x 3 have improved performance.
– In large masks, weighting by distance from the central pixel is beneficial.
Sebastian Thrun CS223B Computer Vision, Winter 2005 41
Today’s Goals
• Features 101 • Linear Filters and Edge Detection • Canny Edge Detector Sebastian Thrun CS223B Computer Vision, Winter 2005 42
Canny Edge Detector
figure(7) edge(bw, 'canny') Sebastian Thrun CS223B Computer Vision, Winter 2005 43
Canny Edge Detection
Steps:
1. Apply derivative of Gaussian 2. Non-maximum suppression • Thin multi-pixel wide “ridges” down to single pixel width 3. Linking and thresholding • Low, high edge-strength thresholds • Accept all edges over low threshold that are connected to edge over high threshold Sebastian Thrun CS223B Computer Vision, Winter 2005 44
Non-Maximum Supression
Non-maximum suppression:
Select the single maximum point across the width of an edge.
Sebastian Thrun CS223B Computer Vision, Winter 2005 45
Linking to the Next Edge Point
Assume the marked point is an edge point. Take the normal to the gradient at that point and use this to predict continuation points (either r or s). Sebastian Thrun CS223B Computer Vision, Winter 2005 46
Edge Hysteresis
•
Hysteresis
: A lag or momentum factor • Idea: Maintain two thresholds k high – Use k high chain and k low to find strong edges to start edge – Use k low to find weak edges which continue edge chain • Typical ratio of thresholds is roughly k high / k low = 2 Sebastian Thrun CS223B Computer Vision, Winter 2005 47
Canny Edge Detection (Example)
gap is gone Original image Strong + connected weak edges Strong edges only Weak edges courtesy of G. Loy Sebastian Thrun CS223B Computer Vision, Winter 2005 48
Canny Edge Detection (Example)
Using Matlab with default thresholds Sebastian Thrun CS223B Computer Vision, Winter 2005 49
Application: Road Finding
• (add roadrunner video here) Sebastian Thrun CS223B Computer Vision, Winter 2005 50
Corner Effects
Sebastian Thrun CS223B Computer Vision, Winter 2005 51
Today’s Goals
• Features 101 • Linear Filters and Edge Detection • Canny Edge Detector Sebastian Thrun CS223B Computer Vision, Winter 2005 52