Transcript CS 223-B Lecture 1 - Stanford University

Stanford CS223B Computer Vision, Winter 2006
Lecture 2
Lenses, Filters, Features
Professor Sebastian Thrun
CAs: Dan Maynes-Aminzade and Mitul Saha
[with slides by D Forsyth, D. Lowe, M. Polleyfeys, C. Rasmussen, G. Loy, D. Jacobs, J. Rehg, A, Hanson, G. Bradski,…]
Sebastian Thrun
CS223B Computer Vision, Winter 2006
1
Today’s Goals
•
•
•
•
Thin Lens
Aberrations
Features 101
Linear Filters and Edge Detection
Sebastian Thrun
CS223B Computer Vision, Winter 2006
2
Pinhole Camera (last Wednesday)
-- Brunelleschi, XVth Century
Marc Pollefeys comp256, Lect 2
Sebastian Thrun
CS223B Computer Vision, Winter 2006
3
Snell’s Law
Snell’s law
n1 sin a1 = n2 sin a2
Sebastian Thrun
CS223B Computer Vision, Winter 2006
4
Thin Lens: Definition
focus
optical axis
f
Spherical lense surface: Parallel rays are refracted to single point
Sebastian Thrun
CS223B Computer Vision, Winter 2006
5
Image plane
Thin Lens: Projection
optical axis
f
z
Spherical lense surface: Parallel rays are refracted to single point
Sebastian Thrun
CS223B Computer Vision, Winter 2006
6
Image plane
Thin Lens: Projection
optical axis
f
f
z
Spherical lense surface: Parallel rays are refracted to single point
Sebastian Thrun
CS223B Computer Vision, Winter 2006
7
Thin Lens: Properties
1. Any ray entering a thin lens parallel to
the optical axis must go through the
focus on other side
2. Any ray entering through the focus on
one side will be parallel to the optical
axis on the other side
Sebastian Thrun
CS223B Computer Vision, Winter 2006
8
Thin Lens: Model
P
Q
QO OR

Z
f
O
OR QO

z
f
Fr
Fl
p
R
Z
f
f
z
OR  QO  QO
Z
f
Sebastian Thrun
f
z
f
z
2
  f  zZ
Z f
CS223B Computer Vision, Winter 2006
9
Transformation
Sebastian Thrun
CS223B Computer Vision, Winter 2006
10
A Transformation
Sebastian Thrun
CS223B Computer Vision, Winter 2006
11
The Thin Lens Law
Zˆ
P
zˆ
Q
f 2  zZ
O
Fr
Fl
p
R
Z
f
f
z
1 1 1
 
zˆ Zˆ f
Sebastian Thrun
CS223B Computer Vision, Winter 2006
12
The Thin Lens Law
1 1 1
 
zˆ Zˆ f
Sebastian Thrun
CS223B Computer Vision, Winter 2006
13
Limits of the Thin Lens Model
3 assumptions :
1. all rays from a point are focused onto 1 image point
• Remember thin lens small angle assumption
2. all image points in a single plane
f'
3. magnification m 
is constant
z0
Deviations from this ideal are aberrations
Sebastian Thrun
CS223B Computer Vision, Winter 2006
14
Today’s Goals
•
•
•
•
Thin Lens
Aberrations
Features 101
Linear Filters and Edge Detection
Sebastian Thrun
CS223B Computer Vision, Winter 2006
15
Aberrations
2 types :
geometrical : geometry of the lense,
small for paraxial rays
chromatic : refractive index function of
wavelength
Marc Pollefeys
Sebastian Thrun
CS223B Computer Vision, Winter 2006
16
Geometrical Aberrations
 spherical aberration
 astigmatism
 distortion
 coma
aberrations are reduced by combining lenses
Sebastian Thrun
CS223B Computer Vision, Winter 2006
17
Astigmatism
Different focal length for inclined rays
Marc Pollefeys
Sebastian Thrun
CS223B Computer Vision, Winter 2006
18
Astigmatism
Different focal length for inclined rays
Marc Pollefeys
Sebastian Thrun
CS223B Computer Vision, Winter 2006
19
Spherical Aberration
rays parallel to the axis do not converge
outer portions of the lens yield smaller
focal lenghts
Sebastian Thrun
CS223B Computer Vision, Winter 2006
20
Distortion
magnification/focal length different
for different angles of inclination
pincushion
(tele-photo)
barrel
(wide-angle)
Can be corrected! (if parameters are know)
Sebastian Thrun
CS223B Computer Vision, Winter 2006
Marc21
Pollefeys
Coma
point off the axis depicted as comet shaped blob
Sebastian Thrun
CS223B Computer Vision, Winter 2006
Marc22
Pollefeys
Chromatic Aberration
rays of different wavelengths focused
in different planes
cannot be removed completely
Marc Pollefeys
Sebastian Thrun
CS223B Computer Vision, Winter 2006
23
Vignetting
Effect: Darkens pixels near the image boundary
Sebastian Thrun
CS223B Computer Vision, Winter 2006
24
CCD vs. CMOS
•
•
•
•
•
•
•
Mature technology
Specific technology
High production cost
High power consumption
Higher fill rate
Blooming
Sequential readout
Sebastian Thrun
•
•
•
•
•
•
•
•
•
Recent technology
Standard IC technology
Cheap
Low power
Less sensitive
Per pixel amplification
Random pixel access
Smart pixels
On chip integration
with other components
CS223B Computer Vision, Winter 2006
Marc25
Pollefeys
Today’s Goals
•
•
•
•
Thin Lens
Aberrations
Features 101
Linear Filters and Edge Detection
Sebastian Thrun
CS223B Computer Vision, Winter 2006
26
Today’s Question
•
•
•
•
What is a feature?
What is an image filter?
How can we find corners?
How can we find edges?
• (How can we find cars in images?)
Sebastian Thrun
CS223B Computer Vision, Winter 2006
27
What is a Feature?
• Local, meaningful, detectable parts of the image
Sebastian Thrun
CS223B Computer Vision, Winter 2006
28
Features in Computer Vision
• What is a feature?
– Location of sudden change
• Why use features?
– Information content high
– Invariant to change of view point, illumination
– Reduces computational burden
Sebastian Thrun
CS223B Computer Vision, Winter 2006
29
(One Type of) Computer Vision
Image 1
Feature 1
Feature 2
:
Feature N
Computer
Vision
Algorithm
Image 2
Feature 1
Feature 2
:
Feature N
Sebastian Thrun
CS223B Computer Vision, Winter 2006
30
Where Features Are Used
• Calibration
• Image Segmentation
• Correspondence in multiple images (stereo,
structure from motion)
• Object detection, classification
Sebastian Thrun
CS223B Computer Vision, Winter 2006
31
What Makes For Good Features?
• Invariance
–
–
–
–
View point (scale, orientation, translation)
Lighting condition
Object deformations
Partial occlusion
• Other Characteristics
– Uniqueness
– Sufficiently many
– Tuned to the task
Sebastian Thrun
CS223B Computer Vision, Winter 2006
32
Today’s Goals
• Features 101
• Linear Filters and Edge Detection
• Canny Edge Detector
Sebastian Thrun
CS223B Computer Vision, Winter 2006
33
What Causes an Edge?
• Depth discontinuity
• Surface orientation
discontinuity
• Reflectance
discontinuity (i.e.,
change in surface
material properties)
• Illumination
discontinuity (e.g.,
shadow)
Slide credit: Christopher Rasmussen
Sebastian Thrun
CS223B Computer Vision, Winter 2006
34
Quiz: How Can We Find Edges?
Sebastian Thrun
CS223B Computer Vision, Winter 2006
35
Edge Finding 101
im = imread('bridge.jpg');
image(im);
figure(2);
bw = double(rgb2gray(im));
image(bw);
gradkernel = [-1 1];
dx = abs(conv2(bw, gradkernel, 'same'));
image(dx);
colorbar; colormap gray
matlab
[dx,dy] = gradient(bw);
gradmag = sqrt(dx.^2 + dy.^2);
image(gradmag);
colorbar
colormap(gray(255))
colormap(default)
Sebastian Thrun
CS223B Computer Vision, Winter 2006
36
Edge Finding 101
• Example of a linear Filter
Sebastian Thrun
CS223B Computer Vision, Winter 2006
37
Today’s Goals
•
•
•
•
Thin Lens
Aberrations
Features 101
Linear Filters and Edge Detection
Sebastian Thrun
CS223B Computer Vision, Winter 2006
38
What is Image Filtering?
Modify the pixels in an image based on
some function of a local neighborhood of
the pixels
10 5
4 5
1 1
Sebastian Thrun
3
1
7
Some function
7
CS223B Computer Vision, Winter 2006
39
Linear Filtering
• Linear case is simplest and most useful
– Replace each pixel with a linear combination of its neighbors.
• The prescription for the linear combination is called the
convolution kernel.
10
5
3
4
5
1
1
1
7

0
0
0
0
0.5
0
0
1.0 0.5

7
kernel
Sebastian Thrun
CS223B Computer Vision, Winter 2006
40
Linear Filter = Convolution
I(.) I(.) I(.)
I(.) I(.) I(.)
I(.) I(.) I(.)
g11 g12 g13
g21 g22 g23
g31 g32 g33
f (i,j) =
Sebastian Thrun
g11 I(i-1,j-1)
+ g12 I(i-1,j) + g13 I(i-1,j+1) +
g21 I(i,j-1)
+ g22 I(i,j)
g31 I(i+1,j-1)
+ g32 I(i+1,j) + g33 I(i+1,j+1)
+ g23 I(i,j+1) +
CS223B Computer Vision, Winter 2006
41
Linear Filter = Convolution
f [m, n]  I  g   I [m  k , n  l ] g[k , l ]
k ,l
with
 g[k, l ]  1
k ,l
Sebastian Thrun
CS223B Computer Vision, Winter 2006
42
Filtering Examples
Sebastian Thrun
CS223B Computer Vision, Winter 2006
43
Filtering Examples
Sebastian Thrun
CS223B Computer Vision, Winter 2006
44
Filtering Examples
Sebastian Thrun
CS223B Computer Vision, Winter 2006
45
Image Smoothing With Gaussian
figure(3);
sigma = 3;
width = 3 * sigma;
support = -width : width;
gauss2D = exp( - (support / sigma).^2 / 2);
gauss2D = gauss2D / sum(gauss2D);
smooth = conv2(conv2(bw, gauss2D, 'same'), gauss2D', 'same');
image(smooth);
colormap(gray(255));
gauss3D = gauss2D' * gauss2D;
tic ; smooth = conv2(bw,gauss3D, 'same'); toc
Sebastian Thrun
CS223B Computer Vision, Winter 2006
46
Smoothing With Gaussian
Averaging
Gaussian
Slide credit: Marc Pollefeys
Sebastian Thrun
CS223B Computer Vision, Winter 2006
47
Smoothing Reduces Noise
The effects of smoothing
Each row shows smoothing
with gaussians of different
width; each column shows
different realizations of
an image of gaussian noise.
Slide credit: Marc Pollefeys
Sebastian Thrun
CS223B Computer Vision, Winter 2006
48
Example of Blurring
Image
Blurred Image
-
Sebastian Thrun
CS223B Computer Vision, Winter 2006
=
49
Edge Detection With Smoothed Images
figure(4);
[dx,dy] = gradient(smooth);
gradmag = sqrt(dx.^2 + dy.^2);
gmax = max(max(gradmag));
imshow(gradmag);
colormap(gray(gmax));
Sebastian Thrun
CS223B Computer Vision, Winter 2006
50
Scale
Increased smoothing:
• Eliminates noise edges.
• Makes edges smoother and thicker.
• Removes fine detail.
Sebastian Thrun
CS223B Computer Vision, Winter 2006
51
The Edge Normal
S  dx 2  dy 2
dy
a  arctan
dx
Sebastian Thrun
CS223B Computer Vision, Winter 2006
52
Displaying the Edge Normal
figure(5);
hold on;
image(smooth);
colormap(gray(255));
[m,n] = size(gradmag);
edges = (gradmag > 0.3 * gmax);
inds = find(edges);
[posx,posy] = meshgrid(1:n,1:m); posx2=posx(inds); posy2=posy(inds);
gm2= gradmag(inds);
sintheta = dx(inds) ./ gm2;
costheta = - dy(inds) ./ gm2;
quiver(posx2,posy2, gm2 .* sintheta / 10, -gm2 .* costheta / 10,0);
hold off;
Sebastian Thrun
CS223B Computer Vision, Winter 2006
53
Separable Kernels
f [m, n]  I  g  I  g X  gY
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01
0
Sebastian Thrun
0
5
10
15
20
25
CS223B Computer Vision, Winter 2006
30
35
40
45
54
Combining Kernels / Convolutions
( I  g )  h  I  ( g  h)

0.0030 0.0133 0.0219 0.0133 0.0030
0.0133 0.0596 0.0983 0.0596 0.0133
0.0219 0.0983 0.1621 0.0983 0.0219
0.0133 0.0596 0.0983 0.0596 0.0133
0.0030 0.0133 0.0219 0.0133 0.0030
Sebastian Thrun

  1 1 
CS223B Computer Vision, Winter 2006
55
Effect of Smoothing Radius
1 pixel
Sebastian Thrun
3 pixels
7 pixels
CS223B Computer Vision, Winter 2006
56
Robert’s Cross Operator
1 0
0 1
0 -1 + -1 0
S=
[ I(x, y) - I(x+1, y+1) ]2 + [ I(x, y+1) - I(x+1, y) ]2
or
S = | I(x, y) - I(x+1, y+1) | + | I(x, y+1) - I(x+1, y) |
Sebastian Thrun
CS223B Computer Vision, Winter 2006
57
Sobel Operator
-1 -2 -1
S1= 0 0 0
1 2 1
Edge Magnitude =
-1
-2
-1
S2 =
2
1
2
1
2
S1 + S1
Edge Direction = tan-1
Sebastian Thrun
0
0
0
S1
S2
CS223B Computer Vision, Winter 2006
58
The Sobel Kernel, Explained
-1
-2
0
0
1
2
-1
0
1
= 1/4 * [-1 0 -1] 
1
2
1
Sobel kernel is separable!
1
0
2
0
1
0
-1
-2
-1
= 1/4 * [ 1 2 1] 
1
1
-2
2
-1
1
Sebastian Thrun
1
0
-1
Averaging done parallel to edge
CS223B Computer Vision, Winter 2006
59
Sobel Edge Detector
figure(6)
edge(bw, 'sobel')
Sebastian Thrun
CS223B Computer Vision, Winter 2006
60
Robinson Compass Masks
-1
-2
-1
0 1
0 2
0 1
0 1 2
-1 0 1
-2 -1 0
1 2 1
0 0 0
-1 -2 -1
2 1 0
1 0 -1
0 -1 -2
1
2
1
0 -1
0 -2
1 -1
0 -1 -2
-1 0 -1
2 1 0
-1 -2 -1
0 0 0
1 2 1
-2 -1 0
-1 0 1
0 1 2
Sebastian Thrun
CS223B Computer Vision, Winter 2006
61
Claim Your Own Kernel!
1
1
1
5
5
5
-1
- 2
-1
1
-2
1
-3
0
-3
0
0
0
-1
-1
-1
-3
-3
-3
1
2
1
Prewitt 1
Kirsch
Frei & Chen
1
1
1
1
2
1
0
0
0
0
0
0
-1
-1
-1
-1
-2
-1
Prewitt 2
Sebastian Thrun
Sobel
CS223B Computer Vision, Winter 2006
62
Comparison (by Allan Hanson)
• Analysis based on a step edge inclined at an angle q (relative to yaxis) through center of window.
• Robinson/Sobel: true edge contrast less than 1.6% different from
that computed by the operator.
• Error in edge direction
– Robinson/Sobel: less than 1.5 degrees error
– Prewitt: less than 7.5 degrees error
• Summary
– Typically, 3 x 3 gradient operators perform better than 2 x 2.
– Prewitt2 and Sobel perform better than any of the other 3x3 gradient
estimation operators.
– In low signal to noise ratio situations, gradient estimation operators of
size larger than 3 x 3 have improved performance.
– In large masks, weighting by distance from the central pixel is beneficial.
Sebastian Thrun
CS223B Computer Vision, Winter 2006
63