Transcript Computer Vision Spring 2006 15-385,-685 Instructor: S. Narasimhan Wean 5403 T-R 3:00pm – 4:20pm.

Computer Vision
Spring 2006 15-385,-685
Instructor: S. Narasimhan
Wean 5403
T-R 3:00pm – 4:20pm
Image Processing and Filtering
(continued)
Lecture #6
Images are Discrete and Finite
f  x, y 
g  x, y 
hx, y 
f
h
Convolution
M
N
g i, j    f m, n hi  m, j  n 
j
m 1 n 1
Fourier Transform
N
M
i
N
F u, v    f m, ne
M
m 1 n 1
Inverse Fourier Transform
f k , l  
M
N
1
F u, v e

MN u 1 v1
 ku lv 
i 2   
M N
 mu nv 
i 2 
 
M N
Averaging
Let’s think about averaging pixel values
For n=2, convolve pixel values with
2D images:
(a) use
1 2 1
1 2 1
1
then
2
or (b) use
1 2 1
1
Which is faster?

1
2
1
a O2n 1

1 2 1
2 4 2
1 2 1
b On 12 
Averaging
The convolution kernel
n2
n8
large n
Repeated averaging

Gaussian smoothing
Gaussian Smoothing
Gaussian kernel
hi, j  
1
2
2
e
1  i2  j2
 
2  2




N pixels
Filter size N   …can be very large
(truncate, if necessary)
g i, j  
1
2
2
 e
1  m 2  n 2
 
2  2




f i  m, j  n 
m 1 n 1
2D Gaussian is separable!
g i, j  
1
2
2
e
m 1
1 m2

2 2
e
1 n2

2 2
f i  m, j  n 
n 1
Use two 1D Gaussian filters
Gaussian Smoothing
• A Gaussian kernel gives less weight to pixels further
from the center of the window
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
90 90 90 90 90
0
0
1
2
1
0
0
0
90 90 90 90 90
0
0
2
4
2
0
0
0
90 90 90 90 90
0
0
1
2
1
0
0
0
90
90 90 90
0
0
0
0
0
90 90 90 90 90
0
0
0
0
0
0
0
0
0
0
0
0
0
0
90
0
0
0
0
0
0
0
0
•0 This
kernel is an approximation of a Gaussian function:
0 0 0 0 0 0 0 0 0
Gaussian Smoothing
original
 2
  2.8
 4
Mean vs. Gaussian filtering
Gaussian Smoothing
by Charles Allen Gillbert
by Harmon & Julesz
http://www.michaelbach.de/ot/cog_blureffects/index.html
Gaussian Smoothing
http://www.michaelbach.de/ot/cog_blureffects/index.html
Border Problem
Border Problem
• Ignore
– Output image will be smaller than original
• Pad with constant values
– Can introduce substantial 1st order derivative values
• Pad with reflection
– Can introduce substantial 2nd order derivative values
Median Filter
• Smoothing is averaging
(a)
(a) Blurs edges
(b) Sensitive to outliers
(b)
• Median filtering
– Sort N 2  1 values around the pixel
– Select middle value (median)
sort
median
– Non-linear (Cannot be implemented with convolution)
Salt and pepper noise
3x3
5x5
7x7
Gaussian noise
Correlation
template
How do we locate the template in the image?
• Minimize
Ei, j    f m, n  t m  i, n  j 
2
m
n

  f
m
2
m, n   t m  i, n  j   2 f m, n t m  i, n  j 
2
2
n
• Maximize
Rtf i, j    t m  i, n  j  f m, n
m
n
Cross-correlation
Cross-correlation
Rtf i, j    t m  i, n  j  f m, n
m
n
Rtf  t  f
Note: t  f  f  t
R ff  f  f
Auto-correlation
Problem:
t
Rtf C   Rtf B  Rtf  A
A
B
C
f
We need Rtf  A to be the maximum!
Normalized Correlation
• Account for energy differences
N tf i, j  
 t m  i, n  j  f m, n 
m
n
1

 

2
2
 t m  i, n  i   f m, n 
m n
 m n


2

1
2
Image Processing in the Fourier Domain
Magnitude of the FT
Does not look anything like what we have seen
Image Processing in the Fourier Domain
Magnitude of the FT
Does not look anything like what we have seen
Convolution is Multiplication in Fourier Domain
|F(sx,sy)|
f(x,y)
*
h(x,y)
|H(sx,sy)|
g(x,y)
|G(sx,sy)|
Low-pass Filtering
Let the low frequencies pass and eliminating the high frequencies.
Generates image with overall
shading, but not much detail
High-pass Filtering
Lets through the high frequencies (the detail), but eliminates the low
frequencies (the overall shape). It acts like an edge enhancer.
Boosting High Frequencies
Most information at low frequencies!
Fun with Fourier Spectra
Image as a Discrete Function
Digital Images
The scene is
– projected on a 2D plane,
– sampled on a regular grid, and each sample is
– quantized (rounded to the nearest integer)
f i, j   Quantize f i, j
Image as a matrix
Sampling Theorem
Continuous signal:
f x 
x
Shah function (Impulse train):
sx  
s x 

 x  nx 
n  
x
x0
Sampled function:
0

f s x   f x sx   f x    x  nx0 
n  
Sampling Theorem
Sampled function:
Sampling
frequency

f s x   f x sx   f x    x  nx0 
1
x0
n  
1  
n
FS u   F u  S u   F u     u  
x0 n  
x0 
FS u 
F u 
A
A
umax
x0
umax
u
1
Only if u max 
1
2 x0
x0
u
Nyquist Theorem
If u max
FS u 
1

2 x0
A
x0
Aliasing
umax
1
u
x0
When can we recover F u  from FS u  ?
1
Only if u max 
(Nyquist Frequency)
2 x0
We can use

 x0
C u   

0
u 1
2 x0
otherwise
Then F u   FS u Cu  and f x   IFTF u 
Sampling frequency must be greater than 2umax
Aliasing
Alias: n., an assumed name
Input signal:
Matlab output:
Picket fence receding
into the distance will
produce aliasing…
WHY?
x = 0:.05:5; imagesc(sin((2.^x).*x))
Alias!
Not enough samples
Image Scaling
This image is too big to
fit on the screen. How
can we reduce it?
How to generate a halfsized version?
Image Sub-Sampling
1/8
1/4
Throw away every other row and
column to create a 1/2 size image
- called image sub-sampling
Image Sub-Sampling
1/2
1/4
(2x zoom)
1/8
(4x zoom)
Good and Bad Sampling
Good sampling:
•Sample often or,
•Sample wisely
Bad sampling:
•see aliasing in action!
Really bad in video
Sub-Sampling with Gaussian Pre-Filtering
G 1/8
G 1/4
Gaussian 1/2
• Solution: filter the image, then subsample
– Filter size should double for each ½ size reduction. Why?
Sub-Sampling with Gaussian Pre-Filtering
Gaussian 1/2
G 1/4
G 1/8
Compare with...
1/2
1/4
(2x zoom)
1/8
(4x zoom)
Aliasing
Canon D60 (w/ anti-alias filter)
Sigma SD9 (w/o anti-alias filter)
From Rick Matthews website, images by Dave Etchells
Figure from David Forsyth
Next Class
• Image Processing and Filtering (continued) –
Edge Detection
• Horn, Chapter 6