Transcript Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean Hall 5409 T-R 10:30am – 11:50am.

Computer Vision
Spring 2012 15-385,-685
Instructor: S. Narasimhan
Wean Hall 5409
T-R 10:30am – 11:50am
Image Processing and Filtering
Lecture #3
Image as a Function
• We can think of an image as a function, f,
• f: R2  R
– f (x, y) gives the intensity at position (x, y)
– Realistically, we expect the image only to be
defined over a rectangle, with a finite range:
• f: [a,b]x[c,d]  [0,1]
• A color image is just three functions pasted
together. We can write this as a “vectorvalued” function:
 r ( x, y ) 
f ( x, y )   g ( x, y ) 


 b( x, y ) 
Image as a Function
Image Processing
• Define a new image g in terms of an existing
image f
– We can transform either the domain or the range of f
• Range transformation:
What kinds of operations can this perform?
Point Operations
Point Processing
Original
Invert
Darken
Lighten
Lower Contrast
Nonlinear Lower
Contrast
Raise Contrast
Nonlinear Raise
Contrast
Point Processing
Original
x
Invert
255 - x
Darken
x - 128
Lighten
x + 128
Lower Contrast
x / 2
Raise Contrast
x * 2
Nonlinear Lower
Contrast
((x / 255.0) ^ 0.33) * 255.0
Nonlinear Raise
Contrast
((x / 255.0) ^2) * 255.0
Neighborhood Operations
Neighborhood operations
Image
Edge detection
Blur
Image Processing
• Some operations preserve the range but change
the domain of f :
What kinds of operations can this perform?
• Still other operations operate on both the
domain and the range of f .
Face Morphing
Linear Shift Invariant Systems (LSIS)
Linearity:
f1
g1
f1  f 2
g2
f2
g1  g 2
Shift invariance:
f x  a 
a
g x  a 
a
Example of LSIS
Defocused image ( g ) is a
processed version of the focused
image ( f )
g
f
Ideal lens is a LSIS f x 
LSIS
g x 
Linearity: Brightness variation
Shift invariance: Scene movement
(not valid for lenses with non-linear distortions)
Convolution
LSIS is doing convolution; convolution is linear and shift invariant
g x  

 f  hx  d
g  f h

f  
x
h 

h  
kernel h


Convolution - Example
f
g
f g
Eric Weinstein’s Math World
Convolution - Example
-1
a x 
bx 
1
1
-1
1
1
c  a b
c x 
1
-2
-1
1
2
Convolution Kernel – Impulse Response
f
g
h
g  f h
• What h will give us g = f ?
Dirac Delta Function (Unit Impulse)
1
 x 
2
Sifting property:




 f x xdx   f 0 xdx

 f 0  x dx  f 0


g x    f   x   d  f x 


    hx   d  hx 

 
 0
Point Spread Function
scene
Optical
System
image
• Ideally, the optical system should be a Dirac delta function.
• However, optical systems are never ideal.
 x 
point source
Optical
System
PSFx
point spread function
• Point spread function of Human Eyes
Point Spread Function
normal vision
myopia
astigmatism
hyperopia
Images by Richmond Eye Associates
Properties of Convolution
• Commutative
• Associative
a b  b  a
a  b c  a  b  c
• Cascade system
f
h1
g
h2
f
h1  h2
g
f
h2  h1
g
Images are Discrete and Finite
f  x, y 
g  x, y 
hx, y 
f
h
Convolution
M
N
g i, j    f m, n hi  m, j  n 
j
m 1 n 1
Fourier Transform
N
M
i
N
F u, v    f m, ne
M
m 1 n 1
Inverse Fourier Transform
f k , l  
M
N
1
F u, v e

MN u 1 v1
 ku lv 
i 2   
M N
 mu nv 
i 2 
 
M N
Averaging
Let’s think about averaging pixel values
For n=2, convolve pixel values with
2D images:
(a) use
1 2 1
1 2 1
1
then
2
or (b) use
1 2 1
1
Which is faster?

1
2
1
a O2n 1

1 2 1
2 4 2
1 2 1
b On 12 
Averaging
The convolution kernel
n2
n8
large n
Repeated averaging

Gaussian smoothing
Gaussian Smoothing
Gaussian kernel
hi, j  
1
2
2
e
1  i2  j2
 
2  2




N pixels
Filter size N   …can be very large
(truncate, if necessary)
g i, j  
1
2
2
 e
1  m 2  n 2
 
2  2




f i  m, j  n 
m 1 n 1
2D Gaussian is separable!
g i, j  
1
2
2
e
m 1
1 m2

2 2
e
1 n2

2 2
f i  m, j  n 
n 1
Use two 1D Gaussian filters
Gaussian Smoothing
• A Gaussian kernel gives less weight to pixels further
from the center of the window (5x5, sigma=0.5)
0.0000
0.0000
0.0002
0.0000
0.0000
0.0000
0.0113
0.0837
0.0113
0.0000
0.0002
0.0837
0.6187
0.0837
0.0002
0.0000
0.0113
0.0837
0.0113
0.0000
0.0000
0.0000
0.0002
0.0000
0.0000
• This kernel is an approximation of a Gaussian function:
Gaussian Smoothing
original
 2
  2.8
 4
Mean vs. Gaussian filtering
Gaussian Smoothing
by Charles Allen Gillbert
by Harmon & Julesz
http://www.michaelbach.de/ot/cog_blureffects/index.html
Gaussian Smoothing
http://www.michaelbach.de/ot/cog_blureffects/index.html
Border Problem
Border Problem
• Ignore
– Output image will be smaller than original
• Pad with constant values
– Can introduce substantial 1st order derivative values
• Pad with reflection
– Can introduce substantial 2nd order derivative values
Median Filter
• Smoothing is averaging
(a)
(a) Blurs edges
(b) Sensitive to outliers
(b)
• Median filtering
– Sort N 2  1 values around the pixel
– Select middle value (median)
sort
median
– Non-linear (Cannot be implemented with convolution)
Salt and pepper noise
3x3
5x5
7x7
Gaussian noise
Correlation
template
How do we locate the template in the image?
• Minimize
Ei, j    f m, n  t m  i, n  j 
2
m
n

  f
m
2
m, n   t m  i, n  j   2 f m, n t m  i, n  j 
2
2
n
• Maximize
Rtf i, j    t m  i, n  j  f m, n
m
n
Cross-correlation
Cross-correlation
Rtf i, j    t m  i, n  j  f m, n
m
n
Rtf  t  f
Note: R ff  f  f Auto-correlation
Problem:
t
Rtf C   Rtf B  Rtf  A
A
B
C
f
We need Rtf  A to be the maximum!
Normalized Correlation
• Account for energy differences
N tf i, j  
 t m  i, n  j  f m, n 
m
n
1

 

2
2
 t m  i, n  i   f m, n 
m n
 m n


2

1
2
Announcements
• Project 1 will be out later this evening.
• WARNING! Start projects early.
Next Class
• Image Processing and Filtering (continued)
• Fourier or frequency domain analysis
• Horn, Chapter 6