Transcript Computer Vision Spring 2006 15-385,-685 Instructor: S. Narasimhan Wean 5403 T-R 3:00pm – 4:20pm.

Computer Vision
Spring 2006 15-385,-685
Instructor: S. Narasimhan
Wean 5403
T-R 3:00pm – 4:20pm
Announcements
• Homework 1 is due today in class.
• Homework 2 will be out later this evening (due in 2 weeks).
• Start homeworks early.
• Post questions on bboard.
Image Processing and Filtering
Lecture #5
Image as a Function
• We can think of an image as a function, f,
• f: R2  R
– f (x, y) gives the intensity at position (x, y)
– Realistically, we expect the image only to be
defined over a rectangle, with a finite range:
• f: [a,b]x[c,d]  [0,1]
• A color image is just three functions pasted
together. We can write this as a “vectorvalued” function:
 r ( x, y ) 
f ( x, y )   g ( x, y ) 


 b( x, y ) 
Image as a Function
Image Processing
• Define a new image g in terms of an existing
image f
– We can transform either the domain or the range of f
• Range transformation:
What kinds of operations can this perform?
Image Processing
• Some operations preserve the range but change
the domain of f :
What kinds of operations can this perform?
• Still other operations operate on both the
domain and the range of f .
Linear Shift Invariant Systems (LSIS)
Linearity:
f1
g1
f1  f 2
g2
f2
g1  g 2
Shift invariance:
f x  a 
a
g x  a 
a
Example of LSIS
Defocused image ( g ) is a
processed version of the focused
image ( f )
g
f
Ideal lens is a LSIS f x 
LSIS
g x 
Linearity: Brightness variation
Shift invariance: Scene movement
(not valid for lenses with non-linear distortions)
Convolution
LSIS is doing convolution; convolution is linear and shift invariant
g x  

 f  hx  d
g  f h

f  
x
h 

h  
kernel h


Convolution - Example
f
g
f g
Eric Weinstein’s Math World
Convolution - Example
-1
a x 
bx 
1
1
-1
1
1
c  a b
c x 
1
-2
-1
1
2
Convolution Kernel – Impulse Response
f
g
h
g  f h
• What h will give us g = f ?
Dirac Delta Function (Unit Impulse)
1
 x 
2
Sifting property:




 f x xdx   f 0 xdx

 f 0  x dx  f 0


g x    f   x   d  f x 


    hx   d  hx 

 
 0
Point Spread Function
scene
Optical
System
image
• Ideally, the optical system should be a Dirac delta function.
• However, optical systems are never ideal.
 x 
point source
Optical
System
PSFx
point spread function
• Point spread function of Human Eyes
Point Spread Function
normal vision
myopia
astigmatism
hyperopia
Images by Richmond Eye Associates
Properties of Convolution
• Commutative
• Associative
a b  b  a
a  b c  a  b  c
• Cascade system
f
h1
g
h2
f
h1  h2
g
f
h2  h1
g
How to Represent Signals?
• Option 1: Taylor series represents any function using
polynomials.
• Polynomials are not the best - unstable and not very
physically meaningful.
• Easier to talk about “signals” in terms of its “frequencies”
(how fast/often signals change, etc).
Jean Baptiste Joseph Fourier (1768-1830)
• Had crazy idea (1807):
•
Any periodic function
can be rewritten as a
weighted sum of Sines and
Cosines of different
frequencies.
• Don’t believe it?
– Neither did Lagrange,
Laplace, Poisson and
other big wigs
– Not translated into
English until 1878!
•
But it’s true!
– called Fourier Series
– Possibly the greatest tool
used in Engineering
A Sum of Sinusoids
• Our building block:
Asin(x   
• Add enough of them to
get any signal f(x) you
want!
• How many degrees of
freedom?
• What does each control?
• Which one encodes the
coarse vs. fine structure of
the signal?
Fourier Transform
• We want to understand the frequency  of our signal. So, let’s
reparametrize the signal by  instead of x:
Fourier
Transform
f(x)
F()
• For every  from 0 to inf, F() holds the amplitude A and phase
 of the corresponding sine
Asin(x   
– How can F hold both? Complex number trick!
F ( )  R( )  iI ( )
A   R( )  I ( )
2
F()
2
Inverse Fourier
Transform
I ( )
  tan
R( )
1
f(x)
Time and Frequency
• example : g(t) = sin(2pf t) + (1/3)sin(2p(3f) t)
Time and Frequency
• example : g(t) = sin(2pf t) + (1/3)sin(2p(3f) t)
=
+
Frequency Spectra
• example : g(t) = sin(2pf t) + (1/3)sin(2p(3f) t)
=
+
Frequency Spectra
• Usually, frequency is more interesting than the phase
Frequency Spectra
=
=
+
Frequency Spectra
=
=
+
Frequency Spectra
=
=
+
Frequency Spectra
=
=
+
Frequency Spectra
=
=
+
Frequency Spectra

1
= A sin(2 kt )
k 1 k
Frequency Spectra
FT: Just a change of basis
M * f(x) = F()
*
.
.
.
=
IFT: Just a change of basis
M-1 * F() = f(x)
*
.
.
.
=
Fourier Transform – more formally
Represent the signal as an infinite weighted sum
of an infinite number of sinusoids

F u    f x e
i 2ux

Note:
dx
eik  cosk  i sin k
i  1
Arbitrary function
Single Analytic Expression
Spatial Domain (x)
Frequency Domain (u)
(Frequency Spectrum F(u))
Inverse Fourier Transform (IFT)

f x    F u e

i 2ux
dx
Fourier Transform
• Also, defined as:

F u    f x e

Note:
iux
dx
eik  cosk  i sin k
• Inverse Fourier Transform (IFT)
1
f x  
2



F u eiux dx
i  1
Fourier Transform Pairs (I)
Note that these are derived using
angular frequency ( e iux )
Fourier Transform Pairs (I)
Note that these are derived using
angular frequency ( e iux )
Fourier Transform and Convolution
Let
g  f h

Then Gu    g x e i 2ux dx





 


f  hx   ei 2ux ddx
  f  e

i 2u
 



 f  e
i 2u
d


d hx   e i 2u  x  dx
 hx'e


i 2ux '

dx'
 F u H u 
Convolution in spatial domain
 Multiplication in frequency domain
Fourier Transform and Convolution
Spatial Domain (x)
Frequency Domain (u)
g  f h
g  fh
G  FH
G  F H
So, we can find g(x) by Fourier transform
g

IFT
G
f

FT

F
h
FT

H
Properties of Fourier Transform
Spatial Domain (x)
Frequency Domain (u)
c1 f x   c2 g x
c1F u   c2Gu 
Scaling
f ax
1 u
F 
a a
Shifting
f x  x0 
ei 2ux0 F u 
Symmetry
F x 
f  u 
Conjugation
f  x 
Convolution
f x   g x 
F   u 
Differentiation
d n f x 
dxn
i2u n F u 
Linearity
F u Gu 
Note that these are derived using
frequency ( e i 2ux )
Properties of Fourier Transform
Example use: Smoothing/Blurring
• We want a smoothed function of f(x)
g x   f x  hx 
• Let us use a Gaussian kernel
hx 
 1 x2 
1
h x  
exp
2
2

2 


• Then
 1
2 2
H u   exp 2u   
 2

Gu   F u H u 

x
H u 
1
2
u
H(u) attenuates high frequencies in F(u) (Low-pass Filter)!
Image as a Discrete Function
Digital Images
The scene is
– projected on a 2D plane,
– sampled on a regular grid, and each sample is
– quantized (rounded to the nearest integer)
f i, j   Quantize f i, j
Image as a matrix
Sampling Theorem
Continuous signal:
f x 
x
Shah function (Impulse train):
sx  
s x 

 x  nx 
n  
x
x0
Sampled function:
0

f s x   f x sx   f x    x  nx0 
n  
Sampling Theorem
Sampled function:
Sampling
frequency

f s x   f x sx   f x    x  nx0 
1
x0
n  
1  
n
FS u   F u  S u   F u     u  
x0 n  
x0 
FS u 
F u 
A
A
umax
x0
umax
u
1
Only if u max 
1
2 x0
x0
u
Nyquist Theorem
If u max
FS u 
1

2 x0
A
x0
Aliasing
umax
1
u
x0
When can we recover F u  from FS u  ?
1
Only if u max 
(Nyquist Frequency)
2 x0
We can use

 x0
C u   

0
u 1
2 x0
otherwise
Then F u   FS u Cu  and f x   IFTF u 
Sampling frequency must be greater than 2umax
Aliasing
Announcements
• Homework 1 is due today in class.
• Homework 2 will be out later this evening.
• Start homeworks early.
• Post questions on bboard.
Next Class
• Image Processing and Filtering (continued)
• Horn, Chapter 6