Transcript Document 7112441

CSCE 641 Computer Graphics:
Image Sampling and Reconstruction
Jinxiang Chai
Review: 1D Fourier Transform
A function f(x) can be represented as a weighted combination of
phase-shifted sine waves 
f ( x) 
 F (u )e
i 2ux
du
Inverse Fourier
Transform

How to compute F(u)?

F (u ) 


f ( x)e i 2ux dx
Fourier Transform
Review: Box Function
f(x)
1
f ( x)  
0
x1
2
x1
2
x
|F(u)|
sin u
F (u ) 
 sin c(u )
u
If f(x) is bounded, F(u) is unbounded
u
Review: Cosine
1.5

1
0.5
0
-1
-0.5
-1
-1.5
cos( 2x)
If f(x) is even, so is F(u)
1
Review: Gaussian
0.1 8
0.18
0.1 3
0.13
0.08
0.8
0.03
0.3
-0.02
e 2
1
e
2 
 u
2 2 2
x2
 2
2
-0.2
e
2 2 2u 2
If f(x) is gaussian, F(u) is also guassian.
Review: Properties
Linearity:
af ( x)  bg ( x)  aF (u )  bG (u )
Time shift:
f ( x  x0 )  e  i 2ux0 F (u )
Derivative:
df ( x )
 uF (u )
x
F (u )
f ( x)dx 
u
Integration:

Convolution:
f ( x)  g ( x)  F (u )G (u )
Outline
2D Fourier Transform
Nyquist sampling theory
Antialiasing
Gaussian pyramid
Extension to 2D
Fourier Transform:
 
F (u , v) 

f ( x, y )e i 2 (ux vy ) dxdy
  
Inverse Fourier transform:
 
f ( x, y ) 

i 2 ( ux  vy )
F
(
u
,
v
)
e
dudv

  
Building Block for 2D Transform
Building block:
Frequency:
Orientation:
Oriented wave fields
e
i 2 ( ux  vy )
u 2  v2
a tan( v, u )
Building Block for 2D Transform
Building block:
Frequency:
Orientation:
Oriented wave fields
Higher frequency
e
i 2 ( ux  vy )
u 2  v2
a tan( v, u )
Some 2D Transforms
From Lehar
Some 2D Transforms
Why we have a DC component?
From Lehar
Some 2D Transforms
Why we have a DC component?
From Lehar
Some 2D Transforms
Why we have a DC component?
From Lehar
Some 2D Transforms
Why we have a DC component?
From Lehar
Some 2D Transforms
Why we have a DC component?
- the sum of all pixel values
 
F (0,0) 
  f ( x, y ) e
 i 2 ( 0 x  0 y )
dxdy
  
From Lehar
Some 2D Transforms
Why we have a DC component?
- the sum of all pixel values
 
F (0,0) 
  f ( x, y ) e
 i 2 ( 0 x  0 y )
dxdy
  
Oriented stripe in spatial domain =
an oriented line in spatial domain
From Lehar
2D Fourier Transform
Why?
- Any relationship between two
slopes?
2D Fourier Transform
Why?
- Any relationship between two
slopes?
Linearity
2D Fourier Transform
Why?
- Any relationship between two
slopes?
Linearity
Why is the spectrum
bounded?
Online Java Applet
http://www.brainflux.org/java/classes/FFT2D
Applet.html
2D Fourier Transform Pairs
Gaussian



  ( x  k xTx , y  k yTy )
k x   k y  
Gaussian


 
 (u 
ku   k v  
df ( x, y )
x
uF (u , v)
df ( x, y )
y
vF (u , v)
ku
k
,v  v )
Tx
Ty
2D Image Filtering
Fourier
transform
Inverse
transform
From Lehar
2D Image Filtering
Fourier
transform
Inverse
transform
Low-pass
filter
From Lehar
2D Image Filtering
Fourier
transform
Inverse
transform
Low-pass
filter
high-pass
filter
From Lehar
2D Image Filtering
Fourier
transform
Inverse
transform
Low-pass
filter
high-pass
filter
band-pass
filter
From Lehar
Aliasing
Why does this happen?
Aliasing
How to reduce it?
Sampling Analysis
fs(x)
f(x)
Sampling
x
x
… -2T -T 0 T 2T …
Sampling Analysis
fs(x)
f(x)
Sampling
x
Reconstruction
x
… -2T -T 0 T 2T …
Sampling Analysis
fs(x)
f(x)
Sampling
x
Reconstruction
x
… -2T -T 0 T 2T …
What sampling rate (T) is sufficient to reconstruct
the continuous version of the sampled signal?
Sampling Theory
• How many samples are required to
represent a given signal without loss of
information?
• What signals can be reconstructed without
loss for a given sampling rate?
Sampling Analysis: Spatial Domain

  ( x  kT )
f(x)
k  
X
… -2T -T 0 T 2T …
x
f ( x) 

  ( x  kT )
k  
?
fs(x)
x
… -2T -T 0 T 2T …
x
Sampling Analysis: Spatial Domain

  ( x  kT )
f(x)
k  
X
… -2T -T 0 T 2T …
x
f ( x) 
x

  ( x  kT )
k  
?
fs(x)
What happens in Frequency
domain?
x
… -2T -T 0 T 2T …
Fourier Transform of Dirac Comb
T

  ( x  kT )
k  
Review: Dirac Delta and its
Transform
 x  0
 ( x)  
0 x  0
f(x)
  ( x)dx  1


 f ( x) ( x)dx  f (0)

x
|F(u)|
1
F (u )  1
u
Fourier transform and inverse Fourier transform are
qualitatively the same, so knowing one direction gives you
the other
Review: Fourier Transform
Properties
Linearity:
af ( x)  bg ( x)  aF (u )  bG (u )
Time shift:
f ( x  x0 )  e  i 2ux0 F (u )
Derivative:
df ( x )
 uF (u )
x
F (u )
f ( x)dx 
u
Integration:

Convolution:
f ( x)  g ( x)  F (u )G (u )
Fourier Transform of Dirac Comb
T

  ( x  kT )
k  

e
k  
 i 2ukT
Fourier Transform of Dirac Comb

e
k  
i 2ukT

k
   (u  )
T
k  
Fourier Transform of Dirac Comb
T
1/T


  ( x  kT )
k
T
  (u  )
k  
k  

e
 i 2ukT
k  
Moving the spikes closer together in the spatial domain
moves them farther apart in the frequency domain!
Sampling Analysis: Spatial Domain

  ( x  kT )
f(x)
k  
X
… -2T -T 0 T 2T …
x
f ( x) 
x

  ( x  kT )
k  
?
fs(x)
What happens in Frequency
domain?
x
… -2T -T 0 T 2T …
Sampling Analysis: Freq. Domain

n
T
  (u  )
F(u)
n  

-fmax
fmax
u
… -1/T
0
1/T …
u
Sampling Analysis: Freq. Domain

n
T
  (u  )
F(u)
n  

-fmax
fmax
u
… -1/T
0
How does the convolution result look like?
1/T …
u
Sampling Analysis: Freq. Domain
 (u )
F(u)

-fmax
fmax
u
… -1/T
0
1/T …
u
Sampling Analysis: Freq. Domain
 (u )
F(u)

-fmax
fmax
u
… -1/T
0
F (u )   (u )
1/T …
u
Sampling Analysis: Freq. Domain
 (u )
F(u)

-fmax
fmax
u
… -1/T
0
F (u )   (u )  G (u )
G(0)?
G(fmax)?
G(u)?
1/T …
u
Sampling Analysis: Freq. Domain
 (u )
F(u)

-fmax
fmax
u
… -1/T
0
F (u )   (u )  G (u )
G(0)
= F(0)
G(fmax) = F(fmax)
G(u)
= F(u)
1/T …
u
Sampling Analysis: Freq. Domain
 (u )
F(u)
1
T
How about  (u  )

-fmax
u
fmax
… -1/T
1/T …
0
F (u )   (u )
Fs(u)
-1/T
-fmax
fmax
1/T
u
u
Sampling Analysis: Freq. Domain
 (u )
F(u)
1
T
How about  (u  )

-fmax
u
fmax
… -1/T
1/T …
0
F (u )   (u )
Fs(u)
-1/T
-fmax
fmax
1/T
u
u
Sampling Analysis: Freq. Domain

n
T
  (u  )
F(u)
n  

-fmax
u
fmax
… -1/T
F (u ) 
1/T …
0

n
  (u  T )
n  
Fs(u)
-1/T
-fmax
fmax
1/T
u
u
Sampling Theory
• How many samples are required to
represent a given signal without loss of
information?
• What signals can be reconstructed without
loss for a given sampling rate?
Sampling Analysis: Freq. Domain

n
T
  (u  )
F(u)
n  

-fmax
u
fmax
… -1/T
F (u ) 

n
  (u  T )
n  
How can we reconstruct the
original signal?
-1/T
1/T …
0
Fs(u)
-fmax
fmax
1/T
u
u
Reconstruction in Freq. Domain
Fs(u)
box(u)

fmax
-fmax
u
-fmax

Fs (u )  box(u )
-fmax
fmax
u
fmax
u
Signal Reconstruction in Freq.
Domain
Fs(u)
Fourier
transform
x
-fmax
… -2T -T 0 T 2T …
u
fmax
Fs (u )  box(u )
f(x)
F(u)
Inverse Fourier
transform
x
-fmax
fs(x)
fmax
u
Signal Reconstruction in Spatial
Domain
sinc(x)
fs(x)

x
… -2T -T 0 T 2T …
f s ( x)  sin( x)
x
Aliasing
Why does this happen?
Sampling Analysis
When does aliasing happen?
f(x)
F(u)
x
-fmax
x
0
u
Fs(u)
fs(x)
… -T
fmax
T
…
-fmax
fmax
u
Sampling Analysis
When does aliasing happen?
1
 2 f max
T
f(x)
F(u)
x
-fmax
x
0
u
Fs(u)
fs(x)
… -T
fmax
T
…
-fmax
fmax
u
Sampling Analysis
When does aliasing happen?
1
 2 f max
T
f(x)
F(u)
x
-fmax
x
0
u
Fs(u)
fs(x)
… -T
fmax
T
…
-fmax
fmax
u
Sampling Analysis
When does aliasing happen?
1
 2 f max
T
f(x)
F(u)
x
-fmax
x
0
u
Fs(u)
fs(x)
… -T
fmax
T
…
-fmax
fmax
u
Sampling Analysis
fs(x)
f(x)
Sampling
x
Reconstruction
x
… -2T -T 0 T 2T …
What sampling rate (T) is sufficient to reconstruct
the continuous version of the sampled signal?
Sampling Analysis
fs(x)
f(x)
Sampling
x
Reconstruction
x
… -2T -T 0 T 2T …
What sampling rate (T) is sufficient to reconstruct
the continuous version of the sampled signal?
Sampling Rate ≥ 2 * max frequency in the signal
• this is known as the Nyquist Rate
Antialiasing
Antialiasing
Increase the sampling rate to above twice the highest
frequency
-fmax
fmax
u
Antialiasing
Increase the sampling rate to above twice the highest
frequency
-fmax
fmax
u
Introduce an anti-aliasing filter to reduce fmax
Fs(u)
-fmax
fmax
u
Antialiasing
Increase the sampling rate to above twice the highest
frequency
fmax
-fmax
u
Introduce an anti-aliasing filter to reduce fmax
Fs(u)
-fmax fmax
u
Sinc Filter
Is this a good filter?
F(u)
-fmax fmax
u
Sinc Filter
Is this a good filter?
F(u)
f ( x) 
Inverse Fourier
transform
sin x
 sin c( x)
x
x
-fmax fmax
u
Sinc Filter
Is this a good filter?
F(u)
f ( x) 
Inverse Fourier
transform
sin x
 sin c( x)
x
x
-fmax fmax
u
Multiplying with a box
function in frequency domain
Convolution with a sinc
function in spatial domain
Sinc Filter
Is this a good filter?
F(u)
f ( x) 
Inverse Fourier
transform
sin x
 sin c( x)
x
x
-fmax fmax
u
Multiplying with a box
Convolution with a sinc
function in frequency domain
function in spatial domain
Good: removes all frequency components above a given bandwidth
Bad:
- an infinite support in spatial domain and hard to do in the spatial domain
- Fluctuation causes ripples
- Negative weights (not good for filtering pixles)
Good Pre-filtering filters
Finite support in frequency domain
- cut-off frequency
Finite support in spatial domain
- implementation in spatial domain
Positive weights
- good for filtering pixels
Other Filter Choices
f(x)
Mean filter
|F(u)|
sinc(u)
u
x
e 2
 u
2 2 2
Sin2c(u)
Triangular filter
x2
0.1 8
Gaussian filter
0.1 3
 2
1
e 2
2 
0.08
0.03
0.18
0.13
e 2
 u
2 2 2
0.8
0.3
-0.02
-0.2
Cannot have a filter with finite support in both spatial and frequency
domain (box, mean, triangular)
- box filter: finite in freq. and infinite in spatial domain
- mean and triangular: infinite in freq. and finite in spatial domain
Gaussian filter provides a good tradeoff between two domains
Other Filter Choices
f(x)
Mean filter
|F(u)|
sinc(u)
u
x
e 2
 u
2 2 2
Sin2c(u)
Triangular filter
x2
0.1 8
Gaussian filter
0.1 3
 2
1
e 2
2 
0.08
0.03
0.18
0.13
e 2
 u
2 2 2
0.8
0.3
-0.02
-0.2
Cannot have a filter with finite support in both spatial and frequency
domain (box, mean, triangular)
- box filter: finite in freq. and infinite in spatial domain
- mean and triangular: infinite in freq. and finite in spatial domain
Gaussian filter provides a good tradeoff between two domains
Gaussian Filter
Fs(u)
-fmax
fmax
u
Gaussian Filter
Fs(u)
0.18
Inverse Fourier
transform
0.13
0.08
0.03
-fmax
fmax
u
-0.02
Gaussian Filter
Fs(u)
0.18
Inverse Fourier
transform
0.13
0.08
0.03
-fmax
fmax
Multiplying with gaussian
function in frequency domain
u
-0.02
Convolution with a gaussian
function in spatial domain
Gaussian Filter
Fs(u)
0.18
Inverse Fourier
transform
0.13
0.08
0.03
-fmax
fmax
Multiplying with gaussian
function in frequency domain
u
-0.02
Convolution with a gaussian
function in spatial domain
- Since the Gaussian function decays rapidly, it is
reasonable to truncate the filter window
- Its standard deviation controls the smoothness of filtered
signal: large σ in spatial domain = lower cutoff frequency
- Easy to implement it in the spatial domain
Filtering in Spatial Domain
=
Filtered image
Input image
Filter function
Gaussian Filtering in Spatial
Domain
Discrete approximation to
Gaussian function with σ =1.0
Filtering in Spatial Domain
Discrete approximation to
Gaussian function with σ =1.0
Filtering in Spatial Domain
Discrete approximation to
Gaussian function with σ =1.0
Filtering in Spatial Domain
Discrete approximation to
Gaussian function with σ =1.0
Filtering in Spatial Domain
Discrete approximation to
Gaussian function with σ =1.0
Filtered_I45 =

pixelswindown
X
Gaussian Pre-filtering
G 1/8
G 1/4
Gaussian 1/2
Solution: filter the image, then subsample
– Filter size should double for each ½ size reduction.
Why?
Gaussian Pre-filtering
Gaussian 1/2
G 1/4
G 1/8
Solution: filter the image, then subsample
– Filter size should double for each ½ size reduction.
Why?
Without Prefiltering
1/2
1/4
(2x zoom)
1/8
(4x zoom)
Image Pyramids
Known as a Gaussian Pyramid
– MipMap (Williams, 1983)
MipMap
Mip (latin phase): many things in a small place
Used for texture mapping
Small-scale texture used for rendering distant objects
Gaussian Pyramid
filter mask
Repeat
– Filter
– Subsample
Until minimum resolution reached
– can specify desired number of levels (e.g., 3-level pyramid)
The whole pyramid is only 4/3 the size of the original image!
Laplacian Pyramid
Laplacian Pyramid (subband images)
- Created from Gaussian pyramid by subtraction
- Encode fine to course structure in a different level
Laplacian Pyramid
What happens in frequency domain?
Where Are They Used?
Texture mapping
- Mipmap
Image processing
- Image composition/blending
- Image compression
Computer vision
- Image matching
- Motion tracking
- Optical flow estimation
Summary
2D Fourier Transform
Nyquist sampling theory
- Sampling Rate ≥ 2 * max frequency in the signal
Antialiasing
- prefiltering
Gaussian pyramid
- Mipmap
Next Lecture
Image Processing
- Filtering noise
- Bilateral filter
- Feature extraction