Transcript 2D Fourier Theory for Image Analysis

2D Fourier Theory
for Image Analysis
Mani Thomas
CISC 489/689
Roadmap
2D image basis
 Fourier basis
 Scale-space representation

 Gaussian
pyramid
 Laplacian pyramid
Image mosaicing
 Gabor filters

Different basis representation

Recall our discussion of basis vectors for
coordinate systems:
 Describe
point as linear combination of orthogonal basis vectors: x = a1v1 + ... + anvn

The standard basis for images is the set of
unit vectors corresponding to each pixel.
A toy example:
Hadamard basis


The standard basis is not the only one we can use to describe an
image
E.g., the Hadamard basis (basis images shown
here for 2 x 2 images, where black = +1,
white = -1)
 For the previous example, we can express the image
with these new (normalized) basis vectors as:
Coefficients of sum = projection of I onto new basis (dot product)
 These are the coordinates of the image in “Hadamard space”


We can also say that I has undergone a Hadamard transform H:
Sinusoidal Basis

Binary-valued, rectangular wave pattern of
Hadamard basis doesn’t capture real
image gradients well
 Idea:
Use smoothly-varying sinusoidal
patterns at different frequencies, angles for
basis images
Fourier Basis

The Fourier basis uses the family of
complex sinusoidal functions
2D DFT

Forward 2D DFT
1 M 1 N 1
 j 2 ( ux / M  vy / N )
F (u, v) 
f
(
x
,
y
)
e

MN x 0 y 0

Inverse 2D DFT
M 1 N 1
f ( x, y)   F (u, v)e  j 2 (ux / M vy / N )
u 0 v 0
 (u,
v) are the frequency coordinates while (x, y) are
the spatial coordinates
 M, N are the number of spatial pixels along the x, y
coordinates
Fourier Basis
Real
(cos) part
(u, v) (1, 0) (1, 1) (0, 5)
Imaginary
(sin) part
v
Fourier transform in Matlab
Discrete, 2-D Fourier & inverse Fourier
transforms are computed by fft2 and
ifft2, respectively
 fftshift: Move origin (DC component)
to image center for display
 Example:

>>
>>
>>
>>
I = imread(‘test.png’);
F = fftshift(fft2(I));
imshow(log(abs(F)),[]);
imshow(angle(F),[]);
%
%
%
%
Load grayscale image
Shifted transform
Show log magnitude
Show phase angle
Phase and Magnitude

Output of the Fourier transform is a complex number


Decompose the complex number as the magnitude and phase
components
In Matlab: u = real(z), v = imag(z), r =
abs(z), and theta = angle(z)
Image pyramid representation

Smoothing means
removing high
frequencies


Smoothing required to
avoid aliasing
Fourier transform of a
Gaussian is a Gaussian

Convolution is a
multiplication Gaussian
suppresses high
frequencies
Gaussian Pyramid



Downsampling: Cut width,
height in half at each iteration:
Upsampling S"(I): Double
size of image, interpolate
missing pixels
Let the base (the finest
resolution) of an n-level
Gaussian pyramid be defined
as P0 = I. Then the ith level
is reduced from the level below
it by:
from Forsyth & Ponce
Gaussian pyramid
Laplacian pyramid

The tip (the coarsest resolution) of an n-level
Laplacian pyramid is the same as the Gaussian
pyramid at that level: Ln(I) = Pn(I)

The ith level is expanded from the level above
according to Li(I) = Pi(I) ¡ S"(Pi+1(I))

Synthesizing the original image: Get I back by
summing upsampled Laplacian pyramid levels
Gaussian and Laplacian
courtesy of Wolfram

Gaussian – Smoothing pyramid


Each level is a smoothed and decimated signal of the previous
Laplacian – Band pass filter of the images

Each level is the difference of a more smoothed and less
smoothed image
Summation Property
If L0, L1 LN is the sequence of laplacians
Li = Gi – EXPAND[Gi+1], 0<i<N
LN = GN
 The steps used to construct the Laplacian
can be reversed to get the original

 Expand
Li and add it to Li-1 to Gi-1
G0 = i=0N Li
Applications – Image Mosaicing
Seamless joining of images to get a larger view
Multi-resolution spline interpolation
Laplacian
level
4
Laplacian
level
2
Laplacian
level
0
left pyramid
right pyramid
blended pyramid
Image mosaicing

Automatic mosaicing
 Cross
correlation to compute translation
between images

Matlab demo – Burt and Adelson’s paper
 http://www.cs.huji.ac.il/course/2003/impr/splin
e83.pdf
Application - Coarse-to-Fine Estimation
warp
+
a

aw
refine
J pixels
u=1.25

u=2.5 pixels
Δa
u=5 pixels
image J
Pyramid of image J
u=10 pixels
image I
Pyramid of image I
Slide from CS 223-B L9 by Richard Szeliski
Gabor filters

Gaussian windowed Fourier Transform
 Make
convolution kernels from product of
Fourier basis images and Gaussians
£
Odd
(sin)
Even
(cos)
Frequency
=
Texture Representation: Filter
Responses

Choose a group of filters
 Edge/Bar
filters: Something like Gabor filters at
different orientations, scales
 Spot filters: Center-surround filters like a
Gaussian/difference of Gaussians at multiple scales

Run filters over image to get a set of response
images
 Each
contains specific texture information
Example: Filter Responses
Filter
bank
Input
image
from Forsyth & Ponce
Texture Similarity based on
Response Statistics

Collect statistics of responses over an
image or subimage
 Mean
of squared response
 Mean and variance of squared response

Euclidean distance between vectors of
response statistics for two images is
measure of texture similarity
Conclusions


2D Fourier Theory
Image pyramid representation
 Gaussian
pyramid
 Laplacian pyramid

Applications of Image Pyramids
 Image Mosaicing
 Gaussian + Laplacian pyramids (Burt and Adelson)

Texture statistics
 Gabor
filters