Transcript Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am.

Computer Vision
Spring 2012 15-385,-685
Instructor: S. Narasimhan
Wean 5409
T-R 10:30am – 11:50am
Image Resampling and Pyramids
Lecture #8
Image Scaling
This image is too big to
fit on the screen. How
can we reduce it?
How to generate a halfsized version?
Image Sub-Sampling
1/8
1/4
Throw away every other row and
column to create a 1/2 size image
- called image sub-sampling
Image Sub-Sampling
1/2
1/4
(2x zoom)
1/8
(4x zoom)
Good and Bad Sampling
Good sampling:
•Sample often or,
•Sample wisely
Bad sampling:
•Aliasing!
Aliasing
Alias: n., an assumed name
Input signal:
Matlab output:
Picket fence receding
into the distance will
produce aliasing…
WHY?
x = 0:.05:5; imagesc(sin((2.^x).*x))
Alias!
Not enough samples
Really bad in video
Wagon-wheel effect
Stroboscopic effect
Sampling Theorem
Continuous signal:
f (x )
x
Shah function (Impulse train):
s(x ) =
s(x )
¥
å d (x - nx )
n = -¥
x
x0
Sampled function:
0
¥
f s (x )= f (x )s(x ) = f (x )å d (x - nx0 )
n = -¥
Sampling Theorem
Sampled function:
Sampling
frequency
¥
f s (x )= f (x )s(x ) = f (x )å d (x - nx0 )
1
x0
n = -¥
1 ¥ æ
nö
FS ( u ) = F ( u ) * S ( u ) = F ( u ) * å d ç u - ÷
x0 n= -¥ è
x0 ø
FS (u )
F (u )
A
umax
A
u
x0
umax
1
Only if
umax
1
£
2 x0
x0
u
Nyquist Frequency
If u max
FS (u )
1
>
2 x0
A
x0
Aliasing
umax
1
u
x0
When can we recover F (u ) from FS (u ) ?
Only if
umax
We can use
1
£
(Nyquist Frequency)
2 x0
ìï x0
C (u )= í
ïî 0
u< 1
2 x0
otherwise
Then F (u )= FS (u )C (u ) and
f (x )= IFT[F (u )]
Sampling frequency must be greater than 2umax
Sub-Sampling with Gaussian Pre-Filtering
G 1/8
G 1/4
Gaussian 1/2
• Solution: filter the image, then subsample
– Filter size should double for each ½ size reduction. Why?
Sub-Sampling with Gaussian Pre-Filtering
Gaussian 1/2
G 1/4
G 1/8
Compare with...
1/2
1/4
(2x zoom)
1/8
(4x zoom)
Aliasing
Canon D60 (w/ anti-alias filter)
Sigma SD9 (w/o anti-alias filter)
From Rick Matthews website, images by Dave Etchells
Image Resampling
• What about arbitrary scale reduction?
• How can we increase the size of the image?
1
2
3
4
5
• Recall how a digital image is formed
– It is a discrete point-sampling of a continuous function
– If we could somehow reconstruct the original function, any new
image could be generated, at any resolution and scale
Image Resampling
• So what to do if we don’t know
– Answer: guess an approximation
– Can be done in a principled way: filtering
1
1
2
2.5
3
4
5
Resampling filters
• What does the 2D version of this hat function look like?
performs
linear interpolation
performs
bilinear interpolation
• Better filters give better resampled images
– Bicubic is common choice
Bilinear interpolation
• A common method for resampling images
Image Rotation
Multi-Resolution Image Representation
• Fourier domain tells us “what” (frequencies, sharpness,
texture properties), but not “where”.
• Spatial domain tells us “where” (pixel location) but not “what”.
• We want a image representation that gives a local description
of image “events” – what is happening where.
• Naturally, think about representing images across varying scales.
Figure from David Forsyth
Multi-resolution Image Pyramids
Low resolution
High resolution
Space Required for Pyramids
Constructing a pyramid by
taking every second pixel
leads to layers that badly
misrepresent the top layer
Even worse for synthetic images
Decimation
Expansion
Interpolation Results
The Gaussian Pyramid
• Smooth with Gaussians because
– a Gaussian*Gaussian=another Gaussian
• Synthesis
– smooth and downsample
• Gaussians are low pass filters, so repetition is redundant
• Kernel width doubles with each level
The Gaussian Pyramid
Low resolution
G4  (G3 * gaussian )  2
blur )  2
G3  (G2 * gaussian
blur
G2  (G1 * gaussian )  2
blur
G1  (G0 * gaussian )  2
G0  Image
blur
High resolution
Pyramids at Same Resolution
The Laplacian Pyramid
Li  Gi  expand(Gi 1 )
Gaussian Pyramid
Gn
G2
G1
Gi  Li  expand(Gi 1 )
Laplacian Pyramid
Ln  Gn
-
=
=
G0
L2
L1
L0
-
=
Applications of Image Pyramids
• Coarse-to-Fine strategies for computational efficiency.
• Search for correspondence
– look at coarse scales, then refine with finer scales
• Edge tracking
– a “good” edge at a fine scale has parents at a coarser scale
• Control of detail and computational cost in matching
– e.g. finding stripes
– very important in texture representation
• Image Blending and Mosaicing
• Data compression (laplacian pyramid)
Fast Template Matching
Fast Template Matching
Blending Apples and Oranges
Image Blending and Mosaicing
Pyramid blending of Regions
Horror Photo
© prof. dmartin
Image Fusion
Multi-scale Transform (MST)
= Obtain Pyramid from Image
Inverse Multi-scale Transform (IMST) = Obtain Image from Pyramid
Multi-Sensor Fusion
Image Compression
Next Class
• Edge Detection