Transcript Biometrics - University of Cambridge

Image Processing
IB Paper 8 – Part A
Ognjen Arandjelović
http://mi.eng.cam.ac.uk/~oa214/
– Image Essentials, Sampling –
Image Processing Paradigm
We can think of image processing as a black box
that takes an input image and produces an output
image.
Feature extraction, for example, does not fall under the umbrella of image
processing.
Image Processing
Image processing deals with computer-based
manipulation of digital images. These include:
 geometric operations (various morphs)
 brightness and contrast correction
 colour enhancement,
 segmentation,
 denoising.
Image Processing Applications
Fingerprint enhancement for analysis and recognition:
Original fingerprint
image
After denoising and
contrast enhancement
After binarization
and morphological
processing
Image Processing Applications
Enhancement of CCTV footage:
Still from original CCTV
footage
After brightness and
contrast enhancement
After denoising
Image Processing Applications
Achieving illumination and pose invariance for
automatic face recognition:
Still from original
authentication video
After face detection
and cropping
After adaptive
high-pass filtering
Discretization
An inherent
cause of information loss when
Face geometry
representing physical signals on a computer:
 Two types:
 Level / numerical
It is impossible to represent infinite numbers (e.g. Pi)
using a finite amount of storage space
 Temporal or spatial
It is impossible to sample a signal at infinite frequency or
at infinite number of locations
Discretization Under a Magnifying Glass
An example on an image of a face:
Pixels = spatial
discretization
Values = numerical
discretization
Numerical Quantization
The luminance
of each pixel can be represented only
Face geometry
with finite precision. This loss of information is called
quantization noise.
We can compute the average
quantization noise energy per pixel:
Minimal luminance difference
that can be represented
Numerical Quantization – An Example
Spatial Discretization – An Example
1D Sampling
In 1D, we model sampling of a continuous function as
multiplication by a train of delta functions:
1D Sampling – Frequency Domain
Multiplication in the spatial domain goes into
convolution in the frequency domain:
A train of delta functions
Fourier spectrum
of the original 1D
signal
Fourier spectrum of the
sampled 1D signal
2D Sampling
Much like in 1D, in 2D we model sampling of a
continuous function as multiplication by a comb of
delta functions:
A comb of delta
functions
Image as a 2D
function / surface
2D Sampling – Frequency Domain
Much like in 1D, in 2D we model sampling of a
continuous function as multiplication by a comb of
delta functions:
A comb of delta
functions
Fourier spectrum
of a 2D signal
Fourier spectrum of
the sampled 2D
signal
Geometric Transformations
Geometric transformations warp images without (in
principle) changing the value of the corresponding
2D signal. Examples include:
 Scaling
 Cropping
 Rotation
 Arbitrary morphs
Geometric Transformations – Apps
Image registration for face recognition (correction of
mild pose variations):
Geometric Transformations – Apps
Image warping for mosaicing of photographs:
How to perform seamless
concatenation of images?
Geometric Transformations – Main Idea
All geometric transformations of images at their core
concern the problem of resampling.
As the original (not sampled) 2D signal cannot be
accessed, we need to first reconstruct it from the set
of samples we have i.e. pixels.
Signal Reconstruction
Remember that in the case of a band-limited signal,
we can obtain a perfect reconstruction using the
ideal low-pass filter:
Frequency spectrum of
the sampled signal
Ideal low-pass
filter
Perfectly reconstructed
signal
What is the problem with this?
The Ideal Low-Pass Filter
Recall the Fourier transform of the ideal low-pass filter
(i.e. the pulse function):
Fourier transform
Ideal low-pass filter
The sinc function
N.B.
The Ideal Low-Pass Filter in 2D
Recall the Fourier transform of the ideal low-pass filter
(i.e. the pulse function):
Fourier transform
Ideal 2D low-pass filter
The 2D sinc function
Reconstruction in Spatial Domain
Using duality, we can see that the original signal can
be reconstructed by convolving its sampled version
with a sinc function:
The Ideal LPF in the Spatial Domain
The reason why we cannot use the ideal LPF in image
processing is of computational nature:
This value, for example depends on all samples!
The idea is to use something more manageable, by making the new sample
dependent only on its neighbourhood.
Linear Interpolation
In linear interpolation, the signal between two samples
is approximated by a straight line:
Clearly not perfect
We can thus say that the constraint is that the line should pass through the two
sample points – there are 2 DOFs (a point on the line and the slope).
Linear Interpolation
We can then write an expression for the value of the
function at an arbitrary location between two original
samples:
Value at the new
sample
Value of “pixel a”
Value of “pixel b”
Bi-Linear Interpolation
Bi-linear interpolation is simply a 2D extension of
linear interpolation:
Original sampling
locations
What is the value
at this location?
Bi-Linear Interpolation
One way of thinking about bi-linear interpolation is as
fitting of a plane between neighbouring non-collinear
sampling points:
Original sampling
locations
What is the value
at this point?
Bi-Linear Interpolation
Alternatively, you can think of bi-linear interpolation as
weighted average of the new sample’s neighbours:
Original sampling
locations
What is the value
at this point?
Cubic Interpolation
We have seen that linear interpolation is far from
perfect. Cubic interpolation does better:
Much better!
Cubic Interpolation Constraints
In cubic interpolation we have the following constraints
for segment between x2 and x3, given 4 consecutive
samples x1, x2, x3, x4:
 C0 Continuity:
Value at x2 and x3 should be exact
(as with linear interpolation)
 C1 Continuity:
Gradient at x2 should be (x3-x1) / 2
and at x3 should be (x4-x2) / 2
The Gradient Constraint
The see why the gradient at say x2 should be
(x3-x1) / 2, consider 1st order Taylor series
expansions around x2:
Now subtract (1) from (2):
Cubic Interpolation in 1D
The expression for cubic interpolation is thus more
complicated:
Bi-Cubic Interpolation
The expression for bi-cubic interpolation gets very
messy, so we do not give it here.
The principle, however, is the same – you can think of
it as cubic interpolation in the x direction, followed
by cubic interpolation in the y direction.
Bi-Linear vs. Bi-Cubic Interpolation
A summary of key differences:
 Cubic produces less smoothing of the
signal than linear
 Cubic is about 4 times more
computationally demanding and is hence
seldom used for resampling of large
images
Impulse Response of Linear Interp.
We can analyse the performance of the two
interpolators by looking at their impulse responses:
An impulse
Impulse response of the linear
interpolation process
Impulse Response of Linear Interp.
By taking the Fourier transform of impulse responses
of the two interpolators, we can see how they affect
different frequencies:
Frequency response of cubic interpolation is much flatter at
higher frequencies – closer to the ideal LPF.
Image Rescaling
Now that we know how to reconstruct the original
signal from a discrete set of samples, we can easily
manipulate images in various ways.
New sampling locations
(use interpolation)
Original sampling locations
Image resizing as resampling – the original 5 х 5
image is resampled to 6 х 6.
Image Rescaling Example
To examine the effects of resampling, let us look at a
small rectangular patch extracted from CCTV
footage:
Magnified patch – 50 х 90 pixels
Image Rescaling Example
Let us now compare the results of bi-linear and bicubic interpolation-based magnification for a factor
of 2:
Bi-linear
Bi-cubic
Note that bi-linear magnification produces a much smoother result.
Rescaling Comparison – An Example
Quantitative insight can be gained by looking at the
image difference of two results:
As expected from theory, the difference is mainly in the high-frequency content.
Downsampling Caveats
Remember that the original signal can be
reconstructed from a set of samples using a LPF
only if it is band-limited:
A train of delta functions
Fourier spectrum
of the original 1D
signal
Fourier spectrum
of the sampled
1D signal
No overlap of spectrum ‘replicas’ (this
overlap is called aliasing)
It is crucial to ensure that the signal is band-limited before downsampling.
Downsampling Caveats – Aliasing
Let’s take a look at what happens with downsampling
without considering the aliasing problem:
Downsampling
Original image (184 х 184 pixels)
Resulting image (40 х 40 pixels)
Note strange downsampling artefacts.
Downsampling Caveats – Aliasing
To ensure that high frequencies are suppressed, the
original image should first be LP filtered:
Gaussian
Smoothing
Original image (184 х 184 pixels)
Low-pass smoothed image
Downsampling Caveats – Aliasing
Now we can downsample as before, without worrying
about aliasing:
Downsampling
Low-pass smoothed image
Resulting image (40 х 40 pixels)
Downsampling Caveats – Aliasing
Compare the results (or try switching anti-aliasing off
in your Acrobat Reader!):
Without LP filtering
With LP filtering
Image Rotation
Now that we know how to reconstruct the original
signal from a discrete set of samples, we can easily
manipulate images in various ways.
New sampling locations
(use interpolation)
Original sampling locations
Image rotation as resampling – the original 5 х 5
image is rotated ~30 degrees clockwise.
Image Rotation – New Samples
The main difference to rescaling is that the locations of
new samples are now slightly more difficult to
compute.
Recall that these are related to the original locations by the
rotation matrix:
Where θ is the angle of rotation.
Image Rotation – Image Size
Note that the size of the resulting, rotated image is in
general different than the size of the original image.
If the input image is of size H х W pixels, then the output image
is of size H' х W' pixels, where:
The ceiling operator
Image Rotation – Image Size
Note that the size of the resulting, rotated image is in
general different than the size of the original image.
Image Rotation – An Example
Original fingerprint
(480 х 400 pixels)
Fingerprint rotated by 60 degrees
(586 х 615 pixels)
– That is All for Today –