Transcript Document

Visual Information Systems
Basic images processing
Image Resolution
• How many pixels
– spatial resolution
• How many shades of grey/colours
– amplitude resolution
• How many frames per second
– temporal resolution
Spatial Resolution
n, n/2, n/4, n/8, n/16 and n/32 pixels per unit length
amplitude resolution
-Shades of Grey
8, 4, 2 and 1 bit images.
Temporal Resolution
– how much does an object move between
frames?
– Can motion be understood unambiguously?
• Nyquist’s Theorem
– A periodic signal can be reconstructed if the
sampling interval is half the period
– An object can be detected if two samples span
its smallest dimension
Colour Representation
• three primaries could approximate many
colours
• red, green, blue
• C= rR+gG+bB
Other Colour Models
•
•
•
•
YMCK
HSI
YCrCb
etc
Colour image and video sequence
• colour can be conveyed by combining different
colours of light, using three components (red, green
and blue): R = r(x,y); G = g(x,y); B = b(x,y), where R,
G, B are defined in a similar way to F.
• The vector (r(x,y), g(x,y), b(x,y)) defines the intensity
and colour at the point (x,y) in the colour image.
• A video sequence is, in effect, a time-sampled
representation of the original moving scene.
• Each frame in the sequence is a standard colour, or
monochrome image and can be coded as such.
• a monochrome video sequence may be represented
digitally as a sequence o 2-D arrays [F1, F2, F3..FN].
Image processing and transform
Objectives:
• To enhance features that are useful
• Obtain key representations (silent features)
of image content
Classification of Image
Transforms
• Point transforms
– modify individual pixels
– modify pixels’ locations
• Local transforms
– output derived from neighbourhood
• Global transforms
– whole image contributes to each output value
Point Transforms
• Manipulating individual pixel values
– Brightness adjustment
– Contrast adjustment
• Histogram manipulation
– equalisation
• Image magnification
Monadic, Point-by-point
Operators
• Monadic point-to-point operator
Brightness Adjustment
Add a constant to all values
g(x,y) = f(x,y) + k
(k = 50)
Intensity Shift
g(x,y) <=
0
a(x,y)+k < 0
f(x,y) +k
0<=f(x,y) +k <=W
W
W<f(x,y) +k
Where k is a user defined variable
Contrast Adjustment
Scale all values by a constant
g(x,y) = a* f(x,y)
(a = 1.5)
General expression for brightness
and contrast modification
g(x,y) = a * f(x,y) +b
If we do not want to specify a gain and a bias, but
would rather map a particular range of grey levels,
[f1,f2],, onto a new range, [g1,g2]. This form of
mapping is accomplished using
(g(x,y)-g1)/(f(x,y)-f1)=(g2-g1)/(f2-f1)
i.e.
g(x,y) = g1 +((g2-g1)/(f2-f1))[f(x,y)-f1]
linear mapping
g
255
0
255
f
Linear and non-linear mapping
• If ‘a’ is a constant, then it is a linear mapping;
• When ‘a’ is a function, then it is a non-linear
mapping
• Non-linear mapping functions have a useful
properties: the gain, ‘a’, applied to input grey
level, can vary. Thus the way in which contrast is
modified depends on the input grey level.
• If a range of grey level is mapped to a wider
ranger of grey level, the contrast is enhanced
• If a range of grey level is mapped to a narrower
range of grey level, the contrast is reduced.
Non-linear mapping
•Generally, logarithmic mapping is for to enhance details in the
darker regions of the image, at the expense of detail in the
brighter regions.
•Exponential mapping has a reverse effect, contrast in the
brighter parts of an image is increased at the expense of
contrast in the darker parts.
Image Histogram
• Measure frequency of occurrence of each
grey/colour value
Grey Value Histogram
12000
8000
6000
4000
2000
Grey Value
256
241
226
211
196
181
166
151
136
121
106
91
76
61
46
31
1
0
16
Frequency
10000
Calculation of an image histogram
Create an array histogram with 2b elements
for all grey levels, i, do
histogram[i] = 0
end for
for all pixel coordinates, x and y, do
Increment histogram [f(x,y)] by 1
end for
Histogram Manipulation
• Modify distribution of grey values to
achieve some effect
Cumulative Histogram
• Records the cumulative frequency distribution of
grey levels in an image.
• A cumulative histogram is a mapping that counts
the cumulative number of pixels in all of the bins
up to the specified bin. That is, the cumulative
histogram Hk of a histogram hk is defined as:
• (k’ can start from 0, if the range is from 0-255)
Histogram Equalisation
• Histogram equalisation is based on the argument that the
image’s appearance will be improved if the distribution of
pixels over the available grey level is even.
• A non-linear mapping of grey level, specific to that image,
that will yield an optimal improvement in contrast.
• Redistributes grey levels in an attempt to flatten the
frequency distribution.
• More grey levels are allocated where there are most pixels,
fewer grey levels where there are fewer pixels.
• This tends to increase contrast in the most heavily
populated regions of the histogram, and often reveals
previously hidden detail.
Histogram Equalisation
• If we are to increase contrast for the most frequently
occurring grey levels and reduce contrast in the less
popular part of the grey level range, then we need a
mapping function which has a steep slope (a>1) at
grey levels that occur frequently, and a gentle slope
(a<1) at unpopular grey levels.
• The cumulative histogram of the image has these
properties.
• The mapping function we need is obtained simply
by rescaling the cumulative histogram so that its
values lie in the range 0-255.
Calculating histogram equalisation
Compute a scaling factor, a=255/number of pixels
Calculate histogram (see previous algorithm)
c[0] = a*histogram[0]
for all remaining grey levels, i, do
c[i] = c[i-1]+a*histogram[i]
end for
for all pixel coordinates, x and y, do
g(x,y) = c[f(x,y)]
end for
Equalisation/Adaptive
Equalisation
• Specifically to make histogram uniform
Grey Value Histogram
12000
8000
6000
4000
2000
Grey Value
256
241
226
211
196
181
166
151
136
121
91
106
76
61
46
31
16
0
1
Frequency
10000
Threshold
• This is an important function, which
converts a grey scale image to a binary
format. Unfortunately, it is often difficult, or
even impossible to find satisfactory values
for the user defined integer threshold value
Threshold
• C(x,y) <=
W a(x,y)>=threshold
0
otherwise
for (int y=0; y<height; y++)
for(int x=0; x<width; x++)
if (input.getxy(x,y) < threshold)
output.setxy(x,y, BLACK);
else
output.setxy(x,y,WHITE);
Thresholding
• Transform grey/colour image to binary
if f(x, y) > T output = W (or 1)
else 0
• How to find T?
Threshold Value
• Manual
– User defines a threshold
• P-Tile – if we know the proportion of the image is
occupied by the object, define the threshold as that
grey level which has the correct proportion
• Mode - Threshold at the minimum between the
histogram’s peaks
• Other automatic methods
Image Magnification
• Reducing
– new value is weighted sum of nearest neighbours
– new value equals nearest neighbour
• Enlarging
– new value is weighted sum of nearest neighbours
– add noise to obscure pixelation
(such operations will be practised in the labs)
Local Transforms
• Consider neighbourhood information
• Convolution
• Applications
– smoothing
– sharpening
– Matching
• Very useful, but computationally costly
Local Operators
• The concept behind local operators is that the
intensities of several pixels are combined together
in order to calculate the intensity of just one pixel.
Amongst the simplest of the local operators are
those which use a set of nine pixels arranged in a
3 x 3 square region. It computes the value for one
pixel on the basis of the intensities within a region
containing 3 x 3 pixels. Other local operators
employ larger windows.
A sub-image
(x-1, y-1) (x, y-1) (x+1, y-1)
(x-1, y)
(x,y)
(x+1,y)
(x-1,y+1) (x, y+1) (x+1, y+1)
Local Operators
Convolution Definition


g' r, c   
 gr  x, c  y   t x, y 
x   y  
Place template on image
Multiply overlapping values in image and
template
Sum products and normalise
(Templates usually small)
Example
Image
… .
…3
…4
…4
…4
…4
… .
.
5
5
6
6
5
.
.
7
8
9
9
8
.
Template
.
4
5
6
5
5
.
. ...
4…
4…
4…
3…
4…
. ...
1
1
1
1
2
1
Result
1
1
1
…
…
…
…
…
…
…
Divide by template sum
.
.
.
.
.
.
.
.
.
6
6
6
.
.
.
.
6
7
7
.
.
.
.
6
6
6
.
.
. ...
. ...
.…
.…
.…
. ...
. ...
Separable Templates
• Convolve with n x n template
– n2 multiplications and additions
• Convolve with two n x 1 templates
– 2n multiplications and additions
Example
• Laplacian template
0 –1 0
-1 4 –1
0 –1 0
• Separated kernels
-1 2 -1
-1
2
-1
Applications
• Usefulness of convolution is the effects
generated by changing templates
– Smoothing
• Noise reduction
– Sharpening
• Edge enhancement
Smoothing
• Aim is to reduce noise
• What is “noise”?
– Noise is deviation of a value from its expected
value
• How is it reduced
– Addition
– Adaptively
– Weighted
Original
Smoothed
Median Smoothing
Sharpening
• What is it?
– Enhancing discontinuities
– Edge detection
• Why do it?
– Perceptually important
– Computationally important
Edge Definition
•An edge is a significant local change in
image intensity.
•Significant: in relation to the proportional
change in image intensity across the edge
•Local: Neighbourhood relationship matters
Algorithms for Edge Detection
• Algorithms for detecting edges - edge detectors
– differentiation based
• estimated the derivatives of the image intensity
function, the idea being that large image derivatives
reflect abrupt intensity changes.
– Model based
• determine whether the intensities in a small area
conform to some model for the edges that we have
assumed.
First Derivative, Gradient Edge
Detection
• If an edge is a discontinuity
• Can detect it by differencing
Roberts Cross Edge Detector
-1 0
0 -1
0
1
1
• Simplest edge detector
0
Prewitt/Sobel Edge Detector
-1 -1 -1
0 0 0
1 1 1
-1 0
-1 0
-1 0
1
1
1
Edge Detection
• Combine horizontal and vertical edge
estimates
Mag  h  v
2
2
and
v
  tan
h
1
Example Results
Example Results
Global Transforms
• Computing a new value for a pixel using the
whole image as input
• Cosine and Sine transforms
• Fourier transform
– Frequency domain processing
• Hough transform
• Karhunen-Loeve transform
• Wavelet transform
Geometric Transformations
• Definitions
– Affine and non-affine transforms
• Applications
– Manipulating image shapes
Affine Transforms
Scale, Shear, Rotate, Translate
Length and areas preserved.
x’ = a b c
x
y’
d e f
y
1
g h i
1
Change values of transform matrix elements according
to desired effect.
[ ] [ ][ ]
a, e  scaling
b, d  shearing
a, b, d, e  rotation
c, f  translation
Affine Transform Examples
Warping Example
Ansell Adams’ Aspens
x' , y '  1.2 x2 y  1.2 x y 2  x
Summary
• Point transforms
– scaling, histogram manipulation,thresholding
• Local transforms
– edge detection, smoothing
• Global transforms
– Fourier, Hough, Principal Component, Wavelet
• Geometrical transforms