Transcript Edge Preserving Smoothing Techniques Prof. PhD. Vasile Gui Polytechnic University

Edge Preserving Smoothing
Techniques
Prof. PhD. Vasile Gui
Polytechnic University
of Timisoara
Content




Introduction
Brief review of linear operators
Linear image smoothing techniques
Nonlinear image smoothing techniques
Introduction
Why do we need image
smoothing?
What is “image” and what
is “noise”?
–
–
Frequency spectrum
Statistical properties
Brief Review of Linear Operators
[Pratt 1991]

Generalized 2D linear operator
g (m, n) 
M 1 N 1
 O( j, k ; m, n) f ( j, k )
j 0 k 0

Separable linear operator:
O( j, k ; m, n)  OR ( j; m)OC (k ; n)
g (m, n) 

M 1
O
j 0
N 1
R
( j; m) OC (k ; n) f ( j , k )
k 0
Space invariant operator:
O( j , k ; m, n)  O(m  j , n  k )  H (m  j , n  k )
g (m, n) 
M 1 N 1
 h(m 
j 0 k 0
Convolution sum
j, n  k ) f ( j, k )
Brief Review of Linear Operators
h22 h21h20
h12 h11h10
h02 h01 h00
H
L-1
2
G
L-1
2
L-1
2
L-1
2
h22 h21h20
h12 h11h10
h02 h01 h00
h22 h21h20
h12 h11h10
h02 h01 h00
NL+1
N
F
N+L-1
h22 h21h20
h12 h11h10
h02 h01 h00
h22 h21h20
h12 h11h10
h02 h01 h00

Geometrical
interpretation
of 2D
convolution
Brief Review of Linear Operators

Circular convolution theorem
N 1 N 1
g (m, n)   h(m  j, n  k ) f ( j, k )
j 0 k 0
1
t g ( u, v )  t f ( u, v ) t h ( u, v )
N

Periodicity of f,h and g with N are assumed
Linear Image smoothing techniques
Box filters. Arithmetic mean LL operator
1
1
1 
h 2
L 

1
1  1
1
1
1  1 1
1
 1 1  1*
  1 L
L 


1  1
1
Linear Image smoothing techniques
Box filters. Arithmetic mean LL operator


Separable
Can be computed recursively, resulting in
roughly 4 operations per pixel
m,n
m,n+1

+
L pixels
Linear Image smoothing techniques
Box filters. Arithmetic mean LL operator
Optimality properties
 Signal and additive white noise
1
g
N
1
z
N

N
1
( f k  nk ) 

N
k 1
N

k 1
1
fk 
N
N
n .
k 1
k
N
n
k 1
k
.
Noise varianceN is
reducedN NN times
N
1
1
  E{ z z}  2 E{ nl nk }  2
N
N
k 1 l 1
1 N N
1 2
2

(
l
,
k
)

=

2 
N
N k 1 l 1
2
z
T
 E{n n } 
k 1 l 1
l
k
Linear Image smoothing techniques
Box filters. Arithmetic mean LL operator


Unknown constant signal plus noise
Minimize MSE of the estimation g:
N
 (g)   ( fk  g)2
2
k 1
 2 ( g )
0
g
1
gˆ 
N
N
f
k 1
k
Linear Image smoothing techniques
Box filters. Arithmetic mean LL operator


i.i.d. Gaussian signal with unknown mean.
( f  ) 2
p( f | )  ct. exp{ 
}
2
2
Given the observed samples, maximize
N
1
p
(
f
|

)

ct
.
exp{


k
2 2
k 1

N
( f
k 1
k
 ) 2 }
Optimal solution: arithmetic mean
Linear Image smoothing techniques
Box filters. Arithmetic mean LL operator

Frequency response:
Non-monotonically decreasing with frequency
1 1 1
1
1
1 
 1
1
1
1

[
111
]

1  hx  hy
 3
9
3 
1 1 1
1
( N 1)/ 2
t (u)  hx (0) 

n  ( N 1)/ 2
hx (n) exp{
1 2
2
t (u )   cos( u )
3 3
N
i 2
un}
N
Linear Image smoothing techniques
Box filters. Arithmetic mean LL operator

An example
     
 8 5 8 8 5 8 
     
 7 7 7 7 7 7 
     
 8 5 8 5 8 5 
     
 6 7 6 7 6 7 
1 1 1
1

 8 5 8 8 5 8  
1 1 1   7 7 7 7 7 7 

9
1 1 1  7 7 7 7 7 7 
 8 5 8 8 5 8 
     
     
1 1 1
1

 8 5 8 5 8 5  
1 1 1   6 7 6 7 6 7 

9
1 1 1  6 7 6 7 6 7 
 8 5 8 5 8 5 
     
     
Linear Image smoothing techniques
Box filters. Arithmetic mean LL operator
Image smoothed with
33, 55, 99
and 11 11 box filters

Linear Image smoothing techniques
Box filters. Arithmetic mean LL operator
Original Lena image
Lena image filtered with
5x5 box filter
Linear Image smoothing techniques
Binomial filters [Jahne 1995]




Computes a weighted average of pixels in
the window
Less blurring, less noise cleaning for the
same size
The family of binomial filters can be defined
recursively
The coefficients can be found from (1+x)n
Linear Image smoothing techniques
Binomial filters. 1D versions
b1 
1
1 1
2
1
b  1 2 1  b 1 * b 1
4
1
4
b  1 4 6 4 1  b 1 * b 1 * b 1 * b 1
16
2
1
1 6 15 20 15 6 1  b 1 * b 1 * b 1 * b 1 * b 1 * b 1
b 
64
6
As size increases, the shape of the filter is closer to a Gaussian one
Linear Image smoothing techniques
Binomial filters. 2D versions
1
1 2 1
1
1
1
b 2  1 2 1* 2  2 4 2
4
4
16
1
1 2 1
1 
1 4 6 4 1 
 4
4 16 24 16 4

1
1  
1 
4
6 24 36 24 6
b  1 4 6 4 1* 6 
16
16   256 

4
4
16
24
16
4
 


1
1 4 6 4 1 
Linear Image smoothing techniques
Binomial filters. Frequency response


1
b  1 2 1

4
2
b4 

t (u ) 

1
1 4 6 4 1

16
1 1
2
 cos( u )
2 2
N
1 1
2
t (u )  [  cos( u )] 2
2 2
N
Monotonically decreasing
with frequency
Linear Image smoothing techniques
Binomial filters. Example
Original Lena image
Lena image filtered
Lena image filtered
with binomial 5x5 kernel
with box filter 5x5
Linear Image smoothing techniques
Binomial and box filters. Edge blurring comparison

Linear filters have to compromise smoothing
with edge blurring
Step edge
Result of size L
box filter
Size L binomial
L
Nonlinear image smoothing
The median filter [Pratt 1991]

Block diagram
f(1)
f1
f(2)
f2
.
.
.
fN
Order
samples
.
.
.
select
f(N)
median
f (1)  f ( 2 )  f (3)   f ( N )
N 1
m
2
N is odd
f(m)
Nonlinear image smoothing
The median filter

Numerical example
3
7
8
2
3
7
4
6
7
2, 3, 3, 4, 6, 7, 7, 7, 8
6
Nonlinear image smoothing
The median filter

Nonlinearity
median{ f1 + f2}  median{ f1} + median{ f2}.
However:
median{ c f } = c median{ f },
median{ c + f } = c + median{ f }.
• The filter selects a sample from the window, does
not average
• Edges are better preserved than with liner filters
• Best suited for “salt and pepper” noise
Nonlinear image smoothing
The median filter
Noisy image
5x5 median filtered
5x5 box filter
Nonlinear image smoothing
The median filter


Optimality
Grey level plateau plus noise. Minimize sum of
absolute differences:
N
( g )   | f k  g |
k 1

Result:
g  median{ f1 , f 2 , f 3 ,, f N }  f ( m )

If 51% of samples are correct and 49% outliers,
the median still finds the right level!
Nonlinear image smoothing
The median filter

Caution: points, thin lines and corners are erased by the median filter
Test images
Results of 33 pixel median filter
Nonlinear image smoothing
The median filter

Cross shaped window can correct some of the problems
Results of the 9 pixel cross shaped window median filter
Nonlinear image smoothing
The median filter


Implementing the median filter
Sorting needs O(N2) comparisons
–
–
–
–
Bubble sort
Quick sort
Huang algorithm (based on histogram)
VLSI median
Nonlinear image smoothing
The median filter

VLSI median block diagram
a
b
Min(a,b)
Max(a,b)
x1
x(1)
x2
x(2)
x3
x(3)
x4
x(4)
x5
x(5)
x6
x(6)
x7
x(7)
Nonlinear image smoothing
The median filter

Color median filter
–
–
–
–
There is no natural ordering in 3D (RGB) color
space
Separate filtering on R,G and B components does
not guarantee that the median selects a true
sample from the input window
Vector median filter, defined as the sample
minimizing the sum of absolute deviations from all
the samples
Computing the vector median is very time
consuming, although several fast algorithms exist
Nonlinear image smoothing
The median filter




Example of color
median filtering
5x5 pixels window
Up: original image
Down: filtered image
Nonlinear image smoothing
The weighted median filter






The basic idea is to give higher weight to some samples, according to
their position with respect to the center of the window
Each sample is given a weight according to its spatial position in the
window.
Weights are defined by a weighting mask
Weighted samples are ordered as usually
The weighted median is the sample in the ordered array such that
neither all smaller samples nor all higher samples can cumulate more
than 50% of weights.
If weights are integers, they specify how many times a sample is
replicated in the ordered array
Nonlinear image smoothing
The weighted median filter

Numerical example for the weighted median filter
1
2
1
2
3
2
1
2
1
3
7
8
2
3
7
4
6
7
2, 2, 3, 3, 3, 3, 4, 6, 6, 7, 7, 7, 7, 7, 8
6
Nonlinear image smoothing
Rank-order filters (L filters)

Block diagram
weights
f(1)
f1
a1
f(2)
f2
a2
.
.
.
ordering
y
.
.
.
 sum
f(N)
fN
aN
N
a
k 1
k
1
y = a1f(1) + a2 f(2)+...+ aN f(N)
Nonlinear image smoothing
Rank-order filters (L filters)

Some examples
a1=1
. . .
Percentile
filters (rank
selection):
Min
aN=1
. . .
0%,
50%
Max
Particular cases
of grey level
morphological
filters
am=1
. . .
. . .
100%
a1=.5
. . .
Median
aN=.5
Mid range
Nonlinear image smoothing
Rank-order filters (L filters)

Optimality
N
( f )   | f ( k )  f |r
Minkovski distance
k 1
Case r = 1: best estimator is median.
Case r = 2, best estimator is arithmetic mean.
Case r  , best estimator is mid-range.
Nonlinear image smoothing
Rank-order filters (L filters)

Alpha-trimmed mean filter
1 /( 2Q  1), for m  Q  k  m  Q
ak  
otherwise
0,
 = (2Q + 1) / N, defines de degree of averaging
= 1 corresponds to arithmetic mean
 = 1 / N corresponds to the median filter
Properties: in between mean and median
Nonlinear image smoothing
Rank-order filters (L filters)
Median of absolute differences trimmed mean

Better smoothing than the median filter and good edge
preservation
M 1  median{ f1 , f 2 , f 3 ,, f N }
M 2  median{| f i  M 1 |}, i  1,2,..., N
output  average{ f i : | f i  M 1 |  M 2 }
M2 is a robust estimator of the variance
Nonlinear image smoothing
Rank-order filters (L filters)



k nearest neighbour (kNN) median filter
Median of the k grey values nearest by rank
to the central pixel
Aim: same as above
Nonlinear image smoothing
Selected area filtering [Nagao 1980]

Kuwahara type filtering

Form regions Ri
Compute mean, mi and variance,
Si for each region.
Result = mi : mi < mj for all j
different from i, i,e. the mean of
the most homogeneous region
Matsujama & Nagao



reg 1
reg 2
reg 4
reg 3
reg 6
reg 5
reg 8
reg 7
Nonlinear image smoothing
Conditional mean

Pixels in a neighbourhood are averaged only if they
differ from the central pixel by less than a given
threshold:
g (m, n) 
L
L
  h(k , l ) f (m  k , n  l ),
k  L l  L
1, if | f (m  k , n  l )  f (m, n) |  th
h( k , l )  
0, otherwise
L is a space scale parameter and th is a range scale parameter
Nonlinear image smoothing
Conditional mean

Example with L=3,
th=32
Nonlinear image smoothing
Bilateral filter [Tomasi 1998]



Space and range are treated in a similar way
Space and range similarity is required for the averaged pixels
Tomasi and Manduchi [1998] introduced soft weights to penalize the
space and range dissimilarity.
h(k , l )  s(k , l )r ( f (m  k , n  l )  f (m, n))
1
g ( m, n) 
h( k , l ) f ( m  k , n  l ),


K k l
K 
  h( k , l )
k
l
s() and r() are space and range similarity functions (Gaussian
functions of the Euclidian distance between their arguments).
Nonlinear image smoothing
Bilateral filter



The filter can be seen as weighted averaging in the
joint space-range space (3D for monochromatic
images and 5D – x,y,R,G,B - for colour images)
The vector components are supposed to be properly
normalized (divide by variance for example)
The weights are given by:
|| x c  x || 2
h(x)  exp{ 
}
s
h(x)  K(d(x c  x); s )
Nonlinear image smoothing
Bilateral filter



Example of Bilateral filtering
Low contrast texture has been removed
Yet edges are well preserved
Nonlinear image smoothing
Mean shift filtering [Comaniciu 1999, 2002]

Mean shift filtering replaces each pixel’s value with the most probable local value, found
by a nonparametric probability density estimation method.
xi  R d
{x i }i 1... n

The multivariate kernel density estimate obtained in the point x with the kernel K(x) and
window radius r is:
1
fˆ (x)  d
nr

 x  xi 
K



 r 
i 1
n
For the Epanechnikov kernel, the estimated normalized density gradient is proportional
to the mean shift:
ˆ f ( x)
r2 
1
 M r ( x) 
d  2 fˆ (x)
nx
x
xi S r ( x )
i
x
S is a sphere of radius r, centered on x and nx is the number of samples inside the sphere
Nonlinear image smoothing
Mean shift filtering




The mean shift procedure is
a gradient ascent method to
find local modes (maxima)
of the probability density
and is guaranteed to
converge.
Step1: computation of the
mean shift vector Mr(x).
Step2: translation of the
window Sr(x) by Mr(x).
Iterations start for each pixel
(5D point) and tipically
converge in 2-3 steps.
Nonlinear image smoothing
Mean shift filtering

Example1.
Nonlinear image smoothing
Mean shift filtering
Detail of a 24x40
window from the
cameraman image
a) Original data
b) Mean shift paths for
some points
c) Filtered data
d) Segmented data
Nonlinear image smoothing
Mean shift filtering

Example 2
Nonlinear image smoothing
Mean shift filtering
Comparison to bilateral filtering


Both methods based on simultaneous processing of both the
spatial and range domains
While the bilateral filtering uses a static window, the mean shift
window is dynamic, moving in the direction of the maximum
increase of the density gradient.
REFERENCES







D. Comaniciu, P. Meer: Mean shift analysis and applications. 7th
International Conference on Computer Vision, Kerkyra, Greece, Sept.
1999, 1197-1203.
D. Comaniciu, P. Meer: Mean shift: a robust approach toward feature
space analysis. IEEE Trans. on PAMI Vol. 24, No. 5, May 2002, 1-18.
M. Elad: On the origin of bilateral filter and ways to improve it. IEEE
trans. on Image Processing Vol. 11, No. 10, October 2002, 1141-1151.
B. Jahne: Digital image processing. Springer Verlag, Berlin 1995.
M. Nagao, T. Matsuiama: A structural analysis of complex aerial
photographs. Plenum Press, New York, 1980.
W.K. Pratt: Digital image processing. John Wiley and sons, New York
1991
C. Tomasi, R. Manduchi: Bilateral filtering for gray and color images.
Proc. Sixth Int’l. Conf. Computer Vision , Bombay, 839-846.