Transcript Document

Signal- und Bildverarbeitung, 323.014
Image Analysis and Processing
Arjan Kuijper
23.11.2006
Johann Radon Institute for Computational and
Applied Mathematics (RICAM)
Austrian Academy of Sciences
Altenbergerstraße 56
A-4040 Linz, Austria
[email protected]
Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at
1/25
Last week
• The diffusion can be made locally adaptive to image
structure. Three mathematical approaches are discussed:
1. PDE-based nonlinear diffusion, where the luminance function
evolves as the divergence of some flow.
2. Evolution of the isophotes as an example of curve-evolution;
3. Variational methods, minimizing an energy functional defined on
the image.
• The nonlinear PDE's involve local image derivatives, and
cannot be solved analytically.
• Adaptive smoothing requires geometric reasoning to define
the influence on the diffusivity coefficient.
• The simplest equation is the equation proposed by Perona &
Malik, where the variable conduction is a function of the local
edge strength.
Strong gradient magnitudes prevent the blurring locally, the
effect is edge preserving smoothing.
• The Perona & Malik equation leads to deblurring (enhancing
edges) for edges larger than the turnover point k, and blurs
smaller edges.
Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at
2/25
Today
• Total Variation
–
–
–
–
–
Rudin – Osher – Fatemi (ROF) Model
Denoising
Edge preserving
Energy minimizing
Bounded variation
Taken from:
Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at
3/25
• Let the observed intensity function u0(x, y) denote the
pixel values of a noisy image for x, y W. Let u(x, y)
denote the desired clean image, so
with n additive white (0,)noise.
• The constrained minimization problem is:
Min(H|I1,I2) = Min (H - l1I1 - l2I2)
Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at
4/25
• We arrive at the Euler-Lagrange equations
0= dH - l1dI1 - l2dI2
• Integrating the expression over W gives l1 = 0.
So the average intensity is kept.
Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at
5/25
• The solution procedure uses a parabolic equation with
time as an evolution parameter, or equivalently, the
gradient descent method. This means that we solve
Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at
6/25
• We must compute l(t). We merely multiply the
equation in W by (u - u0) and integrate by parts over W.
We then have
Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at
7/25
• The numerical method in two spatial dimensions is as
follows:
Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at
8/25
• The numerical approximation is
Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at
9/25
• and ln is defined discreetly via
• A step size restriction is imposed for stability:
Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at
10/25
Results
Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at
11/25
Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at
12/25
•
•
•
•
Original
Noisy
Wiener
TV
Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at
13/25
• Do it slightly
¡kr ksloppy:
¢
r
¢
r
¡
Lt =
L 1 L + ¸(L ¡ L0 )
¡
¢
¡ k + k¡1 +
¢
±L = C D
D L
D L + ¸(L ¡ L0 )
i;j
i;j
i;j
• demo
Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at
14/25
• Write
ut =
¡(±H ¡ ¸±I)
< A; B >=
R
¸=
<±H;±I>
<±I;±I>
A¢B d
• Then I(u) is a constant of motion: I(u0)=I(ut):
@t I(u) =< ±I; ut >= 0
• Then the pde converges to a minimum on the manifold
given by the constraints:
@t H(u) = ¡u2 · 0
t
Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at
15/25
Some extensions
• A blurred noisy image:
u0 = (Au)(x,y) + n(x,y)
where A is a compact operator on L2.
• Multiplicative noise & blurring
u0 = [(Au)(x,y) ] n(x,y)
u0 = (Au)(x,y) + [u(x,y) n(x,y)]
R functional:
• Smarter
H=
Á(jrLj)d
Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at
16/25
Recent Developments in Total Variation
Image Restoration
• T. Chan, S. Esedoglu, F. Park, A. Yip
Handbook of Mathematical Models in Computer Vision
Since their introduction in a classic paper by Rudin, Osher and
Fatemi, total variation minimizing models have become one of the
most popular and successful methodology for image restoration.
More recently, there has been a resurgence of interest and exciting
new developments, some extending the applicabilities to inpainting,
blind deconvolution and vector-valued images, while others offer
improvements in better preservation of contrast, geometry and
textures, in ameliorating the stair casing effect, and in exploiting the
multi-scale nature of the models.
In addition, new computational methods have been proposed with
improved computational speed and robustness.
Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at
17/25
Properties
• Total variation based image restoration models were
first introduced by Rudin, Osher, and Fatemi (ROF) in
their pioneering work on edge preserving image
denoising.
• It is one of the earliest and best known examples of
PDE based edge preserving denoising.
• It was designed with the explicit goal of preserving
sharp discontinuities (edges) in images while removing
noise and other unwanted fine scale detail.
• Being convex, the ROF model is one of the simplest
variational models having this most desirable property.
• The revolutionary aspect of this model is its
regularization term that allows for discontinuities but at
the same time disfavors oscillations.
Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at
18/25
Properties
• The constraint of the optimization forces the
minimization to take place over images that are
consistent with this known noise level.
• The objective functional itself is called the total
variation (TV) of the function u(x); for smooth
images it is equivalent to the L1 norm of the derivative,
and hence is some measure of the amount of oscillation
found in the function u(x).
Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at
19/25
Remark
• The step from
to
is not trivial!
Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at
20/25
BV
• The space of functions with bounded variation (BV) is
an ideal choice for minimizers to the ROF model since
BV provides regularity of solutions but also allows sharp
discontinuities (edges). Many other spaces like the
Sobolev space W1,1 do not allow edges.
• Before defining the space BV, we formally state the
definition of TV as:
where
and
is a bounded open set.
• We can now define the space BV as
• Thus, BV functions amount to L1 functions with
bounded TV semi-norm.
Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at
21/25
TV & Contours
• Why does this work?
Ignoring the constraints we get
@f
@t
=
fvv
fw
= ¡· !
• The TV norm of f can be obtained by integrating along
all contours of f = c for all values of c.
• Thus, one can view TV as controlling both the size of
the jumps in an image and the geometry of the
isophotes (level sets).
Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at
22/25
Caveats
• While using TV-norm as regularization can reduce
oscillations and regularize the geometry of level sets
without penalizing discontinuities, it possesses some
properties which may be undesirable under some
circumstances.
– Loss of contrast: The total variation of a function, defined
on a bounded domain, is decreased if we re-scale it
around its mean value in such a way that the difference
between the maximum and minimum value (contrast) is
reduced.
– Loss of geometry: In addition to loss of contrast, the TV of
a function may be decreased by reducing the length of
each level set.
– Staircasing: This refers to the phenomenon that the
denoised image may look blocky (piecewise constant).
– Loss of Texture: Although highly effective for denoising,
the TV norm cannot preserve delicate small scale features
like texture.
Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at
23/25
Summary
• Total variation minimizing models have become one of
the most popular and successful methodology for
image restoration.
• ROF is one of the earliest and best known examples of
PDE based edge preserving denoising.
• It was designed with the explicit goal of preserving
sharp discontinuities (edges) in images while removing
noise and other unwanted fine scale detail.
• However, it has some drawbacks as shown in the
previous slides
Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at
24/25
Next week
• Non-linear diffusion:
Mean curvature motion
–
–
–
–
–
Curve evolution
Denoising
Edge preserving
Implementation
Isophote vs. image implementation
Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at
25/25