Transcript Numerical Geometry in Image Processing Ron Kimmel www.cs.technion.ac.il/~ron
Computer Science Department Technion-Israel Institute of Technology
Numerical Geometry in Image Processing
Ron Kimmel www.cs.technion.ac.il/~ron Geometric Image Processing Lab
Heat Equation in Image Analysis Linear scale space (T. Iijima 59, Witkin 83, Koenderink 84)
I t
I
(
t
)
I
(
t
)
G
(
t
) *
I
( 0 )
Geometric Heat Equation in Image Analysis Geometric scale space, Euclidean (Gage-Hamilton 86, Grayson 89, Osher-Sethian 88, Evans Spruck 91, Alvarez-Guichard-Lions-Morel 93)
Geometric Heat Equation in Image Analysis Gabor 65 anisotropic reaction-diffusion Geometric, Special Affine. (Alvarez-Guichard-Lions-Morel 93, Sapiro-Tannenbaum 93)
Geometric Heat Equation in Image Analysis Multi Channel, Euclidean Enk 97,…) .(Chambolle 94, Whitaker-Gerig 94, Proesmans-Pauwels-van Gool 94,Sapiro-Ringach 96, Shah 96, Blomgren Chan 96, Sochen-Kimmel-Malladi 96, Weickert, Romeny, Lopez, and van Geometric, Bending.(Curves: Grayson 89, Kimmel-Sapiro 95 (via Osher-Sethian),Images: Kimmel 97)
Bending Invariant Scale Space Invariant to surface bending.
Embedding: The gray level sets embedding is preserved.
Existence: The level sets exist for all evolution time, disappear at points or converge into geodesics.
Topology: Image topology is simplified.
Shortening flow:The scale space is a shortening flow of the image level sets.
Implementation: Simple, consistent, and stable numerical implementation.
Curves on Surfaces: The Geodesic Curvature
From Curve to Image Evolution
Geodesic curvature flow
The Beltrami Framework
Brief history of
c o l o r
line element theories.
A simplified
c o l o r
image formation model.
The importance of channel alignment.
Images as surfaces.
Surface area minimization via Beltrami flow.
Applications: Enhancement and scale space.
Beyond the metric, the Gabor connection
Images as Surfaces
Gray level analysis is sometimes misleading… Is there a `right way’ to link
c o l o r
enhance volumetric data? channels? process texture? We view images as embedded maps that flow towards minimal surfaces: Gray scale images are surfaces in (x,y, I) , and
c o l o r
images are surfaces embedded in (x,y, R , G , B ) . Joint with Sochen & Malladi, IEEE T-IP 98, IJCV 2000.
Spatial-Spectral Arclength Helmholtz 1896: Schrodinger 1920: Stiles 1946: Vos and Walraven 1972: inductive line elements (above), empirical elements (MacAdam 1942, CIELAB 1976). Define: the simplest hybrid spatial-
c o l o r
space: line
C o l o r
Image Formation
F. Guichard 93 Mondrian world: Lambertian surface patches
Lambetian model Image formation N l V
I
(
x
,
y
)
R
(
x
,
y
)
R G
(
x
,
y
)
G B
(
x
,
y
)
B
N
N
, ,
l
l
N
,
l
N
,
l
cos( )
R
cos( )
G
cos( )
B
cos( )
C o l o r
Image Formation
The gradient directions should agree since
Example: Demosaicing
C o l o r
image reconstruction Solution: Edges support the colors and the
c o l o r s
support the edges
Color Image Formation
Lambertian shading model: R (x,y) = G (x,y) = B (x,y) = Thus R G B
c o l o r
indication function.
/ G = R / =constant ratio weighted by an edge
Original
Demosaicing Results
Bilinear interpolation Weighted interpolation
Demosaicing Results
Bilinear interpolation Weighted interpolation
Original
Demosaicing Results
Bilinear interpolation Weighted interpolation
Demosaicing Results
Bilinear interpolation Weighted interpolation
Original
Demosaicing Results
Bilinear interpolation Weighted interpolation
Demosaicing Results
Bilinear interpolation Weighted interpolation
From Arclength to Area
Gray level arclength:
C o l o r
arclength Area
Multi Channel Model
Gray level:
The Beltrami Flow
C o l o r
: where
The Beltrami Flow
Matlab Program
Signal processing viewpoint
Gaussian Smoothing Beltrami Smoothing Sochen, Kimmel, Bruckstein, JMIV, 2001.
Texture:
The Beltrami Flow
Inverse Diffusion Across the Edge
Inverse Diffusion Across the Edge
Summary: Geometric Framework From
c o l o r
image formation to the importance of channel alignment.
From
c o l o r
line element theories to the definition of area in
c o l o r
images.
Area minimization as a unified framework for enhancement and scale space.
Inverse heat operator across the edges.
Related applications:
C o l o r
demosaicing movies segmentation and
www.cs.technion.ac.il/~ron
Open Questions Is there a maximum principle to the Beltrami flow?
Are there simple geometric measures to minimize in
c o l o r
image processing subject to more complicated image formation models?
Can we really invert the geometric heat operator?
Is there a real-time numerical implementation for the Beltrami flow in color?
www.cs.technion.ac.il/~ron