Transcript lec05_snakes - Computer Science

The University of
CS 4487/9587
Algorithms for Image Analysis
Ontario
2D Segmentation (part II)
Deformable Models
Acknowledgements: many slides from the University of Manchester,
demos from Visual Dynamics Group (University of Oxford),
5-1
The University of
CS 4487/9587 Algorithms for Image Analysis
Deformable Models in 2D

Ontario
Active Contours or “snakes”
• “snakes” vs. “livewire”
• (discrete) energy formulations for snakes
• relation to Hidden Markov Models (HMM)

Optimization (discrete case)
• Gradient Descent
• Dynamic Programming (DP), Viterbi algorithm
• DP versus Dijkstra
Extra Reading: Sonka et.al 5.2.5 and 8.2
Active Contours by Blake and Isard
5-2
The University of
“Live-wire” vs. “Snakes”
Ontario
• intelligent scissors
[Mortensen, Barrett 1995]
• live-wire [Falcao, Udupa, Samarasekera, Sharma 1998]
4
3
1
2
Shortest paths on image-based graph connect
seeds placed on object boundary
5-3
The University of
“Live-wire” vs. “Snakes”
Ontario
•Snakes, active contours [Kass, Witkin, Terzopoulos 1987]
•In general, deformable models are widely used
Given: initial contour (model) near desirable object
5-4
The University of
“Live-wire” vs. “Snakes”
Ontario
•Snakes, active contours [Kass, Witkin, Terzopoulos 1987]
•In general, deformable models are widely used
Given: initial contour (model) near desirable object
Goal: evolve the contour to fit exact object boundary
5-5
The University of
Tracking via deformable models
Ontario
1. Use final contour/model extracted at frame
t as an initial solution for frame t+1
2. Evolve initial contour to fit exact object
boundary at frame t+1
3. Repeat steps 1 and 2 for t ‘= t+1
5-6
The University of
Tracking via deformable models
Ontario
Acknowledgements: Visual Dynamics Group, Dept. Engineering Science, University of Oxford.
Applications:
Traffic monitoring
Human-computer interaction
Animation
Surveillance
Computer Assisted Diagnosis in medical imaging
5-7
The University of
Tracking via deformable models
Ontario
Tracking Heart Ventricles
5-8
The University of
gradient descent w.r.t. some
function describing snake’s quality
“Snakes”
Ontario
A smooth 2D curve which matches to image data
 Initialized near target, iteratively refined
 Can restore missing data

initial
intermediate
final
Q: How does that work? ….
5-9
The University of
Preview
f(x)
for simplicity, assume that
"snake” is a vector (or point) in R1
Ontario
assume some energy function f(x)
describing snake’s “quality”
x R
ˆx
local minima
for f(x)
x2 x1
1
x0
gradient descent for 1D functions
xi 1

xi   t  f ' ( xi )
Q: Is snake (contour) a point in some space? ... Yes
5-10
The University of
Parametric Curve Representation
(continuous case)

Ontario
A curve can be represented by 2 functions
ν ( s )  ( x ( s ), y ( s ))
parameter
0s1
closed curve
open curve
Note: in computer vision and medical imaging the term “snake” is commonly associated with
such parametric representation of contours. (Other representations will be discussed later!)
Here, contour is a point in R
C  { ν ( s ) | s  [ 0 ,1 ]}

(space of functions)
5-11
The University of
Parametric Curve Representation
(discrete case)

Ontario
A curve can be represented by a set of 2D points
ν i  ( xi , y i )
parameter
0in
( xn , yn ) 
2n
Here, contour is a point in R _
C  ( ν i | 0  i  n )  ( x 0 , y 0 , x 1 , y 1 ,...., x n  1 , y n  1 )
5-12
The University of
Measuring snake’s quality:
Energy function
Contours can be seen as points C in R
Ontario
2n

(or in R )
C  ( ν 0 , ν 1 , ν 2 ,...., ν n  1 )  ( x 0 , y 0 , x 1 , y 1 ,...., x n  1 , y n  1 )
We can define some energy function E(C) that assigns
some number (quality measure) to all possible snakes
R
2n
(contours C)
WHY?: Somewhat philosophical question, but
E(C)

R
(scalars)
specifying a quality function E(C) is an objective
way to define what “good” means for contours C.
Moreover, one can find “the best” contour
(segmentation) by optimizing energy E(C).
Q: Did we use any function (energy) to measure quality of segmentation results in
1) image thresholding?
NO
2) region growing?
NO
3) live-wire?
YES (will compare later)
5-13
The University of
Energy function
Ontario
Usually, the total energy of snake is a combination of
internal and external energies
E  E in  E ex
Internal energy encourages
smoothness or any particular shape
Internal energy incorporates prior
knowledge about object boundary
allowing to extract boundary even if
some image data is missing
External energy encourages curve onto
image structures (e.g. image edges)
5-14
The University of
Internal Energy
(continuous case)

Ontario
The smoothness energy at contour point v(s) could be
evaluated as
E in ( ( s ))   ( s )
d
2
d 
2
  (s)
ds
2
2
d s
Elasticity/stretching
Stiffness/bending
Then, the interior energy (smoothness) of the whole snake
C  { ν ( s ) | s  [ 0 ,1 ]} is
1
E in 
E
0
in
(  ( s )) ds
5-15
The University of
Internal Energy
(discrete case)
Ontario
C  ( ν 0 , ν 1 , ν 2 ,...., ν n  1 )  
elastic energy
(elasticity)
d
ds
v3
v4
v5
2n
ν i  ( xi , y i )
v2
 vi1   i
v1
v6
v7
v 10
v9
d 
v8
bending energy
(stiffness)
2
ds
2
 (  i 1   i )  (  i   i 1 )   i 1  2  i   i 1
The University of
Internal Energy
(discrete case)
Ontario
C  ( ν 0 , ν 1 , ν 2 ,...., ν n  1 )  
d
ds
d 
2n
ν i  ( xi , y i )
 vi1   i
2
ds
2
 ( i 1   i )  ( i   i 1 )   i 1  2 i   i 1
n 1
E in 

 |  i 1   i |   |  i 1  2 i   i 1 |
2
2
i0
Elasticity
Stiffness
5-17
The University of
External energy
Ontario
The external energy describes how well the
curve matches the image data locally
 Numerous forms can be used, attracting the
curve toward different image features

5-18
The University of
External energy




Ontario
Suppose we have an image I(x,y)
Can compute image gradient  I  ( I x , I y ) at any point
Edge strength at pixel (x,y) is |  I ( x , y ) |
External energy of a contour point v=(x,y) could be
E ex ( v )   |  I ( v ) |   |  I ( x , y ) |
2

2
External energy term for the whole snake is
1
E ex 

E ex ( ( s )) ds
continuous case
C  { ν ( s ) | s  [ 0 ,1 ]}
0 n 1
E ex 
E
i0
ex
( i )
discrete case
C  { νi |0  i  n }
5-19
The University of
Basic Elastic Snake

Ontario
The total energy of a basic elastic snake is
1
E    |
0
dv
1

2
| ds
ds
E     | vi 1  vi | 
2
elastic smoothness term
(interior energy)
continuous case
C  { ν ( s ) | s  [ 0 ,1 ]}
0
n 1
i 0
 |  I ( v ( s )) | ds
2
n 1
 |  I ( vi ) |
2
discrete case
C  { νi |0  i  n }
i 0
image data term
(exterior energy)
5-20
The University of
Basic Elastic Snake
(discrete case)
Ontario
C  ( ν i | 0  i  n )  ( x 0 , y 0 , x 1 , y 1 ,...., x n  1 , y n  1 )
n 1
E in

   Li
2
i
This can make a curve shrink
i-1
(to a point)
i0
Li-1
i+1
Li+1
n 1

Li
   ( x i 1  x i )  ( y i 1  y i )
2
i+2
2
C
i0
n 1
E ex


 |  I ( xi , yi ) |
2
i0
n 1


 | I x ( xi , yi ) |  | I y ( xi , yi ) |
2
2
i0
5-21
The University of
Basic Elastic Snake
(discrete case)

Ontario
The problem is to find contour
that minimizes
C  ( x 0 ,  , x n 1 , y 0 ,  , y n 1 )  R
n 1
E ( C )     ( xi1  xi )  ( yi1  yi )
2
i 0

2n
n 1
2

 | I x ( xi , yi ) |  | I y ( xi , yi ) |
2
2
i 0
Optimization problem for function of 2n variables
• can compute local minima via gradient descent (coming soon)
• potentially more robust option: dynamic programming (later)
5-22
The University of
Basic Elastic Snake
Synthetic example
Ontario
(1)
(2)
(3)
(4)
5-23
The University of
Basic Elastic Snake
Dealing with missing data

Ontario
The smoothness constraint can deal with
missing data:
5-24
The University of
Basic Elastic Snake
Relative weighting

Ontario
Notice that the strength of the internal elastic
component can be controlled by a parameter, 
n 1
E in

   Li
2
i0

Increasing this increases stiffness of curve
large

medium

small

5-25
The University of
Encouraging point spacing

Ontario
To stop the curve from shrinking to a point
n 1
E in

 
2
ˆ
( Li  Li )
i0
• encourages particular point separation
5-26
The University of
Simple shape prior

Ontario
If object is some smooth variation on a known
shape, use
n 1
E in

 
2
ˆ
( i   i )
i0
– where { ˆi } give points of the basic shape

May use a statistical (Gaussian) shape model
E in

 ln N ( | ˆ ) 
T
ˆ
(   )  C  (  ˆ )
5-27
The University of
Alternative External Energies

Ontario
Directed gradient measures
n 1
E ex



u x ,i  I x ( ν i )  u y ,i  I y ( ν i )
i0
• Where u i  ( u x ,i , u y ,i ) is the unit normal to the
boundary at contour point  i
• This gives a good response when the boundary has
the same direction as the edge, but weaker
responses when it does not
5-28
The University of
Additional Constraints
Ontario
• Snakes originally developed for interactive image
segmentation
• Initial snake result can be nudged where it goes wrong
• Simply add extra external energy terms to
– Pull nearby points toward cursor, or
– Push nearby points away from cursor
5-29
The University of
Interactive (external) forces

Ontario
Pull points towards cursor:
n 1
E pull


i0
r
2
| i  p |
2
Nearby points get
pulled hardest
Negative sign gives
better energy for
positions near p
5-30
The University of
Interactive (external) forces

Ontario
Push points from cursor:
n 1
E push


 |
i0
r
i
2
 p|
2
Nearby points get
pushed hardest
Positive sign gives
better energy for
positions far from p
5-31
The University of
Dynamic snakes
Ontario

Adding motion parameters as variables (for
each snake node)

Introduce energy terms for motion consistency

primarily useful for tracking (nodes represent real
tissue elements with mass and kinematic energy)
5-32
The University of
Open and Closed Curves
Ontario
0
0  n
open curve
 n 1
closed curve
n2
n 1
E in 
 (
i0

 i )
i 1
2
E in 
 (
 i )
i 1
2
i0
When using an open curve we can impose constraints on
the end points (e.g. end points may have fixed position)
– Q: What are similarities and differences with the live-wire
if the end points of an open snake are fixed?
5-33
The University of
Discrete Snakes Optimization



Ontario
At each iteration we compute a new snake position within
proximity to the previous snake
New snake energy should be smaller than the previous one
Stop when the energy can not be decreased within local
neighborhood of the snake (local energy minima)
Optimization Methods
1. Gradient Descent
2. Dynamic Programming
5-34
The University of
Gradient Descent

Ontario
Example: minimization of functions of 2 variables
E ( x, y )
x
( x0 , y0 )
y
 E 




E
x

E 
y 
negative gradient at point (x,y) gives direction of the
steepest descent towards lower values of function E
5-35
The University of
Gradient Descent

Ontario
Example: minimization of functions of 2 variables
E ( x, y )
x
update equation for a point
p=(x,y)
p  p  t  E
( x0 , y0 )
y

 x   x 
       t  


y
y
   

E
x
E
y





Stop at a local minima where  E  0
5-36
The University of
Gradient Descent

Ontario
Example: minimization of functions of 2 variables
E ( x, y )
x
y
High sensitivity wrt. the initialisation !!
5-37
The University of
Gradient Descent for Snakes
C
n 1
E ( x 0 , , x n  1 , y 0 , , y n  1 )


Ontario
simple elastic snake energy
 | I x ( xi , yi ) |  | I y ( xi , yi ) |
2
i 0
here, energy is a function of 2n variables
n 1

   ( x i 1  x i )  ( y i 1  y i )
2
2
i0
update equation for the whole snake
C'  C   E   t
C








E
x' 0   x 0    x 0 
 

E
y' 0   y 0    y 0 
...    ...    ...    t
 
  E 
x' n  1   x n  1    x n  1 
y' n  1   y n  1    y E 
 n 1 
5-38
2
The University of
Gradient Descent for Snakes
C
n 1
E ( x 0 , , x n  1 , y 0 , , y n  1 )


Ontario
simple elastic snake energy
 | I x ( xi , yi ) |  | I y ( xi , yi ) |
2
i 0
here, energy is a function of 2n variables
n 1

   ( x i 1  x i )  ( y i 1  y i )
2
2
i0
update equation for each node

ν' i  ν i  Fi   t

Fi
C








E
x' 0   x 0    x 0 
 

E
y' 0   y 0    y 0 
...    ...    ...    t
 
  E 
x' n  1   x n  1    x n  1 
y' n  1   y n  1    y E 
 n 1 

Fi 
 xEi 
  E 
 yi 
5-39
2
The University of
Gradient Descent for Snakes
C
n 1
E ( x 0 , , x n  1 , y 0 , , y n  1 )


Ontario
simple elastic snake energy
 | I x ( xi , yi ) |  | I y ( xi , yi ) |
2
2
i 0
here, energy is a function of 2n variables
n 1

   ( x i 1  x i )  ( y i 1  y i )
2
2
i0
update equation for each node

ν' i  ν i  Fi   t

Fi
C

Fi 
 xEi 
  E 
 yi 
= ?
E
xi
  2  I x  I xx  2  I y  I yx    2  ( x i  1  x i )    2  ( x i  x i  1 )
E
yi
  2  I x  I xy  2  I y  I yy    2  ( y i  1  y i )    2  ( y i  y i  1 )
Q: Do points move independently?
NO, motion of point i depends on positions of neighboring points
5-40
The University of
Gradient Descent for Snakes
C
n 1
E ( x 0 , , x n  1 , y 0 , , y n  1 )


Ontario
simple elastic snake energy
 | I x ( xi , yi ) |  | I y ( xi , yi ) |
2
2
i 0
here, energy is a function of 2n variables
n 1

   ( x i 1  x i )  ( y i 1  y i )
2
2
i0
update equation for each node

ν' i  ν i  Fi   t

Fi
C

Fi 
 xEi 
  E 
 yi 
= ?
E
xi
  2  I x  I xx  2  I y  I yx    2  ( x i  1  x i )    2  ( x i  x i  1 )
E
yi
  2  I x  I xy  2  I y  I yy    2  ( y i  1  y i )    2  ( y i  y i  1 )
from exterior
(image) energy
from interior
(smoothness) energy
5-41
The University of
Gradient Descent for Snakes
C
n 1
E ( x 0 , , x n  1 , y 0 , , y n  1 )


Ontario
simple elastic snake energy
 | I x ( xi , yi ) |  | I y ( xi , yi ) |
2
2
i 0
here, energy is a function of 2n variables
n 1

   ( x i 1  x i )  ( y i 1  y i )
2
2
i0
update equation for each node

Fi
C

ν' i  ν i  Fi   t

Fi 

Fi 
 xEi 
  E 
 yi 
2
 (|  I | )

2
( xi , yi )
motion of vi towards higher
magnitude of image gradients
2 
d v
2
d s
= ?
This term for vi
depends on
neighbors
vi-1 and vi+1
motion of vi reducing
contour’s bending
5-42
The University of
Discrete Snakes:
“Gradient Flow” evolution
Ontario
dC   dt   E
Contour evolution via
“Gradient flow”
update equation for each node
i
C
 i'

ν' i  ν i  Fi   t
i  0 , , n  1
Stopping criteria:
C’

F i  0 for all
i

E  0
local minima of energy E
5-43
The University of
Difficulties with Gradient Descent

Ontario
Very difficult to obtain accurate estimates of high-order
derivatives on images (discretization errors)
• E.g., estimating  E ex requires computation of second image
derivatives I xx , I xy , I yy

Gradient descent is not trivial even for one-dimensional
functions. Robust numerical performance for 2ndimensional function could be problematic.
• Choice of parameter  t is non-trivial
– Small  t , the algorithm may be too slow
– Large  t , the algorithm may never converge
• Even if “converged” to a good local minima, the snake is likely
to oscillate near it
5-44
The University of
Alternative solution for 2D snakes:
Dynamic Programming

Ontario
In many cases, snake energy can be written
as a sum of pair-wise interaction potentials
n 1
E total ( 0 ,  , n 1 )


E i ( i , i  1 )
i0
More generally, it can be written as a sum of
higher-order interaction potentials (e.g. triple interactions).

n 1
E total ( 0 ,  , n 1 )


E i ( i 1 , i , i  1 )
i0
5-45
The University of
Snake energy:
pair-wise interactions
Ontario
Example: simple elastic nsnake
energy
1

E total ( x 0 , , x n  1 , y 0 , , y n  1 )

 | I x ( xi , y i ) |  | I y ( xi , y i ) |
2
i 0
n 1

   ( x i 1  x i )  ( y i 1  y i )
2
2
i0
n 1
E total (  0 , , n  1 )



||  I (  i ) || 
2
i 0
n 1
   ||  i  1   i ||
2
i0
n 1
E total ( 0 ,  , n 1 )


E i ( i , i  1 )
i0
2
2
where E i (  i , i  1 )   ||  I (  i ) ||   ||  i   i  1 ||
Q: give an example of snake with triple-interaction potentials?
5-46
2
The University of
DP Snakes
[Amini, Weymouth, Jain, 1990]
Ontario
v2
v3
v1
v4
v5
v6
control points
First-order interactions
E ( v1 , v 2 ,..., v n )  E1 ( v1 , v 2 )  E 2 ( v 2 , v 3 )  ...  E n 1 ( v n 1 , v n )
Energy E is minimized via Dynamic Programming
5-47
The University of
DP Snakes
[Amini, Weymouth, Jain, 1990]
Ontario
v2
v3
v1
v4
v5
v6
control points
First-order interactions
E ( v1 , v 2 ,..., v n )  E1 ( v1 , v 2 )  E 2 ( v 2 , v 3 )  ...  E n 1 ( v n 1 , v n )
Energy E is minimized via Dynamic Programming
Iterate until optimal position for each point is the center of the box,
i.e. the snake is optimal in the local search space constrained by boxes
5-48
The University of
Dynamic Programming (DP)
Viterbi Algorithm
Ontario
Here we will concentrate on first-order interactions
sites
E1 ( v1 , v 2 )  E 2 ( v 2 , v 3 )  ...  E n 1 ( v n 1 , v n )
v1
E 1 ( v1 , v 2 )
v2
E 2 (v 2 , v3 )
states
1
2
…
m
v3
E 3 (v3 , v 4 )
v4
E 4 (v4 , vn )
vn
E1 (1)  0
E 2 (1)
E 3 (1)
E 4 (1)
E n (1)
E1 ( 2 )  0
E 2 (2)
E3 (2)
E 4 (2)
E n (2)
E1 (3)  0
E 2 (3)
E 3 (3)
E 4 (3)
E n (3)
E1 ( 4 )  0
E 2 (4)
E3 (4)
E 4 (4)
E n (4)
Complexity: O ( nm ), Worst case = Best Case
5-49
2
The University of
Dynamic Programming and
Hidden Markov Models (HMM)

Ontario
DP is widely used in speech recognition
time
word1
word2
word3
word4
ordered (in time) hidden variables (words) to be estimated from observed signal
E1 ( v 0 , v1 )  ...  E i ( v i 1 , v i )  ...  E n ( v n 1 , v n )
 ln{Pr( signal ( t i ) | word i ) }  ln{Pr( word i | word
i 1
)}
5-50
The University of
Snakes can also be seen as
Hidden Markov Models (HMM)
Ontario
Positions of snake nodes are hidden variables
 Timely order is replaced with spatial order
 Observed audible signal is replaced with image

1
n
E1 ( v1 , v 2 )  ...  E i ( v i , v i 1 )  ...  E n 1 ( v n 1 , v n )
 ||  I ( i ) ||  E elastic ( i , i 1 )
5-51
The University of
Dynamic Programming
for a closed snake?
Ontario
Clearly, DP can be applied to optimize an open ended snake
E1 ( v1 , v 2 )  E 2 ( v 2 , v 3 )  ...  E n 1 ( v n 1 , v n )
1
n
Can we use DP for a “looped” energy in case of a closed snake?
E1 ( v1 , v 2 )  E 2 ( v 2 , v 3 )  ...  E n 1 ( v n 1 , v n )  E n ( v n , v1 )
 n 1
n
1
2  4
3
5-52
The University of
Dynamic Programming
for a closed snake
Ontario
E1 ( v1 , v 2 )  E 2 ( v 2 , v 3 )  ...  E n 1 ( v n 1 , v n )  E n ( v n , v1 )
1. Can use Viterbi to optimize snake
energy in case  1  c is fixed.
(in this case the energy above
effectively has no loop)
2. Use Viterbi to optimize snake for
all possible values of c and
choose the best of the obtained m
solutions.
for exact solution
complexity
increases to
O(nm3)
5-53
The University of
Dynamic Programming
for a closed snake
Ontario
E1 ( v1 , v 2 )  E 2 ( v 2 , v 3 )  ...  E n 1 ( v n 1 , v n )  E n ( v n , v1 )
DP has problems with “loops” (even one loop increases complexity).
However, some approximation tricks can be used in practice…
1. Use DP to optimize snake energy
with fixed  1 (according to a
given initial snake position).
2. Use DP to optimize snake energy
again. This time fix position of an
intermediate node  n / 2  ˆ n / 2
where ˆ is an optimal position
obtained in step 1.
This is only an
approximation,
but complexity is
good: O(nm2)
5-54
The University of
Dynamic Programming
for snakes with higher order interactions
Ontario
E1 ( v1 , v 2 , v 3 )  E 2 ( v 2 , v 3 , v 4 )  ...  E n  2 ( v n  2 , v n 1 , v n )
(e.g. if bending energy is added into the “model” of the snake)
Viterbi algorithm can be generalized to 3-clique
case but its complexity increases to O(nm3).
one approach: combine each pair of neighboring
nodes into one super node. Each triple interaction
v4
can be represented as a pair-wise interaction
v5
between 2 super-nodes. Viterbi algorithm
will need m3 operations for each super node (why?)
v3
v2
v1
5-55
The University of
DP snakes (open case)
Summary of Complexity
energy
type
Ontario
complexity
(order of interactions)
n
E
i
(vi )
i 1
n 1
E
i
( v i , v i 1 )
i 1
n2
E
i
( v i , v i 1 , v i  2 )
i 1
E ( v1 , v 2 ,..., v n )
unary potentials
O(nm)
(d=1)
pair-wise potentials
O((n-1)m2)*
(d=2)
triple potentials
O((n-2)m3)*
(d=3)
complete connectivity
O(mn) – exhaustive search
(d=n)
* - adding a single loop increases complexity by factor
md-1
5-56
The University of
Problems with snakes
Ontario
Depends on number and spacing of control points
 Snake may oversmooth the boundary
 Not trivial to prevent curve self intersecting


Can not follow topological changes of objects
5-57
The University of
Problems with snakes

Ontario
May be sensitive to initialization
– may get stuck in a local energy minimum near initial contour

Numerical stability can be an issue for gradient descent and
variational methods (continuous formulation)
• E.g. requires computing second order derivatives

The general concept of snakes (deformable models) does
generalize to 3D (deformable mesh), but many robust
optimization methods suitable for 2D snakes do not apply in 3D
• E.g.: dynamic programming only works for 2D snakes
5-58
The University of
Problems with snakes

Ontario
External energy: may need to diffuse image gradients,
otherwise the snake does not really “see” object
boundaries in the image unless it gets very close to it.
image gradients  I
are large only directly on the boundary
5-59
The University of
Diffusing Image Gradients  I
Ontario
image gradients diffused via
Gradient Vector Flow (GVF)
Chenyang Xu and Jerry Prince, 98
http://iacl.ece.jhu.edu/projects/gvf/
5-60
The University of
Alternative Way to
Improve External Energy
n 1

Use
E ex 
 D( v
Ontario
n 1
i
)
instead of
i 0
• Distance Transform
E ex    |  I ( v i ) |
where D() is
i0
(for detected binary image features, e.g. edges)
binary image features
(edges)
Distance Transform can
be visualized as a grayscale image
Distance Transform D ( x , y )
• Generalized Distance Transform (directly for image gradients)
5-61
The University of
Distance Transform
(see p.20-21 of the text book)
Ontario
Image features (2D)
1
1
1
1
2
3
4
5
Distance Transform
0 1 2 3 4 3
0 1 2 3 3 2
0 1 2 3 2 1
0 0 1 2 1 0
1 1 2 1 0 1
2 2 2 1 0 1
3 3 2 1 0 1
4 4 3 2 1 0
2
1
0
1
2
2
2
1
Distance Transform is a function D () that for each image
pixel p assigns a non-negative number D ( p ) corresponding to
distance from p to the nearest feature in the image I
5-62
The University of
Distance Transform
can be very efficiently computed
Ontario
5-63
The University of
Distance Transform
can be very efficiently computed
Ontario
5-64
The University of
Distance Transform
can be very efficiently computed
Ontario
• Forward-Backward pass algorithm computes
shortest paths in O(n) on a grid graph with regular
4-N connectivity and homogeneous edge weights 1
• Alternatively, Dijkstra’s algorithm can also compute
a distance map (trivial generalization for multiple
sources), but it would take O(n*log(n)).
- Dijkstra is slower but it is a more general
method applicable to arbitrary weighted graphs
5-65
The University of
Distance Transform:
an alternative way to think about

Assuming
Ontario
 0 if pixel p is image feature 
F ( p)  

  O .W .

then D ( p )  min {|| p  q ||  F ( q )}  min || p  q ||
q
q :F ( q )  0
is standard Distance Transform (of image features)



F ( p)
D ( p)
p
Locations of binary image features
5-66
The University of
Distance Transform vs.
Generalized Distance Transform

For general
Ontario
F ( p)
D ( p )  min {  || p  q ||  F ( q )}
q
is called Generalized Distance Transform of
F ( p)
F ( p)
D ( p)
D(p) may prefer
“strength” of F(p)
to proximity
q
p
F(p) may represent non-binary image features (e.g. image intensity gradient)
5-67
The University of
Generalized Distance Transforms
(see Felzenszwalb and Huttenlocher, IJCV 2005)

Ontario
The same “Forward-Backward” algorithm can be
applied to any initial array
• Binary (0 /  ) initial values are non-essential.

If the initial array contains values of function F(x,y) then
the output of the “Forward-Backward” algorithm is a
Generalized Distance Transform
D ( p )  min (  || p  q ||  F ( q ))
q I

“Scope of attraction” of image gradients can be extended
via external energy E   D ( v ) based on a generalized
distance transform of
n 1
ex
i
i0
F ( x , y )  g (|  I ( x , y ) |)
5-68
The University of
Metric properties of
discrete Distance Transforms
Forward
mask
Backward
mask
-
1
0
1
1
0
1
-
1.4
1
1
0
1.4
1.4
Metric
Ontario
Set of equidistant
points
Manhattan (L1) metric
0
1
1
1.4
In fact, “exact” Euclidean Distance
transform can be computed fairly
efficiently (in linear or near-linear time)
without bigger masks
1) www.cs.cornell.edu/~dph/matchalgs/
2) Fast Marching Method –Tsitsiklis, Sethian
Better approximation
of Euclidean metric
Euclidean (L2) metric
5-69
The University of
HW assignment 2

Ontario
DP Snakes
• Use elastic snake model
• Compare
E  E int  l  E ext (value of l is important)
n 1
n 1
E int 

Li
i0
• Compare
2
vs.
E int 
i0
n 1
 L vs. E
E   F ( v ) vs.
• Compare
generalized distance transform of
2
E int 
n 1
ext
i
i 0
i
i0
such that
|L
int
i
2
 Lˆ |
n 1

|L
i0
E ext 
i
|
n 1
 D ( v ) where D is a
i
i0
F  g (|  I |)
D ( p )  min {  || p  q ||  F ( q )}
(value of  is important)
q
• Use Viterbi algorithm for optimization
• Incorporate edge alignment
• Use 3x3 search box for each control point
5-70