Transcript CSci 6971: Image Registration Lecture 5: Feature-Base Regisration January 27, 2004 Prof. Chuck Stewart, RPI Dr.

CSci 6971: Image Registration
Lecture 5: Feature-Base Regisration
January 27, 2004
Prof. Chuck Stewart, RPI
Dr. Luis Ibanez, Kitware
Overview
 What is feature-based (point-based)
registration?
 Feature points
 The correspondence problem
 Solving for the transformation estimate
 Putting it all together: ICP
 Discussion and conclusion
Image Registration
Lecture 5
2
What is Feature-Based Registration?
 Images are described as discrete sets of
point locations associated with a geometric
measurement
 Locations may have additional properties
such as intensities and orientations
 Registration problem involves two parts:
 Finding correspondences between features
 Estimating the transformation parameters
based on these correspondences
Image Registration
Lecture 5
3
Feature Examples: Range Data
 Range image points:
 (x,y,z) values
 Triangulated mesh
 Surface normals are
sometimes computed
 Notice:
 Some information
(locations) is
determined directly by
the sensor (“raw data”)
 Some information is
inferred from the data
Image Registration
Lecture 5
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
4
Feature Examples: Vascular Landmarks
 Branching points
pulmonary images:
 Lung vessels
 Airway branches
 Retinal image
branches and
cross-over points
 Typically augmented
(at least) with
orientations of
vessels meeting to
form landmarks
Image Registration
Lecture 5
5
Points Along Centers of Vessels and Airways
 Airways and vessels
modeled as tubular
structures
 Sample points spaced
along center of tubes
 Note that the entire
tube is rarely used
as a unit
 Augmented
descriptions:
 Orientation
 Radius
Image Registration
Lecture 5
6
“Interest” Points
 Locations of strong
intensity variation in all
directions
 Augmented with summary
descriptions (moments) of
surrounding intensity
structures
 Recent work in making
these invariant to
viewpoint and
illumination.
 We’ll discuss interest
points during Lectures 16
and 17
Image Registration
Lecture 5
Brown and Lowe, Int. Conf. On
Computer Vision, 2003
7
Feature Points: Discussion
 Many different possible features
 Problem is reliably extracting features in all
images
 This is why more sophisticated features are
not used
 Feature extraction methods do not use all
intensity values
 Use of features dominates range-image
registration techniques where “features” are
provided by the sensor
Image Registration
Lecture 5
8
Preamble to Feature-Based Registration: Notation
 Set of moving image features
 Set of fixed image features
 Each feature must include a point location in
the coordinate system of its image. It may
include more
 Set of correspondences
Image Registration
Lecture 5
9
Mathematical Formulation
 Error objective function depends on unknown
transformation parameters and unknown feature
correspondences
 Each may depend on the other!
 Transformation may include mapping of more than
just locations
 Distance function, D, could be as simple as the
Euclidean distance between location vectors.
 We are using the forward transformation model.
Image Registration
Lecture 5
10
Correspondence Problem
 Determine correspondences before estimating
transformation parameters
 Based on rich description of features
 Error prone
 Determine correspondences at the same time as
estimation of parameters
 “Chicken-and-egg” problem
 For the next few minutes we will assume a set of
correspondences is given and proceed to the
estimation of parameters
 Then we will return to the correspondence
problem
Image Registration
Lecture 5
11
Example: Estimating Parameters
 2d point locations:
 Similarity transformation:
 Euclidean distance:
Image Registration
Lecture 5
12
Putting This Together
Image Registration
Lecture 5
13
What Do We Have?
 Least-squares objective function
 Quadratic function of each parameter
 We can
 Take the derivative with respect to each
parameter
 Set the resulting gradient to 0 (vector)
 Solve for the parameters through matrix
inversion
 We’ll do this in two forms: component and
matrix/vector
Image Registration
Lecture 5
14
Component Derivative (a)
Image Registration
Lecture 5
15
Component Derivative (b)
At this point, we’ve dropped the leading factor of 2.
It will be eliminated when this is set to 0.
Image Registration
Lecture 5
16
Component Derivatives tx and ty
Image Registration
Lecture 5
17
Gathering
 Setting each of these equal to 0 we obtain a
set of 4 linear equations in 4 unknowns.
Gathering into a matrix we have:
Image Registration
Lecture 5
18
Solving
 This is a simple equation of the form
 Provided the 4x4 matrix X is full-rank
(evaluate SVD) we easily solve as
Image Registration
Lecture 5
19
Matrix Version
 We can do this in a less painful way by
rewriting the following intermediate
expression in terms of vectors and matrices:
Image Registration
Lecture 5
20
Matrix Version (continued)
 This becomes
 Manipulating:
Image Registration
Lecture 5
21
Matrix Version (continued)
 Taking the derivative of this wrt the
transformation parameters (we didn’t cover
vector derivatives, but this is fairly
straightforward):
 Setting this equal to 0 and solving yields:
Image Registration
Lecture 5
22
Comparing the Two Versions
 Final equations are identical (if you expand
the symbols)
 Matrix version is easier (once you have
practice) and less error prone
 Sometimes efficiency requires handcalculation and coding of individual terms
Image Registration
Lecture 5
23
Resetting the Stage
 What we have done:
 Features
 Error function of transformation parameters
and correspondences
 Least-squares estimate of transformation
parameters for fixed set of
correspondences
 Next:
 ICP: joint estimation of correspondences
and parameters
Image Registration
Lecture 5
24
Iterative Closest Points (ICP) Algorithm
 Given an initial transformation estimate 0
 t=0
 Iterate until convergence:
 Establish correspondences:
 For fixed transformation parameter estimate, t,
apply the transformation to each moving image
feature and find the closest fixed image feature
 Estimate the new transformation
parameters,
 For the resulting correspondences, estimate
t+1
ICP algorithm was developed almost simultaneous by at
least 5 research groups in the early 1990’s.
Image Registration
Lecture 5
25
Finding Correspondences
 Map feature into coordinate system of If
 Find closest point
Image Registration
Lecture 5
26
Finding Correspondences (continued)
 Enforce unique correspondences
 Avoid trivial minima of objective function
due to having no correspondences
 Spatial data structures needed to make
search for correspondences efficient
 K-d trees
 Digital distance maps
 More during lectures 11-15…
Image Registration
Lecture 5
27
Initialization and Convergence
 Initial estimate of transformation is again crucial
because this is a minimization technique
 Determining correspondences and estimating the
transformation parameters are two separate
processes
 With Euclidean distance metrics you can show
they are working toward the same minimum
 In general this is not true
 Convergence in practice is sometimes problematic
and the correspondences oscillate between points.
Image Registration
Lecture 5
28
2d Retinal Example
 White = vessel
centerline points from
one image
 Black = vessel
centerline points from
second image
 Yellow line segments
drawn between
corresponding points
 Because of the
complexity of the
structure, initialization
must be fairly accurate
Image Registration
Lecture 5
29
Comparison
Intensity-Based
Feature-Based
 For a given transformation
estimate, we can only find
a new, better estimate, not
the best estimate, based
on the gradient step.
 We then need to update
the constraints and reestimate
 For given set of
correspondences, we
can directly (leastsquares) estimate the
best transformation
 BUT, the transformation
depends on the
correspondences, so
we generally need to reestablish the
correspondences.
Image Registration
Lecture 5
30
Summary
 Feature-based registration
 Feature types and properties
 Correspondences
 Least-squares estimate of parameters based
on correspondences
 ICP
 Comparison
Image Registration
Lecture 5
31
Looking Ahead to Lecture 6
 Introduction to ITK and the ITK registration
framework.
Image Registration
Lecture 5
32