Transcript ITK Lecture 8 - Neighborhoods Damion Shelton Methods in Image Analysis

ITK Lecture 8 - Neighborhoods
Damion Shelton
Methods in Image Analysis
CMU Robotics Institute 16-725
U. Pitt Bioengineering 2630
Spring Term, 2004
1
Methods in Image Analysis 2004, Damion Shelton
Goals for today
• Understand what a neighborhood is and and
the different ways of accessing pixels using
one
• Use neighborhoods to implement a
convolution/correlation filter
2
Methods in Image Analysis 2004, Damion Shelton
What is a neighborhood?
• You may already be familiar with the
concept of pixels having neighbors
• Standard terminology in 2D image
processing will refer to the 4 neighborhood
(N,E,S,W) and the 8 neighborhood (4
neighborhood + NE, SE, SW, NW)
3
Methods in Image Analysis 2004, Damion Shelton
Neighborhoods in ITK
• ITK carries this concept a bit further
• A neighborhood can be any collection of
pixels that have a fixed relationship to the
“center” based on offsets in data space
4
Methods in Image Analysis 2004, Damion Shelton
Neighborhoods in ITK, cont.
• In general, the neighborhood is not
completely arbitrary
– Neighborhoods are rectangular, defined by a
“radius” in N-dimensions
– ShapedNeighborhoods are arbitrary, defined by
a list of offsets from the center
• The first form is most useful for
mathematical morphology kinds of
operations, convolution, etc.
5
Methods in Image Analysis 2004, Damion Shelton
Neighborhood iterators
• The cool & useful thing about
neighborhoods is that they can be used with
neighborhood iterators to allow efficient
access to pixels “around” a target pixel in an
image
6
Methods in Image Analysis 2004, Damion Shelton
Neighborhood iterators
• Remember that I said access via pixel
indices was slow?
– Get current index = ind
– Upper left pixel index uleft = ind - (1,1)
– Get pixel at index uleft
• Neighborhood iterators solve this problem
by doing pointer arithmetic based on offsets
7
Methods in Image Analysis 2004, Damion Shelton
Neighborhood layout
• Neighborhoods have one primary
parameter, their “radius” in N-dimensions
• The side length along a particular
dimension i is 2*radiusi + 1
• Note that the side length is always odd
because the center pixel always exists
8
Methods in Image Analysis 2004, Damion Shelton
A 2x1 neighborhood in 2D
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
9
Methods in Image Analysis 2004, Damion Shelton
Stride
• Neighborhoods have another parameter
called stride which is the spacing (in data
space) along a particular axis between
adjacent pixels in the neighborhood
• In the previous numbering scheme, stride in
Y is amount then index value changes when
you move in Y
• In our example, Stridex = 1, Stridey = 5
10
Methods in Image Analysis 2004, Damion Shelton
Neighborhood pixel access
• The numbering on the previous page is
important! It’s how you access that
particular pixel when using a neighborhood
iterator
• This will be clarified in a few slides...
11
Methods in Image Analysis 2004, Damion Shelton
NeighborhoodIterator access
• Neighborhood iterators are created using:
– The radius of the neighborhood
– The image that will be traversed
– The region of the image to be traversed
• There syntax largely follows that of other
iterators (++, IsAtEnd(), etc.)
12
Methods in Image Analysis 2004, Damion Shelton
Neighborhood pixel access, cont.
Let’s say there’s some region of an image that has
the following pixel values
1.2 1.3 1.8 1.4 1.1
1.8 1.1 0.7 1.0 1.0
2.1 1.9 1.7 1.4 2.0
13
Methods in Image Analysis 2004, Damion Shelton
Pixel access, cont.
• Now assume that we place the
neighborhood iterator over this region and
start accessing pixels
• What happens?
14
Methods in Image Analysis 2004, Damion Shelton
Pixel access, cont.
myNeigh.GetPixel(7) returns 0.7
so does myNeigh.GetCenterPixel()
1.2
0
1.8
5
2.1
10
1.3
1
1.1
6
1.9
11
1.8
2
0.7
7
1.7
12
1.4
3
1.0
8
1.4
13
1.1
4
1.0
9
2.0
14
15
Methods in Image Analysis 2004, Damion Shelton
Pixel access, cont.
Next, let’s get the length of the iterator and the
stride length
Size() returns the #pixels in the neighborhood
unsigned int c = iterator. Size () / 2;
GetStride returns the stride of dimension N
unsigned int s = iterator. GetStride (1) ;
16
Methods in Image Analysis 2004, Damion Shelton
Pixel access, cont.
myNeigh.GetPixel(c) returns 0.7
myNeigh.GetPixel(c-1) returns 1.1
1.2
0
1.8
5
2.1
10
1.3
1
1.1
6
1.9
11
1.8
2
0.7
7
1.7
12
1.4
3
1.0
8
1.4
13
1.1
4
1.0
9
2.0
14
17
Methods in Image Analysis 2004, Damion Shelton
Pixel access, cont.
myNeigh.GetPixel(c-s) returns 1.8
myNeigh.GetPixel(c-s-1) returns 1.3
1.2
0
1.8
5
2.1
10
1.3
1
1.1
6
1.9
11
1.8
2
0.7
7
1.7
12
1.4
3
1.0
8
1.4
13
1.1
4
1.0
9
2.0
14
18
Methods in Image Analysis 2004, Damion Shelton
The ++ method
• In ImageRegionIterators, the ++ method
moves the focus of the iterator on a per
pixel basis
• In NeighborhoodIterators, the ++ method
moves the center pixel of the neighborhood
and therefore implicitly shifts the entire
neighborhood
19
Methods in Image Analysis 2004, Damion Shelton
Does this sound familiar?
• If I say:
– I have a region of interest defined by a certain
radius around a center pixel
– The ROI is symmetric
– I move it around an image
• What does this sound like?
20
Methods in Image Analysis 2004, Damion Shelton
Convolution (ahem, correlation)!
To do convolution we need 3 things:
1. A kernel
2. A way to access a region of an image the
same size as the kernel
3. A way to compute the inner product between
the kernel and the image region
21
Methods in Image Analysis 2004, Damion Shelton
Item 1 - The kernel
• A NeighborhoodOperator is a set of pixel
values that can be applied to a
Neighborhood to perform a user-defined
operation (i.e. convolution kernel,
morphological structuring element)
• NeighborhoodOperator is derived from
Neighborhood
22
Methods in Image Analysis 2004, Damion Shelton
Item 2 - Image access method
• We already showed that this is possible
using the neighborhood iterator
• Just be careful setting it up so that it’s the
same size as your kernel
23
Methods in Image Analysis 2004, Damion Shelton
Item 3 - Inner product method
• The NeighborhoodInnerProduct computes
the inner product between two
neighborhoods
• Since NeighborhoodOperator is derived
from Neighborhood, we can compute the IP
of the kernel and the image region
24
Methods in Image Analysis 2004, Damion Shelton
Good to go?
1. Create an interesting operator to form a
kernel
2. Move a neighborhood through an image
3. Compute the IP of the operator and the
neighborhood at each pixel in the image
Voila - convolution in N-dimensions
25
Methods in Image Analysis 2004, Damion Shelton
Inner product example
itk::NeighborhoodInnerProduct<ImageType> IP;
itk::DerivativeOperator<ImageType> operator ;
operator->SetOrder(1);
operator->SetDirection(0);
operator->CreateDirectional();
itk::NeighborhoodIterator<ImageType> iterator(operator>GetRadius(), myImage, myImage>GetRequestedRegion());
26
Methods in Image Analysis 2004, Damion Shelton
Inner product example, cont.
iterator.SetToBegin();
while ( ! iterator. IsAtEnd () )
{
std::cout << "Derivative at index " <<
iterator.GetIndex () << is <<
IP(iterator, operator) << std::endl;
++iterator;
}
27
Methods in Image Analysis 2004, Damion Shelton
This suggests a filter...
• NeighborhoodOperatorImageFilter wraps
this procedure into a filter that operates on
an input image
• So, if the main challenge is coming up with
an interesting neighborhood operator, ITK
can do the rest
28
Methods in Image Analysis 2004, Damion Shelton
Your arch-nemesis... image boundaries
• One obvious problem with inner product
techniques is what to do when you reach the
edge of your image
• Is the operation undefined?
• Does the image wrap?
• Should we assume the rest of the world is
empty/full/something else?
29
Methods in Image Analysis 2004, Damion Shelton
ImageBoundaryCondition
• Subclasses of itk::ImageBoundaryCondition
can be used to tell neighborhood iterators
what to do if part of the neighborhood is not
in the image
30
Methods in Image Analysis 2004, Damion Shelton
ConstantBoundaryCondition
• The rest of the world is filled with some
constant value of your choice
• The default is 0
• Be careful with the value you choose - you
can (for example) detect edges that aren’t
really there
31
Methods in Image Analysis 2004, Damion Shelton
PeriodicBoundaryCondition
• The image wraps, so that if I exceed the
length of a particular axis, I wrap back to 0
and start over again
• If you enjoy headaches, imagine this in 3D
• This isn’t a bad idea, but most medical
images are not actually periodic
32
Methods in Image Analysis 2004, Damion Shelton
ZeroFluxNeumannBoundaryCondition
• I am not familiar with how this functions
• The documentation states that it’s useful for
solving certain classes of differential
equations
• A quick look online suggests a
thermodynamic motivation
33
Methods in Image Analysis 2004, Damion Shelton
Using boundary conditions
• With NeighborhoodOperatorImageFilter,
you can call OverrideBoundaryCondition
34
Methods in Image Analysis 2004, Damion Shelton
SmartNeighborhoodIterator
• This is the iterator that’s being used
internally by the previous filter; you can
specify its boundary behavior using
OverrideBoundaryCondition too
• In general, I would suggest using the
“smart” version - bounds checking is good!
35
Methods in Image Analysis 2004, Damion Shelton
An aside: numeric traits
• This has nothing to do with Neighborhoods
but is relevant to the assignment 2
• Question: given some arbitrary pixel type,
what do we know about it from a numerics
perspective?
36
Methods in Image Analysis 2004, Damion Shelton
itk::NumericTraits
• NumericTraits is class that’s specialized to
provide information about pixel types
• Examples include:
– min and max values
– IsPositive(), IsNegative()
– Definitions of Zero and One
37
Methods in Image Analysis 2004, Damion Shelton
Using traits
• What’s the maximum value that can be
represented by an unsigned char?
• itk::NumericTraits<unsigned char>::max()
• Look at vnl_numeric_limits for more data
that can be provided
38
Methods in Image Analysis 2004, Damion Shelton
Next assignment - due 2/12/04
• Design a filter that “inverts” a grayscale
image
• By inverts I mean that dark values (low)
become light values (high) and vice versa
• This filter should work on a variety of data
types, by taking into account the numeric
traits of the input and output types
39
Methods in Image Analysis 2004, Damion Shelton
Assignment, cont.
• I’m pretty sure there’s not an existing
implementation of this filter
• If there it, it’s not okay to use it
• It’s perfectly fine (and encouraged) to use
the filter mentioned in the filter lecture as a
template and rename things
• You should replace the myITKgui filter
with your inversion filter
40
Methods in Image Analysis 2004, Damion Shelton