Transcript week02-connectedComponents.ppt

Binary Image Proc: week 2




Getting a binary image
Connected components extraction
Morphological filtering
Extracting numerical features from
regions
Stockman CSE803 Fall 2009
1
Quick look at thresholding




Separate objects from background.
2 class or many class problem?
How to do it?
Discuss methods later.
Stockman CSE803 Fall 2009
2
Cherry image shows 3 regions




Background is black
Healthy cherry is bright
Bruise is medium dark
Histogram shows two cherry
regions (black background has
been removed)
Stockman CSE803 Fall 2009
Use this gray
value to separate
3
Choosing a threshold




Common to find the deepest valley
between two modes of bimodal histogram
Or, can level-slice using the intensities
values a and b that bracket the mode of
the desired objects
Can fit two or more Gaussian curves to
the histogram
Can do optimization on above (Ohta et al)
Stockman CSE803 Fall 2009
4
Connected components






Assume thresholding obtained binary image
Aggregate pixels of each object
2 different program controls
Different uses of data structures
Related to paint/search algs
Compute features from each object region
Stockman CSE803 Fall 2009
5
Notes on Binary Image Proc







Connected Components Algorithms
Separate objects from background
Aggregate pixels for each object
Compute features for each object
Different ways of program control
Different uses of data structures
Related to paint/search algs
Stockman CSE803 Fall 2009
6
Example red blood cell image




Many blood cells are
separate objects
Many touch – bad!
Salt and pepper
noise from
thresholding
How useable is this
data?
Stockman CSE803 Fall 2009
7
Cleaning up thresholding results




Can delete object
pixels on boundary
to better separate
parts.
Can fill small holes
Can delete tiny
objects
(last 2 are “salt-andpepper” noise)
Stockman CSE803 Fall 2009
8
Removing salt-and-pepper


Change pixels all
(some?) of whose
neighbors are
different (coercion!):
see hole filled at right
Delete objects that are
tiny relative to targets:
see some islands
removed at right
Stockman CSE803 Fall 2009
9
Simple morphological cleanup


Can be done just after thresholding
-- remove salt and pepper
Can be done after connected
components are extracted
-- discard regions that are too small
or too large to be the target
Stockman CSE803 Fall 2009
10
CC analysis of red blood cells




63 separate
objects
detected
Single cells
have area
about 50
Noise spots
Gobs of cells
Stockman CSE803 Fall 2009
11
More control of imaging




More uniform
objects
More uniform
background
Thresholding
works
Objects actually
separated
Stockman CSE803 Fall 2009
12
Results of “pacmen” analysis




15 objects
detected
Location known
Area known
3 distinct
clusters of 5
values of area;
85, 145, 293
Stockman CSE803 Fall 2009
13
Results of “coloring” objects



Each object
is a
connected
set of pixels
Object label
is “color”
How is this
done?
Stockman CSE803 Fall 2009
14
Extracting components: Alg A


Collect connected foreground pixels into
separate objects – label pixels of same
object with same color
A) collect by “random access” of pixels
using “paint” or “fill” algorithm
Stockman CSE803 Fall 2009
15
Extracting “white regions”




Program learns white
from training set of
sample pixels.
Aggregate similar white
neighbors to form
regions.
Regions can then be
classified as characters.
(Work contributed by
David Moore, CSE 803.)
(Left) input
RGB image of
a NASA sign
(Right)
output is a
labeled
image.
Stockman CSE803 Fall 2009
16
paint/fill algorithm



Obj region
must be
bounded by
background
Start at any
pixel [r,c]
inside obj
Recursively
color
neighbors
Stockman CSE803 Fall 2009
17
Events of paint/fill algorithm





PP denotes “processing point”
If PP outside image, return to prior PP
If PP already labeled, return to prior PP
If PP is background pixel, return to prior PP
If PP is unlabeled object pixel, then
1) label PP with current color code
2) recursively label neighbors N1, …, N8
(or N1, …, N4)
Stockman CSE803 Fall 2009
18
Recursive Paint/Fill Alg: 1 region



Color closed
boundary with L
Choose pixel [r,c]
inside boundary
Call FILL
FILL ( I, r, c, L)
If [r,c] is out, return
If I[r,c] == L, return
I[r,c]  L // color it
For all neighbors [rn,cn]
FILL(I, rn, cn, L)
Stockman CSE803 Fall 2009
19
Recall different neighborhoods




4-connected: right, up, left, down only
8-connected: have 45 degree diagonals also
If the pixels [r,c] of an image region R are 4connected, then there exists a path from any
pixel [r1,c1] in R to any other pixel [r2,c2] in
R using only 4 neighbors
The coloring algorithm is guaranteed to color
all connected pixels of a region. Why?
Stockman CSE803 Fall 2009
20
Correctness of recursive coloring




If any pixel P is colored L, then all of
its object neighbors must be colored
L (because FILL will be recursively
called on each neighbor)
If pixels P and Q are in connected
region R, then there must be a path
from pixel P to Q inside of R.
If FILL does not color Q, let X be the
first uncolored pixel on a path from P
to Q.
If a neighbor N of X is colored, then
so must be X! This contradiction
proves the property – all of the path
Pixels P
and Q are
in a
connected
region.
must be colored.
Stockman CSE803 Fall 2009
P
Q
NX
P is colored; Q
is not; N is
colored; X is
not. But if N is
colored, X
must be too!
21
Connected components using
recursive Paint/Fill




Raster scan until object pixel found
Assign new color for new object
Search through all connected neighbors
until the entire object is labeled
Return to the raster scan to search for
another object pixel
Stockman CSE803 Fall 2009
22
Extracting 5 objects
Stockman CSE803 Fall 2009
23
Outside material to cover





Look at C++ functions for raster
scanning and pixel “propagation”
Study related fill algorithm
Discuss how the recursion works
Prove that all pixels connected to the
start pixel must be colored the same
Runtime stack can overflow if region
has many pixels.
Stockman CSE803 Fall 2009
24
Alg B: raster scan control
Visit each image pixel once, going by
row and then column.
Propagate color to neighbors below
and to the right.
Keep track of merging colors.
Stockman CSE803 Fall 2009
25
Have caution in programming



Note how your image tool
coordinates the pixels.
Different tools or
programming languages
handle rows and columns
differently
Text uses EE-TV coordinates
or standard math coordinates
Stockman CSE803 Fall 2009
row
col
Yi
Xi
26
Raster scanning control
Stockman CSE803 Fall 2009
27
Events controlled by neighbors




If all Ni background,
then PP gets new
color code
If all Ni same color
L, then PP gets L
If Ni != Nj, then
take smallest code
and “make” all
same
See Ch 3.4 of S&S
Processing point
Stockman CSE803 Fall 2009
28
Merging connecting regions
Detect and record
merges while
raster scanning.
Use equivalence
table to recode
(in practice,
probably have to
start with color
= 2, not 1)
Stockman CSE803 Fall 2009
29
alg A






versus
Visits pixels more than
once
Needs full image
Recursion or stacking
slower than B
No need to recolor
Can compute features on
the fly
Can quit if search object
found (tracking?)





alg B
“visits” each pixel once
Needs only 2 rows of
image at a time
Need to merge colors
and region features
when regions merge
Typically faster
Not suited for heuristic
(smart) start pixel
Stockman CSE803 Fall 2009
30
Outside material




More examples of raster scanning
Union-find algorithm and parent table
Computing features from labeled object
region
More on recursion and C++
Stockman CSE803 Fall 2009
31
Computing features of regions
Can postprocess results of CC alg.
Or, can compute as pixels are
aggregated
Stockman CSE803 Fall 2009
32
Area and centroid
Stockman CSE803 Fall 2009
33
Bounding box: BB





Smallest rectangle containing all pixels
r-low is lowest row value; c-low is
lowest column value
r-high, c-high are the highest values
Bounding box has diagonal corners [rlow, c-low] and [r-high, c-high]
BBs summarize where an object is; they
can be used to rule out near collisions
Stockman CSE803 Fall 2009
34
Second moments
These are invariant to object location in the image.
Stockman CSE803 Fall 2009
35
Contrast second moments



For the letter ‘I’
Versus the letter ‘O’
Versus the underline ‘_’
r
I
c
Stockman CSE803 Fall 2009
36
Perimeter pixels and length
How do we find these boundary pixels?
Stockman CSE803 Fall 2009
37
Circularity or elongation
Computing P is more trouble than
computing A.
Stockman CSE803 Fall 2009
38
Circularity as variance of “radius”
Stockman CSE803 Fall 2009
39
Radial mass transform
• for each radius r, accumulate the mass at distance r from the
centroid (rotation and translation invariant)
• can iterate over bounding box and for each pixel, compute a
rounded r and increment histogram of mass H[r]
Stockman CSE803 Fall 2009
40
Interest point detection



Centroids of regions can be interesting
points for analysis and matching.
What do we do if regions are difficult
to extract?
We might transform an image
neighborhood into a feature vector, and
then classify as “interesting” vs “not”.
Stockman CSE803 Fall 2009
41
Slice of spine MRI and interesting
points selected by RMT & SVM
Stockman CSE803 Fall 2009
42
3D microvolumes from Argonne high
energy sensor: 1 micron voxels
Ram CAT slice of a bee stinger (left) versus segmented slice
(right). Each voxel is about 2 microns per side.
Stockman CSE803 Fall 2009
43
Scanning technique used
CCD camera
material sample
X-rays
scintillator
Pin head
X-rays partly absorbed by sample;
excite scintillator producing image in
the camera; rotate sample a few
degrees and produce another image;
3D reconstruction using CT
rotate
Stockman CSE803 Fall 2009
44
Different view of stinger
Rendered using ray tracing and pseudo coloring based on the
material density clusters that were used to separate object from
background. (Data scanned at Argonne National Labs)
Stockman CSE803 Fall 2009
45
Section of interesting points from
RMT&SVM
Stockman CSE803 Fall 2009
46
Segmentation of Scutigera
Stockman CSE803 Fall 2009
47
Scutergera: a tiny crustacean
• organism is smaller than
1 mm
• scanned by
• volume segmented and
meshed by Paul Albee
• roughly ten million triangles
to represent the surface
• anaglyph created for 3D
visualization (view with glasses)
Stockman CSE803 Fall 2009
48
Axis of least (most) inertia
• gives object oriented, or
natural, coordinate system
• passes through centroid
• axis of most inertia is
perpendicular to it
• concept extends to 3D
and nD
Stockman CSE803 Fall 2009
49
Approach (in Cartesian/math
coordinates)





First assume axis goes through centroid
Assume axis makes arbitrary angle Θ with the
horizontal axis
Write formula for the angular momentum
(inertia) to rotate a single unit of mass (pixel)
about the axis
Write formula for the sum over all pixels
Take derivative of formula to find Θ for
extreme values
Stockman CSE803 Fall 2009
50
Rotational inertia of pixel
Y
d
[x,y]
Θ + 90
Θ
Axis of
inertia
X
Centroid of pixels
Stockman CSE803 Fall 2009
51
Overall steps



d = [x, y] projected along normal to axis
= -x sin Θ + y cos Θ
Inertia of single pixel is d squared
Inertia of entire region is sum over all pixels
İ = ∑ ( x sin Θ – y cos Θ)^2
= sin^2 Θ ∑ x^2 – 2sin Θ cos Θ ∑ xy +
cos^2 Θ ∑ y^2
= sin^2 Θ a – 2sin Θ cos Θ b + cos^2 Θ c
where a, b, c are the second moments!
Stockman CSE803 Fall 2009
52
Final form





By differentiating İ with respect to Θ
and setting to 0
0 = a tan 2 Θ -2b – c tan 2 Θ
tan 2 Θ = 2b/(a-c) pretty simple
What if all pixels [x,y] lie on a circle?
What if they are all on a line?
Stockman CSE803 Fall 2009
53
As in text with centroid
Stockman CSE803 Fall 2009
54
formula for best axis



use least squares to derive the tangent
angle q of the axis of least inertia
express tan 2q in terms of the 3
second moments
interpret the formula for a circle of
pixels and a straight line of pixels
Stockman CSE803 Fall 2009
55