Transcript CAP5415-Lecture 2

Region Segmentation
Region Segmentation

Find sets of pixels, R1 , R2 ,, Rn such that

R
i
I
i
 i  j, Ri  R j  

All pixels in region i satisfy some constraint
of similarity.
K-Means



Choose a fixed number of
clusters
Choose cluster centers
and point-cluster
allocations to minimize
error
can’t do this by search,
because there are too
many possible
allocations.

Algorithm



fix cluster centers;
allocate points to closest
cluster
fix allocation; compute
best cluster centers
x could be any set of
features for which we can
compute a distance
(careful about scaling)

2 
x j   i 



iclusters jelements of i' th cluster

K-Means
Image Segmentation by K-Means






Select a value of K
Select a feature vector for every pixel (color, texture,
position, or combination of these etc.)
Define a similarity measure between feature vectors
(Usually Euclidean Distance).
Apply K-Means Algorithm.
Apply Connected Components Algorithm.
Merge any components of size less than some threshold to
an adjacent component that is most similar to it.
Image
Clusters on intensity
Clusters on color
K-means clustering using intensity alone and color alone
Image
Clusters on color
K-means using color alone, 11 segments
K-means using
color alone,
11 segments.
K-means using colour and
position, 20 segments
How to find K



Use prior knowledge about image.
Apply the algorithm for different values of
K and test for goodness of clusters.
Analyze Image Histograms.
How to find K
How to find K
How to Find K
Realistic Histograms
How to Find K
Smooth Histogram. (Convolve by averaging
or Gaussian Filter)
How to Find K
Find Peaks and Valleys and perform peakiness test.
Agglomerative clustering



Assume that each cluster is single pixel (i.e.
every pixel is a cluster itself).
Merge Clusters i.e. attach closest to cluster it
is closest to (if possible)
Repeat step 2 until no more clusters can be
merged.
Divisive clustering



Assume that whole image is a single cluster.
Split Clusters along best boundary (if exists)
Repeat step 2 until no more clusters can be
split.
Inter-Cluster distance



single-link clustering: Minimum distance
between an element of the first cluster and one of
the second..
complete-link clustering: Maximum distance
between an element of the first cluster and one of
the second.
group-average clustering: Average of distances
between elements in the clusters.
Segmentation by Split and Merge


Start with an initial segmentation (for example by KMeans).
Define a criteria P for goodness of region such that





P( R )=True, if R satisfies the criteria
P( R )=False, otherwise
For each region R, split R in four regions (quadrants), if
P( R ) = False
Merge any two adjacent regions R and Q
if PQ  R  T rue
Repeat until no more clusters can be split or merged.
Segmentation by Split and Merge
Graph Theoretical Techniques for
Image Segmentation
Graph

A graph G(V,E) is a triple consisting of a
vertex set V(G) an edge set E(G) and a
relation that associates with each edge two
vertices called its end points.
Path

A path is a sequence of edges e1, e2, e3, …
en. Such that each (for each i>2 & i<n)
edge ei is adjacent to e(i+1) and e(i-1). e1
is only adjacent to e2 and en is only
adjacent to e(n-1)
Connected & Disconnected Graph

A graph G is connected if there
is a path from every vertex to
every other vertex in G.

A graph G that is not
connected is called
disconnected graph.
Graphs Representations
a
b
c
e
d
0
1

0

0
1
1 0 0 1
0 0 0 0
0 0 0 1

0 0 0 1
0 1 1 0
Adjacency Matrix: W
Weighted Graphs and Their
Representations
a
b
c
e
6
d
0
1

3



1 3  
0 4  2 
4 0 6 7

 6 0 1
2 7 1 0 
Weight Matrix: W
Minimum Cut
A cut of a graph G is the set of
edges S such that removal of
S from G disconnects G.
Minimum cut is the cut of
minimum weight, where
weight of cut <A,B> is given
as
w A, B    xA, yB wx, y 
Minimum Cut and Clustering
Image Segmentation & Minimum Cut
Pixel
Neighborhood
Image
Pixels
w
Similarity
Measure
Minimum
Cut
Minimum Cut
1H.

There can be more than one minimum cut in a
given graph

All minimum cuts of a graph can be found in
polynomial time1.
Nagamochi, K. Nishimura and T. Ibaraki, “Computing all small cuts in an
undirected network. SIAM J. Discrete Math. 10 (1997) 469-481.
Drawbacks of Minimum Cut

Weight of cut is directly proportional to the
number of edges in the cut.
Cuts with
lesser weight
than the
ideal cut
Ideal Cut
Normalized Cuts1

Normalized cut is defined as
N cut  A, B  


1J.

w A, B
xA, yV

wx, y 


w A, B
zB , yV

wz, y 
Ncut(A,B) is the measure of dissimilarity of sets A
and B.
Minimizing Ncut(A,B) maximizes a measure of
similarity within the sets A and B
Shi and J. Malik, “Normalized Cuts & Image Segmentation,” IEEE Trans. of PAMI,
Aug 2000.
Finding Minimum Normalized-Cut


Finding the Minimum Normalized-Cut is
NP-Hard.
Polynomial Approximations are generally
used for segmentation
Finding Minimum Normalized-Cut
1
Image
Pixels
2
Pixel
Neighborhood
3
w
Similarity
Measure
n-1
n
Finding Minimum Normalized-Cut
W  N  N symmetricmatrix,where
e  Fi  F j
W i, j   

 F2
e
0
 X i  X j  X2
if j  N i 
otherwise
Fi  Fj  Imagefeaturesimilarity
X i  X j  Spatial Proximit y
D  N  N diagonalmatrix,where Di, i   W i, j 
j
Finding Minimum Normalized-Cut
y T D  Wy
 It can be shown that min N cut  miny
y T Dy
such that yi 1,b, 0  b  1, and yT D1  0

If y is allowed to take real values then the
minimization can be done by solving the
generalized eigenvalue system
D  Wy  Dy
Algorithm




Compute matrices W & D
Solve D  Wy  Dy for eigen vectors with the
smallest eigen values
Use the eigen vector with second smallest eigen
value to bipartition the graph
Recursively partition the segmented parts if
necessary.
Figure from “Image and video segmentation: the normalised cut framework”,
by Shi and Malik, 1998
F igure from “Normalized cuts and image segmentation,” Shi and Malik, 2000
Drawbacks of Minimum Normalized Cut



Huge Storage Requirement and time
complexity
Bias towards partitioning into equal
segments
Have problems with textured backgrounds
Suggested Reading



Chapter 14, David A. Forsyth and Jean Ponce, “Computer
Vision: A Modern Approach”.
Jianbo Shi, Jitendra Malik, “Normalized Cuts and Image
Segmentation,” IEEE Transactions on Pattern Analysis
and Machine Intelligence, 1997
Chapter 3, Mubarak Shah, “Fundamentals of Computer
Vision”