Transcript Image Segmentation Techniques

Image Segmentation Techniques
Advanced Multimedia Databases
Jeff Strickrott
Content
• Introduction
• Image segmentation methods
– Template
– Transformation based
– Probabilistic modeling
• Summary
7/20/2015
Image Segmentation
2
Introduction
• What is the goal behind building multimedia
databases?
– We want to ask questions based on content.
• Of the form: what images show the red car in front of
the house?
• Why do we need to segment image?
– To represent content! Content Represented by
objects or spatial regions in the image.
– To support region/content based queries.
7/20/2015
Image Segmentation
3
Template Based Segmentation
• A template is a generalization of the features of
p
p 
p
some object.


11
12
 p21
 p31
p22
p32
13
p23 
p33 
– Spatially distributed.
– Features can be pixel: intensity, color, relations between
regions, etc.
– Detect object through correlation (match object to
template).
– Detect objects the same size as template.
• Requires domain knowledge (to create template).
• Result: Provides semantic segmentation of image.
7/20/2015
Image Segmentation
4
Face Templates Examples
Liu’s Intensity Based Template [Liu 00]
Scasselati Ratio-Metric
Template [Scaz 98]
Two templates for real-time face detection: Average intensity &
Ratio-metric
7/20/2015
Image Segmentation
5
Using Templates
• Algorithm (Naive):
–
–
–
–
Select region of image.
Slide template over region, moving “i” pixels at a time.
Compute correspondence for each location.
Locations with high correspondence are locations of
objects.
– Resize (Downsample) to find faces of different sizes.
– Repeat
• Dynamic programming can be used to make
search more efficient (Liu 00).
7/20/2015
Image Segmentation
6
Results Liu’s & Wang’s Face Temp.
[Liu 00]
Video Scene Segmentation: on new face start new scene.
7/20/2015
Image Segmentation
7
Transformation Based Segmentation:
Image Mapping
• Map Image from intensity space to new
feature space.
• Reason: Easier to detect relevant
information about objects in the image.
• Transform Examples: Fourier Transform,
DCT, Wavelet.
7/20/2015
Image Segmentation
8
Wavelet Image Representation
• Maps 2D image into varying spatial and
frequency resolution space.
• Haar Basis function encodes relationship
between neighboring pixel intensities
(edges).
• O(n), n = the number of pixels.
• Linear transform
(invertible/lossless).
7/20/2015
Image Segmentation
9
Poggio’s People Detection System
• Generate Templates of Objects in Domain from
over-complete Haar wavelet.
– Averaged over transformed images of objects to get
relevant features (multiple views).
– Spatial correspondence between transform coefficients
and object location.
• Template encodes relationship between regions in
image (absolute or ratio of intensities).
• Encode spatial relationship.
7/20/2015
Image Segmentation
10
Standard Haar Wavelet
L = Lowpass
H = Highpass
7/20/2015
Image Segmentation
XYz
X Row
Y Column
Z level
11
People Templates
• Each square one wavelet
• Gray: random patterns
• Black: Relevant edge
information
[Poggio 00]
7/20/2015
Image Segmentation
12
Detection System
• Algorithm:
– Image transformed.
– Template matching
done in wavelet space.
• Shift of wavelet
coefficient = 4 pixels in
image.
– Image must be resized
to find objects not the
size of template.
• Proven also on faces
and cars.
7/20/2015
Real time people detection in
complex scene.
[Poggio 00]
Image Segmentation
13
Probabilistic Modeling
• Generic Model, No Segmentation. We know
nothing about image domain.
• Need generic criteria that captures local and
global information.
– Should perform well on texture and high color
images.
– Should work on sections of images.
– No assumptions about similarity metric or
dimension of feature space.
7/20/2015
Image Segmentation
14
Retrieval as Classification Problem
• Model content of images as class(es) in some set
of classes.
• Bayesian classifier used to minimize the
misclassification of image content.
• Solve problem:
– g*(X) = argmaxi P(X | Y=i) P(Y =i).
– X can be an image, subset of image, other types of data
(text, audio, etc.).
• Will work on any type of features used to
represent a class.
7/20/2015
Image Segmentation
15
Embedded Mixture Model
• Vasconcelos showed feature representation
choice is major factor in minimizing
classification error.
• Proposed solution to Bayesian min problem:
– DCT image in blocks (separate or overlapping)
– Model as Mixture of Gaussian distributions,
this is a weighted sum of Gaussian probability
densities.
1
P( X | Y  i)   w
e
(2 ) 
1
( X   )T  1 ( X   )
2
C
c 1
7/20/2015
i
Image Segmentation
n
16
EMM Performance
• Can model content from images with complex color and
texture features (outperforms histogram based and texture
based models for retrieval).
• Works on compressed images.
• Supports region based queries.
• Classification algorithm runs in O(C2) time. C on the order
of 8-16.
• Prior probability term P(Y =i) can be used for learning via
relevance feedback.
• Expectation Maximization algorithm used for learning
weight, mean and covariance parameters (w, ,S). More
work must be done to make this efficient.
7/20/2015
Image Segmentation
17
Region Based Retrieval
• Content based
Retrieval on a
Region of an
image.
• Database of
mixed objects
(in color).
• Results of
query on Onion
[Vasc 00]
7/20/2015
Image Segmentation
18
Summary
• We have investigated three techniques for
segmenting images that require varying
amounts of domain knowledge.
• Closest similarity with segmentation
methods discussed in class are the
clustering and stochastic modeling methods.
• Two methods work in real-time while fourth
designed for generic domains and offline
classification.
7/20/2015
Image Segmentation
19
References
•
[Liu 00] Liu, Z. and Wang, Y., "Face Detection and Tracking in Video Using Dynamic
Programming," ICIP-2000, Vancouver, Canada.
•
[Poggio 00] Papageorgiou, C. and Poggio, T., A Trainable System for Object Detection
International journal of computer vision, vol. 38, pp. 15-35, 2000.
•
[Scaz 98] Scassellati, B., "Eye Finding via Face Detection for a Foveated, Active Vision
System," AAAI 98
•
[Stau 99] Stauffer, C. and Grimson, W. E. L., "Adaptive Background Mixture Models
for Real-Time Tracking," IEEE Computer Society Conference on Computer Vision and
Pattern Recognition, pp. 246, 1999.
•
[Vasc 00]Vasconcelos, N. and Lippman, A., "A Probabilistic Architecture for Contentbased Image Retrieval," IEEE Conference on Computer Vision and Pattern Recognition
(ICPR 2000), Hilton Head Island, SC, 2000.
7/20/2015
Image Segmentation
20
Segmentation Methods
• Color Histogram
– segment in histogram space
– no domain knowledge
• Split and Merge
– Segment image into regions ()
• Region Growing
– Grow regions around seed point based on similarity
measure.
– Similarity criteria
• Clustering
• Stochastic
7/20/2015
Image Segmentation
21