Transcript GENERAL IMAGE RETRIEVAL USING SHAPE AND COMBINED …

1
General Image Retrieval Using
Shape and Combined Features
Dengsheng Zhang and Guojun Lu
Gippsland School of Computing and
Information Technology
Monash University, Australia
2
Outline
• CBIR—Content-based Image Retrieval
• Shape Feature—Generic Fourier
Descriptor
• Texture Feature—Gabor Filter
• Semi-automatic Segmentation and
Indexing
• Experimental Results
• Conclusions and Future Work
3
Content-based Image Retrieval
• General CBIR Problem: given a query image,
find similar images from database.
• General Methodology of CBIR:
– Finding effective features to represent images
– Index the images in DB with the extracted features
using certain type of data structure
– Matching the query submitted by user with images
in DB using certain type of distance measure and
Interface
• Our focus: finding effective perceptual features
4
Content-based Image Retrieval
• Represent images with content features
– Color: RGB, HSV, LUV
– Shape: moments, Fourier descriptors, scale
space method
– Texture: statistic method, fractal method,
spectral method
• Represent images with combined
features
– Combine several content features to
represent images more effectively than
individual features
5
Generic Fourier Descriptor (GFD)
• Basically, GFD is acquired from 2D Fourier
transform on a polar-raster sampled shape
image.
F (  ,  )  
r
i
r
2i
f (r ,  i ) exp[ j 2 (  
 )]
R
T
6
Generic Fourier Descriptor (GFD)
• After applying the 2D FT on the polar-raster
sampled image, a set of transformed coefficients
are obtained, which is used as the feature vector
PF (0,0) PF (0,1) PF (0, n) PF (m,0) PF (m, n)
GFD  {
,
,...
,...
,...
}
area PF (0,0) PF (0,0)
PF (0,0)
PF (0,0)
where m is the maximum number of the radial frequencies
selected and n is the maximum number of angular
frequencies selected.
7
Gabor Filter
• Basically, Gabor filters are a group of wavelets, with
each wavelet capturing energy at a specific frequency
and a specific direction.
• For a given image I(x, y) with size PQ, its discrete
Gabor wavelet transform is given by a convolution:
K
K
*
Gmn ( x, y)   I ( x  s, y  t ) g mn
( s, t )
s 0 t 0
where
1 x2 y2
g ( x, y) 
exp[ ( 2  2 )] exp( j 2Wx )
2 x y
2 x y
Extracted Energy: E (m, n) 
1
P
Q
 | G
x 0 y 0
mn
( x, y) |, m  0,1,, M - 1; n  0,1,, N - 1
Extracted feature: f = (00 , 00 , 01 , 01 , …, 45, 45)
8
Semi-automatic Segmentation and Indexing
9
Experiments
• A database of 1,000 images from over 40 varieties is
created from a classified collection of 10,000 natural
pictures. The types of images include land animals,
marine animals, flying animals, buildings, aircrafts,
flowers and other real world objects.
• Each image is segmented and indexed using the semiautomatic segmentation tool. For each image, up to 5
objects are allowed to index the image. The similarity
between the query image and the target image is
measured by the similarity between the two most similar
objects in the two images.
10
Performance Measurement
• Precision and Recall
r
numberof relevantretrievedimages
R 
n1 totalnumberof relevantimagesin DB
r numberof relevantretrievedimages
P

n2
numberof retrievedimages
11
Distance Measurement
• The similarity between two images are
measured by the city block distance
between the two feature vectors of the
images.
• For the combined retrieval, assuming the
rank of an image using shape retrieval is
r1 and the rank of the image using texture
retrieval is r2, then the rank of the image
using combined retrieval is given by
(r1+r2)/2.
12
Performance Results
100
GFD
90
Texture
Precision
80
Shape&Texture
70
60
50
40
30
20
10
0
0
10
20
30
40
Recall
50
60
70
80
13
Retrieval Using Shape
14
Retrieval Using Combined Features
Using shape
Using combined
15
Conclusions
• A general image retrieval technique using shape and combined
features has been presented.
• A semi-automatic object segmentation and indexing method has
been presented.
• On average, the retrieval effectiveness of shape is comparable with
that of texture.
• Retrieval using combined shape and texture features is more
powerful than retrieval using individual features.
• Combined features should be an option rather than replacement of
individual features.
• In the future, we plan to segment image automatically into
homogenous texture regions using split and merging technique, so
that indexing of images using shape and texture features can be
done automatically.