Transcript Document

Shape-Representation
and
Shape Similarity
Part 1: Shapes
Dr. Rolf Lakaemper
May I introduce myself…
• Rolf Lakaemper
•
PhD (Doctorate Degree) 2000
Hamburg University, Germany
• Currently Assist. Professor at Department
of Computer and Information Sciences,
Temple University, Philadelphia, USA
•
Main Research Area: Computer Vision
Motivation
WHY SHAPE ?
Motivation
These objects are recognized by…
Motivation
These objects are recognized by…
Texture
Color
X
X
Context
X
Shape
X
X
X
X
X
X
Why Shape ?
Several applications in computer vision use
shape processing:
• Object recognition
• Image retrieval
• Processing of pictorial information
• Video compression (eg. MPEG-7)
…
This presentation focuses on
object recognition and image retrieval.
Motivation
Typical Application: Multimedia: Image Database
Query by Shape / Texture / …(Color / Keyword)
Blobworld
Example 1: Blobworld
http://elib.cs.berkeley.edu/photos/blobworld/start.html
BLOB = “Binary Large Object”,
“an indistinct shapeless (really ?) form”
Blobworld
Blobworld: Query by Shape / Texture / Location / Color
Selected Blob
Query:
by Color and
Texture of
Blob
Result:
Blobs with
similar Color
and Texture
Satisfying ?
Blobworld
Blobworld: Query by Shape / Texture / Location / Color
Selected Blob
Query:
by Shape of
Blob
Result:
…are these
shapes
similar ?
Satisfying ?
Blobworld
Result:
SHAPE recognition seems to
be necessary but not easy !
ISS Database
Example 2: ISS-Database
http://knight.cis.temple.edu/~shape
The Interface (JAVA – Applet)
The Sketchpad: Query by Shape
The First Guess: Different Shape - Classes
Selected shape defines query by shape – class
Result
ISS Database
ISS: Query by Shape / Texture
Sketch of
Shape
Query:
by Shape
only
Result:
Satisfying ?
ISS Database
SHAPE recognition seems to
be possible and leads
to satisfying results !
ISS Database
The ISS-Database will be topic of
part IV of this tutorial
…so stay alert !
Overview
Overview Part 1
•
•
•
•
•
•
•
Why shape ?
What is shape ?
Shape similarity
Metrices
Classes of similarity measures
Feature Based Coding
Examples for global similarity
Why Shape ?
Why Shape ?
•
Shape is probably the most important property
that is perceived about objects. It allows to
predict more facts about an object than other
features, e.g. color (Palmer 1999)
•
Thus, recognizing shape is crucial for object
recognition. In some applications it may be the
only feature present, e.g. logo recognition
Why Shape ?
Shape is not only perceived by visual means:
•
•
tactical sensors can also provide shape
information that are processed in a similar
way.
robots’ range sensor provide shape
information, too.
Shape
Typical problems:
• How to describe shape ?
• What is the matching
transformation?
• No one-to-one correspondence
• Occlusion
• Noise
Shape
• Partial match:
only part of
query appears
in part of
database shape
What is Shape ?
What is Shape ?
Plato, "Meno", 380 BC:
• "figure is the only existing thing that
is found always following color“
• "figure is limit of solid"
What is Shape ?
… let’s start with some properties easier
to agree on:
• Shape describes a spatial region
Shape is a (the ?) specific part of spatial cognition
• Typically addresses 2D space
why ?
What is Shape ?
• 3D => 2D projection
What is Shape ?
• the original 3D (?) object
What is Shape ?
Moving on from the naive
understanding, some questions arise:
• Is there a maximum size for a shape to be a
shape?
• Can a shape have holes?
• Does shape always describe a connected
region?
• How to deal with/represent partial shapes
(occlusion / partial match) ?
What is Shape ?
Shape or Not ?
Continuous transformation from shape to no
shape: Is there a point when it stops being
a shape?
What is Shape ?
Shape or Not ?
Continuous transformation from shape to two
shapes: Is there a point when it stops being
a single shape?
What is Shape ?
But there’s no doubt that
a single, connected region
is a shape.
Right ?
What is Shape ?
A single, connected region.
But a shape ?
A question of scale !
What is Shape ?
• There’s no easy, single definition of shape
•
In difference to geometry, arbitrary shape is not
covered by an axiomatic system
• Different applications in object recognition
focus on different shape related features
•
Special shapes can be handled
• Typically, applications in object recognition
employ a similarity measure to determine a
plausibility that two shapes correspond to
each other
Similarity
So the new question is:
What is Shape Similarity ?
or
How to Define a Similarity Measure
Similarity
Again: it’s not so simple (sorry).
There’s nothing like
THE
similarity measure
Similarity
which similarity measure,
depends on
which required properties,
depends on
which particular matching problem,
depends on
which application
Similarity
…which application
Simple Recognition (yes / no)
... robustness
Common Rating (best of ...)
Analytical Rating (best of, but...)
... invariance to basic transformations
Similarity
…which problem
• computation problem: d(A,B)
• decision problem: d(A,B) <e ?
• decision problem: is there g: d(g(A),B) <e ?
• optimization problem: find g: min d(g(A),B)
Similarity
…which properties:
We concentrate here on the
computational problem d(A,B)
Similarity Measure
Requirements to a similarity measure
• Should not incorporate context knowledge
(no AI), thus computes generic shape
similarity
Similarity Measure
Requirements to a similarity measure
• Must be able to deal with noise
• Must be invariant with respect to basic
transformations
Scaling (or resolution)
Next:
Strategy
Rotation
Rigid / non-rigid deformation
Similarity Measure
Requirements to a similarity measure
• Must be able to deal with noise
• Must be invariant with respect to basic
transformations
• Must be in accord with human perception
Similarity Measure
Some other aspects worth consideration:
• Similarity of structure
• Similarity of area
Can all these aspects be expressed by a
single number?
Similarity Measure
Desired Properties of a Similarity Function C
(Basri et al. 1998)
• C should be a metric
• C should be continous
• C should be invariant (to…)
Properties
Metric Properties
S set of patterns
Metric: d: S  S R satisfying
1. Self-identity: "xS, d(x,x)=0
2. Positivity: "x yS, d(x,y)>0
3. Symmetry: "x, yS, d(x,y)= d(y,x)
4. Triangle inequality: "x, y, zS,
d(x,z)d(x,y)+d(y,z)
• Semi-metric: 1, 2, 3
• Pseudo-metric: 1, 3, 4
• S with fixed metric d is called metric space
Properties
1. Self-identity: "xS, d(x,x)=0
2. Positivity: "x yS, d(x,y)>0
…surely makes sense
Properties
Properties
Properties
In general:
• a similarity measure in accordance with
human perception is NOT a metric. This
leads to deep problems in further
processing, e.g. clustering, since most of
these algorithms need metric spaces !
Properties
Continuity:
“usually useful”, although sometimes not in accordance
with principles of Gestalt properties, e.g. symmetry,
collinearity.
Properties
Properties
Properties
Some more properties:
• One major difference should cause a
greater dissimilarity than some minor ones.
• S must not diverge for curves that are not
smooth (e.g. polygons).
However, these demands are contradictory
(proof is left as an exercise)
Similarity Measures
Classes of Similarity Measures:
Similarity Measure depends on
• Shape Representation
• Boundary
• Area (discrete: = point set)
• Structural (e.g. Skeleton)
• Comparison Model
• feature vector
• direct
Similarity Measures
Boundary
Area (point set)
direct
feature based
Spring model, Cum.
Angular Function,
Chaincode, Arc
Decomposition (ASRAlgorithm)
Central Dist. Fourier
Hausdorff
Moments
…
Zernike Moments
Distance histogram
…
…
Structure
Skeleton
…
---
Feature Based Coding
Feature Based Coding
This category defines all approaches that
determine a feature-vector for a given shape.
Two operations need to be defined: a mapping of
shape into the feature space and a similarity of
feature vectors.
Representation
Feature Extraction
Vector Comparison
Vector Comparison
Vector Comparison
,
Vector Comparison
Vector Comparison
Vector Comparison
Vector Comparison
More Vector Distances:
• Quadratic Form Distance
• Earth Movers Distance
• Proportional Transportation Distance
•…
Vector Comparison
Histogram Comparison
• Vector Comparison
• Histogram Intersection
•…
Vector Comparison
Feature Based Coding
Again:
Two operations need to be defined: a mapping of
shape into the feature space and a similarity of
feature vectors.
We hence have TWO TIMES an information
reduction of the basic representation, which by
itself is already a mapping of the ‘reality’.
Representation
Feature Extraction
Vector Comparison
Vector Comparison
Example 1
Elementary Descriptors
Shape A,B given as
• Area (continous) or
• Point Sets (discrete)
(Elementary Descriptors are 1dimensional feature
vectors)
Vector Comparison
Vector Comparison
Boundary Box (area):
These shapes are equal…
Vector Comparison
Boundary Box (area):
…these shapes differ
Vector Comparison
Example 2
(Discrete) Moments
Shape A,B given as
• Area (continous) or
• Point Sets (discrete)
Moments
Discrete Point Sets
Moments
Moments
Moments
Discrete Moments
Exercise:
Please compute all 7 moments for the following shapes, compare the
vectors using different comparison techniques

Discrete Moments
Result: each shape is transformed to a 7dimensional vector. To compare the shapes,
compare the vectors (how ?).
Vector Comparison
Example 3
Central Distance Fourier
Shape A,B given as
Contour (Boundary)
Vector Comparison
Contour is given as list of euclidean
coordinates:
0,0; 1,0; 2,0; 2,1; 2,2; 3,3; 4,3; 5,2; …
0
1
2
3
4
7 ...
5
6
...
Central Distance Fourier
(MATLAB DEMO)
3D Distance Histogram
Example 4
3D Distance Histogram
Shape A,B given as 3D point set
3D Distance Histogram
(MATLAB DEMO)
Vector Comparison
All Feature Vector approaches have similar properties:
• Provide a compact representation
• this is especially interesting for database indexing !
• Works for any shape
• Requires complete shapes (global comparison)
• Sensible to noise (except Zernike moments which are
computationally demanding)
• Map dissimilar shapes to similar feature vectors (!)
•
They can be used as a prefilter for database applications !
• Make the choice of a similarity function difficult
Direct Comparison
End of Feature Based Coding !
Next:
Direct Comparison
Vector
Direct Comparison
Comparison
Example 1
Hausdorff Distance
Shape A,B given as point sets
A={a1,a2,…}
B={b1,b2,…}
Feature
Vector Based
Comparison
Coding
Hausdorff
Vector Comparison
Distance
Boundary
Vector Comparison
Representation
Hausdorff:
Unstable with respect to noise
(This is easy to fix ! How ?)
Problem: Invariance !
Nevertheless: Hausdorff is the motor behind many applications
in specific fields (e.g. character recognition)
Boundary
Vector Comparison
Representation
Example 2
Chaincode Comparison
Shape A,B given as chaincode
Boundary
Vector Comparison
Representation
Boundary
Vector Comparison
Representation
A binary image can be converted into a ‘chain code’
representing the boundary. The boundary is traversed
and a string representing the curvature is
constructed.
3
2
1
4
C
0
5
6
7
5,6,6,3,3,4,3,2,3,4,5,3,…
Boundary
Vector Comparison
Representation
For curvature classes, a similarity can be defined:
Boundary
Vector Comparison
Representation
To extend this measure to strings, two steps are carried out.
1. Extend the measure to character against string, for
example by summing up individual similarity measures.
2. Employing a matching to compute a correspondence of
sub-strings. Hereby, the matching constitutes from 1-to-1, 1to-many, and many-to-1-matchings. It is computed as string
matching by means of dynamic programming.
Boundary
Vector Comparison
Representation
Digital curves suffer from effects caused by
digitalization, e.g. rotation:
Boundary
Vector Comparison
Representation
Compare chaincodes by string matching
As string-matching is not able to model a matching of digital curves adequately,
more sophisticated matching algorithms are employed in “real applications”
using chain codes:
Weighted Levensthein Distance
Defines an edit distance for transforming one string into another.
Costs are defined for altering, deleting, or inserting a character.
Extended Distance
Formal translation system with costs assigned to individual production rules.
Structural
Vector Comparison
Representation
Example 3
Skeletons
Shape A,B primarily given as area or
boundary, structure is derived from
representation
Structural
Vector Comparison
Representation
Structural approaches capture the
structure of a shape, typically by representing shape as a graph.
Typical example: skeletons
Structural
Vector Comparison
Representation
The computation can be described as
a medial axis transform, a kind of
discrete generalized voronoi.
Structural
Vector Comparison
Representation
The graph is constructed mirroring the
adjacency of the skeleton’s parts. Edges are
labeled according to the qualitative classes.
Matching two shapes requires matching two
usually different graphs against each other.
Structural
Vector Comparison
Representation
Problems of skeletons:
- Pruning
Structural
Vector Comparison
Representation
-Robustness
(MATLAB Demo)
Vector
Shape
Comparison
similarity
All similarity measures shown can not deal with
occlusions or partial matching (except
skeletons ?) !
They are useful (and used) for specific
applications, but are not sufficient to deal with
arbitrary shapes
Solution: Part – based similarity !