Transcript Interactive Systems Laboratories

Computer Vision:
Chamfer System
Dr. Edgar Seemann
[email protected]
1
Interactive Systems Laboratories, Universität Karlsruhe (TH)
cv:hci
Research Group, Universität Karlsruhe (TH)
Computer Vision for Human-Computer Interaction
Silhouette Matching
Dr. Edgar Seemann
2
Research Group, Universität Karlsruhe (TH)
cv:hci
Computer Vision for Human-Computer Interaction
Chamfer Matching [Gavrila & Philomin ICCV’99]
Goal
•
Align known object shapes with image
Object shapes
Real-world image of object
Requirements for an alignment algorithm
•
•
•
•
High detection rate
Few false positives
Robustness
Computationally inexpensive
Dr. Edgar Seemann
3
Research Group, Universität Karlsruhe (TH)
cv:hci
Computer Vision for Human-Computer Interaction
Distance Transform

Used to compare/align two (typically binary) shapes
Shape 1
Shape 2
Distance = ?
1.
Compute for each pixel the distance to the next edge pixel

Here the eculidean distances are
approximated by the 2-3 distance
Dr. Edgar Seemann
Distance transform
8
6
5
3
2
2
3
5
6
5
3
2
0
0
2
4
5
3
2
0
2
0
2
4
4
2
0
2
2
0
2
4
4
2
0
2
2
0
2
4
5
3
2
0
2
0
2
4
6
5
3
2
0
0
2
4
8
6
5
3
2
2
3
5
4
Research Group, Universität Karlsruhe (TH)
cv:hci
Computer Vision for Human-Computer Interaction
Distance Transform
2.
Overlay second shape over distance transform
Distance transform
8
6
5
3
2
2
3
5
6
5
3
2
0
0
2
4
5
3
2
0
2
0
2
4
4
2
0
2
2
0
2
4
4
2
0
2
2
0
2
4
5
3
2
0
2
0
2
4
6
5
3
2
0
0
2
4
8
6
5
3
2
2
3
5
Distance = 14
3.
4.
Accumulate distances along shape 2
Find best matching position by an exhaustive search


Distance is not symmetric
Distance has to be normalized w.r.t. the length of the
shapes
Dr. Edgar Seemann
5
Research Group, Universität Karlsruhe (TH)
Binary image
Distance transform
8
6
5
3
2
2
3
5
6
5
3
2
0
0
2
4
5
3
2
0
2
0
2
4
4
2
0
2
2
0
2
4
4
2
0
2
2
0
2
4
5
3
2
0
2
0
2
4
6
5
3
2
0
0
2
4
8
6
5
3
2
2
3
5
Distance transform of a real-world image
Chamfer Matching
• Compute distance transform (DT)
• For each possible object location
• Position known object shape over DT
• Accumulate distances along the
contour
Distance measure
cv:hci
Computer Vision for Human-Computer Interaction
Chamfer Matching
Dr. Edgar Seemann
6
Research Group, Universität Karlsruhe (TH)
cv:hci
Computer Vision for Human-Computer Interaction
Efficient implementation
 The distance transform can be efficiently computed by two
scans over the complete image
 Forward-Scan


Starts in the upper-left corner and moves from left to right, top to
bottom
3 2 3
Uses the following mask
2
0
 Backward-Scan


Starts in the lower-right corner and moves from right to left,
bottom to top
Uses the following mask
0 2
3
Dr. Edgar Seemann
2
3
7
Research Group, Universität Karlsruhe (TH)
cv:hci
Computer Vision for Human-Computer Interaction
Forward scan
 We can choose different values for the filter mask
 The local distances, d, s and c, in the mask are
added to the pixel values of the distance map and
the new value of the zero pixel is the minimum of
the five sums
 Example:
3
2
3
5
3 2+2 2+3 3+5
2
0
?
?
2 2+0 2
?
?
?
?
?
?
?
Dr. Edgar Seemann
?
?
d
s
s
c
d
8
Research Group, Universität Karlsruhe (TH)
cv:hci
Computer Vision for Human-Computer Interaction
Advantages and Disadvantages
 Fast


Distance transform has to be computed only once
Comparison for each shape location is cheap
 Good performance on uncluttered images (with
few background structures)
 Bad performance for cluttered images
 Needs a huge number of people silhouettes

But computation effort increases with the number of
silhouettes
Dr. Edgar Seemann
9
Research Group, Universität Karlsruhe (TH)
 To reduce the number of silhouettes to consider, silhouettes
can be organized in a template hierarchy
 For this, the shapes are clustered by similarity
cv:hci
Computer Vision for Human-Computer Interaction
Template Hierachy
Dr. Edgar Seemann
10
Research Group, Universität Karlsruhe (TH)
cv:hci
Computer Vision for Human-Computer Interaction
Search in the hierarchy
 Matching the shapes, then corresponds to a
traversal of the template hierarchy
 How can we prune search branches to speed up
matching?

Thresholds depend on:





Edge detector (likelihood of gaps)
Silhouette sizes
Hierarchy level
Allowed shape variation
Thresholds are set statistically
during training
Dr. Edgar Seemann
11
cv:hci
Research Group, Universität Karlsruhe (TH)
Computer Vision for Human-Computer Interaction
Example Detections
Dr. Edgar Seemann
12
cv:hci
Research Group, Universität Karlsruhe (TH)
Computer Vision for Human-Computer Interaction
Video
Dr. Edgar Seemann
13
Research Group, Universität Karlsruhe (TH)
cv:hci
Computer Vision for Human-Computer Interaction
Coarse-To-Fine Search
 Goal: Reduce search effort by discarding unlikely
regions with minimal computation
 Idea:


Subsample image and search
first at a coarse scale
Only consider regions with a
low distance when searching
for a match on finer scales
 Again, we have to find
reasonable thresholds
Dr. Edgar Seemann
Level 1
Level 2
Level 3
14
cv:hci
Research Group, Universität Karlsruhe (TH)
Computer Vision for Human-Computer Interaction
Protector System (Daimler)
Dr. Edgar Seemann
15
Research Group, Universität Karlsruhe (TH)
 So far edge orientation has been completely
ignored
Distance = small
 Idea: Consider edge orientation for each pixel
cv:hci
Computer Vision for Human-Computer Interaction
Adding edge orientation
Dr. Edgar Seemann
16
Research Group, Universität Karlsruhe (TH)
cv:hci
Computer Vision for Human-Computer Interaction
Edge orientation - The math
 Given two shapes S, C, we can express the chamfer
distance in the following manner
 The orientation correspondence between two points is then
measured by
 The combined distance measure:
Dr. Edgar Seemann
17
Research Group, Universität Karlsruhe (TH)
 Adding statistical relevance of silhouette regions
further improves the results [Dimitrijevic06]
cv:hci
Computer Vision for Human-Computer Interaction
Statistical Relevance
Dr. Edgar Seemann
18
Research Group, Universität Karlsruhe (TH)
 Use multiple successive frames to build a spatiotemporal template (T={T1,…,TN})
 Allow spatial variations of dx, dy (due to motion
or camera movements)
cv:hci
Computer Vision for Human-Computer Interaction
Spatio-Temporal templates
Dr. Edgar Seemann
19
cv:hci
Research Group, Universität Karlsruhe (TH)
Computer Vision for Human-Computer Interaction
Example: single-frame vs. 3 frames
Dr. Edgar Seemann
20
Research Group, Universität Karlsruhe (TH)
cv:hci
Computer Vision for Human-Computer Interaction
Quantitative Results
 Red: spatio-temporal templates + statistical
relevance
Dr. Edgar Seemann
21
Research Group, Universität Karlsruhe (TH)
 Restrict detection to a single articulation (when legs are in a
v-shaped position)
 Spatio-Temporal templates:


Allows more reliable detection of motion direction
Avoids confusions and some false positive detections
cv:hci
Computer Vision for Human-Computer Interaction
Video
Dr. Edgar Seemann
22
Research Group, Universität Karlsruhe (TH)
 Originally developed to compare histograms
 Idea: Find the minimal ‘flow’ to transform one
histogram to another
 Example:
cv:hci
Computer Vision for Human-Computer Interaction
Alternatives: Earth Mover’s Distance
Dr. Edgar Seemann
23
Research Group, Universität Karlsruhe (TH)
Chamfer:
EMD Matching
EMD:
• Detect edges in image
• For each possible object location
• Optimize correspondences between
known shape and edge image
Example detection
Distance measure basis
cv:hci
Computer Vision for Human-Computer Interaction
Earth mover’s distance (EMD)
Dr. Edgar Seemann
24
Research Group, Universität Karlsruhe (TH)
cv:hci
Computer Vision for Human-Computer Interaction
EMD – The math
 Variant of the transportation problem (possible solutions: Stepping
Stone Algorithm, Transportation-simplex method)
 Constraints
 EMD-Distance
Dr. Edgar Seemann
25
Research Group, Universität Karlsruhe (TH)
cv:hci
Computer Vision for Human-Computer Interaction
Advantages and Disadvantages
 Optimizes matching between silhouette and edge
structure in image
 Enforces one-to-one matchings (unlike chamfer)
 Allows partial matches
 Can deal with arbitrary features
 High computational complexity

Approximation is possible [Graumann, Darrel CVPR’94]
Dr. Edgar Seemann
26