Transcript Nat

3D Shape Retrieval from
Silhouette Images
Nat Duca
Overview (1)
Input:
Edges:
Output:
Overview (2)

Are silhouettes good descriptions of shape?
–
–


One may not be… but a few are
Can you see 3D structure in the shadow of a hand in the
projector?
Silhouettes are easily entered queries… draw the
contour of your shape
Also, they can be derived from an arbitrary sketch or
image… or shadows, come to think of it
Approach Overview

Sea of cameras: produce multiple views of an object

A silhouette contour of a closed object is a circular
function: can be made rotation-invariant

Multiple silhouettes can be used to improve the
accuracy of a query (i.e. get me the object that has
these two silhouettes)
Sea of cameras

The “sea” of cameras can be sampled over a
hemisphere because silhouettes will be the same
(mod flip)

Regular distribution on a sphere over samples…
Ideal case: random distribution of K camera points
over a hemisphere that changes for each model


My version: custom placed locations of 20 points on
a hemisphere that is constant across all shapes
Contour extraction (1)

Most practical shapes will have:
–
–
Sampling artifacts in the silhouette image
Interior contours (i.e. no one dominant contour)
silhouette
edges
Contour extraction (2)


Perform morphological closing of silhouette
image with a disk structuring element
Use a laplacian-of-gaussian edge detector to
extract contour pixels
Contour extraction (3)




Using a maze walking algorithm, find a closed loop
through the contour pixels to produce a single closed
group of pixels
This is done for all connected components
I bet we could use all of the components to do partial
matching on this shape
… but for now, we chose the largest contour that
results
Silhouette Parameterization


Goals: rotation, scale, translation invariance
RDF (radial distance function)
–
–

f(c) = r where r is maximal object extent in
direction c from centroid
As in spherical harmonics
Arc length parameterization:
–
–
f(c) = r + k(s) i parameterization about centroid
Where k(s) is the stretch required to fit the curve
to the surface
Matching Metric

Let dist(s1, s2) be the sum of squared distances
between the descriptor for two contours (“samples”)

Option 1: best sample from query and model
–
Let dist(mi,q1…K) for some model mi and K samples q1…j be
defined as


The sample sd in model mi that is closest to any qi
Option 2: all samples in query, best in model

Sum over all k of qi
–
The sample sd in model mi that is closest to qi
System Details (1)

I start with Princeton Shape Benchmark geometry
and classifications

Silhouette sea is generated by a glut app in Linux
using depth buffer readback
Images are copied to a machine with Matlab where
morphological closing and edge detection is done


… and then copied back… grr…

Useful stat: 622 models yields 1.5 GB of images, but
only a 7.5mb database
System Details (2)

Edge images are processed by a maze search algorithm to find
the largest closed contour

Contours are sampled evenly to 200 points using Gnu Scientific
Library

The parameterization is FFT’d with FFTw

The resulting descriptors are stored in a binary database

We can set an epsilon value that allows the same sample to be
appear in multiple models if it is similar enough (compression!)

Precision vs recall is generated internally by the database
Control tests: rotation invariance
Match is somewhat sensitive to noise… e.g. higher frequency
noise introduced by the sampling of the contours
Contour invariance with rotation: 400-element descriptor
3.5
3
2.5
2
1.5
1
0.5
Rotation
18
0
17
0
16
0
15
0
14
0
13
0
12
0
11
0
90
10
0
80
70
60
50
40
30
20
10
0
0
Measured distance

Control tests: rotation invariance
Tinkering with filtering and other stuff doesn’t
help much… using the RDF does.
Contour Invariance with Rotation
1.6
1.4
1.2
Linear Interp
1
Cubic Interp
0.8
Cubic + gaussian(1s)
0.6
Cubic + gaussian(2s)
RDF + Linear
0.4
0.2
95
11
0
12
5
14
0
15
5
17
0
65
80
35
50
20
0
5
Difference from Base Contour

Rotation Amount
Control tests: Scale invariance
Contour Invariance with Scale
Distance from base contour
1.6
1.4
1.2
Linear Interp
1
Cubic Interp
0.8
Cubic + gaussian(1s)
0.6
Cubic + gaussian(2s)
0.4
RDF + Linear
0.2
0
25
0.
4
5
0. 0.5
7
5
0. 0.8
1
15
1.
Scale Factor
3
5
1. 1.4
6
5
1. 1.7
Circular parameterization and
Model match option 1
20 classes shown
57 total classes
622 models
Circular parameterization and
Model match option 1
20 classes shown
57 total classes
622 models
Circular parameterization and
Model match option 1
20 classes shown
57 total classes
622 models
Radial distance parameterization and
Model match option 1
1 silhouette used in query
15 classes shown
57 total classes
622 models
Radial distance parameterization and
Model match option 1
2 silhouettes used in query
15 classes shown
57 total classes
622 models
Radial distance parameterization and
Model match option 1
8 silhouettes used in query
15 classes shown
57 total classes
622 models
Radial distance parameterization and
Model match option 2
1 silhouette used in query
15 classes shown
57 total classes
622 models
Radial distance parameterization and
Model match option 2
4 silhouettes used in query
15 classes shown
57 total classes
622 models
Radial distance parameterization and
Model match option 2
4 silhouettes used in query
15 classes shown
57 total classes
622 models
Radial distance parameterization and
Model match option 2
8 silhouettes used in query
15 classes shown
57 total classes
622 models
Radial distance parameterization and
Model match option 2
8 silhouettes used in query,
57 classes shown
57 total classes
622 models