Transcript [PPT file - 2 MB]

Recovering Articulated Object Models
from 3D Range Data
Dragomir Anguelov
Daphne Koller
Hoi-Cheung Pang Praveen Srinivasan
Sebastian Thrun
Computer Science Dept.
Stanford University
Problem overview

Data:



Task: Recover the articulated
object model
1.
2.
3.

Articulated object observed in
different configurations
3D range surfaces (meshes)
obtained by a scanner
Register all surfaces
Cluster the object surface into
rigid parts
Recover joints linking the parts
Applications of articulated
models

Animation, object detection,
scene segmentation
Surface registration

Algorithm input: meshes Z0, Z1, … ZN




3
Correlated Correspondences algorithm [Anguelov et al ‘04]



Pick template mesh X = Z0
Need to recover point-to-point correspondences
between X and instance meshes Zi , i={1, …, N}
Any registration algorithm providing can be used
2
1
Unsupervised nonrigid registration algorithm
Handles large deformations, occlusion (but not clutter)
Algorithm Overview



Computes embedding of mesh Z into mesh X
defined as a joint assignment to correspondences
Encodes correspondence correlations (minimizing
deformation, preserving geodesic distances) as pairwise
potentials
in a Markov network over
Uses Loopy-BP to obtain a good joint assignment
to all variables
Clustering into rigid parts:
Generative model
1. Rigid part points share the same
rigid transformaition in each instance
Part labels a1
aM
…
Transformations
T
Template X
Points
…
x1
Transformed
Model
Instance Z
xM
y1
Point labels
Points
j = Part (xj)
b1
…
yM
…
translation
2. Each part is a connected region of
the template mesh surface
a) Soft contiguity constraints:
bk
z1
rotation
If
and
are connected in X
zk
k = ModelPoint (zk)
Assume  values provided by a
nonrigid registration algorithm
for
b) Hard contiguity constraints:
Each part consists of a single
connected component of the
surface of X
Clustering into rigid parts: Optimization

Model Summary:

Ignoring hard contiguity constraints, generative model defines a
Markov Network over the part labels :

Objective

Iterative Optimization


Point in mesh Zi , generated
by template point xj
E-step: Given transformations , compute the part labels
maximizing Eq. (1) and satisfying the hard contiguity constraints
M-step: Given part labels , compute transformations
Clustering into rigid parts: E-Step

The problem of finding a MAP assignment to
given
which
maximizes Eq. (1) can be expressed as an integer program (IP). A
relaxation of the IP produces a linear program (LP) [Kleinberg et al ’99].


For 2-part models, the optimal LP solution is also the optimal IP solution.
For n-part models, n>2:



In case of a fractional LP solution, a rounding procedure produces an IP solution
within a factor of 2 of the optimal.
The LP can be solved very efficiently by using multiway min-cuts
[Kolmogorov et al ’04]
The LP solution may not satisfy hard contiguity constraints.

A part consisting of m connected components is split into m separate parts
which doesn’t change the objective in Eq. (1).
Clustering into rigid parts: LP formulation


Introduce variables
Log-cost associated with potential
s.t.
becomes
where

Define separation cost of an edge as

Putting everything together:
Relax binary constraints
to obtain LP
where
Clustering into rigid parts:
M-step and Annealing

M-step: find rigid part transformations

Optimization function decomposes into a separate equation for
each


given part labels
:
Eq. (2) is isomorphic to the ICP problem [Besl’ 92]. Close form
estimate of
is obtained by solving a simple LP.
Annealing



Number of parts in solution depends on the tradeoff
where large values of result in fewer parts
When estimate of
is poor and is large, solving E-step LP is
hard, because multiple plausible solutions with similar scores exist
Starting with low values of and gradually increasing performs
best in practice
Clustering into Rigid Parts: Initialization

Need good initial values for the transformations
Method 1: Cluster local rigid transformations [Cheung et al ’03]
• Associate a vector of transformations
with each point
(estimate
using the known correspondences for all instance meshes)
• Cluster points into k parts using adaptive PCA [Archer et al ’02]
• Slow: >10 mins for 2000 point models
Template X
a)
b)
Method 2: Break surface into k arbitrary fragments of
approximately equal size
• Fast: <1s for 2000 point models
• Same final results!
a) Rigid parts after initialization
b) Rigid parts after EM
Optimizing the Joint Positions

A joint is a point belonging to two parts simultaneously: [Cheung et al ’03]
Joint between parts p and q


To solve for the joint, must optimize
Sometimes, the solution can be an entire
space of points


Example: a joint with one degree of freedom
Introduce an additional regularization term,
enforcing that the joint is close to the boundary
between two parts in X
a)
b)
a) Rigid parts after EM
b) Estimated joints
Results: Puppet Dataset
Correctly identifies all 15 puppet parts and the skeleton
Results: Initialization

Testing EM Initialization


Initialize with a different number of parts k
If k > 15, the algorithm consistently finds the correct number of
parts (15)
Results: Arm Dataset
Correctly identifies 4 arm parts and the skeleton
in the presence of non-rigid deformation
Results: Annealing

Results of annealing with increasing value of


Increasing decreases the number of parts (A – D)
Increasing too much results in a partition which tries to reduce
the number of links between parts (E)
Conclusions and Future Work

Algorithm for recovering articulate object models from 3D data



Algorithm properties




Clusters the object surface into rigid parts
Recovers the articulated object skeleton
Probabilistic formulation
Efficient global inference
Robustness to local maxima
Algorithm recovers complex models consisting of a large
number of parts in real world datasets
Future work

Extend the algorithm to allow and explicitly model (small) deformations
of the articulated parts