Transcript Turning to the Masters: Motion Capturing Cartoons Christopher Bregler, Lorie Loeb, Erika Chuang, Hrishi Deshpande SIGGRAPH ’02

Turning to the Masters:
Motion Capturing Cartoons
Christopher Bregler, Lorie Loeb, Erika
Chuang, Hrishi Deshpande
SIGGRAPH ’02
Motivation
 Animation has two
dimensions:


Visual Style
 Style of Drawing or
Model
 Rendering etc
Motion Style
 How Characters
Move
 Amount of
Exaggeration
 Use of Cartoon
Physics
Examples
Motivation
 Motion Capture
restricted to a very
small region (Green)
 Try to isolate the
motion style of
cartoon animation and
apply it to different
visual styles
Challenges: Capture
 Start with a 2-dimensional animated video




Cartoon characters have no markers
Low Frame Rate makes tracking difficult
Identifying limb locations in cartoon characters
is difficult
Cartoon Objects undergo large degrees of
non-rigid deformation throughout the
sequence
Challenges: Retargeting
 Most retargeting techniques based on
skeletal models
 Can capture only 2D information. Retargeting
domain may be 3D models
 Main Idea: Parameterize cartoon motion with
a combination of affine-transformations and
key-weight vectors.
 Claim: Describe a wide variety of motion and
non-rigid shape deformation.
Affine Deformations
Affine Deformations
 Important part of
cartoon motion comes
from


Velocity of entire body
Stretch and Squash in
different directions
 S is a 3xN shape matrix
encoding N points in
homogenous form,
The ball shape V(t) at
time-frame t is defined
with the equation on left
si=[xi,yi,1]T
Key-Shape Deformations
For more complicated motion use a set of characteristic
key-shapes Si . The model then extends to
Extended Linear Warping Space
 In-Between shapes
produce undesirable
visual artifacts
 Can be avoided by
using a large number of
intermediate shapes
 Can use non-linear
interpolating functions
but then inverse
calculation is non-trivial
Extended Linear Warping Space..
 Solution:
 For every combination of hand-picked key-shapes
generate M in-between key-shapes.
 For K shapes: (K-1)*(K-2)*M shapes now
 Use PCA to give:
 Mean shape M
 Eigen Vectors E1…EL.
 Using Sl=M+El and Sl+1=M gives the extended linear
space
Contour Capture
Given sequence of cartoon contours: V(1)…V(t) and
labeled key-shapes S1….Sk need to find affine matrix
and the weights
Contour Capture
Done by a two-step process.
• Estimate the affine parameters.
• Estimate
the key-weights
Iterate.
Some constraints include all key-shapes adding up to 1.
Reason: Good at interpolation and bad at extrapolation
Video Capture
 Input: Sequence of images instead of contours
 Extend the model to directly model image pixel
variations.
Video Capture: Affine case
 S2xN=[s1,s2…sN] :contains (x,y) coordinates of all
pixels in cartoon image region
 I(si): intensity at pixel si
 Io: Image Template (Base Key-Shape)
Approximation
Video Capture: Affine and Key-Shape
 Io: Was the image template (Base Key-Shape) in
affine case
 Replace it with linear combination of key-shapes.
 The error function thus becomes:
Results from Video-Capture
Results (contd..)
Retargeting Motion
 Key-Shape based
 Correspondence also specified between
control points etc:.
Results
 Video
Pros and Cons
 Advantages
 Attempts the problem of cartoon capture
 Motion Style that is expressive is taken into
consideration
 Adopts simple models and hence can be fast
 Disadvantages
 Adopts very simple models.
 Input Key-Shapes and Correspondences need to be
provided.
 Adopts interpolation
 Retargeting to 3D not handled completely
 Most Importantly: No temporal constraint used.
Hence jitter would be a major problem.