Transcript Document 7594803

Advanced Programming for 3D
Applications
CE00383-3
Motion Capture
Lecture 9
Bob Hobbs
Staffordshire university
Definition of Motion Capture
Motion capture is the recording of
human body movement (or other
movement) for immediate or delayed
analysis and playback. “MoCap”
2
History
• Late 1970’s – Rebecca Allen created the first
primitive form of motion capture.
“Rotoscoping”
• 1980-1983- Tom Calvert invented
“Geniometers”
• 1982-1983- Ginsberg & Maxwell invented
“Graphical Marionette”
• 1988- deGraf/Wahrman invented “Mike the
Talking Head”
3
• 1988- Pacific Data Images invented “Waldo
C. Graphic”
• 1989- Kleiser- Walczak invented “Dozo”
• 1991- Videosystem invented “Mat the Ghost”
• 1992- SimGraphics invented “Mario”
• 1992- Brad deGraph invented “Alive”
• 1993- Acclaim invented two character
animation motion capture
4
Applications
• Entertainment
• Medicine
• Arts / Education
• Science / Engineering
5
Entertainment: Live Action Films
• Computer generated characters in live action
films (e.g. Battle Droids and many others in
Star Wars Prequels, Gullum in The Lord of
the Rings, King Kong in King Kong)
6
Entertainment: 3D computer
animations
• Characters in computer animated files (e.g.
Polar Express, Monster House)
7
Entertainment: Video Games
• Video games by Electronic Arts, Gremlin,
id, RARE, Square, Konami, Namco, and
others, (e.g. Enemy Territory)
8
Medicine
• Medicine (e.g., gait analysis, rehabilitation)
• Sports medicine (e.g. injury prevention,
performance analyses, performance
enhancement)
Gait Analysis Service
9
Arts / Education
• Dance and theatrical performances
• Archiving (e.g., Marcel Marceau)
OSU/ACCAD
10
Science / Engineering
• Computer Science (e.g., human motion
•
•
•
11
database, recognitions)
Engineering (e.g., Biped robot developments)
Ergonomic product design
Military (e.g., field exercises, virtual
instructors, and role-playing games)
Types of mocap equipment
• Magnetic systems
• Mechanical systems
• Optical systems
12
Magnetic systems
• Utilize sensors placed on the body
to measure the magnetic field
generated by a transmitter source.
13
Magnetic systems
• Advantages
– Require no special lighting condition.
– Sensors are never occluded.
• Disadvantages
– Require a metal-fee environment.
– Electro-magnetic interference
14
Mechanical systems
• Exoskeleton with angle sensors.
15
Mechanical systems
• Advantages
– Measure joint angles (no marker ID
problems).
– Sensors are never occluded.
• Disadvantages
– Breakable!
– Configuration of sensors is fixed.
– Constrains on joints.
16
Optical systems
• The cameras are equipped with infrared
LED's and filters. (Filters enhance the
contrast of the image.)
• The cameras see reflector markers.
17
Optical systems
• Advantages
– Higher sampling rate.
– Larger capture space.
• Disadvantages
– Markers are sometimes occluded -> marker
ID problems.
– Provide only positional data -> joint angles
need to be computed.
18
Typical Mocap system
• Vicon optical system - Best system in
Academia!
• 8 high-speed MX 13 (up to 1000 fps) and
8 high-resolution MX 40 (4 million pixels)
cameras.
• Capture up to 5 performers at once.
19
Mocap animation
• Motion capture animation is different from
keyframe animation in terms of how
motion is created.
• Same principles apply to mocap
animation & keyframe animation!
• A combination of motion capture
animation and keyframe animation is often
used.
20
Keyframe animation
• A keyframe is a drawing of a key moment in an
•
•
21
animated sequence, where the motion is at its
extreme.
Inbetweens fill the gaps between keyframes.
Every motion is created by animators.
Advantages of mocap animation
• Faster to create (only if an established
production pipeline exists.)
• Secondary motions and all the subtle
motions are captured -> more realism.
• Physical interactions between performers
and props can be captured.
22
Disadvantages of mocap animation
• Cost.
• Manipulating mocap data is often difficult > Re-capturing or key framing a shot with
bad data is often easier.
• Mapping mocap data of a performer to a
character with a different proportion often
causes problems.
23
Process of MoCap
• Data needs to be manipulated
• Transform data into file format
• Build specific models
• Attaching the mesh
24
movement flowchart for games
25
•Planning and Directing Motion Capture For Games
By Melianthe Kines Gamasutra January 19, 2000
URL:
http://www.gamasutra.com/view/feature/3420/planning_and_directing_motion
_.php
processing passive markers
• each camera records capture session
• extraction: markers need to be identified in
•
•
26
the image
– determines 2d position
– problem: occlusion, marker is not seen
• use more cameras
markers need to be labeled
– which marker is which?
– problem: crossover, markers exchange
labels
• may require user intervention
compute 3d position: if a marker is seen by
at least 2 cameras then its position in 3d
space can be determined
Direct Linear Transformation
A
Collinearity Condition
B=cA
27
B
Distortion Effects
The DLT model of Abdel-Aziz/Karara only accounts
for errors in perpendicularity between axes
•Achieves accuracy of 1:2000
Other types of distortion
•Radial
Pin Cushion
Barrel
•Tangential
•De-centering
28
16 parameter DLT
•Equations representing nonlinear distortions
L12-L14 represent
symmetrical lens distortion
L15 & 16 represent
asymmetrical/ de-centering
distortion
29
The Problem
•The original DLT method contained 11 unknowns
Uo,Vo
Xo,Yo,Zo
3 rotation angles
2 scaling factors
d
•Only 10 are actually independent
d and the 2 scaling factors are mutually
dependent variables
30
Modified Direct Linear
Transformation
•The Non-Linear constraint is added to the system to ensure
orthogonality
To solve:
31
•
Compute the 11 parameters normally
•
Use the value of one parameter to remove it from
the equation and estimate the 10 independent
parameters
•
Solve for the value of the removed parameter using
the values of the 10 independent parameters
Non-Linear MDLT
• Adds correction for lens distortion to the
MDLT method
32
retargeting: the character
• the character is
controlled by
skeleton
• to control the
skeleton, need to
specify joint
rotations
• muscles?
33
retargeting
• capture motion on
•
•
34
performer
– positions of
markers are
recorded
retarget motion on a
virtual character
– motion is usually
applied to a
skeleton
– a skeleton is
hierarchical
• linked joints
– need rotation data!
need to convert
positions to rotations
performer→ actor → character
• the actor is used to convert
•
•
•
marker positions to rotational
data
– markers are handles on the
actor
– actor should have similar
proportions as the
performer
joint rotations of the actor are
applied to the character
there are still issues with
proportions
http://www.youtube.com/watch
?v=frNpkIH8_vo
Alias Motionbuilder: actor and markers
35
retargeting problems
36
retargeting problems
37
Production pipeline overview
• Calibrate the system.
• Fit a generic skeleton to the subject’s proportion
•
•
•
(subject calibration).
Capture shots & reconstruct 3D trajectories using the
calibrated subject.
Link the subject specific skeleton to a CG character’s
skeleton and edit motion (in MotionBuilder).
Add skin to the CG character, edit motion, and render
(in Maya).
http://atec.utdallas.edu/midori/Handouts/mocap_pip
eline.htm
38
System Calibration and Capturing and Processing Data
2D Image
Circle fitting
2D Camera Data
Triangulation
3D Markers Positions
Reconstruction
C3D
Trajectories
Positional Data
I
Q
Subject
Calibration
• Capture Range of
Motion (ROM)
•Reconstruct trajectories
of ROM
•Label markers
The processes that you
go through for each
character
VST
(subject template)
VSK
Calibrated
Subject
Mocap Pipeline
Flow Chart
Character Setup
HIK
• Correlate :
Actor
(C3D, HIK)
&
(FBX)
FBX
• Bind skin to the skeleton
MAYA
Skeleton Only
• Edit motion
• Bake motion data to the
skeleton
Rendering
MB
39
MOTION
BUILDER
Character
• Create a skeleton
The processes that
you repeat for
each shot
• Place markers on Actor
( intermediate skeleton )
Markers Set
Character Setup
• Rig the character
Actor Setup
Maya Scene File
• Marge the rigged character (MB) and
the skeleton with motion data (FBX)
• Edit motion (IK/FK blend, Trax)
• Render
FBX
Skeleton with motion data
( joint rotation angles )
References
• Sturman, David. “A Brief History of Motion Capture for Computer Character
Animation.” 13 March 1999. MEDIALAB. 25 Nov. 2005.
http://www.siggraph.org/education/materials/HyperGraph/animation/character_anim
ation
• “When Motion Capture Beats Keyframing.” Sept. 1997. Game Developer.
•
•
•
40
http://www.gdmag.com
http://www.kwon3d.com/theory/dlt/dlt.html
http://www.kwon3d.com/theory/dlt/mdlt.html
http://express.howstuffworks.com/gollum3.htm
Related Work
• Motion capture search
– Arikan and Forsyth 2002; Kovar et al. 2002; Lee et al. 2002; Li et al. 2002;
Metoyer 2002
• Physics and motion capture
– Rose et al. 1996
– Popovic and Witkin 1999; Pollard 1999; Pollard and Behmaram-Mosavat 2000
– Shapiro et al. 2003
• Physically based reactions
– Oshita and Makinouchi 2001; Zordan and Hodgins 2002
– Komura et al. 2004; Mandel 2004
• Industry Software
– Natural Motion’s Endorphin
41