Transcript CSCI480/582 Lecture 8 Chap.2.1 Principles of Key

CSCI480/582 Lecture 31 Chap 5.3
Performance Driven Facial Animation
Apr, 13, 2009
Outline





What differs facial animation from body
animation?
Face model
Facial motions and FACS
Facial motion synchronization with audio
production
Performance-driven techniques
Face Model for Facial
Animation

In modeling face, Four aspects needs to be
considered

Structure: vertex distribution and topology to
represent face in geometry

Appearance: color, texture, light reflection of face
Face Model - Geometry
Dense mesh around the
“holes”
 The density of the mesh
determines the level of details
in facial deformation
 Eyes, ears, and mouth cavity
 Teeth and tongue

Face Model Appearance


Using Texture Mapping
Unwinding the “cylinder” surface by concatenating
multiple camera views to form the texture map
More Advanced Skin
Appearance Model



The skin is translucent
Light is scattered through
multiple layers, then
reflected from a sub-layer
Bidirectional Reflectance
Distribution Function is
often used
Bidirectional Reflectance
Distribution Function(BRDF)



A 4-dimensional function that
defines how light is reflected at an
opaque surface
Input:

incoming light direction wi

Outgoing direction wo
Output:

Ratio of reflected radiance along wo
to irradiance incident along wi
Facial Motion



Expressions involving

All front facial mesh

Eye ball orientation
Speeches involving

Lip and jaw

Tongue
The combination of expressions and speeches
Facial Acting Code System

Originally developed by Ekman and Friessen in 70s

Based on the physiology of facial muscles

A common standard to systematically categorize the physical
expression of emotions
FACS – The Codes for Action Units

Action Descriptors

Inner Brow Raiser -- Frontalis (pars medialis)

2 Outer Brow Raiser -- Frontalis (pars lateralis)

4 Brow Lowerer -- Corrugator supercilii, Depressor supercilii

5 Upper Lid Raiser -- Levator palpebrae superioris

6 Cheek Raiser -- Orbicularis oculi (pars orbitalis)

7 Lid Tightener -- Orbicularis oculi (pars palpebralis)

9 Nose Wrinkler -- Levator labii superioris alaeque nasi

10 Upper Lip Raiser -- Levator labii superioris

…

66 Action descriptors

Demo
Lip-Sync by Viseme Morphing



Use reference video to rotoscope
the shape of the 3D model for key
gestures
Use a set of viseme corresponding
to phonemes, then interpolate
between viseme gestures
Such approach often is not
detailed enough for finer visual
speech animation
Facial Motion Driven by
Mocap Data



Captures subtle details of facial expressions
But, Mocap Data need be to cleaned for good rendering
effect. Often this step is labor intensive.
How can we make Mocap more flexible?
FACS-Viseme Solver

Use a small set of Mocap data to synthesize a
much larger range of facial motions


Each piece of mocap data in the small recording set
represent an Action Unit
Use Action Unit analysis to decompose recorded
Mocap data for easy post editing
N
m ocap(i)   wk AUk
k 1

Use audio-visual mapping analysis to align
mocap segment for new speech