Micro expression Detection using Strain Patterns
Download
Report
Transcript Micro expression Detection using Strain Patterns
Micro expression Detection using
Strain Patterns
-Sridhar Godavarthy
Based on
V.Manohar, D.B. Goldgof, S.Sarkar, Y. Zhang, "Facial Strain Pattern as a
Soft Forensic Evidence", IEEE Workshop on Applications of Computer
Vision (WACV'07),pp 42-42
Microexpressions
• What are microexpressions?
– Subtle movements of the human body
– Quick enough to be completed within the blink
of an eye
– Not large enough to be called a micro
expression
Microexpressions Contd…
• Examples:
•
•
•
•
Raising an eyebrow
Shrugging of shoulders
Pout of lips
Fast blinking of eye
• Non Examples:
•
•
•
•
Talking
Smiling
Laughing
Anger
How do we go about it?
• Majority of the work “referenced” from :
V.Manohar, D.B. Goldgof, S.Sarkar, Y. Zhang,
"Facial Strain Pattern as a Soft Forensic
Evidence", IEEE Workshop on Applications of
Computer Vision (WACV'07),pp 42-42
However, to keep things clear, we discuss
each step in detail.
•
First Step – Obtaining Motion
Field
Feature Based
–
–
–
–
–
Need to identify features – Difficult!
Features may be ill defined( when camouflaged)
Usually requires manual intervention
Produces a sparse motion field
Produce Good correspondence in large motion
• Optical Flow based
– Fully automated
– Dense Motion field.
– Requires constant illumination
Second Step – Strain Computation
Type
• 3D Strain
– Ideal
– No high speed equipment available to capture
range images
• 2D Strain
– Well – not much of a choice
– Authors could use existing data.
Strain Computation - methods
• Finite Element Method
–
–
–
–
Forward modeling when Dirichlet condition is satisfied
Good at handling irregular shapes
Computationally expensive
This method is an approximation to the solution
• Finite Difference Method
– Strain, a tensor, can be expressed derivatives of the
displacement vector
– This can be approximated by a Finite Difference Method.
– Very efficient when carried out on a regular grid.
– This method is an approximation to the differential equation
Finite Difference Method
• Finite Strain tensor
• Cauchy tensor
Video
• Video is a collection of individual images
also known as frames
• In reality: spatial and temporal compression
using properties of the scene.
• Any video can be decoded into a series of
frames.
• 24/30 frames per second of video.
Video Coding
• The science of
encoding a video in a
manner such that
– Minimum number of
bits are used
– Motion compensated
prediction can be
performed from a
previous frame.
Optical Flow
• Pattern of apparent motion of objects/surfaces/edges
caused by relative motion between the
observer(camera) and the scene
Elasticity
• Different materials have different elasticity
• Elasticity can be modeled
stress
Elasticity
strain
Known
Calculate
Optical Strain
• Variation of displacement values obtained from optical flow
– Calculated by taking the derivative of each pixel
Sobel operator (central difference)
Facial Strain
• What is Facial Strain?
– Strain on soft tissue when expressions are
made.
– Anatomical method
– Uses a pair of frames to measure deformation
Strain Measurement
• Finite Difference Method
• Compute spatial derivatives from discrete
points.
– Forward Difference Method
– Central Difference Method
– Richardson extrapolation
The Process
Start
Divide videos into Training
and Testing Sequences
Read
Training
Video
Read
Neutral
Micro Exp
Macro
Exp
Frames
Calculate OF & OS
for Micro &
Macro frames
Decode Video
The Process Contd…
Read Test
Video
Decode Video
Read
Neutral
Frames
Stop
For Every other Frame
Calculate Optical
Flow between
Neutral frame
and each frame
OF
within
desired
range?
No
Reject Frame
Yes
Calculate Optical
Strain between
Neutral frame
and this frame
OS
within
desired
range?
No
Reject Frame
Yes
MICRO
EXPRESSION
Program Output…
Program Output Contd…
RESULTS – Training & Test
Frame: Neutral
Optical Flow
Frame: Micro
Normalized optical flow
RESULTS - Test
Frame: Neutral
Optical Flow
Frame: Micro
Normalized optical flow
RESULTS - Test
FALSE
POSITIVE
Frame: Neutral
Optical Flow
Frame: Macro
Normalized optical flow
RESULTS - Test
Frame: Neutral
Frame: XYZ
NOT
DETECTED
Optical Flow
Normalized optical flow
Decisions and Consequences
• Use OF to reject highly improbable frames
– Improves performance
– Rejects frames with both Micro and Macro
Exp.
– Segment image into regions?
• Threshold for classification was set to be
⅔T.O.S < M.O.S < 5/3 T.O.S
T.O.S = Training Optical Strain.
M.O.S = Measured Optical Strain
Possible Options
• Use only Optical Flow
• Use only Optical Strain
• Vary the thresholds
• Use an alternative for Max(OF/OS)
Future work
• Do not compare all frames( Skip frames
immediately after a positive)
• Identify only one positive in a sequential list
of positives
• Segment images to get separate Micro and
Macro expressions
What I could not do
• Display strain as an image.
• Segment image into regions and have
Regions of Interest
References
• V.Manohar, D.B. Goldgof, S.Sarkar, Y. Zhang, "Facial Strain
Pattern as a Soft Forensic Evidence", IEEE Workshop on
Applications of Computer Vision (WACV'07),pp 42-42
• Vasant Manohar, Matthew Shreve, Dmitry Goldgof and Sudeep
Sarkar, "Finite Element Modeling of Facial Deformation in
Videos for Computing Strain Pattern", International Conference
on Pattern Recognition, Dec. 2008
• Matthew A. Shreve, Shaun J. Canavan, Yong Zhang, John R.
Sullins, and Rupali Patil, "Imaging And Characterization Of
Facial Strain In Long Video Sequences",xxxx
• Malcolm Gladwell,” Blink: The Power of Thinking Without
Thinking”, Back Bay Books (April 3, 2007)
Thank You!
Sridhar Godavarthy
Dept. Of Computer Science and Engineering
University of South Florida
[email protected]
About the Template
Fire was considered the most fearful of weapons in Greek Mythology. This template –
“Fireball” is a depiction of the might and power of fire, punching through ignorance and
enlightening the audience