Transcript Knowledge-based Video Human Motion Recognition

3D Motion Capture Assisted
Video human motion recognition
based on the Layered HMM
Myunghoon Suk & Ashok Ramadass
Advisor : Dr. B. Prabhakaran
Multimedia and Networking Lab
The University of Texas at Dallas
Contents
• Motivation
• Previous Work
• Current Work
– Extracting 2D feature data (MHI)
– Classifying human motions
Motivation
Video Human
Motion data
Cleaned
Semantic Data
+
3D MOCAP data
Easy to get, but
Quite noisy
Recognizing
Video Human Motion
MHI
Quantization
2D Motion
Shape data
Test data
K-means
(WEKA)
Observation
Sequence data
O2
O3 O4 O
m
O1
s1
s2
s3
s4
sn
T
Which Motion?
3D Motion
Capture data
1
2
3
Hidden state-transition
Sequence data
4 … t
HMM
Modeling
Forehand, Backhand, Smash,
Left kick, Right Kick, Left
punch, Right punch
3D Motion Capture Assisted
Video Human Motion Recognition Enhancement
Current Work
• The system for falling-down detection of
elderly or patient at home
• Lower layered HMMs with 3D motion capture
data are to estimate one of atomic activities
(e.g. movement of human hip portion)
• Higher layer recognizes exactly the fallingdown motion with much longer time
granularity
Layered HMM
HMM (B) (Baum-Welch)
Normal Action
Abnormal Action
(Falling down)
Classification Results
Movement directions
HMM (A)
2D Feature Vector
Position of human hip
Horizontal direction
Up direction
Down direction
Background Techniques
• Extracting 2D feature (Computer Vision)
– Motion History Images (MHI)
• Classification (Machine Learning)
– Hidden Markov Model (HMM)
– Layered Hidden Markov Model (LHMM)
Motion History Images
• Keywords:
– Motion Energy Image (MEI)
– Motion History Image (MHI)
– 2D Image feature data with suggestion of possible
actions.
Motion History Images
• Motion Energy Image :– Describes the motion energy for a given view of
action
– Spatial distribution of motion – WHERE
Motion History Images
• Motion History Image :– Pixel intensity
– HOW the spatial distribution has occurred
Motion History Images
2D Image
Feature Date
MEI
WHERE
MHI
HOW
Suggestion of
Possible actions
Motion History Images
• Reference Paper :– Hierarchical Motion History Images for recognizing
Human Motion.
Project - Face detection using HAAR like
Features & AdaBoost algorithm
• deals with the application of one of the four AdaBoost algorithms in
boosting the classifiers based on the paper "Robust Real Time Face
detection by viola & jones“
• OpenCV Visual C++
• Available Source files:
Face detection using available HAAR like Features.
• PreRequisites:
Basic knowledge of using OpenCV library.
Knowledge on AdaBoost(Adaptive Boosting) – A Machine Learning
Algorithm.
• Other References:
http://cmp.felk.cvut.cz/~sochmj1/adaboost_talk.pdf
Project – Contour detection using Background Subtraction
and Edge Detection Techniques
• OpenCV Visual C++
• Available Source files:
Reading a video file.
• PreRequisites:
• Basic knowledge of using OpenCV library.
• Other References:
• Introduction to opencv programming http://www.cs.iit.edu/~agam/cs512/lectnotes/opencv-intro/opencv-intro.html
Thank you
Myunghoon Suk ([email protected])
Ashok Ramadass ([email protected])