Transcript Human Activities Recognition

Amin Rasekh, Chien-An Chen, Yan Lu
CSCE 666 Project Presentation

Introduction
◦ Human Activity Recognition
◦ Active Learning



Goals
Literature Review
Methods
◦ Data Collection and Feature Extraction
◦ Classification Techniques
◦ Query Strategies of active learning


Results
Conclusions
Using sensors to identify human activities such as
walking, jogging, limping.

Motivation

Sensors types

◦ Human survey (study human daily activities)
◦ Medical care (diabetes, elderly, rehabilitation)
◦ Inertial sensors (accelerometer, gyroscope)
◦ Camera
◦ GPS
Smartphone is small and convenient to carry
around and its computational resource is
powerful enough for our purpose.
Passive Learning: What we have studied in class
We can achieve greater accuracy with fewer training labels if we
choose the data from which we learn

Motivation: To minimize the time and labor for labeling
abundant data

Design a simple, light weight, and accurate
system that can learn human activity with
minimum user interaction.
◦ Compare and find a model that best fit our system
in terms of accuracy and efficiency.
◦ Reduce the labeling time and labor works using
active learning.





Use one or multiple camera to do a vision-based recognition
[5,6].
Install multiple inertial sensors on the body. [1, 2, 3,4]
A mixture between vision-based and inertial sensor
system.[7]
Classifiers such as Bayesian Decision Making, KNN, SVM, ANN
were studied before. [10,11]
Features from time domain, frequency domain and wavelet
analysis have been studied.[8,9]

Data Collection
◦
◦
◦
◦

Smartphone: HTC EVO 4G
Sensor: 3D accelerometer,50 Hz
Cellphone in pockets around waist
3 people 5 activities: walking, biking, walking upstairs,
walking downstairs, jogging, limping
Feature Generation (Total 31 features)
◦ Sampling Window: 256 samples (5.12 seconds)
◦ Time Domain:
 Variance, Mean, 25% Percentile, 75% Percentile, Correlation,
Average Resultant Acceleration
◦ Frequency Domain:
 Energy, Entropy, Centroid Frequency, Peak Frequency

Classification Techniques
◦
◦
◦
◦

Quadratic
K-Nearest Neighbors
Support Vector Machines
Artificial Neural Networks
Query Strategies based on Uncertainty
◦
◦
◦
◦
Quadratic:
KNN:
SVM:
ANN
Distance from discriminant curve
Entropy
Distance from the boundary
Discriminant function values
Query is performed for the unlabeled instance
that is nearest to the discriminant curve or SVM
boundary
Random Query
Active Query
Query is performed for the unlabeled instance
that has the maximum entropy:
-0.8
-1
LDA component 2
-1.2
-1.4
-1.6
-1.8
-2
LDA component 4
8.5
walking
limping
jogging
downstair
upstair
walking
limping
jogging
downstair
upstair
8
7.5
-2.2
-2.4
-2.6
-1.8
-1.6
-1.4
-1.2
-1
-0.8
-0.6
LDA component 1
-0.4
-0.2
0
7
8.4
8.6
8.8
9
9.2
9.4
9.6
LDA component 3
9.8
10
10.2
10.4
◦ Sequential Forward Selection (Wrapper)
◦ Algorithm: SVM
◦ 10-Fold Cross Validation for each feature subset
◦ Best Features
 Variance, 25% Percentile, Frequency-Domain Entropy,
Peak Frequency
◦ Classification Rate of SVM+LDA:
78%
◦ Classification Rate of SVM+SFS:
84%
Second LDA Component
30
20
10
0
-10
-20
-30
-50
-45
-40
-35
-30
First LDA Component
-25
Quadratic
KNN
SVM
0.7
Active Learning
Random Sampling
0.65
Classification rate
0.6
0.55
0.5
0.45
0.4
0.35
0
50
100
Number of Instance Queries
150
-0.8
-0.8
-1
-1
Active learning
with SVM
-1.6
-1.8
-2
-1.4
-1.6
-1.8
-2
-2.2
-2.2
-2.4
-2.4
-2.6
-1.8
-1.6
-1.4
-1.2
-1
-0.8
-0.6
-0.4
-0.2
Random sampling
with SVM
-1.2
LDA component 2
-1.4
-2.6
-1.8
0
-1.6
LDA component 1
-1.4
-1.2
-1
-0.8
-0.6
-0.4
LDA component 1
KNN
SVM
Quadratic
0.8
0.75
0.7
Classification rate
LDA component 2
-1.2
0.65
0.6
0.55
0.5
0.45
Active Learning
Random Sampling
0.4
0.35
0
50
100
150
200
Number of instance queries
250
300
-0.2
0

Improving the performance of active learning
for activity recognition problem
◦ Clustering
◦ Hybrid query strategies

Adding more activities such as biking




We achieved a classification rate of over 80%
on 5 human activities using a smartphone.
The result is robust to common positions and
orientations of cellphone.
SVM+SFS gives the best performance and is
promising to run on mobile devices.
Performance of active learning is highly
sensitive to the type of problem
Thank you!
Questions?
1)
2)
3)
4)
5)
6)
7)
8)
9)
10)
11)
L. Bao and S. S. Intille, “Activity recognition from user-annotated acceleration data,” Pers Comput., Lecture Notes in
computer Science, vol. 3001, pp. 1–17, 2004.
U. Maurer, A. Rowe, A. Smailagic, and D. Siewiorek, “Location and activity recognition using eWatch: A wearable sensor
platform,” Ambient Intell. Everday Life, Lecture Notes in Computer Science, vol. 3864, pp. 86–102, 2006.
J. Parkka, M. Ermes, P. Korpipaa, J. Mantyjarvi, J. Peltola, and I. Korhonen, “Activity classification using realistic data from
wearable sensors,” IEEE Trans. Inf. Technol. Biomed., vol. 10, no. 1, pp. 119–128, Jan. 2006.
N.Wang, E. Ambikairajah,N.H. Lovell, and B.G. Celler, “Accelerometry based classification of walking patterns using timefrequency analysis,” in Proc. 29th Annu. Conf. IEEE Eng. Med. Biol. Soc., Lyon, France, 2007, pp. 4899–4902.
T.B.Moeslund,A.Hilton,V.Kr ¨ uger, Asurveyofadvancesinvision-based human
motioncaptureandanalysis,Comput.VisionImageUnderstanding 104 (2–3)(2006)90–126.
T.B. Moeslund, E. Granum, A survey of computer vision-based human motion capture, Comput. Vision Image Understanding
81 (3) (2001) 231–268.
Y. Tao, H. Hu, H. Zhou, Integration of vision and inertial sensors for 3D arm motion tracking in home-based rehabilitation,
Int. J. Robotics Res. 26 (6) (2007) 607–624.
Preece S J, Goulermas J Y, Kenney L P J and Howard D 2008b A comparison of feature extraction methods for the
classification of dynamic activities from accelerometer data IEEE Trans. Biomed. Eng. at press
N. Ravi, N. Dandekar, P. Mysore, and M. L. Littman. Activity recognition from accelerometer data. In AAAI, pages 1541–
1546, 2005.
S.J. Preece, J.Y. Goulermas, L.P.J. Kenney, D. Howard, K. Meijer and R. Crompton, Activity identification using body-mounted
sensors—a review of classification techniques. Physiol Meas, 30 (2009), pp. R1–R33.
Altun, K., Barshan, B., Tun¸cel, O.: Comparative study on classifying human activities with miniature inertial and magnetic
sensors. Pattern Recogn. 43(10), 3605–3620 (2010), doi:10.1016/j.patcog.2010.04.019

Support Vector Machine
30
30
20
20
10
10
0
0
-10
-10
-20
-20
-30
-50
-45
-40
-35
-30
-30
-25 -50
-45
-40
-35
-30
-25