Transcript Slide 1

Activity Recognition Using
Cell Phone Accelerometers
Jennifer Kwapisz, Gary Weiss, Samuel Moore
Department of Computer & Info. Science
Fordham University
July 25, 2010
SensorKDD 2010
1
We are Interested in WISDM

WISDM: WIreless Sensor Data Mining
Powerful portable wireless devices are becoming
common and are filled with sensors
 Smart phones: Android phones, iPhone
 Music players: iPod Touch


Sensors on smart phones include:

Microphone, camera, light sensor, proximity sensor,
temperature sensor, GPS, compass, accelerometer
July 25, 2010
SensorKDD 2010
2
Accelerometer-Based Activity
Recognition


The Problem: use accelerometer data to
determine a user’s activity
Activities include:
Walking and jogging
 Sitting and standing
 Ascending and descending stairs
 More activities to be added in future work

July 25, 2010
SensorKDD 2010
3
Applications of Activity Recognition

Health Applications
Generate activity profile to monitor overall type and
quantity of activity
 Parents can use it to monitor their children
 Can be used to monitor the elderly


Make the device context-sensitive
Cell phone sends all calls to voice mail when jogging
 Adjust music based on the activity


Broadcast (Facebook) your every activity
July 25, 2010
SensorKDD 2010
4
Our WISDM Platform

Platform based on Android cell phones



Android is Google’s open source mobile computing OS
Easy to program, free, will have a large market share
Unlike most other work on activity recognition:


No specialized equipment
Single device naturally placed on body (in pocket)
July 25, 2010
SensorKDD 2010
5
Our WISDM Platform

Current research was conducted off-line



Data was collected and later analyzed off-line
In future our platform will operate in real-time
In June we released real-time sensor data
collection app to Android marketplace

Currently collects accelerometer and GPS data
July 25, 2010
SensorKDD 2010
6
Accelerometers

Included in most smart phones & other devices
All Android phones, iPhones, iPod Touches, etc.
 Tri-axial accelerometers that measure 3 dimensions


Initially included for screen rotation and
advanced game play
July 25, 2010
SensorKDD 2010
7
Examples of Raw Data




Next few slides show data for one user over a
few seconds for various activities
Cell phone is in user’s pocket
Earth’s gravity is registered as acceleration
Acceleration values relative to axes of the
device, not Earth

In theory we can correct this given that we can
determine orientation of the device
July 25, 2010
SensorKDD 2010
8
Standing
July 25, 2010
SensorKDD 2010
9
Sitting
July 25, 2010
SensorKDD 2010
10
Walking
July 25, 2010
SensorKDD 2010
11
Jogging
July 25, 2010
SensorKDD 2010
12
Descending Stairs
July 25, 2010
SensorKDD 2010
13
Ascending Stairs
July 25, 2010
SensorKDD 2010
14
Data Collection Procedure

User’s move through a specific course
Perform various activities for specific times
 Data collected using Android phones
 Activities labeled using our Android app



Data collection procedure approved by Fordham
Institutional Review Board (IRB)
Collected data from 29 users
July 25, 2010
SensorKDD 2010
15
Data Preprocessing


Need to convert time series data into examples
Use a 10 second example duration (i.e., window)


3 acceleration values every 50 ms (600 total values)
Generate 43 total features
Ave. acceleration each axis (3)
 Standard deviation each axis (3)
 Binned/histogram distribution for each axis (30)
 Time between peaks (3)
 Ave. resultant acceleration (1)

July 25, 2010
SensorKDD 2010
16
Final Data Set
July 25, 2010
ID
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
Walk
74
48
62
65
65
62
61
57
31
62
64
36
60
62
61
65
70
66
69
31
54
33
30
62
67
85
84
32
65
Jog
15
15
58
57
54
54
55
54
59
52
55
63
62
0
32
61
0
59
66
62
62
61
5
0
64
52
70
19
55
Up
13
30
25
25
25
16
13
12
27
20
13
0
24
7
18
24
15
20
41
16
15
25
8
23
21
0
24
26
19
Down
25
20
23
22
25
19
11
13
23
12
12
0
15
8
18
20
15
20
15
15
16
10
10
21
16
0
21
22
18
Sit
17
0
13
6
77
11
9
0
13
16
8
8
0
15
9
0
7
0
0
4
12
0
7
8
8
14
11
8
8
Stand
7
0
9
8
27
8
4
0
10
9
9
6
0
10
8
8
7
0
0
3
9
0
0
15
7
17
13
15
14
Total
151
113
190
183
273
170
153
136
163
171
161
113
161
102
146
178
114
165
191
131
168
129
60
129
183
168
223
122
179
Sum
%
1683
37.2
1321
29.2
545
12.0
465
10.2
289
6.4
223
5.0
4526
100
SensorKDD 2010
17
Data Mining Step

Utilized three WEKA learning methods
Decision Tree (J48)
 Logistic Regression
 Neural Network


Results reported using 10-fold cross validation
July 25, 2010
SensorKDD 2010
18
Summary Results
July 25, 2010
SensorKDD 2010
19
J48 Confusion Matrix
Predicted Class
A
c
t
u
a
l
C
l
a
s
s
July 25, 2010
Walk
Jog
Up
Down
Sit
Stand
Walk
1513
14
72
82
2
0
Jog
16
1275
16
12
1
1
Up
88
23
323
107
2
2
Down
99
13
92
258
1
2
Sit
4
0
2
3
270
3
Stand
4
1
2
7
1
208
SensorKDD 2010
20
Conclusions

Able to identify activities with good accuracy
Hard to differentiate between ascending and
descending stairs. To limited degree also looks like
walking.
 Can accomplish this with a cell phone placed
naturally in pocket
 Accomplished with simple features and standard
data mining methods

July 25, 2010
SensorKDD 2010
21
Related Work

At least a dozen papers on activity recognition using
multiple sensors, mainly accelerometers



Activity recognition also done via computer vision
Actigraphy uses devices to study movement


Typically studies only 10-20 users
Used by psychologists to study sleep disorders, ADD
A few recent efforts use cell phones


Yang (2009) used Nokia N95 and 4 users
Brezmes (2009) used Nokia N95 with real-time recognition

July 25, 2010
One model per user (requires labeled data from each user)
SensorKDD 2010
22
Future Work





Add more activities and users
Add more sophisticated features
Try time-series based learning methods
Generate results in real time
Deploy higher level applications: activity profiler
July 25, 2010
SensorKDD 2010
23
Other WISDM Research

Cell Phone-Based Biometric identification1




Same accelerometer data and same generated features but
added 7 users (36 in total)
If we group all of the test examples from one cell phone and
apply majority voting, achieve 100% accuracy
Can be used for security or automatic personalization
Interested in GPS spatio-temporal data mining
1 Kwapisz, Weiss, and Moore, Cell-Phone Based Biometric Identification, Proceedings of the
IEEE 4th International Conference on Biometrics: Theory, Applications, and Systems (BTAS-10),
September 2010.
July 25, 2010
SensorKDD 2010
24
Thank You
Questions?
July 25, 2010
SensorKDD 2010
25