Transcript Document

Real-Time Feature Extraction and
Classification of Prehensile EMG Signals
Master Thesis
Christopher Miller
Supervisor: Marko
Vuskovic
Department of Computer Science
San Diego State University
March 24, 2008
Agenda
•
•
•
•
•
•
•
•
•
Introduction
Electromyography (EMG) Signals
EMG Signal Processing
Classification
Experimental Results
Implementation
Conclusion
Future Work
Questions
Copyright © 2008 by Christopher Miller
2
Introduction
(1 of 2)
• Numerous technological advances in prosthetic hands
• Greater degrees of freedom
• Continue to function as pincers
Copyright © 2008 by Christopher Miller
3
Introduction
(2 of 2)
The purpose of this thesis was to implement a
program that could perform real-time feature
extraction and classification of prehensile EMG
signals for the following grasps:
Copyright © 2008 by Christopher Miller
4
Electromyography (EMG) Signals:
Myoelectric Energy Detection
• Motor units control groups
of muscle fibers
• Brain recruits motor units to
innervate muscles for
movement
• Myoelectric energy
produced as motor units
activate
• Surface electrodes detect
myoelectric energy
Copyright © 2008 by Christopher Miller
5
Electromyography (EMG) Signals:
EMG Amplification
• SENIAM Recommends
–
–
–
–
Pre-gelled Ag/Ag-Cl
Bipolar
0.8” inter-electrode distance
0.4” wide
• Surface electrodes
– Pre-gelled Ag/Ag-Cl
– 1” inter-electrode distance
– 0.875” wide
• EMG amplification device
(Saksit Siriprayoonsak, 2005)
– 4 bipolar channels
– 1 reference channel
Copyright © 2008 by Christopher Miller
6
Electromyography (EMG) Signals:
Muscle Anatomy
Copyright © 2008 by Christopher Miller
7
Electromyography (EMG) Signals:
Mounted EMG Amplifier
Copyright © 2008 by Christopher Miller
8
EMG Signal Processing:
System Flowchart
Signal
Processing
EMG
Amplification
Device
A/D
Converter
Time Sample
Extraction
Classifier
Feature
Extraction
Prosthetic
Hand
Controller
Transformation
Copyright © 2008 by Christopher Miller
9
EMG Signal Processing:
Onset of Movement Detection
•
•
•
•
Lidierth (1986)
Hodges & Bui (1996)
Bonato et al. (1998)
Staude et al. (2001) – AGLRamp, AGLRstep
Copyright © 2008 by Christopher Miller
10
EMG Signal Processing:
Bonato Method
gk 
1

2
0
y
2
k 1

y ,
2
k
ta  mink  1, 3, 5,...: g k  h
Copyright © 2008 by Christopher Miller
11
EMG Signal Processing:
Feature Extraction Methods
•
•
•
•
•
•
•
•
•
•
•
•
•
Mean Absolute Value (Hudgins et al., 1993)
Mean Absolute Value Slope (Hudgins et al., 1993)
Willison Amplitude (Willison, 1964)
Zero Crossings (Hudgins et al., 1993)
Slope Sign Changes (Farry et al., 1996)
Waveform Length (Farry et al., 1996)
Simple Square-Integral
Amplitude of the First Burst (Vuskovic et al., 2002)
Multiple Time Windows (Du et al., 2003)
Short-Time Fourier Transform
Wavelet Transform
Wavelet Packet Transform
Spectral Moments (Vuskovic et al., 2005)
Copyright © 2008 by Christopher Miller
12
EMG Signal Processing:
Feature Extraction Comparison
Method
MAV
MAVSLP
VAR
WAMP
Wave form Length
Zero-Crossings
Slope Sign Changes
Squared Integral
Spectral Moments
Parameters
I=3
H = 29
H = 28
H = 100
m=1
K = 11
flag = 0
400 ms
96.11%
83.89%
90.56%
98.33%
98.89%
96.11%
96.67%
90.56%
92.78%
Copyright © 2008 by Christopher Miller
300 ms
95.56%
81.67%
90.00%
95.56%
98.89%
94.44%
96.67%
90.00%
89.44%
200 ms
93.89%
73.89%
86.67%
92.22%
95.00%
83.89%
86.67%
86.67%
82.22%
13
EMG Signal Processing:
Feature Extraction Methods Employed (1 of 3)
Spectral Moments:
P( f ) 
N 1
C
k   N 1
SS
[k ]e
 j 2π f k
K
1
CSS [k ] 
N
M m  CSS [0]I m (0)  2 CSS [k ]I m (k )
k 1
N  k 1
 s[i]s[i  k ], k   1,  2, ...,  ( N 1)
i 0
M m  2 I m (0) M 0 K
Rm 
  CSS [k ]I m (k )
2
k 1
I-coefficients, calculated in advance:
1
I m (0)  m 1
2 (m  1)


(1) k  1
(1) k
(1) k k   2  2
I 0 (k )  0, I1 (k ) 
, I 2 (k ) 
, I 3 (k )  3
(2k ) 2
(2k ) 2
(2k ) 4


k


m
k
(1) k k   6
m 1


m  1I m2 k 
I
(
k
)


1

2
I 4 (k )  2
,
k

1,
2,
3,
...
m
m 1
2
4
(2k )
2 (2k )
k
Copyright © 2008 by Christopher Miller
14
EMG Signal Processing:
Feature Extraction Methods Employed (2 of 3)
Spectral Moment Parameters
Parameter
k
m
Range
0..N
(N is sample size)
0..4
Ts
Flag
0..N
0..2
Purpose
Time-lag parameter in I-coefficient and autocorrelation
function
Highest moment index to use in feature vector (i.e. m = 2
means that M0, M1, and M2 are included)
Size of time sample to use for moment calculation
0 – use straight moments
1 – use M0 and reduced moments for higher values of m
2 – use strictly reduced moments
Copyright © 2008 by Christopher Miller
15
EMG Signal Processing:
Feature Extraction Methods Employed (3 of 3)
Waveform Length:
N
WL   xk  xk 1
k 2
Copyright © 2008 by Christopher Miller
16
Classification:
Methods
• Artificial Neural Networks (McCulloch-Pitts, 1943)
• ARTMAP Networks (Grossberg et al., 1976)
• Mahalanobis-Distance Based ARTMAP Network
(Xu et al., 2003)
• Maximum Likelihood Estimation (Fisher, 1912)
• Mahalanobis Distance (Mahalanobis, 1936)
Copyright © 2008 by Christopher Miller
17
Classification:
Methods Employed
• Maximum Likelihood Estimation (MLE)
p( x | ) 
1
(2 ) SC
1
2
 1

exp ( x  C )T SC1 ( x  C )
 2

Sc  cov(X )  E[ X   X ( X   X )T ]
• Mahalanobis Distance
MD( x)  (C  x)T  SC1  (C  x)
Copyright © 2008 by Christopher Miller
18
Experimental Results:
Impact of Log Transformation
Copyright © 2008 by Christopher Miller
19
Experimental Results:
Box-Cox Transformation
• Box-Cox (1964):
y ( )
y ( )
 y 1

  , 0
 log y,   0
 ( y  2 ) 1  1
, 1  0


1

 log( y  2 ), 1  0
Copyright © 2008 by Christopher Miller
20
Experimental Results:
λ Optimization
• Mahalanobis-distance Classifier using Spectral Moments
– Flag = 0
– K = 11
– Ts = 250
Copyright © 2008 by Christopher Miller
21
Experimental Results:
Moment Optimization
• Classifiers using Spectral Moments
–
–
–
–
Flag = 0
K = 11
Ts = 400
Lam = 0.06
Copyright © 2008 by Christopher Miller
22
Experimental Results:
Time Sample Optimization
• Mahalanobis-distance Classifier using Spectral Moments
–
–
–
–
M=2
Flag = 0
K = 11
Lam = 0.06
Copyright © 2008 by Christopher Miller
23
Experimental Results:
Channel Reduction (1 of 2)
Copyright © 2008 by Christopher Miller
24
Experimental Results:
Channel Reduction (2 of 2)
• Mahalanobis-distance Classifier using Spectral Moments
–
–
–
–
–
M=2
Flag = 0
K = 11
Lam = 0.06
3 channels
Copyright © 2008 by Christopher Miller
25
Experimental Results:
Feature Comparison (1 of 2)
- Spectral Moments
Waveform Length -
Copyright © 2008 by Christopher Miller
26
Experimental Results:
Feature Comparison (2 of 2)
MAH Classifier Performance at 264 ms Using Various Feature Vectors
Percent Used
for Testing
10%:
20%:
30%:
40%:
50%:
60%:
70%:
80%:
Average of
50% and below
Spectral
Moments
99.34%
99.20%
98.84%
97.96%
96.30%
26.87%
N/A
N/A
98.33%
Waveform
Length
97.79%
97.83%
97.89%
97.70%
97.59%
97.37%
96.93%
95.10%
97.76%
M=0
98.54%
98.31%
97.90%
97.56%
96.92%
95.77%
89.73%
25.78%
97.85%
Copyright © 2008 by Christopher Miller
Hybrid
M=1
98.82%
98.27%
97.46%
96.20%
93.36%
25.88%
N/A
N/A
96.82%
M=2
98.51%
97.37%
95.45%
89.33%
26.07%
N/A
N/A
N/A
81.35%
27
Experimental Results:
Hybrid Approach
Copyright © 2008 by Christopher Miller
28
Experimental Results:
Spectral Moments
• Best classification rate: 97.5%
• Optimal time sample: 378 ms
Copyright © 2008 by Christopher Miller
29
Experimental Results:
Waveform Length
• Best classification rate: 95%
• Optimal time sample: 300 ms
Copyright © 2008 by Christopher Miller
30
Experimental Results:
Waveform Length Online Testing
• Cross-validation
– 95% at 300 ms
• Online validation
–
–
–
–
–
Sphere: 22/25 (88%)
Cylinder: 25/25 (100%)
Precision: 21/25 (84%)
Lateral: 17/25 (68%)
Total: 85%
Copyright © 2008 by Christopher Miller
31
Experimental Results:
Spectral Moments Online Testing
• Cross-validation
– 89.4% at 355 ms
• Online validation
–
–
–
–
–
Sphere: 24/25 (96%)
Cylinder: 25/25 (100%)
Precision: 25/25 (100%)
Lateral: 19/25 (76%)
Total: 93%
Copyright © 2008 by Christopher Miller
32
Implementation:
Program Files
Name
EMGProgGUIFC.m
EMGProgGUIFC.fig
Train.m
GetClass.m
LOOValidation.m
BonatoOnset.m
BonatoRealTime.m
Features.m
Moments.m
GenerateI.m
FeaturesWL.m
FeaturesMAV.m
BoxCox.m
dflt_train.mat
EMGDLLmfile.m
EMGDLL.dll
DAQ_Functions.h
DAQ_Functions.c
Type
Purpose
MATLAB
GUI support functions for program
GUI front-end for program
Function to generate classifier parameters S, C, N
Function to classify data using current classifier
Leave One Out Validation procedure used for training
Function to determine onset of movement in recorded data
Function to determine onset of movement in real-time
Function to create feature vector using spectral moments
Function to compute spectral moments of time signal
Function to generate I coefficients for moment calculation
Function to create feature vector using waveform length feature
Function to create feature vector from mean absolute value
Function to perform Box-Cox transformation
MAT file containing classifier based on research data
DAQ functions MATLAB prototype file
C
Dynamic Link Library (DLL) for DAQ card
National Instruments DAQ functions header
National Instruments DAQ functions implementation
Copyright © 2008 by Christopher Miller
33
Implementation:
Main Screen
Copyright © 2008 by Christopher Miller
34
Implementation:
Training (1 of 3)
Copyright © 2008 by Christopher Miller
35
Implementation:
Training (2 of 3)
• Leave-one-out validation from 200400 ms in 10 ms intervals
• Optimal time explored at 1 ms
intervals
• Both features scored based on
classification rate and time sample
required
• Highest scoring feature selected and
system trained
Copyright © 2008 by Christopher Miller
36
Implementation:
Training (3 of 3)
• May load training from files
• Training recordings automatically stored with classifier
• Training recordings saved to EMGRecordings folder
Copyright © 2008 by Christopher Miller
37
Implementation:
Real-Time Grasp Classification
Onset of
movement not
detected
Rest
Position
Onset of movement
detected
Grasp tension
below threshold
Start
Grasp
Collect
grasp data
Grasp
classified
Hold
Grasp
Grasp tension
above threshold
Copyright © 2008 by Christopher Miller
38
Implementation:
Real-Time Grasp Classification Screenshot
Copyright © 2008 by Christopher Miller
39
Implementation:
Real-Time Grasp Classification Video
Copyright © 2008 by Christopher Miller
41
Conclusion
• Feasible approach demonstrated for real-time classification of
prehensile EMG signals
• More natural approach reduces mental and physical demands
on operator
• Only 3 EMG channels are necessary to classify the 4 grasps
• Waveform Length proved to be valuable for small training sets
• Spectral Moments enabled better performance (93% online)
with larger training sets and optimized parameters
–
–
–
–
–
K = 11
m=2
flag = 0
λ = 0.06
Ts optimized for training set
Copyright © 2008 by Christopher Miller
42
Future Work:
Additional Grasps
• Previous research explored classifying 6 grasps, which
included small and large versions of balls and cylinders
• Two-phased approach to classification is more likely to
succeed
Copyright © 2008 by Christopher Miller
43
Future Work:
Full Control
• Channel 4 of current device can be employed for grasp control
Copyright © 2008 by Christopher Miller
44
Future Work:
Online Learning
• Real-Time program should be written in multi-threading
capable language
• Online learning capability with feedback from prosthetic hand
• Mahalanobis-distance based ARTMAP network suggested
EMG Classifier
Library
(MATLAB DLL
with C Interface)
Real-Time EMG
Classifier GUI
(MATLAB)
Upload
classifier
Classify
EMG Data
Deployed
Real-Time
Program
(C++ or Java)
Train
classifier
Read EMG
Data
EMG
Device
Drivers
(C)
Update
grasp
Robotic
Hand
Controller
(C/C++)
Copyright © 2008 by Christopher Miller
45
Questions
Copyright © 2008 by Christopher Miller
47
Backup Slides
Differential Amplification
Copyright © 2008 by Christopher Miller
49