Transcript NEUROPHONE: BRAINMOBILE PHONE INTERFACE USING A WIRELESS EEG HEADSET Andrew T. Campbell, Tanzeem Choudhury, Shaohan Hu, Hong Lu, Matthew K.

NEUROPHONE: BRAINMOBILE PHONE INTERFACE
USING A WIRELESS EEG
HEADSET
Andrew T. Campbell, Tanzeem Choudhury,
Shaohan Hu, Hong Lu,
Matthew K. Mukerjee!, Mashfiqui Rabbi, and
Rajeev D. S. Raizada
Dartmouth College, Hanover, NH, USA
Motivation
• Mobile phones and neural signals are present are
accessible to many people.
• Recent advances in technology has led to the
development in low-cost EEG headsets.
• Smart phones are now powerful enough to run
sophisticated machine learning algorithms.
• It is thus easy to interface neural signals with mobile
computing paradigms.
Introduction
• This group proposed to used neural signals to control a
mobile phone.
• They developed the NeuroPhone system that translates
and decodes neural signals to drive a mobile app using
off-the-shelf wireless EEG headsets.
• This paper demonstrates their brain-controlled address
app:
• An application that uses the brain signals to select address
contacts to call.
Introduction
• They implement their mobile app using two different
paradigms: P300 dialing and “Wink”-triggered dialing.
• P300 signals are positive transient deflections in EEG that are
elicited in response to a rare or novel stimulus
• The eye “Wink” is a type of EMG signal that is generated in
response to the contraction of skeletal muscle contraction.
Challenges
• Research Grade EEG headsets
• Expensive (Often costing tens of thousands of dollars)
• Offer very robust and reliable EEG signals
• Off-the-shelf EEG headsets
• More affordable ($100-$500)
• Electrode design and amplification are not as robust
• Results in noisy, low-quality signals.
• Require more sophisticated processing techniques to classify neural
events.
• Most Off-the-shelf headsets are wireless and thus encrypt the EEG
signals.
• They are designed for synchronization with a computer (using wireless
dongle).
• They complicate the process of developing a clean brain-mobile
interface.
Challenges
• There is an energy cost for brain-mobile interfacing:
• Continuously streaming raw brain-signals wirelessly
• Running classifiers on the phone introduces heavy processor
loads.
• Brain-mobile phones could likely be used in applications
such as: walking, riding in a car or bicycle, shopping, etc.
• Many of these cases present significant noise artifacts in the EEG
signals.
• These signals will need to be filtered out to improve the brain-mobile
interface
NeuroPhone
• The NeuroPhone system uses the
iPhone to display pictures of
contacts in the phone’s address
book.
• The pictures are displayed and
flashed in random order.
• For the EEG mode, the user
concentrates on a picture of the
person they wish to call.
• For the wink mode, the person
winks with the left or right eye to
make the intended phone call
P300
• Whenever the user
concentrates on a target
stimulus among a pool of
non-target stimulus, the
target stimulus (flash) will
elicit a positive peak in the
EEG at around 300ms after
stimulus onset (P-300).
• The P300 signal can be
found on most EEG
channels
• Common on central and parietal
channels
NeuroPhone - P300 Paradigm
• In This case, there are 6 total stimuli on the
screen (5 non-target and 1 target). The user
visually attends to one of the photos while each
photo is flashed in a random order. Whenever
the target photo flashes, a P300 should be
generated.
Wireless EEG Headset
• Emotiv EPOC headset
• 14 data electrodes (2 reference electrodes)
• Transmits encrypted data wirelessly to a
windows-based machine. (802.11) 2.4GHz
• Low SNR
• Contains build in gyroscope
• ~$300
Pre-Processing
• Signals were band-passed filtered to keep only the
relevant information within the P300 range.
• Signal averaging was performed to increase the SNR
• This improves the quality of the signal while simultaneously adding
lag to the system
Classification
• To reduce complexity, only a subset of relevant channels
are used for classification.
• Wink Mode
• Multivariate, naive Bayesian classifier.
• P300 Mode
• Decision stump classifer
Implementation
• Laptop relay is used for decoding of the encrypted Emotiv
signals
• Encrypted EEG signals are sent from the phone to a laptop for
decryption (via WiFi).
• Decrypted EEG signals are sent back to the phone.
• Signals are sampled at 128 samples per second and transferred to
the phone at 4kbps per channel.
Wink Mode Classification
• Emotiv head-set was put on
backwards to place two electrodes
directly above the eyes.
• Data was collected by having the
subject wink multiple times.
– Data were labeled as “wink” or
“non-wink”
• A Bayesian classifier was trained
by calculating the mean and
variance of each wink and nonwink and building respective
Gaussian models.
– As can be seen, the two models do
not overlap leading to good
classification
P300 Classification
• The Gaussian distributions overlap too much and
therefore cannot be classified with a Bayesian classifier.
• Signals from each of the six stimuli were band-passed
filtered between 0-9Hz.
• The highest signal segment at around 300ms after
stimulus onset is extracted.
• For classification, a decision stump is used where the
threshold is set to the maximum value of the extracted
segment.
Results (Wink-Mode)
• Multiple sessions were collected on three subjects.
• Subjects performed the test while sitting and while
walking
• The classifier was trained on five sessions from a
single subject and then tested on the remaining
subjects. (I think).
• Results are shown in table 1
– Precision: % of classified winks that are actual winks
– Recall: % of actual winks that are classified as winks.
– Accuracy: % of total events that are classified correctly
Results (P300 mode)
• Data was collected with same set of subjects while sitting,
with loud background music and while standing up.
Discussion
• Although data was classified using the P300 mode, large
amounts of averaging is needed to get decent
classification accuracies.
• This “unresponsiveness” of the system proves to be very frustrating
for the end user.
• i.e. it can take 100 seconds to initiate a phone call with only 89%
chance of dialing the right person (with six to choose from).
• This System is currently not in any form to be used by
subjects on a regular basis.
• Looking into single trial classification techniques to speed up the
system.
Phone Loading Statistics
• The CPU usage when running the application:
• 3.3% for the iPhone (iphone 3g?).
• Total memory usage:
• 9.40MB memory used
• (9.14MB are for GUI elements).
• Continuous streaming raw EEG channels to the phone,
and processing signals lead to battery drain (no
quantitative measure given)
• Looking into duty cycling to solve this phone.