Transcript Lesson 8

M.Tech. (CS), Semester III, Course B50
Functional Brain Signal
Processing: EEG & fMRI
Lesson 8
Kaushik Majumdar
Indian Statistical Institute
Bangalore Center
[email protected]
Artificial Neural Network (ANN)

What does a single node in an ANN do?
x1
x2
x3
x4
x5
w12
w22
w32
w42
w52
5
y2
5


exp  b   wi 2 xi 
i 1


w
i 1
x b
i2 i
More Nodes
yj 
x1
x2
x3
x4
1
6
6


1  exp  b j   wij xij  b  w x
i3 i3
y1
i 1

 3 
i 1
y2
y3
y4
out
put
Output
layer
x5
x6
Input layer
b4   wi 4 xi 4
i 1
6
6
b1   wi1 xi1
i 1
Hidden layer 1 if inside, 0
if outside
the closed
region
6
b2   wi 2 xi 2
i 1
Number of Hidden Layers

There must be two hidden layers to identify
the following annulus.
A neural network is basically
a function approximator,
which can approximate
continuous functions by
piecewise linear functions
(interpolation). Neural
networks are also known as
universal approximator.
Separation or Classification

A separation or classification is nothing but
approximating the surface separating the
(mixed) data. In other words it approximates
a continuous function generating the
separating surface.
A classifier will have
to approximate the
function whose graph
is this curve.
Classification by ANN


Most classification tasks are accomplished
by separating the data with curve(s)
consisting only a single line. Therefore for
most classification tasks ANNs with a single
hidden layer is sufficient.
However number of nodes in the hidden
layer is to be determined by trial and error for
optimal classification.
Universal Approximation

For any continuous mapping f :[0,1]n  n  m
there must exist a three-layer neural network
(having an input or ‘fanout’ layer with n
processing elements, a hidden layer with 2n
+ 1 processing elements, and an output layer
with m processing elements) that implements
f exactly. Hecht-Nielsen, 1988.
Duda et al., Chapter 6, p. 283 & 289
Backpropagation Neural Network




By far the most widely used type of neural
network.
It is simple yet powerful neural network even
for complex models having hundred of
thousands of parameters.
Its conceptual simplicity and high success
rate makes it a mainstay in adaptive pattern
recognition.
Offers means to calculate input to hidden
layer weights.
Regularization

It is a deep issue concerning complexity of
the network. Number of input and output
nodes is fixed. But number of hidden nodes
and connection weights are not. These are
free parameters. If there are too few of them
the training set cannot be adequately
learned. If there are too many of them,
generalization of the network will be poor
Regularization (cont.)
(apart from enhanced computational
complexity). That is, its performance on the
test data set will fall down (while on training
data set its performance may remain very
high).
Training seizure pattern
Testing seizure pattern
Hecht-Nielsen, 1988
Backpropagation Architecture
Three layer
General
y1
x1
y2
x2
x3
x4
Hecht-Nielsen, 1988
Backpropagation Architecture
(cont.)
Backpropagation Algorithm
1 c
1
2
J (w )   (tk  zk )  t  z
2 k 1
2
J
w  
w
2
has to be minimized, where t and z
are target and network output vectors
respectively. c is # output nodes.
 is the learning rate.
where
w (m  1)  w (m)  w (m)
m stands for the m’th iteration.
Subasi and Ercelebi, Comp. Meth. Progr. Biomed., 78: 87 – 99, 2005
Epileptic EEG Signal
http://en.wikipedia.org/wiki/Daubechies_wavelet
DB4 Wavelet
DB wavelets do not
have closed form
representation
(cannot be
expressed by an
elegant mathematical
formula, like Morlet
wavelet).
http://www.bearcave.com/misl/misl_tech/wavelets/daubechies/index.html
DB4 Wavelet Generation: Cascade
Algorithm
 (t ) 


g (n) 2 k (2t  n)
n 
N 1
 k 1 (t )   h(n) 2 k (2t  n)
g(n), h(n) are impulse response
functions. Ψ(t) is the wavelet. DB4
will contain only 4 taps or
coefficients.
n 0
1 3
3  3 g (0)  h(3) g (2)  h(1)
h(0) 
h(2) 
4 2
4 2 g (1)  h(2) g (3)  h(0)
1 3
3 3
h(3) 
h(1) 
4 2
4 2
EEG Data





Electrode placement was according to 10 –
20 system.
4 signals selected as F7 – C3, F8 – C4, T5 –
O1 and T6 – O2.
Sample frequency 200 Hz.
Band-pass filtered in 1 – 70 Hz range upon
acquisition.
EEG was segmented at 1000 time point
window (5s).
Feature Extraction by DB4
Wavelets
EEG signals decomposed by
high-pass (called ‘detail
signal’) and low-pass (called
‘approximation’) FIR filtering
Assignment


Preprocess depth EEG signals (to be given)
by wavelet transforms (DB4 wavelet is seen
to be more efficient than other wavelets, see
Subasi & Ercelebi, 2005 and Vardhan &
Majumdar, 2011). This will extract features
from the signals.
Use a three layer (that is, with only one
hidden layer) perceptron neural network to
Assignment (cont.)
classify the features to separate out the seizure
portion from non-seizure portion in the
signals.
References


A. Subasi and E. Ercelebi, Classification of
EEG signals using neural networks and
logistic regression, Comp. Meth. Progrm.
Biomedicine, 78: 87 – 99, 2005.
I. Kaplan, Daubechies D4 wavelet transform,
http://www.bearcave.com/misl/misl_tech/wav
elets/daubechies/index.html
References (cont.)


R. Hecht-Nielsen, Theory of the
backpropagation neural network, INNS 1988,
p. I-593 – I-605. Freely available at
http://s112088960.onlinehome.us/annProject
s/Research%20Paper%20Library/backPropT
heory.pdf
I. Daubechies, Ten lectures on wavelets,
SIAM, 1992. p. 115, 132, 194, 242.
THANK YOU
This lecture is available at http://www.isibang.ac.in/~kaushik