Transcript Pulsed Neural Nets (ppt)

Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
Pulsed Neural Networks
Neil E. Cotter
ECE Department
University of Utah
Overview
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Challenging Problems
Artificial Neural Networks
Learning
Dynamic Networks
Pulsed Networks
Applications
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Challenging Problems
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Challenging Problems
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
Image recognition in varied settings
Speech recognition in noise
Robotic control and navigation
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Artificial Neural Networks
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Biological Neuron
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
t0
Biological Neuron
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
w0
t
t1

w1
t
t2
w2
t
T
t
tn
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Perceptron
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
w0
x0
w1
x1

w2
x2

y
y
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Perceptron Response
y(x1, x2)
x2
y=1
y=0
x1
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Linear Separability
0 0
0
1
0
0
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
1
1
1
0
11
1
1
1
0
0
0
0
0
1
x2
1
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Linear Separability
0 0
0
1
0
0
QuickT ime ™an d a
kT i me™
and a ssor
TIFF ( Uncomp res sed) Quic
deco
mpre
dec om pres s or
are needed
tpictur
o s ee thi s e.
pi c ture.
ar e need ed to see
this
1
1
1
0
11
1
1
1
0
0
0
0
0
1
x2
1
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Logic Gates
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
x1
0
1
(0, 1)
(1, 1)
y=1
y=0
0
0
(0, 0)
(1, 0)
x2
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Parallel Processing
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Sigmoid Neuron
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
w0
x0
w1
x1

w2
x2

y
y
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Sigmoid Neuron Response
, x2)
x2
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
y(x1
x1
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
u1
uL
Neural Network
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
Neuron
Neuron
wt
Neuron
Neuron
wt
Neuron
Neuron
wt
Neuron
Neuron
wt
y1
yM
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
u1
uL
Neural Network
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
Line
AND gate
wt
Line
AND gate
wt
Line
AND gate
wt
Line
AND gate
wt
y1
yM
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Universal Approximation
y
x2
x1
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Universal Approximation
y
x2
01
0
1
1
0
x1
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Universal Approximation
y
x2
x1
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Function Approximations
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
Learning
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Gradient Descent
, w2)
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
w2
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
E(w1
w1
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Local Minima
, w2)
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
w2
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
E(w1
w1
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Backward-Error Propagation
, w2)
w2
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
E(w1
w1
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
Dynamic Networks
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Dynamic Networks
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
Neural Network
•dt
•dt
x1
xN
u1
y1
uL
yM
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
State
0 or 1
1
Learning through Time
v1
x
noise
v16
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
Adaptive Critic
p
filter
reward = ±1
adaptation
w1
x16
x x 
noise
Associative Search

y
w16
±F
x
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
State
0 or 1
x1
Learning through Time
v1
noise
v16
w1
x x 
x16
w16
Adaptive Critic
p
delay
reward =
±1
l
decay
a
b
noise
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
^r
Associative Search

y
±
F
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
Pulsed Networks
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
t0
t
Pulsed Neuron
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
w0
t
t1
t

w1
t
t
t2
w2
t
t
T
t
tn
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Pattern Processing
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
T
t
t0
t1
t2
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
0
x2
Response Regions
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
Perceptron
Pulsed Neuron
x3
x3
0
0
1
1
0
0
x1
x2
0
x1
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Pattern Windows
t0
t1
t2
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
t
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Pattern Windows
t0
t1
t2
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
t
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Dynamic Pulsed Networks
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
Pulsed Neural Network
x1
xN
u1
y1
uL
yM
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
Applications
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Speech Recognition
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
t

Sound
Window
Spectrum
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Auditory System
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
Basilar Membrane
Sound
t
t
Quic k Tim e ™ and a
dec om pre s s or
are neede d to s e e this pic ture .
Research Opportunities
Mathematically describe pattern processing
Devise pattern-learning algorithms
Build circuits
Quic kT i me™ and a
dec om pres s or
are needed t o s ee thi s pi c ture.
Title
QuickTi me™ and a
TIFF ( Uncompressed) decompressor
are needed to see thi s pi ctur e.