Transcript Artificial Neural Networks

Artificial Neural Networks
人工神经网络
Introduction
Table of Contents
• Introduction to ANNs
– Taxonomy
– Features
– Learning
– Applications
I
• Supervised ANNs
– Examples
– Applications
– Further topics
• Unsupervised ANNs
– Examples
– Applications
– Further topics
II
18/07/2015
Artificial Neural Networks - I
III
2
Contents - I
• Introduction to ANNs
– Processing elements (neurons)
– Architecture
•
•
•
•
•
Functional Taxonomy of ANNs
Structural Taxonomy of ANNs
Features
Learning Paradigms
Applications
18/07/2015
Artificial Neural Networks - I
3
The Biological Neuron
• 10 billion neurons in human brain
• Summation of input stimuli
– Spatial (signals)
– Temporal (pulses)
• Threshold over composed inputs
• Constant firing strength
18/07/2015
6
• 10 billion synapses in human brain
• Chemical transmission and
modulation of signals
• Inhibitory synapses
• Excitatory synapses
Artificial Neural Networks - I
4
Biological Neural Networks
• 10,000 synapses per
neuron
• Computational power =
connectivity
• Plasticity
– new connections (?)
– strength of connections
modified
18/07/2015
Artificial Neural Networks - I
5
Neural Dynamics
40
mV
membrane
rest
activation
20
0
Action potential
-20
Action potential ≈ 100mV
Activation threshold ≈ 20-30mV
Rest potential ≈ -65mV
Spike time ≈ 1-2ms
Refractory time ≈ 10-20ms
-40
-60
-80
Refractory time
-100
ms
-120
0
10
18/07/2015
20
30
40
50
60
70
80
90
100
Artificial Neural Networks - I
6
神经网络的复杂性
• 神经网路的复杂多样，不仅在于神经元和突触
的数量大、组合方式复杂和联系广泛，还在于
突触传递的机制复杂。现在已经发现和阐明的
突触传递机制有：突触后兴奋，突触后抑制，
突触前抑制，突触前兴奋，以及“远程”抑制
等等。在突触传递机制中，释放神经递质是实
现突触传递机能的中心环节，而不同的神经递
质有着不同的作用性质和特点
18/07/2015
Artificial Neural Networks - I
7
神经网络的研究
• 神经系统活动，不论是感觉、运动，还是脑的
高级功能（如学习、记忆、情绪等）都有整体
上的表现，面对这种表现的神经基础和机理的
分析不可避免地会涉及各种层次。这些不同层
次的研究互相启示，互相推动。在低层次（细
胞、分子水平）上的工作为较高层次的观察提
供分析的基础，而较高层次的观察又有助于引
导低层次工作的方向和体现其功能意义。既有
物理的、化学的、生理的、心理的分门别类研
究，又有综合研究。
18/07/2015
Artificial Neural Networks - I
8
The Artificial Neuron
Stimulus
ui t    wij  x j t 
j
x1(t)
x2(t)
x3(t)
Response
yi t   f urest  ui t 
x4(t)
x5(t)
wi1
wi2
wi3
wi4
w
j
ij
 x j (t )
yi  f (ui (t))
yi(t)
wi5
Neuron i
urest = resting potential
xj(t) = output of neuron j at time t
wij = connection strength between neuron i and neuron j
u(t) = total stimulus at time t
18/07/2015
Artificial Neural Networks - I
9
Artificial Neural Models
• McCulloch Pitts-type Neurons (static)
– Digital neurons: activation state interpretation
(snapshot of the system each time a unit fires)
– Analog neurons: firing rate interpretation
(activation of units equal to firing rate)
– Activation of neurons encodes information
• Spiking Neurons (dynamic)
– Firing pattern interpretation (spike trains of
units)
– Timing of spike trains encodes information
(time to first spike, phase of signal, correlation
and synchronicity
18/07/2015
Artificial Neural Networks - I
10
Binary Neurons
hard threshold
1.2
Stimulus
output
1
ui   wij  x j
on
0.8
0.6
Response
yi  f urest  ui 
j
0.4
0.2
input
0
-0.2 -10
-8
-6
-4
-2
0
2
4
6
8
10
“Hard” threshold
-0.4
-0.6
heaviside
-0.8
-1
off
-1.2
 z    ON 


f z   

 else  OFF 


= threshold
• ex: Perceptrons, Hopfield NNs, Boltzmann
Machines
• Main drawbacks: can only map binary
functions, biologically implausible.
18/07/2015
Artificial Neural Networks - I
11
Analog Neurons
sigmoid
1.2
Stimulus
output
on
1
ui   wij  x j
0.8
0.6
Response
yi  f urest  ui 
j
0.4
0.2
input
0
-0.2 -10
-0.4
-8
-6
-4
-2
0
2
4
6
8
10
2/(1+exp(-x))-1
“Soft” threshold
-0.6
-0.8
-1
-1.2
off
f z  
2
1
1  e z
• ex: MLPs, Recurrent NNs, RBF NNs...
• Main drawbacks: difficult to process time
patterns, biologically implausible.
18/07/2015
Artificial Neural Networks - I
12
Spiking Neurons
Stimulus
 = spike and afterspike potential
urest = resting potential
(t,u() = trace at time t of input at time 
= threshold
xj(t) = output of neuron j at time t
wij = efficacy of synapse from neuron i to
neuron j
u(t) = input stimulus at time t
ui t    wij  x j t 
j
Response


yi (t )  f urest   (t  t f )   0  t , ui  
t
dz


z


&

0

ON


dt
f z   



else

OFF


18/07/2015
Artificial Neural Networks - I
13
Spiking Neuron Dynamics
neuron output
2.5
y(t)

urest+(t-tf)
V
2
1.5
1
0.5
t
0
0
10
20
30
40
50
60
70
80
90
100
-0.5
-1
18/07/2015
Artificial Neural Networks - I
14
赫布律
加拿大心理学家Donald Hebb出版了《行为的组
织》一书，指出学习导致突触的联系强度和传
递效能的提高，即为“赫布律”。
在此基础上，人们提出了各种学习规则和算法，
以适应不同网络模型的需要。有效的学习算法，
使得神经网络能够通过连接权值的调整，构造
客观世界的内在表示，形成具有特色的信息处
理方法，信息存储和处理体现在网络的连接中。
18/07/2015
Artificial Neural Networks - I
15
Hebb’s Postulate of Learning
• Biological formulation
• When an axon of cell A is near enough
to excite a cell and repeatedly or
persistently takes part in firing it, some
growth process or metabolic change takes
place in one or both cells such that A’s
efficiency as one of the cells firing B is
increased.
18/07/2015
Artificial Neural Networks - I
16
赫布律
• 当细胞A的一个轴突和细胞B 很近，足以对它产
生影响，并且持久地、不断地参与了对细胞B 的
兴奋，那么在这两个细胞或其中之一会发生某
种生长过程或新陈代谢变化，以致于A作为能使
B 兴奋的细胞之一，它的影响加强了。
18/07/2015
Artificial Neural Networks - I
17
Hebb’s Postulate: revisited
•
•
•
Stent (1973), and Changeux and Danchin (1976)
have expanded Hebb’s rule such that it also models inhibitory synapses:
1. If two neurons on either side of a synapse are
activated simultaneously (synchronously), then
the strength of that synapse is selectively
increased.
2. If two neurons on either side of a synapse are
activated asynchronously, then that synapse is
selectively weakened or eliminated.
18/07/2015
Artificial Neural Networks - I
18
Artificial Neural Networks
Output layer
fully connected
Hidden layers
Input layer
sparsely connected
18/07/2015
Artificial Neural Networks - I
19
Feedforward ANN Architectures
•
•
•
•
•
18/07/2015
Information flow unidirectional
Static mapping: y=f(x)
Multi-Layer Perceptron (MLP)
Radial Basis Function (RBF)
Kohonen Self-Organising Map
(SOM)
Artificial Neural Networks - I
20
Recurrent ANN Architectures
• Feedback connections
• Dynamic memory:
y(t+1)=f(x(τ),y(τ),s(τ)) τ(t,t-1,...)
• Jordan/Elman ANNs
• Hopfield
• Adaptive Resonance Theory (ART)
18/07/2015
Artificial Neural Networks - I
21
History
• Early stages
–
–
–
–
–
1943
1948
1949
1958
1960
McCulloch-Pitts: neuron as comp. elem.
Wiener: cybernatics
Hebb: learning rule
Rosenblatt: perceptron
Widrow-Hoff: least mean square algorithm
• Recession
– 1969 Minsky-Papert: limitations perceptron model
• Revival
– 1982 Hopfield: recurrent network model
– 1982 Kohonen: self-organizing maps
– 1986 Rumelhart et. al.: backpropagation
18/07/2015
Artificial Neural Networks - I
22
历史
• 40年代心理学家Mcculloch和数学家Pitts合作提出的兴奋
与抑制型神经元模型和Hebb提出的神经元连接强度的修
改规则，他们的研究结果至今仍是许多神经网络模型研
究的基础。
• 50年代、60年代的代表性工作是Rosenblatt的感知机和
Widrow的自适应性元件Adaline。
• 1969年，Minsky和Papert合作发表了颇有影响的
Perceptron一书，得出了消极悲观的论点，加上数字计算
机正处于全盛时期并在人工智能领域取得显著成就，70
年代人工神经网络的研究处于低潮。
18/07/2015
Artificial Neural Networks - I
23
历史
• 80年代后，传统的Von Neumann数字计算机在模拟视听
觉的人工智能方面遇到了物理上不可逾越的极限。与此
同时，Rumelhart与Mcclelland以及Hopfield等人在神经网
络领域取得了突破性进展，神经网络的热潮再次掀起。
•
•
•
•
•
自适应共振理论（ART）
组织特征映射理论
Hinton 等人最近提出了 Helmboltz 机
徐雷提出的 Ying-Yang 机理论模型
甘利俊一( S.Amari) 开创和发展的基于统计流形的方法
应用于人工神经网络的研究
18/07/2015
Artificial Neural Networks - I
24
ANN Capabilities
•
•
•
•
•
•
•
Learning
Approximate reasoning
Generalisation capability
Noise filtering
Parallel processing
Distributed knowledge base
Fault tolerance
18/07/2015
Artificial Neural Networks - I
25
Main Problems with ANN
• Knowledge base not transparent
(black box) (Partially resolved)
• Learning sometimes difficult/slow
• Limited storage capability
18/07/2015
Artificial Neural Networks - I
26
ANN Learning Paradigms
• Supervised learning
–
–
–
–
Classification
Control
Function approximation
Associative memory
• Unsupervised learning
– Clustering
• Reinforcement learning
– Control
18/07/2015
Artificial Neural Networks - I
27
Supervised Learning
• Teacher presents ANN input-output
pairs
• ANN weights adjusted according to
error
• Iterative algorithms (e.g. Delta rule,
BP rule)
• One-shot learning (Hopfield)
• Quality of training examples is
critical
18/07/2015
Artificial Neural Networks - I
28
Linear Separability in Perceptrons
18/07/2015
Artificial Neural Networks - I
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
29
Learning Linearly Separable Functions (1)
What can these functions learn ?
Bad news:
- There are not many linearly separable functions.
Good news:
- There is a perceptron algorithm that will learn
any linearly separable function, given enough
training examples.
18/07/2015
Artificial Neural Networks - I
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
30
Delta Rule
e  di  yi
wij    e  x j
=learning coefficient
wij=connection from neuron xj to yi
x=(x1,x2,...,xn) ANN input
y=(y1,y2,...,yn) ANN output
d=(d1,d2,...,dn) desired output
(x,d) training example
e=ANN error
• a.k.a. Least Mean Squares
• Widrow-Hoff iterative delta rule
(1960)
• Gradient descent of the error surface
• Guaranteed to find minimum error
configuration in single layer ANNs
• Stochastic approximation of desired
behaviour
18/07/2015
Artificial Neural Networks - I
y1
w11
x1
w12
x2
y2
y3
w13
w14
x3
x4
31
Unsupervised Learning
• ANN adapts weights to cluster input
data
• Hebbian learning
– Connection stimulus-response
strengthened (hebbian)
• Competitive learning algorithms
– Kohonen & ART
– Input weights adjusted to resemble
stimulus
18/07/2015
Artificial Neural Networks - I
32
Hebbian Learning
General Formulation
Hebb postulate
d
wij  F wij , yi , x j 
dt
Kohonen & Grossberg (ART)
F    yi  x j
F    yi  x j  wij 
d
wij    yi  x j
dt
d
wij    yi  x j  wij 
dt
=learning coefficient
wij=connection from neuron xj to yi
y1
• Hebb postulate (1948)
• Correlation-based learning
• Connections between concurrently
firing neurons are strengthened
• Experimentally verified (1973)
w11
18/07/2015
Artificial Neural Networks - I
x1
w12
x2
33
Learning principle for
artificial neural networks
ENERGY MINIMIZATION
We need an appropriate definition of energy for artificial
neural networks, and having that we can use
mathematical optimisation techniques to find how to
change the weights of the synaptic connections between
neurons.
ENERGY = measure of task performance error
18/07/2015
Artificial Neural Networks - I
34
Neural network mathematics
Inputs
Output
 y11  2
1
2
 y 32 
 1  y1  f ( y , w1 )
 2
2
3
y 12  f ( x 2 , w12 ) 1  y 2  2
2
1
2
y   1  y 2  f ( y , w2 ) y   y3  yOut  f ( y , w1 )
 2 
y 31  f ( x3 , w31 )
 y3  y 2  f ( y1 , w 2 )
y3 

1
3
3

y4 
1
1

y

f
(
x
,
w
)
18/07/2015
Artificial Neural Networks - I
35
4
4
4
y11  f ( x1 , w11 )
Neural network mathematics
Neural network: input / output transformation
yout  F ( x,W )
W is the matrix of all weight vectors.
18/07/2015
Artificial Neural Networks - I
36
MLP neural networks
MLP = multi-layer perceptron
Perceptron:
yout  wT x
x
yout
MLP neural network:
1
y 1k 
 w1 k T x  a1k
1 e
y 1  ( y11 , y 12 , y31 ) T
, k  1,2,3
1
y k2 
 w 2 k T y 1  a k2
1 e
y 2  ( y12 , y 22 ) T
, k  1,2
x
yout
2
y out   wk3 y k2  w3T y 2
k 1
18/07/2015
Artificial Neural Networks - I
37
RBF neural networks
RBF = radial basis function
r ( x)  r (|| x  c ||)
Example:
4
f ( x)  e
yout   wk2  e
k 1
18/07/2015
|| x  w1,k || 2

2( ak ) 2
|| x  w|| 2

2a 2
Gaussian RBF
x
Artificial Neural Networks - I
yout
38
Neural network tasks
• control
• classification
• prediction
• approximation
These can be reformulated
in general as
FUNCTION
APPROXIMATION
tasks.
Approximation: given a set of values of a function g(x)
build a neural network that approximates the g(x) values
for any input x.
18/07/2015
Artificial Neural Networks - I
39
Neural network approximation
Task specification:
Data: set of value pairs: (xt, yt), yt=g(xt) + zt; zt is random
measurement noise.
Objective: find a neural network that represents the input /
output transformation (a function) F(x,W) such that
F(x,W) approximates g(x) for every x
18/07/2015
Artificial Neural Networks - I
40
Learning to approximate
Error measure:
1 N
E   ( F ( xt ;W )  yt ) 2
N t 1
Rule for changing the synaptic weights:
E
wi  c 
(W )
j
wi
j
wij , new  wij  wij
c is the learning parameter (usually a constant)
18/07/2015
Artificial Neural Networks - I
41
Learning with a perceptron
Perceptron:
yout  wT x
1
2
N
Data: ( x , y1 ), ( x , y2 ),..., ( x , y N )
2
T t
2
E
(
t
)

(
y
(
t
)

y
)

(
w
(
t
)
x

y
)
Error:
out
t
t
Learning:
( w(t )T x t  yt ) 2
E (t )
wi (t  1)  wi (t )  c 
 wi (t )  c 
wi
wi
wi (t  1)  wi (t )  c  ( w(t )T x t  yt )  xit
m
w(t ) x   w j (t )  x tj
T
j 1
A perceptron is able to learn a linear function.
18/07/2015
Artificial Neural Networks - I
42
Learning with RBF neural
networks
M
2
RBF neural network: yout  F ( x,W )   wk  e
|| x  w1,k || 2

2 ( ak ) 2
k 1
1
2
N
Data: ( x , y1 ), ( x , y2 ),..., ( x , y N )
M
Error: E (t )  ( y(t ) out  yt )  ( wk2 (t )  e
2
|| x t  w1,k || 2

2( ak ) 2
k 1
Learning:
 yt ) 2
E (t )
w (t  1)  w (t )  c 
wi2
2
i
2
i
E (t )
t

2

(
F
(
x
, W (t ))  yt )  e
2
wi
|| x t  w1,i || 2

2 ( ai ) 2
Only the synaptic weights of the output neuron are modified.
An RBF neural network learns a nonlinear function.
18/07/2015
Artificial Neural Networks - I
43
Learning with MLP neural
networks
y 1k 
 w1 k T x  a1k
1 e
y 1  ( y11 ,..., y 1M ) T
MLP neural network:
, k  1,..., M 1
1
with p layers
x
1
yout
y k2 
1
 w 2 k T y 1  a k2
1 e
y 2  ( y12 ,..., y M2 ) T
, k  1,..., M 2
2
...
y out  F ( x;W )  w pT y p 1
1 2 … p-1 p
1
2
N
(
x
,
y
),
(
x
,
y
),...,
(
x
, yN )
Data:
1
2
Error: E(t )  ( y(t ) out  yt ) 2  ( F ( x t ;W )  yt ) 2
18/07/2015
Artificial Neural Networks - I
44
Learning with backpropagation
Learning: Apply the chain rule for differentiation:
• calculate first the changes for the synaptic weights
of the output neuron;
• calculate the changes backward starting from layer
p-1, and propagate backward the local error terms.
The method is still relatively complicated but it
is much simpler than the original optimisation
problem.
18/07/2015
Artificial Neural Networks - I
45
Learning with general optimization
In general it is enough to have a single layer of nonlinear
neurons in a neural network in order to learn to
approximate a nonlinear function.
In such case general optimisation may be applied without
too much difficulty.
Example: an MLP neural network with a single hidden layer:
M
yout  F ( x;W )   w 
k 1
18/07/2015
2
k
1
1 e
Artificial Neural Networks - I
 w1,kT x  ak
46
Learning with general optimization
Synaptic weight change rules for the output neuron:
wi2 (t  1)  wi2 (t )  c 
E (t )
wi2
E (t )
1
t

2

(
F
(
x
,
W
(
t
))

y
)

t
wi2
1  ew
1,iT
x t  ai
Synaptic weight change rules for the neurons of the
hidden layer: w (t  1)  w (t )  c  Ew(t )
1, i
j
1, i
j
1, i
j
E (t )

 2  ( F ( x t , W (t ))  yt )  1,i
1, i
w j
w j

w1j,i
1


w
1 e
1,iT

ew

 
 1  ew
1,iT
x  ai
t

1


w
1 e
1,iT
x t  ai
1,iT
x  ai
t

2

x  ai
t





 w1,iT x t  ai
1, i
w j



 w1,iT x t  ai   x tj
1, i
w j
ew
1,iT
w (t  1)  w (t )  c  2  ( F ( x , W (t ))  yt ) 
1, i
j
18/07/2015
1, i
j
t
Artificial Neural Networks - I
1  e
w
x t  ai
1,iT
x  ai
t

2
 ( x tj )
47
New methods for learning with
neural networks
Bayesian learning:
the distribution of the neural network
parameters is learnt
Support vector learning:
the minimal representative subset of the
available data is used to calculate the synaptic
weights of the neurons
18/07/2015
Artificial Neural Networks - I
48
Reinforcement Learning
•
•
•
•
•
•
Sequential tasks
Desired action may not be known
Critic evaluation of ANN behaviour
Weights adjusted according to critic
May require credit assignment
Population-based learning
– Evolutionary Algorithms
– Swarming Techniques
– Immune Networks
18/07/2015
Artificial Neural Networks - I
49
ANN Summary
Artificial Neural Networks
Feedforward
Recurrent
Unsupervised
Supervised
Unsupervised
Supervised
(Kohonen)
(MLP, RBF)
(ART)
(Elman, Jordan,
Hopfield)
18/07/2015
Artificial Neural Networks - I
50
神经网络的集成
• 1996年，Sollich和Krogh 将神经网络集成定义为：
“神经网络集成是用有限个神经网络对同一个
问题进行学习，集成在某输入示例下的输出由
构成集成的各神经网络在该示例下的输出共同
决定”。
18/07/2015
Artificial Neural Networks - I
51
ANN Application Areas
•
•
•
•
•
Classification
Clustering
Associative memory
Control
Function approximation
18/07/2015
Artificial Neural Networks - I
52
ANN Classifier systems
•
•
•
•
•
•
Learning capability
Statistical classifier systems
Data driven
Generalisation capability
Handle and filter large input data
Reconstruct noisy and incomplete
patterns
• Classification rules not transparent
18/07/2015
Artificial Neural Networks - I
53
Applications for ANN Classifiers
• Pattern recognition
–
–
–
–
–
–
Industrial inspection
Fault diagnosis
Image recognition
Target recognition
Speech recognition
Natural language processing
• Character recognition
– Handwriting recognition
– Automatic text-to-speech conversion
18/07/2015
Artificial Neural Networks - I
54
Clustering with ANNs
• Fast parallel distributed processing
• Handle large input information
• Robust to noise and incomplete
patterns
• Data driven
• Plasticity/Adaptation
• Visualisation of results
• Accuracy sometimes poor
18/07/2015
Artificial Neural Networks - I
55
ANN Clustering Applications
• Natural language processing
– Document clustering
– Document retrieval
– Automatic query
• Image segmentation
• Data mining
– Data set partitioning
– Detection of emerging clusters
• Fuzzy partitioning
• Condition-action association
18/07/2015
Artificial Neural Networks - I
56
Associative ANN Memories
•
•
•
•
•
Stimulus-response association
Auto-associative memory
Content addressable memory
Fast parallel distributed processing
Robust to noise and incomplete
patterns
• Limited storage capability
18/07/2015
Artificial Neural Networks - I
57
Application of ANN Associative
Memories
•
•
•
•
•
Character recognition
Handwriting recognition
Noise filtering
Data compression
Information retrieval
18/07/2015
Artificial Neural Networks - I
58
ANN Control Systems
•
•
•
•
•
•
•
•
•
•
Learning/adaptation capability
Data driven
Non-linear mapping
Fast response
Fault tolerance
Generalisation capability
Handle and filter large input data
Reconstruct noisy and incomplete patterns
Control rules not transparent
Learning may be problematic
18/07/2015
Artificial Neural Networks - I
59
ANN Control Schemes
• ANN controller
• conventional controller + ANN for
unknown or non-linear dynamics
• Indirect control schemes
– ANN models direct plant dynamics
– ANN models inverse plant dynamics
18/07/2015
Artificial Neural Networks - I
60
ANN Control Applications
• Non-linear process control
–
–
–
–
Chemical reaction control
Industrial process control
Water treatment
Intensive care of patients
• Servo control
– Robot manipulators
– Autonomous vehicles
– Automotive control
• Dynamic system control
– Helicopter flight control
– Underwater robot control
18/07/2015
Artificial Neural Networks - I
61
ANN Function Modelling
•
•
•
•
•
•
•
•
ANN as universal function approximator
Dynamic system modelling
Learning capability
Data driven
Non-linear mapping
Generalisation capability
Handle and filter large input data
Reconstruct noisy and incomplete inputs
18/07/2015
Artificial Neural Networks - I
62
ANN Modelling Applications
• Modelling of highly nonlinear
industrial processes
• Financial market prediction
• Weather forecasts
• River flow prediction
• Fault/breakage prediction
• Monitoring of critically ill patients
18/07/2015
Artificial Neural Networks - I
63
Neural Network Approaches
ALVINN - Autonomous Land Vehicle In a Neural Network
18/07/2015
Artificial Neural Networks - I
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
64
Neural Network Approaches
- Developed in 1993.
Output units
Hidden layer
- Performs driving with
Neural Networks.
- An intelligent VLSI image
sensor for road following.
Input units
18/07/2015
- Learns to filter out image
details not relevant to
driving.
Artificial Neural Networks - I
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
65
人脸识别
18/07/2015
Artificial Neural Networks - I
66
手写数字识别
18/07/2015
Artificial Neural Networks - I
67
Summary
• Artificial neural networks are inspired by the learning
processes that take place in biological systems.
• Artificial neurons and neural networks try to imitate the
working mechanisms of their biological counterparts.
• Learning can be perceived as an optimisation process.
• Biological neural learning happens by the modification
of the synaptic strength. Artificial neural networks learn
in the same way.
• The synapse strength modification rules for artificial
neural networks can be derived by applying mathematical
optimisation methods.
18/07/2015
Artificial Neural Networks - I
68
References
• B. Kosko (1992), Neural Networks and
Fuzzy Systems, Prentice-Hall Int. Ed.
• R. Rojas (1992), Neural Networks,
Springer Verlag
• P. Mehra and B. W. Wah 1992, Artificial
Neural Networks: Concepts And Theory,
Ieee Computer Society Press.
• M. H. Hassoun (1995), Fundamentals of
Artificial Neural Networks, MIT Press.
18/07/2015
Artificial Neural Networks - I
69