Transcript Widrow-Hoff Learning

Widrow-Hoff Learning
Outline
•
•
•
•
•
•
1 Introduction
2 ADALINE Network
3 Mean Square Error
4 LMS Algorithm
5 Analysis of Converge
6 Adaptive Filtering
Introduction
• In 1960, Bernard Widrow and his doctoral
student Marcian Hoff introduced the ADALINE
(ADAptive LInear NEuron)network and
LMS(Least Mean Square) algorithm.
Perceptron Network
• Figure: a=hardlim(Wp+b)
ADALINE Network
• Figure: a=purelin(Wp+b)=Wp+b
Single ADALINE
decision boundary
Mean Square Error
Mean Square Error(conti.)
Mean Square Error(conti.)
Error analysis
𝐹 𝐱 =𝑐+
𝐝𝑇 𝐱 +
𝟏 𝑻
𝐱 𝐀𝐱
𝟐
Error analysis(conti.)
d = -2h and A = 2R
definite
=0
Example 1
Example 1(conti.)
Example 1(conti.)
Approximate Steepest Descent
Approximate Gradient
Approximate Gradient(conti.)
Approximate Gradient(conti.)
LMS Algorithm
LMS Algorithm (conti.)
Example 2
Example 2(conti.)
, W(0)=
1𝑤
𝑇
= 0 0
Example 2(conti.)
Example 2(conti.)
Example 2(conti.)
Analysis of Convergence
Analysis of Convergence(conti.)
Analysis of Convergence(conti.)
Example 3
Perceptron rule V.S. LMS algorithm
Perceptron rule V.S. LMS
algorithm(conti.)
Perceptron rule V.S. LMS
algorithm(conti.)
Perceptron rule V.S. LMS
algorithm(conti.)
Adaptive Filtering
Tapped Delay Line
Adaptive Filter
Adaptive Noise Cancellation