Transcript Locally Weighted ELM

1





Experimental Results
ELM
Weighted ELM
Locally Weighted ELM
Problem
2




All training data are randomly chosen
Targets are normalize -1 to 1
Features are normalize 0 to 1
Using RMSE criterion
K
RMSE 

( yˆi  yi ) 2
i 1
K
3





sin(x)
Sinc function:
X=-10:0.05:10 x
Train:351
Test:50
(hidden neuron, h, k)
Original ELM Weighted ELM
(10)
(10,0.01)
1.95E-1
9.41E-5
Locally
Weighted
ELM
(10,1,20)
1.53E-4
4
5





Function: y  1.1 (1  x  2x )  e
X=-5:0.05:5
Train:151
Test:50
(hidden neuron, h, k)
2
x2 / 2
Original
ELM
(10)
Weighted
ELM
(10,0.01)
Locally
Weighted
ELM
(10,1,20)
T2FNN
2.81E-1
1.39E-4
8.15E-4
1.3E-3
6
7





 x22
3
y  x1  e
 (1  x32 )
Function:
X1,x2,x3=-1:0.005:1
Train:351
Test:50
(hidden neuron, h, k)
Original ELM Weighted ELM
(10)
(10,0.01)
1.41E-4
3.09E-6
Locally
Weighted
ELM
(10,1,20)
2.61E-5
8





Machine CPU
Feature:6
Train:100
Test:109
(hidden neuron, h, k)
Original ELM Weighted ELM
Locally
(10)
(10,0.9)
Weighted ELM
(10,1,40)
0.111342
0.103473
0.105663
9





Auto Price
Feature:15 ,1 nominal ,14 continuous
Train:80
Test:79
(hidden neuron, h, k)
Original ELM Weighted ELM
(15)
(10,0.9)
0.201255
0.189584
Locally
Weighted
ELM
(10,0.9,50)
0.193568
10





Cancer
Feature:32
Train:100
Test:94
(hidden neuron, h, k)
Original ELM Weighted ELM
(10)
(3,0.9)
0.533656
0.528415
Locally
Weighted
ELM
(3,1,40)
0.532317
11
Hβ  T
 g (w1x1  b1 )  g (w j x1  b j ) 


H




 g (w x  b )  g (w x  b )
1 N
1
j N
j 

min Hβ  T
β
β  (HT H)1 HT T
H : N  j , hidden layer output matrix
Input
layer
hidden
layer
output
layer
β : j  m , theoutput weight matrix
T : N  m , target
The weights between input layer and hidden layer and the biases of neurons
in the hidden layer are randomly chosen. weight [1,1] , bias [0,1]
12
p
dn 

( xa , i  xn , i ) 2 , n  1 ~ N
i 1
p : the number of feature
n : then th trainingdata
a : thea th testingdata
wnn  exp(0.5  (d n / h) 2 )
0 
 w11 


W  
  , diagonal matrix
 0  wNN 
13
min W Hβ  W T
β
W Hβ  W T
β  (( W H)T W H)1( W H)T W T
14

Ex
X  [0.1;0.2;0.4]
0.4750
0.5 


 
H  0.4502 T  0.7 
0.4013
0.9 
β  (HT H ) 1 HT T  1.5505
0.2365 


Hβ  T   0.002   0.3648
 0.2778
15
假設0.3為testing,target為1
d  [0.2;0.1;0.1]
0
0 
0.99


W 0
0.9975
0 
 0
0
0.9975
β  ((W H)T W H) 1 ( W H)T T  1.5534
0.2355 


W Hβ  W T   0.0007  0.3628
 0.2759
16

Find the k nearest training data to testing
data
 w11  0 


W       , diagonal matrix
 0  wkk 
W Hβ  W T
β  (( W H)T W H)1( W H)T W T
17




Paper數據
Randomly weight and bias
The output of Nearest data
(feature selection…?)
-0.93182 0.312205 0.029309 0.061562
0
0
0
1
-0.95352 0.25826 0.060621 0.061562
0
0.019231 0.005682
2
-0.98056 0.393122 0.022044 0.03028
0
0.019231 0.005682
3
-0.97946 0.211059 0.029309 0.045921
0
0.038462 0.022727
6
-0.9493 0.204316 0.006012 0.092843
0
0.019231 0.034091
18