Transcript Robust Optimization

Robust Optimization
Multi-dimensional Marquardt
Parameters
Multi-dimensional Optimization

2

 f A  c , xi   f i 


i



i

2
ck
 
2
 2
i
2
ck cm
2
 f A  c , xi   f i  1  f A  c , xi 


i
ck

 i
 2
i
1  f A  c , xi  1  f A  c , xi 
i
ck
i
cm
 f A  c , xi   f i  1  f A  c , xi 


i
ck cm

 i
2
 2
i
The approximating function
• A linear fitting function would be
fA c, x 
c
k
Pk  x 
k
In this work the fitting function is essentially
f A (c , x ) 

k
2
 1xc


2
k 1
c k exp   
 
 2  ck  2  


 b x  b 
m
m
m 1
3

Expand the first derivatives about
the value c0.
2
 
2
 
 
 
 cm 
 cm 
c
c0
0

m
 
1



 cm   cn  cm 
c0
c
2
 
2
2
0

k

 c k  c k ,0 
 
 cm  ck 
c
 
2
2
0

 
   c k  c k ,0 


c

c
 cn  cm
k ,m
k
m c
 
2
2
2
0
2
1


c
0
The result is a new estimate for the
values of c
0

m
 
1



 cm   cn  cm 
c0
c
2
c n  c n ,0 
 
2

m
2
 

 c
k
 c k ,0   k , n
k
0
1



 cm   cn  cm 
c0
c
2
 
2
2
0
The usual problem is the inversion
of the matrix
• If a constant cn is such that its effect is a
linear multiple of cm - that is if fA has parts
cn Pn and cmPm with Pn and Pm effectively
the same for all xi - the matrix
2
2
 
has no inverse owing to the
ck cm
fact that there are no unique
values for cn and cm
The Marquardt Parameter
• The solution is to request small constants.
In particular add  c∙c to 2
 f A  c , xi   f i 
2


c
S



m m
i
m


2
   
2

i
   
2
ck cm
2
 2
i
1  f A  c , xi  1  f A  c , xi 
i
ck
i
cm
 2  S m  k ,m
This results in constant estimates
that depend on 
    
    
2
c n     c n ,0 

m
 cm


c
2
 cn  cm
0
2
1


c
0
The general result for any single
constant is
Relative Smoothers
 f  x pred
 f  xm
2


ck
ck

f  x0 
ck
x pred  x 0
 f  x pred
Sk 
ck
x
pred


 x0 
 f  x0 
ck
 f  x0 
2
ck
 f  x0 
2
 Sk
ck
Define p2 and Fr

2
p
   
2
0

 c    c 
m

m ,0
m

2
 
cm
1
c    c c    c 

2
m
m ,0
n
m ,n

2
p
 

2
0
 Fr  0
n ,0
 
2
2
 
cm cn
Fr versus log 
Fr values
• If on a minimization step the actual 2
equals p2, not brave enough
• Fr(Fr)3
• If the actual 2 > the last 2 then
• Fr(9+Fr)/10
• If the values are slowly changing from one
side  Aitkin’s extrapolation.
At the beginning
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
FR,CHI,CHB,CHL 0.999990000
FR,CHI,CHB,CHL 0.999970000
FR,CHI,CHB,CHL 0.999946001
FR,CHI,CHB,CHL 0.999902805
FR,CHI,CHB,CHL 0.999825058
FR,CHI,CHB,CHL 0.999685131
FR,CHI,CHB,CHL 0.999433325
FR,CHI,CHB,CHL 0.998980275
FR,CHI,CHB,CHL 0.998165430
FR,CHI,CHB,CHL 0.996700804
FR,CHI,CHB,CHL 0.994071243
FR,CHI,CHB,CHL 0.989359873
FR,CHI,CHB,CHL 0.980949663
FR,CHI,CHB,CHL 0.966036016
ierr, iloop
1
0
FR,CHI,CHB,CHL 0.993990303
FR,CHI,CHB,CHL 0.989215050
FR,CHI,CHB,CHL 0.980691773
118785.585
118785.230
118784.590
118783.438
118781.362
118777.623
118770.875
118758.679
118736.558
118696.154
118621.239
118476.802
118128.813
116688.910
0.100000000E+67 0.500000000E+
118782.022
118785.585
118778.816
118785.230
118773.045
118784.590
118762.657
118783.438
118743.962
118781.362
118710.315
118777.623
118649.761
118770.875
118540.808
118758.679
118344.823
118736.558
117992.433
118696.154
117359.094
118621.239
116219.779
118476.802
114116.688
118128.813
116611.223
116441.523
115680.541
115987.645
115353.576
114193.243
116688.910
116611.223
116441.523
Aitkin’s extrapolation
•
•
•
•
•
•
•
•
•
•
•
•
•
FR,CHI,CHB,CHL 0.996558072
FR,CHI,CHB,CHL 0.993815191
FR,CHI,CHB,CHL 0.988901771
FR,CHI,CHB,CHL 0.988901771
FR,CHI,CHB,CHL 0.988901771
FR,CHI,CHB,CHL 0.988901771
FR,CHI,CHB,CHL 0.988901771
FR,CHI,CHB,CHL 0.988901771
ierr, iloop
1
0
AITKIN'S EXTRAPOLATION
FR,CHI,CHB,CHL 0.998890177
FR,CHI,CHB,CHL 0.998003427
FR,CHI,CHB,CHL 0.996409757
115637.868
115550.779
115232.958
114790.155
114243.910
113659.796
113455.616
112988.531
115282.377
114922.670
114268.370
113954.076
113516.188
112976.005
112398.373
112196.459
115680.541
115637.868
115550.779
115232.958
114790.155
114243.910
113659.796
113455.616
112951.863
112875.210
112716.318
112863.134
112726.346
112469.960
112988.531
112951.863
112875.210
Non-Linear Irevs
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
FR,CHI,CHB,CHL 0.999968166
FR,CHI,CHB,CHL 0.999942699
FR,CHI,CHB,CHL 0.999896862
FR,CHI,CHB,CHL 0.999814361
IREV= 1 IEND= 151
FR,CHI,CHB,CHL 0.999953590
FR,CHI,CHB,CHL 0.999860777
IREV= 1 IEND= 150
FR,CHI,CHB,CHL 0.999965194
FR,CHI,CHB,CHL 0.999906028
ierr, iloop
1
0
FR,CHI,CHB,CHL 0.999983086
FR,CHI,CHB,CHL 0.999969555
FR,CHI,CHB,CHL 0.999945199
FR,CHI,CHB,CHL 0.999901361
IREV= 1 IEND= 145
FR,CHI,CHB,CHL 0.999975340
FR,CHI,CHB,CHL 0.999926023
IREV= 1 IEND= 144
112081.893
112080.061
112075.828
115490.184
112078.975
112075.471
112068.501
112055.022
112082.543
112081.893
112080.061
112075.828
112070.874
113183.400
112070.627
112055.271
112075.828
112070.874
112067.371
112064.902
112066.973
112056.840
112070.874
112067.371
112064.676
112063.861
112061.774
114265.708
112063.007
112061.264
112057.720
112050.720
112064.902
112064.676
112063.861
112061.774
112059.109
113792.644
112059.010
112050.819
112061.774
112059.109