Transcript Swarm Optimization for Real-Time Adaptation for Variable

Population Based Optimization for
Variable Operating Points
Alan L. Jennings & Raúl Ordóñez,
ajennings1, [email protected]
Electrical and Computer Engineering
University of Dayton
Introduction
Monday, June 6th, 2011
Method
Examples
CEC2011-#274
Conclusion
1
Adjust X
Many dimensions
Change Y
One dimension
The Challenge
As a desired parameter
changes,
Smoothly change other
parameters in real-time,
Smoothly and quickly
Maintain Optimality
1
4
Using particles and
clusters.
Introduction
Monday, June 6th, 2011
0
x3
-2
0.5
x4
0
0.5
y
Output, y
0
-1
1
-0.5
1
1
1
0.5
0.5
0.5
x
x
2
1.5
0
0
0
-0.5
-0.5
-0.5
-1
-1
0
x1
1
Method
-1
-1
Examples
CEC2011-#274
0
0.5
1
Step: Output/Cost
1Final
Particles:
Clusters:
60
Clusters:
11
0
Overlay
1.5
Cost, J
1.5
2
Global search/optimization
of input space,
To form inverse functions
in the output space,
x2
2
The Solution
2
x
While maintaining local
optimality.
x1
0
x1
1
-1
-1
-1
0
x1
11
1
Conclusion
2
Final Clusters: 11
Optimal Inverse Functions, hi
Problem Statement
1.5
0.5
J
-1
0.2
-0.2
-0.4
-0.6
0.4
0.6
0.8
2
• Find continuous functions
x*= hi(yd)
Such that,
• J=g(x*) is local minimum
• yd=f(x*)
over an interval of yd.
1
0
x
x
1
0
-0.5
0.2
0.4 0.6
yd
-1
-1
0.8
0
x1
Cost, J
1
Output, y
1
0
0
Assumptions
-1
• Compact set in x
• g & f are C1, deterministic
and time invariant
• Change in x is easy to implement
• Adequately scaled
• Regions larger than a point where
∇f=0 can result in open domain of hi
Introduction
Monday, June 6th, 2011
Method
-1
-2
1
1
0
x2
-1 -1
Examples
CEC2011-#274
0
x1
1
1
0
x2
-1 -1
0
x1
Conclusion
3
Example Problems
Input: Set point
• Thermostat
• Combining generators
Output: Temperature Cost: Energy
1
• Optimal control trajectory
Linear, SISO system
0
-1
-1
Input: Servo positions nodes in time
Output: Crawl distance, Jump height, ….
Cost: Energy, Max Torque, Profile Height,…
Introduction
Monday, June 6th, 2011
Method
1
This method is different from
• Nominal operating point
• Pareto-Optimal Front
Examples
CEC2011-#274
0
Conclusion
4
Method Overview
Surrogate Function Creation
• Neural Networks
Universal Approximators
Converge in the gradient
Simple to get gradient
• Swarm Optimization
Agents cover n space
Simple motions
Allow for clusters
• Spline Interpolation
Use known optimal
points of clusters
Sample
Function
Swarm Optimization
Initialize Population
Train
Network
Validate
Network
Move Agents:
lower g(x), keep f(x)
Check for removal/
settling conditions
Form clusters
Execution
Select
cluster hi
Get yd
Evaluate hi
Move
x to x*
Introduction
Monday, June 6th, 2011
Method
Examples
CEC2011-#274
Conclusion
5
Particle Motion
• Output gradient
Move in null space
• Cost gradient
Move opposite (in null space)
• Saturation
All gradients saturate
If gradients are large -> fixed step length
If a gradient is small -> step size diminishes
• Boundary constraint reduces step length
• Minimum step for settling
• Remove particles close to another
Quickly reduces population size
Introduction
Monday, June 6th, 2011
Method
Examples
CEC2011-#274
1.5
1.45
1.4
Cost
Output
1.35
1.3
Step
1.25
1.2
1.15
1.1
-1
-0.9
-0.8
-0.7
Conclusion
6
Cluster Formation
• Form cluster from settled particle
• Ascend/Descend Output
Form new point
Apply gradient descent
• End Cluster conditions
Particle doesn’t settle
Output decreases / increases
Settles too far away / close
Introduction
Monday, June 6th, 2011
Method
1.5
1.45
1.4
1.35
1.3
1.25
1.2
1.15
1.1
-1
Examples
CEC2011-#274
-0.9
-0.8
-0.7
Conclusion
7
Simple Example
• Combination of generators
Output: Total power out
Cost: Quadratic function
• Expected result
Each does half the load
Swarm, Step: 1
49Linear
Number
Number
ofofparticles:
particles:
60
0 Number
0
Cost of clusters: 1
Quadratic
Output,
70
60
50
xx
22
40
Optimal Inverse Functions, hi
80
30
x
60
40
20
20
0
20
40
60
80
10
100
0
60
0
10
20
30
50
40
60
70
80
90
x1
J
40
20
0
20
40
60
yd
Introduction
Monday, June 6th, 2011
80
100
Method
Examples
CEC2011-#274
Conclusion
8
Complex Examples
x 21  (x 2-c 1)2
1
0 -1
-1 -1 -0.5
00
xy1d
0.5 1
Quadratic Cost
Introduction
Monday, June 6th, 2011
1
1
-1
-1
0
x1
1
2
x
2
x
2
0
0
-1
-1
0
x1
Optimal
Inverse
Functions,
hi
Final
Clusters:
9
1.5
1
00
y
x 1d
0.5
11
-1
-1
1
0
x1
1
0
1
1
-1
1
0.5
2
22
0
xx
xx
2
-2
-1
-1 -0.5
0
Optimal
Final
Inverse
Clusters:
Functions,
7
hi
1.5
1
0
-1
2
-0.5
0
J
J
-1
2-0.5
0
-1
2
-0.5
0
J
0
0
1
x 1 sin(c 1 x 2)
1
x
1
2
xx
2
2
xx
2
1
-1
1
0.5
-1
10.5
J
0
x
1
x
1
x
0 1
0
i
1.5
1
x
x
Final Clusters: 11
Optimal
Final
Inverse
Clusters:
Functions,
5
hi
1.5
1
(x 1-c 1)  x 22
1
2
• Combination of functions
Peaks
Multiple extremum
1
Saddle points
0
2-dim for verification
-1
-1
0
x
• Expected result
1
Clusters between output
extremum
Optimal Inverse Functions, h
-2
-1
-1 -0.5
Linear/Quadratic
Cost
Method
CEC2011-#274
00
y
x 1d
0.5
Periodic Cost
Examples
11
0 1
-1
0.5
1
0 0
-1
2
-0.5
1
0 -1
-1 -0.5
00
xy 1d
0.5
11
Quadratic Cost
Conclusion
9
Cluster 1 Test
1.5
Output vs Desired
1
1
-1
-1
0
-1
0
1
Yd
Error in output
-3
x 10
5
-5
-1
0
Y
0.5
1
0
Cost difference
1
J-J(Y)
d
a
Y -Y
0
1
2
0
J
Y
a
• Verify Output Accuracy
Plot Actual vs Desired for
test points
• Verify Optimality
Generate neighbors
Plot cost vs output
Subtract expected cost
Cost of outputs
2
x
Cluster Evaluation
-0.5
0.5
0
0
Yd
-1
1
0
Y
-1
-1
1
-0.5
0
x1
0.5
1
-0.5
0
x1
0.5
1
Cluster 2 Test
1.5
Optimal Inverse Functions, hi
1
0.5 1
Yd
Error in output
-3
x 10
5
2
x
2
x
0
0
Introduction
Monday, June 6th, 2011
a
Y -Y
J
1
1
0
-1
0
d
-1
2
-0.5
J
Y
1
0
-1
1
0.5
-0.5
0
yd
0.5
1
1
2
0
-5
0 0.5 1
Y
0
0.5
Yd
Method
Cost difference
1.5
1
0.5
0
1
Examples
CEC2011-#274
0 0.5 1
Y
0.5
2
1
0
0.5
J-J(Y)
x
1
a
1.5
0
x1
Cost of outputs
3
x
Final Clusters: 5
-1
-1
Output vs Desired
1
0
-0.5
-1
-1
Conclusion
10
5 Dim, ill scaled example
• 5 different generators
• Order of magnitude
difference of gradient
• Used exact NN to
eliminate that source or
error
• Resulted in single cluster
that balanced the
incremental cost of all
generators
• 0.1% full range accuracy
Failure Methods
Introduction
Monday, June 6th, 2011
Possible bifurcations,
direction dependent
1
2
0.5
x
• `Kill distance’ may end other
bifurcation branches
• Cluster ends prematurely
• Global optimization
parameters insufficient
• Corners in clusters can impair
cubic interpolation
Piecewise cubic can make
interpolant monotonic
• Difficult to verify in high
dimensions
Testing cluster is reasonably
simple
Final Clusters: 11
1.5
0
Gradient goes to
zero, ends cluster
-0.5
-1
-1
-0.5
Method
0
x1
Examples
CEC2011-#274
0.5
1
Corners in cluster
interfere with
interpolation
Conclusion
12
Questions or Comments?
Global search, using particles & clusters, to
find optimal, continuous output-inverse
functions.
_____________________________________________________________
Tested to work on many
difficult combinations.
_____________________________________________________________
Future: apply to developmental/
resolution increasing control
Introduction
Monday, June 6th, 2011
Method
Examples
CEC2011-#274
Conclusion
13