Particle Swarm Optimization mini tutorial

Download Report

Transcript Particle Swarm Optimization mini tutorial

Particle Swarm optimisation: A mini
tutorial
2002-04-24
Particle Swarm optimisation
[email protected]
The “inventors” (1)
Russell
Eberhart
[email protected]
2002-04-24
Particle Swarm optimisation
[email protected]
The “inventors” (2)
James
Kennedy
[email protected]
2002-04-24
Particle Swarm optimisation
[email protected]
Part 1: United we stand
2002-04-24
Particle Swarm optimisation
[email protected]
Cooperation example
2002-04-24
Particle Swarm optimisation
[email protected]
Initialization. Positions and velocities
2002-04-24
Particle Swarm optimisation
[email protected]
Neighbourhoods
geographical
2002-04-24
social
Particle Swarm optimisation
[email protected]
The circular neighbourhood
1
Particle 1’s 3neighbourhood
2
8
3
7
Virtual circle
4
6
5
2002-04-24
Particle Swarm optimisation
[email protected]
Psychosocial compromise
My best
perf.
pi
Here I am!
x
pg
The best perf. of
my neighbours
v
2002-04-24
Particle Swarm optimisation
[email protected]
The historical algorithm
At each time step t
for each particle
for each component d
vd t  1 
update v t 
 d
the 
 rand0, 1  pi , d  xd t 

velocity
 rand0,  2  p g , d  xd t 

then move x(t  1)  xt   vt  1
2002-04-24
Particle Swarm optimisation
[email protected]
Random proximity
Hyperparallelepiped => Biased
pi
x
pg
v
2002-04-24
Particle Swarm optimisation
[email protected]
Animated illustration
Global
optimum
2002-04-24
Particle Swarm optimisation
[email protected]
Part 2: How to choose parameters
2002-04-24
Particle Swarm optimisation
[email protected]
Type 1” form
v(t  1)   (v(t )   ( p  x(t )))

 x(t  1)  v(t  1)  x(t )
with
  rand(0, 1 )  rand(0,  2 )   '1  '2
p
 '1 pi   ' 2 p g
 '1  ' 2
2

for   4


    2   2  4


else 
2002-04-24
Particle Swarm optimisation
Usual values:
=1
=4.1
=> =0.73
swarm size=20
hood size=3
[email protected]
5D complex space

A 3D section
} Convergence
6
5
4
}
3
-2
2
-1
-4
1
-2
0
00
2
Re(v)
2002-04-24
4
Particle Swarm optimisation
Non
divergence
1
2
Re(y)
[email protected]
Move in a 2D section (attractor)
Im(v)
0.8
0.6
0.4
0.2
-1
-0.5
00
0.5
-0.2
-0.4
1
Re(v)
-0.6
-0.8
2002-04-24
Particle Swarm optimisation
[email protected]
Some functions ...
Griewank
Rastrigin
Rosenbrock
2002-04-24
Particle Swarm optimisation
[email protected]
... and some results
Optimum=0, dimension=30
Best result after 40 000 evaluations
30D function
PSO Type 1" Evolutionary
algo.(Angeline 98)
Griewank [±300]
0.003944
0.4033
Rastrigin [±5]
82.95618
46.4689
Rosenbrock [±10] 50.193877
2002-04-24
Particle Swarm optimisation
1610.359
[email protected]
Beat the swarm!
2002-04-24
Particle Swarm optimisation
[email protected]
Part 3: Beyond real numbers
1
8
2002-04-24
0
2
0
1
3
1
2
4
0
2
3
0
5
1
3
4
1
6
4
0
2
2
1
3
3
2
4
Particle Swarm optimisation
Bingo!
8
[email protected]
Minimun requirements
Comparing positions in the
search space H
( x, x' )  H  H ,  f x   f x'   f x   f x'
Algebraic operators

coefficient , velocity 
velocity

velocity, velocity 
velocity

 position, position 
velocity

 position, velocity 
position
2002-04-24
Particle Swarm optimisation
[email protected]
Pseudo code form
velocity = pos_minus_pos(position1, position2)
}
velocity = linear_combin(,velocity1,,velocity2)
position = pos_plus_vel(position, velocity)
algebraic
operators
(position,velocity) = confinement(positiont+1,positiont)
=>
2002-04-24
Particle Swarm optimisation
[email protected]
Fifty-fifty


xi  1...N

i  j  xi  x j
D / 2
D

xi
 xi 

 1
D / 21
 

N=100, D=20. Search space: [1,N]D
105 evaluations:
63+90+16+54+71+20+23+60+38+15
=
12+48+13+51+36+42+86+26+57+79 (=450)
2002-04-24
Particle Swarm optimisation

[email protected]
Knapsack


x  1...N
 i

i  j  xi  x j



xi  S
iI , I  D, I 1, N 

 

N=100, D=10, S=100,
870 evaluations:
run 1 => (9, 14, 18, 1, 16, 5, 6, 2, 12, 17)
run 2 => (29, 3, 16, 4, 1, 2, 6, 8, 26, 5)
2002-04-24
Particle Swarm optimisation
[email protected]
Graph Coloring Problem
1
2
0
5
5
3
5
1
=
2002-04-24
+
2
-1
1
-1
0
-3
2
1
1
4
1
5
4
5
Particle Swarm optimisation
2
5
[email protected]
The Tireless Traveller
Example of position: X=(5,3,4,1,2,6)
Example of velocity: v=((5,3),(2,5),(3,1))
2002-04-24
Particle Swarm optimisation
[email protected]
Apple trees
Best position
Swarm size=3
n1
n3
Evaluation n1
0
3
6
1
3
2
3
7
n2
0
4
11
n3
17
10
6
7 6
n2
f  n1  n2 2  n2  n3 2
2002-04-24
Particle Swarm optimisation
[email protected]
Part 4: Some variants
2002-04-24
Particle Swarm optimisation
[email protected]
Unbiased random proximity
Hyperparallelepiped => Biased
Hypersphere vs hypercube
1.2
pi
x
1
pg
Volume
0.8
0.6
0.4
0.2
v
0
1
3
5
7
9
11 13 15 17 19 21 23 25 27 29
Dimension
2002-04-24
Particle Swarm optimisation
[email protected]
Clusters and queens
Each particle is weighted by its
perf.
Dynamic clustering
2002-04-24
Particle Swarm optimisation
Centroids = queens =
temporary new
“particles”
[email protected]
Think locally, act locally (Adaptive
versions)
2002-04-24
Particle Swarm optimisation
[email protected]
Adaptive swarm size
There has been enough
improvement
although I'm the worst
I'm the best
but there has been not enough
improvement
2002-04-24
Particle Swarm optimisation
I try to kill myself
I try to generate a
new particle
[email protected]
Adaptive coefficients
v
rand(0…b)(p-x)
The better I
am, the more I
follow my own
way
2002-04-24
Particle Swarm optimisation
The better is my best
neighbour, the more
I tend to go towards
him
[email protected]
Energies: classical process
Rosenbrock 2D. Swarm size=20, constant coefficients
2002-04-24
Particle Swarm optimisation
[email protected]
Energies: adaptive process
Rosenbrock 2D. Adaptive swarm size, adaptive coefficients
2002-04-24
Particle Swarm optimisation
[email protected]
Part 5: Real applications (hybrid)
Medical diagnosis
Electrical generator
2002-04-24
Industrial mixer
Electrical vehicle
Particle Swarm optimisation
[email protected]
Real applications (stand alone)
Cockshott A. R., Hartman B. E., "Improving the fermentation medium for
Echinocandin B production. Part II: Particle swarm optimization", Process
biochemistry, vol. 36, 2001, p. 661-669.
He Z., Wei C., Yang L., Gao X., Yao S., Eberhart R. C., Shi Y., "Extracting
Rules from Fuzzy Neural Network by Particle Swarm Optimization", IEEE
International Conference on Evolutionary Computation, Anchorage, Alaska,
USA, 1998.
Secrest B. R., Traveling Salesman Problem for Surveillance Mission using
Particle Swarm Optimization, AFIT/GCE/ENG/01M-03, Air Force Institute of
Technology, 2001.
Yoshida H., Kawata K., Fukuyama Y., "A Particle Swarm Optimization for
Reactive Power and Voltage Control considering Voltage Security
Assessment", IEEE Trans. on Power Systems, vol. 15, 2001, p. 1232-1239.
2002-04-24
Particle Swarm optimisation
[email protected]
To know more
THE site:
Particle Swarm Central, http://www.particleswarm.net
Self advert
Clerc M., Kennedy J., "The Particle Swarm-Explosion,
Stability, and Convergence in a Multidimensional
Somplex space", IEEE Transaction on Evolutionary
Computation, 2002,vol. 6, p. 58-73.
Clerc M., "L'optimisation par essaim particulaire.
Principes et pratique", Hermès, Techniques et
Science de l'Informatique, 2002.
2002-04-24
Particle Swarm optimisation
[email protected]
Appendix
2002-04-24
Particle Swarm optimisation
[email protected]
Canonical form
vt  1  vt     p  xt 

 xt  1  xt   vt  1
yt   p  xt 
M
   vt 
 vt  1  1
 y t  1   1 1     y t 

 


Eigen values e1 and e2
v t 1  v t   
 yt

  y t 
yt  1   vt   
2002-04-24
Particle Swarm optimisation
[email protected]
Constriction
Constriction coefficients
2
2















2





2





 1 
2

2




 4



       2  2     2       2
 2 

2     2  4
2002-04-24
Particle Swarm optimisation
[email protected]
Convergence criterion


  1e1  1
 1  1

 

  2 e 2  1 
  2 e2  1
  2 e 2
3.5
3
2.5
2
1.5
1
0.5
1
2002-04-24
2
3

Particle Swarm optimisation
4
5
6
[email protected]
Magic Square (1)
 m1,1

 
m
 D ,1

D 

m D, D 


mi , j
m1,


mi , j  x j  i 1

mi , j  1 N 

mi , j  mk ,l
2002-04-24
D 1 
  m
i 1

D



D
i, j
j 1
D 1 
 
j 1
D
 mi 1, j






mi , j  mi , j 1

 i 1
2





0
Particle Swarm optimisation
[email protected]
2
Magic Square (2)
55 30 68
42 49 62
56 74 23
30 61 53
89 32 23
25 51 68
43 51 78
75 33 64
54 88 30
80 3 30
22 72 19
11 38 64
D=3x3, N=100
10 runs
13430 evaluations
27 96 39
73 40 49
62 26 74
2002-04-24
50 43 67
58 55 47
52 62 4
10 solutions
22 70 58
40 75 35
88 5 57
Particle Swarm optimisation
18 25 59
32 53 17
52 24 26
65 28 64
63 55 39
29 74 54
50 65 68
69 42 72
64 76 43
[email protected]
Non linear system
2
2

 x1  x2  1  0


sin10x1   x2  0
Search space
[0,1]2
2002-04-24
1 run
143 evaluations
1 solution
10 runs
1430 evaluations
3 solutions
Particle Swarm optimisation
[email protected]
2002-04-24
Particle Swarm optimisation
[email protected]