Embed Size (px)
Citation preview
2002-04-24
Particle Swarm optimisation
Particle Swarm optimisation: A mini tutorial
2002-04-24
Particle Swarm optimisation
The “inventors” (1)
Russell Eberhart
2002-04-24
Particle Swarm optimisation
The “inventors” (2)
James Kennedy
2002-04-24
Particle Swarm optimisation
The circular neighbourhood
Virtual circle
1
5
7
6 4
3
8 2
Particle 1’s 3-
neighbourhood
2002-04-24
Particle Swarm optimisation
Psychosocial compromise
Here I am!
The best perf. of my neighbour
s
My best perf.
x pg
pi
v
i-proximity
g-proximity
2002-04-24
Particle Swarm optimisation
The historical algorithm
txprand
txprand
tv
tv
ddg
ddi
d
d
,2
,1
,0
,0
1
for each particle
update the
velocity
1)1( tvtxtxthen move
for each component d
At each time step tRandom
ness inside
the loop
2002-04-24
Particle Swarm optimisation
Random proximity
xpg
pi
v
i-proximity
g-proximity
Hyperparallelepiped => Biased
2002-04-24
Particle Swarm optimisation
Part 2: How to choose parameters The right way
This way
Or this way
2002-04-24
Particle Swarm optimisation
Type 1” form
2121 ''),0(),0( randrand
21
21
''
''
gi pp
p
)()1()1(
)))(()(()1(
txtvtx
txptvtv
with
4for 42
22
else
Usual values:=1=4.1=> =0.73swarm size=20hood size=3
Non divergence criterion
Global constriction coefficient
2002-04-24
Particle Swarm optimisation
5D complex space
4
2
0
-2
-4
2
1
0
-1
-2
6
5
4
3
2
1
0
}
}Convergence
Non divergence
A 3D section
Re(y)Re(v)
2002-04-24
Particle Swarm optimisation
Move in a 2D section (attractor)
10.50-0.5-1
0.8
0.6
0.4
0.2
0-0.2
-0.4
-0.6
-0.8
Im(v)
Re(v)
2002-04-24
Particle Swarm optimisation
Some functions ...
Rosenbrock
Griewank Rastrigin
2002-04-24
Particle Swarm optimisation
... and some results
30D function PSO Type 1" Evolutionaryalgo.(Angeline 98)
Griewank [±300] 0.003944 0.4033
Rastrigin [±5] 82.95618 46.4689
Rosenbrock [±10] 50.193877 1610.359
Optimum=0, dimension=30Best result after 40 000 evaluations
2002-04-24
Particle Swarm optimisation
Beat the swarm!
Your current position
Your best perf.
Best perf.
of the
swarm
2002-04-24
Particle Swarm optimisation
Part 3: Beyond real numbers
0
1
2
3
4
0
1
2
3
0
1
2
3
4
1
2
3
4
5
6 0
1
2
0
1
2
3
4
8 8Bingo!
2002-04-24
Particle Swarm optimisation
Minimun requirements
'',)',( xfxfxfxfHHxx
positionvelocityposition
velocitypositionposition
velocityvelocityvelocity
velocityvelocitytcoefficien
,
,
,
,
Comparing positions in the search space H
Algebraic operators
2002-04-24
Particle Swarm optimisation
velocity = pos_minus_pos(position1, position2)
velocity = linear_combin(,velocity1,,velocity2)
position = pos_plus_vel(position, velocity)
(position,velocity) = confinement(positiont+1,positiont)
Pseudo code form
}algebraic operators
=>
2002-04-24
Particle Swarm optimisation
Fifty-fifty
xi 1...N i j xi xj
xi xiD/21
D
1
D /2
N=100, D=20. Search space: [1,N]D
105 evaluations:63+90+16+54+71+20+23+60+38+15=12+48+13+51+36+42+86+26+57+79 (=450)
granularity=1
2002-04-24
Particle Swarm optimisation
Knapsack
xi 1...N i j xi xj
xi SiI , I D, I 1,N
N=100, D=10, S=100, 870 evaluations: run 1 => (9, 14, 18, 1, 16, 5, 6, 2, 12, 17)run 2 => (29, 3, 16, 4, 1, 2, 6, 8, 26, 5)
granularity=1
2002-04-24
Particle Swarm optimisation
Graph Coloring Problem
1
45
2
11
55
3
22
1
1
5
5
50
2
0
-1
4
-3
-1
-1
+
=
pos -plus -vel
2002-04-24
Particle Swarm optimisation
The Tireless Traveller Example of position: X=(5,3,4,1,2,6)Example of velocity: v=((5,3),(2,5),(3,1))
2002-04-24
Particle Swarm optimisation
n1
n3
n2
Apple trees
2322
21 nnnnf
Evaluation n1 n2 n30 1 2 3
Swarm size=3Best position
7 7 6
3 11 66 4 103 0 17
2002-04-24
Particle Swarm optimisation
0
0.2
0.4
0.6
0.8
1
1.2
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29
Unbiased random proximity
xpg
pi
v
i-proximity
g-proximity
Hyperparallelepiped => Biased
DimensionV
olu
me
Hypersphere vs hypercube
2002-04-24
Particle Swarm optimisation
Clusters and queensEach particle is weighted by its
perf.
Dynamic clustering
Centroids = queens = temporary new “particles”
2002-04-24
Particle Swarm optimisation
Think locally, act locally (Adaptive versions)
2002-04-24
Particle Swarm optimisation
Adaptive swarm sizeThere has been enough improvement
but there has been not enough improvement
although I'm the worst
I'm the best
I try to kill myself
I try to generate a new particle
2002-04-24
Particle Swarm optimisation
Adaptive coefficients
The better I am, the more I follow my own way
The better is my best neighbour, the more I tend to go towards him
vrand(0…b)(p-x)
2002-04-24
Particle Swarm optimisation
Energies: classical process
Rosenbrock 2D. Swarm size=20, constant coefficients.
0,00E+00
1,00E+03
2,00E+03
3,00E+03
4,00E+03
5,00E+03
6,00E+03
7,00E+03
8,00E+03
9,00E+03
1,00E+04
Number of evaluations
0
5
10
15
20
25
Potential energyKinetic energySwarm size
=0.00001 after 2240 evaluations
Rosenbrock 2D. Swarm size=20, constant coefficients
2002-04-24
Particle Swarm optimisation
Energies: adaptive process
Rosenbrock 2D. Adaptive swarm size, adaptive coefficients.
0,00E+00
1,00E+03
2,00E+03
3,00E+03
4,00E+03
5,00E+03
6,00E+03
7,00E+03
8,00E+03
9,00E+03
1,00E+04
Number of evaluations
0
5
10
15
20
25
Potential energyKinetic energySwarm size
=0.00001 after 1414 evaluations
Rosenbrock 2D. Adaptive swarm size, adaptive coefficients
2002-04-24
Particle Swarm optimisation
Part 5: Real applications (hybrid)Medical diagnosis Industrial mixer
Electrical generator Electrical vehicle
2002-04-24
Particle Swarm optimisation
Real applications (stand alone)Cockshott A. R., Hartman B. E., "Improving the fermentation medium for Echinocandin B production. Part II: Particle swarm optimization", Process biochemistry, vol. 36, 2001, p. 661-669.
He Z., Wei C., Yang L., Gao X., Yao S., Eberhart R. C., Shi Y., "Extracting Rules from Fuzzy Neural Network by Particle Swarm Optimization", IEEE International Conference on Evolutionary Computation, Anchorage, Alaska, USA, 1998.
Secrest B. R., Traveling Salesman Problem for Surveillance Mission using Particle Swarm Optimization, AFIT/GCE/ENG/01M-03, Air Force Institute of Technology, 2001.
Yoshida H., Kawata K., Fukuyama Y., "A Particle Swarm Optimization for Reactive Power and Voltage Control considering Voltage Security Assessment", IEEE Trans. on Power Systems, vol. 15, 2001, p. 1232-1239.
2002-04-24
Particle Swarm optimisation
To know more
Clerc M., Kennedy J., "The Particle Swarm-Explosion, Stability, and Convergence in a Multidimensional
Somplex space", IEEE Transaction on Evolutionary Computation, 2002,vol. 6, p. 58-73.
Clerc M., "L'optimisation par essaim particulaire. Principes et pratique", Hermès, Techniques et
Science de l'Informatique, 2002.
Particle Swarm Central, http://www.particleswarm.netTHE site:
Self advert
2002-04-24
Particle Swarm optimisation
Canonical form
11
1
tvtxtx
txptvtv
txpty
v t 1 v t y t y t 1 v t y t
ty
tv
ty
tv
11
1
1
1M
Eigen values e1 and e2
2002-04-24
Particle Swarm optimisation
Constriction
42
22
42
22
2
22
2
2
22
1
Constriction coefficients
2002-04-24
Particle Swarm optimisation
Convergence criterion
654321
3.5
3
2.5
2
1.5
1
0.5
1e1 1
2e2 1
1 1
2e2 1
2e2
2002-04-24
Particle Swarm optimisation
Magic Square (1)
DDD
ji
D
mm
m
mm
,1,
,
,11,1
lkji
ji
Dijji
mm
Nm
xm
,,
,
1,
1
0
1
1
2
11,,
1
1
2
1,1,
D
j
D
ijiji
D
i
D
jjiji
mm
mm
2002-04-24
Particle Swarm optimisation
Magic Square (2)
D=3x3, N=10010 runs13430 evaluations
10 solutions
55 30 68 42 49 62 56 74 23
30 61 53 89 32 23 25 51 68
27 96 39 73 40 49 62 26 74
22 70 58 40 75 35 88 5 57
50 43 67 58 55 47 52 62 4
43 51 78 75 33 64 54 88 30
50 65 68 69 42 72 64 76 43
18 25 59 32 53 17 52 24 26
80 3 30 22 72 19 11 38 64
65 28 64 63 55 39 29 74 54
2002-04-24
Particle Swarm optimisation
Non linear system
010sin
01
21
22
21
xx
xx
Search space [0,1]2
1 run143 evaluations
1 solution
10 runs1430 evaluations
3 solutions
