Particle Swarm Optimization (PSO) Algorithmand Its Application in Engineering Design

Optimization

School of Information TechnologyIndian Institute of Technology Kharagpur

September 9, 2005

Sushanta Kumar Mandal

Research Scholar

By

Outline

Introduction to OptimizationOptimization ProcedureDifferent Optimization AlgorithmsDifferent Global Optimization AlgorithmsParticle Swarm Optimization (PSO) AlgorithmApplication of PSO in Design Optimization

Problems

Optimization

As ageless

as time

Calculus

Maximum and minimum of a smooth function is reached at a stationary point where its gradient vanishes.

Optimization is Everywhere

The more we know about something, the more we see where optimization can be applied.

Some personal decision making

- Finding fastest route home or class

- Optimal allocation of time for home work

- Optimal budgeting

Goal of Optimization

Find values of the variables that minimize or maximize the objective function while satisfying the constraints.

Component of Optimization Problem

Objective Function: An objective function which we want to minimize or maximize.

For example, in a manufacturing process, we might want to maximize the profit or minimize the cost.

In fitting experimental data to a user-defined model, we might minimize the total deviation of observed data from predictions based on the model.

In designing an inductor, we might want to maximize the Quality Factor and minimize the area.

Component of Optimization Problem

Design Variables: A set of unknowns or variables which affect the value of the objective function.

In the manufacturing problem, the variables might include the amounts of different resources used or the time spent on each activity.

In fitting-the-data problem, the unknowns are the parameters that define the model.

In the inductor design problem, the variables used define the layout geometry of the panel.

Component of Optimization Problem

Constraints: A set of constraints that allow the unknowns to take on certain values but exclude others.

For the manufacturing problem, it does not make sense to spend a negative amount of time on any activity, so we constrain all the "time" variables to be non-negative.

In the inductor design problem, we would probably want to limit the upper and lower value of layout parameters and to target an inductance value within the tolerance level.

Are All these ingredients necessary?

Almost all optimization problems have objective function.

No objective function. In some cases (for example, design of integrated circuit layouts), the goal is to find a set of variables that satisfies the constraints of the model. The user does not particularly want to optimize anything so there is no reason to define an objective function. This type of problems is usually called a feasibility problem.

Are All these ingredients necessary?

Variables are essential. If there are no variables, we cannot define the objective function and the problem constraints.

Constraints are not essential. In fact, the field of unconstrained optimization is a large and important one for which a lot of algorithms and software are available. It's been argued that almost all problems really do have constraints.

What We Need for Optimization

Models: Modeling is the process of identifying objective function, variables and constraints. The goal of models is “insight” not the numbers. A good mathematical model of the optimization problem is needed.

Algorithms: Typically, an interesting model is too complicated to be able to solve in with paper and pencil. An effective and reliable numerical algorithm is needed to solve the problem. There is no universal optimization algorithm. Algorithm should have robustness (good performance for a wide class of problems), efficiency (not too much computer time) and accuracy (can identify the error)

Flowchart of Optimal Design Procedure

Need for optimization

Choose design variables

Formulate constraints

Formulate objective function

Set up variable bounds

Select an optimization algorithm

Obtain solution(s)

Mathematical Formulation of Optimization Problems

1 2

minimize theobjective function

min ( ), , ,.......,

subject toconstraints

( ) 0

0

n

i

i

f x x x x x

c x

c x

2 2

1 2

2 21 2

1 2

Example

min 2 1

subject : 0

2

x x

x x

x x

Constraints

Inequality constraints: x12 – x2

20 Equality constraints: x1 = 2

Variable Bounds

Maximum and minimum bounds on each design variable.

Without variable bounds the constraints completely surround the feasible region.

Variable bounds are used to confine the search algorithm within these bounds.

Ex: ( ) ( )L Ui i ix x x

Classification of Optimization Methods

Single variable Multi-variable Constrained Non-constrained Single objective Multi-objective Linear Non-linear

Classifications of Optimization Methods

Local and Global Optimizers

A local minimizer, xB*, of the region B, is defined so that:

f(xB*)f(x), xB.

Ex: Gradient based search methods, Newton-Rapson algorithms, Steepest Decent, Conjugate-Gradient algorithms, Levenberg-Marquardt algorithm etc.

Shortcomings: 1)  One requires an initial guess to start with. 2) Convergence to an optimal solution depends on the chosen initial guess. 3) Most algorithms tend to get stuck to a sub-optimal solution. 4)  An algorithm efficient in solving one optimization problem may not be efficient in solving another one. 5) These are useful over a relatively narrow range.

Local and Global Optimizers

The global optimizer, x* , is defined so that f(x*)f(x), xS where S is the search space.

Ex. Simulated Annealing algorithm, Genetic Algorithm, Ant Colony, Geometric Programming, Particle Swarm Optimization etc.

Local and Global Optimizers

Particle Swarm Optimization

Evolutionary computational technique based on the movement and intelligence of swarms looking for the most fertile feeding location

It was developed in 1995 by James Kennedy and Russell Eberhart

Simple algorithm, easy to implement and few parameters to adjust mainly the velocity

A “swarm” is an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Continued…

It uses a number of agents (particles) that constitute a swarm moving around in the search space looking for the best solution.

Each particle is treated as a point in a D-dimensional space which adjusts its “flying” according to its own flying experience as well as the flying experience of other particles

Each particle keeps track of its coordinates in the problem space which are associated with the best solution (fitness) that has achieved so far. This value is called pbest.

Continued…

Another best value that is tracked by the PSO is the best value obtained so far by any particle in the neighbors of the particle. This value is called gbest.

The PSO concept consists of changing the velocity(or accelerating) of each particle toward its pbest and the gbest position at each time step.

Continued…

Each particle tries to modify its current position and velocity according to the distance between its current position and pbest, and the distance between its current position and gbest.

21( ) * ( CurrentPosition ) 2( ) * ( CurrentPositionn n1 1 best,n best,n )randv v c rand p c gnn

CurrentPosition[n+1] = CurrentPosition[n] + v[n+1]

current position[n+1]: position of particle at n+1th

iteration

current position[n]: position of particle at nth iteration

v[n+1] : particle velocity at n+1th iteration

vn+1: Velocity of particle at n+1 th iteration

Vn : Velocity of particle at nth iteration

c1 : acceleration factor related to gbest

c2 : acceleration factor related to lbest

rand1( ): random number between 0 and 1

rand2( ): random number between 0 and 1

gbest: gbest position of swarm

pbest: pbest position of particle

PSO Algorithm

For each particle     Initialize particle with feasible random numberENDDo    For each particle         Calculate the fitness value        If the fitness value is better than the best fitness value (pbest) in history            Set current value as the new pbest    End

Choose the particle with the best fitness value of all the particles as the gbest    For each particle         Calculate particle velocity according to velocity update equation

        Update particle position according to position update equation     End

While maximum iterations or minimum error criteria is not attained

gbest and lbest

global version:vx[ ][ ] = vx[ ][ ] + 2*rand( )*(pbest[ ][ ] – presentx[ ][ ])

+2*rand( )*(pbestx[ ][gbest] – presentx[ ][ ])

local version:vx[ ][ ] = vx[ ][ ] + 2*rand( )*(pbest[ ][ ] – presebtx[ ][ ]) +

2*rand( )*(pbestx[ ][lbest] – presentx[ ][ ])

Particle Swarm Optimization:Swarm Topology

In PSO, there have been two basic topologies used in the literature

– Ring Topology (neighborhood of 3)

– Star Topology (global neighborhood)

I4

I0

I1

I2 I3

I4

I0

I1

I2 I3

PSO Parameters: Velocity

An important parameter in PSO; typically the only one adjusted

Calmps particles’ velocities on each dimenson Determines “fineness” with which regions are

searched If too high, can fly past optimal solutions If too low, can get stuck in local minima

Flow Chart for Extraction Procedure using PSOInput

No. of Partic leNo. of iteration countError = 0.05

Initialize Random ly1. Position of the partic le2. Velocity of the Partic le

For each Partic le

S im ulate Model Param etersY11,Y12,Y21,Y22

Evaluate Fitness Function

Isfitness(Current Pos.)

>fitness(gbest)

?

Measurem entData

Updategbest = current pos.

Yes

No

Isfitness(Current Pos)

>fitness(lbest)

?Update

lbest = current pos.

Yes

No

Ispartic le > Max. no.

?

No

go for nextpartic le

Evaluate Fitness Function Ebased on gbest

Isfitness error < defined error

ExtractedParam eter

Yes

Isiteration count > Max. count

?

No

Yes

No

Updatevlocity

Lim it velocity[Vm in, Vm ax]

Updateposition

Lim it position[Pm in, Pm ax]

Next Iteration

Comparison of Genetic Algorithm and PSO

0 2000 4000 6000 8000 100000 5

10

20

30

40

50

60

70

Number of Iteration

Targ

et v

alue

of E

rror

min

imiz

atio

n

GA

PSO

Tested in a MATLAB Program, P4 1.7GHz CPU, 256M RAM. No. of particle/ population size = 100No. of simulation runs: 10000

Crossover Prob.= 0.9

Mutation Prob. = .01

Model Fitting

Model Fitting

Inductor Optimization

, , ,

1 , , ,arg

, , , 1 arg

maxmin

maxmin

min min

maxmin2 2

maximize Q n d w s

subject to tol L L n d w st et

L n d w s tol Lt et

n n n

d d d

w w d

s s s

d n w s s

Constrained PSO Optimization

Thank You