Particle Swarm Optimization: The Algorithm and Its Applications

IntroductionBiological Inspiration

The AlgorithmApplicationsConclusions

PARTICLE SWARM OPTIMIZATION : THE

ALGORITHM AND ITS APPLICATIONS

Muhammad Adil Raja

Roaming Researchers, Inc.

July 31, 2014

Muhammad Adil Raja Particle Swarm Optimization: Algorithm and Applications



OUTLINE

1 INTRODUCTION

2 BIOLOGICAL INSPIRATION

3 THE ALGORITHM

4 APPLICATIONS

5 CONCLUSIONS




OUTLINE

1 INTRODUCTION


3 THE ALGORITHM

4 APPLICATIONS

5 CONCLUSIONS




OUTLINE

1 INTRODUCTION


3 THE ALGORITHM

4 APPLICATIONS

5 CONCLUSIONS




OUTLINE

1 INTRODUCTION


3 THE ALGORITHM

4 APPLICATIONS

5 CONCLUSIONS




OUTLINE

1 INTRODUCTION


3 THE ALGORITHM

4 APPLICATIONS

5 CONCLUSIONS




ANT COLONY OPTIMIZATION

A valuable technique for mathematical optimization.

Takes inspiration from swarming behavior of birds, animalsor insects.

Useful for discrete and continuous optimization problems.

In telecommunications: Routing and load balancing.




BIOLOGICAL INSPIRATION

Inception – early 90’s.

Proposed by Kennedy and Eberhardt.

Social psychologist and electrical engineer.

Based on observation of bird flocks searching for corn.

Birds are social animals.

Birds are also driven by the goal of community survivalrather than being focused on survival of the individuals.

Bird’s foraging behavior: How birds swarm together as theysearch for food.

Convergence: How the whole swarm converges to goodcorn fields (optimum solutions)




SWARMING BEHAVIOR OF BIRDS

When searching for food birds:

As a single bird finds a good corn source.

Other birds try to converge to it so that they can also grabsome food.

Which other birds: They are the neighboring birds.

Who are the neighbors: Neighborhood functions.

Birds drift together probabilistically, meaning sometimesthey move closer and sometimes they lurch away.

Benefit: Better exploration of corn fields for good corn.




BENEFITS

Indirect communication between birds enables them toconverge to better food sources.

Random probabilistic search enables them to find better,globally optimal, food sources as opposed to substandard,locally optimal, ones.




BASIC IDEAS

A number of simple entities – particles – are placed in thesearch space of some problem or function.

Each particle evaluates the objective function at its currentlocation.Each particle determines its movement through the searchspace by:

1 Combining some aspect of the history of its own currentand best locations with those of one or more members ofthe swarm.

2 And with some random perturbations.




THE ALGORITHM

The next iteration takes place when all particles have beenmoved.

Eventually the swarm as a whole is likely to move close toan optimum of the fitness function.

Like a flock of birds foraging for food.




DATA STRUCTURES

Each individual is composed of three D-dimensionalvectors.

D is the dimensionality of the search space.

The vectors are: The current position ~xi , the previous bestposition ~pi , and the velocity ~vi .

The current position ~xi can be considered as a set ofcoordinates describing a point in space.

On each iteration of the algorithm, the current position isevaluated as a problem solution.

If that position is better than any that has been found so far,then the coordinates are stored in the second vector, ~pi .




USING DATA STRUCTURES

The value of the best function result so far is stored in avariable that can be called pbesti (for previous best)., forcomparison on latter iterations.

The objective, of course, is to keep finding better positionsand updating ~pi and pbesti .

New points are chosen by adding ~vi coordinates to ~xi , andthe algorithm operates by adjusting ~vi .~vi is effectively the step size.




WHY SWARMING IS IMPORTANT?

The particle swarm is more than just a collection ofparticles.

A particle itself has almost no power to solve any problem.

Progress occurs only when particles interact.

Problem solving is a population-wide phenomenon.

It emerges from the individual behaviors of the particlesthrough their interactions.

In any case, populations are organized according to somesort of communication structure or topology.

This is often thought of as a social network.




SWARM TOPOLOGY

The topology typically consists of bidirectional edgesconnecting pairs of particles.

So that if j is in i’s neighborhood, i is also in j’s.

Each particle communicates with some other particles.

And is affected by the best point found by any member ofits topological neighborhood.

This is just the vector ~pi for that best neighbor.

We denote this with ~pg.

The potential kinds of population ""social networks" arehugely varied.




SWARM TOPOLOGY

In practice certain types have been used more frequently.

Velocity is iteratively adjusted to allow the particles tooscillate.

Topologies can be static and dynamic depending on theproblem.




THE ALGORITHM – PSEUDOCODEI

1 Initialize a population array of particles with randompositions and velocities and D-dimensions in the searchspace.

2 Begin loop:3 For each particle, evaluate the desired optimization fitness

function in D variables.4 Compare particleÕs fitness evaluation with its pbesti . If

current value is better than pbesti , then set pbesti equal tothe current value, and ~pi equal to the current location ~xi inD-dimensional space.




THE ALGORITHM – PSEUDOCODEII

5 Identify the particle in the neighborhood with the bestsuccess so far, and assign its index to the variable g.

6 Change the velocity and position of the particle.

~vi ← ~vi + ~U(0, φ1)⊗ (~pi − ~xi) + ~U(0, φ2)⊗ ( ~pg − ~xi) (1)~xi ← ~xi + ~vi (2)

7 If a criterion is met (usually a sufficiently good fitness or amaximum number of iterations), exit loop.

8 End loop




THE ALGORITHM – PSEUDOCODEIII

Where:~U(0, φ1) represents a vector of random numbers uniformlydistributed in [0, φi ] which is randomly generated at eachiteration and for each particle.

⊗ is component-wise multiplication.

In the original version of PSO, each component of ~vi iskept within the range [−Vmax ,+Vmax ]




PARAMETERS

1 Population size.2 Velocity.3 etc.




APPLICATIONS

Image and video analysis.

Design and restructuring of electricity networks and loaddispatching.

Control applications.

Applications in electronics and electromagnetics.

Antenna design.

Power generation and power systems.

Scheduling.

Design applications.




APPLICATIONS

Design and optimization of communication networks.

Biological, medical and pharmaceutical.

Clustering, classification and data mining.

Fuzzy and neuro-fuzzy systems and control.

Signal processing.

Neural networks.

Combinatorial optimization problems.

Robotics.

Prediction and forecasting.




APPLICATIONS

Modeling.

Detection and diagnosis of faults and recovery from them.

Sensors and sensor networks.

Applications in computer graphics and visualization.

Design or optimization of engines and electrical motors.

Applications in metallurgy.

Music generation and games.

Security and military applications.

Finance and economics.




CONCLUSIONS

A great algorithm.

Bio-inspiration is the key.

Emulation of real bird swarming behavior..

Easy to comprehend.

Many variants.

Many applications.

Problem formulation is the real trick.

Inspiration (reference): Particle Swarm Optimization,Riccardo Poli, James Kennedy and Tim Blackwell


Engineering

Particle Swarm Optimization: The Algorithm and Its Applications