Improved Search for Local Optima in Particle Swarm Optimization

May 6, 2015

Huidae ChoWater Resources Engineer, Dewberry Consultants

Part-Time Assistant Professor, Kennesaw State University

2

Overview

Why Find Local Optima? Isolated-Speciation-based Particle Swarm

Optimization (ISPSO) Challenges in Multi-Modal Optimization Stochastic Rainfall Generator Other Applications Conclusions

3

Why Find Local Optima?

Traditional model optimization tries to find “the” global optimum only.

Is the/a global optimum always

what we want?

We want “realistic” solutions. Need a new technique to find many working solutions.

4

Why Find Local Optima? (Cont.)

Flood risk model Want to minimize the risk. Finds only the global

optimum. If factors A and B are

costly?

Factor AFactor B

Risk

5

Isolated-Speciation-based Particle Swarm Optimization (ISPSO)

New optimization method based on Species-based PSO (SPSO)!

Implemented in the R language.

Runs on multi-platforms: MS-Windows, UNIX

Finds local optima as well as the global optimum.

6

ISPSO (Cont.)Particle Swarm Optimization (PSO)

PSO is a metaheuristic based on the movement of possible solutions referred to as particles.

A swarm consists of multiple particles sharing information with each other.

Particles fly through the search space towards optimal solutions.

7

ISPSO (Cont.)Randomness vs. Uniformity

8

ISPSO (Cont.)Particle’s Movement

~vi (t)

~xi (t)

Â~vi (t)

x2

~pi

x1

~pg

~xi (t +1)

~vi (t +1)

ÂÃ1~r1(t)(~pi ¡ ~xi (t))

ÂÃ2~r2(t)(~pg ¡ ~xi (t))

 =2

j2¡ Ã ¡pÃ2 ¡ 4Ãj

; Ã = Ã1+Ã2 >4

~vi (t +1) = Â[~vi (t) +Ã1~r1(t) (~pi ¡ ~xi (t)) +Ã2~r2(t) (~pg ¡ ~xi (t))]

~xi (t +1) =~xi (t) +~vi (t+1)

9

ISPSO (Cont.)Isolated Speciation

10

ISPSO (Cont.)Fitness Assimilation

11

ISPSO (Cont.)Preemptive Competitive Nesting

12

ISPSO (Cont.)Flowchart

Initial population from Sobol’ sequences

Isolated speciation

Update velocities

Check for nesting criteria

Preemptive nesting

Update positions

Fitness assimilation

Stopping criteria?

No

Yes

Start End

13

ISPSO (Cont.)ISPSO vs. SPSO

14

Challenges in Multi-Modal OptimizationHow to Detect the Findings of Local Optima?

SPSO: error between the fitness of the global optimum and the particle’s fitness In real-world problems, in most cases, the fitness of the

global optimum unknown. NichePSO: the standard deviation of fitness values over a

number of iteration Cannot guarantee spatial convergence.

ISPSO: SD of fitness and the normalized geometric mean of the particle’s half-life experience. Guarantees spatial convergence as well as fitness

convergence.

15

Challenges in Multi-Modal Optimization (Cont.)When Failed to Detect Local Optima Correctly?

n Analytically counted by Cho et al. (2008),

Numerically confirmed with ISPSO

NichePSO by

Brits et al. (2007)

1 9 5

2 111 25

3 1,215 625

Number of Global and Local Optima in [-28,28]n

16

Challenges in Multi-Modal Optimization (Cont.)When to Stop the Algorithm?

Don’t want to wait until the maximum number of iterations. As the number of iterations between successive discoveries

increases It becomes more difficult to find more optima. Possibility to find another optimum decreases. Certain threshold.

17

Stochastic Rainfall GeneratorModified Bartlett-Lewis Rectangular Pulse Model (MBLRP)

Stochastically generates synthetic rainfall time series. Replicates statistics of observed rainfall at a given

rain gage. Highly multi-modal: Need to find as many feasible

solutions as possible.

18

Stochastic Rainfall Generator (Cont.)ISPSO vs. NichePSO: 2D Projection of Solutions

19

Stochastic Rainfall Generator (Cont.)ISPSO vs. NichePSO: Histograms of the Normalized Distance between Solutions and True Optima

20

ISPSO vs. AutoCal built in SWAT

Other ApplicationsSWAT Model Calibration

Clear Multi-Modality!

21

Other Applications (Cont.)Uncertainty Analysis within the GLUE Framework

Multi-modal optimization suitable for equifinality Relative sampling density In a SWAT case study, 4,000 model runs vs. 46,000!

22

References

Brits, R., Engelbrecht, A.P., van den Bergh, F., 2007. Locating Multiple Optima Using Particle Swarm Optimization. Applied Mathematics and Computation 189 (2), 1859-1883.

Cho, H., Kim, D., Olivera, F., Guikema, S. D., 2011. Enhanced Speciation in Particle Swarm Optimization for Multi-Modal Problems. European Journal of Operational Research 213 (1), 15-23.

Cho, H., Olivera, F., Guikema, S. D., 2008. A Derivation of the Number of Minima of the Griewank Function. Applied Mathematics and Computation 204 (2), 694-701.

23

Conclusions

Mathematically best solutions are not always practical and feasible.

ISPSO improved SPSO for finding local optima. More reliable criteria for finding solutions and

stopping optimization were introduced. ISPSO outperformed SPSO, its predecessor, and

NichePSO, another PSO-based multi-modal optimizer. Application to model calibration and GLUE.