1

Copyright notices

This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

Concerning copyrighted figures and tables with copyright, the following apply

©2008, 2009 IEEE ©2010 Applied Energy . Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE /Applied Energy.

2

Non-revisiting Stochastic Search

Shiu Yin Yuen, Kelvin

Department of Electronic Engineering, City University of Hong Kong

3

Outline of Talk

Introduction: need for stochastic search; evolutionary computation; no-free-lunch theorems; known problems

Non-revisiting Stochastic Search: the philosophy; non-revisiting genetic algorithm; experimental results; theoretical results

Other Non-revisiting Stochastic Searches Conclusions

4

Need for Stochastic Search

Many real world optimization problems are multi-modal; the landscape is non-smooth; no well defined gradient. They present great difficulties to conventional optimization techniques

Scissors-paper-stone: the “foil an adversary argument”

The nature argument: nature inspired algorithm is suitable for real world problems

5

Evolutionary Computation (EC)

One popular type of stochastic search Idea inspired by nature Can apply to multi-modal, non-smooth, ill defined

gradient situation Simple to learn and give reasonably good result Famous examples:

genetic algorithm, evolutionary strategies, evolutionary programming, genetic programming (natural genetics)

simulated annealing (cooling of hot objects) particle swarm optimization, ant colony, cultural

algorithm (social behavior of a group)

6

No-Free-Lunch Theorems

(Wolpert and Macready 1997)

The average performance of any black box algorithm, when averaged over the set of problems, assumed to be uniformly distributed, is equal

Assume that all algorithms are non-revisiting: a visited search position will not be re-visited

Such a non-revisiting algorithm has never been reported before!

7

Known Problems in EC

The algorithms are revisiting: throws away all past history; only keeps the current population

Premature convergence: the whole population consists of a single type of individual

Parameter control: you need to fine tune your evolutionary algorithm by selecting good parameters; the good parameters may vary over time

Which algorithm for which problem? Does anybody know what it’s actually doing?

8

Non-revisiting (Nr) Stochastic Search

Basic Idea 1) Remember every past experience 2) Use 1) to make intelligent future search decisions

based on the entire previous search history Nr search solves the first three problems Main Objection

Remember everything will take up too much memory Answer

Memory is cheap in IT age and obeys Moore’s law Vast majority of engineering problems involves

expensive function evaluations

9

Genetic Algorithm (GA)

Standard Algorithm

1. Randomly initialize a population of individuals2. Select two individuals by a selection operator3. Crossover the two individuals using a crossover

operator to generate an offspring4. Mutate the offspring using a mutation operator 5. Repeat steps 2-4 until offspring have been generated6. Use a replacement policy involving the parents and

offspring to generate the next generation of individuals

7. Repeat steps 2-6 until the stopping criterion is satisfied

10

Non-revisiting Genetic Algorithm (NrGA)

GA is a standard one but without mutation New solution is stored in a binary space

partitioning (BSP) structure When there is a revisit, mutate randomly within

the nearest unvisited “subspace”; prune the “subspace” if full

(The figure appears in reference [4], 2009 IEEE)

11

An Illustrative Example

Search space [3, 9] x [3, 6] axis resolution is 3 and 2 (i.e. possible values of

the first gene is 3, 6, 9; possible values of the second gene is 3, 6)

GA randomly generates sq = (s(1), s(2), s(3), s(4)) = (s1 = (9,6), s2 = (6,6), s3 = (9,3) and s4 = (9,3)).

s4 is a revisit

12

s1s2

s3, s4

(The figure is adapted from reference [4], 2009 IEEE)

13

Pruning (The figure is adapted from reference [4], 2009 IEEE)

14

Revisit

Parameter-less adaptivelymutate

(The figure is adapted from reference [4], 2009 IEEE)

15

Experiment 1

15 EC benchmark functions with Dimension = 30, 40 and 4 functions with Dimension = 2

Compare with Standard GA: GA (CGA), Real coded GA (RC-

GA) GA with diversity mechanism: Div-GA Covariance Matrix Adaptation Evolutionary

Strategy (CMA-ES) Three Improved PSOs: DPSO, SEPSO, PSOMS

16

Unimodal Noisy Unimodal

Function f1 f2 f3 f4 f5 f6 D 10 20 30 40 10 20 30 40 10 20 30 40 10 20 30 40 10 20 30 40 10 20 30 40

NrGA 2 5 5 6 7 7 7 3 2 2 2 2 2 2 RC-GA CMA-ES CGA Div-GA DPSO SEPSO PSOMS

Multimodal

Function f7 f8 f9 f10 f11 f12 f13 f14 D 10 20 30 40 10 20 30 40 10 20 30 40 10 20 30 40 2 2 2 2

NrGA 2 3 7 RC-GA CMA-ES CGA Div-GA DPSO SEPSO PSOMS

Rotated Multimodal Hybrid Composition

Function f15 f16 f17 f18 f19 D 10 20 30 40 10 20 30 40 10 20 30 40 10 20 30 40 10 20 30 40

NrGA 3 2 4 3 2 2 2 2 RC-GA CMA-ES CGA Div-GA DPSO SEPSO PSOMS

NrGA ranks 1st (or joint 1st) in 39 and is 2nd in 13 out of a total of 64 cases) (Statistically significant via t test)

(The table appears in reference [4], 2009 IEEE)

17

Experiment 2

Heating Ventilating and Air Conditioning (HVAC) Engineering

Determine optimal (x1, x2), where x1 is chilled H20 supply temperature, x2 is supply air temperature

Simulate using TRNSYS software 10 sec. per fitness evaluation (PC 3.2 GHz)

18

This figure appears in reference [10], ©2010 Applied Energy

19Reference [10], ©2010 Applied Energy

20This figure appears in reference [10], ©2010 Applied Energy

21Conventional gradient descent based method, e.g. BFGS, perform very poorly on such landscape

This figure appears in reference [10], ©2010 Applied Energy

22

Comparison of GA and NrGA

This figure appears in reference [10], ©2010 Applied Energy

23

Optimal Settings

This table appears in reference [10], ©2010 Applied Energy

24

Theoretical Results

On 1Max-0Spike problem, Progressive Randomized local search – a simplified NrGA - solve it in expected time O(nlogn), n is the problem size

(1+1) EA solve it in ((n/2)n) For some problem, a search algorithm can

change the expected time complexity from exponential to polynomial by storing visited solutions and disallows revisits

25

Other Non-revisiting Stochastic Search NrGA has been modified to work on a

combinatorial optimization problem: Traveling Salesman Problem. It outperforms GA (compare using wall clock time)

AM stands for parameter-less adaptive mutation used in NrGA, EN, EX, SI and DP stand for other famous mutation used in TSP problems Note that standalone AM is better than other famous TSP mutation operators

(The table appears in reference [5], 2008 IEEE)

26

NrSA outperforms adaptive simulated annealing (ASA) NrSA is better than ASA in 20 out of 24 cases

(The table appears in reference [6], 2008 IEEE)

27

NrPSO outperforms three improved PSOs NrPSO ranks 1st (or joint 1st) in 10 out of 11 cases

(The table appears in reference [7], 2009 IEEE)

28

NrGP outperforms the conventional GP Symbolic regression (function estimation)

Logic circuit synthesis

(The tables appear in reference [3], 2009 IEEE)

29

Conclusions

Introduce a new search method: Non-revisiting stochastic search

It is a useful add-on to existing evolutionary algorithms NrX; X = GA, SA, PSO, GP shows improvement

Previous EA method throws away valuable information; remembering all past information and use them intelligently will significantly enhance search performance

The method is “parameter-less”. It obviates the difficult “parameter control” problem in EA.

30

Acknowledgments

Dr. Chi Kin Chow (EE) – co-inventer of NrSS Dr. Square Fong (Division of Building Science and

Techology) – collaborator in HVAC engineering Dr. Albert Sung (EE) – collaborator in theoretical

analysis of evolutionary algorithm with memory Mr. Leung Shing Wa – collaborator in NrGP

project

All published results can be found in my web page

31

前事不忘，後事之師《戰國策．趙策一 》Past experience, if not forgotten, will serve as a teacher in future decisions

32

References[1] Yuen, S.Y., Chow, C.K., Continuous Non-revisiting Genetic Algorithm, Proc. IEEE

Congress on Evolutionary Computation (CEC) (May 2009) 1896-1903.[2] Yuen, S.Y., Chow, C.K., A Study of Operator and Parameter Choices in Non-revisiting

Genetic Algorithm, Proc. IEEE Congress on Evolutionary Computation (CEC) (May 2009) 2977-2984.

[3] Yuen, S.Y., Leung, S.W., Genetic Programming that Ensures Programs are Original, Proc. IEEE Congress on Evolutionary Computation (CEC) (May 2009) 860-866.

[4] Yuen, S.Y., Chow, C.K., A Genetic Algorithm that Adaptively Mutates and Never Revisits, IEEE Transactions on Evolutionary Computation, Vol 13(2) (April 2009) 454-472.

[5] Yuen, S.Y., Chow, C.K., Applying Non-revisiting Genetic Algorithm to Traveling Salesman Problem, Proc. IEEE Congress on Evolutionary Computation (CEC) (June 2008) 2217-2224.

[6] Yuen, S.Y., Chow, C.K., A Non-revisiting Simulated Annealing Algorithm, Proc. IEEE Congress on Evolutionary Computation (CEC) (June 2008) 1886-1892.

[7] Chow, C.K., Yuen, S.Y., A Non-revisiting Particle Swarm Optimization, Proc. IEEE Congress on Evolutionary Computation (CEC) (June 2008) 1879-1885.

[8] Sung, C.W., Yuen, S.Y., On the analysis of the (1+1) Evolutionary Algorithm with Short-term Memory, Proc. IEEE Congress on Evolutionary Computation (CEC) (June 2008) 235-241.

[9] Yuen S.Y., Chow, C.K., A Non-revisiting Genetic Algorithm, Proc. IEEE Congress on Evolutionary Computation, Singapore (CEC) (Sept. 2007) 4583-4590.

[10] K.F. Fong, S.Y. Yuen, C.K. Chow, S.W. Leung, Energy Management and Design of Centralized Air-conditioning Systems Through the Non-revisiting Strategy for Heuristic Optimization Methods, Applied Energy, Vol. 87(11), (Nov. 2010) 3494-3506. (doi:10.1016/j.apenergy.2010.05.002)