Embed Size (px)
DESCRIPTION
This work provides an in-depth investigation of the effects of population initialization on Differential Evolution (DE) for dealing with large scale optimization problems. Firstly, we conduct a statistical parameter sensitive analysis to study the effects of DE’s control parameters on its performance of solving large scale problems. This study reveals the optimal parame- ter configurations which can lead to the statistically superior performance over the CEC-2013 large-scale test problems. Thus identified optimal parameter configurations interestingly favour much larger population sizes while agreeing with the other parameter settings compared to the most commonly employed parameter configuration. Based on one of the identified optimal configurations and the most commonly used configuration, which only differ in the population size, we investigate the influence of various population initialization techniques on DE’s performance. This study indicates that initialization plays a more crucial role in DE with a smaller population size. However, this observation might be the result of insufficient convergence due to the use of a large population size under the limited computational budget, which deserve more investigations.
Citation preview
Effects of
Population Initialization on
Differential Evolution for
Large-Scale OptimizationBorhan KazimipourXiaodong LiA.K. Qin
Outlines
1. Introduction
2. Background
3. Experiments
4. Future Work
5. Questions
CEC 2014, Beijing, China 2Population Initialization on DE for LSO
Outlines
1. Introduction
2. Background
3. Experiments
4. Future Work
5. Questions
CEC 2014, Beijing, China 3Population Initialization on DE for LSO
What is it all about?
CEC 2014, Beijing, China 4Population Initialization on DE for LSO
Investigating the effect of advanced population initialization techniques on DE for large scale
Large scale is not
discovered
Initialization is important
DE is powerful
What is it all about?
• What is missing in previous works?– Lack: Large scale optimization has received little attention on this topic.
– Ambiguity: Advanced statistical tools have not been employed to validate the significance of improvement.
– Contradiction: Some claims appose others
CEC 2014, Beijing, China 5Population Initialization on DE for LSO
What is it all about?
CEC 2014, Beijing, China 6Population Initialization on DE for LSO
Contribution
Dimensionality
Advanced Statistical Tests
Parameter Configuration
What is it all about?
• Basic Facts
– Differential Evolution (DE) is one the most promising optimizers and winner of many optimization competitions.
– Many researches have claimed that adopting advanced population initialization techniques improve Evolutionary Algorithms (EAs), including DE.
– Those claims have not been deeply studied in Large Scale Optimization (LSO) domain.
• The Goal
– This research aims to study the effect of advanced population initialization techniques on a widely used DE variant when it comes to deal with large scale problems considering the effects of DE parameter setting.
CEC 2014, Beijing, China 7Population Initialization on DE for LSO
Outlines
1. Introduction
2. Background
3. Experiments
4. Future Work
5. Questions
CEC 2014, Beijing, China 8Population Initialization on DE for LSO
Definitions
•
CEC 2014, Beijing, China 9Population Initialization on DE for LSO
Differential Evolution (DE)
CEC 2014, Beijing, China 10Population Initialization on DE for LSO
+ Effective: the winner of many competitions
+ Popular: with many publications and applications
- Population-based algorithm:Sensitive to initial population
- Has parameters to control exploration-exploitation balance:Sensitive to parameter setting
- Suffers from Curse of Dimensionality
Process and Operators
• DE Operators0. Initialization
1. Mutation
2. Recombination
3. Selection
CEC 2014, Beijing, China 11Population Initialization on DE for LSO
0- Population Initialization
• Common Parameters– Number of decision variables or problem dimensionality (given)
– Variable ranges (given)
– Population size
• Technique-Specific Parameters– Chaotic Number Generators: map type and number of iterations
– Uniform Design: number of levels
– Opposition-Based Learning: original population initializer
– …
CEC 2014, Beijing, China 12Population Initialization on DE for LSO
Categorize of Population Initialization
CEC 2014, Beijing, China 13Population Initialization on DE for LSO
Population Initialization
Randomness
Stochastic
Pseudo-Random Number
Generators
Chaotic Number Generator
Deterministic
Quasi-Random Sequence
Uniform Experimental
Design
Compositionality
Non-Composite Composite
Hybrid
Multi-Step
Generality
Generic Application Specific
Categorize of Population Initialization
CEC 2014, Beijing, China 14Population Initialization on DE for LSO
Population Initialization
Randomness
Stochastic
Pseudo-Random Number
Generators
Chaotic Number Generator
Deterministic
Quasi-Random Sequence
Uniform Experimental
Design
Compositionality
Non-Composite Composite
Hybrid
Multi-Step
Generality
Generic Application Specific
1- Mutation
• A DE Mutation Strategy (rand/1)
– r1 and r2 are randomly chosen from population
– F is scaling factor
• Scaling Factor (F)– Controls exploration-exploitation balance
– Too small F values increase the risk of premature convergence
– Too large F values decrease the convergence speed, degrades efficiency and may result in early termination
CEC 2014, Beijing, China 15Population Initialization on DE for LSO
2- Recombination
• Binomial Crossover
• Crossover Rate (CR)– CR determines the number of variables of target vector which must be interchanged
with the corresponding variables of mutant vector
– Small CR values can boost convergence speed when a few decision variables are interacting with each others (separable functions)
– Large CR values are more effective when lots of decision variables are interacting (non-separable functions).
In dealing with black-box problems, we have no idea about the separability of the objective function.
CEC 2014, Beijing, China 16Population Initialization on DE for LSO
3- Selection
• Elite Selection
CEC 2014, Beijing, China 17Population Initialization on DE for LSO
3- Selection
• Elite Selection
CEC 2014, Beijing, China 18Population Initialization on DE for LSO
Yes!No more parameters!
Differential Evolution (DE)
• Important Parameters– NP: Population Size
– CR: Crossover Rate
– F: Scale Factor
CEC 2014, Beijing, China 19Population Initialization on DE for LSO
Differential Evolution (DE)
• Important Parameters– NP: Population Size
– CR: Crossover Rate
– F: Scale Factor
CEC 2014, Beijing, China 20Population Initialization on DE for LSO
Population Initialization Technique
Outlines
1. Introduction
2. Background
3. Experiments
4. Future Work
5. Questions
CEC 2014, Beijing, China 21Population Initialization on DE for LSO
Experiments
• Two- parts Experiment:
A.Parameter Calibration
– Aim: to find the most effective parameter configuration for DE/rand/1/bin to deal with large scale problems
B.Population Initialization
– Aim: to investigate whether advanced population initialization techniques can improve common techniques using the most effective parameter setting.
CEC 2014, Beijing, China 22Population Initialization on DE for LSO
Experiments SetupParts A & B
• Benchmark– CEC 2013 LSGO Benchmarks
– 15 functions
– 1000 dimensions
– Categories
1. fully separable functions (f1 - f3),
2. partially separable functions with a separable subcomponent (f4 - f7),
3. partially separable functions with no separable subcomponents (f8 - f11),
4. overlapping functions (f12 - f14),
5. fully non-separable function (f15).
• Statistical Tests– Iman and Davenport (a.k.a. Friedman rank) test is used for ranking
– Li post-hoc procedure is used as significance test
CEC 2014, Beijing, China 23Population Initialization on DE for LSO
ExperimentsPart A
A. Parameter Calibration– PRNG as population initializer
– 14 population sizes [10,20,30,40,50,60,70,80,90,100,150,200,250,300]
– 3 CR values [0.1, 0.5, 0.9]
– 2 F values [0.5, 0.8]
– 84 configurations X 15 functions X 51 runs
CEC 2014, Beijing, China 24Population Initialization on DE for LSO
Experiments ResultsPart A
• Range of parameter values which perform significantly better than the others based on Li post-hoc procedure
– NP = [80,90,100,150,200,250]
– CR = 0.9
– F = 0.5
CEC 2014, Beijing, China 25Population Initialization on DE for LSO
Experiments ResultsPart A
• Among 84 configurations (on 15 functions in 51 runs), the best configuration based on Iman and Davenport test is found to be:
– NP = 150
– CR = 0.9
– F = 0.5
CEC 2014, Beijing, China 26Population Initialization on DE for LSO
Experiments ResultsPart A
CC: 3,000,000 FE = 50 NP X 60,000 iterations
CS: 3,000,000 FE = 150 NP X 20,000 iterations
CEC 2014, Beijing, China 27Population Initialization on DE for LSO
Experiments ResultsPart A
• What we learn from Part A?
– NP, CR an F must be set carefully.
– NP is more relaxed than CR and F values.
– Most effective values for CR and F are the same in low and high dimensional problems
– Higher dimension problems demand larger populations (even if computational budget is fixed.)
– Note: The findings are based on the dedicated computational budget; Large increment or decrement of this limit may affect the results.
CEC 2014, Beijing, China 28Population Initialization on DE for LSO
ExperimentsPart B
A. Parameter Calibration – PRNG as population initializer
– 14 population sizes [10,20,30,40,50,60,70,80,90,100,150,200,250,300]
– 3 CR values [0.1, 0.5, 0.9]
– 2 F values [0.5, 0.8]
– 84 configurations X 15 functions X 51 runs
B. Population Initialization Techniques– Pseudo-Random Number Generators (PRNG)
– Chaotic Number Generator (CNG)
– Sobol Ste (SBL)
– Good Lattice Point (GLP)
– Opposition-Based Learning (OBL)
– Quasi-Opposition-Based Learning (QOBL)
CEC 2014, Beijing, China 29Population Initialization on DE for LSO
Using the best configuration found
in Part A
Experiments ResultsPart B
No significant improvement is visible
CEC 2014, Beijing, China 30Population Initialization on DE for LSO
Experiments ResultsPart B
• What we learn from Part B?
– Advanced population initializers may improve DE/rand/1/bin, but not significantly.
– When proper values for the control parameters are used, population initialization has only a minor effect.
– Size of population plays more important role than the way it is initialized.
– Note: The findings are based on the dedicated computational budget; Large increment or decrement of this limit may affect the results.
CEC 2014, Beijing, China 31Population Initialization on DE for LSO
Discussions
• Important finding:– Obtained results challenges the general belief of significant advantages of
advanced techniques in high dimensional spaces.
• How we discuss the contradiction?1. Significant effect of parameters: None of the previous studies has tried to
compare population initialization techniques on the well-tuned optimizers.
2. Importance of advanced statistical tools: Some statistically minor improvements of using advanced initializers may wrongly considered as significant contributions.
Note: This study is well-conducted based on a systematic framework and the findings are statistically validated. However, the we are well aware of the need of further investigations to generalise the findings from DE/rand/1/bin to other EAs.
CEC 2014, Beijing, China 32Population Initialization on DE for LSO
Outlines
1. Introduction
2. Background
3. Experiments
4. Future Work
5. Questions
CEC 2014, Beijing, China 33Population Initialization on DE for LSO
Future Work
• Expansion to other EAs:– Repeating this study on other popular EAs for further generalization of
the findings.
• Involving other metrics:– Considering other performance metrics besides final objective values
can help us to investigate whether advanced initializers are able to significantly improve EAs according to other aspects
• Effect of budget– computational budget has significant effects on the performance of EAs
when armed with different population initialization techniques.
CEC 2014, Beijing, China 34Population Initialization on DE for LSO
Outlines
1. Introduction
2. Background
3. Experiments
4. Future Work
5. Questions
CEC 2014, Beijing, China 35Population Initialization on DE for LSO
Thank you☺☺☺☺
Any question or comment?
36CEC 2014, Beijing, China Population Initialization on DE for LSO
