Omni-Optimizer A Procedure for Single and Multi-objective Optimization Prof. Kalyanmoy Deb and Santosh Tiwari

Omni-OptimizerOmni-Optimizer

A Procedure for Single and A Procedure for Single and Multi-objective OptimizationMulti-objective Optimization

Prof. Kalyanmoy Deb andProf. Kalyanmoy Deb and

Santosh TiwariSantosh Tiwari

MotivationMotivation

Generic Programming PracticesGeneric Programming Practices Unified algorithm for all types of Unified algorithm for all types of

optimization problemsoptimization problems An efficient implementation of NSGA-II An efficient implementation of NSGA-II

framework (procedure)framework (procedure) Developing an efficient and self-Developing an efficient and self-

adaptive optimization paradigmadaptive optimization paradigm

Literature SurveyLiterature Survey

CHC (Cross generation elitist selection, CHC (Cross generation elitist selection, Heterogeneous recombination, Cataclysmic Heterogeneous recombination, Cataclysmic mutation) – mutation) – Explicit DiversityExplicit Diversity

GENITOR (Steady state GA), more like GENITOR (Steady state GA), more like (µ+1)-ES so far as selection mechanism is (µ+1)-ES so far as selection mechanism is concerned. – concerned. – High selection pressureHigh selection pressure

NPGA (Niched Pareto Genetic Algorithm), NPGA (Niched Pareto Genetic Algorithm), uses sharing parameter σuses sharing parameter σshareshare – – # of niches # of niches obtained depend on the sharing parameterobtained depend on the sharing parameter

Literature Survey contd…Literature Survey contd… PESA (Pareto Envelope-based Selection Algorithm), PESA (Pareto Envelope-based Selection Algorithm),

Hyper-grid division of phenotypic space, selection Hyper-grid division of phenotypic space, selection based on crowding measurebased on crowding measure

NSGA-II (Non-dominated Sorting Genetic Algorithm)NSGA-II (Non-dominated Sorting Genetic Algorithm) SPEA2 (Strength Pareto Evolutionary Algorithm), Fine SPEA2 (Strength Pareto Evolutionary Algorithm), Fine

grained fitness assignment mechanism utilizing density grained fitness assignment mechanism utilizing density information, Only archive members participate in information, Only archive members participate in mating – mating – Excellent Diversity in phenotypic spaceExcellent Diversity in phenotypic space

NCGA (Neighborhood Cultivation Genetic Algorithm), NCGA (Neighborhood Cultivation Genetic Algorithm), used neighborhood crossover, based on NSGA-II and used neighborhood crossover, based on NSGA-II and SPEA2SPEA2

RPSGAe (Reduced Pareto Set Genetic Algorithm with RPSGAe (Reduced Pareto Set Genetic Algorithm with elitism)elitism)

ENORA (Evolutionary Algorithm of Non-dominated ENORA (Evolutionary Algorithm of Non-dominated Sorting with Radial Slots)Sorting with Radial Slots)

Salient Features of the Salient Features of the AlgorithmAlgorithm

Based on NSGA-II frameworkBased on NSGA-II framework Based on the concept of Pareto dominanceBased on the concept of Pareto dominance Incorporates elitismIncorporates elitism Explicit diversity preserving mechanismExplicit diversity preserving mechanism

Can be used for single-objective as well as Can be used for single-objective as well as multi-objective problemsmulti-objective problems

Can be used for uni-global as well as multi-Can be used for uni-global as well as multi-global problemsglobal problems

Independent of the number of niches that Independent of the number of niches that an optimization problems exhibitsan optimization problems exhibits

Moving beyond NSGA-IIMoving beyond NSGA-II

Restricted Selection SchemeRestricted Selection Scheme Tournament selection based on usual dominationTournament selection based on usual domination Non-dominated sorting based on epsilon dominanceNon-dominated sorting based on epsilon dominance

Crowding Distance AssignmentCrowding Distance Assignment Genotypic as well as Phenotypic space nichingGenotypic as well as Phenotypic space niching Choose best members from above average populationChoose best members from above average population Remove worst members from below average populationRemove worst members from below average population

More robust recombination and variation operatorsMore robust recombination and variation operators Two point crossover for binary variablesTwo point crossover for binary variables Highly disruptive real variable mutationHighly disruptive real variable mutation

Restricted SelectionRestricted Selection Helps in preserving multi-modalityHelps in preserving multi-modality Experiments show that it gives faster overall Experiments show that it gives faster overall

convergenceconvergence

Epsilon Domination PrincipleEpsilon Domination Principle

A finite percentage (based on A finite percentage (based on function value) of solutions function value) of solutions assigned a particular rankassigned a particular rank

Allows somewhat inferior Allows somewhat inferior solutions to remain in the solutions to remain in the populationpopulation

Provides guaranteed diversityProvides guaranteed diversity Helps to obtain multi-modal Helps to obtain multi-modal

solutions in case of single solutions in case of single objective problemsobjective problems

Epsilon is generally taken in the Epsilon is generally taken in the range 10range 10-3-3 ~ 10 ~ 10-6-6

Modified Crowding DistanceModified Crowding Distance Genotypic as well as Phenotypic space nichingGenotypic as well as Phenotypic space niching

Highly Disruptive Mutation Highly Disruptive Mutation OperatorOperator

1 2

11 1

1

1 1 1

2

,

2 (1 2 )*(1 ) 1 if r 0.5

1 2(1 ) 2( 0.5)*(1 ) otherwise

( )

m m

m m

p l u p

u l u l

q

c p q p l

x x x x

x x x x

r r

r r

x x x x

Computational ComplexityComputational Complexity

Restricted selection Restricted selection O O (n(nNN22)) Ranking procedure Ranking procedure O O (M(MNN22)) Crowding procedureCrowding procedure

max{ max{ O O ((MNMN log log NN), ), O O ((nNnN log log NN) }) } Overall iteration-wise complexityOverall iteration-wise complexity

max {max {O O (n(nNN22), ), O O ((MNMN22), ), O O ((nNnN log log NN)})}

Implementation DetailsImplementation Details

Code written in simple C and strictly Code written in simple C and strictly conforms to ANSI/ISO standardconforms to ANSI/ISO standard

Data structure used is a custom doubly Data structure used is a custom doubly linked list (gives O(1) insertion and linked list (gives O(1) insertion and deletion)deletion)

Randomized quick sort used for sortingRandomized quick sort used for sorting Almost all the functions perform in-place Almost all the functions perform in-place

operation (addresses are passed, operation (addresses are passed, significantly decreases stack overheads)significantly decreases stack overheads)

Simulation ResultsSimulation Results

GA parameters for all problems chosen as GA parameters for all problems chosen as followsfollows ηηcc = 20 = 20 ηηmm = 20 = 20 P (crossover) = 0.8P (crossover) = 0.8 P (mutation) = 1/n, where n = # of real variablesP (mutation) = 1/n, where n = # of real variables δδ = 0.001 = 0.001 Population size and number of generations Population size and number of generations

different for different problemsdifferent for different problems

Simulation Results contd…Simulation Results contd… 20 variable Rastrigin function20 variable Rastrigin function

# of function evaluation# of function evaluation Least = 19260Least = 19260 Median = 24660Median = 24660 Worst = 29120Worst = 29120

20 variable Schwefel function20 variable Schwefel function # of function evaluation# of function evaluation

Least = 54950Least = 54950 Median = 69650Median = 69650 Worst = 103350Worst = 103350

Other single objective problems can be found in the Other single objective problems can be found in the paperpaper

In all cases, better results are found in comparison to In all cases, better results are found in comparison to previous reported studiesprevious reported studies

Single objective multi-modal Single objective multi-modal functionfunction

f(x) = sinf(x) = sin22 ( (ππx)x) x x єє [0,20] [0,20]

Single objective multi-modal Single objective multi-modal functionfunction

Unconstrained Himmelblau’s functionUnconstrained Himmelblau’s function

Multi-objective Uni-Global Test Multi-objective Uni-Global Test ProblemsProblems

30 variable ZDT2 (100×100)30 variable ZDT2 (100×100)


10 variable ZDT4 (100×250)10 variable ZDT4 (100×250)


CTP4 (100×7000)CTP4 (100×7000)


CTP8 (100×100)CTP8 (100×100)


DTLZ4 (300×100)DTLZ4 (300×100)

Multi-objective Multi-Global Test Multi-objective Multi-Global Test ProblemProblem

FF11 (x) = summation (sin ( (x) = summation (sin (ππxxii) )) ) xxii єє [0,6] [0,6]FF22 (x) = summation (cos ( (x) = summation (cos (ππxxii) )) ) xxii єє [0,6] [0,6]

Efficient points in phenotypic space

Multi-objective Multi-Global Test Multi-objective Multi-Global Test ProblemProblem

Genotypic space plotsGenotypic space plots

Few Sample SimulationsFew Sample Simulations

F(x) = sinF(x) = sin22 (10,000*pi*x) (10,000*pi*x) Himmelblau’s FunctionsHimmelblau’s Functions ZDT Test ProblemsZDT Test Problems CTP Test ProblemsCTP Test Problems Test Problem TNKTest Problem TNK Multi-global Multi-objective Test Multi-global Multi-objective Test

ProblemProblem

Further Ideas and Future Further Ideas and Future WorkWork

Incorporating PCX instead of SBX for Incorporating PCX instead of SBX for crossovercrossover

Automatically fine-tuning mutation Automatically fine-tuning mutation index so as to achieve arbitrary index so as to achieve arbitrary precisionprecision

Self-adaptation of parameter Self-adaptation of parameter δδ Segregating population into niches Segregating population into niches

without the introduction of DMwithout the introduction of DM Dynamic population sizingDynamic population sizing Using hierarchical NDS for the Using hierarchical NDS for the

crowding distance assignmentcrowding distance assignment

Documents

Omni-Optimizer A Procedure for Single and Multi-objective Optimization Prof. Kalyanmoy Deb and Santosh Tiwari