Optimization withGenetic Algorithms

Walter Reade

October 31, 2002

TAG Meeting

Outline

Background on Optimization Introduction to Genetic Algorithms Using GAs to Solve Difficult Problems A MatLab Implementation Summary / Questions

How Do We Find the Minimum?

Gradient Methods(Steepest Descent)

Move in the direction of steepest gradient.

Simple to implement, guaranteed convergence.

Must know something about the derivative.

Can easily get stuck in a local minimum.

Stochastic Methods

HeuristicUsing “Rules of Thumb”

MetaheuristicA framework of heuristics used to update a set

of solutions during a search.

Simulated Annealing Tabu Search Ant Colony Systems

Genetic Algorithms

Use a population of possible solutions to the search space.

Each solution is encoded in a string called a chromosome (or genome).

Chromosomes are evaluated for fitness each generation (iteration); chromosomes that are more fit have a high probability of surviving.

Genetic Algorithms (cont.)

Once the surviving population is chosen, different “parent” chromosomes are combined to form “child” chromosomes.

Chromosomes may undergo mutation. A new generation is formed, the process is

repeated. By selection, cross-over, and mutation, GAs

search the solution space while creating stronger solutions over each generation.

Fitness and Selection

Roulette Wheel Competition Etc.

Cross-Over

Replaces two parent solutions with two children solutions.

Mechanism for covering large area of search space.

Mutation

Operates on a single chromosome.

Mechanism to improve local search space.

Advantages to using GAs

Flexible and adaptive to a wide variety of problems.

Robust, global search capability. Does not require the solution space to be

smooth, continuous, or differentiable. Can be used in permutation problems. No practical drawbacks.

Slow local convergencePerceived learning overhead

Applications

Function optimization Job shop scheduling Process planning Assembly line optim. Process control Airplane landing Nested design Keyboard layout

Creativity VLSI Traveling Sales Man Chemical kinetics Etc. Etc. Etc.

Solving Difficult Problems

Difficult Problems

Appeared in Jan/Feb 2002 SIAM News in the 100-Dollar, 100-Digit Challenge.

exp(sin(50*x)) + sin(60*exp(y)) + sin(70*sin(x)) + sin(sin(80*y)) - sin(10*(x+y)) + 0.25*(x^2 + y^2)

The genetic algorithm was able to solve this to 10 digits of precision in 2000 generations, which took 25 seconds on a P-III 1.0 GHz. (35% success rate)

Permutation (Order-based) Problems

Time-share ExampleOne condo building at a ski resortFour identical condo units16 week ski season – 64 total owners2nd choice = 2 free lift tickets per person, 3rd choice = 5

free tickets, otherwise $$ and 7 free tickets.5 out of 16 weeks are twice as popularMaximum occupancy = 22

Possible solutions: 1x1067

Results of GA

A previous published result (using SA) found a minimum of 224 after 261 iterations, and no improvement after 1,000,000 iterations.

The GA found a cost of 200 after 2,150 iterations, and a minimum of 172 after 250,000 iterations.

(Author of previous work was “astonished” at the new result.)

Using GAs in MatLab

http://www.ie.ncsu.edu/mirage/GAToolBox/gaot/

MatLab Code% Bounds on the variablesbounds = [-5 5; -5 5];

% Evaluation FunctionevalFn = 'Four_Eval';evalOps = [];

% Generate an intialize population of size 80startPop=initializega(80,bounds,evalFn,[1e-10 1]);

% GA Options [epsilon float/binary display]gaOpts=[1e-10 1 0];

% Termination Operators -- 500 GenerationstermFns = 'maxGenTerm';termOps = [500];

% Selection FunctionselectFn = 'normGeomSelect';selectOps = [0.08];

% Crossover OperatorsxFns = 'arithXover heuristicXover simpleXover';xOpts = [1 0; 1 3; 1 0];

% Mutation OperatorsmFns = 'boundaryMutation multiNonUnifMutation

nonUnifMutation unifMutation';mOpts = [2 0 0;3 200 3;2 200 3;2 0 0];

% Apply the genetic algorithm [soln endPop bestPop

trace]=ga(bounds,evalFn,evalOps,startPop,gaOpts,termFns,termOps,selectFn,selectOps,xFns,xOpts,mFns,mOpts);

Evaluation Function

function [x, soln] = Four_Eval(x, options)

soln = -(exp(sin(50*x(1))) + sin(60*exp(x(2))) + sin(70*sin(x(1))) + ...

sin(sin(80*x(2))) - sin(10*(x(1)+x(2))) + 0.25*(x(1)^2 + x(2)^2));

Time Share Evaluation Functionfunction [assignment, soln] = local_min(assignment, options)

global family_info

cost = 0;occupancy = zeros(16,1);

for i = 1:64 if ceil(assignment(i)/4) == family_info(i,2) % first choice cost = cost + 0; elseif ceil(assignment(i)/4) == family_info(i,3) % second choice cost = cost + 2*family_info(i,1); elseif ceil(assignment(i)/4) == family_info(i,4) % third choice cost = cost + 4*family_info(i,1); else

% didn't get any choice cost = cost + 50 + 7*family_info(i,1); end building = ceil(assignment(i)/4); occupancy(building) = occupancy(building) + family_info(i,1); end

for i = 1:16 if occupancy(building) > 22 cost = cost + 1000; endend

soln = -cost;

Summary

Genetic Algorithms are:PowerfulFlexibleEasy to use and understand

Consider using a GA for your next optimization problem!