Alexandria ACM SC | Introduction to Genetic Algorithms

7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms

1/45

INTRODUCTION TOGENETIC

ALGORITHMS

BY

AHMED MEDHAT OTHMAN

@amedhat3


2/45

AGENDA

1. Introduction

2. Natural Inspired Computing

3. Classical Computation vs. bio-inspired computing

4. Evolution in the real world5. Collaborative discussion

6. Problem solving

7. Genetic Algorithms (GA)

8. Anatomy of a GA

9. Advantages of GA

10. Limitations of GA

11. Related techniques12. Summary

2


3/45

INTRODUCTION

3


4/45

INTRODUCTION

4


5/45

INTRODUCTION

5


6/45

INTRODUCTION

6


7/45

INTRODUCTION

7


8/45

INTRODUCTION

8


9/45

INTRODUCTION

The only intelligent systems on this planet are biological. Biological intelligences are designed by natural

evolutionary processes.

They often work together in groups, swarms, or flocks. They don't appear to use logic, mathematics, complex

planning, complicated modeling of their environment.

They can achieve complex information processing andcomputational tasks that current artificial intelligences

find very challenging indeed.

9


10/45

NATURAL INSPIRED

COMPUTING

10


11/45

NATURAL INSPIRED

COMPUTING

In other words Biologically Inspired Computing. Biological organisms cope with the demands of their

environments.

They uses solutions quite unlike the traditional human-engineered approaches to problem solving.

They exchange information about what theyve discoveredin the places they have visited.

Bio-inspired computing is a field devoted to tacklingcomplex problems using computational methods modeled

after design principles encountered in nature.

11


12/45

CLASSICAL COMPUTATION

VS.

BIO-INSPIRED COMPUTING

12


13/45


VS.


Classical computing is good at: Number-crunching Thought-support (glorified pen-and-paper) Rule-based reasoning Constant repetition ofwell-definedactions.

Classical computing is bad at:

Pattern recognition Robustness to damage Dealing with vague and incomplete information; Adapting and improving based on experience

13


14/45


VS.


Bio-inspired computing takes a more evolutionaryapproach to learning.

In traditional AI, intelligence is often programmed fromabove. The Programmer create the program and imbues it

with its intelligence.

Bio-inspired computing, on the other hand, takes a morebottom-up, decentralized approach.

Bio-inspired computing often involve the method ofspecifying a set of simple rules, a set of simple organisms

which adhere to those rules.

14


15/45

EVOLUTION IN THE

REAL WORLD

Each cell of a living thing containschromosomes - strings ofDNA.

Each chromosome contains a set ofgenes - blocks of DNA.

Each gene determines some aspectof the organism (like eye colour).

A collection ofgenes is sometimescalled a genotype.

A collection ofaspects (like eyecharacteristics) is sometimes called

a phenotype.

15


16/45

EVOLUTION IN THE

REAL WORLD

Reproduction involves recombination of genes from parentsand then small amounts ofmutation (errors) in copying.

The fitness of an organism is how much it can reproducebefore it dies.

Evolution based on survival of the fittest.

16


17/45

COLLABORATIVE

DISCUSSION

What we can learn from nature? Applications of nature in the engineering field.

17


18/45

PROBLEM SOLVING

Suppose you have a problem.

You dont know how to solve it. What can you do? Can you use a computer to somehow find a solution? This would be nice! Can it be done?

18


19/45

PROBLEM SOLVING

Brute-Force Solution

A blind generate and test algorithm:

Repeat

Generate a random possible solution

Test the solution and see how good it is

Until solution is good enough

19


20/45

PROBLEM SOLVING

Can we use this Brute-Force idea?

Sometimes -YES: if there are only a few possible solutions and you have enough time then such a method couldbe used

For most problems - NO: many possible solutions with no time to try them all so this method cannotbe used

20


21/45

PROBLEM SOLVING

Search Techniques

Calculus Base

TechniquesGuided random search

techniques

Enumerative

Techniques

BFSDFS Dynamic

Programming

Tabu Search HillClimbing

SimulatedAnnealing

EvolutionaryAlgorithms

Genetic

Programming

Genetic

Algorithms

Fibonacci Sort

21


22/45

GENETIC ALGORITHMS

22

Initialize Population

satisfy constraints ?

Evaluate Fitness

Select Survivors

Output Results

Randomly Vary Individuals

Yes

No


23/45

GENETIC ALGORITHMS

How do you encode a solution?

Obviously this depends on the problem! GAs often encode solutions as fixed length

bitstrings (e.g. 101110, 111111, 000101)

Each bit represents some aspect of the proposed solutionto the problem

For GAs to work, we need to be able to test any stringand get a score indicating how good that solution is.

23


24/45

GENETIC ALGORITHMS

The set of all possible solutions [0..1000] is called thesearch space orstate space.

In this case its just one number but it could be manynumbers.

Often GAs code numbers in binary producing a bitstringrepresenting a solution.

We choose 1,0 bits which is enough to represent 0..1000

24


25/45

GENETIC ALGORITHMS

Example: encoding 4 parameters Param1 value = 1000 = 8 Param2 value = 1011 = 11 Etc.,

25

Binary Representation


26/45

GENETIC ALGORITHMS

Search Space

For a simple function f(x) the search space is onedimensional.

But by encoding several values into the chromosomemany dimensions can be searched e.g. two dimensionsf(x,y).

Search space can be visualised as a surface orfitnesslandscape in which fitness dictates height.

Each possible genotype is a point in the space. A GA tries to move the points to better places (higher

fitness) in the the space.

26


27/45

GENETIC ALGORITHMS

Fitness landscapes

27


28/45

GENETIC ALGORITHMS

Implicit fitness functions

Most GAs use explicit and static fitness function Some GAs (such as in Artificial Life or Evolutionary Robotics) use

dynamic and implicit fitness functions - like how many obstacles

did I avoid

Individuals fitness

Average fitness of population

28


29/45

GENETIC ALGORITHMS

Example - Drilling for Oil

Imagine you have to drill for oil somewhere along a single1 km desert road.

Problem: choose the best place on the road that producesthe most oil per day.

We could represent each solution as a position on theroad.

Say, a whole number between [0..1000]

29


30/45

GENETIC ALGORITHMS

Where to drill for oil?

30

0 500 1000

Road

Solution2 = 900Solution1 = 300


31/45

GENETIC ALGORITHMS

In GAs these encoded strings are sometimes calledgenotypesor chromosomes and the individual bits are

sometimes called genes

31

512 256 128 64 32 16 8 4 2 1

900 1 1 1 0 0 0 0 1 0 0

300 0 1 0 0 1 0 1 1 0 0

1023 1 1 1 1 1 1 1 1 1 1

Convert to binary string


32/45

GENETIC ALGORITHMS

32

0 1000

Road

Solution2 = 900

(1110000100)

Solution1 = 300

(0100101100)

OI

L

Location

30

5


33/45

ANATOMY OF GA

Selecting Parents

Many schemes are possible so long as better scoringchromosomes more likely selected

Score is often termed the fitness Roulette Wheel selection can be used:

Add up the fitness's of all chromosomesGenerate a random number R in that range

Select the first chromosome in the population that - when allprevious fitnesss are added - gives you at least the value R

33


34/45

ANATOMY OF GA

34

Example population

No. Chromosome Fitness1 1010011010 12

1111100001

2

3 1011001100 34 1010000000 15 0000010000 36 1001011111 57 0101010101 18 1011100111 2


35/45

ANATOMY OF GA

35

Roulette Wheel Selection

1 2 3 1 3 5 1 2

0 18

21 3 4 5 6 7 8

Rnd[0..18] = 7

Chromosome4

Parent1

Rnd[0..18] = 12

Chromosome6

Parent2


36/45

ANATOMY OF GA

36

Crossover - Recombination

0100101100

1110000100

Crossoversingle point -

random

0100000100

1110101100

Parent1

Parent2

Offspring1

Offspring2

With some high probability (crossover rate) applycrossover to the parents.

(typical values are 0.8 to 0.95)


37/45

Why does crossover work?

A lot of theory about this and some controversy The idea is that crossover preserves good bits from

different parents, combining them to produce better

solutions

A good encoding scheme would therefore try to preservegood bits during crossover and mutation

37

ANATOMY OF GA


38/45

ANATOMY OF GA

38

Mutation

1011011111

1010000000

Offspring1

Offspring2

1011001111

1000000000

Offspring1

Offspring2

mutate

Original offspring Mutated offspring

With some small probability (the mutation rate) flip each bit in

the offspring (typical values between 0.1 and 0.001)


39/45

ANATOMY OF GA

39

Many Variants of GA

Different kinds of selection (not roulette) Tournament Elitism, etc.

Different recombination Multi-point crossover 3 way crossover etc.

Different kinds of encoding other than bitstringInteger values

Ordered set of symbols Different kinds of mutation


40/45

ADVANTAGES OF GA

Concepts are easy to understand Genetic Algorithms are intrinsically parallel. Always an answer; answer gets better with time Inherently parallel; easily distributed Less time required for some special applications Chances of getting optimal solution are more

40


41/45

LIMITATIONS OF GA

The population considered for the evolution should bemoderate or suitable one for the problem (normally 20-30 or

50-100)

Crossover rate should be 80%-95% Mutation rate should be low i.e. 0.5%-1% assumed as best The method of selection should be appropriate Writing of fitness function must be accurate

41


42/45

RELATED TECHNIQUES

Genetic programming Evolutionary programming Swarm intelligence

Ant colony optimization Particle swarm optimization Intelligent Water Drops Bees algorithm

Neural Networks42


43/45

SUMMARY

43

Representation Binary strings

Recombination N-point or uniform

Mutation Bitwise bit-flipping with fixedprobability

Parent selection Fitness-Proportionate

Survivor selection All children replace parents

Speciality Emphasis on crossover


44/45

4

4


45/45

THANKS

Contact information Email: [email protected] Website: http://amedhat.info Twitter: @amedhat3 Linkedin: linkedin.com/in/amedhat

5

Documents

Alexandria ACM SC | Introduction to Genetic Algorithms