1

Algoritmos Genéticos aplicados em Machine Learning

Controle de um Robo(em inglês)

2

Robot Control using Genetic Algorithms

3

Summary

• Introduction– Robot Control– Khepera Simulator

• Genetic Model for Path Planning– Chromosome Representation– Evaluation Function– Case Studies

• Conclusions

4

The Robot Controller Problem

• Given a robot and a description of an environment, provide commands (motor speeds) to the robot, in order to achieve a path between two specified locations, which is collision-free and satisfies certain optimisation criteria.

(xi, yi)

(xf, yf)

5

Optimisation Criteria

• Robot should:– attempt near-optimal paths– avoid obstacles– perform straight motion

• Controller should be independent of:– the robot’s environment– target location

6

The Khepera Simulator

• Freeware mobile robot simulator (designed by Olivier Michel, University of Nice Sophia-Antipolis)

• User designed worlds

• Control algorithms can be written in C/C++• Robot’s position and angle reading

• 8 sensors (S0-S7): [0, 1023]• 2 motors (M1, M2): [-10, +10]

S0

S1S2 S3

S4

S5

S7 S6

M1 M2

Front identification

7

Simulator Readings:sensors, position and angle

S0

S1S2 S3

S4

S5

S7 S6

M1 M2

Front identification

S0-S7: [0, 1023]S0-S7: [0, 1023]

1000

1000

X

Y

Robot’s World

S0

S1S2 S3

S4

S5

S7 S6

M1 M2

Front identification

angle of the robot with the world

αα : [-: [-ππ , , ππ ]]

xx

yy

0

obstacle not obstacle very

detected closed

8

Control Mode

• To evolve the robot’s attitudes as it interacts with the environment

• Each robot action determines:– how well the controller performs with

respect a given task;– the next input stimuli to the controller.

• The controller should learnlearn as the robot interacts with the environment

9

Controller Model

Genetic Genetic AlgorithmAlgorithm

evolves robot’s evolves robot’s attitudesattitudes

Sensors

Position

Robot’s Angle

Goal Location

Motor 2 Motor 1

KheperaKheperaSimulatorSimulator

10

Proposed Modelbased on human behavior

IF Obstacle detectedObstacle detected

THEN

Avoid collision, forget targetAvoid collision, forget target

ELSE

SStraight to the target according to the traight to the target according to the

target directiontarget direction

END

11

Sensors Reading Simplification

Sleft Sright

Sback

S0

S1S2 S3

S4S5

S7 S6

Sleft = ( S0 + S1 + S2 ) / 3

Sright = ( S3 + S4 + S5 ) / 3

Sback = ( S6 + S7 ) / 2

12

Determining the Target Direction

(x,y)α = π/2

arrival position: (xf ,yf)

β

β = tan-1

[(yf - y)/(xf - x)]

β - απ/2

π

π/4

0

−π/4

−π/2

3π/4

−3π/4

Goal point toright side

Goal point infront

Goal point toleft side

Goal pointbehind

Direction =Direction =

13

Model

IF ((((SSleftleft > L > L) or () or (SSrightright > L > L) or () or (SSbackback > L > L))))

THENObstacle detected, avoid collision, forget target

Proximity-sensor = highest value (SProximity-sensor = highest value (Sleftleft, S, Srightright, S, Sbackback))

ELSEObstacle not detected (collision-free), straight to the target

Target direction = Target direction = ββ - - αα

END

L=collision threshold=900

14

Genetic Algorithm Modelling

• Problem

• Chromosome Representation

• Evaluation Function• Genetic Operators

• Techniques

• Parameters

15

Chromosome Representation

M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2

TargetDirection(collision free) Obstacle detected

Left Right BackFront Left Right Back

Attitudes (Genes)

1 2 3 4 5 6 7

Which speed should be imposed to each motor in Which speed should be imposed to each motor in each situation the robot is?each situation the robot is?

16

Evaluation Function

( )

10;10;10

**7

1

≤≤≤≤≤≤

= ∑=

AiDiVi

AiDiViFi

• Main objectives:– (V) speed: as high as possible– (D) straight motion: same motor speed for M1 e M2– (A) action: reach a target and avoid obstacles

• Calculated based on the contribution for each gene [1,7], at each step.

17

Speed

max)*2/(|)2||1(| MMMVi +=

• Normalised sum of the absolute value of the motors speeds;

• Vi increases as both speeds increase• Whatever the robot does, it does quickly.

18

Straight Motion

=++=

=]7,6,5,1[,2))max)*2()21((1(

]4,3,2[,1

iMMM

iDi

• It favours high positive speeds to both motors

• When the robot is not oriented to the target (2,3,4), D=1 avoids contradictory learning

19

Action

• It considers the benefit of each gene regarding to:– obstacle avoidance– target closeness

=−=≤∆=∆=≤∆=∆

=]7,6,5[,max1

0,)0(])4,3,2[(,max

0,)0()1(,max

iifSS

AAielsegiandiifggi

AAielsediandiifddi

AAi

t

t

t

t

≠

== ∑

=

0,)(

0,0

1

TPiTPiAAi

TPiAi TPi

tt

TPiTPi = total of steps executed by attitude i = total of steps executed by attitude i

AAiAAi=action’s fitness at stept of attitude i=action’s fitness at stept of attitude i

20

Action

• It considers the benefit of each gene regarding to:– obstacle avoidance– target closeness

=−=≤∆=∆=≤∆=∆

=]7,6,5[,max1

0,)0(])4,3,2[(,max

0,)0()1(,max

iifSS

AAielsegiandiifggi

AAielsediandiifddi

AAi

t

t

t

t

≠

== ∑

=

0,)(

0,0

1

TPiTPiAAi

TPiAi TPi

tt

TPiTPi = total of steps executed by attitude i = total of steps executed by attitude i

AAiAAi=action’s fitness at stept of attitude i=action’s fitness at stept of attitude i

Rates the distance variation to the target between

two consecutive steps, and the maximum distance in one step,

for collision free/front

21

Action

• It considers the benefit of each gene regarding to:– obstacle avoidance– target closeness

=−=≤∆=∆=≤∆=∆

=]7,6,5[,max1

0,)0(])4,3,2[(,max

0,)0()1(,max

iifSS

AAielsegiandiifggi

AAielsediandiifddi

AAi

t

t

t

t

≠

== ∑

=

0,)(

0,0

1

TPiTPiAAi

TPiAi TPi

tt

TPiTPi = total of steps executed by attitude i = total of steps executed by attitude i

AAiAAi=action’s fitness at stept of attitude i=action’s fitness at stept of attitude i

Rates the angle variation between two consecutive steps, and

the maximum angle in one step, for collision freeleft, right, back

22

Action

• It considers the benefit of each gene regarding to:– obstacle avoidance– target closeness

=−=≤∆=∆=≤∆=∆

=]7,6,5[,max1

0,)0(])4,3,2[(,max

0,)0()1(,max

iifSS

AAielsegiandiifggi

AAielsediandiifddi

AAi

t

t

t

t

≠

== ∑

=

0,)(

0,0

1

TPiTPiAAi

TPiAi TPi

tt

TPiTPi = total of steps executed by attitude i = total of steps executed by attitude i

AAiAAi=action’s fitness at stept of attitude i=action’s fitness at stept of attitude i Increases as the distance

to the proximity-sensor increasesin the step

23

Improving the Target Direction Model

Â2 - Â1 3π/4 π/4

Target at LEFT

Target Behind Target in FRONT

Target at RIGHT

-3π/4 - π/4

First Target Direction Model

4 possible target directions

0

π/4

π/2

3π/4

π/2

-3π/4 -π/4

-π/2

8 possible target directions

24

Chromosome Representations

Cada estado corresponde a uma única atitude (par de velocidadesM1 e M2), e cada atitude corresponde a um gene do cromossoma. Target Direction (collision free) Obstacle detected

M1, M2 M1, M2 M1, M2 M1, M2 M1, M2 M1, M2 M1, M2 Front Left Right Back Left Right Back

Chromosome with 7 attitude Genes

Target Direction (collision free) Obstacle detected

M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 Front Front Left Left Right Right Back Back Left Right Back left right front back front back left right

Chromosome with 11 attitude Genes

25

Genetic Algorithm

• Integer chromosome• Population Size =100• Generations = 50• Crossover Rate = 80 %

• Mutation Rate = 4%• Roulette Wheel Reproduction• Elitism• Linear scaling of fitness• 300 Evaluation Steps for each chromosome

• Average of 25 Experiments

26

Genetic Algorithm Performance7 Genes Chromosome

Best Chromosomesin 1 experiment

0

0,5

1

1 ,5

2

2,5

3

3,5

4

4,5

Number of Generations

FIT

NES

S

27

Genetic Algorithm Performance7 Genes Chromosome

Average of Best Chromosomes in 25 experiments

0

0,5

1

1,5

2

2,5

3

3,5

4

4,5

1 3 5 7 9

11 13 15 17

19

21

23 25 27 29

31

33

35 37 39

41

43

45 47 49

Number of Generations

FIT

NE

SS

28

Genetic Algorithm Performance11 Genes Chromosome

Best Chromosomesin 1 experiment

0

0,5

1

1,5

2

2,5

3

3,5

4

4,5

5

1 3 5 7 9

11

13

15

17

19

21

23

25

27

29

31

33

35

37

39

41

43

45

47

49

Number of Generations

FIT

NE

SS

29

Genetic Algorithm Performance 11 Genes Chromosome

Average of Best Chromosomesin 25 experiments

0

0,5

1

1,5

2

2,5

3

3,5

4

4,5

5

1 3 5 7 9

11

13

15

17

19

21

23

25

27

29

31

33

35

37

39

41

43

45

47

49

Number of Generations

FIT

NE

SS

30

Paths Achieved in World 1Case Study 1SITUAÇÃO 1

CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome

31

Paths Achieved in World 1Case Study 2SITUAÇÃO 4

CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome

32

Speed Comparison

0

1000

2000

3000

4000

5000

6000

1 2 3 4 5

Case Studies

Nu

mb

er

of

Ste

ps

11 Genes Chromosome

7 Genes Chromosome

33

Paths Achieved in World 2Case Study 1SITUAÇÃO 1

CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome

34

Paths Achieved in World 2Case Study 2SITUAÇÃO 2

CROMOSSOMA DE 7 GENES CROMOSSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome

350

500

1000

1500

2000

2500

3000

3500

4000

4500

1 2 3

Case Studies

Nu

mb

er

of

Ste

ps

Speed Comparison

11 Genes Chromosome

7 Genes Chromosome

36

84,62%

40,00%

214,29%

Speed Comparison (%)

Case Study 1Case Study 1

Case Study 2Case Study 2

Case Study 3Case Study 3

37

Paths Achieved in World 3Case Study 1SITUAÇÃO 1

CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome

38

Paths Achieved in World 3Case Study 2SITUAÇÃO 2

CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome

39

Paths Achieved in World 3Case Study 3SITUAÇÃO 3

CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome

40

Paths Achieved in World 3Case Study 4SITUAÇÃO 4

CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome

41

Speed Comparison

Speed Comparison – World 37 Genes Cromosome 11 Genes Cromosome

Case Study 1 target not reached 729Case Study 2 1124 618Case Study 3 target not reached target not reachedCase Study 4 target not reached 1655

42

Conclusions

• A simple GA was able to gradually evolve the robot control

• The robot achieved near optimal path towards the goal,

avoiding obstacles

• Retraining is not necessary when the environment changes

• Controller improved performance with the 11 genes model

• The robot has no memory about previous unsuccessful paths

and may get lost

• Other tasks can be included in the model (e.g. energy supply)

• Chromosome codification is limited for few robot’s situations