An overview on the termination conditions in the

evolution of game bots

A. Fernández-Ares, P. García-Sánchez, A.M. Mora, P.A. Castillo, J.J. Merelo, M.G. Arenas

Universidad de Granada (Spain)

A summary of previous worksPablo García-Sánchez, Antonio Fernández-Ares, Antonio Miguel Mora, Pedro A. Castillo Valdivieso, Jesús González, Juan Julián Merelo Guervós: Tree Depth Influence in Genetic Programming for Generation of Competitive Agents for RTS Games. EvoApplications 2014: 411-421

Antonio Fernández-Ares, Pablo García-Sánchez, Antonio Miguel Mora, Pedro A. Castillo, Juan Julián Merelo Guervós: Designing competitive bots for a real time strategy game using genetic programming. CoSECivi 2014: 159-172

Antonio Fernández-Ares, Pablo García-Sánchez, Antonio M. Mora, Pedro A. Castillo Valdivieso, Juan Julián Merelo Guervós, Maria I. García Arenas, Gustavo Romero: It's Time to Stop: A Comparison of Termination Conditions in the Evolution of Game Bots. EvoApplications 2015: 355-368

Index

1. Introductiona. RTSb. Planet Warsc. Genetic

Programming2. GPBot

a. Conditionsb. Actions

3. SCORE - Fitness4. Methodology

a. Termination Criteria5. Experimental Setup

6. Resultsa. Scoreb. Generationsc. Testsd. Benchmarke. Comparative

7. Conclusions8. Future Work

Introduction: RTS

Real-Time Strategy games (RTS-games)● Resources

● Units

● Buildings

Victory: Get all resources, kill all enemy units or destroy enemy buildings

Introduction: Planet Wars

Buildings:Planets

Units:Space ships

Resources:Ships generated in every planet

Introduction: Genetic Programming

Evolutionary Algorithm that evolves binary decision tree➔ Internal nodes: Conditions➔ Leafs: Actions

Individual -> behavioural model (solution)Evaluation -> Playing a game against a rival and getting a scoreAdapted operators

GPBot: Conditions

A logical expression composed by, at least, one extracted standard game state variable and a value between 0 and 1

● myShipsEnemyRatio● myShipsLandedFlyingRatio● myPlanetsEnemyRatio● myPlanetsTotalRatio● actualMyShipsRatio● actualLandedFlyingRatio

GPBot: Actions

The possible actions just involve the movement of an amount of ships from a source to a destination planet

● Attack Nearest (Neutral|Enemy|NotMy)Planet● Attack Weakest (Neutral|Enemy|NotMy) Planet● Attack Wealthiest (Neutral|Enemy|NotMy)

Planet● Attack Beneficial (Neutral|Enemy|NotMy)

Planet● Attack Quickest (Neutral|Enemy|NotMy) Planet}● Attack (Neutral|Enemy|NotMy) Base● Attack Random Planet● Reinforce Nearest Planet● Reinforce Base● Reinforce Wealthiest Planet

Score (Fitness)

SCORE BASED IN TURNS IN

VICTORIES

SCORE BASED IN TURNS IN DEFEATS

SCORE BASED IN

VICTORIES> >

Methodology: Termination Criteria

[NG] Number of Generations ➔ 30, 50, 100, 200 Generations

[AO] Age of Outliers ➔ 1, 1.5, 2, 2.5 times the IQR

[RT] Replacement Rate➔ n/2, n/4, n/8, n/16 with n = N/2

[FT] Fitness Threshold➔ 22, 24, 26, 28

[FI] Fitness Improvement➔ 3, 7, 10, 15

Experimental SetupParameter Name Value

Population Size (N) 32

Number of battles for scoring (NB) 30

Re-evaluation of individuals Yes

Crossover type Sub-tree crossover

Crossover rate 0.5

Mutation 1-node mutation

Mutation rate 0.25

Selection 2-tournament

Replacement Generational

Maximum Tree Depth 7

Runs 36

SCORE of the best individuals (of all runs) grouped by criterion

GENERATION attained (of all runs) grouped by criterion

Kruskal-Wallis Test of samples of each criterion by SCORE an GENERATIONS.

Black means no statistically significant difference has been found.

Percentage of victories (of the 36 best individuals) in benchmark against five different competitive bots available in the literature in 100 maps

Linear regression of the SCORE (fitness) with the results of the benchmark (Percentage of victories)

Average score of the best individual and average reached generations per

termination criteria

Average results of every criterion(relative with respect to NG_30)

Conclusions I

[AO] � Not good. As more restrictive values don’t imply a significant score improvement.[FT]� Best score. But the optimum fitness might not be know. Needs the highest amount of generations. It might not finish.[FI]� Useful to ‘detect’ local optima.[RT]� Provides the best results considering all metrics.

Conclusions II

Fitness Threshold would be the most desirable option➔ It attains the best score.➔ It is quite difficult to find an optimal fitness value to

use (normally it is unknown).➔ Requires more computational budget, and it is

possible that it never ends (the criterion is not met).

Replacement Rate as stopping criterion, since it is a compromise solution which relies in the population improvement without an implicit use of the fitness.

Future work

● New problems (and algorithms) are being addressed.○ P. García-Sánchez, Alberto Tonda, Giovanni

Squillero and JJ. Merelo: Towards Automatic StarCraft Strategy Generation Using Genetic Programming. Accepted at CIG 2015.

● Mechanisms to improve the EA.● Use larger and complex decision trees.

Questions

Thank you!Antonio Fernández-Aresantares@ugr.es - @antaress

Antonio M. Moraamorag@ugr.es - @amoragar

Pablo García-Sánchezpablogarcia@ugr.es - @fergunet