Design of a real time strategy game with a genetic AI

By Bharat Ponnaluri

Abstract

Current strategy game AI Involved combinations of heuristics and constants. The combinations need to be optimized for the AI to be effective Too many possible constants&combinations-->difficult optimization Chess heuristic evaluations

Moves available Is the king safe Pawn structure Material balance

Genetic AI Can produce constants and combinations of heuristic algorithms quickly by

eliminating suboptimal combinations.

Introduction• Design of a simple real time strategy game

– Build troops and try to take over cities– Like Risk in real time– Goal is to have an intelligent AI optimized with a Genetic

Algorithm(GA).

• AI design– Coding of simple building block heuristic evaluation functions– Addition of constants(Agressiveness,stinginess) that determine behaviour– Chromosomes will be random combinations of

• Heuristic evalution functions– Used to evaluate the quality of a possible course of action based on game

data

Personality Traits• Goal: To make the AI more interesting and

humanlike

• Traits– Agressiveness– Paranoia– Stinginess– Vengence– Trustworthiness

Observation• AI will take advantage of other players' behavior

• Examples– Assertivness

• The AI will keep track of when a player retreats

• The AI will attack a player which retreats a lot

– Attacking game leader• One player is threatening to win

• AI will track who tries to stop that player

• Then the AI will let that player do the work

Game Theory• Advantages

– Not very complicated– Easily understandable by AI– Good model of human behavior

Game Theory: War of attrition• Two players fighting over city

• Backing down=loss in percieved assertiveness

• Fighting=massive loss in strength

• t=time that conflict lastsDon't Fight Fight

Don't Fight (0,0,0) (-t+1,-t,0)

Fight (t,-t+1,0) (-t,-t,2*t)

Sample chromosome

    Chromosome       enemyStrength()/2*myStrength()-3*fooA()/43.........+power()

Possible genes    public int fooA()      .... public int enemyStrength()     .... public int myStrength()

 

Heuristic evaluation functions• Heuristic evaluation functions

– Used to evaluate the quality of a possible course of action based on game data

• Example:– Public int getTrafficJamFactor()

• Return ( 1.001^(number of cars))/roadSize

How do GAs work• Replicate evolution

• Steps1.Determine random combinations of heuristics and

constants(chromosomes)

2.Test chromosomes

3.Remove suboptimal chromosomes

4.Mutate surviving chromosomes

5.Swap data between surviving chromosomes

6.If chromosomes are optimal,end algorithm

7.Otherwise, repeat steps 1-7

Advantages and Disadvantages

• Advantages o Can function without user input o Can optimize a large number of constants o Usually is very good at optimizing stuff o Can determine unknown solutions

• Possible disadvantages o GAs are complicated o Take a significant amount of computing power o Networking will increase complexity

Evolutionary Stable Strategies(ESS)• Gene pool of defectors means cooperating players

will lose

• Advantages– Diverse AI personalities

• Disadvantages– May lead to suboptimal/boring AI's

Cooperate Defect

Cooperate 1,1 -10,3

Defect 3,-10 -5,-5

ESS: Example• Population of turtling AI's

• AI's only counterattack or attack weak players

• A more balanced AI will lose

• Playing against a group of turtling AI's is boring

New Heurstics• Find Targets()

– Average proximity– Income power factor– Troop power factor– Potential borders

 

Testing results• Numbers greater than 1 are not good

• From last working version(before new heurstics)

Number of Troops On Map

Rendering Time

Update Time

AI

10 1.100.06 0.55

1500 2.500.07 0.63

2500 2.850.08 0.56

Generating a target number using a mathematical expression

– Combination of numbers(1-9) and mathematical operators(-,+,*,/) adding up to a target number

– Each number=4 bits – Chromosome

• 6.7657+5.7845*4.8568/2.8969 +1 .756857– Fitness value is the absolute value of

(targetNumber)/(expressionValue-targetNumber) – Can be scaled for large numbers &decimals – No attempting to use brute force to test out random

combinations.

Important Classes • Classes

– FormualaGenerator: • Driver Program

– GraphPanel: • Draws the graph of the average fitness and maximum

fitness.

• Makes the GA run periodically in synchornization with the graph.

– Chromosome:• Represents the chromosomes/expressions.

Static Mutation

Dynamic MutationDynamic Mutation

Sample Expressions for 150• Static Mutation

– 1.0292804250299605 + ((2.7400839829565777 / 1.0605221382369576) * 7.741718241469524

* 7.316070011134583) - ((-0.9821438516501173) / 4.043144676725913) + (0.9986661909539591

* 6.634650893629718 * 0.3645726822807098) = 150.026535

• Dynamic Mutation– 1.0292804250299605 + ((2.7400839829565777 / 1.0605221382369576) *

7.741718241469524

• * 7.316070011134583) - ((-0.9821438516501173) / 4.043144676725913) + (0.9986661909539591

• * 6.634650893629718 * 0.3645726822807098) = 150.026535

Analysis of results• The dynamic mutation did not help because the

standard deviations were 0

• The GA generally works but occasionally crashes due to a bug which deletes all the chromosomes

• The graphs show rapid spikes in fitness followed by a rapid decline

– Shows that the GA does not get stuck on local exterma.

• There are long period of nearly static fitness levels.

Important Methods• RemoveFailures:

– Finds all the chromosomes in the population with a fitness that is below average and removes them

• Crossover: – Picks random chromosomes to mate and produce children.– Once enough children are produced, the children are saved and the parents

are deleted

• Mutations– Generates a mutation rate based on the standard deviation of the fitness

values and average change of them– Calls the mutate method and gives it the mutation rate

• DrawGraph: Takes the saved data for the highest fitness rate, average fitness rate, standard deviation, and change of population and outputs the data visually.

Future Plan• Debug implementation of dynamic mutation

• Make genetic algoritm generate formuals for input-output pairs based on common functions

• Create scenarios for my game with obvious good/bad choices for each player

• Run GA for game– Fitness rates will be based on how well the AI's

evaluates the scenarios.– Running a full version of my game will take too

much time.