Applying Genetic Programming to

Stratego

Ryan Albarelli

Overview

Problem definition & MotivationStrategoMinimax AlgorithmProblem RepresentationGP ImplementationTournamentFuture Work

Problem Definition

• Devise a minimax heuristic for Stratego using Genetic Programming

• Devise a method for evaluating heurstic against non-GP generated intelligence

Motivation

• Develop generic minimax heuristic generator for new game solutions

• Add another level to the man vs. machine debate

• Auction X-Box prize on eBay for gas money home

Solution Viability

• Determine heuristic’s playability against other types of intelligence– “by hand” heuristics– human opponents– AI tournament

Stratego

• Turn based strategy board game• Pieces of increasing rank and

decreasing frequency• Hidden information• Initial piece layout determined by each

player• Objective is to capture enemy flag

Board Layout

Minimax Algorithm

• Tree representation• Branch represents possible move• Leafs values generated by heuristic• Depth first search of all possible moves• Simple minimization/maximization at each level• Alpha-Beta pruning• Partial Information

Minimax Tree (depth=2)

6

51

6 1 29 85

Each leaf contains a value generated by the game heuristic

Minimize

Arrows represent move candidates

Maximize

6

Turn ofPlayer 1

Turn of Player 2

Problem Representation

• Heuristic as Expression Tree• Binary operations

– Standard arithmetic– Custom relationships

• Terminals– Piece values– formulas

401 is my Evil Organization!

Experiment Methodology

• Each heuristic expression tree is an individual in the population

• Competition among individuals in the form of a full game to determine fitness.

• Apply genetic operations to survivors.

• Rinse, Repeat

Binary Operations

• Standard arithmetic operations: + - * /

Terminals

• Number of pieces in hand (of Rank X)• “Center of gravity” of your/opponents pieces• Position/rank of pieces on board• Constants (numbers)• Number of adjacent friendly/enemy pieces• Distance of key pieces to enemy key pieces

Expression Tree

*

Flag count500

/

4 CGrav

+

Fitness Evaluation

• Victory against other GP individuals in the pool.

• Age bonus

• Number of moves required to win

• Final piece count

Selection / Competition

• Rank based

• Drop lowest candidates from the pool

• Candidates with only one loss given slight waiver

Genetic Operations

• Expression Trees• Crossover

– Similar sized subtree swapping– Subtree shrinking

• Mutation– Mutate terminals and operations– Adding a operation/terminal

Tournament

• Best solution to be competing in the CS AI tournament in December

Future Work

• Generalize GP for any board game

• Develop method to automatically compile expression trees for faster evaluation

Questions?