AD for GAMES

AD FOR GAMESLecture 4

MINIMAX AND ALPHA-BETA REDUCTION

Borrows from Spring 2006 CS 440 Lecture Slides

MOTIVATION Want to create programs to play games Want to play optimally Want to be able to do this in a reasonable amount of time

TYPES OF GAMES

BackgammonMonopoly

ChessCheckers

GoOthello

Card GamesBattleship

FullyObservable

PartiallyObservable

DeterministicNondeterministic (Chance)

Minimax is for deterministic, fully observable games

HOW TO PLAY A GAME BY SEARCHING General Scheme

Consider all legal moves, each of which will lead to some new state of the environment (‘board position’)

Evaluate each possible resulting board position Pick the move which leads to the best board position. Wait for your opponent’s move, then repeat.

Key problems Representing the ‘board’ Representing legal next boards Evaluating positions Looking ahead

GAME TREES Represent the problem space for a game by a

tree Nodes represent ‘board positions’; edges

represent legal moves. Root node is the position in which a decision

must be made. Evaluation function f assigns real-number

scores to `board positions.’ Terminal nodes represent ways the game

could end, labeled with the desirability of that ending (e.g. win/lose/draw or a numerical score)

BASIC IDEA (SEARCHING) Search problem

Searching a tree of the possible moves in order to find the move that produces the best result

Depth First Search algorithm Assume the opponent is also playing optimally

Try to guarantee a win anyway!

REQUIRED PIECES FOR MINIMAX An initial state

The positions of all the pieces Whose turn it is

Operators Legal moves the player can make

Terminal Test Determines if a state is a final state

Utility Function Other name: static evaluation function

UTILITY FUNCTION Gives the utility of a game state

utility(State) Examples

-1, 0, and +1, for Player 1 loses, draw, Player 1 wins, respectively

Difference between the point totals for the two players

Weighted sum of factors (e.g. Chess) utility(S) = w1f1(S) + w2f2(S) + ... + wnfn(S)

f1(S) = (Number of white queens) – (Number of black queens), w1 = 9

f2(S) = (Number of white rooks) – (Number of black rooks), w2 = 5

...

TWO AGENTS MAX

Wants to maximize the result of the utility function Winning strategy if, on MIN's turn, a win is obtainable

for MAX for all moves that MIN can make MIN

Wants to minimize the result of the evaluation function

Winning strategy if, on MAX's turn, a win is obtainable for MIN for all moves that MAX can make

EXAMPLE Coins game

There is a stack of N coins In turn, players take 1, 2, or 3 coins from the stack The player who takes the last coin loses

COINS GAME: FORMAL DEFINITION Initial State: The number of coins in the stack Operators:

1. Remove one coin 2. Remove two coins3. Remove three coins

Terminal Test: There are no coins left on the stack

Utility Function: F(S) F(S) = 1 if MAX wins, 0 if MIN wins

N = 4K =

N = 3K =

N = 2K =

N = 1K =

N = 0K =

N = 1K =

N = 2K =

N = 0K =

N = 0K =

N = 1K =

N = 0K =

N = 1K =

N = 0K =

N = 0K =

12

3

3 21

2 1 1

1211

N = 0K =

1 F(S)=0F(S)=0 F(S)=0

F(S)=1

F(S)=1

F(S)=1 F(S)=1

MAXMIN

N = 4 K = 1

N = 3K = 0

N = 2K = 0

N = 1K = 1

N = 0 K = 1

N = 1 K = 0

N = 2K = 1

N = 0K = 1

N = 0K = 1

N = 1K = 0

N = 0K = 0

N = 1K = 1

N = 0K = 0

N = 0K = 0

12

3

3 21

2 1 1

1211

N = 0K = 1

1 F(S)=0F(S)=0 F(S)=0

F(S)=1

F(S)=1

F(S)=1 F(S)=1

MAXMIN

Solution

Basic Algorithm

DEMO TIME See: WinMinimaxCS project demo

EXERCISE TIME See board ….

ANALYSIS Max Depth: 5 Branch factor: 3 Number of nodes: 15 Even with this trivial example, you can see that

these trees can get very big Generally, there are O(bd) nodes to search for

Branch factor b: maximum number of moves from each node

Depth d: maximum depth of the tree Exponential time to run the algorithm!

How can we make it faster??????

ALPHA-BETA PRUNING Main idea: Avoid processing subtrees that have

no effect on the result Two new parameters

α: The best value for MAX seen so farβ: The best value for MIN seen so far

α is used in MIN nodes, and is assigned in MAX nodes

β is used in MAX nodes, and is assigned in MIN nodes

ALPHA-BETA PRUNING MAX (Not at level 0)

If a subtree is found with a value k greater than the value of β, then we do not need to continue searching subtrees MAX can do at least as good as k in this node, so MIN

would never choose to go here! MIN

If a subtree is found with a value k less than the value of α, then we do not need to continue searching subtrees MIN can do at least as good as k in this node, so MAX

would never choose to go here!

Algorithm

CSE 391 - Intro to AI 22

ALPHA-BETA PRUNING A way to improve the performance of the

Minimax Procedure Basic idea: “If you have an idea which is

surely bad, don’t take the time to see how truly awful it is” ~ Pat Winston>=2


ALPHA-BETA PRUNING Traverse the search tree in depth-first order For each MAX node n, α(n)=maximum child value found

so far Starts with – Increases if a child returns a value greater than the current

α(n) Lower-bound on the final value

For each MIN node n, β(n)=minimum child value found so far Starts with + Decreases if a child returns a value less than the current

β(n) Upper-bound on the final value

MAX cutoff rule: At a MAX node n, cut off search if α(n)>=β(n)

MIN cutoff rule: At a MIN node n, cut off search if β(n)<=α(n)

Carry α and β values down in search

function Max-Value(s,α,β)inputs:

s: current state in game, A about to playα: best score (highest) for A along path to sβ: best score (lowest) for B along path to s

output: min(β, best-score (for A) available from s)if ( s is a terminal state )then return ( terminal value of s )else for each s’ in Succ(s)α:= max( α, Min-value(s’,α,β))if ( α≥β) then return βreturn α

function Min-Value(s’,α,β)output: max(α, best-score (for B) available from s )

if ( s is a terminal state )then return ( terminal value of s)else for each s’ in Succs(s)β:= min( β, Max-value(s’,α,β))if (β≤α) then return αreturn β



















EFFECTIVENESS OF ALPHA-BETA PRUNING

Guaranteed to compute same root value as Minimax

Worst case: no pruning, same as Minimax (O(bd))

Best case: when each player’s best move is the first option examined, you examine only O(bd/2) nodes, allowing you to search twice as deep!

CONCLUSION Minimax finds optimal play for deterministic, fully

observable, two-player games Alpha-Beta reduction makes it faster

Interesting Link(s): http://www.ocf.berkeley.edu/~yosenl/extras/alphabeta/alphabeta.html

http://www.ocf.berkeley.edu/~yosenl/extras/alphabeta/alphabeta.html

http://www.ocf.berkeley.edu/~yosenl/extras/alphabeta/alphabeta.html

YOUR KBS 2 PROJECT We will focus on deterministic, two-player, fully

observable games You will use minimax, alpha beta pruning and

transposition tables (next college)

HOMEWORK Study chapter 10.5.2 Games form your study

book Study this presentation

HOMEWORK AND PRACTICE Implement “Tic Tac Toe with a Twist” with two human

players.

See DEMO now

During practice you are asked to implement an AI player using minimax for this game.

Documents

AD for GAMES