Embed Size (px)
Citation preview
Artificial Intelligence in Game Design
Lecture 22: Heuristics and Other Ideas in
Board Games
Good and Bad Heuristics
• Heuristic for evaluating board must be accurate– Directly related to “likelihood” of win
– Inversely related to “distance” from win
• Example: TicTacToe heuristicH(board) = 2 × # of possible rows/columns/diagonals where X could win in one move + + 1 × # of possible rows/columns/diagonals where X could win in two moves + - 2 × # of possible rows/columns/diagonals where O could win in one move + - 1 × # of possible rows/columns/diagonals where O could win in two moves
• Gives these two boards same measure
-2-2
X
O X
O X O
O X
X
O X O
Guaranteed loss for AI
Good and Bad Heuristics
• Better heuristic must take this into account!• H(board) =
– if my move next and I have 2 in row MAXINT
– if opponent move next and they have 2 in row -MAXINT
– if my move and opponent has > 1 instance of 2 in row -MAXINT
– if opponent move and I have > 1 instance of 2 in row MAXINT
– else (above function)
• Will work better, but more complex to compute!
Speed to computeheuristic value
Accuracy ofheuristic value tradeoff
Linear Heuristic Functions
• Heuristic is some function of individual pieces on board– Usually weighted in some way
– Overall board value = Σ H(piecei)
i
• Very fast to compute
• Example: Chess– Pieces have “standard values”
– Heuristic = sum of AI pieces – sum of player pieces
9 5 3 3 1
Linear Heuristic Functions
• Often based on position of pieces on board• Examples:
– Games where purpose is to move all pieces to some goal• Backgammon• Sorry• Can often send other player back to “start”
– Heuristic value = total distance of AI pieces from goal – total distance of player pieces from goal
Linear Heuristic Functions
• Terrain based gamesHeuristic value = total value of regions player has influence over– Bias AI towards taking high ground, cover, etc.
BHeight = 50
CHeight = 20
AHeight = 40 D
Height = 30
Green positions have total influence of 7
Red positions have total influence of -3
Total heuristic value for green = 10
Total Influence
A B C D
2.0 8.0 -5.0 -1.0
Linear Heuristic Functions
• Reversi (Othello)– Board value of based entirely on piece positions
• Corners very valuable (can’t be flipped)• Sides somewhat valuable (very difficult to flip)• Middle little value (will easily be flipped)
– H(board) = C1 * number of pieces in corner +
C2 * number of pieces on side +
C3 * number of pieces in middle +
C1 >> C2 >> C3
• Reversi easiest type of game for AI– Low branching factor (5 to 15 possible moves)
– Good heuristics
– Single move can greatly change board• Hard for human player to see• Easy for MinMax lookahead to see
Nonlinear Heuristics
• Based on relationships between pieces on board• Simple example:
– Prefer chess pieces to protect one another
– Piece value = piece value * 1.5 if protected by another
– Piece value = piece value * 0.5 if not protected and threatened by opponent piece
Nonlinear Heuristics
• Drawback:Usually much more expensive to compute– n pieces O (n2) relationships between them
• May be able to explore more moves with simpler (linear) heuristic• Example:
– Simple linear heuristic takes k ms to evaluate per board
– Complex nonlinear heuristic takes 400k ms to evaluate per board
– Average of 20 possible next moves per board• 400 possible next two moves
– On average, could explore two additional moves if use linear heuristic
Linear vs. Nonlinear Heuristics
• Nonlinear heuristics often detect future changes in board– Pieces threatened by others might be captured
– Pieces protected by others less likely to be captured
• Often see effect of nonlinear heuristic with additional levels of linear heuristic
• Example: “knight fork” in chess– Can look for this using nonlinear
heuristic
Linear vs. Nonlinear Heuristics
• Searching 2 more moves in game tree will show result of knight fork– Shows state where rook now gone
– Higher value for linear heuristic based on piece values
– MinMax then gives “knight fork” state a high value
– Using simple heuristic might make this possible
Board with knight fork
Board with knight threatening rook
Black moves king
Other moves give checkmate (value = MAXINT)
Board with blackrook gone
Knight takes rook
High value h
High value h
High value h
Max level
Min level
Linearizing Nonlinear Heuristics
• Worth working hard to rethink nonlinear heuristcs• Example: Make linear in terms of positions rather than pieces
– Backgammon: Prefer pieces in groups of >1 to prevent being sent back to start
– Simply count positions with only one piece rather than comparing pieces
Horizon Effect
• Major weakness of purely linear heuristics based on piece values– “Throwing good money after bad”
• May need to recognize when move does not improve board position• Chess example:
Queen pinned by bishops Could move rook in way But rook captured and will still lose queen
Horizon Effect
Board with queen pinned
Queen takes bishop
Board with queen lost
Bishop takes queen
White down 6 points
Move rook in front
Board with rook lost
Bishop takes rook
White down 5 points
Queen takes bishop
Board with queen and
rook lost
Bishop takes queen
White now down 11 points!
This looks worse at cutoff level, but is actually best in long run!
Data-Driven Approaches
Basing actions on known strategies rather than tree search
• Opening books of initial moves (Chess)– Often 20-30 moves at start of Grandmaster match
• Each book consists of:– List of moves
– Evaluation of final outcome• Should we follow this strategy
• Allows faster processing– No need to search game tree until
end of sequence
– Can just use evaluation as heuristic
Currentboard
Boards in sequence
End of sequence
Now start branching
Opening Books
• Choose as own opening strategy• Must have good final evaluation!
– Make moves according to script
– If opponent follows script, keep following
– If opponent leaves script, start MinMax• Will probably be in same way that benefits us
• Must also recognize when opponent is using an opening book– Keep database of moves in opening books
– Match current board to those in database to find whether it is part of a sequence
– If so, make decision about whether following script is good idea
Other Set Plays
• End Games– Many games have different strategies when few pieces left
• Forcing checkmate in chess• Kings vs. kings in checkers• Getting last pieces home in backgammon
– Recognize based on pieces left
– Follow set strategies
• Set Evaluation Values– No heuristic evaluation of board – instead,
match board to database to get evaluation
– Works best if can match subboards• Example: Edge configurations in Othello• This edge has a known value
Alternative Approaches
• Go – 19 19 board branching factor of 361
– Impossible for MinMax
• Only known approaches based on template matching– Look in local area for configurations
that match known strategies
– Still very open problem• Best AI for Go only plays at
amateur level
