Embed Size (px)
Citation preview
1
Today’s Topics
• Playing Deterministic (no Dice, etc) Games
– Mini-max
– - pruning
– ML and games?
• 1997: Computer Chess Player (IBM’s Deep Blue) Beat Human Chess Champ (Kasparov)
10/6/15 CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
Mini-Max Algorithm
• Generate Search Tree of ALL Possible Games
• Score Leaves (Assume Higher Better for ‘You’)
eg, +1 you win, -1 you lose, 0 if a draw (ie, a tie)
• Assume Opponent Scores ‘Board Configurations’
the Same Way You Do (more on this later)
• Propagate Info from Leaves to Root of Search Tree,
then Choose Best Move– Your Turn: choose MAX score– Opponent’s Turn: choose MIN score (which is best for opponent)
• Only Choose NEXT Move; When Your Turn Again, Repeat– opponent might not have done what you expected
10/6/15 2
CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
Tic-Tac-Toe (https://en.wikipedia.org/wiki/Tic-tac-toe)
10/6/15 3
CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
What If Game More Complex?
Chess: approx 35 moves per term and50 moves/player/game, so O(35100) possible board configurations!
Better estimate: 1047 en.wikipedia.org/wiki/Shannon_number
Number of atoms in the Earth: 1050 www.fnal.gov/pub/science/inquiring/questions/atoms.html
Solution: project ahead as far as possible, then use a ‘static board evaluator’ (SBE) to score in-progress board configurations
10/6/15 4
(Common) Abuse of big-O notation
CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
SBE’s
• No Perfect SBE Exists for Chess (otherwise we’d know if black can always win, etc)
• But a lot of Domain K can be put in the SBE
• You Might Have Learned “Piece Values” in Chess SBE = 1 (#myPawns - #theirPawns) + … + 10 (#myQueens - #theirQueens)
+ points for ‘center of board’ control + etc
•Can Run Mini-Max with SBE
10/6/15 5
CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
Saving Unnecessary Calculations: - Pruning
• Can Prune Away Subtrees by Realizing no Matter their Score, they won’t Impact Choice
• We Won’t Cover the Full - Pruning Algorithm,Since its Applicability Across AI is Limited
• But We’ll Cover some Simple Cases of the Two Types of Pruning
• Ask: Would it Matter if this Node had Value or Value -? If Decision Same in Both Cases, No Need to Compute Node’s SBE Value
10/6/15 6
CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
An -Cutoff Example(view in Slide Show mode to see animation)
10/6/15 7
Us (max)
Them (min)
big subtree
?
2 7 1
2
2
1
Tip: Always fully compute the left-most
subtree of root
If good for Us, Opponent will not go here.If bad for Us, we will take left branch from root.
move
CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
A -Cutoff Example(view in Slide Show mode to see animation)
10/6/15 8
Us (max)
Them (min)
big subtree
?
2 7 9
7
7
9
Recall: Always compute the
left-most subtree of root
If good for Us, Opponent will
take left branch from
root.If bad for Us, we will take left branch.
move
CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
Note
We project ahead as far as we can in the time we have, but only
choose our next move
Next time it is our turn, we repeat - calc(possibly repeating some work, but too much to store)
10/6/15 9
CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
Performance Improvement
Best case: - can double search depth considered is fixed amount of time
(doubling depth exponentially
increases boards considered!)
Worst case: no savings
Note: - and mini-max return SAME answer (- simply avoids wasted work)10/6/15 10
CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
Some Built-in Assumptions
• Projecting Ahead k Moves will Lead to Better SBE Estimates than Simply Applying SBE to the Legal Next Boards– seems reasonable, but isn’t guaranteed
• Our Opponent Thinks the Same Way We Do– no consideration of ‘traps’ and tricking opponent
10/6/15 11
CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
Horizon Effect
Defn: Since We Usually Stop before Game Ends, ‘Danger’ might be Lurking just Beyond our Search Depth
One (Partial) Soln: Once Best Move Chosen, Look Ahead a Few Moves More
if turns out bad, spend more time searching this turn
10/6/15 12
CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
Chess: The ‘E. coli’ of AI?
• Chess Originally Thought to be a Great Testbed for Advancing AI
• Has had Moderate Impact on AI Progress
• Not Much ‘Technology Transfer’ to AI Tasks
• Played Minor Role in ‘ML Revolution’
10/6/15 13
CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
ML and Games
• ML Led to World-Class Backgammon Player Decades Ago
• ML Produced World-Class Poker Player Over Last Few Years
• Had Less Success at Chess and Go, but Recent Promise Shown with ‘Deep [Neural] Networks’
• Deep Networks Recently Learned to Play Well without being given Games’ Rules
10/6/15 14
