Upload
marshall-ford
View
216
Download
1
Embed Size (px)
Citation preview
Neural Networks
• Biological approach to AI• Developed in 1943• Comprised of one or more layers of neurons
• Several types, we’ll focus on feed-forward networks
Neurons
http://faculty.washington.edu/chudler/color/pic1an.gif
http://research.yale.edu/ysm/images/78.2/articles-neural-neuron.jpg
Biological
Artificial
Neural Network Neurons
• Receives n-inputs• Multiplies each input by its weight
• Applies activation function to the sum of results
• Outputs result
http://www-cse.uta.edu/~cook/ai1/lectures/figures/neuron.jpg
Activation Functions
• Controls when unit is “active” or “inactive”
• Threshold function outputs 1 when input is positive and 0 otherwise
• Sigmoid function = 1 / (1 + e-x)
Neural Network Layers
• Each layer receives its inputs from the previous layer and forwards its outputs to the next layer
http://smig.usgs.gov/SMIG/features_0902/tualatin_ann.fig3.gif
Simple Example Application
• Smart Sweepers• Red mine sweepers are best generated so far
• Hit ‘F’ key to switch between displays
• Every generation changes neural net weights by a genetic algorithm
Neural Network LearningBack-Propagation Algorithm:
function BACK-PROP-LEARNING(examples, network) returns a neural network inputs: examples, a set of examples, each with input vector x and output vector y network, a multilayer network with L layers, weights Wj,i , activation function g
repeat for each e in examples do for each node j in the input layer do aj ‰ xj[e] for l = 2 to M do ini ‰j Wj,i aj
ai ‰g(ini) for each node i in the output layer do j ‰ g’(inj) i Wji i
for l = M – 1 to 1 do for each node j in layer l do j ‰ g’(inj) i Wj,i i
for each node i in layer l + 1 do Wj,i ‰ Wj,i + x aj xi
until some stopping criterion is satisfied return NEURAL-NET-HYPOTHESIS(network)
[Russell, Norvig] Fig. 20.25 Pg. 746
Neural Networks and Game AI
• simplify coding of complex state machines or rules-based systems
• potential for AI to adapt as the game is played
• “I am very confident about it as well, because I understand what the neural net is doing. I haven't just created a big mysterious black box, I can map out the internal workings of it.”
Jeff Hannan, creator of AI for Colin McRae Rally 2.0
http://www.generation5.org/content/2001/hannan.asp
Colin McRae Rally 2.0
• Used standard feedforward multilayer perceptron neural network
• Constructed with the simple aim of keeping the car to the racing line
http://www.generation5.org/content/2001/hannan.asp
BATTLECRUISER: 3000AD • 1998 space simulation game
• 'I am a Gammulan Criminal up against a Terran EarthCOM ship. My ship is 50% damaged but my weapon systems are fully functional and my goal is still unresolved' what do I do?
http://www.gameai.com/games.html
http://www.3000ad.com/shots/bc3k.shtml
Black and White
• Uses neural networks to teach the creature behaviors
• “Tickling” increase certain weights
• Hitting reduces certain weights
Usefulness in Games
• Control – motor controller, for example, control of a race car or airplane
• Threat Assessment – input number of enemy ground units, aerial units, etc.
• Attack or Flee – input health, distance to enemy, class of enemy, etc.
• Anticipation – Predicting player’s next move, input previous moves, output prediction
http://www.onlamp.com/pub/a/onlamp/2004/09/30/AIforGameDev.html
Influence Maps Refresher
• From section 7.1• Creates a spatial
database of game world info
• Static or dynamic
Openness Layer
Occupancy Layer
Static Cover Layer
Desirability Layer
Neural Networks and Influence Maps
• Usually influence map layers are combined using a weighted sum
• Neural network can be used instead
• Finding correct weight for each layer usually results in trial and error
• Choosing relevant layers can be difficult
• Potential loss of important information if factors cancel each other
Computational Complexity• Could lead to a very large number of calculations
Influence Map Layer
1
Influence Map Layer
2
Hidden Units
Output Units
Input Units
Optimizations
• Only analyze relevant portion of map
• Reduce grid resolution• Train network during development, not in-game
Design Details
• Need an input for each cell in each layer of the influence map
• Need one output for each cell on map
• Hidden units are arbitrary, usually 10-20 with some guess and test to prune it
Different Decisions and Personalities
• One network for each decision• Implemented as one network with a different array of weights for each decision
• Different personalities can have multiple arrays of weights for each decision
Training
• Datasets from gaming sessions of human vs. human are best
• Must decide whether training will occur in-game, during development, or both
• Learning during play provides for adaptations against individual players
Conclusions
• Neural networks provide ability to provide more human-like AI
• Takes rough approximation and hard-coded reactions out of AI design (i.e. Rules and FSMs)
• Still require a lot of fine-tuning during development
References
• Four Cool Ways to Use Neural Networks in Games• Interview with Jeff Hannan, creator of AI for
Colin McRae Rally 2.0• Interview with Derek Smart, creator of AI for
Battlecruiser: 3000AD• Neural Netware, a tutorial on neural networks• Sweetser, Penny. “Strategic Decision-Making
with Neural Networks and Influence Maps”, AI Game Programming Wisdom 2, Section 7.7 (439 – 46)
• Russell, Stuart and Norvig, Peter. Artificial Intelligence: A Modern Approach, Section 20.5 (736 – 48)