7/28/2019 5.3 AI Agents
1/79
Chapter 5.3Artificial Intelligence:Agents, Architecture, and Techniques
7/28/2019 5.3 AI Agents
2/79
2
Artificial Intelligence Intelligence embodied in a man-made
device
Human level AI still unobtainable
7/28/2019 5.3 AI Agents
3/79
3
Game Artificial Intelligence:
What is considered Game AI? Is it any NPC behavior?
A single if statement?
Scripted behavior?
Pathfinding?
Animation selection?
Automatically generated environment?
Best shot at a definition of game AI?
7/28/2019 5.3 AI Agents
4/79
4
Possible Game AI
DefinitionInclusive view of game AI:
Game AI is anything that contributes to theperceived intelligence of an entity,regardless of whats under the hood.
7/28/2019 5.3 AI Agents
5/79
5
Goals of an
AI Game ProgrammerDifferent than academic or defense industry
1. AI must be intelligent, yet purposely flawed2. AI must have no unintended weaknesses
3. AI must perform within the constraints
4. AI must be configurable by game designersor players
5. AI must not keep the game from shipping
7/28/2019 5.3 AI Agents
6/79
6
Specialization of
Game AI Developer No one-size fits all solution to game AI
Results in dramatic specialization
Strategy Games Battlefield analysis Long term planning and strategy
First-Person Shooter Games One-on-one tactical analysis Intelligent movement at footstep level
Real-Time Strategy games the mostdemanding, with as many as three full-time
AI game programmers
7/28/2019 5.3 AI Agents
7/79
7
Game Agents May act as an
Opponent
Ally Neutral character
Continually loops through theSense-Think-Act cycle Optional learning or remembering step
7/28/2019 5.3 AI Agents
8/79
8
Sense-Think-Act Cycle:
SensingAgent can have access to perfect
information of the game world May be expensive/difficult to tease out
useful info
Game World Information Complete terrain layout
Location and state of every game object
Location and state of player
But isnt this cheating???
7/28/2019 5.3 AI Agents
9/79
9
Sensing:
Enforcing Limitations Human limitations?
Limitations such as Not knowing about unexplored areas Not seeing through walls
Not knowing location or state of player
Can only know about things seen,heard, or told about
Must create a sensing model
7/28/2019 5.3 AI Agents
10/79
10
Sensing:
Human Vision Model for Agents Get a list of all objects or agents; for each:
1. Is it within the viewing distance of the agent?
How far can the agent see? What does the code look like?
2. Is it within the viewing angle of the agent?
What is the agents viewing angle?
What does the code look like?
3. Is it unobscured by the environment?
Most expensive test, so it is purposely last
What does the code look like?
7/28/2019 5.3 AI Agents
11/79
11
Sensing:
Vision Model Isnt vision more than just detecting the
existence of objects?
What about recognizing interestingterrain features?
What would be interesting to an agent?
7/28/2019 5.3 AI Agents
12/79
12
Sensing:
Human Hearing Model Humans can hear sounds
Can recognize sounds Knows what emits each sound
Can sense volume Indicates distance of sound
Can sense pitch Sounds muffled through walls have more bass
Can sense location Where sound is coming from
7/28/2019 5.3 AI Agents
13/79
13
Sensing:
Modeling Hearing How do you model hearing efficiently?
Do you model how sounds reflect off every
surface? How should an agent know about sounds?
7/28/2019 5.3 AI Agents
14/79
14
Sensing:
Modeling Hearing Efficiently Event-based approach
When sound is emitted, it alerts interested
agents
Use distance and zones to determinehow far sound can travel
7/28/2019 5.3 AI Agents
15/79
15
Sensing:
CommunicationAgents might talk amongst themselves!
Guards might alert other guards
Agents witness player location and spreadthe word
Model sensed knowledge through
communication Event-driven when agents within vicinity of
each other
7/28/2019 5.3 AI Agents
16/79
16
Sensing:Reaction Times
Agents shouldnt see, hear,communicate instantaneously
Players notice!
Build in artificial reaction times
Vision: to second
Hearing: to second
Communication: > 2 seconds
7/28/2019 5.3 AI Agents
17/79
17
Sense-Think-Act Cycle:Thinking
Sensed information gathered
Must process sensed information
Two primary methods
Process using pre-coded expert knowledge
Use search to find an optimal solution
7/28/2019 5.3 AI Agents
18/79
7/28/2019 5.3 AI Agents
19/79
19
Thinking:Search
Employs search algorithm to find anoptimal or near-optimal solution
A* pathfinding common use of search
7/28/2019 5.3 AI Agents
20/79
20
Thinking:Machine Learning
If imparting expert knowledge and search areboth not reasonable/possible, then machinelearning might work
Examples: Reinforcement learning
Neural networks
Decision tree learning Not often used by game developers
Why?
7/28/2019 5.3 AI Agents
21/79
21
Thinking:Flip-Flopping Decisions
Must prevent flip-flopping of decisions
Reaction times might help keep it from
happening every frame
Must make a decision and stick with it
Until situation changes enough
Until enough time has passed
7/28/2019 5.3 AI Agents
22/79
7/28/2019 5.3 AI Agents
23/79
23
Acting:Showing Intelligence
Adeptness and subtlety of actions impactperceived level of intelligence
Enormous burden on asset generation Agent can only express intelligence in terms
of vocabulary of actions
Current games have huge sets ofanimations/assets
Must use scalable solutions to make selections
7/28/2019 5.3 AI Agents
24/79
24
Extra Step in Cycle:Learning and Remembering
Optional 4th step
Not necessary in many games
Agents dont live long enough
Game design might not desire it
7/28/2019 5.3 AI Agents
25/79
25
Learning
Remembering outcomes andgeneralizing to future situations
Simplest approach: gather statistics
If 80% of time player attacks from left
Then expect this likely event
Adapts to player behavior
7/28/2019 5.3 AI Agents
26/79
26
Remembering
Remember hard facts Observed states, objects, or players
For example Where was the player last seen?
What weapon did the player have?
Where did I last see a health pack?
Memories should fade Helps keep memory requirements lower
Simulates poor, imprecise, selective humanmemory
7/28/2019 5.3 AI Agents
27/79
27
Rememberingwithin the World
All memory doesnt need to be stored inthe agent can be stored in the world
For example:Agents get slaughtered in a certain area
Area might begin to smell of death
Agents path planning will avoid the area
Simulates group memory
7/28/2019 5.3 AI Agents
28/79
28
Making Agents Stupid
Sometimes very easy to trounce player
Make agents faster, stronger, more accurate
Sometimes necessary to dumb down agents,for example:
Make shooting less accurate
Make longer reaction times
Engage player only one at a time
Change locations to make self more vulnerable
7/28/2019 5.3 AI Agents
29/79
29
Agent Cheating
Players dont like agent cheating
When agent given unfair advantage in speed,
strength, or knowledge Sometimes necessary
For highest difficultly levels
For CPU computation reasons
For development time reasons
Dont let the player catch you cheating!
Consider letting the player know upfront
7/28/2019 5.3 AI Agents
30/79
30
Finite-State Machine (FSM)
Abstract model of computation
Formally:
Set of states
A starting state
An input vocabulary
A transition function that maps inputs andthe current state to a next state
7/28/2019 5.3 AI Agents
31/79
31
Finite-State Machine:In Game Development
Deviate from formal definition1. States define behaviors (containing code)
Wander, Attack, Flee
2. Transition function divided among states Keeps relation clear
3. Blur between Moore and Mealy machines Moore (within state), Mealy (transitions)
4. Leverage randomness5. Extra state information
For example, health
7/28/2019 5.3 AI Agents
32/79
32
Most common game AI software pattern Natural correspondence between states and
behaviors
Easy to diagram
Easy to program
Easy to debug
Completely general to any problem
Problems Explosion of states
Often created with ad hoc structure
7/28/2019 5.3 AI Agents
33/79
33
Finite-State Machine:UML Diagram
Wander Attack
Flee
See Enemy
LowH
ealthNo
Enemy
No Enemy
7/28/2019 5.3 AI Agents
34/79
34
Finite-State Machine:Approaches
Three approaches
Hardcoded (switch statement)
Scripted
Hybrid Approach
7/28/2019 5.3 AI Agents
35/79
35
Finite-State Machine:Hardcoded FSMvoid RunLogic( int * state ) {
switch( state )
{
case 0: //Wander
Wander();
if( SeeEnemy() ) { *state = 1; }break;
case 1: //Attack
Attack();
if( LowOnHealth() ) { *state = 2; }
if( NoEnemy() ) { *state = 0; }
break;
case 2: //Flee
Flee();
if( NoEnemy() ) { *state = 0; }
break;
}
}
7/28/2019 5.3 AI Agents
36/79
36
Finite-State Machine:Problems with switch FSM
1. Code is ad hoc Language doesnt enforce structure
2. Transitions result from polling Inefficient event-driven sometimes better
3. Cant determine 1st time state is
entered4. Cant be edited or specified by game
designers or players
7/28/2019 5.3 AI Agents
37/79
37
Finite-State Machine:Scripted with alternative language
AgentFSM
{
State( STATE_Wander )
OnUpdate
Execute( Wander )
if( SeeEnemy ) SetState( STATE_Attack )OnEvent( AttackedByEnemy )
SetState( Attack )
State( STATE_Attack )
OnEnter
Execute( PrepareWeapon )
OnUpdate
Execute( Attack )
if( LowOnHealth ) SetState( STATE_Flee )if( NoEnemy ) SetState( STATE_Wander )
OnExit
Execute( StoreWeapon )
State( STATE_Flee )
OnUpdate
Execute( Flee )
if( NoEnemy ) SetState( STATE_Wander )}
7/28/2019 5.3 AI Agents
38/79
38
Finite-State Machine:Scripting Advantages
1. Structure enforced
2. Events can be handed as well as
polling
3. OnEnter and OnExit concept exists
4. Can be authored by game designers
Easier learning curve than straight C/C++
7/28/2019 5.3 AI Agents
39/79
39
Finite-State Machine:Scripting Disadvantages
Not trivial to implement
Several months of development
Custom compiler With good compile-time error feedback
Bytecode interpreter
With good debugging hooks and support
Scripting languages often disliked by users Can never approach polish and robustness of
commercial compilers/debuggers
7/28/2019 5.3 AI Agents
40/79
40
Finite-State Machine:Hybrid Approach
Use a class and C-style macros to approximate ascripting language
Allows FSM to be written completely in C++
leveraging existing compiler/debugger Capture important features/extensions
OnEnter, OnExit Timers Handle events Consistent regulated structure Ability to log history Modular, flexible, stack-based Multiple FSMs, Concurrent FSMs
Cant be edited by designers or players
7/28/2019 5.3 AI Agents
41/79
41
Finite-State Machine:Extensions
Many possible extensions to basic FSM
OnEnter, OnExit
Timers
Global state, substates
Stack-Based (states or entire FSMs)
Multiple concurrent FSMs Messaging
7/28/2019 5.3 AI Agents
42/79
42
Common Game AI Techniques
Whirlwind tour of common techniques
7/28/2019 5.3 AI Agents
43/79
43
Common AI Techniques:A* Pathfinding
Directed search algorithm used forfinding an optimal path through the
game worldA* is regarded as the best
Guaranteed to find a path if one exists
Will find the optimal pathVery efficient and fast
7/28/2019 5.3 AI Agents
44/79
7/28/2019 5.3 AI Agents
45/79
7/28/2019 5.3 AI Agents
46/79
7/28/2019 5.3 AI Agents
47/79
47
Common AI Techniques:Flocking
Example of emergent behavior Simulates flocking birds, schooling fish
Developed by Craig Reynolds 1987 SIGGRAPH paper
Three classic rules
1. Separation avoid local flockmates2. Alignment steer toward average heading
3. Cohesion steer toward average position
7/28/2019 5.3 AI Agents
48/79
7/28/2019 5.3 AI Agents
49/79
7/28/2019 5.3 AI Agents
50/79
7/28/2019 5.3 AI Agents
51/79
51
Common AI Techniques:Manager Task Assignment
Manager organizes cooperationbetween agents
Manager may be invisible in gameAvoids complicated negotiation and
communication between agents
Manager identifies important tasks andassigns them to agents
h
7/28/2019 5.3 AI Agents
52/79
52
Common AI Techniques:Obstacle Avoidance
Paths generated from pathfindingalgorithm consider only static terrain,
not moving obstacles Given a path, agent must still avoid
moving obstacles
Requires trajectory prediction Requires various steering behaviors
C h
7/28/2019 5.3 AI Agents
53/79
53
Common AI Techniques:Scripting
Scripting specifies game data or logic outsideof the games source language
Scripting influence spectrumLevel 0: Everything hardcoded
Level 1: Data in files specify stats/locations
Level 2: Scripted cut-scenes (non-interactive)
Level 3: Lightweight logic, like trigger system
Level 4: Heavy logic in scripts
Level 5: Everything coded in scripts
7/28/2019 5.3 AI Agents
54/79
7/28/2019 5.3 AI Agents
55/79
C AI T h i
7/28/2019 5.3 AI Agents
56/79
56
Common AI Techniques:Stack-Based State Machine
Also referred to as push-down automata
Remembers past states
Allows for diversions, later returning toprevious behaviors
C AI T h i
7/28/2019 5.3 AI Agents
57/79
57
Common AI Techniques:Subsumption Architecture
Popularized by the work of Rodney Brooks
Separates behaviors into concurrently runningfinite-state machines
Lower layers Rudimentary behaviors (like obstacle avoidance)
Higher layers
Goal determination and goal seeking Lower layers have priority
System stays robust
7/28/2019 5.3 AI Agents
58/79
C AI T h i
7/28/2019 5.3 AI Agents
59/79
59
Common AI Techniques:Trigger System
Highly specialized scripting system
Uses if/then rules
If condition, then response Simple for designers/players to
understand and create
More robust than general scripting Tool development simpler than general
scripting
7/28/2019 5.3 AI Agents
60/79
7/28/2019 5.3 AI Agents
61/79
7/28/2019 5.3 AI Agents
62/79
P i i AI T h i
7/28/2019 5.3 AI Agents
63/79
63
Promising AI Techniques:Decision Tree Learning
Constructs a decision tree based onobserved measurements from game
world Best known game use: Black & White
Creature would learn and form opinions
Learned what to eat in the world based onfeedback from the player and world
7/28/2019 5.3 AI Agents
64/79
7/28/2019 5.3 AI Agents
65/79
7/28/2019 5.3 AI Agents
66/79
Promising AI Techniques:
7/28/2019 5.3 AI Agents
67/79
67
Promising AI Techniques:N-Gram Statistical Prediction
Technique to predict next value in asequence
In the sequence 18181810181, it wouldpredict 8 as being the next value
Example
In street fighting game, player just did LowKick followed by Low Punch
Predict their next move and expect it
7/28/2019 5.3 AI Agents
68/79
Promising AI Techniques:
7/28/2019 5.3 AI Agents
69/79
69
Promising AI Techniques:Perceptrons
Single layer neural network
Simpler and easier to work with than
multi-layer neural network Perceptrons get stimulated enough to
either fire or not fire
Simple yes/no output
Promising AI Techniques:
7/28/2019 5.3 AI Agents
70/79
70
Promising AI Techniques:Perceptrons (2)
Game example: Black & White
Creature used perceptron for hunger
Three inputs: low energy, tasty food, andunhappiness
If creature ate and received positive ornegative reinforcement, then perceptronweights were modified
Results in learning
7/28/2019 5.3 AI Agents
71/79
7/28/2019 5.3 AI Agents
72/79
Promising AI Techniques:
7/28/2019 5.3 AI Agents
73/79
73
Promising AI Techniques:Production Systems
Formal rule-based system
Database of rules
Database of facts Inference engine to decide which rules trigger
resolves conflicts between rules
Example
Soarused experiment with Quake 2 bots
Upwards of 800 rules for competent opponent
Promising AI Techniques:
7/28/2019 5.3 AI Agents
74/79
74
Promising AI Techniques:Reinforcement Learning
Machine learning technique
Discovers solutions through trial and error
Must reward and punish at appropriatetimes
Can solve difficult or complex problems likephysical control problems
Useful when AIs effects are uncertainor delayed
7/28/2019 5.3 AI Agents
75/79
Promising AI Techniques:
7/28/2019 5.3 AI Agents
76/79
76
Promising AI Techniques:Smart Terrain
Put intelligence into inanimate objects
Agent asks object how to use it
Agents can use objects for which they werentoriginally programmed for
Allows for expansion packs or user createdobjects, like in The Sims
Enlightened by Affordance Theory Objects by their very design afford a very specific
type of interaction
7/28/2019 5.3 AI Agents
77/79
Promising AI Techniques:
7/28/2019 5.3 AI Agents
78/79
78
Promising AI Techniques:Text-to-Speech
Turns ordinary text into synthesized speech
Cheaper than hiring voice actors
Quality of speech is still a problem Not particularly natural sounding
Intonation problems
Algorithms not good at voice acting
Large disc capacities make recording humanvoices not that big a problem
No need to resort to worse sounding solution
7/28/2019 5.3 AI Agents
79/79