1

Artificial Intelligence in GamesWeek 3

Steve Rabinsteve.rabin@gmail.com

www.aiwisdom.com/digipen

TA: Chi-Hao Kuo (chihao.kuo@digipen.edu)TA: Allan Deutsch (allan.d@digipen.edu)

2

Questions1. Name 3 procedural AI architectures.2. Name 3 declarative AI architectures.3. Name the 4 AI architecture “Power Design Patterns”.4. What behavior isn’t captured in a UML State Machine

diagram?5. If multiple states exist within a single state of a finite-state

machine, what is the name of this architecture?6. Name four key EVENTS that a state in the State Machine

Language (SML) might respond to.7. What is the key benefit of the global message response

feature in SML?8. How does SML support load balancing?9. Name 2 ways that SML supports Option Stacks.

3

Behavior Trees

4

Behavior Trees Combines

Hierarchical state machines Scheduling Planning Action Execution

Halo 2 had 50 different behaviors Tree structure of behaviors

Power comes from composability Hierarchies without having to know details of how things

get done Non-leaf behaviors decide which children should run Leaf behaviors get things done (task)

Analogy: State is to FSM as Task is to Behavior Tree

5

Halo 2Behavior Tree

6

Selector Node(execute tasks in order until success)

Attack Taunt Stare

?

Succeed or Fail? Succeed or Fail? Succeed or Fail?

7

Selector Node(execute random tasks until success)

Attack Taunt Stare

~?

Succeed or Fail? Succeed or Fail? Succeed or Fail?

8

Sequence Node(each task in order, unless fail)

Enemy

Visible?

Turn

away

Run

away

9

Sequence Node(each in order, unless fail)

Enemy

Visible?

Turn

away

Run

away

Succeed or Fail?

10

Sequence Node(each in order, unless fail)

Enemy

Visible?

Turn

away

Run

away

Succeed! Succeed or Fail?

11

Sequence Node(each in order, unless fail)

Enemy

Visible?

Turn

away

Run

away

Succeed! Succeed! Succeed or Fail?

12

Sequence Node(each in order, unless fail)

Enemy

Visible?

Turn

away

Run

away

Succeed! Succeed! Succeed!

13

Sequence Node(each in order, unless fail)

Enemy

Visible?

Turn

away

Run

away

Succeed! Succeed! Succeed!

Succeed!

14

Sequence Node(each in order, unless fail)

Enemy

Visible?

Turn

away

Run

away

Succeed! Succeed! Fail

15

Sequence Node(each in order, unless fail)

Enemy

Visible?

Turn

away

Run

away

Succeed! Succeed! Fail

Fail!

16

Nondeterministic Sequence Node(random until all executed, unless fail)

Get

matches

Get

gasolineSucceed or Fail? Succeed or Fail?

17

Parallel Node(all tasks in parallel, until fail)

Run to

Enemy

Battle

crySucceed or Fail? Succeed or Fail?

18

Parallel Node and Decorator Node(for condition checking)

Trash

Visible?Yes or no? Succeed or Fail?

Until fail

Tidy

trash

Fail?

(DecoratorNode)

Decorator Nodeshttp://aigamedev.com/open/article/decorator/

Filters Limit the # of times it can run Prevent from firing too often with a timer Temporarily deactivate Restrict number of simultaneous running behaviors

Managers & Handlers Deal with the error status code and restart

planning or execution Store information for multiple child nodes to access

as a blackboard

19

Decorator Nodes Control Modifiers

Force a return status (always fail or succeed)

Fake a certain behavior (keep running instead of failing or succeeding)

Meta Operations Debug breakpoint

(pause execution and prompt the user) Logging

(track this node execution and print to the console)

20

21

Behavior Tree:Node Recap

?

~?

Selector Node: Each in order until one has success

Selector Node: Random until one has success

Nondeterministic Selector: Random until all have success

Sequence Node: Each in order unless fail

Parallel Node: All task in parallel until fail

22

Behavior Tree(to enter a room)

Move

(into room)

?

Door

open?

Move

(to door)

Move

(into room)?

Barge

door

Door

open?

Open

door

Door

locked?

23

Behavior Tree:Activity Open door if unlocked Otherwise either (doesn’t matter

which): Barge the door Light door on fire

But you first need matches and gasoline You need to douse gasoline on the door You need to ignite door

24

Behavior Tree:Node Recap

?

~?

Selector Node: Each in order until one has success

Selector Node: Random until one has success

Nondeterministic Selector: Random until all have success

Sequence Node: Each in order unless fail

Parallel Node: All task in parallel until fail

25

Behavior Tree(to enter a room)

Entering... Open door...

Barge door...

?

~?

Douse door Ignite door

Get matches Get gasoline

26

Behavior Treecommunicating with blackboard

Enemy

visible? ?Select enemy

(write to blackboard)

Engage enemy

(read from blackboard)

Always

succeed

High ground

available?

Move to

high ground

Example from Game AI ProBehaviorTree* bt = BehaviorTreeBuilder() .activeSelector() .sequence() //Attack the player if seen! .condition(IsPlayerVisible) .activeSelector() .sequence() .condition(IsPlayerInRange) .filter(Repeat, 3) .action(FireAtPlayer) .action(MoveTowardsPlayer) .sequence() //Search near last known position. .condition(HaveWeGotASuspectedLocation) .action(MoveToPlayersLastKnownPosition) .action(LookAround) .sequence() //Randomly scanning nearby. .action(MoveToRandomPosition) .action(LookAround) .end();

27

28

Behavior Trees More scalable than simple FSMs Similar to HFSMs but with rule-based

transitions More information:

Halo 2 article http://www.gamasutra.com/gdc2005/features/200

50311/isla_01.shtml Behavior Tree Overview

http://aigamedev.com/hierarchical-logic/bt-overview

29

Behavior Trees Criticism Clunky with state-based behavior

Hard to design in term of states AI based on behavior trees tends to avoid

this behavior

Doesn't deal nicely with interruptions Cumbersome to implement Examples:

Interrupting patrol route to hide or raise alarm Switching strategies when ammo low

SWITCH SLIDES!

30

31

Utility

32

Decision Making with Utility Score choices to choose the best Utility score is NOT value

Do I want: $4.99 cat toy – 2 days to be delivered $2.99 cat toy – 5 days to be delivered

33

Decision Making with Utility:Normalized & Expected Utility Utility Scores are Normalized: [0, 1]

Makes scores comparable

Expected Utility Utility * Probability

Maximum Expected Utility Choose one with highest Expected

Utility

34

Decision Making with Utility:Maximum Expected Utility Attack with dagger

Utility: 0.6 Probability of hit: 85%

Attack with two handed sword Utility: 0.9 Probability of hit: 60%

Attack with dagger: 0.6 x 0.85 = 0.51 Attack with sword: 0.9 x 0.60 = 0.54

35

Combining Utility Scores:Mix and Match

36

Transforming Utility with Functions

37

Transforming Utility with Functions:Hunger in The Sims

38

Smart Terrain

39

Smart Terrain Accredited to Will Wright and The Sims Intelligence resides in terrain/objects in

addition to characters Behaviors and possible actions associated with an

object are embedded in that object Example: TV object

File for TV model also contains instructions for Watching it, Turning it on and off Conditions under which a particular Sim might want to

watch it How the Sim reacts while watching it

40

Smart Terrain Characters have motivations and needs

Terrain offers various ways to satisfy those needs

Each piece of terrain broadcasts to nearby characters what it has to offer

For example, when a hungry character walks near a refrigerator, the refrigerator's "I have food" broadcast allows the character to decide to get some food from it

41

Smart Terrain Character is guided from action to action

realistically, driven only by simple, object-level programming

Refrigerator "I have food"

Food "Must be cooked"

Microwave "I can cook food"

42

Smart Terrain:The Sims Architecture in Detail

Fix Dinner

Disposal (neat)Placement

Surface

SeatingCookPreparation

43

Smart Terrain:The Sims Architecture in Detail

Data

-Needs

-Personality

-Skills

-Relationships

Person Physical

-Hunger

-Comfort

-Hygiene

-Bladder

Mental

-Energy

-Fun

-Social

-Room

Motive Engine

Personality

-Sloppy to Neat

-Shy to Outgoing

-Serious to Playful

-Lazy to Active

-Mean to Nice

Skills

-Cooking

-Mechanical

-Logic

-Body

-Etc

44

Smart Terrain:The Sims Architecture in Detail

Stats

Hunger +20

Comfort -12

Hygiene -30

Bladder -75

Energy +80

Fun +40

Social +10

Room -60

Mood +18

PersonToilet

Mood+26

Urinate (+40 Bladder)

Clean (+30 Room)

Unclog (+40 Room)

Bathtub

Mood +20

Take Bath (+40 Hygiene, +30 Comfort)

Clean (+20 Room)

Find Best Action

45

Smart Terrain:The Sims Architecture in Detail

46

Smart Terrain:The Sims Architecture in Detail

"Happyscape"

47

Smart Terrain:The Sims Architecture in Detail

"Happyscape"

48

Smart Terrain:The Sims Architecture in Detail

"Happyscape"

49

Subsumption Architecture

50

Subsumption Architecture Term invented by Rodney Brooks

(MIT) in mid to late 1980s AI model invented by Alexandre Parodi Designed for behavior based robots

J. J. Nitao and A. M. Parodi (1986) "A Real-Time Reflexive Pilot for an Autonomous Land Vehicle", IEEE Control Systems, vol. 6, no. 1, February, pp.14-23.

R. A. Brooks (1986) "A Robust Layered Control System For A Mobile Robot", IEEE Journal Of Robotics And Automation, RA-2, April. pp. 14-23.

51

Subsumption Architecture One school of thought is that

robots should model the world internally and operate on this internal model

Rodney Brooks argued World too complex to model internally

"The world is its own best model" Robots should operate by sensing the

changing environment

52

Subsumption Architecture Cleanly separates the behavior of a

single character into concurrently running behaviors Each behavior could be a finite-state machine

Higher layers Goal determination and goal seeking

Lower layers Obstacle avoidance and other rudimentary

behaviors Priority is with lower layers

Can inhibit inputs or overrule outputs Lower layers form complete working system

53

Subsumption Architecture:Rodney Brooks' "Allen" Robot Bottom layer: Avoidance

Avoids static and dynamic obstacles Close objects cause robot to scurry (steer) away Speed controlled by inverse square of distance to

nearest obstacle Middle layer: Wandering

Picks a destination and travels for 10 seconds Top layer: Exploring

Uses sonar to identify distance places to explore If found, it would go toward it and suppress the middle

wander layer (but summed with bottom layer)

54

Subsumption Architecture:Robot Example

Avoid Obstacle

Highest Layer (goals)

Lowest Layer (reflexes)

Motors

Wander

Explore World

Create a Map

Sensor

Data

55

Subsumption Architecture:Evaluation Key Benefits of Architecture

Robustness Goal preservation

Problems Can't be modularized easily Low flexibility at runtime

56

Subsumption Architecture: Known games that used

subsumption: Halo series Oddworld series Jedi Knight: Dark Forces 2

Slightly different implementation, but still spirit of subsumption Increases flexibility and modularization

57

Subsumption Architecture:Game Implementation (Oddworld series)

Movement

Action

Behavior

Tactics

58

Subsumption Architecture:Game Implementation Movement layer forces animation

constraints Character can't go directly from "Falling

through Air" state to "Stand" state without first going to "Land" state

If hit by a punch, plays "Knock Back" animation and slides the character back a step

If character runs into a wall, it's stopped and a "Knock Back" is played

If character loses contact with ground, it falls Each state moves the character and plays the

correct animation

59

Subsumption Architecture:Game Implementation Action layer

Short term goals over a finite period of time Example actions: Goto, Fight, and Die

Actions can succeed or fail (unlike motions) Goto action asks nav system for list of

waypoints and sends requests to movement layer

run, turn, run ,run, climb down ladder, run, stop Fight action might have a time limit

Failed if enemy can't be engaged Succeeded if enemy dies or engaged until time limit

Die action chooses between different death options in the movement layer

For example, die while falling, jumping, or fighting

60

Subsumption Architecture:Game Implementation Behavior layer

Goals with indefinite time periods Constructed out of actions

Examples Patrol, Investigate Noise, Attack Enemy X

Patrol behavior is sequence of actions Go to location A Look around Go to location B Call in: Alpha Sector Secured! Go to location C Repeat

Attack behavior Actions like pursue character, collect nearby weapons,

look for good spots to attack from and move to them

61

Subsumption Architecture:Game Implementation Tactical layer: Defend area example

Update() {if( low health ) {

request Retreat behavior; return;}if( behavior in progress ) && ( not timed out )

return;if( no enemy spotted ) {

if( suspicious noise ) request Investigate Noise behavior; return;else request Patrol behavior; return;

}else {

if( no alarm ) { if( close to enemies ) request Retreat behavior; return; else request Sound Alarm behavior; return;}request Attack Enemy behavior; return;

}}

62

Subsumption Architecture:Game Implementation Upper layers make requests Each layer should not make its own

state change decisions Instead allow upper layers to make state

changes through requests If character is investigating a noise and

sees an enemy, it might retreat to sound an alarm or it might attack immediately The decision is not inside the "Investigate

Noise" behavior but is pushed up to the tactical layer

63

Subsumption Architecture:Game Implementation Summary

Lower layers ignore wishes of upper layers in order to enforce rules of the game

Rule-enforcing code centralized, easy to debug, and hard to break with higher-level AI code

Character remembers what it was doing if setbacks occur

Because state is stored at every level

64

Subsumption Architecture:Game Implementation Multiple modules per layer might be

needed Example: Need ability to shoot while

running Instead of adding states like "Running and

Shooting", "Standing and Throwing" One solution is to split the body into two

parts: legs and torso If animation needs both parts (like kicking)

then the torso module is suppressed

65

Subsumption Architecture:Game Implementation

Body Movement

Action

Behavior

Tactics

Torso Movement

66

Scripting Many games support AI scripting - Why? Pros

No recompiling of code base, designers empowered, extensible to players

Cons Harder to debug, non-programmers required to program,

time commitment and risk to create tools Choice of language

Existing languages: Python or Lua Make your own language

Dungeon Siege: Skrit Unreal Engine: UnrealScript Jak and Daxter: GOAL

Hybrid: State Machine Language Trigger system

67

Scripting: Custom Scripting Languages ID Software: QuakeC

Quake series, Doom 3, games that use Quake Engine Epic Games: UnrealScript

Unreal Tournament and games that use Unreal Engine BioWare: NWscript

Neverwinter Nights Lucas Arts: SCUMM (Script Creation Utility for Maniac

Mansion) 50+ games

Maxis: Simantics The Sims, The Sims 2

Gas Powered Games: Skrit Dungeon Siege, Dungeon Siege 2, Supreme Commander

Naughty Dog: GOAL (Goal Oriented Action Language) Crash Bandicoot series and Jak and Daxter series

WizBang Studio Productions: Adlib Activision's HyperBlade, Microsoft Baseball series

68

Scripting: Lua Currently most popular off-the-

shelf scripting language Games that have used Lua

World of Warcraft, Far Cry, Crysis, Psychonauts

Baulder's Gate, Homeworld 2, MDK2 Grim Fandango, Escape from Monkey Island

69

Scripting:Degree of Scripting / Data-DrivenLevel 0: Hardcode everything in C/C++Level 1: Data in files specify stats/locationsLevel 2: Scripted cutscene sequencesLevel 3: Lightweight logic specified by tools or

scripts (trigger system)Level 4: Heavy logic in scripts that rely on core

C/C++ functionsLevel 5: Everything coded in scripts – full

alternative language to C/C++ (GOAL)

70

Trigger System Simple

Easy to write tools and hooks Can be entirely in GUI (StarCraft level editor)

Easy for designers and players Should be impossible to crash game

If CONDITIONS then RESPONSES If player steps

ON_FLOOR_TILE(54)_and_HOLDING(BAD_LUCK_MEDALLION) then MUSIC(EERIE)_and_SPAWN(SNAKES, 20)

Perfect for mission-based games like FPS, RTS or RPGs

71

StarCraft Campaign Editor:Based on Trigger System Concept

72

Trigger System:Centralized Trigger System Create a centralize system to organize

triggers Instead of scattered and ad hoc

Saves CPU time Agents can subscribe to triggers/events they

are interested in Triggers occur and register themselves Loop through agents to see if they are

listening to any triggers that occurred Promotes event-driven, rather than polling

73

Trigger System:One Implementationstruct TriggerRecordStruct{ EnumTriggerType eTriggerType; //The condition unsigned long nTriggerID; //Unique id of this event unsigned long idSource; //What created the event Vector vPos; //Event position float fRadius; //Influence radius of event unsigned long nTimeStamp; //When the event occurred unsigned long nExpirationTime; //Fixed event duration};

enum EnumTriggerType{ kTrig_None = 0, kTrig_Explosion = (1 << 0), kTrig_EnemyNear = (1 << 1), kTrig_Gunfire = (1 << 2)};

74

Trigger System:One Implementationenum EnumTriggerType{ kTrig_None = 0, kTrig_Explosion = (1 << 0), kTrig_EnemyNear = (1 << 1), kTrig_Gunfire = (1 << 2)};

EnumTriggerType is a bit field Efficient way to identify applicable triggers

For example, blind agent only aware of Explosions and Gunfire

dwTriggerFlags = kTrig_Explosion | kTrig_Gunfire; if( trigger->triggerType & dwTriggerFlags )

{ //Applicable trigger}

75

Trigger System:One Implementation triggerID

Unique ID used to refer to trigger (to remove) sourceID

Game object that created trigger Might be character who fired weapon or a landmine

that exploded vPos, fRadius

Agent only reacts if within trigger's radius nExpirationTime

Only valid for a specified amount of time Zero expiration time means it lives forever

bDynamicPos Flag that specifies trigger is moving and position must

be updated

76

Trigger System:One Implementation

class CTriggerSystem

{

public:

unsigned long RegisterTrigger( eTriggerType, nPriority,

idSource, vPos, fRadius,

fDuration, bDynamicPos );

void RemoveTrigger( unsigned long nTriggerID );

void Update();

private:

TRIGGER_MAP m_mapTriggerMap; //STL multimap sorted by priority

bool m_bTriggerCriticalSection;

};

77

Trigger System:One Implementation

unsigned long RegisterTrigger( eTriggerType, nPriority, idSource, vPos, fRadius, fDuration, bDynamicPos ){ //Create a trigger record and fill it in TriggerRecordStruct* pTriggerRecord = new TriggerRecordStruct( eTriggerType, idSource, vPos, fRadius, fDuration, bDynamicPos );

//Trigger records are sorted by priority m_mapTriggerMap.insert( TRIGGER_MAP::value_type( nPriority, pTriggerRecord) );

//Return the unique identifier for this trigger return pTriggerRecord->nTriggerID;}

78

Trigger System:One Implementation Update function (heart of the trigger system)

Removes expired triggers Refreshes dynamic positioning triggers Notifies agents of triggers that are relevant

Update(){ loop( triggers ) { if( trigger expired ) DeleteTrigger( trigger ); else if( trigger->bDynamicPos ) UpdatePos( trigger->vPos ); } loop( agents ) { if( time for Agent update ) { loop( triggers ) //From highest to lowest priority { if( Agent doesn't respond to trigger ) continue; if( this is the Agent ) continue; if( distance > trigger->fRadius ) continue; if( Agent->HandleTrigger( trigger ) ) break; } } }}

79

Trigger System:Versus Scripting Languages Can be specified within a GUI Very accessible to players Constrained to prevent crashes Quicker to implement Easier to debug

80

The Inverse Ninja Law The effectiveness of a group of ninja is

inversely proportional to the number of ninja in the group

e = effectiveness n = number of ninja k = as-yet-undetermined proportionality constant

n

ke

81

Stormtrooper Effect Minor characters have no chance against

the main characters Despite

Being close range Having overwhelming numbers Full armor Military-grade weapons Professional military training

They can't hit anything

82

Red Shirts Stock character whose sole purpose is to

violently die shortly after being introduced

Typically, a landing party would consist of several main characters Often Kirk, Spock, McCoy And one or two never-before-seen red-

shirted ensigns Who would be dead by the end of the mission,

usually within minutes

83

What would happen if Imperial Stormtroopers and Redshirts got in a firefight?

Can't hit anything Destined to die

VS

84

Rule-Based Architecture

85

Rule-Based Architecture:Basic Cycle

Match

ActConflict

Resolution

Changes to

Working Memory

Selected Rule

Rule Instantiations that

Match Working Memory

86

Rule-Based Architecture:Complete Picture

Match

ActConflict

Resolution

Changes to

Working Memory

Actions

Sensors

87

Rule-Based Architecture:Example Rules and Working Memory

Rules Get Item Rule

Condition(s) visible = true type = <type>

Action Get Item( <type> )

Working Memory Self

health = 1000 weapon = blaster

Item visible = true type = health range = 400

Item visible = true type = hyperblaster range = 500

88

Rule-Based Architecture:Example Rules (1)Rule: RetreatFromEnemyIfHealthLow

Condition(s) self health < 200 enemy visible = true, name <name>

Action RetreatFromEnemy( <name> )

Rule: RetreatFromEnemyIfOverwhelmed Condition(s)

enemygroup visible = true, groupsize > 3 Action

RetreatFromEnemy( <name1> )

Rule: WanderIfNoEnemyAndHealthLow Condition(s)

self health < 200 enemy visible = false

Action Wander()

89

Rule-Based Architecture:Example Rules (2)Rule: GetWeaponAmmoLow

Condition(s) self ammo = low, weapon <weapon> item visible = true, type <weapon>

Action GetItem( <weapon> )

Rule: GetWeaponIfDifferent Condition(s)

self ammo = high, weapon <weapon> item visible = true, type <weapon> != <nweapon>

Action GetItem( <nweapon> )

Rule: GetWeaponToDeprive Condition(s)

enemy visible = true, weapon <weapon> item visible = true, weapon <weapon> != <nweapon> assessbetter <nweapon>, <weapon>

Action GetItem( <nweapon> )

90

Rule-Based Architecture:Conflict Resolution Which rule that matches should fire? Which instantiation of a rule should fire?

Separate instantiation for every match of variables in rules

91

Rule-Based Architecture:Conflict Resolution FiltersSelect between instantiations based on filters Refractory Inhibition

Don't fire same instantiation that has already fired Data Recency

Select instantiations that match most recent data Specificity

Select instantiations that match more working memory data

More reasons to do action Random

Select randomly between the remaining instantiations Rule Order

Pick the first rule that matches Rule Importance

Pick rule with highest priority

92

Rule-Based Architecture:Specificity Example Explanation

Select instantiations that match more working memory data

More reasons to do action Example

if( "see weapon I don't have" && "have high ammo for weapon") then "get item"

if( "see item" ) then "get item"

93

Rule-Based Architecture:Efficiency Where does the time go?

90-95% goes to Match Matching all rules against all of

working memory each frame is too slow

Key observation # of changes to working memory

each frame is small

94

Rule-Based Architecture:Efficiency Only process changes to working

memory Save intermediate match information Recompute match for rules affected by

change Use hashing to speed up

95

Rule-Based Architecture:SOAR Experience Competent Quake 2 SOAR bot

800+ rules Could fight, get weapons/health, and ambush

player Problems

Can be memory intensive Can be computationally intensive Hard to debug Hard to fix bugs without affecting other

behavior

96

Rule-Based Architecture:Microsoft Games Studio John Laird consulted for Microsoft Plan to make rule-based system

engine for all Microsoft team sports games However, Microsoft Sports Group

disbanded one year later...