3 States and a Plan: The AI of F.E.A.R. Jeff Orkin Monolith Productions/ MIT Media Lab

3 States and a Plan:The AI of F.E.A.R.

Jeff Orkin

Monolith Productions/

MIT Media Lab

Survey Says!

Survey Says!

FSM: 3 States

FSM: 3 States

Goto Animate

UseSmartObject

FSM: 3 States

Goto Animate

Animate

FSM: 2 States!

Goto Animate

FSM: Transition Logic

void StateAttack::Update(){

//...

if( Ammo == 0 ){

pState = Reload(bCrouch);return;

}

//...}

Shogo, 1998

No One Lives Forever, 2000

F.E.A.R., 2005

Halo 2, 2004Allowable behaviors for infantry, drivers, and passengers

FSM vs Planning

FSM Planning

- How - What

FSM vs Planning

FSM Planning

- How - What

- Procedural - Declarative

Motivation

Here’s the Plan:

• STRIPS Planning Overview

• Planning in F.E.A.R.

• Differences from STRIPS

• Squad Behaviors & Communication

• Beyond F.E.A.R.

What is Planning?

• Planning is a formalized process of searching for sequence of actions to satisfy a goal.

• Process is called “Plan Formulation.”

STRIPS Planning

…in a nutshell

STRIPS Planning

STRIPS =

STanford Research Institute

Problem Solver

STRIPS Planning

• STRIPS Goal: Desired state of the world to reach.

• STRIPS Actions:– Preconditions– Effects

States

States: FSM

Attack Search

States: PlanningRepresented as a logical sentence:

AtLocation(Home) ^ Wearing(Tie)

Represented as a vector:

(AtLocation, Wearing) = (Home, Tie)

States: PlanningExample: Lemonade Stand

( weather, #lemons, $$ )=

( , , ) or

( , , )

STRIPS Planning Example

STRIPS Planning Example

STRIPS Planning Example

STRIPS Planning Example

STRIPS Planning Example

STRIPS Planning Example

State: (phone#, recipe, hungry?)

Goal: ( -- , -- , NO )

( , -- ,YES) ( , -- ,NO )

( -- , ,YES) ( -- , ,NO )

STRIPS Planning Example

State: (phone#, recipe, hungry?)Goal: ( -- , -- , NO )

( , ,NO )

( , ,YES) ( , ,NO )

STRIPS Planning Example

STRIPS Planning Example

State: (phone#, recipe, hungry?)

ActionPreconditions: ( , -- , -- )

Effects:

Delete List: Hungry( YES )

Add List: Hungry( NO )

STRIPS Planning Example

State: (phone#, recipe, hungry?)

ActionPreconditions: ( , -- , -- )

Effects:

Hungry( NO )

Hungry( YES ) ^ Hungry( NO )

STRIPS Planning Example

Action: Buy( object )Preconditions: ( -- )

Effects:

Delete List: --

Add List: Own( object )

Own( ) ^ Own( ) ^ Own( )

STRIPS Planning Example

STRIPS Planning Example

STRIPS Planning Example

STRIPS Planning Example

STRIPS Planning Example

FPS Statistics

Average lifespan of FPS enemy AI:

FPS Statistics

Average lifespan of FPS enemy AI:

12.23 seconds

FPS Statistics

Average lifespan of FPS enemy AI:

12.23 seconds

Number of enemy AI who die in

FPS’s per day:

FPS Statistics

Average lifespan of FPS enemy AI:

12.23 seconds

Number of enemy AI who die in

FPS’s per day:

19,789,203

FPS Statistics

Sally Struthers says “Save the AI!”

Here’s the Plan:

• STRIPS Planning Overview

• Planning in F.E.A.R.

• Differences from STRIPS

• Squad Behaviors & Communication

• Beyond F.E.A.R.

Design Philosophy

Designer’s job is:

Create environments that allow AI to showcase their behaviors.

Designer’s job is NOT:

Script behavior of individual AI.

Planning Video #1

Planning Video #1

Planning Video #1

Planning Video #1

Planning Video #1

Planning Video #1

Planning Video #2

Planning Video #3

Planning Video #4

Planning Video #4Soldier Assassin Rat

Benefits of Planning

1. Decoupling Goals & Actions

2. Layering Behaviors

3. Dynamic Problem Solving

Decoupling Goals & Actions

Decoupling Goals & Actions

AttackFromCover

tt

t

t

t

Priority: 8 .0Sensors: V is ion, CoverNodes

Patrol

tt

t

t

t

Priority: 1 .0Sensors: PatrolNodes

Investigate

tt

t

t

t

Priority: 3 .0Sensors: HearDisturbance

Search

tt

t

t

t

Priority: 6 .0Sensors: V is ion, SearchNodes

Decoupling Goals & Actions

Decoupling Goals & Actions

+ =

Decoupling Goals & Actions

AttackFromCover

tt

t

t

t

Priority: 8 .0Sensors: V is ion, CoverNodes

Patrol

tt

t

t

t

Priority: 1 .0Sensors: PatrolNodes

Investigate

tt

t

t

t

Priority: 3 .0Sensors: HearDisturbance

Search

tt

t

t

t

Priority: 6 .0Sensors: V is ion, SearchNodes

Decoupling Goals & Actions

Decoupling Goals & Actions

Decoupling Goals & Actions

Work

tt

t

t

t

Priority: 8 .0

Death

tt

t

t

t

Priority: 100.0

Stunned

tt

t

t

t

Priority: 80.0

Decoupling Goals & ActionsWork

RecoilStand

Up

SitDown

Anim ate

Goto

Priority: 8 .0

Death

Priority: 100.0

Stunned

Priority: 80.0

Working Memory

Benefits of Planning

1. Decoupling Goals & Actions

2. Layering Behaviors

3. Dynamic Problem Solving

Layering Behaviors

Layering Behaviors

Layering Behaviors

Layering Behaviors

Layering Behaviors

Layering Behaviors

Layering Behaviors

Layering Behaviors

Layering Behaviors

Layering Behaviors

Layering Behaviors

Layering Behaviors

Layering Behaviors

Layering Behaviors

Layering Behaviors

Layering Behaviors

Layering Behaviors

Kill Enemy

Attack Ranged

Layering Behaviors

Kill Enemy

Attack Ranged

Attack Melee

Layering Behaviors

Kill Enemy

Attack Ranged

Attack Melee

Dodge Shuffle

Dodge Roll

Dodge

Layering Behaviors

Kill Enemy

Attack Ranged

Attack Melee

Dodge Shuffle

Dodge Roll

Dodge

Cover

Goto

Dodge Covered

Layering Behaviors

Kill Enemy

Attack Ranged

Attack Melee

Dodge Shuffle

Dodge Roll

Dodge

Cover

Goto

Dodge Covered

Blind Fire

Layering Behaviors

Kill Enemy

Attack Ranged

Attack Melee

Dodge Shuffle

Dodge Roll

Dodge

Cover

Goto

Dodge Covered

Blind Fire

Ambush

Layering Behaviors

Kill Enemy

Attack Ranged

Attack Melee

Dodge Shuffle

Dodge Roll

Dodge

Cover

Goto

Dodge Covered

Blind Fire

Ambush

Layering Behaviors

Benefits of Planning

1. Decoupling Goals & Actions

2. Layering Behaviors

3. Dynamic Problem Solving

Dynamic Problem Solving

Dynamic Problem Solving

Differences From STRIPS

1. Cost Per Action

2. No Add/Delete Lists

3. Procedural Preconditions

4. Procedural Effects

Cost Per Action

Cost Per Action

2 8

Cost Per Action

2 8

*A

Cost Per Action

A

Cost Per Action

A* Navigation Planning

Nodes: NavMesh Polys World States

Edges: NavMesh Poly Actions

Edges

Goal: NavMesh Poly World State

Cost Per Action

A

Differences From STRIPS

1. Cost Per Action

2. No Add/Delete Lists

3. Procedural Preconditions

4. Procedural Effects

No Add/Delete Lists

State: (phone#, recipe, hungry?)

ActionPreconditions: ( , -- , -- )

Effects:

Delete List: Hungry( YES )

Add List: Hungry( NO )

No Add/Delete Lists

State: [phone#, recipe, hungry?]

ActionPreconditions: [ , -- , -- ]

Effects: [ -- , -- , NO ]

No Add/Delete Lists

AttackPreconditions: [ -- , -- , ]

ReloadEffects: [ -- , -- , ]

No Add/Delete Lists

[ -- , -- , -- ] 4-byte values

TargetDead [bool]WeaponLoaded [bool]OnVehicleType [enum]AtNode [HANDLE] - or -AtNode [variable*]

No Add/Delete Lists

Goal Cover: [ -- , CoverNode63, -- ]

Action GotoEffect: [ -- , variable*, -- ]

No Add/Delete Lists

[ -- , -- , ]

No Add/Delete Lists

[ -- , -- , ]

Differences From STRIPS

1. Cost Per Action

2. No Add/Delete Lists

3. Procedural Preconditions

4. Procedural Effects

Procedural Preconditions

class Action{

// [ -- , -- , -- ]WORLD_STATE m_Preconditions;WORLD_STATE m_Effects;

bool CheckPreconditions();

};

Procedural Preconditions

Differences From STRIPS

1. Cost Per Action

2. No Add/Delete Lists

3. Procedural Preconditions

4. Procedural Effects

Procedural Effects

class Action{

// [ -- , -- , -- ]WORLD_STATE m_Preconditions;WORLD_STATE m_Effects;

bool CheckPreconditions(); void ActivateAction();

};

Procedural Effects

void ActionFlee::ActivateAction()

{

// ...

}

Goto Animate

<x,y,z>

Squad Behaviors

Global Coordinator

Global Coordinator

Squad 1

Squad 2

Squad 3

Squad Behaviors

1. Simple

2. Complex

Squad Behaviors

1. Simple– Suppression fire– Goto positions– Follow leader

2. Complex– Flanking– Coordinated strikes– Retreats– Reinforcements

Simple Squad Behaviors

• Get-to-Cover

• Advance-Cover

• Orderly-Advance

• Search

Simple Squad Behaviors

1. Find participants.

2. Send orders.

3. Monitor progress.

4. Succeed or fail.

Get-to-Cover

1. Find participants.

2. Send orders.

3. Monitor progress.

4. Succeed or fail.

Player

Get-to-Cover

1. Find participants.

2. Send orders.

3. Monitor progress.

4. Succeed or fail.

Player

Get-to-Cover

1. Find participants.

2. Send orders.

3. Monitor progress.

4. Succeed or fail.

Player

Get-to-Cover

1. Find participants.

2. Send orders.

3. Monitor progress.

4. Succeed or fail.

Player

Get-to-Cover

1. Find participants.

2. Send orders.

3. Monitor progress.

4. Succeed or fail.

Player

Get-to-Cover

1. Find participants.

2. Send orders.

3. Monitor progress.

4. Succeed or fail.

Player

Get-to-Cover

1. Find participants.

2. Send orders.

3. Monitor progress.

4. Succeed or fail.

Player

Get-to-Cover

1. Find participants.

2. Send orders.

3. Monitor progress.

4. Succeed or fail.

Player

Squad Behaviors

1. Simple– Suppression fire– Goto positions– Follow leader

2. Complex– Flanking– Coordinated strikes– Retreats– Reinforcements

Complex Squad Behaviors

• There are none!!

Complex Squad Behaviors

Player

Complex Squad Behaviors

Player

Complex Squad Behaviors

Player

Squad Behavior Implementation

• Key is separation between squad layer and individual behaviors.

• Possibly formalize with Hierarchical Task Network (HTN) planning.

Squad Communication

“I need reinforcements!”

Squad Communication

Dialogue, instead of announcements: – Announcement: “Ouch!”– Dialogue:

• AI1: “What’s your status?”• AI2: “I’m hit!” (or “I’m alright!”)

Squad Communication

Dialogue, instead of announcements: – Announcement: “Where did he go?”– Dialogue:

• AI1: “Can you see anything?”• AI2: “No sir!” (or “He’s behind

the boxes!”)

Squad Communication

Dialogue also explains lack of action: – Dialogue:

• AI1: “He’s aiming at you!

Get out of there!”• AI2: “I’ve got nowhere to go!”

Squad Communication

“…Not only do they give each other orders, but they actually DO what they’re told!”

Squad Communication

“…Not only do they give each other orders, but they actually DO what they’re told!”

Reality:• Squad behavior has bird’s eye view of the

situation.• Find appropriate dialogue sequence after

the fact.

Here’s the Plan:

• STRIPS Planning Overview

• Planning in F.E.A.R.

• Differences from STRIPS

• Squad Behaviors & Communication

• Beyond F.E.A.R.

“… just like the marines in Half-Life 1”

“… just like the marines in Half-Life 1”

1998 2005

MIT Media Lab:Cognitive Machines Group

http://www.media.mit.edu/cogmac

Dialogue Integration

“What’s your status?”

Unified Planning for Actions and Speech

“Look out! Grenade!”Precondition:

Effect:

Unified Planning for Actions and Speech

OpenDoor -or- “Open the door!”Precondition:

door is closed

Effect:

door is open

Plan Recognition

UnlockDoor

CreateKey

LightFire GotoForge

GetGold

Plan Recognition

Plan as Context Free Grammar (CFG):

UnlockDoor CreateKey

LightFire GotoForge

GetGold

CreateKey

GotoForge :

:

:

Plan Recognition

PLAN RECOGNITION VIDEO

Questions?Jeff Orkin

jorkin@blarg.net

Monolith is hiring!•Engineers•Artists•Designers

http://www.lith.com/jobs