CSCE 590E Spring 2007

AI

By Jijun Tang

Announcements

April 16th/18th: demos Show progress/difficulties/change of

plans USC Times will have reporters in the

class High school outreach

Anyone can contact their high school admin to arrange direct talks to students

Of course, in SC only

Homework

P 655 questions 1 and 2 5 points total Due April 16th

Motion Extraction

Moving the Game Instance Linear Motion Extraction Composite Motion Extraction Variable Delta Extraction The Synthetic Root Bone Animation Without Rendering

Moving the Game Instance

Game instance is where the game thinks the object (character) is

Usually just pos, orientation and bounding box

Used for everything except rendering Collision detection Movement It’s what the game is!

Must move according to animations

Linear Motion Extraction

Find position on last frame of animation Subtract position on first frame of animation Divide by duration Subtract this motion from animation frames During animation playback, add this delta

velocity to instance position Animation is preserved and instance moves Do same for orientation

Composite Motion Extraction

Approximates motion with circular arc Pre-processing algorithm finds:

Axis of rotation (vector) Speed of rotation (radians/sec) Linear speed along arc (metres/sec) Speed along axis of rotation (metres/sec)

e.g. walking up a spiral staircase

Variable Delta Extraction

Uses root bone motion directly Sample root bone motion each frame Find delta from last frame Apply to instance pos+orn Root bone is ignored when rendering

Instance pos+orn is the root bone

The Synthetic Root Bone

All three methods use the root bone But what is the root bone? Where the character “thinks” they are

Defined by animators and coders Does not match any physical bone

Can be animated completely independently Therefore, “synthetic root bone” or SRB

Animation Without Rendering

Not all objects in the world are visible But all must move according to anims Make sure motion extraction and replay

is independent of rendering Must run on all objects at all times

Needs to be cheap! Use LME & CME when possible VDA when needed for complex animations

Mesh Deformation

Find Bones in World Space Find Delta from Rest Pose Deform Vertex Positions Deform Vertex Normals

Example

Find Delta from Rest Pose

Mesh is created in a pose Often the “da Vinci man” pose for humans Called the “rest pose”

Must un-transform by that pose first Then transform by new pose

Multiply new pose transforms by inverse of rest pose transforms

Inverse of rest pose calculated at mesh load time

Gives “delta” transform for each bone

Deform Vertex Positions

Each vertex has several bones affect it (the number is generally set to <=4). Vertices each have n bones n is usually 4

4 bone indices 4 bone weights 0-1 Weights must sum to 1

Deformation usually performed on GPU Delta transforms fed to GPU

Usually stored in “constant” space

Deform Vertex Normals

Normals are important for shading and are done similarly to positions

When transformed, normals must be transformed by the inverse transpose of the transform matrix Translations are ignored For pure rotations, inverse(A)=transpose(A) So inverse(transpose(A)) = A For scale or shear, they are different

Normals can use fewer bones per vertex Just one or two is common

Inverse Kinematics

FK & IK Single Bone IK Multi-Bone IK Cyclic Coordinate Descent Two-Bone IK IK by Interpolation

Single Bone IK

Orient a bone in given direction Eyeballs Cameras

Find desired aim vector Find current aim vector Find rotation from one to the other

Cross-product gives axis Dot-product gives angle

Transform object by that rotation

Multi-Bone IK

One bone must get to a target position Bone is called the “end effector”

Can move some or all of its parents May be told which it should move first

Move elbow before moving shoulders May be given joint constraints

Cannot bend elbow backwards

Two-Bone IK

Direct method, not iterative Always finds correct solution

If one exists Allows simple constraints

Knees, elbows Restricted to two rigid bones with a rotation

joint between them Knees, elbows!

Can be used in a cyclic coordinate descent

IK by Interpolation

Animator supplies multiple poses Each pose has a reference direction

e.g. direction of aim of gun Game has a direction to aim in Blend poses together to achieve it Source poses can be realistic

As long as interpolation makes sense Result looks far better than algorithmic IK with

simple joint limits

Network and Multiplayer

Multiplayer Modes:Event Timing

Turn-Based Easy to implement Any connection type

Real-Time Difficult to implement Latency sensitive

Protocol Stack: Open System Interconnect

R outer

Sender R eceiver

Ne two rk

D a ta L in k

Physica l

Ne two rk

D a ta L in k

Physica l

App lica tio n

Pre se n ta tio n

Session

T ra nspo rt

Ne two rk

D a ta L in k

Physica l

App lica tio n

Pre se n ta tio n

Session

T ra nspo rt

Ne two rk

D a ta L in k

Physica l

G am e E vents

G am e P ac k et iz at ion

Connec t ion & Data E x c hange

Input UpdatesS tate Updates

S eria liz at ionB uffering

S oc k ets

TCPUDP

IP

E thernet (M A C)

W ired (C5, Cable)F iber O pt ic sW ire les s

Real-Time Communications:Peer to Peer vs. Client/Server

Broadcast Peer/Peer Client/Server

Connections 0Client = 1Server = N

N = Number of players

Broadcast Peer/Peer Client/Server

Send 1 N-1Client = 1 Server = N

Receive N-1 N-1Client = 1Server = N

1

1

N

x

x

P 1 P 2

P 1

P 2 P 3

P 1 P 4

P 2 P 3

P 1

P 2 P 5

P 3 P 4

2 p lay er s1 c o n n ec tio n

3 p lay er s3 c o n n ec tio n s

4 p lay er s6 c o n n ec tio n s

5 p lay er s1 0 c o n n ec tio n s

Security:Encryption Methods

Keyed Public Key Private Key Ciphers

Message Digest Certificates IPSec

Security:Copy Protection

Disk Copy Protection Costly Mastering, delay copies to ensure

first several months’ sale Invalid/Special Sector Read

Code Sheets Ask code from a line in a large manual

Watermarking

Privacy

Critical data should be kept secret and strong encrypted: Real name Password Address/phone/email Billing Age (especially for minors)

Using public key for transforming user name and password

Artificial Intelligence:Agents, Architecture, and Techniques

Book Material

The book CD has a lot of material in the chapter content

A state machine language for example Please try it

Artificial Intelligence

Intelligence embodied in a man-made device

Human level AI still unobtainable The difficulty is comprehension

Game Artificial Intelligence:What is considered Game AI?

Is it any NPC (non-player character) behavior? A single “if” statement? Scripted behavior?

Pathfinding? Animation selection? Automatically generated environment? Best shot at a definition of game AI?

Possible Game AIDefinition

Inclusive view of game AI:

“Game AI is anything that contributes to the perceived intelligence of an entity, regardless of what’s under the hood.”

Goals of anAI Game Programmer

Different than academic or defense industry

1. AI must be intelligent, yet purposely flawed2. AI must have no unintended weaknesses3. AI must perform within the constraints4. AI must be configurable by game designers

or players5. AI must not keep the game from shipping

Specialization ofGame AI Developer

No one-size fits all solution to game AI Results in dramatic specialization

Strategy Games Battlefield analysis Long term planning and strategy

First-Person Shooter Games One-on-one tactical analysis Intelligent movement at footstep level

Real-Time Strategy games the most demanding, with as many as three full-time AI game programmers

Game Agents

May act as an Opponent Ally Neutral character

Continually loops through the

Sense-Think-Act cycle Optional learning or remembering step

Sense-Think-Act Cycle:Sensing

Agent can have access to perfect information of the game world May be expensive/difficult to tease out

useful info Game World Information

Complete terrain layout Location and state of every game object Location and state of player

But isn’t this cheating???

Sensing:Enforcing Limitations

Human limitations? Limitations such as

Not knowing about unexplored areas Not seeing through walls Not knowing location or state of player

Can only know about things seen, heard, or told about

Must create a sensing model

Sensing:Human Vision Model for Agents

Get a list of all objects or agents; for each:1. Is it within the viewing distance of the agent?

How far can the agent see? What does the code look like?

2. Is it within the viewing angle of the agent? What is the agent’s viewing angle? What does the code look like?

3. Is it unobscured by the environment? Most expensive test, so it is purposely last What does the code look like?

Sensing:Vision Model

Isn’t vision more than just detecting the existence of objects?

What about recognizing interesting terrain features? What would be interesting to an agent?

Sensing:Human Hearing Model

Humans can hear sounds Can recognize sounds

Knows what emits each sound Can sense volume

Indicates distance of sound Can sense pitch

Sounds muffled through walls have more bass Can sense location

Where sound is coming from

Sensing:Modeling Hearing

How do you model hearing efficiently? Do you model how sounds reflect off

every surface? How should an agent know about

sounds?

Sensing:Modeling Hearing Efficiently

Event-based approach When sound is emitted, it alerts

interested agents Use distance and zones to determine

how far sound can travel

Sensing:Communication

Agents might talk amongst themselves! Guards might alert other guards Agents witness player location and spread

the word Model sensed knowledge through

communication Event-driven when agents within vicinity of

each other

Sensing:Reaction Times

Agents shouldn’t see, hear, communicate instantaneously

Players notice! Build in artificial reaction times

Vision: ¼ to ½ second Hearing: ¼ to ½ second Communication: > 2 seconds

Sense-Think-Act Cycle: Thinking

Sensed information gathered Must process sensed information Two primary methods

Process using pre-coded expert knowledge

Use search to find an optimal solution

Thinking:Expert Knowledge

Many different systems Finite-state machines Production systems Decision trees Logical inference

Encoding expert knowledge is appealing because it’s relatively easy Can ask just the right questions As simple as if-then statements

Problems with expert knowledge Not very scalable

Finite-state machine (FSM)

Production systems

Consists primarily of a set of rules about behavior

Productions consist of two parts: a sensory precondition (or "IF" statement) and an action (or "THEN")

A production system also contains a database about current state and knowledge, as well as a rule interpreter

Decision trees

Logical inference

Process of derive a conclusion solely based on what one already knows

Prolog (programming in logic)

mortal(X) :- man(X). man(socrates).

?- mortal(socrates). Yes

Thinking:Search

Employs search algorithm to find an optimal or near-optimal solution Branch-and-bound Depth-first Breadth-first

A* pathfinding common use of search Kind of mixed

Depth and breadth-first

Thinking:Machine Learning

If imparting expert knowledge and search are both not reasonable/possible, then machine learning might work

Examples: Reinforcement learning Neural networks Decision tree learning

Not often used by game developers Why?

Thinking:Flip-Flopping Decisions

Must prevent flip-flopping of decisions Reaction times might help keep it from

happening every frame Must make a decision and stick with it

Until situation changes enough Until enough time has passed

Sense-Think-Act Cycle:Acting

Sensing and thinking steps invisible to player

Acting is how player witnesses intelligence Numerous agent actions, for example:

Change locations Pick up object Play animation Play sound effect Converse with player Fire weapon

Acting:Showing Intelligence

Adeptness and subtlety of actions impact perceived level of intelligence

Enormous burden on asset generation Agent can only express intelligence in terms

of vocabulary of actions Current games have huge sets of

animations/assets Must use scalable solutions to make selections

Extra Step in Cycle:Learning and Remembering

Optional 4th step Not necessary in many games

Agents don’t live long enough Game design might not desire it

Learning

Remembering outcomes and generalizing to future situations

Simplest approach: gather statistics If 80% of time player attacks from left Then expect this likely event

Adapts to player behavior

Remembering

Remember hard facts Observed states, objects, or players

For example Where was the player last seen? What weapon did the player have? Where did I last see a health pack?

Memories should fade Helps keep memory requirements lower Simulates poor, imprecise, selective human

memory

Rememberingwithin the World

All memory doesn’t need to be stored in the agent – can be stored in the world

For example: Agents get slaughtered in a certain area Area might begin to “smell of death” Agent’s path planning will avoid the area Simulates group memory

Making Agents Stupid

Sometimes very easy to trounce player Make agents faster, stronger, more accurate

Sometimes necessary to dumb down agents, for example: Make shooting less accurate Make longer reaction times Engage player only one at a time Change locations to make self more vulnerable

Agent Cheating

Players don’t like agent cheating When agent given unfair advantage in speed,

strength, or knowledge Sometimes necessary

For highest difficultly levels For CPU computation reasons For development time reasons

Don’t let the player catch you cheating! Consider letting the player know upfront

Finite-State Machine (FSM)

Abstract model of computation Formally:

Set of states A starting state An input vocabulary A transition function that maps inputs and

the current state to a next state

FSM

In Game Development

Deviate from formal definition1. States define behaviors (containing code)

Wander, Attack, Flee

2. Transition function divided among states Keeps relation clear

3. Blur between Moore (within state) and Mealy machines (transitions)

4. Leverage randomness

5. Extra state information, for example, health

Good and Bad

Most common game AI software pattern Natural correspondence between states and

behaviors Easy to diagram Easy to program Easy to debug Completely general to any problem

Problems Explosion of states Often created with ad hoc structure

Finite-State Machine:UML Diagram

W an d er Attac k

F lee

S ee E n em y

L ow HealthN o E n em

y

N o E n em y

Approaches

Three approaches Hardcoded (switch statement) Scripted Hybrid Approach

Hardcoded FSM

void RunLogic( int * state ) { switch( state ) { case 0: //Wander Wander(); if( SeeEnemy() ) { *state = 1; } break; case 1: //Attack Attack(); if( LowOnHealth() ) { *state = 2; } if( NoEnemy() ) { *state = 0; } break;

case 2: //Flee Flee(); if( NoEnemy() ) { *state = 0; } break; }}

Problems with switch FSM

1. Code is ad hoc Language doesn’t enforce structure

2. Transitions result from polling Inefficient – event-driven sometimes better

3. Can’t determine 1st time state is entered4. Can’t be edited or specified by game

designers or players

Scripted with alternative language

AgentFSM{ State( STATE_Wander ) OnUpdate Execute( Wander ) if( SeeEnemy ) SetState( STATE_Attack ) OnEvent( AttackedByEnemy ) SetState( Attack ) State( STATE_Attack ) OnEnter Execute( PrepareWeapon ) OnUpdate Execute( Attack ) if( LowOnHealth ) SetState( STATE_Flee ) if( NoEnemy ) SetState( STATE_Wander ) OnExit Execute( StoreWeapon ) State( STATE_Flee ) OnUpdate Execute( Flee ) if( NoEnemy ) SetState( STATE_Wander )}

Scripting Advantages

1. Structure enforced

2. Events can be handed as well as polling

3. OnEnter and OnExit concept exists

4. Can be authored by game designers Easier learning curve than straight C/C++

Scripting Disadvantages

Not trivial to implement Several months of development

Custom compiler With good compile-time error feedback

Bytecode interpreter With good debugging hooks and support

Scripting languages often disliked by users Can never approach polish and robustness of co

mmercial compilers/debuggers

Hybrid Approach

Use a class and C-style macros to approximate a scripting language

Allows FSM to be written completely in C++ leveraging existing compiler/debugger

Capture important features/extensions OnEnter, OnExit Timers Handle events Consistent regulated structure Ability to log history Modular, flexible, stack-based Multiple FSMs, Concurrent FSMs

Can’t be edited by designers or players

Extensions

Many possible extensions to basic FSM OnEnter, OnExit Timers Global state, substates Stack-Based (states or entire FSMs) Multiple concurrent FSMs Messaging

Common Game AI Techniques

A* Pathfinding Command Hierarchy Dead Reckoning Emergent Behavior Flocking Formations Influence Mapping …

A* Pathfinding

Directed search algorithm used for finding an optimal path through the game world

Used knowledge about the destination to direct the search

A* is regarded as the best Guaranteed to find a path if one exists Will find the optimal path Very efficient and fast

Command Hierarchy

Strategy for dealing with decisions at different levels From the general down to the foot soldier

Modeled after military hierarchies General directs high-level strategy Foot soldier concentrates on combat

US Military Chain of Command

Dead Reckoning

Method for predicting object’s future position based on current position, velocity and acceleration

Works well since movement is generally close to a straight line over short time periods

Can also give guidance to how far object could have moved

Example: shooting game to estimate the leading distance

Emergent Behavior

Behavior that wasn’t explicitly programmed

Emerges from the interaction of simpler behaviors or rules Rules: seek food, avoid walls Can result in unanticipated individual or

group behavior

Flocking

Example of emergent behavior Simulates flocking birds, schooling fish

Developed by Craig Reynolds 1987 SIGGRAPH paper

Three classic rules1. Separation – avoid local flockmates2. Alignment – steer toward average heading3. Cohesion – steer toward average position

Formations

Group movement technique Mimics military formations

Similar to flocking, but actually distinct Each unit guided toward formation

position Flocking doesn’t dictate goal positions

Flocking/Formation

Influence Mapping

Method for viewing/abstracting distribution of power within game world

Typically 2D grid superimposed on land Unit influence is summed into each grid cell

Unit influences neighboring cells with falloff Facilitates decisions

Can identify the “front” of the battle Can identify unguarded areas Plan attacks Sim-city: influence of police around the city

Mapping Example

Level-of-Detail AI

Optimization technique like graphical LOD Only perform AI computations if player will

notice For example

Only compute detailed paths for visible agents Off-screen agents don’t think as often

Manager Task Assignment

Manager organizes cooperation between agents Manager may be invisible in game Avoids complicated negotiation and

communication between agents Manager identifies important tasks and

assigns them to agents For example, a coach in an AI football team

Obstacle Avoidance

Paths generated from pathfinding algorithm consider only static terrain, not moving obstacles

Given a path, agent must still avoid moving obstacles Requires trajectory prediction Requires various steering behaviors

Scripting

Scripting specifies game data or logic outside of the game’s source language

Scripting influence spectrumLevel 0: Everything hardcoded

Level 1: Data in files specify stats/locations

Level 2: Scripted cut-scenes (non-interactive)

Level 3: Lightweight logic, like trigger system

Level 4: Heavy logic in scripts

Level 5: Everything coded in scripts

Scripting Pros and Cons

Pros Scripts changed without recompiling game Designers empowered Players can tinker with scripts

Cons More difficult to debug Nonprogrammers required to program Time commitment for tools

State Machine

Most common game AI software pattern Set of states and transitions, with only one

state active at a time Easy to program, debug, understand

Stack-Based State Machine

Also referred to as push-down automata

Remembers past states Allows for diversions, later returning to

previous behaviors

Subsumption Architecture

Popularized by the work of Rodney Brooks Separates behaviors into concurrently running

finite-state machines Well suited for character-based games where

moving and sensing co-exist Lower layers

Rudimentary behaviors (like obstacle avoidance) Higher layers

Goal determination and goal seeking Lower layers have priority

System stays robust

Terrain Analysis

Analyzes world terrain to identify strategic locations

Identify Resources Choke points Ambush points Sniper points Cover points

Trigger System

Highly specialized scripting system Uses if/then rules

If condition, then response Simple for designers/players to

understand and create More robust than general scripting Tool development simpler than general

scripting

Promising AI Techniques

Show potential for future Generally not used for games

May not be well known May be hard to understand May have limited use May require too much development time May require too many resources

Bayesian Networks

Performs humanlike reasoning when faced with uncertainty

Potential for modeling what an AI should know about the player Alternative to cheating

RTS Example AI can infer existence or nonexistence of

player build units

Example

Blackboard Architecture

Complex problem is posted on a shared communication space Agents propose solutions Solutions scored and selected Continues until problem is solved

Alternatively, use concept to facilitate communication and cooperation

Decision Tree Learning

Constructs a decision tree based on observed measurements from game world

Best known game use: Black & White Creature would learn and form “opinions” Learned what to eat in the world based

on feedback from the player and world

Filtered Randomness

Filters randomness so that it appears random to players over short term

Removes undesirable events Like coin coming up heads 8 times in a row

Statistical randomness is largely preserved without gross peculiarities

Example: In an FPS, opponents should randomly spawn

from different locations (and never spawn from the same location more than 2 times in a row).

Genetic Algorithms

Technique for search and optimization that uses evolutionary principles

Good at finding a solution in complex or poorly understood search spaces

Typically done offline before game ships Example:

Game may have many settings for the AI, but interaction between settings makes it hard to find an optimal combination

Flowchat

N-Gram Statistical Prediction

Technique to predict next value in a sequence

In the sequence 18181810181, it would predict 8 as being the next value

Example In street fighting game, player just did

Low Kick followed by Low Punch Predict their next move and expect it

Neural Networks

Complex non-linear functions that relate one or more inputs to an output

Must be trained with numerous examples Training is computationally expensive making th

em unsuited for in-game learning Training can take place before game ships

Once fixed, extremely cheap to compute

Example

Planning

Planning is a search to find a series of actions that change the current world state into a desired world state

Increasingly desirable as game worlds become more rich and complex

Requires Good planning algorithm Good world representation Appropriate set of actions

Player Modeling

Build a profile of the player’s behavior Continuously refine during gameplay Accumulate statistics and events

Player model then used to adapt the AI Make the game easier: player is not good at

handling some weapons, then avoid Make the game harder: player is not good at

handling some weapons, exploit this weakness

Production (Expert) Systems

Formal rule-based system Database of rules Database of facts Inference engine to decide which rules trigger –

resolves conflicts between rules Example

Soar used experiment with Quake 2 bots Upwards of 800 rules for competent opponent

Reinforcement Learning

Machine learning technique Discovers solutions through trial and erro

r Must reward and punish at appropriate ti

mes Can solve difficult or complex problems li

ke physical control problems Useful when AI’s effects are uncertain

or delayed

Reputation System

Models player’s reputation within the game world

Agents learn new facts by watching player or from gossip from other agents

Based on what an agent knows Might be friendly toward player Might be hostile toward player

Affords new gameplay opportunities “Play nice OR make sure there are no

witnesses”

Smart Terrain

Put intelligence into inanimate objects Agent asks object how to use it: how to

open the door, how to set clock, etc Agents can use objects for which they

weren’t originally programmed for Allows for expansion packs or user created

objects, like in The Sims Enlightened by Affordance Theory

Objects by their very design afford a very specific type of interaction

Speech Recognition

Players can speak into microphone to control some aspect of gameplay

Limited recognition means only simple commands possible

Problems with different accents, different genders, different ages (child vs adult)

Text-to-Speech

Turns ordinary text into synthesized speech Cheaper than hiring voice actors Quality of speech is still a problem

Not particularly natural sounding Intonation problems Algorithms not good at “voice acting”: the mouth

needs to be animated based on the text Large disc capacities make recording human

voices not that big a problem No need to resort to worse sounding solution

Promising AI Techniques:Weakness Modification Learning

General strategy to keep the AI from losing to the player in the same way every time

Two main steps1. Record a key gameplay state that precedes a

failure

2. Recognize that state in the future and change something about the AI behavior AI might not win more often or act more intelligently,

but won’t lose in the same way every time Keeps “history from repeating itself”