Upload
dung
View
17
Download
5
Embed Size (px)
DESCRIPTION
Artificial intelligent, influence maps, machine learning, game theory,
Citation preview
Lecture 17 – Influence Maps
CSci 4511 – Intro to AI3.21.11baylor
University of Minnesota
We begin our story with a bunch of problems…
Terrain Reasoning
• Old fashioned war• Strategies:
– Strengthen front– Attack flank
• Pincer– Ambush rear
Terrain Reasoning
• Old fashioned war• Strategies:
– Strengthen front– Attack flank
• Pincer– Ambush rear
• But where are those?
Problem 1. Finding Flanks
• Old fashioned war• Strategies:
– Strengthen front– Attack flank
• Pincer– Ambush rear
• But where are those?
Group units close to each other
Problem 1. Finding Flanks (contd.)
• Old fashioned war• Strategies:
– Strengthen front– Attack flank
• Pincer– Ambush rear
• But where are those?
Draw line through cores(so can tell who’s behind or side)
Problem 1. Finding Flanks (contd.)
• Old fashioned war• Strategies:
– Strengthen front– Attack flank
• Pincer– Ambush rear
• But where are those?
Perpendicular line is the FrontSides = Flanks, Behind = Rear
Flank
Flank
Rear
Fron
t
Problem 1. Finding Flanks (contd.)
• Old fashioned war• Strategies:
– Strengthen front– Attack flank
• Pincer– Ambush rear
• But where are those?
How do we make a computer understand that?
Flank
Flank
Rear
Fron
t
MOUT• Don’t get to play tanks and trenches much
anymore– Stupid nuclear bombs
• MOUT – Military Operation in Urban Terrain– Cities– Filled with hiding spots– And civilians
• And enemies who look like civilians• We care because of the CNN effect. And stuff
• Modern training is for individual combat, non-lethal combat, checkpoints, etc
Problem 2. City Defense
building,1 door,2 windows
wall,2m / 7ft
wall,1m / 3ft
deepwater
bridge w.1m / 3ft walls
stairs,from water
?Where do we put:• Snipers• Sentries• Sentry guns• Heavy weapons• Traps• Ambush
Image courtesy of William van der Sterren
Problem 2. City Defense
building,1 door,2 windows
wall,2m / 7ft
wall,1m / 3ft
deepwater
bridge w.1m / 3ft walls
stairs,from water
?
Sniperlong range view,enclosed space,
Checkpointchokepoint
Bad for Sniperlong range view but exposed
We can see this but how can a computer “see” it?
Terrain Reasoning
• These problems are instances of terrain reasoning
• Computers can’t “see”– Sees raw data, not groupings, relationships– Humans very good at this
• Understanding the space is important
Problem 3. Path Finding
Plenty of algorithms for finding paths…
Path Finding (contd.)
• Plenty of things should influence path• Some are semi-permanent
– Terrain type– Light (for games where you sneak around)– Sound (ditto)
• Some are dynamic– Enemies– Buildable/destroyable walls (RTSes, Tower Defense,
Red Faction)– Fog of War– Treasure– Mobile Targets (e.g., ball in soccer)
Problem 3. Path Finding
Plenty of algorithms for finding paths…But they don’t take into account things like risk
Building
• Several game types require building– Most obviously Real Time Strategies (RTSes)
• Factors– Place walls where they safeguard the most /
most important items– Place buildings near defense– Place buildings near resources
Placing farms near each other make them easier to defendwith walls, nearby troops
Problem 4. Building location
Building
• Several game types require building– Most obviously Real Time Strategies (RTSes)
• Factors– Place walls where they safeguard the most /
most important items– Place buildings near defense– Place buildings near resources
• Also, Area of Effect (AoE) attacks
Placing farms near each other make them easier to defendwith walls, nearby troops
…but also make themmore vulnerable toAoE attacks
Problem 5. Bomb placement
Building (contd.)
• Lots of games need to be able to place things in a good place– So they need to understand their environment
• Includes types that haven’t been made yet– Urban renewal games– Lemonade stands– Parties (e.g., Hello Kitty Party Pals)– Shopping games (PaperDoll Tower Defense)– Synergy games
• Clownfish & Anemones, Cinnabon & Weight Watchers
6. Ever-Popular Real Estate Game!
• We want to build a house
• Close to roads good• Elves are good, but
not too close• Don’t build near
haunted castles
Where should we build?
How Do We Solve These?
7. One More Example…
• Should we attack that army?
Us
Them
Lots more ofThem
Attenuated Threat Radius
• Assume each unit is worth a certain amount (danger/attack strength)
• Each has a “threat radius”– Represents reach of unit (if ranged)– Also spots unit can reach quickly
• Threat drops off with distance
Note fancy big words!
Red TankBase Value: 10Propagation Divisor: 2Propagation Distance: 3
7. One More Example…
• Add unit values to map, Sum values
Us
Them
Lots more ofThem
7. One More Example…
• Should we attack that army?
Us
Them
Lots more ofThem
One ofours
7. One More Example…
• Quantitative representation– Computer can
reason with this• Computationally
simple and fast– Just adds and
multiplies– Linear growth in
computation time
8. Reminder: Go is Hard
• 10761 games– vs. 1040 vs. 10120
• Champion: Not Yet• Brute Force does
not scale very well
for computers
Influence Maps
• Created in 1969 by Albert Zobrist– Computers didn’t “understand” spatial patterns
• This capability is important to human problem solving
– Looking to capture human-like “visual organization”• Divide the world into a grid• Each piece adds a number to the grid• Values “flow” through the grid
– Like oil/water (borders) or electric charge (intensity)• Add it all up and you understand the world
– Note: You still have to decide how to use the info
Multiple Maps• Can have more than one type of influence
– Represent different concerns / factors– Can use all of them at the same time
• Sum across influence types / maps• Can select relevant maps at run time
– Can be specific to goal, agent, state, personality, etc.• All started with Go
– Goals: Pieces grouped into armies, influence of armies, segmentation of board
– Maps: Borders, # empty squares in segment (safety), breathing spaces in chain, # stones in chain, # neighbors
Zobrist, A. L., "A model of visual organization for the game of GO," AFIPS Conf. Proc., 1969
Anything You Want
In GO• Influence• Concentration• Tension• Instability
For an RTS• Borders• Resources• Defensibility• Line of fire• Fog of war
Types of Influence Maps• Passability• Resources• Exploration• Threat• Light level• Historical• Emotion (fear, beauty, etc.)• Anything you make up• Desirability Map
– Summation of the above
Light Map, Action Quake 2
Propagation• Can propagate however you want
– Predefined actor halo, rules– Auto-propagation - linear, exponential, Gaussian, etc.
• Can have different “conductance” for each tile– Walls or mountains could block– Roads or supply lines might spread influence more
Roadsspreadinfluence
Lakesblock it(for tanks)
Problems Revisited
6. Ever-Popular Real Estate Game!
Elf
Evil Castle
Road that looks like water
Place A Grid Over The Map
Convert Map Into Something Easy For Us To Reason With
O O O O OO O
O O O
O O O OO
R R RR
RR
RE
XO
Convert Grid To Numbers
1 1 1
1
1 1
1
0 0 0 0 0
00
0000
0
00
00
0 0
Normal land is worth 0Road is worth 1
Live close to elves, but not too close
1
-1-1
-1
-1-1
1100
1
1
1
11
00
0 0
00
100
Stay away from evil castles
-2-2
-2
-2 -2
-2
-2
-2
-1 -1 -1 -1
-1
-1
-1
0 0 0 0 0
0
0
0
0
Each Object Affects The Area Around It
-2-2
-2
-2 -2
-2
-2
-2
-1 -1 -1 -1
-1
-1
-1
0 0 0 0 0
0
0
0
0
Squares next to castle worth -2
Squares next to those squares worth -1
Add The Values Together
0 0
0 0
0 0
0
1 1 1
1 -2 -1
-2 -2 -1 -1
-2
-2 -2 -1 2
-2
We Build At The Highest Number0 0
0 0
0 0
0
1 1 1
1 -2 -1
-2 -2 -1 -1
-2
-2 -2 -1 2
-2Right here
We Build At The Highest Number
Right here
Placing farms near each other make them easier to defendwith walls, nearby troops
…but also make themmore vulnerable toAoE attacks
Problem 5. Bomb placementProblem 4. Building location
Shortest distance doesn’t use cover, good way to get shot
A* runs a lot faster when you do it this way
Problem 3. Path Finding
Waypoints are marked as being visible to the enemy, factors into the node cost
A lot more work for A*
Note: Color, like danger, fades with distance
Problem 1. Finding Flanks
• Old fashioned war• Strategies:
– Strengthen front– Attack flank
• Pincer– Ambush rear
How a human sees the situation
Flank
Flank
Rear
Fron
t
0 = border / no man’s land
Ferretman, AI Game Programming Wisdom volume 1
Problem 2. City Defense
building,1 door,2 windows
wall,2m / 7ft
wall,1m / 3ft
deepwater
bridge w.1m / 3ft walls
stairs,from water
?
Sniperlong range view,enclosed space,
Checkpointchokepoint
Bad for Sniperlong range view but exposed
We can see this but how can a computer “see” it?
What Makes A Good Sniping Spot?• Inconspicuous• Overlooking distant locations• Overlooking key locations• Target locations lack cover• Hard to reach from target locations• Nearby cover for sniper• Hard to surprise• Flank protected• Have freedom of movement• “Focus”
Data We Need
• Line of sight• Travel time from one point to another• Distance from one point to another• Which part of area do we belong to
– Are we in a specific room, courtyard, etc.
Waypoint relations• (potential) “interaction” or influence
between two waypoints• directional, leveled [0..1]
Examples• nearby cover from threat• hard to reach from target• overlooking key locations• hard to surprise
Case: Tactical Assessment
0
0.5
1
1.5
0 100 200
Relation: nearby cover from threat• the amount of nearby cover
from threat and nearby positions
Case: Tactical Assessment
0.0
0.5
1.0
weapon performance
protection
2/9 = 0.22
Case: Tactical Assessment
nearbycoverfromseen threatseast ofposition
Focus• concentration of relations in certain
direction (sphere sector)–If long LoS in one direction but
not another, focus is high–If long LoS in all directions, no focus
• Useful for determining flank protection
Case: Tactical Assessment
No focus
Partial focus (blocked 3 directions, open 3 directions)
Case: Tactical Assessment
focuseast(E,NE,SE, Eup, Edown)
Case: Tactical Assessment
localproperty:light level
Terrain Characteristics Used
• William’s formula for CGF for AQ2– 17 ingredients– Checks in 26 directions
• Specifics for this example (map)– 480 waypoints– 10 seconds to calculate on a PIII/500
• Done off-line– 24KB of sniping ratings
Technical Notes (so far)
Making Your Own Formula
• How William created the CGF for AQ2 formula– 10% Analysis – 40% Experimenting– 50% Tuning
• Store Sniping behavior at each waypoint– Time spent sniping
• Time spent at location• Time spent zoomed in• Shots fired
– Damage done while sniping– Damage taken while sniping
• Breakdown by– Sniping spot– Weapon used– Direction faced
Adding History
Case: Tactical Assessment
positivesnipingexperience,after40 rounds,non-directional
Case: Tactical Assessment
snipingeast,usinggeometry+ physics+ learning
Case: Tactical Assessment
trafficdata (all teams)after40 rounds
Case: Tactical Assessment
snipingeast,usinggeometry+ physics+ learning+ tactical
game playanalysis
waypoints
line of sight / fire
level design terrain analysis game round analyse activity + rerank locationsgame round
game round
paths + travel time
sniper spotanalysis [~20 s]
sniperactivity
snipingratings
sniping activitytracking [ 0.0s]
reinforcementlearning [0.1s]
traffic +kills ...
sniperactivityphysics
Results
Case: Tactical Assessment
Bonus Problems!
9. Track and Predict
• Target runs behind something and now we can’t see him
• Want to predict where he’s going to pop out
9. Track and Predict
house
window
AI actor
threat
door
Situation (top view)
Slides stolen directly from William
?
?
9. Track and Predict
Where to look for the threat? What to aim for?
!?
9. Track and Predict
Option: assume threat static after moving out of sight(often incorrect)
!?
9. Track and Predict
Option: assume threat continues last observed movement(incorrect after a little while)
The waypoint grid(overlay)
9. Track and Predict
The waypoint grid(AI “view”)
9. Track and Predict
Extrapolate movement along waypoints(valid paths, movement speed known).
Threat no longer “leaves” the house!
9. Track and Predict
Use known line-of-sight for “negative”observations.Exclude some likely threat movements.
9. Track and Predict
9. Track and Predict• How can we use the waypoint graph?
– Restrict guesses to valid movements– Estimate the threats’ movement speed– Exclude visible potential movements
that were not observed
• Predict a likely path– Path best aligned with last observed direction– Using per waypoint traffic statistics– Towards most attractive nearby objective– Towards strongest defensive position
• WWiD (unrelated to terrain reasoning)– Could ask “what would i do in their situation?”
9. Track and Predict
Where to aim?• Where you think the threat will likely reappear
Good guesses• Visible spot closest to threat• First visible spot on predicted path• If you think he’s coming back for you…
– First visible spot on shortest path back to you• Any nearby portal (door, window, hole, ...)
1
2
3
Threat turns to attack (1) or moves to nearest attack position (2), or continues movement (3)
9. Track and Predict
10. Grenade Handling
• Can’t (or don’t want to) shoot at enemies? Throw grenades!
• Best part – grenades bounce!• Humans good at this, AI bad• You do not want to compute this at run-time
– If you’re going to rebound, there are sooomany places you can throw
– Physics are imprecise so you gotta Monaco it• Precompute, store results in terrain map
– Mark the good spots in your map
Information to track to determine value of spot as a grenade tossing spot
AI is much easier with good debugging tools
Have no direct shot at enemy. But we could throw a grenade from the green spot
We know we can hit this because we pre-computed it long before the game began
Hey, how’d he do that?
Grenades: A Warning
• To throw grenades, need to know– Where to stand– Point to aim at
• Grenades and other bouncy things require you to be very precise– Clean ray trace is not good enough
• If you are just a little off, very bad things can happen
It’s very important to be very precise. Or very bad things could happen
good
bad
oops, we clipped the wall
11. Team Maneuvers
• Choosing an avenue of approach• Bounding overwatch
– One person moves, the other covers him• Keeping clear lines of fire• Keeping to cover• Flanking
11. Team Maneuver
how toattack thethreats asa team?
1
23
11. Team Maneuver
Approach• Initially, sequentially for each member
– evaluate nearby waypoints (within r ) for position improvement
– claim best position p– move up
to position p• Repeat on
memberinitiative
Things To Evaluate For Each Waypoint
• Travel time to / from threat, power-up, ...• Nearby cover and flank protection• Tactical rating of position• Min projected distance to team member
– Spread out a bit• Keep out of friendly line-of-fire• Maintain visual contact with enemy
11. Team Maneuver
how toattack thethreats asa team?
1
23
11. Team Maneuver
evaluate foreach waypoint:-distance to
threat-spread-nearby cover-not claimed-etc.
and claimbest position…
1
23
11. Team Maneuver
member1claims positionand will move up
1
23
11. Team Maneuver
member2evaluatesand picksbest positiongivenexisting andclaimedpositions
12
3
11. Team Maneuver
same formember3
however, inthe search radius, thebest spotmight not bethat good(LOF blocked)…
12
3
11. Team Maneuver
upon arrival,member3indeed seesa blockedline-of-fire
he quicklyexecutesthe sameprocedurefrom hisnew spot…
!#%&!12
3
11. Team Maneuver
now member3moves upagain toa good position…
in an obstaclerich environment,you’ll get flankingfor free…
12
3
Questions?