Embed Size (px)
DESCRIPTION
CS510 AI and Games. Final Report on Dec. 09 2008 Juncao Li. Agenda. About Advanced Protection My Design and Work Results and Evaluations Conclusion and Discussion. Advanced Protection (AP). Author: Soren Johnson Turn-based strategy game Player makes strategy each turn - PowerPoint PPT Presentation
Citation preview
CS510 AI and Games
Final Report on Dec. 09 2008
Juncao Li
Computer Science, Portland State University 2
Agenda
• About Advanced Protection• My Design and Work• Results and Evaluations• Conclusion and Discussion
Computer Science, Portland State University 3
Advanced Protection (AP)
• Author: Soren Johnson• Turn-based strategy game
– Player makes strategy each turn– AI plays against player’s strategy
• Static AI V.S. adaptive AI – Player learns the strategy of static AIs– Adaptive AI adapts based on users performance
Computer Science, Portland State University 5
AP (Cont.)Figure A-1: Terrain Types
Terrain Color Farming Bonus Defense Bonus Impassable?Water
B l u e 0 0 Y
W e t l a n d s L i g h t G r e e n 2 0 N
F o r e s t D a r k G r e e n 1 0 N
H i l l s B r o w n 0 1 N
M o u n t a i n s G r a y - 1 2 N
Figure A-2: Human Units
Unit Cost SalvageAttack
StrengthDefense Strength
Farming Rate
Special Ability
D r o n e $ 2 0 $ 2 0 0 1 1 -
M i n e $ 5 0 $ 2 5 0 0 0 ( 1 )
F a r m e r $ 1 0 0 $ 8 0 0 1 1 -
I n f a n t r y $ 1 0 0 $ 9 0 3 6 0 -
S e t t l e r $ 1 5 0 $ 1 2 0 2 4 3 -
J a m m e r $ 1 5 0 $ 9 0 0 2 0 ( 2 )
A r m o r $ 2 0 0 $ 1 5 0 8 1 0 0 -
A r t i l l e r y $ 4 0 0 $ 3 0 0 3 2 0 ( 3 )
Computer Science, Portland State University 6
AP (Cont.)Figure A-3: Chaos's Minions
Unit CostAttack
StrengthDefense Strength
Movement Rate
Hearing Radius
Special Ability
Scout$ 1 0 0 1 1 - ( 4 )
S c a v e n g e r $ 1 0 0 0 1 1 - ( 5 )
C o m m o n B a r b a r i a n $ 5 0 6 3 1 2 -
A m p h i b i o u s B a r b a r i a n $ 6 0 6 3 1 2 ( 6 )
M o b i l e B a r b a r i a n $ 7 0 5 3 2 3 -
J u m p B a r b a r i a n $ 7 0 6 2 1 2 ( 7 )
K a m i k a z e e B a r b a r i a n $ 7 0 0 1 0 2 2 ( 8 )
A r m o r e d B a r b a r i a n $ 2 0 0 1 5 1 0 1 3 ( 9 )
Computer Science, Portland State University 7
AP (Cont.)
• Features– Each minion has a brain (automaton) encoded by
a 128-bit string– Four behaviors depending on the input:
• Move forward, turn right, turn left and do actions
– 250 brains for a minion to choose• 20 hardcoded, 230 from genetic algorithm• During the play, brains are rated based on their
performance against the player
– Dynamically choose best-fit brains• Each player has their own easy and hard minion brains • Players does not loose or win too much
Computer Science, Portland State University 8
Minion InputsFigure 3: Scout Inputs
Input Meaning000
N o i n p u t
0 0 1 F a c i n g i m p a s s a b l e t e r r a i n
0 1 0 S e n s e s j a m m i n g r a d i u s
0 1 1 S e n s e s a r t i l l e r y r a d i u s
1 0 0 F a c i n g d r o n e o r f a r m e r
1 0 1 F a c i n g i n f a n t r y , s e t t l e r , j a m m e r , o r a r t i l l e r y
1 1 0 F a c i n g a r m o r
1 1 1 F a c i n g m i n e
Figure 4: Scavenger Inputs
Input Meaning000
F a c i n g h u m a n
0 0 1 F a c i n g i m p a s s a b l e t e r r a i n
0 1 0 S e n s e g o l d
0 1 1 F a c i n g g o l d
1 0 0 N o f a r m i n g b o n u s
1 0 1 + 1 f a r m i n g b o n u s
1 1 0 + 2 f a r m i n g b o n u s
1 1 1 + 3 f a r m i n g b o n u s
Figure 5: Barbarian Inputs
Input Meaning000
N o i n p u t
0 0 1 F a c i n g i m p a s s a b l e t e r r a i n
0 1 0 S e n s e s s c o u t b r o a d c a s t
0 1 1 F a c i n g s c o u t b r o a d c a s t
1 0 0 F a c i n g d r o n e o r f a r m e r
1 0 1 F a c i n g i n f a n t r y o r s e t t l e r
1 1 0 F a c i n g a r m o r
1 1 1 F a c i n g j a m m e r o r a r t i l l e r y
Computer Science, Portland State University 12
Agenda
• About Advanced Protection• My Design and Work• Results and Evaluations• Conclusion and Discussion
Computer Science, Portland State University 13
My Goal & Why
• Design AIs that play as players– Try to learn the player’s strategy– Try to perform best on all 250 brains
• Because Chaos selects brains based on player’s strategy• Win (defined later) brains as much as it could
• Design benchmark for the AIs on each side– How well each AI performs
• Learning about players helps game designs– Player will not like a game that they can never win!– Player will not like a game that they can easily win!
Computer Science, Portland State University 14
Finite State Machine (FSM)
• Code the game strategy in a FSM• Inputs or what matters
– Treasury: Human and Chaos– Terrain and current human nodes on the map
• Output– Where to place the units– What units to place– Based on current state
• Allocate more farmers for earning money• Allocate more infantry for attacking Chaos
Computer Science, Portland State University 18
FSM (Cont.)
• My FSM is static• Hard to make it dynamic and adaptive
– Too many possibilities (24*24 size map plus human ……)
• Works well as a benchmark– How well the Chaos AI performs
• Train Neural Network (NN)– Hopefully, teach the NN to recognize the map
Computer Science, Portland State University 19
Use Neural Network (NN)
• Input nodes: MAP_SIZE+2 – Map size: 24*24 = 576– Human+Chaos treasury: 1+1
• Output nodes: MAP_SIZE– Human units placement
• One hidden layer: MAP_SIZE+2• Normalize the inputs/outputs range: [0.0, 1.0]• Use other’s NN code:
– AI Game Engine: well developed, but not efficient– Tim Jones book: simple and efficient NN
Computer Science, Portland State University 20
Train the NN
• Firstly, use the static FSM to train the NN– Random generated maps– Random initial Human/Chaos treasury – FSM generates outputs as the training case
• Secondly, improve the NN– Randomly generate strategy– If it performs better than current NN– Train NN with the strategy
Computer Science, Portland State University 21
Understand the NN Outputs
• NN doesn’t always tell the exact answer• So we have to “guess”
Human_T MatchHuman(float type){float threashold = 0.2;type = type*HUMAN_TYPES;if(type<=threashold || type>HUMAN_TYPES+threashold)
return NO_HUMAN;
if(fabs(type-float(DRONE))<threashold)return DRONE;
else if(fabs(type-float(MINE))<threashold)return MINE;
else if(fabs(type-float(FARMER))<threashold)return FARMER;
else if(fabs(type-float(INFANTRY))<threashold)return INFANTRY;
else if(fabs(type-float(SETTLER))<threashold)return SETTLER;
else if(fabs(type-float(ARMOR))<threashold)return ARMOR;
else if(fabs(type-float(JAMMER))<threashold)return JAMMER;
else if(fabs(type-float(ARTILLERY))<threashold)return ARTILLERY;
return NO_HUMAN;}
CNNPlayer::DoTurns(){ ……//normalize the outputfloat low = actual[0];float high = actual[0];float avg = 0;for(i=0; i<OUTPUT_NEURONS; i++){
if(low>actual[i])low=actual[i];
if(high<actual[i])high=actual[i];
avg+=actual[i]/OUTPUT_NEURONS;}
for(i=0; i<OUTPUT_NEURONS; i++){if(actual[i]<=0)
actual[i] = 0;else{
actual[i] = (actual[i]-low)/(high-low);//actual[i] = actual[i]/avg;
}}……}
Computer Science, Portland State University 25
Agenda
• About Advanced Protection• My Design and Work• Results and Evaluations• Conclusion and Discussion
Computer Science, Portland State University 26
Evaluations
• Test based on each turn– Only for each turn, the Chaos’ brain is certain– My AI performs well (win) if and only if:
• Given random initial treasury and map• It has advantage against most Chaos’ brains (money
earned)
• Good performance on each turn leads to final win
Computer Science, Portland State University 27
Statistics of My FSM
Initial Treasury
We want this!
Not well dealing with money between this range
Computer Science, Portland State University 28
Statistics of My NN AI
Initial Treasury
We want this!
Computer Science, Portland State University 29
Agenda
• About Advanced Protection• My Design and Work• Results and Evaluations• Conclusion and Discussion
Computer Science, Portland State University 30
Conclusion
• Studied AP– Adaptive turn-based strategy game
• Static FSM– Benchmark– Train NN
• NN– Developed a demo – Trained by FSM– Need more time to train it
Computer Science, Portland State University 31
Discussion
• Static FSM– Efficient to code– Pros: do exactly what you want– Cons: do exactly what you want
• NN– Like dealing with a child
• Patient• Be careful what you feed to it• Try to understand a child’s language• More trained, usually better performance
Computer Science, Portland State University 32
Discussion (Cont.)
• Coding & debugging issues– Error in coding NN is not easy to be detected
• Wrong training data• Inconsistent between training inputs and using inputs
– Be careful about important parts• Training data• Training process
– Set breakpoint to check– Code review– Train it by the same example, see if it adapts as
expected
