27
StarCraft Winner Prediction Presented by: Yaser Norouzzadeh Authors: Y. Norouzzadeh, S. Bakkes, P. Spronck Tilburg Center for Cognition and Communication Tilburg University Tilburg, the Netherlands

StarCraft Winner Prediction

Embed Size (px)

Citation preview

Page 1: StarCraft Winner Prediction

StarCraft Winner PredictionPresented by: Yaser Norouzzadeh

Authors: Y. Norouzzadeh, S. Bakkes, P. Spronck

Tilburg Center for Cognition and CommunicationTilburg University

Tilburg, the Netherlands

Page 2: StarCraft Winner Prediction

2

Table of Contents• Introduction• Data• Features• Method• Results• Conclusion

[email protected]

Page 3: StarCraft Winner Prediction

3

Table of Contents• Introduction• Data• Features• Method• Results• Conclusion

[email protected]

Page 4: StarCraft Winner Prediction

4

StarCraft Environment

[email protected]

Player B basePlayer A base

Page 5: StarCraft Winner Prediction

5

StarCraft Races

[email protected]

Page 6: StarCraft Winner Prediction

6

StarCraft Match Types

[email protected]

TvZ

PvZPvT

Page 7: StarCraft Winner Prediction

7

Winner Prediction• Complexity:

• many action choices• managing units concurrently• different strategies• various match types (PvT, PvZ, TvZ, PvP, ZvZ, and TvT)

• Winner:• The first player who destroy all enemy units

• Application:• Evaluation function for AI bots

[email protected]

Page 8: StarCraft Winner Prediction

8

Table of Contents• Introduction• Data• Features• Method• Results• Conclusion

[email protected]

Page 9: StarCraft Winner Prediction

9

Dataset Information• Expert replays collected by Synnaeve and Bessiere 2012• Database provided by Robertson and Watson 2014 • Filter condition:

If (game-length < 10 min) or (game-length > 50 min) or (game-winner == Null) Remove replay

Match Type PvT PvZ TvZ PvP ZvZ TvT

Number of replays 2017 840 812 392 199 395

Number of replays(After filtering) 1490 579 612 263 115 298

[email protected]

Page 10: StarCraft Winner Prediction

10

Winning Rate

PvT PvZ TvZ48%

49%

50%

51%

52%

53%

54%

55%

56%

57%

Wining rate percent in non-symmetric match types

[email protected]

Page 11: StarCraft Winner Prediction

11

Map Types• 60% of maps have a size of 128 × 128 tiles• Different maps are used

[email protected]

Page 12: StarCraft Winner Prediction

12

Table of Contents• Introduction• Data• Features• Method• Results• Conclusion

[email protected]

Page 13: StarCraft Winner Prediction

13

Feature sets• Time-Dependent Features (TDF)• Time-Independent Features (TIF)

Sample size per match typeMatch types PvT PvZ TvZ PvP ZvZ TvT

Feature samples 24k 9k 9k 3k 1k 4k

[email protected]

Page 14: StarCraft Winner Prediction

14

Extracting Time-Dependent Features

10 seconds Features during 10s

180 seconds TDF(mean,var,dif)

[email protected]

Page 15: StarCraft Winner Prediction

15

Time-Dependent Features• Proposed features:• Number of frequent commands (move, build, tech, hold, siege, and burrow)• Number of micro/macro commands• Number of control/strategy/tactic commands• Number of unique regions that include a building• Difference of building values of all regions

• Well-known features:• Unspent resources: available resources on average at any given time• Income: total resources that are collected over T• APM

[email protected]

Page 16: StarCraft Winner Prediction

16

Extracting Time-Independent Features

Map TIF

[email protected]

Page 17: StarCraft Winner Prediction

17

Time-Independent Features (TIF)• Number of regions• Ratio of buildable tiles• Ratio of walkable tiles• Average of choke distances• Height level ratio (low, low doodads, high, high doodads, very high,

very high doodads)• Map dimension in tiles

[email protected]

Page 18: StarCraft Winner Prediction

18

Table of Contents• Introduction• Data• Features• Method• Results• Conclusion

[email protected]

Page 19: StarCraft Winner Prediction

19

Problem Formulation• Binary classification: win(1)/lose(0)• Classification methods• Gradient Boosting Regression Trees (GBRT)• Random Forest (RF)

• Approaches:• Individual model for each match type (6 models)• Mixed models

• For symmetric match types (PvP, ZvZ, and TvT)• For non-symmetric match types (PvT, PvZ, and TvZ)

• General model for all match types

[email protected]

Page 20: StarCraft Winner Prediction

20

Table of Contents• Introduction• Data• Features• Method• Results• Conclusion

[email protected]

Page 21: StarCraft Winner Prediction

21

A=APM and economy featuresB=time-dependent featuresC=time-independent features

Winner Prediction by Individual models

Model Features PvT PvZ TvZ PvP ZvZ TvT

baseline 0.55 0.51 0.56 0.50 0.50 0.50RF A,B,C 0.591 0.611 0.502 0.502 0.515 0.491

GBRT A,B,C 0.595 0.623 0.502 0.502 0.507 0.483

RF A,B 0.644 0.634 0.624 0.643 0.587 0.639

GBRT A,B 0.637 0.634 0.624 0.639 0.581 0.635

[email protected]

Page 22: StarCraft Winner Prediction

22

Winner Prediction by Mixed Models and General Model

Model Features Non-symmetric Symmetric General

RF A,B,C 0.575 0.497 0.591

GBRT A,B,C 0.577 0.499 0.593

RF A,B 0.639 0.637 0.639

GBRT A,B 0.635 0.634 0.635

[email protected]

Page 23: StarCraft Winner Prediction

23

Top-10 TDF In Individual Models

• Income and unspent resources always amongst top three (Except ZvZ)• In ZvZ, micro commands have strongest predictive value• Control commands (move, gather, build, …) and region values are the next strongest predictive value

[email protected]

Page 24: StarCraft Winner Prediction

24

Top-10 TDF In Mixed/General Model

[email protected]

Page 25: StarCraft Winner Prediction

25

Table of Contents• Introduction• Data• Features• Method• Results• Conclusion

[email protected]

Page 26: StarCraft Winner Prediction

26

Conclusion• Winner prediction is possible for all match types.• Mixed models also manage to predict the match winner as individual

models.• In all models, top-10 features are more or less the same.• Economic features (income and unspent) are strongest features across

match types

[email protected]

Page 27: StarCraft Winner Prediction

27

Thank you for your attention.

[email protected]