REINFORCEMENT LEARNING IN STARCRAFT 2

Preview:

Click to see full reader

Citation preview

REINFORCEMENT

LEARNING IN

STARCRAFT 2Björn Boyd Isacsson

AI IN GAMES

When is it used

Improvement needed

Players are too good

Difficult and time-consuming creation

General only

STARCRAFT 2

Why?

What is StarCraft 2?

AI in StarCraft

Default

Human-made

Galaxy Script

REINFORCEMENT LEARNING

Why?

Games

Galaxy limitations

What is Reinforcement Learning?

REINFORCEMENT LEARNING

THE BASICS

States and Actions

Reward-based

Iterations

Optimisation vs Exploration

Put it all together!

Problems?

REINFORCEMENT LEARNING

FIXING THE PROBLEMS

Continuous gameplay

Sampling

Lots of actions

Limiting the actions

“Infinite” states

Table → Function

Q[s][a] → Q(s, a) = α + β * x1 + γ * x2 + …

Find values for α, β, γ…

THE AI IMPLEMENTATION

Sample time

0.1 seconds

Actions

Move left

Move right

Attack closest target

Function

Distance

Reinforcement Learning AI Health

Enemy Health

Action chosen

MY OPTIMAL

ONE UNIT – 10 ITERATIONS

ONE UNIT – 100 ITERATIONS

MANY UNITS

Implementing many units

One AI

Multiple AIs

MANY UNITS – 100 ITERATIONS

MANY UNITS – 1000 ITERATIONS

LIMITATIONS

Reinforcement Learning

Requires instant reward

Many iterations = Lots of time

No guaranteed optimal

Galaxy Script

Memory!

CONCLUSION

There is potential

Improvements could expand the possibilities

Doesn’t fit every scenario

Recommended

Reinforcement Learning
Technology
Reinforcement Learning: Learning algorithms
Documents
Cooperative Inverse Reinforcement Learning...Cooperative Inverse Reinforcement Learning Dylan Hadfield-Menell CS237: Reinforcement Learning May 31, 2017
Documents
Replicating DeepMind StarCraft II Reinforcement Learning ......Replicating DeepMind StarCraft II Reinforcement Learning Benchmark with Actor-Critic Methods Bachelor’sthesis RomanRing
Documents
Reinforcement Learning Lecture Inverse Reinforcement Learningipvs.informatik.uni-stuttgart.de/mlr/wp-content/uploads/2017/07/09... · Reinforcement Learning Inverse Reinforcement
Documents
Reinforcement Learning and Deep Reinforcement Learningcse.ucdenver.edu/.../Class-22-Reinforcement-learning-DL.pdf · 2018. 11. 28. · Outlines 1 Principles of Reinforcement Learning
Documents
Adaptive High-Level Strategy Learning in StarCraft
Technology
On Reinforcement Learning for Full-length Game of StarCraft
Documents
Reinforcement Learning for the Beginning of Starcraft II Game
Documents
Introduction to Reinforcement Learning · reinforcement learning are able to keep up and even surpass humans in real-time in games as complex go and starcraft, which was said to be
Documents
Generalization in Reinforcement Learning: Successful ...papers.nips.cc/paper/1109-generalization-in-reinforcement-learning... · Generalization in Reinforcement Learning: Successful
Documents
Inverse Reinforcement Learning CS885 Reinforcement
Documents
Eick: Reinforcement Learning. Reinforcement Learning Introduction Passive Reinforcement Learning Temporal Difference Learning Active Reinforcement Learning
Documents
Inverse Reinforcement Learning - Peoplecbfinn/_files/bootcamp_inverserl.pdf · Apprenticeship Learning via Inverse Reinforcement Learning. Good introduction to inverse reinforcement
Documents
Tutorial: Deep Reinforcement Learning - Machine Learning ...hunch.net/~beygel/deep_rl_tutorial.pdfTutorial: Deep Reinforcement Learning - Machine Learning
Documents
Reinforcement Learning - uni-freiburg.degki.informatik.uni-freiburg.de/.../recordings/reinforcement.pdf · Reinforcement Learning 3 What is Reinforcement Learning? Learning from interaction
Documents
Reinforcement Learning & Apprenticeship Learning
Documents
Bayesian Reinforcement Learning - mlg.eng.cam.ac.ukmlg.eng.cam.ac.uk/rowan/files/BayesianReinforcementLearning.pdf · Introduction Bayesian Reinforcement Learning Bayesian Reinforcement
Documents
Reinforcement Learning Chapter 13 What is Reinforcement Learning? Q-Learning Examples 1
Documents
Deep Reinforcement Learning - Environments Tour€¦ · Deep Reinforcement Learning - Environments Tour Machine Learning Gdańsk, 02.10.2017 - Adam Wróbel Starcraft 2 DeepMind toolset
Documents