(GAM403) From 0 to 60 Million Player Hours in 400B Star Systems

© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

David Walsh, COO

Dav Stott – Senior Server Developer

October 2015

GAM403

From 0 to 60 Million Player Hours

Frontier Developments plc

in 400 Billion Star Systems

Boss level challenge 1

• Have similar impact as Elite did in 1984 using

• Modern CPU and GPU power

• Network connectivity

• Massively multiplayer

Multiplayer architecture

edServerP2P

Multiplayer architecture

Shared Galaxy

AWS Elastic Beanstalk PHP

MongoDB

Redis

Amazon

DynamoDBAmazon

S3

STUN / TURN servers

•NAT traversal and network relay services

nslookup

turn.server

Amazon Route 53

Latency routing

eu-west-1

EC2

eu-west-1a

EC2

eu-west-1b

TURNTURN

•Uses worldwide AWS data centers to reduce P2P latency

ap-southeast-2

EC2

ap-southeast-

2a

EC2

ap-southeast-

2b

TURN TURN

eu-west-1a

edServer and Auto Scaling

edServer.exe

Amazon CloudWatch

Players

Frames/second

metrics.py

EU #19

Amazon EC2 edServer builds

hosted in Amazon S3

watchDog.py

EC2 management

API

edServer and Auto Scaling

eu-west-1

EU1 EU2 EU3

EU4 EU5 EU6

EU7 EU8 EU9

EU10 EU11 EU12

EU13 EU14 EU15

EU16 EU17 EU18

??

EU20 EU21 EU22

EU2

3

EU24 EU25

EU26 EU27 EU28

edServer and Matchmaking

Giving players the best possible online experience

GET /server

API

EU/GB

Travelled to Barnards Star

In Wing0 5 10 15 20 25

Server OC 1

Server EU 2

Server EU 3

Alpha Centauri

Sol

Barnards Star

Wings

edServer and load balancing

Quiet servers are easier to terminate

Stack players up to server load capacities

Central web API, autoscaling, and authentication

eu-west-1

Availability Zone A

DynamoDB

RDS

Amazon ElastiCache

Auto Scaling group

PHP

S3

PHP

Availability Zone B

A few learning points

aws autoscaling suspend-processes --scaling-processes AZRebalance --region eu-west-1 --auto-scaling-group-name <ASGNAME>

boto.ec2.autoscale.connect_to_region().terminate_instance(instanceId, true)

A few learning points

EC2 API rate limit

PHP’s Composer is vulnerable at deployment time

Orchestration and configuration in source control

Commanders and ships

Relational Document

MySQL MongoDB 3

id 103911108

credits 9119226

name A.N.Other

alive 1

currentShip 0

currentShipType FerDeLance

{"_id": {"commander_id": 103911108, "slot": 0},"name": "FerDeLance","modules": {

"HugeHardpoint1": "Hpt_BeamLaser_Huge","MediumHardpoint1": "Hpt_PulseLaserBurst_Small","MediumHardpoint2": "Hpt_PulseLaserBurst_Small","Armour":"FerDeLance_Armour_Grade1","FrameShiftDrive": "Int_Hyperdrive_Size4_Class5"

},"health": {"hull": 1000000, "shieldup": true},"fuel": {"capacity": 8, "lvl": 6.554478},"reserve": {"lvl": 0.9162},

}

MongoDB 3 on EC2

eu-west-1

t1.

micro

Amazon

Route 53

mongo-a 10.1.2.3

mongo-b 10.2.2.3

mongo-c 10.3.2.3

mongo-

backup

10.1.2.4

mongo-

arbiter

10.2.2.4






• Set in the real 1:1 sized Milky Way galaxy

• Accurately simulate 400 B star systems

Rings

Moons

Binaries

Name: LHS 6309 A 1.

Class: TerrestrialClassification_EarthLike.

Mass: 1.8877(Gas:0.000, Volatiles:0.685, Rock:0.212, Metal:0.102)

Radius: 1.475 Earths.

Surface temp: 144.821 K.

Black body temp: 144.821 K.

Approx magma temp: 3740.786 K.

Semi-major axis: 0.025 AU.

Eccentricity: 0.076.

Inclination: -0.000 radians?]

Surface pressure: 4961.263 Atm.

Volcanism type: WaterGeysers.

Atmosphere composition: eAtmosphereClass_EarthLike(Nitrogen(1.000),

Oxygen(0.000), Neon(0.000)).

Description: Tidally locked Earthlike body with thick atmosphere and major water

geysers volcanism.

Data storage

EC2 SSD instance storage

Elasticache Redis

RDS MySQL and Mongo

S3

Sophisticated simulation

Sp

ee

d

Co

nsis

ten

cy

Only store “interesting” data








• Exploration, dynamic trading, outfitting

Exploration

Ship navigation computers (DynamoDB)

First discovery credits (S3)

{

"bodies": {"M": {"36028797039542902": { "N": "7" }}},

"cmdr_id": {"N": "119161"},

"ss_addr": {"N": "20578934"}

}

{

"system": {"name": "Sagittarius A*"},

"bodies": {

"36028797039542902": {

"name": "Sagittarius A* A",

"cmdrname": "Zulu Romeo",

"discovered": "2014-12-01 13:48:22“

}

}

Game event logs

eu-west-1

Availability Zone A

Auto Scaling group

PHP PHP

Availability Zone B

Amazon SQS

Queue 1

Queue 2

Trading and outfitting

Overlays:

Default stock of 1000

Player buys 200

Current stock of 800

Gets stored as 0.8

Gradually restocks over time

Until it returns to 1000 when it gets deleted

Players

Politics

Supply and demand

Core simulation

Supply and demand

eu-west-1

Availability Zone A Availability Zone B

Auto Scaling group

PHP PHP

{“buy”:{“fish”:200, “mkt”: “Lave”}}


{“sell”:{“fish”:275, “mkt”: “Sol”}}

{“buy”:{“tea”:20, “mkt”: “Lave”}}

{“sell”:{“cobalt”:200, “mkt”: “Lave”}}

{“buy”:{“tea”:500, “mkt”: “Lave”}}

{“sell”:{“cobalt”:40, “mkt”: “Lave”}}


{“sell”:{“tea”:500, “mkt”: “Sol”}}

{“lave”: {“fish”: 0.8,

“cobalt”: 1.1,

“tea”: 0.5}

{“sol”: {“fish”: 1.2,

“tea”: 1.5}

Game analytics

Game analytics

Shared

API

SQS

Queue

dispatchers

ElasticsearchKibana 4

Kibana 3

Custom

reports On premise servers








• Exploration and dynamic trading

• Player-driven story

Player-driven story

eu-west-1


Auto Scaling group

PHP PHP

{

“faction: "9469194806713“,

"boom": 148,

"bust": 7,

"famine": 10,

"influence": 49

}

Sobek Boys

vs

Federal Bridge

{boom: +1} {inf: -4}

Community Goals

Community goals and leaderboards

eu-west-1


Auto Scaling group

PHP PHP

ZADD "goal:1" 42 "Cmdr Dav"ZADD "goal:1" 89 "Cmdr Jameson"ZADD "goal:1" 100 "Cmdr Thrust"

ZREVRANGE "goal:1" 0 -11) "Cmdr Thrust"2) "Cmdr Jameson"3) "Cmdr Dav"

ZREVRANK "goal:1" "Cmdr Dav"(integer) 2

Powerplay










• Sell and distribute the game ourselves

Store and distribution

https://frontierstore.net

RDS MySQL

Login servers

S3

Game builds

Amazon

CloudFront











• Add platforms, features, seasons

Close quarters combat championships

eu-west-1


Auto Scaling group

PHP PHP

• Instant action PvP

• Join lobbies

• Background matchmaker











• Add platforms, features, seasons

Thank you!

Technology

(GAM403) From 0 to 60 Million Player Hours in 400B Star Systems