Xbox360 matchmaking & predictions

Sharad Agarwal (microsoft research)

Chris Butcher (bungie)

Youngki Lee (intern)

Jitu Padhye (microsoft research)

Xbox360 Internet games

• most games P2P

– Xbox Live server assists in rendezvous

– 1 console is game host

– 15 other consoles are clients

• picking groups of 16 out of O(million)

– challenging

– called matchmaking

10/17/2008 2

matchmaking

• user wants to play game of type X

• weighted dice determines if it is game host

• if client

– asks Xbox Live Server for consoles hosting X

– probes each console sequentially

– picks one with best network quality

– if not found, reduce constraints of X, try again

10/17/2008 3

packet pair background

• need to estimate latency, available BW

• easier to estimate RTT, capacity

• packet pair

– 2 back-to-back packets

– measure capacity of bottleneck link

– suffers with interrupt coalescence

• not a problem on our platform

– suffers if other packet gets in between

• use median of ~4 packet pairs

10/17/2008 4

Halo background

• Halo series of games

– FPS shooter

– play locally

– play on Internet

• Halo3 recently released

• Internet play

– sensitive to delay

– must have enough BW

10/17/2008 5

motivation

1. understand player population• geography, numbers, play time, etc.• not much known about such large population

2. understand packet pair performance• prior work : tiny testbeds, planetlab, etc• our work : real, live, global network of O(million) nodes

3. matchmaking : reduce time, improve accuracy

• don’t want users to wait• slow game : poor user experience• can spend extra time improving accuracy

10/17/2008 6

Part 1:

population, QoS analysis

data

• from Xbox live server– Halo3 : alpha, beta, delta, release

• session data (per game attempted)– time, session-id, src IP

• probe data– session-id, dest IP

– # packet-pairs {sent, rcvd}

– latency {min, med}, avg {up, down} capacity

10/17/2008 8

basic stats

10/17/2008 9

14.nov.2007 3.jan.2008 (50 days)

sessions 39,803,350

distinct IPs 5,658,951

total probes 126,085,887

• sub-sampled by 20%

player locations

10/17/2008 10

player locations

11

diurnal behavior

10/17/2008 12

0

10

20

30

40

50

60

70

# o

f se

ssio

ns

(x1

00

0)

time (per hour)

sessions per IP

10/17/2008 13

1

4

16

64

256

1024

4096

16384

65536

262144

1048576

1 4 16 64 256 1024 4096

# o

f IP

ad

dre

sse

s

# of sessions

delay (RTT) values

10/17/2008 14

0.01

0.1

1

1 10 100 1000 10000

cum

ula

tive

fre

q. (

pro

bes

)

delay (ms)

capacity values

10/17/2008 15

192 Kbps

1.6Mbps

5.8 Mbps

10Mbps

0

1

2

3

4

5

6

0 2000 4000 6000 8000 10000

freq

ue

ncy

(x1

,00

0,0

00

)

capacity (kbps)

Part 2:

time-based QoS predictor

time-based predictor

• between client A and client B– t1 : probe

– t2 : estimate• can we use previous delay value?

• can we use previous capacity value?– to disqualify this pair, or

– to select this pair, or

– do quick re-probe

– how close do t1 and t2 have to be?

• why would this work?– if bottleneck is last mile

• delay should be similar over time

• capacity should be similar over time

17 October 2008 17

delay prediction

10/17/2008 18

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

10

.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9 1

1.1

1.2

1.3

1.4

1.5

1.6

1.7

1.8

1.9 2

Mo

re

cum

ula

tive

fre

q. (

src-

dst

IPs)

coefficient of variation

Within 5 min

Within 30 min

Within 6 hr

Within 1 day

No Constraints

Baseline

downstream capacity prediction

10/17/2008 19

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

10

.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9 1

1.1

1.2

1.3

1.4

1.5

1.6

1.7

1.8

1.9 2

Mo

re

cum

ula

tive

fre

q. (

src-

dst

IPs)

coefficient of variation

Within 5 min

Within 30 min

Within 6 hr

Within 1 day

No Constraints

Baseline

Part 3:

prefix-based QoS predictor

why?

10/17/2008 21

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 5 10 15 20 25 30

cum

ula

tive

fre

q. o

f p

airs

# of probes

IP Pair

Prefix Pair

prefix-based predictor

• clients A1, A2 in prefix A; clients B1, B2 in prefix B• determine prefixes by BGP table (12/27/2007 RouteViews)

– t1 : probe A1 to B1

– t2 : estimate A2 to B2

• can we use previous delay value?

• can we use previous capacity value?– to disqualify this pair, or

– to select this pair, or

– do quick re-probe

– how close do t1 and t2 have to be?

• why would this work?– if bottleneck is last mile

• perhaps clients in same prefix have similar last mile

17 October 2008 22

delay prediction

10/17/2008 23

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

10

.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9 1

1.1

1.2

1.3

1.4

1.5

1.6

1.7

1.8

1.9 2

Mo

re

cum

ula

tive

fre

q. (

src-

dst

pre

fixe

s)

coefficient of variation

Within 5 min

Within 30 min

Within 6 hr

Within 1 day

No Constraints

Baseline

delay prediction SIQR

10/17/2008 24

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

5

10

15

20

25

30

35

40

45

50

55

60

65

70

75

80

85

90

95

10

0

Mo

re

cum

ula

tive

fre

q. (

src-

dst

pre

fixe

s)

size of range (ms)

SIQR(25%-75%)

10%-90%

downstream capacity prediction

10/17/2008 25

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

10

.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9 1

1.1

1.2

1.3

1.4

1.5

1.6

1.7

1.8

1.9 2

Mo

re

cum

ula

tive

fre

q. (

src-

dst

pre

fixe

s)

coefficient of variation

Within 5 min

Within 30 min

Within 6 hr

Within 1 day

No Constraints

Baseline

Part 4:

geography-based predictor

geography-based predictor

• between client A and client B– t1 : estimate

• calculate locations of A & B

• large distance between A & B imply high delay?

• large distance between A & B imply low capacity?– to disqualify this pair, or

– to select this pair, or

– do quick re-probe

• why would this work?– delay : if speed of light / number of hops is

significant factor

– capacity : if last mile is not bottleneck

17 October 2008 27

why?

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 2000 4000 6000 8000 10000 12000

cum

ula

tive

fre

q. (

pro

bes

)

distance (miles)

10/17/2008 28

delay prediction

10/17/2008 29

summary

• online gaming is really popular– player characterization

– ( geographic density, time of day, games per console, …)

– network characterization– ( delay distributions, capacity, …)

– packet pair consistency

• efficient and accurate matchmaking is important– using past history to predict future performance

– pick better hosts, reduce probe time & overhead

– using IP topology information– remove nodes with likely poor performance

– using geography information

10/17/2008 30

Xbox360 matchmaking & predictions

Sharad Agarwal (microsoft research)

Chris Butcher (bungie)

Youngki Lee (intern)

Jitu Padhye (microsoft research)