Identifying MMORPG Bots: A Traffic Analysis Approach (MMORPG: Massively Multiplayer Online Role Playing Game) Kuan-Ta Chen National Taiwan University Jhih-Wei

Identifying MMORPG Bots:Identifying MMORPG Bots:A Traffic Analysis ApproachA Traffic Analysis Approach

(MMORPG: Massively Multiplayer Online Role Playing (MMORPG: Massively Multiplayer Online Role Playing Game)Game)

Kuan-Ta Chen

National Taiwan University

Jhih-Wei JiangPolly HuangHao-Hua ChuChin-Laung LeiWen-Chin Chen

Collaborators:

2Identifying MMORPG Bots: A Traffic Analysis Approach

Talk OutlineTalk Outline

Motivation

Trace collection

Traffic analysis and bot identification schemes

Performance evaluation

Scheme Robustness

Conclusion


Game BotsGame Bots

AI programs that can perform many tasks in place of gamers

Can reap rewards efficiently in 24 hours a day break the balance of power and economies in the game world

Therefore bots are forbidden in most games


Bot DetectionBot Detection

Detecting whether a character is controlled by a bot is difficult since a bot obeys the game rules perfectly

No general detection methods are available today

The state of practice is identifying via human intelligence (as bots cannot talk like humans)

Labor-intensive and may annoy innocent playersThis work is dedicated to automatic

detection of game bots(without intrusion in players’ gaming experience)


Key ContributionsKey Contributions

We proposed to detect bots with a traffic analysis approach

We proposed four strategies to distinguish bots from human players based on their traffic characteristics


Bot Detection: A Decision ProblemBot Detection: A Decision Problem

Game client Game server

Traffic stream

Q: Whether a bot is controlling a game client given the traffic stream it generates?A: Yes or No


Ragnarok Online -- a screen shotRagnarok Online -- a screen shot

Figure courtesy of www.Ragnarok.co.kr

Ragnarok Online

One of the most popular MMORPGs (they claimed 17 million subscribers worldwide recently)

Notorious for the prevalence of the use of game bots


Game Bots in Ragnarok OnlineGame Bots in Ragnarok Online

Two mainstream bot series:

Kore -- KoreC, X-Kore, modKore, Solos, Kore, wasu, Erok, iKore, and VisualKore

DreamRO (popular in China and Taiwan)

Both bots are standalone (game clients not needed), fully-automated, script-based, and interactive


DreamRODreamRO -- A Screen Shot -- A Screen Shot

World Map

View Scope

Character Status

Character is here


Trace CollectionTrace Collection

Category Trace # Participants Average Length

Network

Human players

8 traces 2 rookies2 experts

2.6 hours ADSL, Cable Modem,Campus Network

Bots 11 traces 2 bots 17 hours

Player skills

Character levels / equipments

Network connections

Network conditions (RTT, loss rate, etc)

Heterogeneity was preserved

206 hours and 3.8 million packets were traced in total


Traffic Analysis of Collected Game Traffic Analysis of Collected Game TracesTraces

Traffic is analyzed in terms of

Command timing

Traffic burstiness

Reaction to network conditions

Four bot identification strategies are proposed


Command TimingCommand Timing

Observation

Bots often issue their commands based on arrivals of server packets, which carry the latest status of the character and environment

game client game server

time

Client response time (response time)

Time difference between the release of a client packet and the arrival of the most recent server packet

State update

t1

Client commandt2

Response timeT = t2 – t1


CDF of Response TimesCDF of Response Times

DreamRO> 50% response times are extremely small

KoreZigzag pattern (multiples of a certain value)


Histograms of Response Times Histograms of Response Times (DreamRO traces)(DreamRO traces)

1 ms

multiple peaks

1 ms multiple peaks

Many client packets are sent in response to server packets


Histograms of Response TimesHistograms of Response Times

Regularity in the distribution of bots’ response times

Scheme #1: Command Timing

A traffic stream is considered from a bot if it has …

Quick response times (< 10 ms) clustered

Regularity in the distribution of response times, i.e., if any frequency component exists


Traffic BurstinessTraffic Burstiness

Traffic burstiness

An indicator of how traffic fluctuates over time

The variability of packet/byte counts observed in successive periods

Index of Dispersion for Counts (IDC)The IDC at time scale t is de ned as

I t =Var(Nt)E (Nt)

;

whereNt indicatesthenumber of arrivals in intervalsof time t.


Example: Wine Sales and IDCExample: Wine Sales and IDC

The period is approximately 12 months

The IDC at 12 months is the lowest


The Trend of Traffic BurstinessThe Trend of Traffic Burstiness

Traffic generated by human players, of course, has no reason to exhibit such property

Conjecture for Bot Traffic

1. Each iteration of the bot program’s main loop takes roughly the same amount of time

2. Each iteration of the main loop sends out roughly the same number of packets

3. Bot traffic burstiness will be the lowest in the time scale around the time needed to complete each iteration


Examining the Trend of Traffic Examining the Trend of Traffic BurstinessBurstiness

Regularity in the distribution of bots’ response times

Scheme #2: Trend of Traffic Burstiness

A traffic stream is considered from a bot if …

the IDC curve has a falling trend at first and after that a rising trend, and

both trends are detected at time scales < 10 sec


The Magnitude of Traffic BurstinessThe Magnitude of Traffic Burstiness

Difficultyno “typical” burstiness of human player traffic

Solutioncompare the burstiness of client traffic with that of the corresponding server traffic (as servers treat all game clients equally)

Scheme #3: Burstiness MagnitudeA traffic stream is considered to be generated by a bot if the client traffic burstiness is much lower than the corresponding server traffic burstiness

Conjecture

Bot traffic is relatively smooth than human player traffic


Human Reaction to Network Human Reaction to Network ConditionsConditions

Conjecture for Human Player Traces

1. The network delay of packets will influence the pace of game playing (the rate of screen updates, character movement)

2. Human players will unconsciously adapt to the game pace (the faster the game pace is, the faster the player acts)

server

Traffic jam!!

Is there any relationship between network delay and

the pace of user actions?


Packet Rate vs. Network DelayPacket Rate vs. Network Delay

Scheme #4: Pacing

A traffic stream is considered from a bot if …correlation between pkt rate vs. network delay is non-negative

Human player traces: downward trend


Performance EvaluationPerformance Evaluation

Evaluate the sensitivity of input size by dividing traces into segments, and computing the above metrics on a segment basis

Metrics

Correct rate the ratio the client type of a trace is correctly determined

False positive rate the ratio a player is misjudged as a bot

False negative rate

the ratio a bot is misjudged as a human player


Performance Evaluation ResultsPerformance Evaluation Results

[Burstiness magnitude]always achieves low false positive rates (< 5%) and yields a moderate correct rate (≈ 75%)

[Command timing and Burstiness trend]Correct rates higher than 95% and false negative rates lower than 5% given an input size > 2,000 packets


An Integrated ApproachAn Integrated Approach

In practice, we can carry out multiple schemes simultaneously and combine their results according to preference

Conservative approach:command timing AND burstiness trend

Aggressive approach:command timing OR burstiness trend


An Integrated Approach -- ResultsAn Integrated Approach -- Results

Aggressive approach (2,000 packets): false negative rate < 1% and 95% correct rateConservative approach (10,000 packets): ≈ 0% false positive rate and > 90% correct rate

Aggressive


Robustness against Counter-AttacksRobustness against Counter-Attacks

Just like anti-virus software vs. virus writers

Our schemes only rely on packet timings

An obvious attack is adding random delays to the release time of client packets

Command timing scheme will be ineffective

Schemes based on traffic burstiness are robust Adding random delays will not eliminate the bot

signature unless the added delay is longer than the iteration time by orders of magnitude or heavy-tailed

However, adding such long delays will make the bots incompetent as this will slowdown the character’s actions by orders of magnitude


Simulating the Effect of Random Delays Simulating the Effect of Random Delays on IDCon IDC


SummarySummary

Traffic analysis is effective to identify game bots

Proposed four bot decision strategies and two integrated schemes for practical use

The proposed schemes (except the one based on command timing) are robust under counter-attacks

Thank You!Thank You!

Kuan-Ta Chen

Documents

Identifying MMORPG Bots: A Traffic Analysis Approach (MMORPG: Massively Multiplayer Online Role Playing Game) Kuan-Ta Chen National Taiwan University Jhih-Wei