Cheats, Anti-Cheats, and Machine Learning

‘THE MULTI-BILLION DOLLAR INDUSTRY THAT’S

IGNORED’PART I: CHEATS, ANTI-CHEATS, AND MACHINE LEARNING

SOLUTIONS

info@whitewizardstudio.com

INTRO

HP ESS TVM – RESEARCHER / PERSON OF MANY HATS

OWNERS – WHITE WIZARD STUDIO LLC

FORMER PROFESSIONAL GAMER – CPL, CEVO

@RSEVEY

PRINCIPAL RESEARCHER, VERACODE

SECURITY ENGINEER, APP SEC SPECIALIST

SOFTWARE DEVELOPER

AUTHOR / INSTRUCTOR

@J_MONTY

THIS TALK

• THE TALK THAT ALMOST WASN’T

• BH USA 2014, PAX DEV

• VALVE, FACEPUNCH STUDIO

GAMING INDUSTRY – OPEN GAMING ALLIANCE

ACTIVE GAMERS – FROM THE OPEN GAMING ALLIANCE

GAMER

• 67% OF US HOUSEHOLDS PLAY VIDEO GAMES

• AVERAGE AGE: 35

• 54% MALE / 46% FEMALE

• 62% PLAY GAMES ONLINE

SECURITY ISSUES

CheatingAnti-Cheats, EULA, Invasive nature

Digital Distribution Systems

Steam, Origin, Battle.net

DRM

Online Economy

Micro transactions, stores, Twitch

Game Engine

Application Security

CHEATINGCheat Maker

Cheat Distribut

or

User

CHEATING ECONOMY

Cheat Distributor Registered Users Estimated Paid Users Estimated Monthly Gross10% - 20% active paid

Aimjunkies.com 172,315 155,084 $186,100.8 - $372,201.6

Tmcheats.com 171,833 154,649 $185,578.8 – $371,157.6

Artificialaiming.net 142,319 113,855 $136,626 - $273,252

Fpscheats.com 375,733 281,799 $338,158.8 - $676,317.6

Ilikecheats.com 276,871 207,653 $249,183.6 - $498,367.2

Catalyst-hax.com 118,448 82,913 $99,495.6 – $198,991.2

Callofdutyhacks.com 225,709 169,281 $203,137.2 - $406,274.4

Hackersadvantage.com 8,774 7,896 $9475.2 - $18,950.4

CHEAT ECONOMY

• ~$1,407,752 TO ~$2,815,509 PER MONTH

• ~$16,893,024 TO ~$33,786,108 ANNUALLY

MORE THEORY

• 2.2 MILLION VAC BANNED ACCOUNTS

• IF EACH CHEATED FOR 1 MONTH

$21.9 MILLION• AVERAGE GAME COST - $20

• VALVE MAKES IF EVERYONE RE-BUYS:

$43.7 MILLION

ANTI-CHEATS EXAMPLES

VAC, Warden, Punkbuster

Monitor RAM, Processes

Doesn’t work.

Signature

Fairfight

Statistical based

Easy to avoid – just don’t use aimbot

Server

Valve’s Overwatch

Humans review highlights, match games only

Humans are.. Human.

Human

“This specific VAC test for this specific round of cheats was effective for 13 days, which is fairly typical.”

https://www.reddit.com/r/gaming/comments/1y70ej/valve_vac_and_trust/

“For most cheat developers, social engineering might be a cheaper way to attack the system than continuing the code arms race, which means that there will be more Reddit posts trying to cast VAC in a sinister light.”

“Our response is to make it clear what we were actually doing and why with enough transparency that people can make their own judgements as to whether or not we are trustworthy.”

Brian Dye, Senior Vice President for Information Security at Symantec

“82 percent of all malware it detects stays active for a mere hour, and 70 percent of all threats only surface once, as malware authors rapidly change their software to skirt detection from traditional antivirus solutions. ‘The function signature-based AV serves has become more akin to ghost hunting than threat detection and prevention’…”

http://www.pcworld.com/article/2150743/antivirus-is-dead-says-maker-of-norton-antivirus.html

FireEye “Antivirus Is Dead”PC World article in May of 2014

“Anti-virus is dead.”

VALVE, BLIZZARD, AND OTHERS

• ANTI-CHEAT JUSTIFICATION IS EXTREME

• BLIZZARD – THE WARDEN

• MONITORS RAM, PROCESSES, BROWSER TABS

• VALVE – VALVE ANTI CHEAT

• SENDS DNS INFORMATION BACK TO VALVE

• BLIZZARD AND VALVE – “INFORMATION IS HASHED BEFORE SENT”

END USER LICENSE AGREEMENTS - VALVE

END USER LICENSE AGREEMENTS - BLIZZARD

PUNKBUSTER EULA

THE CHEATS (OR ‘H4X, HAX, HACKS’)

• PAID CHEATS HAVE DRM SYSTEM

• SOME ARE HOSTED ON AMAZON EC2

• AMAZON DOESN’T CARE

THE LOADERS

HOW THEY’RE WORKING

• TL;DR THEY’RE ROOTKITS

• OPERATE AT RING 0

• THUS ANTI-CHEATS ALSO OPERATE AT RING 0.

CS:GO CHEATING

DEMO

OUTCOME

• ACCOUNT STILL NOT VAC BANNED

• 10 COMPETITIVE MATCHES, OBVIOUS CHEATING

• OVERWATCH BANNED

• BOTTOM-LINE:

• VAC DOESN’T WORK, YET CAN BE HIGHLY INVASIVE

LET’S REVIEW

• PROBLEM: CHEATING STILL NOT EFFECTIVELY DETECTED

• ANTI-CHEATS ARE INVASIVE

• CHEAT MAKERS AND DISTRIBUTORS HAVE LOTS OF MONEY

PROPOSED SOLUTION

• ANTI-CHEAT BASED OFF MACHINE LEARNING

• DOESN’T HAVE TO BE PERFECT

• JUST HAS TO BE BETTER THAN CURRENT SYSTEM, AND HUMANS

ML LOVES DATA

• FIRST PROBLEM… GETTING DATA

• FACEPUNCH NOR VALVE WOULD GIVE US ANY DATA

• HARD TO HARVEST IT ALL OURSELVES.

•  VACBANNED.COM

• CHEATER DATASET

• COMPETITIVE LEAGUES

• NON-CHEATER DATASET

UNAVOIDABLE DATASET ISSUES

• VAC BANNED FROM WHICH GAME?

• IMMEDIATE VAC BAN, NO STATS

• NON-CHEATING PLAYER SET CONTAINS CHEATERS

• STEAM WEB API LIMITATIONS

• 400% ACCURACY?? WUT?

• MISSING IMPORTANT FEATURES

ML ANTI-CHEAT SYSTEM

• CLASSIFICATION PROBLEM (CHEATING / NOT CHEATING)

• SUPERVISED LEARNING

• BOOSTED DECISION TREE

• CHOOSING FEATURES

• “INDIVIDUAL MEASURABLE HEURISTIC PROPERTY OF A PHENOMENON BEING OBSERVED”

• K:D, ACCURACY, # OF MVP AWARDED, ETC.

DECISION TREE – TITANIC EXAMPLE

DEMO

Q & A