Game Analytics

User

INTERACTIVE DIGITAL ENTERTAINMENT

Development, game economics

Persuasion, value, learning

Communication in games

Play experience, design

User behavior,data mining

Digital storytelling, online behavior

Personal background MSc. In Natural Sciences

Large-scale trends and evolutions in time/space, Geographic Information Systems

PhD in Computer Science Empirical evaluation of games, HCI, user testing, game telemetry

Post doc. At the Center for Computer Games Research, IT University Copenhagen Play experience, biometrics, game data mining, game development

RA/project lead, Department of Informatics, Copenhagen Business School Game piracy, behavioral economics, co-creation – more game telemetry

data mining

Assistant Prof., Department of Communication, Aalborg University Yet more game data mining, more game development, more innovation

Co-Founder & Lead Game Analyst, GameAnalytics Tools and consulting on application of game telemetry to development

Research breakdown

90% applied research

10% theory (play experience, play personas)

Collaboration with industry – real needs

Collaboration with international colleagues 1 single-authored publication ...

Game user research

Game User Research

Focus: How users interact with IDE applications and each other + the business side

Game User Research – answering e.g.: Who are the users of interactive digital

entertainment products? What do they do and where, with whom and

why? How do we develop products for different users?

Game User Research

Multi-disciplinary ”field” Researchers from CS, HCI, communication,

design, media, psychology, AI, art, economics, development ...

Emergent field – lack of established theory

Exponential growth in research publications

Backed by a growing industry where users are central

Game User Research

Four main lines of investigation in GUR:

Usability: Can the user operate the controls?

Playability: Is the user having a good experience?

Behavior: What is the user doing while playing?

Development: Integrating GUR in business practices

Game User Research

Why interesting? New field of research Emerging methodologies + theories Plenty of tough problems

Collaboration Broad relevance Multi-disciplinary

Affects millions of people Industry interest Latest technologies

New field

Multi-disciplinary

Impact

User

INTERACTIVE DIGITAL ENTERTAINMENT

Development, game economics

Persuasion, value, learning

Communication in games

Play experience, design

User behavior,data mining

Digital storytelling, online behavior

Understanding games and playersImproving development & testing

User behavior

Patterns in play behavior

Play personas

Spatial user behavior

Behavior correlations with PX/PsyPys/design

Game metrics

What are game metrics?

Metrics = Business Intelligence [BI]

BI is derived from computer-based methods for identifying, extracting and analyzing business data for strategic or operational purposes Across market-, geographic- and temporal

distance

Supports decision making (Decision Support Systems)

What are game metrics?

Quantitative measures about any aspect of games Players: gameplay, customers, monetization, Production: team size, pipeline, milestones,

markets Technical performance: servers, infrastructure

Any other relevant quantitative measure

(e.g. management)

Analysis of game metrics = game analytics

[No accepted definition (working on a standard)]

What are game metrics?

Metrics are measures, e.g.: Average playtime per player Number of ”Swords of Mayhem +5” sold Daily Active Users

% server uptime/stability Avg. network latency

Bugs reported/bugs resolved /day Customer support call avg. length

Players

Performance

Process

Players

Players

Performance

Process

Process

Why game metrics analysis? Big data: populations not samples

Understanding all players Research/development out of the lab and into

the real world

Big depth: Detailed recording of all aspects of play Includes communication, navigation, cross-

games ... Combining GUR data sources for in-depth

research

Challenges

Behavioral telemetry inform what players are doing, only by inference why

Finding the right features to track is not obvious

Managing the allure of numbers

Game Data Mining

Game data mining

Game data mining = data mining of game metrics

Gartner Group: “the process of discovering meaningful new correlations, patterns and trends by sifting through large amounts of data stored in repositories, using pattern recognition technologies as well as statistical and mathematical techniques”

Game Data Mining: approachesCommon approaches in game data

mining:

Description Characterization

DiscriminationClassificationEstimationPredictionClusteringAssociation

Description

Simple description of patterns in data

Accomplished using Explorative Data Analysis

Example: how rapidly does the ”warrior” class advance through levels?

Answers many questions from designers and producers

Description

Drill down/across

Prediction Using a large number of known values to predict

possible future values

How many players will an MMORPG have in 3 months?

When will a F2P break the 1 million player threshold?

When will people stop playing?

One of the most widely used data mining methods in game analytics Persistent world games MMOs F2P

Clustering

Orders data into classes, but the class labels are unknown (unsupervised)

Groups formed according to internal similarity vs. across-group dissimilarity

Subjective element

Problems applying algorithms to game metrics

Player Behavior in Tomb Raider: Underworld

w/ Alessandro Canossa, Georgios Yannakakis, Julian Togelius, Hector

Perez, Tobias Mahlman

Behavior in TRU

Goal: Using gameplay (behavior) metrics to classify the behavior of users

Uses:

1. Comparing behavior with design intent2. Optimization of game design 3. Debugging of playing experience 4. Adaptation: Real-time dynamic adaptation to

player type

Behavior in TRU

Tomb Raider: Underworld (2008) AAA-level commercial title Data from 1.5 million users via Square

Enix Hundreds of variables

Metrics should fit purpose Selected variables fitting key game

mechanics Jumping, completion time, causes of

death …

Behavior in TRUAnalysis:

Clustering algorithms (PCA, k-means) Self-Organizing Map (unsupervised)

Revealed a 4 distinct behaviors (94% users)

Players use the entire design space

Behaviors translated into design terms

Behavior in TRU 8.68% (Veterans): Very few death events

(environment). Fast completion times. Generally perform very well in the game.

  22.12% (Solvers): Die rarely, very rarely use

the help system. Slow completion. Slow pace of play.

46.18% (Pacifists): Largest group of players, dies from enemies. Fast completion time, minimal help requests. Good navigation skills, not experienced with FPS-elements in TRU.

16.56% (Runners): Die often (enemies, environment), uses the help system, very fast completion time

Behavior in TRU

Towards big data:1st study: 1365 players2nd study: 30,000 players3rd study: 203,000 players4th study (in prep): 1.6 million

players5th study (in prep): across

games

From dozens to hundreds of variables

Behavior in TRU

Can we predict when people stop playing?

Use: uncovering design problems; engagement

Approach TRU: 7 levels + prologue 10,000 randomly selected players 7 groups of metrics (400+ variables) Training data: lvl 1

Simple logistic regression best fit: 77.3% (base: 39)

Behavior in TRU Decision trees (prediction)

Use: predicting player behavior; transparent models – ideal for communicating across stakeholders

Level-2 rewards Rewards > 10 ▪ Level-3 playtime▪ -> playtime > 43 minutes : 4 ▪ -> playtime < 43 minutes : 7

Rewards < 10 : 2

Lvl 2 rewards and playtime lvl 3 predictors of quitting

Fun Facts aboutCharacter Names

w/ Christian Thurau

Do paladins always have names like ”Healbot”?

Do Warlocks always have names like ”Ûberslayer?”

Are mages always called ”Gandalf”?

Are there any kind of ?

Character names

Character names7,938,335 WOW characters (5 years

logging)

Name, Race, Class, Playtime, Guild, Server Type, Domain, etc. ...

Some findings

3,803,819

unique names(a surprising lot)

More diverse than real-world names - despite naming constrictions

Looks like naming is important to players – only unique feature you have

RP-characters most diverse (83% unique – rest ~58%)

Any relationships between name and game features?

Class

Any relationships between name and game features?

Race

Races

”Pretty” ”Bestial”

2D-isomap projection (dimensionality reduction technique) ”pretty” races named differently than

”bestial” races Not due to differences in m/f character

ratios

Gnomes and dwarfs named as ”bestial” races?

==

RP vs. PvP/PvE servers

Names on US servers different from EU servers

Except for RP realms (larger overlap btw. EU/US)

Can we predict names?

What is the chance that ”Gimli” will be a dwarf?

Estimated conditional propabilities of a given class/race/server type given a particular character name

Class and Race best predictors, but server type and faction also hints at naming decisions

Some names are very good predictors, others are not -> so yes, Gimli will likely be a dwarf

Sources of inspiration

Sources of inspiration

1000 most common names

128,058

(not a lot, but still 100* bigger than any other study)

38 coding categories foundSome names multiple categories/hard to classify (e.g. ”Raziel”)

Sources of inspiration Regular vanilla real-world names most

common (Sara, Mia, etc.) [186]

Mythology – notably Greek [164] Anubis, Odin, Ares, Loki, Nemesis

Popular culture – games, cartoons, film ... [174] Naruto, Sakura, Tidus, Valeria, Revan, Zelda

Fantasy literature (Tolkien rules supreme) [39] Earendil, Sonea, Morgoth, Aragorn

A lot of names in breach of ToU

Sources of inspiration

697 of 1000 names categorizedRest: Nouns, verbs of unspecified

nature

Semantic nature to categorize:

”Negative”: Nightmare, Sin, Fear, Requiem

”Positive”: Hope, Love, Pure ”Neutral”: Who, Moonlight, Magic, Snow

Sources of inspiration

Names with negative semantic meaning 6 times more common than positive semantic

Are gamers depressed?

Or do ”dark” names just sound cooler?

Perspectives

Lots of ”why”´s unanswered:

Why are certain names more popular than others?

Why do the Mage class exhibit a greater variety of names than other classes?

Why do some players pick names of characters from the same game they are playing?

Need to talk to the players ...

Future

Future

Demand in the IDE industry Unique openness to research-

industry collaboration

Attractive research challenges Complex, mixed-methods, multi-

disciplinary, big data

Future Better methods and algorithms (all forms of

metrics)

Correlating behavior, PX and design Spatial game analytics

User profiling: behavior, personality, motivations ...

Decoding and predicting behavior

Maturing development practices (from joint warehousing to GUR)

”Guerrilla metrics”-methods

Playtime and power laws w/Fraunhofer IAIS

6+ games 5+ game ”types”

Same patterns? 90%+ prediction

Power law: When the frequency of an event varies as a power of

some attribute of that event (session length)

Does all playtime behavior follow a power law?

Thank you

Want to know more? Blog.gameanalytics.com

andersdrachen.wordpress.com [slide deck available here]

IGDA GUR SIG – LinkedIn group, 350+ members

The GUR SIG Mendeley Library – mixed industry/research

GDC archives – industry SOTA

Research publications – ACM, IEEE, Springer digital libraries + new book on game telemetry out 2012