View
117
Download
2
Category
Tags:
Preview:
DESCRIPTION
These are the slides from my presentation at the excellent AI/Game conference in Vienna July 2014 (http://gameaiconf.com/), covering two topics: 1) a broad introduction to the practice of game analytics; 2) a description of how to use cluster analysis to build profiles of players based on game telemetry.
Citation preview
Game AnalyticsUnderstanding
What is analytics?
Analytics is the process of Business Intelligence
Analytics is the process of discovering and communicating patterns in data towards solving problems in business Supporting decision management Driving action Improving performance Research & Development - Or for purely frivolous and artistic reasons!
game analytics
Game analytics = subdomain of analytics: game development and game research
The game as a product: user experience, behavior, revenue, system …
The game as a project: the process of developing the game
Evidence-driven support for decision making
Levels of application
Strategic GA: The global view on how a game should evolve based on analysis of user behavior and the business model. Defining a monetization model, scoping DLC
Tactical GA: informs game design at the short-term, A/B test of a new game feature, prediction, profiling
Operational GA: Analysis and evaluation of the immediate, current situation in the game. Removing a bug, adapting game to user behavior in real-time,
reacting to cheating/piracy
Different types of data/metrics
User
Vastly most common data source!
User
User metrics Metrics related to the users, viewing them as either customers (revenue sources) or players (behaviors)
Metrics related to the game system interacting with players (e.g. AI director)
Metrics related to artificial agents behaving as players (bots, mobs, etc.)
3* goals with user analytics
ARPU, DAU, MAU -> customer focus -> revenue goal
Avg. Playtime, Completion Rate -> player focus -> user experience goal
Navigation, strategies, responses, agent behavior -> system focus -> game optimization
Game Analytics in practiceThe Knowledge Discovery Process in GA
Attribute definition
Data acquisition
Data pre-processing
Metrics developm
entAnalysis
and evaluation
Visualization
Reporting
Knowledge deployme
nt
GA Knowledge Discovery Cycle
Answering questions
Hypothesis testing
Confirming ideas/looking for confirmation
Example: classification
Problem: require existing knowledge/theory
Explorative analysis
We do not know what is going on/want to find out what is going on
Example: clustering
Problem: “feature creep” -> resource demanding
Answering questions
In practice often …
Mixture of explorative and hypothesis-driven (and highly iterative)
Also consider:
Descriptive stats enough? (KISS)Spatial – non-spatial solution?
Sampling or not?Static vs. dynamic deliverable?
Which behaviors to track?
Making decisions
Predefined metrics (DAU, MAU…)? Useful for KPIs Cannot stand on their own Says nothing beyond themselves (no exploration) Emergent patterns: temporal dynamics
“Top-down and core out”: user experience, system, revenue Common mechanics -> genre/game specific Core mechanics -> peripheral mechanics
3* general advice
Law of Diminishing Returns
Look at players and system together
Keep an eye out for unexpected behaviors (clustering)
Profiling via cluster analysisUnexpected Behaviors
Lots of
data
Clustering
Dealing with high-dimensional, (massive) datasets
Clustering is used for reducing dimensionality and finding commonalities
Explorative: What is going on? Why? The most common questions in game analytics …
The goal
Formal: Group objects so that intra-cluster similarity is high and inter-cluster similarity is low.
Practice: Interpretable clusters that accurately encapsulate player/AI/system/X behavior
Example: k-means clustering
Applications
Compare and benchmark games
Understanding player behavior Evaluate if there is sufficient variety in gameplay Detect churn behavior and link with root causes Discover what playstyles people use Detecting extreme behaviors
Applications cont.
Develop profiles for adaptative systems Skill profiling -> then cater to them via AI Experience management/experience personalization Finding the factors most important to characterize user
behavior
Monitor development in players´ profiles to track behavior changes: target novice -> expert▪ Useful for evaluating learning curves fx.
Example: BF2BC2
SIVM: finding extreme profiles
Assassins Veterans Target dummies
Assault-Recon Medic-Engineer Driver Assault wannabee
Example: BF2BC2
Each different playstyles, and different things that keep them in the game:
”Driver”: drives, flies, sails – all the time and favors maps with vehicles
”Assassin”: kills – afar or close – no vehicles!
”Target dummies”: unskilled novices – high dropout unless they quickly transfer to another cluster
K-means example: Glitch
K-means example: Glitch
Foundations
Many algorithms tailored to different problems in different fields What a ”cluster” is varies depending on the model
Any assignment of objects to clusters can be either hard or soft.
Hard: Player belongs to the “Rainbowdash” cluster Soft: Player belongs 73% to “Rainbowdash”, 27% to “Fluttershy”
Foundations
NOT an automatic process – iterative procedure, human decisions
Same data can lead to different outcomes depending on algorithm and parameters
No “correct” algorithm
Established model can (potentially) be automated
Algorithm types
Hierarchical clustering : agglomerative methods based on proximities K-means: popular, simple, intuitive, but … Clusters represent averages and are not always
interpretable in terms of the behavior of “real” objects/players
Centroid clustering: represent clusters in terms of central vectors which do not need to be actual objects
Algorithm types
Distribution-based clustering: uses statistical distribution models such as Gaussian mixtures. Clusters reflect how likely it is that objects belong to the
same distribution.
Density clustering: determines areas of high density and applies local density estimators so that clusters (i.e. regions in data space) can have arbitrary shapes.
Finding the right algorithm
1) Are the data high-dimensional and/or sparse?
If so, consider models tailored to sparse data (AA or NMF).
Finding the right algorithm
2) What is the overall goal? To build general models of behavior or to detect extreme behaviors (e.g. cheating, gold-farming)?
For the former, consider centroid-seeking models (k-means, k-medoids)
For the latter, consider models such as AA.
Finding the right algorithm
3) Are the data numerical or relational?
For the latter, use spectral clustering or kernel methods.
Finding the right algorithm
4) Are the players tightly grouped in the data space?
If so, k-means might have difficulties distinguishing them.
If so, consider density-based approaches that do not operate on Euclidean distances.
Finding the right algorithm
5) Are the data noisy?
If so density-based methods might be appropriate as they are better tunable
(but also require more knowledge/expertise)
Key challenges in clustering
Validation: validating clusters
Interpretation: what do the clusters signify?
Time/progress-dependency: players change behavior and progress
Data type mixing: normalization generally advisable
Would you like to know more?
Handout: Reference list covering the topics of this presentation(at reception desk, also on andersdrachen.com)
Clustering Game Behavior Data – a more detailed guide (printed copies at reception, arxiv.com)
Introducing Clustering I-IV – a beginners guide (blog.gameanalytics.com; gamasutra.com)
anders@gameanalytics.com / @andersdrachen
“You cannot improve what you cannot measure”
Lord Kelvin
Recommended