Embed Size (px)
Citation preview
Demystifying DataPALLAS HORWITZSENIOR DATA SCIENTISTBLUE SHELL GAMES
What is wrong with being overwhelmed by data?• Problem:• Only data team members are comfortable accessing and analyzing
the data• Data teams spend too much time pulling data instead of driving
ROI positive insights• Executive team does not value data driven decisions
• Solution:• Identify the analytic needs of every department• Create infrastructure and culture that empowers departments to
perform simple analyses autonomously
Blue Shell Games, LLC
Problem 1: “I can’t access the data”Common Causes:• The visual report is broken or out of
date• The user doesn’t have access to the
database• The user doesn’t know where to find
the relevant data
Problem 2: “We don’t have that data”Common Causes:• Asking the wrong question• Lack of Metrics QA• The requested data is not actionable
Problem 3: “A/B Testing is too hard”Common Causes:• Business intuition leads to the same
conclusions as the A/B test results• Test results are too fuzzy and p-values
are statistically insignificant• Even statistically significant results
won’t change the product roadmap
Solution: Empower everyone to be a data consumer• Identify the needs of
every department• Anyone can do simple
SQL• Data Standardization
Document:• Database Logins• Template Queries
Data
Product
Engineering
QA
Community
Product Solution
Result: • Only the “right” questions are asked
• “Is my feature making money?”
• High Level Statistics• DAU, Installs, ARPDAU, Payers,
Conversion• Trends: DoD, WoW, 2Wo2W,
MoM
• Read Every Spec!
Engineering Solution• “What do I need to
implement?”• Act as translator• Metrics Spec
Result: • Useful data is collected and proxies are available for
missing metrics
QA Solution• “How do I test this?”• A feature is not complete
until metrics are implemented and QA’d
• Setup user friendly metrics logs
Result: • Implemented metrics act as intended
Community Solution• “How do I keep players
engaged?”• Tailor contests to in-game
activity• Have contest entries
queryable
Result: • Marketing decisions are informed by actual player
behavior
Data Solution• “How can I contribute to
the bottom line?”• Naïve customer
segmentation can greatly impact revenue
• Only implement tests that will affect change
User Segment Revenue Delta p-value
non payer -34% 0.45
payer 137% 0.02
small whale 65% 0.16
whale -43% 0.23
all -6% 0.8
Result: • Data improves the bottom line and drives product
change
Pallas [email protected]@gmail.com
