View
27
Download
0
Category
Preview:
Citation preview
© 2016 Apteligent • www.apteligent.com
What Data Science Can Teach You About App Performance & User Experience • March 2017
Mobile App User Experience Myths, Debunked
© 2016 Apteligent • www.apteligent.com
Agenda
Myth #1: Crash Data is All I Need
Myth #2: Release Planning Happens Once
Myth #3: User Behavior & Business Insights are separate from App Performance
Finally: Where is this data coming from?
Andrew LevyCo-Founder & Chief Strategy Officerandrew@apteligent.com@andrewmlevy
© 2016 Apteligent • www.apteligent.com
Myth #1:
Crash Data is All I Need
Cras
hes +
Erro
rsNet
work P
roble
msSlo
w Perfo
rman
ceDev
ice Pr
oblem
sUI Is
sues
User C
ontex
t
Intractable Number of Permutations of Issues Impacting User Experience
Even if you don’t buy into this, what can data science teach us about the cause of crashes?
© 2016 Apteligent • www.apteligent.com
20% of Crashes are Correlated with a Network Issue
Network issues account for a large percentage of crashes
© 2016 Apteligent • www.apteligent.com
88% of Network Calls Were Successful Before a Crash
Mobile developers are typically careful to detect and handle a failed network call.
What seems counterintuitive is that the majority of crashes happen after successful network calls
© 2016 Apteligent • www.apteligent.com
10% of Successful Network Calls Before a Crash Returned No Data
89% of the time speeds were lower than expected72% of the time latency was lower than average
Since latency was lower than average, we would expect the speed to increase. Since the majority of calls were successful, the lower speed must be driven by a lower than expected bytes received.
bpl (speed) = (bytes out + bytes in) / latency
© 2016 Apteligent • www.apteligent.com
Myth #2:
Release Planning
Happens Once
• Where are we relative to our competitors?• Gather relevant KPIs & performance benchmarks based on geography and App Store
category
• What are currently the most popular hardware & software configurations?• Which platforms should you support and how will your app perform in those environments?• Plan for new device and operating system launches
• Understand global bandwidth constraints
• Choose service providers and SDKs for analytics, performance, and monetization
Take a Data-Driven Approach to Drive Your Experience Strategy
© 2016 Apteligent • www.apteligent.com
Example: Launch and Android Shopping App in India
Key Performance, Device & OS Metrics. Sample pulled from Apteligent Release Planner
• 60% of Android Shopping apps have a launch time of < 2.0 seconds
• The overall average launch time is 3.09 seconds. Android Shopping apps have an average crash rate of 3.38%, this is 35% lower than the global average of 5.22%
• 52% of apps in this category have less than < 1.5% crash rate
• The most popular operating system is Android 6.0 with 25% usage share.
• The top 3 devices in India make up 12% of the market.
• Of the top 20 devices, the worst performing by latency are: Samsung Galaxy Note 3 (675 ms), Intex Cloud Q11 4G (649 ms), and the OnePlus 2 (599 ms).
© 2016 Apteligent • www.apteligent.com
New Device Releases 2H 2016
© 2016 Apteligent • www.apteligent.com
Compare Yourself Against Relevant Competition & Prioritize Dev Time
89% of the time speeds were lower than expected
© 2016 Apteligent • www.apteligent.com
Myth #3:
User Behavior &
Business Insights are
Separate from App
Performance
Understand Why User Behavior Changes and the Relationship Between Performance and Business Metrics
Users drop at every step of the customer acquisition funnel
© 2016 Apteligent • www.apteligent.com
Poor Performance Drives a Huge Increase in Churn
• Crashes increase churn by as much as 6x
• Users are up to 8x less likely to return the next day after a crash
• Revenue critical App Store categories like Shopping and Finance are very susceptible
Rolling Retention
Nth Day Retention
“Hard Churn”
Day-1 Retention: “Did they come back the next day?”
© 2016 Apteligent • www.apteligent.com
Relationship Between App Loads and Crashes : Churn
© 2016 Apteligent • www.apteligent.com
Example mCommerce Scenario
It is much more expensive to acquire a new user than to retain an existing user, anywhere from 5x to 25x
• Customer acquisition cost is $50 x 600 means $30,000 just to win back another 600• Assume $300 LTV, you just lost $180,000 in future revenue• If you had a goal to gain 300 customers, and you’ve lost 600, you’ve just tripled
your customer acquisition cost to reach that goal
Quickly isolated the issue, spent X hours of an engineers time (and $Y dollars, much less than the numbers outlined above)
A crash just caused your app to lose 600 customers
© 2016 Apteligent • www.apteligent.com
About Apteligent
Data Generated from Tens of Thousands of Apps
Userflow Failure Network Services Error Crashes Slow apps
Apteligent Data Platform
© 2016 Apteligent • www.apteligent.com
About Apteligent
Data Generated from Tens of Thousands of Apps
Transaction Failure Network Services Error Crashes Slow apps
Apteligent Data Platform Optimize Business Result
Get Ecosystem Insight
Manage App Lifecycle
• Retention/Churn
• Detect Issues• Determine root
cause• Assess impact
• Prioritize• Manage releases• Monitor vital signs
• Benchmarks and trends
• Carrier QoS
• Device adoption• Bespoke analysis
• Engagement• Monetization• Data ingestion
• Correlation• Prediction• Recommendation
© 2016 Apteligent • www.apteligent.com
Feel free to contact me:andrew@apteligent.com
Recommended