Slow Mobile Apps Ruin Everything

© 2014 New Relic, Inc | US +888-643-8776 | www.newrelic.com | www.twitter.com/newrelic | blog.newrelic.com

eBook

Slow Mobile AppsRuin EverythingHow to Drive Growth with Better Mobile Performance

http://www.newrelic.com

http://www.twitter.com/newrelic

http://blog.newrelic.com


INTRODUCTION

CHAPTER 1:

CHAPTER 2:

CHAPTER 3:

CHAPTER 4:

CHAPTER 5:

CONCLUSION

03

04

07

10

14

16

19

The State of the Mobile App Economy

Why Yesterday’s Mobile Tools Won’t Cut It

Why Mobile APM Matters and How to Do It Right

The Need for Seamless Integration

Driving Business Growth with Mobile APM

eBook

Table of Contents

Slow Mobile Apps Ruin Everything: How to Drive Growth with Better Mobile Performance




eBook Slow Mobile Apps Ruin Everything: How to Drive Growth with Better Mobile Performance

© 2014 New Relic, Inc | US +888-643-8776 | www.newrelic.com | www.twitter.com/newrelic | blog.newrelic.com03

IntroductionThe mobile app economy is booming, and opportunity is everywhere. By 2017,

Gartner predicts the app economy will reach 268 billion apps downloaded, and

$77 billion in revenue. 1

But the opportunity to succeed is also the opportunity to fail. There are millions

of mobile apps already available for users, with thousands more being launched every

month. Users have their pick of what apps to download, and if your app isn’t

performing, there’s nothing stopping them from deleting your app and moving on.

In fact, it’s been reported that 80 and 90% of mobile apps downloaded are deleted

after one use, and 60% of the apps in the Apple App Store have never been down-

loaded at all.2 These stats are grim even for apps that perform perfectly.

When freezes, crashes, and slow response times occur, however, the results get even

more devastating—these are the top three reasons why users write a bad review.

Not only can these performance issues cause existing users to delete your app and

contribute to bad online reviews, they can prevent new users from downloading your

app in the first place.

That’s why having visibility into the performance of mobile apps—in real time and

from real users—is so critical; it helps solve customer problems and create great

user experiences.

This eBook examines the current state of mobile application performance monitoring;

the challenges this is creating for mobile app development; and how the new forms of

mobile APM are helping developers create better user experiences for customers and

grow their businesses.

1 “Forecast: Mobile App Stores, Worldwide, 2013 Update,” Gartner, September 2013 (https://www.gartner.com/doc/2584918?ref=ddrec)2 “App Store infested with zombie software, claims analytics startup Adeven,” Gigaom, July 2012 (http://gigaom.com/2012/07/31/app-store-infested-with-zombie-software-claims-analytics-startup-adeven)




http://gigaom.com/2012/07/31/app-store-infested-with-zombie-software-claims-analytics-startup-adeven


To understand why mobile APM is so important, and so necessary for app developers, let’s first take a deeper look at the drivers and potential roadblocks in the mobile app economy.

CHAPTER 1

The State of the Mobile App Economy






The mobile app space is one of the most healthy and growing parts of the tech industry

today. Demand for mobile devices is only increasing—from 5.7 devices per household

in 2013, to 6+ mobile devices per household in 2014.3 More people than ever before will

be able to afford their first mobile device this year. And there are more types of mobile

devices than ever to choose from, whether old standards like smartphones and tablets,

or new wearable smart watches and glasses.

One of the main reasons for the popularity of all these mobile devices is mobile apps.

Demand for mobile apps is booming in every aspect of life, from entertainment to

the enterprise.

6+ MOBILE DEVICESPER HOUSEHOLD IN 2014

1.2 BILLION PEOPLE AROUND THE WORLD USE APPS

4.4 BILLION PEOPLE PROJECTED TO USE APPS IN 2017

$77 BILLION IN REVENUE

BY 2017, GARTNER PREDICTSAPP ECONOMY WILL REACH

Currently, around 1.2 billion people around the world use apps. This number is estimated

to increase 29.8% each year, reaching 4.4 billion users in 2017.4 The global app

economy is estimated to be growing at a rate of 28% between 2012 and 2016. Clearly,

the demand for apps is high and is only going to climb higher.

Although the majority of app downloads are free apps, app developers are finding

a way to make money through channels such as in-app purchases and mobile

advertising, as well as contract development. Contract development may be an

especially viable income stream for app developers, because companies in the app

economy currently employ an average of 4.5 mobile developers, and this number

will likely grow in the years to come.

3 “2014 Trend Predictions for the Mobile Web” Netbiscuits Blog, November 2014 (https://www.gartner.com/doc/2126015/market-trends-mobile-app-stores) 4 “Fast growth of apps user base in booming Asia Pacific market,” Portio Research, March 2013




https://www.gartner.com/doc/2126015/market-trends-mobile-app-stores



But despite the opportunities in the app space, there are also some big challenges

that lie ahead for mobile developers.

The Fragmentation Problem One of the biggest problems in the mobile app development space is that you can’t

just create one version of your app for all devices. According to a study from Flurry,5

you’ll need to support 331 device models to reach 90% of active users. You’ll need to

support 156 device models to reach 80% of users, and 37 device models to reach 60%

of users. And this study only covers smartphones and tablets, and current OSes such

as iOS, Android, and Windows.

New OSes such as Mozilla’s mobile Firefox OS, a mobile Ubuntu OS, and Sailfish,

a proprietary OS created by former Nokia engineers, are already anticipated to launch

this year. With new smart wearable tech soon to be available, the fragmentation

problem is only going to get exponentially worse.

Currently, most developers are not very aware of or don’t have access to metrics that

will show them data sorted by device, OS, and carrier, yet those are exactly the kinds

of metrics app developers need in order to pinpoint exactly where problems are arising

when apps crash, freeze, or have slow response times.

According to Paul Ryan, CTO at Yellow Pages Group (YPG), some of the greatest

mobile challenges his company has faced directly correlated with the fact that

they were supporting every major mobile platform. YPG wanted to get in front

of as many users as possible, but the company found that troubleshooting across

all those different platforms was extremely difficult for them, especially as the

live versions behaved in ways they’d never seen during testing.

When you are supporting an app on multiple devices, carriers, and OSes, mobile

APM becomes that much more important to the success of your app. But as you will

see in the next chapter, many of the methods developers are using to currently

monitor mobile app performance are just not enough.

5 “Are Indie App Developers Becoming an Endangered Species?” Flurry Blog, March 2013 (http://blog.flurry.com/bid/94811/Are-Indie-App-Developers-Becoming-an-Endangered-Species)




http://blog.flurry.com/bid/94811/Are-Indie-App-Developers-Becoming-an-Endangered-Species


While the booming mobile app economy provides a world of new opportunities for app developers (as explored in Chapter 1), it also provides a number of new challenges for developers to deal with, especially in the performance space. In the following pages, we’ll explore the two most popular methods of mobile performance monitoring and why they fall short in today’s complex mobile environments.

CHAPTER 2

Why Yesterday’s Mobile Tools Won’t Cut It






Since mobile app development is relatively new, developers are still trying to get

a handle on how to best support users. To them, the concept of mobile APM is

still quite alien. The most common methods of mobile app performance monitoring

today are crash reporting and user testing.

These tools do provide valuable data and should not be discounted. It’s definitely

important to understand when and how your apps crash, and how you can resolve

these issues. But any normal app that has been built well probably crashes only 1% of

the time. What’s happening during the other 99% of the time when the app isn’t

crashing? Given that most users will simply delete your app and move on if

it fails to perform well, fixing issues after the fact just isn’t enough.

Many times, developers have to wait to hear complaints from users to find out that an

issue is occurring—and by that time, multiple users have probably left one-star ratings

and bad reviews for the app in the App Stores, which can be greatly damaging to

a company’s brand and reputation.

When a company gets a one-star review in one of the app stores, they generally need

at least a couple of five-star reviews to counteract it, according to Pierre-Luc Simard,

CTO at Mirego, a mobile design and development firm. “Waiting for users to complain

can be a very damaging approach to testing a mobile application,” says Simard.

To be able to provide stellar performance, there is a strong need to understand and

have visibility into how users are interacting with an app. Being able to dissect the

information and figure out which parts are hogging most of the time is key to being

able to optimize an app for best performance. With most mobile APM solutions

today, there’s a lack of visibility into user interactions. Developers can’t really see what

happens on the backend because the necessary metrics aren’t available. And this

visibility just happens to be one of the most important aspects of mobile APM that

developers have still not fully caught on to.

Crash Reporting As mentioned earlier, most well-built apps probably spend only 1% of the time

crashing. Being able to get visibility into this data is excellent because it provides

developers with valuable insights to help reduce the number of crashes, and

provide an optimal end-user experience from that aspect. But what about

the other 99% of the time when an app is not crashing?

RunKeeper was using several different tools to monitor the performance of its app:

Crashlytics for crash reporting and analytics, Google Analytics for product analytics,

Modit for servers and Pingdom for the website. But even with all these tools, RunKeeper

still wasn’t getting the visibility into its app performance that it needed to diagnose

and fix problems, according to Joe Bondi, Co-founder and CTO of RunKeeper.

It is very important to have deeper visibility into what’s happening when your app

is not crashing as well because users don’t want apps that show never-ending

spinners, or take too long for screen loads, or throw errors in the backend that the

developer has no visibility into. Developers don’t want to be in a ‘you don’t know

what you don’t know’ situation.

User Testing User testing is another valuable tool developers use to prevent problems with their apps,

but the problem with user testing is that it’s almost impossible to test every edge-case

and catch all the bugs in your app before launching. Synthetic performance monitoring

allows developers to test their apps on multiple devices running different carriers in

different parts of the world, but there is nothing like the real deal – real users, scattered

in various different regions, running all these use cases that developers probably never

thought about. Problems can’t be detected if developers don’t know what to look for

in the first place. If you can’t simulate the error, it can be impossible to fix.






At Unwired Revolution, a pilot end user reported an increase in ‘server errors’ when

using Unwired’s mobile app, but the company couldn’t track the issue or reproduce

it in testing. They ended up having developers comb over log files and debug traces

to try and find the problem, wasting many hours of developer time that could have

been much better spent.

But even when you are able to detect the bug, it can be just as difficult to then track

down the root cause. Unwired Revolution found that when doing user testing for

enterprise on-premise solutions, that it was extremely difficult for the company to

anticipate and simulate the environmental factors that impact integration. And those

environmental factors can lead to unique issues that can take a very long time to debug.

Traditional methods of error diagnosis and performance management simply cannot

keep up with the many variables of mobile app performance. There are too many

users, too many interactions, too many OSes and device types out there, and not

enough time to test them all.

Mobile application testing can be a lot of guesswork, based on the limited evidence

at hand. And there’s currently no way to send out a new version of a mobile app to

a small subset of users. When new mobile app versions are uploaded to the Google

Play or Apple App Store, every single user gets that new version, and can see how

buggy it is. The only way you can get feedback on the version is from reviews posted

on the app stores. “It’s like driving blind, except you don’t know if you’ve wrecked

the car until weeks afterward,” says Mirego CTO Pierre-Luc Simard.

Developers and app providers need to be proactive about detecting problems with

their apps, and fixing those problems before users even have a chance to notice.

In today’s competitive landscape, developers can’t afford to be left behind.





Mobile APM is a crucial part of a business’ app strategy in the competitive app market of today. In the following pages, we’ll explore why good mobile APM is so important, and how to monitor the performance of your app the right way.

CHAPTER 3

Why Mobile APM Matters and How to Do It Right






Good mobile APM goes beyond just crash reporting, waiting for customer complaints and feedback, and checking app stores for poor reviews and issues that customers are talking about.

You need real-time, real-user information in order to offer real-time solutions to your customers and to create a better user experience.

More often than not mobile apps interface with at least one if not more backend web services to access information. You can have lots of issues at this stage of the process.

And with the more common methods of mobile APM like crash reporting, if instances are taking too long to respond on the backend or if a screen is taking too much time to

load (and doesn’t result in a crash), there’s no way to find out what actually happened at the code-level. But with newer forms of mobile APM, you can trace the instance all the

way from the device through the network and to the backend service, and pinpoint exactly where the problem is.

A good mobile APM solution will, among other things, offer real-time, real-user monitoring of your app, offer a variety of metrics, be able to compare different versions of

your apps, and provide the ability to trace an instance from end to end.






Here’s a closer look at some features that a good mobile APM tool should include:

HTTP requests Your app may not be what’s slowing you down—the APIs may be to blame. Look for

a mobile APM tool that can monitor all the interactions your app has with external

services, and give visibility into how web services and API requests are performing.

For API-intensive apps, media or animation-heavy apps, it’s especially important

to be able to see response times, data transfers, and throughput.

HTTP errors With your mobile APM solution, you should be able to get visibility into HTTP request

errors and network failures, and track error rates by error type and domain. Additionally,

you should be able to drill down into details to find the cause of individual errors. This

way you can quickly work to resolve the issue and make sure your API or media-

intensive apps—and compliance-sensitive apps, in particular—are working optimally.

Carriers & geography When you’re managing large-scale distributed apps, you want a mobile APM tool

that can break down performance issues by carrier and region. If your app is slowing

down only on certain carriers or in certain regions, you need to know, in order to

fix the problem. This type of visibility is especially useful for development teams in

charge of external and internal enterprise apps.

Version analysis You need a mobile APM solution that will let you compare different versions of your app,

so you can see right in front of you if the changes you made to the newer version of

your app are actually creating a better user experience, or if unexpected interactions

are causing poor performance. Version analysis is especially important for consumer

apps that are custom designed or have media-rich or animation-heavy content, as

well as for apps that need frequent updates to refresh dynamic content or to keep up

with OS updates.

Device-level metrics An efficient mobile APM solution will provide insight into the app’s device-level

data, allowing developers to track how their local databases, device memory and

the UI are affecting the app’s performance. This helps developers identify the right

problems in their gaming or animation-heavy apps, for example, and solve them

immediately.






Custom metrics An effective mobile APM solution recognizes that each mobile app has its own unique

requirements, and will allow you to custom instrument and monitor any specialized

metrics based on functionalities unique to your app.

Potential use cases for custom metrics include wearables/smart object-integrated apps

and business apps with specialized transactions. Custom metrics are especially useful

for monitoring code that the mobile APM tool can’t normally detect, monitoring code

not controlled by your team, monitoring scripts, and monitoring custom events.

Interaction traces To truly pinpoint where performance issues are occurring, knowing how your users

are interacting with your app, and how that is affecting performance is a must. With

newer mobile APM features like User Interaction Traces you can trace every part of

how a user is interacting with your mobile app, from end-to-end. A user interaction

trace collects data on memory usage, database metrics, CPU time, and network

response times and errors. It also includes method level-traces for each individual task

that an interaction is comprised of thereby giving developers access to code-level

information. Having access to user interaction traces is especially important if your app

is custom-designed for a niche function, if it’s API intensive, or it makes heavy use of

media and animation.

Mobile device profiles Tablets, smartphones, wearable tech—as more and more mobile devices hit the market, it

will be even more important for app developers to have a good mobile APM solution that

provides device performance metrics, so developers can compare adoption across

different devices, see which devices their apps work well on, and which ones have

a tendency to be problematic. A good mobile APM solution will also let you track

how your apps are performing on different operating systems, and tell you where

you can optimize for best performance by monitoring response times, throughput,

error rates and user interaction times.

Get the full storywww.newrelic.com/case-studies/runkeeper

Mobile fitness platform RunKeeper uses smartphone

location technology to help fitness enthusiasts track,

measure, and improve their progress toward their fitness goals. The app runs on both iOS

and Android and has more than 27 million users worldwide.

As RunKeeper has increased in popularity, app performance has become increasingly difficult

to predict. RunKeeper used a number of performance monitoring tools, but a significant gap

remained in the company’s ability to understand the root cause of emerging issues. It wasn’t

until RunKeeper implemented a real-time monitoring solution that it was able to address

those issues and improve customer experience.

Mobile Use Case: RunKeeper




http://www.newrelic.com/case-studies/runkeeper


Mobile APM is a crucial part of your overall app performance monitoring strategy. But it’s only one piece of a larger picture. In the following pages, we’ll explore why it’s important that your mobile APM integrates seamlessly with the rest of your app monitoring environment.

CHAPTER 4

The Need for Seamless Integration






Good mobile APM needs to have seamless integration with the rest of your monitoring

environment—web, server, network—as all these pieces complement each other and

affect each others’ performance.

Application performance monitoring not only includes the mobile app itself but multiple

external pieces, such as web services and servers that interact with and influence your

app as well. Different metrics are being monitored in each component, and they talk to

each other.

When you have integration between these different areas of performance management

and monitoring, it helps zero in on exactly where performance problems are that much

more quickly.

A good APM solution will allow you to monitor the performance of each of these

pieces and cross-trace across them easily to help identify how they are influencing

the performance of one another in order to provide a good end-user experience.

Ideally you’ll want a mobile APM tool that will integrate well enough with your web APM

tool to show both on one single-view platform. You’ll be able to see the entire picture

of your mobile and web apps working together, and from there, implement end-to-

end solutions more quickly and more effectively. You’ll also be able follow the greater

performance trends of your entire app infrastructure, and work to deliver a better

experience to all your customers.

Unwired Revolution is a mobile solutions integrator

specializing in security, management, support,

connectivity, and application enablement for enterprise

mobile initiatives across all major industries. In recent

years, the company has turned much of its attention

toward the development of custom mobile apps that help clients’ internal teams collaborate

more productively and effectively.

With a good mobile APM tool integrated with its other monitoring solutions, Unwired

Revolution was able to get clear visibility into bottlenecks, helping determine how to

allocate subsequent development cycles to improve app performance, and greatly reduced

the time needed to diagnose and fix interoperability issues.

Mobile and Web Use Case: Unwired Revolution

Get the full storyhttp://www.newrelic.com/case-studies/unwiredrevolution




http://www.newrelic.com/case-studies/unwiredrevolution


Mobile app performance directly affects the user experience, which means it also directly affects a business’ bottom line through retail performance, ROI, and brand perception. Companies who embrace mobile APM can reap a number of benefits, including faster problem diagnosis and resolution, improved customer experiences, and saved developer time that can then be turned toward innovation.

CHAPTER 5

Driving Business Growth with Mobile APM






Web performance has become a key priority in small companies and enterprises alike.

As enterprises, in particular, increasingly embrace mobile, mobile APM will become an

enterprise priority as well. Here are a few key reasons why it’s a smart move to invest in

mobile APM.

Faster problem diagnosis With mobile APM, companies can identify problems with their apps in real time before

a large part of their end-user base even becomes aware of them. Faster diagnosis

leads to faster resolution of app performance problems. Developers don’t waste time

in going through trial-and-error phases to locate the right problem—the problem is

immediately and clearly laid out to them including code-level details. This significantly

reduces the mean time to resolution of problems and saves precious developer time.

Better use of internal resources A company using a good mobile APM tool will eliminate unnecessary support and

development team hours spent on diagnosis and repair. The support team will have

access to information that helps them diagnose exactly what problems users are

experiencing, and the development team will have access to data that helps them

solve these problems quickly. Those formerly lost hours can then be used towards

innovation in developing new features, improving existing functionality and creating

new features that make the app more beneficial to customers.

More satisfied customers Businesses who proactively monitor their apps can ensure that customers receive

a consistently excellent experience every time they use the app. Satisfied users

lead to better brand reputation and app store reviews, both of which are crucial in

attracting new customers.

Better brand perception When your customers are happy with their experience, it shows—in better online

reviews, greater referral rates, and overall better brand perception as your app earns

a reputation for reliable performance.

Greater ROI The above factors—better use of internal resources, more satisfied customers

and better brand perception—combine to yield a greater return on investment to

any company that invests in a quality mobile APM tool, and implements the results

effectively. A good mobile APM tool also helps companies minimize revenue loss by

reducing mean time to resolution of problems that could directly result in revenue loss;

for example, a purchase transaction that fails to work for no apparent reason.

MOBILEAPM

GREATERROI

FASTERPROBLEM

DIAGNOSIS

BETTERUSE OF

INTERNALRESOURCES

MORESATISFIED

CUSTOMERS

BETTERBRAND

PERCEPTION






The True Cost of Customer Dissatisfaction

When mobile users have a poor experience using your app, the consequences can often be devastating. Here’s why:

• 91% of unhappy customers won’t choose to do business with your company again6

• The average cost of attracting new customers is five times as much as it takes to keep current ones7

• On average, one unhappy customer will tell between eight and 16 people about their bad experience, and 20% of unhappy customers will tell more than 20 people8

• With an assumed influence level of 25%, that translates into four lost customers for every customer dissatisfied with their experience

6 TARP 7 Lee Resource Inc8 Lee Resource Inc.






ConclusionAs the mobile app economy grows, and more and more apps become available to users, app developers will need better methods of mobile APM in order to stay competitive

in the marketplace. App developers who rely only on old methods of mobile APM risk losing the good will of current customers and alienating potential new customers with poor

performing apps. But with the new mobile APM tools available, there’s no reason for app developers not to have access to real-time monitoring of app performance, giving them

the ability to provide a quality app experience to users.

Mobile APM AssessmentAre you concerned about your current method of mobile APM? Take our test below to see if it’s time for a new solution.

Did you answer “No” or “Not Sure” to any of these questions? If so, now’s the time to start looking at your options.

For a good place to start, visit: www.newrelic.com/mobile.

QUESTION

Are we measuring real end-user experiences on our mobile app today?

Are we meeting customer SLAs?

Is our app delivering consistent response times across devices, carriers, operating systems and geographies?

Is our app delivering a consistent user experience, regardless of device, operating system or carrier type?

Can we trace the performance of a particular user interaction with our app from end-to-end?

Can we see how the local database, memory or CPU is affecting the app’s performance?

Yes No Not Sure




http://www.newrelic.com/mobile



About New RelicNew Relic is a software analytics company that makes sense of billions of metrics about millions of applications in real time. Our comprehensive SaaS-based solution provides

one powerful interface for web and native mobile applications and consolidates the performance monitoring data for any chosen technology in your environment. Our 90,000

customers use our cloud solution every day to optimize more than 200 billion metrics for 3 million applications. When your brand and customer experience depend on the

performance of modern software, New Relic provides insight into your overall environment. Learn more at newrelic.com.

New Relic, San Francisco HQ188 Spear Street, Suite 1200

San Francisco, CA 94105

New Relic, Portland111 SW 5th Avenue, Suite 2800

Portland, OR 97204

New Relic, Seattle2101 4th Avenue, 19th Floor

Seattle, WA 98121

Tel: +1.888.643.8776

[email protected]

www.newrelic.com

New Relic, Dublin34-39 Nassau Street, 3rd Floor

Dublin 2, Ireland




http://newrelic.com

mailto:support%40newrelic.com?subject=


©2008-14 New Relic, Inc. All rights reserved.