HIGH-QUALITY SOLUTIONS Melbourne | Sydney | Canberra | briSbane | HobarT | adelaide | Singapore

www.revolutionit.com.au

45%

Executive summary

Mobile applications are now a major channel to conduct business, improve marketing communications and reach out to customers. In Australia, Android and Apple reports over 25 billion apps have been downloaded from the App Store since its launch; and Google reports Android app downloads have broken the 10 billion mark and are growing rapidly.

Testing mobile applications uses many of the same quality assurance and testing practices used for other software applications. However, there are important distinctions and challenges to overcome. What’s more, how these challenges are addressed often underpins a mobile application’s success.

Key challengesOperating System Fragmentation: There are six prominent operating systems in the market. In Australia, Android and Apple iOS hold over 90% of the market share. Clearly any testing project needs to test both. However, the major challenge to address concerns the different characteristics and configurations of the different versions of the same OS.

For example: Android 2.3 is designed specifically for Smartphones and does not support soft-touch keys; whereas Android 4.0 can be used in both Smartphones and Tablets and supports soft-touch keys. Likewise, Apple has different features, user experiences and navigations in every iOS version it has released.

To date, Android has released nine major new versions; Windows Mobile has released four and Apple iOS has released seven.

Performance Issues: An app’s performance is often the key to customer satisfaction. An app should not only be fully optimised to handle peak hours effectively, it should also enable users to navigate its pages with minimal time and effort.

Key Challenges in Mobile App Testing

Typically, the best performing apps should deliver:Log-in: 2 to 3 secondsNavigation: Less than 2 secondsBanking Transactions: 2 to 3 secondsRetail Shopping: Less than 4 seconds

Many performance issues are due to bandwidth limits, as well as variable network types – another challenge in mobile app testing.

Network Issues: Users encounter different mobile network orientations moving around the home or office, and while travelling in cars, trains and planes. Apps should handle frequent network changes without their performance or usability being compromised.

Apps can change behaviour during:Flights: In an aircraftData handshake points: 2G converts to 3G or 3G to 4G and vice versaNull area zones: Areas where the network suddenly disappearsLimited service areas: Limited network availability in lifts and tunnelsSession management: App data interacts with a non-responding server

Android BlackBerry RIM

Apple iOS Symbian

Windows Mobile Samsung BADA

Android and Apple iOS have 90% of the OS market share

HIGH-QUALITY SOLUTIONS Melbourne | Sydney | Canberra | briSbane | HobarT | adelaide | Singapore

Network Issues Continued: Therefore, testing needs to address variable bandwidth, offline, flight and sleep-mode scenarios, and to validate user sessions moving between different network conditions.

Device Characteristics: A mobile’s hardware components can make apps behave differently. Therefore, mobile app testing needs to take into account the following device characteristics: chipset, processor speed, RAM, GPS, camera, screen size, touch capabilities and voice control.

Instances where a particular app behaves differently on similar handsets include:

• A Samsung S4 has a Qualcomm Snapdragon chipset with 2GB of RAM; whereas a Samsung Duos has a Qualcomm MSM 7227 chipset with 768MB of RAM

• Apple iPhones have one-touch capabilities; whereas there are three different touch options in HTC, Sony and Samsung mobiles

• The processor speed of Sony Xperia S is 1.4 GHz and does not support 4G; whereas a Xperia Z1 has a speed of 2.2 GHz and does support 4G.

User Experiences: The goal of usability testing is to ensure users can finish a particular task they’ve undertaken with an app. Importantly, they must be able to do so easily and without becoming frustrated. Potential customers may simply stop using an application if it’s not intuitive to use. Key challenges include navigation, layout and exception handling.

A user generally decides in less than 20 seconds whether to use an app or discard it. Therefore, mobile apps need to be designed and tested with user experiences in mind. If an app delivers a poor user experience, organisations can lose customers and revenue.

Recent examples of apps with poor user experiences include:

1. Ikea used the same UI for both iOS and Android – this caused fonts to display poorly

2. A piano app with advertisements displaying in the wrong place causes wrong key presses.

Conformance: Most apps are distributed through open and public app stores. Each has their own set of rules and regulations, from marketplace guidelines to enterprise policies which prohibit types of content.

Apple’s App Store has a 24-point checklist with guidelines to follow before registering the app. This includes use of APIs, non-functionality on iPad, app size, content and duplicity. Google’s Play Store specifies the app purchase system, advertisement policy, content and system interference.

GPRS 2.5G <100 kbps

EDGE 384 kbps

3G 7.1 mbps

HSPA 21 mbps

4G/LTE 100 mbps

Below are the different GSM network types and their data speed:

HIGH-QUALITY SOLUTIONS Melbourne | Sydney | Canberra | briSbane | HobarT | adelaide | Singapore

Security: Mobile applications need to be free from vulnerabilities and protected against data breaches. More than 70% of apps involved in security breaches are either Banking or Retail apps. These apps contain user credentials, personal data and card details. General attack points are data storage systems, caches and platforms. The famous Zeus Trojan on Android affected millions of banking users by stealing one-time passwords sent by banks to authenticate mobile transactions.

Testing apps needs to not only focus on their features but also on verifying the security of their underlying platforms.

Automation Tools: Automated testing provides a mechanism to consistently repeat a test procedure and verify the results. It’s effective for both regression testing and during development (to ensure new features are not resulting in unintended changes to existing features).

Using simulators, automation tools and physical devices at the right time reduces costs and improves application quality. As a rule of thumb, simulators and automation tools should cover approximately 75% of written test cases.

Therefore, a viable toolset needs to satisfy a number of criteria:

• Current and future target platforms

• Script reusability

• Lead time for new OS versions and devices

• Results and reporting

• Integration capabilities

• Service and support

• Costs

ConclusionTo create powerful mobile testing strategies, there are many approaches and techniques to leverage. Revolution IT enables clients to determine the right test strategies and software tools to ensure all success criteria are met for a particular mobile application testing project.

Why Revolution IT?We believe mobile applications need to prove their worth in terms of utility, performance, security and development costs. Therefore development teams need testing capabilities and quality controls in place. This is precisely where our skills and experience in mobile application testing can help.

Our over-arching goals are to enable our clients to build their capabilities and to curb their costs. By testing rigorously and appropriately, we minimise application rework, improve functionality, address load and performance issues, and efficiently secure invaluable business and customer data. Plus we’re partners and trainers for the best testing tools available, including HP, Neotys, Perfecto Mobile, Zap Technologies and other major software vendors.

Revolution IT - Delivering high-quality solutions for high-performance enterprise.At Revolution IT, we don’t just make software work, we make it perform superbly. From planning through to operations, we use proven methodologies to reduce IT spend, meet aggressive timelines and deliver outstanding application quality. What’s made our name in the industry is our underlying focus on quality. To find out more about how our services can benefit your business, talk to us today on (03) 96002566 or info@revolutionit.com.au or info@revolutionit.com.sg

Automated testing provides a mechanism to consistently repeat a test procedure and verify the results. It’s effective for both regression testing and during development.