Stop guessing, start testing – mobile testing done right - Timo Euteneuer

About Me

TestObject GmbH Co-Founder I Sales

Mobile Quality Crew’s Co-Founder I Organizer Contact: @testobject I timo@testobject

Session

1)  Mobile app testing: Why it is important

2)  Challenges

3)  Different testing approaches

4)  Automation as key factor for success

Familiar?

48 % of users won‘t use an

app again after encountering a

major issue!

Source: Compuware Global Study Mobile Apps 2013

Better Find Bugs Before Users Do

Bad Quality = Less Revenue

Quality

Ratings

Downloads

Revenue

App Testing Challenges

Device Fragmentation

http://opensignal.com/reports/2015/08/android-fragmentation/

OS Fragmentation

http://opensignal.com/reports/2015/08/android-fragmentation/

Screen Size Fragmentation

http://opensignal.com/reports/2015/08/android-fragmentation/

Android iOS

... More Challenges

§  Mobile means your users are not in a protected environment

§  Network (Wifi, 3G, GPRS ...) §  Sensors, Bluetooth ...

Learnings

§  Your users are not very patient

§  Quality has direct impact on apps‘ success

§  Challenges: Fragmentation and mobile

use of apps

§  Implication: Mobile QA is a central part of development

There must be an easy solution to handle this ?!

...nope!

What Do I Have to Test?

§  What to test: §  Usability §  Functionality §  Performance §  Edge Cases

§  Approx. 50 use cases (per platform) §  Approx. minimum 20 target devices

Testing Framework

Development QA Production

Unit Tests

Functional Tests

Usability

Sanity

Monitoring

Automation

Manual Edge Cases

Checkups

Manual Approaches

§  Checkups, Usability, Edge Cases, Performance §  Pros: You can test the actual user behavior

§  Cons: Time consuming, inefficient

Continuous Delivery = Continuous Testing

Build

Test

Deploy

Enhance

UI Automation Approaches

§  Regression, End-to-end, Integration §  Pro: High test coverage, scalable, time-to-market,

releases resources for manual testing

§  Con: Initial investment, requires technical know-how, maintaining of scripts, no edge cases

Automated Testing

ROI

Investment

Test Cycles

Resources

UI Automation Components

§  CI & Build: Jenkins, Bamboo, Travis CI, Gradle, Ant, Maven

§  Scripting Tool: Proprietary or Open Source Tools §  Infrastructure: e.g. Spoon, Selenium Grid,

SauceLabs, TestObject ...

UI Automation Frameworks

iOS Android Web Language

Calabash

x x - Ruby

Robotium - x - Java

Espresso

- x - Java

Appium

x x x Java, JavaScript, Python, Ruby ...

Code Example Appium

Example Test Execution

Learnings

§  Quality is crucial but mobile QA is complex

§  Different automation and manual testing approaches are required

§  For decent test coverage and time-to-

market UI test automation is key

Thank you!