CI/CD for Mobile Apps

Streamlining the Dev, Test & Deliver Cycle in Enterprises

Today’s Speakers

Tal BarmeirCEO

Experitest

Ton BensimhonHead of Product

Experitest

Ruth ZamirMarketing Director

Experitest

22

Agenda

3

QA challenges in implementing mobile CI/CD in enterprises

CI/CD best practices for mobile app testing

Case study

Mobile market velocity and

variety impacts development

• Competition drives frequent app releases

(2 weeks, B2B and B2C)

• Device matrix large and constantly evolving

• High cost of error - App/play stores create a

distribution gap that makes rollback

impossible

4

High stakes make enterprises rethink the way they operate

5

Continuous Integration:

Frequently merging code to a mainline:

• Breaking down development effort into

bite-size chunks

• Ensuring code works

• Fast feedback while things are fresh

What it CI/CD?

Continuous Delivery:

Ensuring that an app is ready for

deployment:

• Packaging

• end-to-end testing

6

Poll

7

But there are challenges…

Source: Capgemini world quality report 2016

8

The CI/CD Pipline

Build

Validation

Deploy to test

environment

Testing - Automated

Unit testing

Verification - Automated

testing (functional, non

functional)

Deploy to

Prod.

Coding Integration &

debuggingDebugging

Measurement

and feedback Go!

Agenda

9

QA challenges in implementing mobile CI/CD in enterprises

CI/CD best practices for mobile app testing

Case study

Implement a mobile device cloud allowing remote access

to a wide selection of devices, emulators and simulators

Create a common test environment for developers and

testers, complete with collaborative tools

Integrate your test environment with CI tools and case

management systems for end-to end hands free automation

Implement continuous testing with large scale test

automation, covering all mobile app test cases

Centrally manage devices, applications, projects and

users

Leverage end-to-end QA data analytics for rapid root

cause analysis and constant QA status

CI/CD best practices for mobile app testing

10

Operate

11

Implement a mobile device cloud

allowing remote access to a wide

selection of devices, emulators

and simulators

1Create a common test

environment for developers

and testers, complete with

collaborative tools

2

Remote Access to Mobile Devices & Emulators Centralized device management, enabling remote users (testers,

developers) to access devices and simulators from any location

Fully integrates to all testing, dev & CI environments

Online cloud hosted at

Experitest datacenters

Onsite cloud located at

your enterprise premises

Supports unit, manual, automated & performance testing

12

Supports all mobile devices and any OS

13

Hundreds of Mobile Devices Hosted or onsite

Allow developers and testers to access a large pool of real mobile

devices, emulators and simulators

Full control of the remote mobile devices - as if it was in the

tester’s hand

Web access from any browser - no setup or installation required

All mobile OS versions available, incl. beta versions

14

Unified environment for development and QA Provide developers with advanced testing tools, easy test development and access to real remote devices early in the development process

Integration with development environments

support such as Xcode , eclipse, Android Studio,

Support UI and unit testing with XCUITest,

Espresso and UIAutomator

Support for open source frameworks such as

Appium, Calabash & Cucumber

15

Native support for simulators, emulators & remote devices

Develop or debug a mobile application on simulators, emulators or remote

mobile devices using any development and open-source environment

Native support for emulators and iOS simulators▪ Easy test development with test recorder and object spy

▪ Test non-standard UI elements, integrations with system

apps or 3rd party applications, integrations with device

components, or application components built with SDKs

Access remote real devices as if they were locally

connected

Demo

Native Simulator and

Emulator Support

18

DevOps Collaboration tools

Share a live session with other tester/developer

Debug on a remote device using any development

environment, as if the device was locally connected

Debug the device where the fault occurred

Detailed step-by-step video reports from QA to

easily reproduce bugs

Expedite root cause analysis and bug detection

Demo

Share Remote Session

DEMO

Share Session with QA Video

Demo

Remote Debugging

DEMO

Develop and Debug on Remote Devices Video

23

Integrate your test environment

with CI tools and case

management systems for end-to

end hands free automation

3Implement continuous testing

with large scale test automation,

covering all mobile app test cases4

24

Poll

25

CICD integration

Scale your mobile projects with any fully automated 24/7

continuous testing environment

Seamless integration into CI environments: Hudson,

Jenkins, TeamCity, Quality Center, Bamboo and TFS, Ant,

Centralized, cloud-based execution management

▪ Grid capabilities

▪ Dynamic test scheduling, execution, and distribution

▪ View of all executions in real time

▪ Direct execution from the CI

▪ Ability run multiple CI projects (multiple applications)

▪ RESTful API to mass deploy application on targeted

devices

Centralized test reporting

26

Large scale parallel execution

Accelerate testing cycles with a centralized automated test

execution environment

Large scale parallel execution of tests on remote

mobile devices

Centralized test execution management

Managed directly from the CI environment

Real-time view of all executions

Dynamic and smart resource allocation according

to testing needs

Centralized execution results and test reports

Demo

Integration with CI/CD, Large

Scale Parallel Execution

28

29

Centrally manage devices,

applications, projects and users 5

30

Management Capabilities

Centralized device, application project and user management

Full administration capabilities:

▪ Device management - Schedule device reservations, and filter

device selections

▪ User management - Define user roles, permissions, and device

access

▪ Project management - Assign devices, and applications to users

for specific projects and create a secure separation between

projects

▪ Application management – Mass installation, mass deletion,

cleanup, version control

Reporting and alert management capabilities:

▪ Dashboard for quick overview

▪ Reporting by device, user, project, application, etc.

▪ Email alerts

Demo

Mobile device Lab

Management

33

Leverage end-to-end QA data

analytics for rapid root cause

analysis and constant QA status6

Centralized Test Reporting Consolidate test results to gain end-to-end quality insights and drive

improvements

End-to-end quality view of your entire

QA program

34

Actionable insights with indication of

high risk areas and root cause analysis

Driving the fault investigation and

resolution process

35

Centralized Test Reporting

Gain up-to-date quality status and key insights for all your apps

Ensure your testing program covers all required scenarios

and platforms

Shorten investigation time with automated root-cause

analysis

Create customized reports based on your key focus areas

Integrate to all testing frameworks and defect-

management systems

Agenda

36

QA challenges in implementing mobile CI/CD in enterprises

CI/CD best practices for mobile app testing

Case study

37

Large North American Financial Institute

The problem:

• Limited results (25% coverage)

• Bottlenecks due to manual testing and manual processes

Suggested solution:

• Onsite mobile device cloud, serving:

• Remote testers

• Onshore + offshore automation engineers

• Remote developers for unit testing and debugging

• Integration with CI tools (Jenkins) and case management systems

• Advanced automation and manual testing tools

• High scale parallel execution

(SeeTestCloud, SeeTest Automation, SeeTest Manual, SeeTestGrid, SeeTestReporter-beta)

Site B

Site A

Site CMobile device

cloud

Developers,

automation

engineers

Offshore testers

Case Study

38

Outcome:

• 2 week delivery cycle

• 200 devices located in on-site mobile device cloud – site A

• Test automation conducted by engineers – Site B

• Offshore team performs automated and manual testing in

off-hours – Site C

• Jenkins triggers nightly test suite execution via grid:

thousands of tests on 50 devices in parallel: under 4 hours

• Developers in Site A perform unit tests and debug on the

device where the error occurred

• High, 24/7 device utilization manual testing, automated

testing and debugging

• Quality dashboard supports version readiness decisions

“Using SeeTestCloud allows me to

get better visibility on the quality

and the capacity of all our

different teams, and the progress

of every app.”

General Manager,

Mobile Technologies

Agile quality operation, supporting rapid, high quality releases

Case Study

✓ Leverage a mobile device cloud

✓ Create a common environment for developers and testers

✓ Integrate your test environment with CI/CD tools

✓ Perform continuous testing for all mobile app test cases

✓ Centrally manage devices, applications, projects and users

✓ Leverage end-to-end QA data analytics

Best practices to help you deliver great mobile apps rapidly

39

40

Questions?

Thank You!

41