Hp perfecto webinar - UFT Mobile

Preview:

Citation preview

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

How to Build an Enterprise Grade Mobile Testing Strategy

Eran Kinsbruner Product Marketing @Perfecto Mobile

Antoine Aymer Product Marketing @HP

A

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 2

How different is Mobile?

A

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 3

Type

Click Double click Right click Drag

Tap & Multi tap Double / long tap

Drag Swipe

Pinch & zoom

Locate

Type

Rotate / Tilt

Snapshot

Authenticate

Communicate

Speak

A

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 4

CES 2014, Smart Innovation

BYOD Inflight Entertainment

Contactless payment

Smart

e-bike

Smart gardener

Smart clothes

Smart nappy

Smart appliance

Augmented reality

A

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 5

Starbucks’ app

Features: • Pay for your purchases • Track and redeem rewards • Reload Starbucks Card’s balance • Get directions to the nearest store • Explore coffees, beverages, food and

nutritional information • Send eGifts instantly to your contacts • Stay informed by being the first to

hear of special offers

Improve in-store customer experience

Generate customers loyalty

Increase sales

A

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 6

But this is not enough… A

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 7

Use case #1

SaaS-based applications are expected to generate a growth rate of 17.4%.

Sport App connected with smartwatch

Unit test

• Unit test • 100% emulation

Interoperability

• Data transfer via Bluetooth • 100% manual/human testing

Industrialisation

• Functional testing on major devices • 80% automated testing

A

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 8

Why Mobile Testing looks so difficult?

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 9

It is the jungle!

versus versus

Development platform

Various OS types, versions, device/tablet, OEMs

Evolving

Browsers

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 10

DEVICE

TESTING

MARKET

Performance Network (>400 operators)

Contextual issues Peripherals Impact on back-end Emulator vs real mobile

Automation vs manual Device management What Devices to test on? Bug-fix latency Unavailable services Off-shore

Time to market

Fragmented landscape (OS, version, input, screen, style)

USER Know the user Users’ feedback

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 11

Why Mobile Testing looks so difficult

A

48% lack

expertise

38% lack tools

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 12

Mobile testing disciplines

Functional

• Validation of Functionality

• Smoke / Regressions Testing

• Offline access testing

• Negative Testing

Non Functional

• Network Strength / Outage / Recovery

• Different NW Types

• Peripheral Testing

• Services

Interoperability (IOP)

• Voice / SMS interrupts

• Notifications

• Battery /Cable Removal

Memory Leak

• Memory Usage

• Memory Leaks

• Garbage Collection

Performance Testing

• CPU Usage testing

• Network Usage

• Page Render time or activity Render time

Usability Testing

• User Experience

• Competitive Analysis

• Expert Review

Installation Testing

• New App Install

• Uninstall and Reinstall

• Upgrade testing

Security Testing

• OWASP Vulnerabilities

• Dynamic Testing

• Static Code Analysis

• Data Encryption

Language Testing

• Validation for Locales

• Images and Text

• Currencies, time zones etc.

• Context

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 13

Definition of Fully Tested

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 14

The Truth Is Out There! Which of the following statements reflect your development team's approach to testing mobile apps?

4%

21%

27%

31%

34%

55%

55%

55%

60%

We don't formally test or instrument our appsbefore deploying them; we fix changes as needed

We integrate crash reporting/analytics into ourdeployed application to monitor and fix problems

We automate our testing process as part of acontinuous integration process

We use a cloud-based service to test multipledevices

We test devices that are deployed on multiplecellular carriers

We do a quick test on an emulator/simulator or ona single device

We test across multiple devices that run the sameversion operating system

We test across multiple OS versions

We test our app across multiple device orientations

Source: Forrester, Q1 2013 Base: 765 Software developers developing mobile applications

A

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 15

Mobile is Agile on Steroids

A

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 16

The ideal Mobile testing picture

Cum

ula

tive

bu

sin

ess

valu

e

Time

Sprint 4 Sprint 3 Sprint 2 Sprint 1

CI CI CI CI

V1.0 V1.1 V1.2

Release, Iteration, Nightly, Continuous

End-to-End test cycle

On any device, by any team, from any location

Longevity testing (independent of sprint cycles)

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Agile and Testing -- 5 o’clock on Friday the 30th, do you know if your code is “done”?

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 18

The Truth Is Out There!

52% spend

more than an hour debugging/fixing

code

49% of

developers spend more than an hour a day testing new

code

Source: Forrester, Q1 2013 Base: 765 Software developers developing mobile applications

only 15% spend that much time writing test automation code

A

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 19

Agile mandates automation

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 20

Use case #2

SaaS-based applications are expected to generate a growth rate of 17.4%.

Top US Bank

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 21

Use case #3

SaaS-based applications are expected to generate a growth rate of 17.4%. Type of Testing

3 weeks

Mobile banking, 12 Millions unique users native application and mobile enabled website

Impact

manual

Deployment

In-house

A

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 22

Use case #3

SaaS-based applications are expected to generate a growth rate of 17.4%. Type of Testing

3 weeks

Mobile banking, 12 Millions unique users native application and mobile enabled website

Automated

Deployment

Cloud

48 hours

86% time savings

60% cost savings

Impact

A

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 23

Automation requirements

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 24

What can get in the way?

Silos App Style Cross-Platform

Differences Use cases

web store

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 25

Cross-platform differences

Menu panel

Text colours

Images

Name position

Labels

Button images

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 26

Building an Enterprise Grade Mobile Testing Strategy

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 27

Sprint 4 Sprint 3 Sprint 2 Sprint 1

CI CI CI CI

Start

Transform Grow

Predictability, scalability

What fits my workflow?

Coverage, Reliability & Governance

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 28

Finding the right balance

Coverage

Devices

Service Level

Governance

Existing Tools

Time to Market Locations &

Networks

A

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 29

HP UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 30

na

tiv

e

#1 HP UFT Add-in #2 Deployment #3 Devices

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 31

HP UFT Extension Integrated with HP ALM

Real mobile devices (and emulators) Perform almost any operations

Device agnostic Develop, run and schedule one

test across multiple platform Visual & Object analysis

Manageable Full IT control on authentication, access permission and activities

Predictable availability

Global Accessible from any location (VPN mandatory with MCE)

Secured Full tractability of actions. Device cleanups and encryption abilities Continuous testing

Support fast, agile delivery

Scalable From single, small project to hundreds of simultaneous users

Complex scenario advanced built-in functionalities such voice injection and recognition, NFC, location based #1 HP UFT Add-in

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 32

#2 Deployment

Public Cloud

Virtual Private Cloud

MobileCloud Express

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 33

Use case #4: Virgin Media

Virgin Media UK Offices V

irg

in M

edia

Co

rpo

rate

Net

wo

rk

On-premise lab

Team members

Core team

Off-shore

Virtual Private Cloud Dedicated devices in various geographies

Multiple devices in various geographies

Public Cloud

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 34

#3 Devices

List of supported devices: http://www.perfectomobile.com/portal/cms/services/handsets.html

Wide device spectrum HTC, Apple, Nokia, Motorola, Samsung, SonyEricsson, LG,

BlackBerry, Palm, Kindle

Full device control Fast charging cycle, audio injection, reboot, clean/wipes

Scalable Additional handsets are added on-

demand

Global The handsets resides in various datacenters in the world: US, Canada, UK, France, Germany, Spain, South Africa, India.

Deploy your app Testers can deploy their application on the device, test it as if you were holding the handset in your hands.

Powerful replay all tests are video recorded to allow

easy recreation of defects

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 35

SCO

PE

AU

TOM

ATIO

N

Choose the right devices

Define relevant tests across all target devices

Traditional non-mobile automation rules still

apply

2

(before the demo)

Be intimate with manual testing

Think about future maintenance

Implement continuous integration

Gain mobile objects experience

Always sanity test your previous version

1

3

4

6

5

7

8

Recommended