How to Test on the Right Mobile Platforms

Choose the Right Digital Platforms for Dev & Test

January, 2016

Presenters

05/01/2023 2© 2015, Perfecto Mobile Ltd. All Rights Reserved.

Eran KinsbrunerDir. Technical Evangelist

Gil KidronDir. Product Management

Gidi Pridor VP Marketing

A few housekeeping items

1. Submit questions using the Q&A panel.

2. Slides and recording sent

after the webinar.

3. Please complete survey after the

presentation.

“What devices/platforms/browsers should I test on to ensure efficient

market coverage?”

Answering the most common question

Agenda for Today


Discussion: How to plan your own coverage strategy

Web test coverage insights

& recommendation

s

Trends & insights from Digital Test Coverage Index

Edition 3

Coverage Strategy is Fundamental All-Around

For the Business it’s market coverage

• Great end user experience

• Build requirement for IT

• Optimize user engagement & rating

For the Dev/QA it’s test coverage

• Optimize process

• Discover bugs early, mitigate risk

• Continuously test on what matters

What does it take to cover the REAL end user experience?

v

Device & Platform

Model

OS Version

Screen Size

browserv

Environment

Location

Network

Phone events

Other apps

v

Conditions

Signal

Load

CPU

Device

Test Cases

iPhone 4iPhone 4siPhone 5iPhone 5siPhone 6iPhone 6+iPad MiniiPad MiniiPad Gen3iPad Gen4Samsung S4Samsung S5Samsung Note IIIAndroid 2.4Android 4.4

Improving Test Coverage Through Automation

Environment

While DrivingCell Site handoff

GPS LocationsUnder Load

4GLTE

Wifi

Improve Overall Quality - Run Test Automation as part of CI- Run Parallel Unattended Tests - Run Anytime & More Frequently

Test Coverage Optimization Process

2. Lab Sizing

Teams Projects

SDLC Cycle

Requirements

How many?

v

Me

Analytics

Market

My Space

Popular & emerging

1. Coverage Mix

Industry trends & insights

Which?

Test Coverage Optimization Process

Me

Analytics

Market

My Space

Popular & emerging

Industry

Current state

Internal

External

Future state

The Index

What’s new?• Web browsers index• Canada and Australia index

Digital Test Coverage Index 3nd Edition

Did you send a gift to your travel agent this Christmas?


Survey

One Experience on All Digital Platforms


Using multi-screens one after the other to accomplish one task requires one experience across all of those screens

Mobile is the #1 platform to start the digital experience but it is only a part of a broader set of digital platforms

Source: Google


2015 Retrospective – A Lot To Learn From…


The new index

Web test coverage

Index

Recommended OS versionPixels per Inch

Trend

U.S Index Q4 2015

Apple & Samsung Leads

Legacy & Critical Devices

Canadian Index Q4 2015

Blackberry holds ~8% Share

Apple & Samsung Leads The Market

Chrome & IE On Win 7 dominate

Digital Test Coverage Index - Q4 2015 Key Insights & Trends


iOS 9 stabilized on 70% market share, iOS 8 holds ~20%

Android stronger in Europe (EU5) than in U.S, Canada

Windows Phone clearly stronger In Europe (10% share)

Disruptive technologies (Touch ID, Force Touch) enhances

fragmentation and test coverage (e.g. Samsung S7 vs. S6, iPhone

6S vs. iPhone 6)

Mobile Insights

Digital Test Coverage Index - Q4 2015 Key Insights & Trends


Desktop browsers auto update forces an instant response for Dev/Test

Laggards desktop OS users shift towards early adopters **Microsoft heavily pushes for Windows 10 adoption

Users transition to new Chrome & Firefox OS version is swift (3-4 Days)

Web Insights


A word about Screen properties

• Screen properties (Size, Resolution, HW) Impacts • CPU, Battery, Performance

• Often large screen size reduces PPI (iPad Pro e.g.)

• Mixing various PPI levels assures better UX coverage (Responsive web, intensive UI apps)

Source: Mobile Jury

http://www.mobilejury.com/featured/screen-resolution-affects-battery-life-user-experience/8812/

Mobile Market Calendar

Force Touch

Large screens trendingSamsung 18.4’’, iPad Pro 12.9’’Growing customer demand

1st Tablet by Google, 10’’, Android 6.x

Web Index Insights

© 2015, Perfecto Mobile Ltd. All Rights Reserved. 2305/01/2023

2016 – The Web Chase

51

5049

48

52

53 5455

4344

4546

4748 49

50 5152

10.0.210.0

9.0.9

9.0.89.0.79.0.6

9.0.59.0.49.0.3

10.0.1OS X 10.12

Today’s Browsers are Auto-Updated by Default


Dec 2nd

Dec 5th

3 days

46 47

Dec 15th

Dec 19th

4 days

42 43

New Version Adoption Rate (days)


USA 3 4 5

Europe 3 4 7

UK 3 4 5

Germany 3 4 7

Australia 3 4 6

India 3-4 4 7

News Flash


Microsoft Cleaning the Stables


…and Cleaning


The EDGE Factor


?

Too Many Choices


48

51

50

49

48

52

53 5455

4344

45

4647

4849

5051

5210.0.2

10.0

9.0.9

9.0.8

9.0.7

9.0.6 9.0.59.0.4

9.0.3

10.0.1

U.S Index Q4 2015

Discussion – Planning your own Test Coverage

• How to use the index when planning your own test coverage

strategy? How to combine it with your own analytics data?

• How often should you revisit the coverage plan

• Who should be involved and sign off on the coverage plan?

• The role of automation in increasing your test coverage

• Are user conditions part of the coverage strategy?


Reminder

Please Complete the survey

Learn More: www.perfectomobile.com

http://www.perfectomobile.com/

Thank You

• 1) Assign a person in charge of continuously planning test coverage .• merging business requirements (which are often based on analytics)

with Index report and market stats is a valuable to do item• 2) CI & automation is key to continuous quality. Build the lab based

on the right coverage and cross platform test automatically to assure no regressions introduced when lab is refreshed by new device, new OS version, new browser/desktop permutation• 3) The index is a recommendation but it includes valuable platform

characteristics such as recommended OSV, screen properties, legacy & new devices etc. - even if you decide to include additional devices or change the mix based on your analytics, please make sure to keep the various mix of the above properties to assure high quality across screens, platforms etc. • 4) Though reference devices do not often tend to be that popular

from a market share perspective (e.g. Nexus devices) - their importance to the test coverage does not come from their market share but from the frequent OS updates these devices receives - making them the 1st in the market to run on the latest Android OS. These devices are a must in any test lab in today's market• 5) From a desktop and web perspective there is a clear auto update

mechanism for leading web browsers (e.g. Chrome) where the end users are not even aware to. The fast adoption of these new platforms due to them being transparently updated, forces Dev/Test teams to continuously update their lab and test on early Beta/Dev versions of these browsers so they are ready to support them on Day 1


Software

How to Test on the Right Mobile Platforms