14 Mobile Testing - QAI Global Servicesminisites.qaiglobalservices.com/stc2012/Paper_ Best_Practice/14_M… · Mobile Device Testing It is the process to assure the quality of mobile

Mobile Testing

© 2010 Wipro Ltd - Confidential

Rupesh Garg

Agenda

3

1 What is Mobile Testing?

2 Testing – Mobile Application Testing

Automation – Mobile Application Testing

© 2010 Wipro Ltd - Confidential2 © 2010 Wipro Ltd - Confidential2

3

4 Tools for Testing Site on Mobile Devices

Automation – Mobile Application Testing

5 Challenges to Mobile Testing

What is a Mobile Testing?

Mobile-device testing process is to assure the quality of mobile devices, like mobile

phones, PDAs, etc. The testing will be conducted on both hardware and software.

And from the view of different procedures, the testing comprises R&D testing,

© 2010 Wipro Ltd - Confidential3

And from the view of different procedures, the testing comprises R&D testing,

factory testing and certificate testing.

Mobile-device testing involves a set of activities from monitoring and trouble

shooting mobile application, content and services on real handsets. Testing includes

verification and validation of hardware devices and software applications.

What is a Mobile Application?

A mobile application is any application that runs on a

handheld device, like a personal digital assistant or a

smart phone, and connects to the network wirelessly.


Mobile Application Types

Giguere (1999) provided a way to categorize mobile applications on the basis

of the connectivity model of the application to the backend system.

• Applications that are stand-alone

• Applications that connect to the backend through synchronization


• Applications that connect to the backend through synchronization

software

• Applications that connect to the backend through a wide-area wireless

network

• Applications that connect to the backend using special networks.

• Other Applications

Applications that are stand-alone: These applications run on the handheld

device itself without connecting to the network. An example of a standalone

application is a calculator running on a Windows Pocket PC.

Applications that connect to the backend through synchronization software:

These applications use synchronization software like Microsoft Active



These applications use synchronization software like Microsoft Active

Synch to connect to a parent computer or network. An example of such an

application is Microsoft Outlook for Pocket PC that synchronizes data

between the handheld device and the host computer through

synchronization software.

Applications that connect to the backend through a wide-area wireless

network:

These applications use either circuit-switched or packet-switched wide-area wireless

networks to connect to a data source or other network resource.

Applications that connect to the backend using special networks:



Applications that connect to the backend using special networks:

These applications connect to the back-end through special networks like

Specialized Mobile Radio (SMR) or paging networks.

Other Applications:

There applications include those that connect to the back-end using short-range

wireless networks, such as Bluetooth or infrared.

Mobile Applications Life Cycle

LifecycleManagement

ManagedServices

SolutionAlternatives

WiproValueAdditions

Best Practices

Key Aspects of US Bank

Requirements

Feature Testing

Integration Testing

IOT Testing

System Testing

Usability Testing

App Launch

Design & Develop

Deploy to Prod

Performance

Regression


8

Mobile Testing

Certification

Performed at Internal Labs

Performed at Field

Performed at IOT Labs

Performed at Operator Lab

Field Testing

Device Launch

M&A

Go/No Go Usability

Testing

Testing – Mobile Testing

Test StrategyOptimize

Test DesignProductivity Improvements

Test ExecutionCertification / sign-off

Test Planning-Scenario optimization using in-house tools- Test planning

-for device testing-for platform and component

Strategy Definition

- Scenario definition and

traceability to business

requirements

- Device Optimization

- Device Platform

Testing

- Functional Testing on

simulator

- Application Certification

(MSN, Symbian Signed)

- Operator Certification


9

component integration-Bluetooth applications-Streaming testing-Web browser testing

- Test automation planning using in-house tools

- Device Platform

optimization

- Test environment

requirement definition

- Operator Certification

- Field Testing in Key

Operator Networks

- Application and protocol

testing

- Authorization & access

control

- Performance testing

(server, device)

Mobile Device Testing

It is the process to assure the quality of mobile devices, like mobile

phone, PDA etc. The testing will be conducted on both hardware and

software.

And from the view of different procedures, the testing comprises R&D


And from the view of different procedures, the testing comprises R&D

testing, Factory Testing and Certificate Testing.

R&D Testing:

R&D test is the main test phase for mobile device, and it happens during

the developing phase of the mobile devices. It contains hardware testing,

software testing, and mechanical testing , and mechanical testing.




Factory Testing:

Factory Testing is a kind of sanity check on mobile devices.

It's conducted automatically to verify that there are no

defects brought by the manufacturing or assembling.

Certificate Testing:

Certificate Testing is the check before a mobile device goes


to market. Many institutes or governments require mobile

devices to conform it's specifications and protocols to

make sure the mobile device will not harm users' health

and have the compatibility with devices from other

manufactures. Once the mobile device passes the

checking, a certificate will be issued to it.

Mobile Platform

A mobile operating system, also known as a mobile OS, a mobile

platform, or a handheld operating system, is the operating

system that controls a mobile device or information appliance

similar in principle to an operating system such as Windows,

Mac OS, or Linux that controls a desktop computer or laptop.


However, they are currently somewhat simpler, and deal more

with the wireless versions of broadband and local connectivity,

mobile multimedia formats, and different input methods.

Typical examples of devices running a mobile operating system are

smart phones, personal digital assistants (PDAs),tablet computers

and information appliances.

Smartphone operating systems:

Operating systems that can be found on smartphones include

Mobile Platform


Operating systems that can be found on smartphones include

Nokia's Symbian, Google's Android, Apple's iOS, RIM's BlackBerry

OS, Microsoft's Windows Phone, Linux, Palm/HP's WebOS,

Samsung's Bada, Nokia's Maemo and MeeGo among many others.

Android, Bada, WebOS and Maemo are built on top of Linux, and

iOS is derived from the BSD and NeXTSTEP operating systems,

which are all related to Unix.

Share of worldwide 2010 Q4

Smartphone sales to end users by

operating system, according to

Canalys.

The most common operating systems

Mobile Platform


The most common operating systems

(OS) used in smart phones by Q3 2010

sales are: The Symbian OS and its

successor Symbian platform from the

Symbian Foundation (36.6% Market

Share Sales Q3 2010)[7] (open public

license)

Mobile Application Testing Types:

Compatibility testing:

Compatibility testing assures that a given application works as

intended with selected devices with different screen sizes,

resolutions, and internal hardware (memory size, processor


resolutions, and internal hardware (memory size, processor

speed, and button/input differences).

Interoperability testing:

Interoperability testing includes testing of different

functionalities on different mobile agents such as messaging, call

setup, sharing conference, and video messaging etc.


Functionality testing:

Functionality testing of a mobile device includes controls,

storage media handling options, and other operational aspects.

Functionality testing of a mobile application is a black-box type

of testing to assure that the application is functioning as per the



of testing to assure that the application is functioning as per the

business specifications.

Usability testing:

Usability testing encompasses mobile interface testing,

application navigation testing, and intuitiveness of the

application, consistency, and soberness of color scheme

Performance testing:

The load, capacity, and stress testing of mobile application

servers. The objective of load testing is to determine

whether the application can sustain the required number of



whether the application can sustain the required number of

users with acceptable response times. The objective of

capacity testing is to benchmark the maximum load of

concurrent users an application can sustain before

experiencing system failure while the objective of stress

testing is to validate an application’s stability and reliability

over an extended period of time.

Localization and Internationalization testing:

The objective of this testing is to assure that the localization and

internationalization features are supported by the devices and

applications as per specifications. Communication testing: The



applications as per specifications. Communication testing: The

objective of communication testing is to verify the protocols and

handshakes in a call flow between multiple users.

Data exchange and synchronization testing:

This testing focuses on data exchange and synchronization among

different mobile devices, PDAs, and PCs using infrared frequency

and USB ports.

Critical Success Factors for Mobile Applications

Testing:

The critical factors that determine the success of mobile testing program

are:

Diversity of the Device Environment


Hardware Configuration & Network related challenges

Rapid Application Development (RAD)

Use of Test Automation

Use of emulators and actual devices

Testing for mobile environment and application complexity

Automation:

Automation involves implementing the sequence of steps that needs

user interaction in mobile phone while executing a test case, to be

performed by a script or system without user interaction. When no

automation is possible the test case is tested manually. The aim of

Modest is to automate all the identified test cases.

Two grades of automation are


Two grades of automation are

Partial Automation

Full Automation

Partial Automation Full Automation

Partial Automation refers to

performing atleast one activity

done by human in the test case

implementation via script or

system.

Full Automation refers to

performing all the activities done

by human involved in test case

implementation via script or

system.

Automation:


Eg. The user may be requested to

press ‘Pass’ or ‘Fail’ button on

seeing the output of the test case

Eg. The validation for the output

can be embedded in the script

itself and the test case can be

loaded automatically once one

test case gets completed.

Approaches to Automation

There are two ways of approaches suggested in automation. They are

Phone Independent Approach

Phone Support Approach


Phone Independent Approach :

deals with test cases implementation where activities like loading of

test cases can be done one after the other automatically as one

gets completed.

Phone Support approach :

deals with test cases implementation where the launching of test cases or a

WAP session can be brought about using some scripts and after execution

the test result in the browser can be captured and compared internally with

Approaches to Automation


the test result in the browser can be captured and compared internally with

the stored output for confirmation of success of the test cases .This can be

brought about by using equivalent script coding.

Some Automation Tools for Mobile Application

Testing

1.TestQuest Pro

2. DeviceAnywhere

3.HP QTP with DeviceAnyWhere Addin

4.M-eux test extension for HP QTP

5. TestComplete

6. Perfectomobile


6. Perfectomobile

7. UISpec

8. FoneMonkey

Tools for Testing Site on Mobile Devices


An excellent free iPhone tester,

iPhoney isn’t exactly an emulator,

but allows developers to create

320x480px websites for use on the

iPhone. It allows you to test images

and code in a pixel-perfect Apple-

1)iPhoney:


and code in a pixel-perfect Apple-

Safari-powered environment, with

all the normal features including

Portrait and Landscape modes,

fullscreen, zoom and plugins.

2)W3C mobileOK Checker

This checker is a web-based automated validation tool that

checks to see how mobile-device-friendly your website is. The

tests are checked against the specification developed by W3C.

W3C mobileOK Basic Tests


3) iPad Peek

This handy web-based tool allows you to see how yourwebsites look when rendered on the iPad. It’srecommended that you use a WebKit-based browsersuch as Apple Safari or Google Chrome to have asaccurate a simulation as possible — or at the very least,a CSS3-capable browser that supports transformationproperties (like Opera) because it uses them to renderthe page in Portrait mode.


the page in Portrait mode.

4)Modify Headers Add-on for Firefox:

There are add-ons for Firefox that can manipulate the data

sent to servers to make it seem as if the user agent is

browsing on a mobile device, even when they’re not. To do

this, you need an add-on called Modify Headers (for

Firefox).


Once you have downloaded and installed this add-on, the

new options, Tools > Default User Agent and Tools > Modify

Headers should appear in your browser.

Modify Headers Add-on for Firefox:


5)Adobe Device Central CS5

As part of the latest Adobe Creative Suite, Device Central

emulates the operation of mobile devices on your desktop

allowing you to test HTML and Flash from the comfort of

your desktop.

To open a page or site, select Device Profiles > Browse

Devices, then right-click and choose Add To Test Devices.


Devices, then right-click and choose Add To Test Devices.

Now that you’ve added a device, simply go to File > Open

(for local testing) or File > Open URL (for remote testing).

Though far from being free, if you’re a professional

designer or developer, you’ll have a high chance of being

able to access Adobe’s creative suite of applications.

Adobe Device Central CS5


Google Mobilizer is a simple web tool that lets you input a web page address

and then makes the page mobile-web-friendly by trimming the content

down to its bare essentials. This is an excellent tool for seeing where you can

make performance optimizations on your site.

6)Google Mobilizer


7)Gomez

Gomez mobile readiness test gives you a score between 1 and

5 based on an analysis of over 30 proven mobile web

development techniques, ranging from style sheet use (e.g.

media queries) to caching techniques and standards-

compliant code. The results are displayed in an easy-to-

understand document that offers advice on how to make your


understand document that offers advice on how to make your

site better.

Unfortunately, to be able to test your site, you must enter in a

bunch of information including your email address, country,

zip code, and phone number

Gomez


MobiReady is another online testing site that allows

you to enter a URL so that it can perform a set of

evaluations, including Page Test, Markup Test and

Site Test of the web page. This is a slightly more

detailed version of Gomez above and provides a

comprehensive test result page including dotMobi

8)MobiReady


comprehensive test result page including dotMobi

compliance, W3C mobileOk tests, device emulators,

detailed error reports, HTTP tests and a code

checker.

However, the results aren’t as succinct as Gomez’s

results, a disadvantage when it comes to presenting

your information to your less technically astute

clients/employers.

MobiReady


9)DotMobi Emulator

The dotMobi emulator gives you a live preview ofyour site from a (fairly small, to be honest) rangeof different mobile phones. This mobile testingtool is great if you need to test your page on oldermobile devices.


What can be annoying to potential users of thistool is that it requires the Java browser plugin forit to work.

10)Opera Mini Simulator

With over 120 million smart phones sold that hasOpera pre-installed[1] and an edition available foralmost any OS, making sure that your site works onOpera Mini (the mobile version of the Opera webbrowser) is a good practice to include in your testingworkflow.


workflow.

Opera has obliged by offering a free web-basedemulator for testing and evaluating your web designon a mobile device that uses Opera Mini.

This tool requires the Java browser plug-in tofunction.


References:

1. Wipro Knet

2. www.DeviceAnyWhere.com

3. www.HP.com


4. www.jamosolutions.com

5. www.automatedqa.com

Thank

You!!!

© 2010 Wipro Ltd - Confidential

You!!!

Documents

14 Mobile Testing - QAI Global Servicesminisites.qaiglobalservices.com/stc2012/Paper_ Best_Practice/14_M… · Mobile Device Testing It is the process to assure the quality of mobile