Analysis of Usbability Metrics for Mobile Applicationsvnsgu.ac.in/dept/publication/ETIT/ETITPaper4_Pages 28 to...VNSGU Journal of Science and Technology – V 5(1) - 2016 - 30 Figure

[VNSGU JOURNAL OF SCIENCE AND TECHNOLOGY] Vol.5. No. 1, July, 2016 28 - 52,ISSN : 0975-5446

Analysis of Usbability Metrics for Mobile Applications

PATEL Nidhi N.

Site,Nathdwara

Technical University, Kota

DALAL Pankaj Site, Nathdwara Rajasthan

Rajasthan Technical University, Kota

Abstract

One can find a rapid growth in popularity of mobile devices and mobile application in recent years.

This in turn has recognised usability as important factor of quality to be evaluated. To increase

usage and popularity of mobile application developer needs to improve its quality and to improve

quality one needs to improve usability. One important step of Usability Evaluation ( UE ) of mobile

application is to get user’s perception about the application as usability measure is completely user

dependent. The aim of this paper is to suggest a new SDLC based on usability for improving the

usability factor of mobile applications.

Keywords: Usability; Usability Attributes, Usability metrics, SDLC, Mobile Applications,

Usability based SDLC.

1. Introduction

The popularity of mobile application has inspired the developers to develop application for almost

all fields and tasks to be performed through mobiles. Various tasks are being introduced day by day

due to enhancement of new mobile technology which allows the users to perform tasks speedily,

remotely and accurately. The quality metric is having a vital role in recent years as the usage of

Mobile applications has been increased greatly. The usability metric in a mobile context has gain

importance among the quality metrics. Usability along with the functionality is the core factor

behind rising and falling of Mobile applications.

Research studies & reviews have been made in the area of improving usability and most of them

concluded with providing guidelines for designing of User Interface in order to improve usability.

But usability is not only about User Interface.

Usability is user dependent factor of quality. It is must to get users’ opinion about the application to

improve usability of Mobile application. No user testing can find out users’ perception about

Mobile applications like what user feels while using various Mobile applications, what they like,

what they don’t like, which features they would like to change and why, etc.Hence it is compulsory

to get user opinion about the application because if the users do not find the application usable then

he/she would not use the application. To get user’s view about mobile application we have

performed a survey.This paper aims to present some activities to be performed at all stages of

SDLC in order to improve the usability of mobile applications based on survey data analysis, UE-

Usability Engineering & UELC-Usability Life Cycle.

VNSGU Journal of Science and Technology – V 5(1) - 2016 - 29

2. Literature Review

2.1 What is Mobile Application?

Mobile Applications are software/set of program that runs on a mobile device and performs certain

tasks for the user. We can say mobile apps are add-on software for smart phones which are

designed to assist us in our everyday life. In most of mobile devices some Mobile Applications are

pre-installed on mobile and other applications can be downloaded from the internet and can be

installed it in mobile phone. Different Mobile Applications run on different platforms like Android,

iOS, Windows, BlackBerry (RIM), Symbian, etc.

The Mobile Application delivers a wide mixture of application area. People use Mobile

Application for various purposes like to connect with internet(browsing), interact with

world(chatting), getting information from distance place(reading news), social

communication(facebook, twitter, etc), identifying location, road navigation, vehicle tracking,

finding out any place using GPS, control home devices via mobile, online shopping, mobile

banking and e-Ticketing, listing music, playing games, watching videos/television, learning

recopies, calculating loan instalments, voice recording, language translation, order food parcel from

restaurant, etc.

2.2 Types of Mobile Applications

Mobile Applications can be divided in following 3 categories [9]:

1. Web Applications

2. Native Applications

3. Hybrid Applications

2.3 What is Usability?

The term usability was being introduced in early 1980’s to replace the term “user friendly” while

measuring the quality of software [16].

The definition given in the ISO standard for software qualities (ISO 1991b) is product and user-

oriented: “a set of attributes of software which bear on the effort needed for use and on the

individual assessment of such use ...” According to ISO 9241, Part 11, usability is “the extent to which a product can be used by

specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a

specified context of use.”

Different people use the word usability in different context. Most commonly usability is used to

refer to ease of use. In this context it falls under Usefulness, which comes under Practical

acceptability under System Acceptability as shown in Fig-2.1 below [15].

As shown in Fig-2.1 the usefulness of a product can be measured by two key features – its utility

and its usability. Utility refers to a product’s capability to carry out an intended function. Usability

refers to how easy users find it to accomplish that intended function. The figure also defines

sub features of usability which can be defined as attributes of usability as shown in Fig- 2.2. This

attributes are quite similar to what Zhan and Adipat [21], Ahmed[1],Neilsen[22] have mentioned.


Figure -2.1 Usability as ease of use.

2.4 Why Usability is important?

In the current competitive market of Mobile Application, usability is a necessary condition for

survival. If user finds the application difficult to use, user would leave it as number of other

options are available for them. If the homepage fails to clearly state what an application serves

and what users can do with the application, user would not use the application at all. If users get

confuse and lost while using the application, user will simply stop using the application. If the

application's information is hard to read or doesn't answer users' key questions, they leave. There

are plenty of other applications available offering the same service to user; so for user leaving is the

first option when users encounter a difficulty.

The first law of e-commerce is that if users cannot find the product, they cannot buy it either. For

companies who sell their products thru Mobile Application, it is compulsory to make the

application easy to understand and easy to use to improve their sales. This two ease of use &

understanding are attributes of usability, means that application need to have usability as a quality

factor.

2.5 Usability Evaluation (UE)

Usability Evaluation is the process by which usability of Mobile Application is evaluated from

various aspects, like the application is easy to use or not, whether the users are able to efficiently

use the products to perform their desired tasks or not, and that users enjoy using the products, the

application is providing good UI or not, it is having high accessibility & learnability or not, etc.

R. Bernhaupt has classified following UE methods [17]:

• User testing (in the laboratory and the field)

• Inspection oriented methods (like heuristic evaluation and cognitive walkthrough)

• Self-reporting and inquiry oriented methods (like diaries and interviews)

• Analytical Modelling (task model analysis and performance models)

Ivory [12] has also mentioned the UE methods along with techniques used in each class.


Figure - 2.2 Attributes of Usability

2.5.1 Usability tests

They are performance measurements to determine whether usability goals have been achieved or

not [17]. It describes the activity of performing usability tests in a laboratory or at field with a

group of users and recording the results for further analysis. Various testing methods used in

usability testing are: thinking aloud protocol, performance testing, log file analysis, etc.

2.5.2 Inspection:

Inspection oriented UE methods are commonly used in industry because they are fast and cheap.

Most commonly used methods are: heuristic evaluation and cognitive and pluralistic walkthrough.

It has been found that inspection-oriented methods lack validity when applied to mobile devices

because they do not consider the contextual factors that affect user-system interaction. The success

of these methods lies in the expert’s ability to interpret the context of use and to draw meaningful

conclusions [17].

2.5.3 Inquiry:

To evaluate mobile devices and applications, questionnaires and self-reporting methods are

additionally used to survey users’ behaviours and usage of the systems. As traditional


questionnaires conducted at the end of the usage are able to show several hindering effects, new

variations of these methods have been developed [17].

2.6 Usability Metrics

Nielsen [22] has mentioned 5 usability metrics: Efficiency of use, learnability, memorability,

errors/safety and satisfaction. Boehm’s model incorporates 19 different quality factors

encompassing product utility, maintainability, and portability [1]. The MUSiC model proposed by

Bevan in 1995 was concerned specifically with defining measures of software usability, many of

which were integrated into the original ISO 9241 standard. However, a strictly performance-based

view of usability cannot reflect other aspects of usability, such as user satisfaction or learnability

[14]. Azham [4] reviewed the existing metrics for desktop application and then he tried to develop

a new conceptual model which suggests metrics to be calculated for usability evaluation of M-

Application. Nigel [14] has also mentioned the MUSiC method which gives effectiveness,

efficiency, productive period, learnabilty and satisfaction as usability metrics. Formulas for

measuring the same are also provided [14]. Jeff [24] has mentioned some more metrics like Task

time, Pages view/click, Error, etc.

3. Research Methodology

3.1 Purpose of Research

Usability along with the functionality is the core factor behind rising and falling Mobile

Applications. Research studies & reviews have been made in the area of improving usability and

most of them concluded with providing guidelines for designing of UI in order to improve

usability. But usability is not only about UI. When we are talking about usability, we are talking

about efficiency, learnability, memorability, productivity, satisfaction, effectiveness etc. All of

these attributes need to be considered to improve usability. These attributes are measured by

various usability metrics (like user efficiency, user satisfaction, user expectation, task time, error

rate, page views/clicks, etc.). Hence to improve usability we need to consider all these usability

attributes in early phases of SDLC which is not the case with currently adopted methodology for

Mobile Applications development.

Problem Statement

Usability is user dependent factor of quality. It is must to get users’ opinion about the application to

improve usability of Mobile Application. No user testing can find out users’ perception about

Mobile Applications like what user feels while using various Mobile Applications, what they like,

what they don’t like, which features they would like to change and why, etc. The analysis of

usability metrics identifies reasons for low values of usability. This information about users’ views

can be used to introduce new activities in early phases of SDLC to improve usability of the final

product.

Research Question

What metrics should be analysed in order to improve usability of Mobile Applications by

introducing which new activities into SDLC phases?

For answering this research question, a number of secondary research questions have to be

answered like:


What is Mobile Application?

Which Usability Metrics are used to evaluate usability of Mobile Applications?

What metrics should be analysing in order to improve usability of Mobile Applications?

What activities to be introduced in SDLC phases in order to improve usability of Mobile

Applications?

Research Objective:

The above research question will be answered by achieving the following objectives:

Primary Objective:

Identifying and analysing usability metrics in order to improve usability of Mobile Applications by

suggesting activities to be included in early phases of SDLC.

It Requires:

To study about Mobile Applications.

To study about usability evaluation metrics for Mobile Applications.

To indicate the metrics to be considered for usability improvement.

To introduce new activities into SDLC phases to improve usability.

Research Design

The aim is to improve usability of Mobile Applications by introducing new activities at early

phases of SDLC. The analysis of usability metrics identifies reasons for low usability values. This

result leads to some activities to be performed in early phases of SDLC in order to improve

usability of Mobile Applications. The figure 3.1 shows the workflow, the research design to be

followed to achieve the above mentioned objectives.

Figure 3.1- Research Design


3.2.1 Sampling Design:

Usability is a user dependent factor of quality. When usability metrics tend to measure usability,

they are actually measuring users’ experience & reviews about usage of Mobile Applications.

Hence for our analysis of usability metrics we will use users’ reviews about Mobile Applications.

Users’ views about Mobile Application usage would be changing with time, experience, evolving

technology and availability. Hence we will not use secondary data instead we will collect users’

views as our primary data for analysis. Also we are planning to cover as many possible users of

different categories, occupancies, age groups, areas and different mentalities for various types of

Mobile Applications, various types of mobile devices, various types of mobile OS. So we will

perform a survey based on questionnaire to collect our data.

The target of our survey is to get responses while covering people from different areas, different

age groups, with different occupancies and different requirements from Mobile Applications.

While covering as many possible users, as many possible Mobile Applications are also need to be

covered with different devices, operating systems, categories & versions as well. Hence the

sampling universe would be infinite. The sampling unit would be group of people who are using

smart phones and various Mobile Applications for various tasks. Here we are not mentioning the

source list as our sampling universe is infinite.

3.2.2 Questionnaire Design:

The parameters of interest are usability metrics so our survey questionnaire includes questions for

usability metrics for collecting users’ views as data. The survey includes 44 question regarding 6

different usability metrics which are:

A. User Satisfaction Metric (Q8 to Q14)

B. User Efficiency Metric (Q15 to Q21)

C. Task Time Metric (Q22 to Q26)

D. Error Metric (Q27 to Q30)

E. Page Views/Clicks Metric (Q31 to Q34)

F. User Expectation Metric (Q35 to Q44)

The Questionnaire i.e. survey form is shown in appendix. The users have been provided options for

giving answers in terms of scale like: (more, average, less), (all of them, some of them, none of

them), etc. The purpose is to make it easy to divide the metrics into submetrics as follows:

User Satisfaction Metric: Fully Satisfied, Average Satisfied and Not Satisfied.

User Efficiency Metric: High Efficient, Efficient and Low Efficient.

Task Time Metric: Less Time, Average Time, More Time.

Error Metric: Less Errors, Average Errors, More Errors.

Page Views/Clicks Metric: Less Clicks, Average Clicks, More Clicks.

3.3 Data Collection

We have performed a survey for data collection. The survey was performed in 2 ways:

online and manual. The online survey was performed through Google form service, which is free

online survey service. It returns back the responses into form of an excel sheet. The social network

of Facebook was also used for online survey. While for offline survey, hard copies of survey forms

were distributed and got them filled back by respondents. The replies of manual survey is then


added to the excel sheet of online survey. Total 500 people had participated in the survey. The

collected data of 500 respondents is shown in appendix.

3.4 Data Analysis

The collected data is then divided into 4 groups based on users’ occupancy: Job group, Business

group, Student group and Housewife group depending on their occupancy. Following steps are

performed on the collected data:

1. The values of usability metrics & submetrics are calculated for all groups.

2. Analysis is performed for identifying reasons for getting low values of these metrics.

3. Descriptive Statistics is calculated for all metric values to find out analysis to be performed

further.

4. Correlation analysis is performed to find out whether any relationship exists between the

usability metrics or not.

5. ANOVA & t-Test analysis is performed to find out difference of user efficiency metric for

job, business & student group.

6. Regression analysis is performed to find out the exact relationship between user efficiency

& task time, error, page views/clicks metrics.

7. The analysis result is interpreted and mapped to SDLC phases by introducing new

activities into early phases.

All the statistical analysis is performed by using Microsoft Excel; no specific software has been

used.

4. Data Analysis

The collected data for Job group, Business group, Student group and Housewife group is presented

in Table - 4.1.

Table 4.1 responses received from survey

At this point we eliminated the housewife group as there was not enough number of responses for

this group to be analysed. Metric wise data analysis for rest of each group is performed.

4.1 Survey Analysis

All the user responses are divided into 3 groups Fully/High, Average/Medium, Less/Low according

to the answers and average is calculated for all 3 groups. Table 4.2 shows the average submetric

values. Based on this data the questions from questionnaire are found due to which the Fully

Satisfied, High Efficient, Less Time, Less Errors and Less Clicks metrics got low values.

The common reasons for getting low values of user satisfaction metric are as follows:

User Groups Reviews Business Count 105

Housewife Count 32 Job Count 217

Student Count 146

Grand Total 500


By using Mobile Applications how do you feel?

How many times do you get confused while using application?

How many times do you find the application unnecessarily complex?

Table -4.2 Average values of submetrics for all Groups

The common reasons for getting low values of user efficiency metric are as follows:

Are you able to use Mobile Application without errors in first attempt?

How many features do you remember for any application after a gap of time?

The common results for getting low values of task time metric are as follows:

How much time do you take for using any application for first time?

The common reasons for getting low values of error rate metric are as follows:

While using Mobile Application how many errors do you repeat frequently or always?

The common results for getting low values of page views/clicks metric are as follows:

How many click/key press do you make while using any application for first time?

The mapping Table 4.3 shows the mapping of all these common reasons to usability attributes. This

mapping result identifies the usability attributes that needs to be considered in early phases of

SDLC to improve the values of respected usability metrics which in turn would improve usability

of final product.

Metric Average of Response Received

Business Group Job Group Student Group

User

Satisfacti

on

Metric

Fully

Satisfi

ed

Avera

ge

Satisfi

ed

Not

Satisfi

ed

Fully

Satisfi

ed

Avera

ge

Satisfi

ed

Not

Satisfi

ed

Fully

Satisfi

ed

Avera

ge

Satisfi

ed

Not

Satisfi

ed

68 71 10 133 156 16 92 98 12

User

Efficienc

y

Metric

High

Efficie

nt

Efficie

nt

Low

Efficie

nt

High

Efficie

nt

Efficie

nt

Low

Efficie

nt

High

Efficie

nt

Efficie

nt

Low

Efficie

nt

43 56 14 81 128 26 63 81 14

Task

Time

Metric

Less

Time

Avera

ge

Time

More

Time

Less

Time

Avera

ge

Time

More

Time

Less

Time

Avera

ge

Time

More

Time

29 61 15 74 120 22 34 87 25

Error

Metric

Less

Errors

Avera

ge

Errors

More

Errors

Less

Errors

Avera

ge

Errors

More

Errors

Less

Errors

Avera

ge

Errors

More

Errors

41 57 8 107 100 10 57 75 15

Pages/Vi

ew

Click

Metric

Less

Clicks

Efficie

nt

Click

More

Clicks

Less

Clicks

Efficie

nt

Click

More

Clicks

Less

Clicks

Efficie

nt

Click

More

Clicks

35 62 8 85 106 26 52 73 21


These reasons themselves give the points to be considered to improve the mapped usability

attribute. Table 4.4 shows the reasons and the concluded result points to be considered in SDLC.

Table -4.3 Attribute Mapping Table

Reasons for low values Usability Attribute

By using Mobile Applications how do you feel? Satisfaction

How many times do you get confused while using application? Efficiency

How many times do you find the application unnecessarily

complex? Effectiveness

Are you able to use Mobile Application without errors in first

attempt? Learnability

How many features do you remember for any application after

a gap of time? Memorability

How much time do you take for using any application for first

time Learnability

While using Mobile Application how many errors do you

repeat frequently or always? Memorability

Table 4.4 Summary of analysis result points

Reasons for low values Result Point

How many times do you get confused while using

application?


complex?

How many features do you remember for any application

after a gap of time?

Irrelevant or more

information makes the

applications complex and the

users might get confused.

Are you able to use Mobile Application without errors in

first attempt?

How much time do you take for using any application for

first time?

Improper help/manual may

lead to confusion which may

result into errors in first

usage.

The descriptive statistics for the metric data is shown in Table 4.5. In Table 4.5 we can see that the

Mean & Median values are very close to each other for all the metrics. This suggests that we can

select regression analysis as a technique for the purpose of data analysis.

Table 4.5 descriptive statistics result

User

Efficiency

Task Time Error Page

views/clicks

User

Satisfaction

Mean 56.2222222 39.8888889 52.2222222 52 72.88888889

Standard Error 12.4049472 9.06832773 12.4150198 10.80637672 17.65521991

Median 56 29 57 52 71

Mode 81 #N/A 57 #N/A #N/A


Standard Deviation 37.2148417 27.2049832 37.2450593 32.41913015 52.96565973

Sample Variance 1384.94444 740.111111 1387.19444 1051 2805.361111

Kurtosis 0.24933933 -

0.83839653

-

1.31108744

-0.908444963 -1.128376546

Skewness 0.68626137 0.84633146 0.22001346 0.312244845 0.168080248

Range 114 75 99 98 146

Minimum 14 12 8 8 10

Maximum 128 87 107 106 156

Sum 506 359 470 468 656

Count 9 9 9 9 9

4.2 Correlation Analysis

The result of correlation analysis is shown in Table 4.6. It is clear from the Table 4.6 that both the

user efficiency & user satisfaction are strongly related to error and page views/clicks. Also the user

efficiency is proportionate to user satisfaction i.e. if user efficiency increases, user satisfaction

increases and vice versa. This implies that consideration of either of them in further analysis will

not affect our result.

Table 4.6 correlation analysis summary

User

Efficiency Task Time Error

Page vies/

clicks

User

Satisfaction

User Efficiency 1

Task Time 0.287702 1

Error 0.923976 0.519889 1

Page views/clicks 0.972774 0.429015 0.962978 1

User Satisfaction 0.965017 0.373534 0.977224 0.96070375 1

4.3 ANOVA Analysis

Now further we are applying ANOVA analysis to our data to find out whether job, business and

student users are equally efficient or not only for the data of user efficiency metric as shown in

Table 4.7.

Hypothesis 1Job users, business users and students are equally efficient in using the Mobile

Applications.

H0: μ1 = μ2 = μ3

H1: at least one of the means is different.

Table 4.7 ANOVA Summary

SUMMARY Count Sum Average Variance

High Efficient 3 187 62.33333 361.3333

Efficient 3 265 88.33333 1336.333

Low Efficient 3 54 18 48


Job User 3 235 78.33333 2606.333

Business User 3 113 37.66667 462.3333

Student User 3 158 52.66667 1202.333

Table 4.8 ANOVA Analysis Summary

Source of Variation SS Df MS F P-value F crit

Rows 7588.222 2 3794.111 15.91193 0.012467 6.944272

Columns 2537.556 2 1268.778 5.321062 0.07463 6.944272

Error 953.7778 4 238.4444

Total 11079.56 8

Table 4.8 shows summary of ANOVA analysis where F > F crit ( 15.91193 > 6.944272 ) so the

hypothesis is rejected. The means of the three populations are not equal. That means job users,

business users and student users all not equally efficient in using Mobile Applications. At least one

of them is different.

4.4 t-Test Analysis

We found as a result from ANOVA analysis all 3 groups of users are not equally efficient. To find

out which pair is different we need to perform t- test. We have to perform t-test for 3 times for 3

pairs: job & business, job & student, business & student. Starting with job & business groups we

define our second hypothesis as follows:

Hypothesis 2 Job users are as efficient as the business users.

H0: μ1 - μ2 = 0

H1: μ1 - μ2 ≠ 0

Table 4.9 shows result of the t-test for job & business groups for efficiency metric. Table 4.14

shows that none of the conditions are false i.e. neither t stat < -t Critical two tail nor t stat > t

Critical two tail as per two-tail-test. So the Hypothesis is not rejected.

This implies that Hypothesis is correct means the job users are almost as efficient as the business

users, though the mean values for both the populations(78.333, 37.67) are quite different.

Table 4.9 T-Test Job & Business Analysis Summary

Job

User

Business

User

Mean 78.33333333 37.66666667

Variance 2606.333333 462.3333333

Observations 3 3

Hypothesized Mean Difference 0

Df 3

t Stat 1.271523366

P(T<=t) one-tail 0.146584509

t Critical one-tail 2.353363435

P(T<=t) two-tail 0.293169019

t Critical two-tail 3.182446305


Hypothesis 3 Job users are as efficient as the student users.

H0: μ1 - μ2 = 0

H1: μ1 - μ2 ≠ 0

Table 4.10 t-Test Job & Student Analysis Summary

Job

User

Student

User

Mean 78.33333333 52.66667

Variance 2606.333333 1202.333

Observations 3 3

Hypothesized Mean

Difference 0

Df 4

t Stat 0.720350091

P(T<=t) one-tail 0.255579002


P(T<=t) two-tail 0.511158005


Table 4.10 shows result of the t-test for job & student groups for efficiency metric. Table 4.10

shows that none of the conditions are false i.e. neither t stat < -t Critical two tail nor t stat > t

Critical two tail as per two-tail-test. So the Hypothesis is not rejected.

This implies that Hypothesis is correct means the job users are almost as efficient as the student

users, though the mean values for both the groups(78.33, 52.67) are quite different.

Hypothesis 4 Business users are as efficient as the student users.

H0: μ1 - μ2 = 0 , H1: μ1 - μ2 ≠ 0

Table 4.11 shows result of the t-test for business & student groups for efficiency metric.

According Table 4.11 the two-tail-test the first condition i.e. t stat < -t Critical two tail is true. So

the Hypothesis is rejected. This implies that Hypothesis is not correct means the business users and

student users are not equally efficient in using the Mobile Applications.

Table 4.11 T-Test Business & Student Analysis Summary

Business

User

Student

User

Mean 37.66667 52.66667

Variance 462.3333 1202.333

Observations 3 3

Hypothesized Mean

Difference 0

Df 3

t Stat -0.63678


P(T<=t) one-tail 0.284767


P(T<=t) two-tail 0.569533


The output of t-test on job, business & student groups confirmed the result achieved by ANOVA

test i.e. job, business & student users are not equally efficient. This implies that different types of

users with different characteristics would have different efficiency & learnability.

4.5 Regression Analysis

From the results correlation analysis it is clear that user satisfaction and user efficiency metrics

follows the same patters i.e. both tends to increase or decrease simultaneously. Hence we are not

considering user satisfaction metric for regression analysis even.

Now we will apply regression analysis to the remaining metrics user efficiency, task time, errors

and Page Views/Clicks metrics. After applying regression analysis onto these 4 metrics data results

obtained are as below:

Table 4.12 Regression Analysis Result

Regression Statistics

Multiple R 0.983453706

R Square 0.967181192

Adjusted R Square 0.947489908

Standard Error 8.527811002

Observations 9

The Table 4.12 shows the Regression Analysis Results; here the value of R Square is 0.97

which implies that 97% variation of User Efficiency is explained by the variations of Task

Time, Error and Page views/clicks. The more it is closer to 1 the better the regression fits the

data.

Table 4.13 Regression Analysis Anova Summary

Df SS MS F Significance F

Regression 3 10715.93775 3571.9793 49.1172218 0.000392811

Residual 5 363.6178025 72.72356

Total 8 11079.55556

The Table 4.13 shows the ANOVA summary of Regression Analysis which denotes the value of

Significance F is 0.0004 which is very lesser then 0.05 which makes it sure that the results are

reliable.

Table 4.14 Regression Analysis Summary

Coefficients Standard Error t Stat P-value

Intercept 3.402518521 6.888694571 0.4939279 0.64228806


Task Time -0.23006604 0.13650897 -1.685355 0.15273342

Error 0.071098162 0.334105371 0.2128016 0.83988568

Page Views/clicks 1.120843815 0.362991903 3.0877929 0.02722757

Table 4.15 Regression Analysis Residual Output

Observation

Predicted User

Efficiency Residuals

1 89.2568592 -8.256859197

2 126.5609867 1.439013321

3 28.19398645 -2.193986449

4 38.87516154 4.124838462

5 62.91340185 -6.913401852

6 9.487063736 4.512936264

7 57.91674679 5.083253208

8 70.54073369 10.45926631

9 22.25506006 -8.255060065

The Table 4.14 describes the Regression Analysis Summary. In this table the first column

represents Coefficients from which the regression line developed is:

y = User Efficiency = 3.403 -0.23 * Task Time + 0.071 * Error + 1.121 * Page Views/Clicks

In other words, for each unit increase in task time, User Efficiency decreases with 0.23 units. For

each unit increase in Error, User Efficiency increases with 0.071 units and for each unit increase in

Page Views/Clicks User Efficiency increases with 1.121.

The Table 4.15 represents the residual output of regression analysis. The second column of Table

4.15 shows predicted values of user efficiency based on values of task time, errors and page

views/clicks by regression analysis. The third column shows the difference between the actual

value and predicted values of user efficiency for our data set.

The line graph in the figure-4.1 shows the comparison between actual user efficiency and predicted

user efficiency from the regression line developed from the data.

As one can see that there is very small difference between actual user efficiency and predicted user

efficiency, which implies that the user efficiency is dependent on task time, error and Page

Views/Clicks as per the developed regression line. This implies that the metrics to be considered

are: Task Time Metric, Error Metric and Page Views/Clicks Metric.

Table 4.16 shows the result point table which includes all the result points developed from data

analysis mapped with the respected usability attributes.


No of Observations

Figure–4.1 Comparison between Actual User Efficiency & Predicted User Efficiency

4.6 Result Interpretation:

These result points are used for proposing some activities to be performed for improvement of

usability while developing Mobile Application. These proposed activities are based on survey

result points. Modern SDLC methodology proposed by Valacich [30] is selected as base for

introducing new activities. Usability engineering (UE) along with UELC-Usability Engineering

Life Cycle are also integrated into SDLC. Mayhew [26] has suggested Usability Engineering Life

Cycle, which defines phases/activities related to usability that are required to perform during

various phases of SDLC in order to integrate usability engineering into SDLC.

Various phases & activities of SDLC are as follows for Mobile Applications development:

4.6.1 Project Selection and Planning

The first phase is concerned with the feasibility of the new application. It consists of

finding the need and importance of developing the Mobile Application. An argument whether to

continue with development of new application or not is made and finally if the argument results

into positive conclusion than the application is developed by following rest of the activities in rest

of phases of SDLC.

The need for new application may result from:

The desire to reach more customers.

The realization that use of Mobile Application would finally increase the profit.

4.6.2 Analysis

Here the functionalities of the application under development are figured out and while doing some

special activities are introduced to consider usability.

0

20

40

60

80

100

120

140

1 2 3 4 5 6 7 8 9

Use

r Ef

fici

en

cy

Uset Actual Efficiency vs User Predicted EfficiencyPredicted User Efficiency


Table 4.16 Data Analysis Result Points

Result

Point RP)

Derived results Derived From Usability

Attributed

RP1 Different users with different characteristics would

have different efficiency & learnability.

ANOVA & t-Test

Efficiency

RP2 User efficiency is positively related to user

satisfaction. If efficiency increases satisfaction

would increase and vice versa.

Correlation

Analysis Satisfaction

Efficiency

RP3 User efficiency is dependent on task time, error

rate & page views/clicks.

Regression

Analysis Learnability

Memorability

Efficiency

RP4 If Task time required by users to perform the task

increases user efficiency decreases and vice versa.

Regression


Memorability

Efficiency

RP5 If error rate increases user efficiency increases and

vice versa.

Regression


Memorability

Efficiency

RP6 If page views/clicks increase than user efficiency

increases and vice versa.

Regression


Memorability

Efficiency

RP7 Irrelevant or more information makes the

applications complex and the users might get

confused.

Survey Analysis

Effectiveness

RP8 Improper help/manual may lead to confusion

which may result into errors in first usage.

Survey Analysis Learnability

Efficiency

These activities are:

4.6.2.1 Requirement Gathering & Specification

The process of requirement gathering is a challenge in case of Mobile Application development

compare to traditional application development because of various reasons like: Changing

requirements of users, continuous evolution in technology, limitations of mobile device (screen

size, battery life, etc.) and large number of users. We suggest using questionnaire to collect users’

views that give idea about how they prefer the product to work like, to look like, etc.

Generally the Mobile Applications are evolved based on users’ feedback resulting in new versions.

Identifying users’ views in the beginning of application development would decrease the number of

iterations required in evolving the application and also the cost & time of development and training

users.

4.6.2.2 Target User Analysis

Mobile Applications are developed almost for all areas and hence it has users of various classes

with various skills, nature & expectations. According to RP1 of Table 4.16, different users with


different characteristics would have different efficiency & learnability. Now both efficiency &

learnability is having great impact on usability. This implies that while developing an application

the developer needs to understand the characteristics of the target users in order to improve the

usability of the application. Knowing the target users would also help the developers to satisfy the

users which again improve the usability as user satisfaction directly affects usability.

The target user Analysis is introduced to identify the target users and their characteristics. This

would help the developer and designer to understand the user. Under this analysis users’

characteristics are identified like: age, gender, education, occupancy, purpose of using Mobile

Applications, experience of using Mobile Applications, frequency of using Mobile Application,

learnability & capability of using Mobile Application, usage constraints, etc.

4.6.2.3 Task Analysis

The RP4 of Table 4.16 says that the task time required by users to perform the task greatly affects

the user efficiency and hence affects usability. In order to improve the usability designers have to

design the task modules and UI in such a way that would decrease the task time required by the

users to perform the task. For that designers have to first understand the way of working of users,

what users would do to complete the task. This information can be collected by task analysis. By

performing task analysis designers would find the sequence of steps performed by users in order to

perform a task. Some tasks would be such that it would require sequences of steps to be performed

while some tasks can be performed by a single click/tap.

One more objective of introducing task analysis is to help/support users while performing the task

through Mobile Applications by providing some extra information/functionality not identified

during requirement specification which would increase effectiveness and hence usability of Mobile

Applications.

4.6.2.4 Workflow Analysis

Workflow analysis would further analysis the results of task analysis to find out information like:

which task requires a sequence of steps to be performed, what are the common steps amongst the

task sequences, on performing the task does it lead to any other task if yes which task, the tasks can

be abandoned in between or not, users can jump onto jump onto some other task while performing

one task, etc.

All these information is useful to designers for developing the designs in order to make them

effortless for users. So to improve task analysis analyst should perform workflow analysis.

4.6.2.5 Identify Usability Goals

Durrani [25] has suggested Usability Engineering Life Cycle- UELC. In which he has suggested to

define usability goals in the requirement phase but we have included this phase in analysis phase.

These usability goals cover usability attributes like: learnability, memorability, efficiency,

effectiveness & satisfaction. The usability goals define what has to be done in order to achieve high

values of usability attributes for e.g. “application can be learnt in less than 2 minutes (learnability),

user is able to perform N tasks without errors (efficiency)”.

This makes it clear that defining usability goals requires information about target users & their way

of thinking and we are collecting this information through task analysis & user analysis in analysis

phase so we have added the usability goals into analysis phase as activities after task analysis &


user analysis. These goals are identified from user analysis & task analysis. These usability goals

are than used in design phase for developing the UI.

4.6.2.6 Usability Evaluation Metrics

Usability engineering suggests to measure usability characteristics at all phases of SDLC so we

have added this activities at the end of analysis phase, before moving onto design phase [25].

During this activity the analysis made in previous activities is evaluated in terms of usability.

Prototypes or mock-ups are generated based on analysis results (user analysis, task analysis & work

flow analysis) and then tested to measure the usability metrics. These calculated values of usability

metrics specify the expected values of metrics to be achieved after the completed Mobile

Application is used by target users.

Table 4.24 shows various evaluation metrics to be calculated. These metrics are selected from

survey questionnaire in such a way to cover all usability attributes. The first column of Table 4.13

shows the 3 main measures of usability efficiency, effectiveness and satisfaction while the second

column shows the related questions included in questionnaire as part of survey.

4.6.3 Design

The third phase is systems design. During systems design, analysts convert the description resulted

from analysis phase into design specification. In this phase database design, program design and UI

design specification are developed from analysis specifications developed in last phase. The

activities for UI design are:

Table 4.24 Summarized Usability Evaluation Metrics

Usability Evaluation Metrics

Template Related Survey Questions

Efficiency

Easy to use

Are you able to use Mobile Application without errors in first

attempt?

While using Mobile Application how many features can you use

without errors?

Easy to remember

After using Mobile Application how many features do you

remember?

How many features do you remember for any application after a

gap of time?

Easy to learn

At first time how do you use Mobile Application?

When do you use manual/help while using Mobile Application?

How much time do you need to achieve expertise for using most

application?

Easy error recovery How much time do you spend for correcting errors while using

any application?

Satisfaction

Usefulness Does the application work the way you want?

Are you satisfied with the applications?

Enjoyable By using Mobile Applications how do you feel?

Do you find the UI of application user friendly?


Trustworthy Would you recommend the applications to your friends?

Effectiveness

Effortless

How much time do you take for using any application for first

time?

How much time do you take for reading manual/help while using

any application?

Simple & effective interface Do you find the UI of application user friendly?

Less complex

How many times do you get confused while using application?


complex?

4.6.3.1 CMD – Conceptual Model Design

Durrani [25] has mentioned use of CMD into design phase in UELC for integrating usability into

SDLC. This model represents the way most people think reason and use the M- Applications. It is

developed from the analysis (user analysis, task analysis & work flow analysis) made in last phase.

4.6.3.2 DUID – Detailed User Interface Design

Durrani [25] has suggested including the DUID phase into design phase of UELC for improving

usability. This activity includes designing of screen layouts, dialogue boxes, feedback, inputs &

outputs as well. Designers make use of CMD-activity of design and the usability goals developed

in requirement phase for improving usability of application. It is the most effective way of

integrating users’ thinking into interface.

While designing UI designer must follow some design points to avoid complex & confusing UI

designs. All these points are derived from result points. The number at the end of each point

denotes the respected result point that makes the base for introducing the design point.

Various design points to be considered are:

Keep the design as simple as possible.

Maximize the visibility.

Don’t overload user’s memory.

Provide only relevant information to the task.

Minimize search time. – RP4

Reduce users’ effort. – RP6

Prevent terrible user errors. – RP5

Provide enjoyable, satisfying & easy to use interface. – RP2

4.6.3.3 Target User Acceptance test

Now at the end of design phase it is time to measure the design from usability point of view

according to usability engineering, for that user acceptance tests are introduced. In this activities

paper prototypes or mock-ups have to be made up based on design specifications developed in

previous activities and test them on target users.

The behaviour of the users’ have to be recorded while performing the tests to collect information

regarding usability attributes like: learnability, memorability, complexity, effort required,

satisfaction, etc. From these measured attributes values usability metrics can be calculated. These

RP7


calculated values have to be compared with the expected values resulted from evaluation metrics-

activities of analysis phase, if the measured values are less than expected values than applications

needs to be redesign.

We have introduced the activities to measure usability at the end of each phase so that variance in

usability metrics values compare to usability metrics values of last phase can be identified

4.6.4 Implementation & Testing

This phase includes 2 activities implementation and coding. Implementation consists of coding the

program based on design specification. Testing consists of testing the application is tested by

applying various testing techniques. We have introduced testing of usability metrics at each phase

hence testing will not require much effort & time in this phase. And the implementation of Mobile

Figure 4.2 Comparison between Modern SDLC & usability based


Applications are done by the users so implementation part does not need to be handled in case of

Mobile Applications.

4.6.5 End User Reviews

In the modern SDLC the implementation & testing phase is the final phase. But here we have

added one more everlasting phase considering the evolving nature of Mobile Applications. To

implement this designer has to design a feedback provision for users. Once the Mobile Application

is placed in market the users would start using them and providing feedback about the applications.

Based on this feedback the applications would be modified and new version is placed in market for

users. This phase is kept everlasting considering that users would like to use modified and more

functional applications.

Consolidation of all above activities implies whole new SDLC which is mainly concentrating on

identification & improvement of usability as a quality attribute from the first phase of SDLC till the

last phase. Hence we have given a name: UBSDLC – Usability Based SDLC for Mobile

Applications development to this new SDLC.

The proposed UBSDLC methodology is shown in comparison with modern SDLC in figure-4.2.

The blue eclipse in the figure-4.2 shows various phases of modern SDLC. The pink trapezoid

boxes represent the common activities in both methodologies. The green boxes shows the activities

which are included in SDLC phases and are not included in UBSDLC phases. While the orange

rounded boxes shows the activities which are proposed in UBSDLC and which are not there in

modern SDLC.

5. Conclusion

In the current competitive market of M-Application, usability is a necessary condition for survival.

The application must have usability as a quality factor. Users want Mobile Applications to be

simple and fast. Just one bug or usability issue can spoil the entire user experience. And with so

much competition in this new era, if users don’t have an excellent experience with your

application, they will switch to a rival product faster. If user finds the application difficult to use,

user would leave it as number of other options are available for them. Thus usability is a crucial

attribute of quality to be considered while developing M-Applications. Hence we had analyzed

usability metrics to propose some activities in order to improve usability of M-Applications.

5.1 Proposed Usability Based SDLC Methodology

We have proposed UBSDLC Methodology for development of Mobile Applicationsby considering

usability as a core factor. This methodology has been developed by integration usability

engineering and UELC into modern SDLC phases. While doing this we have also included

activities derived from survey results. The figure 5.1 shows the proposed Usability Based SDLC

Methodology.

5.2 Limitations

In this methodology we have considered the constraints which make the development of M-

Application different from traditional application development like: Connectivity problems,

Mobility issues, Cross platform compatibility, Limitations of mobile device, Limited resources, etc.

On considering all these issues some more activities are included as sub phases almost in all phases

of Usability Based SDLC which are as shown in figure 5.1. The purple boxes shows the activities


related to mobile issues, which are: Data Access Analysis, Data Distribution Analysis, Security

Analysis, platform Analysis in analysis phase and Data Access Design, Synchronization & Auto

Backup Design in design phase.

Figure – 5.1 Proposed Usability Based SDLC Methodology


We have only suggested the activities to handle the issues but we have not provided the exact

definition of each of these phase related to mobile issues. This is the limitation of the proposed

UBSDLC methodology for developing M-Applications.

5.3 Future Research Work

The future research work can be done for defining the activities to be performed into sub phases

related to mobile issues in UBSDLC. What has to be done, how these activities are to be carried

out, what would be the input of the phases, what would be the output, how to interpret the output

and how to use the output, etc, can be defined to complete the UBSDLC methodology to make it

practically applicable.

References

[1] Ahmed, S., Mohammad, D., Rex, B.K., Harkirat K.P. Usability measurement and metrics:

A consolidated model, Software Quality Journal,14, (2), pp. 159-178,2006.

[2] Anne Kaikkonen, Aki Kekäläinen, Mihael Cankar, Titti Kallio, and Anu Kankainen,

Usability Testing of Mobile Applications: A Comparison between Laboratory and Field

Testing , Journal of Usability Studies, 2005.

[3] Ashwin, Mobile Application Testing Summary , Ashwin Group Professional, Aug-2012.

[4] Azham Hussain, Maria Kutar, Usability Metric Framework for Mobile Phone Application ,

2009.

[5] Caritor, Testing Mobile Business Application-A Hybrid Approach , Mar-2007.

[6] Florence Balagtas-Fernandez, Heinrich Hussmann, A Methodology and Framework to

Simplify Usability Analysis of Mobile Applications , IEEE/ACM International Conference

on Automated Software Engineering, 2009.

[7] Jakob Nielsen , Ten Heuristics for Usability .

[8] Keynote , Testing Strategies and Tactics for Mobile Applications , Dec-2012.

[9] Lionbridge, Mobile Web Apps vs. Mobile Native Apps: How to Make the Right Choice

,Lionbridge, 2012.

[10] Md. Rashedul Islam, Md. Rofiqul Islam, Tahidul Arafhin Mazumder, Mobile Application

and Its Global Impact , International Journal of Engineering & Technology IJET-IJENS,

Vol:10 No:06, Dec – 2010.

[11] Melissa D. Mulikita, Mobile Application Testing ,Seminar Informatik,June-2012.

[12] Melody Yvette Ivory, An Empirical Foundation for Automated Web Interface Evaluation ,

2001.

[13] Mohan Kumar, Manish Chauhan, Best Practises in Mobile Application Testing , Building

Tomorrow's Enterprise, April -2013.

[14] Nigel Bevan, Measuring usability as quality of use , Software Quality Journal, 4, 115-150

(1995).

[15] Nigel Bevan, Usability is Quality of Use , Proceedings of the 6th International Conference

on HCI, July - 1995.

[16] Nigel Bevan, Jurek Kirakowskib and Jonathan Maissela, What is Usability , Proceedings of

the 4th International Conference on HCI, September 1991.

[17] R. Bernhaupt, et al., Usability evaluation methods for mobile applications ,Handbook of

research on user interface design and evaluation for mobile technology, Vol. 2. pp. 745-758,

2008.

[18] Rachel Harrison, Derek Flood and David Duce, Usability of mobile applications: literature

review and rationale for a new usability model , Journal of Interaction Science, 2013.

[19] Xavier Ferré and Natalia Juristo, Helmut Windl, Larry Constantine, Usability Basics for

Software Developers , IEEE Software, Jan-Feb-2001.


[20] Xiaoxiao Ma, Bo Yan, Guanling Chen, Chunhui Zhang, Ke Huang, and Jill Drury, A

Toolkit for Usability Testing of Mobile Applications , 2011.

[21] Zhang, D, & Adipat, B. , Challenges, Methodologies and Issues in the Usability Testing of

Mobile Applications , International Journal of Human-Computer Interaction, 18(3), 293-

308, 2005.

[22] Nielsen J., Usability engineering , Morgan Kaufmann Pub., 1994.

[23] C.R. Kothari, Research Methodology : Methods and Techniques , ISBN (13) : 978-81-224-

2488-1, New Age International (P) Ltd.

[24] Jeff Sauro, A Practical Guide to Measuring Usability: 72 Answers to the Most Common

Questions about Quantifying the Usability of Websites and Software , ISBN (10) :

1453806563, Create Space Independent Publishing Platform, November, 2010.

[25] Qaiser Durrani, Samia Qureshi, Usability Engineering Practice in SDLC , ICCIT, 2012.

[26] Deborah J. Mayhew, The Usability Engineering Lifecycle – A practitioner’s handbook for

user interface design .

[27] Luis Corral, Alberto Sillitti, Giancarlo Succi, Software Development Processes for Mobile

Systems: Is Agile Really taking over the Business? , Centre for Applied Software

Engineering.

[28] Srinivas Ramadath, Maurice Collins, Mobile Application Development: Challenges and

Best Practices , Accenture, 2012.

[29] Harleen K. Flora1, Dr. Swati V. Chande, A Review and Analysis on Mobile Application

Development Processes using Agile Methodologies , International Journal of Research in

Computer Science, Vol. 3, Issue 4, 2013.

[30] Joseph S. Valacich, Joey F. George, and Jeffrey A. Hoffer, Essentials of Systems Analysis

and Design , Prentice Hall Publication, 2009.

Documents

Analysis of Usbability Metrics for Mobile Applicationsvnsgu.ac.in/dept/publication/ETIT/ETITPaper4_Pages 28 to...VNSGU Journal of Science and Technology – V 5(1) - 2016 - 30 Figure