62
Evaluating and Improving Software Usability 902 : Thursday, 9:30am - 10:45am Philip Lew www.xbosoft.com

Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Embed Size (px)

Citation preview

Page 1: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Evaluating and

Improving

Software Usability 902 : Thursday, 9:30am - 10:45am

Philip Lew

www.xbosoft.com

Page 2: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Understand,  Evaluate  and  

Improve  

2  

Page 3: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Agenda  

•  Introduc7on  

–  Importance  of  usability  

– What  is  usability?  

– What  is  UX?  

•  Usability  Modeling  and  Measurements  

•  Case  studies  on  measuring  and  improving  

•  Summary  

 

Page 4: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

What  is  Quality?  

Pertaining  to  soEware  J  

4  

Requirements…  

Page 5: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Importance  of  Usability  

Page 6: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

•  Usability  is  important  especially  for  web  applica7ons  

– And  now  especially  mobile  applica7ons  

•  Saas,  mobile  are  in  alignment  with  and  soon  becoming  equivalent  to  cloud  •  Most  prevalent  implementa7on  method  in  the  next  5  years  

•  Without  good  usability:  

– Users  will  leave  the  applica7ons  

– For  mobile,  if  they  can’t  learn  in  30  seconds,  they  won’t  come  back  

Why  is  Usability  Important?  

Page 7: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Web  and  Mobile  have  Changed  the  

Terrain  •  Business  models  

have  changed  

–  Instead  of  paying  

upfront  and  

‘owning’  the  

soEware  

– Pay  as  you  go,  pay  

by  subscrip7on  

•  Behavior  and  

expecta7ons  have  

changed  

Page 8: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

What  is  Usability  

In  rela7on  to  quality  

In  rela7on  to  user  experience  

Page 9: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

When  you  think  Usability…  

•  Naviga7on  

•  Efficiency  

•  Responsiveness-­‐performance  

•  Learnability  

•  AZrac7veness  

•  Understandability  

Anything  

else  come  to  

mind?  

Page 10: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Current  Research  –  Usability  

Quality

Usability Satisfaction

ISO 9126-1

ISO 9241-11

Usability Design

Experts ISO 25010

User  

Experience  

Product Owners

Page 11: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Usability  -­‐  Key  Characteris7c  of  

Product  Quality  

Source: ISO 25010

Page 12: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Quality  in  Use  ISO 25010

Page 13: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Notes  on  Sa7sfac7on  and  Usability  

•  Sa7sfac7on  is  a  subjec7ve  feeling  dependent  on  many  things  other  than  usability:  

–  A  user  can  be  highly  sa7sfied  but  the  applica7on  with  low  usability.  

–  An  applica7on  can  be  highly  usable  (high  usability)  but  the  user  is  not  sa7sfied!  

Highly usable software

My password doesn’t work

don’t have what I want I’m unsatisfied

I’m satisfied!

! low usability

software

Finished my work today

Nice weather today

Page 14: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Usability-­‐Major  Component  of  Quality  Quality  and  Usability  in  the  so1ware  development  lifecycle  

•  Can  be  measured  from  the  design  point  of  

view  or  of  the  product  

•  Can  be  measured  ‘in-­‐use’  with  real  users  

Usability  

Usability  

Page 15: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Evolu7on  of  SoEware  quality  

Software processes

Code

Process quality

Software Quality

(internal)

Software Quality

(external)

Software Quality

(in use)

Product ?

CMMI assessment

model

white box testing

black box testing ?

CMMI Assessment

Company

Programmer Tester End User

Type of quality

What is measured

How measured?

Who measures?

ISO 9000 ISO 9241 ISO 9126 ISO 25010

Page 16: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Source: ISO 25010

What  is  Usability-­‐Effect  of  the  

SoEware  Product  

Degree to which specified

users can achieve specified

goals with effectiveness,

efficiency and satisfaction in a

specified context of use.

Page 17: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Usability  in  Actual  Usage  

• User  role  

• Objec7ve  

• Task  

• Environment  

• Domain  

• …  

specified  users  

specified  goals  

specified  context  of  use  

What  else  can  you  think  of?  

Page 18: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Usability

UX, Usability and Quality in Use •  Difficult to understand their relationships

•  Lack of consensus in meaning in literature

•  Not totally clear how related to quality

•  Standard definition for user experience is still not available

and  User  Experience

Page 19: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Some  UX  Defini7ons  

UX DESCRIPTION

D1 a continuous process of user engagement with the product [10] D2 entire set of affects that results in user-product interaction [11] D3 the evolution of usability [12] D4 elaboration of the satisfaction component of usability [13] D5 a categorization of “do-goals” (pragmatics) and “be-goals”(hedonics) [3], [14]

D6 infinite small experiences relating to people, products and contexts [6] D7 consequence of user’s and product’s characteristics when interacted in a

specific environment [1] D8 degree to which specified users can achieve actual usability, safety, and

satisfaction in use in a specified context of use [15] D9 A person's perceptions and responses that result from the use or anticipated use

of a product, system or service [16]

D10 the degree of positive or negative emotions that can be experienced by a

specific user in a specific context during and after product use and that

motivates for further usage [17]

Page 20: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

User Experience (UX)

•  Pragmatic (do-goals) : refers to the product's

perceived ability to support the task achievement

and focuses on the product’s utility and usability in

completing tasks, the ‘do-goals’ of the user.

•  Hedonic (be-goals) : product's ability to support

the user’s achievement of ‘be-goals’, such as

being satisfied, pleasure.

•  Real goal of the end user is fulfilling be-goals such

as being autonomous, competent, related to

others, stimulated, etc.”

•  Be-goals are the driver of UX

Hassenzahl

Page 21: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

21  

Rela7ng  Usability  and  User  Experience  

Quality  

characteris7c  

508  

compliancy  Func7onality  

User  Interface  

Usability  Learnability  

Product  

aZributes  

Text/Sound  

aZributes  

Appropriate  

Func7ons  Easy  to  use   Easy  to  learn  

UX  Pragma7c  

Do  Goals  Effec7ve  and  Efficient  

UX  Hedonic  

Be  Goals  Sa7sfac7on  (pleasure,  sense  of  community…)  

Usability  

Measurement  Effec7veness  in  use   Efficiency  in  use   Learnability  in  use  

Sa7sfac7on  

Measurement  Pleasure   Comfort  

Sense  of  

Community  Trust  

Context  

USER  EXPERIENCE  

Page 22: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Now  that  we  understand  it  ALL…  

We  Need  to  Define  Usability  For  Our  

Own  Purposes  

22  

Page 23: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Let’s  Define  Usability/UX  

From  the  Product  Viewpoint  

Usability  

Save  Loca7on  

Characteris7c  1   Characteris7c  2   Characteris7c  n  

Subcharacteris7c  1   Subcharacteris7c  2   Subcharacteris7c    n  

AZribute  1   AZribute  2   AZribute  n  

Understandable  Naviga7on  

Control  Stability   Previous-­‐Next  

Home  Loca7on   Ease  of  finding  

Page 24: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Defining  Usability  from  an    

Effect-­‐Real  usage  Point  of  View  

Quality  

Errors  

Effec7veness   Efficiency   Characteris7c  n  

Accuracy   Completeness   Subcharacteris7c    n  

AZribute  2   AZribute  3   AZribute  1  

Usability

Sa7sfac7on  

Page 25: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Some  Poten7al  AZributes  

Measurements  for  ‘in  use’  Usability  •  Effec9veness  

– Comple7on  rates  

– Error  rate  

– Help  usage  

•  Efficiency  

– Task  7me  

– Backtracking  

•  Learnability  

– Learning  rate  

– Task  7me  devia7on  

Can  you  think  

of  any  others,  

par9cular  to  

your  

applica9on?  

Page 26: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

26  

Adding  in  UX  

Sa9sfac9on  in  use  

•  Degree  to  which  users  are  sa7sfied  in  a  specified  context  of  use.  Sa7sfac7on  is  the  user’s  response  to  interac7on  with  the  product,  including  amtudes  towards  use  of  the  product.  Subdivided  into  sub-­‐characteris7cs:  

– Likability  (cogni7ve  sa7sfac7on)  

– Pleasure  (emo7onal  sa7sfac7on)  

– Comfort  (physical  sa7sfac7on)  

– Trust  

UX Component

Page 27: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

   External  Quality  Requirements  (for  

Shopping  Cart  Entity)  

1 Usability  

1.1 Understandability  

1.1.1 Icon/label  ease  to  be  recognized    

1.1.2 Information  grouping  cohesiveness  

1.2 Learnability  

1.2.1 ………………………………………………………..    

1.3 Ease  of  Use  

1.3.1 Control  permanence    

1.3.2 Helpfulness      

 

 Example  of  Product  Quality  Model  

with  AZributes  (Operability in ISO 25010)

Page 28: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

28  

Learnability  

•  Degree  to  which  the  soEware  product  enables  users  to  learn  its  applica7on  -­‐  7  principles  (ISO  9241-­‐10)  

–  Suitability  for  the  task  -­‐  should  be  suitable  for  the  user’s  task  and  skill  level    

–  Self-­‐descrip7veness-­‐  should  be  clear  what  the  user  should  do  next  

–  Controllability    -­‐  the  user  should  be  able  to  control  the  pace  and  sequence  of  the  interac7on    

–  Conformity  with  user  expecta7ons  -­‐  should  be  consistent  

–  Error  tolerant  -­‐  forgiving  

–  Suitability  for  individualiza7on  -­‐  should  be  able  to  be  customized  to  suit  the  user    

–  Suitability  for  learning  -­‐  should  support  learning  

Page 29: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

29  

Ease  of  use  

•  Degree  to  which  the  soEware  product  makes  it  easy  for  users  to  operate  and  control  it.  

– controllability  

– error  tolerance  (by  operator)    

– conformity  with  user  expecta7ons  

Page 30: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

30  

Ease  of  Use  -­‐  Helpfulness  

•  Degree  to  which  the  soEware  product  

provides  help  when  users  need  

assistance  including  help  that  is:  

– easy  to  find    

– comprehensive  

– effec7ve  

Page 31: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Example:  Usability  Measurement  

31  

ACribute  

Scale How Calcula9on Goal Current

Help  

completeness Percent    of  

Menu  items  

with    

help

Compare  

menus  and  

help  items

% 90% 40%

Ease  of  

access Keystrokes  

to  

find/use  a  

feature/

func7on/

informa7on

Sample  50  

items average 3.5 12

Consistency Number  

loca7ons  for  

same  buZon

Examine  

menus  and  

doc.

integer 1 5

What types of usability measures are these?

Page 32: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

32  

Using  a  measurement  model  

Measurements  

Metrics  

Evalua7on  

Measurement  

Func7on  

Indicator  

Evalua7on  Evalua7on  

Sub-­‐characteris7cs  

Evalua7on  Evalua7on  Characteris7c  

Usability  

composed of

composed of

Page 33: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Usability-­‐Review  

•  Can  be  measured  from  the  design  point  of  

view  or  of  the  product  

•  Can  be  measured  ‘in-­‐use’  with  real  users  

usability   usability  in  use  

UX  

Page 34: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

34  

Measuring  Usability  

Methods  and  models  

Now  that  we  know  what  it  is,  and  

what  to  measure,  then  HOW?  

Page 35: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Measurable  AZributes  

•  Usability  characteris7c  

•  Descrip7on  and  purpose  

•  How  to  measure  

•  What  is  measured  

•  Measurement/Calcula7on  

•  Range  (min,  max)  

•  Metric  

•  Objec7ve  (goal)  versus  Current  (indicator)  

Once you have a

model (what you

are going to

measure), then

you start doing IT!

Page 36: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

UX/Usability  

Measurement  Methods  

Let’s  get  started  

Logging  

Heuris9c  Evalua9on  

Observa9on  

Labs  

Focus  groups  

Ques9onnaires  

Page 37: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

37  

Measurement  methods  

Expert  Evalua9on  (Heuris9c  evalua9on)  

•  Define  a  set  of  rules  or  criteria  with  measurements  and  evaluate  against  them  

Web-­‐based  logs  

•  Collect  user  ac:vity  data  

– Mistakes  and  errors  

– How  long  it  takes  to  complete  tasks  

– Comple:on  rates  

Ques9onnaires  

•  Quan7ta7ve  subjec7ve  measurement  of  UX  characteris7cs  

Page 38: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Heuris7c  Evalua7on  

Using  a  checklist  

But  a  liZle  more  complicated  than  

yes/no  

38  

Page 39: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

39

Heuristic Evaluation of a Pharmacy

application Each usability attribute has a quantifying metric.

•  For  example,  for  Predic9ve  textual  aCribute,  users  should  be  able  to  understand  a  buCon’s  results  prior  to  pressing  it.    

•  Direct  metrics  need  to  be  designed,  i.e.:    

–  (0)  No  support  at  all  

–  (1)  Par9al  

–  (2)  Complete    

•  Need  a  mapping  from  0,  1,  2  to  something  more  understandable,  i.e.  2  =  100,  and  1  =  60  

•  Need  an  indicator  to  interpret  the  level  of  sa9sfac9on  met  with  decision  criteria  with  acceptability  ranges  in  a  percentage  scale:    

–  0-­‐40  (unsa9sfactory  –red)  means  changes  must  take  place  with  high  priority;  

–  40-­‐70  (marginal  –yellow)  indicates  a  need  for  improvement  ac9ons;    

–  70-­‐100  indicates  a  sa9sfactory  level  –green-­‐  for  the  analyzed  aCribute.    

Page 40: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

40

2.2.2 Error Recovery Support

" In the current state, users filling a new prescription are supported well in error recovery (2.2.2) in automatic cursor positioning

Page 41: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

41

Heuristic Evaluation – Pharmacy Software The purpose of the evaluation •  Understand the external quality level of the learnability sub-characteristic for

filling a new prescription

Table 2 - excerpt of the whole current evaluation

Page 42: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

External Quality Requirements Measure EI value P/GI value

Global Quality Indicator 61.97%

1 Usability 60.88%

1.1 Understandability 83%

1.1.1 Icon/label ease to be recognized 100%

1.1.2 Information grouping cohesiveness 66%

1.2 Learnability 51.97%

1.2.1 ……………………………………………… …

1.3 Operability 49.50%

1.3.1 Control permanence 100%

1.3.2 Expected behaviour 50%

2 Content Quality 63.05%

2.1 Content Suitability 63.05%

2.1.1 Basic Information Coverage 50%

2.1.1.1 Line item information completeness 2 50%

2.1.1.2 Product description appropriateness 50%

2.1.2 Coverage of other Contextual Information 76.89%

2.1.2.1 ……………………………………………….. …

2.1.2.2 Return policy information completeness 33%

Heuris7c  Evalua7on  of  Usability-­‐Shopping  Cart  

Page 43: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

JIRA Heuristic Usability Evaluation

Page 44: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Usability  Logging  

Collec7ng  user  ac7vity  and  behavior    

Tradi7onally  used  for  other  things  

like…    

44  

Page 45: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Usability  Logging  

•  Iden7fy  users  by  using  session  ID  to  iden7fy  a  unique  user.    

•  Itera7vely  insert  code  into  the  applica7on  

•  Collect  data  

•  Analyze  the  data  for  each  aZribute  in  different  dimensions  and  aggrega7ons  

•  Determine  the  need  for  further  calcula7ons  and  what  aZributes  to  measure  further  

•  Revise  the  data  we  are  collec7ng,  adding  or  decreasing  granularity  

Page 46: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Some  easy  and  some  hard  

46  

Page 47: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

47

Jira Evaluation Summary

Page 48: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Using  Both    

Product  Perspec7ve  and  the  User  Perspec7ve  

•  If  we  find  some  aZribute  with  low  

performance  from  the  user  perspec7ve  

•  Then  we  can  go  back  to  the  product  

perspec7ve,  figure  out  why,  and  improve  

48  

Page 49: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

49

JIRA Evaluation by Attribute

Excerpt

Page 50: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

For  example:Login  BuZon  Visibility  

(product  usability  aZribute)  

50  

Page 51: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Using  Ques7onnaires  

The  process:  

1. Use  your  model  of  what  you  want  to  measure  and  improve  

2. Design  the  ques7onnaire  according  to  the  model  

3. Execute  the  ques7onnaire  

4. Analysis  

Page 52: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

One  UX  Model  

Page 53: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Ques7onnaires  Used  

SUMI  (So1ware  Usability  Measurement  Inventory  )  

QUIS  (Ques:onnaire  for  User  Interface  Sa:sfac:on  )  

PSSUQ  (Post-­‐Study  System  Usability  Ques:onnaire  )  

ASQ  (A1er-­‐Scenario  Ques:onnaire  )  

PUTQ  (Purdue  Usability  Tes:ng  Ques:onnaire  )  

PHUE  (Prac:cal  Heuris:cs  for  Usability  Evalua:on  )  

SUS  (System  Usability  Scale  )  

IUI  (Isometrics  Usability  Inventory)  

Page 54: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

54  

Page 55: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

55  

Page 56: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

What  do  you  do  with  all  this?  

56  

Page 57: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Itera7ve  Improvement  

•  Measure  external  quality  –  usability  with  

heuris7cs  

•  Measure  in-­‐use  usability  with  logging  or  

ques7onnaires  

•  Go  back  to  external  and  improve  

Page 58: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Goal  is  to  Understand,  Evaluate  and  Improve  

58  

Page 59: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Next  Steps  

•  Produce  an  ac7on  plan  

– What  usability  aZributes  are  important  to  your  

organiza7on?  

•  Develop  a  model  

– What  data  can  you  collect/Which  technique  can  

you  use  

•  Maybe  some  elements  of  the  model  drop  out-­‐can’t  be  

measured  that  easily  

•  Start  collec7ng  and  developing  benchmarks  

Page 60: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Conclusion  

•  Usability  and  UX  are  both  abstract  concepts  

– Cri7cal  component  of  quality  

•  Defining  is  different  for  each  organiza7on  

•  Need  a  model  for  your  organiza7on  

•  The  model  is  the  founda7on  of  what  to  

measure  

•  Once  you  can  measure,  then  you  can  evaluate  

and  improve  

Page 61: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Resources  

61  

These are all journal publications that can be accessed via these links…

Page 62: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking

Thanks  

Ques7ons  and  Answers    

Please  fill  out  an  evalua:on  form  and  drop  it  in  the  

collec:on  basket  located  at  the  back  of  the  room.  

   

Philip  Lew  

 

@xbosoE  

 

[email protected]  

phone:  408-­‐350-­‐0508  

www.xbosoE.com