View
50
Download
0
Category
Tags:
Preview:
DESCRIPTION
Software Usability Course notes for CSI 5122 - University of Ottawa. Section 4: Usability in the Software Engineering Process Timothy C. Lethbridge < tcl@site.uottawa.ca > http:// www.site.uottawa.ca /~ tcl /csi5122. An Assertion. - PowerPoint PPT Presentation
Citation preview
Software UsabilityCourse notes for CSI 5122 - University of Ottawa
Section 4:
Usability in the Software Engineering Process
Timothy C. Lethbridge
<tcl@eecs.uottawa.ca>
http://www.eecs.uottawa.ca/~tcl/csi5122
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 2
I assert: Lack of usability is the most critical problem facing software engineering
In other words, the biggest gains in
• software quality
• productivity
• cost-reduction
• user satisfaction
• profitability, etc.
… would come from focusing on usability and related issues.
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 3
Evidence for the assertionFailures of projects due in significant part to usability
• E.g. FAA air traffic control systems
Observations of ‘great’ software vs. not-so-great with respect to usability• Great:
—Google (many of its products, but some getting worse)—Microsoft Office (some of it)—Mac-OS (much of it, but weaknesses creep in)
• Not-so-great:—Much in-house custom software—Software development environments (many aspects)—Web sites (far too many of them)
• I have observed many users formally and informally
My survey of practitioners (next slide)
Analysis of benefits and cost savings
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 4
Practitioner survey about 75 topics taught to computer science studentsLethbridge, T.C. (2000), “What Knowledge is Important to a Software Engineer?”, IEEE Computer, May, pp. 44-50.
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 5
More on the survey
HCI was second in terms of “knowledge gap”• Where importance most exceeds current knowledge
The top 5 (out of 75)• Negotiation• HCI/user interfaces• Leadership• Real-time system design• Management
I believe not much has changed since 1998
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 6
Sources of resistance to adopting usability practices
Primarily: Inadequate education of students, practitioners and managers
Which leads to:• Lack of introspection ability of software developers
—Unable to think like users• Lack of integration of usability into core development
processes• Persistent beliefs
—Most software is usable enough—Usability can be fixed at the “end”—Usability should be left to the HCI experts
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 7
Causes of the problems in education
Faculty don’t have the background either
The field of HCI is seen as distinct and separate
The field is perceived as too soft:
• There is a tendency to focus teaching and research on hard deterministic areas
There is little industry push for research in this area
There is little employer pull for greater education
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 8
Some partial solutionsThe SE and HCI communities need to work more closely together
• Core UI development and evaluation topics should be considered jointly part of the two fields
Improve education by ensuring HCI permeates curriculum models, certifications, accreditation, etc.
Enable corporations to be certified for their usability capabilities
• Would help “pull” education up by creating a need for professionals
• Clients might come to learn that better software comes from such companies
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 9
Should aspects of HCI be considered also integral to Software Engineering?
I argue yes:
• Usability should have no special status as compared to reliability, efficiency, maintainability, etc.
• Design involving users and their needs must drive software development
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 10
Should aspects of HCI be considered also integral to Software Engineering?
Yes, but this does not preclude HCI specialists• Analogies:
—A software engineer must be capable of designing architectural elements for storing data and information
- But there will always need to be database specialists
—Similar argument can be made for- Security experts, performance experts, requirements
experts, etc.
Core UCD techniques (coming slides) can be part of the SE field and part of a broader HCI field
User-Centred DesignRevisited
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 11
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 12
User centered design (UCD)
A term encompassing a range of software engineering approaches that focus on the user
We have talked about many of the techniques
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 13
UCD main stages
Pre-design
Design stage
Post-design
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 14
UCD Pre-design techniques:
Understand classes of user
Understand tasks
Perform a competitive analysis
• i.e. usability testing of competing products
Set usability goals
• using the metrics we discussed earlier
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 15
UCD Design-stage techniques
Parallel design
• Many developers generate UI ideas in parallel on their own
• They meet together and pick the best ideas
• Has maximum chance of discovering good ideas, without constraining the possibilities too soon
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 16
UCD Design-stage techniques - 2
Involving users
• Critiquing design
• Discovering problems
—with the UI
—with task descriptions
—through user testing
• Contributing ideas
• But users are not designers so don't ask them what do you want?
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 17
UCD Design-stage techniques - 3
Coordinating the interfaces of all parts of the system
• Including documentation, training etc.
Applying guidelines and heuristic rules
Prototyping
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 18
UCD Design-stage techniques - 4
Performing empirical testing using simulations and prototypes
• To discover problems
Measuring usability of the simulations and prototypes
Performing iterative design
• Design
• Evaluate
• Redesign
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 19
UCD Post-Design stage
Gather on-going feedback about potential improvements
Another term related to UCD
“Outside-in development”
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 20
Economics of Usability
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 21
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 22
An example of economic analysis in UCD: Economics of heuristic evaluation
We will look at
• Number of people needed
• Cost-benefit analysis
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 23
How many evaluators should you use in a heuristic evaluation?
Given:
• N = total number of problems to be found
—Clearly depends on:- Quality of the original design or work performed so far
- Size and complexity of the system
λ= proportion of all usability problems found by a single evaluator:
—Studies have shown this averages 34% (range 19% to 51%)
The percent of the total number of different usability problems found by aggregating reports from i evaluators is:
• PercentProblemsFound(i) = 1-(1-λ)i
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 24
We can plot the effect of this formula for different λ
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 25
We can also look at specific figures for the average case where λ=34%
Evaluators Percent Problems found1 342 563 714 815 876 927 95
Given a desired percentage of problems to be found, we can determine the number of evaluators needed
• by inverting the formula:• i = ln(1-P)/ln(1-λ)
It is not economical to try to find every problem but where should we stop?
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 26
How much cost savings can we achieve by finding a given problem?
This depends on many factors, but
• The worst problems are the easiest to find and will be found first
• As we put more effort into finding problems, we will find problems that are less and less severe
We can model the cost savings from fixing a problem as seconds of wasted time per user session
• Very approximate, but useful
—Some problems will only occur once, then never again because the user will learn to avoid them
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 27
Visualizing the diminishing returnsEstimated number of seconds per session saved by fixing the nth worst problem found in a particular project.
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 28
Deciding what to fix and the resulting savings
From the previous chart, find the total savings if we just solve the first n problems
We can divide this by the n to find the average savings per problem had we just found the first n
• A typical answer for this would be 2 seconds
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 29
Deciding what to fix and the resulting savings
Some data from the previous chart, where there were about 200 problems eventually found
nth problem s/session saved average savings had we justfound found the first n problems1 10.6s 10.6s10 7.4s 8.9s20 4.9s 7.4s40 2.2s 5.4s60 1.0s 4.1s80 0.45s 3.2s100 0.20s 2.6s In a large organization, we can gather data to calibrate our own charts and tables like the above.
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 30
Deciding how many evaluators to include:Costs
• Fixed costs to organize the evaluation: (no matter how many evaluators):
—Planning
—Preparing materials
—Writing report and communicating results to developers
—Typical estimate for a small system- = 75 hours work
- = $4500
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 31
Costs continued
• Variable costs for each evaluator
—Weighted salary for time spent
—Cost of analyzing the evaluator's report
—Typical estimate for an average system- 8 hours work for each evaluator ($480)
- 2 hours work analyzing the report ($120)
- Total = $600
• Total costs = $4500 + 600i
—Where i is the number of evaluators
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 32
Deciding how many evaluators to include:Benefits
Total benefits vary substantially depending on nature of system.
In some published studies, the benefit averaged $15000 per problem
On the following slides are two benefit calculation examples
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 33
Benefits example 1:small mass-market system
Three factors to consider:
1. Time saved per session per problem
• Will obviously vary from problem to problem and session to session
• Reasonable average 2s saved per session per problem (similar to data shown earlier)
2. Typical number of sessions of use per system
• Assume an average of 50 sessions
3. Number of users (licenses)
• Assume an average of 3000 users
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 34
Benefits example 1 - continuedTherefore: total time = 2s * 50 sessions * 3000 users
= 300000 seconds per problem!= 83.3 hours of saved time
If weighted at $60 per hour:• Potential benefit is $5000 per problem
Assuming only half the problems will get fixed:• Benefit = $2500 per problem found
Or added value to a license per problem fixed= 2s * 50 sessions= 100s= $1.66
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 35
Benefits example 2:small in-house system
The same three factors to consider:
1. Same average time savings per session (2s)
2. More average sessions over lifetime (200)
3. Fewer users (40)
Therefore: total time
= 2s * 200 sessions * 40 users
= 16000 s / problem
= 4.4 hours of saved time
Using the same weighted value of $60 per hour:
• Potential benefit is $266 per problem
• or $133 per problem actually fixed.
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 36
Cost benefit analysis
So will a usability study be economical?
• Let’s put the costs and benefits together
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 37
Example 1: Mass-market system
Insp Total benefit % probs total ben/cost
cost /prob found benefits ratio
2 5700 2500 56 280000 49
3 6300 2500 71 355000 56
4 6900 2500 81 405000 59
5 7500 2500 87 435000 58
6 8100 2500 92 460000 57
7 8700 2500 95 475000 55
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 38
Example 2: In-house system
Insp Total benefit % probs total ben/cost
cost /prob found benefits ratio
1 5098 132 34 8976 1.8
2 5700 133 56 14896 2.6
3 6300 133 71 18886 3.0
4 6900 133 81 21546 3.1
5 7500 133 87 23142 3.1
6 8100 133 92 24472 3.0
7 8700 133 95 25270 2.9
Odds and Ends
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 39
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 40
Prototyping
A prototyping tool should:
• Be easy to learn and use
• Possess good visual capabilities
• Allow easy extension
• Give full support for the types of interface being developed
• Support modular programming
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 41
Types of prototyping
Horizontal prototyping
• Complete UI, but no underlying functionality
Vertical prototyping
• In-depth functionality of only certain parts
Scenario prototyping
• Selected paths only
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 42
The design rationale document
Allows you to keep control of the various alternatives considered and decisions made
Explain your usability decisions so others know the reasoning you followed
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 43
Discount usability engineering
User and task observation• Cheap prototyping and scenarios• Simplified thinking aloud• Heuristic evaluation
Key goal, communicate qualitative information to developers in order to improve interfaces
• as opposed to qualitative measures
A study was performed of iterative design involving discount usability engineering
• Improvements of 38% after each stage
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 44
Consumability - Closely allied to usability
Ability for customers to get rapid return on investment.
Factors include
• all aspects of usability
• installability / uninstallability
• lack of inter-product dependency/interaction
• low footprint: lack of run-time resource consumption
• backward-compatibility
• integratability
• appropriate prioritization of iterations and features to yield business value
• low purchase cost
Rating corporate maturity with respect to usability
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 45
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 46
The Model for Usability Maturity (MUM)(proposed by Lethbridge!)
A 5-level CMM-like categorization of organizations
—Based on their interaction with human end-users in software development
—(Each level beyond 1 builds on the previous levels)
• Level 1: Haphazard
• Level 2: Defined input from users / usability awareness
• Level 3: Iterative interaction with users / design for usability
• Level 4: Controlled and measured involvement of users
• Level 5: Continually improving usability
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 47
Level 1: Haphazard
If you can get users to use the system, then it is considered good enough!
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 48
Level 2: Defined input from users / usability awareness
Users involved in requirements reviews• Feedback from users at reviews is incorporated into the
next stage
Design team members have basic training in usability
Design team adheres to usability standards for look and feel
Reuse of well-understood controls, styles etc.
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 49
Level 3: Iterative interaction with users/ design for usability
Users actively involved in decision making
Use case / task analysis
Competitive analysis
Design with careful attention to usability guidelines
Usability design decisions are carefully analyzed and the decisions are recorded
• E.g. options analysis, tables of pros and cons
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 50
Level 3: Continued
User input and feedback with repeated prototypes
• From paper prototypes to functional prototypes
Informal qualitative observations of users or heuristic evaluations
Discount usability engineering
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 51
Level 4: Controlled and measured involvement of users
User input and feedback at all stages
Design team has trained usability experts or cognitive scientists
Careful cost-benefit optimization including usability
Setting of quantitative usability objectives
• For all parts of the system that will be subject to regular or critical-situation use
• For learnability, efficiency of use, etc.
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 52
Level 4: Continued
User-centered design with formal usability studies
Measurement of usability so as to determine progress towards goals
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 53
Level 5: Continually improving usability
Active development of new UI understanding and innovations
Formal experiments to validate new UI modalities• metaphors, controls, widgets, styles etc.
Anthropological studies of human tasks• to enable optimized UI design
Scientific study of• Users and their work practices • Usability and the software engineering process
Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 54
Potential benefits of MUM
It provides a visible framework that companies can use to incrementally improve their practices
If even a few companies adopted it and were certified, they might have a competitive advantage in some situations
Recommended