1

1

CS 544 Discount Usability Engineering

Heuristic Evaluation

Acknowledgement: Some of the material in these lectures is based on material prepared for similar courses by Saul Greenberg (University of Calgary), Ravin Balakrishnan (University of Toronto), James Landay(University of California at Berkeley), monica schraefel (University of Toronto), and Colin Ware (University of New Hampshire). Used with the permission of the respective original authors.

2

Usability Inspectionis the generic name for a set of methods based on having evaluators inspect or examine usability-related aspects of a user interfaceusability inspectors can be usability specialists, software development consultants with special expertise, end users with task/content knowledge, or other professionals4 ways to evaluate an interface:

– automatically (compute usability measures using evaluation software)

– empirically (usability testing with real users)– formally (using models to calculate usability measures)– informally (rule of thumb and general knowledge of evaluators)

usability inspection is an informal evaluation

2

3

Discount Usability EngineeringDeveloped by Nielson, based on using four

techniques:User and task observationScenariosSimplified thinking aloudHeuristic evaluation

4

Articulate:•who users are•their key tasks

User and task descriptions

Goals:

Methods:

Products:

Brainstorm designs

Task centered system design

Participatory design

User-centered design

Evaluate

Psychology of everyday things

User involvement

Representation & metaphors

low fidelity prototyping methods

Throw-away paper prototypes

Participatory interaction

Task scenario walk-through

Refined designs

Graphical screen design

Interface guidelines

Style guides

high fidelity prototyping methods

Testable prototypes

Usability testing

Heuristic evaluation

Completed designs

Alpha/beta systems orcomplete specification

Field testing

Interface Design and Usability Engineering

3

5

Discount usability engineeringCheap– no special labs or equipment needed– the more careful you are, the better it gets

Fast– on order of 1 day to apply– standard usability testing may take a week

Easy to use– can be taught in 2-4 hours

6

Heuristic EvaluationDeveloped by Jakob NielsenPrinciples used to “evaluate” a system for usability problemsBecoming very popular

user involvement not requiredcatches many design flaws

Small set (3-5) of evaluators examine UI– independently check for compliance with usability principles

(“heuristics”)– different evaluators will find different problems– evaluators only communicate afterwards

findings are then aggregatedCan perform on working UI or on sketches

4

7

Why multiple evaluators?Every evaluator doesn’t find every problemGood evaluators find both easy & hard ones

8

Heuristic Evaluation ProcessEvaluators go through UI several times– inspect various dialogue elements– compare with list of usability principles– consider other principles/results that come to mind

Usability principles– Nielsen’s “heuristics”– supplementary list of category-specific heuristics

competitive analysis & user testing of existing products

Use violations to redesign/fix problems

5

9

Neilson’s Heuristics (original)H1-1: Simple & natural dialogH1-2: Speak the users’ languageH1-3: Minimize users’ memory loadH1-4: ConsistencyH1-5: FeedbackH1-6: Clearly marked exitsH1-7: ShortcutsH1-8: Precise & constructive error messagesH1-9: Prevent errorsH1-10: Help and documentation

10

H2-1: Visibility of system statuskeep users informed about what is going on

– example: pay attention to response time 0.1 sec: no special indicators needed, why? 1.0 sec: user tends to lose track of data 10 sec: max. duration if user to stay focused on action for longer delays, use percent-done progress bars

searching database for matches

6

11

H2-1: Visibility of system statuskeep users informed about what is going on– Appropriate visible feedback

What did I select?

What mode am I in now?

How is the system

interpreting my actions?

12

H2-2: Match between system & real world

speak the users’ languagefollow real world conventions

(bad) example: Mac desktop– Dragging disk to trash

should delete it, not eject it!

7

13

H2-2: Match between system & real world

speak the users’ languagefollow real world conventions

e.g. withdrawing money from a bank machine

14

H2-3: User control & freedom“exits” for mistaken choices, undo, redodon’t force down fixed paths

How do I get out of this?

8

15

H2-3: User control & freedom“exits” for mistaken choices, undo, redodon’t force down fixed paths

Strategies:– Cancel button (for dialogs waiting for user input)– Universal Undo (can get back to previous state)– Interrupt (especially for lengthy operations)– Quit (for leaving the program at any time) – Defaults (for restoring a property sheet)

16

H2-4: Consistency & standardsConsistency of effects

– same words, commands, actions will always have the same effect in equivalent situations

predictability

Consistency of language and graphics– same info/controls in same location on all screens/dialog boxes

– forms follow boiler plate– same visual appearance across the system (e.g. widgets)

e.g. different scroll bars in a single window system!

Consistency of input– consistent syntax across complete system

9

17

H2-4: Consistency & standards

These are labels with a raised appearance.

Is it any surprise that people try and click on them?

18

H2-5: Error preventiontry to make errors impossiblemodern widgets: only “legal commands” selected, or “legal data” entered

Provide reasonableness checks on input data– on entering order for office supplies

5000 pencils is an unusually large order. Do you really want to order that many?

10

19

H2-6: Recognition rather than recallComputers good at remembering things, people aren’t!Promote recognition over recall

– menus, icons, choice dialog boxes vs command lines, field formats– relies on visibility of objects to the user (but less is more!)

20

H2-7: Flexibility and efficiency of useExperienced users should be able to perform frequently used operations quicklyStrategies:

– keyboard and mouse acceleratorsabbreviationscommand completionmenu shortcutsfunction keysdouble clicking vs menu selection

– type-ahead (entering input before the system is ready for it) – navigation jumps

e.g., going to window/location directly, and avoiding intermediate nodes

– history systems WWW: ~60% of pages are revisits

11

21

H2-7: Flexibility and efficiency of useKeyboard

accelerators for menus

Customizable toolbars andpalettes for

frequent actions

Split menu, with recently used fonts on top

Scrolling controls for page-sized increments

Double-click raises object-specific menu

Double-click raises toolbar

dialog box

22

H2-8: Aesthetic and minimalist designNo irrelevant information in dialoguesBad example:

12

23

H2-9: Help users recognize, diagnose, and recover from errors

error messages in plain languageprecisely indicate the problemconstructively suggest a solution

24

Errors we make

Mistakes– arise from conscious deliberations that lead to an

error instead of the correct solutionSlips– unconscious behaviour that gets misdirected en

route to satisfying goale.g. drive to store, end up in the office

– shows up frequently in skilled behaviourusually due to inattention

– often arises from similarities of actions

13

25

Types of slips

Capture error– frequently done activity takes charge instead of one intended

occurs when common and rarer actions have same initial sequence– change clothes for dinner and find oneself in bed (William James, 1890)– confirm saving of a file when you don’t want to delete it

I can’t believe I pressed

Yes...

26

Types of slips

Description error– intended action has much in common with others that are possible

usually occurs when right and wrong objects physically near each other– pour juice into bowl instead of glass – go jogging, come home, throw sweaty shirt in toilet instead of laundry basket– move file to trash instead of to folder

Loss of activation– forgetting what the goal is while undergoing the sequence of actions

start going to room and forget why you are going therenavigating menus/dialogs and can’t remember what you are looking forbut continue action to remember (or go back to beginning)!

Mode errors– people do actions in one mode thinking they are in another

refer to file that’s in a different directorylook for commands / menu options that are not relevant

14

27

Designing for slips

General rules– Prevent slips before they occur– Detect and correct slips when they do occur– User correction through feedback and undo

Examples– mode errors

have as few modes as possible (preferably none)make modes highly visible

– capture errorsinstead of confirmation, make actions undoableallows reconsideration of action by user

– e.g. Mac trash can can be opened and “deleted” file taken back out– loss of activation

if system knows goal, make it explicitif not, allow person to see path taken

– description errorsin icon-based interfaces, make sure icons are not too similar, check for reasonable input, etc.

28

Generic system responses for errors

General idea: Forcing functions– prevent / mitigate continuation of wrongful action

Gag– deals with errors by preventing the user from continuing

eg cannot get past login screen until correct password entered

Warn– warn people that an unusual situation is occurring– when overused, becomes an irritant

e.g., – audible bell – alert box

Do nothing– illegal action just doesn’t do anything– user must infer what happened

enter letter into a numeric-only field (key clicks ignored)put a file icon on top of another file icon (returns it to original position)

15

29

Generic system responses for errors

Self-correct– system guesses legal action and does it instead– but leads to a problem of trust

spelling correctorLets talk about it

– system initiates dialog with user to come up with solution to the problemcompile error brings up offending line in source code

Teach me– system asks user what the action was supposed to have meant– action then becomes a legal one

30

H2-10: Help and documentation

Help is not a replacement for bad design!

Simple systems:– walk up and use; minimal instructions

Most other systems:– feature rich– some users will want to become “experts” rather than “casual” users– intermediate users need reminding, plus a learning path

Many users do not read manuals– prefer to spend their time pursuing their task

Usually used when users are in some kind of panic, need immediate help– indicates need for online documentation, good search/lookup tools– online help can be specific to current context– paper manuals unavailable in many businesses!

e.g. single copy locked away in system administrator’s officeSometimes used for quick reference

– syntax of actions, possibilities...– list of shortcuts ...

16

31

Types of help

Tutorial and/or getting started manuals– short guides that people are likely to read when first obtaining their

systemsencourages exploration and getting to know the systemtries to get conceptual material across and essential syntax

– on-line “tours”, exercises, and demosdemonstrates very basic principles through working examples

Reference manuals– used mostly for detailed lookup by experts

rarely introduces conceptsthematically arranged

– on-line hypertextsearch / findtable of contentsindexcross-index

32

Types of help (cont.)Reminders– short reference cards

expert user who just wants to check factsnovice who wants to get overview of system’s capabilities

– keyboard templatesshortcuts/syntactic meanings of keys; recognition vs. recall; capabilities

– tooltipstext over graphical items indicates their meaning or purpose

17

33

Types of help (cont.)Context-sensitive help– system provides help on the interface component

the user is currently working withMacintosh “balloon help”Microsoft “What’s this” help

– brief help explaining whatever the user is pointing at on the screen

Wizards– walks user through typical tasks– but dangerous if user gets stuck– Remember the MS “paperclip”?

34

How to perform evaluationAt least two passes for each evaluator

– first to get feel for flow and scope of system– second to focus on specific elements

If system is walk-up-and-use or evaluators are domain experts, no assistance needed

– otherwise might supply evaluators with scenarios

Each evaluator produces list of problems– explain why with reference to heuristic or other information– be specific and list each problem separately

18

35

How to perform evaluation (cont.)Why separate listings for each violation?– risk of repeating problematic aspect– may not be possible to fix all problems

Where problems may be found– single location in UI– two or more locations that need to be compared– problem with overall structure of UI– something that is missing

hard w/ paper prototypes so work extra hard on thosenote: sometimes features are implied by design docs and just haven’t been “implemented” – relax on those

36

Severity ratingsUsed to allocate resources to fix problems Estimates of need for more usability effortsCombination of– frequency– impact– persistence (one time or repeating)

Should be calculated after all evals. are inShould be done independently by all judges

19

37

Severity ratings (cont).0 - don’t agree that this is a usability problem1 - cosmetic problem 2 - minor usability problem3 - major usability problem; important to fix4 - usability catastrophe; imperative to fix

38

SummaryAdvantages

– the “minimalist” approacha few general guidelines can correct for the majority of usability problemseasily remembered, easily applied with modest effort

– discount usability engineeringcheap and fast way to inspect a systemcan be done by usability experts, double experts, and end users

Problems:– principles are more or less at the motherhood level

can’t be treated as a simple checklistsubtleties involved in their use

– doesn’t necessarily predict user/customers overall satisfaction

– may not have the same “credibility” as user test data (one solution to this is to include the design team and developers in the usability evaluation, as in a “pluralistic walkthrough”)

20

39

SummaryResearch result:

– between 4-5 evaluators appear to be sufficient in most cases to identify 80% of total usability problems

– user testing and usability inspection have a large degree of non-overlap in the usability problems they find (i.e., it pays to use both methods)

Cost-benefit:– usability engineering activities are often difficult to justify and

carry out in a timely way, but many activities can be done quickly and cheaply, and produce useful results

– usability inspection turns less on what is “correct” than on what can be done within development constraints

– ultimate trade-off may be between doing no usability assessment and doing some kind

40

You now know10 Heuristics for evaluating designs:

– H2-1: Visibility of system status – H2-2: Match between system & real world– H2-3: User control & freedom – H2-4: Consistency & standards– H2-5: Error prevention– H2-6: Recognition rather than recall– H2-7: Flexibility and efficiency of use– H2-8: Aesthetic and minimalist design– H2-9: Help users recognize, diagnose, and recover from errors – H2-10: Help and documentation

Principles can be used to systematically inspect the interface for usability problems

21

41

Readings and ReferencesBGBG 80-86 (Evaluating Systems and Their User Interfaces)

BGBG 170-181 (Usability Inspection Methods)

Optional:Neilsen, J. (1993). Usability Engineering. Read Chapter 5: Usability Heuristics, 115 – 163.