Baobab Health 2015 Usability Inspections

Usability Inspections: Heuristic Evaluation

Harry Hochheiser Biomedical Informatics

[email protected]

Harry Hochheiser, [email protected] Baobab Health, February 2015

mailto:[email protected]


Goal of Usability Assessments

• Can users complete tasks?

• Appropriateness of mental models

• Comparative efficiency

• Subjective satisfaction

• How do we assess?


No bright lines

DesignFunctional

system Paper Prototype Release

Usability Inspections

Usability Studies

Empirical User Studies, Case Studies, Longitudinal Studies, Acceptance Tests

Low cost, low validity Higher cost, validity


Usability Inspections

• “clean-room” static examination of usability • Methodically scrutinize interfaces in search of potential

problems

• Pros: – Inexpensive – no users, relatively easy – Identify major issues at a relatively early stage

• Cons: – May miss problems: generally find < 50% – All results are hypothetical – don’t know which problems might

really lead to errors


Common Questions across inspection types

● Who inspects? ●Background ●How many?

● Which tasks? ● How to interpret findings?

●Problems are not “real” problems experience by users ●How important are they?


Broad classes of inspections

Heuristic Evaluations: How well does an interface conform to guidelines for interface design?

Walkthrough: Analytic examination of interface and interaction requirements, usually informed by some model of the user

Many variants...


Who inspects? Heuristic inspection

Usability experts Domain experts Combination? (Double experts) Users should participate as users when possible

3-5 experts? (Nielsen)

Or more...

Work alone, or in teams..

Walkthroughs May require more cognitive background

Domain expert feedback helpful

Conducted by a team?


Tasks

Heuristic inspections Set tasks Open-ended exploration

Walkthroughs Generally, specific tasks


Nielsen's Heuristics Nielsen, 1994 - http://www.nngroup.com/articles/ten-usability-heuristics/

• Visibility of system status • Match between system and real world • User control and freedom • Consistency and standards • Error prevention • Recognition rather than recall • Flexibility and efficiency of use • Aesthetic and minimalist design • Help users recognize, diagnose, and recover from errors • Help and documentation


Results Nielsen and Molich 1990 , Nielsen 1993

• Few false positives • Find < 50% of errors • “In general, we would expect aggregates of five

evaluators to find about two thirds of the usability problems which is really quite good for an informal and inexpensive technique like heuristic evaluation.”


Results Nielsen and Molich 1990


Results Nielsen and Molich 1990

Source of the “you only need 5 evaluators” rule… Hotly contested…


Which Evaluators? Nielsen, 1992

• “Double Experts” - Domain and Usability expertise – considered best

• Not always available – use some of each, or teams


Heuristic Evaluation ProcedureEvaluators work alone

(except for when they work in teams)

Optional observer can help explain confusing issues and to record issues.

Go through interface several times List specific tasks

Note discrepancies between interface and heuristic Note any concerns - even those that don’t exactly fit heuristics

Individual evaluators meet to aggregate results agree on interpretation and prioritization



• Visibility of system status •“The system should always keep users informed about what is going on, through appropriate feedback within reasonable time”

• Match between system and real world “The system should speak the users’ language, with words, phrases, and

concepts familiar to the user, rather than system-oriented terms. Follow real-

world conventions, making information appear in a natural and logical order.”

• User control and freedom

“Users often choose system functions by mistake and will need a clearly marked

"emergency exit" to leave the unwanted state without having to go through an

extended dialogue. Support undo and redo.”



• Consistency and standards “Users should not have to wonder whether different words, situations,

or actions mean the same thing. Follow platform conventions.”

• Error prevention “Even better than good error messages is a careful design which prevents a

problem from occurring in the first place. Either eliminate error-prone

conditions or check for them and present users with a confirmation option

before they commit to the action.”

• Recognition rather than recall “Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.”



• Flexibility and efficiency of use “Accelerators -- unseen by the novice user -- may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.”

• Aesthetic and minimalist design • “Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.”



• Help users recognize, diagnose, and recover from errors “Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.”

• Help and documentation “Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.”


How to Interpret?

Use severity judgments to prioritize fixes Frequency of problem Impact of problem Persistence – will users be repeatedly bothered? Multiple independent raters increase reliability

Bigger questions – does this design work at all? As with usability studies, try to generalize

Don't solve lots of small problems if the design is inherently problematic


Severity Ratings

0. No problem 1. Cosmetic 2. Minor 3. Major 4. Catastrophe

Measure both impact and frequency?


Reporting heuristic violations http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/ 6-831-user-interface-design-and-implementation-spring-2011/lecture-notes/MIT6_831S11_lec23.pdf

• Goal - communicate problems • Identify successes • Be constructive • Be specific

http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/


Heuristic Evaluation reports http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/ 6-831-user-interface-design-and-implementation-spring-2011/lecture-notes/MIT6_831S11_lec23.pdf

• Problem • Heuristic • Description • Severity • Recommendations • Screenshot

http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/


Example: NIH Undiagnosed Disease Program UDPICs LIMS tool


Example: NIH Undiagnosed Disease Program UDPICs LIMS tool

Heuristic Visibility of system status; Consistency and standards; Recognition rather than recall

Description The “gear” icon is often used to indicate a menu of systems settings and preference. Although this is not a standard, it is a widely-used convention. UDPICS uses this gear icon to access a menu that contains a variety of options - some of which are settings and others providing additional system functionality Figure 2). This may confuse users who select the menu looking only for settings selections, or for those who look elsewhere for options found in this menu. The use of the gear item also raises concerns for administrative users, who might be confused by the gear icon on the one hand and the “settings and preferences” menu on the left-hand navigation bar.

Severity Minor

Suggested Revisions

One possible solution would be to remove all non-settings items from this menu, adding additional choices to the otherwise underutilized menu bar. Even if the left-hand navigation bar is available only to administrators, placing all settings menu in a single place and adjusting the available options based on user privileges might be easier for users.


Making use of heuristic evaluation results?

● Have member of development team present during inspection?

Can be done with usability studies and other inspections as well They should observe, not analyze

● Debriefing session to explore fixes Consider redesigns where appropriate.


Other Inspection Methods http://www.useit.com/papers/heuristic/inspection_summary.html

Heuristic estimation – compare interfaces

Feature Inspection – look for long sequences of operations, unnatural sequences, those requiring extensive skill...

Consistency inspection – compare designs for consistency

Standards Inspection

http://www.useit.com/papers/heuristic/inspection_summary.html


Which Heuristics?

• Multiple versions of Nielsen’s list • Other versions

– Special cases?

• More heuristics – more focus – ..but less study

• Too many heuristics – cumbersome, complicated


Web Usability Heuristicshttp://www.andybudd.com/archives/2007/01/heuristics_for_modern_web_application_development/index.htm

• Design for user expectations • Clarity • Minimize Unnecessary Complexity and Cognitive Load • Efficiency and Task Completion • Provide Users with Context • Consistency and Standards • Prevent Errors • Help users, notice, understand, and recover from errors • Promote a pleasurable and positive web experience


Usability Evaluation of an Electronic Medication Administration Record (eMAR) Application Guo, et al. 2011

• eMAR tasks – Access to drug information, policies, and procedures related to

meds – Context-specific patient clinical results – Facilitate communication between nurses and pharmacists – Verification of the five “rights” (patient, drug, dose, route, and

time)

• Heuristic Evaluation for usability • Using Zhang’s heuristics and severity ratings


Heuristic evaluation of eMAR Guo, et al. 2011

• Tasks – Login – Order and modify medication – Verify medication orders – Access drug references – Administer medication – Edit eMAR – Generate reports and review

• Procedure – Train evaluators – Conduct evaluation – Calculate agreement – Build consensus – Review with staff – conduct tasks



• Observations – classified by heuristic, – grouped into usability problems – Organized by task






Agreement?

• What if multiple reviewers disagree? • Can calculate inter-rater reliability

• Cohen’s kappa • Raw agreement

• Goal here is to find potential problems - • being thorough is more important than agreeing

Education

Baobab Health 2015 Usability Inspections