Upload
jeff-belden-md
View
113
Download
5
Tags:
Embed Size (px)
DESCRIPTION
Jeff Belden MD and Janey Barnes PhD co-presented at HIMSS Virtual Conference June 2010. You can hear the audio recording if you are a HIMSS member, available online.
Citation preview
1
Usability Methodsto Improve EMRs
Janey Barnes, PhDUser-View, Inc.
Raleigh, NC
Jeff Belden, MD Family & Community MedicineUniversity of Missouri
June 9, 2010
2
Conflict of Interest Disclosure
Janey Barnes, PhD• Contracted Research:
• Allscripts• Patagonia Health• Duke Health Systems
3
Conflict of Interest Disclosure
Jeff Belden, MD • Dr. Belden is on faculty at University of Missouri -
Columbia, and an EMR usability consultant with the Tiger Institute, a technology collaborative between the University of Missouri and Cerner Corporation.
• Dr. Belden has no financial interest.
4
Session Objectives
• Describe stages of software development in which usability methods can be most effective.
• List and describe usability methods applicable to each stage of EMR development.
• Analyze the cost/benefit of implementing usability methods within a user-centered design process.
5
Software Development Process
6
Waterfall modelof software development
7
Waterfall
8
“Agile” modelof software development
9
Agile
10
Usability methods work for both
11
don’t forget the buyer’s sideof usability
12
Selection Implementation
AgileWaterfall
13
Want to be a change-agent?
15
User Centered Design Process
Discover Define Design Develop Beta Launch Post-Launch
16
Discover Define Design Develop Beta Launch Post-Launch
Incorporating User Centered Designin Software Development Process
•Affinity Diagramming•Personas•Use case scenarios
•Requirements•User stories
•Keep testing
•User acceptance testing•Incorporating feedback•Maintain design intent
•Summative testing
•and testing
•Prototyping•Card sorting•Concept exploration•Usability Testing
17
a few words about usability when selecting &
implementing
Selection Implementation
Discover Define Design Develop Beta Launch Post-Launch
Software Selection ActivitiesUsability Methods of Interest• Site visit for:
• User surveys of current users like you• Performance testing
• Observe one of their experienced (but not an ace) users• Do clinical scenario test with real data• Define target measures• Set goals
18Selection Implementation
Discover Define Design Develop Beta Launch Post-Launch
Implementation ActivitiesUsability Methods of Interest• Task analysis & user observation• Card sorting (picking categories, nomenclature)• Design guidelines & Style guides• Paper prototyping• Have clinician create documents for several patients
19Selection Implementation
Discover Define Design Develop Beta Launch Post-Launch
21
Discovery ActivitiesContextual inquiry1. Affinity diagramming2. Personas3. Use case scenarios
• For each method we present• Goal of the method• Details for planning & executing each method• Best timing the software development schedule• How to deliver impact to the product
Discover Define Design Develop Beta Launch Post-Launch
22
Contextual Inquiry
Persona
Discover Define Design Develop Beta Launch Post-Launch
23
Goals• understand the user’s:
• tasks• workflows• environment
• description of a specific person (a target user). Details make them real.
Discover Define Design Develop Beta Launch Post-Launch
Contextual Inquiry
Persona
Planning • Observe users onsite• Be a fly on the wall• Before, during & after• Half day…or longer!• Take photos of environment.
Execution• Watch. Observe. Learn. • Make notes. • Stand, sit, to side, in middle. • Stay out of way• Collect any artifacts.
Discover Define Design Develop Beta Launch Post-Launch
Contextual Inquiry
Persona
• Timing• Early• Requirements gathering• Before sprint planning• Before “iteration 1”
• Having Impact• By watching, observing, learning
Contextual Inquiry
Persona
Discover Define Design Develop Beta Launch Post-Launch
26
Contextual InquiryUse Case Scenarios
27
Contextual InquiryUse Case Scenarios
Goals• Understand the user’s:
• tasks• workflows• environment
• Specify how users carry out their tasks in a specified context.
Discover Define Design Develop Beta Launch Post-Launch
28
Contextual InquiryUse Case Scenarios
Discover Define Design Develop Beta Launch Post-Launch
Planning • Observe users onsite• Be a fly on the wall• Before, during & after• Half day…or longer!• Take photos of
environment.
Execution• Watch. Observe. Learn. • Make notes. • Stand, sit, to side, in
middle. • Stay out of way• Collect any artifacts.
29
Contextual InquiryUse Case Scenarios
Discover Define Design Develop Beta Launch Post-Launch
• Timing• Early• Requirements gathering• Before sprint planning• Before “iteration 1”
• Having Impact• By watching, observing, learning
30
Contextual Inquiry
How long should this take?
Discover Define Design Develop Beta Launch Post-Launch
5 days first time
Day 1
PlanDay 2
PlanDay 3
On-siteDay 4
CompileDay 5
Compile
Further visits could be…
1 dayOn-site
1.5 daysCompile
31
Contextual Inquiry
How long should we plan?
Discover Define Design Develop Beta Launch Post-Launch
Plan• Teach team to observe• Not sell, or teach, or fix• Have tools ready
5 days first time
Day 1
PlanDay 2
PlanDay 3
On-siteDay 4
CompileDay 5
Compile
32
Contextual Inquiry
How long should we stay to observe?
Discover Define Design Develop Beta Launch Post-Launch
It depends• How varied are user groups?• How familiar are you with territory?• Stay until you see patterns emerge
5 days first time
Day 1
PlanDay 2
PlanDay 3
On-siteDay 4
CompileDay 5
Compile
33
Contextual Inquiry
How long to compile findings?
Discover Define Design Develop Beta Launch Post-Launch
Compile into• Personas• Use case scenarios• Work flows• Task flows
5 days first time
Day 1
PlanDay 2
PlanDay 3
On-siteDay 4
CompileDay 5
Compile
35
Definition Activities
1. Requirements 2. User Stories
• Goal• Planning• Execution• Timing• Impact
Discover Define Design Develop Beta Launch Post-Launch
36
Requirements gatheringGoal• Identify usability requirements that can be
tested later• For each chosen task and user type estimate:
• acceptable task time & optimum target• how to score effectiveness - agreeing what errors user might
make• the effectiveness target• the satisfaction target.
Discover Define Design Develop Beta Launch Post-Launch
37
Requirements gathering
• Arrange workshop with users, developers.
• Review tasks• For each task & user type
estimate• Task time & optimum goal• How to score effectiveness• Effectiveness target• Satisfaction Target
• Decide on usability requirements among:• Understandability• Learnability• Operability• Attractiveness
• Recruiting physicians is challenging.
Planning Execution
Discover Define Design Develop Beta Launch Post-Launch
38
Requirements gathering
Timing• Early
Impact• Shows importance of usability early on• Provides concrete objectives & testable criteria
for usability
Discover Define Design Develop Beta Launch Post-Launch
39
User Stories
Discover Define Design Develop Beta Launch Post-Launch
40
User StoriesGoal
• More approachable than a formalized use case
• Slim, give high-level requirements
• Just a sentence or three.• “A surgeon needs latest evolving lab results while moving
through the hospital, without stopping to log-in at a PC”.
Discover Define Design Develop Beta Launch Post-Launch
41
User StoriesPlanning• Arrange short meeting with
customer
• Bring 3x5 cards for each story
Execution
• User writes user stories
• 3x5 note card• Rewrite until clear• Developer may use
questions to get user going
Discover Define Design Develop Beta Launch Post-Launch
42
User Stories
Timing• Early
Impact• Short, quick to generate• Low maintenance• Keeps contact with user• Helps estimate the development work effort
Discover Define Design Develop Beta Launch Post-Launch
44
Design & User Feedback Activities
1. Information Design• Prototyping• Card sorting
2. Interaction Design• Prototyping
3. Visual Design• Concept exploration
Discover Define Design Develop Beta Launch Post-Launch
45
Design & User Feedback Activities
Prototyping
Discover Define Design Develop Beta Launch Post-Launch
46
Prototyping
Goal• clarify requirements• create draft of
interaction designs & screen designs to be rapidly simulated & tested
Discover Define Design Develop Beta Launch Post-Launch
47
Prototyping
Planning• Arrange workshop with
• Users• Developers• Facilitator & scribe
• Four stages• Concept design• Interaction design• Screen design• Screen testing
Discover Define Design Develop Beta Launch Post-Launch
48
PrototypingExecution• Concept design
• Sketch possible approaches
• See if they meet agree objectives
• Interaction design• Use Post-It notes,
writing each suggested screen or activity
• Group them, name clusters, arrange sequences
Discover Define Design Develop Beta Launch Post-Launch
49
PrototypingExecution• Screen design
• Brainstorm screen designs with users
• Ask user to carry out a realistic task, pointing among screen sketches
• Screen testing• Mock-up rough designs.
• Walk thru steps, explaining what happens, or get fancier with paper menus, dialog boxes, etc.
Discover Define Design Develop Beta Launch Post-Launch
50
Prototyping
Timing• Early• Before codeImpact• Find usability problems early before code is
written• Communication between users & designers is promoted• Paper is cheap & quick
Discover Define Design Develop Beta Launch Post-Launch
51
Design & User Feedback Activities
Card sorting
Discover Define Design Develop Beta Launch Post-Launch
52
Card SortingGoal• Aids information design• Discover latent structure in an unsorted list
of ideas• Example: “Group these smart-phone functions into
ones that you need, and ones that would be optional. Then rank the needed ones in order of importance.”
Discover Define Design Develop Beta Launch Post-Launch
53
Card SortingPlanning• Make separate cards for
each statement• Number cards on back• Find at least 6 users
Execution• Shuffle the deck each time.• Have users group cards• Two styles of card sort
• Closed (you supply category)• Open (user names groups)
• Note results. Include names user gave groups, proximity info or comments user made.
Discover Define Design Develop Beta Launch Post-Launch
54
Card SortingTiming• Design phase• Before codingImpact• Helps discover users’ mental model• Shows how ideas or concepts should be
presented
Discover Define Design Develop Beta Launch Post-Launch
Speaker handoff
56
Design & User Feedback Activities
Visual DesignConcept Exploration • Goal
• Communicating information organization• Communicating information priority• Engaging brand
Discover Define Design Develop Beta Launch Post-Launch
57
Visual Design
It’s not just pretty colors…
Discover Define Design Develop Beta Launch Post-Launch
58
Visual Design
Contrast | Repetition |Alignment |ProximityContrast | Repetition | Alignment |
ProximityContrast | Repetition | Alignment | Proximity
Discover Define Design Develop Beta Launch Post-Launch
59
Visual Design
Good visual design communicates: Info organizationInfo priority
and reduces cognitive load
Discover Define Design Develop Beta Launch Post-Launch
60
Design & User Feedback Activities
Visual DesignConcept Exploration • Planning: Iterative Process• Execution: Research methods to
quantify effectiveness of Visual Design • Timing: In parallel with information &
interaction design• Impact: Critical! Visual Design is not
just “make it pretty”
Discover Define Design Develop Beta Launch Post-Launch
Use CasesNew customer: Salesman wants to add new customer information, but doesn’t have a truck to spec at this time.
Edit customer: Salesman needs to update the street address of an existing long-time customer.
New Truck/Existing Customer: Salesman needs to create a new spec and relate to an existing customer.
Existing Truck/Existing Customer: Salesman needs to relate existing truck spec to current customer.
Existing Truck/New Customer:Salesman need to input new customer information and related to existing spec
Affinity Diagramming
PrototypingLow-resolution High-resolution
64
Design & User Feedback Activities
Usability Tests
Discover Define Design Develop Beta Launch Post-Launch
65
Usability Tests
Discover Define Design Develop Beta Launch Post-Launch
66
Design & User Feedback Activities
Usability Tests - one-on-one sessions where participant user performs key/at risk tasks while researcher gathers data related to performance
Goals
Discover Define Design Develop Beta Launch Post-Launch
• Formative• Design decisions• Identify usability issues
• Summative• Evaluate usability goals• Validate design
Planning • Test Objectives• Test Design• Test Plan including data analysis plan
• Test Participants• Test Environment• Test Report
Execution• Pilot Test• Logistics• Carry out plan
including test and data analysis
• Share with team & IMPACT design
Usability Tests
Formative and Summative
67
68
Timing
Usability Tests
Formative and Summative
Discover Define Design Develop Beta Launch Post-Launch
Formative• Earlier• As soon as you have use cases in a prototype
Summative• Later• Enough• Stability• Early enough to make IMPACT
Fit to sprints• UX a sprint ahead• UX doing formative activities for next sprint and summative activities for current sprint
• Having Impact• Actionable & Prioritized
Recommendations• Here are 3 examples…
Usability Tests
Formative and Summative
Discover Define Design Develop Beta Launch Post-Launch
Usability TestsActionable & prioritized recommendations
Discover Define Design Develop Beta Launch Post-Launch
High Priority
Users (patients) cannot find Clinician’s name in the list of physicians (appointment task). Put names in alphabetical order
High Priority
Users (physicians) cannot find “New Prescription” action button. Make “New Prescription” action button visually apparent.
High Priority
Users (physicians) cannot find patient’s name on open chart quickly. Put the patient’s name in a visually distinct location on the screen and be consistent on all screens.
71
Development Activities
1. User Acceptance Testing (Software)2. Incorporating user feedback3. Maintaining the intent of the design
Discover Define Design Develop Beta Launch Post-Launch
Keep the door open betweenUX and Development!
72
Development Activities1. User Acceptance Testing (Software)
• Goal: Validate that the application works as intended
• Planning: Begins during Define Phase• Execution: Automatic versus Manual• Timing: Nearing the end• Impact: Too late to IMPACT the application?
Discover Define Design Develop Beta Launch Post-Launch
73
Development Activities2. Incorporating user feedback3. Maintaining the intent of the design
• Goal: SUPPORT development team• Planning: Little planning time – rely on process• Execution: SUPPORT• Timing: Nearing the end• Impact: CRITICAL time
Discover Define Design Develop Beta Launch Post-Launch
74
Beta Testing Activities
Discover Define Design Develop Beta Launch Post-Launch
GoalFrom Usability Perspective:
• Live usability test to inform current application and next release
• System-wide usability test • Preparation for implementations
Usability Activitiesduring Beta Testing
75
Beta Testing Activities
Discover Define Design Develop Beta Launch Post-Launch
Planning • Test Objectives• Test Design• Test Plan
including data analysis plan• Test Participants• Test Environment• Test Report
Execution• Pilot Test• Logistics• Carry out plan
including test and data analysis• Share with team &
IMPACT design, implementation
Usability Activitiesduring Beta Testing
76
Beta Testing ActivitiesUsability Activities
during Beta Testing
Discover Define Design Develop Beta Launch Post-Launch
Timing: We are nearing the endImpact: Not too late to impact the application
77
Launch Activities
Discover Define Design Develop Beta Launch Post-Launch
Goal• From Usability Perspective:
• Live usability test to inform current application and next release
• System-wide usability test
Continued Testing
78
Launch ActivitiesContinued Testing
Discover Define Design Develop Beta Launch Post-Launch
Planning • Test Objectives• Test Design• Test Plan
including data analysis plan• Test Participants• Test Environment• Test Report
Execution• Pilot Test• Logistics• Carry out plan
including test and data analysis• Share with team &
IMPACT design, implementation
79
Launch Activities
Discover Define Design Develop Beta Launch Post-Launch
Timing: We are nearing the endImpact: Not too late to impact the application
Continued Testing
80
Post Launch ActivitiesLearnings Discover Activities
DiscoverDefine
Design
Develop
Beta
Laun
ch
Post-
Launch
Post-
Launch
Implementation
Implementation
Selection
81
Cost / Benefit
82
Cost/Benefit
Key to cost-effective product usability is to
Plan & Managethe usability activities within the development process.
83
Cost/Benefit
specific activity designed to gather data to address specific research question
•Affinity Diagramming•Personas•Use case scenarios
•Requirements•User stories
•Keep testing
•User acceptance testing•Incorporating feedback•Maintain design intent
•Summative testing
•and testing•Prototyping•Card sorting•Concept exploration•Usability Testing
Discover Define Design Develop Beta Launch Post-Launch
Formative Summative Formative
84http://bit.ly/UXmethods
85
CostsIncluding Usabilityin Development Cycle
Planning Design Development Release
Design Alternatives
Cost of Changes
86
Costs Usability in the Development Cycle 1 of 2
• Discount Usability vs Premium Usability• Time (labor)
Employee vs Contractor (expendable)• Design vs Evaluation• Test system setup• Participant recruitment and compensation
87
Costs Usability in the Development Cycle 2 of 2
• Lab equipment & usability lab facility• Travel expenses• Planning Time !!
• Managers must make the time to plan the integration of usability into the development process.
• Sometimes too busy to stop and determine how to integrate usability during development.
88
Measurable Benefitsof Usability Integration
• Improved productivity• Shorter time to
complete tasks• Fewer errors during
tasks• Improved user
satisfaction• Lower training costs
• Reduced tech support & documentation costs
• Increased patient safety• Litigation deterrence• User preferences
• Brand loyalty• Marketing voice
89
Tradeoffs between Usability Costs & Benefits
• Measurable benefits not well documented• “Post” data is often not collected due to
budget and lack of vision• Proprietary data used competitively and not
available to the public• Usability community has made efforts to
quantify benefit of Usability interventions
90
Tradeoffs between Usability Costs & Benefits
• Case scenarios presented as examples that hopefully demonstrate the cost-benefit tradeoffs across several types of HF activities.
• Data for case scenarios taken from:Good Ergonomics Is Good Economics (1996). Hal
Hendrick. Proceedings of the HFES 40th.Usability Professionals’ Association (UPA) http://
www.upassoc.org
91
Best Bang for the BuckKey Usability Activities
• Contextual Inquiry• Usability Testing (1 on 1) with
prototype or early code• Enough functionality to test• Early enough to make changes
Discover Define Design Develop Beta Launch Post-Launch
92
Session Objectives1. Describe stages of software development in which
usability methods can be most effective.• Requirements & Design
2. List and describe usability methods applicable to each stage of EMR development.• Contextual Inquiry & Usability Testing
3. Analyze the cost/benefit of implementing usability methods within a user-centered design process.• Early usability methods are most cost effective in terms
of planning, carrying out, and incorporating findings in software.
Links• Usability Methods Table | Usability.net• HIMSS EHR Usability Links | Online resources
Questions?For further info, contact:Janey Barnes PhD | [email protected]
Jeff Belden MD | [email protected]