Upload
melvin-wilkinson
View
217
Download
2
Embed Size (px)
Citation preview
© 2012 IBM Corporation
Toshiba Burns-Johnson and Cheryl D. Loughlin2012 Enterprise Computing Community Conference at Marist College
Usability Study: The Key Ingredient in Designing Effective Help Systems
© 2012 IBM Corporation2
Repurposing an iPAD: What a clever idea!
http://www.wimp.com/dadipad/
© 2012 IBM Corporation3
Agenda
Purpose and perceptions of help systems
Introduction to our study
Method for conducting our study
Results of our study
Benefits of conducting a usability study
Resources
© 2012 IBM Corporation4
What’s the purpose of a help system?
© 2012 IBM Corporation5
Purpose of a help system
Translates software function into tasks that users understand
Teaches users concepts required to perform tasks
Familiarizes users with the user interface
Addresses the information needs of a diverse audience
Helps users resolve error conditions
© 2012 IBM Corporation6
Do you use help?
© 2012 IBM Corporation7
Impact of negative perceptions of help systems
Under Utilization of Help Systems
Negative External Views
Negative Internal Views
Decreased Enthusiasm in Help System Development
© 2012 IBM Corporation8
Agenda
Purpose and perceptions of help systems
Introduction to our study
Method for conducting our study
Results of our study
Benefits of conducting a usability study
Resources
© 2012 IBM Corporation9
Introduction: The usability study
What is it?– A controlled experiment where users are observed using a
product or help system to determine the effectiveness and efficiency of the design in terms of human interaction and satisfaction.
Why use it?– To validate the design of a product or help system– To uncover unexpected user behaviors – To uncover unforeseen problems – To learn users' expectations
© 2012 IBM Corporation10
Modernizes the “face” of z/OS
Provides an integrated help system
Makes z/OS accessible to a new generation of system programmers
Reduces z/OS complexity
Introduction: IBM® z/OS® Management Facility (z/OSMF)
© 2012 IBM Corporation11
Introduction: The help system
© 2012 IBM Corporation12
Agenda
Purpose and perceptions of help systems
Introduction to our study
Method for conducting our study
Results of our study
Benefits of conducting a usability study
Resources
© 2012 IBM Corporation13
Role in IBM
56%22%
22%
Information Developer
System Tester
Usability Specialist
Role in IBM
56%22%
22%
Information Developer
System Tester
Usability Specialist
Method: Demographics
Years in Current Role
22%
56%
22%
1-56-1011+
Years in Current Role
22%
56%
22%
1-56-1011+
Gender
56%
44%
MaleFemale
Gender
56%
44%
MaleFemale
Product Experience
44%
56%
NoneSome
Product Experience
44%
56%
NoneSome
n = 9 n = 9
n = 9 n = 9
© 2012 IBM Corporation14
Method: Creating user scenarios
Identified actions that require support of the help system
Created 28 questions that users might have while completing each action
Incorporated the 28 questions into 10 task-based user scenarios
© 2012 IBM Corporation15
Method: Sample user scenario
User Interface: Define a new desktop and add a metric to the desktop. This desktop should contain only generic sysplex resources.
Help: What’s the benefit of defining generic sysplex resources?
© 2012 IBM Corporation16
Method: Conducting feedback sessions
The participants:
Accessed the software directly
Navigated to the page where they would complete the user interface portion of the scenario
Accessed the help system from that page to answer the help portion of the scenario
Verbalized their thoughts as they completed each task
© 2012 IBM Corporation17
Method: Conducting feedback sessions
We, the researchers:
Conducted the review with one participant at a time
Imposed no time limit for completing the scenarios
Emphasized that we were testing the help system, not the participant
Documented the completion rate and any errors or problems identified
© 2012 IBM Corporation18
Method: Categorizing and prioritizing issues
Each issue was assigned a:
Category– Help System, User Interface, or Both
Problem type– Clarity, Conciseness, Structure, or Missing Content
Priority – Low, Medium, or High
Resolution complexity level – Easy, Moderate, or Difficult
© 2012 IBM Corporation19
Agenda
Purpose and perceptions of help systems
Introduction to our study
Method for conducting our study
Results of our study
Benefits of conducting a usability study
Resources
© 2012 IBM Corporation20
Results: Types of issues identified
6
8
45
10
12
0
2
4
0 00
2
4
6
8
10
12
14
Help User Interface Both
Impact Area
Num
ber
of
Issu
es
Clarity Missing Content Structure Conciseness
6
8
45
10
12
0
2
4
0 00
2
4
6
8
10
12
14
Help User Interface Both
Impact Area
Num
ber
of
Issu
es
Clarity Missing Content Structure Concisenessn = 42
© 2012 IBM Corporation21
Results: Example issues
Impact Area Problem Type Example of Issues
Help System
Structure Participants reported that paragraphs are more difficult to scan than tables, lists, and section headers.
Conciseness Participants reported that there was too much text overall.
User Interface Clarity Participants identified icons that were unclear.
BothStructure Participants reported that it was unclear how to
suppress messages.
© 2012 IBM Corporation22
Results: Priority of issues identified
Priority Example of Issues
Low Participants reported not being aware that the help system was context sensitive.
Medium Participants had a difficult time locating other help topics that were related to a specific topic.
High Participants reported that the help topics were long and verbose.
10%
52%
38%
LowMediumHigh
10%
52%
38%
LowMediumHighn = 42
© 2012 IBM Corporation23
Results: Complexity of resolutions
Complexity Level Example of Resolutions
Easy Add concise text to the help system or interface to clarify an icon, an action, or a dependency between tasks.
Moderate Simplify help topics so that they address only one concept.
Difficult Restructure the entire help system so that it doesn’t require users to know what page they are on in the interface.
47%
43%
10%
EasyModerateDifficult
47%
43%
10%
EasyModerateDifficultn = 42
© 2012 IBM Corporation24
Agenda
Purpose and perceptions of help systems
Introduction to our study
Method for conducting our study
Results of our study
Benefits of conducting a usability study
Resources
© 2012 IBM Corporation25
What are the benefits of conducting a usability study of a help system?
© 2012 IBM Corporation26
Benefits of conducting a usability study of a help system
Improve the help system
Improve the usability of the software
Increase the accessibility of the software
Identify additional user requirements
Increase customer satisfaction
Reduce customer calls to technical support
© 2012 IBM Corporation27
Ultimate benefit: Positive perceptions of help systems
Increased Utilization of Help Systems
Positive External Views
Positive Internal Views
Increased Enthusiasm in Help System Development
© 2012 IBM Corporation28
Agenda
Purpose and perceptions of help systems
Introduction to our study
Method for conducting our study
Results of our study
Benefits of conducting a usability study
Resources
© 2012 IBM Corporation29
Questions
© 2012 IBM Corporation30
Resources
User Centered Design at IBM– https://www.ibm.com/software/ucd/ucd.html
IBM z/OS Management Facility– Web Site
• http://www.ibm.com/systems/z/os/zos/zosmf/ – Modernizing the Mainframe article
• http://www.ibm.com/software/ucd/gallery/zosmf_systems.html– Demos
• http://publib.boulder.ibm.com/infocenter/ieduasst/stgv1r0/index.jsp?topic=/com.ibm.iea.zosmf/plugin_coverpage.html