Embed Size (px)
DESCRIPTION
Usability tests are meant to find usability problems. If your question is, “where are the usability problems in this design”, usability testing is right for you. With usability testing, can study how well someone can get from point A to point B and where are the problems along the way. Finding usability problems is the focus, and the method works great.But, we are finding that many of the questions business sponsors and stakeholders have are not about finding usability problems. The questions they have are more about the overall usefulness of a design, its potential for success, and how well it meets expectations.This presentation will define usefulness research, show how it is different from usability tests, and offer different approaches for asking the right questions of users. Whether you think this is slap-your-forehead obvious or a method that needs to be expanded and refined, we seek to have a lively conversation.
Citation preview
Prepared by:
Michael Hawley – Chief Design Officer
Daniel Berlin – Experience Research Director
May 25, 2011
Beyond Usability Testing: Assessing the Usefulness of Your DesignUPA Boston Mini-Conference 2011
Have You Run Usability Tests?
Usability Testing
Participants attempt to complete a set of defined tasks.
Researchers learn what to improve by observing and interpreting think-aloud.
Virzi, R.A., Refining the Test Phase of Usability Evaluation: How Many Subjects is Enough? Human Factors, 1992. 34(4): p. 457-468.
Trend
Business sponsors turn to us as UX professionals with questions that are not about usability problems.
Rather, their questions are about overall user experience strategy, value and usefulness.
Usability:
Find interruptions in workflow that prevent users from performing tasks quickly and efficiently.
Usefulness:
Inform a re-structure of the application to best align with workflow.
Determine where to position productivity tips and help buttons within the application for best utilization.
Find optimal level of personalization and customization that users would take advantage of.
Usability:
Assess effectiveness of navigation system in guiding users to desired pages.
Evaluate descriptiveness and clarity of links.
Gauge ability of page layouts to orient users to relevant content.
Usefulness:
Identify content that is missing which will help overcome objections or answer critical questions?
Understand how branded labels and content themes contribute to the overall experience or detract from it.
Determine level/types of promotions and interstitials that are acceptable to users.
Understand how different audience personas prefer to consume information for the particular domain.
Usability:
Determine optimal level of difficulty to encourage advancement to multiple levels of the game.
Assess discoverability of game features and controls.
Usefulness:
Find the optimal rate of point accumulation and alignment with prize levels.
Understand best use of social media within or around the game.
Determine the threshold for ads, interstitials and registration for game play.
Usability:
Assess if users can figure out how to add a comment, share content, or use a tagging mechanism to find what they are looking for.
Usefulness:
Determine the most compelling and appealing topics or categories for conversation.
Understand the level of involvement the sponsoring company should have in the social experience, if any.
Balance branded or non-branded experience for optimal trust of the site.
Determine the elements or attributes that should allow comment and review.
Usability:
Find any confusion points or interruptions that prevent users from registering.
Find misleading or ambiguous terminology.
Usefulness:
Determine the most persuasive elements that will compel the target audience to register.
Understand a design’s impact a user’s perception of the brand.
Position the offering and messaging against the company’s competitors.
Determine missing content that can help target audience make an informed decision about the product.
Sound Familiar?
Natural Reaction
Turn to what we know: Usability Testing(one-on-one interviews, design artifact, and tasks)
Are You Forgetting Contextual Inquiry and Foundational Research?Discovery research and needs analysis is valid, but:
• Time and budget for separate research is not always an option
• Many participants need design artifacts to elicit appropriate reaction and commentary
Our Goal
Leverage the strengths of usability testing but adjust our approach when objectives differ from finding usability problems.
Three Components
Phase
Pre-Task Questions
Tasks
Post-Task Questions
Three Components
Phase Usability Usefulness
Pre-Task Questions
Demographics, level of expertise, prior experience. Goal: validating groups and classify results.
Daily task flow, pain points, expectations, desires, scenarios.Goal: set mindset for usefulness evaluation of the design
Tasks
Post-Task Questions
Three Components
Phase Usability Usefulness
Pre-Task Questions
Demographics, level of expertise, prior experience. Goal: validating groups and classify results.
Daily task flow, pain points, expectations, desires, scenarios.Goal: set mindset for usefulness evaluation of the design
Tasks Pre-defined tasks, minimal moderator intrusion, Goal: find usability problems, measure time on task and completion percentage.
Emphasis on participant-directed tasksGoal: understand how proposed design aligns with user needs
Post-Task Questions
Three Components
Phase Usability Usefulness
Pre-Task Questions
Demographics, level of expertise, prior experience. Goal: validating groups and classify results.
Daily task flow, pain points, expectations, desires, scenarios.Goal: set mindset for usefulness evaluation of the design
Tasks Pre-defined tasks, minimal moderator intrusion, Goal: find usability problems, measure time on task and completion percentage.
Emphasis on participant-directed tasksGoal: understand how proposed design aligns with user needs
Post-Task Questions
Level of satisfaction with the design, and points of confusion or ambiguity.Goal: measure usability
Comparison with expectations and value. Task retrospectives.Goal: discuss opportunities for improvement.
Summary
Foundational research is still important.
Usability testing is still important.
However, recognize when you have different goals and adapt the research method as necessary.
Additional Information
Complete Presentation Slides
•http://www.slideshare.net/hawleymichael
•http://www.slideshare.net/banderlin
Dan Berlin
@banderlin
Michael Hawley
@hawleymichael
Contact Information
![The Role of Usability on the Implementationteresarojano.net/system/files/articulos/The_Role_of_Usability_on_the... · [9] recognize the importance of assessing usability in educational](https://img.pdfslide.us/doc/110x75/602e584d2cec2737653c87ed/the-role-of-usability-on-the-implem-9-recognize-the-importance-of-assessing-usability.jpg)
