View
1
Download
0
Category
Preview:
Citation preview
1
Lecture 2Lecture 2
Concepts of usabilityConcepts of usabilityUsability testingUsability testing
How would you evaluate your How would you evaluate your knowledge levelsknowledge levels
1. My understanding of usability concept
Very bad Very goodOK
2
How would you evaluate your How would you evaluate your knowledge levelsknowledge levels
2. My knowledge on usability testing
Very bad Very goodOK
What is usability?What is usability?How to measure it?How to measure it?
3
Usability conceptUsability concept
ISO 9241:effectiveness (% of goal achieved)
+ efficiency (time to complete a task, or the error rate, or the amount of effort)
+ satisfaction (subjective rating scale)= Usability
What is context?What is context?
SystemSystemStakeholder analysis, role and benefitStakeholder analysis, role and benefitUserUser
Experience, knowledge and skillExperience, knowledge and skillPersonal attributesPersonal attributes
TaskTaskEnvironmentEnvironment
Technical, physical, organizationalTechnical, physical, organizational
4
RelationshipRelationship
Work environment
user
Computer system
software
Internal quality
External qualityQuality in use
Quality in useQuality in use
The capability of the software product The capability of the software product to enable specialed users to achieve to enable specialed users to achieve specialed goals with effectiveness, specialed goals with effectiveness, productivity, safety and satisfaction in productivity, safety and satisfaction in specialed contexts of use.specialed contexts of use.
ISO/IEC FDIS 9126ISO/IEC FDIS 9126--1 (2000)1 (2000)
5
Quality in useQuality in use
Effectiveness, productivity, safety, satisfaction
FunctionalityAccuracySuitability
Interoperabilitysecurity
ReliabilityMaturity
Fault toleranceRecoverability
availability
UsabilityUnderstandability
LearnabilityOperability
attractiveness
EfficiencyTime behaviour
Resourceutilisation
maintainabilityAnalysability
Changability, stability, testability
PortabilityAdaptability, installability
Co-existence, replaceability
SatisfactionSatisfaction
Satisfation: Satisfation: the level of comfortthe level of comfort that that the users feel when using a product the users feel when using a product and how acceptable the product is to and how acceptable the product is to users as a means of achieving their users as a means of achieving their goalsgoals
Subjective aspectSubjective aspectDifficult to measureDifficult to measureOften be strongly correlated with Often be strongly correlated with effectiveness and efficiencyeffectiveness and efficiency
6
Satisfaction levelsSatisfaction levels
FunctionalityFulfill the appropriate functionality
The context and the environment in which it will be used
UsabilityEfficiency, effectiveness, satisfaction
PleasurePeople’s emotion
Value, hope, taste, fear, etc
FunctionalityFulfill the appropriate functionality
The context and the environment in which it will be used
UsabilityEfficiency, effectiveness, satisfaction
PleasurePeople’s emotion
Value, hope, taste, fear, etc
Emotional feeling of pleasure Emotional feeling of pleasure
PropertiesPropertiesFeatures/functionalityFeatures/functionalityUsabilityUsabilityAestheticsAestheticsEffective performanceEffective performanceReliabilityReliabilityHighHigh--status productstatus productConvenienceConvenienceSizeSizeBrand nameBrand name
EmotionsEmotionsSecurity/comfort Security/comfort Confidence Confidence Pride Pride ExcitementExcitementSatisfactionSatisfactionEntertainmentEntertainmentFreedomFreedom
Sentiment/nostalgiaSentiment/nostalgia
7
Usability testingUsability testing
Usability testing determines whether the users can Usability testing determines whether the users can find and use the features in the amount of the time find and use the features in the amount of the time and effort they willing to expend searching.and effort they willing to expend searching.
Primary goal Primary goal –– improve the usabilityimprove the usabilityReal usersReal usersDo the real taskDo the real taskObservationObservationAnalysisAnalysis
Usability testingUsability testing
Usability is not: Usability is not: Quality assuranceQuality assuranceZero defectsZero defectsUtility of design featureUtility of design featureIntrinsic in productsIntrinsic in products
8
Making a listMaking a list
Usability testingUsability testing Quality assuranceQuality assurance
Lab. Test vs. field testsLab. Test vs. field tests
Lab testLab testWell designedWell designedNot choiceNot choiceaim is to test aim is to test whether a product whether a product meets certain meets certain predefined usability predefined usability criteriacriteria
Field testField testacceptability of a acceptability of a productproduct
however the benefits of having both types of test dataoutweigh the negative factors
9
Usability test methodsUsability test methods
Focus on Lab. testsFocus on Lab. tests
Usability testing lab.Usability testing lab.
10
Methods overviewMethods overview
Inspection methodInspection methodUser testingUser testing
11
Methods overviewMethods overview
Inspection methods: need to be carried out Inspection methods: need to be carried out by cognitive expert. They are good in by cognitive expert. They are good in finding problems finding problems
Heuristic evaluation: better predictorHeuristic evaluation: better predictorCognitive walkthrough: finding endCognitive walkthrough: finding end--user user problems towards easeproblems towards ease--of useof use
Actual user testing is still very importantActual user testing is still very importantUser testing and inspection methods do User testing and inspection methods do not have a high degree of overlapping not have a high degree of overlapping findingsfindings
Selecting methodsSelecting methods
Not to rely on any one method for Not to rely on any one method for determining the usability of product, determining the usability of product, but user a variety of methods.but user a variety of methods.
12
Learn about userLearn about user
Self definition (job, tasks, tools, and mental Self definition (job, tasks, tools, and mental models)models)individual (personal, physical and cultural individual (personal, physical and cultural characteristics, as well as motivation)characteristics, as well as motivation)stage of use (How they use products over time stage of use (How they use products over time and the choices they make about the levels of and the choices they make about the levels of expertise they want or need to achieve expertise they want or need to achieve
Characteristics of user populationsCharacteristics of user populationsNovicesNovicesAdvanced beginnersAdvanced beginnersCompetent performersCompetent performersExpert performersExpert performers
User and task analysisUser and task analysis
What usersWhat users’’ goals aregoals areWhat processes they use to achieve their What processes they use to achieve their goalsgoalsWhat characteristics shape the way they What characteristics shape the way they perform tasks and achieve goalsperform tasks and achieve goalsAffect from previous experienceAffect from previous experienceWhat is the most important to user or what What is the most important to user or what is most helpful to them in performing tasksis most helpful to them in performing tasksWhat impact the environment has on their What impact the environment has on their ability to perform tasks.ability to perform tasks.
13
UserUser’’s goals and taskss goals and tasks
UserUser’’s tasks are not necessary users tasks are not necessary user’’s s goalsgoalsIf the task interferes with the goal, users If the task interferes with the goal, users become frustrated.become frustrated.
the goal stackthe goal stackFalse goals (easily achieved by programmers)False goals (easily achieved by programmers)Corporate goals (companyCorporate goals (company’’s benefit)s benefit)Practical goals (the bridge between the companyPractical goals (the bridge between the company’’s s goals and the usergoals and the user’’s goals)s goals)Personal goals (true for everyone)Personal goals (true for everyone)
Site visitSite visit
Follow a user for a dayFollow a user for a dayQuestion user while they workQuestion user while they workTalking after taskTalking after taskThink aloudThink aloudCritical incident techniqueCritical incident techniqueScenarioScenarioCued recallCued recall
14
Constraints of site visitConstraints of site visit
Time constraintsTime constraintsBudget constraintsBudget constraintsPolitical and ethical constraintsPolitical and ethical constraints
After a site visitAfter a site visit
Report:Report:User analysisUser analysisTask analysisTask analysisEnvironmental analysisEnvironmental analysisMethodologyMethodologyRecommendationRecommendation
Personas Personas DailyDaily--use scenarios (often used)use scenarios (often used)NecessaryNecessary--use scenarios (infrequently used)use scenarios (infrequently used)
User learningUser learning
15
Iterative testing: Paper prototypeIterative testing: Paper prototype
goodgoodCheap, easy, can Cheap, easy, can do it oftendo it oftenTry new ideas and Try new ideas and conceptsconceptsTeam activitiesTeam activitiesEnhanced Enhanced communication communication among team among team membersmembersCommon baselineCommon baseline
BadBadComplex productsComplex productsCollect response Collect response timetimeinteraction qualityinteraction qualityTeam Team understandingunderstandingLonger time for Longer time for testingtesting
Purpose of usability testingPurpose of usability testing
Different stages of product Different stages of product development, The test is different.development, The test is different.
16
Planning for usability testingPlanning for usability testing
TeamTeamProduct Product GoalsGoalsMeasurementsMeasurementsUsersUsersTasksTasksResults analysisResults analysisTest planTest plan
LimitationsLimitations
TimeTimeMoneyMoneyTesting conditionsTesting conditionsWho wants to know the resultsWho wants to know the resultsWhat is best learned from usability What is best learned from usability testing?testing?
17
PlanningPlanning
Initial team meeting Planning meeting Results meeting
Pilot testwalkthroughRecruiting participants
Completion of test plan Approval of the test plan
Testing conditionTesting condition
Where?Where?Field?Field?Lab?Lab?Remote?Remote?
(this is depending on the product (this is depending on the product development stages and the location of development stages and the location of the users)the users)
SetSet--up for testingup for testing
18
What can we learn from usability What can we learn from usability test?test?
Usability test can not answer many Usability test can not answer many questions, such as color preference, questions, such as color preference, font size, use of particular graphics, font size, use of particular graphics, array array oof products available, since the f products available, since the answers to such questions cannot be answers to such questions cannot be obtain from 5 to 8 subjects.obtain from 5 to 8 subjects.
Preparing for usability testingPreparing for usability testing
Screening questionnaires:Screening questionnaires:The background information of users The background information of users towards the use of the product.towards the use of the product.These questions shall be sent to the These questions shall be sent to the users and have the answer back before users and have the answer back before the test.the test.
PrePre--test questionnairestest questionnairesThe questions answered before the test The questions answered before the test started.started.
19
Preparing for usability testingPreparing for usability testing
Creating scenariosCreating scenariosScenarios are description of typical Scenarios are description of typical tasks you want the participants to tasks you want the participants to perform.perform.Avoid creating steps or instructions, Avoid creating steps or instructions, focus on processfocus on processAvoid using product language, use Avoid using product language, use useruser’’s languages language
Objective measurementObjective measurement
Try to obtain if you canTry to obtain if you can
20
PostPost--task and posttask and post--test test questionnairesquestionnaires
Avoid bias responsesAvoid bias responsesNeutral questionsNeutral questionsCan be qualitative and quantitative Can be qualitative and quantitative responsesresponses
Roles of the team membersRoles of the team members
Based on their expertiseBased on their expertiseConsistentConsistentFacilitator: give user direct instructionFacilitator: give user direct instructionCamera/video operatorCamera/video operatorLogger: capture the measurementsLogger: capture the measurements
21
Important factorsImportant factors
Avoid biasAvoid biasMaintain consistencyMaintain consistency
Walkthrough Walkthrough
It is a chance to test the equipment, It is a chance to test the equipment, the scenarios, the questionnaires, the scenarios, the questionnaires, and the checklists for the team and the checklists for the team members.members.Team memberTeam member’’s practice s practice Intend to find the flaws in the planIntend to find the flaws in the plan
22
Pilot testPilot test
Test of the testTest of the testFind out practical problemsFind out practical problemsA few days earlier than real testA few days earlier than real test
Conducting the UConducting the U-- testtest
Team should avoid contacting with Team should avoid contacting with usersusersBriefing:Briefing:
Any misconception can be clarified.Any misconception can be clarified.Fill the forms.Fill the forms.Using videotape (?)Using videotape (?)UnbiasUnbias
People try to please the othersPeople try to please the others
23
Handling questionsHandling questions
If, user ask a question, donIf, user ask a question, don’’t answer it t answer it directlydirectly
““did I do that right?did I do that right?””““what do you think should happened?what do you think should happened?””or or ““tell me what makes you wonder tell me what makes you wonder whether you did it right?whether you did it right?””
Try to led the user shear his Try to led the user shear his experience with you.experience with you.
Intervene Intervene
Only when you see the user get lostOnly when you see the user get lostCareful! Careful!
Questions like: Questions like: ““what are you trying to what are you trying to do now?do now?”” or or ““what do you think the what do you think the system is doing now?system is doing now?””
HelpHelpNot to give the direction of how to solve Not to give the direction of how to solve the problem.the problem.Provide problem solving strategies.Provide problem solving strategies.
24
Collect dataCollect data
Category the dataCategory the dataIf two users have a problems, the If two users have a problems, the problem has to be taking careproblem has to be taking careIf only one single user has that If only one single user has that problem, depend how many users are problem, depend how many users are tested!tested!Find the explanation of the special Find the explanation of the special case.case.Conflict data? Believe what you see!Conflict data? Believe what you see!
AnalysisAnalysis
Determining the causes of problems.Determining the causes of problems.Make sure that one solution doesnMake sure that one solution doesn’’t t create some other problems.create some other problems.Scope and severity of problemsScope and severity of problems
25
Usability reportUsability report
Who is going to read the report?Who is going to read the report?
Standard form for industry Standard form for industry Usability report (1)Usability report (1)
Title PageTitle PageExecutive SummaryExecutive SummaryIntroductionIntroductionFull Product DescriptionFull Product DescriptionTest ObjectivesTest Objectives
26
Standard form for industry Standard form for industry Usability report (2)Usability report (2)
MethodMethodParticipantsParticipantsContext of Product Use in the TestContext of Product Use in the TestTasksTasksTest FacilityTest FacilityParticipantParticipant’’s Computing Environments Computing EnvironmentTest Administrator ToolsTest Administrator ToolsExperimental DesignExperimental Design
ProcedureProcedureParticipant General InstructionsParticipant General InstructionsParticipant Task InstructionsParticipant Task Instructions
Standard form for industry Standard form for industry Usability report (3)Usability report (3)
Usability MetricsUsability MetricsEffectivenessEffectivenessEfficiencyEfficiencySatisfactionSatisfaction
ResultsResultsData AnalysisData AnalysisData ScoringData ScoringData ReductionData ReductionStatistical AnalysisStatistical AnalysisPresentation of the ResultsPresentation of the ResultsPerformance ResultsPerformance ResultsSatisfaction ResultsSatisfaction Results
27
Standard form for industry Standard form for industry Usability report (4)Usability report (4)
AppendicesAppendicesReferencesReferences
ExerciseExercise
A short description of tasksA short description of tasksPlanning of usability test in detailPlanning of usability test in detailThe structure of usability reportThe structure of usability report
28
Usability test plan (1)Usability test plan (1)
What kind of knowledge they you need to What kind of knowledge they you need to have before you can carry out the usability have before you can carry out the usability tests of the product? Where and how can you tests of the product? Where and how can you get the knowledge? Do you need any skill get the knowledge? Do you need any skill training as well? You need to specify the training as well? You need to specify the discussion as detail as possible.discussion as detail as possible.What kind of team members do you like to What kind of team members do you like to have when you are asked to organize the have when you are asked to organize the usability tests?usability tests?Would you prefer to carry the test in the lab, Would you prefer to carry the test in the lab, or in the field, or both? Why?or in the field, or both? Why?
Usability test plan (2)Usability test plan (2)
What kind of preparations do you need to do What kind of preparations do you need to do before you can design the tests?before you can design the tests?What can be the best schedule of the plan? What can be the best schedule of the plan? Who shall be your tests subjects? How many of Who shall be your tests subjects? How many of them in each group? How can you find them?them in each group? How can you find them?How many tests you plan to carry out? How How many tests you plan to carry out? How would you design your testing scenarios would you design your testing scenarios (making as detail as possible of your testing (making as detail as possible of your testing scenario)? What kinds of design features and scenario)? What kinds of design features and functions you are planning to test?functions you are planning to test?
29
Usability test plan (3)Usability test plan (3)
Prepare your testing tools. What kind of tools Prepare your testing tools. What kind of tools you need? How shall they look like? What you you need? How shall they look like? What you plan to measures? Please make a detail design plan to measures? Please make a detail design of one lab test as detail as possible.of one lab test as detail as possible.Data analysis. Based on your plan of tests and Data analysis. Based on your plan of tests and your plan of measurements, can you make the your plan of measurements, can you make the analysis of the results? What kind of analysis of the results? What kind of conclusions you are expecting from the data conclusions you are expecting from the data analysis.analysis.Preparing you usability testing report. Making Preparing you usability testing report. Making the structure of how your usability testing report the structure of how your usability testing report shall be looked like.shall be looked like.
Exercise Exercise
After you finish the reportAfter you finish the reportRead the document about Read the document about ””7 common 7 common mistakesmistakes””A checklist of usability reportA checklist of usability report
Recommended