CRIA-Les5-Afnametest 2012 Engels (1)

7/31/2019 CRIA-Les5-Afnametest 2012 Engels (1)

1/52

Create a Rich Internet Application

Lesson 5 Usability testing

Informatica Communicatie Academie

Hogeschool Arnhem Nijmegen


2/52

Learning aims

At the end of this lesson you know

-what usability testing is

-whywe do a usability test

-howto prepare, organize and analyze a test-about important measurable variables

-all info about your next assignment


3/52

Contents

Part-01 What is Usability Testing

Part-02 Measures and variables

Part-03 1E Phase Usability TestPreparation of the test

Conducting the test

Structuring data

Processing data


4/52

Design overview


5/52

What is usability testing

Usability testing is not:

Testing functionalities (although this partly applies!) Beta-testing Doing research

Focus group*

*A focus group is a group of (in general) 6 to 12 people, led by a discussion leader,that discusses a given subject. Attendants of the group can be clients, users orexperts. The group is not interviewed, but a vivid interaction between attendants isthe main driver. By an interactive discussion a more valuable result can be expected

then in the case of separate interviews with people within the group. A focus groupis a valuable instrument for discovery. It reveals hidden needs on top of the explicitneeds.


6/52


7/52

Place of the Usability Test

User- & Task analysis

Design

PrototypeUsability Testing

Endproduct

Repeat

Not satisfactory

Satisfactory

Also the basis for

your action plan!


8/52

Wat is Usability Testing

The test is based on goals set in preparation of the test

Test-persons are real users and do realistic tasks(as you designed them)

Testers

- observe and register what users do and say

- analyze registered data

-measurements, users remarks, observations

- identify usability-problems

- formulate advice for improvements


9/52

Characteristics Usability Test

Time for a little video

http://www.2c.nl/nl/usability_onderzoek/usability_test/video.php

http://www.usefulusability.com/10-must-see-usability-videos

And software

http://silverbackapp.com/

Or search for usability software testing on Google
http://www.2c.nl/nl/usability_onderzoek/usability_test/video.phphttp://www.2c.nl/nl/usability_onderzoek/usability_test/video.phphttp://www.usefulusability.com/10-must-see-usability-videoshttp://silverbackapp.com/http://silverbackapp.com/http://www.usefulusability.com/10-must-see-usability-videoshttp://www.usefulusability.com/10-must-see-usability-videoshttp://www.usefulusability.com/10-must-see-usability-videoshttp://www.usefulusability.com/10-must-see-usability-videoshttp://www.usefulusability.com/10-must-see-usability-videoshttp://www.usefulusability.com/10-must-see-usability-videoshttp://www.usefulusability.com/10-must-see-usability-videoshttp://www.usefulusability.com/10-must-see-usability-videoshttp://www.usefulusability.com/10-must-see-usability-videoshttp://www.2c.nl/nl/usability_onderzoek/usability_test/video.phphttp://www.2c.nl/nl/usability_onderzoek/usability_test/video.php


10/52

Phases Usability Test

This lesson:1. Plan and prepare the usability test.

2. Take the usability test.

3. Collect test data from the usability test.

Next lesson:

Analyze data of the test.

Report test results.


11/52

Contents




Conducting the test

Structuring data

Processing data


12/52

Sorts of Variables

Qualitative(kwalitief)

Quantitative(kwantitatief)

Nominal

Ordinal

Discrete

Continuous


13/52

Qualitative Nominal

Latin: nomen = name

Concerns the naming of the registered parameter

Example:

Gender: Man / Woman

City:Arnhem

Animal sorts: Dog, cat etc


14/52

Qualitative Ordinal

Aggregation of information that can be organized in logicalcategories without reference to absolute measurable meaning orinterpretation

Ordinal measurements only showrelative discriminationbetween categories

Examples:Education levels:VMBO, HavoGrades in education: 1-10Scales/scores/rankings:

very difficult - little difficult - neutral little easy very easy

It shows that there is a difference between little and very, but without telling you howmuch difference


15/52

Quantitative Discrete

Quantitative measurements result in numbers

Discrete variables comprise of real full number only:1,2,3,4,5 etc

Example:

Flipping a coin can only give you one side in a full numberof cases. You cant get it 2,5 times! So thats why this

quantitative number can only be a discrete one.


16/52

Quantitative Continuous

Continuous variables can have any kind of value:1,3 5.000,92 0,001 - etc

Example:

If you register the weight of an item it can be any value evenwhen limited between given thresholds like any valuebetween 10 and 100: 30,5 40 60,58


17/52

Quantitative Continuous

Unlike discrete variables that can only hold discrete pointson a scale, continuous variables can have any value.

For example:

To become a fire fighter ones weight must be between 75and 100 kilo. Weight is an example of a continuousvariable. The weight of a fire fighter van have any value

between 75 and 100 kilo.


18/52

Qualitative or quantitative?

1. Determine what you want to measure and whichvalues are allowed;

2. Determine how you want to respond to values that are

not permitted;

Needed topresentyour results and drawconclusionsfrom a usability test!

It is important to determine what you want to measurebefore you start!


19/52

Contents




Conducting the test

Structuring data

Processing data


20/52

Planning and preparation

Determine aims

Participants are real usersDetermine users and participants

Participants perform realistic tasksSelect tasksWrite scenarios

Write interview questions

Determine how usability will be measured


21/52

Determine aims

Often products are (very) complex.

Ask yourself: What do I want to achieve?

Write questions that reveal important issues:How fast and accurate can users find a specific taskby clicking the right button?

Write down aims for these issues:

Users must be able to accomplish a task at once withoutusing a method of trial and error.


22/52

Determine users and participants

The target population can be taken from the user profilesdescribed in the user analyses.

Sample size: about 4-6, minimal 3

Create some subgroups


23/52

Number of users

1 test with 8 user

5 errors found in total

2 tests with 3 users each

9 errors found in total


24/52

Which Use Cases to choose?

The aim of your usability test is your starting point.

Ask yourself: What problems can be expected?

Check your Use Casesand scenarios (StructureDocument)

Put some Use Cases in a sequence that seems to be naturalfor the user. Write (test)scenarios that are understandablefor the user.

N.B. These Use Cases perform tasks which the user will betesting.


25/52

Write (Use Case) scenarios

Adescription of a taskthat will be performed by the tester.

Contains enough information to perform a task.

Uses terms with which the user is familiar.

Is unambitious.

Is NOT a list of user actions.

( )


26/52

Scenario example (wrong)

Add a new examination.Enter todays date.

Enter Project4 in field Teacher and Interface projectin field Description.

Enter 25 in field number of questions and 1 in fieldparts.

Enable all controls.

Enter IC2B in field class.

( i h )


27/52

Scenario example (right)

The results of an examination for the course userinterfaces have to be processed in SVS. You have beenteaching this course to class IC2B.

The examination consisted of 25 questions for which there

is only 1 version.

You want SVS to warn you if an answer contains a valuethat is outside the range of possible values or if a questions

was not answered at all.

Add a new examination to SVS to process your results.


28/52

Write interview questions

Possible questions:

Pre-testBefore the test, to get a clear picture of the tester.Necessary when using subgroups.

Post-taskAfter each task, to ask the tester his opinion about a task.Gives a wealth of information! Use this always!

Post-testAfter the test, to ask the tester his opinion about theproduct/website.

h b l ll b d


29/52

Determine how usability will be measured?

Possible measurements (per task) are:Number of errors;Number of menus and screens displayed;Number of times the tester asked for Help or support;If the tester was able to accomplish a task (without asking

for help);Remarks from the tester about the product/website tested;Time to accomplish a task;.

Determine objective criteria (for each variable)Determine what is acceptable and what not

h d


30/52

Inhoud




Conducting the test

Structuring data

Processing data


31/52

Important

Structuring and composition test team

How to start

Think aloud

Help?

List of problems identified

O i i


32/52

Organization testteam

Possible roles:

Tester - user who is performing the usability test

Facilitator - tells user what to do, facilitate, encourageto think aloud, listen what the user says

Observer - body language (qualitative)

Observer - time, mistakes, remarks (quantitative)

S f h


33/52

Start of the test

Introduction of the persons involved.

Explanation of the process.

Make clear that the product is tested and not theparticipant.

Ask to think aloud.

I t t d i T k t t


34/52

Important during Take test

Composition of the test team

Start of the test

Think aloud

Helping?

Problem list

C t t


35/52

Contents




Conducting the test

Structuring data

Processing data

St t i d t


36/52

Structuring data

Legenda:

Question 1: Was het moeilijk om uit te vinden hoe u uw risico op een hart of vaatziekte kon

meten?

Question 2: Waar bent u begonnen met zoeken naar de risicometer vanaf de homepage?

Question 3: Wat vond u van het gebruik van de risicometer?

P i d t


37/52

Processing data

Conclusions:

From the retrieved data we can conclude that:

1. Male testers were more able to find the Risicometer;

2. Female testers did do all the tasks in an orderly sequence and

started with the Medische informatie part;3. Male testers started by entering a search request;

4. The use of the Risicometer is rather easy;

5. Two participants complained because not all values can be

entered. Someone who is younger than 40 (age) cannot usethe Risicometer

(value cannot be entered).

St t i d t


38/52

Structuring data

Legenda:

Question 1: Kon u de functie om zorgverleners te vergelijken makkelijk vinden?

Question 2: Waren de keuzes die u moest maken duidelijk?

Question 3: Kon u de informatie vinden die u zocht en was deze informatie duidelijk?

Question 4: Waren de knoppen om verschillende zorgverleners te vergelijken duidelijk?

Question 5: Zou u, als u ooit zorgverleners op kwaliteit moet vergelijken, www.kiesbeter.nlnog een keer gebruiken hiervoor?

Processing data


39/52

Processing data

Conclusions:

From the previous table we can conclude that:

1. The functionality to do the test was easy to find;

2. Some testers followed alternative routes to find the

functionality;

3. The received information was clear to all testers. One tester

did not recognize the outcome of the test;

4. Two testers think that the site is not user friendly. Two testers

think that the site is bad.

In general we can conclude that the steps to do the test are

straight forward and easy to follow. The side is not really user

friendly. Two participants will not visit the site and two doubt if

they will visit the site again.

Be carefull


40/52

Be carefull

Write down positiveremarks as well.

Everybody is eager to hear the good news.

Write them down in your conclusions!

Contents


41/52

Contents

What is Usability Testing

Measures and variables

1E Phase Usability TestPreparation of the test

Conducting the test

Structuring data

Processing data

Data presentation


42/52

Data presentation

Always in a table and/or graph!

Purpose: present the data in an honest way

No Puzzle

The

- Contents (data)

- Presentation (data)

- Drawn conclusions

Counts

Not like this


43/52

Not like this

Not like this


44/52

Not like this

This is nice


45/52

This is nice

Legenda:

Question 1: Was het moeilijk om uit te vinden hoe u uw risico op een hart of vaatziekte kon met

Question 2: Waar bent u begonnen met zoeken naar de risicometer vanaf de homepage?

Question 3: Wat vond u van het gebruik van de risicometer?

This is nice as well


46/52

This is nice as well.

Task Name Totally agree Agree Do not know Do not agree Totally disagree

1 Renting a Bike

1.1 Bike are easy to select 3 2 3 2 0

1.2 Pimping the bike makes fun 2 5 3 0 0

1.3 Suggested Bike type conforms to my profile 0 1 6 1 2

1.4 Accesoires are easy to select 1 4 5 0 0

0

1

2

3

4

5

6

7

Totally agree Agree Do not know Do not agree Totally disagree

Suggested Bike type conforms to my profile

Antwoord

This could be an approach


47/52

This could be an approach

Task Name Totally agree Agree Do not know Do not agree Totally disagree

1 Renting a Bike

1.1 Bike are easy to select 3 2 3 2 0

1.2 Pimping the bike makes fun 2 5 3 0 0

1.3 Suggested Bike type conforms to my profile 0 1 6 1 2

1.4 Accesoires are easy to select 1 4 5 0 0

0

1

2

3

4

5

6

7

Totally agree Agree Do not know Do not agree Totally disagree

Suggested Bike type conforms to my profile

Antwoord

Relate conclusions to observations:

- The suggested bike in relation to the profile does not fit.

Advice: Improve usability Profiling a User and Selecting a Bike.

Other mistakes


48/52

Other mistakes

3D graphs (do mostly make no sense)

Overload of graphs

Wrong use of diagrams (pie charts etc.)

If you want you can use


49/52

If you want you can use

Location measures

Median = midst observation

Arithmetic Average = sum/n

Range

Variation = max min

See also: (strongly advised!)

http://www.youtube.com/watch?v=qKPhtBIsyIY&feature=related

http://www.youtube.com/watch?v=5wO8QjfYsLI&feature=related

Assignment
http://www.youtube.com/watch?v=qKPhtBIsyIY&feature=relatedhttp://www.youtube.com/watch?v=5wO8QjfYsLI&feature=relatedhttp://www.youtube.com/watch?v=5wO8QjfYsLI&feature=relatedhttp://www.youtube.com/watch?v=qKPhtBIsyIY&feature=related


50/52

Assignment

Assignment 5.1

Start working on the first part of your usability test(look in your study guide)

Be aware: Use the provided structure document!Dont invent it yourselves (itll cost youtime)

Let us have a look.

Summarized


51/52

Summarized

1. Design realistic tasks for the users.

2. Set up the observation.3. Describe to the user the purpose of the observation.4. Demonstrate the equipment in the room.5. Explain how to "think aloud".6. Explain that you will not provide help.

7. Describe the tasks and introduce the product.8. Ask if there are any questions before you start.9. Begin the observation.10.Conclude the observation.11. Use the results.

Tog On Interface, Bruce Tognazzini, Addison-Wesley, 1991, ISBN 0-201-60842-1The Art of Human Computer Interface Design, Brenda Laurel, Addison-Wesley,

1991,ISBN 0-201-51797-3

Knowing more?


52/52

Knowing more?

About face Alan Cooper

A practical guide to usability testing Tom Brinck

Usability for the web - JS Dumas & JC Redish

Documents

CRIA-Les5-Afnametest 2012 Engels (1)