Usability Testing – Part II

Teppo Räisänenhttp://www.oamk.fi/~teraisan/

Teppo.raisanen@oamk.fi

Practical Advice

When a test person arrives, a written query of his/her background information is often useful

It is not uncommon, that because of misunderstandings the test person does not belong to targeted user group

Practical Advice

Often also written agreements are signed NDA Agreement of participation Agreement about recording

(video/audio) the test session

Interpretation of the Results

Vast amounts of information are collected during testing

For this information to be useful, it must be correctly transformed and interpreted

Just gathering technical data (response times etc.) is usually not enough

Interpretation of the Results

Often the most critical findings are reported before in-depth analysis Product development cycles have

become increasingly faster The results of quick analysis can be

used for important corrections and updates

Interpretation of the Results The test data should be organized

in an uniform way E.g. handwritten comments and

audio tapes are transformed into digital formats

Numerical data is collected and processed

Backup copies are made

Interpretation of the Results Several statistical methods are used for

intepretation In case of usability problems it should

be investigated if just an individual test person had a spesific

problem problem is common to a spesific sector of

target group problem is common to all users

Interpretation of the Results After an usability problem has been

recognized the origin of the problem should be tracked

After origin has been tracked a solution to the problem is suggested

Problems are usually prioritized Not all problems in all cases can be

solved within resources available

Usability Lab Facilities that are needed vary a

lot according to the case Just simple room might be enough Basic equipment consists of

computer system for testing video camcorder video monitor (backup equipment)

Usability Lab Often two rooms divided by a see-

through mirror are used The test person is not able to see to

the observers’ room In many cases more than one

camrecorder are used Information shown on computer

screen is usually not taped but saved internally

Usability Lab

If two rooms are used, the rooms are often soundproof

Test person’s voice is transmitted to the control room via speaker system

Instead of see-through mirror, the visual information about test session can be displayed using control room’s monitors

Usability Lab

A good lab should be easily modified according to the needs

Often some amount of staging is used to make test sessions more realistic Background audio tapes Lights Anything within the limits of imagination

Field Tests Modern usability research stresses the

importance of use context Therefore usability testing should also

take place within actual use context Lab conditions are often quite different

from real world of target users It has even been predicted, that

traditional lab tests will become extinct

Field Tests

A cost-effective way is to use customer’s facilities for field testing

Some modern mobile test labs use advanced methods for recording user’s actions pulse measurement eye movement tracking

Automatic Collecting of Log Data

In case of software products it is usually quite easy to implement a tool for collecting log data during tests Actions will be recorded automatically Analyzing the data still requires

human resources Google Analytics

Automatic Collecting of Log Data

Many kinds of sources can be recorded Data about the performance of the

system Response times Network usage

User’s actions, e.g. choices of commands in spesific situations

Use of menu commands/kb shortcuts

Example: Subjective Enjoyability

Enjoyability is one aspect of usability Opinions about product’s enjoyability

are often quite individualistic Enjoyability can be investigated using

interviews after test sessions Test person can for example be asked

to fill a questionnaire

Example: Subjective Enjoyability

Simple 12345 Complicated

Technical 12345 Untechnical

Reliable 12345 Unreliable

Easy to use 12345 Difficult to use

Friendly 12345 Hostile

Safe 12345 Unsafe

Robust 12345 Fragile

High quality 12345 Low quality

Pleasant 12345 Unpleasant

Well-defined 12345 Ill-defined

Example: Subjective Enjoyability Results of questionnaires can be

statistically analyzed E.g. simple average values In many cases test persons have

preconceptions about products and brands

Therefore it can be useful to commit anonymous testing

Anonymous Testing

Test person is not told the company behind the product

It is important not to give even a hint

Often outside consultants are used