Upload
optimal-experience
View
104
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Ilana Kaplan presented on multi-national user testing at UXNZ in Wellington, November 2013.
Citation preview
Ilana Kaplan UX NZ 2013
Multi-national user testing: Tools, tricks and pitfalls
Multi country testing becoming more common place Understanding international customers is more
critical than ever as emerging market economies continue their rapid expansion
On the flip side, companies that don’t understand their international customers are at a strategic disadvantage and at greater risk of making missteps that could undermine their success in diverse markets
http://uxmag.com/articles/6-key-questions-to-guide-international-ux-research
Multi country testing becoming more common place International UX research is important even in
countries or regions with customers who speak the same language, as regional differences in nomenclature , cultural norms, and customer needs can dramatically impact the success of a product, and….
http://uxmag.com/articles/6-key-questions-to-guide-international-ux-research
Multi country testing becoming more common place International UX research is important even in
countries or regions with customers who speak the same language, as regional differences in nomenclature , cultural norms, and customer needs can dramatically impact the success of a product, and….
Legislation (medical devices) One size fits all – or does it? Soft launch in one county before we launch in
others Benchmarking against international competitors
Testing options
Moderated Unmoderated
In person Remote
Unmoderated Testing optionso First clicko Card sortingo Surveyso Task basedo Diaries
Cost
Data
Qu
al
Qu
ant
Less More
Testing options
Moderated Unmoderated
In person Remote
o Traditional user testingo Eye tracking
Testing options
Moderated Unmoderated
In person Remote
o Observer shares monitor and participant uses system
Testing options
Moderated Unmoderated
In person Remote
Cost
Data
Qu
al
Qu
ant
Less More
Medical deviceCoordinated out of USA
Face to face moderated usability testing• Observer travelled to each
country • Testing coordinated by
local UX partner
Global consumer site Coordinated out of Australia
Phase 4Face to face moderated usability testing in Australia and USA
• Study designed in Australia
• Skype briefing and debriefing
• Testing conducted by local partners in each country
Phase 2Unmoderated Treejack testing in USA, Australia and Germany
Phase 3Unmoderated ChalkMark testing in USA, Australia and Germany
• All recruit and analysis out of Australia, with help from German speaking staff
Phase 1Face to face moderated research on visual concepts
• Study designed in Australia
• Skype briefing and debriefing
• Testing conducted by local partners in each country
International websiteNZ company operating
internationally
1. User Survey on site in USA, Australia and NZ (around 1000 participants)
2. UX partners in Canada, the UK, and Hong Kong identified best practice examples in their country
3. Multi-national usability testing in Australia and NZ (34 particpants)
4. Eye-tracking in NZ5. Expert reviews
Smart Blood Pressure Monitor Study 2013
Qualitative insights into the purchase of a smart blood pressure monitor, the out-of-the-box experience and using the device over 10 days
Digital health and wellness
• More and more, patients are becoming consumers, spending more time monitoring their own health and well being and taking increased responsibility for themselves
• There is a whole new genre of Apps and devices that support and encourage this movement
• Electronic blood pressure monitors fit into this
• UX alliance partners set out to understand some of the short term factors which could impinge on the usefulness of these device
20 participants from 17 countries
BrazilCanadaJapanNew ZealandItaly
IndiaPolandDenmarkSwitzerland AustraliaUSA
FranceFinlandRussiaIrelandUnited KingdomGermany
The study – 3 stages
Purchasing the device
Out-of-the-box
experience
Using the
device
UX professionals purchase the device and record the experience
Observe and record the participant’s out-of-the-box experience
Participants measure their blood pressure and complete a survey daily, for 10 days
Smart Blood Pressure Monitor Study 2013: ‘Out-of-the-Box’ Experience
Unpacking device Assembling App First attempt
Purchase accessory
“It was hard to insert the batteries and there was no explanation about this in the manual.”
“It looks like a apple product. White package… it looks like an expensive product.”
“Reading on the package I realised that it wasn't iPhone 5 compatible. I didn't have [an] adapter though.”
“It was not clear which application you have to use with the device, and how to use it.”
Insights• It was not clear if an adapter was needed • Users struggled to read instructions in a foreign language• Most users had difficulties assembling the device
• Many users found that batteries were needed to charge the device after attempting to charge via connection to a computer
• Different apps were available for download which confused users• Most users were not successful reading their blood pressure the first time
“I spent something like 10 minutes trying to force the box open. I missed a transparent sticker that prevented the box to slide.”
“It worked fine from the very first time.”
“Not quite as slick as it could be.”
“The participant received a high reading. No information was given to prompt him to consider the cause.”
“There was no Russian version of the manual [or] list of languages on the cover of the manual so [the] participant had to look through the entire brochure.”
Multi-national user testing 21
Testing options – how to choose?
Moderated Unmoderated
Web/software UI More quantitative based Large, distributed sample or low
incidence Low(er) budget High penetration of Internet
access
Physical artifact Need rich qualitative feedback Need to have the human
connection Ensure high-level of consistency Uncertain of quality or
environment
Cultural differences In multi country moderated
testing
Planning
Running sessions
Planning
Check out the local calendar –
make sure you find out what really happens
In Japan, at the start of May there is
a week-long public holiday called
“Golden Week”; in August, there is
another week-long public holiday
called “Bon Festival”. Most
companies shut down so difficult to
recruit and test.
In Canada, hockey is a
religion! It’s difficult to recruit
evenings during the hockey
season.
In Brazil, don’t even think of recruiting
during Carnival (February or March, it
varies).
In France, avoid
running test sessions
in the first half of
May, around July
14th (Bastille Day)
In Australia and
many other
countries recruiting
participants is
difficult from mid
December to January
because of
Christmas and New
Year holidays.
Planning
Pick the time for sessions carefully
In Japan, evening and weekend sessions are common
because office workers are usually unable to attend daytime
sessions during weekdays.
In the US, it is difficult to schedule usability studies on the
weekend. Most US participants place a high premium on
their weekend time and reserve it for family, friends, or
weekend errands and chores.
In Moscow, many people have flexible working schedules;
some of them work late in the evening, so session times
can be varied. Free time is valuable (especially in large
cities), so it’s difficult to convince people to participate on
weekends.
Brazilians often run late – traffic and other occurrences that
prevent them from being in control contribute to laid-back
attitude to punctuality.
Planning
Consider local demographics
What does elderly mean?
Which languages do you consider?
Participants born abroad?
Planning
Plan ahead for recruitment.
Get help from someone local How many? Incentives?
• Italy over recruits by 20-30%
• UK 1 standby for each 6 participants
• Poland may one extra
• China, no shows rare
Planning
• Develop detailed
testing docs for
recruitment, running,
data collection and
reporting
• Translation
Allow enough time to plan
Protocol
• Recordings
• Sharing
files/streaming
• Cameras
• Test equipment
Technolog
y • Jetlag
• Where’s the lab?
• Time for a dry run
Travel
Running sessions
Does it matter who moderates?
English isn’t always English. In Australia, it’s
not unusual to use a local facilitator, even
when the client is English speaking and has
travelled to Australia to observe the sessions.In South Africa, it is most important to have
the right cultural fit between moderators and
participants. As such, moderators are not
typically experienced usability professionals.
What’s polite in one
culture can be
offensive in another
Running sessions‘Thinking aloud’ is very difficult in some cultures and too easy in others • The Brazilians and Irish love to talk, so thinking
aloud is no problem!
• Chinese users are often not willing to talk while
doing the tasks.
• Indians typically think aloud.
• For Germans and the British, small talk at the
front of sessions is very important to get
participants warmed up.
• In the UK, some participants are direct, while
others may be more circumspect. In some cases,
participants have struggled with tasks and yet
are still politely positive about it.
• In Japan, there is a cultural bias for men not to
say “I don’t know” or “I don’t understand,”
• Italians like to express their opinion on everything
- they are experts in all matters and when you ask
to think aloud during a task, you always get
opinion on the layout!
In conclusion
• There are valuable UX research options across a range of budgets
• Local culture, beyond language influences design, but also how you do
your research
• Allow enough time to plan, and
• Consider working with local partners
Ilana Kaplan UX NZ 2013
Dank U
Thank
You
Thank
you !
Merci
Kiitos
Grazie
mahal
o
DankeAny
question
s?