Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
IN DEGREE PROJECT MEDIA TECHNOLOGY,SECOND CYCLE, 30 CREDITS
, STOCKHOLM SWEDEN 2016
The transition to a responsive website: A user study
JOSEFINE LINDQVIST
KTH ROYAL INSTITUTE OF TECHNOLOGYSCHOOL OF COMPUTER SCIENCE AND COMMUNICATION
ABSTACT
An anonymous company has been redesigning its website to a responsive site and during this process the users have the
option to stay on either site. This study examined the reasons as to why users stay on the non-responsive site and how the
responsive site could be improved to make them switch to it. In order to understand how users were effected by first
impressions and age as well as how adaption to technology works, previous research in these areas have been presented. A
survey and user test were conducted to find the answers to the research question.
Because the transformation to a responsive site meant that the visual structure changed, it affected the users’ existing habits.
The results and conclusion were that there were three major groups stating different main reasons on what would make them
go to the responsive site. The strategy for making users on the non-responsive site more satisfied with the responsive site
would be to first implement missing functionality. The next step would be to invite all users and adding guides to help them
to learn how to find what they usually look for.
SAMMMANFATTNING
Ett anonymt företag har arbetat med att designa om sin hemsida till att bli responsiv och under den här processen har
användarna haft möjlighet att stanna på båda websidorna. Den här studien undersökte varför vissa användare stannade på den
icke responsiva sidan och hur den responsiva sidan kan förbättras för att få dem att byta till den. För att förstå hur användare
påverkades av första intryck och ålder samt hur anpassning till teknik fungerar har tidigare forskning inom dessa områden
presenterats. En enkät och användartest utfördes för att svara på frågeställningen.
Eftersom en övergång till en responsiv sida innebar att den visuella strukturen ändrades påverkades användares redan
existerande vanor. Resultatet och slutsatsen var att det fanns tre huvudgrupper som angav olika anledningar till vad som
skulle få dem att gå över till den responsiva sidan. Strategin för att få användarna på den icke responsiva sidan mer nöjda
med den responsiva sidan skulle vara att först implementera de funktioner som saknas. Nästa steg skulle vara att bjuda in alla
användare och att skapa guider för att hjälpa dem att lära sig hur de hittar det de oftast letar efter.
The transition to a responsive website: A user study Josefine Lindqvist
Royal Institute of Technology
Stockholm, Sweden
ABSTRACT
An anonymous company has been redesigning its website
to a responsive site and during this process the users have
the option to stay on either site. This study examined the
reasons as to why users stay on the non-responsive site and
how the responsive site could be improved to make them
switch to it. In order to understand how users were effected
by first impressions and age as well as how adaption to
technology works, previous research in these areas have
been presented. A survey and user test were conducted to
find the answers to the research question.
Because the transformation to a responsive site meant that
the visual structure changed, it affected the users’ existing
habits. The results and conclusion were that there were
three major groups stating different main reasons on what
would make them go to the responsive site. The strategy for
making users on the non-responsive site more satisfied with
the responsive site would be to first implement missing
functionality. The next step would be to invite all users and
adding guides to help them to learn how to find what they
usually look for.
Author Keywords
User study; user satisfaction; HCI; website redesign;
technological adaption
INTRODUCTION
Responsive design
Web design used to be a simpler task with only a few
screen sizes to have in mind while developing, but over
time it has become more complicated. Today, there are
multiple different screen sizes and devices to adjust a
website to. According to Fox [11] responsive design can be
used to counter this problem, which means that the website
adjusts the content based on the resolution of the screen.
This approach makes websites fit all kinds of screen sizes,
even on tablets and phones.
Glassman and Shen [8] explains that responsive design also
highlights priorities when designing a website. Instead of
doing a fully formed website developed for desktop
practices, this approach begins with finding the most
essential part of a website. This is to fit content and services
to a small internet accessible device, such as phone or
tablet. This approach makes the content contain core
functionality in its simplest form, while scaled up versions
could have extended features. With this, heavy or broad
content that is non-essential can be avoided on mobile
devices but could be enabled on desktop to give a richer
user experience. [8]
The project
The anonymous company’s product is a Swedish online
marketplace available on both desktop and mobile devices.
The users are the ones selling and buying on the site and the
sellers can offer both auctions and fixed priced items. Each
week one million unique users visit the site to browse, bid,
buy or sell items online. Not only private customers access
this site but companies also use it to drive their businesses.
The age range of the user base varies with active users from
18 years old to around 85 years old.
The company has worked on transforming their website to a
responsive site, including new design and features. The
reason behind this redesign was mainly because the mobile
traffic has increased in recent years and because the website
was designed for desktops they were not well displayed in a
mobile phone. This responsive site has been in development
for some time. On mobile devices this version of the site is
available to everyone. However, on desktop the site is
currently divided into two parts: My account and the rest of
the site. At My account the members can for example deal
with their bids, bought items and sold items. At the rest of
the site the member can for example search for items and
upload new auctions to sell. Because the responsive version
is currently in development, only half of the users on
desktop can see the option to go to the responsive version
of My account while the rest of the site is accessible to
everyone. Except these two parts, the old non-responsive
site is accessible as well and the desktop users have the
option to stay on the non-responsive site during the
transition. The different version can be seen in Figures 1-4.
Even though there are only a few functions missing on the
responsive site, around 10% of the users who can see the
link are choosing to stay on the non-responsive site. This
study explored why they are staying there and how the
responsive site can be improved to make them switch to it.
To do this, the field of Human-Computer Interaction (HCI)
will be in the center. The HCI area focuses on
understanding what creates a good experience for the users
and how products can become more user-friendly [10]. To
be user-friendly, the users must feel that a product is useful
Figure 1: This figure shows how My account looks at the
non-responsive site. This is shown to the users who
cannot access the responsive version.
Figure 2: This figure shows how search results looks
like at the non-responsive site. Users can access this site
by clicking a link or by going to a specific URL.
Figure 3: This figure shows how My account looks at the
responsive site. This version is only shown to half of the
desktop users the rest sees the non-responsive version.
Figure 4: This figure shows how search results looks like
at the responsive site. This is accessible to all users.
and that it would enhance their job performance. It is also
important for the product to be perceived as easy to use,
which means that the user can interact with a system
without effort. By exploring emotional aspects when users
interact with products and by adapting products to their
needs and capabilities, the experience can be improved and
better usability can be achieved. Because the user has an
increasingly larger role in design development, this study
will focus on their experience and satisfaction. [19]
RELATED WORK
Responsive design and website redesign methods
Earlier related work in the area of responsive web design
highlights how best practices should be defined in this area
as well as that research are needed to find out how
responsive design performance could be improved. [2,17]
User studies have had its focus on testing responsive design
practices on mobile devices [4,5] rather than testing how
existing users are effected by the visual change and
structure a new site can have.
Several case studies have been published describing library
websites redesign process with the support of user tests
[3,20]. These kinds of tests can be particularly good to
identify and understand how people use a certain site. In
one study [20] the test was conducted with prepared tasks
that represent the actions a user could take on the
website. To cover the possible actions there can be between
six to twelve tasks to perform on the website. Furthermore,
the number of participants can be small, with between five
to fourteen users. Tests with five users often identifies 85
percent of the problems. That is why it is more cost-
efficient to iterate multiple times to find all problems rather
than to do only one test with many users. [20]
In another study [3], focused on user-centered design when
redesigning a library website, iterative user tests were
conducted to include the users’ needs and wants into the
design. The researchers asked the users to think aloud while
doing pre-determined tasks on both the old and the new
site. This gave them the opportunity to incorporate the
users’ opinions into the product rather than base it on
assumptions.
Barnes and Vidgen [1] highlights that when benchmarking
the process of a redesign, surveys can be used. Surveys
should be conducted regularly, especially when a larger
change is done to a website. Also, websites are often
complex and have multiple user groups. If an improvement
is done to one group, it affects the other groups as well and
they might perceive it differently. Perhaps the change will
lessening the ease of use for them. [1]
Technology adoption
When redesigning a website, it is important to have an
understanding of how first impressions and familiarity can
affect the user. A user makes up their mind about a website
after exposures as short as approximately 50 ms according
to Lindgaard et al [16]. They further explain that this can
result in complications. If users have a positive first
impression they may disregard problems encountered later,
i.e. errors or other issues may be overlooked. The reverse
effect occurs when the first impression is negative. Even if
the experience of the website is positive after the first
negative exposure, the first impression will make the user
reluctant to accept it. [16]
A theory named Mere Exposure Effect, explaining that
people find familiar things more attractive, has had its
research well-documented for more than fifty years
according to Ruggieri and Boca [18]. The authors clarify
that the effect is a stable and verified phenomena in social
psychology but the understanding of this effect is still
incomplete. In the same study, which was about placement
of advertising, the researchers found that even one exposure
could lead to a positive attitude towards a brand. The
processing that occurs, with a single product placement
exposure, is enough to give a person the feeling of
familiarity that is later recognized as a preference for the
brand. Other studies [13] have revealed that with every
repeated exposure of a novel object people tend to find it
more attractive. It is a positive linear relationship between
attractiveness and exposure frequency.
In another study about interaction and familiarity [9], the
result was that with each interaction the user gains a better
understanding based on experiences and learnings. This
means that the user knows how to navigate and can find
things he or she wants to do and a familiarity is created with
the site. According to Cox and Cox [6] users preference for
complexity may change if the user is exposed to a website
multiple times. Furthermore, preferences for more complex
designs often increase with repeated exposure while few
exposures lead to preference for simple design.
User acceptance is an important part when redesigning a
website because it is the users who will perform tasks at the
site. The Technology Acceptance Model (TAM) is a model
that aims to describe acceptance of users in information
systems [14,15]. This model explains underlying factors to
technology acceptance and the user’s adoption processes.
The user’s intention is a way to predict their behavior and
the intention is determined by the user’s assessment of the
system’s usage, Therefore, perceived usefulness and
perceived ease of use are two key elements in this which is
highlighted in a study by Lee and Lehto [14] as well as in
another study by Lin [15]. The first term means to what
extent a user believes a system enriches their job
performance and the second term refer to what level a user
believes a system is free of effort.
The effect of ageing
Because the company has a broad age range among the
users it is important to think about elderly people and how
they can be affected by new technology and design. A study
about age and technology [12] presented that older adults
have a lower degree of adoption towards technology.
According to the study older users do not have the
experience or training that younger people have today with
technology, which often makes them novices in the area.
Furthermore, older users often use outdated equipment
compared to the younger generations. Their computers,
mobile phones etc. can be gifts from younger adults who
have upgraded their own equipment. Because of this, the
technology is limited in terms of what kind of applications
it can run. [12]
Older adults may also have problems with age-related body
changes which is described in the same study [12]. When
users perceptual-, motor-, and cognitive-abilities decrease it
will affect their use of technology. A reduced vision is a
common age-related change, which makes it harder to e.g.
read on a screen. Also, their reduced cognitive abilities
affect the short-term memory, speed of processing and
problem solving ability, which are critical when learning
new technologies. This can cause difficulties for older
adults when dealing with web navigation and dynamic
content changes. [12]
Purpose
The purpose of this study was to identify the users on the
non-responsive site and to find the reasons as to why they
are staying there. The research question is:
Why are some users staying on the non-responsive site and
what can be done to make them use the responsive site?
This study is limited to the desktop version of the site. It
does not include studying the mobile versions because on
mobile devices the responsive site is accessible to everyone
and are therefore not relevant to this study.
METHOD
Both qualitative and quantitative data have been gathered in
this study to answer the research question. The methods
conducted was data gathering by distributing a survey and
having user tests with users from the non-responsive site.
Identifying the users
The first step was to identify the users on the non-
responsive site. For this Google Analytics (GA) was used,
which is a web analytic service that tracks and reports
website traffic. To identify the sellers, member-IDs of users
who have successfully listed at least one auction through
the non-responsive site during November 1st to January
31st were extracted with GA. In the same way the buyers’
member-IDs were exported by knowing that they had
confirmed a bought item through the non-responsive site.
However, because the number of buyers in the same time
span was unmanageable, the decision was made that it was
sufficient to extract buyer data from the most recent month,
January.
To better understand who these users were and to gather
quantitative information, their member-IDs were used to
extract demographic data, such as age, gender and location.
This was done in a program named QlikView, which is a
business intelligence & visualization software. Other
relevant factors were also extracted, e.g. their e-mails, if
they accepted to receive e-mail, how much a user sells/buys
per month, how much Gross Merchandise Volume (GMV)
they have produced etc.
Survey
To gather information on what the identified users think of
the responsive site, an online survey was conducted. The
focus of the survey was to understand the users, why they
use the non-responsive site as well as what would make
them go to the responsive site. It also included questions
about what they gain from staying the on the non-
responsive site and what their issues are with the responsive
site. The survey consisted of questions with mostly free-text
inputs, the rest was multiple-choice options. This was to
encourage the users to think freely on what they want,
instead of limiting them to predefined alternatives. In the
survey result, the sellers’ and the buyers’ answers were
separated to see if there was any difference between them.
The survey was sent to all identified sellers who accepted to
receive e-mails. On the other hand, because the buyers were
a larger number of users the decision was made to make a
selection out them to be able to handle the expected data
outcome from the survey. The survey was therefore sent to
around half of the identified buyers who had confirmed
most bought items and accepted being contacted by e-mail.
Because the survey mostly had free-text inputs the answers
needed to be analyzed. Therefore, the responses on each
question were read and categorized into different groups
depending on their inputs.
User tests
In order to gather deeper insights on design and
functionalities of the site user tests was conducted. The
purpose of these user tests was to understand how sellers
and buyers on the non-responsive site work on the site and
to find their pain points at the responsive site. Questions
and tasks on both the responsive and non-responsive site
were prepared beforehand.
To observe how users work with the site on their own
devices these user tests took place at the user's home, if
possible. To represent the users on the non-responsive site,
five different types of users were invited. As mentioned in
related work section, five users can often identify 85
percent of the problems [10]. The different types of users
were: a corporate big seller, a private big seller, a private
small seller, a private big buyer and a private small buyer.
A big seller is someone who list more than 50 times a week
and a small seller list only a few per month. A big buyer
means someone who buy around two times a week and a
small buyer purchase around one time a month. Three of
the user tests took place at the user's office or home and two
took place at the company’s office.
During the user tests the users were asked to think aloud
which requires the users to express their thoughts as they
were engaging in completing tasks. If the users stop
verbalizing for some time they were reminded to think
aloud. This data gathered from this is qualitative and
therefore it is preferred to have a smaller number of users
performing this method during user tests [7]. The user tests
were recorded using a video camera during home visits
because then the users did not need to have certain
recording software installed on their private computers.
During the user tests at the office, screen recording software
was used on the company owned computer. The video
recording was to support taking notes and making sure
everything that was said or done could be followed up later.
Graph 1: Age distribution in intervals for both sellers and buyers. Total number of buyers and sellers were 13247 respectively
39341.
All users got to decide if they wanted to be recorded or not
after the purpose of the video was explained.
RESULT
Identifying users
The results of the initial data gathering from GA identified
13247 sellers who used the old site to upload new auctions.
2,9% of them were corporate sellers, meaning they were
companies using the site to sell items, and the rest was
private users. The gender distribution was 51% female and
49% male among both the private and corporate sellers. The
number of identified buyers was 39341 who had bought
items from the non-responsive site. Out of these, 99,7%
were private and the rest were corporate users. The gender
distribution was 53% male and 47% female.
When looking closer at age intervals the distribution was
rather even among the sellers. All intervals contained
between 12% and 22% of the sellers, with the most users in
the age range 48-57 years old, as seen in Graph 1. The
intervals with the fewest users were the younger users
under 27 and older users above 68 years old. Among the
buyers it is a more uneven spread compared to the sellers.
The age intervals contained between 9% and 25% of the
buyers, but it is still the age range 48-57 that contain the
most buyers. The interval with fewest users was users under
27 years old and over 68 years old.
Survey
Sellers
The seller survey got a total of 2015 responses in the five-
day time span the survey was open, which represented
15,2% of the identified sellers. The majority of the sellers
answered that they were missing key functionalities on the
responsive site and that the older design fit them and their
usage. A fewer number of users said that they used the site
that appeared first.
Most of the feedback on the functionality was about the
lists in My account, where the sellers said they missed
information and actions to take that existed in the lists on
the non-responsive site. One other major functionality the
sellers wanted to be improved was the page where they
leave feedback to buyers, e.g. because of smaller bugs and
that they wanted to leave feedback on multiple items on one
page to save time.
One of the reasons behind why the sellers stated that the
design was better on the non-responsive site was because of
the lack of a clear overview. It can be exemplified by these
quotes:
“Text, boxes and images are smaller
compared to the new site, which
means you get a better overview
(almost everything fit on the screen so
you do not have to scroll a lot)”
“It was a better overview of sold and
not sold, I could easily choose to look
what was not paid or items I have not
left feedback on.”
Others just stated that the overview was better on the non-
responsive site and that it was easier to use but with no
further explanation of why that is.
The answers from the question “what would make you go
to the responsive site?” were grouped into three major
reasons. Reason number one was users stating that they
would never switch and would rather quit using the site
than to use the new one. Reason number two were users
who missed functions and the last reason was users who
wanted to go there but did not know how.
The largest group was those who said they would never
want to go to the responsive site. Some quotes on this was:
“Nothing! When it disappears I will
stop advertise! It will not be simpler
than that!”
“I’m happy with the old site and I
will probably quit using the site if the
old disappears”
“No I do not want to go to new design
because I’m used to the old one… I
find everything with a click. I have
been on the site since the start so I
speak with experience. My wish is to
KEEP THE OLD DESIGN!”
These people were generally upset with having to deal with
a new site for various reasons. One of the reasons was
because they have to learn something new and that will take
time. They mentioned being used to the non-responsive site
and was therefore happy with it. They were also saying that
it takes too much time to list auctions and that it is not
worth the time. It is more efficient for them to list larger
number of items on the non-responsive site compared to the
responsive one. There were also users saying that they
would only go to the responsive site if the old site is shut
down, and even then a few would rather consider quitting
then to switch.
The second largest group were those who were willing to
go to the responsive site if the functions they use are
implemented there. They said that the responsive site is
unusable right now because the functions they use are not
incorporated on the site.
The last group wanted to go to the new site but did not
know how to do it.
“If it is easier to find”
“Why aren't we being redirected
there directly?”
These quotes were typical for this group of users. They
wanted to find the site more easily by being helped to get
there by clear instructions or a guide. A few also said that
they wanted to be automatically redirected to it.
Buyers
In the buyer survey a total of 1956 users answered in five
days, which represent 24% response rate to the survey.
Most of the buyers stated in the survey that they were sent
to the non-responsive site automatically and a few did not
see the difference between them or did not even know that a
new site existed. For those who actively stayed on the non-
responsive site they were doing it because they are used to
it and they have their working habits. Some stated that this
makes it easier to use and to understand. They also
mentioned that the overview is better on the non-responsive
site, with less scrolling and clicking to navigate. In addition
to this, there are important functions that are missing which
stops them from switching to the responsive site.
The answers to what would make them go to the responsive
site were for the buyers also grouped into three major
reasons. The largest number of users stated that the entrance
to the responsive site should be easier to find. This is some
examples of quotes:
“If I knew how to do it”
“If I could find it… why are there
several sites?”
“If I could easily switch. Clear
explanation of how I change.”
A few specifically mentioned having clearer instructions
explaining how to go to the new site. Other wanted a guide
on the responsive site explaining what is new and how
things can be done.
The second largest group of users would like the overview
and the layout to be better to be able to use the responsive
site. They did not state how this can be achieved with the
exception of reducing the number of clicks required on the
responsive site and to remove white space in the auction
lists in My account to make the lists more compact. These
were typical quotes for this group:
“Hard to believe that someone can
get me to start using the new site
when there is so much clutter and so
incredibly immense. “
“Clearer overview. Easier to find.
Less messy.”
The last and third largest group wrote that they would never
go to the responsive site and that they would quit their
membership if the non-responsive site is closed. The reason
for this is that they do not believe that something can
replace the non-responsive site and that something that is
already perfect should and cannot be replaced.
User tests
During the tests, the users consisted of three males (52, 54
and 72 years old) and two females (53 and 70 years old). A
summary of the age, gender, type of user and how much the
test user sell/buy can be seen in Table 1.
AGE GENDER TYPE OF USER SELL/BUY PER WEEK
52 Male Big seller corporate Several hundred items/week
53 Female Big seller 50 items/month
72 Male Small seller 1 item/month
53 Male Big buyer 2 items/week
71 Female Small buyer Occasionally
Table 1: This table show the test users’ age, gender, what type
of user they were and how much they sell/buy at the site.
The users had varied technological knowledge which was
something they got to interpret themselves but was also
observed during the user tests. Their difference in
technological knowledge made their interactions with the
site quite different. The older people was more careful
when navigating on the site compared to the younger, even
though the age gap was not that large.
When it came to the equipment they used, all of them had
both mobile phones and computers. Their screens and
computers were relatively new and the computer screens
were between 12-27 inches. The one having a 12-inch
screen also had a bigger screen at work which he used
frequently. All except one had mobile phones that could
access internet.
Two of the users were keen to stay on the old site. One,
who was a corporate seller who sell several hundred smaller
items a week, said that:
“I have to be effective to sell a lot”
He said that he felt that the responsive site did not
encourage big sellers because it is slower to put up similar
auctions again. In the non-responsive site there is a function
where he can restart auctions with edits that it is missing on
the responsive one. He would never upload a totally new
auction because it requires filling out all information again
and that takes time. After he have sold something he use a
list called a “picklist” where all sold items are shown
grouped by buyers and it is printable. This list on the non-
responsive site can be filtered and it is all in one page,
however on the responsive site it is quite different. There is
no filtering and it has pages, which made his picklist 27
pages with 50 items on each page. He states that this is
unusable for him because it would take too much time to
look through it.
The other person, who's very keen to stay, was the big
seller. She worked differently compared to the corporate
seller. She did not use the picklist even though she sold 50
items a month, instead she used the “sold” list to manually
copy the addresses to the buyers and pasting it in her own
document, which she later can print to place on packages.
She had tried the responsive version several times but did
not like it because she has her working habits and was used
to the non-responsive way. She said that:
“Why change the old site when it
works?”
The next user was a small seller and he did not see the
reason why he should switch when they design-wise look
the same. Because he was a small seller he did not need the
functionality of the picklist for example, that is why he used
the other seller lists. He just wanted to be able to see his
active and sold auctions in the same list, which he cannot
on the responsive site. He said that it is unnecessary to
divide them into two different lists when he often only have
one or two items to view at the same time. He also
mentioned that he would like bigger images to know what
item he is dealing with which he wanted on both sites.
Except this, he did not see any functionality differences
when comparing the both sites and he did not understand
why the site was rebuilt when he said this.
The big buyer bought items around two times a week and
he used both the mobile phone and his laptop to go to the
site. This made him quite confused because the mobile
version is the same as the responsive site and the functions
that were new were only working on his mobile phone. He
believed that he was on the new site on both devices and
that they were just structured differently. He said that he
thought that the company did as other sites, where they
release updates without you having an option of choosing
what version you want. He said the mobile version was
better when viewing auctions in the “wish list”, a list
whenever you can add your favorite auctions, because he
could filter the list on ongoing and ended auctions.
However, because he was one of the users that could not go
to the new My account-page he could not use the
responsive site on his desktop. When talking about change
and to learn how to use a new site he said that everything
new is always an obstacle to overcome but you get used to
it eventually.
The last user was a small buyer who bought items
occasionally, mostly for her grandchildrens’ birthdays and
during Christmas. She used the whole non-responsive site
because it is the first that is shown when she goes to the site
in the browser. She was also one of the users who did not
have the option to go to the responsive version of My
account. After she tried the responsive site with a test
account she said that she did not understand why there are
two sites and she is uncertain if she would go to the new
one if she could. She liked the new design of viewing her
own feedback because it was clearer, but she said that she is
used to the old navigation and she knows how it works.
Then she states that:
“Of course you can get used to
anything maybe"
DISCUSSION
The purpose of this study was to first identify the ones
using the non-responsive site. The next step was to
understand why users choose to stay on the non-responsive
site and what can be done to make them more satisfied with
the responsive one.
Identified users on the non-responsive site
The results from identifying the users on the old site made
it clear that all ages and genders are using the non-
responsive site. Even though the buyers’ age distribution
was not as even as the sellers it is still even enough to point
out that all ages use it. The most common age interval was
the ages between 48-57 for both buyers and sellers, which
is when age is beginning to take its toll on the body. As
stated in the related work, body changes come with age and
perceptual-, motor-, and cognitive-abilities decrease among
those who are older. Older users often use outdated
technology and do not have the same base as younger
people when it comes to learning and to adapting to new
things. This could be seen in the user tests by the older
user’s careful behavior while navigating. However, while
visiting users their technical equipment could be observed
and the two users visiting the company’s office got to
describe their equipment. The results of this was that they
did not have outdated equipment. This was a surprising
discovery because it was not expected, however because of
the small sample this cannot be said for every older user.
The reasons for staying on the non-responsive site
The survey and user tests identified three main reasons
when it comes to leaving the non-responsive site to use the
responsive one. The largest group among the sellers and the
last of the three major reasons among the buyers said that
they would never leave and that they would rather quit
using the site then use the responsive one. Related work in
this area that was examined in this study did not highlight
the consequences of having a new visual design and
structure when transforming a site to become responsive.
The result from this study is that both old and young users
said that it is a hassle to learn something new and that it
will take them extra time to perform their tasks at the site.
This group also said that they did not see the benefits with
having a responsive site even though more and more traffic
goes through mobile devices.
The fundamentals of HCI and TAM is that users must feel
that they are gaining something from using a product and
that it enhances the user’s performance. This does not seem
to be the case right now when users have a hard time
learning to navigate on the responsive site. Two persons
also mentioned in their user tests that they did not see the
benefit of a new site when the non-responsive site works for
them and when it looks the same.
Because the user groups said that they would never use the
responsive site, they most likely had a negative experience
from the start. Therefore, it could be argued that they are
reluctant to see what the advantages are with the responsive
site. However, studies have pointed towards people getting
more familiar with each usage and exposure to a site. With
this, the users in this group needs to repeatedly use the site
in order to learn how it works and get a positive experience
from it. Still, this is complicated matter because they can
choose to go back to the non-responsive site by themselves
which means that they can avoid getting familiar with the
responsive site.
The second largest group among the buyers and the third
largest group among the sellers, said that they would like be
on the responsive site. This group also said that they would
like some guidance on how the site works and what is new
to be able to find it easier. Currently only 50% get the
opportunity to see a link to the whole site and there are both
advantages and disadvantages with dividing it like this.
Because users make up their mind after only a few
milliseconds, the first impression often stays and effect the
experience. Since the site is not completely finished there is
always a risk of people trying it out and are getting a bad
experience because certain things are missing. Therefore, it
is an advantage to show the link to a limited amount of
users first because the things that need improvement can be
identified and corrected without effecting the rest of the
users’ experience. On the other hand, a disadvantage is that
users become confused while using the responsive design
because they are sent to the non-responsive site when
functions are missing, and sometimes they cannot find their
way back again. This can happen when they want to, for
example, book shipping, contact highest bidder or use other
functionalities that is not implemented on the responsive
site yet. To be able to make everyone use the whole site, all
the known missing functionalities needs to be implemented
and the problems that have been discovered need to be
fixed as well to get the uses to be satisfied.
The last reasons was different for the sellers and the buyers.
One was because functionality was missing and the other
was that the overview was worse on the responsive site. As
previously mentioned, missing core functionality is mainly
the reason to why not everyone can go to the responsive
site. The survey and user tests identified additional
functionalities that was missing apart from known ones and
also identified problems with existing functionalities, e.g.
the picklist and restarting auctions. That the overview was
worse on the responsive site compared to the non-
responsive one was a new discovery but not many users
stated why it was worse. The known reasons such as white
space in the lists is because the design approach was doing
mobile devices first which resulted in the lists being empty
in the middle when scaled up. This is something that needs
to be considered to make it more desktop friendly.
Designing for two user groups
In this study the users from the non-responsive site are in
focus, but it is important while designing not to forget the
existing user’s behaviors and needs. If something is to be
changed on the responsive site to please the non-responsive
users, the users on the responsive site needs to accept this
change as well. It can become a delicate matter balancing
the two different user groups because of their varying
opinions and needs. The best way would be to find a
solution that would fit both of them, but the risk is that it
instead fit no one because the design is a compromise. The
same problem can occur while dealing with small and big
buyers/seller because they use the site quite differently.
Heavy sellers often use the “picklist” because the view is
more compressed which means that they can oversee more
items at a time. However, a small seller does not have the
same need as heavy sellers. As the small seller said in the
user test, he wants to have everything in the same lists both
ongoing and sold auctions. If this was to be implemented, it
would be unusable by the bigger sellers. These different
types of user need to be taken in consideration as well for
every design decision.
What can be done to increase the satisfaction on the responsive site?
While still having the different user groups in mind,
missing functionalities is something that needs to be
implemented. This will make all those who are using these
functionalities be more satisfied with the responsive site.
This study also discovered functionality that exist on both
sites but are not satisfactory enough on the responsive site.
This needs to be researched further to discover what the
problems are with these and how to solve them. When the
implementation of missing functionality is completed, the
link inviting users can be shown to all users. If this is done
earlier it can affect the user’s first impressions negatively
because of the known missing functionalities. At the same
time as showing the link to all users, the guides should be
done to help the users with finding everything they usually
use.
The hardest users to satisfy are the users who do not see a
benefit with using the responsive site and said that they
would never use it. Because the number of exposures effect
the way a site is perceived hopefully more exposures could
lead to more users to be convinced to switch site. If some
users continue to stay on the non-responsive site despite
repeated exposures, perhaps additional features need to be
implemented for them to see the value of using the
responsive site. However, this needs to be examined with
additional surveys, interviews or user tests with the users.
Method discussion
Because of the large number of users that stay the non-
responsive site the survey was an appropriate approach to
gather as much insights as possible in a limited time frame.
One advantage of using a survey is that it can reach many
users located all over Sweden in a short amount of time
compared to other methods such as workshops, telephone
interviews etc. However, surveys have their limitations as
well because it is a one-way communication and it is
unclear if the respondents understand the questions asked.
There is also no means of asking quick follow-up questions
to their responses to further understand their answers. This
is why the survey was supplemented with user tests. The
user tests were also conducted because the users’ way of
working and usage of the site could be observed while they
were performing specific tasks. While surveys can gather
large quantity of data, this method was used to dig even
deeper to gain further insights about the users and their
needs. On the other hand, user tests are time consuming and
can only be done with smaller amounts of participants. In
this case, the users taking part in the test were older because
younger users did not have the time during the day to either
participate in their home or at the company’s office. This
was unfortunate because the result from the survey showed
that the age distribution was quite even. Therefore, it is
important to have in mind that the results from the user test
might be skewed. However, because the younger users
could make their voices heard trough the survey they were
not forgotten.
Future work
This was an initial study to discover user needs and
problems with the responsive site. To keep track of how the
progress goes of transferring users to the responsive site,
more user studies need to be conducted. This is to be able to
see how the satisfaction is among the users and to be able to
adjust the site to their usage. When doing a redesign of a
complex site such as this company with its diverse user
base, it is important to be able to give value to the users. If
the users are not satisfied they will look for other forums to
buy and sell online.
The results of this study can be a start of discovering how
redesigns can be done in the future. In this specific case the
users had the option to go to the responsive site by
themselves, but other solutions could be researched to find
the most effective and satisfying one for the user when
redesigning a site.
CONCLUSION
To conclude, the reasons behind users staying on the non-
responsive site were mainly because of three things;
functionality was missing, they could not find the
responsive site or they were more comfortable with the way
the non-responsive site works.
To make user from the non-responsive site more satisfied
with the responsive site and move them over, the first step
would be to implement the functionality that are missing.
Because the user base is varied, the implementation of the
missing functionality need to be carefully planned to fit the
user’s needs on both the responsive and the non-responsive
site. When all functionality is implemented, all users can be
invited to use the responsive site with guides helping them
find the functionalities they use. However, there were users
saying that they would never switch sites and to get them
satisfied with the responsive site more research needs to be
done in order to find out what would make them satisfied or
comfortable with the responsive version.
ACKNOWLEDGMENTS
A special thank you to the company, its employees and
members for supporting this study with all their knowledge
and expertise. My gratitude also goes towards my
supervisors at KTH and the company.
REFERENCES
1. S.J. Barnes and R.T Vidgen. 2001. Assessing the
Effect of a Web Site Redesign Initiative: An SME
Case Study. International Journal of Management
Literature 1, January.
2. Meltem Huri Baturay and Murat Birtane. 2013.
Responsive Web Design: A New Type of Design
for Web-Based Instructional Content. Procedia -
Social and Behavioral Sciences 106, December:
2275–2279.
http://doi.org/10.1016/j.sbspro.2013.12.259
3. Danielle a. Becker and Lauren Yannotta. 2013.
Modeling a library website redesign process:
developing a user-centered website through
usability testing. Information Technology and
Libraries 32, 1: 6.
http://doi.org/10.6017/ital.v32i1.2311
4. Kim Bohyun. 2013. Responsive Web Design ,
Discoverability , and Mobile Challenge. Library
Technology Reports 49, 6: 29–40.
5. Henrik Bygdeman. Från desktop till responsiv
design – en fallstudie kring utformningen av
publikationsdatabasen på Skolverkets webbplats.
Csc.Kth.Se. Retrieved from
http://www.csc.kth.se/utbildning/kth/kurser/DD143
X/dkand11/Group2Mads/Rapport_Sebastian_Hew_
Toni_Huotari.pdf
6. Dena Cox and Anthony D Cox. 2002. Beyond first
impressions: The effects of repeated exposure on
consumer liking of visually conplex and simple
product designs. Spring 30, 2: 119–130.
http://doi.org/10.1177/03079459994371
7. Sharon Eveland Jr., William and Dunwoody. 2000.
Examining Information Processing on the World
Wide Web Using Think Aloud Protocols. Media
Psychology 2, 3: 219–244.
http://doi.org/10.1207/S1532785XMEP0203
8. Robert Fox. 2015. Being responsive. OCLC
Systems & Services: International digital library
perspectives, 28, 3: 119 – 125.
9. David Gefen. 2000. E-commerce: the role of
familiarity and trust. Omega 28, 6: 725–737.
http://doi.org/10.1016/S0305-0483(00)00021-9
10. Michael Glanznig. 2012. User Experience
Research: Modelling and Describing the Subjective.
Interdisciplinary Description of Complex Systems
10, 3: 235–247. http://doi.org/10.7906/indecs.10.3.3
11. Nancy R NR Glassman and Phil Shen. 2014. One
Site Fits All: Responsive Web Design. Journal of
Electronic Resources in Medical Libraries 11, 2:
78–90.
http://doi.org/10.1080/15424065.2014.908347
12. Vicki L. Hanson. 2010. Influencing technology
adoption by older adults. Interacting with
Computers 22, 6: 502–509.
http://doi.org/10.1016/j.intcom.2010.09.001
13. Paul Hekkert, Clementine Thurgood, and T. W
Allan Whitfield. 2013. The mere exposure effect for
consumer products as a consequence of existing
familiarity and controlled exposure. Acta
Psychologica 144, 2: 411–417.
http://doi.org/10.1016/j.actpsy.2013.07.015
14. Doo Young Lee and Mark R. Lehto. 2013. User
acceptance of YouTube for procedural learning: An
extension of the Technology Acceptance Model.
Computers and Education 61, 1: 193–208.
http://doi.org/10.1016/j.compedu.2012.10.001
15. Chin Chao Lin. 2013. Exploring the relationship
between technology acceptance model and usability
test. Information Technology and Management 14,
3: 243–255. http://doi.org/10.1007/s10799-013-
0162-0
16. Gitte Lindgaard, Gary Fernandes, Cathy Dudek,
and J. Brown. 2006. Attention web designers: You
have 50 milliseconds to make a good first
impression! Behaviour & Information Technology
25, 2: 115–126.
http://doi.org/10.1080/01449290500330448
17. S. Mohorovicic. 2013. Implementing responsive
web design for enhanced web presence. 36th
International Convention on Information &
Communication Technology Electronics &
Microelectronics (MIPRO): 1206–1210.
18. Stefano Ruggieri and Stefano Boca. 2013. At the
roots of product placement: The mere exposure
effect. Europe’s Journal of Psychology 9, 2: 246–
258. http://doi.org/10.5964/ejop.v9i2.522
19. Manuel Sánchez-Franco and José Roldán. 2005.
Web acceptance and usage model. International
Journal of Retail & Distribution mangement 34, 1:
21–48. http://doi.org/10.1108/02656710210415703
20. Robert L Tolliver, David S Carter, Suzanne E
Chapman, et al. 2006. Website redesign and testing
with a usability consultant: lessons learned. OCLC
Systems & Services: International digital library
perspectives 21, 3: 156 – 166.
www.kth.se