View
35
Download
3
Category
Tags:
Preview:
Citation preview
Kasey AlpertInto to Web PublishingJacquie LamerUsability Test Assignment3/10/2015
Usability Test for KansasCity.com
Test Preparation
I prepared for the test by first going through each of the tasks that the participants
would be asked and complete them as if I was testing myself. The first task was to find what
films were nominated for best picture for the Academy Awards. I typed “best picture academy
awards” into the search tool that is in the header on KansasCity.com. The first article that
showed up was a photo gallery for the Oscars. I had to scroll through 6 pages on the photo
slideshow to find the names of the films that were nominated for best picture. The next
assigned task was to find out when the Oscars air on TV. Before I went to go search for the
information, I noticed that what I was looking for was in the caption of the photo that was still
up from the last task. The assignment for task 3 was very easy, in my opinion. The task was to
find the section of the site titled “816”, click on an article and go through the process of
ordering reprints of it. I first hovered over the “Living” tab, and then when I didn’t find what I
was looking for I went over to the “News” tab and found the “816” section. There is a circular
button with an arrow encased in it at the top of the article, and I hovered over it. I was right in
guessing that this button would display a link to order reprints. Once the link had been clicked,
it takes you to a different website where reprints can be ordered. I thought finding the reprint
link was straightforward and simple. However, task 4 proved to be more difficult. The task was
to determine the events that will take place over the following week in Jackson and Cass
counties. I went through a couple different searches, typing “upcoming events in Jackson
county” and “events calendar” through the search tool, but they weren’t helpful. I went to the
“News” tab and then clicked on “816”, and I found an article that had some recent events that
happened in Jackson and Cass County, but the article didn’t mention future events. In the end, I
couldn’t find exactly what the task was asking for. The next task was to search for a job in a
specific city using a keyword. There is a “Classified” tab that I went to that places a search for a
job using a keyword, and the city and state can also be specified. When I clicked search, the
jobs page didn’t show up. Instead KansasCity.com reloaded back to the home page. So instead
of searching the jobs through the Kansas City Star’s website, I clicked a link titled “View All Top
Jobs”, which is located right under the job search under the “Classified” tab. This took me to
careerbuilder.com where I could then search for a job in Kansas City. The last task was to share
a job listing on social media. Completing this task proved to be difficult. I could not locate a link
to share a listing on any social media site. Some job listings had the company Facebook or
Twitter feed on the side, but the only way I could find to share a job listing was to email it.
Choosing Participants
Tester 1: Karla Everman
Karla Everman is 19 years old, and is from the Kansas City area and a fulltime
student and part time tutor at Northwest Missouri State University. I chose her as a
tester because I knew that she would have an interest in participating. She spends
approximate 8 hours a day on the internet with 10% of her time spent browsing and
90% on social media. Karla is a high experience tester.
Environment for Tester 1
Location of test: The location of the test was in Karla’s room at her desk. Her
room was chosen because this is where she does most of her internet browsing.
Physical environment: There was an overhead light that was on, as well a
another overhead light in the background and a lamp nearby. Karla had her
phone on vibrate and was placed on the desk, to the left of her computer. The
room was silent; there were no other people in the room and the air
conditioning was off.
Technical environment: The computer used was a HP EliteBook laptop with a
screen resolution of 1366x768. Karla used Google Chrome for the test and did
not have any add ons installed. The internet connection speed was
approximately 5Mb/s.
Tester 2: Brenna McLeod
Brenna is a 21 year old from the Kansas City area and is a full time student at
Northwest Missouri State University. She spends about 40 hours a week on the internet,
and only 20% of that time is on social media. I chose her as a tester because I thought
she would be a good high experience user because she works part time with her job that
is back home, and all of her projects are on the computer.
Environment for Tester 2
Location of test: The location of the test was in Brenna’s room at her desk. Her
room was picked because that is where most of her internet browsing is done.
Physical environment: There was an overhead light on, a lamp, another
overhead light on, and Christmas lights were on. There was a window open, and
some slight noise could be heard from the people outside.
Technical environment: The computer used is a HP EliteBook laptop. The screen
resolution is 1366x768 and the internet connection was about 5Mb/s. For the
majority of the test Chrome was the browser that was used, and an adblock ad-
on was in-use. Internet Explorer was used for the last half of the assigned tasks.
Test Results
Initial Site Thoughts:
Tester 1 Report: Karla got to the site by typing the url into a google search. Once
she got to the site, she read through all of the articles that were displayed on the
home page. She scrolled by using the mousepad. When Karla got to the bottom
of the page, she noticed the bottom navigation and thought that it was a good
thing for the site to have.
Tester 2 Report: Brenna first noticed the temperature display when she reached
Kansascity.com. She said that, “there are lots of big things, and a lot going on but
it doesn’t seem too crowded”. She also used the mousepad to scroll and she said
that it seemed like she was scrolling forever. Once Brenna reached the bottom of
the page she remarked that, “it doesn’t seem too hard to find things specific, but
it could be crazy if you’re just browsing”.
Task 1: Determine the following information about the 87th Annual Academy Awards.
Tester 1
What films are nominated for best picture?: Karla first went over the
“Entertainment” tab and read through all of the topics before clicking on
“Movies”. She scrolled using the mousepad, and went through the entire page
until she got to the bottom. When she didn’t find what she was looking for she
went back to the top of the page and hovered over the “News” tab, then the
“Home” tab, reading through each of the topics under the tabs. She then went
back over to the “Entertainment” tab and clicked on “TV”. Karla started reading
through each of the titles of the articles and got distracted by the titles that were
displayed on the right side of the page as she scrolled through the whole page.
Her phone vibrated at this point, and she seemed to give up on the task after she
checked her phone. We moved on to the next task.
- Answer: Didn’t finish task.- Completed: No.
When will the Oscars air on Television?: She first went to the “Entertainment”
tab and after reading through the topics, and then clicked on “Star TV Listings”.
She scrolled using her mouspad, and read through the tv listings. Karla took time
looking through the listings, searching for when the Oscars would be airing. She
went through many pages of the tv listings before saying “there should be an
easier way of doing this”. She thought maybe she had passed the listing for the
Oscars so she went back up to the top of the page and started going through the
listings again. At this point I knew that the task was not going to be completed so
I advised that we move on.
- Answer: Did not find the answer.- Completed: No.
Tester 2
What films are nominated for best picture?: Brenna first went to the
“Entertainment” tab and clicked on the section “Movies”. She started scrolling
using the mousepad and was looking at the headlines first. She asked: “can I use
the search bar in the site”? I told her that that was acceptable and she typed in
“academy awards and movies” into the search tool. She said that it would be
helpful to filter out the results. Brenna then searched “academy awards”, and
when she didn’t find what she wanted she searched “Oscars”. She then spent
some time reading through all of the headlines to see if an article contained the
information that she needed. Brenna then did a couple more searches, typing
“87th Oscars” then trying “best picture”. The top link that came up when she
searched for the best picture was a photo gallery that had Oscar information.
She said, “I don’t think this is right”. She did another search for “Oscars best
picture” and scrolled through the results. She was starting to get frustrated and
said “it’s really buried; not the first thing that comes up”. At this point my phone
vibrated, and I hoped that it hadn’t distracted her. She was still looking through
articles and eventually found the answer.
- Answer: “American Sniper, Birdman, Boyhood, The Grand Budapest Hotel, The Imitation Game, Selma, The Theory of Everything, and Whiplash”.
- Completed: Yes.
When will the Oscars air on Television?: She searched “Oscars air time” in the
search tool. She mentioned the photo gallery that she had seen during her
search in the previous task, and said that it probably contained the information
she was looking for. She clicked on a different photo gallery that contained
pictures of celebrities from the red carpet event of the Academy Awards. Brenna
clicked through a couple of images until one of them said the date that they
were taken.
- Answer: “February 22”- Completed: Yes.
Recommendations to improve UX for this task, linked to Jakon Nielsen’s 10
Usability Heuristics: Both testers thought that the site was busy, and caused
some distractions. I would suggest cleaning up the design of the placement of
the articles. The heuristic that most aligns with this idea is “aesthetic and
minimalist design”. When searching through a topic, such as “816”, there are
multiple places where articles are displayed. The featured articles should be off
to the side more and not displayed throughout the page. It becomes distracting
and overwhelming when browsing.
[see next page for screenshot example]
Original
Proposed
Task 2: Find the section of the site titled “816” which covers community news
Tester 1
In this section, find a story of interest and go through the steps to order
reprints of the article. Go as far as possible without actually ordering: Karla
hovered over the “Living” tab and read through the available topics, then she
went over to the “Entertainment” tab and did the same thing. Then she found
that the “816” topic is located under the “News” section. Once she got into the
“816” topic she quickly found an article of interest. After she found the article,
her phone went off and she quickly checked it before returning to her task. She
immediately hovered over the arrow button and clicked on the order reprints
link. Then Karla clicked on the “Request reprint” link on the external page and
quickly found the form to fill out.
- Completed up to ordering: Yes.
Tester 2
In this section, find a story of interest and go through the steps to order
reprints of the article. Go as far as possible without actually ordering: Brenna
started by looking in the “Home” tab before finding the section in the “News”
tab. Once she clicked on “816” she scrolled down to the bottom of the page and
noticed the link that read “Print” in the bottom navigation. She said “that’s
probably to order the paper”, so she went and clicked on an article. There was a
pop-up that said she must sign in as a member to get unlimited access. When
she exited the pop-up the site just reload back to the home page. So, she closed
down Chrome and opened up Internet Explorer. The first thing she said when
she got to Kansascity.com was that the website “looks obnoxious with the ads”.
When she used chrome, she had an adblock on. She then got back to the same
exact article by doing all the steps already stated above. Once Brenna got to the
article, she scrolled down to the bottom of the page while saying that she was
“looking for text that says reprint”. Then, she scrolled back to the top and found
the icon with the arrow on it and clicked the link to order reprints. She followed
the link all the way until the form was on the screen. After, Brenna mentioned
that she thought that task would be simple, but it turned out not to be.
- Completed up to ordering: Yes.
Recommendations to improve UX for this task, linked to Jakob Nielsen’s 10
Usability Heuristics: Hiding the reprint link in a button makes it difficult for some
users to find. I would suggest placing the link directly on the article page, so that
once a visitor reads the article, and reaches the bottom of it, they can clearly see
the text to order a reprint. A link to print the article should also be added to stay
consistent with expectation. This would be an example of the “Recognition
rather than recall” heuristic.
[see next page for screenshot example]
Task 3: Determine what events will take place in the following week in Jackson and Cass counties.
Tester 1
Determine what events will take place in the following week in Jackson and
Cass counties: Karla went over to the “News” tab and clicked on the “Local
News” section. Then it occurred to her that she could have used the search tool
within the website to help guide her search. She typed “Jackson County” into the
search tool and then scrolled through the results. When she didn’t find what she
wanted she typed “upcoming events in Jackson co”. This search provided Karla
with events in Jackson Co., but they were not upcoming ones. She kept scrolling
through the search results and went on to page 2. There on page 2 she found a
Original
Proposed
The icons that are at the end of an article
calendar of events for Jackson and Cass Co. She read through it and got
distracted by the titles of the events.
- Answer: Read off each of the titles until I told her to stop.- Completed: Yes.
Tester 2
Determine what events will take place in the following week in Jackson and
Cass counties: Brenna first looked under the “News” tab and got distracted by
the different topics. She said she was looking through all of the tabs for a
calendar, and as she was looking, ads were popping up which were distracting.
She ended up clicking the “local news” section under the “News” tab. Brenna
said that she couldn’t find a calendar anywhere. So, she typed “calendar” into
the search tool and then clicked on an article that was on how to submit a
calendar, which wasn’t helpful. Then she did a few searches, typing
“neighborhood news”, then “upcoming events”. Brenna then filtered the results
by latest date posted. With this still not delivering the results she wanted, she
became frustrated and annoyed. Brenna said that, “I just want there to be a
calendar”. Then she wanted to move on to the next task and didn’t want to try
to figure this one out anymore. So, she gave up.
- Answer: None, gave up.- Completed: No.
Recommendations to improve UX for this task, linked to Jakob Nielsen’s 10
Usability Heuristics: There is no calendar for visitors to look at. Both of my
testers would have benefited from being able to visit a calendar that listed all of
the upcoming events. The upcoming events page is buried deep within the site,
and I am sure that many visitors have never been able to find it, or find it only
after looking for a long time. I would suggest that the site follows the heuristic
“Match between system and the real world”, and add a calendar that can be
visible to visitors.
[see next page for screenshot example]
Add a link that when clicked, will take the visitor to a calendar of events.
Original
Proposed
Task 4: Determine the following information related to finding a job.
Tester 1
Search for a job in any specific city using any keyword: First Karla went to the
“Classifieds” tab and then went over all of the other tabs before beginning to
type into the search bar. She didn’t finish typing because the titles of the articles
distracted her. She started scrolling down the page and then read through the
bottom navigation links when she got there. Then she went back over all of the
tabs again before clicking the “jobs” link at the top right of the page. At this point
it took her to an external page hosted by careerbuilder.com. Karla typed in her
keyword “Accountant” and also typed in a city and selected a category as well.
Then she scrolled through most of the job listings before clicking on one.
- Answer: Was able to find a job by typing in a keyword.- Completed: Yes.
Find a job listing and share it using social media: With the search results still up,
Karla clicked on one. She clicked on the company Facebook icon which took Karla
to the company Facebook page. She looked around and then hit the back button
to go back to her search results. She then scrolled through the twitter feed that
is directly on the Kansas City site. Karla then clicked the link to view all jobs and
then hit the back button. Her phone vibrated and she picked it up, hit a button to
silence it, and took if off the desk. Then she clicked on a different job link than
the one before, but she was having trouble finding how to share the listing. She
decided that she would try to email it instead, and then she gave up on searching
anymore.
- Answer: Couldn’t find out how to share a job listing.- Completed: No.
Tester 2
Search for a job in any specific city using any keyword: Brenna hovered over the
“Business” tab before clicking on the “Jobs” link at the top right of the page. It
then took her to the page where she could search for jobs. She typed in
“Psychology jobs” and put “Sarasota, FL” for the location and was able to find
many job listings.
- Answer: Was able to find a job using a keyword.- Completed: Yes.
Find a job listing and share it using social media: With all of the listings from the
previous task still up on the screen, Brenna clicked on a listing. She noted that
“there are ads everywhere” on the screen. She noticed that the listing could be
emailed. She clicked the company Facebook link and then went back to the job
listing she had clicked. Brenna said that she “expected it to be next to the email,
for continuity”. When she scrolled through the listing she noticed how all of the
text was on the left side of the page, and thought that the text should really be
covering more of the site. When she couldn’t figure out how to share the listing
she said that “this looks like a terrible site”. This site is, however, not directly on
Kansascity.com. It is on careerbuilder.com but there is a header that links back to
Kansascity.com.
- Answer: Was not able to share a listing on social media.- Completed: No.
Recommendations to improve UX for this task, linked to Jakob Nielsen’s 10
Usability Heuristics: When viewing any article on KansasCity.com, there were
buttons before the article started that were for sharing to social media sites. To
stay consistent, the jobs site needs to have the same kind of buttons, even if this
is on careerbuilder.com and not kansascity.com. This aligns with the
“Consistency and Standards heuristic.
These are the icons on the job listings. There is nothing to indicate a share button for social media.
These are the buttons on kansascity.com. There are clear buttons for Facebook, Twitter, and Google+.
This is what I would suggest gets done to the buttons. A share button should be added, so that visitors who want to share a job link on a social media page they can.
Conclusions
Completion rate: Karla Everman had a completion rate of 50%. She completed 3
out of the 6 assignment tasks. Brenna McLeod had a 67% completion rate. She
completed 4 out of the 6 tasks.
Similarities and differences: Both of my testers completed the tasks in many
similar ways. They both used the search tool to help them find information on
the site. They both also took the time to scroll through the pages while trying to
find the right article or information. Both Karla and Brenna used the jobs link at
the top right corner of the site instead of going under the “Classifieds” tab. My
testers also went about completing the tasks in different ways. Brenna right off
the bat used the search tool, while Karla didn’t use it until after completing 3
tasks. Karla also spent more time reading through the articles and the other
content on the site than Brenna did. Brenna didn’t let the frustration of the tasks
get to her as much as Karla did. Once Karla got frustrated, she was ready to be
done, while Brenna tried harder to finish the task. Karla tended to get more
distracted with the site than Brenna did. One of the obvious reasons why they
went about completing the tasks differently is because they are different people.
Different people search the internet different ways. I also think that the amount
of distractions a person can take affects their ability to focus. Some people are
able to have focus when there are elements that could pose as a distraction. This
was clear with my testers. Karla was easily distracted with all of the articles and
content on the site, while Brenna has a little more focus. The threshold for
focusing on the task at hand and not getting distracted made itself known during
this test. It is important to remember that different people get distracted by
different things. So, making sure that a site doesn’t have a lot going on and has a
minimalist design (following the aesthetic and minimalist design heuristic) is a
good way to ensure that the website has good user experience.
Recommended