Tumblr Usability Test Report...Tumblr Usability Test Report Created by The Renegade Unicorns Kayla Boyett Candice Horde Walker Williams November 23, 2015 1 Table of Contents

Tumblr Usability Test Report

Created by The Renegade Unicorns

Kayla Boyett

Candice Horde

Walker Williams

November 23, 2015

1

Table of Contents Introduction…………………………………………………………………………………………………………………………………………..2 Executive Summary…………………………..…………………………………………………………………………………………………..2 Methodology…………………..…………..…………..…………..…………..…………..…………..…………..…………..……………….3

Sessions……………..…………..…………..…………..…………..…………..…………..……………………..…………..………3 Participants………………..…………..…………..…………..…………..…………..…………..….………..…………..………..4 Overview of Test Participants………………..…………..…………..…………..…………..……….…..…………..………5

Evaluation Tasks/Scenarios……………..…………..…………..…………..…………..…………..…………..…………….……..…..5 Results……………..…………..…………..…………..…………..…………..…………..…………..…………..…………..…..………..……6 Task Ratings…………………..…………..…………..…………..…………..…………..…………..…………..…………..…………..…….7 Overview of Results on Desktop Application for HP……………..…………..…………..……………………..……9 Overview of Results on Mobile Application for iPad………………..…………..…………..…………..………...10 Task Completion Success Rate………………..…………..…………..…………..…………..…………..…………..…….……..….10 Errors…………………..…………..…………..…………..…………..…………..…………..…………..…………..…………..…………….10 Overview of Errors in Desktop Application for HP……………..…………..…………..…………..…..………..…11 Overview of Errors in Mobile Application for iPad……………..…………..…………..…………..……….…..…12 Overall Metrics……………..…………..…………..…………..…………..…………..…………..…………..…………..…………..…..12

Post Task Questionnaire…….…………..…………..…………..…………..…………..…………..…………..…………...13 Comments……………..…………..…………..…………..…………..………..…………..…………..…………..…………..…14 Recommendations……………..…………..…………..…………..…………..…………..…………..………………..…..….15

Conclusion………………..…………..…………..…………..…………..…………..…………..…………..…………..…………..…..…..15 Appendix……………..…………..…………..…………..…………..…………..…………..…………..…………..…………….……..……16

2

Introduction

This document contains research that tests the usability of Tumblr, a social media site and app that is predominantly used by teenagers and young adults. The immediate goal of the research was to collect valid and reliable qualitative/quantitative data on the current Tumblr user experience while gathering data to improve the usability of both the browser website and iOS application for Tumblr. Our future goals are to utilize our results to improve the efficiency, functionality, and productivity of the site in order to create the best overall user experience possible. Tumblr was founded in 2007 by David Karp and has headquarters in New York City, New York. Tumblr’s parent foundation is Yahoo! Inc. The purpose of Tumblr is to connect individuals across cities, states, and countries by users blogging texts, video, photos, etc. and giving other users the opportunity to like or reblog these posts.

Executive Summary

All tests were completed in the Usability Research Lab supplied by Texas Tech University. Each test was conducted by Walker Williams, Candice Horde, or Kayla Boyett, and when necessary, supervised by Dr. Carter. Tests for the browser version of Tumblr were performed on a desktop computer located in the English and Philosophy Building’s Usability Research Lab. We used an iMac running Windows 7 Enterprise Operating System in order to complete record on the Morae Usability Testing Software. We used Google Chrome for the Tumblr desktop application. We also utilized the Mobile Eye Guide Tracking Software in tandem with the Tumblr mobile application for iOS 9.1 on the iPad Mini.

The Tumblr desktop application tests lasted approximately 45 minutes for each user, including the setup. Similarly, the Tumblr mobile application tests with eye-tracking lasted around the same amount of time, despite lacking half the tasks because the eye-trackers required more preparation.

The purpose of the testing was to find out if Tumblr veteran users can operate the site efficiently. Normally this wouldn’t be an issue that would require testing, but Tumblr updates their site so regularly, often times changing the layout and moving certain buttons and links to different spots. This can adversely affect the user base, especially since Tumblr help documentation is very slow to update along with site changes.

For our pool of test participants, we researched the main demographics of Tumblr and created general user profiles out of that data. Our main personas that we developed were an avid fan-fiction writer, a social justice warrior, and a hipster young adult that is focused on eccentric aesthetics. The majority of

3

Tumblr users are women and people of color. Results concluded that overall Tumblr users could easily post content fairly quickly. The main issues that users tested into were:

● Turning on and off anonymous questions in privacy settings ● Changing their theme to endless scrolling ● Picking a theme ● Seeing the prompt

This document contains task overview, in depth data regarding testing, results and analyses of collected quantitative and qualitative data, and recommendations. In the appendix section we will have the test plan, personas, pre and post-test questionnaires, and screen-shots of common errors that could be visually tracked.

Methodology

Sessions Test participants were recruited by the Renegade Unicorns, and they all knew at least one member of the team personally or had them in a class. The test participants included: Ana, Orlando, Candice, and Caleb. Once these individuals were recruited, the Renegade Unicorns informed them that they would be participating in a usability test concerning Tumblr’s user experience on both the desktop and the mobile application.

At the beginning of the test session participants were required to fill out a preliminary questionnaire to gauge their previous knowledge of Tumblr and various other social media websites. In each session the participants were led through a series of tasks administered by one of the test administrators in the Texas Tech Usability Lab. We completed the mobile and desktop sessions separately, and each lasted approximately half an hour, excluding preparation time. For the Morae testing, we had a sample folder for users to grab content from to provide external validity that would allow us to test the usability of Tumblr rather than the users’ ability to post original content.

4

After each task, the administrator asked the participant to rate the interface on a 5-point scale with measures ranging from Very Easy to Not Very Easy At All. Post-task scenario subjective measures included:

Ease of Task Ratings: ● 1 = Not Easy At All ● 2 = Not Very Easy ● 3 = Moderately Easy ● 4 = Pretty Easy ● 5 = Very Easy

After the last task was completed, the test administrator asked the participant to rate the website overall by using a 5-point scale (1 = Horribly Executed to 5 = Thoroughly Executed) based on Jakob Nielsen 10 Usability Heuristic heuristic completion. The results for the post questionnaire will be further explained in the Overall Metrics section.

*See Appendix for the raw data of the subjective and overall questionnaires.

Participants Each participant was taken from the available pool of students at Texas Tech University. All participants had to meet the preconditions of the Usability Test, which included:

● familiarity with computers and tablets

● Current Tumblr users

● Ages from 15 and 25

● Coinciding schedules with the time allotted for Usability

Out of four total participants two were male and two were female. Many steps were taken in order to maintain the vigor required of an effective Usability Test. Although we could not find any African-American students that met all of the necessary criterion, we made sure half of our participants were Hispanic in order to maintain the external validity. Another characteristic of our participants that support the external validity is the major role of most Tumblr users. The test participants are broken down into three purposes: post fan-fiction, spread awareness of social issues, or to simply show other users artistic, expressive photos, memes, and videos. Each participant falls under the three categories. Lastly, in order to fulfill internal validity, each participant qualifies as an expert user of Tumblr with at least three hours of usage a day.

5

Overview of Test Participants

Ana Candice Orlando Caleb

Age 21 21 20 19

Race/Ethnicity Hispanic Caucasian Hispanic Caucasian

Major Tumblr Role

FanfictionWriter

Artsy Media Poster

Social Justice Warrior

Artsy Media Poster

Usability Testing Participation Experience

No No No No

Usual Tasks on Tumblr

Reblog: - Videos - Memes - Funny Political Posts Post: - Fanfiction

Reblog: - “Dank” memes - Music - Videos

Reblog: - Equality Posts - Aesthetic Photos - Funny Videos - Few Memes Saves: - GIFs for later use

Reblog: - Photos - Memes

Hours on Tumblr Per Day

3 5 12 2

Evaluation Tasks/Scenarios

For our user testing we had users complete several small tasks that any user might complete on the Tumblr site. The tasks selected by Walker Williams are as follows: Desktop Application:

● Create a text post, and post a video, sound clip, and photo. ○ create a text post about their favorite ice cream ○ Post a video from the sample folder

○ Post a sound file from a url supplied in the sample folder ○ Post a photo from the sample folder

● Edit their photo to have a caption. ● Change the privacy settings for their “Ask” questions.

○ Turn “Allow anonymous questions” on/off ● Search for a specific user.

6

○ Search for the user that is shown in a screenshot photo supplied in the sample folder ● Ask the specific user found a question. ● Answer the question that appears in the inbox.

○ Use the GIF found in the test folder ● Adjust settings for for the homepage

○ Pick a theme with a high contrast ○ Set homepage to endless scrolling

Mobile Application:

● Create a text post, and post a video, sound clip, and photo. ○ create a text post about their favorite ice cream ○ Post a video from the sample folder

○ Post a sound file from a url supplied in the sample folder ○ Post a photo from the sample folder

● Edit their photo to have a caption. ● Search for a specific user.

○ Search for the user that is shown in a screenshot photo supplied in the sample folder ● Ask the specific user found a question. ● Answer the question that appears in the inbox.

○ Respond with text only

Results All of the following results are organized by the task in order to account for all disparities in the quantitative data. Since we did not want to test the user’s ability to find files, all extraneous materials were placed in a sample folder for internal validity. Problems are considered as “common” once 50% or more of the participants have the same issues. Ease of Task Ratings: Following each task completed, our participants rated task ease based on the following 5-point scale: 1 = Not Easy At All 2 = Not Very Easy 3 = Moderately Easy 4 = Pretty Easy 5 = Very Easy

7

Ease of Task User Ratings Ana Orlando Candice Caleb Average

Rating Text Post 4 4 3 5 4

Video Post

3 5 5 5 4.5

Sound Post

5 5 5 5 5

Photo Post

5 5 5 5 5

Edit Post 5 5 5 5 5

Change Ask Privacy

5 4 2 3 3.5

User Search

5 5 5 5 5

Ask 5 5 5 5 5 Answer 5 3 5 3 4.5

Pick a Theme

5 4 5 3 4.25

Enable Endless Scrolling

3 5 1 4 3.25

8

Overview of Results of Desktop Application on HP

Ana Orlando Candice Caleb Average Time on Task

Average Rating

Text Post 54 seconds 45 seconds 1 minute 50 seconds

36 seconds 1 minute 2 seconds

4

Video Post

1 minute 12 seconds

21 seconds 20 seconds 23 seconds 34 seconds 4.5

Sound Post

34 seconds 36 seconds 19 seconds 30 seconds 29 seconds 5

Photo Post


Edit Post 8 seconds 21 seconds 14 seconds 30 seconds 18 seconds 5 Change Ask Privacy

15 seconds 57 seconds 2 minutes 1 minute 15 seconds

1 minute 11 seconds

3.5

User Search


Ask 58 seconds 24 seconds 38 seconds 20 seconds 35 seconds 5 Answer 1 minute 15

seconds 1 minute 34 seconds

1 minute 24 seconds

1 minute 40 seconds

1 minute 51 seconds

4.5

Pick a Theme

1 minutes 21 seconds


44 seconds 2 minutes 40 seconds

1 minute 47 seconds

4.25

Enable Endless Scrolling


11 seconds Task was not completed

2 minutes 2 seconds

91 seconds 3.25

9

Overview of Results of Mobile Application on iPad Ana Orlando Candice Caleb Average Time

on Task Average Rating

Text Post 1 minute 30 second


Video Post


Photo Post


Edit Post 50 seconds 20 seconds 13 seconds 1 minute 13 seconds

39 seconds 4.5

User Search

58 seconds 33 seconds 17 seconds 1 minute 19 seconds

47 seconds 5

Ask 1 minute 33 seconds


Answer 38 seconds 27 seconds 47 seconds 44 seconds 39 seconds 5

Task Completion Success Rate

Almost all of our task had a 100% completion rate excluding Task 11: Enable Endless Scrolling which had a 75% success rate due to one participant not completing the task on the Morae desktop testing. Our tasks were generated to test the efficiency of the task and the speed in which it could be completed rather than if the task could actually be completed.

Errors

Desktop Application on HP ● The following tasks had errors that the users discovered: ● Task 1 - Author a text post about favorite ice cream. ● Task 2 - Post a video. ● Task 3 - Post a sound file. ● Task 7 - Turn on/off anonymous questions prompt. ● Task 10 - Answer a question from their inbox. ● Task 11 - Pick a theme for profile. ● Task 12 - Set profile to endless scrolling.

10

Overview of Errors in Desktop Application on HP

Error Average Time on

Task Average Rating

Appendix Reference

Task 1 – Author a text post about favorite ice cream

Seeing “+” to pull photo from computer

1 minute 2 seconds 4 Problem expressed in audio and long “Average Time on Task”

Task 2 – Post a video

Seeing the “This is my video” prompt

34 seconds 4.5 Error Screenshot 1

Task 3 – Post a sound file

Seeing the “I’m not a robot prompt”

29 seconds 5 Problem expressed in audio

Task 7 – Turn on/off anonymous questions prompt

Finding which tab “anonymous questions” was under

1 minute 11 seconds 3.5 Error Screenshot 2

Task 10 – Answer a question from their inbox

Finding the “inbox” symbol

1 minute 51 seconds 4.5 Problem expressed in audio and long “Average Time on Task”

Task 11 – Pick a theme for profile

Searching through themes and picking one

1 minute 47 seconds 4.25 Problem express in audio and long “Average Time on Task”

Task 12 – Set profile to endless scrolling

Finding what tab “endless scrolling” was under

91 seconds 3.25 Error Screenshot 3

Mobile Application on iPad

The following tasks had errors that the users discovered: ● Task 4 - Edit caption. ● Task 6 - Ask a question of the user they searched.

11

Overview of Errors in Mobile Application for iPad

Task Error Average Time on Task

Average Rating

Appendix Reference

Task 4 – Edit caption Finding the button to edit post

39 seconds 4.5 Problem expressed in audio

Task 6 – Ask a question of the user they searched

Could not find button that specifically said “Ask Question”

43 seconds 4.75 Problem expressed in audio

Overall Metrics

Jakob Neilsen’s 10 Usability Heuristics include:

Heuristic 1 - Visibility of System Status Heuristic 2 - Match Between System and the Real World Heuristic 3 - User Control and Freedom Heuristic 4 - Consistency and Standards Heuristic 5 - Error Prevention Heuristic 6 - Recognition Rather than Recall Heuristic 7 - Flexibility and Efficiency of Use Heuristic 8 - Aesthetic and Minimalist Design Heuristic 9 - Help Users Recognize, Diagnose, and Recover from Errors Heuristic 10 - Help and Documentation Tumblr Heuristic 5-Point Scale Used: 1 - Horribly executed. Does not meet Heuristic at all. 2 - Badly executed. Does not meet Heuristic. 3 - Somewhat executed. Partially meets the Heuristic. 4 - Almost executed. Roughly meets the Heuristic. 5 - Thoroughly executed. Completely meets the Heuristic.

12

Post-Task Overall Questionnaire

Ana Orlando Candice Caleb Average Rating Heuristic 1

4 4 4 4 4 - Roughly meets Heuristic

Heuristic 2

3 2 3 5 3.25 - Partially meets Heuristic

Heuristic 3


Heuristic 4

3 5 3 4 3.75 - Roughly meets Heuristic

Heuristic 5


Heuristic 6


Heuristic 7

3 5 4 3 3.75 - Roughly meets Heuristic

Heuristic 8


Heuristic 9

3 1 3 4 2.75 - Partially meets the Heuristic

Heuristic 10

3 5 3 5 4 - Roughly meets Heuristic

Comments: *Users omitted from this section chose not to comment on their rating of the Heuristic. Heuristic 1 - Visibility of System Status Orlando and Caleb feel that the system’s visibility and feedback information work fine. Heuristic 2 - Match Between System and the Real World

Orlando thinks Tumblr is unorganized and messy, and Candice attributes the lack of system and real world consistency to older developers who haven’t adapted to current concepts and language.

Heuristic 3 - User Control and Freedom While Orlando feels that the user’s have to play a guessing game to backtrack information, Caleb thinks the system is straight-forward with the available undo and redo.

13

Heuristic 4 - Consistency and Standards Orlando thinks that not the entire website is consistent, but terms are while Caleb thinks it is straightforward. Heuristic 5 - Error Prevention Orlando thinks that errors seldom show up, but when they do there is no way to report them and Caleb thinks that there are too little errors for this to be an issue. Heuristic 6 - Recognition Rather than Recall Ana says the system changes too much with updates. Orlando feels Tumblr doesn’t have clear instructions on how to use it, however, contrastingly, Caleb thinks it is straightforward. Heuristic 7 - Flexibility and Efficiency of Use No comments Heuristic 8 - Aesthetic and Minimalist Design Orlando thinks the website is minimalist in ways that aren’t helpful and that there is too much going on but Caleb likes the design. Heuristic 9 - Help Users Recognize, Diagnose, and Recover from Errors Orlando says that the system does not guide you in navigation at all. Heuristic 10 - Help and Documentation Orlando thinks that Tumblr needs simplified rules and regulations.

14

Recommendations Based on our results, our hypothesis of Tumblr causing issues by changing their layout was confirmed. Veteran users ran into usability issues while using common functions of Tumblr. 2 general fixes to all established usability problems are outlined below:

Change Justification

● Keep documentation properly updated.

● Make sure new settings/options that are added are easy to find. By placing them under common related tabs i.e. Privacy settings under the “Settings” tab and the “Endless Scrolling” button under the homepage main design tab

Users had issues finding some integral options. These were veteran users who should not have had issues under normal circumstances.

Conclusion We urge Tumblr to listen to their community and any further changes they make to be strictly aesthetic. Since Tumblr has a very large presence in social media and depends on user satisfaction exposure and expansion, we hope they take these recommendations into consideration in order to increase user satisfaction. The majority of our test group agree that some functions on Tumblr that aren’t immediately available on the homepage are difficult to find. While posting and editing are relatively easy to use as they are available with clearly marked icons on the main page, tasks such as changing privacy settings or enabling endless scrolling for a them were somewhat more difficult. Implementation of recommendations should be very simple. Tumblr is notorious for moving settings or categorizing items differently in successive updates. They would have much greater user satisfaction if they were to just pick an organization option and maintain that design concept.

15

Appendix

1. Usability Test Plan

Tumblr: Quality User Experience Project Version 1.1 – October 20, 2015 Table of Contents Background…………………………………………………………………………………………………………….........2 Methodology………………………………………………………………………………………………………………...2 Participants………………………………………………………………………………………..………………………....2 User Profile………………………………………………………………………………..………………………………....3 Training…………………………………………………………………………..................................................3 Procedure…………………………………………………………………………………………………………………......3 Roles…………………………………………………………………………………………………………………………......5 Trainer/Facilitator……………………….………………………………………………………………..……………....5 Data Loggers/Test Observers……………………………………………………….………………………....…....5 Ethics……………………………………………………………………………………………………………………………..5 Usability Tasks………………………………………………………………………………………………………………..6 Usability Metrics/Goals………………………………………………………………………………………………....7 Allotted Completion Time……………………………………………………………..……………………………....7 Completion Goals……………………………………………………………………………………………………….….7 Critical Errors/Non-critical Errors………………………………………………….………………………....…...7 Subjective Evaluations/Measures.......................................................................................7 Problem Severity………………………………………………………………………….………………………………..8 Reporting Results………………………………………………………………………………………....……………….8 Appendix……………………………………………………………………………………...………………………………10 Figures……………………………………………………………………………………………………………………….…10 Preliminary Questionnaire……………………………………………................……………………………….11 Post-session Questionnaire…………………………………………………………………………………..……..12

16

Background

This document constructs a research plan that will test the usability of Tumblr, a social media site and app that is predominantly used by teenagers and young adults. The immediate goal of our research is to collect valid and reliable qualitative/quantitative data on the current Tumblr user experience. With this data our group will be able to identify possible usability issues that negatively impact the average user’s experience. Our future goals are to utilize our results to improve the efficiency, functionality, and productivity of the site in order to create the best overall user experience possible.

As stated above, Tumblr is a social media site that also has an app, which means it can be used by anyone with an electronic device. Tumblr was founded in 2007 by David Karp and has headquarters in New York City, New York. Tumblr’s parent foundation is Yahoo! Inc. This became official on June 20, 2013. Currently Tumblr hosts over 256 million blogs as of the beginning of October of this year. The purpose of Tumblr is to connect individuals across cities, states, and countries by users blogging texts, video, photos, etc. and giving other users the opportunity to like or reblog these posts. All tests will be completed in the usability research lab supplied by Texas Tech University and supervised by Dr. Carter or other lab assistants. During the testing procedure the Morae Usability Testing Software will be utilized to log data.

Methodology Participants will be led into the Texas Tech University Usability Lab by the Trainer/Facilitator and sat down at the desktop with the Windows 7 Enterprise Operating System. The browser that the participants will use will be the

Participants The demographics in figure 1 (go to appendix) show that most of the users are teenagers, ages 13 to 18, and young adults, ages 19-15. Figure 2 (go to appendix) goes on to further characterize Tumblr users as majorly African-American and Hispanic, outweighing the Caucasian users 15 to 9. From past user experience and the frequency of particular posts broadcasted by the site, we have narrowed down the teenage/young-adult Tumblr purposes: post fan-fiction, spread awareness of social issues, or to simply show other users artistic, expressive photos, memes, and videos.

Other than asking friends and/or family for participants, we plan to post an ad on TechAnnounce outlining the general idea and objectives of our test plan. We will also include all the necessary qualifications for the research in the ad, and subsequently choose from the people who respond for the ones who are most qualified. We want to optimally use 12 participants in total. 2 teenagers and 2

17

young adults for each possible purpose listed at the end of the above paragraph. At least 7 or 8 of the chosen participants will be either Hispanic or black so that we remain true to the current demographics who are currently using the social media site.

User Profile All of the chosen participants must:

● be very familiar with computers and tablets

● be currently interested in or already using Tumblr

● be between the ages of 15 and 25

● be able to complete the testing procedure within the time frame the Usability lab will be

open

Participant responsibilities include:

● Attempt to complete ALL task scenarios given

● Provide answers to ALL usability questions i.e. preliminary/post-session questionnaires,

and task completion questions

● Speak all thought processes aloud during the usability test

● Remain honest and cooperative throughout the testing process

Training The participants will receive an overview of the usability test procedure, and explain all necessary information to use the equipment and software. We will also briefly explain how to use the desktop-browser and the mobile platform versions of Tumblr if the participants don’t already have that knowledge.

Procedure Participants will take part in the usability test at the Texas Tech Usability Research Lab in the English and Philosophy Building. A desktop with the Windows 7 Enterprise OS with the Web site/Web application and supporting software will be used in a typical office environment. The usability testing room will only have the current participant and the participant’s interaction with the Web site/Web application will be monitored by the facilitator seated in the same office. Note takers and data logger(s) will monitor the sessions in observation room, connected by video camera feed using the Morae usability testing software.. Both the screen feed and the actual test subject will be filmed. Test observers will be able to monitor both feeds as well as also entering notes throughout using a dual screen setup.

18

Participants will take part in the usability test via remote screen-sharing technology. The participant will be seated at their workstation in their work environment. Verbal communication will be supported via microphones linked between the test observing station and the testing station.

The facilitator will brief the participants on the Web site/Web application and instruct the participant that they are evaluating the application, rather than the facilitator evaluating the participant. Participants will sign an informed consent that acknowledges: the participation is voluntary, that participation can cease at any time, and that the session will be videotaped but their privacy of identification will be safeguarded. The facilitator will ask the participants if they have any questions or thoughts.

Participants will complete a pretest demographic and background information questionnaire as well as a pretest questionnaire regarding the usability. The facilitator will explain that the amount of time taken to complete the test task will be measured and that exploratory behavior outside the task flow should not occur until after task completion. At the start of each task, the participant will be asked to read aloud the task description from the printed copy and begin the task. Time-on-task measurement begins when the participant starts the task.

The facilitator will instruct the participant to ‘think aloud’ so that a verbal record exists of their interaction with the Web site/Web application. The test observers will observe and enter user behavior, user comments, and system actions in the data logging application. Tasks involving uploading certain files will be taken from from a sample folder which will be provided to test participants via USB flash drive. The facilitator will clarify which file needs to be selected for each task.

After each task, the participant will complete the post-task questionnaire and elaborate on the task session with the facilitator. After all task scenarios are attempted, the participant will complete the post-test satisfaction questionnaire. The post-test questionnaire will use Jakob Nielsen’s 10 Heuristics to gauge the usability of the website regarding the designated tasks. Feedback will be gathered using the Morae software for the desktop testing and our data loggers will manually gather data during the mobile app testing.

19

Participants will take part in the usability test via remote screen-sharing technology. The participant will be seated at their workstation in their work environment. Verbal communication will be supported via telephone.

Roles The roles involved in a usability test are as follows. The individuals within our test group are ascertaining the following roles below:

● Facilitator/Trainer: Kayla Boyett

● Test Observers/Data Loggers: Walker Williams, and Candice Horde

Listed below are the defined duties for each group member.

Trainer/Facilitator ● Provide training overview prior to usability testing.

● Provides overview of study to participants.

● Defines usability and purpose of usability testing to participants.

● Assists in conduct of participant and observer debriefing sessions.

● Responds to participant's requests for assistance.

Data Loggers/Test Observers ● Silent observer at the computer

● Identifies problems, concerns, coding bugs, and procedural errors

● Makes commentary notes on the user’s progress and completion as the test proceeds

Ethics Each member of our team that will take part in the usability tests are required to abide by the following ethical rules:

● All data and results must be delivered as they are initially recorded. No alterations to the data

of any kind will be permitted.

● Each participant has to sign a release form allowing us to use the data in any way we see fit. In

this form they will be releasing all rights to the information collected.

20

Usability Tasks The usability issues found during our Heuristic Evaluation include:

● To determine design inconsistencies and usability problem areas within the user interface and

content areas. Potential sources of error may include:

● Navigation errors – failure to locate functions, excessive keystrokes to complete a function,

failure to follow recommended screen flow.

The tasks listed below are a brief overview of what will be required to be completed by the test participants. Tasks were chosen to test the overall design of Tumblr. The tasks that will be completed by the participants include:

● Posting pictures, videos, music, and text posts

○ User will post a picture with a caption

○ User will post a video with a caption

○ User will post a song

○ User will author a text post

● Editing posts

○ Edit a previous post created prior to the test

● Changing privacy settings for users to ask questions.

○ Locate the privacy settings

○ Change the settings to “Allow Anonymous Asks”

● Asking/Answering questions

○ Ask an anonymous question

○ Answer a question with an animated .gif

● Searching for specific users

○ Search through Tumblr

○ Go to a page directly with url

● Designing participants homepage

○ Change profile picture

○ change theme

○ Add a custom link

21

Each tasks is designed to take approximately 1-10 minutes depending on the various tasks. The overall test should be around 10-15 minutes.

Usability Metrics and Goals This section outlines how we will use Qualitative and Quantitative data to compare expected performance goals against actual participant performance. It set parameters for completion success, allotted completion time, critical/non-critical errors and subjective evaluations. The goals for our usability test is to define how user-friendly Tumblr is to veteran users. Tumblr is often updated with a new layout. It is often criticized by users for making small changes that affect the user’s flow. The post-task questionnaire has been consolidated into asking the users confidence level and efficiency following every task they complete. *All tasks will be will be rated during the preliminary and post-session questionnaires. Subject evaluations below reflect the kinds questions that will be asked during the test after the completion or non-completion of each task.

Tasks Allotted Completion Time

Completion Goals

Critical (C)/ Non-critical (NC) Errors

Subjective Evaluations/ Measures

Posting 5 minutes for each sub-task

Posts are submitted and appear on timeline

Confusion of what file formats to upload (NC) Posting the wrong content (C)

Confidence level/Efficiency

Editing posts 3 minutes Changes to respective post are

N/A Efficiency

22

shown on timeline

Privacy settings

6 minutes Switch button is set to desired option

Searching under the wrong tabs (NC) Changing the wrong setting (C)

Efficiency

Asking & Answering Questions

3 minutes for each sub-task

Answer or question is posted

N/A Confidence Level/Efficiency

User Search 3 minutes The indicated user profile is found

Finding the wrong user (C ) Search using wrong criteria (NC)

Efficiency

Homepage Design

10 minutes All aspects of homepage are personalized

N/A Confidence Level

Problem Severity Due to the nature of the test, issues shouldn’t be encountered that would permanently affect the participant in a negative way. However, we have devised a ranking system to classify errors that might slow the participant from completing the task.

● Severe Issue - Over 50% of participants were not able to complete task/s due to issue.

● Moderate Issue - Over 30% of participants were not able to complete task/s due to issue or a

moderate amount of time was needed to complete task.

23

● Trivial Issue - Over 10% of participants were not able to complete task/s due to issue or issue

was a minor inconvenience that didn’t vastly affect task/s.

Reporting Results The Usability Test Report will be provided at the conclusion of the usability test. The report will be comprised of a research description, methods section, and all data found from our usability tests, as well as supplementary data that describes demographics of Tumblr to coincide with our test group. At the end of the report we will include the analyzation of data and recommendation to fix any issues found. The report will be due November 2nd.

Appendix Figures

Figure 1. Demographics from Business Insider

24

Figure 2. Demographics from Pew Research Center

25

Preliminary Questionnaire

1. Have you ever participated in usability research prior to this occasion? If so, when and where? 2. How often do you use Tumblr daily (hours)? 3. What tasks do you engage in on Tumblr i.e. post videos, memes, music etc. or chat with friends? Are any of these tasks primarily done on Tumblr? 4. What are the top 3 things you struggle with on Tumblr right now? Explain. 5. Do you use any other social networks? If so, which one do you use more often (including Tumblr) and why?

26

Post-session Questionnaire This list of Heuristics or principles was taken from Jakob Nielsen’s 10 Heuristics, and will be used to rate the overall user experience of Tumblr during this session. Rate each Heuristic with the scale provided below. If you have any further comments please add them in the space provided under each principle. Scale: 1-5 1…. Horribly executed. Does not meet Heuristic at all. 2… Badly executed. Does not meet Heuristic. 3… Somewhat executed. Partially meets the Heuristic. 4… Almost executed. Roughly meets the Heuristic. 5… Thoroughly executed. Completely meets the Heuristic. Visibility of system status

The system should always keep users informed about what is going on, through appropriate feedback

within reasonable time.

Rate:

Comments:

Match between system and the real world

The system should speak the users' language, with words, phrases and concepts familiar to the user,

rather than system-oriented terms. Follow real-world conventions, making information appear in a

natural and logical order.

Rate:

Comments:

27

User control and freedom

Users often choose system functions by mistake and will need a clearly marked "emergency exit" to

leave the unwanted state without having to go through an extended dialogue. Support undo and redo.

Rate:

Comments:

Consistency and standards

Users should not have to wonder whether different words, situations, or actions mean the same thing.

Follow platform conventions.

Rate:

Comments:

Error prevention

Even better than good error messages is a careful design which prevents a problem from occurring in

the first place. Either eliminate error-prone conditions or check for them and present users with a

confirmation option before they commit to the action.

Rate:

Comments:

28

Recognition rather than recall

Minimize the user's memory load by making objects, actions, and options visible. The user should not

have to remember information from one part of the dialogue to another. Instructions for use of the

system should be visible or easily retrievable whenever appropriate.

Rate:

Comments:

Flexibility and efficiency of use

Accelerators -- unseen by the novice user -- may often speed up the interaction for the expert user

such that the system can cater to both inexperienced and experienced users. Allow users to tailor

frequent actions.

Rate:

Comments:

Aesthetic and minimalist design

Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of

information in a dialogue competes with the relevant units of information and diminishes their relative

visibility.

Rate:

Comments:

29

Help users recognize, diagnose, and recover from errors

Error messages should be expressed in plain language (no codes), precisely indicate the problem, and

constructively suggest a solution.

Rate:

Comments:

Help and documentation

Even though it is better if the system can be used without documentation, it may be necessary to

provide help and documentation. Any such information should be easy to search, focused on the user's

task, list concrete steps to be carried out, and not be too large.

Rate:

Comments:

30

2. Personas

31

Figure 2.1: Writing/Opinion Post Enthusiast Persona

32

Figure 2.2: Aesthetics Enthusiast Persona

PersonaTemplate

PersonatypeTumblrHipsterPhotographyandMusicGuru.

NameJacksonBeauregardD’ArtagnanMiller

Age16yearsold

LocationLubbock,Texas

TechnicalcomfortComputersciencemajor,betterthanaverageskill.

IdealJobTitleStartingandowninghisownbusinessinthetechworld.

Backstory

Tellusabitabouttheirlives • OriginallyfromSantaRosa,Californiafromalargefamily,parentshavebeentogetherfortwentyyears.• HomeschooledinCalifornia.• Earlyhighschoolgraduate,beganattendingTexasTechUniversityat16yearsold,currentcollegesophomore.• Currentlyhasa4.0GPAwhileinComputerSciencemajordegreeprogramintheWhitacreCollegeofEngineering.

Motivations

Whatconcernsdotheyhave?Whydotheyneedthiswebsite/service?Howhavetheyfoundorheardaboutthewebsite?

• FoundoutaboutTumblruponcomingtocollegethroughcollegeroommate.• EnjoysbloggingonTumblr,primarilyonhisSamsungGalaxyS6.Spendsabout3hoursperdayonhisphone.• AlsoblogsonhisMacBookPro,andhisprimarilyusinghisMacBookProfor6to8hoursperday.• Blogsandrepostsprimarily“hipster”photographyandundergroundindiebands,alongwithoccasionalTVshows.

Frustrations What’sstoppingthemfromchoosingtheservice/websiteorannoyingthem?

• Constantchangingpositionsofbuttonsareannoyingandinconvenientforblogging.

• InconsistencybetweenthemobileapplicationandthebrowserversionofTumblrisalsoinconvenient.

• CannotpostGIFfilesfrommobileapplicationonphone.

• Nosearchbarwithinblogpostsmakesfindingcertainmaterialdifficult.

Theiridealexperience Theirstoryincludingfeaturesandcontentwhichwillhelpthemhavea

greatexperience

• Consistentmobileapplicationforeaseofusewhenpostingisideal.

• WantstobeabletouseTumblrasquicklyandefficientlyaspossible.

• PostingGIFfilesfromthemobileapplicationisdesiredverymuch.

• Wantstobeabletofindbuttonsinthesameplaceaftereveryupdate.

Quote

“Wearethebiggestkindofsmall.

”

33

Figure 2.3: Tumblr Social Justice Warrior Persona

Persona: Tumblr Social Justice Warrior Name: Rebekka White Job: Student/Unemployed Current Year: Freshman Goals: Not really any. She’s still figuring herself out. Environment: University of Arkansas, lives on campus. Demographics: White, middle class, has two brothers, mother and father, no romantic

relationships

Biography Bekka, as her friends call her, is a first year student at the University of Arkansas. She grew up in Fayetteville and decided to go to UA because she wasn’t ready to leave home. Bekka grew up with average grades and was a member of the marching band. She was introduced to Tumblr when she was a sophomore in high school. Blogging was all the rage at her high school, so she had to look into this “Tumblr” thing. She was all about cat memes and twerking videos until one day when a post about white privilege came up on her dash. She was then thrust into the real world where inequality is rampant. From that day on, she became a proponent of equal rights, whether it be about gender identity, race, sexuality, or just feminism in general. With this new concept of empathy, Bekka took her newfound attitude into her personal life. Living in Arkansas, she got into a lot of arguments over minority groups. She would always be the first one to point out a person’s micro-aggressions or major aggressions. When she graduated, she realized she didn’t really have any plans for her future. In a panic, she applied for University of Arkansas, which let her in because all kids from her school district had guaranteed acceptance. Despite how liberal the college campus is, Bekka is still hard at work finding flaws and how to be more politically correct. While she doesn’t join any activists groups because she “doesn’t have time,” she will attack anyone on Tumblr that isn’t as accepting as her.

34

3. Screenshots Screenshot 1

Figure 3.1: This image outlines the test participant missing the security prompt under the video.

35

Screenshot 2

36

Figures 3.2 and 3.3: The first image is the tab all users went to in order to find the “Privacy Settings” while the second photo is an indication how where the users cohesively searched next – the sub-menu. Screenshot 3

Figure 3.4: This image shows the participant looking for the “endless scrolling” option. Like other participants in our pool, she initially searched in the same page you edit the theme of the homepage.

Documents

Tumblr Usability Test Report...Tumblr Usability Test Report Created by The Renegade Unicorns Kayla Boyett Candice Horde Walker Williams November 23, 2015 1 Table of Contents