Maggie Shatt, Chelsey Glasson, John Ford, Mark Moledor & Suneetha Sivasankaran

HCDE 5183/10/20112011

StumbleUpon.comUser Profile Redesign Project

This report provides a summary of the results of three user-requirements research activities and, in response to the findings, presents a redesign of the StumbleUpon.com User Profile page.

ContentsOverview.....................................................................................................................................................2

Activity 1: Contextual, Semi-Structured Interviews.....................................................................................3

Background..............................................................................................................................................3

Method....................................................................................................................................................3

Results.....................................................................................................................................................4

Next Steps...............................................................................................................................................5

Activity 2: Online Survey..............................................................................................................................6

Background..............................................................................................................................................6

Method....................................................................................................................................................6

Results.....................................................................................................................................................7

Next Steps...............................................................................................................................................7

Activity 3: Focus Group................................................................................................................................7

Background..............................................................................................................................................7

Method....................................................................................................................................................7

Results.....................................................................................................................................................8

Next Steps.............................................................................................................................................12

Redesign Proposal.....................................................................................................................................13

Recommendations.................................................................................................................................13

Redesign....................................................................................................................................................13

Conclusion.................................................................................................................................................14

Appendices................................................................................................................................................15

Appendix A: Interview...........................................................................................................................15

Appendix B: Interview Data...................................................................................................................17

Appendix C: Online Survey Question Set and Data................................................................................19

Appendix D: Focus Group Participant Sketches.....................................................................................21

Appendix E: Focus Group Questions......................................................................................................22

1

Overview StumbleUpon is a discovery engine that uses peer-sourcing and social networking capabilities to provide its users with the ability to easily discover, share, and rate web pages, photos, and videos that are in alignment with their tastes and interests. Also, through its social networking features, StumbleUpon encourages its users to find likeminded people to connect and share information with. This ability is one of the things that sets StumbleUpon apart from typical social networking sites and discovery engines and it is pivotal to StumbleUpon’s long-term survival.

A preliminary heuristic evaluation of the social networking aspect of StumbleUpon revealed that the current design of a user’s profile page is confusing and difficult to navigate. Accordingly, through this redesign project, our goal is to improve the profile page—the centerpiece of the StumbleUpon social networking experience. The profile page redesign ultimately does an improved job of emphasizing the most popular aspects of StumbleUpon’s social networking features, as indicated by current users. Additionally, the profile page redesign incorporates new elements that make it easier for users to find and connect with other likeminded Stumblers. These improvements, if implemented, will ultimately increase users’ social connections and, therefore, return visits and on-going use of StumbleUpon.

Three user requirements research activities were conducted to inform the profile page redesign. The goal of these activities was to determine how users interact with StumbleUpon’s social networking features, to gauge user satisfaction with the profile page’s current design and functionality, and to brainstorm areas for improvement.

The first research activity was contextual, semi-structured interviews, which provided a rich, in-depth understanding of how users interact with StumbleUpon’s social networking features as well as what they like and dislike about them. The data collected from the interviews was used to design our second research method, an online survey, which validated findings from the semi-structured interview with a larger audience. Together, the data from the interviews and survey informed the creation of three profile page prototypes which were tested in a focus group. Feedback from the focus group, combined with results from the interviews and survey, drove the development of our final user profile page redesign.

The profile page redesign incorporates most of the following recommendations which arose from the three research activities:

Make terminology less confusing Include only key information (as determined by users) in profile summary Cluster similar information Make it easier to determine what information we both like (i.e., weighted representation) Integrate a dynamic, interactive experience Add private messaging Incorporate recent user activity (e.g., this site was ranked one hour ago) Add a comments/reviews feed

2

Activity 1: Contextual, Semi-Structured Interviews

BackgroundThe first research activity conducted consisted of contextual, semi-structured interviews. The goal of the interviews was to gain a general understanding of how users interact with StumbleUpon’s social networking features as well what they implicitly and explicitly like and dislike about the experience.

Method

Participants

Six participants, consisting of personal and professional acquaintances, participated in the interviews and were recruited via e-mail and phone. Their ages ranged from 18 to 44 years, and one was female and five were male. Participants did not receive compensation for participating in the interviews and a screener was used to ensure that they met the following criteria: like surfing the internet, using social networking sites, and having some experience using StumbleUpon as a registered user. As with all three research activities, these criteria were important for the interviews because we wanted to be certain that participants had the required skills and experience to complete our tasks within the allotted timeframe and engage in meaningful, reflective conversation. This would be difficult, if not impossible, with anyone not meeting the criteria.

Procedure

The 45-minute interviews were conducted in 5 parts: 1) introduction, 2) warm-up, 3) tasks, 4) post-task questions, and 5) wrap-up. Throughout the interviews, a Master/Apprentice relationship was maintained in order to encourage candid feedback from participants while preventing the halo effect that is common to user-centered design research methods. Details of each part of the interviews are provided below.

Part 1: Introduction During the introduction phase of the interview, participants were provided with a basic overview of the study (i.e., goal of the study, what to expect from the interviews). They were also asked to think aloud during the interview and told that we were interested in their candid feedback. Finally, they were reassured that they were not being tested and informed that we were not involved in the design of StumbleUpon and will not take offense to negative comments.

Part 2: Warm-UpDuring the warm-up phase of the interview, participants were asked some basic questions about their background and preferences in order to get them comfortable with talking to us and to give us insight into their background and perspective.

Part 3: TasksIn this phase, participants were asked to complete six predetermined tasks. The tasks all concerned key steps involved in utilizing the social networking aspects of StumbleUpon and were informed by the

3

aforementioned preliminary heuristic evaluation. Where necessary, interviewers asked follow-up questions in response to participants’ experience while completing the tasks.

Part 4: Post-Task QuestionsAfter participants completed the six tasks, they were asked a series of questions that did not pertain to any one specific task but were nevertheless important (e.g., “How is “Meet a Stumbler” similar to or different from the experience of social networking sites you use frequently”). These questions provided participants with an opportunity to provide feedback after having gone through all of the tasks. Please refer to Appendix A for a complete list of the tasks and questions asked during and after the tasks.

Part 5: Wrap-upFinally, participants were thanked for their time and asked for additional feedback and comments before concluding the interview.

Results

AnalysisThe data collected from all six interviews was sorted using an affinity diagram. Feedback was summarized onto over 100 sticky notes and posted onto a white board. Through discussing and organizing the notes, main groups and sub-groups for the feedback emerged. These are reflected in Figure 1.

4

FindingsThe three main categories that emerged from the affinity diagram were: 1) site design and features, 2) personal connections, and 3) ideas/suggestions. These three categories were primarily composed of feedback concerning confusing terminology, site layout and design, inadequate profile information, and privacy (Figure 1).

Among other things, the feedback from the interviews revealed elements of the StumbleUpon social networking experience that are particularly unsatisfying. For example, several participants said the profile page does not provide the information necessary to decide whether or not to connect with another Stumbler. Additionally, participants shared that they would like additional ways to communicate with fellow Stumblers. Finally, several participants shared feedback concerning who they would or would not like to connect with on StumbleUpon and how they would prefer to search for those connections. Please see Appendix B for a detailed list of the findings from the interviews.

Next StepsThe data collected from the user interviews was used to drive the design of subsequent user-requirements research activities: online survey and focus group. For the survey, feedback from the interviews determined what questions to ask in addition to what options to present in response to those

5

Figure 1

questions. For the focus group, findings from the interviews influenced several design decisions in the three prototypes presented to focus group participants.

Activity 2: Online Survey

Background With the information gathered from the semi-structured interviews, an eight-question online survey was created with the primary goal of validating and prioritizing user preferences revealed in the interviews.

Method

Participants As with the interviews, the target user for the survey was someone who surfs the internet, uses social networking sites, and has experience with social bookmarking sites. We reached out to target users by posting the survey on social networking sites with the disclaimer that it was intended for those who use social bookmarking sites. Thirty-six participants ultimately took the survey. Twenty-three of the 36 participants indicated they are registered StumbleUpon users.

Procedure Findings from the interviews were used to direct the creation of the survey questions. For example, in response to the feedback that the profile page has inadequate information to assess whether or not to connect with a fellow Stumbler, we asked survey participants what information they would want to see. Specifically, we asked them to choose from a list that included existing options available as well as options explicitly and implicitly suggested by interview participants. Among other questions, survey participants were also asked what activities they most frequently engage in when utilizing social networking capabilities on social bookmarking sites. Appendix C provides a list of the survey questions as well as an overview of the responses collected.

Since it was important to reach a large number of survey participants, a web-based survey that could be quickly distributed via several channels was used. The survey was distributed via email, Facebook, LinkedIn, and community forums—locations in which we knew the population consisted of those who matched the abovementioned profile. With a brief message explaining our study, users of social bookmarking sites were invited to take the survey. The survey was made available for five days.

Results

AnalysisThe University of Washington’s CatalystWebQ online survey tool was used to design, collect and analyze the survey data.

FindingsAnalysis of the survey data revealed the following key insights:

6

In order of importance, people prefer to share their profile name, interests and age on their profile page.

In order of importance, people prefer to see someone's interests, favorites, reviews, and comments when wanting to connect with someone.

People more frequently connect with friends and people with similar interests than with friends of friends, family members, people they don't know, and co-workers. Most do not connect with family members and co-workers.

Of the most common activities, people most frequently read content shared/posted by others. They also exchange private messages and suggestions often.

People don’t have a preference on whether their online status is visible to others. The preferred ways of searching for connections include: search by name (16 respondents),

search my contact list from Facebook, Gmail or another site (12 respondents), and search by email address (10 respondents).

Next StepsThe survey data confirmed the most important elements on a StumbleUpon user profile page. These findings served as the basis for our prototype designs tested in the focus group.

Activity 3: Focus Group

BackgroundThe final user-requirements research activity was a focus group. Three redesign prototypes of the StumbleUpon user profile page were presented and participants reacted to them through discussion, survey questions, and a sketching exercise.

The redesign prototypes differed in layout and appearance but catered to the preferences shared by survey and interview participants and emphasized elements of the user profile page participants said were most important to them. For example, we took into account in all of the designs that, aside from friends, most StumbleUpon users prefer to connect with people with similar interests. Prototype A offered a cover-flow interface, Prototype B offered a tag cloud interface, and Prototype C offered three column interface.

Method

ParticipantsSix focus group participants were recruited and they all met the user profile in that they were internet users, used social networking sites, and were registered users of StumbleUpon. They were recruited through personal and professional connections and were incentivized with food.

ProcedureThe focus group was held on Sunday, February 13th, in an office conference room in downtown Seattle. During the 90 minute session, participants were shown each prototype individually; starting with

7

prototype A and ending with prototype C. The prototypes were projected onto a screen for viewing and participants were also given printed copies to refer to. Immediately after being shown a prototype, participants were asked a series of open-ended questions (see Appendix E) as a group and were then asked to individually rate it on a 7-point scale. Finally, after reviewing all three of the prototypes and a wireframe of the current StumbleUpon profile page, participants were asked to individually sketch on paper or white board what their ideal profile page would look like.

Results

AnalysisAside from the moderator, each researcher took notes during the focus group. Immediately following the focus group, the notes and participant sketches were reviewed and analyzed.

FindingsBelow is a summary of the key findings from each prototype, the results of the Likert-scale questions, and an analysis of the participant sketches.

Prototype APrototype A (Figure 2) offers dynamic interaction by implementing a cover-flow style interface. Additionally, the prototype expands upon StumbleUpon’s usage of the “Badge” moniker to award “Interest Icons” as a means of providing incentive for interacting with other Stumblers. The prototype

8

Figure 2

incorporated feedback collected during the interviews in providing an easier way to compare interests by providing more information concerning a user’s favorite sites, videos and photos. The cover-flow style interface was inspired by a cursory review of popular websites such as iTunes.

Key Findings Participants responded favorably to the cover-flow design because of the increased size of the

website screenshots and its interactive nature. The awards concept was received favorably as a means of establishing a Stumbler’s credibility

and promoting interaction among Stumblers. The comment feed received positive feedback as a means of establishing dialogue over sites of

mutual interest. Participants also appreciated that the prototype eliminated the number of clicks required to

access certain information.

In summary, not only did participants feel it offered a cleaner, easier-to-use interface than the current design, but they also felt Prototype A offered a more pleasing, engaging experience.

Quotes “I like this because it’s like flipping through a magazine. The current version feels like a book in

which you have a lot of content you are forced to read to figure out what you actually want.” “Like Xbox, people would compete to earn the badges.” “There is more information on one page, so there is no need to click several times to get what

you want.”

9

Prototype BPrototype B (Figure 3) offers a weighted visual representation of a fellow Stumbler’s favorite websites (similar to a tag cloud experience but using images instead of words). In other words, the more a user’s favorites were in alignment with their interests, the larger the thumbnails of their favorites. Also, profile details were in the right navigation pane and a profile summary can been seen when hovering over the profile picture, centered within the thumbnail cloud. As with Prototype A, Prototype B was primarily inspired by feedback collected during the interviews concerning providing an improved way of comparing similar interests. Prototype B also incorporated feedback from the survey pertaining to what information should be included in the profile summary.

Key Findings Participants liked the idea of having the option to view a weighted representation of fellow

Stumbler’s favorite websites in relation to their interests, but didn’t want this as a default view. They felt the screen looked too cluttered and that it might be difficult to view this content on

certain devices. Several said the thumbnail cloud was too unpredictable and might be difficult to navigate.

In summary, of all the prototypes, participants shared the highest number of reservations and negative comments to Prototype B. This seemed to be because the prototype did not to fit their mental model for what a profile page should look like. Interestingly, the prototype inspired a lot of creative thinking

10

Figure 3

among participants, resulting in several helpful suggestions concerning terminology and potential features.

Quotes “I like this prototype but wouldn’t be able to get my mom to use it.” “I wonder if this page could get cluttered. Lack of consistency could get confusing.”

Prototype CPrototype C (Figure 4) offers an incremental redesign of the current site in which a user’s profile information is arranged according to the level of importance that each feature received in the user survey. A profile summary was in the left navigation allowing higher page placement of the user’s favorites. Additionally, the search field and Stumble buttons were placed directly above the user’s favorites.

Key Findings Participants liked the clean look and felt that the site was well organized and user-friendly. One participant did not like the Following section in the right column, as she felt the information

provided in the section did not add to her understanding about the user. Some participants felt the right and left-hand columns should be combined. One participant’s

felt this information should be on the right side of a profile in order to deviate from standards of other social networking sites.

11

Figure 4

In summary, participants appreciated the simplistic design of Prototype C but felt it didn’t do a good enough job of chunking related information.

Quotes “This flows better—current site is more broken up.”

Participant Sketches With one exception, the participant sketches were in alignment with feedback provided during the focus group discussion. Although participants expressed interest in the Badge concept, only 1 included badges in his sketch. Below are the key insights from the sketches:

5 of the 7 sketches integrated tabs 2 included an area for comments All clustered similar information together (e.g., profile information, discovered content,

comments and reviews, etc.) 3 provided profile information in a right-hand navigation pane, 3 had the profile information in a

left-hand navigation pane, and 1 had profile information in a top navigation pane 1 integrated interest badges 3 integrated a cover-flow like experience 3 created a space to explicitly display one’s interests 2 included a button on the profile page for findings connections

See Appendix D for images of all of the participant sketches.

Rating Scale Question ResultsOn a scale of 1 to 7, with 1 being “didn’t like” and 7 being “liked a lot,” participants gave prototype the highest cumulative and average score via the abovementioned Likert-scale question. (Figure 5)

12

Figure 5

Next StepsThe most liked elements of all three prototypes, along with the findings from the interviews, surveys, and focus group were considered in creating the redesign of the profile page.

Redesign Proposal

RecommendationsThe following are the top recommendations for the StumbleUpon profile page redesign based on findings from the interviews, survey and focus group:

Recommendation

Business Benefit

User Benefit

Cost Evidence

Include reviews/comments feed 2 2 $ surveyCluster profile information 2 2 $$ interviews, survey, focus groupPresent weighted representation of similar interests 2 2 $$ interviews, survey, focus group

Present more interactive experience 2 2 $$$

interviews revealed favorites as most desired interaction; demographics and industry trends

Offer private message capability 1 2 $$$ interviews, survey, focus group

Prioritized RecommendationsIn order to prioritize our recommendations, we performed a cost-benefit analysis (Figure 6) and recommend the following three as the ones with highest business and user impact while requiring the least cost to implement. Further analysis would be needed to confirm this.

1. Include reviews/comments feed

2. Cluster profile information 3. Weighted representation of

similar interests placed prominently

13

Figure 6

RedesignOur proposed redesign (Figure 7) of the profile page incorporates most of the recommendations from our research findings.

Recent user activity and a comments feed have been added. By clustering the key elements of the profile summary together, we’ve reduced the amount of

scanning users have to do to find information. The text cloud in the left navigation makes it easier to determine similar interests immediately. The cover-flow design offers a dynamic, interactive experience. The option to send a message to this user is more prominent in the profile summary.

14

Figure 7

ConclusionThrough three user research methods, we gained an in-depth understanding of StumbleUpon users, what social networking features are most important to them, how their experience can be improved to encourage connecting and we explored their opinions of new ideas for redesigns. Our proposed redesign incorporates these research findings and user recommendations.

There were a number of other issues discovered in these research processes that require additional research. These issues include:

Terminology can be confusing and ambiguous.

o We recommend a closed card sort to discover what terms make sense to users

Privacy is a big concern

o We recommend a competitor analysis to determine trends and what users expect

Lessons Learned Planning the interview and the questionnaire was crucial to get us all on the same page, as a

semi-structured interview is not scripted completely.

Developing the survey question set proved difficult at first because there was a lot of feedback we wanted to validate. However, we realized the importance of keeping it short in order to encourage completion.

It is important to allow enough time for a focus group moderator to become familiar with the product being reviewed as well as any prototypes that are presented.

15

Appendices

Appendix A: InterviewIntroduction to Participants

Thank you for participating in the interview today. In order to promote consistency among everyone we will be interviewing, I am going to read some information off of a script before I get started with the interview.

I have asked you here to learn about how you use StumbleUpon to meet and interact with other like-minded StumbleUpon users. I will first ask you a few questions about your social networking preferences after which I will ask you to perform some activities on the site. After completing these activities, I will ask a few more questions about your experience. Our conversation should take about 45 minutes.

Although I will be observing you as you interact with the site, please perform today’s activities as you normally or naturally would without observation. I am not testing you or judging your performance—I am only interested in your experience interacting with certain features of the site. Please try to think aloud as you’re working, just say whatever is going through your head as you are going about your activities. It’s like a running commentary.

If you have any questions, feel free to interrupt me anytime. Also if you need to take a break anytime during the interview, please let me know.

Let’s get started.

Pre-Activity InterviewBefore we get started, I would like to ask you a few questions about your computer background and social networking experience:

1. What is your job title?2. What is your primary operating system? (Windows or Mac)3. How long have you been a StumbleUpon user?4. How often do you use StumbleUpon?5. What are the top three things that you like about online social networking?

Let’s start exploring the social aspect of StumbleUpon.

Interview and Activities

Activity 1: You’ve already created a StumbleUpon account; now connect with me by using my email ID. Once you are connected you can follow me.

Observe: Did they go to Find Friends first? Were they able to find where to add the email address to connect to you?Ask:

16

How do you feel about the option of selecting friends by searching your contacts? What do you think about the information provided in the tabs under the user? Rate the Tabs in the order of importance for you.

Activity 2: Follow another Stumbler. Observe: Do they immediately go to the Stumblers tab? Do they go to “Visitors”?Ask: What are the common interests that you share with the person you stumbled upon? What would you like to see to inform you better about the person you stumbled?

Activity 3: Who is following you?Observe: Do they go to the followers tab?Ask: What do you think about the information in this page? Is it useful to you?

Activity 4: Who are you following?Observe: Did they go to the following tab?Ask: What information does this page provide for you? What else would you would like to see on this page?

Activity 5: Click on the Suggestions tab. What information does this page provide for you? What else would you would like to see on this page?

Activity 6: Customize your profileObserve: Where do they go first to customize their profile? Did you need to prompt them to go to settings page?Ask: Did you have any issues with changing your profile?

Post-Activity Interview Questions1. How is “Meet a Stumbler” similar to or different from the experience of social networking sites

you use frequently?2. Overall, what aspect of “Meet a Stumbler” did you find most appealing? Least appealing? 3. What value does “Meet a Stumbler” add to the StumbleUpon experience? 4. Would you like to correspond with other Stumblers in a way not presented? If yes, how?5. How likely are you to want to “Meet a Stumbler” 6. How confident were you with StumbleUpon’s ability to keep your information private?

17

Appendix B: Interview DataA. Site Design & Features

1. Terminology (20 comments)These issues were found due to the conflict in meaning of the names used by the site. Some of the terms that were under misconstrued were:

Visitors Followers/Following Likes vs. Tags Suggestions vs. Find Friends Discoveries vs. Discover Stumble (connections or websites)?

2. User Interface Design (11 comments)These are some of the issues found with the UI elements of the website.

Grid view preferred as default Too many levels of tabs throughout the site Overlooked “search by e-mail” field Bad theme color implementation Too much white space Didn’t notice Home icon Overlooked “Search for Stumbler” field Website information sometimes cut off

3. Privacy concerns (16 comments) Connecting w/strangers I don’t want it accessing my Gmail and Facebook contacts Spamming More secure than regular surfing?

B. Personal ConnectionsIssues found with user experience while connecting with others

1. Matching/Connections (6 comments) More comfortable with being connected with friends of friends Wanted more control over connection choices (e.g. provide list of choices, connect via

mutual reviews) Can’t search by actual name, need specific profile name Unclear as to how matches are made Search by e-mail option is not discoverable/visible Search by name is not discoverable/visible

2. Profile Didn’t know I had a profile Customize profile page is unorganized and impersonal Can’t edit profile by clicking on profile picture Too much personal information requested Information should be cached Settings page not discoverable Settings and Hi__ links redundant

3. Following Preferences Commonalities not obvious Would want to connect with friends and family—not strangers Choose followers based on similar site thumbs up Do not want to correspond with other Stumblers

18

I don’t need to find friends here Don’t know the reasoning behind suggested Stumblers to follow Tags are a poor way of comparing interests

C. Ideas/Suggestions Apart from the issues we also had opinions and ideas from the users that were separate from the issues.

To improve the social networking experience, user would like to connect with Stumblers via IM

Using wizard format for profile updates was expected. More information was expected in the “We Both Like,” common interests section. A recent activity overview would be nice Add a new favorites following page Point out what information is missing (don’t have any followers, do this … click here to

add a photo) Would be nice to have a video that explained social networking functions More info with pictures

D. Other Comments User like it that there were no ads. “We both like’ tab was considered cool. Users liked finding contacts through Gmail and Facebook. Information in following and followers tabs were good Users liked the way tags were implemented. The tabs in the user’s main page were useful. Users liked being able to invite and share with others via StumbleUpon. “I like that I can browse followers without connecting with anyone.” Being able to add reviews was appreciated

19

Appendix C: Online Survey Question Set and Data

20

21

Appendix D: Focus Group Participant Sketches

22

Appendix E: Focus Group Questions

1. What is this prototype telling you?

2. Looking at all of the elements, what can you do here?

3. Which parts are self-explanatory and which parts are confusing?

4. What are your likes/dislikes of the design?

23