assignment #9: usability study and website designccarra1/Writing/Usabilty.pdf · assignment #9: usability study and website design executive summary ... However, the image accompanying

http://iwantsandy.com usability study

group #3christina carrasquilla

sarah houghstacy rempel

assignment #9: usability study and website design

executive summaryThis report will examine the usability of the website http://iwantsandy.com and based on the findings will suggest a homepage redesign. The testing will cover the main topics of the website including sign up, service provided, examples, widgets, accolades, 24/7 customer service, and the help guide. Testing was performed on the first week of November 2008 with 3 members on the test crew and 9 users tested. The test lasted around 10-15 minutes per user and was performed at the homes of the test crew. The findings were that users could locate and understand the main functions of the website http://iwantsandy.com such as the sign up, the services provided, the examples, the accolades, and the help guide but were slightly confused with the more advanced functions of the website such as the advertising widgets and 24/7 customer service forum.

page 1


page 2

table of contents

executive summary—1

purpose of the study—3

participants—3

method—3step one: select a project—3

step two: user profile—3

step three: heuristic evaluation—4

step four: study planning—5

step five: prototyping and scenarios—5

findings and recommendations—5sign up—5

service provided—5

examples—6

widgets—6

accolades—6

24/7 customer service—7

help guide—7

post questionnaire—7

original homepage—9

suggested homepage redesign—10

conclusion—11

appendix a: participant questionnaire—12

appendix b: consent form—14

appendix c: tasks—15

appendix d: post questionnaire—17

appendix e: script—19


page 3

purpose of the studyThis site was chosen because of its potential to improve. Overall, we felt that the site provides a valuable service but that it is presented in an unprofessional way. Inconsistent colors, fonts, and verbage throughout the site look unorganized which hurts a site that is trying to help the user get organized. However, the service the site provides is performed very well and with some basic graphic uniformity can be greatly improved. The homepage suggests the main usability goals of the site are to get the users to sign up, to explain the service provided, to provide examples of the service, to provide advertising widgets, to list accolades given to the website, to provide 24/7 customer service and a user help guide.

participantsThis study tested 9 users that were contacted in person. The average profile of the users was:

Age- • 33-37

Sex-• Female

Country of Origin- • United States

Primary Language-• English

Income Level-• $41-45K

Highest Education Level-• Some College

Operating System-• Windows XP

Web Browser- • Internet Explorer

Computer Experience-• 14.5 years

Hours Daily Spent on Computer-• 7-8 hours

http://iwantsandy.com• Experience- None

method

step one: select a projectChoose a website to do a usability study on and familiarize yourself with all pages including identifying the major goals of the site.

step two: user profileBuild a profile of your end user according to the following categories:

Age or date of birth1. Gender2. Location3. Country of Origin4. Languages Spoken5. Income level6. Education level7. Operating system used/preferred8. Browser used/preferred9. Computer Experience10. Internet/Web/email experience11. Experience with this site12.


page 4

step three: heuristic evaluationPerform a Heuristic Evaluation by answering the following questions:

Visibility of system status• Does the system keep users informed about what is going on, through appropriate 1. feedback within reasonable time?Does the site are tell users “Where am I?” and “Where can I go next?”2. Is each page is branded and does it indicate which section it belongs to?3. Stress Test4.

Match between system and the real world• Does the system speak the users’ language?5.

User control and freedom• Does the site support undo and redo?6. Is there a “home” button on every page?7. Are users forced into certain fonts, colors, screen widths or browser versions?8. Does the site have animated GIFs?9. Does the site follow HTML standards and other specifications?10.

Consistency and standards• Is the wording in the content and buttons consistent?11. Do the page titles match the links that lead to the page?12. Does the site use custom link colors (a color other than blue)? 13. Is the navigation where you expect to find it?14. Is the search box located where you expect to find it?15. Are any site features unusable?16.

Error prevention• Do you receive any error messages when using the site?17. Do forms offer a submission confirmation?18.

Recognition rather than recall• Does the navigation clearly indicate which page you are using?19. Are there “bread crumbs” defining the path to the current page?20. Are the link labels descriptive?21. Does the site appear to use linked images?22. If so, was it obvious the image was linked?23.

Flexibility and efficiency of use• Are pages of the site easy to bookmark?24. Are frames used for the site?25. Are temporary URL’s used?26. If forms are used, is GET used instead of POST?27.

Aesthetic and minimalist design• Do dialogues contain relevant information?28. Is there extra information on the page that could be eliminated?29. If users need additional information, are there progressive levels of detail within the site?30.

Help users recognize, diagnose, and recover from errors• Are error messages expressed in plain language, indicating the issue and offering a 31. solution?

Help and documentation•


page 5

Is documentation available for users when they need help?32. If so, is it easy to read, focusing on the user’s task and concrete steps to be carried out?33. Is help integrated into the site, making it easy for users to find and use?34.

step four: study planningThis usability test will consist of:

Welcome• User will complete a pre-test questionnaire (appendix a), which will include basic -demographic information, along with a consent form (appendix b).

Observation Time• Users will have an opportunity to explore the site, without interruption. -

Tasks• Users will be asked to complete a set of tasks (appendix c). -

Post questionnaire• A questionnaire will be administered where the users can provide feedback about their -experiences (appendix d).

step five: prototyping and scenariosA script (appendix e) will be followed by each member of the test crew.

findings and recommendations

sign upThe main purpose of the site is to get users to sign up for an account. The sign up section was easily understood by the users. They were able to quickly locate and click on of the two sign up links. However, the image accompanying on the front page was unclear and user where unsure of what it is displaying and why. During testing, users were asked and indicated the following:

Where would you go to sign up for an • http://iwantsandy.com account?89% clicked on the correct link on the first click -100% eventually clicked on the correct link -7 seconds was the average time it took to click on the correct link -

Does the site lead you to sign up?• 89% said yes -11% said no -

service providedA main focus of the site is to explain the service of the site to its users. The service provided section of the website was also easily understood by the users. They were able to quickly locate and click on the “How I Work” link. However, once there, some users felt like the language was too cute. During testing, users were asked and indicated the following:

Where would you go to learn if the site provides a service you may need?• 67% clicked on the correct link on the first click -100% eventually clicked on the correct link -12 seconds was the average time it took to click on the correct link -

Did you have trouble finding this?• 89% said no -


page 6

11% said yes -

examplesShowing examples of how the service can be used is an important feature to educate users on how to use the site and how it might address their specific needs. The participants were asked a series of questions regarding finding examples of how to use the service.

“Where would you go to find an example of how to use the services this site provides?” • 100% clicked on the correct link on the first click -7 seconds was the average time it took to click on the correct link -

“Do you need examples in addition to the service explanation?” • 67% of the participants wanted examples in addition to the service explanation -33% felt that the explanation alone was sufficient -

“Do the examples provided match your situation?” • 67% fell within the categories listed -33% of the participants felt the example categories did not match their situation -

widgetsWikipedia defines a widget as a portable chunk of code that can be installed and executed within any separate HTML-based web page by an end user without requiring additional compilation. They are derived from the idea of code reuse. Widgets often take the form of on-screen tools that draw on Internet databases to show the information on the users ‘widget’ so a web-browser does not need to be used. http://iwantsandy.com makes widgets available on its site so that calendar reminders can be brought to the user through whichever widget dashboard they may use. Participants were asked questions regarding finding and implementing the widgets offered by http://iwantsandy.com.

Where would you go to find this site’s advertisement widgets?” • 33% of participants achieved the successful end state in one click -56% took two clicks -11% took five clicks to find the appropriate page -17 seconds was the average time to complete this task -

“Was this confusing?” • 89% of the participants found the previous question to be confusing -11% clearly understood what was being asked -Overall, it was found that although a few users had heard of a widget, they weren’t sure -what it was; therefore, were unsure of where it would be logically located

“Do you know what a widget is? If yes, could you implement one?” • 44% of participants knew what a widget was -56% did not -The users who knew what a widget was, were not all able to implement one -

accoladesAccolades can build trust with potential users and is an important feature to build service credibility and security. Participants were asked questions regarding locating accolades given to http://iwantsandy.com.

“Where would you go to read accolades earned by this site?”• 100% of participants were successful in locating this information in one click -11 seconds was the average completion time -


page 7

“Is seeing accolades necessary for you to sign up for the site?”• 67% of participants did not feel it was necessary to see accolades in order to sign up -33% felt it was necessary -

24/7 customer serviceParticipants reported that they would need live support, but then indicated that they would use a help guide over this resource. If the live customer support was easier to locate, then participants may be more likely to use this feature. During testing, users were asked and indicated the following:

“Where would you go to contact a customer service representative for help?”• 22% in one click -22% in two clicks -22% in three clicks -11% in five clicks -11% in 10 clicks -60 seconds was the average time for finding this page -

“Was this hard to find?”• 78% reported that customer support was difficult to find -22% two stated that it was easy to locate -

“Do you require 24/7 service?” • 55% answered that they would need the support -44% answered that they would not need 24 hour live support -

help guideUsers would prefer using a help guide in order to find the answers that they are looking for on the site, and it appears that this is relatively easy to find. During the study, users had an easier time with finding the help guide, so this may have contributed to the answers they provided. During testing, users were asked and indicated the following:

“Where would you go to find the help guide?”• 100% clicked on the correct link on the first click -7 seconds was the average time for finding this page -9 clicks was the average of to get to this page -

“Was this confusing?”• 0% of the participants reported that the help guide was difficult to find -

“Are you more likely to use a help guide or a live person?”• 100% reported they would more likely use the help guide than 24 hour live support -

post questionnaireFollowing the main usability questions, participants were asked and indicated the following:

The questions were easily understood• 78% Agree -11% Neither Agree or Disagree -11% Disagree -

The homepage,• http://iwantsandy.com, contained adequate initial information78% Agree -22% Disagree -

The colors used were appropriate for the site•

page 8


11% Strongly Agree -56% Agree -22% Neither Agree or Disagree -11% Disagree -

The navigation was grouped in a logical manner• 56% Agree -22% Neither Agree or Disagree -22% Disagree -

The pages contained sufficient information• 89% Agree -11% Disagree -

Topic and page headers were easy to understand• 78% Agree -22% Disagree -

Terms used throughout the site were understandable• 33% Agree -11% Neither Agree or Disagree -56% Disagree -

There were an adequate number of images• 67% Agree -11% Neither Agree or Disagree -22% Disagree -

Images were clear and sharp• 78% Agree -22% Disagree -

The site was visually appealing• 89% Agree -11% Disagree -

The pages loaded quickly• 89% Agree -11% Disagree -

On a scale of 1-10, 10 being the best, how would you rate this site based on what you • experienced today?

11% Rated 4 -11% Rated 5 -22% Rated 6 -44% Rated 7 -11% Rated 8 -

Additional comments• Thought the top utility links were too obscure and didn’t like that the screen shot on the -front page was too small to really see an example of use.Interesting concept, but a little confusing - could be better done. I don’t like that so much -of it is based on text messaging.This site is really good service idea and I’m going to start using it. -Front image (table in the middle) is hard to see so not sure what is shows , sandbox and -buzz are not familiar terms.

page 9


Needs more than the one image, sandbox and community terms don’t make any sense, -how I work isn’t the best description, overall the site is too cute and uses cutesy terms and explanations.Names of sandbox and buzz are confusing terms. -I could do this service myself. -Interesting concept. I liked the look and feel of the site. -I want to start using the site, but I keep feeling there is a catch somewhere. -

original homepage

suggested homepage redesign

page 10


Life moves too fast for your calendar.

sign up for free log in

Home How I work Examples Extras News Help

Hi! I’m Sandy, your new assistant. I'll remember the details so you can focus on what's important.

• Kick off your morning knowing what's on for the day.

• Keep your day moving forward with reminders right when you need them.

• Books to read, people to call, your cousin's birthday — I'll remember it all.

• Stay organized with everyone in your life — effortlessly.

All without leaving your email inbox.

Everyone deserves an assistant...

Already a client? Log in

Sign up for free!

][ [ [ [ [] ] ] ] ][

About us · Blog · Buzz · Policy · Terms of use · PrivacyA Values of n, Inc product · Powered by Stikkit · Copyright © 2006-2008

Really can't stand Comic Sans? Sans | Avec the sans

conclusionBased on the usability study of the website http://iwantsandy.com we find that users could locate and understand the main functions of the website such as the sign up, the services provided, the examples, the accolades, and the help guide but were slightly confused with the more advanced functions of the website such as the advertising widgets and 24/7 customer service forum. Most of the problems were due to confusing terms such as “sandbox” for widgets (although most completely unaware of what a widget is) and “buzz” for accolades, and “community” for 24/7 customer service (especially when there is also a help guide).

Our recommendations for a follow up study would be to implement our suggested changes and re-test with a much larger number of participants.


page 10


page 12

appendix a: participant questionnaireThe following answers will be confidential and are for demographic purposes only.

1. Age:18 - 2223 - 2728 - 3233 - 3738 - 4243 - 4748 - 52

2. Sex:MaleFemale

3. Country of origin: _________________________

4. Primary language spoken: _________________________

5. Income level:$20,000 - $25,000$26,000 - $30,000$31,000 - $35,000$36,000 - $40,000$41,000 - $45,000$46,000 - $50,000$51,000 - $55,000$56,000 - $60,000

6. Highest education level completed:High SchoolSome CollegeBachelor’s DegreeMaster’s DegreeDoctoral Degree

7. Which operating system do you use?Windows 2000Windows XPWindows VistaMac OS ClassicMac OS XLinux

8. Which web browser do you prefer?InternetExplorerSafari

FirefoxOpera

9. How many years have you been using a personal computer? ______

10. On average, how much time do you spend on a computer each day?Not at all0 - 2 hours a day3 - 4 hours a day5 - 6 hours a day7 - 8 hours a day9 - 10 hours a day

11. How much experience do you have with the iwantsandy.com website?NoneLittleModerateExperienced


page 13

appendix b: consent form

iwantsandy.com Usability Testing

Thank you for participating in our usability testing http://iwantsandy.com. Your usability testwill be recorded by webcam for informational purposes only. Your feedback andparticipation will beneficial to the future growth of this site. This video will not beviewed by anyone outside of the evaluation team.

I understand that a video recording will be made of my usability testing. I grant theusability test group permission to use these recordings for the purposes mentionedabove, and waive my right to review or inspect the tapes prior to their disseminationand distribution.

Please print name: _________________________________

Signature: ________________________________________

Date: ____________________________________________


page 14

appendix c: tasksTask 1: Sign Up

Script: “Where would you go to sign up for an • http://iwantsandy.com account?”

Start state: • http://iwantsandy.com

Success Criteria (end state): • http://iwantsandy.com/signup

Analysis:• Notes: -Number of clicks to complete: -Time to complete: -

Additional Debriefing Questions•

Script: “Does the site lead you to sign up?”•

Task 2: Service Provided

Script: “Where would you go to learn if this site provides a service you may need?”•

Start state: • http://iwantsandy.com/signup

Success Criteria (end state): • http://iwantsandy.com/how_i_work


Additional Debriefing Questions• Script: “Did you have any trouble finding this?” -

Task 3: Examples

Script: “Where would you go to find an example of how to use the services this site • provides?”

Start state: • http://iwantsandy.com/

Success Criteria (end state): • http://iwantsandy.com/examples


Additional Debriefing Questions• Script: “Do you need examples in addition to the service explanation?” -Script: “Do the examples provided match your situation?” -

Task 4: Implement Advertisement Widgets

Script: “Where would you go to find this site’s advertisement widgets?”•


Success Criteria (end state): • http://iwantsandy.com/sandbox#widgetbox

Analysis:•


page 15

Notes: -Number of clicks to complete: -Time to complete: -

Additional Debriefing Questions• Script: “Was this confusing?” -Script: “Do you know what a widget is? If yes, could you implement one?” -

Task 5: Site Accolades

Script: “Where would you go to read accolades earned by this site?”•


Success Criteria (end state): • http://iwantsandy.com/buzz


Additional Debriefing Questions• Script: “Is seeing accolades necessary for you to sign up for the site?” -

Task 6: 24/7 Online Customer Service

Script: “Where would you go to contact a customer service representative for help?”•


Success Criteria (end state): • http://getsatisfaction.com/iwantsandy


Additional Debriefing Questions• Script: “Was this hard to find?” -Script: “Do you require 24/7 service?” -

Task 7: Help Guide

Script: “Where would you go to find the help guide?”•


Success Criteria (end state): • http://iwantsandy.com/help?context=%2Fhelp%3Fcontext%3D%252Fbuzz

Analysis:• Notes: -Number of clicks to complete: -

Additional Debriefing Questions• Script: “Was this confusing?” -Script: “Are you more likely to use a help guide or a live person?” -


page 16

appendix d: post questionnaire

Please answer the following questions based on your experience using http://iwantsandy.com.

Task Questions/Issues:

1. The questions were easily understood.Strongly disagreeDisagreeNeither agree nor disagreeAgreeFeedback:_______________________________________________

Site Design:

2. The homepage, iwantsandy.com contained adequate initial information.Strongly disagreeDisagreeNeither agree nor disagreeAgreeFeedback:_______________________________________________

3. The colors used were appropriate for the site.Strongly disagreeDisagreeNeither agree nor disagreeAgreeFeedback:_______________________________________________

4. The navigation information was grouped in a logical manner.Strongly disagreeDisagreeNeither agree nor disagreeAgreeFeedback:_______________________________________________

5. The pages contained sufficient information.Strongly disagreeDisagreeNeither agree nor disagreeAgreeFeedback:_______________________________________________

6. Topic and page headers were easy to understand.Strongly disagreeDisagreeNeither agree nor disagreeAgreeFeedback:_______________________________________________


page 17

7. Terms used throughout the site were understandable.Strongly disagreeDisagree

Neither agree nor disagreeAgreeFeedback:_______________________________________________

8. There were an adequate number of images.Strongly disagreeDisagreeNeither agree nor disagreeAgreeFeedback:_______________________________________________

9. Images were clear and sharp.Strongly disagreeDisagreeNeither agree nor disagreeAgreeFeedback:_______________________________________________

10. The site was visually appealing.Strongly disagreeDisagreeNeither agree nor disagreeAgreeFeedback:_______________________________________________

11. The pages loaded quickly.Strongly disagreeDisagreeNeither agree nor disagreeAgreeFeedback:_______________________________________________

Overall Impression Of The SiteOn a scale of 1 to 10, 10 being the best, how would you rate this site based on what youhave experienced today?1 2 3 4 5 6 7 8 9 10

Comments/SuggestionsAll comments are welcome.________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________


page 18

appendix e: script

INTRODUCTIONBefore testing begins, the testing crew will briefly introduce themselves and explain how testing will work and how long the test will take.

Test Crew 1: “Hi, We are Christina, Sarah and Stacy and we are doing a group project to test a websites usability. We have a few brief questions to ask in order for us to analyze the current site’s usability and then suggest improvements. It will only take 10 minutes and you do not need any previous knowledge about the site or usability.”

Test Crew 2: “In order to evaluate the site’s navigation, we will ask you where you would click to complete a task based on the functions that the site provides.”

Test Crew 3: “In other words, where would you click to do (blank)?”

SCENARIOYou are trying to find information on the website http://iwantsandy.com.

TASKSThe testing crew will have a computer on hand that will be open to the website http://iwantsandy.com. Each member of the testing crew will present a task and ask theparticipant where they would click to find the information.

Task 1: Sign Up

Script: “Where would you go to sign up for an • http://iwantsandy.com account?”

Start state: • http://iwantsandy.com

Success Criteria (end state): • http://iwantsandy.com/signup


Additional Debriefing Questions•

Script: “Does the site lead you to sign up?”•

Task 2: Service Provided

Script: “Where would you go to learn if this site provides a service you may need?”•

Start state: • http://iwantsandy.com/signup

Success Criteria (end state): • http://iwantsandy.com/how_i_work


Additional Debriefing Questions• Script: “Did you have any trouble finding this?” -


page 19

Task 3: Examples

Script: “Where would you go to find an example of how to use the services this site • provides?”


Success Criteria (end state): • http://iwantsandy.com/examples


Additional Debriefing Questions• Script: “Do you need examples in addition to the service explanation?” -Script: “Do the examples provided match your situation?” -

Task 4: Implement Advertisement Widgets

Script: “Where would you go to find this site’s advertisement widgets?”•


Success Criteria (end state): • http://iwantsandy.com/sandbox#widgetbox


Additional Debriefing Questions• Script: “Was this confusing?” -Script: “Do you know what a widget is? If yes, could you implement one?” -

Task 5: Site Accolades

Script: “Where would you go to read accolades earned by this site?”•


Success Criteria (end state): • http://iwantsandy.com/buzz


Additional Debriefing Questions• Script: “Is seeing accolades necessary for you to sign up for the site?” -

Task 6: 24/7 Online Customer Service

Script: “Where would you go to contact a customer service representative for help?”•


Success Criteria (end state): • http://getsatisfaction.com/iwantsandy

Analysis:•


page 20

Notes: -Number of clicks to complete: -Time to complete: -

Additional Debriefing Questions• Script: “Was this hard to find?” -Script: “Do you require 24/7 service?” -

Task 7: Help Guide

Script: “Where would you go to find the help guide?”•


Success Criteria (end state): • http://iwantsandy.com/help?context=%2Fhelp%3Fcontext%3D%252Fbuzz

Analysis:• Notes: -Number of clicks to complete: -

Additional Debriefing Questions• Script: “Was this confusing?” -Script: “Are you more likely to use a help guide or a live person?” -

CONCLUSIONTest Crew: Thank you for participating in our usability survey. If you have any further questions, please contact us.


page 21

Documents

assignment #9: usability study and website designccarra1/Writing/Usabilty.pdf · assignment #9: usability study and website design executive summary ... However, the image accompanying