USABILITY REPORT FINALsga72/docs/iScreen-Usability... · 2009-03-09 · INFO 610 George Abraham & Christine Wania 1 I-Screen: USABILITY REPORT Introduction Context The College of

INFO 610 George Abraham & Christine Wania

1

I-Screen: USABILITY REPORT

Introduction

Context

The College of IST recently installed an interactive kiosk called iScreen, designed to

serve as an information resource for student/visitors to the College of IST. The

content is designed to answer some of the common questions students/visitors might

have, for example: faculty office locations, building hours etc.

The hardware comprises of a touch-screen capable 42” Plasma monitor, featuring a

walk-up-and-use interface. The interface needs to support novice as well as repeat

users in terms of content and accessibility.

Problem statement

Ethnographic study of prospective and actual users revealed system/hardware-

specific, system environment-specific and interface-specific problems that hinder

usability of the iScreen (refer Portfolio Pt.1). In this report, we focus on the study of

interface-specific problems.

Goal

Our goal is to research the information seeking behavior and the information needs

of students and visitors, so as to customize content on the iScreen. We try and

determine a way to present this information to aid effective use.

Approach

Based on the ethnographic findings related to the interface, we conducted semi-

structured interviews with prospective and current users. In parallel, we designed

and performed usability tests followed by a debriefing to understand and target the

usability issues faced by the users. The following figure presents an overview of our

study design.

Fig.1 Study design overview.

The study results were analyzed quantitatively and interpreted qualitatively to

generate recommendations for improving the usability of the iScreen. The following

sections describe in detail the product under study, research method utilized, and

the implications of our findings for (re)design.

Ethnographic observations

Interview design

Usability tasks Questionnaire

Findings and design

implications


2

Product description

Description of the environment

The iScreen is located in the lobby of the College of IST, Rush Building. The

environment is the lobby of the Rush Building. The lobby has several conversation

spaces and seating where students meet to socialize, eat and/or work. The lobby

area is the first space any building visitor enters. There is little on the walls except

for a bulletin board and the interactive iScreen.

User population its intended for

The current content is targeted at common queries students and visitors at the

College of IST might have, like for e.g. Directory assistance, faculty office location,

classroom location etc. The interface and content needs to support both novice and

repeat users. It has been designed as a walk-up-and-use kind of interface and

should require minimal or no training. The information resource on the iScreen is

presently organized in the following manner:

Main (Touch screen)

• Directory

o Faculty

o Staff

o Main

• Building info

o Hours of operation

o Telephone numbers

o Main

• IST calendar

o IST events

o Academic calendar

� Graduate Fall schedule

o Main

• Course schedule

o Fall courses

o Main

• Academic FAQ

o Undergraduate

o Graduate

� More questions

o Main

Hardware and behavior

The hardware comprises of a 42” Panasonic plasma screen mounted 4 ft. above floor

level on the wall. In its inert state the iScreen displays a number of different screens

in a cyclical manner. One screen simply displays the Drexel University logo, followed

by two screens displaying pictures of past events at the College of IST. The

remaining few screens are advertisements for the College of IST degree programs.

Each screen image is displayed for a set amount of time (approximately around 5

seconds, this was not measured) and then the next screen is displayed. In the

bottom left-hand corner of each of these screens is text that reads “Touch Screen to

Begin.” This text is relatively small and very difficult to read unless you are standing

right in front of the iScreen within a short distance (5 feet).


3

Study design

Participant profile

Number of participants

The aim of our study was not to achieve statistical significance in the quantitative

sense. Rather, we aimed at ensuring that we had at least 4-6 representative users

for our interview and representative 4-6 users for our usability tests (Mayhew,

1999). The participants were selected based on the target population for the

system/product, primarily students and visitors.

Segmentation

We identified three types of student users and one visitor category. The three types

of students belonged to the Bachelors, Masters, and Doctoral program. We did not

differentiate between students within each level of study. It can be argued that a

senior would have different information requirements as compared to a freshman or

a sophomore. In an ideal condition this would be considered, but our study focus was

at a higher level. During the interview and debriefing, we recorded responses

regarding information requirements specific to the user-group to account for these

finer grained differences.

Key characteristics and capabilities

In total there were eight participants for our study, four for the interview and four for

the usability test. Each of them fit the generic profile of a target user. This was

important especially for the usability test. We did not want to make participants

perform tasks that would not be part of their routine. The participants needed to be

familiar with abbreviations like INFO 435 etc. that were used commonly to represent

a course name. It was not necessary for the participants to have previous experience

with the interactive screen. If they had previous experience with the iScreen, we

accounted for this during our interview and debriefing by asking a few open-ended

questions to get further insight regarding usage and experience level.

Selection process

In an ideal situation we would have advertised on bulletin boards to recruit

participants for this study after receiving IRB approval. Because of the lack of time to

complete this study we needed to improvise. We would have liked to include one

undergraduate student, one graduate student, and two visitors as participants for

both the interview and the usability study. We did not have much luck finding such

participants, therefore we settled for whoever agreed to volunteer. An overview of

the participants in our study can be seen in Table 1. The users that participated in

the interview were not the same as those that participated in the usability testing.

Participant Type Interview Usability Testing

Undergraduate Student 1

Graduate Student 3 3

Visitor 1

Table 1: Participant details for interview and usability testing


4

Interviews

Purpose

The purpose of the interview was to learn about the general information seeking

behaviors of students in the College of IST. In addition we were looking to find out

what students know about the interactive iScreen located in the lobby of the College

of IST.

Questions posed

The interview contained a number of questions related to students’ information

seeking behaviors and their knowledge of the interactive iScreen. We also asked

questions based on the information presented in the iScreen. A copy of the interview

questions can be seen in Appendix A. As we saw fit throughout the interviews we

asked related questions that were not included in the original question list. The

interview protocol was to elicit responses regarding:

• Profile of the user- What year of study and program enrolled for?

• Previous experience- Have you used the iScreen before?

• Information resources known to them- Like for e.g. College website etc.

• Information seeking behavior- Who to ask/ where to go.

• Information needs- Common information requirements

• Preferred information format- How should content be presented?

• Suggested improvements

Question selection

The interview was used as mini-requirements gathering process to facilitate better

usability test design, and also to inform us regarding information that should be

represented in the content but not there yet. This also helps us in deciding what

information needs to be prioritized on the interactive screen.

Test environment

It was not necessary to conduct a contextual interview because the questions were

more abstract in nature, and the primary focus was on information behavior. Test

We audio taped the interviews and transcribed the audio. The participants were

informed that their identities would be kept anonymous unless specified otherwise.

Also, no permanent audio record was maintained once it was transcribed.

Key findings

Interview data

As mentioned earlier, the focus of our interviews with participants was to determine

the users’ information seeking behavior, information requirements, iScreen

functionality awareness and improvements to the interface. In this section, we

present our findings based on qualitative interpretation of the participant responses.

We have considered the fact there might be some bias in the responses based on the

fact that the participant might have used the iScreen before. Among the four

participants we interviewed, two of them had used it in the past, and two of them

had no idea that the iScreen was interactive.

1. Information seeking behavior


5

All the participants tended to differentiate between information related to their

program of study and general inquiries. In all the cases, they confirmed that they

knew whom to talk to regarding program related questions, in this case, advisors.

For general inquiries participants preferred to consult the IST website or call upon

the information desk at the College of IST.

2. Information requirements/format

Three of the four participants said they would be unable to dial a department phone

based on the 4-digit extension number alone. They did not know the how to call from

a campus phone, or from an external line based on the information provided.

Almost all of them expressed that the room numbers alone were not sufficient to

determine the physical locations. This is because the building is in V-shape with two

wings. Providing room numbers alone was not sufficient for determining which wing

the room belonged to.

3. iScreen functionality awareness

The purpose was to determine if the iScreen in its current form attracted the

attention of the student and visitors. Since it is located in a prominent and well-

lighted area there was a high chance that the participants had seen it even if they

had not used it. Two of the participants had not idea that it was interactive, and two

of them used it because they paid enough attention to notice a “Touch screen to

begin” icon. When asked if they knew if the screen was interactive one of the

participants replied, “Was it a touch screen? No way! I had no idea”.

Based on the interviews it became apparent that the users came to know about the

content only if they went up to the screen and touched it. In its inert phase, the

iScreen did not communicate its purpose or content. Another aspect of the

awareness problem that emerged from the interviews was that the name iScreen did

not communicate much to the users. Most of the participants suggested a more

semantically appropriate information screen or information kiosk instead.

4. Suggested improvements

The participants suggested improvements to enhance usability on two aspects,

aesthetics/interface and content. Aesthetic improvements included having icons in

addition to text that communicate the content. They emphasized the role of colors

and size of the icons that would attract the users’ attention without having to come

close to the screen. This information should be displayed during the inert phase of

the information screen.

Some of the participants gave suggestions to include faculty profiles, student

profiles, and videos to improve interactivity. One of them even suggested sound to

enhance the user experience. We feel that having sound to confirm clicks would

serve as an adequate feedback to the user.

Other suggestions were to have a list of classes in session or classes that are

scheduled for that particular day displayed during the inert phase. Train schedules,

and traffic information were also suggested, which we feel are excellent inclusions.


6

Usability test

Purpose

The purpose of the usability test is to collect and analyze information about the

effectiveness, efficiency, and user satisfaction of the College of IST interactive

iScreen.

The goals of the usability testing include:

• Determining how easy it is to use the iScreen

• Determining how easy it is to learn to use the iScreen

• Determining how easy it is to find information using the iScreen

• Determining if the information available in the iScreen suits the needs of the

users

• Determining what information is not provided that may be useful

Task list

We developed a task list based on the information available in the iScreen. We also

verified that all the tasks could be completed before conducting any usability tests.

The task list can be seen in Appendix B. The tasks were determined after analyzing

the structure of information in the iScreen. (The menu structure was discussed in the

background section in the beginning of this paper.) In an effort to be thorough in our

testing we constructed tasks that would force users to access every menu option.

The task list embedded in the menu structure can be seen in table 2.

Table 2. Task list embedded in Menu Structure

Main (Touch screen)

• Directory

o Faculty> Scott Silverstein office location and phone extension (Task 1)

o Staff> Colin Saunders office location and extension (Task 11)

o Main

• Building info

o Hours of operation> Hours of operation of CRC (Task 3)

> CRC helpdesk phone # (Task 8)

o Telephone numbers> Teaching assistants office number (Task 6)

> CRC helpdesk phone# (Task 8)

o Main

• IST calendar

o IST events> Events scheduled this week (Task 7)

o Academic calendar

� Graduate Fall schedule > Last day to withdraw (Task 5)

> Fall Quarter exam begins (Task 9)

o Main

• Course schedule

o Fall courses > professor for INFO 435 (Task 2)

> Location date and time for INFO 646 (Task 10)

o Main

• Academic FAQ

o Undergraduate > Advising walk-in hours (Task 4)

> Who to contact regarding advisors (Task 12)

o Graduate

� More questions> Who to contact regarding advisors (Task 12)

o Main


7

Test environment

The iScreen is in the lobby of the College of IST, therefore we assumed that the main

users would be students and visitors of the College of IST. The iScreen is intended to

be a walk-up-and-use interface therefore we are interested in studying novice users.

Based on the intended user population we included three students of the College of

IST and one visitor in our study. (We tried to find more visitors but did not have any

luck.)

Test set-up

The test was conducted in the actual environment as we felt that the system

environment was important to the study. We wanted to study the system in use, and

interpret the impact it might have on task performance or effectives qualitatively.

Conducting the test in the real environment would help bring out realistic concerns a

participant might have with the information presented.

Investigator tools

Based on the task list we determined that we wanted to measure the time, number

of clicks, and whether or not the task was completed. In addition we decided to

record the path a user takes to find the information in order to allow us to record

errors. We also agreed to record anything that seemed interesting including anything

the user commented on while performing the tasks.

Based on the task list and what we were interested in measuring, we created a form

to record data during the usability test. After conducting the first usability test we

slightly refined the documentation form. We realized it was not necessary to record

the number of clicks if we were recording the path the user took to find the

information. Since we had the site-map, we knew beforehand the actual number of

nodes that need to be traversed to find the task related information. We used this as

measure to determine if the information was difficult to find. We also had difficulty

recording the time because users found the information very quickly and we did not

have great equipment for recording the time, therefore we decided not to record the

time in subsequent tests. The documentation form can be seen in Appendix C.

We created the following steps to guide us through the usability test.

• Introduce ourselves to the user

• Describe the purpose of the test

• Inform the user that we are testing the system, not the user

• Describe what the user will do (12 tasks)

• Inform the user of how long it will take

• Inform the user that they may stop at any time

• Explain to the user that we would like them to think aloud

• Give the user the task list

• Record data

• Give the user the questionnaire

• Debrief the user

• Thank the user


8

Key findings

Usability test

All four users were able to complete all twelve tasks successfully. Table 3 displays

the tasks in which users committed an error while completing the specified task. In

table 3 the task number appears in the first column of the table. For each task if the

user committed an error the word “ERROR” appears in the corresponding cell in the

table. If the user completed the task without error the corresponding cell is left

blank. For example subject 3 completed tasks 1, 2, 3, 7, 8, 9, 10, and 12

successfully without error, while she committed an error while completing tasks 4, 5,

6, and 11. Examining this table by looking across the rows to focus on which tasks

most of the participants had difficulty completing was enlightening.

Task

Number Subject 1 Subject 2 Subject 3 Subject 4

1 ERROR ERROR 2 ERROR 3 4 ERROR ERROR ERROR ERROR 5 ERROR ERROR 6 ERROR ERROR ERROR ERROR 7 8 9 10 11 ERROR ERROR ERROR 12 ERROR

Table 3. Participant errors while completing tasks

All four participants committed an error while completing tasks 4 and 6. Three users

encountered error while trying to complete task 11. Two users encountered error

while trying to complete tasks 1 and 5. One user encountered error while completing

task 2 and another user encountered error while completing task 12. We analyzed

the paths that each user traversed while trying to complete these tasks.

The correct path and the user paths for these tasks can be seen below. Examining

the user traversal paths can give us insight into were the users expected to find the

information they were looking for. Some users also expressed their thoughts during

the usability testing.

Task 4. Find advising walk-in hours

Correct Path: Academic FAQ – Undergrad FAQ

S1: Building Info – Hours – Main – Academic FAQ – Undergrad FAQ

S2: Academic FAQ – graduate FAQ – undergrad FAQ

S3: Academic FAQ – graduate FAQ – undergrad FAQ

S4: IST Calendar – Academic Calendar - Academic FAQ– undergrad FAQ


9

For task 4 S1 looked first in building information, possibly assuming that all listed

hours of operation would be found in building information. S2 and S3 are both

graduate students therefore it is not surprising that they both looked first at

graduate FAQs. S4 looked first at the IST calendar to see if the hours would be

displayed there. The errors that both S1 and S4 made are very understandable.

Task 6. Find the phone number for the Teaching Assistant’s office

Correct Path: Building Info – Phone Numbers

S1: Directory – Staff – Faculty – Building Info – Phone Numbers

S2: Directory – Staff – Faculty – Building Info - Phone

S3: Directory – Staff – Academic FAQ – Grad FAQ – Building Info - Phone

S4: Directory – Faculty - Building Info - Phone

For task 6 all the subjects looked in the directory first, assuming that all phone

numbers would be listed in the directory. This is an understandable error. Two

subjects also commented during the usability test that they did not understand why

the phone numbers were separated. They expressed that they assumed all phone

number would be in the directory.

Task 11. Find Colin Saunders’ office location and phone extension

Correct Path: Directory – Staff

S1: Directory – Faculty – Staff

S2: Directory – Faculty - Staff

S3: Directory – Faculty - Staff

S4: Directory –Staff

For task 11 three of the four subjects looked in the directory and then looked in the

faculty section before looking in the staff directory. This is an understandable error.

Users should not be expected to know what position a person holds in order to find

out their office location and phone number.

Task 1. Find Scott Silverstein’s office location and phone extension

Correct Path: Directory - Faculty

S1: Directory – Staff - Faculty

S2: Directory – Faculty

S3: Directory – Faculty

S4: Directory – Staff – Directory – Faculty

The errors made in task 1 are the same as those in task 11. This is an

understandable error. Users should not be expected to know what position a person

holds in order to find out their office location and phone number.


10

Task 5. Find the last day to withdraw from a course

Correct Path: IST Calendar – Academic Calendar - Fall

S1: Academic FAQ – Main – Course Schedule – IST Calendar – Academic Calendar

S2: IST Calendar - Fall

S3: Academic FAQ – Undergrad FAQ - IST Calendar - Events

S4: IST Calendar – Fall

For task 5 S1 and S3 first looked to the FAQ list, assuming that this would be a FAQ.

The other two subjects went directly to the calendar. This may be an indication that

this type of information should also be displayed as a FAQ.

Examining the errors made by users provided information that the users may not

have expressed while completing the tasks or after completing the tasks. This data

was evidence of where users had the most difficulty and helped us determine what

may need to be redesigned. Examining this data also revealed that not all users are

the same. There were differences in how visitors, undergraduate students, and

graduate students used the system.

Questionnaire

We designed a questionnaire to be administered after the user completed the tasks.

The purpose of the questionnaire was to collect the users’ opinions about

effectiveness, efficiency, and their satisfaction after actually using the system.

Research has shown that efficiency, effectiveness and satisfaction are independent

variables, and hence, have to be measured independently (Frokjar, 2000). We also

took into consideration issues like learnability and navigability in the design of the

post-test questionnaire. The questionnaire can be seen in Appendix D. The

questionnaire was designed with closed-ended and open-ended questions. The close-

ended question responses were collected using a 5-point Likert scale because these

were subjective measures.

The questionnaire was administered after the users completed the twelve tasks. All

four users responded that they knew the iScreen was interactive before the study.

Two of the users had used the iScreen before this study and the other two had not. A

summary of the user responses can be seen in figure 1. We grouped these

statements based on which usability aspect we thought they measured. The raw

counts of the user responses can be seen in Appendix E.

Overall users expressed that it was easy to learn to use the system. This may be

because you only need to be able to touch the screen to use it. The majority of users

expressed that they found it easy to navigate through the system even though most

of them did not like where the icons/buttons were located.

Overall the users were not very satisfied with the efficiency of the system. A few

users also expressed aloud while performing the tasks that “it’s taking a long time to

find these things.” Although it was difficult for us to time these tasks because we

thought users were finding things quickly the users felt that it was taking too long.

Only one user expressed that it was easy to find the information they needed.


11

0%

10%

20%

30%

40%

50%

60%

70%

80%

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

Statement

Pe

rce

nta

ge

of

us

ers

ag

ree

ing

wit

h s

tate

me

nt

Effectiveness

Efficiency

Satisfaction

Ease of use

Learnability

Navigability

Table 3. Questionnaire Response

Statement

1. The interface communicated the content found in the system

2. The organization of information on the system screens is clear

3. This system has all the functions and capabilities I expect it to have

4. Using the system for answering some common questions is a good idea

5. I would use the system in the future

6. It was easy to read the information on the screen

7. It is easy to find the information I needed

8. I felt it took too long to find the information I was looking for

9. Overall, I am satisfied with how easy it is to use this system

10. I feel comfortable using this system

11. The interface of this system is pleasant

12. I like using the interface of this system

13. Overall, I am satisfied with this system

14. It was simple to use this system

15. It was easy to learn to use this system

16. Whenever I make a mistake using the system, I recover easily and quickly

17. It is easy to navigate

18. The icons/buttons were conveniently located

The main dislike expressed by the users in the open-ended questions was that the

screen they were currently on disappeared too quickly. In one user’s words “it comes

out of the current screen very quickly, by the time I decide to press something, it

goes back to the flashing screens.” The main thing users expressed the iScreen being

useful for was for finding faculty phone numbers and office locations. Finding the

faculty and staff phone numbers was one of the more difficult tasks for the users to

perform based on the number of errors made in these tasks. Therefore this is one


12

aspect that should be redesigned. Some users suggested improving the design by

adding a search feature for faculty. Another thing that users expressed was that

there needs to be something more to draw one’s attention to the screen.

Implications for design

Based on the interview responses, usability test and post-test questionnaire we

suggest some changes that could be implemented to improve usability of the

information screen.

1. Improve the phone directory interface to facilitate browsing capabilities

a. Facilitate last name based searching

During the usability test it became apparent that both students and visitors were

finding it difficult to find people because the current content layout implicitly

implies that the user would know who is a faculty and who is a staff (refer Task 1

& 11, Appendix B). The site map of current layout is as follows:

Main-Directory-Faculty or Staff

We suggest that this interface should be improved so that one could search/sort

list of names based on last name alone.

An alternative interface could be:

Main-Directory- A|B|C|D….|Z

In the results it could be indicated if the person listed is a faculty or staff like

Doe, John (Staff).

b. List important phone numbers as default page of the phone directory

Some important phone numbers and contact information could be presented on

the default page of the phone directory, like Teaching assistants (TA) office

number, Computing resource center (CRC) phone number, IST information desk,

Campus security, Bursars office without using abbreviations for the above. This

could be done in addition to listing it under important phone numbers in Building

Info (Refer Fig.2, Pg XX).

c. Provide help in dialing the extension and finding the office.

The current directory listing provides only the extension number and room

number of the contact person. Only two among the eight people who participated

knew how to use a campus phone or call based on the extension number alone. A

one line help should be displayed that provides instruction to the user on how to

place a call using a campus phone or an external line. Providing some help to the

user for finding the faculty office would be beneficial. The Rush Building is a V-

shaped building with two wings that house offices and classrooms. Letting the

user know which wing to proceed to when they exit out of the stairway, or

elevator could speed up the process of locating it.

Note: Another optional improvement could be including faculty profiles if

someone clicks on the faculty name in the directory search result. This is based


13

on the observation that during the usability test the participants continued to

click on the faculty name hoping to get some more information.

2. Improve visibility and layout of F.A.Q section

Tasks 4 & 12 were related to finding information on advising hours, and advisors.

All four usability test participants made an error in getting to this information.

One of the reasons for this was that the current FAQ section followed the

questions-followed-by-answers format. This seemed to be too much information

too quick worsened by the fact the screen returned to main screen before one

could finish reading. In case of graduate FAQ, this information required two

pages, and the participants tended to miss seeing the link to the next page.

We suggest that the FAQ section default page lists just the questions categorized

as follows:

Q.No. Undergraduate FAQ Graduate FAQ

1 What are the advising walk-in hours? What is the minimum GPA

requirement?

2 Who is my advisor? Who to contact to find out

graduation requirements?

This listing can be ordered based on what the students ask most frequently.

3. Using icons and colors to communicate content/attract user attention

Seven of our eight participants were aware that the information screen was

interactive, but only four of them had actually used it. Among the four users only

one of them had actually used it to look for something. The other three users had

touched it but did not have an idea of what it actually contained.

The fact that they knew that the information screen is interactive is influenced by

the fact that the College sent out an email to the student and faculty to inform

them that it was interactive.

Our interview and usability test elicited suggestions that the information screen

could use customized icons and colors to communicate content quickly even if

viewed from a distance, in this case the entrance. Some of the suggestions were

for motion graphics to attract attention. May be having white/light grey text on

black background could improve readability of the interface.

4. Maximize the use of the information screen during the inert phase (when not

touched)

Interviews and questionnaires with prospective and current users communicated

the need for having some meaningful information displayed on the screen.

Currently the screen shows guest lectures scheduled, and pictures of past events,

which is good but does not exploit the full potential. Displaying list of classes

scheduled for that day, room changes, or even SEPTA train schedules for

commuter students would be good to increase the chances of students/visitors to

use it as an information resource.


14

5. Increase the amount of time a user-selected screen is displayed.

Most of the participants expressed dissatisfaction when the screens kept

returning to the main screen while they were reading the information. This forced

them to search for information all over again. In addition to this, the touch

sensitivity of the touch screen caused the screen to behave unpredictably at

times. The screen touch-sensitivity needs to be adjusted and tested after

studying the pattern of use. Also it would be beneficial if the user-selected

screens are visible for a longer period of time.

6. Change the name iScreen to Information Kiosk

Although the name IST-iScreen may seem catchy, it does not have a significant

impact on the participants. Most of them when asked, what came to their mind

when the word iScreen was used said, “a nice cold treat”. We tested words like

“Information Kiosk” and “Information Screen” which met with significantly higher

approval. All eight of our participants preferred a semantically appropriate word

to describe the screen and its contents.


15

Appendix A

Interview Questions

What program are you in?

How many years have you been in the program?

When you have questions about the College of IST who do you usually ask?

How often do you use the College of IST website to look for information?

Are you familiar with the room numbers in this building?

How would you make a call to one of the faculty offices based on the extension

number?

Did you know the iScreen was interactive? YES NO

Have you used the iScreen before? YES NO

If Yes, what were you looking for? And did you find it?


16

Appendix B

Task List

Kindly complete these tasks using the iScreen

1. Find Scott Silverstein’s office location and phone extension.

2. Find the professor for INFO 435 001 Info Services.

3. Find the hours of operation for the Computing Resource Center.

4. Find advising walk in hours.

5. Find the last day to withdraw from a course.

6. Find the phone number for the Teaching Assistant’s office.

7. Find the events scheduled for this week.

8. Find the Computing Resource Center help desk phone number.

9. Find when the Fall Quarter exams begin.

10. Find the location, days and time the INFO 646 510 IS Management course is

offered.

11. Find Colin Saunders’ office location and phone extension.

12. Find out who to contact for details about faculty advisors.


17

Appendix C

Documentation Form for Usability Testing

1. Find Scott Silverstein’s office location and phone extension.

Success

Path

Notes

2. Find the professor for INFO 435 001 Info Services.

Success

Path

Notes

3. Find the hours of operation for the Computing Resource Center.

Success

Path

Notes

4. Find advising walk in hours.

Success

Path

Notes

5. Find the last day to withdraw from a course.

Success

Path

Notes

6. Find the phone number for the Teaching Assistant’s office.

Success

Path


18

Notes

7. Find the events scheduled for this week.

Success

Path

Notes

8. Find the Computing Resource Center help desk phone number.

Success

Path

Notes

9. Find when the Fall Quarter exams begin.

Success

Path

Notes

10. Find the location, days and time the INFO 646 510 IS Management course is

offered.

Success

Path

Notes

11. Find Colin Saunders’ office location and phone extension.

Success

Path

Notes

12. Find out who to contact for details about faculty advisors.

Success

Path

Notes


19

Appendix D

Questionnaire

Please answer the following questions.

Did you know the iScreen was interactive? YES NO

Have you used the iScreen before today? YES NO

If Yes, what were you looking for? And did you find it?

Please answer the following questions about the iScreen by placing a check

in the appropriate box.

STRONGLY

AGREE

AGREE NEITHER AGREE OR DISAGREE

DISAGREE

STRONGLY DISAGREE

The interface communicated the content found in the system

□

□

□

□

□

Overall, I am satisfied with how easy it is to use this system

□

□

□

□

□

It was simple to use this system □

□

□

□

□

I feel comfortable using this system □

□

□

□

□

It was easy to learn to use this system □

□

□

□

□

Whenever I make a mistake using the system, I recover easily and quickly

□

□

□

□

□

It is easy to find the information I needed □

□

□

□

□

The organization of information on the system screens is clear

□

□

□

□

□

The interface of this system is pleasant □

□

□

□

□

I like using the interface of this system

□

□

□

□

□

This system has all the functions and capabilities I expect it to have

□

□

□

□

□

Overall, I am satisfied with this system

□

□

□

□

□

Using the system for answering some common questions is a good idea

□

□

□

□

□

I felt it took too long to find the information I was looking for

□

□

□

□

□

I would use the system in the future □

□

□

□

□

It is easy to navigate

□ □ □ □ □

The icons/buttons were conveniently located

□ □ □ □ □

It was easy to read the information on the screen

□ □ □ □ □


20

What were things you liked or disliked about the iScreen?

Why or why not would you use the iScreen in the future?

What information did you find useful?

Was there information not included in the iScreen that you may find useful?

What could we have on the iScreen that would catch your attention when you walked

by?

What improvements would you suggest?


21

Appendix E

Questionnaire Responses

ST

RO

NG

LY

A

GR

EE

AG

RE

E

NE

ITH

ER

A

GR

EE

OR

D

ISA

GR

EE

DIS

AG

RE

E

ST

RO

NG

LY

D

ISA

GR

EE

The interface communicated the content found in the system 2 2

Overall, I am satisfied with how easy it is to use this system 1 3

It was simple to use this system 1 1 2

I feel comfortable using this system 1 2 1

It was easy to learn to use this system 3 1

Whenever I make a mistake using the system, I recover easily and quickly 2 2

It is easy to find the information I needed 1 2 1

The organization of information on the system screens is clear 2 2

The interface of this system is pleasant 2 2

I like using the interface of this system 1 2 1

This system has all the functions and capabilities I expect it to have 4

Overall, I am satisfied with this system 1 1 2

Using the system for answering some common questions is a good idea 1 1 2

I felt it took too long to find the information I was looking for 1 2 1

I would use the system in the future 1 1 2

It is easy to navigate 3 1

The icons/buttons were conveniently located 1 2 1

It was easy to read the information on the screen 2 2

Documents

USABILITY REPORT FINALsga72/docs/iScreen-Usability... · 2009-03-09 · INFO 610 George Abraham & Christine Wania 1 I-Screen: USABILITY REPORT Introduction Context The College of