NIDDK WEB SITE U NALYSIS

NIDDK WEB SITE USABILITY ANALYSIS

August 20, 2005

Prepared By:

Bilen Aynu, Rob Fay, & Nneka Okereke INFM 702

User Interaction with Information Systems University of Maryland

College Park, MD 20742

Prepared For:

Roberta Albert Computer Technology Branch

OD/NIDDK/NIH/DHHS Two Democracy Plaza, Room 940

6707 Democracy Boulevard Bethesda, MD 20892-5462

NIDDK Web Site Usability Analysis

Page 2

Executive Summary

Objectives of the Assessment

The NIH National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK)

web team is in the beginning stages of redesigning the Institute’s www.niddk.nih.gov domain.

The NIDDK’s primary motivation for the redesign is to both make the web site more informative

and usable for the customer, but also to introduce upgraded technologies so that the site will be

updated in a more timely fashion using a content management system (CMS). The current

NIDDK site uses static web pages to inform customers.

The purpose of this assessment is to provide objective feedback to the web team and

other interested stakeholders regarding the usability of the current site and to make

recommendations for improvements. Our team conducted an evaluation that includes current

research, a competitive analysis, trusted heuristic ratings, and detailed personas.

Importance of the Assessment

The INFM 702 team believes that a usability analysis will provide some rich feedback to

the NIDDK web team and to the Institute’s stakeholders. Although the NIDDK has collected

some web survey information that polled both the public and the NIDDK staff regarding the

current site, we believe that our evaluation could not happen at a better time. Since proper

information architecture activities and usability studies must take place before a content

management system is implemented, we feel that our evaluation and subsequent

recommendations fill a very vital need.

Key Findings and Recommendations

Although only seven percent of visitors to the NIDDK site focus on the section geared for

researchers, the team feels that the current NIDDK site focuses primarily on these stakeholders


Page 3

and focuses less on the information needs of its general public stakeholders. The team has found

that the site is very text-intensive and does not seem to support readability at a sixth grade level.

Fortunately, the preponderance of text makes for quick page load times and makes for easy

viewing on mobile devices. However, all reports are presented in PDF format and do not

provide alternate viewing options such as viewing them as a web page.

The INFM 702 team first recommends that the NIDDK accept and use the personas

included in this deliverable. The NIDDK design team needs to have each stakeholder in mind

when approaching the redesign. In addition, we make a bold recommendation: The NIDDK

needs to include in its mission a responsibility for informing the general public. We’re confident

this is already generally accepted, but the team argues that expanding its mission will allow the

NIDDK to better respond to the increasing information needs of its customers.

Practically, the INFM 702 team recommends that the NIDDK a) provide a consistent

interface across NIDDK pages, including the health information and intramural sites, b) better

organize the text elements using easily understood bite-sized words, c) make better use of white

space, and d) break up text with images, graphics, and media that might more easily

communicate important information to all consumers of the site. Additionally, we recommend

that the NIDDK web team refer to our competitive analysis, using the NCI (www.cancer.gov)

and NSF (www.nsf.gov) sites as best of breed models.

Finally, since the Institute is moving towards implementing a content management

system, we feel that now is the perfect time to a) introduce content ownership so that content

owner names and dates posted are provided with available content and to b) develop rich

metadata elements to aid the consumer in “findability” – through better search (taxonomies, best

bets, thesauri) and navigation (labeling, layouts, topics, etc.).


Page 4

Table of Contents

Executive Summary ...............................................................................................................2 Objectives of the Assessment..........................................................................................2 Importance of the Assessment.........................................................................................2 Key Findings and Recommendations..............................................................................2

Table of Contents...................................................................................................................4 Introduction............................................................................................................................5

Background......................................................................................................................5 Importance and Context of the General Topic ................................................................6 Overview/Current Knowledge of the Topic ....................................................................7 Assessment Objectives ....................................................................................................9 Definition of Key Terms .................................................................................................10 Thesis/Point of View .......................................................................................................10

Evaluation of the Usability of the NIDDK Web Site ............................................................11

Description of the Research Issue ...................................................................................11 Method of Data Collection Approach for Conducting the Evaluation............................13 Underlying Assumptions .................................................................................................15 Analysis Techniques Used and Justification ...................................................................15 Development and Pre-testing of Data Collection Instruments ........................................17 Data Collection, Analysis, and Findings .........................................................................18

Heuristic Evaluations..................................................................................................18 Nielsen’s 10 Usability Heuristics .........................................................................20 Shneiderman’s 8 Golden Rules of Interface Design ............................................20 Guidelines for Usability Based on the 4 Learning Styles ....................................20

Persona Development .................................................................................................21 Competitive Analysis..................................................................................................28

Summary/Conclusion.............................................................................................................32

Findings and Conclusions................................................................................................32 Identified Limitations of the Study..................................................................................33 Recommendations ...........................................................................................................33 Future Research Directions .............................................................................................35

References..............................................................................................................................36 Appendices.............................................................................................................................38

Appendix 1: Nielsen’s 10 Usability Heuristics ...............................................................38 Appendix 2: Shneiderman’s 8 Golden Rules of Interface Design ..................................40 Appendix 3: Guidelines for Usability Based on the 4 Learning Styles...........................41 Appendix 4: Competitive Analysis .................................................................................42 Appendix 5: NIDDK Web Site Main Pages....................................................................44


Page 5

Introduction

Background

For over one hundred years, the National Institutes of Health (NIH) has played an

important role in improving the health of the nation. There are currently 27 institutes and centers

providing leadership and financial support to researchers throughout the nation and the world.

With over 17,000 employees, the NIH funds over 28 billion annually in research. Of the

Congressional funding it receives, the NIH uses approximately 80% to support publicly funded

research (extramural research) and 10% is used to support NIH employee-conducted research

(intramural research).

The mission of the National Institute of Diabetes and Digestive and Kidney Diseases

(NIDDK) is to “conduct and support research on many of the most serious diseases affecting

public health” (http://www.niddk.nih.gov/welcome/mission.htm). Much of the Institute’s

research focuses on medical topics including kidney disease, diabetes, digestive diseases, and

nutrition and obesity. In addition, according to its web site, the NIDDK

…supports much of the clinical research on the diseases of internal medicine and

related subspecialty fields as well as many basic science disciplines. The NIDDK

supports basic and clinical research through investigator-initiated grants, program

project and center grants, and career development and training awards. The

Institute also supports research and development projects and large-scale clinical

trials through contracts. (http://www.niddk.nih.gov/welcome/mission.htm)


Page 6

Importance and Context of the General Topic

The NIDDK is currently planning a redesign of its www.niddk.nih.gov domain (see

Appendix for NIDDK main web page images). The primary motivator for the redesign is to both

make the web site more informative and usable for the consumer, but also to introduce upgraded

technologies so that the site will be updated in a more timely fashion using a content

management system (CMS). The current NIDDK site uses static web pages to inform

consumers.

The reason our team decided to conduct a usability evaluation of the NIDDK site was

because one of the INFM 702 team members was approached by the NIDDK webmaster to act as

an information architect resource for the redesign project. As a result, this team member

approached the webmaster to see if the team could conduct a usability analysis while the

redesign was in its initial stages. Our team hopes that this analysis can provide some substantive

feedback to the NIDDK redesign team as they choose a content management tool and prepare a

new design. Due to time restrictions, our team’s evaluation focused on the site’s overall

usability characteristics – we primarily evaluated the high level pages of the site.

According to the NIDDK web site redesign committee, initially, the project will focus on

the “Research Funding Opportunities” portion of the site, the NIDDK Home page, and the “look

and feel” of the entire web site. The web site “look and feel” redesign will apply to the entire

niddk.nih.gov domain regardless of whether or not the content has been revised. A great deal of

focus will be placed on the design phase of this project to guarantee that the new site is easy to

navigate, and information is easy to read and find. In addition, a major requirement for the

selected “Interactive Publishing” tool is that it be user friendly and intuitive to use. This will

reduce the learning curve for contributors and limit the number of helpdesk calls.


Page 7

Overview/Current Knowledge of the Topic

There are no clear guidelines for evaluating the usability of health information web sites.

The increasing number and variety of user types who are using the web to access health

information make the challenge more complex. In her Washington Post article, Agnvall (2005)

stated that consumers in greater numbers are turning to the Internet for health information. The

article looked at research released in May 2005 by the Pew Internet & American Life Project

found that about 95 million American adults use the Internet to find health information. Pew

research also previously found that nearly a fifth of Web seekers say they have gone online to

diagnose or treat a medical condition without consulting a doctor. These users don’t like to be

bogged down in detail, which is one reason why the average user was attracted to commercial

sites such as MayoClinic.com, because it presented information in a manageable way. On the

other hand, non-commercial sites such as the National Institutes of Health, presented an

overwhelming amount of information. Since the information may not be presented in a

manageable way, consumers may feel too bogged down in detail.

Some research readily exists that evaluates the effectiveness and usability of health

information sites. For example, Consumer WebWatch issued a report in 2002 stating that

consumers did not evaluate the credibility of Web sites using rigorous criteria when assessing a

Web site's credibility. Instead, consumers focused on superficial criteria, such as attractiveness

of site design (Forster, 2002).

Furthermore, Sillence, Briggs, and Fishwick (2005) go on to indicate that medical

information sites are often “heavily jargon-laden” and difficult to read (p. 1026). The results of

their study found that customers were heavily influenced by the look and feel of a site (83%) and


Page 8

by the degree of personalization of a site. The authors provided the following three

recommendations:

• Guideline 1 – make the purpose of the site clear;

• Guideline 2 – allow personalization, tailored experience; and

• Guideline 3 – include markers of social identity (Sillence, Briggs, and

Fishwick, 2005, p. 1026).

Siegel and Wood (2003) offer insight into the National Library of Medicine’s (NLM)

web strategy, including how this strategy has radically changed over the last 7 years. Originally,

the NLM focused on “meeting the information needs of health practitioners, researchers, and

librarians” (2003, p. 227). More recently, perhaps influenced in part by e-government

legislation, the National Library of Medicine expanded its mission to include focusing on the

information needs of the general public, including “patients, families and friends of patients,

students, and the general health consumer” (2003, p.227). Consequently, it became much more

important for the Institute to critically evaluate how they make information available to all its

stakeholders via the Internet, so they created and recommend a multidimensional approach to

web evaluation that includes usability testing, gathering user feedback, collecting web usage

data, collecting web performance data, and initiating outreach projects to gain qualitative

feedback on the usability of a site.

The usability field overall is maturing in the area of federal health web sites. The

National Cancer Institute developed the "Research-Based Web Design and Usability Guidelines"

to provide guidance to other federal agencies (Barr, 2003). The 187-page guideline covers issues

such as accessibility, home page design, site navigation, writing, graphics and content

organization. Each guideline comes with a score indicating the "strength of evidence" behind the


Page 9

recommendation and a score for "relative importance" to help Internet developers make

judgments about competing priorities (Barr, 2003).

Assessment Objectives

The INFM 702 team believes that a usability analysis will provide some rich feedback to

the NIDDK web team and to the Institute’s stakeholders. One piece of information the INFM

702 team received early on was a NIDDK web usage report that calculated the average monthly

page visits. Results indicated that eighty percent of customers visit the NIDDK site to find

health information. On the other hand, only seven percent of customers visit the site to find

research funding information. These contrasting statistics seem very much at odds with the

NIDDK mission to both conduct and support research. Therefore, we feel that the results of our

assessment might encourage the NIDDK to rethink its current mission (conduct and support

research) to include a web site specific mission that focuses on all of its consumers of

information.

Secondly, we feel that providing a third party objective analysis will be extremely

beneficial since two of the members of the INFM 702 team have no affiliation with the NIDDK,

do not have a deep understanding of what the Institute does, and can provide feedback from the

general public’s perspective.

In addition to the benefit of providing objective third party feedback, the team will

provide research-based findings that can be used to increase the usability of the redesigned site.

Since a majority of the redesign participants are scientists and Ph.D.s (in a medical or scientific

field), our team feels that providing research findings will lend a certain level of credibility to

our findings and recommendations.


Page 10

Definitions of Key Terms

• B2C – Business-to-Consumer services including information or financial data exchange through an information system.

• CMS – Content Management System • Competitive Analysis – The process of looking at competitive or similar organizations to

see how the usability of their web sites compares with the NIDDK web site. • Extramural Research – This refers to the non-NIH research community that receives

federal funds (grants, contracts) to conduct health-related research or provide related services.

• FAQ – Frequently Asked Questions • HCI – Human-Computer Interaction; the study of how humans interact with computers,

primarily studying the effects of how easily a person can use a given information system. • ICGI - Interagency Committee on Government Information; established as a result of the

E-Government Act of 2002. • Information Architecture – The process or method of organizing information to make it

easier for people to find and use. • Institute – One of the 27 NIH entities that supports health-related research according to

its specific science or support-related mission. • Intramural Research – This refers to the NIH employees that work for specific Institutes

to conduct research related to the mission of their Institute. • NICHD – National Institute of Child Health and Human Development • NIDDK – The National Institute of Diabetes and Digestive and Kidney Diseases • NIH – The National Institutes of Health (NIH), a part of the U.S. Department of Health

and Human Services, is the primary Federal agency for conducting and supporting medical research.

• NLM – National Library of Medicine; one of the NIH institutes • NSF – National Science Foundation; not part of NIH but provides federal grant funding

for scientific research • OMB – Office of Management and Budget • RFA – Request for Application; NIH mechanism for asking for submissions for specific

research funding initiatives. • Stakeholder – Any individual or group that has a vested interest in the NIDDK web site.

This includes the general public (taxpayer), Congress, the research community, NIDDK and NIH leadership and oversight, and NIDDK employees and affiliated contractors.

• Usability – A qualitative and/or quantitative measurement to describe how well a consumer can find and use information from an information system, and their likelihood of revisiting the system to fulfill their future information needs.

Thesis/Point of View

The INFM 702 team understands that the mission of the NIDDK is primarily to conduct

and support medical research as it relates to issues including diabetes, kidney diseases, digestive


Page 11

diseases, and nutrition. With this in mind, it seems that the current web site is usable for some

stakeholders but not for others.

Our team believes that in order to make a web site more useful to all stakeholders, it is

vital to conduct an evaluation that provides the NIDDK web redesign team and related

stakeholders with current usability research, a competitive analysis, trusted heuristic ratings, and

detailed personas. Clearly, the redesigned web site must reflect, not only the mission of the

Institute, but it must provide customer satisfaction to all stakeholders, not only specific

audiences.

Evaluation of Usability of the NIDDK Web Site

Description of the Research Issue

The primary mission of the NIH (and the NIDDK) is to provide funding for medical

research. Approximately 80% of the NIH’s Congressionally allocated funds are used to finance

extramural research and related services, whereas approximately 10% supports intramural

research. Although the Institute’s mission emphasizes research funding, its web presence needs

to embrace the larger mission of informing every stakeholder.

Traditional web usability studies generally focus on commercial B2C sites. The primary

reason for this is because commercial entities have a strong interest in making sure they can

maximize their profits by providing customers with a very intuitive and pleasing experience

using their information system. Not-for-profit sites, however, often receive less research

attention with regard to usability. One explanation for this lack of research may be because a

non-profit’s goods and services are not offered to the consumer based on a business model of

maximizing profit. In addition, these organizations often do not have substantial human or

financial capital resources to ensure that their web presence has high usability. Recent


Page 12

legislation, however, has recognized the need for the general public to be able to easily access

government information via the Internet.

For example, on December 17, 1999, President Clinton released "Electronic

Government," a memorandum for the heads of executive departments and agencies. The

directive of this memorandum was for executive departments and agencies, in conjunction with

the private sector as appropriate, to help citizens gain one-stop access to existing government

information and services, and to provide better, more efficient, government services and

increased government accountability. The President felt that:

• The public should be able to access government information by the type of service or

information that people may be seeking; and

• Providing information at an agency level might be burdensome since the public would

need to know which agency provides the service he or she needs.

In addition, the E-Government Act of 2002 established the Interagency Committee on

Government Information (ICGI). As a result, the ICGI formed a Content Management

Workgroup to make recommendations on issues related to federal public web sites (ICGA,

2004). The ICGA recommendations include:

1. Citizens must be able to identify official government web sites and trust that those web

sites will provide current and accurate government information;

2. Federal public web sites must be written and organized from the audiences’ point of

view;

3. Federal public web sites must be designed and written to ensure they are easy to access

and use;

4. To promote seamless government, federal organizations must work to simplify and unify


Page 13

information across the government;

5. Federal organizations must establish priorities and a schedule for posting content on their

public web sites;

6. Organizations must continue to comply with existing federal laws, regulations, and

policies; and

7. Developing web content policies and requirements for federal public web sites is an

ongoing process, requiring structure (ICGA, 2004, p. 1)

As a result of this and other government initiatives, government web sites in particular

are being held more accountable for the information they present, including how easily

information can be accessed by the general public.

Method and Data Collection Approach for Conducting the Evaluation

In addition to reviewing relevant literature, the INFM 702 team used the following three

processes to conduct the evaluation of the NIDDK web site.

1. Heuristic Evaluations

2. Persona Development

3. Competitive Analysis

The team conducted three heuristic evaluations that sought to quantify subjective

evaluations using specific criteria. Our first two evaluations were based on established heuristics

created by Jakob Nielsen (Nielsen, 2000) and Ben Shneiderman (Shneiderman & Plaisant, 2004).

We found limitations in the first two standards and decided to create our own third heuristic

based on the Four Learning Styles (Jester, 2000). All heuristic criteria were rated by all three

members of the team using a 1 (poor) to 5 (excellent) scale.


Page 14

Second, we created detailed personas based on the stakeholders that we felt the site was

visited by. We decided to create personas that reflected the web page traffic data provided by the

NIDDK:

Table 1.0 – NIDDK Unique Web Page Visits Per Month

We quickly noticed that out of the 1.5 million average monthly page visits, approximately eighty

percent of customers seek out health information. By contrast, only seven percent seek out

research funding information. In addition, even fewer people visit the other sections of the site,

including the intramural research pages, the reports, and the clinical trials information pages.

The final approach to our methodology included conducting a competitive analysis of

similar web sites. We felt that by seeing the good, the bad, and the ugly of similar sites, we

might therefore be able to provide some rich information to the NIDDK regarding best practices

available to them. We decided to briefly review three sites using the Nielsen heuristic: the

National Cancer Institute (NCI), the National Institute of Child Health and Human Development

(NICHD), and the National Science Foundation (NSF). The first two sites are sister NIH


Page 15

Institutes and the latter is a peer site that provides a similar function of conducting and

supporting research related to its science focus.

Underlying Assumptions

Our team stepped into this project with the assumption that we could provide feedback to

the NIDDK, unhindered by any limitations. Two of the team members approached this

evaluation knowing little about the National Institutes of Health (NIH) or its mission, so we felt

that we could therefore be able to provide a relatively objective evaluation, specifically

representing the views of the general public stakeholder.

The NIDDK wisely set up an oversight committee to oversee its redesign effort. This

committee established both a redesign subgroup and a content subgroup with the expressed

intent to focus on issues of technology selections for the new site and to focus on how to

organize, present and manage content. Although some thought has been given to the information

architecture of the site, our team feels that the committee members may be less informed

regarding standards for web design, usability, and user experience.

Analysis Techniques Used and Justification

Heuristic Evaluations

The team adapted the underlying criteria of all of the heuristics by adding specific

descriptors to them. The purpose of adding detailed and relevant descriptions for the heuristics

was so that we could provide a richer NIDDK evaluation. Nielsen (1996) recommends that

multiple persons review the same site for usability. Therefore, all three team members evaluated

the NIDDK web site using the three heuristics.


Page 16

The first heuristic the group used was Jakob Nielsen’s 10 Heuristics of Usability

(Nielsen, 2000). Jakob Nielsen is an acknowledged expert in information system usability,

supported by his extensive collection of research on the issue. His ten heuristics provide a

common standard for measuring the usability of the NIDDK web site. Second, the team decided

to use Ben Shneiderman’s heuristic entitled “The 8 Golden Rules of Interface Design” for our

evaluation (Shneiderman & Plaisant, 2004). Since Dr. Shneiderman is considered an expert in

Human Computer Interaction (HCI) research, the group felt that his rules of thumb would

provide us with additional criteria with which to evaluate the NIDDK site.

Our team determined that there were some limitations to the previous two heuristics.

Both focus on information access and usability, but neither specifically focuses on the learning

styles of individuals. Universal standards of usability may not translate to all persons if these

people learn differently. Therefore, in order to address this deficiency, the group decided to

adapt the 4 Learning Styles (Jester, 2000) into a new usability heuristic. This third heuristic will

be described in greater detail in the next section.

Personas

In order to develop consistent and relevant personas, the group agreed upon certain

criteria to create attributes and characteristics that need to be included while developing the

personas. Each group member created at least two personas, with a total of seven personas.

Competitive Analysis

Another way to make sure that the redesigned site is as good as it can be is to look at

other related sites to find best practices. The team picked three related peer web sites to conduct

a competitive analysis. Each group member did an evaluation of the usability of the web site

compared to the NIDDK and submitted findings. Ratings were based on Nielsen’s heuristic.


Page 17

Development and Pre-testing of Data Collection Instruments

A heuristic evaluation is the most general of the usability inspection methods and is also

the easiest to learn and apply. Unfortunately, heuristic evaluation itself does not provide a

systematic way to generate fixes to the usability problems or provide a way to assess the

probable quality of any redesigns. However, because heuristic evaluations aim at explaining

each observed usability problem with reference to established usability principles, it will often be

fairly easy to generate a revised design according to the guidelines provided by the violated

principle for good interactive systems. Also, many usability problems have fairly obvious fixes

as soon as they have been identified (Nielsen, 2000).

In addition to reviewing NIDDK-provided survey results of NIDDK employees and the

general public’s opinion of the current site, the team applied three usability heuristics to the

NIDDK web site. The main purpose of conducting the web site usability assessment using the

three heuristics is to provide an objective third party evaluations and recommendations to the

NIDDK redesign committee. The three heuristics are:

1. Jakob Nielsen’s 10 Heuristics for Web site evaluation

2. Ben Shneiderman’s 8 Golden Rules for Interface Design

3. Guidelines for Usability Based on the 4 Learning Styles

The first two heuristics the team used were already developed. The team developed a

third heuristic using the 4 learning styles (Jester, 2000). The reason the first two heuristics were

not sufficient is because the two focus on information access and usability but do not consider

learning styles of customers. Since the NIDDK web site users have varying levels of expertise in

the topic on which information is sought; they also vary in their learning styles. The task of

searching for information has very different requirements to those who are looking for health


Page 18

information and for those who are looking for research grants (from an individual's perspective).

Just as the team did for the previous two heuristics, our adaptation of the 4 Learning Styles

meant that we added our own descriptions for each criterion. Please see the Appendix for

specific descriptions. Jester (2000) defines the 4 Learning Styles as:

• Visual/Verbal Learning Style – This person learns best when information is

presented visually and in a written language format. This learner likes to write down information

that is seen, often in summary fashion, as a way to aid retention.

• Visual/Nonverbal Learning Style – This person learns best when information is

presented visually and in a picture or design format. This person likes to see images, charts,

graphs and diagrams to better visualize the information that is being represented. This learner

may draw pictures or color code test as a way to aid retention.

• Tactile/Kinesthetic – This person learns best when physically engaged in a "hands

on" activity. This person is an experiential learner who often uses flashcards and “experiences”

information using alternate media choices (visual, audio) to aid retention.

• Auditory/Verbal Learner – This person learns best when information is presented

auditory in an oral language format. This person benefits from speaking to and listening to

others and audio files or using a screen reader would greatly aid this learner’s retention of the

presented information.

Data Collection, Analysis & Findings

Heuristic Evaluations

The main data collection instrument used was a heuristic evaluation data collection table.

This table included 2 – 5 descriptions of the evaluation criterion for each heuristic used (see

Appendix to see specific evaluation criteria). The web sites were evaluated in terms of each


Page 19

description using a range from 1 to 5. The rating used was 1 (poor); 2 (needs improvement); 3

(good); 4 (better than average); 5 (excellent). Table 2 provides a representative sample of one

criteria selected, along with specific descriptors of each. Each member of the team evaluated the

site using these descriptors.

Table 2.0 - Heuristic Evaluation Data Collection Table

The data collection table also had columns for subjective comments and recommendations for

criteria that scored below three. Completed data collection tables are found in the appendix. Our

team performed a quantitative data analysis by averaging the three individual ratings for each

description of each heuristic and then averaging the overall rating for the heuristic. In addition,

we performed a qualitative data analysis by compiling comments for criteria that scored below

three (these comments can be found in detailed tables, located in the Appendix). Below are the

average ratings for the three heuristics.


Page 20

Table 3.0 - Nielsen’s 10 Usability Heuristics

The overall average rating for Nielsen’s 10 Heuristics for the NIDDK web site was 3.30

using a rating scale of 1 (poor) to 5 (excellent).

Table 4.0 - Shneiderman’s Eight Golden Rules of Interface Design

The overall average rating for Shneiderman’s Eight Golden Rules for the NIDDK web

site was 3.95 using a rating scale of 1 (poor) to 5 (excellent).

Table 5.0 - Guidelines for Usability Based on the Four Learning Styles


Page 21

The overall average rating for Guidelines for Usability Based on the Four Learning Styles

for the NIDDK web site was 2.33 using a rating scale of 1 (poor) to 5 (excellent).

Persona Development

In addition to providing both qualitative and quantitative heuristic evaluations, our group

created personas that would reflect that customers served by the NIDDK web site. It is widely

regarded that if representative personas can be created and used, developers are more likely to

create a usable site for all customers. Seven personas consisting of names, photos, and brief bios

were created so that the NIDDK could redesign its site with specific customers in mind. The

personas represents behavior patterns that will help the NIDDK redesign committee understand

the goals, motivations, and behaviors of the people who will use the NIDDK web site and how

the interface will fit into meeting those needs. The purpose of this exercise was to help explore

the NIDDK web site’s target customers. One current concern is that the primary members of the

web redesign working groups might focus on specific audiences such as users who are looking

for research funding rather than considering the many types of people who might visit the site for

information. However, our team feels that if these personas are considered, the redesign team

might successfully make the site usable for all customers. Below is a listing of the seven created

personas:

Dr. Steven Arias

Dr. Arias is in desperate need of a grant through RFA. He is 50 years

old and is worried about losing his job. He has already had to fire his

technician. Dr. Arias has tried for RFA’s in the past, but has been unscored


Page 22

three times. He needs to speak to someone in his field, and find other initiatives he can apply

for. Before applying for funding again. Dr. Arias hopes he can get in touch with one of the

NIDDK program staff to get some ideas on how he might improve his application.

Not knowing much about Internet search, he used google.com and stumbled onto the

NIDDK web site. On the site, he first looked at the FAQs to check for existing solutions. Then

he viewed the site map and clicked the big headings for clues. Finally, he used the contact form

to submit his inquiry for assistance.

Dr. Arias has a x486 pc running Windows NT. Since he is completing his sabbatical, he

is working from a remote location and only has dial up access. He has not spent much time

“surfing” the web. He has little patience, due to his desperate situation. He has to be able to get

answers to his questions quickly.

Dr. Arias lives in Atlanta, Georgia with his wife of 30 years Susan. He loves to play golf

in his spare time.

Dr. Helmut Heisenberg

Dr. Heisenberg is in his 60’s, with significant experience in cancer

research, has spent the last 20 years at NCI. Dr. Heisenberg is well written,

having published four papers in the last year. He is a member of the National

Academy of Sciences.

During his recent research, Dr. Heisenberg made an interesting finding that he feels may

apply to diabetes research. He has no idea what areas in diabetes are “hot” topics now, so he

needs guidance.

Dr. Heisenberg would like to hire three new post-docs to work with him in this new area.


Page 23

The doctor normally has his team do his Internet research for him, so his web skills are not

strong. But, he has the ambition and desire to find what he is looking for. He uses a Macintosh

running OS X at his office on the NIH Campus in Bethesda, MD.

Dr. Heisenberg lives in Bethesda, Maryland with his wife Marilyn. Both have had what

could be considered by all means a good marriage. And in their 35 years of marriage, they have

traveled to over 30 countries throughout the world.

Because of his longstanding career with the NIH, Dr. Heisenberg knows that the NIDDK

web site must exist, but he is not particularly skilled in Internet searching. However, his

enthusiasm about his new discovery would not allow him wait for his team to do the search for

him. He spends part of his time looking at the NIDDK web site from his office, using a high-

speed connection on his Macintosh.

He wants to browse the NIDDK web site to look for new research findings in Diabetes.

On the NIDDK web site, he is first drawn to the following headings and functions: Diabetes,

What’s New, Highlights, and the search function. He is primarily interested in seeing the

recency of information found. He would desire graphic representations of certain things.

Although Dr. Heisenberg is still very strong and agile, his vision has started to fail and so he

would most certainly prefer the texts in bold formats with well-contrasted colors. And because

of his limitations in the use of the internet, he will be very frustrated if he has to click several

times deeper in other to get specific information.


Page 24

Seth Warren

Mr. Warren works in the construction business out in a rural setting

about 50 miles west of Chicago, IL. He is 37 years old, married to his high

school sweetheart, Lisa, and has two children (Daniel, 10, Becky, 8). He

stumbled across the NIDDK site after searching for “nih diabetes obesity” on the Yahoo! Search

page. He is particularly interested to learn more about diabetes and its affects on the elderly

since his mother now lives with the family due to complications with diabetes.

Mr. Warren doesn’t currently have a computer at home, although his children would like

to get a new computer. Instead, he must travel to the nearby county library to access a Pentium

II PC with Windows 98 with a high-speed connection. He spends very little time on the Internet,

generally limiting it to checking email once a week or so.

Dr. Elizabeth Marquette

Dr. Marquette is 42 years old and has been with the NIDDK as a

program staffer for 6 years. She has been with the Institute for 10 years,

previously working in the Review section. Dr. Marquette is a single mother

of two, commuting to her Bethesda office from Frederick, Maryland.

She spends a great deal of time on the NIDDK web site, but needs to

find information very quickly. Her day is fast-paced to say the least.

During her research on the NIDDK web site, she needs to be able to find out who is the

Urology PD, for example. She can never remember the P01 cap, so this information needs to be


Page 25

readily available. Also, she knows that there is something new having to do with animal sharing,

but has no idea where this information is located.

Because Dr. Marquette is often on the go either attending meetings and conferences or

tending to her parental responsibilities, she tends to spend some of her available time responding

to emails and doing some web research using her Blackberry and mobile phone. In addition,

because of occasional mobile interference and the very small screen size, she wants to be able to

locate needed information quickly. Dr. Marquette is quite web-savvy, with a background in

designing a previous version of the NIDDK web site.

Jenny Spinova

Jenny is a fifth grader with two younger brothers (Jaime -

6, Elias – 4). Her parents were killed on 9/11, so she and her

brothers now live with her grandmother (mom’s side) –

Hermosa, who is 67 years old. They live in Hoboken, New

Jersey.

Jenny is a very bright student. She is currently enrolled in a magnet school where she

excels in science and math and she enjoys spending time in the computer lab. She primarily

speaks Spanish at home with her grandmother, but she is fluent in both Spanish and English.

Since it is national nutrition month, Jenny’s teacher has asked that her students write a

report on steps that both old and young can take to remain healthy. Since Jenny’s grandfather

(dad’s side) works for the NIDDK, he suggested she look at the site to find some good

information. Unfortunately, Jenny’s grandfather does not use the site too much, but he knows


Page 26

she must be able to find some information. He is at least able to give her the address of the home

page.

Fortunately, Jenny is relatively skilled at navigating the Internet, so she finds the site

while at school, using a pc with Windows XP and a high-speed connection. She does not know

how to use the search function, but instead, tends to browse.

Unfortunately, the home page has lots of text with little graphics, so she becomes

somewhat intimidated by all the links. She decides to click through a couple of links, but finds

that some of the words and abbreviations are confusing. She eventually finds some pages in

Spanish, but they appear to primarily provide information on food recipes. She eventually gives

up her search.

Sally Smith

Sally Smith, originally from the DC metro area, is a 25 year old

graduate student at the University of Pennsylvania, studying the effects of

diabetes on kidney function.

Since she will be graduating in May, Sally needs to find information on pre-doctoral

scholarships and where to find the pre-doctoral scholarship application forms. She logs on to the

National Institutes of Health web site (http://www.nih.gov) and then looks under the grants

section (http://grants1.nih.gov/grants). She then finds the link for the NIDDK web site.

Sally is overwhelmed by this process and would really like to talk to someone in person.

Up to this point, she doesn’t have a mentor to guide her. Sally’s goal is to make sure she has all

the right information to complete the scholarship application accurately. After Sally downloads


Page 27

the scholarship application, she will be looking for contact information so she can reach some

one in person.

Sally’s use of the Internet is second nature; she is never more comfortable than with a

mouse in her hand. She has a high-speed cable connection and uses a Mac.

Joann Bentley, Ph.D.

Dr. Joann Bentley is a 32 year old female that has just been hired as

an Assistant Professor of Biology at University of Maryland, College Park.

She found out about the NIDDK web site through her colleagues and she

needs to find information on appropriate grants, and really doesn’t know

where to begin. “What is NIDDK looking for?” is the question that runs through her mind

constantly as she searches on the web site.

Dr. Bentley needs to be directed to information on career awards, including eligibility,

DK-specific information, application forms and deadlines. She needs basic research grant

information, but doesn’t know that is what it’s called. She also needs to find a current list of

funding initiatives. Dr. Bentley is confused because she is not familiar with the terminology that

she is finding on the NIDDK web site. Quite frankly, she is unfamiliar with the whole federal

grants process, and is more uninformed about the NIH grants process.

Dr. Bentley feels confident with her web navigation abilities; she uses the computer in

her office that is a high-speed Windows machine to do this research. This is the first time she

has needed to find these kinds of information, and may be more comfortable if there was a


Page 28

person she could talk to directly. She has no idea how to contact someone who can guide her in

the right direction.

Competitive Analysis

In addition to providing the three heuristic evaluations and creating seven relevant

personas, the team felt it would be useful to conduct a competitive analysis. Federal agencies

can often become insulated, not knowing what other agencies may be doing or keeping up with

current best practices in web development. Therefore, our team looked at three sister sites to see

if there were best practices that might be followed.

Our competitive analysis essentially looked at the high-level usability issues with each

site, rating each using the Nielsen heuristic applied earlier to the NIDDK site. Our team

provided usability scores (as shown on Chart 1 below) for three federal government web sites

and contrasted these evaluations with the previously rated NIDDK evaluation (see the Appendix

for more detailed quantitative score attributes). The National Institute of Health and Child

Development (NIHCD) and the National Cancer institute (NCI) are other NIH Institutes and the

National Science Foundation (NSF) is a peer agency that also conducts and supports scientific

research.

Chart 1.0 - Competitive Analysis


Page 29

Due to the limitations in time, the competitive analysis evaluations were not very

detailed. Only a first level analysis which is meant to draw a quick comparison to that of the

NIDDK web site was done. Therefore only a quick overview of the comparison is presented

below:

The National Institute of Health and Child Development (NIHCD)

- Unlike the NIDDK and its inconsistent navigation (specifically between the main pages,

the intramural site, and the health information sites), the user can identify visibly where

he or she is on each page.

- The styles, fonts and headings were standard from page to page.


Page 30

- The logo, contact information, and links to the home page are in the same location across

each page.

- The site map is well defined and detailed information is provided.

The NIHCD web site probably included the least amount of information on its site, but its

simplicity was probably its greatest strength. The site was rated slightly higher than NIDDK,

primarily because it used consistent navigation items throughout the site. The site also reflected

Nielsen’s standard in terms of aesthetic and minimalist design. Overall, this site was rated

slightly above average, on par with the current NIDDK site.

The National Cancer institute (NCI)

- Unlike the NIDDK site the NCI site had standardized task sequences


Page 31

- The NCI site ensured that embedded links are descriptive. The NIDDK site had no

descriptions on any of its links on the homepage.

- The NCI site used unique and descriptive headings

- The NCI site made good use of thumbnail images to preview larger images.

Unfortunately, the NIDDK site homepage did not have any images to break up text other

than its logo.

Besides being an award winning web site for usability, the NCI site rated extremely high,

in fact higher than all other compared sites, using Nielsen’s 10 heuristics. The NCI web site also

certainly made better use of powerful images and pictures to aid users’ recognition and

understanding of the NCI’s mission. It was also perceived that the NCI site probably employed

more detailed personas by the simple designs, pictures of families etc. Unfortunately, the NCI

site, like all the rest compared, did not pass the W3C validation test.

The National Science Foundation (NSF)


Page 32

- The NSF site has a search function that allows users to search by various content types

such as a) news, b) discoveries, c) funding, d) publications, and e) awards. Every other

site that the team evaluated did not employ this method of search.

- The NSF site has consistent navigation, as well as look and feel throughout the site.

- The site uses breadcrumbs very well so users know where they are at all times.

- The site allows users to search for information by subject area, by NSF division and by

customer type. This site was very flexible in its approach to providing multiple ways for

users to locate information.

Overall, the NSF site uses easy to understand terminology and provides a great mix of

text, graphics, and media for different learning types. This site took great pains to make sure that

information appealed to various learning styles, was accessible by various stakeholder types, and

was easy to navigate. This site also had a substantial help section. We would consider this to be

a model site for the NIDDK redesign team to consider.

Summary/ Conclusion

Findings and Conclusions

The overall findings of our usability evaluation indicate that the current site was rated

better than average in terms of its usability. However, our team feels that the site could be

redesigned for even greater usability. One of our key findings suggests that the NIDDK site may

have been unintentionally too focused on its research funding audience, even though these

people only constituted approximately seven percent of the site’s visitors.

Therefore, our team feels that the redesign must focus on each stakeholder equally,

considering the learning styles and information needs of each customer.


Page 33

Identified Limitations of the Study

The primary limitation of this project was that our team only had six weeks to conduct

this usability study. As a result, only three persons conducted the analysis without having the

time to conduct a formal usability study (usability lab, videotape, etc.). In addition, personas

were created without the convenience of being able to interview persons to identify persona

types. The seven personas were created as a result of both NIDDK staff feedback and the team’s

own ideas of the Institute’s target customers.

Another limitation resulting from the short time frame was that the INFM 702 team only

had time to review high-level pages. There simply was little time to look into deeper levels of

the site.

Finally, the team was provided with results of a public web survey of the NIDDK site.

This survey provided an overall customer satisfaction score that was very high – even higher

when compared to customer satisfaction with NIH sites as a whole as well as federal government

sites as a whole. Time again limited us because we simply did not have time to determine why

our usability results seemed to conflict with the public web survey results (public responded with

83% satisfaction with NIDDK site – better than NIH or federal sites in general).

Recommendations

As a result of this usability assessment, our team has a few recommendations to make the

NIDDK site better. We feel that these recommendations could not happen at a better time since

the NIDDK is in the beginning stages of its site redesign process and good information

architecture must precede the implementation of a content management system.

Our first recommendation simply asks the NIDDK to consider using some of the results

of our usability study. For instance, we used initial feedback from NIDDK staff and created


Page 34

detailed personas, representative of the various stakeholders that might visit the NIDDK site.

We ask that these personas be referred to frequently when attending both content and redesign

subcommittee meetings. It is only by this constant reminder that the NIDDK redesign staff

makes sure that the redesigned site meets its usability objectives for its intended audience.

Our team purposefully limited the number of NIDDK staff personas and did not consider

the results of a survey of NIDDK employees, simply because the ICGI recommends that Internet

sites are for the public, not for employees (Interagency Commission on Government Information,

2004). Instead, the NIDDK should consider making its Intranet presence much more robust to

meet the needs of its employees.

In general, the INFM 702 team recommends the following Information Architecture

components:

• Create a consistent interface across all pages (including the health information and

intramural sites);

• Organize text groupings using easily understandable bite-sized words;

• Make better use of white space;

• Break up text with useful images (see NCI and NSF sites for best practices), graphics,

and media that might more easily communicate to persons with different learning styles;

• Focus on search and “findability” by spending time creating metadata to aid in “best

bets,” classification, search, and retrieval;

• Change the “Quick Links for Investigators” drop-down so that it instead provides quick

links for various stakeholder types, minimally including a) New Investigators, b) Current

Awardees, c) Press/Media, and the d) General Public.


Page 35

• In addition to a site map, allow users to find in multiple ways, including by browsing

subject categories (A-Z) and possibly including a definitions/acronym/abbreviations

page; and

• Apply content ownership procedures by posting the date content was posted to the site.

In addition, include the content owner name and contact details so customers can ask

questions directly.

Lastly, we recommend that the NIDDK consider revising or expanding its mission so that its

focus moves from simply conducting and supporting research to also meeting the needs of all of

its Internet patrons equally by providing information that is easily accessible and understandable.

Future Research Directions

Finally, our team feels that there is a lot of usability research that has been largely

unexplored. For example, although much research has focused on the usability of commercial

systems, only within the last half dozen or so years have research efforts focused on the usability

of non-profit sites, including government Internet sites.

Although Internet sites are used to inform customers and should offer organization,

simplicity, and “findability,” sites are offering richer interactive experiences than ever before.

Our team argues that usability is related to personal learning styles. Therefore, future studies

should focus on this relatedness.

Furthermore, usability cannot be universally measured, in part because it can be

considered both an art and a science. Therefore, we recommend that future studies focus on

cultural and international differences with respect to usability as Internet sites focus more on

their expanding multicultural and international customer base.


Page 36

References

Agnvall, E. (June 21, 2005). Sites for sore eyes (legs, etc.); Ads, lack of disclosure, missing data

compromise even the best consumer health web sites, new ratings show [FINAL Edition].

Washington, DC: The Washington Post, p. F.01.

Barr, S. (October 28, 2003). HHS web guidelines created with designers, users in mind.

Washington, DC: The Washington Post, p. B.02.

Forster, S. (Oct 29, 2002). Consumers using wrong criteria to evaluate medical web sites. The

Wall Street Journal Online. New York: Wall Street Journal (Eastern edition), p. D.3.

Interagency Committee on Government Information. (June, 2004). Recommended policies and

guidelines for federal public web sites: Final report of the Interagency Committee on

Government Information. Office of Management and Budget.

Jester, C. (2000). The four learning styles. Pleasant Hill, CA: Diablo Valley College. Accessed

August 2005 at: http://www.metamath.com/lsweb/dvclearn.htm.

National Institute of Diabetes and Digestive and Kidney Diseases. Bethesda, MD. Accessed

August, 2005 at: http://www.niddk.nih.gov.

Nielsen, J. (2000). Designing Web Usability. Indianapolis: New Riders.

Nielsen, J. (1994). Heuristic evaluation. In Nielsen, J., and Mack, R.L. (Eds.), Usability

Inspection Methods. New York: John Wiley.

Shneiderman, B. and Plaisant, C. (2004). Designing the User Interface, 4th ed. New York:

Addison Wesley.

Siegel, E. R. and Wood, F. B. (2003). The National Library of Medicine’s strategy for assessing

the impacts of health information web sites. Information Services & Use, 23, 227-234.


Page 37

Sillence, E., Briggs, P., and Fishwick, L. (May 2005). Guidelines for developing trust in health

web sites. WWW 2005, 1026-1027.


Page 38

Appendix 1: Nielsen’s 10 Usability Heuristics (NIDDK)


Page 39

Appendix 1: Nielsen’s 10 Usability Heuristics (NIDDK) – Continued


Page 40

Appendix 2: Shneiderman’s Eight Golden Rules of Interface Design (NIDDK)


Page 41

Appendix 3: Guidelines for Usability Based on the Four Learning Styles (NIDDK)


Page 42

Appendix 4: Competitive Analysis Using Nielsen’s 10 Usability Heuristics


Page 43

Appendix 4: Competitive Analysis Using Nielsen’s 10 Usability Heuristics - Continued


Page 44

Appendix 5: NIDDK Web Site Main Pages

NIDDK Home Page


Page 45


Welcome Page


Page 46


Health Information Page


Page 47


Research Funding Opportunities Page


Page 48


Clinical Research Page


Page 49


NIDDK Laboratories Page


Page 50


Reports, Testimony, and Plans Page

Documents

NIDDK WEB SITE U NALYSIS