MRN30,11

816

Management Research NewsVol. 30 No. 11, 2007pp. 816-828# Emerald Group Publishing Limited0140-9174DOI 10.1108/01409170710832250

Determining the importance ofkey criteria in web usability

J. Michael Pearson and Ann PearsonCollege of Business and Administration, Southern Illinois University,

Carbondale, Illinois, USA, and

David GreenCollege of Business, Morehead State University, Morehead, Kentucky, USA

Abstract

Purpose – This research aims to investigate the relative importance of five key criteria in assessingweb usability. These criteria are navigation, download speed, personalization, ease of use, andaccessibility. It is believed that these factors play a significant role in assessing web usability.Design/methodology/approach – This study utilized a scenario-based, multi-criteria decision-making approach. The method (policy capturing) utilizes multiple scenarios to determine the relativeimportance of the criteria of interest. Based on the responses to these scenarios (assessing webusability), linear regressions, and Tukey’s honestly significant differences were utilized to determinethe relative importance of the five web usability criteria.Findings – The research indicated that, as expected, ease of use was the criteria that the respondentsconsidered most important in assessing web usability. Interestingly, personalization andcustomization was consistently reported as the least important criteria. It was also found thatmales and females view these web usability criteria differently.Practical implications – The findings from this study have practical implications for web sitedesigners.Originality/value – The results indicate that less effort/resources should be devoted topersonalization and customization, and more in making sure that web sites are easy to use andhave clear navigation capabilities.

Keywords Internet, Design, User interfaces, Gender

Paper type Research paper

IntroductionThe World Wide Web (WWW) is a medium that allows users arbitrary connections inan open environment; within this environment, users have computer skills rangingfrom novice to expert (Tarafdar and Zhang, 2005). With all the available web sites andthe diverse set of user skills, what motivates users to choose one site over another? If auser finds a site difficult to use, cannot find the desired product on an business-to-consumer (B2C) web site, or is not clear on what a site offers, the user will typicallyleave that site (Nielsen, 2003a, b).

Statements about the importance of Web site usability come at a time whenconsumers are increasing their spending at B2C web sites. E-commerce estimates forthe first quarter of 2005 from the Census Bureau of the Department of Commerce foundthat online sales rose to $143.2 billion, a 22 per cent increase over 2004 (Burns, 2006).Based on this rapid expansion of e-commerce sales, the importance of web siteusability is apparent. Previous studies have found that a usable web site creates apositive attitude toward online stores, increases stickiness and revisit rates, andeventually stimulates online purchases. A usable web site also provides benefits toe-business companies by reducing web site development, support, and maintenance

The current issue and full text archive of this journal is available atwww.emeraldinsight.com/0140-9174.htm

An earlier version of this manuscript appeared in the conference proceedings for theNational Decision Sciences Institute, 2006.

Key criteria inweb usability

817

costs (Becker and Mottay, 2001). However, several studies have reported that most websites contain numerous usability problems. Difficult to understand formats, difficultyin navigation, customer disorientation, and lack of interaction and reliability arefrequently mentioned problems (Jhang et al., 2000; Nerurkar, 2001; Shneiderman, 2005).Each of these factors contributes to transactions not being completed and the resultingloss of sales revenue for the sponsoring e-organization.

The objectives of this research are two-fold. The first objective is to betterunderstand web usability and the criteria that influence successful web design; andsecondly, to determine if gender impacts the relative importance of these usabilitycriteria.

Literature reviewUsability is the most traditional concept of study in human–computer interaction (HCI)research (Olson and Olson, 2003; Karat, 2003). Usability has been defined as a‘‘the measure of the quality of a user’s experience when interacting with a product orsystem – whether a Web site, a software application, mobile technology, or any user-operated device’’ (Anonymous, 2006). The importance of HCI and usability has becomeincreasingly important with the development of the WWW and its role in e-commerceactivities. In fact, Jakob Nielsen, a noted usability expert, suggests that organizationsshould spend 10 per cent of the development budget on usability. Nielsen, in a study of42 organizations who redesigned their web site with usability as a primary concern,concluded that the sales conversion rates increased by 135 per cent and that traffic onthe web site increased by 150 per cent (www.useit.com/alertbox/20030107.html). Thesefindings provide support as to the importance of developing web sites that have highusability.

Web usability can be defined as making the design simple enough so thatcustomers, who by nature tend to be goal-driven, can accomplish their task as quicklyand painlessly as possible (www.webcredible.com). Shneiderman (2005) suggests thatusability can be a balancing act – inadequate functionality will render the applicationuseless while complexity and clutter make an interface difficult to use. Nielsen(2003a, b) states that it is more important for design to meet the needs of the customerrather than be attractive and fun. If the customer finds the site too difficult to use, therewill not be a purchase or return visit.

In their research on web customer satisfaction, McKinney et al. (2002) state that aweb site will be abandoned if the consumer has difficulty searching or retrieving theirneeded information, even if the web site provides the information necessary tocomplete the intended task. They state that the web site must compensate for lack ofphysical contact experienced by online shoppers and at the same time make theshopping experience easy and enjoyable. The user’s impression of the web site’susability impacts the user’s impression of the products available at the site.

Although the HCI literature has examined several aspects of web site usability(Turban and Gehrke, 2000; McKinney et al., 2002; Torkzadeh and Dhillon, 2002), it hasonly been recently that information systems literature has focused on web siteusability in the context of understanding B2C e-commerce. Two studies, Palmer (2002)and Agarwal and Venkatesh (2002), investigated the underlying dimensions of web siteusability. Palmer defined usability based on five dimensions derived from usability andmedia richness literature (download delay, navigability, content, interactivity, andresponsiveness), while Agarwal and Venkatesh utilized the Microsoft UsabilityGuidelines to define web site usability through five different dimensions (ease of use,

MRN30,11

818

made for the medium, emotion, content, and promotion). Each study resulted in aninstrument that, it was suggested, could be used to assess a web site’s usability.

Green and Pearson (2006, 2007) found that while both instruments (Agarwal andVenkatesh, 2002; Palmer, 2002) provided some degree of reliability and robustness inmeasuring web site usability, a modified instrument consisting of navigation,customization and personalization, download speed, accessibility, and ease of useprovided a more valid and more robust measure of web site usability. Interestingly, intheir study, content was not a statistically significant predictor of web site satisfactionor intent to return to the web site. They concluded that content was not a necessarycomponent of web site usability – that users considered content to be part ofusefulness. This differs from most previous research in this area, but intuitively makessense. Based on previous work on web usability (Turban and Gehrke, 2000; Agarwaland Venkatesh, 2002; McKinney et al., 2002; Palmer, 2002) and the findings of Greenand Pearson (2007), this study seeks to determine the relative importance of these fivedimensions (navigation, customization and personalization, download speed,accessibility, and ease of use) in an individual’s assessment of web site usability.

NavigationNavigation has been an issue for a long time. In a study comparing organized andunorganized screen designs, shneiderman (1987) found that novice users made twice asmany navigation errors and had to think twice as long when using an unorganizedscreen. In their review of literature on web usability, Turban and Gehrke (2000) foundthat navigational controls were important for web sites. They found that consumersprefer web sites that lend themselves to navigation efficiency. Tarafdar and Zhang(2005) found navigation to influence web site usability because without goodnavigation, users tend to experience cognitive overload.

Nielsen (2003a, b) stresses the importance of keeping the user in mind whendesigning web site navigation. Nielson compared the navigation scheme between twomodels: one built on the mental model characteristic of most users and the other builtaccording to the company’s internal thinking. Results showed that a navigation schemebased on the users’ mental model was used nine times more frequently than thenavigation scheme preferred by the company’s internal thinking. Nielsen adds thatusers will take paths totally different from what designers expect – such as completelyskipping the home page. Navigation that is simple, efficient, user-centered, and flexiblewill help the customer achieve intended goals and increase the likelihood of returnvisits.

Personalization and customizationStudies have found customization and personalization, dynamically fitting a site to theuser’s needs (Agarwal and Venkatesh, 2002; Palmer, 2002), to be important factors inweb site success. Recently, Liang et al. (2006-2007) found that personalization canincrease user satisfaction with an interactive web site. They also indicated, however,that personalization of a web site can be overdone; i.e. too much personalization resultsin lower user satisfaction and information overload. In a study involving web sitesfrom four different industries, Agarwal and Venkatesh (2002) found thatpersonalization, also referred to as ‘‘made-for-the-medium’’, was important for websites that were hoping to establish an ongoing relationship with the customer.Personalization and customization provides the ability to control the amount ofinformation pushed at users, but it is often not used in web sites as Huang (2003)

Key criteria inweb usability

819

discovered in his study of 50 web sites owned by large international corporations.Despite the large volume of information available on these sites, only 6 per cent of thecorporate web sites included customization and personalization capability, anunexpected result from the hypothesized 50 per cent.

Download speedDownload speed can refer to the telecommunication infrastructure and in that regard,download speed is beyond the control of most studies. For example, download speed isaffected by many components in the chain from server to browser – one of those linksis connection to the Internet. Content of a web site such as images, video clips, and/oraudio clips can affect the download speed in initially accessing a site and in subsequentrequest of pages within the site. Research has shown that users find download delay animportant factor in web site usability (Palmer, 2002). Turban and Gehrke (2000)suggest that consumers prefer simple web sites that provide service and content oversites containing flashing banner ads, revolving images, and a multitude of multimediaeffects that slow download speed. Tarafdar and Zhang (2005) found that waiting toolong to access online information can lead to anxiety for users and loss of satisfactionwith the web site. Other usability studies have shown that a wait of just a few secondsis the maximum time web users will wait for a page to download.

AccessibilityAccessibility is important for web site usability as users will not be able to completetransactions if they are unable to use the site. Accessibility, when pertaining to a website, means that information has been made available for use by potential users of thatparticular web site, including individuals with disabilities. Recent studies have foundthat many web sites (public and private) do not provide an adequate level ofaccessibility (Stowers, 2002; Hackett et al., 2005; Parmanto and Zeng, 2005). Hackettet al. (2005) suggest that as web developers add complex features to dynamic web sites,accessibility to that web site is reduced. Nielsen (2000) warns that following everyaccessibility guideline can involve excessive development time for a business, but hesuggests that developing reasonably accessible web sites should be a priority. Web sitedesigners should test the sites with text-only browsers and make sure that at aminimum, all information is being displayed. Users will be more satisfied withaccessible sites and are more likely to make return visits.

Ease of useEase of use along with perceived usefulness has been shown to be importantpredictors of technology acceptance (Venkatesh et al., 2003). Other studies havesuggested that ease of use is an important component in web site usage (Lee, 2004;Muthitacharoen et al., 2006). In their study, Agarwal and Venkatesh (2002) found easeof use to be an important variable in determining web usability. Based on thesepredictors, the following research question is investigated:

RQ1. What is the relative importance of navigation, customization andpersonalization, download speed, accessibility, and ease of use in assessingWeb site usability?

MRN30,11

820

GenderWhile in previous studies, gender was found to have an effect on computer usage,increased educational levels for women and the proliferation of computers into thehome may have caused gender to have less of an affect on computer usage within theorganization (Harrison and Rainer, 1992; Knight and Pearson, 2005). Additionally, withthe increasing number of women in the workforce, and the continued growth ofcomputer usage required by employers, questions arise as to whether gender affectscomputer usage. However, other studies have found that this gender-based technologygap still exists. For example, Joiner et al. (2005) found a significant and negativecorrelation between gender and the use of the Internet; Simon and Peppas (2005) foundthat males have more positive attitudes than females related to both rich and lean websites; and Wasserman and Richmond-Abbott (2005) found that men and women differon what services they utilize on the Internet. Based on these contradictory findings, webelieve it is important to investigate a second research question:

RQ2. Does gender impact the relative importance of navigation, customizationand personalization, download speed, accessibility, and ease of use inassessingWeb site usability?

Research methodologyThis study utilized a multi-criteria decision-making approach, policy capturing, whichallows the researcher to determine the relative importance of the independent variables(navigation, ease of use, personalization and customization, download speed, andaccessibility) have on the dependent variable of interest (web usability). Policycapturing allows the researcher to collect data based on a series of scenarios that reflectdifferent levels of the decision criteria and the respondent’s decision corresponding toeach scenario. Previous studies have shown that this approach can accurately measurethe relationship between an individual’s decisions and the relative weight of the criteriaused to arrive at those decisions (Graves and Karren, 1992; McNichols and Zimmerer,1985; Webster and Trevino, 1995; Pearson et al., 1997). See Appendix for a moredetailed explanation of policy capturing.

In order to identify the relative importance of each of the five web usability criteriatested in this study, a linear regression model that utilized all captured scenarios wascalculated. The number of possible data points for analysis can be calculated by takingthe number of scenarios evaluated by each individual multiplied by the total number ofparticipants. To determine if significant differences existed between the calculatedbetas, an analysis of variance was conducted. If the F-test failed, meaning thatsignificant differences existed between the regression betas, Tukey’s honestlysignificant differences (HSD) was utilized to determine which of the five criteriadiffered statistically from the others.

To determine if gender changed the relative importance of the five web usabilitycriteria, a one-way analysis of variance was performed for each pooled subcategory(male vs female). This allowed us to determine if significant differences existed in therelative importance of these web usability criteria based on gender. Based on theresults of this analysis, we were able to ascertain the validity of RQ 2.

SampleThe participants in this study were undergraduate students enrolled at five differentbusiness schools located within the USA. These business schools were selected

Key criteria inweb usability

821

because of contacts one of the authors had at each institution; classes sampled weredependent on the classes being taught by these contacts. Therefore, this could beconsidered a sample of convenience.

Many studies in the management information systems (MIS) field use students asparticipants for the purpose of data collection. McKnight et al. (2002) argue thatstudents can be used as subjects in research for which they may have specificexperiences, such as in the context of B2C e-commerce or utilization of web sites.Students as web site users provide an excellent proxy for this type of study as most arefrequent users/visitors of the Internet and theWWW.

Questionnaire developmentTwo versions of a multi-part questionnaire were developed based on work donepreviously by one of the authors (Pearson et al. 1997). In both versions, the first part ofthe questionnaire consisted of seven questions designed to solicit information about therespondent and his or her Internet usage. The second part contained the policy-capturing portion of the study and consisted of 16 scenarios describing five differentcriteria associated with web usability (see Figure 1). Each version of the questionnairecontained one-half of the possible scenarios. The values of the decisions concerningweb site usability, which ranged from 1 (Strongly Disagree) to 7 (Strongly Agree), werethe dependent values in the regression calculations.

Prior to administering the questionnaire, each version was administered to a groupof ten undergraduate business students. As a result of this pilot test, minormodifications were made to how the scenarios were presented (i.e. a counter was addedto provide a progress indicationwhen completing the different scenarios).

Results of analysisThe survey instrument was administered to multiple classes at each university over atwo-week period. This approach provided 215 responses. Five questionnaires weredropped from further analysis because of incomplete responses. Additionally, thereliability or internal consistency of the remaining responses was evaluated byexamining the adjusted R2 of individual within-subject regressions. The adjusted R2

ranged from 0.05 to 0.962, averaging 0.738. Thirty-two (32) responses were deletedfrom further analysis due to adjusted R2 values below 0.50. The average adjusted R2

changed to 0.793. The remaining 178 responses were used to evaluate the researchquestions formulated earlier in this paper.

Figure 1.Scenario example

MRN30,11

822

Demographics of studyOne hundred seven (107) of the respondents were male and all of the respondentsindicated that they utilized the WWW (i.e. education, browsing, work, etc.). A total of94.9 per cent of the respondents were under the age of 25; and 61.8 per cent of therespondents accessed the WWWmore than 15 h per week. The respondents identifiedbroken links (52.2 per cent), the inability to find specific information (50 per cent), anddownload speed (37.1 per cent) as the major problems when accessing theWWW.

RQ1: criteria impacting web usabilityIn order to investigate RQ1, we performed a linear regression based on the responsesprovided by 178 individuals who had demonstrated a consistent application of theirdecision criteria in their decision-making process. The dependent variable in thisanalysis was the responses provided from the policy-capturing scenarios relating towhether the respondent thought the hypothetical web site was usable. Theindependent variables were the different conditions for the five criteria as presented inthe policy-capturing simulations. This resulted in 2,848 observations for analysis (178respondents � 16 simulations per respondent). The following equation provides arepresentation of the regression model tested:

WU¼ Intercept�BiðNAVÞ�BiðCUSTÞ�BiðDLOADÞ�BiðACCESSÞ�BiðEOUÞ�"

The results of the linear regression provided the following results:

WU¼ 0:52þ 1:35 NAVð0:138Þ

þ0:726 CUSTð0:042Þ

þ1:27 DLOADð0:122Þ

þ1:12 ACCESSð0:095Þ

þ1:60 EOUð0:255Þ

þ "

The adjusted R2 for the regression equation was 0.653[1]. This suggests that a largepercentage of the variability in the respondents’ web usability decision can beexplained by their views on the five criteria tested in this study. Based on the calculatedF statistic (F 5, 2841 ¼ 1,070.5, p < 0.01), we can conclude that this model is a significantand valid predictor of the dependent variable. The results support the belief that thefive criteria are significant predictors of the web usability decision. EOU (0.255) andNAV (0.138) had the largest contribution in explaining web usability. Interestingly,CUST (0.042) explained the least amount of variability in predicting the respondents’web usability decision. This would suggest that ease of use and navigation are themost important criteria in determining web site usability while personalization andcustomization is the least important. This finding supports much of the technologyadoption literature, as previous studies in this area have consistently found that ease ofuse is a significant predictor of intention to adopt and to utilize a new technology. Thefinding that navigation is also a significant predictor of web site usability is notsurprising; from an intuitive perspective, web sites that have broken links or poor website design would not be considered very user friendly. Perhaps the most surprisingfinding was the low emphasis placed on personalization and customization by therespondents of this study. Organizations have been spending a considerable amount oftime and money trying to add this very feature to their web sites. It appears that theseorganizations would be better served to redesign their web sites to enhance ease of useand clear navigation.

To determine the relative importance of the five decision criteria (see Table I), weaveraged the regression betas obtained from the with-in subject regressions calculatedfor each of the 178 participants. To determine if significant differences existed between

Key criteria inweb usability

823

the five criteria, an analysis of variance was conducted. The results showed asignificant difference among the mean values of the decision criteria (F4,708 ¼ 27.22,p < 0.05). The Tukey’s HSD test was then calculated to determine which of the fivecriteria differed statistically from the other. The results suggest that the respondentsfound the mean value for ease of use to be statistically different from the other fourusability criteria; navigation and accessibility were uniquely grouped; and downloadspeed and customization and personalizationwere also statistically linked.

RQ2: impact of genderTo see if gender influenced the relative significance of the web usability criteria, weconducted a one-way analysis of variance using gender as a blocking factor. Theresults are presented in Table II. Two usability criteria, navigation and ease of use,were found to have significant differences based on gender. Females placed greateremphasis on both of these web usability criteria than did males. These findings,especially on ease of use, are consistent with previous studies in which women consider

Table I.Relative importance ofWeb usability criteria

Web criteriaMeans of

regression betas Tukey’s HSD

Ease of use 1.54 ANavigation 1.28 BAccessibility 1.25 BDownload speed 1.03 CCustomization and personalization 0.84 C

Note: The different letters (A, B, C) indicate different groupings based on Tukey’s HSD procedure

Table II.One-way analysis ofvariance blocked on

gender

Sum ofsquares df Mean square F Significance

NAV Between groups 1.720 1 1.720 5.021 0.026Within groups 60.277 176 0.342Total 61.996 177

CUST Between groups 0.048 1 0.048 0.111 0.739Within groups 76.275 176 0.433Total 76.323 177

DLOA Between groups 2.211 1 2.211 3.480 0.064Within groups 111.839 176 0.635Total 114.050 177

ACCE Between groups 0.316 1 0.316 0.835 0.362Within groups 66.562 176 0.378Total 66.877 177

EOU Between groups 3.053 1 3.053 5.797 0.017Within groups 92.687 176 0.527Total 95.740 177

MRN30,11

824

these constructs to be important in their intention to utilize the Internet (Morris andVenkatesh, 2000).

LimitationsThe present research is limited by a number of factors. The first limitation pertains tothe decision-modeling procedure used for this study. Although respondents werepresented with five decision criteria derived from relevant web usability literature,other criteria may also influence the respondents’ determination of web usability.Another possible limitation of this study could be the method by which the scenarioswere presented in the research instrument. Finally, the sample was drawn from astudent population, which could limit the generalizibility of the findings.

ContributionsThis study has the potential to have important applications to organizations involvedin e-commerce. Several studies have found that usability is an important component ofweb site success. If a user finds a site difficult to use (usability), then they typically willnot ‘‘stick around’’ to determine if the content (usefulness) meets their requirements. Bydetermining which criteria of web usability are most important to the users of theInternet and WWW, organizations can modify the design of their web sites and,hopefully, increase the volume of transactions that will completed at these web sites.For example, the results of this study indicated that females have a stronger preferencefor web sites that are easy to use and have clear navigation capabilities; this suggeststhat organizations that have women-centric web sites should emphasize these featuresin the design of their web site. Organizations have to understand who targetedcustomers are before moving into the e-commerce arena.

This study also has important implications for academics who wish to research inthis area. By using a multi-criteria decision-making approach to data capture andanalysis, we are able to validate or invalidate the findings of perception-based researchas it relates to the relative importance of web usability criteria. If this study supportsthe findings of previous perception-based research, then we can have confidence thatperception-based research provides accurate and valid measures of an individual’sdecision-making process. Another area of interest for academics would be toinvestigate what other criteria influence a user’s assessment of web site usability; thisstudy found that the five criteria tested explained approximately 64 per cent of thevariance in this assessment. These criteria should also be tested with a more diversegroup of web users to see if the findings of this study are robust across different agegroups, technological competence, and cultures.

Note

1. The numbers in parentheses under the independent variables represent the amount ofvariance explained by that particular independent variable in predicting web usability.

References

Agarwal, R. and Venkatesh, V. (2002), ‘‘Assessing a firm’s web presence: a heuristic evaluationprocedure for the measurement of usability’’, Information Systems Research, Vol. 13 No. 2,pp. 168-86.

Anonymous (2006), ‘‘Step-by-step usability guide’’, available at: www.usability.gov (accessed3 October 2006).

Key criteria inweb usability

825

Becker, S and Mottay, F. (2001), ‘‘A global perspective on web site usability’’, IEEE Software,Vol. 19 No. 1, pp. 54-62.

Burns, E. (2006), ‘‘Online retail sales grew in 2005’’, available at: www.clickz.com/stats/sectors/retailing/article.php/3575456 (accessed 3 October 2006).

Cooksey, R.W. (1996), ‘‘The methodology of social judgment theory’’,Thinking & Reasoning, Vol. 2No. 2&3, pp. 141-74.

Feldman, D.C. and Arnold, H.J. (1978), ‘‘Position choice: comparing the importance oforganizational and job factors’’, Journal of Applied Psychology, Vol. 63 No. 6, pp. 706-10.

Graves, L.M. and Karren, R.J. (1992), ‘‘Interviewer decision processes and effectiveness:an experimental policy capturing investigation’’, Personnel Psychology, Vol. 75 No. 2,pp. 313-41.

Green, D.T. and Pearson, J.M. (2006), ‘‘Developing a model for B2C web site usability ande-commerce acceptance’’, in Proceedings of Web 2006: The 5th Conference on E-Business(Pre-ICIS SIG-eBiz),Milwaukee,WI.

Green, D. and Pearson, J.M. (2007), ‘‘The examination of two web site usability instruments foruse in B2C e-commerce organizations’’, Information and Management, submitted forpublication.

Hackett, S., Parmanto, B. and Zeng, X. (2005), ‘‘A retrospective look at website usability’’,Behavior & Information Technology, Vol. 24 No. 6, pp. 407-17.

Harrison, A. and Rainer, R. Jr (1992), ‘‘The Influence of individual differences on skill in end-usercomputing’’, Journal of Management Information Systems, Vol. 9 No. 1, pp. 93-112.

Huang, A. (2003), ‘‘An empirical study of corporate web site usability’’, Human SystemsManagement, Vol. 22 No. 1, pp. 23-36.

Jhang, J., Jain, H. and Ramamurthy, K. (2000), ‘‘Effective design of electronic commerceenvironments: a proposed theory of congruence and an illustration’’, IEEE Transactions onSystems, Man, and Cybernetics, Vol. 30 No. 4, pp. 456-71.

Joiner, R., Gavin, J., Duffield, J., Brosman, M., Crook, C., Durndell, A., Maras, P., Miller, J., Scott,A.J. and Lovatt, P. (2005), ‘‘Gender, internet identification, and internet anxiety: correlatesof internet use’’, CyberPsychology & Behavior, Vol. 8 No. 4, pp. 371-8.

Karat, J. (2003), ‘‘The evolution of user-centered focus in the human–computer interaction field’’,IBM Systems Journal, Vol. 42 No. 4, pp. 532-41.

Knight, M. and Pearson, J.M. (2005), ‘‘The changing demographics: the diminishing role of ageand gender in computer usage’’, Journal of Organizational & End User Computing, Vol. 17No. 4, pp. 49-65.

Lee, J. (2004), ‘‘Discriminant analysis of technology adoption behavior: a case of internettechnologies in small businesses’’, Journal of Computer Information Systems, Vol. 44No. 4, pp. 57-66.

Liang, T.P., Lai, H.J. and Ku, Y.C. (2006-2007), ‘‘Personalization content recommendation and usersatisfaction: theoretical synthesis and empirical findings’’, Journal of ManagementInformation Systems, Vol. 23 No. 3, pp. 45-70.

McKinney, V., Yoon, K. and Zahedi, F. (2002), ‘‘The measurement of web-customer satisfaction: anexpectation and disconfirmation approach’’, Information Systems Research, Vol. 13 No. 3,pp. 296-315.

McKnight, D., Choudhury, V. and Kacmar, C. (2002), ‘‘Developing and validating trust measuresfor e-commerce: an integrative typology’’, Information Systems Research, Vol. 13 No. 3,pp. 296-315.

McNichols, C. and Zimmerer, T. (1985), ‘‘Situational ethics: an empirical study of differentiates ofstudent attitudes’’, Journal of Business Ethics, Vol. 4 No. 3, pp. 175-80.

MRN30,11

826

Morris, M.G. and Venkatesh, V. (2000), ‘‘Age differences in technology adoption decisions:implications for a changing work force’’, Personnel Psychology, Vol. 53 No. 2, pp. 375-403.

Muthitacharoen, A., Palvia, P.C., Brooks, L.D., Krishnan, B.C. and Otondo, R.F. (2006),‘‘Reexamining technology acceptance in online task behaviours’’, Electronic Markets,Vol. 16 No. 1, pp. 4-15.

Nerurkar, U. (2001), ‘‘Web user interface design: forgotten lessons’’, IEEE Software, Vol. 18 No. 6,pp. 69-71.

Nielsen, J. (2000),DesigningWeb Usability, New Riders Publishing, Indianapolis, IN.

Nielsen, J. (2003a), ‘‘Usability 101: introduction to usability’’, Useit.com, available at:www.useit.com/alertbox/20030825.html (accessed 1 November 2006).

Nielsen, J. (2003b), ‘‘Return on investment for usability’’, Useit.com, available at: http://useit.com/alertbox/20030107.html (accessed 25 April 2007).

Olson, G.M. and Olson, J.S. (2003), ‘‘Psychological aspects of the human use of computing’’,Annual Review of Psychology, Vol. 54 No. 1, pp. 491-517.

Palmer, J. (2002), ‘‘Web site usability, design, and performance metrics’’, Information Systems

Research, Vol. 13 No. 2, pp. 151-16.

Parmanto, B. and Zeng, Q. (2005), ‘‘Metric for web accessibility evaluations’’, Journal of Society forInformation Science and Technology, Vol. 56 No. 13, pp. 1394-404.

Pearson, J., Crosby, L. and Shim, J.P. (1997), ‘‘Measuring the importance of ethical behaviorcriteria’’, Communications of the ACM, Vol. 40 No. 9, pp. 94-100.

Shneiderman, B. (1987), Designing the User Interface, Addision-Wesley Publishing Company,Reading, MA.

Shneiderman, B. (2005), Designing the User Interface, 4th ed., Pearson-Addison Wesley,New York, NY.

Simon, S.J. and Peppas, S. (2005), Attitudes towards product website design: a study of the effectsof gender’’, Journal of Marketing Communications, Vol. 11 No. 2, pp. 129-44.

Stowers, G.N.L. (2002), ‘‘The state of federal websites’’, Pricewaterhouse-CoopersEndowment for the Business of Government, E-Government Series, available at:www.businessofgovernment.org/pdfs/StowersReport0802.pdf (accessed 25 April 2007).

Tarafdar, M. and Zhang, J. (2005), ‘‘Analyzing the influence of web site design parameters on website usability’’, Information Resources Management Journal, Vol. 18 No. 4, pp. 62-80.

Torkzadeh, G. and Dhillon, G. (2002), ‘‘Measuring factors that influence the success of internetcommerce’’, Information Systems Research, Vol. 13 No. 2, pp. 187-204.

Turban, E. and Gehrke, D. (2000), ‘‘Determinants of e-commerce web site’’, Human Systems

Management, Vol. 19 No. 2, pp. 111-20.

Venkatesh, V., Morris, M.G., Davis, G.B. and Davis, F.D. (2003), ‘‘User acceptance of informationtechnology: toward a unified view’’,MIS Quarterly, Vol. 27 No. 3, pp. 425-78.

Wasserman, I.M. and Richmond-Abbott, M. (2005), Gender and the internet: causes of variationin access, level, and scope of use’’, Social Science Quarterly, Vol. 86 No. 1, pp. 252-70.

Webster, J. and Trevino, L.K. (1995), ‘‘Rational and social theories as complementary explanationsof communication media choices: two policy capturing studies’’, Academy of Management

Journal, Vol. 38 No. 6, pp. 1544-73.

Zedeck, S. (1977), ‘‘An information processing model and approach to the study of motivation’’,Organizational Behavior & Human Performance, Vol. 18 No. 1, pp. 47-77.

Key criteria inweb usability

827

Further reading

Anonymous (2004), ‘‘Web usability: the basics’’, available at: http://webcredible.co.uk/user-friendly-resources/web-usability/basics.html (accessed 3 October 2006).

About the authorsJ. Michael Pearson is an Associate Professor of Information Systems at Southern IllinoisUniversity at Carbondale. Pearson has presented papers at regional, national and internationalconferences. He has published over 40 articles in journals such as the Communications of theACM, Information & Management, Journal of Strategic Information Systems, Journal ofInformation Systems, Journal of Computer Information Systems, Decision Support Systems,Review of Business, Journal of Internet Commerce, Information Resources Management Journal,and Public Administration Quarterly. His current research interests are Internet business,technology adoption at the organizational level, and social connectivity. J. Micheal Pearson is thecorresponding author and can be contacted at: jpearson@cba.siu.edu

Ann Pearson is currently a PhD student at Southern Illinois University at Carbondale. Herresearch interests include innovating with technology, web usability, and web functionality. Shehas 15 years of work experience in the information systems area. Ms Pearson has worked asprogrammer and in web development. She also worked as a software trainer for a health carefirm.

David Green is an Assistant Professor of Computer Information Systems at Morehead StateUniversity. He received his PhD from Southern Illinois University at Carbondale. His researchhas been published in the Journal of Computer Information Systems and the InternationalJournal of Electronic Marketing and Retailing as well as the Proceedings of the HawaiiInternational Conference on System Sciences, the Americas Conference on Information Systems,and theAnnual Meeting of the Decision Sciences Institute. His current research interests are in theareas of human–computer interaction, information security, and enterprise systems success.

Appendix. Policy capturingPolicy capturing is a method employed by researchers to assess how decision makers useavailable information when making evaluative judgments (Zedeck, 1977). The purpose of thismethodology is to capture an individual’s decision-making policies, or how they weight,combine, or integrate information (Zedeck, 1977). It involves asking decision makers to judge aseries of scenarios describing various levels of the explanatory factors, or cues, and thenregressing their responses on the cues. The estimated coefficients indicate the relativeimportance of the various cues in the decision-making process. Past research has indicated that,because of cognitive limitations, decision makers typically make decisions on relatively smallnumber of key criteria (Cooksey, 1996). This, and findings that policy capturing more accuratelymeasures the relative weight of key decision criteria (Feldman and Arnold, 1978; Pearson et al.,1997), make policy capturing an attractive tool to utilize in multi-criteria decision-makingsituations.

Policy capturing can be used to address two general types of research questions – idiographicand nomothetic. Idiographic questions concern how separate individuals use information tomake choices, while nomothetic focus on outcomes aggregated across many decision makers.This study took a nomothetic approach to evaluating the role five-specific criteria (navigation,customization and personalization, ease of use, accessibility, and download speed) have on website usability.

The policy-capturing exercise in this study consisted of 32 unique scenarios related to website usability. The scenarios were created by combining appropriate manipulations for the fivevariables of interest in this study. These scenarios were created by combining the five variableswithin a full factorial design (25) and by randomly varying the order in which the scenarioinformation was presented. This approach ensures that the independent variables areuncorrelated. In order to avoid issues of fatigue by the respondents; each survey instrumentcontained 16 randomly selected scenarios (see Figure 1).

MRN30,11

828

Policy capturing develops an additive linear model of the relationships between anindividual’s decisions and the criteria (cues) used to arrive at those decisions. This relationshipcan be stated as:

Yj ¼Xn

i¼1

biXij where j ¼ 1; 2; . . . . . . ; n

In this study, Yj are the decisions provided by the individual respondent, Xij are the criteriarelated to web usability, and the bi represent the importance attached to each of the criteria by thedecision maker when evaluating the criteria and determining the usability of the hypotheticalweb site.

In order to test the reliability or internal consistency of an individual’s use of the decisioncriteria (cues), a within subject regression was utilized. This requires that a linear regressionmodel be calculated for each individual based on the decision responses provided to each of the16 scenarios. The reliability and internal consistency is evaluated by examining the adjusted R2

of this linear regression model (McNichols and Zimmerer, 1985). Adjusted R2 values below 0.50indicate a random application of the respondents’ decision criteria (Graves and Karren, 1992).Respondents with an adjusted R2 below this value were deleted from further analysis. This stepassures that, on a whole, respondents were consistent in their application of the decision criteriain their decision-making process.

When taking a nomothetic approach, the researcher is interested in determining the relativeimportance of the decision criteria (cues) with respect to the dependent variable for the group as awhole. In order to determine the relative importance of these decision criteria, the responses forall of the scenarios from all of the respondents are utilized to calculate a linear regression model.In this study, this linear regression model would be:

WU ¼ Intercept� BiðNAVÞ � BiðCUSTÞ � BiðDLOADÞ � BiðACCESSÞ � BiðEOUÞ � "

Calculating this regression equation allows the researcher to determine, based on the betas (Bi)obtained, the relative strength of each decision criteria when predicting the usability of a website. By utilizing a statistical tools such as one-way analysis of variance and Tukey’s HSD, itbecomes possible to determine if these decision criteria are significantly different and if so, how.

To purchase reprints of this article please e-mail: reprints@emeraldinsight.comOr visit our web site for further details: www.emeraldinsight.com/reprints