User Interface Guideline

8/2/2019 User Interface Guideline

1/24

User interface guidelines and standards: progress,issues, and prospects

P. Reeda,*, K. Holdawayb, S. Isenseec, E. Buied, J. Foxe, J. Williams f,A. Lundg

a Lucent Technologies, 7282 Silverhorn Drive, Evergreen, CO 80439, USAbUIS Consulting Inc., 3112 Fox Hollow, Round Rock, TX 78681, USA

cBMC Software, 10415 Morado Circle, Austin, TX 78759, USAdComputer Sciences Corporation, 15245 Shady Grove Rd., Rockville, MD 20850, USA

ePacific Northwest National Laboratory, 901 D Street, SW, STE 900, Washington, DC 20024, USAfBellcore, 6 Corporate Place, PYA 1M186, Piscataway, NJ 08854, USA

gUS WEST Advanced Technologies, 4001 Discovery Drive, Suite 340, Boulder, CO 80303, USA

Abstract

This article reviews progress in the development of standards and guidelines for humancomputer

interaction, including those developed within international and US standards bodies. Guidance for

incorporating software ergonomics standards and guidelines into software design and development

processes is discussed. Several different techniques that have been defined for assessing the confor-

mance of a product to guidelines are reviewed. In addition, the strategies employed by formally

approved standards developed in ISO and ANSI for determining conformance are discussed. Finally,

we discuss the prospects and challenges for software ergonomics standards and guidelines that must

be addressed as the pace of technological change continues to accelerate. 1999 Elsevier Science

B.V. All rights reserved.

Keywords: Software ergonomics standards; User interface design; User interface guidelines; Software usability

standards; Software ergonomics in design and development; Usability conformance assessment; Tools for work-ing with guidelines

1. Introduction

Since the mid-1980s, formal standards and published guidelines for software user

interfaces and HumanComputer Interaction (HCI) have increased in importance as the

Interacting with Computers 12 (1999) 119142

INTCOM 1143

0953-5438/99/$ - see front matter 1999 Elsevier Science B.V. All rights reserved.

PII: S0953-5438(99)00008-9

www.elsevier.com/locate/intcom

* Corresponding author. Tel.: 1-303-670-6955; fax: 1-303-670-6955.

E-mail addresses: [email protected] (P. Reed), [email protected] (K. Holdaway), [email protected] (S. Isensee), [email protected] (E. Buie), [email protected] (J. Fox), [email protected]

(J. Williams), [email protected] (A. Lund)


2/24

use of computers has become pervasive in work settings and other environments. Substan-tial benefits for both end-users and employers can result from the use of software user

interface standards and guidelines including: increased productivity, reduced mental and

physical stress, reduced training expense, improved usersystem interoperability across

applications, and improved overall product quality and aesthetics.

The purpose of this paper is to: (1) provide an overview of standards and guidelines for

humancomputer interaction, including the International Organization for Standardiza-

tion (ISO) and American National Standards Institute (ANSI) standards development

activities in the area of software ergonomics, (2) discuss how software ergonomics stan-

dards and guidelines may be incorporated into software design and development

processes, (3) review the techniques that have been defined for assessing the conformanceof a product to guidelines and how the formally approved standards developed in ISO and

ANSI address conformance, and (4) examine the prospects and challenges for software

ergonomics standards and guidelines that must be addressed as the pace of technological

change continues to accelerate.

There are a wide range of resources available to system developers seeking guidance

regarding the design of software user interfaces. Numerous books, e.g. Nielsen [1], prin-

ciples, e.g. Mayhew [2], style guides e.g. Apple Computer [3], standards e.g. ISO [4], and

guidelines, e.g. Brown [5] have been published in the extant literature. Depending on the

requirements for a particular project, some or all of these sources may be consulted.

The least formal of these sources are books (often containing principles) and guide-

lines, which usually do not undergo any systematic consensus-building process, and which

usually provide general guidance that may be applied to most user interfaces. Style guides

are generally published by software producers (or consortium of software producers) that

wish to define a specific look and feel, to a fairly high degree of detail, for applications

developed on a defined hardware/OS/middleware platform. Style guides are formal to the

extent that there is usually a systematic review process conducted with representatives

from consortia members, application developers, and other directly concerned interests,

but there is no formal, due process defined for achieving consensus among all potentially-

affected interests (e.g. end-users).Standards such as those in ISO and ANSI are distinguished by documented standards

development procedures that have been designed to ensure that all companies and

P. Reed et al. / Interacting with Computers 12 (1999) 119142120

Fig. 1. First documented use of the shall statement to create a mandatory standard.


3/24

individuals that are materially- or directly-effected by a particular standard will have an

opportunity to represent their interest and participate in the standards development

process. These procedures require consensus-building processes to be utilized before

any standard is approved. Each individual design guidance statement in a formal standardis either a requirement or a recommendation. A requirement must include the word

shall its statement providing design guidance, and anyone who claims conformance

with the overall standard must conform with each and every requirement (i.e. shall

statement) contained in the standard. In contrast, a recommendation must use the

word should in its statement of design guidance, and a software product does NOT

have to conform with each and every recommendation in order to claim conformance with

the overall standard. As shown in Fig. 1, humans have been challenged to comply with

standards, with varying degrees of success, for many centuries!

2. History and organization of ergonomics standards and committees

As recently as a dozen years ago there were few if any standards to be found which

could be directly applied to the software design of user interfaces. There were some

guidelines available, such as The Mitre Corporation Guidelines on Designing User Inter-

face Software published in 19831986, but no national or international efforts by formal

standards bodies were underway. In the 19851986 timeframe however, several events

occurred which became the catalyst to the formation of groups of interested professionals

who felt the time was right to consider whether or not work should begin on standards for

software user interfaces. Some of these events were:

The rapid development and delivery of different graphical user interfaces for personal

computers. While these new GUIs promoted ease of use and increased productivity,

they brought inconsistencies, incompatibilities, and questions concerning the sound

scientific basis for some of the techniques used;

The emergence of standards for the hardware or physical aspects of the user interfaces

such as VDT character legibility, screen refresh rates, or keyboard characteristics, but

no attempts at standards for the cognitive aspects of the interface. The software issues

seemed to be overlooked;

Widespread concerns over health and safety issues relative to emissions from VDTscaused others to be equally concerned about mental stress resulting from poorly

designed software. While the concerns over emissions have largely gone away, the

concerns about mental workload and mental stress have remained.

These and other similar events prompted human factors and user interface design

professionals, working through the auspices of recognized standards organizations, to

create subgroups within the standards organizations that would specifically address the

software ergonomic issues. Two of these groups, one a US national group and the other an

international group (described below), are active today and are producing practical stan-

dards which can guide developers in developing human centered user interfaces.Today there are approved or draft standards for many aspects of software user inter-

faces. For example; how should menus be constructed, how and when should color be

P. Reed et al. / Interacting with Computers 12 (1999) 119142 121


4/24

used, what factors should be considered when using multi-media in the interface? Standards

on these and many other issues dealing with the design of user interfaces are available or

under development within formal standards organizations throughout the world.

2.1. International software ergonomics standards

International Standards Organization (ISO), Technical Committee 159 (TC159-Ergo-

nomics), Sub Committee 4 (SC4-Ergonomics of Human System Interaction), Working

Group 5 (WG5-Software Ergonomics and Human Computer Dialogs). WG5 first met in

1985 to decide what could or should be done in developing software user interface

standards. Other Working Groups within the parent organization SC4 were busy devel-

oping standards for hardware user interface issues such as front of screen characteristics or

keyboard design, but no one was working on software concerns. So, WG5 became an

official ISO group chartered to develop ergonomic software standards for the use of VDTs

doing office work. Over the next several meetings they outlined 8 additional software parts

(see Appendix A) to be added to the 9 hardware parts already defined and underway within

the overall SC4 standard ISO 9241Ergonomic Requirements for Office Work with

Visual Display Terminals (VDTs).

WG5 is comprised of professionals from government, academia, industry, and user

groups representing approximately 12 different countries. The standards are developed

within the committee using consensus as the method for agreement on proposed material.

Once the committee believes the draft standard is ready for public review, it is sent to the

member bodies of ISO where it is reviewed, voted, and commented upon three separate

times. After each vote and comment period, the draft is returned to WG5 where theymodify the document to accommodate as many of the comments as can be agreed upon.

Finally the standard is approved as an ISO international standard and is available for use

by anyone. Although ISO standards are intended to be used on a voluntary basis, some-

times they are cited in legislation or in a procurement process. When this happens the

standards cease to be voluntary and instead become a mandatory condition of doing

business.

A summary of the titles and content of the parts of the standard being developed by

WG5 can be found in Appendices A and B. Information about the availability of ISO

standards can be found at www.iso.ch or www.ansi.org or in the source column of

Appendix A, B and C. Also, Smith [6] provides an excellent treatment of ISO andANSI ergonomic standards for computer products.

2.2. U.S. national software ergonomics standards

The Human Factors and Ergonomics Society (HFES) Human Computer Interaction

(HCI) committee was formed in 1985 by a group of interested User Interface (UI) profes-

sionals who were concerned with the lack of empirical and other reliable evidence support-

ing proposed software user interface standards. Because of the concentration of user

interface design expertise in the US, it was believed that the US should have some

mechanism for contributing to the ISO work. This group initially consisted of approxi-mately 15 people from government, academia, and industry, all members of the HFES.

They banded together in the common interest of promoting UI standards based upon



5/24

scientific research and empirical evidence rather than abstract principles and ad hoc

current implementations. For the first several years the focus was on reviewing the ISO

documents and providing technical reports to ISO to aid them in their developing stan-

dards. These technical reports subsequently became the base material for several parts ofthe 9241 standard being developed by ISO WG5.

After years of contributing to the ISO activity, the HFES HCI committee felt the work

by ISO needed to be extended in several areas and updated to include more recent

technologies and research. In addition, they felt that a US standard was needed to represent

the United States perspective since the US is a major world wide developer of user inter-

face software. The HFES-200 committee was accepted as an accredited Standards Devel-

oping Organization (SDO) by the American National Standards Institute (ANSI). This

standard has been given the title HFES 200Ergonomic Requirements for Software User

Interfaces.

In 1994, the Human Factors and Ergonomics Society (HFES) obtained approval fromthe American National Standards Institute (ANSI) to develop a national standard that

addresses Ergonomic Requirements for Software User Interfaces. HFES-200 is the

title designating this project as well as the committee (formerly known as the HFES

HCI Committee) formed to develop the standard. Both ISO and the HFES also have

committees working on standards which address hardware user interface topics (HFES-

100see Appendix C).

HFES 200 will adapt parts 1217 of the 9241 ISO standard by updating them to

reflect current research, correcting any information considered to be in error, or

modifying recommendations that may have a uniquely U.S. requirement. In addition,

the standard will contain new material which was felt to be lacking in the ISOstandard. It will add major sections which deal with: use of color, computer acces-

sibility, and voice input/output. In its formative years, the HCI committee decided

that UI recommendations must be based upon a critical minimum level of sound

evidence before they could be included in their standard. A three level test of

evidence was developed which is applied to each proposed recommendation. Is

there documented scientific research? Is there documented empirical evidence? Is there

general agreement among the committee members and in documented the state of the

practice? One or more of these criteria must be affirmed before a recommendation can be

accepted.

The HCI committee works (like ISO) on a consensus method of seeking agree-

ment on the content of the draft standard. The committee will release the proposed

national standard for public review through a canvass process. This ensures that

interested and materially affected individuals can vote, comment and suggest

changes to the document. The HCI committee will review the votes and comments

and attempt to accept as many of the suggestions as can be agreed upon. If the negative

votes were in the majority, a second draft will be prepared and second canvass will be

conducted. This process continues until general agreement is secured. It is expected the

standard will be ready for the first canvass public review around late 1999. For a listing of

the content of HFES 200 see Appendices A and B. Information about the availability ofHFES standards can be found at www.hfes.org or in the source column of Appendices B

and C.



6/24

3. Using HCI standards in product/system development

Whether a projects software life cycle process is waterfall, spiral, or something else

altogether, it will accommodate and benefit from the application of HCI standards. Thissection provides some guidance for using standards during the software design and devel-

opment process. Additional guidance on humancomputer interaction design in product

development can be found in Karat [7] and Hix and Hartson [8]. Standards use involves

eight steps:

Define project objectives

Manage expectations

Select standard(s) to use

Tailor the standard(s) to your project

Determine compliance approach

Apply the tailored standard in designing the interface

Assess compliance of interface design

Conduct usability testing.

In addition, some specific success strategies can help projects get the most out of using

standards.

3.1. Define project objectives

The first step is to identify what the project aims to achieve in using HCI standards. Here

are some possibilities: Provide a familiar look and feel. Standardizing the look and feel of software allows

users to transfer the skills they learn on one piece of software to another. Training costs

are minimized.

Provide consistency. Standardization may occur in varying scopes. Examples include

the various components of an application, applications that will be used together, and

an operating system and all the software that runs on it. The more broadly a standard

can be applied, the greater the benefits.

Use human factors findings. Standards take advantage of the large body of human

factors research and accepted practice. The authors of the standards assimilate and

interpret the research, turning it into guidelines (best practice) for designers to follow.

Streamline development. Standards make many design decisions routine. This frees

designers to spend time on decisions that are more difficult or critical.

Evaluate usability. Standards provide one basis for judging the usability of products.

All else being equal, a product that meets an HCI standard should be more usable than

one that does not.

Comply with requirements. Standards compliance for the software may be required

by the buyer (e.g., in many US government contracts) or by law (e.g., in Europe).

3.2. Manage expectations

The second step is to manage project expectations, as many organizations may have



7/24

inflated ideas of what standards can do. First, be aware of some general limitations of

standards:

No guarantees: Following an HCI standard will not guarantee that a product or system

is usable. Standards cannot address sufficiently the information structure and content,often the hard part of HCI design.

Evolution: User interface knowledge and technology are continually changing and

improving, and so are standards. Although many of their guidelines are based on

consistent human characteristics and are enduring, others need to be replaced by

more current advice when new research and practices suggest better techniques.

There are times when a designer should try innovative solutions before the standards

have caught up. (See Section 5 for a more in depth discussion of this issue).

Generality: Standards provide guidance to meet a broad range of circumstances, not to

help design for specific intended users and their tasks. It is usually beneficial to tailor

the general guidelines into task- or system-specific design rules.

Leading edge? Standards provide guidance that is stable and well accepted. They are

not intended to be leading edge.

State-of-the-art? Long development times can put standards somewhat out of date

before they are released. It is not unusual for an HCI standard to spend several years or

more in development.

Strong? The consensus process involved in standards development ensures that

guidance is well thought out and meets the needs of many parties. However, many

proposed guidelines are controversial and are eliminated, made optional (should

rather than shall), or qualified. Platform-specific? HCI standards are designed to be platform independent. For exam-

ple, ISO 9241 applies to all GUIs and character based interfaces. Platform indepen-

dence ensures that standards are nonproprietary and can be applied widely; however,

the guidance must be general and can be difficult to interpret.

Universal? Design situations vary and a given recommendation is unlikely to be

optimum in all cases. Applying a standard requires the designers judgment.

3.3. Select standard(s) to use

Which standards should be followed, and whether they are required or optional, depend

on project application domain and location, and on project requirements. First, determine

whether you are looking to start with high-level principles, more specific guidelines, or

organizational conventions.

Principles specify high-level goals a user interface should meet (e.g. minimize memory

load). Guidelines specify techniques that can be used to accomplish these goals and that

have been shown to have appropriate usability characteristics (e.g., provide a keyboard

accelerator for every important menu item). Standards sometimes provide principles (e.g.,

ISO 9241 Part 10) and usually provide guidelines.In contrast, conventions specify which of a set of acceptable design techniques or

elements an organization is going to adopt and how it is going to apply them to its



8/24

Fig. 2. Factors concerning the users, their tasks and environment, and the hardware and software implementation drive th

project.


9/24

product(s) (e.g., use the letter T as the accelerator for the Templates menu item). Orga-

nizations typically develop their own design conventions.

Second, determine which type(s) of standard applies to your project:

International (e.g., ISO, ITU) standards are applicable worldwide. Regional (e.g., CEN) standards are applicable within a given region such as Europe.

National standards (e.g., ANSI, DIN, AFNOR, SIS, BSI, CSA) are applicable only

within a given country.

Military or government standards (e.g., DoD, MIL, NASA, ESA) are applicable to

products and systems being developed for specific military or government organiza-

tions.

Platform (e.g., Mac OS, CUA, Windows, Motif) style guides are applicable to all

applications developed for a given operating environment.

Independent (e.g., books by individual authors, company-specific) standards are either

very general or are applicable to the domain for which they are written.

3.4. Tailor the standard to your project

Tailoring is the process of creating a detailed, project-specific HCI specification from

the set of design guidelines in the selected standard(s).

First, select the guidelines that apply to your project. If, for example, the project team

selects ISO 9241 and the user interface is going to include menus but not direct manip-

ulation, Part 14 will directly apply and Part 16 will be irrelevant. Within Part 14, the

project team should identify the specific shalls and shoulds that apply to your users,their tasks and environment(s), and relevant implementation constraints. Fig. 2 shows the

factors that you should consider when determining which guidelines apply to your HCI.

See Section 4 for detailed information on determining applicability.

The other factor that will drive guidelines selection is compliance. You may need to

follow the shalls but not the shoulds; you may have to follow both shoulds and

shalls; or you may not be required to comply with any specific guidelines in the stan-

dard.

Next, translate the selected guidelines into specific design rules for your project. Review

the selected guidelines in the context of the users, their tasks and environment, your

implementation constraints, and the overall requirements for the product or system. Useprototyping as necessary to refine and validate the design rules.

3.5. Determine compliance approach

Standards compliance is most effectively applied during product design and implemen-

tation. If the product or system is not evaluated for compliance until the end of the

development cycle, changes will be difficult to make.

Compliance with some guidelines may be difficult to determine by examining the

interface. They may require knowledge of the users, the designers intent, and the devel-

opment and testing procedures. Examine the tailored standard to identify the best compli-ance approach for each guideline.

For each guideline in the tailored standard, determine the following:



10/24

Level of compliance required: What is the level of compliance that is desired or

required for your project? As mentioned earlier, you may have to follow just the

shalls, or both shoulds and shalls; or compliance may be at your discretion.

Method of assessment appropriate for that guideline: Is the guideline best suited forcompliance assessment by content analysis, documented evidence, observation, analy-

tical evaluation, or empirical evaluation?

The degree of compliance documentation also varies. Detailed information about

compliance with each guideline and the method used to judge compliance may be

required; a letter stating that the product is compliant may be required; or no statement

may be required.

As the standard is tailored, develop and maintain a checklist to help in assessing

compliance. If a checklist was provided with the standard, start with a copy of that and

refine it, or create a checklist for the selected standard(s). (See Section 4 for detailed

information on evaluating compliance).

3.6. Apply the tailored standard in designing the interface

The application of standards to HCI design is most effective when it is part of an overall

usability engineering process for the product or system. Standards may guide this process

as well as the characteristics of the interface itself. For specific information, see

ISO 13407, Human Centered Design of Interactive Systems.

First, ensure that everyone who will be involved in making HCI design decisions is

familiar with the tailored standard and the checklist.

As you make, modify, and implement HCI design decisions, consult your tailored

standard and your checklist to ensure that your design follows the standard.

Issues will probably arise during implementation that will necessitate revising the

tailored standard. Hardware and/or software configurations may change; functions may

need to be added or removed. User interactions (e.g., messages about the status of the

system or the software) will need to be designed that could not have been foreseen or

designed in advance. A process for revising the tailored standard should be established, in

which all parties affected by a change are informed about it and (where appropriate) are

involved in determining design decisions.

3.7. Assess compliance of interface design

As stated earlier, compliance with some guidelines may not be assessable from an

examination of the interface, but may require knowledge of the users, the designers intent,

and the development and testing procedures. (See Section 4 for detailed information on

assessing compliance).

3.8. Conduct usability testing

Standards can promote usability but by themselves cannot guarantee it. Even with theassistance of organizations (such as test houses) or tools (such as checklists or software

products) that can verify or even certify standards compliance, you still cannot be certain



11/24

that your product or system is adequately usable until you have subjected it to usability

testing.

After assessing standards compliance, conduct usability testing to evaluate how well the

design, prototype, or built product/system supports the effectiveness, efficiency, and satis-faction of the intended user population as they perform their tasks in the intended context

of use. Have a representative sample of users perform realistic tasks, and collect two types

of information:

Qualitative: The problems and sources of confusion that the users have in doing the

assigned tasks, and an identification (if possible) of HCI design changes that could

alleviate the problems.

Quantitative: Measures of the effectiveness (completeness and accuracy) and effi-

ciency (resources expended with respect to completeness and accuracy) of the user

tasks performed, and of the users satisfaction in using the product or system to perform

these tasks.

Use this information to identify and prioritize the areas of the HCI that need further

change, and to drive the design of the necessary changes. For more information on

usability and usability evaluation, see ISO 9241, Part 11.

3.9. Success strategies

Here are some strategies for success when applying standards:

Follow the guidelines in the standard throughout product design and development;

dont wait until the end to think about standards Make sure all appropriate members of the design team are aware of the standard. It

seldom works well to depend on a standards czar to achieve compliance.

Use tools whenever possible. For example, some GUI builders take care of setting up

the menu bar, adding keyboard accelerators, etc.

Do not follow standards blindly. Conduct usability testing to verify that the guidelines

are correct for your situation and that the interface is usable.

4. Compliance and conformance evaluation techniques

A number of methods have been defined in the field of software ergonomics for deter-

mining the applicability of a standard or guideline and for determining whether the

requirement or recommendation has been met. The particular choice of a method depends

on various project drivers and constraints. Established methods (extracted from the human

factors literature on evaluation techniques) include:

Content analysis This refers to the analysis of any documents which may describe

the general and specific properties of user interface and the system. Such documents

may include design documents containing system and user requirements, manuals, user

guides, etc. Content analysis is used to determine whether a standard or guideline isapplicable on the basis of its relevancy to the particular system under development or

being evaluated.



12/24

Documented evidence This refers to any relevant documented information about the

task requirements or characteristics, flow of work, user skills, user aptitudes, existing

user conventions or biases, test data from the design of similar systems, etc. Such

information may be used to determine whether a given recommendation is applicableas well as whether a standard or guideline has been met due to previously document

supporting evidence as to the appropriateness of a particular solution.

Observation This consists of the examination or inspection of a user interface

element for the presence of a particular observable property (e.g., applicability is

based on the observation that a source document is used for input or a recommendation

has been met because it is observed that instructions are provided on the form). Obser-

vations can be made by anyone who has the necessary skill to systematically check the

interface and determine if it has the particular properties associated with the applic-

ability of, or compliance with, a given conditional standard or recommendation. Due to

their obvious nature, such observations readily can be confirmed by another person. Analytical evaluation Refers to Informed judgments concerning the properties of

a user interface by a relevant expert (i.e., of those properties). This method is typically

used for the evaluation of properties which can be judged only in the context of other

information or knowledge. In addition, analytical evaluation may be appropriate when

the system exists only in terms of design documents, user populations are not available

for empirical evaluation, or time and resources are constrained. Analytical evaluation

can be used to determine whether a particular recommendation is applicable, e.g., to

determine if a temporary save function would be appropriate to the users task. In

addition, analytical evaluation can be used to determine if a particular standard or

recommendation has been met, e.g., analytical evaluation might be used to determinewhether a form has distinctive labels. In the above case, distinctive is the judgmental

aspect. Analytical evaluation can be performed by any suitably qualified person who

has the necessary skill and experience to judge the relevant property of the interface.

Where these properties concern the application of ergonomic principles, the expert

should possess appropriate skills in software ergonomics. If the properties concern

the work environment, system characteristic, or other aspects of the design, the

judge needs to be an expert in the particular relevant domain. It is important to note

that analytical evaluation can verify the tenability of a design, but cannot validate the

design. Validation can be accomplished only by using empirical evaluation.

Empirical evaluation Is the application of test procedures using representative end

users to determine the applicability of a standard or recommendation. This method is

most appropriate when a prototype or the actual system is available and potential or

actual user population representatives are available. Many kinds of test procedures

could be used, but in each case the test subjects must be representative of the end

user population and be of sufficient number that the results can be generalized to the

user population as a whole. For example, empirical evaluation to determine the applic-

ability of whether error feedback should be provided as soon as the user completes a

field in a form could be done by having typical users enter and correct field values under

both the condition of providing the feedback immediately after the field is completedand providing the feedback prior to transmitting the completed form. The task perfor-

mance of end users using a software application could be analyzed to determine



13/24

whether the various standards or recommendations have been met. Such tests could be

performed both during the development process (e.g. by prototyping) and after the

design and implementation of the system (e.g. by system evaluation techniques) and

could be based on both objective and subjective user data. Although empirical evalua-tions typically are used to determine whether standards/guidelines have been met by

comparing the test results against specific software ergonomic standards or recommen-

dations, it also may be necessary to evaluate test results in terms of effectiveness. That

is, the dialogue supports the user in his/her task in a manner which leads to improved

performance, results in a difficult task being performed with less difficulty, or enables

the user to accomplish a task that he/she would not have been able to otherwise). It

should be noted that empirical evaluation should be conducted by individuals posses-

sing appropriate skills in testing methodology and evaluation techniques.

The above methods for determining conformance can be applied to any evaluation of

software ergonomic standards or guidelines. Although shalls are given more importance

than shoulds in most standards, to optimize the quality of the user interface it still is

important to evaluate whether the applicable shoulds have been met. In addition, some

software procuring organizations may require that all of the recommendations as well as

the requirements be met.

4.1. Evolution of the conditional approach for specifying software ergonomic

standards/guidelines

The conditional approach to stating ergonomic guidelines for software applications

was initially developed by the Human Factors Society Human Computer Interaction (HCI)Standards Committee (see Williams [8]). During the process of defining guidelines rele-

vant to user interface design from various literature sources, the HCI Standards Committee

noted that a major problem with most proposed guidelines was that the applicability of a

particular guideline depended on a number of variables. The most common variables were:

the type of user (e.g. expert vs. novice), the task activity being accomplished, the system

configuration being used, and the environment. As a result, the HCI Standards Committee

began to use an ifthen structure to state the conditions relevant to the applicability of a

guideline where ever sufficient evidence allowed. Since a number of the initial drafts of the

dialogue parts of ISO 9241 were submitted by the HCI Standards Committee as a US

contribution, this approach carried over into the software parts of ISO 9241.

The conditional approach is also used to determine whether or not a major section/

clause of a standard is applicable. For example, if the application does not using pointing

devices, recommendations concerning the use of pointing devices would not apply. The

following provides some examples of conditional statements extracted from some of the

ISO 9241 software parts:

User characteristics:

If casual or intermittent users may enter data on the form, instructions should be

provided on the display (or easily accessible through a help facility).If users are experienced and/or need fast access to specific menu options, one or both

of the following methods should be applied:



14/24

Task characteristics:

If the task requires error management by the user, the dialogue should provide a

means (information and/or function) of enabling the user to continue.

If information concerning unavailable options is not required for the task or othersupporting activities (e.g., training), only options available to the user.

System capabilities:

If a system or application has modes (i.e. a specific user action has different results

depending on the state of the system), users should be able to discriminate the current

mode.

If the system response to option execution will be delayed (more than three seconds

after initiation), an indication should be provided to the user of the system.

Interface design characteristics:

If options are placed in columns, options and groups of options should be visually

distinct from one another and should be arranged to minimize search time.

If brief error messages are displayed, users should be able to request more detailed

on-line information or should be referred to additional off-line information.

4.2. Traditional view of conformance to a standard and problems for software

Most standards, particularly hardware standards, specify attributes that can be objec-

tively measured or directly observed in some way (e.g., voltage, dimensions, pressure).Conformance to such standards can be demonstrated by measuring the product or device to

determine whether it complies with the requirements stated in the standard. In addition,

these standards can establish those attributes that are required (shall statements) from

those that are desirable, but not required (should statements).

Unfortunately, in the domain of software user interface standards, it is rarely possible to

unambiguously state what is required and what is desirable. The conditional nature of

many software ergonomic standards statements requires the applicability of a particular

statement to depend on a number of variables. Due to these often-complex dependencies,

most software ergonomic experts are unwilling to designate such statements as a require-

ment because of the difficulty of precisely determining the dependencies. An additionalconcern is that developers will only pay attention to requirements and ignore recommen-

dations. It is also important to note that organizations that utilize a consensus process for

developing standards (e.g., ISO and ANSI) have a much more difficult time obtaining

agreement on shalls than do organizations which follow a less open review process (e.g.,

ITU, government agencies).

4.3. A pragmatic approach to conformance with software user interface guidelines

Utilizing the conditional approach and the evaluation methods described above, a two

step process for determining conformance to software ergonomic standards/guidelineswas developed based on extensive work in ISO. The first step of this process is to deter-

mine the applicability of the standard or guideline and the second step is to assess



15/24

Fig. 3. ISO 9241 conformance evaluation process flow chart.


16/24

whether the requirement/recommendation has been met by the developer, i.e. compli-

ance (see Fig. 3 for a flow diagram of the process). Applicability can be determined both

as part of the early design process when the designer is specifying elements of the inter-

face, and later by both the designer and evaluators as part of any heuristic evaluation of thedesign. It is strongly recommended that the evaluation be made using the most detailed

method which is appropriate to the requirement or recommendations. For example, it

would not be necessary to do usability testing (empirical evaluation) to determine that

menu options keyboard equivalents (underlined character) are shown when it can be

readily observed. The intent of the pragmatic approach is to minimize the time and effort

required of an organization to determine whether they comply with a standard or guide-

line. The pragmatic approach to conformance has been incorporated in the software parts

of ISO 9241 and is planned to be utilized in the HFES 200 standard as well.

4.4. Conformance to the ISO 9241 software standard

All of the dialogue parts of ISO 9241 now use a common approach to conformance as

was agreed by the national bodies on Part 14. This approach includes the two step process

(Applicability and Adherence) as a sample procedure, and allows users of the standard to

use other processes. It should be noted that the term adherence was agreed because

compliance was thought to be too related to requirements and would not apply to

recommendations. The only actual statement of conformance in 9241 Part 12 through

17 is: If a product is claimed to have met the applicable recommendations in ISO 9241-

(14), the procedure used in establishing requirements for, developing, and/or evaluating

(menu dialogues) shall be specified. The level of specification of the procedure is a matterof negotiation between the involved particles.

The two step process recommends that the user of the standard first determine which of

the recommendations are applicable and then determine whether those recommendations

that are deemed applicable have been met. It should be noted that the approach described

in ISO 9241 suggests applying the method most relevant and cost effective to determining

the applicability and adherence for a particular recommendations. For example, if the

recommendation specified an attribute that could be directly observed (e.g. left justifica-

tion of a menu option list), the method indicated as the most relevant would be observa-

tion. Thus the sample procedure is intended to be as cost effective as possible.

Several other means for determining conformance also have been devised. Some of the

test houses in Europe (e.g. TUV Rheinland) have been evaluating products for their

conformance to the software as well as the hardware parts of ISO 9241. The actual method

used for these evaluations is not known since the test houses have not revealed the exact

procedure used. A somewhat different approach has been used by Lloyds Register who

has developed a procedure to assess the application of the ISO 9241 software parts as part

of the overall software quality assessment procedure. Another approach to evaluation of

the software ergonomic parts is the 9241 Evaluator developed in Germany (Oppermann,

R and Reiterer, H. [9]). In their paper, Oppermann and Reiterer provide an overview of

different evaluation techniques and describe the 9241 Evaluator as an expert-basedmethod for conformance testing of the 9241 standard. While an important step towards

the development of a performance support system for standards evaluation, the 9241



17/24

Evaluator is intended to be used only by experts after the system has been developed (at

least to a prototype stage). However, the ISO 9241 software parts state that the intended

users include designers as well as evaluators and that the standards should be used during

all stages of system design and development.All of the approaches to conformance described above satisfy the conformance clause

stated in the software parts of ISO 9241. The HFES 200 Committees current direction is

to use the same basic approach to conformance as stated in ISO 9241. However, it is

important to note that many organizations (particularly in the United States) are stating the

same recommendations as can be found in the ISO and emerging ANSI software ergo-

nomic standards as requirements (i.e., shall statements). For example, many of the

statements found in the recently released Federal Aviation Administration The Human

Factors Design Guide section on human computer interfaces contain shalls.

4.5. Utilizing the pragmatic conformance approach for evaluating other softwareergonomic standards or guidelines

As noted above, the two step process and the methods used to determine applicability

and adherence can be used to evaluate many types of standards and guidelines. The

process is particularly relevant where standards or guidelines have been qualified in

terms of the conditions in which they apply (e.g., conditional recommendations as

described above). Currently many style guides require users to complete a check list

indicating both which style guide requirements and recommendations apply and which

have been met. Although most style guide requirements and recommendations are look

and feel type statements which usually can be evaluated for compliance by observation,many recommendations concern design strategies which could be evaluated by means of

many of the other applicability and adherence methods described above. The pragmatic

conformance approach, particularly when a checklist is incorporated as is used in Annex A

of the 9241 software parts, also provides an audit trail which can be used to satisfy

ISO 9000 and ISO 9001 requirements.

5. Future challenges and the evolution of HCI standards

Two issues commonly identified in discussions of standards are: (1) Standards are not

useful., and (2) Standards stifle creativity. A fundamental issue underlying each ofthese statements is the belief that technology is changing so rapidly and standards take so

long to create that they either do not address the current set of technologies in which design

is taking place or they should not address the technologies. The argument that they should

not address the current environment is based on the assumption that the behavior that we

need to measure in order to provide empirical justification for standards is only now

emerging for these technologies. Standardizing prematurely may result in standardization

of the wrong things. Further, it would follow that creating standards that apply to future

technologies should be impossible.

5.1. Moores law, the Internet, and HCI standards

There is a great deal of face validity to these arguments. Many human computer



18/24

interface designers have moved from programming with punch cards and paper tape to

designing animations, collaborative video conferencing, and virtual reality interfaces,

within the time it takes a child to complete their schooling. The explosive growth of the

Internet and novel technologies based on the Internet is well known. A recent conferencemarking the first 50 years of computing invited pioneers and seminal thinkers in the

computing field to use the last 50 years as a foundation for predicting the next 50. Bell

and Gray [10] pointed out that with processor speed, storage capacity, and transmission

rates doubling every 18 months consistent with Moores Law, the clear trend is the emer-

gence of essentially zero-cost, specialized, fully networked computers embedded in

virtually everything around us, and attached to our bodies. Bell and Gray state The

only limits will be our ability to interface computers with the physical world-that is, to

design the interface between cyberspace and physical space. It would appear that

virtually any interface that can be conceived will be possible to build.

Nathan Myhrvold [11], at the same conference, pointed out that software is also follow-ing Moores Law. He stated So we are talking 20 years hence about computers being able

to do in 30 seconds what one of todays computers at a comparable price would take a year

to do. But 40 years hence we are talking about a computer doing in 30 seconds what one of

todays computers would take a million years to do. So what will we do with all that

power? Well, the answer is software. The increasing pace of software evolution will drive

corresponding changes in HCI design. Those looking at a standards process that attempts

to integrate years of design guidance about what works and does not work in user inter-

faces with a given technology may be justifiably skeptical, especially given that it may

take years for the standard to be approved once it is defined. We already know that in the

time it has taken to approve standards addressing personal computer technology, we havemoved into designing for virtual reality. As we move forward, we will be designing the

next generation of interfaces without the benefit of standards utilizing behavioral research

focused on human interaction with next-generation technology. Fortunately for designers,

human capabilities and limitations in perception and performance do not evolve as rapidly

as modern computing technology!

Practically speaking, therefore, can standards be written in an environment driven by

Moores Law? Can standards be written that will be useful for designers working with

technologies emerging for the Internet, computers appearing in novel forms such as tele-

phones and televisions, and technologies with which we have had even less experience?

Should they be written? We believe they can and should be written. For useful standards to

be written for the next generation of interfaces, however, we will need to continue to

improve the process of writing and maintaining the standards and the sophistication of our

taxonomy of standards will need to increase.

5.2. Lessons we have learned

In the last few years, several important lessons have been learned that will shape

effective standards activities in the future. First, a key assumption that we reached in

developing the standards for ANSI and the work for ISO is the assumption of humility.For standards to accommodate rapidly changing circumstances, they need to build on what

should be done more than what shall be done. In other words, they need to provide



19/24

guidance and flexibility, such that the final arbiter of a design is the user. Users adapt

effectively to the situation in which they find themselves. The combination of the design

elements and the structure of the design as it supports the users task can have as much

influence on user satisfaction and performance as individual aspects of the design that havebeen designed according to a standard. Applying the guidelines may even require tradeoffs

between guidelines, such that a given guideline must violated in order to apply a guideline

that is more important in the circumstance. The conditional approach to specifying

design guidance must be extended to provide new guidelines that better address the

context of application will be needed, and we will need to continue to write guidelines

that can be adapted to and provide guidance in changing circumstances.

A second lesson that will shape standards in the future is that it is important to continue

to improve our ability to define standards that are useful and yet that are as technology

independent as possible. Fundamentally, human beings do not change as rapidly as the

technologies they are using. At a very basic level, our needs remain the same. The humanperceptual system works the same now as it did 15 years ago. The properties of the

cognitive system remain the same. This, I believe, is the key to standards in the future.

If we can define standards in terms of the reliable phenomena of human behavior, and

eventually in terms of any theoretical consensus, we are more likely to define standards

that are useful when applied to emerging technologies. This implies that a more tightly

integrated dialogue between academia and industry will be needed, with research direc-

tions being set at least in part as a result of direction from those writing or requiring

standards.

A third lesson is that the best standards emerge from data rather than politics. While we

began the standards effort attempting to build standards largely from well controlledlaboratory studies, over time we have found that data and conclusions emerging from

practice are also important in shaping a useful standard. Harnessing the wealth of data that

is automatically generated in the current environment of design mutation and evolution

on the Internet and in the world of consumer products would be expected to further inform

the standards process. It would be reasonable to predict that a day of activity on the

Internet results in more people being exposed to more design alternatives than all the

labs where testing is currently taking place could produce in a year.

5.3. Improving our approaches to user interface guidelines and standards

Standards efforts, like usability work in general, may need to harness the trend of trying

a design and rapidly fixing it in response to feedback from the field in order to respond

more quickly to emerging technologies. Indeed, one approach to increasing the ability of

standards to reflect emerging technologies is to explore new ways of developing standards,

new processes. Perhaps the solution is in finding new ways to involve the stakeholders in

the development process, finding ways to access emerging data being collected in other-

wise inaccessible corporate labs, and shortening cycles of approval. The challenge, of

course, is that national and International standards involve consensus building. It is not

clear that we have identified the processes yet that meet the needs of technical standardsdevelopment as well as the political and cultural realities of standards approval.

Another approach for developing useful standards in the future is to define standards



20/24

that are virtually technology independent. These standards would define how task proper-

ties should influence design primitives, and their interactions. An example of this would be

a set of standards that recommends list length for the task of selecting a category from a list

presented auditorially, when the contents and names of the categories are not known aheadof time. The approach that would be most effective to these standards would be the

integration of behavioral science findings and theory focusing on the capabilities and

limitations of human beings, and applied research identifying the fundamental elements

of tasks as shaped by the properties of the user.

The next level of standards would be standards applying to combinations of these

primitives associated with generations of technologies. An example of this kind of stan-

dard would be standards that describe the design of a form that supports pointing and

clicking, or the category of window currently typically used in Macintosh and Windows95

applications. This is the level of standard that is most typically defined today. These kinds

of standards currently are defined based on research using the technologies for which thestandards are being written (which means the standards appear well after they are needed

by designers), and based on the years of experience and logical thinking of those writing

the standards. In the future, this level of standard could initially be shaped in part by

applying the higher level standards to relevant properties of the specific technology

(coupled with that same experience and logical thinking that is currently being applied).

The resulting standards could be validated and extended (where the higher levels do not

sufficiently address the design options) through usability studies of specific implementa-

tions. With a sound base of technology-independent standards, it should be easier to

produce flexible, useful standards for emerging technologies in a timely way.

This structure depends on a richer understanding of human behavior in applied settingsthan we currently have, and it has the potential to provide a mechanism for useful design

guidance to precede technology-dependent data collection. A process governor will have

to be added in order to restrain standards bodies from spinning too far from the empirical

base underlying the two levels. While lower level standards should be consistent with

higher level standards, they would also be expected to be inherently technology depen-

dent. These standards would include standards that attempt to address consistency of

applications built for a given operating system and standards that attempt to define unique

brands within such environments.

An area that is missing from even this view of standards, however, is the relationship

between design for usability and emotion. Increasingly it is being recognized that even a

successful business application needs to be attractive and often fun. Applications built

using emerging multi-media technologies increasingly explicitly engage the total person.

When the aesthetic elements of the design result in an application being learned more

completely and used more often, these elements should be included in the definition of

usability and should be instantiated in standards. Macroergonomic considerations of when

applications are adopted and product usability as it interacts with brand identity also

suggest that the definition of usability will be broadened in the future. To the extent

that standards serve as an instantiation of accumulated knowledge about how to create

usable design, the standards will also address a wider range of design considerations.Extending our knowledge of the relationship between aesthetic elements, design that

impacts enjoyment, the creative design process itself and performance, and design as it



21/24

relates to the diffusion of technology is needed. Combined with the standards approaches

that we have just defined, these information sources will enrich the body of available

design guidance, and produce a set of standards that are technology independent and that

can be effectively applied to technologies that will emerge in the future.

Appendix A. Process standards


Process standards (HCI/GUI design, evaluation, and verification)

Standard Title Contents Source

ISO 9241 (Part 11) Ergonomic

requirements for

office work withvisual display

terminals

9241 Part 11: guidance on

usability

International Standardization

Organization, 1 Rue de Varembe

Case Postale 5b, CH-1211, Geneva20, Switzerland;

Tel.: 41-22-734-1240;

URL: www.iso.ch

MIL-STD-1801 Usercomputer

interface

(Various requirements for

verification)

Wright-Patterson AFB ATTN:

ASD/ENES Wright-Patterson AFB,

OH 45433

ISO/CD/13407 Human-centred

design for

interactive systems

Assessing the benefits of

human-centred design.

Interactive system

development processes and


Planning the human-centred

design process. Human-

centred design activities.

Managing and documenting


International Standardization

Organization, 1 Rue de Varembe

Case Postale 5b, CH-1211, Geneva

20, Switzerland;

Tel.: 41-22-734-1240;

URL: www.iso.ch


22/24

Appendix B. Software user interface standards

Software standards and guidelines (HCI/GUI appearance and behavior)

Standard Title Contents S

ISO 9241 (Parts 10-17) Ergonomic requirementsfor office work with

visual display terminals

Part 10: Dialogue principles.Part 11: Guidance on usability.

Part 12: Presentation of

information. Part 13: User

guidance. Part 14: Menu

dialogues. Part 15: Command

dialogues. Part 16: Direct

manipulation dialogues. Part 17:

Form filling dialogues.

IO

V

C

S

T

U

HFES-200 ANSI Project (under

development; canvass draft due

in late 1998)

Software user interfaces Same as ISO 9241, Parts 10-17,

plus color accessibility for

people with disabilities voice

input/output

H

E

9

T

U

ESD-TR-86-278 Guidelines for designing

user interface software

Data entry, data display,

sequence control, user

guidance, data protection, data

transmission

E

A

C

M

MIL-STD-1472D (Section 5.15) Human engineering

design criteria for

military systems,

equipment and facilities

Data entry, data display,

interactive control, feedback,

prompts, defaults, error

management

C

m

A

A

P1201.2 This failed two ballotsand was withdrawn. It is here

because other documents may

reference it.

Graphical user interfacedrivability Keyboards, pointing devices,menus, controls, windows, user

guidance, common user actions

IH

P

T

f


23/24

Appendix C. Hardware user interface standardsHardware standards and guidelines

Standard Title Contents Source

ANSI/HFES-100-1988

(currently under revision)

Human factors

engineering of visual

display, terminal

workstations

Working environment visual

display design, workstation

design, keyboard design,

measurement techniques

Human

Society

Monica

Tel.:

Fax:

URL: w

ISO 9241, Parts 3-9 Ergonomic requirements

for office work withvisual display terminals

Visual display requirements,

keyboard requirements,workplace requirements,

environmental requirements,

display requirements with

reflections, requirements for

displayed colours, requirements

for non-keyboard input devices

Internat

OrganizCase Po

Geneva

Tel.:

URL: w

MIL-STD-1472D

(selected parts)

Human engineering

design criteria for

military systems,

equipment and facilities

(various) Comma

comma

Redston

NASA-STD-3000

(selected parts)

Mansystems

integration standard

(various) Johnson

SP34 N


24/24

References

[1] J. Nielsen, Usability Engineering, Academic Press, New York, 1993.

[2] D. Mayhew, Principles and Guidelines in Software User Interface Design, Prentice-Hall, Englewood Cliffs,

NJ, 1992.

[3] Apple Computer, Macintosh Human Interface Guidelines, Addison Wesley, Cambridge, MA, 1995.

[4] ISO 9241-14: 1997 (E), Ergonomic requirements for office work with visual display terminals (VDTs):

Menu dialogues, International Organization for Standardization, Geneva, Switzerland, 1997.

[5] C.M. Brown, Human-Computer Interface Design Guidelines, 1988.

[6] W. Smith, ISO and ANSI Ergonomic Standards for Computer Products, Prentice-Hall, Upper Saddle River,

NJ, 1996.

[7] J. Karat (Ed.), Taking Software Design Seriously: Practical Techniques for HumanComputer Interaction

Design Academic Press, New York, 1991.

[8] J.R. Williams, in: G. Salvendy, M. Smith (Eds.), Guidelines for dialogue design, in Designing and Using

HumanComputer Interfaces and Knowledge Based Systems, Elsevier Science, Amsterdam, 1989.

[9] R. Oppermann, H. Reiterer, Software evaluation using the 9241 Evaluator, Behaviour and InformationTechnology 16 (4/5) (1997) 232245.

[10] G. Bell, J.N. Gray, The revolution yet to happen, in: P.J. Denning, R.M. Metcalfe (Eds.), Beyond Calcula-

tion: The Next Fifty Years of Computing, Copernicus, New York, 1997, p. 3.

[11] N. Myhrvold, The future of software, the software industry, and possibly, the features of Windows 47,

Speech given at ACM97, The Next 50 Years of Computing, March 3, 1997. Viewable at www.vxtreme.

com/live/acm97/archived/nathanmyhrvold/nmyhrvold.html.


Documents

User Interface Guideline