Assessing the Usability

of Human-Computer

Interfaces

Timothy E. Lindquist, Virginia Tech

Engineers may be able todesign a better interface if theytake into account the control

structures underlying theinterface syntax.

T here are different criteria to eval-uate the human-to-system inter-

face. We use two in this article, "ef-fort-to-learn" and "effort-to-use."Attention has already been devoted toevaluating the "effort-to-learn" and"effort-to-use" the dialogue syntax.Examples include work on help facil-ities' and the Dialogue ManagementSystem.2 Another approach, such asthat of Ledgard, 3 has been to considernaming conventions in command lan-guages. These studies focus on theform, content, and devices used inhuman-computer communication.We think there is an additional aspectthat is equally important.

In this article, we want to examinethe dialogue structure underneath thesyntax whose logic must be traversedeach time human-computer interac-tion takes place. This structure is re-vealed if we characterize the human-computer interface in the same way wecharacterize software-to-software in-terfaces. Specifying software-to-soft-ware interfaces involves specifyingeach of its components.4 Software in-terface components are the routinesinvolved and the data elements sharedat the interface. Accordingly, in ahuman-computer interface, the com-ponents are

(1) the commands invoked by theuser,

(2) the manual actions performed bythe user,

(3) the information manipulated bycommands or the user, and

(4) the decisions made by the user todetermine the dialogue sequence.

The specification of each interfacecomponent must also include its syn-tax, its semantics, and its protocolswith other interface components.

For a human-computer interface,the syntax includes the command entryformat, for example, menu, paramet-ric, or fill-in-the-blanks; the device,such as keyboard, mouse, and voice;and the responsiveness of the system,for example, immediate commandcompletion.The semantics provide the meaning

of individual interactions by detailingthe functionality of interface com-ponents. Command semantics de-scribe, for example, what a commanddoes and what a response meansrelative to the use of the command.

Protocols with other interface com-ponents are rules that govern howusing one interface component affectsusing another. For example, a com-piler on a time-sharing system is not in-voked until a source file has beencreated. To use a debugger, the sourceprogram must first be compiled withspecial options before interactivelyexecuting statements, examining data,or changing data. Protocols make ex-plicit any control or data dependenciesamong interface components. While acontrol dependency might specify thatone action must be performed beforeanother, a data dependency indicatesthat some information must be avail-able in a specific state before an actiontakes place.

In this article, we present this struc-ture underlying the dialogue between ahuman and computer,5 and, using aprocedural language, we encode thesemantics and protocols. Included in

0740-7459/85/0001/0074$01.00 © 1985 IEEE IEEE SOFTWARE74

the representation is the sequence inwhich the user performs actions, alongwith the decisions determining the se-quence. Called a dialogue structure,the representation operationally con-veys interface semantics and protocolswithout addressing syntax. We alsopresent, with examples, the criteriathat could be used to evaluate thedialogue structures.

Coding the dialoguestructureThe semantics and protocols of the

interactions that take place between auser and the computer can be depictedin an algorithmic form very similar to ahigh-level language program. The al-gorithms characterized in this sectiondescribe the human-computer inter-face to a system by detailing what theuser must do in order to employ thecomputer to solve a specific problem.The step-by-step actions performed bythe user, whether carried out by thecomputer or manually by the user, arecoded in the dialogue structure.

The language used to express dia-logue structures has a syntax that islimited to- definitions and controlstatements. Definitions appear at thetop of the algorithm and provide ameans to associate identifiers with realworld entities. The body follows thedefinitions and describes actions takenby the user in performing the task.

DIALOGUE STRUCTURE title(parameters) I'DECLAREname-and_actiondefinitions

BEGINbody

END

Declaration section. Identifiers areused to name both data manipulatedand actions invoked by the dialoguestructure.

Name definitions. When describingthe human-computer interface to aneditor, the following definitions wouldoccur.

current : line;search.Ada: file OF line;

Here, the names current and search.Ada are being defined. Current namesa line in the text file search. Ada, which

is to be manipulated during the editingsession. The definitions include thetypesfile and line to show how they arerelated. That is, files are orderedgroups of lines that can be accessed bystoring and retrieving information lineby line. It is appropriate that we omit amore detailed definition of line sincelines are atomic units of manipulationfor the editor being described.When constructing dialogue struc-

tures, data names should be definedfor each entity that the user must re-member. For example, suppose thatan editor requires that the user find aline number and remember it for all ac-tions to be performed on the line. Thenit would be appropriate for the namecurrent to be defined. In fact, we mighthave the following sequence of actionsin the dialogue structure.

current:= findLnext(current, misspell);change(current,misspell,correct_spell);

In an editor that -doesn't require theuser to remember line numbers, thesame actions may be performed with-out explicitly including the name cur-rent, as follows.

findcLnext(misspell);change(misspell,correct .spell);

Action definitions. Definitionsshould exist for each operation that theuser must perform while completing atask. Actions are activities performedby the user, or those performed by thecomputer at the user's request. For ex-ample, suppose a user must sort a listof items by hand and then request oneor more computerized operations foreach element on the list. Sorting is anaction that, in this scenario, must becarried out by the user. Thus, this ac-tion would be defined as a humanoperation in the following manner.

HUMAN-OPERATION sortiist-sort a list of requests by key

The definition, with appropriate com-mentary, would indicate that the hu-man will manually sort incomingrequests to produce an ordered, list ofrequests.

Suppose the user is presented with alist that is a group of requests to an in-ventory system. One request might befor temporary storage of an incom-plete part. Computerized operationsthat carry out such a request could in-volve a search for adequate storagefollowed by a move of the material to aselected location. Name definitions forsuch actions are

COMPUTER-OPERATION space;COMPUTEILOPERATION store;

Although actions are usually in-voked wvith arguments, an action's dec-laration does not specify parameters.A single action might have differentnumbers of arguments and types ofarguments each time it is invoked.Allowing default values for param-eters may cause differing numbers of

An actions declaration doesnot specify parameters.

arguments. An editing command maynot require that the user enter a linenumber, and if none were specifiedthen the current line would be as-sumed. Another example, indicatinghow different types of arguments maybe used, is the range specified in someeditor commands. This argumentmight be either a single line number, arelative line number, a range of (rela-tive) line numbers, or a range delimitedby search strings.When constructing dialogue struc-

tures, we must consider the detail ad-dressed by the operations (that is, thelevel of abstraction to which the dia-logue structure should be written). Thelevel used affects software analysis.Two distinct dialogue structures withdifferent levels of abstraction couldcorrectly specify a human-computerinterface. Each would result in a dif-ferent complexity when using softwareanalysis. For example, the human op-eration sort-list, discussed above,could be further decomposed into themotor and mental discriminationsneeded to perform the sort. If thedialogue structure were expressed at

January 1985 75

this lower level, its complexity wouldbe greater (when measured by soft-ware metrics) than if written using thesort-list action.

Although no rule exists to assurethat all dialogue structures for a singleinterface define the same actions,guidelines can be used to assure a simi-lar level of abstraction. For computeroperations, each action should be at ahigh enough level to generate at leastone user-viewable response (for exam-ple, an error message or normal re-sponse), and each action should be at alow enough level to correspond to asingle unit of user input (for example,a command line). The level of abstrac-tion for human operations depends onthat used for computer operations.Human operations should be definedat the highest level possible that allowsinteraction with computer operations.The inputs-to and outputs-from com-puter operations should be generatedby, or used in, human operations.These guidelines define an acceptablerange for the level of abstraction. Forcomputer operations, the range isbounded by units of input and output,and, for human operations, the rangeis determined by data jointly manipu-lated by the user and computer. Whencomparing different interfaces for thesame task, this bounded range on thelevel of abstraction assures adequateconsistency for comparison.

Instructions in a dialogue structure.The instructions describe the use of aninterface by detailing the sequence in-which actions are to be performed.The names defined in the dialoguestructure are manipulated using selec-tive and iterative program statementsto indicate decisions, and specify theorder in which human-performed andcomputer-performed actions takeplace.

The basic statement of the dialoguestructure's body is the invocation of anaction, which is denoted by parentheti-cally including the arguments togetherwith the action's name. Although wegave examples of invocation above,two further examples distinguish func-tional from procedural use of actions.Suppose that the following name def-

initions exist in a dialogue structureindicating that the user's interface in-cludes a computer operation to re-name a file and a human operation toselect the next file in a directory. Thedefinitions for these names might be

COMPUTER-OPERATION rename-file;HUMAN-OPERATION select-next;

The action rename-file takes two in-puts, old and new, and although it hasno explicit outputs, its procedural ef-fect is to associate a new name with anexisting file. The action select-nexttakes as input a directory of file-namesand functionally returns as output asingle (selected) file-name. The bodyof a dialogue structure defining thesenames might contain the following un-related invocations, which show howactions may be invoked either func-tionally or procedurally.next-file: = select-next(my-directory);rename-file(search.pas,search.Ada);

The basic statement of thedialogue structure's body is the

invocation of an action.

In the first invocation, the assignmentoperator, :=, is used to indicate thatnext-file obtains the file-name re-turned by select-next. In the secondcase, no value is returned by the pro-cedure rename-file, but it does taketwo arguments, search.pas and search.Ada.The computer operation to rename

a file might not have the same syntaxfor entry as implied by the dialoguestructure's body. By coding invoca-tions using the name of the action,followed by parenthetical arguments,details of the syntax (format, device,and interaction level) used in interact-ing with the computer are hidden fromthe dialogue structure. As mentionedabove, an aspect of usability that is in-dependent of the syntactic appearanceof the interface is considered in this ar-ticle. Entry of the rename commandcould be by fill-in-the-blanks, para-

metric input, or voice input. In eachcase the dialogue structure would bethe same. The dialogue structure al-lows a syntax-independent evaluationof the usability of an interface.

When invoking a computer action,an implicit wait is assumed to occuruntil the action has completed. Thisimplies that the human is inactivewhile the computer is busy carryingout an action. Although this is a rea-sonable assumption, in many systemsan asynchronous mode of activity ex-ists in which the computer and thehuman continue to act independently.An asynchronous system is often char-acterized by computer commands towhich there are multiple acknowledg-ments. An example of this asynchro-nous mode of operation is the Geniesystem.6 Genie's user acts as an airtraffic controller by issuing commandsto vehicles being displayed on a graph-ics screen. After typing a commandline

AIRPLANE 203 TURN LEFTHEADING 270

the user continues other activities untila few seconds later when it can be de-termined on the display that airplane203 has actually turned to a heading of270. Although Genie acknowledgesthat the command has been entered,the computer and human continue ac-tivity simultaneously. We use a limitedset of control structures that do notallow for asynchronous operation ofthe computer with the human. Thus,one action is assumed to be completedprior to the initiation of another.Two statement types are necessary

to express decision-based and iterativestructures. It is often necessary for auser to select from a group of actionsbased on some condition of the taskbeing performed. Two constructs areused for this purpose, the case and theif-then-else statements. An example ofthe case statement (the if-then-else isused in a later example) can be seen inan inventory system, in which a singleuser is handed requests for inventorytransactions. Depending on the re-quest, the user may take different ac-

IEEE SOFTWARE76

tions in communicating with the sys-

tem.

CASE current-request ISWHEN store = >

avail:= space(part,quantity,workstation);store-part(part,quantity,avail);

WHEN retrieve-part= >retrieve(part,quantity,to-location);

END CASE;

If the user's task consists of manual-ly sorting a number of requests basedon some criteria, and then carrying outeach request, then the dialogue struc-ture representing this would be itera-tive. Requests are serviced individually

Figurel.Matiixviewof "effort-to4learn" factors and interface components.

sorted-list:= sort(request-list);FOR each-request IN sorted-list LOOPCASE type(each-request) ISWHEN store = >

avail:= space(part,quantity,near);store-part(part,quantity, avail);

WHEN retrieve-part = >retrieve(part,quantity,to-location);

END CASE;END LOOP;

individually until the entire sequencehas been exhausted.

Others have used formal languagespecification techniques to describe thehuman-computer interface. Reisner,for example, has used grammars tospecify both the physical and mentalactions of human-computer interac-tion.7 Jacob's work on using formalspecifications for human-computer in-terfaces8 has contributed to the ideaswe present here. Dialogue structuresaddress a high-level method of describ-ing human-computer interfaces with-out including their syntax, which is a

need recognized by Jacob. Althoughwe have presented the meta-languagefor expressing dialogue structures in-formally, their correspondence to for-mally described programming lan-guages is clear.

Usability of an interfaceThe categorization of human-com-

puter interfaces into syntactic, seman-

tic, and protocol aspects provides us

with an opportunity for further analy-sis. By identifying these aspects, andby demonstrating how they may beseparately specified, avenues becomeavailable for studies aimed at isolatingthe key factors of interface usability.

"Effort-to-learn" and "effort-to-use" both contribute to the usability ofthe human-computer interface. Wedefine the effort to learn an interfaceas the effort needed to become a pro-ficient user. The effort to use an inter-face is the efficiency with which theinterface can be used once learned.

The syntactic, semantic,and protocol aspects ofthe interface each have

their own "complexity."

Viewing these criteria as contributingto the effectiveness of an interface isconsistent with other work in interfaceevaluation. For example, Good de-scribes an evaluation of the ease of use

of a document-processing system byconsidering its ease of learning, its easeof use once learned, the anxiety itcreates, and user attitudes. 9

We can classify factors contributingto the effort required to learn a hu-man-computer interface into proper-

ties associated with the individual user

and properties independent of theuser. Three such factors are

(1) similarity of the learned inter-face to other known interfaces;

(2) retention of similar interfaces;and

(3) complexity of the interface.Similarity and retention depend on the

user, but complexity is a property ofthe interface.Exposure to other interfaces can be

a positive or negative factor in learninga new interface. Previous use of com-mands with similar functionality can

aid in learning a new interface; how-ever, previous use of similar com-mands with a different functionalitycan hinder learning. For each interfacecomponent, there is a positive or neg-ative learning influence determined bythe similarity or difference of the inter-face components that the user haslearned. The degree of retention meas-ures the effect that past exposure toother interfaces has on learning thenew interface.The syntactic, semantic, and pro-

tocol aspects of the interface each havetheir own "complexity." Determiningand validating a useful measure of theeffort to learn an interface must in-clude the factor of its complexity. It isnot enough to assess an interface by itssuitability to a particular application,especially when designing an interfacefor users with diverse backgroundsand experience levels. To determinethe learnability of an interface we needto objectively measure its complexity.This complexity factor offers thegreatest potential for developing pre-dictive measures of usability, as Reis-ner suggests.7

Figure I is a matrix view in whichone dimension is the interface com-

ponent and the other is the aspect of"effort-to-learn." According to thisview, the "effort-to-learn" factors

January 1985

Similarity Retention Complexity

Syntax

Semantics

Protocols

77

Figure 2. Program entry and syntactic analysis: text editor and compiler.

such as similarity, retention, and com-plexity are orthogonal to the syntactic,semantic, and protocol aspects of theinterface. Predictive assessment tech-niques need to be developed and vali-dated for the syntactic, semantic, andprotocol aspects of interfaces. In theprevious section, we demonstratedhow the semantics and protocols can

be encoded into a dialogue structure.In the remainder of this article, we

discuss the relevance of assessing thecomplexity of dialogue structures byusing software analysis techniques thatcurrently exist.

Complexity of a dialogue structure.For a dialogue structure, learning ef-fort is the effort needed to understandthe commands, inputs, responses,

human actions, and decisions of thedialogue structure. Thus, the com-

ponents of the dialogue structure

(distinct from the interface syntax) are

learned. The user has to learn whatthese components do and how to put

the components together to use the in-terface. These "effort-to-learn" as-

pects can be obtained from the seman-tics and protocols expressed in thedialogue structure.

The semantic complexity of an in-dividual component of a dialoguestructure excludes interactions and in-terdependencies with other compon-ents. Each component, for which thereis an action defined in the dialoguestructure, has a semantic complexity.Components are learned by inputs,outputs, and functionality. Editorcommands, for example, are learnedby matching arguments with the re-sults obtained. Learning the function-ality of a command is enhanced byknowing how different inputs producevarious outputs. The functionality of achange command can be understoodon the surface, but learning howspecific inputs, such as range optionsand pattern-matched search strings,cause different results contributes tothe complexity.

Learning protocol complexity in-volves learning the logic for combininginterface components. The semanticcomplexity of the interface includes,for example, knowing what decisionsneed to be made and how to makethem. Protocol complexity, however,includes when decisions are made(with respect to other dialogue struc-ture actions), where informationneeded in making the decision is ob-tained, and how the result of a decisionis used in determining the next action:In learning to use an interactive debug-ger, the user must learn what condi-tions make "single-step-execution" ofinstructions appropriate and whatconditions make "execute-to-next-breakpoint" appropriate. Further, theuser must learn that it is necessary toset breakpoints before using "execute-to-next-breakpoint." Learning con-trol dependency, which involves learn-ing the order that actions take place,further exemplifies the protocol com-plexity represented by the dialoguestructure.

Programming taskTo demonstrate the utility of the

analysis technique, we compare thehuman interface for programmers byexamining syntax-specific editors andtext editors. These tools represent vast-ly different modes of program entry

IEEE SOFTWARE

DIALOGUE STRUCTURE program_entry_edit_compile ISDECLARE

entry-list: lisLofteditor-commands;current: a-single-command;filename: file_containing_source;options: list_of_compiler_options;design: program-design;

HUMAN OPERATION identify-error, correction-scenario,empty, more-errors, next, construct,repositioning_necessary;

COMPUTER OPERATION invoke-editor, enter_input_mode,enter-statement, leave_input-mode,write_and_exit, invoke-compiler,position_within_source;

BEGINentry-list = construct(design);invoke-editor; enterJinput_mode;WHILE (NOT empty(entry_list)) LOOP

enter-statement(next(entry-Jist)); END LOOP;leave-Jnput.mode; write_and_exit(filename);invoke_compiler(filename, options);WHILE (more-errors) LOOP

FOR ALL errors LOOPidentity-error;correction_scenario(entry_list);

END LOOP;invoke_editor(filename);WHILE (NOT emply(entry_list)) LOOP;

current := next(entry_list);IF (repositioning-necessary(current))THEN position_within_source(current); END IF;

enter_statement(current);END LOOP;write_and_exit(filename);invoke_compiler(file, options);

END LOOP;END program_entry_edit_compile;

78

and translation. Typically in an in-teractive system, the user invokes atext editor to enter the program intothe file system, and then invokes thecompiler to translate it into an ex-ecutable form. Another mode becom-ing more widely used is the syntax-specific editor. Here the user's pro-gram is syntactically analyzed line byline as it is entered. While the texteditor and compiler interact with eachother at the level of entire files, thesyntax-specific editor contains a trans-lator that allows interaction at the levelof source language statements.

Although only a few syntax-specificeditors exist, users advocate theirbenefits over typical interactive-systemtools. Systems such as the CornellProgram Synthesizer '° have been fav-orably accepted by user communities.The use of highly interactive environ-ments originated with program con-struction environments for dynamiclanguages, such as Lisp on Interlisp. "In addition to providing entry facil-ities, these environments also providesupport for 'controlled execution ofprograms in much the same way as isdone in machine language debuggers.Few studies have been performed thatsupport the use of these systems, inpart because the technology forbuilding them for compiled and block-structured languages is still develop-ing. In response to this need, onePascal environment, Peep, 12 wasdeveloped for the purpose of conduc-ting experiments on programmersusing highly interactive environments.While experimentation is needed, theuse of software metrics to assess dia-logue structures is now used to com-pare a sample structure for syntax-specific editors with a typical texteditor and compiler.

Dialogue structures for program en-try. The human-computer interface totext editors and compilers is typicaland conveniently characterized inde-pendent of the command format. Theinterface to syntax-specific editors,however, is not as easy to characterize,and our treatment is limited to pro-gram entry to avoid the vast differ-ences in the way that environments

Figure 3. Program entry with a syntax-specific editor.

allow control of execution.The dialogue structure for an inter-

active text editor and a compiler is bro-ken into three subtasks. The user firstmanually constructs the source of theprogram in a form that can be directlyentered. Next, the user invokes a texteditor to create the source file. Third,he uses the compiler and editor to cor-

rect any syntax errors. The dialoguestructure showing the interface for a

typical system is given in Figure 2.The user first constructs, generally

off line, the program to be entered. Al-though the program may not be in thesyntax of the language, the output ofthis construction is a sequence ofstatements, called entry-list, that can

be directly entered via the editor.While the list is actually made up ofsource program statements, thesestatements are assumed to be in theformat of an editor input command.The next subtask is the entry of theprogram source onto a file. Once theeditor has been invoked and the com-plete list of statements entered, theuser writes the file and exits the editor.At this point, the user requests that thefile be compiled, by naming the file

and providing options. After compila-tion, analysis and correction of errors

begins.For each error encountered during

the compilation, two user actions are

performed. The first is to identify theerror. The user must decide what theerror is and what has caused the error.

Identification is distinct from con-

structing a correction for the error.

After recognizing the error, the user

writes out a set of steps necessary forcorrecting the error using the texteditor available. Corrections are madeby attaching changes to the list, en-

try_ list. Once a scenario for correctingerrors has been assembled into the en-

try list, the editor is invoked and eacherror is corrected using the computerprocedures position_within_sourceand enter-statement. Often severalcompilers are needed to detect and cor-

rect all syntax errors, and this is shownby repeating the compile and editingsessions while errors continue.The dialogue structure for the syn-

tax-specific editor contains two sub-tasks as shown in Figure 3. After con-

structing an initial list of program

statements to be entered, the user

January 1985

DIALOGUE STRUCTURE program_entry_syntax_specific ISDECLARE

entry-list, new-entries: list_of_commands;filename: file-containing_source;current: a-single_command;design: program-design;

HUMAN OPERATION empty, syntax-error, join-list, construct,next, correction-scenario, identify-error;

COMPUTER OPERATION invoke_syntaxcspecific-editor,enter-statement, write-file;

BEGINentry-list = construct(design);invoke-syntax_specific-editor(filename);WHILE (NOT empty(entry_list)) LOOP

current := next(entry-list);enter_statement(current);IF (syntax_error)THEN identify-error;

new-entries = correction-scenario;entry-list = join_list(new_entries, entry-list);

END IF;END LOOP;write_file(filename);

END program-entry_syntax_specific;

79

interactively enters and corrects them.Constructing program statements isdone in the same manner as with thetext editor. Once the list has been con-structed, the syntax-specific editor isinitiated, and an iterative enter-and-correct loop is begun. The action en-ter-statement may take a form distinctfrom that used in Figure 3, as may anyof the procedures. For the syntax-spe-cific editor, a separate command, orgroup of commands, may need to beentered with each statement.The syntax-specific editor responds

to each statement entered indicating,or forcing, proper syntax. Within theenter-and-correct loop, a human-per-formed test is made determiningwhether the editor has found an error.Each time that an error is found, theuser responds in much the same way asdetailed for the text editor. First, theerror is recognized, and then a correc-tion for the error is devised. In Figure3, the correction is named new_entriesto indicate that a single error might re-quire several corrections. Join-list is ahuman function that merges the cor-rections with the remaining entry list.The loop terminates when the list hasbeen exhausted, indicating that all er-rors have been removed, and all state-ments have been entered. At this point,the file may be written, or debuggingmay begin.

Applying metrics. Three existingevaluation techniques can be made toassess dialogue structure semantic andprotocol complexity. These includeassessments of the decision structure,magnitude, and information flowthrough the dialogue structure. As ex-amples, we present McCabe's metric, 13

as an estimate of the complexity of de-cisions made when using an interface,and Halstead's Software Science, 14 asan estimate of the magnitude of thedialogue structure. McCabe's metric(decision structure) addresses the pro-tocol complexity of a dialogue struc-ture, and Halstead's Software Science(magnitude) addresses the semanticcomplextity of the structure. Addi-tionally, a metric such as Henry's In-formation Flow15 could be modifiedto estimate the intricacy of interactions

IEEE SOFTWARE

McCabe's metricMcCabe's software metric, 13 called Cyclomatic Complexity, addresses

the control complexity of a program. Using a graphical representation ofthe program, the metric assesses the program's decision structure bycounting the number of basic decision paths. Many researchers have re-lated these basic paths to testability and maintainability. The relationshipto testability is supported since the metric counts the number of paths cor-responding to linearly independent circuits in the program's graph. Basicpaths are commonly used for devising test cases. The relationship to main-tainability relies on the observation that a software change can be instru-mented only after the decision structure of the affected module isunderstood. Thus, the more complicated the decision structure of the pro-gram, the more difficult the program is to maintain.

McCabe's metric divides the program into blocks of statements that willbe executed to completion once begun. These statement blocks and theprogram's decisions become nodes that are connected using directededges. The cyclomatic number of the resulting graph is the number oflinearly independent circuits in the graph. In practice, Cyclomatic Com-plexity can be calculated (excepting McCabe's suggestion to count all rela-tional conditions once) by counting the number of decision predicates inthe program and adding one. McCabe's measure has received consider-able attention in software literature, and Myers has published a notable ex-tension to the technique. 16

Halstead measuresOne method to evaluate a dialogue structure's semantic complexity is

Halstead's Measures.14 Halstead viewed algorithms as having measur-able interrelated characteristics. To measure these characteristics, Hal-stead applied laws, similar to physical laws, to four program counts. Twocounts are the total number of operators and the number of uniqueoperators in an algorithm. The other two counts are the total number ofoperands and the number of unique operands in an algorithm.

Halstead's Volume, Level, and Effort measures are of interest in assess-ing dialogue structure semantic complexity. Halstead defines the volumeof an algorithm to be the total number of uses of operators and operandstimes the number of bits needed to represent the program. There are two in-terpretations of this volume: first, the volume is the number of bits neededto represent the program; and, second, the volume is the number of mentalcomparisons needed to generate the program.

Halstead also defines a program's level of abstraction to be the ratio ofits potential Volume to actual Volume. The level of abstraction increaseswith the number of distinct operands and decreases as the number ofdistinct operators increases. At the highest level of abstraction(value = 1), the program consists of a single operation (possibly with anassignment). Lower levels of abstraction (values less than 1) indicate thatmore detail exists in the program itself.

Halstead's Effort measure is an assessment of the effort required tocreate and understand a program. The measure, in mental discrimina-tions, is the ratio of the Volume to the Level of abstraction.

80

among actions used in the dialoguestructure. While this would be anotherassessment of protocol complexity,such an example is not constructed inthis article.

Decision complexity. The dialoguestructures constructed in the precedingsection have a decision structure basedtotally on the user. The decisions inthese algorithms either select from al-ternative blocks or guard the actions ofan iterated block. The user makes alldecisions by taking the informationprovided by human or computer oper-ations as input. The user evaluates thepredicates of the algorithm and takesthe appropriate action. A prerequisiteto using a specific human-computerinterface is understanding the under-lying decision (or control) structure.Each time the interface is used to solvea problem, this decision structure istraversed.

McCabe's measure reveals the sizeof the decision structure that the user

must traverse when using the interface.The algorithm for Figure 2 (the texteditor combined with compiler) has a

complexity of six, and the complexityfor the syntax-specific editor is three.While using McCabe's measure indi-cates that the text editor has a more

complex decision structure, otherstructures, for example, those havinghighly nested decisions, in which eachdecision depends on those surroundingit, have a higher complexity than re-

vealed by McCabe's measure. A fur-ther examination beyond the scope ofMcCabe's metric can reveal more in-formation about nested decisions andtheir relationships. To determinewhether a decision depends on thosesurrounding it requires an analysis ofthe content of the decisions.

Semantic complexity. Halstead'sVolume, Level, and Effort measures

must be carefully interpreted as theyare applied to dialogue structures. Forthe algorithms of Figures 2 and 3, theHalstead Volumes are 176 and 90, re-

spectively. In dialogue structures, theVolume augments McCabe's decision-complexity measure by indicating thesize of the structure's components that

Figure 4. Semantic and protocol complexity assessments.

are executed each time the interface isused. Halstead's Volume is an exampletechnique for assessing the semanticcomplexity of dialogue structures. Vol-ume accounts for the number of se-

mantic elements that must be learned.It indicates the complexity of each ac-

tion, however, only through its explicitarguments. Thus, the semantic com-

plexity contributed by implicit argu-

ments, and variations in complexityamong different actions, must also betaken into account.

We would expect, and indeed it isthe case, that the Halstead Level of ab-straction of the edit-compile structureis lower than that of the syntax-specificeditor (0.04 versus 0.14). One reason

for this is that operators are used sever-

al times in the edit-compile structurewhile this is not as true in the syntax-specific structure. The compile com-

mands and find command are goodexamples. Although the compile com-mand only occurs twice in the algo-rithm, one occurrence is within an iter-ation. The find command (position-within-source) occurs with a nested it-eration as well.

Halstead's Effort for the editor-compiler is higher since its Volume ishigher and its Level of abstractionlower than those of the syntax-specificeditor (E for Figure 2 is 4400, and E forFigure 3 is 643). Halstead's Effort in-dicates that the higher the Volume andthe lower the Level of abstraction of aprogram, then the more difficult it is tounderstand. In dialogue structures, theLevel of abstraction varies predomi-

nately through the operands. The high-er the flow of information through thestructure, the lower its Level ofabstraction. The operations used are

mostly unique action names. Similarto Volume, Effort does not accountfor the varying semantic complexityof each unique dialogue-structureaction and argument, but it does in-dicate a semantic magnitude, as-

suming all actions and arguments re-

quire the same effort to learn.The McCabe and Halstead mea-

sures assess the protocol and semanticcomplexities of dialogue structures inthe same manner as they assess soft-ware. The matrix of Figure 4 sum-

marizes the application of these mea-sures to the edit-compile interface ofFigure 2 and the syntax-specific inter-face of Figure 3. While each of thesemeasures addresses a property ofsoftware, they do not individually, or

even together, provide a completeassessment tool. Further, results ofapplying software metrics to human-computer interfaces must be inter-preted taking into account the distinc-tions between software and dialoguestructures. Software contains instruc-tions carried out completely by thecomputer, but dialogue structurescontain both actions and decisionsthat are performed by the user and ac-

tions that are performed by the com-puter.

Efficiency of use. To show howother software analysis techniquesmay further apply to assessing usabil-

January 1985

Semantic Protocolcomplexity complexity

Volume Level Effort Decision

Edit- 176 .04 4400 6compile 1

Syntax- 90 .14 643 3specific I__ _I

81

ity, the following question may be ad-dressed. Once an interface has beenlearned, what is the measure of its useefficiency? If this question was askedof the interface's syntax, for example,such measures as number of key-strokes, thought time, and responsetime might be appropriate indicators.To measure the time required to tra-verse the dialogue structure of the in-terface, we would have to analyze howlong the user takes to perform actionsand make decisions. We could thenestimate use efficiency by analyzingthe dialogue structure's bounds oniterations. Although it is doubtful thatthe values we obtained for the time auser takes to perform actions or makedecisions would be significant, byassessing the computational complexi-ty of the dialogue structures we mightfind factors that could be used to com-pare different structures.

B y viewing a human-computer in-Bterface as having syntax, seman-tics, and protocols, we have a morecomplete view of human-computer in-teractions. In this article, we have pre-sented a method for representing thesemantics and protocols by codingthem into a dialogue structure. Thiscoding combines the user's manual ac-tions with the computer's automatedactions. The dialogue structure detailsthe sequence in which actions takeplace and the order in which the usersmust make decisions. In the article, wehave separated the effort to learn aninterface's dialogue structure intouser-independent and user-dependentterms. We have presented dialoguestructure complexity as the user-inde-pendent component of the effort tolearn, and we have indicated how soft-ware metrics such as those of Halsteadand McCabe can be applied to revealthis complexity. And, finally, we havedemonstrated this assessment of usa-bility by describing dialogue structuresfor two types of program-entry sys-tems, text editors with compilers andsyntax-specific editors. r

AcknowledgmentsThis research was supported by the Of-

fice of Naval Research under ContractNumber N00014-81-K-0143, and WorkUnit Number NR SRO-101. The researchwas supported by the EngineeringPsychology Programs, Office of NavalResearch, under the technical direction ofJohn J. O'Hare.

References1. A. M. Cohill and R. C. Williges,

"Computer Augmented Retrieval ofHELP Information for Novice Users,"Proc. Human Factors Society Meet-ing, Oct. 1982.

2. R. W. Ehrich, "DMS-A System forDefining and Managing Human-Computer Interfaces," Automatica,Vol. 19, No. 6, 1983, pp. 655-662.

3. H. Ledgard et al., "The NaturalLanguage of Interactive Systems,"Comm. ACM, Vol. 23, No. 10, Oct.1980, pp. 556-563.

4. T. E. Lindquist, J. L. Facemire, andD. G. Kafura, "A Specification Tech-nique for the Common APSE Inter-face Set," J. Pascal, Ada, andModula-2, Vol. 3, No. 5, Sept/Oct1984, pp. 25-31.

5. T. E. Lindquist, "The Application ofSoftware Metrics to the Human-Com-puter Interface," Proc. Compcon,Sept. 1983, pp 239-244.

6. T. E. Lindquist, R. G. Fainter, andM.T. Hakkinen, "GENIE: A Modifi-able Computer-Based Task for Ex-periments in Human-Computer In-teraction," to be published in Int'l J.Man-Machine Studies.

7. P. Reisner, "Formal Grammar as aTool for Analyzing Ease of Use: SomeFundamental Concepts," HumanFactors in Computer Systems, Ablex.

8. J. K. Jacob, "Using Formal Specifica-tion in the Design of a Human-Com-puter Interface," Comm. ACM, Vol.26, No. 4, Apr. 1983, pp. 259-264.

9. M. Good, "An Ease of Use Evalua-tion of an Integrated Document Pro-cessing System," Proc. Human Fac-tors Computer Systems, Mar. 1982,pp. 142-147.

10. T. Teitelbaum and T. Reps, "TheCornell Program Synthesizer: ASyntax-Directed Programming En-vironment," Comm. ACM, Vol. 24,No. 9, Sept. 1981, pp. 563-573.

11. W. Teitelman, INTERLISP referencemanual, Xerox PARC, Palo Alto,Calif., 1975.

12. C. S. Ku, "The Design and Implemen-tation of a Language Environment forEvaluating the Programmning Task,"Masters thesis, Virginia Tech, Va.,1982.

13. T. J. McCabe, "A ComplexityMeasure," IEEE Trans. Soft. Eng.,Vol. 2, No. 4, Dec. 1976, pp. 308-320.

14. M. H. Halstead, Elements ofSoftwareScience, Elsevier North-Holland, NewYork, 1977.

15. S. Henry and D. G. Kafura, "Soft-ware Metrics Based on InformationFlow," IEEE Trans. Soft. Eng., Vol.7, No. 5, Sept. 1981, pp. 510-518.

16. G. J. Myers, "An Extension to the Cy-clomatic Measure of Program Com-plexity," SIGPLAN Notices, Oct.1977.

Timothy E. Lindquist is an assistant pro-fessor of computer science at Virginia Techin Blacksburg, Virginia. He is interested inprogramming languages and environments,human-computer interactions, and verifi-cation/validation. For the past three years,he has been actively involved in work onAda environments.

Lindquist received his BS from PurdueUniversity and his MS and PhD in com-puter science from Iowa State University.He is a member of IEEE and ACM.

Questions about this article may be ad-dressed to the author at the Department ofComputer Science, Virginia Tech, 562McBryde Hall, Blacksburg, VA 24061.

IEEE SOFTWARE82