Design and the Analytic Method: The Future of Human Computer Interaction (HCI)

  • Published on

  • View

  • Download

Embed Size (px)


  • 8/14/2019 Design and the Analytic Method: The Future of Human Computer Interaction (HCI)


    Design and the Analytic Method: The Future of HumanComputer Interaction (HCI)

    Laurian C. Hobby

    Center for HCI, Computer ScienceVirginia Tech660 McBryde

    Blacksburg, Virginia

    Abstract. When designing or choosing an analytic method, the designer shouldchoose a method based on the criteria of having a good cost benefit trade-off,ease of use, and reproducibility of results. Even these criteria can lead a designmethod astray if it does not have the right focus of attention. HCIs future indesign depends on shifting the focus away from simply the user, to add work context which incorporates the user and the work being done.Technomethodology is only one of many possible design methods that attemptsto meet these criteria and has the proper focus. This paper explores these issuesand how a merging of social and tangible computing can create a superior

    analytic design methodology that will guide HCI into the future of design.

    1 Introduction

    The world and its systems are complex. To understand them correctly requires theinquisitive mind to analyze their structure and content in detail. To be analytic is tothink in terms of elemental parts or basic principles [10], or to be a logician of sorts. However, the analytical process is much more than that - it is to come to theright conclusion by breaking the problem into manageable parts in a structured andmethodical way. To further understand how to think analytically, it is beneficial tothink of its counterpart, how to think empirically, and what the two say about eachother. Empiricism, to define it, is the use of metrics and measures to accept or reject ahypothesis. In comparing the two approaches, it may seem that an empirical method issterile. This is because the rules of analytic methods can often be ill-defined, whereasthe rules of empirical methods are more likely rigorous and sturdy. A typical examplewould be to think of the evaluation of software: to use a checklist of neededcharacteristics would be an analytic method, whereas to take a measurement of how

  • 8/14/2019 Design and the Analytic Method: The Future of Human Computer Interaction (HCI)


    2 Laurian C. Hobby

    many people were able to complete a task with a piece of software would be anempirical method. There are benefits and costs of using an analytic method for design,therefore, the process used to select and develop an analytic method should becarefully selected. How this method compares to general design methods, and how itcan help to initiate or inspire formal design methods, should also be explored. The restof this paper will investigate these ideas in depth and come to a set of conclusions.

    2 What Makes a Good Analytic Method?

    When it comes to the creation of a software system the criteria for which analyticmethod to use is unclear. It is easy to apply general rules to software systems design,such as generalizability, precision, and realism [ 21]; but for selecting an analyticmethod for the assessment, the criteria need s to be more specific. Typical examples of analytic methods to choose from, (although they mostly , if not only, focus on onephase of design , e.g., evaluation, ) are usability inspections [ 23, 24], model-basedanalysis [ 17, 25], and claims analysis [ 32, 7] (for more information on all three see[29]). When to use which method , and what criteria to apply in order to create a goodanalytic method , are still unclear in the current literature. However, t he primarycriteria for what makes a good analytic method are listed and discussed in the nextthree subparagraphs in order to help clarify and ground what is a superior analytic


    2.1 Criteri on 1: The Cost Benefit Tradeoff

    The cost benefit tradeoff, when viewed at a high level, is similar to asking whether theanalytic method is cost effective , i.e., a re the benefits gained from using this methodequal or better to the cost of using it? The costs and the benefits are computed andcompared . As long as the comparison is in favor of the benefits, then th is analyticmethod may be considered as one that is good. An example of a software designproject where the cost benefit ratio was in favor of performing the analytic method

    was the Goals, Operators, Methods, and Selection Rules project ( GOMS ). BonnieJohn and her colleagues worked on a project using the GOMS methodology where inthe acti vity levels of phone operators were desired to be faster /quicker . A newsoftware system was proposed to replace the original system. John was able toaccurately predict that the new computing system was going to cause the phoneoperator to be slower with the proposed system than with the original system . She wasable to do this with a minimal cost of time and effort because she was able to break

  • 8/14/2019 Design and the Analytic Method: The Future of Human Computer Interaction (HCI)


    Design and the Analytic Method: The Future of Human Computer Interaction (HCI) 3

    down the tasks that the phone operators would be doing with the new system with ashort amount of effort and time [17] . Thus, the costs were low when comparing themto the costs of the creation of bad software. This is an example where the cost of performing the cost benefit analysis was low; however, the benefits of understandingthat the new system would not improve the situation were high . When the cost of applying an analytic method results in significant benefits, then it fits the criteri on forbeing a superior analytic method.

    2.2 Criteri on 2: The Ease of Use

    Put simply, the method should be easy to use. One of the major criticisms of someanalytic method s are that t hey can produce variable results which are dependent onthe quality /ability of the person (the designer) using them, if they are not easy to use .Analytic methods, in essence, are based or grounded on a set of theoretical constructsthat need to be interpreted by the person during their application . This interpretationand the results thereof , are directly dependent on the expertise and training level of thedesigner. This creates a situation where given the same problem to be solved , andusing the same analytic method, two designers of different expertise levels have a highprobability of generating widely different outcomes if the method is difficult to apply .For example, in claims analysis, designers create claims that stipulate the upsides anddownsides of using a particular interface object such as a menu. The difficulty withusing this method is that expert designers create much better claims than novicedesigners if there is no guided tool support. Specifically, novice designers have a hardtime creating accurate downsides for claims about design artifacts [ 18]. As can beseen from this example, the expertise of the designer can be a factor in design, eventhough similar results should be obtained no matter what the knowledge level.Therefore, the ease of use of the method in producing good results should be similarno matter what the designers background knowledge and level of expertise . Inempirical design, as a comparison, the results of such methods are more repeatableirrespective of the expertise level of the designer. (For more information see [ 29].)This is partially due to the ambiguous nature of analytic method s and the products

    produced; empirical methods may take a certain level of expertise for the results to beinterpret ed, but the same interpretation will be arrived at each time. Nevertheless,even though analytic products may be nebulous, the designers level of backgroundexpertise should not be a factor in the success of a method.

  • 8/14/2019 Design and the Analytic Method: The Future of Human Computer Interaction (HCI)


    4 Laurian C. Hobby

    2.3 Criteri on 3: Reproducible

    The last and perhaps the most important criteri on for a superior analytic method is forthe process to produce repeatable results. This is similar to having ecological validity[21] in that the method should be rigorous enough to be used by different people overdifferent periods of time in different settings and to still be able to produce reliablerepeatable results. Although this seems to be the holy grail of design, it is specificallythe most important one for an analytic method because of the high variability withinthe current methods. This may be due to the fact that designers naturally bring in theirown biases and expectations into analytic design. For example, usability analysis isthe analytic method where the operation of a created computer system is tested andproblems with the software (example: the user is not able to find a box to click) arereported and fixed. Rolf Molich, Meghan Ede, Klaus Kaasgaard, and BarbaraKaryunkin showed that evaluating the same software at nine different organizationsresulted in 310 different usability problems being reported; however, only two of thoseproblems were the same [Molich]. Although this example is an extreme case , it showsthe high variability that can exist in using analytic methods , and because of this, ananalytic method that is able to produce re peatable results will be a superior one. (Formore information on variability see [15, 16].)

    3 The Evolution of Analytic and Design Methods

    In comparison to the centuries of electrical engineering development, softwareengineering is a relatively new science, and Human Computer Interaction (HCI) iseven newer. The question What makes a good software design? has been recognizedas the million-dollar question, and with the rising interest in the Science of Designthere has been a lot of interest and work trying to answer which is the bestmethodology (or possibly methodologies) for designers to select for their task of taking a set of specifications and creating a software design. As with most voyages of discovery there are phases of discovery, each building on the previous work. Therehave been many novel design methods proposed in the past years that meet one or twoof the three criteria discussed in Section 2 above, but very few that meet all three. (Tocite a few: Activity Theory [1], Cognitive Dimensions [3], Usability Engineering [11],and Patterns [4]). There have been many theories that have subsequently becomeaccepted as laws (e.g., Fitts Law [20]) and some that are so fundamental to the studyof HCI that they are no longer looked at as stand-alone study items as they havebecome standard text book material (e.g., Information Theory [27]). This knowledgehas led to the generation of questions such as: What design method is the right one to

  • 8/14/2019 Design and the Analytic Method: The Future of Human Computer Interaction (HCI)


    Design and the Analytic Method: The Future of Human Computer Interaction (HCI) 5

    choose? Is the use of an analytic model inherently what makes a design method good?Clearly the random selection of the chosen design method will not be correct becausethe design methodology will likely have the wrong focus. To explore all these issues, alook into the past is the best place to investigate how these questions and distinctionshave arisen and what answers it can lead us to for the future of design an analyticmethod that meets the criteria and has the right focus.

    3.1 The History of the Analytic Method and Design

    All design methods are based on an analytic method. Although design methods may

    have analytic components to them, as seen in the examples below, it is not necessary.For example, the Commercial Off-The-Shelf (COTS) method of design is consideredto be an empirical design method [2], but it should be recognized that the COTSdesign was likely founded on an analytic method. A second example would be toconsider an analytic method such as Pattern Languages [4]. Pattern languages aresimilar to COTS in that the designer can create a design using patterns that have beendefined before; however, to create those patterns in the first place an analysis wouldhave to have been performed to quantify the patterns in existence. Thus, it seems, alldesign methods being employed today, whether empirical, analytical, or a mixture of both, can trace their lineage to one or more analytic methods. It must also berecognized that this traceability does not automatically mean that the designmethod(s) will produce good design results. The focus, or the paradigm, upon whichthe design method was originally established is the main criterion for whether or notthe design method will be successful in todays computing environment.

    The first design method to be widely accepted within the computing industry was theWaterfall Model. During this period of design evolution the hardware platform wasconsidered to be augmented by the software. The software merely helped thehardware get its job done. So this model was based upon the idea that the computerhardware was the locus of attention and computation, instead of the human being.Within this method, the designer was guided through the steps of how to gather

    requirements, derive requirements, and then construct software in an iterative fashion.Although this method was widely adopted, it later went on to take significantcriticism due to its design limitations and resulted in an eventual paradigm shiftwithin the software engineering community. Nevertheless, the Waterfall Model isimportant as it was the first to incorporate an analytic phase into the design process.In the analysis phase the designer would create a requirements document that wouldspecify the needs and costs of creating a piece of software. Even with theincorporation of an analytic phase however, the indiscriminate use of a method that

  • 8/14/2019 Design and the Analytic Method: The Future of Human Computer Interaction (HCI)


    6 Laurian C. Hobby

    may have an incorrect focus on where the analysis should be taking place, is one of the main factors that resulted in the failure of this model to be applicable to solve allclasses/types of software design problems.

    Hence, a second paradigm shift arose where the focus of the resulting design was onthe human user and not computer hardware. It is generally agreed that this shift wasmostly due to the influx of personal computers into the academic and businessenvironments. This shift resulted in design methodologies such as the Spiral Modeland Evolutionary Prototyping. What these new designs had in common was that theyincorporated the input/output methods of the human users more closely as the

    computer systems themselves were designed. There has also been a gradualincorporation of involving the user more and more into the design process.

    It is important to note that this paradigm may also have started to fail. It has beenpostulated by Alan Crabtree [8, 9] that the reason for this failure is its reliance onformal methodologies to guide the designer: The problem, then, is that theunderstanding of work generated through the use of formal design methods tend to bevery abstract , focusing more on what shoul...


View more >