[IEEE IPCC 2005. Proceedings. International Professional Communication Conference, 2005. - Limerick, Ireland (July 7, 2005)] IPCC 2005. Proceedings. International Professional Communication Conference, 2005. - Defining usability: quality of use or quality of experience?

  • Published on

  • View

  • Download

Embed Size (px)


  • 2005 IEEE International Professional Communication Conference Proceedings

    0-7803-9028-8/05/$20.00 2005 IEEE.

    Defining Usability: Quality of Use or Quality of Experience?

    Niamh McNamara University College Corkn.mcnamara@ucc.ie

    Jurek Kirakowski University College Corkjzk@ucc.ie


    This paper is concerned with the impact of changing conceptualizations of usability from quality of use to quality of experience. At present, there are many calls to expand the usability construct, although there is no clear consensus as to how this can be achieved. The lack of such a consensus might result in the usability construct being broadened to such a degree as to become meaningless. The implications of this for the evaluation of electronic consumer products are discussed.

    Keywords: HCI, usability, electronic consumer products, user experience.


    The area of interest of this paper is evaluating the usability of everyday electronic consumer products such as mobile phones, DVD players, VCRs, washing machines, etc. An increasing number of products that people use in their daily lives contain embedded technology. This computerization of everyday life has created new challenges for the discipline of Human Computer Interaction (HCI). Although these devices were first introduced some 30 years ago, they remain difficult to use and grow increasingly complex. Therefore, the need for some form of usability evaluation is obvious.

    Broadly speaking, the term usability refers to how easy a product is to use. It was first introduced in the 1980s to replace the clich term user friendly[1]. While it is central to HCI, the discipline lacks a standard definition of the construct and consequently there are a number of competing perspectives on how it should be conceptualized. The dominant perspective defines usability as quality of use and this has informed much of the

    research carried out on desktop systems. However, there is some debate concerning the applicability of this view of usability to new technology, including electronic consumer products. It is suggested that viewing usability as quality of use neglects the subjective aspects of using technology, such as emotional evaluations and considerations of a devices aesthetic value, and thus the move should be made to conceptualizing usability as quality of experience. This paper discusses the implications of broadening the usability construct from quality of use to quality of experience and then offers a framework for evaluation.

    Usability as Quality of Use The quality of use perspective on usability examines the interaction between the user and the product. It emerged as an alternative to a perspective that dominated early research and still exerts some influence today, which views usability as a product property. The underlying assumption in these early conceptualizations is that usability is dependent on the presence or absence of particular product features. However, such a conceptualization is static and situation-specific and does not guarantee that a product will be usable for all possible user groups and usage scenarios.

    The quality of use perspective, however, hypothesizes that usability varies according to whois using the product, where it is being used, and the purpose for which it is being used. Collectively, the various aspects of the interaction that need to be considered when specifying usability are known as the context of use [2]. This view on usability is enshrined in the ISO 9241-11 definition of usability as the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use [6].


  • 2005 IEEE International Professional Communication Conference Proceedings

    This perspective views usability as an abstract but measurable construct that is central to the processes of Usability Engineering and User-Centered Design. The need for systematic measurement of usability requires operationalisation of the construct and a number of dimensions have been proposed. These include Effectiveness, Learnability, Flexibility, and Attitude [14]; Learnability, Memorability, Errors, and Satisfaction [11]; and Effectiveness, Efficiency, and Satisfaction [6]. An interesting feature of these operationalisations is that all include a subjective element. It is typically recommended that both objective and subjective data be collected when evaluating a product. Objective measures typically examine the accuracy and speed with which the product can be used while subjective measures assess user reactions and attitudes towards the product. Unfortunately, increasingly shorter product development timescales and the need for cost-effectiveness often means that subjective data is not gathered as its collection is regarded to be time-consuming and resource-intensive.

    Those who characterize usability as quality of usenot only propose conceptual definitions of the construct by identifying specific dimensions to be included when measuring usability, they also endeavor to develop a theory of the construct by situating it in a wider model of user acceptance of technology alongside other concepts such as utility, functionality, likeability, and usefulness [14], [11]. Nonetheless, critics of the quality of use perspective argue that it ignores subjective aspects of technology use, such as fun and enjoyment. This is seen as damaging to the discipline as a whole, and has led to calls for a re-examination of the construct in general [7], [12], [15].

    Usability as Quality of Experience Recently, quality of experience has entered the discussion.This perspective proposes to address aspects of technology usage that are omitted by the quality of use perspective. It should be noted that this is not the first time this perspective has been criticized for omitting important aspects of technology usage. Since Davis [3] proposed his Technology Acceptance Model (TAM), a number of authors have criticized HCI for focusing exclusively on usability and neglecting other factors that might influence technology acceptance and usage. According to Davis [3], considering a products perceived usefulness may be more important in

    determining whether or not that product will be used than examining its perceived ease of use orusability. However, there is one important difference that distinguishes the concepts proposed by proponents of TAM from those proposed by advocates of the quality of experience approach. In Keinonens terms, concepts such as usability, utility, and usefulness are factors that influence extrinsic motivation to use technology. Those who call for a focus on the user experience propose to investigate how individuals are intrinsicallymotivated to use technology [8] and so concepts that reflect the more subjective aspects of technology usage such as engagement, pleasure, presence, and fun are receiving more attention.

    There are three main approaches to investigating quality of experience evident in the literature. The first approach involves adding new dimensions to the usability construct. In a sense, this involves revising the operationalisations of usability discussed earlier to include more subjective dimensions. For example, Logan [9] suggests that usability should be split into behavioral usability,which refers to the ability to complete some functional or goal-directed task within a reasonable time and emotional usability, which refers to the degree to which a product is desirable or serves a need beyond the traditional functional objective. Therefore, this approach seeks to broaden the usability construct to include additional concepts rather than discarding it completely and starting afresh.

    The second approach involves the proposal of new concepts that are hypothesized to be important to consider when designing technology but that are distinct from usability. For example, Jordan [7] contends that usability is inherently limited by placing too much emphasis on cognitive issues. Rather than exploring how the use of technology elicits positive feelings in the user, HCI has concentrated on the avoidance of negative emotions [7]. He maintains that the pleasure-based approach to design that he proposes, (encourages) a holistic view of the user, judging the quality of a design on the basis of the wider relationship between a product and the people for whom it is designed. Jordan describes a three-level hierarchy of consumer needs, based on Maslows [10] Hierarchy of Needs, with Functionality at the first level, Usability at the second, and Pleasure at the third. Pleasure is here defined as the emotional, hedonic and practical benefits associated with


  • 2005 IEEE International Professional Communication Conference Proceedings

    products. Jordan maintains that the consumers most basic need is for products to work. After this, people expect products to be easy to use. Finally, once people become accustomed to usable products, they want something more, i.e. pleasurable products. Such products can be created by linking product properties to particular user emotions [7].

    Finally, a third approach, proposed by Wright & McCarthy [16], takes a more theoretical and philosophical look at the concept of experience. They argue that experience is an under-developed concept and is often used without a deeper understanding of what the term actually means [16]. While other approaches (such as that outlined by Jordan [7]) attempt to design a particular user experience by linking product properties to user emotions, this approach advocates the use of structured techniques to talk about experience. To this end, Wright & McCarthy [16] propose a framework with which to analyze experience based on the work of philosophers John Dewey and Mikhail Bakhtin. This framework gives researchers a method of talking about experience, and consists of two parts: describing an experience and making sense in experience. It criticizes attempts to design an experience and maintains that experience cannot be reduced to its fundamental elements [16].

    Clearly, there are many strands to the New Usability involving a number of new approaches and concepts. There is considerable debate, however, as to whether these concepts should complement usability evaluations or, indeed, replace such evaluations.

    Going beyond usability the concept of experience

    Although there are many calls to go beyond usability, no one quite knows what beyond is [9]. This situation is made all the more difficult by the lack of any standard definition of usability. Thus, the introduction of the concept of experience adds further complexity to an already complex construct.

    This is particularly evident in attempts to broaden the usability construct by including concepts such as aesthetics, image, impression, and ambience, among others. There is little, if any, explanation of how these concepts are related to each other or to usability with the result that the usability construct

    could be broadened to such a degree as to become meaningless or else narrowed severely to the same ultimate effect.

    It is suggested here that usability is a concept distinct from that of experience. Indeed, this is similar to the position that Jordan [7] takes in his proposed hierarchy of consumer needs. However, his operationalisation of the pleasure construct is rather narrow and shares many similarities with early conceptualizations of usability as a productproperty in the way that it links product properties to user emotions. Conceptualizing usability in this way is considered inappropriate and it is unlikely that a conceptualization of experience as dependent on particular product attributes will prove any more successful. As Wright & McCarthy [16] argue, experience is as much about what the individual brings to the interaction as it is about the product itself. The concept of experience is being increasingly used in the HCI literature, however without a deeper understanding of what exactly constitutes experience it will remain little more than a buzzword in the same vein as user-friendly in the 1980s [5].

    Implications for Evaluation Clearly, changing conceptualizations of the usability construct have implications for the evaluation of new technology, including electronic consumer products. Investigating the wider relationship between people and technology would no doubt be beneficial to the design of technology and the discipline of HCI as a whole, as it is clear that there are aspects of usage that need to be considered because the way in which technology is used changes. Nonetheless, the fact remains that such products are also difficult to use and grow increasingly complex, suggesting that calls for the abandonment of the usability construct are somewhat premature.

    It is suggested here, in contrast to Jordans [7] hierarchical view, that there are 3 aspects that need to be considered simultaneously when designing and evaluating technology. This is illustrated in Fig. 1. Each of these 3 areas asks a different question about technology usage using a different language of discourse.

    Functionality is a technical issue. In this instance, the goal of evaluation is to answer the question what will the product do?


  • 2005 IEEE International Professional Communication Conference Proceedings

    Figure 1. 3 areas of technology usage

    Usability is a user issue and therefore the product needs to be tested with real users. The criteria of support and goal facilitation are important here in answering the question, can I make the product do what I want it to do? Finally, examining the userexperience investigates the wider relationship between the product and the user in order to consider the individuals personal experience of using it. The question asked in this area may be how do I relate to this product?

    These areas are not arranged in the form of a hierarchy. As such, a presentation not only implies that one aspect of usage is more important to consider than others, but also it implies that one level builds on the other, which it does not. For instance, one could examine the user experience of using a product regardless of whether that product is usable or not. While the various aspects might influence each other to some extent, for example, the implementation of device functions impacts on the usability of the product and the usability of the product impacts on the experience of using it, independent evaluations can, and should, be carried out in each area. According to this functionality or experience should not be incorporated into usability evaluations, nor should they be included in definitions of the construct.

    It is important to note that each area of investigation incorporates its own unique set of methodologies and the significance of choosing an appropriate methodology when carrying out an evaluation in any of the 3 areas cannot be overemphasized. The disadvantage with some

    approaches bel...


View more >