ELECTRONIC PAPER-PROTOTYPING TOOLS · 2007-09-11 · characteristics of prototyping approaches. This thesis has extended this work by applying the full Cognitive Dimensions framework

SUPPORTING ANNOTATIONS IN

ELECTRONIC PAPER-PROTOTYPING TOOLS

Amir M Naghsh

A thesis submitted in Partial fulfilment of the requirements for the Degree of Doctor of Philosophy at Sheffield Hallam University

August 2007

Amir M Naghsh i

Amir M Naghsh ii

To my grand parents who inspired me through my life, especially to Shahzadeh Ayati who always had faith in me.

Amir M Naghsh iii

Communication

The most obvious factor that distinguishes humans from other animals is the variety of ways in which we communicate. Human Interactions are a rich blend of language, facial and body movement, and symbols; we also assess the context of any message to obtain the maximum amount of information. Our style of communication is one of the clearest markers of identity. The language and gestures that we use show the community or nation to which we belong; in addition, each of us has a personal vocabulary of commonly used words and gestures reflecting our individual concerns." Lord Robert Winston, Human, page 301

Amir M Naghsh iv

Amir M Naghsh v

Amir M Naghsh vi

ABSTRACT

It is generally accepted that user involvement is essential for the successful design of an interactive system. Enabling users to envisage or make sense of design proposals (whether those proposals originate with ‘professional designers’ or from the users themselves) is an essential element of all participatory approaches to design. Users can only make informed choices when the proposals being discussed are meaningful to them. In this process, stakeholders exchange a large number of documents and models, and may spend a great deal of time meeting to reach a common ground. This becomes challenging in design projects where stakeholders are fully dispersed (both geographically and in regards of competence).

Previous research indicates that electronic paper-prototyping can be used to rapidly create simple prototypes of interactive systems, such as websites. Recent studies point to various ways in which the design process depends on communicative activities around collaboratively produced prototypes. Little is known about how to provide and maintain a variety of communication channels around electronic paper-prototypes to enable end-users and other stakeholders to contribute to design dialogues.

This thesis examines a range of tools for early prototyping of interactive systems, each of which provides some facilities that may be useful in enabling electronic paper prototyping, but each of which also suffers limitations in terms of their ability to support communication activities. The thesis presents theoretical and empirical analysis to explore the feasibility and requirements for creating an electronic paper prototyping tool that can facilitate computer supported collaboration in the early stages of designing interactive systems.

Collaboration can be facilitated by applications for sharing and annotation. This thesis reports research conducted on the use of annotation in asynchronous collaboration systems. Furthermore, this thesis presents design, development and evaluation of Gabbeh, an electronic paper prototyping tool, designed to facilitate communication and collaboration through annotating the prototype at the early stages of design. The core innovation in Gabbeh is that it allows different stakeholders to add arbitrary annotations in the form of comments either when the prototype is being designed or when it is being executed.

Amir M Naghsh vii

This thesis reports the results of observational and longitudinal studies of using Gabbeh and explores how different stakeholders may use annotations in an electronic paper prototyping tool. Finally, this thesis suggests the important areas for further investigations.

Amir M Naghsh viii

Amir M Naghsh ix

ACKNOWLEDGMENTS

I wish to express my sincere thanks to all those who have encouraged and backed me up while writing this thesis.

First, I would like to express my sincere gratitude to my director of studies, Dr. Andy Dearden, for his continuous support of me at Sheffield Hallam University. I would like to thank Andy for giving me the opportunity to begin this work and for then giving me the skills, continues guidance and experience to finish it. Without his knowledge and help, this thesis would never have begun and without his support and understanding it certainly would never have got been finished.

I would also like to thank my advisors, Dr. Mehmet B. Ozcan and Prof. Jawed Siddiqi for their great comments. I would especially like to single out Mehmet, who gave me more feedback and time out of his busy schedule than anyone next to Andy.

I am also grateful to all those who took part in my studies or answered my questions, my anonymous thanks to them all.

Friends are something without which I could not have made it through this time in my life. I would like to thank Haleh, Rashmee, Mohammad, Pav, Georgios and Sara who have provided pleasant distraction over the years.

I would like to offer my special and sincere thanks to my parents for all years of support and forgiveness for not being there when they needed me the most. I would like to impress my gratitude to my sister Ati, my aunt Elaheh, and my uncle Mostafa who gave me support, encouragement and space to finish this thesis. Last but not least I would like to thank my aunt, Dr. Malihe Shahidan, who was always there to meet and talk about my ideas and mark up my papers and chapters.

To all the significant others whom I couldn't mention here, please accept my gratitude. I might fade in your memory but you will always grow in mine with every step I make.

Amir M Naghsh x

Amir M Naghsh xi

CERTIFICATION

I certify that this thesis, and the research to which it refers, are the product of my own work and that they do not incorporate without acknowledgment, any materials previously submitted for a degree or diploma in any university or institute. I also certify that this thesis do not contain any material previously published or written by other people except where due acknowledgment is made in the text.

I would like to clarify that some of the listed publications have been conducted in collaboration with Dr. Andy Dearden, Dr. Mehmet B. Ozcan and Prof. Jawed Siddiqi. The paper presented at the Fifteen Annual Meeting of the Psychology of Programming Interest Group (PPIG) (Dearden, Siddiqi and Naghsh, 2002) has considered a limited number of Cognitive Dimensions to explore different characteristics of prototyping approaches. This thesis has extended this work by applying the full Cognitive Dimensions framework to compare the electronic paper prototyping tools. The enhanced analysis has been presented in Chapter 4 of this thesis.

Early design of Gabbeh has been reported at the annual conference of British Human Computer Interaction group (HCI’04) (Naghsh and Ozcan, 2004) and at the Participatory Design Conference (PDC’04) (Dearden, Naghsh and Ozcan, 2004). Gabbeh has been enhanced further and the first version and second version of Gabbeh are presented in Chapters 6 and 8 of this thesis.

The publication at the workshop of Design, Specification and Verification of Interactive Systems (DSVIS’05) (Naghsh, Dearden and Ozcan, 2005) has reported the design of the first version as well as observation studies of Gabbeh. These are presented in detail in Chapters 6 and 7 of this thesis.

Amir M Naghsh xii

Amir M Naghsh xiii

TABLE OF CONTENTS

ABSTRACT VI

ACKNOWLEDGMENTS IX

CERTIFICATION XI

TABLE OF CONTENTS XIII

LIST OF FIGURES XVII

LIST OF TABLES XIX

LIST OF PUBLICATION XX

CHAPTER 1. INTRODUCTION 2

1.1. USER PARTICIPATION AND PROTOTYPING 2 1.2. COMMUNICATION IN DESIGN 4 1.3. THE THESIS 5 1.4. OUTLINE 6

CHAPTER 2. REVIEW OF PROTOTYPING 8

2.1. INTRODUCTION 8 2.2. WHAT TO PROTOTYPE? 8 2.2.1. HOW THE PROTOTYPE IS USED? 9 2.2.2. THE FORM OF PROTOTYPES 10 2.3. LOW-FIDELITY APPROACHES 12 2.3.1. STORYBOARDS 12 2.3.2. PAPER PROTOTYPING 13 2.3.3. PICTIVE 14 2.3.4. RAPID SOFTWARE PROTOTYPING 15 2.4. HIGH-FIDELITY APPROACHES 15 2.4.1. RAPID APPLICATION DEVELOPMENT 15 2.4.2. EXTREME PROGRAMMING (XP) 16 2.4.3. EXECUTABLE FORMAL SPECIFICATION 17 2.5. MEDIUM-FIDELITY APPROACHES 19 2.6. ELECTRONIC PAPER PROTOTYPING TOOLS 20 2.6.1. PATCHWORK 20 2.6.2. SILK AND DENIM 21

xiv

2.6.3. INDESIGN 25 2.6.4. FREEFORM 27 2.7. SUMMARY 27

CHAPTER 3. COMPARING EXISTING ELECTRONIC PAPER PROTOTYPING TOOLS 29

3.1. INTRODUCTION 29 3.2. PAPER PROTOTYPING 30 3.3. INDESIGN 31 3.4. DENIM 33 3.5. FREEFORM 35 3.6. DISCUSSION 38

CHAPTER 4. COMPARING PROTOTYPING TECHNIQUES AND TOOLS 40

4.1. INTRODUCTION 40 4.2. RELATING THE COGNITIVE DIMENSIONS TO PROTOTYPING 41 4.2.1. VISCOSITY 41 4.2.2. ABSTRACTION 42 4.2.3. CLOSENESS OF MAPPING 43 4.2.4. CONSISTENCY 43 4.2.5. DIFFUSENESS 44 4.2.6. ERROR-PRONENESS 44 4.2.7. HARD MENTAL OPERATIONS 45 4.2.8. HIDDEN DEPENDENCIES 45 4.2.9. PREMATURE COMMITMENT 46 4.2.10. PROGRESSIVE EVALUATION 46 4.2.11. ROLE-EXPRESSIVENESS 46 4.2.12. SECONDARY NOTATION 47 4.2.13. PROVISIONALITY 48 4.2.14. VISIBILITY AND JUXTAPOSABILITY 48 4.3. A REVIEW OF TOOLS USING COGNITIVE DIMENSIONS FRAMEWORK 49 4.3.1. PAPER PROTOTYPING 49 4.3.2. DENIM 52 4.3.3. INDESIGN 54 4.3.4. FREEFORM 56 4.4. DISCUSSION 58 4.5. CONCLUSION 59

CHAPTER 5. ANNOTATION 61

5.1. INTRODUCTION 61 5.2. ANNOTATION 61 5.3. EXAMPLES OF ANNOTATION TOOLS IN USE 63 5.4. FORMS AND FUNCTIONS OF ANNOTATION 63 5.4.1. PERSONAL ANNOTATIONS 64 5.4.2. PUBLIC ANNOTATIONS 68 5.5. CHARACTERISTICS FOR ANNOTATION TOOLS 70 5.5.1. OPERATIONAL CHARACTERISTICS 70

xv

5.5.2. USER INTERFACE CHARACTERISTICS 72 5.6. CONCLUSION 72

CHAPTER 6. GABBEH 74

6.1. INTRODUCTION 74 6.2. DESIGNING AN ANNOTATION TOOL 75 6.2.1. CONSISTENCY TO CURRENT DESIGN AND EASY TO USE 75 6.2.2. FLUIDITY OF FORM 75 6.2.3. IN SITU ANNOTATION 76 6.2.4. INFORMAL USE 76 6.2.5. RETRIEVING AND FILTERING 77 6.2.6. AVAILABILITY 77 6.3. DENIM 78 6.3.1. THE DENIM USER INTERFACE 78 6.3.2. ARCHITECTURE OF DENIM 81 6.4. GABBEH 87 6.4.1. THE GABBEH USER INTERFACE 87 6.4.2. IMPLEMENTING GABBEH 96 6.5. CONCLUSION 105

CHAPTER 7. OBSERVATIONAL STUDIES 106

7.1. INTRODUCTION 106 7.2. OBJECTIVES 106 7.3. LAYOUT 107 7.4. PILOT STUDY 111 7.5. REVISED 112 7.6. RESULTS 113 7.6.1. WERE COMMENTS USED? 113 7.6.2. WERE DISCUSSIONS SUPPORTED? 116 7.6.3. HOW WERE COMMENTS USED? 118 7.7. DISCUSSION 123 7.7.1. MANAGING LARGE NUMBERS OF COMMENTS 123 7.7.2. IDENTIFYING AND MANAGING DISCUSSIONS 123 7.7.3. SUPPORTING PROGRESS AWARENESS AND TRACKING 124 7.8. CONCLUSION 124

CHAPTER 8. SUPPORTING DISCUSSION IN GABBEH 126

8.1. INTRODUCTION 126 8.2. ENHANCING THE DESIGN OF THE COMMENTING TOOL 127 8.2.1. PROGRESS AWARENESS 127 8.2.2. SUPPORTING DISCUSSIONS 127 8.2.3. MANAGING LARGE NUMBERS OF COMMENTS 129 8.3. GABBEH VERSION 2 130 8.3.1. PROGRESS AWARENESS 130 8.3.2. SUPPORTING DISCUSSIONS 133 8.3.3. MANAGING LARGE NUMBERS OF COMMENTS 136 8.4. CONCLUSION 138

xvi

CHAPTER 9. LONGITUDINAL STUDY 139

9.1. INTRODUCTION 139 9.2. OBJECTIVES 139 9.3. LAYOUT 140 9.4. RESULTS 141 9.5. HOW WERE DISCUSSIONS USED? 145 9.5.1. PROGRESS AWARENESS 145 9.5.2. DISCUSSING DESIGN ISSUES 149 9.5.3. DIRECTING ATTENTION 151 9.5.4. RECORDING DESIGN ISSUES 152 9.6. REVIEWING AND REFLECTING THE COMMENT CLASSIFICATION 153 9.7. DISCUSSION 156 9.7.1. PROGRESS AWARENESS 156 9.7.2. RICHNESS OF COMMUNICATION CHANNEL 157 9.7.3. EMPOWERING USERS 157 9.8. SUMMARY 158

CHAPTER 10. CONCLUSION 159

10.1. SUMMARY 159 10.2. CONTRIBUTIONS 159 10.2.1. COGNITIVE DIMENSIONS 159 10.2.2. GABBEH 160 10.2.3. A CLASSIFICATION OF ANNOTATIONS 160 10.2.4. RECOMMENDATIONS FOR SUPPORTING ANNOTATION IN ELECTRONIC PAPER PROTOTYPING TOOLS 161 10.3. FUTURE WORK 163 10.3.1. ENHANCEMENT TO THE CURRENT VERSION OF GABBEH 163 10.3.2. DESIGN RATIONALE 164 10.3.3. SUPPORTING ANNOTATION IN SITUATED SETTINGS 165 10.3.4. DISTRIBUTED PARTICIPATORY DESIGN 165

REFERENCES 167

APPENDIX A 181

APPENDIX B 185

APPENDIX C 191

APPENDIX D 204

xvii

LIST OF FIGURES

Figure 2-1 Horizontal and Vertical dimensions of prototyping. (Nielsen, 1993, p. 94) ______________9 Figure 2-2 PICTIVE setting (Muller, 1991) ______________________________________________14 Figure 2-3 ZAL Interfaces____________________________________________________________18 Figure 2-6 Make a dialog box appear when the button is pressed. (Landay, 1996a) _______________22 Figure 2-5 DENIM, shown here in “Storyboard View” _____________________________________23 Figure 2-6 Denim - run mode, a simple browser providing back, forward and refresh button. The user can interact with the model by clicking on the hyperlinks____________________________________24 Figure 2-7 Damask user interface (Lin, 2003) ____________________________________________25 Figure 2-8 Active area of a sketch within InDesign ________________________________________26 Figure 2-9 Showing links between screens within InDesign. Scanned sketches are shown in left part of screen ___________________________________________________________________________26 Figure 2-12 Design view (Left), Storyboard view (Right) ____________________________________27 Figure 3-1 A screen state used in the paper prototype with addition of an interface element in the form of a message dialogue box____________________________________________________________30 Figure 3-2 Navigational view in InDesign _______________________________________________32 Figure 3-3 Simulating interaction ______________________________________________________34 Figure 3-4 Page view in FreeForm_____________________________________________________36 Figure 3-5 Storyboard view in FreeForm ________________________________________________37 Figure 6-1 DENIM window___________________________________________________________79 Figure 6-2 DENIM design mode at storyboard view _______________________________________80 Figure 6-3 DENIM design mode at page view ____________________________________________80 Figure 6-4 DENIM in run mode _______________________________________________________81 Figure 6-5 DENIM relies on Satin _____________________________________________________82 Figure 6-6 UML diagram of DENIM scenegraph__________________________________________83 Figure 6-7 Tree data structure of DENIM scenegraph ______________________________________84 Figure 6-8 A visual representation of DENIM scenegraph. DenimSheet boundary is highlighted with a red line, DenimLabel with a black line, DenimSketch with a green line and TimedStroke with a blue line. Also TypedText and ScribbledText are highlighted within the sketch ___________________________85 Figure 6-9 Diagram of how strokes are handled as gesture and ink ___________________________86 Figure 6-10 Login box: allowing stakeholders to select different versions of Gabbeh _____________87 Figure 6-11 Gabbeh toolbox, Commenting pencil for creating and editing comments (fifth from left), pad for filtering the comments (sixth from left) _______________________________________________88 Figure 6-12 Comments can be created by writing words or phrases directly on the design sheet at any zoom levels _______________________________________________________________________89 Figure 6-13 the comment dialogue box allowing the designer to select the colour as well as entering the typed text to create a comment ________________________________________________________89 Figure 6-14 Using the drawing or commenting pencil to associate a comment with one or many design objects ___________________________________________________________________________90 Figure 6-15 Representing a comment shared between components from two pages in the design_____90 Figure 6-16 Design sheet crowded with comments_________________________________________92 Figure 6-17 Using the commenting pencil for managing the visibility of comments _______________92 Figure 6-18 A coloured node represents the comment once it is set to invisible___________________93 Figure 6-19 Managing comments in Gabbeh _____________________________________________93 Figure 6-20 The run mode interface for Gabbeh. File button is to open a design file, Run button to start the execution of the design model and save button to save the added comments to the design model __94 Figure 6-21 Limited functionality browser with commenting features __________________________94 Figure 6-22 Viewing comments in the 'run mode' __________________________________________95 Figure 6-23 Adding comments when Gabbeh is executed____________________________________95 Figure 6-24 Dependency Graph for DenimPanel __________________________________________97

xviii

Figure 6-25 DenimComponent extending DenimIntrinsicComponent _________________________100 Figure 6-26 Introducing new tools in Gabbeh ___________________________________________101 Figure 7-1 Illustration of the layout of the evaluation exercises in two different location. The only communication channel was through the email and prototype while there was no face-to-face meeting________________________________________________________________________________107 Figure 7-2 This is a picture taken in the pilot study demonstrating the designer-users working together using a graphic tablet and laptop connected to each other. Both designer-users were contributing to the design by using the tablet, keyboard and the mouse. Halfway through the design, two designer-users swapped places ___________________________________________________________________109 Figure 7-3 Picture taken from the first design exercise (group1). The picture is demonstrating one designer (left) in charge while the other designer (right) is more distanced from the computer. The video record showed both designer-users discussed the problem but only one (left) was creating the design. While the other one (right) was mainly giving feedback and making sure that the agreed design was developed and in some occasions give assistance by typing on the keyboard____________________110 Figure 7-4 The comments were used in all exercises. This figure is demonstrating the use of comments as Gabbeh was used to develop a prototype (Group 2, Session 2) ____________________________115 Figure 7-5 This thread of two was identified in the design developed by the group 2. The designer-users have changed part of the design in response to users comment. Here the design-users are explaining the new design which receives the user’s approval___________________________________________117 Figure 7-6 This thread was identified in the design from the pilot study. The designer-users explain the design while requesting the user’s approval _____________________________________________117 Figure 7-7 Examples of clarifying comments (group 2, session 2). The blue comment is clarifying the design. Designer-users have made comment to explain what would happen once the user clicked on one of the items in the publication lists. It has allowed designer-users to avoid designing a new page to simulate the dynamic behaviour and its functional output in the form of a new page. _____________119 Figure 7-8 An example of verifying comments as part of a thread ____________________________120 Figure 7-9 The end-user requesting a new page to be added - An example of altering the design____121 Figure 7-10 Red comment shows an example for understanding comments helping the end-user to understand the provisional part of design_______________________________________________122 Figure 8-2 A discussion example from observational studies ________________________________128 Figure 8-3 Short comments were used to approve the design________________________________129 Figure 8-3 The dialogue box for allowing the users to provide a name ________________________131 Figure 8-4 The new associated point to a comments is highlighted on the page _________________132 Figure 8-5 Comments displayed in the run-view when no comment is selected.__________________133 Figure 8-6 Displaying a comment when it is selected______________________________________134 Figure 8-7 Editing an existing comment ________________________________________________134 Figure 8-8 comment inserted as a reply to an existing comment _____________________________135 Figure 8-9 Closed view of a discussion_________________________________________________135 Figure 8-10 Expanded view of a discussion _____________________________________________136 Figure 8-11 A comment with flag _____________________________________________________137 Figure 8-12 filtering toolbar _________________________________________________________137 Figure 9-1 Page "design", sent to user in iteration one ____________________________________147 Figure 9-2 Page "design", sent to user in iteration 9, evolving through 8 iterations ______________147 Figure 9-3 A discussion that is representing comments from user and designer giving instruction to each other ___________________________________________________________________________148 Figure 9-4 Showing a discussion, there has been a two iterations gap between second and third comment. ________________________________________________________________________148 Figure 9-5 An example of debating a design issue ________________________________________150 Figure 9-6 Another example of debating a design issue ____________________________________150 Figure 9-7 User’s feedback__________________________________________________________151 Figure 9-8 Clarifying design through discussion _________________________________________152 Figure 9-9 A discussion, recording a design issue ________________________________________153 Figure 9-10 A discussion on technical issues ____________________________________________153 Figure 9-11 Confirmation by designer _________________________________________________155 Figure 9-12 Confirmation by Designer_________________________________________________156 Figure 9-13 Confirmation by user_____________________________________________________156

xix

LIST OF TABLES

Table 4-1 Summary of 14 Cognitive dimensions in relation to prototyping ______________________58 Table 5-1Mapping annotations form into functions ________________________________________68 Table 7-1 This table presents the number of comments created for each session as well as presenting the overall comments made while using Gabbeh in comparison to DENIM________________________114 Table 7-2 List of identified discussions for each session divided in two main groups of Designer-initiated or User-initiated discussions_________________________________________________________117 Table 7-3 Total number of comment-phrases associated to each category for the three evaluation studies________________________________________________________________________________122 Table 7-4 Number of comment-phrases associated to each category for the pilot study ___________123 Table 9-1 Results for the comments associated to the whole design ___________________________142 Table 9-2 Number of discussions associated with each page of the design _____________________143 Table 9-3 Discussions grouped based on the number of sessions_____________________________144 Table 9-4 Discussions grouped based on number of the comments involved ____________________144 Table 9-5 Number of comment phrases associated to each category __________________________154

xx

LIST OF PUBLICATION

Naghsh, A. M., Dearden, A. and Ozcan, M. B. (2006). Investigating annotation in electronic paper-prototypes. In: Gilroy, Stephen W. and Harrison, Michael D. (eds.). DSV-IS'05 Interactive Systems, Design, Specification, and Verification, 12th International Workshop. Lecture Notes in Computer Science, Springer, pp. 90-101.

Naghsh, A. M. and Dearden, A. (2005). Supporting collaboration in electronic paper prototyping. In: The Workshop on Cognition and Collaboration Analyzing Distributed Community Practices for Design at CHI 2005, Portland, Oregon, USA.

Naghsh, A. M. and Ozcan, M. B. (2004). GABBEH - A tool for computer supported collaboration in electronic paper prototyping. In: A., Dearden and L., Watts (eds.). HCI'04 the Annual Conference of British HCI Group: Design for Life, Leeds, UK, Springer, pp. 77-80.

Naghsh, A. M. (2004). An electronic prototyping tool to facilitate annotations in the early sages of design. In: Doctoral Consortium at Participatory Design Conference 2004, Toronto, Canada.

Naghsh, A. M. (2004). Towards computer supported collaboration in electronic paper prototyping. In: Doctoral Consortium at CSCW'04 the ACM Conference on Computer Supported Cooperative Work, Chicago, USA.

Naghsh, A. M. and Dearden, A. (2004). A demonstration of GABBEH: A tool to support collaboration in electronic paper prototyping. In: CSCW'04 the ACM Conference on Computer Supported Cooperative Work, Chicago, USA.

Dearden, A., Naghsh, A. M. and Ozcan M. B. (2004). Support for participation in electronic paper prototyping. In: The Proceedings of the Participatory Design Conference, CA, USA, CPSR Palo Alto, pp. 105 – 108.

Dearden, A., Siddiqi, J. I. and Naghsh A. M. (2003). Using cognitive dimensions to compare prototyping techniques. In: The First Joint Conference of EASE & PPIG (15th Annual Meeting of the Psychology of Programming Interest Group), Keel, UK.

Amir M Naghsh 1

Amir M Naghsh 2

Chapter 1. INTRODUCTION

1.1. USER PARTICIPATION AND PROTOTYPING

The conventional wisdom within the information systems community suggests that user participation is central to the successful development of information systems [Ives and Olson 1984].

Involving users directly in the software development process has found its clearest expression in the participatory design (PD) approach. PD emerged in the Scandinavian countries that were concern with bringing democracy into the work place (Floyd et al., 1989). The focus of PD has since shifted to a belief that direct user participation in the development process is a key factor in a successful design (Ehn, 1993). Such participation must proceed iteratively from the early stages of design (Rosson et al., 1988) by involving different stakeholders (e.g. designers, users and developers) who are collaboratively engaged in producing the design proposals (Bodker et al., 1993). Therefore, it is an essential element of all participatory approaches to enable users to envisage or make sense of design proposals (whether those proposals originate with ‘professional designers’ or from the users themselves). Users can only make informed choices when the proposals being discussed are meaningful to them.

Prototyping is one popular method of helping users (and designers) to understand possible alternatives. Prototyping is suitable for gaining experience of new applications by implementing parts of software requirements to learn more about actual requirements or about alternative designs that can be used in the refinement of requirements.

“A prototype is a practical implementation of a system built expressly to learn more about a problem or solution to a problem” (Davis, 1992).

The traditional approaches to software development such as the waterfall model pay little attention to user involvement in the design process. These approaches generate prototypes that are mainly intended to help designers and software engineers in providing a description of future systems. In contrast, Bodker and Gronbæk (1991), in

Amir M Naghsh 3

explaining their cooperative approach describe prototyping as a useful process for uncovering unspecified and unstated aspects of users’ work and encouraging them to contribute more in improving the prototype through the design process. They found prototyping useful in simulating the dynamic behaviour of a future system.

“Users need to be actively involved in prototyping - passive participation in demonstrations and unplanned evaluations of prototypes is insufficient to get benefits from prototyping.” (Gronbæk, 1991, p. 31)

Bodker and Gronbæk (1991) explain that cooperative prototyping has its roots in exploratory approaches, as a cooperative activity between users and designers, rather than an activity of designers utilising users’ stated requirements.

The cooperative prototyping approach aims to establish a design process where both users and designers are participating actively and creatively, drawing on their different qualifications. To facilitate such a process, the designers must somehow let the users experience a fluent work-like situation with a future computer application; that is, users’ current skills must be brought into contact with new technological possibilities. (Bodker and Gronbæk 1991, p. 200)

It is not essential for users and designers to understand each other as long as they can relate to the prototype and materials used in the design process (Ehn, 1988). Therefore, a key issue in enabling users to be more involved and creative in the design activities is to use tools and materials which offer family resemblance to the tools and materials that users work with in their ordinary work places (Ehn & Kyng, 1991). The use of pencil and paper is an established participatory approach for designing interactive systems and has been suggested to encourage user participation in the design process (Ehn & Kyng, 1991; Preece et al., 2002).

Whilst paper-prototyping has many advantages in promoting user participation, it also has some limitations. In particular:

• The lack of an explicit representation of the navigational structure could make it difficult for users to understand and revise the dynamic behaviour of paper- prototypes (O’Neill et al., 1999);

• It is difficult to review a paper-prototype when users and designers are dispersed and are not able to arrange a face-to-face meeting;

Amir M Naghsh 4

• Paper-prototypes may be difficult to relate to other representations being used within design, such as detailed specifications of behaviour and functionality;

• The lack of support for design memory could make it difficult for users to search through sketches in the future to find out why a particular design decision was made (Landay, 1996a); and

• Paper-prototypes are hard to modify. By increasing the number of sketches the designer must frequently redraw the common features of the design for all the sketches (Landay, 1996a).

As pen-based interaction devices have become more widely available, some systems have applied pen-based interaction to explore ways in which some of the benefits of paper prototyping could be realised in interactive systems design. Examples include SILK, DENIM and DAMASK (Landay, 1996b; Lin et al., 2000; Newman et al. 2003; Lin, 2003) and FreeForm (Plimmer & Apperley, 2003). These systems might be described as supporting a form of ‘electronic-paper prototyping’. Other work has also explored ways in which the benefits of paper-prototyping might be realised in software prototyping environments, without relying on pen-based interaction (van de Kant et al., 1998; Nixon, 2001). These approaches overcome some of the limitations of paper-prototyping. In particular, these systems can make the dynamic behaviour of the proposed system easier for users to perceive and can permit the prototype to be distributed electronically.

However, the designs of these existing tools are primarily oriented towards the needs of software developers directly involved in creating designs, rather than considering how other stakeholders can provide inputs to design.

1.2. COMMUNICATION IN DESIGN

Earlier studies in design work (Bellotti and Bly, 1996; Button and Sharrock, 1996; Henderson, 1992; Pycock and Bowers, 1996; Wojahn et al., 1998) address various ways in which communicative activities support design processes. Button and Sharrock (1996) noted that the design process depends on communications, and on transformation process involving design artefacts (e.g. prototypes). Perry and Sanderson (1998) described the interactions between designers and other stakeholders as key dimensions in the design process. The communication dimension and the role and transformation of artefacts in design work intersect, in that the artefacts are subject to discussion, negotiation, and alteration and help to mediate and organise the

Amir M Naghsh 5

communication. Bäumer et al. (1996) in a series of case studies found that one of the important goals in building prototypes was to make the intended solution easily communicable and allow different stakeholders to contribute to it.

Näslund (1997) discussed the different groups of stakeholders who are involved within a software development project. He explained that the number of potential stakeholder groups who may be involved during the design process is extensive. These groups may be external or internal to the organisation in which the design group is located. Members of all these stakeholder groups work together to review and discuss artefacts to improve them to reach common ground on the basis that one artefact can and should be developed.

While a key dimension in the design process is the interactions among stakeholders, it has become a major difficulty and challenging when stakeholders are dispersed due to the distributed nature of projects (Heeks et al., 2001; Audy et al., 2004; Coar, 2004). Morris et al. (1999) explained that people use the technology to communicate asynchronously and overcome the time and space constraints in distributed settings. Wojahn et al. (1998) noted the importance of providing affordable ways to communicate about collaboratively produced artefacts and described making annotations as a common communication practice which could be easily shared, and passing such shared annotations among members of a team is a common practice even among those who work in the same office room or building. Bottoni et al (2004) explained that people used annotation when using paper documents to make design decisions, communicate comments with other team members or highlight important design issues.

Recent studies (Cadiz et al., 2000; Brush et al., 2001; Weng and Gennari, 2004) investigated how annotations can facilitate asynchronous communication. Cadiz et al. (2000) explained that annotations that are shared deliberately support communication and collaboration between members of a group.

1.3. THE THESIS

While participatory design approaches are primarily focused on facilitating projects with co-located stakeholders, this thesis aims to investigate how the use of annotation in electronic paper-prototyping tools could facilitate asynchronous communication to encourage user involvement in early design, particularly when stakeholders are fully distributed. Overall this thesis makes following contributions:

Amir M Naghsh 6

The first contribution of the thesis is to show how Cognitive Dimensions can be used to compare existing electronic paper-prototyping tools and identifies secondary notation as a feature that is not provided in any of previously existing tools.

The thesis argues the importance of providing secondary notation in supporting communicative activities in a distributed setting to offer the stakeholders an opportunity to communicate and conduct discussion at early stages of design.

The second contribution of the thesis is in demonstrating the feasibility of supporting communication in electronic paper-prototyping through annotation. The thesis reports on the design, implementation and evaluation of Gabbeh: an electronic paper-prototyping tool that allows different stakeholders to add arbitrary annotations in the form of comments when the prototype is being designed or when the prototype is being executed. The thesis reports on the results of an observational study using Gabbeh in a distributed setting and shows that when the annotation facility is available, more comments were exchanged among participants.

The third contribution of the thesis is a possible classification of the uses made of annotations in designing interactive systems.

The final contribution of the thesis is to suggest a set of features (based on reviewed literature and evaluation studies) that need to be considered when providing annotation features to support asynchronous communication in electronic paper-prototyping tools.

1.4. OUTLINE

Chapter 2 gives an introduction to prototyping in designing interactive systems and reviews the existing prototyping methods and tools. The chapter considers the traditional concept of fidelity to review methods and tools belonging to three categories of low-fidelity, high-fidelity and medium-fidelity.

Chapter 3 investigates the use of electronic paper prototyping tools by conducting a pragmatic investigation. The chapter argues that the traditional concept of fidelity is not sufficiently fine-grained to analyse important similarities and differences between the available set of prototyping methods and tools.

Chapter 4 uses the Cognitive Dimension to evaluate the electronic paper prototyping approaches. Based on the findings presented in the chapter, it highlights the importance of providing “secondary notation” in electronic paper-prototyping tools. The chapter

Amir M Naghsh 7

shows that existing electronic paper-prototyping tools do not support secondary notation and are therefore limited in their ability in supporting communication among different stakeholders.

Chapter 5 reviews various studies investigating the use of annotation in text books to digital documents. The chapter presents the findings of previous studies on investigating the use of annotation in supporting asynchronous collaboration around digital documents and reports on the form and functions of annotation, as well as the characteristics for such annotation tools.

Chapter 6 discusses the design and implementation of an electronic paper-prototyping tool with an annotation facility. The chapter reports on a list of requirements as a possible design for such an electronic paper-prototyping tool based on the reviewed literature in chapter 5 and the outcome of a number of design sessions. Iterates further, the chapter introduces Gabbeh and reports the user interface of Gabbeh and its implementation.

Chapter 7 reports on the evaluation of the comment facility introduced in Gabbeh. The chapter explains the objectives and layout used in the observational studies. Further, the chapter presents the findings and identifies a possible classification of uses of annotations during the design process. Finally, this chapter reports the analysis of the findings and explains areas of concern with the first version of Gabbeh.

Chapter 8 discusses the design of the commenting tool and required enhancements to the design to support communication in form of discussions. The chapter reports the further implementation of Gabbeh and presents the improved user interface and features introduced in the second version of Gabbeh.

Chapter 9 reports on a longitudinal case-study conducted to understand how Gabbeh supports communication through annotation in a distributed setting. The chapter presents the outcome of the study and analysis. Finally the chapter uses the data to validate the classification of uses of annotation and extends it further.

Chapter 10 concludes the thesis, discusses the impact of the research and summarises the contributions and lessons learned from previous chapters. Finally it assesses the research and points towards the areas for future work.

Amir M Naghsh 8

Chapter 2. REVIEW OF PROTOTYPING

2.1. INTRODUCTION

This chapter reviews the prototyping literature concerning the use of prototypes in the development process. The chapter further reviews prototyping tools and approaches and discusses how such tools and approaches improve user participation.

2.2. WHAT TO PROTOTYPE?

The entire idea behind prototyping is to save on the time and cost to develop something that can be tested with real users. These savings can only be achieved by somehow reducing the prototype compared with the full system: either cutting down on number of features in the prototype or reducing the level of functionality of the features such that they seem to work but do not actually do anything. (Nielsen, 1993, p. 94).

To achieve some of these savings Nielsen (1993) introduced two dimensions of vertical and horizontal prototyping.

Nielsen (1993) explained vertical prototyping as a narrow version of the finished system which only has a reduced number of features but includes in-depth functionality for the few selected features. Therefore, vertical prototyping can only evaluate a limited part of the finished system while allowing users to perform real tasks using selected features probably with some real data.

Nielsen explained horizontal prototyping as a surface layer that includes the entire user interface of the finished system but with reduced or no functionality. A horizontal prototype without underlying functionality allows users to feel the entire interface while not letting them perform any real tasks. The main advantage of horizontal prototypes is in allowing stakeholders to concentrate on the bigger picture by assessing the interface as a whole. This type of prototype can be implemented quickly with the use of less formal tools in comparison to vertical prototyping (see Figure 2-1).

Amir M Naghsh 9

Figure 2-1 Horizontal and Vertical dimensions of prototyping. (Nielsen, 1993, p. 94)

In addition to the dimensions already explained, prototyping approaches can be categorised from two points of view; one is based on the prototyping process and the other is based on the fidelity of the prototype.

This chapter presents a review of traditional ways of categorising prototyping approaches from these two points of view. Furthermore, the chapter reviews a number of prototyping methods and tools that encourage user involvement and participation in an iterative design process.

2.2.1. HOW THE PROTOTYPE IS USED?

Davis (1992) has described a framework for categorising the prototyping processes based on the development processes. Davis explained that there are three broad categories of prototyping processes in the literature of software engineering:

a) Throwaway prototyping: A throwaway prototype is built quicker than evolutionary and operational prototyping and implements poorly understood requirements. The throwaway prototype is not used as part of the finished system and it is used in an experimental manner to learn about “which requirements are real and which are not” to be considered in the development of the final system.

b) Evolutionary prototyping: This is built to a higher quality standard. The prototype evolves to become the final system. It implements only confirmed

Amir M Naghsh 10

requirements and is used to explore those requirements that exist but have not been captured. It is also used experimentally.

c) Operational prototyping: An evolutionary prototype is constructed and made into a baseline by developing requirements that are well understood. The copy of this baseline would be sent to other user sites. A trained prototyper would be sent to the user site to monitor users when interacting with the software system and record any problems or requirements which would free the user from having to record them. Changes to the top of baseline software would be made by the prototyper in a throwaway manner. The user then uses the new system composed of the baseline system and throwaway prototype. Useless changes would be discarded and any other changes would be reported to the development team and incorporated into the next version of the baseline.

2.2.2. THE FORM OF PROTOTYPES

Prototypes traditionally have been categorised based on the level of fidelity of the prototype (Walker et al., 2002; Leone et al., 2000; Rudd et al., 1996; Wong, 1992). The determining factor in the fidelity of a prototype is the degree to which that prototype presents application and the interaction of the system from the user's point of view (Tullis, 1990).

Rudd et al (1996) described low-fidelity prototypes with limited or no functionality. They are constructed quickly and allow designers and users to focus on high-level overviews of a final system by providing a representation of the general look and perhaps the feel of the interface, rather than the visual details and styles (Landay and Myers, 2001).

Low-fidelity prototypes are created to increase users' participation and users' feedback at the early stages of design by communicating, educating and informing ideas. Nielsen (1992) suggested that the unfinished look of low-fidelity prototypes encourages users to suggest more changes to the design. Virzi (1989) claimed low-fidelity prototypes are an effective approach for exploring the design space. Nielsen (1993) explained that low-fidelity prototypes can be used in horizontal prototyping and therefore make it quicker to generate the entire user interface.

High-fidelity prototypes represent the core functionality of the final system (Rudd et al., 1996). They are interactive and users can operate and even perform some real tasks with the prototype. Leone et al. (2000) described how the look of high-fidelity prototypes can be made in a way that users could find them similar to the final systems

Amir M Naghsh 11

which are typically created in later stages of the design process. Their creation requires more time and effort compared to low-fidelity prototypes. Nielsen (1993) suggested that high-fidelity prototypes could be used in vertical prototyping which gives a closer look and feel to the finished system.

Studies in comparing the low and high fidelity prototypes to evaluate the usability of the design have showed that low-fidelity prototypes are as effective as high-fidelity prototypes for evaluation by users (Virzi et al., 1996; Walker et al. 2002).

Other research has compared the low and high-fidelity prototypes to identify their advantages to determine at what stage in the development process they could be used. Retting (1994) argued that designers often use low-fidelity prototypes early in the design process. Many researchers (Preece et al., 2002; Muller, 1992; Ehn and Kyng, 1991; Ehn, 1989) have suggested the use of low-fidelity prototypes for creating an early impression of the possible design of an interactive system.

“Informal observations and discussions with designers show that they like to sketch out ideas. Sketching allow them to be more creative and exploratory than if they were using a computer. Sketching also made it easier to rapidly prototype and iterate on user interfaces” (Hong et al, 2002).

Rauch et al. (1997) also discussed the advantages of low and high-fidelity prototypes and argued that low-fidelity is more appropriate for the early stages of the design process for exploring the design while high-fidelity is more appropriate for the later stages of the design process.

The finished look of high-fidelity prototypes has some disadvantages for the early stages of the design process such as discouraging users from suggesting changes (Nilsen, 1992) or causing distraction by shifting the users’ focus to the low level details such as colour or font size (Hong et al., 2001; Wong 2002). However, despite the disadvantages that high-fidelity prototypes may cause at early stages of design, a study conducted by Newman and Landay (2000) interviewing eleven professional designers found out that designers may move to high-fidelity prototypes early in design since clients may find low-fidelity prototypes unprofessional.

Many researchers (Engelberg and Seffah, 2002; Leone et al., 2000) focussed on the gap between low and high fidelity prototypes and suggested a medium-fidelity prototype.

Amir M Naghsh 12

“In our experience, medium-fidelity prototyping allows for a greater flexibility in employing a prototype of higher fidelity [compared to the low-fidelity] and navigation capability earlier in the design process. […] this type of prototype can be built faster, shown to users sooner and iterated by developers throughout the design process (Leone et al., 2000, p. 234)”.

Some of the examples of tools and approaches supporting medium-fidelity prototyping include Microsoft PowerPoint, HTML documents and HyperCard.

To bridge the gap between high and low-fidelity prototypes, many studies (Liu and Khooshabeh, 2003; Hong et al., 2001; Landay and Myers, 1995; Uceta et al., 1998) suggested the importance of keeping the informal appearance of the low-fidelity prototypes while increasing the fidelity and their automation. They studied the use of sketches and paper-prototypes as examples of low-fidelity prototypes and argued that increasing the fidelity can improve the simulation and interaction of the paper-prototype while the informal representation can encourage more user participation.

Some of the works in this area (Landay, 1996b; Damn et al., 2000; Lin et al., 2000; Wilson et al., 1997) have explored ways in which some of the benefits of paper-prototyping could be realised in software environments. These recent tools can be referred to as "electronic paper-prototyping" that can be used in horizontal prototyping and offer more functionality and better navigation to improve the evaluation of the design by users. Tools supporting electronic pape- prototyping approaches generate prototypes that can be categorised as a form of medium-fidelity prototyping. Such prototypes can be used as an alternative to high and low-fidelity prototyping approaches.

Below, this chapter reviews a number of tools and methods divided into three categories of low, high and medium-fidelity. In addition, the chapter focuses on the electronic paper-prototyping tools as a separate section to review them in more details. The chapter discusses each of the approaches from the point of view of improving user involvement and the development process that it may support.

2.3. LOW-FIDELITY APPROACHES

2.3.1. STORYBOARDS

Andriole (1989) imitated film production by introducing Storyboarding as a prototyping approach for user requirements validation. Curtis and Vertelney (1990)

Amir M Naghsh 13

provided a detailed introduction to storyboard prototyping explaining that storyboards consist of a sequence of paper sketches illustrating a scenario. Each of these sketches can represent a screen of a future system or a scene of a user interacting with a device. A sequence of sketches could illustrate a function that a future system may perform or could show how a user interacts with a device when performing a task.

Paper storyboards are portable and can be exchanged with other stakeholders. Storyboards do not need a member of the design team to simulate the dynamic behaviour of the design. Storyboards can be used as part of the throwaway prototyping processes at the early stages of design.

2.3.2. PAPER PROTOTYPING

Another approach that makes the use of paper sketches is paper-prototyping. Unlike the storyboards that illustrate only one possible scenario in the form of a linear sequence of sketched screens, paper-prototypes include a possible user interface that illustrates many of the functions that a future system may perform.

Using paper-prototyping gives the opportunity to designers and users (who are not necessarily programmers) to get involved in generating a prototype and developing the design of an interactive system. Interface elements such as menus, windows, dialogues and icons can be sketched on paper or created in advance using cards, pens and office materials (Muller, 1992). When the paper-prototype has been prepared a member of the design team sits before a user and ‘plays the computer’ by moving interface elements around in response to the user’s actions. The user makes selections and activates interface elements by using their finger as a mouse and writing ‘typed’ input. A further person can assist the session by providing task instructions and encouraging the user to express their thoughts and impressions. Other observers may make notes or a video record may be created.

The individual playing the role of the computer must be fully aware of the functionality of the intended system in order to simulate the computer. However, due to the use of paper and a human operator, this form of prototype cannot be used reliably to simulate real system response times.

Rettig (1994) noted that breaking the user-developer barriers by encouraging communication and collaboration between the designers and users in the early stages of the development process is one of the most important advantages of the paper-prototyping method. Consequently usability problems could be detected at a very early stage in the design process.

Amir M Naghsh 14

This method is effective when the basic control types of a future interface are known but user feedback on a suitable layout is required. The method does not (necessary) require users to draw the interface although they can supplement the design with additional elements or annotations to add contents (Rettig 1994, p. 23).

Paper-prototypes are quick to build and refine and are used in throwaway and operational development processes. Paper-prototypes provide a valuable and cost-effective means of evaluating and iterating design options before a team gets committed to one implementation. However, because of their simplicity, they are not suitable for evaluation of fine design detail (Retting, 1994).

2.3.3. PICTIVE

PICTIVE (Plastic Interface for Collaborative Technology Initiatives through Video Exploration) was created at Bellcore by Muller [1991] combining low technology office objects with higher technology video recording.

PICTIVE is a sophisticated method that provides the users with a set of paper, cardboard or plastic interface elements which they can lay out on a flat surface in what they feel is an appropriate way while designs are discussed and developed as a group or individually.

PICTIVE, by using ordinary office materials, offers a more equal opportunity to both users and developers to get involved in designing interfaces of an interactive system rather than the evaluation of a designed interface.

Figure 2-2 PICTIVE setting (Muller, 1991)

Amir M Naghsh 15

PICTIVE uses video as a recording technique to reassure participants that their ideas and views in sessions are going to be used. This provides an informal and complete record of design.

The success of the exercise relies on the presence of a co-ordinator in the meeting. The main role of this person is to ensure that the group stay focused upon the design problem and ensuring that every member of the group is given the opportunity to stand up and present his or her own ideas.

2.3.4. RAPID SOFTWARE PROTOTYPING

Use of high-level programming languages and software tools was suggested (Gronbæk, 1989, Wilson and Rosenberg, 1988) to implement prototypes that represent the entire interface of a new interactive system (horizontal prototyping). These prototypes can be considered as low-fidelity since they require little programming effort in creating them and they support limited or no functionality to process data. These prototypes can be constructed quickly at the early stages of design. Since software tools and frameworks for implementing the final system are used to create rapid prototypes, they enable the prototypes to be reused in the production of the intended system or evolve to become the final system. Therefore, it is possible to apply either evolutionary or throwaway process to them.

These prototypes have the feel and look of the finished system, they support iterative design by obtaining users’ feedback and using them in the refinement of the prototype. However, Gronbæk (1989)’s research, studying nine projects, he noted that these prototypes are mainly intended to help designers and developers work.

The users did not adequately evaluate the horizontal prototypes, because of the limited functionality, lack of motivation, and bad conditions. Consequently the horizontal prototypes were reduced to be just substitutes for parts of the traditional written requirements specification (Gronbæk, 1989, p. 9).

2.4. HIGH-FIDELITY APPROACHES

2.4.1. RAPID APPLICATION DEVELOPMENT

Techniques that focus on creating and modifying executable software prototypes are widely used in commercial software development in order to obtain early feedback from users (Rapid Application Development) (Martin, 1991; Agarwal, 2000).

Amir M Naghsh 16

Rapid Application Development (RAD) can be used as a high-fidelity prototype when a few selected functions are implemented in detailed as part of the vertical prototyping which supports processing real data.

The use of vertical prototypes in two of the projects seemed to stimulate quite useful user response, which could be utilised in making stepwise development and test of a new computer system with ongoing end-user involvement (Gronbæk, 1989, p. 9).

Such prototypes require more effort, time and cost in creating them. These prototypes are often used in evolutionary prototyping where they can evolve from being a narrowed vertical portion of a system to become the entire future system.

Gronbæk (1989) explained that although such high-fidelity prototypes seem to support user-involvement, they are selective and to implement such prototype for the entire system it requires a large amount of resources. In addition, as Nielsen (1992) explained the finished look of such prototypes and the cost and time involve to incorporate changes into the prototype could discourage end users from making suggestions.

2.4.2. EXTREME PROGRAMMING (XP)

XP is another approach in software development that like RAD generates high-fidelity prototypes that evolve and become the final system in an evolutionary process. In contrast to RAD, which involves long development cycles (Gronbæk, 1989), XP applies shorter development cycles where an executable version of the system is delivered by the end of each cycle (Beck and Andres, 2004). XP aims to reduce the cost of change by adopting changing requirements in between its shorter iteration cycles.

XP highlights the importance of user involvement and allows a potential user of the system to be included as part of the development team on the site. The on-site user is representing an entire community who will use the system. XP assign a series of tasks to the on-site user, including providing feedback by the end of each cycle, driving the product by setting the priorities and answering the development queries on the site. The user's feedback is used to generate the requirements which are called stories in XP.

Although the possibility of integrating more user-centred approaches with XP has been suggested (Sharp and Robinson, 2004), current practice in XP does not permit direct participation of real end-users in designing the system; instead it suggests the use of users’ representatives. In addition, high level skills required for XP processes also do

Amir M Naghsh 17

not allow the on-site user to be involved along with the programmers participating actively and creatively in developing the system as Bodker and Gronbæk (1991) suggested in their cooperative approach.

2.4.3. EXECUTABLE FORMAL SPECIFICATION

Using formal specifications is another way to generate ideas about how an interactive system can be designed, and it helps to evaluate the quality of a solution at an early stage.

“Formal Methods are mathematically based techniques for describing system properties. Such formal methods provide frameworks within which people can specify, develop and verify system in a semantic rather than ad hoc manner (Wing, 1990, p. 8).”

Formal specifications describe the core functionality of the final system. Executable formal specifications are interactive and can be used for exploration and evaluation of the system. They are not as quick and easy to create as paper-prototypes but some of them can be made in such a way that users can find them similar to the final system (Palanque and Bastide, 1997).

It is been argued by a number of researchers (Fuchs, 1992; Plat et al., 1992; Palanque and Bastide, 1997) that specifications in formal languages such as Z, could be used as part of the prototyping process for safety critical systems or where the behaviour of a system is hard to describe except mathematically. Such a mathematical model can be used in explaining the detailed aspects about a design's behavioural properties. Users' feedback and involvement is essential in validation of these specifications.

Fuchs (1992) explains that executable specifications represent the conceptual and behavioural model of the system that is going to be implemented which can allow users and developers to interact with it.

Although researchers have suggested allowing users and developers to interact with executable formal specifications in order to validate requirements (Cooper et al. 2002; Bastide et al., 2003; Parry and Ozcan, 2000; Ozcan et al., 1998; Siddiqi et al., 1997; Fuchs, 1992), there is not much of evidence showing real end-users interacting with executable systems.

Formal specifications could be used in different stages of the development process, but they would not form any part of the finished deliverable system. Therefore, formal

Amir M Naghsh 18

methods are usually used as part of a throwaway process. The following section presents examples of formal executable specifications.

2.4.3.1. EXAMPLES

ZAL (Z Animation in Lisp) is a Lisp based environment, which constructs an executable version of a Z specification (Roast and Siddiqi, 1997; Siddiqi et al., 1997). TranZit (Morrey et al., 1998) is an editor for Z that makes it possible for developers and the designer team to move from informal to formal by using an initial specification written in English from user needs and construct a Z specification (Siddiqi et al., 1999). TranZit contains a transformer engine that converts the Z specification into Lisp syntax that will be used by ZAL. Once the converted version of Z has been transformed into ZAL, it is possible to assign values to the ZAL representation of the requirements in the form of text input and output to validate the Z model (Figure 2-3). ZAL representation is mainly based on mathematical notation and it is limited in providing user interfaces to encourage user involvement and support evaluation.

Figure 2-3 ZAL Interfaces

Further research has suggested the use of visualisation in displaying the mathematical notation in the form of familiar graphical representation (Cooper et al., 2002; Ozcan et al., 1998). Visualisation in Z (VIZ) (Parry and Ozcan, 2000) is an extension to ZAL which provides alternative representations in the form of familiar graphics that offer a better understanding for the end users. First, objects and their actions in a ZAL representation are identified, and then graphics from a component library are used to generate static and dynamic visual representation of identified objects. Then VIZ

Amir M Naghsh 19

attaches the generated graphical models to the underlying ZAL representation and allows user evaluation.

Palanque and Bastide (1997) used Petri nets a graphical and mathematical tool and its diagrammatic representations to model a system specification by emphasising the dynamic behaviour of the system and the tasks of users interacting with the system.

They explained that they have used an object structured dialect of Petri nets which is called interactive cooperative objects or ICO formalism (Bastide and Palanque, 1995). Object-Oriented features of ICO formalism used in modelling the structure and static aspects of the interactive system, while the Petri nets is used to explain the dynamic behaviour of the system.

The use of formal description based on Petri nets has been refined further into a tool called PetShop (Bastide et al., 2003). The main application of this tool is for interactive critical systems such as air traffic control. PetShop offers an iterative development process and uses an interface builder to visualise the underlying formal ICO model for evaluating the prototype by allowing users to interact with it.

Although such tools as VIZ and PetShop enable developers to validate the formal specification by visualising them early in the design and allowing end users to interact with them, the complexity of the underlying formal model does not allow the end user to participate directly in generating the prototype.

2.5. MEDIUM-FIDELITY APPROACHES

Leone et al. (2000) suggested HTML as a method for medium fidelity prototyping to quickly generate user interfaces and demonstrate to the end users. They explained how using HTML addressed some of the limitations of low and high-fidelity prototyping. Unlike high-fidelity prototypes it was quick, and cheap to construct and easier to maintain. It would support an iterative process and capture user requirement in early design. Unlike low-fidelity paper-prototypes they can support close navigation to the final system and allow users to interact with them. They reported that they received more constructive feedback from medium-fidelity prototypes in comparison to high and low fidelity prototypes.

Engelberg and Seffah (2002) used PowerPoint as a tool to support medium-fidelity prototyping. Similar to HTML, PowerPoint was used to prototype the interactive and navigational aspects of interfaces in early design to capture users feedback. Although

Amir M Naghsh 20

using PowerPoint offered benefits such as being easy to learn, quick in constructing the prototype and supporting hyperlinks, they found it weak for designing and sketching dynamic menus, designing a page that is larger than a PowerPoint predefined page size, and also difficult in modifying larger models.

Another tool that can be used in medium-fidelity prototyping is HyperCard. This was an application program from Apple Computers and it has been out of production since 2004 (and is not supported by Apple anymore). HyperCard uses a stack of cards, each of which can contain text, picture and other interface elements available from the tool box. It offers functionality by supporting scripts that can be attached to the buttons. The simplest script is similar to the hyperlink which can link two different cards.

2.6. ELECTRONIC PAPER PROTOTYPING TOOLS

Landay (1996a) argues that an interactive user-interface design tool that supports electronic sketching (an electronic pape- prototyping tool) would give designers more freedom to change sketches and more flexibility in creating and evaluating a design prototype. This flexibility is particularly important in the early stages of design itself, when designers need the freedom to sketch rough design ideas quickly, the ability to test designs by letting users interact with sketches, and the flexibility to fill in the design details as they make choices.

‘Electronic paper-prototyping’ tools try to realise the benefits of paper-prototyping in encouraging user participation by allowing designers and users to participate directly and creatively when designing prototypes (Bodker and Gronbæk, 1991).

This section introduces a number of electronic paper-prototyping tools that can be used in medium fidelity prototyping.

2.6.1. PATCHWORK

Patchwork, designed by van Kant et al. (1998) is a prototyping tool, where designers express their ideas in the form of storyboards. Patchwork consists of patches that are bitmaps and can be in any shape. These patches are easily modifiable and rough-looking building blocks. Many of the patches are digital representations of low technology office materials. It is also possible to use digital photography to import sketches from paper, whiteboard or mock-ups into patches.

Patchwork supports storyboards. The designer can use the workspace to create a prototype. Any time during the design process, the designer can switch from design

Amir M Naghsh 21

mode to run mode to evaluate the design. In the run mode, it is possible to interact with the prototype by clicking on the patches that are linked to other screens. Each screen in the storyboard consists of a background and a number of patches. In addition, patchwork allows the designer to sketch on the background of the storyboard or the patches using coloured pens. It is explained that a limited set of colours have been provided to prevent the designers from focusing on design details too early.

Patchwork is intended as basic tool to encourage user participation in building throwaway prototypes early in design.

2.6.2. SILK AND DENIM

SILK stands for Sketching Interfaces Like Crazy (Landay and Myers, 2001). SILK is written in common Lisp and runs on top of Garnet system1. Garnet is a user interface development environment for common Lisp. Designers can run SILK on a graphic tablet such as a TabletPC or a Wocam Cintiq. It allows designers to quickly sketch out GUI interfaces where designers are not required to specify details that are not known or are not important at the early stages of design (e.g. Size, Colour, Shape, etc).

SILK then retains the sketch-like look of the components. It provides a storyboard view where designers can view more than one sketched interface and identify navigational links between them. These sketches can then be executed on the basis of the state transitions, allowing a user to experience the proposed navigational behaviour of a design.

Further, SILK can transform the sketches into operational interfaces by recognising the gestures as visual basic components (e.g. list, button, etc). SILK can be applied in throwaway and evolutionary prototyping.

1 See http://www.cs.cmu.edu/~garnet/

Amir M Naghsh 22

Figure 2-4 Make a dialog box appear when the button is pressed. (Landay, 1996a)

DENIM2 (Newman et al., 2003) is an outgrowth of SILK which has been implemented in Java environment. Denim is a sketching tool to assist website designers to build throwaway prototypes at the early stages of design.

Similar to SILK, DENIM also runs on a graphic tablet such as a TabletPC or a Wacom Cintiq. However, Denim does not support recognition of drawn gestures. Instead it supports more views that allow designers to use them to sketch out the overall structure of a site (site map view); sketch the contents of the pages as a set of ‘scribbles’ (page view); define hyperlinks from scribbles in one page to another page (storyboard view); and then execute the resulting hypertext in a reduced functionality browser.

Figure 2-5 presents a screenshot from DENIM. The slider bar to the left of the screen allows the site to be viewed at different levels of detail – varying from a site overview that simply identifies the pages included, through a navigation view where the overall navigation can be examined; down to a detailed view where fine details of individual pages can be manipulated. At the bottom of the screen is a toolbar that includes a pencil - for sketching new pages, new content within pages, or adding links; an eraser; a hand – for moving the working surface to bring different areas into view; and a stamp for adding typed text fields. To add links, the user selects the pencil and draws a line from a scribble in one page to the destination page.

Denim also supports abstractions and allows the designer to create templates for the design. Modifying the template can result in making global changes to all the pages

2 Denim version 1 is considered for the thesis that has been available since 2002. However, this section includes references to Denim version 2 that became available in late 2006.

Amir M Naghsh 23

that have used that template. The template can also be used to create components such as a menu navigation bar that can be used within the pages.

Figure 2-5 DENIM, shown here in “Storyboard View”

DENIM also supports storyboards and allows designers and users interact with design through a run mode (see Figure 2-6). It displays sketched pages in a limited functionality browser and enables the users and designers to navigate the designed site by clicking on active areas.

Amir M Naghsh 24

Figure 2-6 Denim - run mode, a simple browser providing

back, forward and refresh button. The user can interact with the model by clicking on the hyperlinks

DAMASK (Lin, 2003) is an extension of DENIM tool that is developed further to support the early stage design for cross device user interfaces such as mobile phones, PDAs and PCs (see Figure 2-7). DAMASK includes a series of design patterns from the book: The Design of Sites (van Duyne et al., 2002). Each pattern has a different solution for each device.

Amir M Naghsh 25

Figure 2-7 Damask user interface (Lin, 2003)

Damask allows the designer to split the design view into multiple screens to view the design for different devices at the same time. To create a cross-device interface, designers have to first design the interface for one of the devices and then specify a design pattern to be used. The selected pattern will generate the user interface for other devices. Therefore, the designer doesn’t need to sketch out the interface for each device. Damask also allows designers to define custom patterns which can be shared by other team members.

2.6.3. INDESIGN

InDesign (Nixon, 2001) supports storyboards to support navigational representations of paper-prototypes. It displays scanned paper sketches in individual screens and lets users and designers navigate through these screens by clicking on the active area of each one. InDesign connects one screen to another by using a navigational link between two screens and adding a label on a displayed sketch within the screen to show the actual place of the navigational link.

Amir M Naghsh 26

Figure 2-8 Active area of a sketch within InDesign

Figure 2-9 Showing links between screens within InDesign. Scanned sketches are shown in left part of screen

Amir M Naghsh 27

2.6.4. FREEFORM

FreeForm (Plimmer & Apperley, 2003) is a tool for designing Visual Basic forms by interacting with an electronic whiteboard. Users draw their designs using specialised whiteboard pens, and when a specific button is pressed the marks are recognised and replaced by user interface widgets. Forms can be associated with navigation links which can be positioned within the storyboard view. Evaluation is then possible in FreeForm in a specialised mode, in which navigation links are indicated by specially coloured target areas which provide the dynamic behaviour of the prototype.

FreeForm, similar to SILK, allows the evolution of the design by recognising gestures that correspond to particular Visual Basic interaction components such as tables, drop-down menus, text fields etc. A button is provided to convert the finished design into a set of Visual Basic forms.

Figure 2-10 Design view (Left), Storyboard view (Right)

2.7. SUMMARY

Methods and tools which have been explained in this chapter are designed in a way to support iterative design. Iterative design involves creating an artefact, evaluating and reviewing it, and then repeating the process several times.

High-fidelity prototyping approaches such as executable software prototypes, executable formal specifications and XP are intended for use by software engineers. Although some of the high-fidelity prototyping approaches (e.g. XP) support iterative design through shorter cycles, it can still be argued that they have longer cycles to evaluate and iterate design options than low-fidelity tools and methods discussed.

Medium fidelity prototypes have been designed to overcome some of the disadvantages of low-fidelity prototypes. These approaches are intended for use by UI

Amir M Naghsh 28

designers and support quick prototyping and iteration. Some of the examples of medium fidelity prototyping tools and approaches are Microsoft PowerPoint, Hyper-card and HTML documents.

Active user participation in designing interactive systems has been identified as one of the primary keys in developing useful and useable software products (O’Neill, 2001). Although the medium fidelity prototypes have been designed to overcome some of the disadvantages of low-fidelity prototypes, none of the tools mentioned earlier realise the benefits of low-fidelity approaches in encouraging direct user involvements.

Therefore it is essential for users to feed into the design from the early stages of the design process, because that is when fundamentally different ideas can and should be generated and observed. Low-fidelity prototyping methods such as paper-prototypes give the opportunity to members of different stakeholder groups, mainly users, to get involved in generating a prototype rather than just the evaluation of a developed prototype.

However, paper-prototypes have disadvantages as explained in Chapter 1, such as: the difficulty in reviewing them when stakeholder teams are distributed; lack of explicit representation of the navigational structure; lack of support for design memory and difficulty in modifying them.

The electronic paper-prototyping tools discussed in this chapter allow designers to sketch user-interface elements or rough layout for web pages and identify navigational links between states or pages by drawing arrows. These sketches can then be executed on the basis of the state transitions, allowing a user to experience the proposed navigational behaviour of the design. These approaches try to realise the benefits of paper-prototyping in encouraging user participation.

However, none of the above studies report on comparing the mentioned electronic paper-prototyping tools. Therefore to be able to compare these tools, the next chapter investigates the use of electronic paper-prototyping tools as part of a pragmatic investigation.

Amir M Naghsh 29

Chapter 3. COMPARING EXISTING ELECTRONIC PAPER PROTOTYPING TOOLS

3.1. INTRODUCTION

To investigate the use of electronic paper prototyping tools, a pragmatic investigation was conducted to explore a simple design task using different Electronic Paper Prototyping tools. DENIM, FreeForm and InDesign were considered for the investigation, while SILK and Patchwork were excluded from this investigation. SILK never has been released to the public, due to the resources Garnet and common Lisp require to be able to run SILK. Patchwork was excluded since there was no access available to either of its source code or the executable. DAMASK was also excluded because of its high level of similarity to DENIM. Instead, paper prototyping was included in order to be able to compare the tools that try to realise paper prototyping benefits.

The design task was a simple student marks program. The student marks program had to be designed in such a way as to keep a record of each university course and each enrolled student. The main purpose of the system was to allow users to assign students to a course module and allocate a mark to each student. In addition, it had to allow users to add, edit and remove records of students and course modules.

Other requirements included enabling users to use the system to generate the following reports:

• A list of units taken by each individual student;

• A list of students assigned to a given course module;

• The results for each individual student indicating all modules taken and assigned marks;

Amir M Naghsh 30

• The results for all students associated with one course module, also indicating whether each student has passed or failed the module.

To conduct the investigation, a paper-prototype of the student marks systems was created. The paper-prototype was used as the starting point in the form of a visualised requirement to generate the following prototypes of the student marks program by applying different methods and tools:

• an executable prototype using DENIM;

• an executable prototype using InDesign;

• an executable prototype using FreeForm.

3.2. PAPER PROTOTYPING

To create the paper prototype, A4 sized paper was used to sketch out the layout of each individual screen state. Special icons were sketched on paper, cut out and put onto the sketched interfaces to model the computer screen which then made it possible to change the arrangement of the interface elements quickly and ‘on the fly.’ To interact with each individual modelled screen pencil and post-it notes were used.

Figure 3-1 A screen state used in the paper prototype with addition of an interface element in the form of a message

dialogue box

Amir M Naghsh 31

Displaying sequences of screens did not really help to demonstrate the representation of the navigational structure, but going through different sequences of the interfaces made it possible to test multiple versions of the system’s interaction design.

The paper prototype was quickly constructed and simple skills were used in the creation of the prototype. Using the paper prototype with other researchers, giving feedback and making dialogues about the design seemed to be an easy task when evaluating it.

The sketches have to be organised in a way to allow the designer to find them as quickly as possible in order to respond to the user when interacting with the paper prototype. Although this was not a large design task in terms of sketched screens, organising the sketches and interface parts was a concern. Since each sketched screen of the paper prototype allows the user to interact with it in many different ways, it became more challenging to organise the screens as their number was increasing. It became difficult for the designer to remember what other screens are related to each individual screen, and interacting with which part of a screen requires the designer to display another screen.

The other main difficulty was remembering the history of a presented series of screens which make up an interaction sequence. An interaction sequence starts from a sketched screen and allows the user to perform a task through interacting with a number of screens and reaching the final screen. Remembering which screens were displayed before reaching the current sketch is difficult when there are too many sketched screens in a paper prototype.

Keeping track of all sketched screens that needed modifications to reflect a change in the design proved to be a difficult task. For example changing the location of the logo requires all the sketched screens to be updated, but adding a new item to a sub-menu requires the designer to find all the screens with that sub-menu to alter the design.

After completion, the paper prototype as a visualised requirement was used to create prototypes using InDesign, DENIM and FreeForm.

3.3. INDESIGN

To create a prototype using InDesign, sketched interfaces on paper were scanned and put into the image library of InDesign.

Amir M Naghsh 32

To create an interactive sequence of sketched screens, the relation between screens was recognised and links between them were created (see Figure 3-2). Being able to execute the prototype gave the opportunity to have a better understanding of how the dynamic behaviour of the final system might appear. However, use of hyper-links between screens was more of a way to change the state from one screen to another to follow a storyboard or illustration of a scenario. In particular, it was not possible to interact with each individual screen layout and interfaces in any form of input.

Figure 3-2 Navigational view in InDesign

Since InDesign does not support pen-based technology, it was not possible to alter the design and layout of a screen once it was scanned and transferred into the image library. It required the effort of finding the relevant sketched interface on the paper, making the changes on the paper and scanning the interface again to transfer it into InDesign repeatedly. Once the scanned sketch was available in the image library the screen holding the older version of the scanned sketched had to be retrieved in order to assign the updated scanned sketch to it. Keeping track and finding relevant screens holding specific sketches became difficult as the size and number of screens was increasing during the process.

In addition, the naming style of the screens which was combination of “screen” and a number indicating the order of the screen’s creation was not aiding the understanding of the screen’s content. Since it was not possible to change the screens’ name to

Amir M Naghsh 33

something more meaningful, retrieving screens of interest became more difficult as the number of screens was increasing. To find an interesting screen the content of many screens which seem to have similar layouts at the storyboard level had to be verified by changing the view to the page which had more details.

Alternatively, a graphic editor to alter the scanned screens could be used; however it still would require the screen holding the older version of the interface to be found before assigning the altered version of the sketched interface.

More importantly, since the links which connected different screens were not attached to the underlying sketches, once an altered sketched interface was assigned to an old screen, all the links from that screen needed to be updated.

3.4. DENIM

To create a prototype using DENIM, screens of paper prototype were used as a starting point and were re-sketched in the DENIM environment using a graphic tablet. In DENIM the designer had to name each screen.

The most used zoom levels were sitemap, storyboard and page levels. The sitemap level gave a view of the prototype as connected labels. The labels were used to indicate the screen names. This representation was used to review connected pages and identify other relations between screens.

The storyboard zoom level was also often used for viewing several pages simultaneously and observing the navigational relationship between the pages. Tools such as pen were used to create identified hyperlinks between screens by linking a phrase or shape from one screen to another. The page level was mainly used while sketching the screen contents or when further modifications to a screen were required. The other two major levels at the extreme ends of the zoom bar were rarely used.

Supporting pen-based interaction in DENIM made it much easier to modify and alter the layout of the interfaces in comparison to InDesign. In addition to the pen-based interaction, a text-field simulating an input box and typed-text were used when designing the content of the pages in DENIM.

One of the advantages of the paper prototype was allowing interaction with the layout of screens by moving around sketched interface elements (see Figure 3-1), which was not supported in DENIM environment. Therefore, it was required to duplicate the

Amir M Naghsh 34

whole sketch in order to simulate the interaction of some part of a sketched layout (see Figure 3-3). For instance, to illustrate the procedure of adding a student name to a module, the same page was duplicated with a minor change from one page to another page.

Figure 3-3 Simulating interaction

Duplicating an interface screen was not a straightforward task and many states of the same screen increased the design’s size and consequently the time and effort needed for further modifications. In particular, multiple duplicates of the same page with similar names caused confusion that required zooming further into the page using the storyboard level to locate the correct page by viewing its content.

Another issue with DENIM was the difficulty with the positioning of the sketches when changing the zoom level. For example, when the zoom level was changed to zoom in, the pages were positioned so far apart that it became difficult to find other pages and it was necessary to pan or zoom back out. The other way round, when changing the zoom level to zoom out, the pages would become too close to each other, covering one another, which required the designer to select and move some of the pages around to be able to have them all visible.

Amir M Naghsh 35

Being able to move the sketches around at the storyboard zoom level, made it possible to arrange a number of pages of interest, to group them based on their content, or to review and compare them together. However, unlike the paper-prototype the pages at the storyboard zoom level did not provide detailed content of the sketches and required further zooming in to view such detailed contents. The zoom levels such as page view were more useful in providing an appropriate level of details, while they were restricted with a maximum number of one or two pages that they could display at one time. On the other hand, the site map zoom level had the benefit of displaying many pages next to each other, but the content of the sketches was not provided at this level.

Selecting the sketched pages and then moving them around the DENIM sheet, proved to not be as easy a task as it was in paper-prototypes. It was possible to select an individual page at any of the zoom levels, however the facility to select a group of pages was only provided at the sitemap zoom level, where the content of the sketches was not displayed. This has the restriction of making a selection based on the understanding of the page label or making the extra effort to zoom in and zoom out to learn about the page content.

DENIM’s simple browser with reduced functionality was used to execute the prototype and interact with it at any time during the design process. DENIM supported more inputs in the form of left and right mouse clicks. It also allowed a potential user to type into the typed-text object in the run mode. These simulated a closer look and feel to the dynamic behaviour of the final system in comparison to InDesign.

3.5. FREEFORM

To create a prototype in FreeForm, sketched screens of paper prototype were also used as a starting point to re-sketch them in FreeForm by using a graphic tablet. FreeForm provided two representations of the sketched pages: page view and storyboard view. The layout and details of each of the screens were sketched at the page view (see Figure 3-4). To identify navigational links between screens the storyboard view was used, which displayed a number of pages at a time (see Figure 3-5).

Unlike DENIM and paper-prototype, the arrangement of screens at the storyboard level was restricted and it was very difficult to move the pages around to find a number of pages of interest in order to group them together. The sketched pages in storyboard view were displayed in the order of their creation in the form of a grid with three columns and many rows. Therefore, at all times only a limited number of screens

Amir M Naghsh 36

depending on the size of the monitor could be displayed. The scroll bar had to be used to access other sketches.

Two empty slots to contain sketched pages were provided at the storyboard view in FreeForm, one at the beginning and other one at the end of the list of displayed pages. The two empty slots were provided as temporary locations to be used when rearranging the order of displayed pages at the storyboard level. To arrange a number of pages together in FreeForm, the designer could only move one page at the time, making an empty slot for another page to replace it. It became a hideous task to rearrange pages when the prototype grew in its size.

Another difficulty with FreeForm is the way that sketched screens are named. Each newly created page is named “sketch” with a number indicating the order of which that page was created. Since it was not possible to change the page names to indicate a more sensible meaning, it became difficult to keep track of what a page (for example called “sketch 5”) was referring to. This became more problematic as the number of screens in the prototypes grew.

Figure 3-4 Page view in FreeForm

Amir M Naghsh 37

Figure 3-5 Storyboard view in FreeForm

FreeForm supports a specialised mode to execute the prototype, in which navigation links were indicated by specially coloured target areas to allow users to interact with a executed version of the prototype. Similar to DENIM and InDesign the run mode in FreeForm was provided to support a closer look and feel to the dynamic behaviour of the finished system. In addition, when the prototype was executed it was possible to add free ink marks onto an overlay of each page to simulate the user filling in the boxes. However, the marks were held on a single overlay, and a button was provided to clear the overlay or it was cleared once FreeForm exited the execution mode without allowing the content of the overlay to be recorded as part of the design.

Unlike DENIM and InDesign, FreeForm provides a function to map and transform the scribbles in sketched pages to Visual Basic components. The transformation of sketches to components seemed to be taking more time than creating them from scratch in Visual Basic. It was time consuming particularly because the content of each individual sketched page had to be mapped before being transformed to a Visual Basic form. Once the page’s content is mapped, a label is added next to each of the scribbles that have been mapped indicating what that scribble was mapped to. By clicking on each label, a drop-down menu is displayed giving the option to choose other possibilities in case FreeForm has guessed the wrong component. Therefore, the user has to go through each individual page, to verify the content that has been mapped piece by piece. It is possible to skip this verification step and modify the forms in Visual Basic; however it will be a more difficult task since access to the sketched pages is not provided any more at that point.

Amir M Naghsh 38

In addition, FreeForm did not transform identified links between sketched pages. Once the transformation was complete, the designer was left with a number of Visual Basic’s forms that were required to be reviewed again to identify their relation to each other and how they have to be linked.

3.6. DISCUSSION

The type and properties of prototyping tools and methods can have a significant effect on the design process and the effectiveness of involved activities (Ehn & Kyng, 1991). Therefore, it is essential to understand the properties of prototyping tools.

Although all the discussed electronic paper prototyping tools can be viewed as supporting the medium fidelity prototyping, the pragmatic investigation showed these tools have many differences as well as similarities.

Unlike FreeForm and DENIM, InDesign does not support pen based technology. FreeForm allows user input when it is in the run mode which is not provided by other two tools. FreeForm allows the sketched pages to be transformed into Visual Basic forms while DENIM and InDesign do not support this recognition. On other hand, it is much easier to move the sketched pages around to arrange related pages together in DENIM and InDesign.

The question of how to categorise the similarities and differences among these tools which belong to medium fidelity prototyping arises. The first question that arises is how to use the concept of fidelity to categorise these tools? Does FreeForm have a higher fidelity in contrast to other two tools because it is allowing user inputs? Or does DENIM have a higher fidelity because it allows the use of text-fields which are very similar to what would be used in the finished system?

The concept of fidelity is mainly focused on the degree to which a prototype presents application and the interaction of a system from the user’s point of view. Therefore, even if it was possible to make a distinction in terms of the fidelity that different electronic paper prototyping are offering, a second question arises - is the traditional concept of fidelity sufficiently fine-grained to highlight the similarities and differences between the electronic paper prototyping tools?

If FreeForm could be considered to have a higher level of fidelity in comparison to DENIM, since it can produce Visual Basic forms, can the concept of fidelity highlight that FreeForm requires more effort in comparison to DENIM to allow its user to link

Amir M Naghsh 39

sketches in the storyboard view? Or how important is it to provide the transformation of sketched pages into Visual Basic forms, when it can cause a high level of errors in the recognition process and requires the same amount of time as sketching them to verify them? Can this concept explain that it is much more difficult to select and move the pages in FreeForm in comparison to DENIM and InDesign? Or for example, how could the concept of fidelity explain that it is more difficult to modify the sketches in InDesign in comparison to other two tools?

While these electronic paper prototyping tools offer similar but not the same level of fidelity, it is still not possible to compare such important similarities and differences that they offer by using the traditional concept of fidelity. Hence, it is crucial to apply other frameworks to explore the characteristics of different electronic paper prototyping tools. The next chapter applies cognitive dimension framework as an alternative to the concept of fidelity.

Amir M Naghsh 40

Chapter 4. COMPARING PROTOTYPING TECHNIQUES AND TOOLS

4.1. INTRODUCTION

As explained in Chapter 3, the concept of fidelity is not sufficiently fine-grained to compare the similarities and differences among different electronic paper-prototyping tools. This chapter applies the cognitive dimension framework that have previously been used to evaluate software systems, programming languages, and software design notations (Roast, 1998; Roast et al., 2000; Clark, 2001; Kutar et al., 2002; Khazaei and Triffitt, 2002) as an alternative way to explore the characteristics of different electronic paper-prototyping approaches.

Green & Blackwell (1998) offer ‘Cognitive Dimensions’ as a useful conceptual framework for assessing notational systems. This approach considers a range of different vocabulary that can be used in describing the usability of a notational system.

Cognitive dimensions are a tool to aid non-HCI specialists in evaluating usability of information-based artefacts (summative evaluation). Since they are addressed to non-specialists, they consciously aim for broad-brush treatment rather than lengthy, detailed analysis. Their checklist approach helps to ensure that serious problems are not overlooked (Green and Blackwell, 1998, P. 5).

To apply the framework, first the information structure of artefacts must be built. The Cognitive Dimensions uses the concepts of notation, environment and medium to structure the information artefacts and define a notational system.

The notation comprises the perceived marks or symbols that are combined to build an information structure. The environment contains the operations or tools for manipulating those marks. The notation is imposed upon a medium, which may be persistent, like paper, or evanescent, like sound (Green and Blackwell, 1998, P. 8).

Amir M Naghsh 41

Tools and methods used in the development of prototypes for interactive systems can be viewed as ‘notational systems’, where tools and their facilities can be viewed as the environment, material and marks that are used in generating the prototype can be viewed as the notation. Finally, the means such as paper, graphic tablet, PC and whiteboard that are used to interact with the prototype can be considered as the medium.

The next section gives a brief review of the full set of dimensions supported with examples and followed by an analytical discussion to describe their relevance for analysing electronic paper-prototyping environments.

4.2. RELATING THE COGNITIVE DIMENSIONS TO PROTOTYPING

4.2.1. VISCOSITY

Resistance to change: the cost of making small changes.

Repetition viscosity: a single goal-related operation on the information structure (one change 'in the head') requires an undue number of individual actions.

Knock-on viscosity: one change 'in the head' entails further actions to restore consistency (Green and Blackwell, 1998, p. 12)

Viscosity means how much work is required to affect a small change. A good example for repetition viscosity is the time and cost required to change the area code of a city in a paper phone book. Updating the area code is a single change that requires an undue number of actions. All the numbers in each page of a phone book have to be checked and changed manually if they have the specific area code.

A timetable with a high interdependency is the example used by Green and Blackwell (1998) for knock-on viscosity. Changing a lecture location from classroom A to classroom B requires the lecture in classroom B to move to classroom C and lecture in classroom C needs to be moved and so on. A single change to the timetable causes reorganisation of everything else.

Viscosity has interactions with other dimensions, in particular abstraction, secondary notation, hidden dependency and premature commitment (Green and Blackwell, 1998).

Amir M Naghsh 42

A simple example of how viscosity can be applied to the prototyping environment is having to make a general change manually when the prototyping environment has no tools to make global updates. An example could be when the user wants to add a new option to the menu bar. In a paper-prototype, the designer has to modify the menu on each individual sketched interface while in a software prototype the designer only modifies the template for menu bar and it will update all the screens which contains the related menu bar.

4.2.2. ABSTRACTION

An abstraction is a class of entities, or a grouping of elements to be treated as one entity, either to lower the viscosity or to make the notation more like the user’s conceptual structure (Green and Blackwell, 1998, p. 24).

Abstraction changes the underlying notation by reducing the viscosity of a notation. An example is the use of styles in the word processor. For example, changing the font size for “heading 1” style will change all the titles that use this heading style. A user can add a new style as well as learning an existing one and applying it to the whole document.

The Abstractions barrier is determined by minimum number of abstractions which must be mastered by users before using the system, as well as their readiness, or desire to accept new abstractions. Abstraction hungry systems can only be used when a large number of abstractions are defined and set up prior to any work. Abstraction tolerant systems allow users to define abstractions but do not require user-defined abstraction to make the system work. Abstraction hating systems do not permit user-defined abstractions.

Green and Blackwell (1998) explain abstractions as investments for the future. They explain by changing the notation now it will be easier to use it in future, or makes what’s built in the notation more re-usable or easier to modify since abstractions can reduce the viscosity. Green and Petre (1996) suggest abstractions as the classic solution to viscosity problems. Introducing more abstractions means more components can be treated as a group thus reducing the repetition viscosity.

However, Blackwell et al (2001), Green and Petre (1996) along with Green and Blackwell (1998) explain disadvantages of abstractions as being costly and requiring a long time to set up. In addition, systems that enforce abstraction are potentially difficult to learn and have a higher level of hard mental operation. Difficulty in learning how abstractions are set up and are used could permanently exclude some of

Amir M Naghsh 43

the stakeholders. Therefore, systems with an abstraction hunger can be inappropriate for exploratory tasks since only skilled designers can use them.

4.2.3. CLOSENESS OF MAPPING

Closeness of representation to domain (Green and Blackwell, 1998, p. 39)

Blackwell et al (2001) explain this dimension as how closely the notation is related to the result it is representing.

Green and Blackwell (1998) provide two ‘thumbnail illustrations’ to explain in this dimension. A close mapping: the visual programming language LabVIEW, designed for use by electronics engineers, is closely modelled on an actual circuit diagram, minimising the number of concepts that need be learnt. A distant mapping: in early versions of Microsoft Word, to count the characters in a file, it had to be saved to disc – whereupon it would calculate the size of the file and number of its characters.

Closeness of mapping determines the visual and behavioural similarities between the symbols in a notation and the result it is describing. This work interprets this dimension as the mapping of visual and behaviour similarities between the prototype and the finally delivered software.

It is important that a relatively close mapping is available to some degree between the interfaces in a prototype and the final interfaces in the finished system. The closeness of mapping in behaviour respect is also suitable which can help users in interacting with the prototype and have a better understanding of how the final system will perform.

Although closeness of mapping is encouraged, too mush of closeness of mapping between the prototype’s visual and the final system is not recommended by researchers (Hong et al., 2000; Nielson, 1993) in the early stages to avoid users from focussing attentions at low-level details of design such as fonts colour or the background picture in a website.

4.2.4. CONSISTENCY

Similar semantics are expressed in similar syntactic forms [within the notation] (Green and Blackwell, 1998, p. 39).

Green (1983) argued that a consistent programming language was easier to learn. Blackwell et al. (2001) indicated that users could learn the structure of an information

Amir M Naghsh 44

artefact through recognising and learning patterns in notation and preventing such patterns from being visible to users could compromise the usability.

A good example for consistency is mobile phones’ menus. In a consistent version, the same keys are used to move up and down between different levels of menus. For instance, in mobile phones the top right key on the keypad is always used to go one level up in the menus.

To relate consistency to the prototyping environment, the similarities between the marks that generate the prototype (notation) and the similarities between the facilities that are supported by the prototyping tools (medium) can be considered.

4.2.5. DIFFUSENESS

Verbosity of language (Green and Blackwell, 1998, p. 39).

Some notations can use long-winded description in the form of many symbols, series of commands or a lot of space to achieve the results that can be achieved more compactly using other notations. Notations with high diffuseness can limit users who are carrying out tasks which require working memory, such as exploratory design (Green and Blackwell, 1998). Such notations force users to use their working memory for remembering the long-winded descriptions. For example, the abstraction hungry systems can have a high diffuseness level by forcing users setting up many levels of abstraction when using the notation.

In the prototyping environment, the number of actions and marks that are required when generating part of the prototype can be considered as the diffuseness.

4.2.6. ERROR-PRONENESS

Notation invites mistakes (Green and Blackwell, 1998, p.40).

Green and Petre (1996) have used the conventional way of categorising errors in two distinguished category as ‘mistakes’ and ‘slips’. They explain a slip as doing something that the user ‘didn’t mean’ to do. It happens when user knew what to do but - due to the notation’s poor ability to discriminate, inadequate syntax-checking, bad dialogue design or memory overload - did it wrong. While they explain a mistake as an error that could happen in the design of parts which are deeply difficult.

The determining factor for error-proneness dimension for the prototyping approaches is how much the notational system is forcing the stakeholders in making errors. For instance, it is very unlikely in paper-prototyping that when a designer has a clear idea

Amir M Naghsh 45

of the layout of an interface, the designer makes errors due to the difficulties in creating marks to design the interface and sketches something that was not intended.

4.2.7. HARD MENTAL OPERATIONS

High demand on cognitive resources (Green and Blackwell, 1998, p. 40).

A notation can make things difficult to be worked out in the head. Hard mental operations is how much effort is required to understand a notation. The hard mental operation of a notation can be considered in terms of the effort that is required in performing three activities of writing, reading and executing the notation.

A simple example is the level of thinking required by the user to read, write and execute a prototype created in a formal environment such as Z in comparison to a paper-prototype. The prototype created in Z has a higher hard mental operation since the user needs to make more effort and thinking to understand it in comparison to the paper-prototype.

4.2.8. HIDDEN DEPENDENCIES

A hidden dependency is a relationship between two components such that one of them is dependent on the other, but that the dependency is not fully visible. In particular, the one-way pointer, where A points to B but B does not contain a back-pointer to A (Green and Blackwell, 1998, p. 17).

A good example of hidden dependencies is hyperlink between HTML pages. The HTML link is a one-way link and it is not visible at the end point which is the page that is being linked to. Hidden dependencies slow down the process of finding information. In the example of HTML pages, since the hyper-links are hidden from the other end, to determine what pages are pointing to a specific page is costly and requires long extensive search processes. Therefore, it is important to reduce the hidden dependency or tolerate it to reduce the effort required for information findings.

When creating prototypes many parts of the design are dependent on one another. For example, there are hidden dependencies between the pages that share similar layout in paper-prototype. The designer has to keep record of pages that are similar, so that when the layout of one page has changed, the same adjustment is incorporated into the other similar pages.

Amir M Naghsh 46

4.2.9. PREMATURE COMMITMENT

Constraints on the order of doing things force the user to make a decision before the proper information is available (Green and Blackwell, 1998, p. 21).

Green and Petre (1996) explain that premature commitment could happen when (a) the notation contains many internal constraints or dependencies, (b) there are order constraints in terms of what can be created first and forces the user to make a decision before all the information is available, and (c) enforcing the user to look ahead in an inappropriate order.

A simple example is route planning and not being aware of road traffic. For example, a driver would not know anything of road works and possible delays until he/she reaches that specific location on his/her way.

Premature commitment in the prototyping environment can be considered as the degree of commitment that users and designers are forced to make to initial choices that have been decided early in design while not all the information was available. For example, it would be a premature commitment to decide on number of screens for illustrating an interaction in a paper-prototype prior to having a clear understanding of the interaction and the functionalities that it includes.

4.2.10. PROGRESSIVE EVALUATION

Work-to-date can be checked at any time (Green and Blackwell, 1998, p. 40).

Blackwell et al (2001) explain that the notational system can facilitate evaluation by letting users to check on work-to-date frequently. Green and Blackwell (1998) indicate the importance of such frequent evaluation even for experienced users in difficult times, such as working under pressure or constant interruptions.

For example, spreadsheets re-compute repeatedly at each opportunity, therefore users can develop a solution gradually while checking it at the same time.

It is essential for designers to be able to run the prototype with users at every given opportunity and obtain users’ feedback to evaluate the design. For instance, it is possible to review the paper-prototype with users at any time during the design process.

4.2.11. ROLE-EXPRESSIVENESS

The purpose of a component (or an action or a symbol) is readily inferred (Green and Blackwell, 1998).

Amir M Naghsh 47

Role-expressiveness means how easy it is to explain ‘what is this bit for?’ when using a notation. In other words, it means how easy it is to understand the use for a part of the notation or how it relates to other parts when using the notation. For example, a GUI builder which offers role-expressiveness should allow the programmer to understand what each component does when looking at the icons in the toolbox.

In the prototype environment, role-expressiveness can be considered as the level of understanding that the user or designer could have in using the marks that create the prototype or in using the facilities provided by the tool to perform tasks.

4.2.12. SECONDARY NOTATION

Extra information carried by other means than the official syntax.

Redundant recoding gives a separate and easier channel for information that is already present in the official syntax. Escape from formalism allows extra information to be added, not present in the official syntax. (Green and Blackwell, 1998, p. 29).

Systems supporting secondary notation let users to record things that have not been considered by the designer within the notation. The user can use the secondary notation freely as informal additions that do not have any official meaning according to the formal notation. Such ability to add arbitrary additional material is an essential aspect of human communication that can be used as a channel between user and designer to express extra information.

A simple example of redundant recoding is when designing a sitemap of a website; usually the pages that belong to same section are grouped together. Another example for secondary notation is the use of coloured post-it notes for categorising a textbook, or comments in programme codes for future use.

Closely related to secondary notation is the possibility of an escape from the formalism altogether. No programming language captures everything that a programmer has to say about a program. If programs are to be used as a means of communication, some other mechanism must be provided with which to escape from the bounds of formalism (Green and Petre, 1996, p. 32).

Another example is offering secondary notation to reduce the understanding time of a notation. Crosby et al. (2002) suggested that providing comments (escape from formalism) and the use of a good naming style (redundant recoding) could reduce the

Amir M Naghsh 48

understanding time. Further a Blinman and Cockburn (2005) study showed that in software development systems which were supported by a good naming style and interfaces, documentations were understood better in comparison to the systems without such supports.

In the prototyping environment, secondary notation can be considered as the forms of additional information that are added to the prototype that were not included in the first place. An example is the comments that users and designers scribble on the sketched pages in a paper-prototype while reviewing the prototype.

4.2.13. PROVISIONALITY

Degree of commitment to actions or marks.

Pencils are used by architects, typographers and other designers to make faint blurry marks, meaning ‘something more or less like this goes more or less here’, as well as precise hard marks. (Green and Blackwell, 1998, p. 41)

Provisionality can reduce the premature commitment. Notations that support undefined or vague marks are said to offer high provisionality by making provisional actions and offering multiple design options before any commitment to the design. A high degree of provisionality is appropriate when the prototype is not intended as a representation of any finished system and it would be used to simulate and explore new ideas early in the design.

4.2.14. VISIBILITY AND JUXTAPOSABILITY

Juxtaposability: ability to place any two components side by side.

Visibility: ability to view components easily (Green and Blackwell, 1998, p. 34).

For example paper-prototype has high level of Juxtaposability since it is possible to select any number of sketched pages and rearrange them to view any two pages at the same time. However, a paper-prototype can have a low visibility since seeking a page of interest could require the user or designer to search through a large pile of sketched pages.

Another example of Juxtaposability is the split option provided by Microsoft Word that allows the user view two different part of same document at the same time. The word documents that use the table of contents have a higher level of visibility since it is easier to find a page of interest.

Amir M Naghsh 49

4.3. A REVIEW OF TOOLS USING COGNITIVE DIMENSIONS FRAMEWORK

Having developed a framework for analysis, it is now possible to examine a range of established prototyping techniques and tools including paper-prototyping and different electronic paper-prototyping tools reviewed in chapter 2. Initial analysis was reported at PPIG 2003 (Dearden et al, 2003). Below is a more complete analysis of using Cognitive Dimension framework to review prototyping notation by exploring eleven dimensions: viscosity, abstraction, closeness of mapping, hard mental operations, hidden dependencies, premature commitment, progressive evaluation, provisionality, role-expressiveness, secondary notation, visibility and juxtaposability.

4.3.1. PAPER PROTOTYPING

In terms of the cognitive dimensions discussed above, paper-prototyping relies on providing a low viscosity representation to encourage more user involvement at the early stages of design. The low viscosity of paper-prototyping is provided by the use of simple tools such as pencil and paper which users can adapt easily and modify the design to express their desires. In particular, paper-prototyping is offering a low knock-on viscosity by enabling users to easily modify any part of the design at one time. Maintaining low viscosity lower the costs associated with repeated iterations in the early stages of the development process. Prototyping assisting such a low viscosity (such as paper-prototyping) are more suitable in encouraging users to participate with supplying feedback in the early stages. However, as Lin et al. (2000) points out one of the drawbacks of the paper-prototype is the time and effort required to make changes to a design that need to be reflected across multiple sketches. In other words, a paper-prototype may suffer from a high repetition viscosity.

Paper-prototyping does not enable the user to modify the notation to define new facilities or terms within the notation. In other words paper-prototyping is an abstraction hating notation. Having a low abstraction prevents paper-prototyping to overcome its high repetition viscosity problems by allowing many components to be treated as one entity for the ease of later modifications and incrementation. However, as Green and Blackwell (1998) explain thinking in abstraction is hard and systems supporting them are difficult to learn. Therefore, the low abstraction of paper-prototyping makes it easier for users to become involved in the design process by not requiring them to learn any skills in advance or spend any time in setting up the notation.

Amir M Naghsh 50

It is also important that a relatively close mapping is available between the domain (the appearance of a graphical user-interface) and the notation (paper sketches) so that the users can easily learn to read the notation. On the other hand, one stated benefit of paper-prototypes is that because the detail of colours, precise positioning, fonts etc. is not available from a sketch, this avoids users focussing attention on such low level details of design. This might be interpreted as an argument against too close a mapping. Some authors have highlighted limitations in paper-prototyping in terms of its ability to faithfully present certain aspects of interaction. Rettig (1994) highlights the problem of representing aspects such as system response times. O’Neill et al. (1999) discuss the lack of an explicit representation of the navigational structure, which may result in users suggesting modifications to the layout of individual screens, but being unable to modify or critique overall dialogue structure. Hence, paper-prototyping offers a closeness of mapping only in respect to certain aspects of the system being developed.

Since a member of the design team is required to ‘play the computer’ to enable users to interact with the paper-prototype, it would be difficult for users to keep track of actions that could result in displaying a sketched screen. In addition, the potential end users cannot interact with the paper-prototype without the help of someone who knows the system inside out to arrange a correct order of sketches to set up a scenario. This hidden dependency between sketches restricts the end-user to interact only with the layout of one sketch at a time rather than the navigational structure of the whole system.

Further it can be argued that paper-prototyping to some extent has a hard mental operation. The lack of dynamic representation closely mapped to the finished system could force the user and designer to work out the dynamic behaviour of the finished system in their heads. The design needs to consider all the different possible orders for sketches to allow the user to interact with the prototype. The user has to try to work out the overall navigational structure of the system from what the designer presented.

The use of squiggles or other vague lines in paper-prototyping can give a paper-prototype a high degree of provisionality. It is easier to create marks whose meaning is undefined or deliberately ambiguous. Such provisionality can act as a positive resource for the paper-prototypes to be explored during the early design stage with the intention of stimulating new ideas. In this case, there is often a clear view that the prototype is not intended as a representation of any finished system.

Amir M Naghsh 51

Paper prototyping offers secondary notations in the form of scribbles on the paper, or added ‘post-it’ notes. In paper-prototyping, post-it notes and similar secondary notational devices can be used to indicate reasons for particular design choices, critiques of particular elements, or indications that further work is required or planned.

Juxtaposability may be very important both during exploration – to consider alternative designs for the same thing, and during evaluation – to allow for evaluations of consistency in design. Paper prototyping allows for juxtaposability of different parts of the prototype in a way that may be difficult to achieve with some software prototyping systems. In addition, the difficulty in finding a page of interest in pile of sketched pages makes the paper-prototype to offer a low level of visibility.

Using pencil and paper to draw sketches is an everyday familiar activity which makes Paper-prototype to have role-expressiveness and consistency. The process of sketching on paper is an easy task which does not require the user to perform many actions which makes the paper-prototype have a low diffuseness.

As explained earlier it is unlikely that use of pencil and paper force the designer to make errors by sketching an interface which was not intended, in particular when the designer had a clear idea of the interface prior to sketching it. However, being unable to execute the paper-prototype could make it difficult to design solutions for the dynamic behaviour of the final system. Also the difficulty in finding a page of interest in a paper-prototype could force the designer to make errors when simulating the dynamic behaviour of the prototype. Therefore the paper-prototype has some degrees of error-proneness.

Green and Petre (1996) express the importance of the possibility of progressive evaluation as essential for end users and other stakeholders who are not experts. As explained in chapter 2 paper-prototyping supports iterative design and allows users and designers to interact with the prototype throughout the design process. The possibility of evaluating the paper-prototype iteratively enhances the paper-prototyping with a high progressive evaluation.

The low viscosity and high provisionality of paper-prototyping makes it simple and cheap to use at early stages of design. This encourages more user involvement by enabling users to make choices without the worry of committing to these choices. This facilitates the paper-prototyping with a lower degree of premature commitment within the wider design process.

Amir M Naghsh 52

4.3.2. DENIM

Reviewing DENIM using the framework above, it is clear that for changing the content or layout of individual pages, DENIM will have a similar level of viscosity in comparison to the paper-prototyping. Further development of DENIM (Lin et al., 2002) allows abstraction to reduce the repetition viscosity. The later version of DENIM includes component to support recurring page elements.

Components provide a mechanism for the designer to create and use reusable widgets and fragments of interface designs. (Lin et a.l, 2002, p 4)

The later version of DENIM relies on providing abstraction by enabling users to modify the notation to define new components. Such components could be used in the design of menu or navigation bars. Providing abstraction in DENIM makes it easier for later modification to the use of pencil and paper. However, this later version of DENIM is abstraction tolerant and does not force the users to learn or define anything before being able to use the tool.

While offering abstraction reduces the repetition viscosity, it has the trade-off of increasing the hard mental operation and the hidden dependency. An unfortunate consequence of using components is that it becomes difficult to identify where in the model these components have been used and, if a component requires to be changed, which part of the model is going to be effected.

In terms of the visual appearance of the DENIM page sketches, they offer a higher level of closeness of mapping than a paper-prototype by supporting some aspects of evolution of the prototype to make it look more like the domain (the eventual web-pages). For example, the stamp tool can be used to replace a scribble with a typed text-field. Also initial suggestions for relations between pages declared in the ‘site map’ level, can be refined to identify particular scribbles as the ‘hotspots’ that are the source of page links. Links can be refined by adding specified time delays (for example to indicate that some processing will be taking place) or by mapping them to specific mouse actions (e.g. requiring a double-click to activate the link). A further development of DENIM called Quill (Hong et al., 2002) allows for additional interaction devices to be added to a design such as drop-down menus, thus it allows a higher closeness of mapping for the visual appearance.

As explained earlier, paper-prototyping has limitation in terms of its ability to present certain aspects of interaction. Being able to execute DENIM in a limited functionality browser offers a major advantage in comparison to paper-prototyping, since it is no

Amir M Naghsh 53

longer necessary for the designer to ‘play the computer’ during the evaluation sessions, which allows the designer to focus more on the users’ responses. The execution of the model also provides a closer mapping to the interaction that users may experience in the completed system, than is possible using pencil and paper. Similarly, adding in time delays and specifying particular mouse actions to execute a link can also lead to a closer mapping.

DENIM supports iterative design and provides the same level of progressive evaluation as the paper-prototype. Being able to execute the prototype created by DENIM at every opportunity allows the user and designer to evaluate the prototype frequently while building it bit by bit. This process allows the stakeholders who are not experts to become involved in the evaluation process.

DENIM permits the kinds of provisional marks that can be made with paper and pencil. Hence, DENIM offering a high degree of provisionality may represent a design direction that is well suited to the initial exploration of possible designs.

DENIM offers a limited support for communication of findings. DENIM does not provide any form of secondary notation either within the design view or during execution of the model. The only marks that can be made are scribbles, which cannot be distinguished from the intended content of the design itself. In particular, this approach does not allow users and other stakeholders to add their own comments to the design. Landay & colleagues have developed 'The Designers Outpost' (Klemmer et al., 2001) which uses a tangible user-interface based on a whiteboard, employing post-it notes as the primitive interaction objects. Using the Designers Outpost annotations can be associated with the display of a DENIM model, but the currently published version of DENIM does not allow such annotations to be imported.

Use of gestures as commands gives DENIM a lower level of role-expressiveness than paper-prototyping. For example, when a wrong form of stroke is used to indicate a gesture, DENIM does not recognise the stroke and cannot match it with any of its gestures, however DENIM response does not help the user to understand what would be the correct stroke that is associated to the interested gesture. In addition, the procedure of creating a page in DENIM does not offer role-expressiveness. The designer has to scribble a page name on to the DENIM sheet at the storyboard level to create a page. The created page always has a default size. However, at the page level DENIM provides a more appropriate way to create a page by allowing the designer to

Amir M Naghsh 54

draw a rectangle that represents the size of a page and then associate a name to the created page.

DENIM also has a lower level of consistency than the paper-prototype. The designer is required to perform different actions in order to use same method at different zoom levels in DENIM. An example would be the way that pages are created in DENIM as it is explained in an earlier paragraph. Another example is the ability to select a group of pages in DENIM that is only provided in some of zoom levels (storyboard and above).

It becomes difficult to find an individual page when the design models is large and has too many sketched pages. To find a page of interest the designer has to repeatedly zoom in, zoom out, and pan the design sheet to view pages that are placed far from each other. Thus, it increases the level of DENIM diffuseness. Difficulty in finding a page also decreases the level of visibility in DENIM.

DENIM offers similar level of error-proneness and premature commitment to the use of pencil and paper. However, it can be argued that. DENIM allows the user to move and juxtapose individual pages and maintain the level of juxtaposability offered by paper-prototyping. This can be useful when attempting to evaluate consistency across a large design.

4.3.3. INDESIGN

InDesign relies on the low viscosity of paper-prototyping. However, once the image has been scanned into InDesign, the tool does not provide any mechanism for editing which makes it suffer from a high level of viscosity and premature commitment. Once the sketches are scanned, the designer is committed to them and for any amendments to the sketches, the designer has to go back to the paper version to make those changes and re-scan them. Designers may also try to apply standard graphical editor to modify individual pages or states to overcome this problem. This process also makes it difficult to execute the prototype frequently after each small change since the changed sketches have been scanned first. Thus InDesign has a lower degree of progressive evaluation in comparison to DENIM and paper-prototype. InDesign supports a similar model for evaluation to that provided by DENIM, although the evaluation device does not match that of a browser as closely – for example, no back button is provided. The execution of the scanned images means that the dynamic behaviour of the prototype created in InDesign can have a closer mapping than paper-prototype to the eventual system.

Amir M Naghsh 55

The ability to add pencil marks to images before they are scanned allows for a similar degree of provisionality to that provided by paper-prototyping.

InDesign, does not support abstraction. InDesign offers same level of role-expressiveness, visibility and consistency compared to the use pencil and paper since it is a direct use of scanned paper sketches. The images that are scanned in may also be printed, or imported into a graphics editor to allow for juxtaposability during evaluation. However, the InDesign environment itself does not provide methods to juxtapose screens directly.

The use of navigational links at the storyboard level in InDesign can make it easier to identify relation among pages thus reduces its hidden dependency to some extent compared to a paper-prototype. However, since the hyperlinks are not glued to the scanned content of pages, any alteration of the content requires further actions of adjusting the position of hyperlinks according to the updated scanned content. Therefore, it increases the hidden dependency much more in comparison to the paper-prototype. In addition, to update the hyperlinks and relate them to the updated content a higher level of thinking is required by the designer, which increases the hard mental operation.

InDesign also has a high level of diffuseness. To associate hyperlinks to a page, firstly, the designer has to identify the relationships between the pages at the storyboard level, and then the links can be created between them. However, since the created hyperlinks are not attached to actual content, the designer has to zoom in to each individual page to adjust the position of the hyperlink trigger on top of the relevant content within the page.

The difficult procedure of associating a hyperlink to the content of a page also increases the error-proneness. Particularly, since InDesign does not provide meaningful names that would represent the content of a page, there is a high possibility that the designer may place the triggers - which are identified with name of the pages that they link to - on top of an incorrect content within the page especially when there are too many of them within a page.

The InDesign environment does not support any form of secondary notation and it only relies on the annotation and comments made on the sketches before they were scanned. InDesign could be viewed as an environment for executing a paper-prototype by scanning each individual sketch. However, scanning sketches make design iterations longer and more difficult in comparison to the paper-prototyping.

Amir M Naghsh 56

4.3.4. FREEFORM

As with DENIM, FreeForm exploits the low viscosity of pen-based interaction to support design exploration. It allows users to select a set of marks and copy, move or paste them to another part of the design, which can be considered as a low level of abstraction.

Forms can be associated with navigation links which can be positioned within the storyboard view. Evaluation is then possible in FreeForm in a specialised mode, in which navigation links are indicated by specially coloured target areas. This provides progressive evaluation as well as a close mapping to the dynamic behaviour that users may experience in the completed system. In terms of the visual appearance of forms, FreeForm offers a higher level of closeness of mapping by using a recognition system to map the pen marks to specific user interface components. FreeForm emphasises the evolution of the design by recognising gestures that correspond to particular Visual Basic (VB) interaction components such as tables, drop-down menus, text fields etc. A button is provided to convert the finished design into a set of VB forms. Once the pen marks are transformed to the VB components, the user can change them by using the provided toolbox in VB. However, previously too close mapping has been discouraged to prevent users focusing attention on low-level details of design.

FreeForm permits provisional marks similar to ones made by pencil and paper. However, because FreeForm is using such a recognition system, a mark that was intended as provisional may later be interpreted by FreeForm and replaced with a (non-provisional) user interface widget. Thus, it can be argued that FreeForm offers a lower degree of provisionality to paper-prototyping or DENIM. For example, a provisional mark can be used for a text-field or text-box, but when it comes to recognition the user is required to make a choice and commit to one of them, while it is possible that the user still does not have sufficient knowledge to make such a decision.

Similar to InDesign, FreeForm does not support meaningful names associated to the pages in design. It also does not attach the hyperlink trigger to the page content. Thus, FreeForm also suffers from a high hard mental operation, hidden dependency and diffuseness.

Once the recognition process is finalised, the stakeholders are required to go through the whole design to validate every individual recognised pen mark. Mistakes made as the result of the recognition process means a higher level for error proneness” and

Amir M Naghsh 57

premature commitment. The user is required to think ahead to use marks which can be mapped to specific user interface components.

FreeForm also supports some level of juxtaposition of forms by dragging them within the storyboard view, however the very difficult procedure of moving pages in the storyboard also increases its diffuseness. FreeForm offers same level of visibility and consistency as InDesign does.

When FreeForm is in run mode, it is possible to add pen marks onto an overlay of each screen. Plimmer & Apperley (2003) describe the use of this facility as a way to simulate the user filling in text fields etc. Such an approach could be used to give users a secondary notation with which to support communication about the design. However, the marks are held on a single overlay, and a button is provided to clear the overlay. Therefore, the marks cannot be treated as separate 'comments'. In addition, when FreeForm returns to the 'design' mode, the marks are not visible.

Amir M Naghsh 58

4.4. DISCUSSION

Table 4-1 presents the results to the review of electronic paper-prototyping tools using the cognitive dimension framework. The results show many differences as well as similarities among the three compared to electronic paper-prototyping tools.

For instance, the FreeForm has the highest level of closeness of mapping compared to other two. This can make freeform a valuable tool for helping users to understand how the finished system may look like in early design.

The lower level of hidden dependency, hard mental operation, diffuseness and premature commitment in comparison to FreeForm and InDesign, makes DENIM an easier tool to used by designers as well as users. DENIM provides a low level of consistency and role-expressiveness in comparison to other two tools, which can make it difficult for the novice users (either designers or end-users) when beginning to use DENIM.

Paper-Prototype

DENIM InDesign FreeForm

Viscosity Low: knock-on

High: Repetition

Low: knock-on

High: Repetition

Medium: knock-on

High: Repetition

Low: knock-on

High: Repetition

Abstraction Hating Tolerance Hating Tolerance Closeness of Mapping High:

Layout

Low: Navigation

High High Higher

Consistency High Low Medium Medium Diffuseness Low Medium High High Error proneness Low Low High High Hard Mental Operation Low/Med Medium High High Hidden Dependencies Medium Medium High High Premature Commitment Medium Low High High Progressive Evaluation High High Low Medium Role-Expressiveness High Low Medium Medium Secondary notation Yes No No No Provisionality High High High Medium Juxtaposability Higher High Medium Low Visibility Medium Medium Low Low

Table 4-1 Summary of 14 Cognitive dimensions in relation to prototyping

Amir M Naghsh 59

4.5. CONCLUSION

The methods and tools developed to date are designed in a way that facilitates the works of one of the stakeholder groups. However, it may be argued that some of these methods have been designed to encourage users' participation (i.e. paper-prototyping) yet they do not fully support direct user’s participation in designing interactive systems for the members of different stakeholder groups to have hands on involvement in generating a prototype and developing the design.

To maximise user participation, it is important to maintain a low level of viscosity, abstraction, diffuseness and avoid hard mental operation to make the notation simple for different stakeholder groups. On other hand, offering a balanced degree of closeness of mapping to avoid users focusing attention to low level details. It is important to support provisionality and progressive evaluation to allow stakeholders to interact with the prototype in a highly iterative process.

As explained earlier, the ability to add arbitrary additional material is an essential aspect of human communication and it is important to provide secondary notation to enable users to understand the design and participate in authoring and evaluating the design by communicating and collaborating around the notation.

The analysis showed that electronic paper-prototyping tools (such as DENIM, FreeForm and InDesign) are similar in regards to many of the dimensions and can support different levels of participatory and exploratory design in the early design. However, none of the listed electronic paper-prototyping tools support secondary notation. These tools do not permit users or other stakeholders to add extra information to give feedback directly through the medium of the prototype. Instead, any comment or feedback must be held separately (for example in an audio recording or minutes of a meeting), resulting in a difficulty in identifying the items to which any comment refers. This problem will be particularly acute if some stakeholders are not co-located with the designers.

The importance of communication and interactions among stakeholders is emphasised in the introduction chapter as a key dimension in the design process, which would become a major difficulty and challenging when stakeholders are dispersed. Green and Petre (1996) refer to prototypes as means of communications and explained that secondary notation could be used as the communication channel for different stakeholders to express extra information. The escape from formalism of secondary

Amir M Naghsh 60

notation allows different stakeholder groups to add extra information to a prototype that can be shared with other stakeholder groups.

Wojahn et al. (1998) described making annotations as a common practice to communicate about collaboratively produced artefacts that can be easily shared. The digital annotation is defined as the production of additional information related to a document as a whole or to some parts of it (Bottoni et al, 2004). Consequently, annotations as the extra information that are carried by the artefact can be viewed as one form of secondary notation in electronic paper-prototypes.

Therefore, it is essential for electronic paper-prototyping to support possible forms of secondary notations such as annotation, in order to enhance communication among different stakeholders. The lack of the ability to annotate the design can severely limit the ability of such prototyping tools to support communication between different stakeholders at the early stages of design.

The next chapter reviews various studies investigating the use of annotation in textbooks to digital documents.

Amir M Naghsh 61

Chapter 5. ANNOTATION

5.1. INTRODUCTION

The traditional notion of annotation has recently evolved from paper document to digital documents such as multimedia, 3D models and websites. This Chapter gives an introduction to annotation and presents various studies investigating the use of annotation in text books and digital documents. Further the chapter reports on the forms and functions of annotations in textbooks and digital documents, presenting them into two categories of personal and public annotation. Finally, the chapter reports on studies that investigated characteristics for annotation tools to support asynchronous communication around digital documents.

5.2. ANNOTATION

“Add explanatory notes to, furnish with notes” (Pearsall and Trumble, 2002). Annotation refers to a note that describes, explains, or evaluates a particular document or as extra information associated with a particular point in a document. Annotation is a natural human activity that is used in day-to-day life. Brockmann et al. (2001) conducted a research on examining some general aspects of “humanities” and found annotation as one of the elements. Brockmann et al. refer to annotation as a support for reading and explain that:

“scholars produce extensive marginal notes, annotating photocopies or personal copies or attaching adhesive notes to a text” (Brockmann et al., 2001, p. 7).

In general pen annotation on paper is a common practice of everyday life. Making annotations is a natural activity that is performed in reading and reviewing processes. When reading text books, a reader often makes notes, underlines important parts, places asterisks, highlights or sketches upon the document. In a design environment, designers often use post-it notes and other removable materials to annotate an artefact.

Amir M Naghsh 62

Since annotation is a natural activity performed in reading and reviewing text-documents, the way people annotate on paper and mostly text-documents have influenced most of the studies that have been carried out on above items. In 1984 Nielsen suggested that, in future, literature would only exist in an online form, as part of an online journal or textbook. He concluded, therefore that computer systems for such online literature should support the possibility of annotation for each individual reader. Nielsen (1984) explained that there are several forms of annotations, and it might be difficult for computer systems to support all of them.

Annotation is often for personal use but in many contexts even personal annotation could be used to support collaboration even when it is not done with others in mind. Personal annotations can be shared, for example by circulating annotated paper documents at work. Marshall explained (1997) that when a marked-up document or used book is shared the sharing is serendipitous.

Cadiz et al. (2000) found that the way that an annotation system is used is affected when users are aware that their annotations will be shared and viewed by others and they were more concerned when making annotation. Cadiz et al. (2000) reported that users expressed that they did not make annotation if they felt that it was a minor or “nitpicky” comment they were making. Their results also showed that users were concerned while making a comment if it could be misunderstood by others or it could misrepresent them as a harsh or unkind person. Cadiz et al. (2000)'s findings indicated that users might not repeat the same annotation, if that comment was already made earlier by another user. However, Wojahn et al. (1998) reported in earlier research that users annotate the annotations. Their results indicated that users are more likely to annotate an existing annotation that has been made by another user or reviewer rather than annotating on an issue which has not already been annotated and discussed by an initial reviewer. Such results were also found in Marshall’s (1997) research on second-hand college text-books when students were highlighting comments made by the previous owner of the text-book.

O’Hara and Sellen (1997) found annotation on paper convenient in reading and reviewing activities, since paper provides a “malleable” surface and it is easy sharing through its “ready-mobility”. Marshall (1997) explained that pen annotations are more natural to people than annotation made by mouse and keyboard. She referred to annotation as an “unselfconscious activity” and found that many annotation tools make interruptions to the higher task that the user is carrying out. The development of tablet computers has motivated researchers in digital annotations to learn from the way in

Amir M Naghsh 63

which people annotate paper documents and apply that in the digital world. The tablet technology makes it possible for users to annotate on a digital document in a similar way to annotating on a printed one.

In addition to tablet technology Morris et al. (1999) explained that people use other digital technologies to overcome time and space constraints. They compared new technologies to old communication mediums. For example some of the advantages of email over letter writing were referred to such as speed, automatic filing, easy and quick duplication and distribution. And also disadvantages such as email spamming. The fast development in technology has made it possible to have asynchronous communication around digital documents through annotating them. A study conducted by Jung et al. (2002) also indicated that annotating digital documents could generate a valuable history of dialogues which could be used for future retrieval.

5.3. EXAMPLES OF ANNOTATION TOOLS IN USE

As explained in the previous section annotation can be made either for a personal or public purpose. Also annotation can be generated in synchronous or asynchronous ways. Recent years have been witness to a number of research projects on annotations focusing on various items in the digital world. Some of these items are:

• Text documents, such as digital books and articles: Quilt (Fish et al. 1988), Microsoft word and Prep editor (Neuwirth et al. 1990).

• Shared world-wide web :MADCOW (Bottoni et al. 2004), AnnoteImage (Brinkley et al. 2002), PAIS (Lober et al. 2000), Annotea (Kahan and Koivunen, 2001)

• Data visualisation and models in 3D: Redliner (Hanson and Wernert 1997), Space Pen (Harmon et al. 1996)

• 2D images: I2Cnet (Chronaki et al. 1997), AnnoteImage (Brinkley et al. 2002)

• Other multimedia environment, such as movies and audios:VideoAnnEx (Naphade et al., 2002), Madeus (Muriel et al. 1998)

• Structured data, such as databases and programming syntaxes

5.4. FORMS AND FUNCTIONS OF ANNOTATION

Although many researchers explored the use of annotations for different items in the digital world, there have not been that many studies exploring the characteristics of

Amir M Naghsh 64

using annotation for most of the items mentioned in the previous section. The item that has received more attention than others is the use of annotation in digital text documents as well as paper text books. Many studies have been carried out focusing on how readers annotate text documents and the practices which creates these annotations. To aid better understanding of different forms of annotations, this section summarises the results of some of these studies which focus on personal and public annotations.

5.4.1. PERSONAL ANNOTATIONS

People use annotation for different purposes in everyday life. One of the particular situations that most people have experienced is annotating textbooks and articles. Every individual has a unique style in making annotations, and several investigations have been made to understand how people in general annotate textbooks. Nielsen (1984) conducted a study to investigate how readers annotate textbooks and manuals. He performed “a more anthropologically flavoured investigation of the notes made by computer science students in their own personal textbooks during normal study activities.” (Nielsen 1984, p. 3)

The results of his studies were used to present a taxonomy of annotations. He explained that such taxonomy could help to have a better understanding of how computer systems should support annotation for digital (on-line) literatures. Nielsen (1984) divided the annotations in traditional textbooks into five major types. They are Highlighting, Structuring, Cross References, Formal additions and Informal additions. Each type was also subdivided further.

Highlighting “is the setting apart of some parts of the original text from other parts of this text” which can serve to “distinguish parts of the text of perceived special importance or the keywords to stand out for the quick scanning of the pages.” (Nielsen 1984, p.3)

Underlining and over-marking by pen or colour markers are two traditional ways of making highlights in text documents.

Nielsen found it possible to have all types of highlighting supported by computer systems. Some of the advantages of using highlighting in digital documents are explained by him, which are: the possibility of hiding the highlighted parts when they are not needed. This would stop readers from being distracted while reading the text. Also such systems can use highlighting to filter the digital document. An example could be using a colour marker for highlighting all the keywords and then using the

Amir M Naghsh 65

system to retrieve all the parts with such keywords. It also could be used to retrieve all the important parts with a particular colour which are related to a specific assignment.

Structuring “is the explicit annotation of some structure in the original text.” Nielsen referred to list numbering in the margin, or keywords instead of numbers as a traditional way of structuring highlighting. An example is when a student finds only some parts of the text relevant to an assignment and wants to read only those parts. Instead of structuring the notes by using different highlighting colours, the student can use key words in the margins to distinguish parts of the text for later study.

Cross references “is reference to some other text.” Nielsen explained that it is possible to distinguish between references that are to the same page, another page in the same textbook or another textbook. A current example of what Nielsen described as cross-references can be the cross-reference facility in Microsoft word for index pages and also references within the documents. Such cross-references in a digital document allow user to go to a relevant section of text without needing to leaf through the document.

A Formal Additions “is an addition having some specific structure.” Nielsen pointed few examples of this type that are: quantitative diagrams such as bar charts, formal diagrams such as flowcharts and formal tables. Nielsen explained that a computer system can help the reader by supporting editors for generating annotations.

Informal additions “are annotations without any specific structure. Typically they consist of natural language text or free hand drawings (Nielsen 1084, p. 7)”. Furthermore Nielsen divided informal additions into the following categories:

• Corrections of errors in the text are often needed in traditional books, such errors can easily be updated once they have been discovered in a digital form.

• Word explanations are required when the author uses not very well known words or the document is in a foreign language.

• Exclamations and question marks in the margin are often used to show support of or disagreement by the reader to the text. It could also be used for the parts which are difficult to understand or need remembering for future use.

• Answers to exercise which are usually written in gaps between paragraphs or in the margins close to the actual question for easy referencing.

Amir M Naghsh 66

• Interpretations and elaborations of existing text that are tied to a specific point in the text.

• Informal drawings in the form of objects and sketches.

• And new independent text that could be included in the new version of existing text.

A more recent study focusing on forms and functions of personal annotations in textbooks was conducted (Marshall, 1997; Marshall and Brush, 2002 and 2004) through performing extensive field studies. Marshall (1997) focus of research was to discover common practices and patterns when creating personal annotations. In addition to this, she studied the practices used for making personal annotations and the implications that they can bring to digital libraries once they are transformed. She also studied the value that personal annotations can have for later readers when they are shared. Marshall (1997) conducted her research through a study on annotation in paper books, since annotation tools and practices around paper have been well developed. She used students of a major American university for her study around college textbooks.

Results by Marshall (1997) along with O’Hara and Sellen (1997) showed that annotations not only serve one function but often multiple functions. Therefore, Marshall (1997) tried to “enumerate the most evident functions” used by personal annotations made in college text books and categorised them into six groups. Later she mapped all the annotation forms into the identified functions from her studies.

Procedural signals, this form of annotation was used by students to give clues and explanations for different sections of the text book. For example it was used to indicate which sections of the text book were relevant to an assignment or indicating important parts of the text book which needed more than one revision.

Placemarks, Student used annotations to indicate the exact phrases from the text book for later use. Mainly students used placemarks as an aid to memory. Some of the uses were when students wanted to use a specific quote from the text book in the final report or for definitions which were needed to be memorized for the exam.

In-situ locations for problem-working, Students used annotations to work out problems once they came across them, rather than postponing it to some time later. Marshall indicated that this sort of annotation occurred more often in the copies of

Amir M Naghsh 67

Organic Chemistry and Calculus, where students could work out the problems in margins next to theorems and equations.

Interpretive activity, students used marginal notes, scribbles and free hand notes to record their interpretations from the text book. Marshall, based on her findings across different types of textbooks, categorised this function into three types. The first identified type was tinterpretations explaining the meaning of a single world or phrase within the text. These were used more often where the subject was a foreign language. The second identified type was interpretation of the structure. And the third identified type was the interpretation of a section or part of the page. Marshall found out that students used annotation to interpret the content of a page more than the other two types. This is was the most common type used by students for different subjects. It was noted that this type of annotation could be more valuable to future readers (Marshall 1997).

Trace of the readers attention, students used annotation such as highlighting or underlining when they were reading a difficult subject which formed a series of sequences. Such annotations traced a reader’s attention while he/she was reading such materials.

Incidental reflections, students made annotations which did not have any relation to the textbook’s subject but more reflected the circumstances of time and place when that student was reading the textbook. Students used the textbook’s paper as a way to reflect on what was happening at the time in the student’s surroundings. An example given by Marshall was the phrase “I Love You” in the early pages of a calculus book.

The taxonomy that was introduced by Nielsen (1984) covered both functions and

forms of the annotation but a visible distinction between form and function of

annotations in a textbook were not identified. Unlike Nielsen, Marshall (1997)

distinguished between forms and functions of annotations and presented a table to map

from forms to functions. However, she noted that it is not always possible to have a

single one-to-one map between a form and function since a form (e.g. highlighting)

could be used for more than one function. Error! Reference source not found. is an

extension to Marshall’s table for mapping from forms to functions which also includes

annotation taxonomy that was introduced by Nielsen (1984).

Amir M Naghsh 68

Forms Functions

Nielsen Marshall Nielsen Marshall

Note

Highlighting / Underlining and over-marking

Underlining / Highlighting higher level structure, symbols

Hierarchical and differential highlighting

Procedural signals

Gives structure for future attention and use (very abstract)

Marginal symbols

Short highlights/ over-marks such as circles/ short marginal symbols

Exclamations and question marks

Place-marks and aiding memory

Indicating an exact point or phrase within the text to: Memorise, use it in later reports or to show support/ disagreement

Annotations in gaps or margins

Marginal annotations near figures or tables

Answers to exercises

Problem working

Answering questions / working out problems near figures and theorems for easy referencing

List numbering, keywords

Word or phrases between lines or in margins

Structuring Interpretation of structure (type two)/(Procedural signals)

Similar to first row but form of the annotation is changed and makes this form a more appropriate way for structuring

Annotations in form of text or scribble in the margins and maybe mapped to a word of interest

Word explanations

Interpretation to a single word (type one)

Translations of words in the margins

In addition to short and long notes, Informal drawing in form of objects or sketches

Short and long notes, in margins or as added as a new independent text (post-it)

Interpretations and elaborations

Interpretations (type three)

Interpretation to a section or part

Extended highlighting or underlining Simple highlighting

Trace of reader’s attention

Using only one means to highlight/ or being able to replace them with another

Scribbles / short notes

None Cross reference

None Referencing to other texts

None Note, drawing and other marking

None Incidental reflections

Reflections of the surroundings

Table 5-1Mapping annotations form into functions

5.4.2. PUBLIC ANNOTATIONS Marshall (1997) explained that personal annotations can become public and shared

through the reuse of the second hand textbooks. Marshall continued her investigation

on personal annotation in textbooks to recognize their values once they become public

and shared by others. Finally she suggested a strong set of implications for design of

Amir M Naghsh 69

annotation tools to facilitate readers work in a digital library setting and noted that an

implemented personal annotation tool for digital libraries is necessary to carry such

investigation forward (Marshall 1997). Some of these facilities which need to be

supported by digital libraries are as follow:

In situ annotation, distinguishable from the source, supports readers in annotating

while they read in such a way that their annotation will be in a format distinguishable

from the actual material that they are reading.

Non-interpretive markings, readers use many annotation forms which are non-

interpretive, including highlighting and underlining.

Fluidity of form, maintain the fluidity that annotation has on the paper by supporting

freeform capabilities.

Informal coding, readers usually have their own way of coding when they annotate,

for instance the way that different colour is used for highlighting could only make

sense to the reader. It is important to add enough to an annotation for digital setting to

maintain such an activity.

Smooth transitions between public and private annotations, to enable readers to

decide if they want to share their annotations. It was noted that even private annotation

can have value for the future readers of the second hand textbook. Also supporting the

reader to choose or remove annotations made by others as well as the ones that reader

has made.

Integration of reading as an activity, it should not distract the reading activity.

O’Hara and Sellen (1997) found annotating on paper is an integral part of reading

which can easily be integrated as one activity while online annotations are distractive

to reading.

Further research by Marshall and Brush (2004) found that only a small number of

private annotations can become public and shared. They found that users made some

changes in the way they made online annotation in comparison to the way that they

Amir M Naghsh 70

made personal annotations. Marshall and Brush (2004) suggested that online

annotation tools have to support two more required facilities.

Finding and filtering, helping users to find and filter annotations which the users want

to re-use and share with others. Such a facility is influenced mainly by the

effectiveness of the relevant user interface.

Support the transition from personal to shared annotations, Marshall and Brush

(2004) found that better user interfaces are required to manage and modify the anchor

of selected personal annotation to facilitate such a transition.

5.5. CHARACTERISTICS FOR ANNOTATION TOOLS

5.5.1. OPERATIONAL CHARACTERISTICS

A large number of studies (Brush et al., 2001; Cadiz et al., 2000; Neuwirth et al., 1990) have investigated the use of annotation in supporting asynchronous collaboration and communication around digital text documents. These investigations were mainly focused on the capabilities that annotation systems need to adopt in order to enhance asynchronous communication and collaboration around text documents through developing and investigating such annotation tools.

Cadiz et al. (2000) along with Brush et al. (2001) have developed and tested systems for asynchronous collaboration around digital text documents and both reported notification and anchoring of annotations as two important areas which require further research and should be supported in asynchronous electronic annotation systems. While Brush et al.'s (2001) investigations were mainly focused only on two areas of notification and anchoring of annotations from the user’s point of view, Cadiz et al.'s (2000) research reported a more complete but abstract list of factors which influence the use of an asynchronous electronic annotation systems and the need for further investigations.

Cadiz et al. (2000) conducted a large case study of digital annotations in a software specification group focusing on writing specification documents. They investigated the strengths and weaknesses of the existing collaborative annotation facility provided in Microsoft Office 2000 and identified existing capabilities and possible improvements. The major problems identified by Cadiz et al. include:

Amir M Naghsh 71

5.5.1.1. TECHNICAL ORPHANING

Cadiz et al. reported that the primary reason that the users stopped working with the tool in their study was caused by orphaning of annotation.

Because the annotations system anchors annotations by computing a unique signature for each paragraph, the system can fail to match an annotation to the correct location when the text is edited. When this happens, the annotation is “orphaned” and displayed at the bottom of the browser (Cadiz et al., 2000, p. 315).

They explain orphaning is a frustrating problem, since generated annotations become meaningless once they are out of context.

5.5.1.2. PROGRESS AWARENESS

The results (Cadiz et al., 2000) indicated that the users were not satisfied with the way that notification worked. It was not possible for users to easily determine who has done what from the notification email. Instead they needed to log in to the annotation system used to check every single annotation which was made. Brush et al. (2002) found the same results which indicated that, it is preferable to have more information in the notification messages which could support multiple communication channels, as well as allowing users to customize the content of the notification messages.

5.5.1.3. RESPONSIVENESS

Cadiz et al. reported that shortcomings of notification systems also caused a lack of responsiveness in users. Users did not find the iteration time (turn around) quick enough for short responses and therefore they would not use it in such cases. However, slow responses did not stop the generation of useful dialogues between users.

5.5.1.4. SHARED ANNOTATIONS

Cadiz et al. reported that once users were aware that their comments would be shared by others they would change their approach in making annotations. They were less likely to make small and not very important annotation. Also they tried not to make comments which could be interpreted wrongly.

5.5.1.5. RICHNESS

Users also expressed the view that they would not use annotation for reporting complex and critical problems. Instead they preferred to use more iterative and interactive tools which would allow them to express themselves properly (Cadiz et al. 2000).

Amir M Naghsh 72

5.5.2. USER INTERFACE CHARACTERISTICS

In addition to the studies that were more focused on the practice involved in asynchronous annotation tools, research (Morris et al. 1999, Neuwirth et al, 1990 and Wojahn et al. 1998) has also been conducted on the characteristics of user interfaces that can affect the way users collaborate and annotate digital documents. Wojahn et al (1998) investigated how various interfaces effect annotation and communication and proposed a set of requirements for annotation interfaces:

There should be a minimum of motion required to make an annotation. Annotation interfaces vary considerably in how many actions the user must take to create an annotation.

The document text should be easily distinguishable from the annotation. This allows other team members, who may not have seen either the original document or the annotations, to adjust themselves to the document quickly.

The annotations should easily become visible and invisible while reading the document. They should be accessible.

The relationship between the primary documents and the annotations should be easy to determine. It would allow readers to recognise which annotations refer to a particular part of the document.

Different annotation should be distinguishable to aid team members in interpreting annotations from different authors and at different stages of the process.

5.6. CONCLUSION

This chapter has reviewed the research carried out on personal annotation, with a specific reference to the annotation made in textbooks. Previous research has shown that annotating text is a common activity when reading (Marshall 1997). O’Hara and Sellen (1997) found that readers use annotation to pinpoint important parts of a text document and to help themselves to understand the text. Such remarks are also useful for future tasks. Marshall (1997) found that annotations were used in many ways in college textbooks including highlighting or underlining the most important parts to aid the memory, book marking important sections and rephrasing the information into the readers’ own words to aid interpretation. These annotations are often used as a personal way of tracking references and keeping notes.

Amir M Naghsh 73

Reported studies on personal annotation and annotation in textbooks in this chapter (Marshall 1997, Marshall et al. 2004, Nielsen 1984) has given a good understanding of the form and functions that users can adopt while making annotation in textbooks.

Annotations are often beneficial to others as well, even when they are not purposely made to be shared. Further research by Marshall et al. (2004) investigated the possibilities of transferring personal annotation into public annotation. They reported a list of capabilities which an online annotation system in a digital library context can adopt to enhance such transition activity.

This chapter has also described the research conducted on the use of annotation in asynchronous collaboration systems around text documents. These studies (Cadiz et al., 2000; Morris et al., 1999; Neuwirth et al., 1990; Wojahn et al., 1998) have investigated the use of shared annotation and reported on the characteristics that an asynchronous system should adopt to support communication through annotations. The next chapter considers these findings and reflects on them to identify a possible design for a desirable annotation tool that can be used in designing interactive systems.

Amir M Naghsh 74

Chapter 6. GABBEH

6.1. INTRODUCTION

This chapter discusses the design and implementation of an electronic paper prototyping tool with an annotation facility. To support the participatory design approach in designing such an annotation facility a number of design sessions with other researchers were conducted.

From the tools that were reviewed in chapter 2 and 3, InDesign, FreeForm and DENIM were the only tools supporting electronic paper prototyping. The analysis in chapter 4 indicated the support for pen based interaction technology and allowing stakeholders to interact and modify the design model directly as one of the key dimensions in encouraging more user participation. Therefore, InDesign was rejected since it does not support pen based interaction technology.

DENIM and FreeForm were the only two tools which support pen based interactions. Cognitive Dimension framework was applied to compare the characteristics of DENIM and FreeForm as explained in chapter 4. The theoretical analysis results showed that FreeForm in contrast to DENIM, suffers from a higher error proneness, hard mental operation, diffuseness, hidden dependency and premature commitment, while DENIM in contrast to FreeForm, benefits from a higher progressive evaluation, provisionality and juxtaposability.

In addition, DENIM has the benefit of an open-source licence and source code available online. DENIM also is supported with in-depth documentation of its design and implementation which is vital for extending the tool. Therefore, DENIM was chosen to be extended to support communication and collaboration in designing interactive systems at early stages of design.

It was decided that the extension of DENIM should be called Gabbeh which is a Persian hand-woven rug. Gabbeh was chosen to follow the fabric nature of naming of Silk, DENIM and Satin. Also in some ways, Gabbeh the rug reflects exploratory and participatory design (see appendix A for more detailed information on Gabbeh).

Amir M Naghsh 75

This chapter firstly reports on a possible design for an electronic paper prototyping tool with annotation facility and follows it by describing DENIM and the technology used by DENIM in more details. And finally it explains the design and implementation of Gabbeh.

6.2. DESIGNING AN ANNOTATION TOOL

Members of the research team (the researcher and two supervisors) conducted two brainstorming design sessions which were video recorded. The materials used in these sessions were mainly pencil, pen, paper and other office materials as well as printed screen shots of DENIM tool.

Prior to the design sessions, the research team reviewed video recording of previous interviews conducted in the Communication and Computing Research Centre at Sheffield Hallam University on designing interactive systems, in particular web sites. Reviewing the video recording revealed some of the uses of the comments in designing web sites. The findings of previous studies in addition to the findings of the theoretical studies in chapter 4 and literature review of annotation in chapter 5 were fed into the two brainstorming design sessions to generate discussions.

The video recording of brainstorming sessions were reviewed and analysed. Based on the findings and reviewed discussions the following list of requirements was identified as one possible design model for a desirable annotation tool. However further evaluation is required to validate such a design model.

6.2.1. CONSISTENCY TO CURRENT DESIGN AND EASY TO USE

Support users with a simple tool to add, edit and remove an annotation. As explained in chapter 5, there should be a minimum of action required to make, edit or delete an annotation (Wojahn et al., 1998). While the tool should be as simple as possible to be used by different stakeholders, it also should be consistent with the design of the selected electronic paper prototyping environment.

6.2.2. FLUIDITY OF FORM

Annotation is explained in chapter 5 as a common activity when using paper prototypes. The fluidity of the use of pen on paper allows stakeholders to use annotation as a way of communication when using paper prototypes. O’Hara and Sellen (1997) found that annotating on paper is an integral part of the reading which does not distract reading activity. Therefore, it is essential that annotation should not distract the actual design activity and exploring and evaluating the proposals and

Amir M Naghsh 76

requirements. It has to be designed in such a way as to maintain the same fluidity that annotation has on the paper and enable users to make comments as an integral part of the design process which easily can be integrated as one activity.

As Marshall (1997) explained, it is essential to support freeform capabilities offered by pen-based interaction technologies in addition to text comments to maintain the fluidity that annotation has on paper.

6.2.3. IN SITU ANNOTATION

As reviewed in chapter 5, one of the facilities requiring support in digital libraries was explained by Marshall (1997) as providing support for readers to annotate while they read in such a way that their annotation can be distinguished from the actual material that they read. Therefore, in a design activity stakeholders should also be able to annotate design at all stages, whether they are creating and exploring the design or evaluating it.

In addition, the user interface of such an annotation facility should have the following characteristics that were proposed by Wojahn et al. (1998):

• The actual design or prototype should be easily distinguishable from the annotation itself. This allows other stakeholders, who may not have seen either the original design or the annotations, to adjust themselves to the design quickly.

• Different comments should be distinguishable to aid stakeholders in interpreting comments from different team members at different stages of the design process.

• The relationship between the design component (or number of design components) and the comment should be easy to determine. It would enable stakeholders to recognise which annotations refer to a particular part of the design.

6.2.4. INFORMAL USE

Stakeholders usually have their own way of coding when they annotate. They use a combination of different pens with different colours, post-it notes and other office materials which may only make sense to the members of the team which have created them (Marshall 1997). It is important to add enough to a commenting tool to maintain

Amir M Naghsh 77

such an activity. Some examples could be enabling users to choose between different colours, font size or freeform styles when creating a comment.

6.2.5. RETRIEVING AND FILTERING

More evolution and iteration of prototypes means more communications and therefore more comments attached to the prototype. To enable stakeholders to track responses to their comments and be aware of the design progress of a specific part of the design through the attached comments, it is important to allow stakeholders to retrieve and filter comments once they have been created to re-use and share them with others (Marshall 1997). Or they can retrieve comments to refer to them in later decision making. It is essential to support a filtering tool to filter all retrieved comments which are related to a specific problem at any time. Such facility is mainly influenced by the effectiveness of the relevant user interfaces.

6.2.6. AVAILABILITY

The design sessions found that annotations should be supported in all representations offered by a prototyping tool. In particular, when different stakeholders interact with different representations of a design it is vital to maintain consistency in making comments available in all of the offered representations.

The results of design sessions also indicated that there should not be any boundaries for creating annotation. This includes the ability of the user to annotate limitlessly anything and anywhere on the paper prototype as well as being able to add comment notes when there is no room for annotation in a specific area of the design.

Therefore, it is important to maintain such unlimited annotation by enabling the comments to be associated with any arbitrary number of components in a design model. Some specific examples are:

• To annotate any component in the design model (such as links, pages, scribbles, etc);

• To annotate any collection of components, for example a comment which could refer to a collection of pages;

• Any number of components share one comment;

• Any number of comments refer to one component;

Amir M Naghsh 78

• Comment on a specific sequence of components (for example a sequence of pages in an e-commerce website: home -> shop -> basket -> check-out);

• To attach a sketch or a file to a comment;

• To use the comment to link the design model to other representations.

6.3. DENIM

The importance of reviewing the way DENIM works was recognised as being able to implement Gabbeh. This section explains DENIM user interface and its underlying architecture which later has been extended into Gabbeh.

6.3.1. THE DENIM USER INTERFACE

As explained earlier in chapter 2 DENIM is a Java based application which has been developed in Berkeley University under an open-source licence. The primary DENIM window is shown in Figure 6-1. The window has three main areas:

The centre area is a canvas where the user creates Web pages, sketches the contents of those pages, and draws arrows between pages to represent their relationship to one another.

On the left is a slider that reflects the current zoom level and allows the level to be set.

The bottom area is a toolbox that holds tools for drawing, panning, erasing, and inserting text fields. Users “pick up” a tool by tapping on it, “drop” a tool by tapping in an empty area in the toolbox, and “swap” tools by tapping on another tool. There are several tools: a hand tool for panning, a pencil and eraser for sketching and erasing the design, and a text field stamp for inserting text fields.

Amir M Naghsh 79

Figure 6-1 DENIM window

In DENIM users can sketch out the overall structure of a web site (a collection of pages), sketch the contents of the pages as a set of ‘scribbles’, draw strokes (in form of a arrow) to define a relationship between two pages. DENIM provides navigational and organizational arrows to demonstrate relationships between pages. Navigational arrows represent hyperlinks in the HTML sense hyperlinks from scribbles in one page to another page. Organizational arrows are used to represent a conceptual relationship between two pages (see Figure 6-2). There are five main zoom levels in DENIM, which are identified on the zoom slider with icons representing the type of view available at that level. The slider bar allows the site to be viewed from a site overview that simply identifies the pages included, through a navigation view where the overall navigation can be examined, down to a detailed view where fine details of individual pages can be manipulated (see Figure 6-3).

Amir M Naghsh 80

Figure 6-2 DENIM design mode at storyboard view

Figure 6-3 DENIM design mode at page view

DENIM can execute the resulting hypertext in a reduced functionality browser. The browser displays one page at a time, like a real Web browser, except the pages displayed are the sketches that the designer has created (see Figure 6-4). If a component within a page is the source of a hyperlink, it is rendered in blue. Clicking on these design components displays the page that is the target of the link in a similar way to normal browsers. The browser also has Back, Forward, and Refresh buttons.

Amir M Naghsh 81

Figure 6-4 DENIM in run mode

6.3.2. ARCHITECTURE OF DENIM

DENIM has been developed as an application on top of Satin and extends many of its features. Satin (Landay et al.) is also a Java based application that was largely co-developed with DENIM at Berkeley University. Satin makes heavy use of Java2D libraries and some use of Swing GUI toolkit. Although Satin provides an abstraction of Java2D, developers can use the Java2D instead when it is desirable (see Figure 6-5).

Satin was developed mainly to apply natural activities such as sketching and writing in an electronic prototyping environment. DENIM uses Satin’s toolkit support for informal ink applications.

Amir M Naghsh 82

Figure 6-5 DENIM relies on Satin

Satin provides a tree-like structure for manipulating and rendering graphical objects which is called scenegraph. The scenegraph manages the way that graphical objects and strokes are processed and displayed. Sheet is the root of the scenegraph and a node in the Satin’s scenegraph is called “GraphicalObject”. There are several sub-classes that extend the GraphicalObject. The simplest GraphicalObject is TimedStroke which is set of points and time that are drawn. Satin assembles TimedStroke from mouse or pen movement and provides reusable mechanisms for handling and processing strokes.

DENIM uses the same scenegraph used by Satin as its data structure. The tree scenegraph allows DENIM to know what is on the screen. Figure 6-6, Figure 6-7 and Figure 6-8 display three different representations for the DENIM scenegraph which contains DenimSheet, DenimPanel, DenimLabel, DenimSketch, TimedStroke and other components and instances used within DenimSketch. DenimSheet extends Sheet class from Satin as the root of the DENIM scenegraph. DenimPanel is what represents the screen in GUI or the webpage in a website. DenimPanel is the parent to DenimLabel and DenimSketch and it keeps them together when the design is zoomed in and out, or when one of them is moved and the other one needs to move with it too.

Amir M Naghsh 83

Figure 6-6 UML diagram of DENIM scenegraph

Amir M Naghsh 84

Figure 6-7 Tree data structure of DENIM scenegraph

Amir M Naghsh 85

Figure 6-8 A visual representation of DENIM scenegraph. DenimSheet boundary is highlighted with a red line, DenimLabel with a black line, DenimSketch with a

green line and TimedStroke with a blue line. Also TypedText and ScribbledText are highlighted within the sketch

Each GraphicalObject has one or more views which know the visual boundaries of the object and how to render the object. Each GraphicalObject also has one or more interpreters for handling strokes. Interpreters can be added to any graphical region displayed on the sheet and process the ink strokes directed to that region before they are rendered to the screen. Each GraphicalObject has two sets of interpreters to handle strokes in two categories for gesture and ink.

When strokes are drawn they are dispatched to the root GraphicalObjects at the tree (which is the sheet) and recursively traverse down the tree, processed only by the GraphicalObjects that contain the complete stroke. This method allows the stroke to be processed first as the gesture before processing it as ink. It also provides global behaviour for gestures. (See Figure 6-9).

Amir M Naghsh 86

Figure 6-9 Diagram of how strokes are handled as gesture and ink

Amir M Naghsh 87

6.4. GABBEH

Early design of Gabbeh prior to its first version was presented in the HCI2004 conference (Naghsh and Özcan, 2004). This section reports on the user interface of the first version of Gabbeh. It further describes the implementation of that version of Gabbeh in detail and reports the reasons for selecting some of the requirements mentioned to be developed in this version.

To address the issues raised by sessions for designing an electronic paper prototyping tool that has annotation facilities, Gabbeh as an extension to DENIM was developed to support communication and collaboration among stakeholders by annotating the design. The core innovation in Gabbeh is that it allows different stakeholders to add arbitrary annotations in the form of comments when the model is being designed or when the model is being executed. It is important to note that this version of Gabbeh (version 1) does not meet all the requirements recognised in section 6.2.

6.4.1. THE GABBEH USER INTERFACE

Gabbeh offers annotation features in both, ‘design mode’ and ‘run mode’. The ‘design mode’ consists of the design sheet and a limited functionality browser. The ‘design mode’ enables the designers to create design proposals by using available features in the toolbox and multiple representations offered through zooming. In addition, it allows designers to interact and evaluate such design proposals by using the limited functionality browser.

Figure 6-10 Login box: allowing stakeholders to select different versions of Gabbeh

Amir M Naghsh 88

The ‘run mode’ consists of a limited browser. Since the end-user might not have access to a graphic tablet, the design sheet is excluded from the ‘run mode’ version to allow end-users to work only with a simple browser with annotation features that can be run on a standard PC. A stakeholder depends on being an end-user or a designer can choose one of the modes by selecting an option in the log in dialogue box (see Figure 6-10).

Figure 6-11 Gabbeh toolbox, Commenting pencil for creating and editing comments (fifth from left), pad for filtering the

comments (sixth from left)

6.4.1.1. THE DESIGN MODE

6.4.1.1.1. CREATING COMMENTS

Gabbeh allows designers to add arbitrary scribbles (free-hand notes) to a comment using a commenting pencil (see Figure 6-11) similar to the free-hand writing tool used to create elements in a web page in ‘design mode’.

The designer can create a comment at any of five zoom levels. There are two ways that a designer can use the commenting pencil to create comments. The first way is to use the commenting pencil to write words in free-hand style on the canvas. Gabbeh uses the Satin toolkit to group the ink strokes that are close to each other into words and phrases. Then Gabbeh automatically creates a comment using the phrase (see Figure 6-12). The defined background colour for a created comment is blue which can be changed easily by using same commenting pencil tool (explained later). Comments are given a background colour. This is intended to allow development teams to distinguish between different types of comments, or perhaps between comments from different speakers. The usage of colour is deliberately left open.

Amir M Naghsh 89

Figure 6-12 Comments can be created by writing words or phrases directly on the design sheet at any zoom levels

The second way of creating a comment is to use the commenting pencil to open a dialogue box which allows the designer to type in the comment and insert it on the canvas (see Figure 6-13).The commenting pencil has to be clicked once on an area on the design sheet where the designer wants to insert the comments. When the commenting pencil is clicked once, Gabbeh opens a dialogue box and allows the designer to type in the comment text as well as selecting the background colour of the comment.

Figure 6-13 the comment dialogue box allowing the designer to select the colour as well as entering the typed text to create a

comment

6.4.1.1.2. ASSOCIATING COMMENTS TO OBJECTS

A comment in Gabbeh can be associated with any arbitrary number of design components, such as panels, labels, texts and scribbles. Arrows have been used to associate comments with design components. An arrow between a comment and a design component represent a relationship which is called annotation arrows. Annotation arrows indicate that the particular design component has comments attached to it.

To create an annotation link, the designer draws a stroke by using the pencil or the commenting tool between the comment and design component. Gabbeh checks if it is an arrow. To be an annotation arrow, it has to start from a comment and end on a design component. Once Gabbeh recognises the stroke as an arrow, it will change its colour similar to the comment background. This process can be repeated as many times as the designer wants in order to share the same comments for different components (see Figure 6-14).

Amir M Naghsh 90

Figure 6-14 Using the drawing or commenting pencil to associate a comment with one or many design objects

Gabbeh also allows a comment to be associated with a different number of pages or design components located within different pages (see Figure 6-15).

Figure 6-15 Representing a comment shared between components from two pages in the design

6.4.1.1.3. EDIT COMMENTS

The commenting pencil can be used to change the colour of a comment regardless of its style being freehand or typed text. The designer can use the commenting pencil to click on the interested comment, and then Gabbeh will open the dialogue box similar to the one shown in Figure 6-13 and allow the designer to change the colour to a different

Amir M Naghsh 91

one for the comment. Also if the comment is in typed text format, the same dialogue box allows the designer to change the text in addition to the colour.

The commenting tool can also be used to add more strokes to a phrase used in comment by writing relatively close enough to the comment. However, it is not possible to edit the phrase in freehand style once it is created. To edit a freehand comment the designer can delete the comment and rewrite the comment with new changes, or convert it into typed text using same dialogue box that has been used to edit the comment’s colour.

6.4.1.1.4. DELETE COMMENTS

The eraser tool from toolbox can be used where required to delete a comment. Once a comment is deleted all of the associated links going out from that comment are deleted too. However, it is possible to use the eraser tool to only delete association arrows and not the comment. For example in Figure 6-15 the eraser could be used to delete only one of the arrows which is not required any more.

In addition to the eraser tool, the delete gesture could also be used. Another alternative would be using the delete option from the pie menu.

6.4.1.1.5. FILTERING COMMENTS

Using Gabbeh indicated that as the prototype evolves the number of comments attached to it increases rapidly. Preliminary exercises indicated that the design could become crowded with a large number of comments and it is vital to be able to make comments invisible and only allow the interested comments to be visible and displayed. Figure 6-16 shows a design case which became crowded with a large number of comments at early iterations. The comments displayed in Figure 6-16 are displayed in the storyboard view of Gabbeh. These comments are attached to a prototype of a website which has only four pages. Therefore, it was suggested that it is essential to be able to make comments visible or invisible and filter them to allow designers to concentrate on the actual design and comments relevant to that part of the design.

Amir M Naghsh 92

Figure 6-16 Design sheet crowded with comments

The designer can use two tools to manage the visibility of the comment in the ‘design mode’. The commenting pencil is one of the tools that can be used to hide comments or make them visible. The designer can use the commenting tool over any individual comment to open the dialogue box to manage the visibility of the selected comments (see Figure 6-17). The dialogue box allows the designer to choose from two options of showing and hiding a comment. When the comment has been set to be invisible, only the starting point of the arrow that associates the comment to a design component remains visible (see Figure 6-18).

Figure 6-17 Using the commenting pencil for managing the visibility of comments

Amir M Naghsh 93

Figure 6-18 A coloured node represents the comment once it is set to invisible

Gabbeh also supports a simple filtering tool in the design view to offer a way of managing the visibility of a larger number of comments. The current design gives two options. One way is to click the filtering tool somewhere on the design sheet to manage the visibility of all comments attached to the design sheet; the other option is to use the filtering tool by clicking on the label of an interested page to manage only the comments attached to that one page (see Figure 6-19). The filtering tool also allows the designer to select the colour of comments that their visibility status should change.

Figure 6-19 Managing comments in Gabbeh

Amir M Naghsh 94

6.4.1.2. THE RUN MODE

End users may execute Gabbeh using a separate limited functionality browser to review a design. To make Gabbeh easier for end-users, the design sheet is excluded from the version of ‘run mode’, which is installed at end-users site (see Figure 6-20). It allows the end-users to work only with a simple browser with annotation features (see Figure 6-21).

Figure 6-20 The run mode interface for Gabbeh. File button is to open a design file, Run button to start the execution of the design model and save button to save the added comments to

the design model

Figure 6-21 Limited functionality browser with commenting features

Gabbeh allows end-users to view and add comments while they are reviewing the design in ‘run mode’. This functionality is intended to allow end-users to give feedback through the prototyping medium. The end-user can choose to view the comments by selecting the view comments button from the menu bar (see Figure 6-21) and Gabbeh displays comments in a side window adjacent to the side of the page (see Figure 6-22).

Amir M Naghsh 95

Figure 6-22 Viewing comments in the 'run mode'

Figure 6-23 Adding comments when Gabbeh is executed

Gabbeh displays the comments location within the page using coloured numbers on the page (see Figure 6-22). If the comment is only associated with the page, Gabbeh only displays the comment in the side window.

Users are able to add comments when Gabbeh is being executed, which can then be visible in the ‘design view’. Figure 6-23 shows an example of how users can add a comment in ‘run mode’ by clicking on the "Add Comment" button which opens a comment dialogue box similar to the one in the ‘design mode’. Once comment is

Amir M Naghsh 96

created it will be displayed in the side window as well. In this version, comments created in ‘run mode’ are associated with a page, but not with a specific position on the page.

6.4.2. IMPLEMENTING GABBEH

The architecture of DENIM is decomposed and spread into many fine-grained packages as can be seen in appendix D. Therefore, understanding the overall system is challenging. For example, if one decides to identify all the related classes to DenimPanel it would be very difficult through the current packaging of DENIM. In addition to the difficulty of identifying such relationships DENIM classes have a high coupling which makes it very difficult to modify or extend. Once one class is modified it is more likely that several classes are affected by that change in other classes and packages in DENIM. Figure 6-24 shows classes related to the DenimPanel which are affected once a DenimPanel is modified.

Amir M Naghsh 97

Figure 6-24 Dependency Graph for DenimPanel

Amir M Naghsh 98

For example, the io package (see Figure 6-24) uses a predefined form for DenimPanel to convert it to a document object model (DOM) element and save it to a file. DOM uses a predefined way for accessing and manipulating DENIM components such as DenimPanel to save them into a file that can be accessed later and read to load the DENIM model. The structure used to convert DenimPanel is:

<DenimPanel>

<ID>…</ID>

<AffineTransform>…</AffineTransform>

<BoundingBox>…</BoundingBox>

<DenimLabel>…</DenimLabel>

<DenimSketch>…</DenimSketch>

</DenimPanel>

Any modification to DenimPanel structure implies a revision of all the methods for converting the DenimPanel to DOM element and creating DenimPanel from DOM element.

DENIM, through the use of the Satin toolkit supports the development of new interpreters and recognisers. DENIM also supports the creation of custom components to be added within DenimSketch. But the high coupling of DENIM and its concrete architecture makes it difficult to modify the scenegraph and structure of DENIM shown in Figure 6-6. For example to add a new object within DenimSheet similar to DenimPanel, all the interpreters, some of the classes in the command, view, io, action and component packages will need modification. Such modification is required because all the classes related to DenimPanel shown in Figure 6-24 are designed and developed based on DENIM scenegraph (see Figure 6-6) which means that they only support the use of DenimPanel as a recognised object to be added to DenimSheet with specific references to DenimPanel from each class and package.

Therefore, the development of the commenting facility is a difficult task, as introducing a new instance into DENIM scenegraph requires major modification of its structure. So, it is time consuming and costly to implement a quick prototype of the proposed design to validate the identified requirements and explore the design of the commenting tool further.

Amir M Naghsh 99

Consequently, due to time constrains and the complexity of DENIM it was decided to extend DENIM based on what is already available within the DENIM structure to support the commenting facility and allows stakeholders to annotate the prototype during the design process.

This section describes the development of Gabbeh and indicates the areas of design which were not developed further due to the reasons mentioned above.

6.4.2.1. INTRODUCING COMMENT AND COMMENT-MARK

Since it was recognised that modifying the DENIM structure is difficult and not practical with the given resources during this research project, available entities in DENIM structure were investigated and DenimLabel was identified to be extended further to implement the requirements described in section 6.2.

DenimLabel was chosen instead of DenimPanel or DenimSketch, since it is the only entity in DENIM that can be created by using scribbled text (free ink) or typed text which supports fluidity of form. It also supports the requirements explained in sections 6.2.3 and 6.2.6 by allowing the DenimLabel to be created at any zoom level and added to any location on the sheet, while DenimSketch can only be created at particular zoom-levels like page or detail.

On the other hand DenimPanel, as explained before, is a rectangle which keeps DenimLabel and DenimSketch together. DenimPanel is created when a rectangle is drawn on the sheet. There are three interpreters which allow DenimPanel to be created directly and once it is created, a default DenimLabel “page” is associated with DenimPanel and it is only possible to modify it by using the text editor. DenimPanel is also created when a DenimLabel is created. This way it is DenimLabel which is created first by using a typed text or scribbled text and then associated to a new DenimPanel.

As explained it is not possible to create a new object as an extension of DenimLabel due to the concrete structure of DENIM. Therefore, a flag as a variable is added to DenimLabel which allows two sets of behaviours for DenimLabel depending on whether the flag is on or off. By default once the DenimLabel is constructed the flag is unchecked and the normal methods of DenimLabel can be used by other classes. Once the flag is checked, DenimLabel would be treated as a comment and in addition to the DenimLabel methods the new set of methods added to DenimLabel could be called as well. This can be seen as extending DenimLabel into DenimComment without altering the DENIM scenegraph.

Amir M Naghsh 100

In addition to the modification of DenimLabel, a DenimCommentMark object was also created to be added to the DenimSketch to highlight the location which the comment is referring to on the DenimSketch. By default, DenimCommentMark is set to invisible since the arrow points at the position that the comment is associated with (see Figure 6-17 and Figure 6-18). The current structure of the “run mode” does not allow the arrows to be displayed and therefore DENIM-Comment-Mark is used to indicate a comment’s position by displaying the comment ID on DenimSketch (see Figure 6-22). DenimCommentMark is created as an extension to DenimIntrinsicComponent since an intrinsic component can consist of components rather than the GraphicalObjects.

Figure 6-25 DenimComponent extending DenimIntrinsicComponent

To support the use of comments, in addition to the modification to DenimLabel and creating the DenimCommentMark, a number of tools and interpreters have to be modified or created to allow stakeholders to use such tools to draw strokes which can be handled by interpreters to create comments.

6.4.2.2. COMMENT TOOLS

ToolsArea is the container for all the tools in DENIM. Tools are extensions of the java swing JPanel which are placed in the ToolsArea and can be selected with a mouse click or a tap of pen on the tablet. Once a tool is selected the mouse cursor changes to the selected tool and the tool in the ToolsArea is set to invisible. To drop the tool back into the ToolsArea, the user can click on an empty area of the ToolsArea or on another tool to exchange them, for example exchange pen with hand. Once a tool is grabbed, a set of interpreters associated to that tool will be activated to handle strokes created by that tool.

Amir M Naghsh 101

Figure 6-26 Introducing new tools in Gabbeh

Two tools are added (see Figure 6-11) to the toolbox in addition to the original tools introduced in DENIM. The new two tools for annotation extend class tool (see Figure 6-26). The comment pen is added for creating and editing comments while the comment view and filter tools are created for managing the visibility of comments. CommentInterpreter and CommentFilterInterpreter are the two main interpreters created to enable each tool to perform their actions. The following sections describe these interpreters in addition to the modified ArrowInterpreter.

6.4.2.3. COMMENT INTERPRETER

To provide the comment facility in Gabbeh a number of interpreters have been added to DenimSheet in addition to the ones from DENIM. CommentInterpreter is added to DenimSheet once Gabbeh is started and became enabled once the black pencil for creating comment is grabbed.

Once the CommentInterpreter is enabled it can handle drawn strokes to recognise if the user is using the comment pencil tool to create a comment in a free-ink style or wants to open the comment dialogue box to create a text comment. As explained before DenimLabel is used to create a comment which consists of ScribbleText or TypedText (see Figure 6-6).

Amir M Naghsh 102

If the comment pencil tool is used to create comments in a free-ink style the CommentInterpreter determines whether strokes drawn on the DenimSheet should be treated as a new comment or should be added to an existing comment. The CommentInterpreter contains DENIM’s ScribbledTextInterpreter which converts TimedStrokes into one ScribbleText object which then can be used to create a label or comment in Gabbeh. For example “Blue” in Figure 6-12 is a ScribbleText object constructed of two TimedStrokes “B” and “lue”. The ScribbledTextInterpreter also determines whether bordering strokes are close enough to an existing ScribbleText to become part of it or not. In the case of the “Blue” example the second TimedStroke was drawn close enough to the first one to create one ScribbleText.

Once the ScribbleText is created it is not possible to edit it. The current structure of DENIM does not allow editing TimedStroke and therefore it is not possible to edit comments in free-ink format created in Gabbeh. Any modification to StrokeAssembler in Satin which generates TimedStroke to enable editing of strokes would change the underlying structure of Satin and therefore DENIM. This in turn would require major modifications to both of them which is far beyond this work.

Therefore in this version of Gabbeh to edit a comment that is created in free-ink style, it has to be deleted and created again. However, if the comment tool is used to create a TypedText comment, it is possible to use the same tool to open a dialogue box to edit the text or colour for that comment.

The structure of DenimText as defined earlier only allows one of the forms of scribble or typed text. In addition, interpreters handling TimedStroke in DENIM and Gabbeh can only process them as part of a ScribbleText object. Therefore, it is not possible to construct a comment using a ScribbleText and TypedText.

It is not possible to merge the two types of scribble and typed text into one comment but it is possible to convert the scribble text into typed text at any time. The comment pencil tool can be used to open the comment dialogue box and therefore CommentInterpreter can convert a ScribbleText comment (free-ink style) into TypedText comment or edit a TypedText comment as well as editing the colour of comments of both types.

6.4.2.4. ARROW INTERPRETER

Gabbeh provides annotating arrows in addition to navigational and organisational arrows introduced in DENIM. The annotating arrows represent a link between a comment object and a GraphicalObject or a location within the visual boundaries of a

Amir M Naghsh 103

GraphicalObject. Therefore, DENIM’s ArrowInterpreter has been extended to determine whether or not to recognise a stroke created by a comment pencil tool or normal DENIM pencil as an arrow and further to interpret it as an annotating arrow if it is originated from a comment. If the stroke is drawn from a GraphicalObject for example a label and ends in a comment, the ArrowInterpreter does not interpret it as an arrow. When the stroke is recognised as an annotating arrow, the arrow’s colour will be changed to match the comment’s colour and also a DenimCommentMark instance is inserted at the destination point that the arrow is pointing at. The DenimCommentMark instance is only visible in the “run-mode” to allow the user to identify the point that a comment is referring to.

As explained in section 6.2.6 the comment facility should allow for an annotation to be added to any component (e.g. links) in the design model. Since the arrow itself is not a GraphicalObject, it is not possible to associate a comment to an arrow by using an annotation arrow. The arrow is the extension of TimedStroke which is redrawn every time DenimSheet is rendered (for example when it is panning or zoomed). Modifying arrow structures is not possible at this stage as it effects too many classes and therefore the requirements for adding comments on links is omitted from the list of feature which are needed to be developed for this version of Gabbeh.

6.4.2.5. COMMENT FILTER INTERPRETER

To support filtering and retrieving comments it is essential to be able to make all the comments invisible and only display the comments that a user is interested in. Therefore to manage the visibility of comments the CommentFilterInterpreter was implemented. The CommentFilterInterpreter manages the visibility of comments through a dialogue box implemented for this purpose.

The CommentFilterInterpreter is associated with the filter tool which has already been described. This version of Gabbeh allows stakeholders to use the filter tool to manage the visibility of all comments associated to the DenimSheet or only a collection of comments associated to a selected DenimPanel. Once the tool is used to dispatch a stroke, CommentFilterInterpreter determines the location that stroke originated from. If it originated within a DenimLabel then Gabbeh allows the stakeholder to manage the visibility of comments for the parent Panel of that Label, otherwise if it has been originated somewhere on the DenimSheet then it would be allowed to control the visibility of all comments attached to the sheet. If the stroke originated within a DenimSketch the interpreter does not process it and passes it to the next interpreter in the queue or attaches it to the child GraphicalObject.

Amir M Naghsh 104

The effectiveness of the CommentFilterInterpreter is highly influenced by the user interface of the control dialogue box. The current interface only allows filtering comments based on their colours since no other identifier is associated with comments. In addition to time and resource limitation, the requirement for the filter and view tool was shallow and abstract. Therefore, a simple interface was implemented to introduce such a possibility as part of the commenting facility and allows the stakeholders to explore the requirement for such a tool by using it when it is needed.

6.4.2.6. RUN MODE

To allow comments to be displayed in the run mode, DenimRunSheet and DenimRunWindow are both modified. As can be seen in Figure 6-21 two buttons are added into the menu bar of DenimRunWindow. One of the buttons is to enable stakeholders to add new comments and another is to hide and view comments when needed. Since the DenimPanel associated to a collection of comments needs to be visible at all times, a JPanel has been added beside the DenimRunSheet to display comments. Using two panels to display comments allow the DenimRunSheet panel to display a static view for the DenimPanel at all times while the comment panel could be scrolled to view all the comments associated with a specific DenimPanel.

However, using two separate JPanels does not allow drawing the arrows to link comment to a position within the DenimPanel. To enable arrows in “run mode” a third panel must be placed over two panels to map a comment to its associated location by drawing arrows. However, the current focus is not on the usability and beautification of Gabbeh and such modifications require more resources and time. Therefore, the possibility of such implementation is considered as future work when the core design of Gabbeh is validated.

Since it is assumed that end-users only have access to a simple desktop, the free ink capabilities are not supported in the run mode. Therefore it is not possible to create ScribbleText comments. The add button only allows stakeholders to create a TypedText comment in the run mode. Also use of two separate panels to display the DenimPanel and comments makes it difficult to draw arrows from comments to position on DenimSketch and therefore to indicate the location of comments on the sketch DenimCommentMark has been used. The comment marks become visible for the DenimPanel which is displayed, once the view comment button is selected.

Comments are displayed based on their creation time, new comments are displayed first. The comment ID is the main source for sorting comments. This version of

Amir M Naghsh 105

Gabbeh only allows creating comment at the page level and it does not allow adding comments at the detailed level and within the DenimSketch. However adding comments at the detailed level and sorting and filtering comments in the run mode are essential requirements which should be included in future versions.

6.5. CONCLUSION

This chapter has explained the implementation of Gabbeh to be able to validate the proposed design for an electronic paper prototyping tool and refining the design criteria and improving them as part of a formative evaluation. This chapter has extended the previously introduced design of DENIM further to allow different stakeholders to add arbitrary annotations in the form of comments when the model is being designed or executed.

While the time and resource constraints were the main limitations, the development mainly focused on implementing a prototype to investigate and explore the design criteria of an electronic paper prototyping tool such as Gabbeh and its role in facilitating communication and collaboration among dispersed stakeholders. It has been noted that to investigate the usability and enhance the beautification of Gabbeh more time and resources are required which is out of the scope of this work and can be addressed in later stages after the design criteria is investigated and validated.

The features such as annotation on transitions, displaying arrows in the run, a mix style of text and free-ink for comments and other features which were omitted from this version are important and should be implemented once the resources are available, but it is not essential at this stage as the aim is to discover to what extent Gabbeh could facilitate the communication among dispersed stakeholders.

Amir M Naghsh 106

Chapter 7. OBSERVATIONAL STUDIES

7.1. INTRODUCTION

Once a working prototype of Gabbeh with the essential criteria was implemented it was vital to evaluate Gabbeh to be able to validate the proposed design with the co-operation of participants as stakeholders, as well as refining the design criteria and improving them as part of a formative evaluation.

This chapter reports on the evaluation of the Gabbeh version 1, explaining the objectives, evaluation approach, layout of the study and participants involved in the study. Finally, it presents the outcome of the evaluation study and discusses the results of the observations.

7.2. OBJECTIVES

Gabbeh was developed as a quick prototype to discover to what extent an electronic paper prototyping tool can facilitate asynchronous communication among distributed stakeholders at the early stages of design.

The evaluation of Gabbeh was conducted to validate the design criteria as well as finding ways to evolve and refine them. This emphasised the need for a formative evaluation of Gabbeh to verify if it met stakeholders’ needs and also to explore the design further.

A summative evaluation is undesirable at this stage since the development of Gabbeh is at its early stages of exploring and validating requirements which are higher priorities compared to evaluating the usability of the design. In addition, considering the time and resource constraints and the high cost of a usability test, it was decided to not include usability testing in this evaluation. However, it is recognised that once the design proposal is validated by stakeholders it is vital to improve the usability of Gabbeh.

Members of the research team conducted two design sessions to identify objectives for the evaluation study as well as arranging the layout of the study.

Amir M Naghsh 107

Three main objectives were identified for this evaluation study.

• The main and most important objective was to determine if the comments were going to be used in Gabbeh or not.

• Communicating design ideas among stakeholders is one of the important tasks at early stages of design. Therefore, the second goal was to discover if a tool like Gabbeh would be used to generate any form of communication such as discussions or dialogues between different stakeholders. Particularly, would stakeholders use the comment facility to respond to each other in form of a discussion?

• As explained in chapter 5, little is known on the use of annotation in designing interactive systems while the attention has been focused on the use of annotation in textbooks and multi authoring text documents. The final objective for this evaluation was to discover how comments are used when designing an interactive system such as a website.

7.3. LAYOUT

The evaluation study was conducted by simulating design exercises. DENIM was used in the evaluation studies as an environment without a comment facility against Gabbeh to explore the use of the commenting facility once it is available to stakeholders. As previously explained this work is mainly interested on asynchronous communication among stakeholders in a dispersed setting. Therefore, the layout was adopted in a way to support a dispersed setting where different stakeholders were situated in two separate locations (see Figure 7-1).

Figure 7-1 Illustration of the layout of the evaluation exercises in two different location. The only communication channel

Amir M Naghsh 108

was through the email and prototype while there was no face-to-face meeting

Three participants were involved in each exercise. Participants were divided into groups of two “designer-users” and one “end-user”. The designer-users and end-users were located at two separated offices. The only communication channel between them was through email and the prototype itself (see Figure 7-1).

The two designer-users worked together on a Graphic Tablet connected to a laptop. They had access to two displays (Graphic Tablet and Laptop screen), keyboard and an extra mouse (see Figure 7-2 and Figure 7-3). A video camera was used to record the designer-users activities as well as discussions between them. The end-user had access to a desktop PC to review the prototype and give feedback to the designer-users. Another video camera was installed at the end-user’s location to record the participant activities.

Each design exercise consisted of two sessions. Each session was between 1.5 and 2.5 hours long. Only one session was allocated to a day and each exercise was split over two days. A design task was allocated to each of the sessions. Designer-users were asked to perform the assigned task in the first session by using DENIM and the task assigned to second session by using Gabbeh and same order was used throughout the evaluation.

Each individual “designer” went through two training sessions and received approximately one hour training on using DENIM and one hour on using Gabbeh. Designers received training on DENIM prior to the design task allocated to DENIM and they received training on Gabbeh prior to the allocated task to Gabbeh. The purpose of the training was to help designer-users to become familiar with each tools’ features and with using a Graphic Tablet, as this was a new device to some participants. The training was provided well in advance, two to three days before each session of design exercise to allow the designer-user to arrange more training to use the tool if the designer-user felt it was necessary.

The order of using DENIM first then Gabbeh allowed designers to use DENIM without being aware of the features offered in Gabbeh. Designer-users became familiar with the strengths and weaknesses of DENIM and adopted their way of working with what it had to offer. Next, designer-users were introduced to Gabbeh and were introduced to its comment facilities. At this stage whilst designer-users were familiar with DENIM,

Amir M Naghsh 109

they had the opportunity to decide if the new features introduced in Gabbeh are useful to them and can improve their working method.

Figure 7-2 This is a picture taken in the pilot study demonstrating the designer-users working together using a

graphic tablet and laptop connected to each other. Both designer-users were contributing to the design by using the

tablet, keyboard and the mouse. Halfway through the design, two designer-users swapped places

Amir M Naghsh 110

Figure 7-3 Picture taken from the first design exercise (group1). The picture is demonstrating one designer (left) in

charge while the other designer (right) is more distanced from the computer. The video record showed both designer-users discussed the problem but only one (left) was creating the

design. While the other one (right) was mainly giving feedback and making sure that the agreed design was developed and in

some occasions give assistance by typing on the keyboard

The other possible order of using Gabbeh first would allow designer-users to experience the same weaknesses that they have discovered by using Gabbeh when using DENIM, in addition to the shortcoming that they could have experienced due to the absence of the commenting facility. Being aware that certain features are stripped from the tool could have influenced the natural way that designer-users would use DENIM. Therefore, designer-users were not introduced to Gabbeh until they had received training on Gabbeh prior to the second session. However, designer-users were informed at the beginning of the layout of the evaluation study and the fact that the design tool would be different in the two sessions.

As previously explained, the main objective of this evaluation study was to discover if the comment facility was going to be used by participants once it was available to them. The evaluation study did not intend to investigate the effectiveness of the comment facility on enhancing the communication. In other words, the study was not investigating if the introduction of the comment facility might have resulted in generating more communications or comments compared to a tool such as DENIM that

Amir M Naghsh 111

does not support such a facility. Therefore, it was decided that is not required to swap the order of design tasks between the two sessions.

At the starting point for each exercise the designer-users were given a task sheet describing requirements for a website and were instructed that they were to develop a preliminary design and gather feedback from the user throughout the design session. Design sessions lasted between two and three hours. Participants in the study were paid a small fee for their time. After each session an informal debriefing session was held with the designer-users, to ask for their feedback and discuss any issues. This was also recorded.

7.4. PILOT STUDY

The first exercise was a pilot study. The designer-users for the pilot study consisted of a member of staff of a computer science department with some experience in web design, a professional interactive system developer and the end-user was a member of staff from the business school. They used DENIM to design a ‘research-group’ website and Gabbeh to design a ‘personal’ website (see Appendix B for the task sheets).

The task sheet was a brief description of the requirement for a website printed on one side of an A4 paper. Since the tasks were brief requirements it was vital to enable the end-user to give sensible feedback in the form of adjustments and clarification of the requirements to the designer-users. Therefore, the end-user was required to have a good understanding of the described problem in the task sheet.

Prior to each session the task sheet relevant to that session was given to the end-user to adopt the task. The first task ‘research-group’ website was generated by the research team, considering that the end-user is a member of academic staff and should be able to relate the task to her knowledge in the work environment. For the second task a ‘personal’ website, the end-user was asked to write a description of what she would want to have in her personal website and it was used to generate the second task. Both tasks were chosen in a way to relate to the end-user knowledge and allow the end-user to influence the design by making interpretations of the task.

At the beginning of each session, the designer-users were given a task sheet, briefing a website requirement and were instructed that any further question regarding the task should be forwarded to the end-user. However, they were encouraged to generate a preliminary prototype based on the given task sheet and make inquiries about the requirements while forwarding the first version of the prototype. Designer-users were

Amir M Naghsh 112

instructed to adopt an iterative design approach to generate a prototype in the 1.5 to 2 hours given time. They were instructed to keep the end-user informed and seek for end-user’s feedback when appropriate.

The results of the pilot study suggested that the end-user was not engaging strongly in the design process. In particular, the end-user made very few comments about the proposed design. Although, there were comments made by the end-user but they were more in form of agreements and accepting the design proposal the way it was presented. Also it took more time than it was expected for the end-user to send her feedback to the designer-users. Not being a real customer could mean that the end-user did not care to make enough effort to relate to the task and become more creative and involved in exploring the requirements. However, the end-user found that the given time to review the design and give feed back was not sufficient for understanding and giving feedback on the design proposal and answering designer-users questions. The time allocated to end-user was 10 minutes for each iteration but it was exceeded every time and once up to 20 minutes.

7.5. REVISED

Difficulties of using a real end-user were explained in the previous section. Therefore, it was decided that the author play the end-user for the later exercises. Since the author had already worked with both design tasks and was involved in generating them, thus he has a more in depth insight to the given tasks and better understanding of the problem. This artificial customer was intended to offer a better simulation of end-user.

For the remaining three exercises, six participants were recruited and trained individually. Six participants were divided into three pairs of ‘designer-users’. The first pair consisted of two members of staff of the School of Computer Science who, were experienced in web design. The remaining two pairs were final year undergraduate students of a Software Engineering Degree, who were returning from industrial placement and were experienced with designing websites. Each group was asked to design a ‘visit Sheffield’ site using DENIM and a ‘research group’ website using Gabbeh (See Appendix B for the design tasks). The same order of tasks was maintained throughout the study.

Amir M Naghsh 113

7.6. RESULTS

This section reports on the result of the design exercises which have also been reported and presented at the DSVIS 2005 conference (Naghsh et al, 2005).

7.6.1. WERE COMMENTS USED?

The results showed that all participants used the commentary feature of Gabbeh. Also, the content of emails was reduced to a few lines informing the end-user of a new update of the design and in some cases informing the end-user of which comments they needed to check within the design. For example, one of the emails posted by designer-users was only a few lines:

“We have made the amendments....We were slightly unsure about couple of points. Please read the comment in red”.

The same designer-users sent longer emails when using DENIM:

Just need to clear up a few issues.

1. We dont understand point 5 of your requirements

Can you please explain this?

2. Can you please provide us with more details about requirements 4 and 5 on the original "Website Design Task Brief". We have provided a page with a list of group projects. We also provided links to these projects from researchers pages. Does this meet the requirement?

To investigate the use of comments it was required to calculate the number of comments created in each session. To calculate the number of comments created in Gabbeh, the number of individual comments attached to the sheet was calculated. For example, Figure 7-4 shows five comments in Gabbeh. In addition, to calculate the number of comments attached to emails, the content of each email was broken down based on the subject to identify individual comments. For example, the above email which was sent when using DENIM was divided into four comments as follows:

• We dont understand point 5 of your requirements

• Can you please provide us with more details about requirements 4 and 5 on the original "Website Design Task Brief"

Amir M Naghsh 114

• We have provided a page with a list of group projects

• We also provided links to these projects from researchers pages. Does this meet the requirement?

Sessions (Gabbeh – DENIM)

Group1 Group2 Group3 Total of groups

Gabbeh: End-User 10 6 6 22

Gabbeh: Designer-Users 14 10 11 35

Gabbeh: 24 16 17 57

DENIM: End-User 7 4 6 17

DENIM: Designer-Users 4 1 3 8

DENIM: 11 5 9 25

Total (of one group) 35 21 26 82

Table 7-1 This table presents the number of comments created for each session as well as presenting the overall comments

made while using Gabbeh in comparison to DENIM

The results presented in Table 7-1 provide a clear answer to the first question that commentary tool is likely to be used when it is available (also see Figure 7-4). The results in Table 7-1 shows that designer-users made four times more comments when using Gabbeh compared to DENIM.

Amir M Naghsh 115

Figure 7-4 The comments were used in all exercises. This figure is demonstrating the use of comments as Gabbeh was

used to develop a prototype (Group 2, Session 2)

Reviewing video recordings revealed that the Gabbeh commenting facility allowed designer-users to create comments once they recognised the need for it during the design process. In DENIM, designer-users did not keep any record of comments during the design process, and they would wait until the design reached an appropriate point to send to the end-user and write the comments in an email. It was observed that at the time of sending the email it was not an easy task for designer-users to remember all the design issues during the design. For example after the first version of design was ready by group 1, one of the designers started typing an email to the end-user and half-way through the email he asked the other designer to help him to remember the design issues that they had already discussed and to confirm what they needed to decide to send to the end-user.

The results showed that although designer-users created more comments using Gabbeh, it was not the case that the commentary tool was always used when it might have been appropriate to do so. For example, analysis of the video recordings revealed the following discussion:

Designer 1: … that would link to a research page which is then going to have all the researchers name, I think that’s what they say at the moment. Does this

Amir M Naghsh 116

make sense, what do you recommend? Do you want to take a different approach?

Designer 2: I would go a different way to represent this, the only thing that I think of is to have all the researchers name on the home page … … But put it like this and we will see what they (refereeing to user) going to say. We are going to get some feed back anyways. … (group 1, session 1)

In this case, the designer-users went ahead without adding any comments, but assumed that the user would respond if they were particularly dissatisfied. However, when working with DENIM, one of the same two designer-users was heard saying: “Now we need a commenting tool to ask user what they think [referring to a specific part of design]”.

7.6.2. WERE DISCUSSIONS SUPPORTED?

To answer this second question all the comments within emails and the prototypes were reviewed. As explained in chapter 6, each comment has a reference ID (see Figure 7-5). This ID is an automated number, which is generated in Gabbeh when each comment is created. The larger ID number means that comment was created later in design process, which helped to identify the possible threads.

In addition, to considering the context of comments for each page, the recorded videos were also reviewed to learn about the time and situation that the comment was created. This made it possible to identify threads of comments in the form of discussion. Results presented in Table 7-2 shows that there were six designer-initiated discussions made in Gabbeh (see Figure 7-5 and Figure 7-6) and two in DENIM. For example, the following thread of comments was made in group 2, session 2 when participants were using Gabbeh. The thread consists of three comments starting with the designer-users:

Designer-users: “We provided a photograph of each researcher together with links to their profile and a quick overview of their interests at here”

User “can we have a group pic instead of individual ones here?”

Designer-users: “We provided an area for some text about the group as well as a group photograph.”

Amir M Naghsh 117

Figure 7-5 This thread of two was identified in the design developed by the group 2. The designer-users have changed part of the design in response to users comment. Here the

design-users are explaining the new design which receives the user’s approval

Figure 7-6 This thread was identified in the design from the pilot study. The designer-users explain the design while

requesting the user’s approval

Sessions (Gabbeh – DENIM) Group1 Group2 Group3 Total of groups

Gabbeh: End-User 2 2 3 7

Gabbeh: Designer-Users 2 2 2 6

Gabbeh: 4 4 5 13

DENIM: End-User 2 1 1 4

DENIM: Designer-Users 1 0 1 2

DENIM: 3 1 2 6

Total (of one group) 7 5 7 19

Table 7-2 List of identified discussions for each session divided in two main groups of Designer-initiated or User-

initiated discussions

Amir M Naghsh 118

Reviewing the video recording and the results of the analysis presented in table Table 7-2 clearly answered the second question and showed that the comment tool was used to generate distinct discussions where one or more comments were made in response to another. In addition, designer-users used the colour scheme to separate their comments from the end-user comments. For example group 1 used red colour for all responses whilst group 2 used different colours (but not the same as the end-user) for different iterations.

7.6.3. HOW WERE COMMENTS USED?

To answer this question each individual comment was identified and the recorded videos were reviewed to retrieve the time that the comment was made, the author(s) of the comment and the key points of the discussion that was happening at the time of the creation of the comment. The recorded videos were also used to break comment objects, that had multi sentences, down into smaller comment-phrases which represented a single specific issue.

Each individual comment was printed out on a piece of paper and all the information related to that comment such as the time, author and other key points was written on that paper. This was done to allow comments to be arranged in different ways. A big surface was used to display all the comments. The comments were then clustered to identify categories of interest. The instructive comments such as “please ignore the first comment” were not considered in this clustering since they were used to overcome some of the functional limitations of Gabbeh.

7.6.3.1. CLARIFYING

The results showed that a number of comments were made particularly to explain the design. The designer-users made comments to explain the functionality or the dynamic behaviour of a part of design. This avoids the designer-users developing the design in detail, whilst indicating how the design may be developed. Figure 7-7 shows an example of this category. The two comments in brown and blue are used to clarify the design. The brown comment is simply explaining what the end-user should be expecting from this page:

“This page holds an image of the researcher and their interests in more detail, also included on this page is a list of their interests and current work. The courses that they teach on are also included here “

While the blue comment, is mainly explaining the functionality of a specific part of the design that is indicated by an arrow pointing to that specific area of design. Designer-

Amir M Naghsh 119

users instead of designing a new page have used the comment to explain what the end-user should expect to see after clicking on the link.

“Link will grab all publications by name and year supplied as an attribute of the link”

Figure 7-7 Examples of clarifying comments (group 2, session 2). The blue comment is clarifying the design. Designer-users have made comment to explain what would happen once the user clicked on one of the items in the publication lists. It has

allowed designer-users to avoid designing a new page to simulate the dynamic behaviour and its functional output in the

form of a new page.

7.6.3.2. VERIFYING

The results showed that in addition to the use of comment for clarifying the design, designer-users made comments to report on specific parts of a design and request for end-user verification. Also they might ask for the confirmation to changes that have been done based on a previously created comment by the end-user.

The results suggested that verifying comments are concentrated on specific issues which require end-user feedback. Figure 7-8 shows an example of a verifying comment. The end-user has requested to have link to publication and be able to view them.

“End-User: Have a link to website and the location of publication, to have view of the paper(s).”

Amir M Naghsh 120

In response, the designer-user have changed the design to include what the end-user has requested and made a comment highlighting the change as well as seeking the end-user’s confirmation on the change.

“Designer-User: Here we provide links to each paper, is this what you wanted?”

Figure 7-8 An example of verifying comments as part of a thread

7.6.3.3. EXPLORING

Designer-users used comments to ask questions that would help them to obtain more detail about the end-user’s needs or desires. They used comments to explore the design. Exploring comments were made to obtain feedback on design on a higher level or when designer-users could not understand the problem to propose any solution for it, whilst verifying comments were mostly used to gain users confirmation on a specific part of the design. For example designer-users who could not decide on what they have to design for a given requirement, when they sent the first version of their work, they also asked the end-user in the email:

“Can you please provide us with more details about requirements 4?”

Comments on clarifying, verifying and exploring categories were all made by the designer-users.

Amir M Naghsh 121

7.6.3.4. ALTERING

Observation showed that the end-user made comments to alter the design through the design process. Altering includes comments on a change or correction of the existing design or an instruction to include a new element in the design. Figure 7-9 shows a comment made by the end-user requesting the designer-users to add a new page to the design. The red comment shows the end-user confirming the need for having a list of publication but wants a separate page to display the publications.

Figure 7-9 The end-user requesting a new page to be added - An example of altering the design

7.6.3.5. CONFIRMING

Feedback provided by end-users approving or disapproving of aspects of the design. It was mainly used in response to verifying comments made by designer-users. The yellow comment in Figure 7-8 is a response to the red comment by design-users requesting end-user feedback. The confirmation is a short “yes, that’s what I wanted” comment.

7.6.3.6. UNDERSTANDING

The results showed that the end-user can not always make sense of a design proposal sent to him/her. Since there could be a lack of clarifying comments about different part of design, end-user can often find it difficult to understand or relate the design to a problem. Therefore, comments were used in form of questions to help end-users to understand the design better to relate it to his/her needs.

The red comment in Figure 7-10 shows an understanding comment. The first line of the comment is simply asking the designer-users what this box with the cross in it is. After investigating the creation time for this comment and other comments that were created afterwards, the designer-users responded by adding a comment (comment ID

Amir M Naghsh 122

32) referring to the box as “pic/graphic” as well writing the word “picture” on the box itself.

Figure 7-10 Red comment shows an example for understanding comments helping the end-user to understand

the provisional part of design

Altering, confirming and understanding comments were made by end-users.

7.6.3.7. ORGANISATION

In addition to above categories, a few comments, three in total were used to discuss the organisation of the design process itself. For example, designer-users created a comment to instruct the end-user what to check in the new version of their design:

Designer-user: “Please read the comment in red.”

7.6.3.8. OVERVIEW OF COMMENTS IN EACH CATEGORY

Number of comments in each category is shown in Table 7-3 and Table 7-4. (see appendix D for all the comments in each category).

Designer-users End-Users

Clarifying Verifying Exploring Altering Confirming Understanding

DENIM 1 2 4 13 1 3

Gabbeh 22 10 3 15 3 3

Total 23 12 7 28 4 6

Table 7-3 Total number of comment-phrases associated to each category for the three evaluation studies

Amir M Naghsh 123

Designer-users End-Users

Clarifying Verifying Exploring Altering Confirming Understanding

DENIM 1 1 2 2 1 2

Gabbeh 5 4 1 4 4 1

Total 6 5 3 6 5 3

Table 7-4 Number of comment-phrases associated to each category for the pilot study

7.7. DISCUSSION

Analysis of the video recordings and debriefing sessions showed a number of concerns with the current design of Gabbeh. Three important issues are considered below.

7.7.1. MANAGING LARGE NUMBERS OF COMMENTS

As a prototype evolves, the number of annotations and comments attached to it also increases and it becomes more difficult to manage them. Observations showed that of the four groups of designers-users only two groups used the filtering tool provided. One group of designer-users reported that they found it difficult to deal with the positioning of the comments in the ‘design-mode’. Instead, they preferred to have all the comments shown at one place. To overcome the problem these designer-users made all the comments invisible on the design sheet and instead used the ‘run-mode’ browser to view the comments for each page, before returning to the design mode to continue their work.

In the debriefing discussion following the exercises designer-users explained they used the “run mode” to view comments because they could identify the sequence of comments by going through comment IDs. The review of video recording revealed that all three groups of designer-users used the ‘run mode’ to view comments after the first iteration, once they had received the end-user comments.

7.7.2. IDENTIFYING AND MANAGING DISCUSSIONS

Analysis of the video tapes and of the comments made, showed that there were distinct discussions in which one or more comments is made in response to another. However, it was difficult to identify these threads without referring back to the video recordings

Amir M Naghsh 124

to clarify the relationships between the comments. Clearly, there is a requirement to identify comments as replies to previous comments. The colour scheme of comments was used in a variety of ways, particularly for separating designer comments from end-user comments. One group of designers used different colours for different iterations of the design. Thus, their second email to the end-user consisted of the single instruction: “Please read the comment in red”.

7.7.3. SUPPORTING PROGRESS AWARENESS AND TRACKING

One problem that became apparent after a number of iterations was the need to develop some form of version control over the prototype and the comments. For example, an end-user might make a comment requesting a change, which is then implemented by the designer-users. If the comment is not removed from the design record, then it may not refer to anything in the current design. Clearly, there ought to be some way of managing the history of the comment to show that it has been addressed.

This suggests that each comment could be regarded as having a life-cycle starting with its creation. This life-cycle might be different for comments from different categories. Understanding such life-cycles will be an area for further work.

7.8. CONCLUSION

The exercise showed that the comment facility is used once it is available to designer-users. It also has suggested a classification for the comments made in Gabbeh and DENIM. The evaluation study has suggested that comments created in designing interactive systems such as website could be divided into seven categories as follows. Designers-users made comments for:

• clarifying the design detail;

• exploring unspecified and not well understood requirements;

• verifying the parts that were already designed.

End-users used comments for:

• understanding by asking questions about the design;

• altering the design;

• confirming the changes in the design.

Amir M Naghsh 125

Additionally, some comments were used to discuss the organisation of the design process itself.

The evaluation identified that it is not possible to discover to what extent Gabbeh could enhance the communication among of stakeholders. To be able to make strong conclusions on the use of discussions through annotations in designing interactive systems, the need for a longitude study with involvement of a real end-user was recognised. However, the evaluation suggested that this version of Gabbeh requires enhancement in important areas of the management of large collections of comments, identifying discussions and understanding the life-cycle of comments within a larger design process. The following chapter reports on further design studies and investigations on enhancing Gabbeh to meet some of the findings mentioned in this chapter.

Amir M Naghsh 126

Chapter 8. SUPPORTING DISCUSSION IN GABBEH

8.1. INTRODUCTION

As explained previously, Gabbeh was developed to discover to what extent an electronic paper prototyping tool with commenting facility could enhance communication and collaboration among different stakeholders. The formative evaluation of Gabbeh version 1 showed that the new comment facility was used once it was made available to users and designers. However, the observational studies revealed the difficulties involved in simulating the design task and recognised the need for a longitudinal evaluation study.

Particularly, the observational studies were found insufficient in order to discover to what extent the commenting facility could have been used to enhance communications through supporting dialogues and discussions between dispersed stakeholders. The analysis of observational studies identified three main areas in Gabbeh which made it difficult to be used in a larger design task with many more iterations.

• progress awareness,

• identifying and managing discussions (supporting discussions),

• managing large numbers of comments.

To maintain the reliability of Gabbeh through the evolutionary process, it was essential to retain the number of changes to its minimum whilst improving the prototype. This chapter explains the Gabbeh version 2 that was used in the longitudinal study. Although, there are other areas of interest such as usability issues and features that the researcher would liked to incorporate into Gabbeh, due to the time and resources limitation a decision was made to not consider them at this stage.

Amir M Naghsh 127

8.2. ENHANCING THE DESIGN OF THE COMMENTING TOOL

8.2.1. PROGRESS AWARENESS

Designing interactive systems such as websites is an iterative process allowing a prototype to evolve through many iterations during the design process. The observational studies showed that the number of comments associated to a design grow large in just one or two iterations. Such a growth would be even bigger when the size of the design and the number of stakeholders increases. This means stakeholders need to deal with a large number of comments whether created by them or in response to them to be able to track the progress of the design.

Version control was identified as one essential requirement which has to be supported. There are a number of researchers (Appan et al., 2005; Bottoni et al., 2004; Brush, 2002; Cadiz et al., 2000; Jung et al., 2002; Leland et al., 1988; and Weng et al., 2004) investigating the version control for systems supporting asynchronous annotations in dispersed settings. To enable the stakeholders track the design process, they need to be aware of the comments created in different iterations. They are also required to be aware of how their comments were received by other stakeholders, if there is any response to their comments or if their comments have been incorporated into the design.

Therefore, it is essential to provide additional information associated to a comment to be able to identify when and at what iteration a comment has been created, who has created the comment, and if it has been addressed or responded to. It should be possible to distinguish new comments from the old comments. The progress awareness should allow a stakeholder to track comments to identify whether there are responses to a comment or if it has been incorporated into the design or it has not been dealt with yet.

The observational studies also indicated that designer-users and end-users needed to be able to associate comments that were created in the run mode to a specific position on the page. The stakeholders needed to map the comment to a location in order to direct other stakeholders’ attention to a specific part of the design that they were providing feedback on, or a part of the design that had been changed.

8.2.2. SUPPORTING DISCUSSIONS

As explained in chapter 5, the conducted studies in supporting annotation, for example: Quilt (Fish et al. 1988); MADCOW (Bottoni et al. 2004); Space Pen (Harmon et al.

Amir M Naghsh 128

1996) and AnnoteImage (Brinkley et al. 2002) have paid little attention to supporting discussions through annotations.

However, the results of the evaluation study in chapter 7 identified threads of comments in the form of distinct discussions. However, it was very difficult to identify these discussions and threads since the first version of Gabbeh did not allow for a comment to be associated to another comment. Therefore, video recordings were used to clarify the relationships between the comments. For example Figure 8-1 shows a collection of comments created by designers and users associated to one page in a design model from the evaluation studies. Reviewing the video recording revealed that there is a relationship between comments 5, 25 and 28 in the form of a discussion. The investigation showed that end-user created comment 25 in response to designers’ comment 5, and comment 28 was created by designers as a reply to comment 25.

Figure 8-1 A discussion example from observational studies

A commenting tool should allow stakeholders to create comments as replies to other comments. Two such comments should be linked to each other to allow the stakeholders to identify the relationship between them in later stages of the design.

Amir M Naghsh 129

In addition, it was noticed occasionally that responses to a comment consisted of few words of agreeing or disagreeing. Comment “Yes, that’s what I wanted” in Figure 8-2 is one example of using a short comment to approve the design. Therefore the need for a simple way of supporting or disagreeing with a comment was recognised to allow stakeholders to respond to a comment without the need of creating a new comment. The design of the tool should allow each stakeholder to agree or disagree with any comment and further it should be able to determine the number of agree and disagree votes for an interested comment.

Figure 8-2 Short comments were used to approve the design


The number of comments and annotations associated to a prototype increases rapidly as the prototype evolves and therefore it becomes difficult to manage the comments. The design of an electronic paper prototyping tool such as Gabbeh should enable stakeholders to retrieve comments which share some common properties during the design process.

The comment’s properties can be explained in the form of additional data associated to comments. For example, the name(s) of the person who has made the comment, the date and time that comment was created at, the iteration that comment was created in and the colour of comment can be considered as some of the properties that can be associated to comments. For instance, in one of the previous observational studies, different stakeholders used different colours to distinguish between each other comments during the design process.

Therefore, a more featured filtering tool is required to allow the stakeholders to retrieve comments based on one or a collection of common properties. The use of such a filtering tool is strongly influenced by its supported features as well as the effectiveness of its user interface. However, as explained earlier, issues regarding the usability of the filtering tool is out of this work’s scope.

Amir M Naghsh 130

8.3. GABBEH VERSION 2

The observation of the evaluation studies indicated that designer-users were more comfortable with the positioning of the comments in the run mode, and the run mode was used more often to review the comments in one screen whilst the other screen (graphic tablet) was used to alter the design. Since all of the designer-users were interacting with two screens at the time, one to alter the design (design view) and one to interact with comments (run view), it was decided to design Gabbeh version 2 by assuming that designers will use two screens at the same time. The design mode is displayed on the graphic tablet and the run mode is displayed in the standard screen allowing designers and users to interact with comments. This prevents the design view from getting crowded with a large number of comments which would cover the actual prototype.

Furthermore, this section explains the further development of Gabbeh and describes the implementation of essential design criteria to enable Gabbeh to facilitate discussions in the run view.


8.3.1.1. NAME DIALOGUE BOX

To distinguish comments from different stakeholders a new feature is introduced into Gabbeh to allow a user to enter his/her name. Every time Gabbeh starts up an input dialogue box will pop up requesting the user to input a name (see Figure 8-3). The current version of Gabbeh does not check if it is a returning user or not and the users should make sure to use same user name every time using the Gabbeh. The provided name by the users will be associated with the comments that are created by that user.

At this stage of the development more attention has been given to adding the necessary features into Gabbeh in the simplest and quickest way instead of exploring the best suitable design for such features.

Amir M Naghsh 131

Figure 8-3 The dialogue box for allowing the users to provide a name

8.3.1.2. REVISION

The structure of Gabbeh version 1 does not support a client-server application that can support dispersed stakeholders either to allow them to connect to a server and work with a shared version of the prototype model, or to be able to connect to the server to upload their version that can be merged into the original version available on the server. Therefore it was assumed that stakeholders would take turn to use the Gabbeh and modify the prototype. A simple version counter is also provided into Gabbeh to enable the stakeholders to identify at what stage in the design a comment was created.

Every time Gabbeh is started up, a dialogue box is displayed to confirm the correct revision. It is also possible to change the revision number through the pie menu. The revision number allows the users to decide if a new iteration or revision has started and will change the revision number once the prototype is loaded into the Gabbeh before altering the design. This new revision number will be associated with all the generated comments by the user at that stage.

The usage of the revision option is left open in this version of Gabbeh to allow the stakeholders to adapt it to their way of working.

Amir M Naghsh 132

8.3.1.3. MAPPING COMMENTS IN THE RUN MODE

The observational study indicated the importance of being able to associate comments to a point within a page in run mode. Further development of Gabbeh allows users to select a comment and then associate the comment to a position on the page.

To associate a comment to a point on the page, first a comment has to be selected. This can be an existing comment or a new comment that just been created using the add comment button as explained in chapter 6. Once the comment is selected, then the link option in the tool bar should be selected and enabled. When the link button is set to enabled, the user can use the mouse cursor or the pen on the graphic tablet to indicate the position that comment has to be associated with. After selecting the position on the page, a coloured number same as the comment ID will be displayed at that position (see Figure 8-4).

Figure 8-4 The new associated point to a comments is highlighted on the page

Figure 8-4 shows a selected comment as part of a discussion with the comment id “9”. The comment is associated to an area on the page which is highlighted by the coloured

Amir M Naghsh 133

number. However, since the comment also belongs to a discussion, the areas associated to the first comment that has initiated the discussion are displayed too. In this case it is the areas highlighted by “2” associated to the first comment in the discussion with the red colour. This is intended to allow stakeholders to identify where initially the discussion related by clicking on any comments belonging to the discussion.

8.3.2. SUPPORTING DISCUSSIONS

8.3.2.1. SELECTING A COMMENT

A click of the mouse on the comment selects a comment. When a comment is selected, it becomes highlighted with a blue border. In addition, the comment id, author name, revision number and the date that the comment was created become visible at the top of the comment (see Figure 8-6 and Figure 8-5).

The location marks associated to all comments are displayed when no comment is selected in the run view (see Figure 8-5). However, as it is shown in Figure 8-6, when a comment is selected, only marks associated to that comment are displayed on the page. This allows the stakeholders to identify which part of the design the comment is referring to. The design of the CommentMark in the second version of Gabbeh consists of the comment id surrounded with a coloured line representing the comment colour. An alternative design could include coloured numbers with different sizes and maybe animated.

Figure 8-5 Comments displayed in the run-view when no comment is selected.

Amir M Naghsh 134

Figure 8-6 Displaying a comment when it is selected

8.3.2.2. REPLY TO COMMENTS / EDIT COMMENTS

Double clicking on any comment would open a comment dialogue box which allows a user to either edit the comment or reply to the comment (see Figure 8-7). When a user double clicks on a comment, two types of information associated to the comment are used to compute whether the user can edit the comment or reply to it. The two types of information are: the name of the person who has created the comment and the revision number of the comment.

When a user double clicks on a comment, if it has been created by him/herself within the same revision then it is possible for him/her to edit the comment (see Figure 8-7). Otherwise, if the comments has been created by someone else or it has been created in an older revision then the dialogue box would be used to allow the user to reply to the comment. This allows a user to use same functionality to edit his/her recent comments whilst being able to use it to reply to other users’ comments.

Figure 8-7 Editing an existing comment

Amir M Naghsh 135

The comment created as a reply will be displayed under the source comment within a grey background to indicate that this comment was a response to source comment (see Figure 8-8).

Figure 8-8 comment inserted as a reply to an existing comment

8.3.2.3. INTERACTING WITH DISCUSSIONS

To decide on how to relate the comments in the form of a discussion in Gabbeh, community forum systems which support discussions such as vBulletin3 were considered. Consequently, a tree hierarchy model similar to that commonly used in community forums was selected.

Related comments in the form of replies are grouped together with the first comment at the top and the rest of the replies grouped together under the first comment with a grey background. This group of comments is called a discussion in this thesis. A discussion consists of at least two comments. The first comment in the discussion, the one which has initiated the discussion is always visible at the top of the group.

Two views are supported by Gabbeh for a discussion. In the closed view, the first and latest comments are displayed. Figure 8-9 shows the closed view of a discussion. The “+” sign next to the first comment in a discussion indicate the closed view for a discussion can be expanded.

Figure 8-9 Closed view of a discussion

3 See http://www.vbulletin.com/

Amir M Naghsh 136

To expand the discussion body, the user can click on the “+” sign and open the discussion. Figure 8-10 shows an expanded view for a discussion with the rest of the comments between first and last comment. The “+” sign changes to “-” to indicate the expanded view for the discussion. To close the discussion back, the user has to click on the “-”.

Figure 8-10 Expanded view of a discussion


8.3.3.1. FLAG COMMENTS

It was noted that the commenting tool should allow each stakeholder to agree or disagree with a comment. In addition, as explained earlier, the current structure of Gabbeh does not allow integration of progress awareness to enable stakeholders to track the design process and indicate if their comments have been responded to.

Due to time and resource constraints which have made it not possible to implement the whole proposed design, it was decided to provide a flag option in this version of Gabbeh to allow the stakeholders to highlight a comment when it is required. For example, if a stakeholder wants to show his/her support towards a comment, or

Amir M Naghsh 137

indicate that a comment as not read. The usage of the flag option was left open to user to adapt with their way of work.

To flag a comment, the user can right click on the comment. Once a comment is flagged, the letter “F” in colour red is displayed next to it (see Figure 8-11). It is possible to set or clear the flag for any comment created at any stage of design regardless of who has been its author.

Figure 8-11 A comment with flag

1.4.1.1. FILTERING TOOL

A more featured filtering tool than the one available in the first version of Gabbeh is supported to allow stakeholders to retrieve comments based on the colour, name, flag and revision number. Gabbeh set the default value of the version to current and colours to all. The “current” value for the revision as it can be seen in Figure 8-11 only retrieve and display comments and discussions created or updated in the current revision or the one before it. This way Gabbeh displays the most recent comments and discussions. However, the user can click on a version button to change the revision value to blank which means all comments and discussions will be displayed. Also comments created in specific revisions can be retrieved by entering the revision number into the revision box then clicking on the update button which is indicated with a face icon (see Figure 8-12). Revision numbers can be entered as a collection of numbers such as “2,4,7” or in the form of “3-6” meaning all revisions from 3 to 6.

Figure 8-12 filtering toolbar

Gabbeh allows comments to be filtered and retrieved by its creator as well. Stakeholder name can be entered into the name text box and then click on the update button. In

Amir M Naghsh 138

addition, comments can be filtered based on their colours. When the colours option is selected in the filtering toolbar, it means that all the colours will be displayed. To select specific colours, the colours check box should be deselected. This will enable the other colour checkbox and allow the user select the interested ones.

An option is also supported to allow the users to filter the comments with a flag. An extra option provided is the “all” button, that when it is selected, all the created comments associated to the design model will be displayed into the comment panel. This is intended to allow the stakeholders to view all comments in the run view.

8.4. CONCLUSION

This chapter has explained the further implementation of Gabbeh mainly to improve Gabbeh to be used as part of a longitudinal evaluation study. This chapter has extended the previously introduced design of Gabbeh further to reflect some of the required enhancement to the design of Gabbeh in three main areas of identifying and managing discussions, progress awareness and managing large numbers of comments.

Whilst the time and resource constraints were the main issues, only some of the enhancements, essential to support discussions have been incorporated into the second version of Gabbeh. However, it is important to emphasise that features such as being able to support comments, having a voting mechanism to facilitate decision making and other identified design issues in Chapter 6, as well as usability issues of these designs, are important and should be implemented once the resources are available but are not essential at this stage.

Amir M Naghsh 139

Chapter 9. LONGITUDINAL STUDY

9.1. INTRODUCTION

As explained previously, the observational studies revealed the difficulties involved in simulating the design task and in discovering to what extent the commenting facility could be used such that the need for a longitudinal evaluation study was recognised.

This chapter reports on using Gabbeh version 2 in a longitudinal case-study in a dispersed setting. This chapter presents the outcome of the case-study and discusses the results of the evaluation.

9.2. OBJECTIVES

The main objective in developing Gabbeh was to build a quick prototype to discover to what extent an electronic paper prototyping tool could facilitate asynchronous communication among distributed stakeholders at early stages of design.

Previous evaluation studies suggested that stakeholders would use the comment facility for communicating ideas in the form of a discussion. However, it was not possible to reach strong conclusions based on the results of the observational studies because of difficulties involved in simulating a real design task.

In addition, the analysis of observational studies identified areas of Gabbeh which needed further improvement to allow Gabbeh to be used in a larger design task with many more iterations. To enable Gabbeh to be used in a longitudinal study Gabbeh was developed further as explained in Chapter 8 to facilitate communication by enabling users and designers to manage comments and discussions.

Once the Gabbeh version 2 was available, it was possible to conduct a longitudinal study with three main objectives identified to extend previous studies.

The first objective was to discover if participants would use the enhanced comment facility to communicate design ideas in a dispersed setting in form of a discussion.

Amir M Naghsh 140

The second objective was to explore how such discussions were used in the design process.

The third objective was to investigate if it was possible to categorise comments using the suggested classification developed in Chapter 7 for comments.

9.3. LAYOUT

The longitudinal study involved two participants. One of the participants was the potential end-user, a 30 year old Chemical Engineer who had used software systems such as Microsoft Office and internet explorer on a day-to-day basis. The other participant was a professional designer, a 27 year old website designer, with BA and MA qualifications in product and user interface design. The end-user was located in Sheffield whilst the designer was located in Cardiff at the time of the evaluation.

At the time of the study, the user was looking for some consultancy for an idea of an e-commerce web site to design and sell t-shirts online. Since the results of the observational study indicated the importance of using real life design tasks, it was decided to conduct the longitudinal evaluation by using the t-shirt website idea as the case-study. Using the t-shirt idea had two main advantages. Firstly it would encourage the end-user to be more involved in the design process since the user was a direct beneficiary of the study. Secondly, the possibility of developing the case-study into a live website would also increase the designer’s interests in the case-study.

The end-user was asked to provide a document explaining the t-shirt idea and all the features which the end-user was expecting to have in such a website. The end-user provided one page (A4 size) explaining the idea, explaining some motivation and giving an abstract idea of what was expecting of the website (see Appendix C).

The end-user received approximately one hour’s training on how to use the Gabbeh in the run mode. Also, the run mode of Gabbeh was installed on the end-user’s portable computer and provided with a couple of prototypes generated by Gabbeh from previous design exercises to allow the end-user to become familiar himself the tool.

The designer also received one hour’s training on how to use Gabbeh and the design-mode of Gabbeh was installed on the designer’s computer. The designer-user was provided with a 15” Graphic Tablet in addition to the portable. As explained in chapter 8, Gabbeh version 2 was enhanced by using two screens, the graphic tablet screen to allow the designer to sketch the web pages, and the other regular screen (the portable

Amir M Naghsh 141

computer’s) to allow the designer to review the design as well as annotating it and managing the discussions.

The user and designer had met prior to the study at a couple of social gatherings and had some basic familiarity with each other. However, it was explained to both of them that they were not supposed to have any contact with each other either by telephone or face to face, and the only permitted communication channel was through exchanges of emails and exchanges of Gabbeh models.

Once the designer and user were comfortable in using Gabbeh the requirement document created by the end-user was sent to the designer. The designer was reminded that the document had been created by the user and any queries should be directed to the end-user. As mentioned before, the designer was also reminded that there could be a possibility of further work to implement the actual website based on the achievements of the case-study.

In addition, the user and designer were informed that there was no deadline involved for finishing the case study. They were also briefed that there was no time limitation imposed to perform any one of the iterations and they could spend as much time as liked before sending the design to the other party.

Each of the participants was asked to forward any communication that they had with each other to the researcher. To reduce the chance of missing any part of this communication, a dummy email address was created for the user to communicate with the designer. The email account was also accessible by the researcher.

Both participants received an informal debriefing interview over the phone at the end of the case-study which was recorded on audio tape.

9.4. RESULTS

The case-study lasted 25 days starting from the date that the design task was sent to the designer. During the 25 days of the case-study the prototype was exchanged between the designer and user. The time that was spent by the designer or user to contribute to the design prior to each of the exchanges was considered as one session. In total, the design process consisted of 10 sessions where 5 of them were designer sessions and the other 5 were user sessions.

Amir M Naghsh 142

Each iteration in the design process was considered to be the designer and user contributions made through a designer session followed by a user session. Through each of the iterations an updated version of the design was produced. Since the design process consisted of 5 iterations, therefore 5 revisions of the prototype were generated. By end of the design process, the prototype had 10 pages (see appendix C).

To report the results of the case-study all 95 comments made by designer and user were imported into an spreadsheet with all the relevant information such as date, author, colour, version number, the comment id and the discussion number (if the comment was part of any discussion).

A discussion was considered to be a group of comments where each comment was a reply to another comment. In other words, either the designer or the user could reply to each others’ comments that had been made in prior sessions. Therefore, the discussion was considered to have consisted of at least two comments from two different stakeholders where the first solo comment was considered to be the initiator and the reply comment was considered to be the response. However, it was possible to reply to an individual comment many times in one session. Although each individual comment could have received many replies, each individual comment as reply could only be associated with one comment. Therefore the discussions would grow in the form of a tree, where an initiator comment could receive multiple responses and each response could receive further multiple responses and so on.

In total, the designer and user created 87 comments to generate 28 discussions attached to 9 pages of the design model. In addition, results indicated that only 8 comments from the total of 95 comments did not receive any response (see Table 9-1).

Discussion Comments in Discussions

Solo Comments

Total number of comments

Total 28 87 8 95

Table 9-1 Results for the comments associated to the whole design

Table 9-2 shows some results for each one of the 10 pages in the prototype. The first column shows the number of discussions associated with each page. The second

Amir M Naghsh 143

column shows the total number of comments that are part of a discussion for each page and the third column shows the number of solo comments for each of the pages.

Page No. of

discussions No. of comments in

discussions Solo

comment

home 4 20 0

t-shirt 5 13 1

design 6 18 1

design(2) 1 5 1

basket 1 2 3

new user 2 4 0

about us 6 18 1

news 0 0 0

contact us 2 5 1

feedback 1 2 0

total 28 87 8

Table 9-2 Number of discussions associated with each page of the design

The results indicated that each discussion was the result of comments which were generated in a number of designer and user sessions. A discussion consists of a number of comments that are created in different sessions; therefore a discussion is associated with a number of sessions whilst a comment is associated to only one session. Table 9-3 groups the discussions based on the number of sessions that are associated with it. For example, the last row shows that only 5 of the 28 discussions consisted of comments which were generated in 5 different sessions. The outcome of the evaluation

Amir M Naghsh 144

shows that about a quarter of the discussions were associated with at least four sessions or more.

No. of sessions No. of discussions

2 13

3 8

4 2

5 5

Table 9-3 Discussions grouped based on the number of sessions

In addition, the results also indicated the number of comments associated with each of the discussions. Table 9-4 shows that 8 of the discussions consisted of at least four or more comments. Furthermore, the results suggest that both user and designer used the comment facility to conduct conversations and discuss design issues throughout the design process.

No. of comments in discussion

No. of discussions

Initiated by the Designer

Initiated by the User

2 13 2 11

3 7 5 2

4 3 1 2

5 3 1 2

6 1 1 0

7 1 0 1

Total - 28 10 18

Table 9-4 Discussions grouped based on number of the comments involved

Amir M Naghsh 145

The results displayed in Table 9-4 show that most of the discussions were initiated by the user. 18 discussions were initiated by the user whilst only 10 discussions were initiated by the designer. However, it is important to note that 11 out of 18 discussions initiated by the user consisted of two comments created in two sessions. The majority of these discussions consisted of a comment created by the user requesting an alteration to the design and a response by the designer clarifying or acknowledging the changes. For example, the following comment by user is a request to an alternation by requesting more options to be added to the design, and the designer clarifies that such options will be added into the future design:

User: “Can you add an option for sending news emails to the user email address, for promoting our new designs or products?” (Session 4)

Designer: “yes it will be added” (Session 5)

In another example, the user has requested some alterations to the order of the menu bar and the designer has responded back by acknowledging that the user’s request has been incorporated into the design.

User: “I think the order should be like this: Home, t-shirt, Design, contact us, corporation, I suggest you to visit good websites such as amazon!” (Session 2)

Designer: “ok” (Session 3)

Results clearly showed that designer and user have both used the commenting facility to communicate ideas and to support discussions. Reviewed discussions suggest that both designer and user used the comment facility to respond to each others comments.

9.5. HOW WERE DISCUSSIONS USED?

Further analysis identified four particular ways that user and designer used discussions during the design process.


Analysis suggests that both the user and designer have used the comment facility and the discussions as a way of tracking the design progress and indicating the status of the design in each of the iterations.

Both participants stated in the debriefing interviews that on receiving the design model they would begin by reviewing the discussions and new comments to learn about the

Amir M Naghsh 146

design and then look into the design to find the issues that comments were referring to. For example the user stated that:

“I would check the [new] comments first, then I would check the design and if I was not happy I would look into the old comments and then make a new comment to explain what I was expecting to see.”

New comments created by each participant were treated as a way of tracking the design progress. The designer explained modification to the design by creating a new comment to inform the user of the changes. The designer mentioned that the comments were used to either inform the user that the requested changes were incorporated into the design or the design was modified and it required user’s feedback. In both cases the designer used the comments to notify the user of changes.

Figure 9-1 and Figure 9-2 display the page “design” from two different versions of the generated prototype. The page shown in Figure 9-1 was created as part of the first version and evolved through the design process to become what it would look like in Figure 9-2 in the last version. Through the process the designer and the user used the comments to instruct each other to what each of them needed to look for in the design. Figure 9-3 shows a discussion which took place during the sessions 2 to 4. The discussion was initiated by the user for the page “design” (see Figure 9-1). In the next session, the designer has modified the design to incorporate the user’s request. In addition to changes to the page as can be seen in Figure 9-2, the designer also has responded to the user’s comment by acknowledging the changes and instructing the user to review new parts of the design. Furthermore, the user has responded to confirm that new additions were reviewed and instructs the designer with more feedback that can be found in the relevant page.

Amir M Naghsh 147

Figure 9-1 Page "design", sent to user in iteration one

Figure 9-2 Page "design", sent to user in iteration 9, evolving through 8 iterations

Amir M Naghsh 148

Figure 9-3 A discussion that is representing comments from user and designer giving instruction to each other

The results showed that the designer also used the flag facility to support progress awareness by highlighting the comments which were not yet incorporated into the design. In session three, the designer indicated in a email that it was not possible to incorporate all the comments created by the user into the design.

Designer: “I didn’t have the time to go through all of your comments. I have flagged the ones which I have not done yet” (Session 3: Email)

Figure 9-4 shows a discussion which was initiated in session one and was ended in session eight. The first two comments in the discussion were created in session one and two. Then the designer flagged the second comment without making any changes to the design model. Later in the fifth session the designer came back to incorporate that issue into the design model and cleared the flag of the second comment and acknowledged the changes incorporated into the design by creating the third comment.

Figure 9-4 Showing a discussion, there has been a two iterations gap between second and third comment.

Amir M Naghsh 149

9.5.2. DISCUSSING DESIGN ISSUES

The analysis of the results and interviews showed that discussions were used to generate dialogues on design issues.

One of the ways that discussions were used was to allow designer and user to debate design problems. Particularly, discussions were used in the cases where they could not agree on a solution. Using the discussions allowed the designer and the user to express different points of view and reach a common ground towards a specific design issue. Being able to discuss design issues would allow the designer to first explore the design with the user before making too many changes to the design.

Figure 9-5 and Figure 9-6 show two examples where the designer and the user debated a design problem. In both cases they used the comments to explain their point of view while justifying it by directing each others attention to a specific part of design or to other external resources, such as existing websites.

Figure 9-5 shows a conversation between the designer and the user, discussing the use of the term “Have fun” on the front page of the prototype. The discussion was conducted by the user comment from the first session and was continued in the following two sessions with designer and user exchanging their points of view. The discussion allowed the designer to understand the user’s point of view and incorporate the changes into the design after discussing it in four sessions.

Figure 9-6 shows another example of the designer and the user debating a design problem. However, in this case it is the user who agrees with the designer by the end of the discussion. This is an example where the designer did not make any changes to the design through the sessions that they were debating, and only made one final modification once the a decision was made by the end of the discussion.

Amir M Naghsh 150

Figure 9-5 An example of debating a design issue

Figure 9-6 Another example of debating a design issue

Amir M Naghsh 151

9.5.3. DIRECTING ATTENTION

Whilst discussions were used by the designer and the user to direct each other’s attention when debating design issues (as explained in the previous section), they were also used to direct the user’s attention to a specific design issue, in order to encourage the user to provide feedback on the relevant issue. For example Figure 9-7 shows a discussion initiated by the designer clarifying a part of the design whilst exploring the design further by asking for the user’s feedback. The analysis of the interviews suggested that being able to preview the design while going through the discussions helped the user to have a better understanding of the design and therefore could provide instructive feedback. The user stated in the debriefing interview that:

“I could see how the design was and then relate the comments to the design….. … .. Previewing the design and reading the comments helped a lot in imagining how the final thing would be.”

Figure 9-7 User’s feedback

The results also indicated that discussions were used to allow the designer to provide information about design issues. The discussions allowed the designer to clarify the design where the user’s responses indicated that the user had not understood a specific part of the design. Figure 9-8 shows a discussion where the designer is clarifying the design without the need to incorporate the new changes into the design. The responses created by the user in Figure 9-8 were already reflected into the design and designer had considered them prior to the user’s latest feedback. The discussion then had been used to allow the designer to direct user’s attention to a specific part of the design in order to help him understand the issue.

Amir M Naghsh 152

Figure 9-8 Clarifying design through discussion

9.5.4. RECORDING DESIGN ISSUES

The analysis suggests that both the user and designer used the discussions as a source of information associated with the design. Reviewing recorded interviews showed that the designer and user would check their old comments before making new comments. The users stated:

User: “I would look back at [the designer] and my old comments to see what I have said before. If I was using emails I could not go back to check other old emails every time, but this way it was easy to go back to check what we were talking about.”

The analysis also showed that discussions were used to extend the design model by recording design issues that had not been possible to incorporate into the design model at this stage. Figure 9-9 shows a discussion which expresses a part of the design which is not possible to implement at this stage. The discussion allows the designer to explain to the user that such a feature would exist in the finished system. Further when implementing the actual website, these comments, in addition to the design model, can be used to demonstrate how the finished system will be.

Amir M Naghsh 153

Figure 9-9 A discussion, recording a design issue

Figure 9-10 shows that discussions allow the designer and user to communicate technical issues related to a part of the design and make decisions during the design process to be retrieved at later stages.

Figure 9-10 A discussion on technical issues

9.6. REVIEWING AND REFLECTING THE COMMENT CLASSIFICATION

To extend the identified classification for comment from Chapter 7, it was important to apply the classification for all the comments associated with the case-study and identify any comment which did not fit into the previously identified categories. In Chapter 7, seven categories were identified based on the analysis of the observational studies.

To test the classification, all the comments created by the user and designer were printed out on the paper to allow a better way of clustering them. Since many of the comments created by participants were multi-sentence comments, some of them were broken down into comment phrases. A comment phrase represents only one specific

Amir M Naghsh 154

issue. It can include all the sentences from a comment object in Gabbeh or include only part of a comment object.

User: “It is very good. I think we are making progress. However, I think the photo should be in the middle of the page and the comment will come beneath it.” (Iteration 3)

For example, the above comment was broken into two comment phrases; one comment phrase is indicating that user is confirming the changes in the design and the second comment phrase is suggesting further alterations to the design.

Comment phrase one: “It is very good. I think we are making progress.”

Comment phrase two: “However, I think the photo should be in the middle of the page and the comment will come beneath it.”

Clarifyi

ng

Verifyi

ng

Explori

ng

Alteri

ng

Confirmi

ng

Understand

ing

Organisi

ng

Justifyi

ng

Design

er 33 8 3 0 6 0 1 7 58

User 0 0 0 31 11 7 1 4 54

total 33 8 3 31 17 7 2 11 11

2

Table 9-5 Number of comment phrases associated to each category

Table 9-5 shows the number of comments associated with each category. One new category called “Justifying” was identified in the analysis which was added to the previously defined seven categories. In addition, the definition of the confirming category was revised since it was primarily used by the user in the previous study while the results of the case-study showed it had been used by both the designer and the user. In total 112 comments were identified, of which, only 11 did not fit into the identified classification for comments from Chapter 7. (See appendix C for a complete list of comments).

Amir M Naghsh 155

9.6.1.1. JUSTIFYING

The results showed that both the user and designer used the comments to give reasons and explanations on specific design issues. When a disagreement occurred over a design issue, the user and designer used comments to justify their way thinking as the correct and appropriate solution to the design issue. The ‘justifying’ comments usually represented the points of the view of an individual towards a design issue and the reasons behind it. Some examples are provided here from the case study.

Designer: “i dont want to put 'coporation' again, its off putting, a coporation sounds like a PLC, when really we want to get the mass market - about us, helps the customer to 'relate' to us (its a pyschological marketing attempt)”(page 7, iteration 2)

User: “People should think that a professional group who are doing marketing, finance, delivery etc. are behind this website. So those who have concern about the company will come to this page. Therefore this page should be very professional with strong message insurring visitors.”

Designer: “still to me coporation sounds "we are bigger than you" when that is not the essense of the site, unless there is a way (like amazon) to allow the customers' name to appear, therefore they will privaligedAn alternative suggestion "company" .“

9.6.1.2. CONFIRMING

In addition to the feedback by the user to confirm or disagree with parts of the design, the analysis showed that the designer also used comments to acknowledge or confirm changes to design in response to ‘altering’ comments made by the user.

Therefore, ‘confirming’ comments are the feedback provided by the user or designer to agree or disagree with a part of the design that is implemented or will be considered in the finished system. Figure 9-11 and Figure 9-12 show two ‘confirming’ comments created by the designer in response to changes that were requested by the user.

Figure 9-11 Confirmation by designer

Amir M Naghsh 156

Figure 9-12 Confirmation by Designer

Figure 9-13 shows another confirmation comment which was created by the user in response to a discussion over a design issue.

Figure 9-13 Confirmation by user

9.7. DISCUSSION

Reviewing the recorded interviews showed that the user and designer used the commenting tool to communicate ideas during the design process. They both showed their interest in the asynchronous communication. The designer indicated that it was easier to manage the time and therefore spend better quality time in the design process. While the user said it made it possible to spend more time on reviewing the design and “think over it”.

Furthermore, the analysis of the results and the debriefing interviews identified a number of concerns with the design of Gabbeh which need to be considered in future enhancements.


The analysis showed that the designer created many comments to inform the user of the progress made in the design model. The results suggested the importance of being able to identify which comments had been incorporated into the design and which had

Amir M Naghsh 157

not. The designer used the flag feature as a way of tracking which comments were incorporated in to the design.

In addition, to enhance the progress awareness it was recognised that it should be possible to allow the user and designer to review different versions of design at the same time. It was noted that reviewing old comments in discussions is not sufficient to enable a designer or user to understand the whole picture regarding a design issue. It was recognised that some comments become meaningless once the design is changed. Therefore, since a discussion consists of a number of comments created in different iterations it is essential to be able to access relevant versions of the design for each comment when going through the discussion.

As the design model evolves, the number of pages of the design increases and it becomes more difficult to track which page has been updated by going through all of them. Both the user and designer indicated that it would be useful to be able to see what pages in the design model had been updated.

9.7.2. RICHNESS OF COMMUNICATION CHANNEL

The results of debriefing interviews indicated the need for a richer communication channel. The user stated that it would be useful to be able to use ‘smiley faces’ to show emotions. The user mentioned it was difficult to say I don’t like a part of a design at all and it could be easier if I could show my disagreement in form of a ‘smiley face’.

The results also showed that the user and designer used the commenting tool to instruct each other to visit other websites which they found useful. It was realised that it is important that the comment content could be linked to other external sources and representations. In addition, it should be possible to attach a picture or file to a comment or discussion to be used by other stakeholders.

9.7.3. EMPOWERING USERS

The analysis showed that the user was interested in contributing the design by getting involved in making changes in the sketches in addition to providing feedback through comments. In the debriefing interview the user mentioned that “I wanted to make marks on the actual page”. The user indicated that in few occasions it seemed to be easier to draw on an existing sketch in order to express a point of view instead of explaining it as a long comment. Therefore, it is important to allow users to annotate the prototype while reviewing it in the run-mode.

Amir M Naghsh 158

9.8. SUMMARY

This chapter reported the results and analysis of a longitudinal case-study in a dispersed setting. The results showed that the comment facility would be used when available to communicate design ideas and issues, and to support discussions.

More importantly, the findings of the analysis indicated that the comment facility was not used only used to support discussions. It was also used to:

• support a progress awareness to track the design changes;

• discuss and debate design conflicts to reach common ground;

• direct attention to external resources or to a specific part of the design to enhance understanding and encourage more instructive feedback;

• record design decisions as the design is progressing.

In addition, the results of reviewing the comments classification complemented the findings of Chapter 7 and in addition to that, it introduced a new category called justifying and it modified the definition of confirming category to include both the user and designer comments.

Amir M Naghsh 159

Chapter 10. CONCLUSION

10.1. SUMMARY

In the preceding chapters, the thesis has examined existing electronic paper prototyping tools that support participatory design at early stages of design. To encourage direct participation of distributed stakeholders, the thesis has investigated the use of annotations to facilitate asynchronous communication in electronic paper prototyping tools. This chapter reflects on the contributions of the thesis in this area and offers suggestion for future work.

10.2. CONTRIBUTIONS

10.2.1. COGNITIVE DIMENSIONS

The first contribution of the thesis was to show how Cognitive Dimensions could be used to compare existing electronic paper prototyping tools. Chapter 3 conducted a pragmatic investigation in order to explore some of the characteristics of existing electronic paper prototyping tools. Furthermore, the chapter presented the results of pragmatic study and concluded that the traditional concept of fidelity is not sufficiently fine-grained to analyse the important differences and similarities between available sets of electronic paper prototyping tools.

Chapter 4 applied the Cognitive Dimensions framework to explore the characteristics of prototyping approaches. The results of theoretical analysis showed that to support a participatory design approach, it is important to maintain a low level of viscosity, abstraction, and diffuseness and to avoid hard mental operation. On the other hand, offering a balanced degree of closeness of mapping was suggested to avoid stakeholders focusing attention on low level details. The results of the analysis also indicated the importance of providing provisionality and progressive evaluation in order to support a highly iterative design process which allows stakeholders to interact with the prototype from the early stages in design.

In addition, chapter 4 has argued the importance of providing possible forms of secondary notation such as annotation in order to enhance communication among

Amir M Naghsh 160

different stakeholders. However, the results of the theoretical analysis have distinctively highlighted that none of the existing electronic paper prototyping tools supported secondary notation. The lack of the ability to provide secondary notation can severely limit the ability of such prototyping tools to support communication between different stakeholders at the early stages of design.

10.2.2. GABBEH

The thesis demonstrated the feasibility of supporting communication through providing annotations in electronic paper prototyping tools. Chapter 6 and 8 reported the design and implementation of Gabbeh, an electronic paper prototyping tool that allows different stakeholders to add arbitrary annotations in the form of comments when the prototype is being designed or executed.

The thesis reported the results of two studies that were conducted to evaluate Gabbeh. Chapter 7 presented the results of an observational study and Chapter 9 showed the results of a longitudinal study. Results of both studies showed that the comment facility was used once it was available to stakeholders. More importantly, the results showed that supporting annotation in electronic paper prototyping tools can facilitate communication since the results highlighted that stakeholders used the comment facility to respond to each others’ comments in the form of discussions.

Additionally, the findings of the analysis indicated that the comment facility was used to generate discussions:

• to support progress awareness in order to track the design changes;

• to discuss and debate design conflicts to reach common ground;

• to direct attention to external resources or to a specific part of the design in order to enhance understanding and encourage more instructive feedback;

• to record design decisions as the design progresses

10.2.3. A CLASSIFICATION OF ANNOTATIONS

The results presented in chapter 7 and 9 were considered to identify a possible classification of the uses made of annotations in designing interactive systems. The findings of the observational study followed by the longitudinal study suggested that comments created in designing interactive systems such as websites could be divided into eight categories as shown below. Stakeholders made comments for:

Amir M Naghsh 161

• clarifying design detail,

• exploring unspecified and not well understood requirements,

• verifying the parts that were already designed.

• understanding by asking questions about the design,

• altering the design,

• confirming the changes in the design.

• Justifying their way of thinking and giving reason for a design issue.

• Organising the design process.

The results of the studies showed that stakeholders created different types of comments when responding to each other as part of a discussion. For instance, an altering comment created by the end-user requesting a change generated a discussion where both the designer and end-user used comments to justify their way of thinking as well as asking questions to explore the design. Finally, the designer clarifies the changes to be incorporated into the design and asks for the end-user’s verification where the end-user confirms the changes.

10.2.4. RECOMMENDATIONS FOR SUPPORTING ANNOTATION IN ELECTRONIC PAPER PROTOTYPING TOOLS

Based on the findings of the theoretical studies in Chapter 4 and literature review of annotation in Chapter 5, the thesis suggested a list of recommendations for designing annotation tools. Chapter 6 presented this list of recommendations in the form of a possible design that could be used in designing electronic paper prototyping tools that support annotations. The proposed design was considered in the implementation of the first version of Gabbeh. Furthermore, the results of the evaluation studies contributed to the list of recommendations of proposed design and consequently further enhancing the second version of Gabbeh which was reported in Chapter 8.

Based on the suggested list of recommendations and results of the two evaluation studies, the thesis suggests the following list of requirements to be considered when designing annotation facilities in electronic paper prototyping:

Amir M Naghsh 162

10.2.4.1. CONSISTENCY TO CURRENT DESIGN

The annotation tool should be consistent with the current design of the electronic paper prototyping tool while it also has to be designed in a way which requires a minimum number of actions and skills for stakeholders to use it.

10.2.4.2. FLUIDITY OF FORM

Using the annotation tool should not distract the actual design activity where the stakeholders are exploring and evaluating the proposals and requirements. It has to be designed in a way that maintains the same fluidity that annotation has on the paper and enable users to make comments as an integral part of the design process. In other words, it should be possible to amalgamate the design activity and creation of comments as one activity.

10.2.4.3. IN SITU ANNOTATION

The design of annotation tools should allow the stakeholders to annotate the prototype while performing the design activity in such a way that their annotations are easily distinguished from the actual design or the prototype. It should also be possible to differentiate the annotations from one another, for instance it should indicate that they were created at different stages of design. The relationship between the design component (or number of design components) and the annotation should also be easy to determine.

10.2.4.4. INFORMAL USE

The design of the annotation tool has to consider that stakeholders usually have their own way of coding when they annotate. The design should provide a rich medium to maintain such an activity.

10.2.4.5. RETRIEVING AND FILTERING

The annotation tool should allow for additional information (e.g. author’s name or date/time of creation) to be associated with annotations as their properties. The tool should allow the stakeholders to distinguish different annotations through these properties. In addition, a filtering tool is required to manage large number of annotations and which enables stakeholders to retrieve annotations that share some common properties.

10.2.4.6. AVAILABILITY

The annotations should be available in all representations of a prototype. The annotation tool should allow the stakeholders to annotate anything without limit and

Amir M Naghsh 163

anywhere on the prototype by enabling the annotation to be associated with any arbitrary number of components in a prototype.

10.2.4.7. PROGRESS AWARENESS

The design of annotation should allow the stakeholders to be able to identify when and at which iteration an annotation has been created, who has created it, and if it has been addressed or responded to or not. It should be possible to distinguish new annotations from old ones. The progress awareness should allow a stakeholder to track comments to identify whether there are responses to a comment or if it has been incorporated into the design or has not yet been dealt with.

In addition, since different annotations are created in different iterations of the design process, they can refer to different versions of the same prototype. When stakeholders review annotations created in earlier iterations they should be able to review previous versions of a prototype where the annotation was created.

10.2.4.8. SUPPORTING DISCUSSIONS

To facilitate communication the annotation tool should support discussions. The tool should allow stakeholders to create annotations as reply to other annotations. Such two annotations should be linked with each other to allow the stakeholders to identify the relationship between them in later stages of the design.

The tool should allow the stakeholders to easily identify annotation associated with a discussion. They should be able to determine which annotation has initiated the discussion and what the latest responses in the discussion are.

10.3. FUTURE WORK

10.3.1. ENHANCEMENT TO THE CURRENT VERSION OF GABBEH

10.3.1.1. INTRODUCING FURTHER FEATURES

Because of the time and resource constraints it was not possible to provide all the features suggested in the list of recommendations of proposed design into Gabbeh. The remit of future work is to incorporate outstanding requirements from the proposed design and the enhancements that were identified during the two evaluation studies into the next version of Gabbeh.

The results of the longitudinal study in chapter 9 referred to the end-user explaining that in some cases, instead of explaining the design issue in a comment, it would have been easier for the user to amend the sketched pages to reflect his viewpoint.

Amir M Naghsh 164

Therefore, it is essential to enable the stakeholders using Gabbeh in run-mode to make changes to the sketched pages, or be able to sketch in comments and attach them to the prototype.

The results of evaluation studies indicated the need for a richer communication channel. The stakeholders should be able to agree or disagree with a comment without the need to create new comments. The end-user in the longitudinal study stated that it would have been useful to be able to use smiley faces to indicate whether the user liked or disliked a part of the design. Use of smiley faces was considered to be an easier way to show disagreements in particular situations.

Progress awareness is another area which requires further enhancement to meet the suggested requirements in an earlier section. Essentially, it should allow stakeholders to identify which comment is new, and which comments have been incorporated into the design. Further enhancements should enable the stakeholders to go through different versions of the design when reviewing comments. Additionally further development might consider introducing a method of informing the stakeholders of the pages that have been updated without the need of going through all the sketched pages.

10.3.1.2. USABILITY ISSUES

Another area of future work that should be investigated further is evaluating the usability of Gabbeh. The limitation of time and resources did not allow this research to investigate the usability issues involved in designing the annotation tool. However, the effectiveness of the design of the annotation tool and filtering features are largely dependent on the provided user interfaces. To improve user participation the interfaces of the facilities such as filtering tools, and the displaying panel for discussions have to be enhanced though further usability studies.

10.3.2. DESIGN RATIONALE

In designing any computer system, many decisions are made as the product goes from a set of vague customer requirements to a deliverable entity. Often it is difficult to recreate the reasons, the rationale, behind various design decisions. Design rational is the information that explains why a computer system is the way it is (Dix et al., 1993, p. 248).

One of the areas that can be explored as advancement in this research is the use of annotation, particularly discussions, to support some form of design rationale. The results of the longitudinal study presented in chapter 9 showed examples of a designer using the discussions to refer to an earlier comment provided by the user to clarify the

Amir M Naghsh 165

changes that were incorporated into the design. Therefore, there is a potential to use recoded discussions in the form of a history associated with different parts of the design. The comments associated with a discussion can reflect the rationale of the changes incorporated into that specific part of design which the discussion is associated with.

To investigate the use of discussions in the design rationale, features such as being able to support comments, or having a voting mechanism to facilitate decision making can be considered as further enhancements.

10.3.3. SUPPORTING ANNOTATION IN SITUATED SETTINGS

To encourage a greater user involvement, participatory design techniques make use of stakeholders’ vast experience of collaborating around traditional tables by using tools and materials that offer family resemblance to what they use in their ordinary workplace. Recent research in CSCW (Masoodian et al., 2007; Morris et al., 2006; Ryall et al., 2004) has considered interactive tabletops an ideal environment for co-present collaborations for small groups. An area of future work can be considered on introducing an electronic paper prototyping tool (such as Gabbeh) into a tabletop environment, intended to increase stakeholders’ participation and mutual learning when collaborating synchronously.

Supporting direct input from multiple users was identified as the essential enhancement into Gabbeh in order to enable it to be used for further investigations in this area of research. Therefore, in collaboration with the I-MAG laboratory at Grenoble University, multi user entry has been introduced into Gabbeh to prepare the base for future work in this direction (Naghsh and Bailly, 2005).

10.3.4. DISTRIBUTED PARTICIPATORY DESIGN

A number of researchers have investigated the use of participatory design in distributed design teams (Jemtrud et al., 2006; Danielsson and Danielsson, 2005; Naghsh and Ozcan 2004; Everitt et al., 2003; Kensing and Blomberg, 1998). A Series of workshops on Distributed Participatory Design (Naghsh et al., 2008; Danielsson et al., 2006) have been organised on this emerging area of research to overcome the challenges of performing participatory design in distributed design teams and to understand the usefulness and constraints of this approach. It is possible for the findings of this thesis to be considered in the design of tools that may support distributed participatory design. Further research may examine the possibility of electronic paper-prototyping

Amir M Naghsh 166

tools with annotation facilities (e.g. Gabbeh) in supporting distributed participatory design.

Amir M Naghsh 167

REFERENCES

Agarwal, R., Prasad, J., Tanniru, M. and Lynch, J. (2000). Risks of rapid application development. Communications of the ACM, 43 (11), ACM Press, pp. 1.

Andriole, S.J. (1989). Storyboard prototyping: A new approach to user requirements analysis. QED Information Sciences, Inc., Wellesley, MA, United States.

Andy, D., Naghsh, A.M. and Ozcan, M.B. (2004). Support for participation in electronic paper prototyping. In: The Proceedings of the Participatory Design Conference, Toronto, Canada. pp. 105 – 108.

Appan, P., Shevade, B., Sundaram, H. and Birchfield, D. (2005). Interfaces for networked media exploration and collaborative annotation. In: IUI '05: Proceedings of the 10th International Conference on Intelligent User Interfaces, San Diego, California, United States. pp. 106-113.

Audy, J., Evaristo, R. and Watson-Manheim, M.B. (2004). Distributed analysis: The last frontier? In: HICSS '04: Proceedings of the 37th Annual Hawaii International Conference on System Sciences (HICSS'04) - Track 1, pp. 100-102.

Bastide, R. and Palanque, P.A. (1995). A petri net based environment for the design of event-driven interfaces. In: Proceedings of the 16th International Conference on Application and Theory of Petri Nets, Torino, Italy, pp. 66-83.

Baumer, D., Bischofberger, W.R., Lichter, H. and Zullighoven, H. (1996). User interface prototyping—concepts, tools, and experience. In: ICSE '96: Proceedings of the 18th International Conference on Software Engineering, Berlin, Germany. pp. 532-541.

Beck, K. and Andres, C. (2004). Extreme programming explained: Embrace change. 2nd ed., Addison-Wesley Professional, Boston, MA, United States.

Amir M Naghsh 168

Bellotti, V. and Bly, S. (1996). Walking away from the desktop computer: Distributed collaboration and mobility in a product design team. In: CSCW '96: Proceedings of the 1996 ACM Conference on Computer Supported Cooperative Work, Boston, Massachusetts, United States. pp. 209-218.

Blackwell, A.F., Britton, C., Cox, A.L., Green, T.R.G., Gurr, C.A., Kadoda, G.F., Kutar, M., Loomes, M., Nehaniv, C.L., Petre, M., Roast, C., Roe, C., Wong, A. and Young, R.M. (2001). Cognitive dimensions of notations: Design tools for cognitive technology. In: BEYNON, M., NEHANIV, C.L. and DAUTENHAHN, K. (eds.). The 4th International Conference on Cognitive Technology, Warwick, UK pp. 325-341.

Blinman, S. and Cockburn, A. (2005). Program comprehension: Investigating the effects of naming style and documentation. In: AUIC '05: Proceedings of the Sixth Australasian Conference on User Interface, Newcastle, Australia. pp. 73-78.

Bodker, S., Gronbaek, K. and Kyng, M. (1993). Cooperative design: Techniques and experiences from the scandinavian scene. Lawrence Erlbaum Associates,.

Bodker, S. and Gronbaek, K. (1991). Cooperative prototyping: Users and designers in mutual activity. Int.J.man-mach.stud., 34 (3), Academic Press Ltd, pp. 453-478.

Bottoni, P., Civica, R., Levialdi, S., Orso, L., Panizzi, E. and Trinchese, R. (2004). MADCOW: A multimedia digital annotation system. In: AVI '04: Proceedings of the Working Conference on Advanced Visual Interfaces, Gallipoli, Italy. pp. 55-62.

Brinkley, J., Jakobovits, R. and Rosse, C. (2002). An online image management system for anatomy teaching. In: The American Medican Informatics Association Annual Symposium, Washington, DC, United States, pp. 983.

Brockman, W., S., Neumann, L., Palmer, C., L. and Tidline, T.J. (2001). Scholarly work in theHumanities and theEvolving InformationEnvironment. Washington DC., Digital Library Federation, Washington DC, United States.

Brush, A.J.B., Bargeron, D., Gupta, A. and Cadiz, J.J. (2001). Robust annotation positioning in digital documents. In: CHI '01: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Seattle, Washington, United States. pp. 285-292.

Amir M Naghsh 169

Button, G. and Sharrock, W. (1996). Project work: The organisation of collaborative design and development in software engineering. Comput.supported coop.work, 5 (4), Kluwer Academic Publishers, pp. 369-386.

Cadiz, J.J., Gupta, A. and Grudin, J. (2000). Using web annotations for asynchronous collaboration around documents. In: CSCW '00: Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work, Philadelphia, Pennsylvania, United States. pp. 309-318.

Chronaki, C.E., Zabulis, X. and Orphanoudakis, S.C. (1997). I2Cnet medical image annotation service. International journal of medical informatics, 22 (4), Elsevier, pp. 337-347.

Clarke, S. (2001). Evaluating a new programming language. In: KADODA, G. (ed.). The Thirteenth Annual Meeting of the Psychology of Programming Interest Group, pp. 275-289.

Coar, K. (2004). The sun never sits on distributed development. Queue, 1 (9), ACM Press, pp. 32-39.

Cooper, D., Stepney, S. and Woodcock, J. (2002). Derivation of Z refinement proof rules. University of York, York, UK.

Crosby, M.,E., Scholtz, J. and Wiedenbeck, S. (2002). The roles beacons play in comprehension for novice and expert programmers. In: KULJIS, J., BALDWIN, L. and SCOBLE, R. (eds.). The 14th Annual Workshop of the Psychology of Programming Interest Group, London, UK. pp. 58-73.

Curtis, G. and Vertelney, L. (1990). Storyboards and sketch prototypes for rapid interface visualization. In: CHI Tutorial, Seattle, Washington, United States.

Damm, C.H., Hansen, K.M. and Thomsen, M. (2000). Tool support for cooperative object-oriented design: Gesture based modelling on an electronic whiteboard. In: CHI '00: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, The Hague, The Netherlands. pp. 518-525.

Danielsson, K. and Danielsson, U. (2005). Distributed participatory design: A case study. In: HCI International Conference, Las Vegas, United States.

Amir M Naghsh 170

Danielsson, K., Naghsh, A.M. and Dearden, A. (2006). The first workshop of distributed participatory design. In: NordiCHI 2006, Oslo, Norway.

Davis, A.M. (1992). Operational prototyping: A new development approach. IEEE softw., 9 (5), IEEE Computer Society Press, pp. 70-78.

Dearden, A.,J., Siddiqi, J.I. and Naghsh, A.M. (2003). Using cognitive dimensions to compare prototyping techniques. In: The First Joint Conference of EASE & PPIG (15th Annual Meeting of the Psychology of Programming Interest Group), Keel, UK.

Dix, A., Finlay, J.E., Abowd, G.D. and Beale, R. (2004). Human-computer interaction. 3rd ed., Prentice-Hall, Inc, Upper Saddle River, NJ, United States.

Duyne, D.K.V., Landay, J.A. and Hong, J.I. (2002). The design of sites: Patterns, principles, and processes for crafting a customer-centered web experience. Addison-Wesley Longman Publishing Co., Inc, Boston, MA, United States.

Ehn, P. (1993). Scandinavian design: On participation and skill. In: In Schuler, D. and Namioka, A., (Eds.) Participatory Design. Principles and Practices, pp. 41-78.

Ehn, P. (1990). Work-oriented design of computer artifacts. Lawrence Erlbaum Associates, Inc, Mahwah, NJ, United States.

Ehn, P. (1988). Playing the language-games of design and use-on skill and participation. In: Proceedings of the ACM SIGOIS and IEEECS TC-OA 1988 Conference on Office Information Systems, Palo Alto, California, United States. pp. 142-157.

Ehn, P. and Kyng, M. (1991). Cardboard computers: Mocking-it-up or hands-on the future. Lawrence Erlbaum Associates, Inc, pp. 169-196.

Engelberg, D. and Seffa, A. (2002). A framework for rapid mid-fidelity prototyping of web sites. In: Proceedings of the IFIP 17th World Computer Congress - TC13 Stream on Usability, pp. 203-215.

Everitt, K.M., Klemmer, S.R., Lee, R. and Landay, J.A. (2003). Two worlds apart: Bridging the gap between physical and virtual media for distributed design collaboration. Human factors in computing systems, 5 (1), pp. 553-560.

Amir M Naghsh 171

Fish, R.S., Kraut, R.E. and Leland, M.D.P. (1988). Quilt: A collaborative tool for cooperative writing. In: Proceedings of the ACM SIGOIS and IEEECS TC-OA 1988 Conference on Office Information Systems, Palo Alto, California, United States. pp. 30-37.

Floyd, C., Mehl, W., Reisen, F., Schmidt, G. & Wolf,G. (1989). Out of scandinavia: Alternative approaches to software design and system development. In: Human-Computer Interaction 4, pp. 253-350.

Fuchs, N.E. (1992). Specifications are (preferably) executable. Software engineering journal, 7 (5), Michael Faraday House, pp. 323-334.

Green, T.R.G. (1983). Learning big and little programming languages. In: WILKINSON, A.C. (ed.). Classroom Computers and Cognitive Science, .

Green, T.R.G. and Blackwell, A.F. (1998). A tutorial on cognitive dimensions. Available on-line at: http://www.ndirect.co.uk/~thomas.green/workStuff/Papers/index.html

Green, T.R.G. and Petre, M. (1996). Usability analysis of visual programming environments: A 'cognitive dimensions' framework. Journal of visual languages and computing, 7 (2), pp. 131-174.

Gronbaek, K. (1991). Prototyping and active user involvement in system development: Towards a cooperative prototyping approach. Ph.D. Thesis. Computer Science Department, Aarhus University.

Gronbaek, K. (1989). Rapid prototyping with fourth generation systems: An empirical study. 5 (2), OFFICE: Technology and People, pp. 105-125.

Hanson, A.J. and Wernert, E.A. (1997). Constrained 3D navigation with 2D controllers. In: VIS '97: Proceedings of the 8th Conference on Visualization '97, Phoenix, Arizona, United States. pp. 175-ff.

Harmon, R., Patterson, W., Ribarsky, W. and Bolter, J. (1996). The virtual annotation system. In: VRAIS '96: Proceedings of the 1996 Virtual Reality Annual International Symposium (VRAIS 96), pp. 239.

Heeks, R., Krishna, S., Nicholson, B. and Sahay, S. (2001). Synching or sinking: Global software outsourcing relationships. IEEE softw., 18 (2), IEEE Computer Society Press, pp. 54-60.

Amir M Naghsh 172

Henderson, K. (1991). Flexible sketches and inflexible data bases: Visual communication, conscription devices, and boundary objects in design engineering. Science, technology & human values, 16 (4), pp. 448-473.

Hong, J.I., Landay, J.A., Long, A.C. and Mankoff, J.C. (2002). Sketch recognizers from the end-user's, the designer's, and the programmer's perspective. In: American Association for Artificial Intelligence, Stanford, CA, United States. pp. 73-77.

Hong, J.I., Li, F.C., Lin, J. and Landay, J.A. (2001). End-user perceptions of formal and informal representations of web sites. In: CHI '01: CHI '01 Extended Abstracts on Human Factors in Computing Systems, Seattle, Washington. pp. 385-386.

J., W. and R. D. (1988). Rapid prototyping for user interface design. In: M., Helander (ed.). Handbook of human-computer interaction. Elsevier, Amsterdam, North-Holland, pp. 859-875.

Jemtrud, M., Nguyen, P., Spencer, B., Brooks, M., Liu, S., Liang, Y., Xu, B. and Zhang, L. (2006). Eucalyptus: Intelligent infrastructure enabled participatory design studio. In: WSC '06: Proceedings of the 38th Conference on Winter Simulation, Monterey, California. pp. 2047-2054.

Jourdan, M., Layaida, N., Roisin, C., Sabry-Ismail, L. and Tardif, L. (1998). Madeus, and authoring environment for interactive multimedia documents. In: MULTIMEDIA '98: Proceedings of the Sixth ACM International Conference on Multimedia, Bristol, United Kingdom. pp. 267-272.

Jung, T., Gross, M.D. and Do, E.Y. (2002). Annotating and sketching on 3D web models. In: IUI '02: Proceedings of the 7th International Conference on Intelligent User Interfaces, San Francisco, California, United States. pp. 95-102.

Kahan, J. and Koivunen, M. (2001). Annotea: An open RDF infrastructure for shared web annotations. In: WWW '01: Proceedings of the 10th International Conference on World Wide Web, Hong Kong, Hong Kong. pp. 623-632.

Kant, M.v.d., Wilson, S., Bekker, M., Johnson, H. and Johnson, P. (1998). PatchWork: A software tool for early design. In: CHI '98: CHI 98 Conference Summary on Human Factors in Computing Systems, Los Angeles, California, United States. pp. 221-222.

Amir M Naghsh 173

Kensing, F. and Blomberg, J. (1998). Participatory design: Issues and concerns. Comput.supported coop.work, 7 (3-4), Kluwer Academic Publishers, pp. 167-185.

Khazaei, B. and Triffitt, E. (2002). Applying cognitive dimensions to evaluate and improve the usability of Z formalism. In: SEKE '02: Proceedings of the 14th International Conference on Software Engineering and Knowledge Engineering, Ischia, Italy. pp. 571-577.

Klemmer, S.R., Newman, M.W., Farrell, R., Bilezikjian, M. and Landay, J.A. (2001). The designers' outpost: A tangible interface for collaborative web site. In: UIST '01: Proceedings of the 14th Annual ACM Symposium on User Interface Software and Technology, Orlando, Florida. pp. 1-10.

Kutar, M., Britton, C. and Barker, T. (2002). A comparison of empirical study and cognitive dimensions analysis in the evaluation of UML diagrams. In: The 14th Annual Meeting of the Psychology of Programming Interest Group Workshop, Uxbridge, UK.

Landay, J.A. (1996a). Interactive sketching for the early stages of user interface design. PhD. Computer Science Department, Carnegie Mellon University.

Landay, J.A. (1996b). SILK: Sketching interfaces like krazy. In: CHI '96: Conference Companion on Human Factors in Computing Systems, Vancouver, British Columbia, Canada. pp. 398-399.

Landay, J.A. (1995). Interactive sketching for user interface design. In: CHI '95: Conference Companion on Human Factors in Computing Systems, Denver, Colorado, United States. pp. 63-64.

Landay, J.A. and Myers, B.A. (2001). Sketching interfaces: Toward more human interface design. Computer, 34 (3), IEEE Computer Society Press, pp. 56-64.

Landay, J.A. and Myers, B.A. (1996). Sketching storyboards to illustrate interface behaviors. In: CHI '96: Conference Companion on Human Factors in Computing Systems, Vancouver, British Columbia, Canada. pp. 193-194.

Landay, J.A. and Myers, B.A. (1995). Interactive sketching for the early stages of user interface design. In: CHI '95: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Denver, Colorado, United States. pp. 43-50.

Amir M Naghsh 174

Leland, M.D.P., Fish, R.S. and Kraut, R.E. (1988). Collaborative document production using quilt. In: CSCW '88: Proceedings of the 1988 ACM Conference on Computer-Supported Cooperative Work, Portland, Oregon, United States. pp. 206-215.

Lin, J. (2003). Damask: A tool for early-stage design and prototyping of cross-device user interfaces. In: UIST'03, Vancouver, Canada. pp. 13-16.

Lin, J., Newman, M.W., Hong, J.I. and Landay, J.A. (2000). DENIM: Finding a tighter fit between tools and practice for web site design. In: CHI '00: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, The Hague, The Netherlands. pp. 510-517.

Lin, J., Thomsen, M. and Landay, J.A. (2002). A visual language for sketching large and complex interactive designs. In: CHI '02: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Minneapolis, Minnesota, United States. pp. 307-314.

Liu, L. and Khooshabeh, P. (2003). Paper or interactive?: A study of prototyping techniques for ubiquitous computing environments. In: CHI '03: CHI '03 Extended Abstracts on Human Factors in Computing Systems, Ft. Lauderdale, Florida, United States. pp. 1030-1031.

Lober, W.B. and Brinkley, J.F. (2000). XML-based image annotation: PAIS - a personal annotated image server. In: Slice of Life, .

M. Naphade, C.-Y. Lin, J. R. Smith and B. Tseng, S.B. (2002). Learning to annotate video databases. In: SPIE Symp. on Electronic Imaging: Storage and Retrieval for Image and Video Databases, San Jose, CA, United States.

Marshall, C.C. (1997). Annotation: From paper books to the digital library. In: DL '97: Proceedings of the Second ACM International Conference on Digital Libraries, Philadelphia, Pennsylvania, United States. pp. 131-140.

Marshall, C.C. and Brush, A.J.B. (2004). Exploring the relationship between personal and public annotations. In: JCDL '04: Proceedings of the 4th ACM/IEEE-CS Joint Conference on Digital Libraries, Tuscon, AZ, United States. pp. 349-357.

Marshall, C.C. and Brush, A.J.B. (2002). From personal to shared annotations. In: CHI '02: CHI '02 Extended Abstracts on Human Factors in Computing Systems, Minneapolis, Minnesota, United States. pp. 812-813.

Amir M Naghsh 175

Martin, J. (1991). Rapid application development. Macmillan Publishing Company, New York.

Masoodian, M., McKoy, S. and Rogers, B. (2007). Hands-on sharing: Collaborative document manipulation on a tabletop display using bare hands. In: CHINZ '07: Proceedings of the 7th ACM SIGCHI New Zealand Chapter's International Conference on Computer-Human Interaction, Hamilton, New Zealand. pp. 25-31.

Morrey, I., Siddiqi, J.I., Hibberd, R. and Buckberry, G. (1998). A toolset to support the construction and animation of formal specifications. J.syst.softw., 41 (3), Elsevier Science Inc, pp. 147-160.

Morris, J.H., Neuwirth, C.M., Regli, S.H., Chandhok, R. and Wenger, G.C. (1999). Interface issues in computer support for asynchronous communication. ACM comput.surv., 31 (2es), ACM Press, pp. 11.

Morris, M.R., Paepcke, A., Winograd, T. and Stamberger, J. (2006). TeamTag: Exploring centralized versus replicated controls for co-located tabletop groupware. In: CHI '06: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Montral, Qubec, Canada. pp. 1273-1282.

Muller, M.J. (1991). PICTIVE—an exploration in participatory design. In: CHI '91: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, New Orleans, Louisiana, United States. pp. 225-231.

Naghsh, A.M. (2004). An electornic prototyping tool to facilitate annotations in the early sages of design. In: Doctoral Consortium at Participatory Design Conference, Toronto, Canada.

Naghsh, A.M. (2004). Towards computer supported collaboration in electronic paper prototyping. In: Doctoral Consortium at CSCW'04 the ACM Conference on Computer Supported Cooperative Work, Chicago, United States.

Naghsh, A.M. and Andy, D. (2005). Supporting collaboration in electronic paper prototyping. In: The Workshop on Cognition and Collaboration Analyzing Distributed Community Practices for Design at CHI 2005, Portland, Oregon, United States.

Naghsh, A.M. and Andy, D. (2004). A demostration of GABBEH: A tool to support collaboration in electronic paper prototyping. In: CSCW'04 the ACM

Amir M Naghsh 176

Conference on Computer Supported Cooperative Work, Chicago, United States.

Naghsh, A.M. and Bailly, G. (2005). Denim-Gabbeh multi-user technical report. Communication & Computing research Centre, Sheffield Hallam University, Sheffield, UK.

Naghsh, A.M., Danielsson, K., Gumm, D. and Warr, A. (2008). The second workshop of distributed participatory design. In: CHI 2008, Florence, Italy.

Naghsh, A.M., Dearden, A. and Ozcan, M.B. (2006). Investigating annotation in electronic paper-prototypes. In: GILROY, S.W. and HARRISON, M.D. (eds.). DSV-IS'05 Interactive Systems, Design, Specification, and Verification,12th International Workshop, Newcastle upon Tyne, UK. pp. 90-101.

Naghsh, A.M. and Ozcan, M.B. (2004). GABBEH - A tool for computer supported collaboration in electronic paper prototyping. In: A., D. and L., W. (eds.). HCI'04 the Annual Conferance of British HCI Group: Design for Life, Leeds, UK. pp. 77-80.

Naslund, T. (1997). Computers in context –But in which context? In: Kyng, M. & Mathiassen, L. (Eds). Computers and Design in Context, Cambridge, MA., United States. pp. 171-200.

Neuwirth, C.M., Chandh, R., Charney, D., Wojahn, P. and Kim, L. (1994). Distributed collaborative writing: A comparison of spoken and written modalities for reviewing and revising documents. In: CHI '94: Conference Companion on Human Factors in Computing Systems, Boston, Massachusetts, United States. pp. 202.

Neuwirth, C.M., Kaufer, D.S., Chandhok, R. and Morris, J.H. (1990). Issues in the design of computer support for co-authoring and commenting. In: CSCW '90: Proceedings of the 1990 ACM Conference on Computer-Supported Cooperative Work, Los Angeles, California, United States. pp. 183-195.

Newman, M.W. and Landay, J.A. (2000). Sitemaps, storyboards, and specifications: A sketch of web site design practice. In: DIS '00: Proceedings of the Conference on Designing Interactive Systems, New York City, New York, United States. pp. 263-274.

Amir M Naghsh 177

Newman, M.W., Lin, J., Hong, J.I. and Landay, J.A. (2003). DENIM: An informal web site design tool inspired by observations of practice. Human computer interaction, 18 (3), pp. 259-324.

Nielsen, J. (1993). Usability engineering. Academic Press, New York.

Nielsen, J. (1992). Finding usability problems through heuristic evaluation. In: CHI '92: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Monterey, California, United States. pp. 373-380.

Nielsen, J. (1984). How readers annotate textbooks and manuals. 182 Computer Science Dept., Aarhus University,.

O'Hara, K. and Sellen, A. (1997). A comparison of reading paper and on-line documents. In: CHI '97: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Atlanta, Georgia, United States. pp. 335-342.

O'Neill, E. (2001). User-developer cooperation in software development : Building common ground and usable systems. Springer, London, UK.

O'Neill, E. (1998). User-developer cooperation in software development: Building common ground and usable systems.

O'Neill, E., Johnson, P. and Johnson, H. (1999). Representations and user-developer interaction in cooperative analysis and design. Human-computer interaction, 14 (1), pp. 43-91.

Ozcan, M.B., Parry, P.W., Morrey, I. and Siddiqi, J.I. (1998). Visualisation of executable formal specifications for user validation. Selected papers on services and visualization: Towards user-friendly design, 1358 Lecture Notes In Computer Science, Springer-Verlag, pp. 142-157.

Palanque, P. and Bastide, R. (1997). Synergistic modeling of tasks, users and systems using formal specification techniques. Interacting with computers, 9 (2), Elsevier, pp. 129-153.

Palanque, P.A. and Bastide, R. (1995). Formal specification and verification of CSCW using the interactive cooperative object formalisms. In: HCI'05 People and Computers, Huddersfield, UK. pp. 213-232.

Parry, P. W. and Ozcan, M. B. (2000). Development of a visual requirements validation tool. In: Limerick, Ireland.

Amir M Naghsh 178

Pearsall, J. and Trumble, B. (2002). Oxford english reference dictionary. Oxford University Press, 2ed edition; Oxford, UK.

Perry, M. and Sanderson, D. (1998). Co-ordinating joint design work: The role of communication and artefacts. Design studies, 19 (3), Elsevier Science Ltd, pp. 273-288.

Pilatti, L., Audy, J.L.N. and Prikladnicki, R. (2006). Software configuration management over a global software development environment: Lessons learned from a case study. In: GSD '06: Proceedings of the 2006 International Workshop on Global Software Development for the Practitioner, Shanghai, China. pp. 45-50.

Plat, N., Katwijk, J.V. and Toetenel, H. (1992). Application and benefits of formal methods in software development. Software engineering journal, 7 (5), Software Engineering Journal, pp. 335-347.

Plimmer, B. and Apperley, M. (2003). Evaluating a sketch environment for novice programmers. In: CHI '03: CHI '03 Extended Abstracts on Human Factors in Computing Systems, Ft. Lauderdale, Florida, United States. pp. 1018-1019.

Preece, J., Preece, J., Rogers, Y. and Sharp, H. (2002). Interaction design: Beyond human-computer interaction. John Wiley & Sons, Inc, New York, NY, United States.

Pycock, J. and Bowers, J. (1996). Getting others to get it right: An ethnography of design work in the fashion industry. In: CSCW '96: Proceedings of the 1996 ACM Conference on Computer Supported Cooperative Work, Boston, Massachusetts, United States. pp. 219-228.

R., B., D., N. and P., P. (2003). A tool-supported design framework for safety critical interactive systems. Interacting with computers, 15 (3), Elsevier, pp. 309-328.

Rettig, M. (1994). Prototyping for tiny fingers. Communications of the ACM, 37 (4), ACM Press, pp. 21-27.

Roast, C., Khazaei, B. and Siddiqi, J.I. (2000). Formal comparison of program modification. In: J., S. and J., P.J. (eds.). IEEE Symposium on Visual Languages, pp. 165-171.

Amir M Naghsh 179

Roast, C.R. (1998). Modelling unwarranted commitment in information artefacts. In: CHATTY, S. and DEWAN, P. (eds.). Engineering for Human-Computer Interaction, pp. 70-90.

Roast, C.R. and Siddiqi, J.I. (1997). Usability requirements as specification constraints: An example ofWYSIWYG. Software engineering. IEE proceedings, 144 (2),.

Rosson, M.B., Kellogg, W. and Maass, S. (1988). The designer as user: Building requirements for design tools from design practice. Communications of the ACM, 31 (11), ACM Press, pp. 1288-1298.

Rosson, M.B., Maass, S. and Kellogg, W.A. (1987). Designing for designers: An analysis of design practice in the real world. In: CHI '87: Proceedings of the SIGCHI/GI Conference on Human Factors in Computing Systems and Graphics Interface, Toronto, Ontario, Canada. pp. 137-142.

Rudd, J., Stern, K. and Isensee, S. (1996). Low vs. high-fidelity prototyping debate. Interactions, 3 (1), ACM Press, pp. 76-85.

Ryall, K., Forlines, C., Shen, C. and Morris, M.R. (2004). Exploring the effects of group size and table size on interactions with tabletop shared-display groupware. In: CSCW '04: Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work, Chicago, Illinois, United States. pp. 284-293.

Sharp, H. and Robinson, H. (2004). An ethnographic study of XP practice. Empirical softw.engg., 9 (4), Kluwer Academic Publishers, pp. 353-375.

Siddiqi, J.I., Morrey, I.C., Roast, C.R. and Ozcan, M.B. (1997). Towards quality requirements via animated formal specifications. Ann.softw.eng., 3 J. C. Baltzer AG, Science Publishers, pp. 131-155.

Siddiqi, J.I., Morrey, I., Hibberd, R. and Buckberry, G. (1999). Understanding and exploring formal specifications. Ann.softw.eng., 6 (1-4), J. C. Baltzer AG, Science Publishers, pp. 411-432.

Triffitt, E. and Khazaei, B. (2002). A study of usability of Z formalism based on cognitive dimensions. In: KULJIS, J., BALDWIN, L. and SCOBLE, R. (eds.). The Fourteenth Annual Meeting of the Psychology of Programming Interest Group, pp. 15-28.

Amir M Naghsh 180

Tullis, T.S. (1990). High-fidelity prototyping throughout the design. In: Human Factors Society 34th Annual Meeting, pp. 266.

Uceta, F.A., Dixon, M.A. and Resnick, M.L. (1998). Adding interactivity to paper prototypes, proceedings of the , 506-511. In: Human Factors and Ergonomics Society 42nd Annual Meeting, pp. 506-511.

Walker, M., Takayama, L. and Landay, J.A. (2002). High-fidelity or low-fidelity, paper or computer medium? In: Human Factors and Ergonomics Society 46th Annual Meeting: HFES2002, pp. 661-665.

Weng, C. and Gennari, J.H. (2004). Asynchronous collaborative writing through annotations. In: Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work, Chicago, Illinois, United States. pp. 578-581.

Wilson, S., Bekker, M., Johnson, P. and Johnson, H. (1997). Helping and hindering user involvement — a tale of everyday design. In: CHI '97: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Atlanta, Georgia, United States. pp. 178-185.

Wing, J.M. (1990). A specifier's introduction to formal methods. Computer, 23 (9), IEEE Computer Society Press, pp. 8-23.

Wojahn, P.G., Neuwirth, C.M. and Bullock, B. (1998). Effects of interfaces for annotation on communication in a collaborative task. In: CHI '98: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Los Angeles, California, United States. pp. 456-463.

Wong, Y.Y. (1992). Rough and ready prototypes: Lessons from graphic design. In: CHI '92: Posters and Short Talks of the 1992 SIGCHI Conference on Human Factors in Computing Systems, Monterey, California. pp. 83-84.

Amir M Naghsh 181

APPENDIX A

This appendix presents a more detailed background on Gabbeh, a hand-woven Persian rug.

Amir M Naghsh 182

Amir M Naghsh 183

Amir M Naghsh 184

Amir M Naghsh 185

APPENDIX B

This appendix presents the design tasks used in the observational study, as well as the results of the study which was used for the analysis of Chapter 7.

RESEARCH GROUP WEBSITE TASK BRIEF

Please could you design a website using the Denim tool.

The website is intended to provide information about a group of researchers at Sheffield Hallam University.

Contents should include:

• An attractive home page

• A page for each individual researcher to describe their work - this should include personal information about the researcher and links for students to find out about courses that the researcher teaches

• Pages describing research projects that the group is involved in

• A page listing publications for the whole group (by year)

• Pages for individual researchers publications, perhaps organised according to projects or titles

• A page of events in which people in the group are involved

• Links to the host institution & research institute

• Photos of the people involved in the research & other images related to the research (e.g. of prototypes etc.)

You may have some additional suggestions that could enhance the site, given its aims.

Amir M Naghsh 186

VISIT SHEFFIELD WEBSITE TASK BRIEF

Your design should take into account the fact that many of the intended users may be using slow dial-up lines.

Please could you design a website for www.visitsheffield.co.uk, using the DENIM tool.

The website is intended to attract tourists and other visitors to Sheffield.

• Contents should include:

• An attractive home page

• Information about how to get to Sheffield (by road, rail, air etc.)

• Information about the region around Sheffield

• Information about how to get around Sheffield & the region

• Information about accomodation with suitable links to providers

• Information about Leisure activites in Sheffield, e.g. sports, nightlife, attractions etc.

• Information about restaurants / eating & drinking etc.

• A news area that can be regularly updated.

You may have some additional suggestions that could enhance the site, given its aims.

Your design should take into account the fact that many of the intended users may be using slow dial-up lines.

Amir M Naghsh 187

List of comments collected from the observational study:

No Comment Group Session Creator Classification

1 A better and more descriptive "Title" for home page such as Sheffield Hallam University Research Group (SHURG)

1 DENIM end-user Altering

2 Add a picture on top of the page which shows all the members of the team 1 DENIM end-user Altering

3 Second page, changing title to Team of Researchers 1 DENIM end-user Altering

4

In Projects, we need a page with 3 boxes that identifies the research subjects: (i.e., Software Development, Networking and possibly have room for adding one or more research area to the page)


5 Having a scurrility option on research projects 1 DENIM end-user Altering

6 We dont understand point 5 of your requirements 1 DENIM designer-

user Exploring

7 Can you please provide us with more details about requirements 4 and 5 on the original "WebSite Design Task Brief".

1 DENIM designer-user Exploring

8 We have provided a page with a list of group projects. 1 DENIM designer-

user Clarifying

9 We also provided links to these projects from researchers pages. Does this meet the requirements?

1 DENIM designer-user Verifying

10

In option 4: We want to have 4 selections, rather than 3 (for our future research) and we prefer to have them in a boxes to makes them more attractive


11

in Option 5: We need security on each project subject as we want each researcher to have security on their research area. Please let me know if it is not still clear?


12 Interactive set of photos 1 Gabbeh designer-user Clarifying

13 if this is link to shu search engine I need to have box to input the keyword to search for.

1 Gabbeh end-user Understanding

14 link opens the search engine in a new page 1 Gabbeh designer-

user Clarifying

15 Add Shu logo on top of this page 1 Gabbeh end-user Altering 16 please ignore the first comment 1 Gabbeh end-user Organising

17 add a list of publication without abstract in this pgae 1 Gabbeh end-user Altering

18 list of publication grouped by year 1 Gabbeh designer-user Verifying

19 have link to website and the location of publication to have view of paper 1 Gabbeh end-user Altering

20 here we provide links to each paper. Is that what you wanted? 1 Gabbeh designer-

user Verifying

Amir M Naghsh 188

21 yes that’s what I wanted 1 Gabbeh end-user Confirming 22 add two schedule for two semesters 1 Gabbeh end-user Altering No Comment Group Session Creator Classification

23 timetables are added to this page for each semester 1 Gabbeh designer-

user Verifying

24 news related to projects and success stories 1 Gabbeh designer-

user Clarifying

25 Can you please provide us with more details about consultancy page? 1 Gabbeh designer-

user Exploring

26 Here we have provided a list of consultancy options. Does this meet your need?

1 Gabbeh designer-user Verifying

27 each of these options should be a link to another page 1 Gabbeh end-user Altering

28 here we provide a link and short summary for each option. 1 Gabbeh designer-

user Verifying

29 no more changes is required 1 Gabbeh end-user Confirming

30 maybe you can separate the news page so it can be more interactive and updated now and then?

1 Gabbeh end-user Understanding

31 a link to publication page is added 1 Gabbeh designer-user Verifying

32 here we provided a list of your projects and experiments 1 Gabbeh designer-

user Clarifying

33 abstracts are moved to the publication page 1 Gabbeh designer-

user Clarifying

34 new pages are added for these options 1 Gabbeh designer-user Verifying

35 list of case studies with a brief overview 1 Gabbeh designer-user Clarifying

36 On the home page, there seems to be a lot of text, can you make it more interactive?


37 I didn't fine any information on accommodation?! Can you add one? 2 DENIM end-user Altering

38 how to get to Sheffield should cover all the means of getting to Sheffield 2 DENIM end-user Altering

39 add more information about the activities in Sheffield 2 DENIM end-user Altering

40 please check the updates 2 DENIM designer-user Organising

41 this home page consists of images and links to each individual page that describes each ac

2 Gabbeh designer-user Clarifying

42

we provide a photograph of each researcher together with links to their profiles and a quick preview of their interests at here.


43 An introduction on what researchers do on day to day basis here at Sheffield Hallam university


Amir M Naghsh 189

44 This page will generate the publications by year depending on where the time line is set to by the user of the site



45 We provide an area for some text about the group in addition to group photo 2 Gabbeh designer-

user Verifying

46 can we have a group link instead of individual one 2 Gabbeh end-user Altering

47 could you add links to host university and other research centres in this page? 2 Gabbeh end-user Altering

48 links to sponsored research inst 2 Gabbeh designer-user Verifying

49

this page hold an image of the researcher and their interests in more details, also include don this page is a list of their interests and current work. the courses that they teach are also included here


50 can you put the researcher name instead of about me? 2 Gabbeh end-user Altering

51 There should be a page listing all publications for this researcher. 2 Gabbeh end-user Altering

52 link will grab all publications by name and year supplied as an attribute of the link


53 current projects that are carried out by post graduate students at Sheffield Hallam university


54 can we have a small introduction on each project and also list of the people who are walking on that project

2 Gabbeh end-user Altering

55 This page will generate the publications by year depending on where the time line is set to by the user of the site –


56 Good idea... the time line 2 Gabbeh end-user Confirming

57 What do you mean by other visitors? 3 DENIM designer-user Exploring

58 Do you want to search the facilities as well as browsing? 3 DENIM designer-

user Exploring

59 We have not used animation sounds frames for the dialup line limitation. Is there feedback on this?

3 DENIM designer-user Verifying

60

Other users would include all the people who need to find some information regarding Sheffield. Students, businessmen and women and so on

3 DENIM end-user Understanding

61 I think we should not have problem with the speed 3 DENIM end-user Confirming

Amir M Naghsh 190

62 Can we have some pictures of key places on the home page? 3 DENIM end-user Altering

63 what do you mean by search and browsing? 3 DENIM end-user Understanding

64 add a page for how to get to Sheffield 3 DENIM end-user Altering


65 what's the search box for in the accommodation page? 3 DENIM end-user Understanding

66 We leave the news area in the homepage.. how regularly they will be updated?

3 Gabbeh designer-user Exploring

67 Do you need any automation for the news area? 3 Gabbeh designer-

user Exploring

68 what is the big box? 3 Gabbeh end-user Understanding

69 can you add e about us page or an space on this page to give a overview of the research group?


70 a group picture 3 Gabbeh designer-user Clarifying

71 this links to a new page 3 Gabbeh designer-user Clarifying

72 graphic/pic 3 Gabbeh designer-user Clarifying

73 list of staff names and link to their homepages 3 Gabbeh designer-

user Clarifying

74 can we separate these people in different groups like researchers, staff , post graduates and so on?


75 this is your profile 3 Gabbeh designer-user Clarifying

76 can I have a link to my publications and one to show the projects that I am working on?


77 requested sections were added under my research. 3 Gabbeh designer-

user Verifying

78 information regarding a course, e.g. web based 3 Gabbeh designer-

user Clarifying

79 can we have a summary for each project? 3 Gabbeh end-user Altering

80 This page list the publications by year and by project 3 Gabbeh designer-

user Clarifying

81 can you give more details for each publications? 3 Gabbeh end-user Altering

82 the project page, project overview and people involved 3 Gabbeh designer-

user Clarifying

Amir M Naghsh 191

APPENDIX C

This appendix presents the design task used in the longitudinal study, as well as the results of the study which was used for the analysis of Chapter 9. This appendix also presents the final version of the prototype that was generated by end of the longitudinal study.

A T-SHIRT WEBSITE

Today popularity of online purchasing is increasing. Customers prefer to buy their needs online. Internet has provided a limitless opportunity to shop online. An internet user can surf online, finds its own needs, compares prices and decides in time. There is no pressure on the customer by the shop assistant to decide whether they want the product or not. People can shop in the middle of night. Internet is not limited to where you are, who you are or time. The other advantage of online shopping is buying and sending present to others. This service is not offered by the most of the shops in high street. Therefore there is a need for a rapid and user friendly website offering wide ranges of services for its users. As clothing is a big market including all classes of the people, we are looking forward to selling high quality t-shirts on line. The website should be powerful and easy to use. Every individual apart from its age and gender should be able to use it and shop online. It is needed that the website provides an excellent service for those professional who are looking for unique services. On the other hand, the website should offer strong designing tool for those who are not satisfied with ready products and desire their own design. Therefore the website covers two categories:

• Ready t-shirts category

• Design tool

READY T-SHIRTS

This category contains t-shirts designed by the Company. The category should be subdivided in different classes depends on the users interest.

Amir M Naghsh 192

DESIGN TOOL

This section of the website offers the user a powerful tool to design its own t-shirt. This is the main feature of the website. The user can select type of the t-shirt, colour and size. Also the user can add its own logo, sign or by using website toolbar can create a one. Also the users should be able to select position of its own logo on the t-shirt.

After selecting the t-shirt, the user can save its own favourite design or t-shirt; or proceed to purchase it. Each user can create an account. The account shows billing history, order history, purchase history and users information. Also there should be a section for tracking an order.

Amir M Naghsh 193

RESULTS Page Author ID Reply Comment Classification

designer 85 This will be in colour and more attractive! Clarifying

user 117 85 It is very good. I think we are making progress. Confirming

However, I think the photo should be in the middle of the page and the comment will come beneath it. Altering

designer 134 117 Ok, I have moved the picture to the middle, Clarifying

is it what you want? Verifying

user 148 134 A recent survay shows a new visitor only Justifying

spends 20 sec on the first page, deciding

to continue surffing or leaving! It is very important to impress the visitor at first sight. So this page is very important.

Words should be selected wisely to attract

any group.

designer 3 Will there be just a 'tshirt' image or 't-shirt on 'cool guy' and/or 'cool girl'? Exploring

user 23 3 It should attract people in first click, Understanding

so on this page Several photos could be used, Altering

each photo can represent a catogery. designer 86 23 Maybe something like this? I think Verifying

it is good to have top sellers here. Clarifying user 118 86 It is good Confirming

the photo change randomly. Altering

designer 135 118 yes, it would be more effective and it will be like that in the Clarifying

actual website. Keep this in mind that this is only a

simple prototype and it is does not support that kind of annimation

but once we agree on the design, I can begin

the development of the actual site which would support

that sort of annimation. user 150 135 I look forward to that feature. Confirming

Hom

e

Amir M Naghsh 194

user 24 I don't like "Have Fun!" Confirming

designer 49 24 have fun is a much more customer Justifying oriented way to get them to create! see sites like www.innocentdrinks.com they do this, and it works very well

as a marketing tool user 64 49 Have fun is a common phrase which Justifying

everyone uses it.I am looking for something especial, something that no oneelse has used it before. Such as:"select, Altering design, buy & wear!"

designer 74 64 I have changed it incorporating your suggestion Clarifying

user 119 74 We are getting close! Confirming user 69 49 Yes you are right for drinks, for example Justifying

no one buys drinks online from that website,

that might works for that kind of the company

but I want to sell t-shirts online. That company is

selling its products in stores not online. The website

is part of their customer service. Thus, we want

to sell online, so should attract people as well as

it should be professional.

designer 73 69 I see what you mean, but its about identifying Justifying

with the customer, and creating an "online customer service" - anyways, if you dont want 'have fun' Exploring

please send through an alternative phrase

user 126 It would be nice if the website shows name Altering

of the returning user on the top of the page,

such as: "Welcome back....". Also there should be

an option to reset the name when other user

is using same computer! designer 138 A welcome message was added to Clarifying

the design page for the returining users, but I also have added that to here.

Hom

e

Is this what you wanted? Verifying

Amir M Naghsh 195

user 151 It would better to say: Altering If you are not Name" please click here".

user 159 How about the idea of creating a category Altering

named "innocent t-shirt" which the user can

add a logo or a message related to a charity

or peaceful group?

designer 166 159 sure, here I am only providing a simple Clarifying prototype, you can change the content when I begin the designing of the actual website.

designer 16 One section is "catagories" what other sections would you like here Exploring

Male/female? Size? Colour? Anything else?

user 32 16 On this page we need to state only main catogeries: Altering

The user should be able to find her/his favourite catogery, for example :

adult t-shirt, under 18, Fun tshirt, plain t-shirt

Romance t-shirt etc designer 55 32 Ok, the catagories can be added in, Clarifying

and these will probably be represented in pictures of the catagories

user 70 Users should directed to a new page after Altering

selecting their category. Then they should

be able to select size, colour etc. Please visit

tshirthell.com for more ideas. designer 78 70 yes, this will be the case Clarifying

user 128 Where are the new pages for each category? Understanding

designer 168 128 not designed yet Clarifying

user 120 I want the photos change randomly and be updated Altering

with the news designs.

T-sh

irt

designer 140 120 it will be like that in the actual site Clarifying

Amir M Naghsh 196

user 152 140 So would you please make links from this Altering

page to the actual site. designer 169 152 I have to remind you that it is only a Clarifying

simple prototype by actual site I mean the website which I will develope later from this prototype

user 153 It will be great if the name of the user be shown Altering

on each page or make an option for the

new users to make account if they decide

T-sh

irt

to do so.

user 72 I was thinking about returned users. How the Understanding

page will be shown for them. designer 101 72 they can log in using a username and Clarifying

password.

(click on the link to see how it would be when users log in) Organising

user 121 101 Please see my comments on the new pages. Organising

user 129 Can the website allow all format of the photos Understanding

to be uploaded? Is there any limit for file size?

designer 143 129 well, it is a more of a technical issue, Clarifying

yes it would allow all the formats, but in regards

to the limit, it would deponds on your hosting

contract. I might be able to get more information

on different available deals in the market and give you some prices

and then you can decide on the limit.

user 30 Here, the user should be able to choose Altering

t-shirts models and designs such as: Coulor, Size, Patterns etc.

By choosing each section the preview window

should be updated.

Could you please put a toolbar for the design Altering

tools.

designer 58 30 The middle box here is the design tools - Clarifying

Des

ign

my apolgies thats was not clear

Amir M Naghsh 197

user 68 How about changing the heading to: Altering

Design & Buy designer 91 68 I don't think that sounds good for Justifying

a heading but I have changed it! Clarifying

user 123 91 Would you please synchronise all the headings. Altering

designer 142 123 I have changed the Design & Buy, to "Design" Clarifying

The user is not really buying anything in this page, Justifying

User can design a pattern here and then add the

finished design in to the basket, and that's where

user can buy the t-shirt. user 155 142 Agree! Confirming

designer 20 Here is the page for customers to customise their choices Clarifying

designer Will there be an option for them to upload thier own pictures? Verifying

user 29 20 Yes, we need that feature Confirming

user 31 Also we need a feature for the user to Altering

be able to open an account so he could save

his work in case of the future use.

designer 57 31 I have added a log on for existing customers Clarifying

user 63 57 How about new users? They should be able to open a new account. Altering

designer 79 63 there should be a button for that, yes. Clarifying

user 158 Can you change "save" to "save as..." Altering

Des

ign

designer 108 A list of pictures uploaded by user Clarifying

I think it is important to have, make it quicker for the user to reuse

a logo, if he does not have a fast internet

connection to upload it everytime

user 124 108 We can give the users ability to name their Altering

designs and save it under their own name!

designer 144 124 yes, I think that's what I meant. as you can Clarifying

D

esig

n an

d B

uy

see there are two options, one is saved

Amir M Naghsh 198

designs, and the other is saved pictures. Saved design is a list of desing names.

user 156 144 Is it possible to have a preview window next Understanding

to the designed names, just to remind the user

a taste of the designed t-shirt. designer 164 156 Yes it is possible, the preview screen Clarifying in middle of the page is designed to allow a user to preview previously

designed t-shirts as well as new ones.

designer 107 A list of saved to designs, Clarifying user can re-use them, or D

esig

n an

d B

uy

edit them further

user 131 The user should be forwarded to a secure page Altering

for entering credit card number.

designer 172 131 yes it will be like that in the actual website Clarifying

user 130 The user should be able to track old orders or Altering

purchases as well as a complete history of

all transactions.

user 132 Also it would be great if the system keeps Altering

a record of the user's card number in case

for speeding up the purchase (if and only if

the user wish) which can appear as an

Bas

ket

option. designer 110 This option is for Clarifying

user 157 Can you add an option for sending news emails Altering

to the user email address, for promoting our new designs or products?

designer 165 157 yes it will be added Confirming

user 122 Before saving the user should accept the company Altering

New

Use

r

policy. designer 146 122 Ok Confirming

Amir M Naghsh 199

user 28 Please change page name from "about us" to Altering

Corporation designer 51 28 i dont want to put 'coporation' again, Justifying

its off putting, a coporation sounds like a PLC, when really we want to get the mass market - about us, helps the customer to 'relate' to us (its a pyschological marketing attempt)

Abo

ut U

s

user 66 51 People should think that a professional Justifying

group who are doing marketing, finance, delivery etc.

are behind this website.

So those who have concern about the company

will come to this page. Therefore this page should be very

professional with strong message insurring visitors.

designer 75 66 still to me coporation sounds "we are bigger than you" Justifying

when that is not the essense of the site, unless there

is a way (like amazon) to allow the customers' name

to appear, therefore they will privaliged

An alternative suggestion "company" user 125 75 ok, "company" is good in that case. Confirming

designer 5 > doing good things? Clarifying

A page highlighting your companies 'donation' of 10p per shirt to a good charity

user 25 5 Does this mean a new page? Understanding

Have you designed anything for it? designer 52 25 Yes, three more pages are needed Clarifying

user 160 How about saying "innocent t-shirts"? Altering

designer 170 160 it can go in to the Clarifying

designer 15 Do you want a page on 'personnel' within the company? Verifying

user 27 15 No, I don't think that would be needed Confirming

designer 76 27 ok Confirming

Abo

ut U

s

designer 4 > news Clarifying

Amir M Naghsh 200

this will feature a separate page of latest news

Is this something that you want? Verifying

user 26 4 I need a link here to a new page containing Altering

company's news. designer 77 26 news is also inclusive of company news Clarifying

designer 6 > our story Clarifying

The story behind the 'company' formation

user 67 I think the order should be like this: Altering Home

t-shirt Design contact us corporation

Abo

ut U

s

I suggest you to visit good websites such as

amazon! designer 99 67 ok Confirming

designer 19 Should you provide a page on "good links" Verifying

user 33 19 No, This feature is not necessary. Confirming But what do u mean by good links? Understanding

designer 17 Do you want a page where 'customers' can leave their comments? Verifying

user 34 17 yes, feedback feature is important for the Confirming

company. But I was thinking to have it in Altering

a seperate page. designer 59 34 yes, i agree, feature should be on Confirming

a separate page, titled "comments" Clarifying

designer 60 I think you have to feature address, Justifying otherwise people wont feel confident

C

onta

ct U

s

to buy from "unknown" companies, especially as it is a new company

user 161 Can the system send a thanking message to the Altering

user after sending his cooments to show

Com

men

ts

our appreciation for his/her feedback?

designer 171 161 Yes Confirming

Amir M Naghsh 201

FINAL DESIGN

Amir M Naghsh 202

Amir M Naghsh 203

Amir M Naghsh 204

APPENDIX D

Figure D DENIM UML diagram

4

Documents

ELECTRONIC PAPER-PROTOTYPING TOOLS · 2007-09-11 · characteristics of prototyping approaches. This thesis has extended this work by applying the full Cognitive Dimensions framework