Worlds of information: designing for engagement at a public multi … › u › jacucci › p2267-jacucci.pdf · Worlds of Information: Designing for Engagement at a Public Multi-touch

Worlds of Information: Designing for Engagement at a Public Multi-touch Display

Giulio Jacucci12, Ann Morrison1, Gabriela Richard3, Jari Kleimola1, Peter Peltonen1, Lorenza Parisi4, Toni Laitinen1

1Helsinki Institute for Information Technology HIIT, Aalto University, Finland, [email protected] 2School of Art and Design, Aalto University, Helsinki, Finland,

3Educational Communication and Technology, New York University, USA, [email protected] 4Facoltà di Scienze della Comunicazione, Sapienza Università di Roma, Italy

ABSTRACT In designing for engagement at a public multi-touch installation, we identified supporting multiple users and allowing for gradual discovery as challenges. In this paper, we present Worlds of Information, a multi-touch application featuring 3D Worlds, which provide access to different content. These 3D widgets gradually unfold and allow for temporal navigation of multimedia in parallel, while also providing a 2D plane where media can be shared. We report on a field trial at an exhibition using questionnaires and video ethnography. We studied engagement through questions adapted from Flow, Presence and Intrinsic Motivation questionnaires, which showed that users, overall, had a positive and social experience with the installation. The worlds effectively invited multiple users and provided for parallel interaction. While functionality was discovered gradually through social learning, the study demonstrates the challenges of designing multi-touch applications for walk-up-and-use displays.

Author Keywords Multi-touch interface, parallel interaction, design for engagement, public display, 3D multi-touch, field trial.

ACM Classification Keywords H5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous.

General Terms Human Factors, Design, Experimentation

INTRODUCTION Despite the considerable attention given to large multi-touch displays in research, their application is still largely driven by the novelty of the interface. Even with a long

trajectory in the history of the development of multi-touch systems dating back to the mid eighties, we are still at the early stages in a learning curve of best uses and the unique added value of such systems. A recent example is CityWall, a large multi-touch display installed from May 2007 to the present in a central location in Helsinki—the first time in an outdoor setting. This provided citizens playful access to pictures of their city on a single timeline. We previously reported on observations of encounters at the display, finding that most of the sessions included multiple people interacting simultaneously [20]. The use was characterised as being driven by the playfulness, ease of use and novelty of the interface. The singular timeline created conflict with parallel use of the display.

In this paper, we address the problem of designing for engagement and parallel interaction with a walk-up-and-use display. Our hypothesis is that a multi-touch interface with multiple 3D widgets (worlds) can support parallel interactions at a public display. A major feature of the revised design is the incorporation of parallel worlds as 3D widgets in a newly created application called Worlds of Information. We discuss how to design for and evaluate engagement in a public walk-up-and-use installation. We report on a field trial in an exhibition where Worlds of Information was exhibited for three days. The field trial utilises video ethnography and questionnaires to provide formative insights and implications to our design solution. In particular we analyse to what extent the design supports parallel interaction and allows for a gradual discovery of the functionality in order to sustain interaction.

RELATED WORK We are interested in contributing to designing applications on public multi-touch screens with particular focus on engagement and group use. To this end, we first review research on collocated interaction on interactive surfaces and then review work on multi-touch solutions that propose widgets in support of advanced collocated interaction.

Collocated Interaction on Interactive Surfaces Many studies of collocated collaboration without computer support are relevant to our analysis and have inspired our work, such as Robertson’s [22] work with a taxonomy of

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. CHI 2010, April 10–15, 2010, Atlanta, Georgia, USA. Copyright 2010 ACM 978-1-60558-929-9/10/04....$10.00.

CHI 2010: Public Displays April 10–15, 2010, Atlanta, GA, USA

2267

embodied actions or Isenberg et al.’s [15] study that pointed to the importance of supporting individual temporal arrangements of tasks.

Studies of public wall and tabletop displays. Studies on public displays generally are done in the field and address user approach and observation, positioning of displays, bystander behaviour, and public versus private or individual versus group use [3,4]. Table-top studies tend to be more experimental and focus on more detailed aspects; for example: examining the influence of the position of a tabletop relative to pupils [21]; exploring the types of tabletop territories, like personal, group, and storage territories [26]; investigating multimodal interaction [29]; analysing turn taking problems and role taking [23]; and probing how pairs frequently and fluidly engage and disengage with group activity [28].

Previously, we detailed the social usage of an interactive public installation, CityWall [20]. We described how passersby approach and form crowds, engage in parallel and collaborative interaction, social learning, conflict management, negotiations of transitions and handovers, as well as playful and performative interaction [17,19]. The use was characterised as being ephemeral, driven by the playfulness of the interface, and permeated by unnecessary conflicts as the interface did not support parallel interaction well.

Guidelines and design Principles. Some of these findings are reflected in several design guidelines and principles. Scott et al. [25] suggest particular design guidelines for digital tabletop display interfaces for co-located collaboration: fluid transition between activities, interpersonal interaction, transitions between personal and group work, and simultaneous user actions. These resonate with the guidelines of Tang et al. [28] to support a flexible variety of coupling styles (see also [12]) and lightweight annotations, which provide fluid transitions between coupling styles and mobile high-resolution personal territories. Recently, Hornecker, et al. [14] presented design principles for shareability. They note the central role of access and entry points for tangible interaction. All of these factors produce the shareability of the system, which refers to how a system engages a group of collocated users in shared interactions around the same content. While guidelines and principles are useful, they still need to be translated to particular solutions. Our contribution is a design solution addressing, amongst other things, shareability, transitions between personal and group work, simultaneous user actions, and personal territories.

Related interactive surface widgets Some widgets have been developed that allow for the creation of multiple and simultaneous spots of interaction of the same data. DTLens [10] is a zoom-in-context, multiuser, two-handed, multi-lens interaction technique that enables group exploration of spatial data with multiple individual lenses on the same direct-touch interactive

tabletop. Interface Currents introduced by Hinrichs et al. [13] are containers that provide a controllable flow of interface items that can be, for example, positioned all around the boundary of the screen. In order to make mode (or view) switching more convenient, Everitt et al. [9] introduce Modal Spaces, which divide the display into four semantic workspaces (or modal regions) that each interpret the same gestural commands on the same items differently. These implementations are applied for tabletop and collaborative work. While our research addresses a wall display with a public installation, we find the metaphors of lens and currents relevant to our work, even though they presently only address parallel interaction and engagement at a walk-up-and-use display in a limited manner.

THE DEVELOPMENT OF WORLDS OF INFORMATION

Design Challenges Multi-touch interfaces are a new solution for walk-up-and-use displays. According to our experience with passersby, local institutions and stakeholders, it is important to allow for a variety of content, themes, and categories within the display. Multi-touch can potentially provide intuitive interaction capabilities and create a playful environment that engages users. However the challenge is to move beyond ephemeral interactions, driven by the playfulness of the interface, and to encourage users to pay attention to the content also exploring more complex functionality. Multi-touch is groundbreaking because it affords multiple hands and users to manipulate the same surface. Parallel interaction is beneficial in the way that it fosters social learning, social experience and creates the attractive honey pot effect [14].

Engaging experiences. A “Walk-up-and-use” system needs to be so self-explanatory that first-time or one-time users need no prior introduction or training [7]. We saw that by implementing our design changes we would disrupt this ease of use, and while we also aimed to extend the scope of the interactions beyond this early learning curve, we looked for a balance, with the aim of extending the depth of the experience. The engaged experience we were after can be thought to involve aspects of presence, flow and intrinsic motivation [2,6,16,30,27]. We wished to retain ease of first use, and structure complexity in a scaffolded way, unpacking the functionality and content gradually as one mean of enabling sustained interaction [18]. Our aim was to immerse our participants in both solo and joint activities, enabling social and spatial immersion in a mixed reality environment, where participants could act with the contents ‘as if it was real’ (as we see with presence research). We see similar phenomena in accounts of flow [1]—optimal experiences in which ‘attention can be freely invested to achieve a person's goals,’ which results in the merging of action and awareness, as well as a consequent lack of self awareness and distortion of sense of time. When participants are engaged with both the product at hand and with others in the collaboration, we can characterise this as group flow [24], similar to what is investigated with social


2268

and spatial presence research. Similarly, we wanted the user and group experience to be intrinsically motivated, or inherent, for its own sake and an end-in-itself. John T. Guthrie [11] makes a case for connection between high levels of engagement and intrinsic motivation, with the promotion of goal-orientated activities that involve understanding content, using effective strategies, and making links between old and new knowledge (in opposition to performance-related activities).

Parallel interaction. Parallel interaction includes the possibility for giving access to multiple content and to support the parallel interaction of multiple users. In our earlier 2D interface implementation, we treated the entire display as a single interaction space, meaning that one user’s actions often had effects on the actions of another user. For example, resizing an image to a very large size might overlap another user’s focus of interaction, and moving the single timeline means disruptions for others because all the photos in the content then start moving left or right accordingly. As with Isenberg et al. [15], we needed to allow individuals and groups to be engaged in different types of processes at the same time (or to work together using the same processes).

Design Solution

Worlds of Information Working towards these design goals, we sought to develop an interface that would allow us to present large amounts of multi-themed content – originating from a multiplicity of content sources – in a way that also affords parallel interaction. We investigated several solutions as to how these views could be presented in the interface, including (i) dividing the screen into vertical panes, (ii) using overlapping transparent layers and manipulation handles, and (iii) using timelines in either x or z-axis dimensions. Because these solutions did not work well in practice, we settled for an alternative that shares the entire interaction space between interacting users. We found that using multiple virtual 3D container objects (spheres or widgets), sitting on the display side by side, would offer a feasible solution. Each virtual 3D sphere could provide an individual interaction access point, with an independent timeline, and a collection of these 3D spheres would then enable parallel interaction within a shared display space. As the overlying theme of the work was environmental awareness, worlds proved appropriate conceptual and functional 3D metaphors for the containers, and were shapes that could readily expand to add more layers of information. The envisaged two-meter screen could easily

accommodate multiple spinning spheres, or as we now came to think of them, Worlds of Information, each with its own theme (see Figure 1). We used six individual globes that contained themed information, in the form of images, videos and text. These worlds housed images of Helsinki since 2007; images of the venue of the installation; videos of state of the art multi-touch systems; SMS, MMS, and email messages sent to the system; help animations; and images from participants of a nearby installation. The hardware of the installation is built similarly to CityWall [20], with its framework and application running on top of .Net 3.5 and Windows Presentation Foundation (WPF) frame-works, which were written using a combination of object oriented and declarative languages (C# and XAML).

Opening and Navigating a World Stretching the sphere over a certain threshold size opens the world, while resizing to the opposite direction will shrink the world back to the collapsed state. In the opened state, the container sphere is coated with 2D plates, each holding an information item belonging to the themed timeline. An opened world can be further enlarged, moved along and spun around the x- and y-axis, in order to browse the photo, video and text items attached to the sphere. Spinning the world rapidly around its y-axis (i.e., to the left or right) allows navigating back and forth through time to view older and newer content according to the theme of the world: the current layer of coated items is replaced with another layer of items from preceding or succeeding dates. To this end, the sphere consists of multiple stacked layers of content, which can be exposed by peeling or spinning actions (see Figure 2A). It is also possible to jump directly to a specific date by activating the navigator menu (see Figure 2B). The menu items in the equatorial circle represent days of a month, while the items in the longitudinal circle denote months and years. Both circles can be spun around to make a selection of the date that is presented with the larger frontal item.

Pieces of Information The pieces of information contain images, videos and text items that are tagged with a title, author and timestamp metadata. They are visualised using 2D plates that are inclined in 3D to cover the spherical surface of a world. A single content item can be selected for closer inspection:

Figure 1. Different worlds with different themes. The worlds are

shown here in their collapsed state, hinting the themes with distinct wrapping textures, while hiding the actual contents.

Figure 2. A. Navigating backwards in time, current layer is

exploded out while a new layer is faded in. B. Navigator menu, date label at the bottom of the sphere.


2269

this action rotates the world to an angle which brings the item to the front of the sphere, where it appears larger that the other items (see Figure 3A). This close-up position allows them to be resized and flipped around to read the associated comments (Figure 3B). It is also possible to make copies of the close-up item, and add these to the communal 2D front plane of the interface. To support co-operative interaction tasks further, we overlaid the entire display space with a virtual transparent interaction area (see Figure 4). In addition to the copied content, the front plane holds recent text messages, which can be moved, resized, rotated, played and dismissed by any user. This horizontally scrollable layer corresponds to the 2D content area of our earlier implementation. Consequently, the ability to enlarge the items and the worlds, and to overtake the whole display area, ensured that we maintained the accidental parallel and associative interactions that had enabled sociability between relative strangers at our previous implementation.

Help System The help topics are presented as spheres that are in constant motion. The main help world travels slowly around the interactive space, emitting slogans that encourage people to try the interface (occurring only in periods of inactivity at

the interface). The system also contains small help spheres travelling at a faster speed. The idea is that people become engaged in catching these help spheres. Once caught, the sphere would open up and show its contents (main help), or play a short animation (topic) that explain the gestural language of the interface (Figure 5).

Gesture Language The challenge of the gestural language design was that the virtual objects in the 3D space require six degrees of freedom (DOFs) to be manipulated in full detail (i.e., translations and rotations in all three dimensions). In contrast, the multi-touch input is sampled from a 2D surface, giving only 3 DOFs (translation in the x/y dimensions and rotation around the z-axis). The functionality of the system introduced in the previous sections, can be summarised as shown in the first column of Table 1: 1) drag – moving fingers on the surface, 2) stretch – two or more fingers moving in opposite directions or towards each other, 3) circle – two or more fingers moving in a circular fashion, and 4) flick – a rapid dragging action, 5) press - touching the surface and holding the finger down for 2 seconds. The mappings are presented in the three rightmost columns of Table 1. To distinguish these operations we had to count the number of touch points on the sphere: 3 fingers or less triggered spinning, and 4 fingers or more triggered the movement operation. During the development we performed a series of informal lab tests. Table 1, for example, contains several cells marked with (x). These mappings were not included in the language, because we noted that it became increasingly difficult to control the 3D model when the number of DOFs was increasing. Finally, we also experimented in using double tap gestures in place of press gestures. We had to choose the latter because external disturbances resulted in falsely triggered taps.

FIELD TRIAL AT AN EXHIBITION To evaluate Worlds of information we conducted a field trial at an exhibition. This setting was considered appropriate to observe how our system would be experienced and used in a public context. The field trial was aimed in particular at a formative evaluation as part of iterative design cycles. The theme of the exhibition was “Science at the service of Society,” and 80 different

Figure 4. 2D front plane and 3D worlds co-exist

Figure 5. Examples of opening, closing and spinning gestures

Figure 3. A. Content selected and enlarged. B. Flipped around

to read associated comments.

Operation World (3D) Item (3D) Item (2D) open/close stretch press stretch spin drag move drag (x) drag resize stretch stretch stretch rotate (x) (x) circle throw flick flick flip flick copy (x) drag (x) play/stop press

Table 1. Gesture mapping. Column 'item (3D)' lists gesture mappings of a content item attached to a world, while column

'item (2D)' lists mappings of a front plane item


2270

projects from around Europe related to this theme were selected from about 250 applicants to showcase their research. IPCity was one of the selected projects, and we exhibited a portable version of Worlds of Information. For this exhibition, we expanded upon ways that participants at the exhibition could input by adding SMS, MMS, emails and tagging local to Paris.

Data collection Both video data and surveys were collected as part of the field evaluation. The video data provided observations of participants in-situ across three days of the exhibition. In an effort to reduce bias, a visiting researcher completed the video analysis, and two visiting researchers analysed the survey data. The surveys were not compulsory and a convenience sample of 101 users completed them.

Survey Data The surveys contained descriptive information about the participants, and part of the survey utilised previously validated questions from presence and flow scales to analyse users' perceptions. Our questionnaires were designed to cross-check the relationship between the states of flow, presence, immersion and intrinsic motivation, as indicators of levels of engagement. Eighteen Likert-type items, rated on a scale of 1–7 were analysed. Participants completed shortened versions of a MEC Spatial Presence Questionnaire (MEC-SPQ), a GameFlow questionnaire and an Intrinsic Motivation Inventory (IMI) to gauge reactions to the display [2,6,16,30,27]). For Presence, we asked users to come up with five words to describe the experience and measured concentration, errors, activated thinking, and imagining space. For IMI, we measured interest/enjoyment, perceived competence, pressure/tension, and effort/importance. For Flow, we measured challenge-skills balance, goals, concentration on task, and sense of control. For social presence, we added questions under development and validation through our research project that investigates presence and interaction in urban environments.

We examined the data to understand how users perceived the system and whether there were differences between types of users. There is a similarity in the kinds of states being queried with presence, flow and intrinsic motivation research, even though different language is used. For example, where presence inquires into levels of activated thinking, flow queries conditions required to achieve an optimal state, and intrinsic motivation queries how people perceive they did, and how motivated they were to play with the work for its own sake. These are comparable states of experience, alongside concentration and enjoyment, also queried across all questionnaires. Social presence (awareness and sharing with others) has similar parameters as engagement with others (asked in both Presence and Flow questions). Flow, engagement, presence and intrinsic motivation are elusive concepts, and, as such, hard to measure. It is difficult to measure how engaging the user experience really is, and so we cross-checked with similar

categories from different evaluation methods, rather than just pursuing one system in order to mine for richer information.

Video Data For each of the three days of the exhibition, several hours of video were recorded. All of the three days of video data were analysed using the third day of the exhibit as a purposive sample for more in-depth analysis. In addition, two hours of continuous video footage was analysed using Erickson’s [8] method of “microanalysis.” This technique is particularly useful when trying to understand the common and distinct elements of events that occur. The video data was examined to understand how individuals, groups and pairs configured themselves around the system; what system states occurred as a result of the interaction; how users worked together or separately; what sort of interaction techniques users employed; how users learned how to use the system; and how the interaction sessions were structured.

Limitations First of all, the surveys were not compulsory so we were only able to obtain a convenience sample. Further, some of the items on the surveys had not been previously validated, and only a sub-sample of previously validated items was included. Additionally, the field trial took place during an exhibition, which is a limited public setting, with certain kinds of users. These factors limit the generalisability of our findings. However, for the purpose of our redesign, the setting and the sample provided crucial information that would be informative to future development of our display.

User Experience and Impressions from the Survey Data We received 101 filled in surveys, of which 64% of respondents were males, and 37% were females. Ages of respondents ranging from eleven to sixty seven years. The average age of individuals who completed the questionnaire was 29. Overall, the user population that completed the surveys would be considered frequent ICT users. In response to expertise in ICT, 54.5% reported having average expertise, 25.7% reported having expert knowledge and 18.8% reported having basic knowledge. Respondents spent, on average, 32 hours with ICT though individual use, ranging from 0-80 hours (Median=30, Mode=60). Of those who responded to educational background (81%), almost all had received or were receiving post-secondary education (83%) with the exception of secondary school student respondents (15%) and 2 adults. The majority of respondents claimed frequent use of the web (95%) and mobile phones (91%).

Positive Impressions with Some Issues Eighty-nine percent of users responded to what they liked about the system. The most cited reasons for liking the system were its simplicity/ease of use (12.9%), interactivity (12.4%), tactility or multi-touch (10.1%), fun or playful nature (7.3%), novelty (6.7%), technology (6.2%) and


2271

intuitiveness (5.1%). Other responses cited its versatility, futuristic nature, fluidity, social capacity, and multi-user compatibility. Seventy percent of users responded to what they didn’t like about the system, of which 22% stated they didn’t dislike anything. The most cited reasons for disliking the system included multi-touch feedback or reactivity problems (16.3%), poor definition of images (13.5%), incomprehension of the system (10.8%) and problems with the interface (6.8%). In response to presence, the most cited words or feelings conveyed by users were fun/play/amusement (13.4%), amazement (7.6%), innovative (6.7%), high tech (6.1%), interactive (5.5%), simplicity/ease of use (5.2%), fascinating/involving (4.9%), tactile (4.7%) and interesting (4.4%).

An Immersed and Interested Experience Likert-type scales measured IMI, GameFlow and social presence. Thirteen statements were adapted from the IMI and GameFlow scales as well as the IPCity forging new territory questionnaire (see Table 2). Items 5, 7, 8 and 9 were adopted from the flow scale; items 10, 11, 12 and 13 were adopted form the IMI scale and items 1, 2, 3, 4 and 6 were inspired by Presence questionnaires. An additional five statements were adopted from the presence scale (see Table 3). One sample t-tests with an assumed mean of 4 (the midpoint of the Likert scales) were used to analyse the items to understand the significance of the responses to the users’ experiences with the system. Most of the responses for the IMI, GameFlow and social experience items were statistically significant, demonstrating that, on average,

users found the system fun, easy-to-use and understandable, and that they were relaxed using it. Further, they felt their skills increased over time. Similarly, four of the five items on the presence and user experience items were statistically significant. In general, users felt that they didn’t experience technical issues, that they concentrated on the tasks and that the system activated their thinking. We compared answers to all of the Likert-scale responses through 3 different sets of analysis. We first ran t-tests comparing professions to see whether there was a difference between responses for people who worked or studied in the technology field (engineers, computer scientists and designers) versus those who indicated they did not. Then we ran one-way ANOVAs comparing individuals who labelled themselves as basic, average or expert ICT users. Finally, we ran t-tests comparing males and females in their responses. For the profession comparison, there was a statistically significant difference between those who worked in the field and those who did not (see Table 4), with those in the field finding it less interesting and those outside the field feeling less competent using the system. There were no statistically significant differences for the other analyses. We were also interested in understanding whether there was a difference in experience between those who would use the system in the future (N=75) and those who would not or weren’t sure (N=12). There were statistically significant differences in responses with the following items: “I understood what the immediate tasks were and what I needed to do to achieve them” (t(82)= -3.03, p=.003), “I felt relaxed while doing the tasks” (t(81)= -2.325, p=.023), “I thought the tasks were very interesting” (t(81)= -2.89, p=.006), and “by the end I felt I competent using CityWall” (t(80)= -4.098, p<.0005), with those who would not return responding statistically significantly lower than those who would.

Findings from the video analysis For the participants we video taped and observed using the

Item M M Diff SD t df 1. I concentrated on the tasks and/or technology 4.97 .966

(**) 1.835 4.938 87

2. I did not experience technical problems when using the system 4.96 .956

(**) 2.076 4.393 90

3. The system activated my thinking 5.16 1.165 (**) 1.740 6.386 90

4. When someone shows me a floor plan, I am able to imagine the space easily

5.39 1.393 (**) 1.850 7.104 88

5. I concentrated on the content 4.04 .044 2.098 .201 89

Table 3. Other Presence Questions. (*)= p<.05. (**)= p<.01.

Item M Mean Diff SD t df

1. I felt I was sharing the same inside-the screen space (virtual) with others. 5.28 1,277

(**) 1.931 6.41 93

2. I often changed by actions in response to those of others. 3.89 -.110 2.030 -.516 90

3. I felt I could move objects and images around in the virtual space freely.

5.91 1.909 (**) 1.487 12.38 92

4. I felt there was a shared experience between the people I was with. 4.99 .989

(**) 1.925 4.96 92

5. I found the system fun. 6.28 2.277 (**) 1.195 18.47 93

6. I understood the spatial relationships between the objects in the environment.

4.79 .793 (**) 1.992 3.82 91

7. My skill level increased as I progressed. 4.82 .815

(**) 1.809 4.32 91

8. I understood what the immediate tasks were and what I needed to do to achieve them.

5.10 1.098 (**) 1.729 6.089 91

9. I lost all sense of time while doing things. 4.26 .261 2.132 1.17 91

10. I felt relaxed while doing the tasks. 4.95 .945 (**) 1.797 5.016 90

11. I thought the tasks were very interesting. 5.12 1.123

(**) 1.798 5.96 90

12. It was important to me to do well at this task. 4.02 .022 1.803 .116 91

13. By the end, I felt competent using the system. 4.81 .811

(**) 1.890 4.07 89

Table 2. IMI, GameFlow and Social Presence Questions. (*)= p<.05. (**)= p<.01.

Item 11. tasks very interesting t= 2.45 p=.017

13. by the end I felt competent t=-2.04 p=.03

Group N M SD N M SD Work in ICT 32 4.55 1.76 33 5.34 1.56 Work outside ICT 51 5.51 1.75 51 4.49 2.01

Table 4. Two items differed for the profession profile.


2272

system, the ages ranged from infants (with families) to people in their 80s. An exact number of participants and their demographic information was not obtained, but the surveys were used to obtain descriptive information of a subset of the sample.

Supporting Multiple Configurations and Parallel Interaction We observed 20 configurations of use around the system, which are illustrated in detail in Figure 6. While each of these observations were unique instances of use, they often involved the same participants reconfiguring themselves around the system based on changes in the system, or their engagement. Often the configurations would perform like a dance, with users working alone, then collaborating and then working alone again, or vice versa. A configuration was labelled as individual if one person engaged in focused manipulation of one object or area of the screen without interacting with or avoiding interaction with other users. It was labelled as pair if 2 users began to manipulate an object or objects together or talk and interact with each other while manipulating objects. It was labelled as group if 3 or more users engaged in the same manner as a pair. Results are depicted in Figure 6.

We also analysed the overall individual use, pair wise use and group use, finding that the most frequent use was individual (47%), followed by pair wise (35%) and group (18%). Figure 8 shows the distribution of occurrences for different configurations. We can analyse the support for multiple use by grouping the above as interaction spots. Following this, we grouped as 1 interaction spot occurrences of 1 pair, 1 individual and 1 group, 2 interaction spots combinations or two of the latter, and so on. As can be seen from Figures 7 interaction spots account

for most occurrences with 4 or more still having a sizeable portion. Coupled with understanding the most common configurations of users around the system and how the users would configure themselves, we also were interested in understanding the average length of time that a user would interact with the wall and the average number of configurations they would take on. Analysis revealed that, on average, an individual would stay at the wall for 2 minutes and 33 seconds, and be part of 6 configurations. However, analysis also revealed that there were distinct differences between adults and children. Once the two groups were analysed separately, we found that children tended to interact with the wall much longer than adults. Children, on average, interacted with the wall for 4 minutes and 21 seconds, while adults stayed for 1 minute and 29 seconds, on average. As a result, children tended to take on 8.7 configurations, while adults took on 4.4.

Gradual Engagement We identified five distinct system states that influenced interaction and configuration by users. The states of the system are not linear, and the system can go back and forth (depending on use) through any of the states, with the exception of the first. During state 1, the system is at its initial state, where the worlds are closed and discrete interaction zones are apparent. During state 2, one or more of the worlds are open, but the interaction zones are still discrete. During state 3, the interaction zones are mostly separate but partially overlapping meaning that one or more worlds or objects partly intersects another initially distinct space. During state 4, at least one world or object is overtaking one-third or one-half of the space but there is still at least one separate interaction zone. During state 5, one or more worlds or objects completely takes over the space.

Everything starts with one finger. Users were most likely to attempt to manipulate objects with one finger initially, especially when not influenced by other users’ of the wall. One-finger interaction was often not a problem for users when trying to manipulate an open world or a picture. During those times, users would often be able to rotate a world or pull out a picture, which was most amenable to one-finger interaction.

Difficulties opening the worlds. One-finger interaction didn’t lend itself well to opening a world. In the cases where someone was successful, it was often by accident (i.e., pressing on the world for a long period of time or moving a finger around the world, hoping to spin it but instead opening it), or by observing someone else.

Figure 7. Configurations into number of interaction spots

Figure 6. User Configurations around the Wall, with a total of

195 occurrences.


2273

From one finger to two handed interaction. Often, one-finger interaction would become partial or full one-handed interaction or two-handed, one-finger interaction as users would attempt to enlarge or compress pictures or worlds. Often full one-handed interaction or two-handed, one-finger interaction starts accidentally (unless the user had observed someone else successfully using the technique) and becomes a more and more refined intentional manipulation. Intermittently, users would start with one full or partial hand interaction, but this typically happened in cases where they encountered the screen at state 3 or 4 and attempted to move an object or picture already situated on the screen. The use of two full hands for manipulating objects was less intuitive for users (unless they had observed someone else using the technique) but it was the most effective for enlarging objects, especially the worlds (see Figure 8). Users who stayed at the wall longer usually ended their session with two full-handed interaction, and influenced other users to do the same. A typical user would not start with two-full-handed interaction unless influenced by another user. One of the cases we observed involved a woman who attempted to open a world in the same way that she enlarged a picture (see Figure 9). She initially started with one finger interaction, flicking pictures around on the screen. Then she decided to open and close her hand on one of the pictures and discovered that it opened in response. She decided to try that interaction technique on a closed world. Unfortunately, she wasn’t successful in doing so. As a result, she adjusted to two-handed, one-finger interaction, which resulted in successfully opening the world. A second case studied involved a pair (see Figure 10). Two men were working together, manipulating objects with one full hand and talking. They accidentally started working with the same picture and realised that by having each of their hands on the screen, they were able to make a picture larger. One man learns, as a result, that he could use two full hands to make that same picture smaller.

Social Learning There were four types of techniques that users employed to understand the system: individual exploration, cooperative exploration, passive observation then attempt and imitation. Individual exploration is defined as one user testing out techniques with the system independently without observing or working with others. Cooperative exploration defines users who work together in pairs or groups to understand the system. Passive observation then attempt is defined when users watch others using the wall and attempt to imitate their use or try out their own strategies. Imitation is defined when users go directly to the wall (without observing others initially) and imitate how other users work with the wall while they are there. Most users would use a combination of two or more of these techniques when using the wall. The most frequently cited learning techniques were cooperative exploration and passive observation then attempt, which often worked in tandem. In another case, a man observes others using the wall but starts his interaction

with one finger. A young boy at the wall starts talking with him about the wall and shows him a technique he has used. As a result, he starts to successfully work on his own section of the wall. A woman comes to the wall and starts to work with him. In the process, he learns a new technique (how to turn a picture around to view comments) and shows her what he has learned. Imitation and individual exploration were less frequently cited, probably due to the nature of use at the wall, which was often continuous. Imitation worked successfully when employed in a similar vein to passive observation then attempt because users could learn from others around them. Individual exploration varied in success depending upon the interaction techniques users employed. For example, in the first case study, the user had observed several users successfully opening and closing worlds with two full hands, but she decided to start with one finger interaction. However, having watched others in their attempts may have helped her determine that two-handed, one-finger interaction would later be a better choice for opening the worlds. This may have been an attempt by her to understand if the system would be amenable to another interaction technique.

DISCUSSION AND CONCLUSIONS We set out to develop a multi-touch display for a walk-up-and-use interaction. We identified challenges for designing for engagement and for parallel interaction. Supporting multiple users and multiple content has been addressed concurrently by providing several separate worlds.

Figure 8. Interaction with two full hands was the most effective for enlarging objects, especially the worlds.

Figure 9. Case 1: A woman learning two-handed interaction.

Figure 10. Case 2: Pair learning two-handed interaction.


2274

Engagement was addressed through gradual discovery of content and functionality, which is particularly challenging in a public display due to the short character of sessions.

Information Worlds as Multi-touch 3D widgets Our solution adds a 3rd dimension to multi-touch interfaces that are generally 2D and applies the metaphor of Worlds, which is different from other metaphors used for similar purposes [10,13,9]. The interface solution we proposed worked but uncovered four problems and implications in particular for this type of display. (1) Users should be accompanied through the exploration of the functionality, for example, through the help spheres, which can be brought contextually to the attention of the user at the right moment or made more intuitive in design. (2) Gradual unfolding and discovering means that the interface should be adaptive to the situation and be able to provide a similar starting phase to all users. For example, the Worlds could be animated to go back more promptly to the starting collapsed state to be able to offer people exploration from the beginning [10]. Additionally, the starting collapsed state should be made more intuitive to open. (3) Advanced functionality should be brought to users’ attentions intervening in the interaction. Spinning and Timeline navigation, which was not found intuitive to users most of the time, should be made more visible and easier to understand. Similarly, the methods for uploading and sending content should be more obvious. (4) Finally, the display should provide support for managing territoriality in parallel interaction, controlling size and position of worlds. This could be used to better support stability of some of the configurations of people at the display by limiting the behaviour of the Worlds.

The Worlds Inviting Multiple Users We have shown how the multi-touch 3D widget supported parallel interactions. The observational data demonstrated that the most frequent configurations of users involved multiple individuals working in groups or pairs, and the instances of individual use that were highest were in tandem with another individual, pair or group. This demonstrates that the system frequently accommodated multiple users, and different coupling styles [10,12]. Another finding was that users were influenced by others, both through observation and collaborative exploration, as pairs and groups often influenced each other on the wall. Further, survey data indicated that users felt that they engaged in shared experience with others, but did not change their actions in response to them, indicating that they could share the space without compromising individual exploration.

Designing for the walk-up-and-use experience The responses to the survey data indicate that, overall, users responded positively to the system, finding it engaging, interesting and understandable. Engagement with the content was not significantly reported, indicating that further attention should be paid to the information users

interact with. Users started with one-handed or one-finger interaction and were less likely to engage in two-handed interaction without observing or interacting with others. Overall, the analysis supports Worlds of Information as a system that enabled different levels of use where users could explore the functionality individually or socially. Even though the system was found to be engaging and easy-to-use, on average, different groups of users (for example, people working in ICT) found the system less interesting and non-ICT professionals felt less competent, showing that supporting different levels of competence did not work perfectly. However, users, on average, found the interface intuitive and playful, which has also been found in former studies [20]. Further, users responded that motivation to play was intrinsic (without external reward). The 3D Spheres and the metaphor of the worlds proved to be effective solutions to provide mobile territories [10] and access and entry points [14]. In particular Worlds, when they are unused, invite passersby to interact, explicitly, even if someone else is interacting with another world. By adding another layer of complexity with gestures that move beyond the now familiar pinch, expand and rotate movements, we hoped to entice our participants to become more immersed in uncovering interaction techniques by pursuing varied options. By allowing worlds to overlap, participants were required to be aware of each others activity, and we looked to initiate forms of mutual engagement [10], where individuals can spark their curiosity together, and can lose themselves in a joint activity. Walk-up-and-use display can greatly benefit from multi-touch. However we found that not all users fully exploit the multi-finger and multi-hand features. The challenges ahead include providing easy access to relevant content through effective navigation mechanisms. The gradual discovery of more complex functionality should be supported adopting adaptive interface strategies.

ACKNOWLEDGMENTS This research has been co-funded by the 6th Framework Research Programme of the EU, through the IPCity project (FP-2004-IST-4-27571). We would like to thank Rodolfo Samperio, Tommi Ilmonen and the participants at ECS.

REFERENCES 1. Csikszentmihalyi, M. Flow: The psychology of optimal

experience. Harper & Row, New York, 1990. 2. Benyon, D., Smyth, M., O’Neill, S., McCall, R. and

Caroll, F. The Place Probe: Exploring a Sense of Place in Real and Virtual Environments. Presence, 15, 6 (2006), 668–687.

3. Brignull, H. and Rogers, Y. Enticing people to interact with large public displays in public spaces. In Proc. INTERACT 2003, IOS Press (2003), 17–24.

4. Brignull, H., Izadi, S., Fitzpatrick, G., Rogers, Y. and Rodden, T. The introduction of a shared interactive


2275

surface into a communal space. In Proc. CSCW 2004, ACM (2004), 49–58.

5. Bryan-Kinns, N., Healey, P. and Leach, J. Exploring Mutual Engagement in Creative Collaborations. In Proc. C&C’07, ACM (2007), 223–232.

6. Deci, E. L., Ryan, R. M. The "what" and "why" of goal pursuits: Human needs and the self-determination of behavior. Psychological Inquiry, 11 (2000), 227–268.

7. Elrod, S. Bruce, R., Gold, R., Goldberg, D., Halasz, F., Janssen, W., Lee, D., McCall, K., Pedersen, E., Pier, K., Tang, J. and Welch, B. Liveboard: a large interactive display supporting group meetings, presentations, and remote collaboration. In Proc. CHI 1992, ACM Press (1992), 599–607.

8. Erickson, F. The interface between ethnography and microanalysis. In LeCompte, M.D., Millroy, W.L. and Preissle, J. (Eds.) The handbook of qualitative research in education. Academic Press, San Diego, CA, 1992, 201–225.

9. Everitt, K., Shen, C., Ryall, K. and Forlines, C. Modal spaces: spatial multiplexing to mediate direct-touch input on large displays. Ext. Abstracts CHI 2005, ACM Press (2005), 1359–1362.

10. Forlines, C. and Shen, C. DTLens: multi-user tabletop spatial data exploration. In Proc. UIST 2005, ACM Press (2008), 119–122.

11. Guthrie, J.T. Contexts for engagement and motivation in reading. Reading Online, 4, 8 (2001).

12. Gutwin, C. and Greenberg, S. A Descriptive Framework of Workspace Awareness for Real-Time Groupware. Computer Supported Cooperative Work, 11, 3 (2002), 411–446.

13. Hinrichs, U., Carpendale, S., Scott, S.D. and Pattison, E. Interface Currents: Supporting fluent collaboration on tabletop displays. In Proc. SG 2005, Springer (2005), 185–197.

14. Hornecker, E., Marshall, P. and Rogers, Y. From entry to access: how shareability comes about. In Proc. DPPI 2007, ACM Press (2007), 328–342.

15. Isenberg, P., Tang, A. and Carpendale, S. An Exploratory Study of Visual Information Analysis . In Proc. CHI 2008, ACM Press (2008), 1217–1226.

16. Jackson, S.A., and Marsh, H.W. Development and validation of a scale to measure optimal experience: The Flow State Scale. Journal of Sport and Exercise Psychology, 18 (1996), 17–35.

17. Jacucci, G. Interaction as Performance. Cases of Con-figuring Physical Interfaces in Mixed Media. Oulu University Press (2004), Oulu, Finland.

18. Jacucci, G., Spagnolli, A., Chalambalakis, A., Morrison, A., Liikkanen, L., Roveda S., Bertoncini, M. Bodily Explorations in Space: Social Experience of a Multimodal Art Installation. In: Interact 2009, Springer, (2009), 62–75.

19. Jacucci, G., Wagner, I. Performative Uses of Space in Mixed Media Environments. In Davenport, E., Turner P., Spaces, Spatiality and Technologies, Springer (2005), 191-216.

20. Peltonen, P., Kurvinen, E., Salovaara, A., Jacucci, G., Ilmonen, T., Evans, J., Oulasvirta, A. and Saarikko, P. "It's Mine, Don't Touch!": Interactions at a Large Multi-Touch Display in a City Centre. In Proc. CHI 2008, ACM Press (2008), 1285–1294.

21. Rick, J., Harris, A., Marshall, P., Fleck, R., Yuill, N., and Rogers, Y. Children designing together on a multi-touch tabletop: an analysis of spatial orientation and user interactions. In Proc. IDC 2009, ACM Press (2009), New York, 106–114

22. Robertson, T. Cooperative Work and Lived Cognition: A Taxonomy of Embodied Actions. In Proc. ECSCW 1997, Kluwer Academic Publishers (1997), 205–220.

23. Russell, M., Drews, D. Sue, A. Social Aspects of Using Large Public Interactive Displays for Collaboration. In Proc. UbiComp 2002, Springer (2002), 229–236.

24. Sawyer, K. Group creativity: Music, theater, collaboration. Lawrence Erlbaum Associates (LEA), NJ, USA, 2003.

25. Scott, S. D., Grant, K. D. and Mandryk, R. L. System guidelines for co-located, collaborative work on a tabletop display. In Proc ESCW 2003, Kluwer Academic Publishers (2003), 159–178.

26. Scott, S.D., Sheelagh, M., Carpendale, T. and Inkpen, K.M. Territoriality in collaborative tabletop workspaces. In Proc. CSCW 2004, ACM Press (2006), 294–303.

27. Sweetser, P. and Wyeth, P. Gameflow: a model for evaluating player enjoyment in games. In Proc. CIE 2005, ACM Press (2005), 1–24.

28. Tang, A., Tory, M., Po, B., Neumann, P. and Carpendale, S. Collaborative coupling over tabletop displays. In Proc. CHI 2006, ACM Press (2006), New York, 1181–1190.

29. Tse, E., Greenberg, S., Shen, C., Forlines, C. and Kodama, R. Exploring true multi-user multimodal interaction over a digital table. In Proc. DIS 2008, ACM Press (2008), New York, 109–118

30. Vorderer, P., Wirth, W., Gouveia, F. R., Biocca, F. Saari, T., Jancke, F. MEC-SPQ: Report to the European Community, Project Presence: MEC (IST-2001-37661), 2004.


2276

Documents

Worlds of information: designing for engagement at a public multi … › u › jacucci › p2267-jacucci.pdf · Worlds of Information: Designing for Engagement at a Public Multi-touch