Xambo

Issues and techniques for collaborative music making on multi-touch surfaces

Anna Xambó[email protected]

Supervisors Robin LaneyDepartment/Institute Department of ComputingStatus Visiting research student (4 months)Probation viva -Starting date -

A range of applications exist for collaborative music making on multi-touch surfaces. Some of them have been highly successful, but currently there is no systematic way of designing them, to maximize collaboration for a particular user group. We are specially interested in applications that will engage novices and experts. Traditionally the challenge in collaborative music instruments is to satisfy the needs of both [1]. For that purpose, we developed a collaborative music making prototype for multi-touch surfaces and evaluated its creative engagement.

Applications for musical multi-touch surfaces are not new. A pioneering work is the ReacTable [2, 3], which allows a group of people to share control of a modular synthesizer by manipulating physical objects on a round table. Iwai’s Composition on the Table [4] allows users to create music and visuals by interacting with four tables which display switches, dials, turntables and sliders. Stereotronic Multi-Synth Orchestra [5] uses a multi-touch interface based on a concentric sequencer where notes can be placed. What is less addressed is the evaluation of creative engagement in these applications. There are numerous theoretical accounts of the nature of emotional engagements with art and artefacts. Current models are based on a pragmatist view, which conceptualises the aesthetic and affective value of an object as lying not in the object itself, but in an individual’s or a group’s rich set of interactions with it [6, 7]. In the context of pleasurable creative engagement and the collective composition of music, Bryan-Kinns et al. [8] see attunement to others’ contributions as the central principle of creative engagement. The phenomena of personal full immersion in an activity, also known as ’fow’ [7], has been extended to groups as means of heightening group productivity [9].

Our approach is, frst, to study the issues and techniques of multi-user instruments and multi-touch applications in general, second, to design a simple application in an initial attempt to clearly analyse some of these issues, and third, to evaluate its creative engagement. For that purpose, a prototype was built which allowed groups of up to four users to express themselves in collaborative music making using pre-composed materials. Bykeeping the prototype minimal, we were able to investigate the essential aspects of engaging interaction.

Case studies were video recorded and analysed using two techniques derived from Grounded Theory (GT) and Content Analysis (CA). For the GT, which is a qualitative research method employed in the social sciences that derives theoretical explanations from the data without having hypotheses in mind [10], we adopted an open coding strategy of identifying key moments of the video

2010 CRC PhD Student Conference

Page 118 of 125

interactions; grouping the codes by concepts and generating general explanations from the categorization of the concepts. Given that this approach is based on creative interpretation, we added more evidence by complementing GT with CA. Content Analysis (CA) is defned by Holsti (1969) as ”any technique for making inferences by objectively and systematically identifying specifed characteristics of messages” [10]. This defnition includes content analysis of text, videos, music or drawings. There are varied approaches to CA using quantitative, qualitative or both techniques. Our approach is derived from ethnographic content analysis or qualitative content analysis [11], an approach to documents that emphasises the role of the investigator in the construction of the meaning of texts. We took same steps as in the open coding, but in the frst step we used instead structured codes to help us identify key points of the video-recorded interactions.

The case study protocol was the following: The users were expected to perform three musical tasks of different character as well as an informal discussion in order to generate suffcient data to analyse several aspects of behaviours using the prototype. A questionnaire was also conducted and evaluated. The main focus of the analysis was on the evaluation of the collaborative interactions enabled by the prototype. The questions we wanted to address were:

1. What were the modes participants found to collaborate with oneanother;

2. What were the diffculties that participants encountered and the extentto which they found the exercise engaging;

3. What was the degree of satisfaction at the end result.

From transcription of the video speech and behaviours, and then the process of open coding, we identifed the following concepts: collaboration, musical aesthetics, learning process and system design. After that, we analysed the same data using the nomenclature chosen from two existing theoretical frameworks. The frst one is a general framework of tangible social interaction which includes the concepts of tangible manipulation, spatial interaction, embodied facilitation or expressive representation [12]. The second one is focused on the engagement between participants in music collaboration, which considers the following features: mutual awareness, shared and consistent representations, mutual modifability and annotation [8]. We found that some of the content analysed was already discussed in the open coding process, which provides consistency. Data was also collected using a questionnaire, which was designed to probe such issues as how aware each participant had been of other instruments; the diffculty of the tasks, and how much they felt they had enjoyed and concentrated on them; and the extent to which they considered they had operated as a team and felt part of a collaborative process. Responses were recorded using numerical scores, but the questionnaire also asked for qualitative feedback on how participants organised themselves as a group and the nature of any rules they created. We also recorded anonymously the participants age, gender, previous experience, love of music, and the instrument they had been allocated on the table.

Within a user-centered design approach of active participation of users in the process of designing the prototype, the most prominent aspects that have emerged as enhancements of multi-touch applications in music collaboration are:

• Responsiveness. The responsiveness determines the perceived emotiveness. This parameter should be adequately related to the


Page 119 of 125

application performance in terms of time and computer resources used. A consistent audiovisual feedback will enhance the perceived response of the application.

• Shared vs. individual controls. Both shared and individual spaces are needed. Shared features would strength mutual awareness and mutual modifability. Individual spaces would strength personal opinion, musical identity and musical expression.

The fndings of this study help us understand engagement in music collaboration. Qualitative video analysis and the questionnaires provide indication of participants having mutual engaging interaction in terms of being engaged with the music collaboratively produced and also being engaged with others in the activity. High degree of satisfaction at the end result is evidenced mostly by the gestural mode. The evidence found of participants exchanging ideas constantly indicates that the prototype strongly facilitates conversation, which, as noted earlier, is important in terms of group productivity.

In the future, we are interested in how many, and what type of, affordances such applications should offer in order to maximise engagement. We are also interested in validating our evaluation method. To that end, there is scope to improve the responsiveness of the prototype and to redesign the distribution of shared and individual controls. Furthermore, there is a plan to add individual continuous controls for sound parameter modifcations in order to encourage a process-oriented composition. The mutual experience might be enhanced and collaboration deepened, by adding common controls, as well. A balance between adding more features and keeping simplicity must be kept in order to attract both novices and experts alike.

[1] T. Blaine and S. Fels, “Collaborative musical experiences for novices,” Journal of New Music Research, vol. 32, no. 4, pp. 411–428, 2003.

[2] S. Jordà, M. Kaltenbrunner, G. Geiger, and R. Bencina, “The reacTable*,” in Proceedings of the International Computer Music Conference (ICMC 2005), (Barcelona, Spain), 2005.

[3] S. Jordà, G. Geiger, M. Alonso, and M. Kaltenbrunner, “The reacTable: Exploring the synergy between live music performance and tabletop tangible interfaces,” in TEI ’07: Proceedings of the 1st international conference on Tangible and embedded interaction, (New York, NY, USA), pp. 139–146, ACM, 2007.

[4] T. Iwai, “Composition on the table,” in International Conference on Computer Graphics and Interactive Techniques, SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques, ACM, 1999.

[5] http://www.fashionbuddha.com/, 15/3/2010.

[6] M. Blythe and M. Hassenzahl, The semantics of fun: differentiating enjoyable experiences, pp. 91–100. Norwell, MA, USA: Kluwer Academic Publishers, 2004.

[7] M. Csikszentmihalyi, Beyond Boredom and Anxiety: Experiencing Flow in Work and Play. Jossey-Bass, 1975.


Page 120 of 125

[8] N. Bryan-Kinns and F. Hamilton, “Identifying mutual engagement,” Behaviour and Information Technology, 2009.

[9] K. Sawyer, Group Genius: The Creative Power of Collaboration. Basic Books, 2007.

[10] J. Lazar, J. Feng, and H. Hochheiser, Research Methods in Human-Computer Interaction. Wiley, 2010.

[11] D. L. Altheide, “Ethnographic content analysis,” Qualitative Sociology, vol. 10, pp. 65–77, 1987.

[12] E. Hornecker and J. Buur, “Getting a grip on tangible interaction: A framework on physical space and social interaction,” in CHI ’06: Proceedings of the SIGCHI conference on Human Factors in computing systems, (New York, NY, USA), pp. 437–446, ACM Press, 2006.


Page 121 of 125

Education

Xambo