Embed Size (px)
DESCRIPTION
Final PhD Thesis for Camille Baker December 2012
Citation preview
Title page
MINDtouch - Ephemeral Transference:
‘Liveness’ in Networked Performance with Mobile Devices
Camille BAKER
A thesis submitted in partial fulfilment of the requirements of the University of East London
for the degree of Doctor of Philosophy
October 2010
2
i / thesis abstract
This practice-based thesis investigates the four key qualities of 'liveness’, ‘feltness’, ‘embodiment’ and 'presence' in mobile media performance, in order to shed light on the use qualities and sensations that emerge when mobile technologies are used in tandem with wearable devices in performance contexts. The research explores mobile media as a non-verbal and visual communication tool that functions by repurposing the mobile phone device and its connection to a wireless network, not only for communication but explicitly for the expression of ‘emotion’ in the form of a video file representing an interpersonal connection shared over distance. The research aims to identify and supplement existing scholarly discourse on the nature of these four key strands of kinaesthetic philosophy made ‘live’ in the online network, applying knowledge gained through the practice of enhancing participant experience of the use of simple ubiquitous mobile tools with bespoke biofeedback sensors and an online repository for the playback of users’ visual expressions. This enhanced toolkit enables participants to share personal relationships and social interactions in an immediate way, with collaborators at a distance. The selected methodology of active research using kinaesthetic tools in live performance seeks to identify and clarify new ways of simulating or emulating a non-verbal, visual exchange within a social participatory context, with particular attention paid to a sense of ‘feltness’ as an element of ‘presence’ or ‘liveness’, and with attention to the experience of a sense of ‘co-presence’ arising in real-time collaborative mobile performances at a distance. To best explore these concepts as well as the bodily sensations involved for participants, the thesis analyses original data gleaned from a larger R&D project (conducted in tandem with this thesis project, sponsored by the BBC) as its major case study. The project, called MINDtouch , created a series of unique practice-based new media performance events played out in real-time networked contexts. The MINDtouch events were framed as a means for participants to simulate dream exchange or telepathic thought transfer using mobile phones and biofeedback devices, linked to a bespoke video file protocol for archiving and sharing visual results. The corporeal, non-verbal forms of communication and visual interaction observed when participants use such devices within participatory performance events is examined by way of demonstrating the impact of specific live encounters and experiences of users in this emerging playing field between real-time and asynchronous, live and technologised forms expressing liveness/presence/distance. The thesis benefits from access to the larger MINDtouch project and its original data, providing this research with a set of process-based evidence files both in video and transcript form (contained in the thesis appendices). By analysing this unique data set and applying the theoretical contexts of kinaesthetic philosophies where appropriate, the thesis demonstrates both the practical and the critical/contextual effectiveness of the media facilitation process for the participants, and shares their senses of ‘liveness’ and ‘presence’ (of themselves and of others) when using technology to externalise visual expressions of internalised experiences. This thesis makes an original contribution to scholarship in the fields of Performance and New Media, with additional contributions to the cognate fields of Philosophy and Technology, and locates its arguments at the locus of the fields of Performance Art, Mobile Performance/Locative Media, Philosophies of the Body and Communications. The thesis uses methods, practices and tools from Phenomenology, Ethnography, Practice-As-Research, and Experience Design, bringing together the relevant aspects of these diverging areas of new media research and media art/performance practices. The research demonstrates that there is a need for new technological tools to express viscerally felt emotion and to communicate more directly. It is hoped that this study will be of use to future scholars in the arts and technology, and also that it may help to demonstrate a way of communicating rich emotion through felt and embodied interactions shared with others across vast distances (thus supporting political movements aimed at reducing global travel in the age of global warming).
3
i i / acknowledgements
I wish to acknowledge all the people who have made both this academic thesis and the larger research
project possible. In such a large project, there are many to thank.
Primarily, I would like to thank Professor Lizbeth Goodman for all her hard work both professionally and
personally in terms of coaching, encouragement and provision of opportunities, for being an amazing
friend and confidante, for hunting down and finding funding and resources for me and all the
SMARTlab PhD cohort; for all her love and support, and all the tireless work she does for all of us and
for the community and the domain of education.
I must of course thank the invaluable team of SMARTlab supervisors who joined me on this doctoral
project in the last year of the process: an A team recruited specially to assist me in my last dash to the
finish line: Dr Ryya Bread, Dr Sher Doruff and Dr Allan Parsons.
I would also like to thank my friends from the adhoc online PhD support group provided both within
and beyond the SMARTlab cohort, which began before I left Canada in the spring of 2007: Dr Lone
Koefoed Hansen, Cynthia Poremba, and Valerie LaMontagne – friends and valuable peers, reviewers
and supporters. Although we had different topics, our work was related enough to allow us to share
reading, theory and links to ideas and networks. The group provided a vital source of encouragement
and peer support, not only on the research but also on the personal issues that often overlapped with
our work. This group also provided consistent and intriguing intellectual engagement and
encouragement, as did the cohort of PhD students and friends from SMARTlab, including the high level
peer support developed in the later stages of thesis writing from the SMARTlab “senior citizens” (e.g.
those of us who had spent longest in the programme by year 4 and were thus nearest to completion),
for their amazing support and insightful inputs. Their positive influence on both my life and my studies
has been considerable. Thanks belong with all the ‘SMARTgang’ and particular thanks go to: Kasia
Molga, Anita Mckeown, Denise Doyle, Taey Kim, Kate Sicchio and Turlif Vilbrant.
A project of this technological complexity could not have been achieved without the help of numerous
engineers and experts in online systems and tools. In this regard, I want to thank Michael Markert for
his diligent work in technical development and innovative contributions to the larger BBC project that
informs this thesis: particularly to the invention of unique ways to link the electronic and software tools
required for visualising body data and video on mobile phones. Michael contributed considerably to the
technical production of the BBC MINDtouch Project, despite his physical distance from the team.
Thanks go to Michael for his patience and diligence as he worked through the project’s challenges with
the state of play in current technology. While he was not able to provide exactly what I had envisioned,
his efforts paid off greatly in helping to make as much of the vision a reality as was technically possible
in the timeframe of the project. Similar thanks are due to Manjit Bedi for his significant technical
contributions in the late project stage to mixing visuals using software, with additional work from Evan
4
Raskob. I would like additionally to thank Dr Marc Price, Huw Williams and the BBC for their financial
support, and for providing a framework within which an art project and a scholarly dissertation could
both take shape, within the larger remit of an R&D project for the BBC. Marc Price’s intellectual
support, insight and advice as co-supervisor were also appreciated throughout the process of
envisioning and then writing a thesis linked to, but separate from, the BBC project.
In addition to the direction provided by SMARTlab itself, I would like to thank Dr Susan Kozel for her
continual involvement in my graduate studies and conceptual development, since my Master’s studies
in Interactive Arts at Simon Fraser University (Vancouver, Canada: 2002–2004) through to the rooting of
this new work in the UK at SMARTlab. Her rigour of thought and consistent thoroughness in feedback
has helped and taught me immensely as a thinker, writer and artist/curator. Her attention to my work
was in particular a support and source of comfort to me in the early stages of the research, as I not
only went from one level of study to the next, but also from my home country to another
country/home/life and career. Susan’s ongoing friendship and personal support, and her role as an
effective professional artistic/curatorial collaborator between my graduate degrees, has been greatly
appreciated and acknowledged throughout the thesis. I would also like to thank Dr Thecla Schiphorst
for her positive input in my early work; collaborating with her and Susan Kozel on the whisper[s]
project (discussed herein) had a significant influence on my current direction and career.
ii.1/ collaborative research notes and professional credits While the thesis submitted for examination is a solo work that puts forth my own arguments, this work
was achieved in the larger context of a practice-based R&D project and performance experiment using
mobile technologies, sponsored by the BBC. While the ideas and arguments of the thesis are entirely
mine, and while the conceptualisation, direction and performance of the practical components of the
major case study were also mine, still a group of collaborators contributed to the larger project in
specific ways that bear a mention here.
I want to acknowledge and thank profusely all the people who worked on MINDtouch , and assisted
me – usually gratis – with the technology, workshops and performances, especially: Perparim Rama
and the 4MGroup, Tara Mooney, Rachel Lashebikan, Pawel Borkowski, Evan Raskob, Fred Brown and
Steve Lauder of the Matrix East Lab at UEL (under the direction of Professor Haim Bresheeth), Toby
Borland, Clilly Castiglia, Stanislava Mislanova, Vanessa Wiegand and Mike Murphy, along with
engineers/programmers: Dr Li Zhang and Jeremi Sudol. The first performance experiment in July 2009
was guided by Cathy O’Kennedy, Kate Sicchio and Anita Mckeown, to whom thanks are also due.
Other performance experiments involved too many people to mention, as did the archiving and analysis
of the many hours of footage provided by the group’s generous participants, who gave me permission
to use their video expressions.
There is not space here to extend my thanks further in print. There are many more to be made, but
they must go without saying here.
5
i i i / statement of originality I Camille Baker hereby declare that all the intellectual ideas and research materials included for examination in this thesis are entirely mine, and that the collaborations on the larger BBC research project (not included for examination) have been duly credited. Signed: research candidate, Camille Baker _____________________________ Date: ________________________ Countersigned: Director of Studies and PI of the BBC Project for UEL: Professor Lizbeth Goodman __________________________________ BBC R&D Principal Investigator/sponsor: Dr Marc Price __________________________________________________
6
table of contents
t i t le page ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
i/ thesis abstract .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
i i / acknowledgements ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 ii.1/ collaborative research notes and professional credits .................................................................4
i i i / statement of or ig inal i ty .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
table of i l lustrat ions ( images and f igures) in text .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
iv/ statement of the art ist as researcher .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
v/ sty le notes ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 v.i/ fonts.........................................................................................................................................12 v.ii/ tense/first and third person voices ............................................................................................12 v.iii/ italics .......................................................................................................................................12
1/ introduct ion: ephemeral transference – backgrounds and contexts ... . . . . . . . . . . . . . . . . . . . 13 1.1/ ‘setting the stage’ (and a note on placement of the distributed literature survey) ......................13 1.2/ MINDtouch – summary of the major case study and its aims/outcomes ..................................17 1.3/ motivations, roles and voice.....................................................................................................20
1.3.1/ motivations.......................................................................................................................20 1.3.2/ key terms defined: 'liveness', 'feltness', 'embodiment', 'presence', 'co-presence', 'ephemerality', 'network', 'transference' .....................................................................................22 1.3.3/ voice ................................................................................................................................25 1.3.4/ roles .................................................................................................................................26
1.4/ methodologies.........................................................................................................................27 1.5/ performance context (Literature Review Part 1)........................................................................29 1.6/ mobiles, databases and network freedom ...............................................................................31
1.6.1/ mobile and database media..............................................................................................31 1.6.2/ dreams of networked freedom..........................................................................................33
1.7/ philosophical landscape (Literature Review Part 2) ...................................................................36 1.8/ ‘connective tissue’ ...................................................................................................................38
1.8.1/ exegesis/thesis structure ..................................................................................................40
Part I : mobi le media pract ice in cr i t ical context .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2/ hybr id methodologies of pract ice ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 2.1/ art-as-research ........................................................................................................................43 2.2/ phenomenology.......................................................................................................................51
2.2.1/ journaling phenomenologically ..........................................................................................58 2.3/ ethnography ............................................................................................................................59
2.3.1/ interview process..............................................................................................................60 2.3.2/ performance ethnography ................................................................................................61
2.4/ experience & sensorial design..................................................................................................62 2.5/ embodied design / ‘workshopping’..........................................................................................64 2.6/ structured improvisation ..........................................................................................................67 2.7/ visual methodologies ...............................................................................................................68 2.8/ technology and software design methods................................................................................70
3/ part ic ipatory performance ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 3.1/ participation as performance ...................................................................................................72 3.2/ politics of participation .............................................................................................................75
7
3.3/ presence in performance ........................................................................................................ 81
4/ v ideo art + l ive cinema performance ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 4.1/ a video art journey................................................................................................................... 87 4.2/ live cinema.............................................................................................................................. 88
Part I I : theory into praxis .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
5/ mobi le v ideo aesthet ics, non-verbal / non-l inguist ic .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 5.1/ the emergent aesthetics of mobile video ................................................................................. 96 5.2/ non-linguistic mobile video communication ........................................................................... 110 5.3/ generative and database video ............................................................................................. 117
6/ networks, te lepresence + connect iv i ty .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 6.1/ telepresence and telematic performance............................................................................... 122 6.2/ electronic threads between us .............................................................................................. 124 6.3/ access / protocols / privileges............................................................................................... 128
7/ presence: permutat ions... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 7.1/ presence: feltness sensing .................................................................................................... 131 7.2/ emotional presence............................................................................................................... 133 7.3/ distance + intimacy: mobile digital presence + connection ................................................... 136 7.4/ presence and telepathy......................................................................................................... 139 7.5/ collective consciousness, spiritual presence and out-of-body-experiences ........................... 142
8/ embodiment and affect .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 8.1/ embodiment – towards a working definition .......................................................................... 149 8.2/ embodiment of space / absence and the space between ..................................................... 153 8.3/ experience senses, perception and affect ............................................................................. 161 8.4/ liveness – when is now?........................................................................................................ 170
Part I I I : conclusions & references for further study ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
9/ thesis conclusion: .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178 9.1/ implications and new directions ............................................................................................ 178 9.2/ in conclusion......................................................................................................................... 182
bibl iography ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
Appendix I/ the MINDtouch project (BBC R&D sponsored) .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
A.1/ - technical appendix and background source mater ia l on mobi le performance, pract ice and process ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
A.1.1/ mobile video collection workshops .................................................................................... 194 A.1.1a/ biosensors and electronics .......................................................................................... 196 A.1.1b/ garment design ........................................................................................................... 200 A.1.1c/ performance events ..................................................................................................... 202 A.1.1d/ participant expectations .............................................................................................. 205
A.1.2/ outcomes, analysis + interpretation: performances events ................................................. 206 A.1.2a/ workshop outcomes and observations ........................................................................ 207 A.1.2b/ performance discoveries, results + analysis ................................................................. 211 A.1.2c/ demographics ............................................................................................................. 212 A.1.2d / event survey feedback................................................................................................ 213 A.1.2e/ event analysis: production report ................................................................................. 215 A.1.2f/ participant response analysis ........................................................................................ 223 A.1.2g/ overall performance assessment ................................................................................. 227
8
A.1.3/ visual mixing software + streaming development process...................................................230 A.1.4/ video content categories + analysis ....................................................................................236 A.1.5/ mobile/ networks and connectivity......................................................................................238
Appendix I I/ workshop mater ia ls + technical e lements ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241 A.2.1/ technical timeline for technical development .......................................................................242 A.2.2 / workshop participant consent form ..................................................................................243 A.2.3 / workshop participation and non-exclusive licensing agreement .........................................246 A.2.4/ Participant Profile Information .............................................................................................248 A.2.5/ step-by-step videophone expressions workshop procedure...............................................249 A.2.6/ workshop mind quieting script............................................................................................250 A.2.7/ post-workshop questionnaire .............................................................................................252 A.2.8/ post-workshop interview questions ....................................................................................255 A.2.9/ workshop video file content database.................................................................................256 A.2.10/ technical updates and software guides.............................................................................266
Appendix I I I / performance event mater ia ls .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293 A.3.1/ streaming server set-up and operational notes ..................................................................294 A.3.2/ technical workplan..............................................................................................................296 A.3.3/ performance preparation checklist......................................................................................297 A.3.4/ performance preparation outline .........................................................................................298 A.3.5/ event technical preparation.................................................................................................299 A.3.6/ additional logs of facebook, website and online interaction .................................................300 A.3.7/ first performance: introductions script.................................................................................311 A.3.8/ performance mind-quieting script .......................................................................................312 A.3.9/ group guiding information...................................................................................................314 A.3.10/ performance event participant consent form.....................................................................320 A.3.11/ post performance audience survey...................................................................................321 A.3.12/ event participant experience feedback themes .................................................................323 A.3.13/ performance survey feedback ..........................................................................................326
.................................................................................................................................................326
Appendix IV/ col laborator and technical feedback ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334
A.4.1/ testimonials on network issues ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335 A.4.2/ screenshots of forum discussions on Nokia Realplayer issues ............................................336
Appendix V/ DVD Support Mater ia ls: Table of Contents ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338
A.5/ Notes ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339 A.5.1/ Disc 1 - participant workshop video clips ...........................................................................339 A.5.2/ Disc 2 - Workshop documentation videos and stills............................................................340 A.5.3/ Disc 3 - Performance videos and final ‘VJ’ mixes................................................................341 A.5.4/ Disc 4 - Promotional materials ............................................................................................342 A.5.5/ Disc 5 – Technical materials ...............................................................................................343
Appendix VI/ publ ished papers re levant to the thesis .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344 A.6.1/ ‘Liveness’ and ‘presence’ in bio-networked mobile media performance practices: emerging perspectives (Dec 2008) ...............................................................................................................345 A.6.2/ aesthetics of mobile media art (July 2009) ..........................................................................365
9
table of i l lustrations (images and figures) in text Figure 1 © 2007 C. Baker - early technical diagram of technical development phases....................... 19 Figure 2 © 2007 C. Baker – chart of project roles............................................................................... 26 Figure 3 © 2010 C. Baker – Diagram of thesis knowledge domains.................................................... 39 Image 1 © 2007 C. Baker – still from Dublin workshop, October 2007 ............................................... 66 Image 2 © 2007 C. Baker – participants in video collection workshops .............................................. 73 Images 3 and 4 © 2003 Blast Theory – Uncle Roy All Around You ..................................................... 76 Images 5 and 6 © 2007 Blast Theory, Riders’ Spoke ......................................................................... 77 Image 7 © 2004 - still from PDX_01 by NomIg in live performance.................................................... 90 Image 8 © 2004–2008, still from A/V performance [SILENT ROOM] film-poème, skoltz_kolgen.......... 91 Images 9 and 10 © 2007 Sue Costabile still from audiovisual performance ........................................ 92 Images 11 and 12 © 2005-2006 stills from A/V performance - Sara Kolster and Derek Holzer .......... 93 Image 13 © 2004 Steve Hawley – still from Speech Marks............................................................... 103 Image 14 © 2008 Mark Amerika – still from Mobile Phone Video Art Classics................................... 103 Image 15 © 2009 Mark Amerika Immobilité ...................................................................................... 104 Image 16 © 2005 Dean Terry ........................................................................................................... 105 Images 17 and 18 © 2006 Max Schleser – Max With a Kaitei........................................................... 106 Image 19 © 2005 Giselle Beiguelman, screen shot from sometimes never ....................................... 107 Images 20 and 21 © 2005 Giselle Beiguelman, screen shot from sometimes never ......................... 107 Image 22 © 2006 Tina Gonsalves – FEEL.TRACE ............................................................................ 108 Image 23 © 2004 Tina Gonsalves with Tom Donaldson – MEDULLA INTIMATA .............................. 109 Image 24 © 2007 C. Baker – still from participant video during Dublin workshop, October 2007...... 112 Image 25 © 2005 – whisper[s] project screenshot from a video I shot at SIGGRAPH conference ..... 113 Image 26 © 2005 – whisper[s] project, showing the electronics, conductive fabric and designs ....... 113 Image 27 © Phillips Design, Skin Probes .......................................................................................... 114 Image 28 © Philips Design – Skintile................................................................................................. 115 Image 29 © 1995 Char Davies, Ephemere ....................................................................................... 116 Image 30 © 1995 Kathleen Rogers, Psi Net ..................................................................................... 117 Image 31 © 2002 Lev Manovich, Soft Cinema ................................................................................. 119 Figure 4 © 2009 Camille Baker......................................................................................................... 130 Figure 5 © 2010 Camille Baker – key ideas and theorists on embodiment ........................................ 169 Figure 6 © 2009 Camille Baker......................................................................................................... 170 Images 33, 34, and 35 © 2007 C. Baker –participants video stills from workshops .......................... 195 Image 36 © 2007 Mind Media – Nexus 4 Bluetooth Sensor transmitter........................................... 197 Images 37, 38, 39, 40 and 41 2007 Mind Media: electrode cable and sensors ............................... 197 Images 42, 43, and 44 © 2008 C. Baker: Michael Markert developing the custom electronics. ........ 197 Image 45 © 2008 C. Baker: custom biofeedback visualisation software. .......................................... 198 Image 46 © 2007 collaborator Brady Marks: vision of the mobile interface for social VJing............... 199 Image 47 © 2009 C. Baker: Quartz Composer early software patch view......................................... 199 Image 48 © 2009 C. Baker: Alpha / Garment 1 with embedded sensors.......................................... 201 Images 49, 50, and 51 © 2009 C. Baker: Beta / Garment 2 with embedded sensors....................... 201 Figure 5 © 2009 C. Baker - early technical diagram of the networked performance project .............. 202 Figure 6 © 2009 C. Baker –staging and organisation performing groups live context........................ 203 Images 52 and 53 © 2009 C. Baker – stills from July 2009 first performance ................................... 204 Images 54 and 55 © 2009 C. Baker................................................................................................. 204 Images 56, 57 and 58 © 2009 C. Baker........................................................................................... 205 Image 59 and 60 © 2007 C. Baker – still images of participants in video collection workshops ........ 207 Images 61 and 62 © 2007 C. Baker - from Vancouver workshop June 2007 ................................... 208 Image 63 and 64 © 2007 C. Baker - stills from Vancouver workshop June 2007 ............................. 208 Images 65 and 66 © 2007 C. Baker workshop still Dublin July 2007 ................................................ 209
10
Images 68 and 69 © 2007 C. Baker - stills from participants’ videos Vancouver, August 2007 .........210 Images 70, 71 and 72 © 2007 C. Baker - stills from participants’ videos Dublin, October 2007 ........210 Image 73 © 2010 C. Baker – screen shot of the Quartz Composer patch for video mixing................230 Figure 7 © 2010 C. Baker – physiological sensors mapped to life-force qualities & visual effects.......231 Image 73 © 2010 C. Baker – zoomed in screen shot of patch for video mixing from sensor data......232 Figure 8 © 2009 C. Baker – demonstrating the technology connections ...........................................233 Figure 9 © 2010 C. Baker – demonstrating the server-side processing .............................................234 Figure 10 © 2009 Evan Raskob (collaborator) – sketch of networked interactions .............................235 Figure 11 © C. Baker – chart of the activities, content and connection, database and sensors .........237 Image 74 © 2010 C. Baker – spreadsheet of database of participant videoclips ...............................238
11
iv/ statement of the artist as researcher I am an artist-performer/researcher/curator within various art forms: interactive and performance
installation, music composition and performance, video art, web animation, and experience design. I
have been immersed in many forms of media creation and performance since 1994, including roles as:
new media installation and performance curator; videographer, animator and documentary producer;
video artist; photographer; media art instructor; web designer/developer, CD ROM and multimedia
designer; visual arts curator; sculptor and modern dancer/performer. I was in Masters in Appl ied
Science in Interact ive Arts completed in April 2004, within the School of Interactive Arts and
Technology at Simon Fraser University, Vancouver, Canada. This research and thesis project was an
immersive media installation that explored the concept of initiating a mind/body to computer
communication aided by biosensors to trigger media to calm the mind to enable telepathy in
participants.
Prior to moving to the UK between 2004 and 2007, I was involved in: starting and developing a
performance media society, the Escape Artists Society (www.escapeartists.ca), which put on several
performance events in 2005/2006. From 2002–2005, I was researcher for the wearable art installation
and performance project whispers/between bodies project (http://whisper.iat.sfu.ca) with Dr Thecla
Schiphorst and Dr Susan Kozel. This involved research in performance and wearable design, biosensor
testing and development, prototype garment construction, movement development with wearable
computing, as well as website development and video editing. I was also the primary Event Producer/
Interactive and Performance Curator (2004), the Lead Curator/Conference Director (2003), and the
Visuals Coordinator (2002) for the New Forms Festival (www.newformsfestival.com), which involved
extensive research in media art and performance, practices and history; exhibition, event production
and staging coordination; arts funding administration; and curation. I was editor-in-chief of an online
pop-culture/relationship magazine (Tales of Slacker Bonding) in 2000–2003;
I was a singer/songwriter, performer, musician and producer with three primary musical projects from
1992 to 2007: Spiritual Heroine, EPX and ultrapuss, as well as singing for many side projects, and
student dance and film projects. After my undergraduate degree in Sociology/Anthropology I began as
an Employment and Life Skills instructor for at-risk youths and adults, from 1992–2000. In high school
and into university I studied contemporary dance, and from 2001 to the present I have been an active
yoga practitioner. Both these and other body practices have factored heavily in and influenced my art
practice.
I decided to undertake both the PhD research and MINDtouch practice as a set of related but
separate work projects, in order to combine my particular background, artistic history and interests into
a unique practice. I was led to this work, and this combined focus, by years of work experience as a
performer, musician, media artist/designer, curator and employment facilitator. I have been enthused
and motivated to work with mobile phones and video in a networked performance context since the
idea occurred and began to develop in 2000. This PhD research and the related BBC project thus
12
provided a valuable opportunity to further explore and develop my own voice within the context of
performance, entwining video art and facilitation skills, and in the process creating a new form for
others to share and experience.
v/ style notes
The style of the thesis complies with the university’s regulations regarding house style in all regards. A
number of specific stylistic devices have been used to increase the level of aesthetic expression in line
with the content and domain media of the thesis.
v.i/ fonts
The thesis uses new iterations of both the Arial and Helvetica fonts. The fonts employed are,
specifically, Arial Rounded MT Bold for headings – in various point sizes to reflect the hierarchy of
content categories throughout the text. Helvetica Neue Light is also used at 10.5pt, with a 1.25
paragraph spacing, for the body of the text, in lowercase for the Table of Contents and lowercase italic
for the Table of Figures.
v.ii/ tense/first and third person voices
While practice-based research in the Arts and Humanities has, in recent years, made it more
acceptable to use the first person voice for parts of any academic project that require an explicit
inclusion of the personal experience of the author and her/his practice, still it remains the case that the
extent of use of the first person voice sits rather uneasily within academic discourse. This thesis
attempts to strike a balance of voices. It uses the first person where appropriate to describe the
aspects of the practice that draw on the phenomenological and personal. This usage is, however,
limited to those aspects of the writing where the ‘personal is of political, practical or philosophical
relevance. A more detached third person voice is otherwise employed.
v.iii/ italics
Text appears in italics in sections where a more personal voice is required: this technique is intended to
help the reader to move seamlessly and without confusion between parts of the academic argument
that stand on their own as ‘objective’ argument, in standard font, and those sections where a
phenomenological reading imbued with a personal voice adds another layer to the argument.
13
1/ introduction: ephemeral transference – backgrounds and contexts
If you could exchange your sleeping dream imagery, feelings and sensations, with your
friends and loved ones, what would it be like? If you could not only share and exchange,
but remix and collage them, what would it look like?
The mobile media performance project MINDtouch is based on a vision of the exchange of dream
imagery and embodied sensations through mobile media, wearable biofeedback sensors and
participatory, collaborative, networked activities as live media performance events. This telematic,
physical–virtual event-system is intended as a simulation of synaesthetic and embodied dream
experiences or to mimic a telepathic affective exchange, creating a paradox in the notions of
liveness/presence and of feeling/being-there. The use of live body data as a medium to mix concurrent
streams of mobile video in a multi-thread, non-linguistic conversation that creates a collaborative visual
dialogue, could enable a more embodied, meaningful and personalised exchange between remote
groups of people.
1.1/ ‘setting the stage’ (and a note on placement of the distributed literature survey)
MINDtouch explores embodied, non-linguistic communication and interaction, using wearable
biosensing devices and mobile phones as the ‘interfaces’ to remotely connected media experiences,
within social and performance events. In addition, the MINDtouch research has attempted self-
reflexivity by being cognisant of the effectiveness of the facilitation of enabling participants to
experience (or not) liveness and presence within these new mobile media social contexts.
MINDtouch is a performative research project that explores the experience of liveness and presence
within a series of live, iterative, in-person, staged and virtual ‘scratch’ participatory mobile media social
events. These ‘scratch’ or performance experiments involved improvisation and experimentation in
generating live, collaborative visualisations using mobile mixed video triggered by biosensor data from
participants’ bodies. The aim was to investigate how people ‘connect to each other’ with technology, in
alternate, embodied ways, to create a simulation of telepathic [1] dream exchange. Within the project, I
sought to understand how bodily sensations, perceptions, interactions and responses might be
meaningfully utilised to find unique ways to visualise the body/mind [2] activity, and to experience that
activity in a collaborative performance environment.
1 Here I use the term ‘telepathic’ to mean thought or experience transfer through the mind. It will be defined further on in this thesis in Part II in the exploration of Presence. I want to be clear at the onset that I am not trying to prove telepathy in this research, but rather I am trying to emulate it, and that requires a suspension of disbelief on the concept of telepathy for the reader.
2 The term ‘body/mind’ is chosen to acknowledge Massumi’s studies and theory, neurobiologist Antonio Damasio, Benjamin Libet and other philosophers, showing that the mind not only exists in the brain but also in the parts of the body, especially the senses, and is discussed further in Chapter 7.
14
To identify how or whether liveness and presence can be sensed during mobile performance activities,
MINDtouch used video as the mode of expression to explore levels of engagement, embodiment,
affect and sensation within live social and mediated environments using mobile and wearable devices. I
have attempted, through ‘iterative’ or experimental performances [3], to examine the sensing of
liveness and presence engagement, using biofeedback and mobile devices, during a few social events.
The aim of this study has been to understand what the affect, reception, and experience(s) of liveness,
presence of the performer and the audience might be, at the same time revealing their intrinsic
embodiment within the network, during the structured events and activities devised, accessible only
phenomenologically. As it has turned out, key to getting to this part of the research has been tackling
the technological challenges to enable this mobile participatory interaction to happen smoothly for
participants.
Critical to the investigation have been the active attempts to enable individual and collective
perceptions and embodied sensations of liveness and presence, within the context of the virtual,
invisible-space of mobile networks. Thus, the project has explored new ways to simulate, emulate, or
even facilitate telepathic connections or at least the sensing of feltness, presence and/or liveness, co-
presence and collaboration within a mobile performance. I have created events to enable presence and
liveness using physiological sensors to intensify the embodied interaction and engagement. During the
performance events the physical presence of participants was transformed, transduced or translated
into a digital video form, to allow them to ‘touch’ and ‘play’ with others, remotely and non-locally [4].
This was visually expressed and represented through their videophone expressions of their presence.
The various project threads were unified through the coding to enable the phones and sensors to
communicate with one another in as close to real-time as was possible.
Mediating between disciplines of performance and mobile video creation has been important in this
work, because the project has primarily been about devising and implementing methods to embody
the technological and communication networks and in a feedback loop. The first is from the body
directly with the sensors, and the second is through the interpretations of the body in the videos by
participants with the mobile phones. These impressions were then sent out to others to intermingle the
expression of physicality across distance. In this way, the two modalities of sending expressions of
physicality and embodiment through the network, the resulting visual mixes, as well as the audience
experience of the outcome, each required a specialised treatment and method of facilitation, not to
mention discipline for manifesting each, as will explained much more herein. Spanning these disciplines
has been also important for me personally and creatively, as I have been trying to find and implement
ways to address and integrate my many passions and artistic endeavours.
3 As will be explained in this document. I will show how I organised these events to examine these qualities in practice and within social contexts.
4 Radin and other Quantum physicists define ‘non-local’ as physical objects (or people) that may seem unrelated or separate, but are connected, transcending the limitation of space and time; or events that take place simultaneously either through space and time, in space or in time (Radin,1999: 277). This will also be discussed briefly in Part II.
15
Telepathy, for purposes of this project, is considered and further argued as the desirable and long-
sought-after ability to develop, as Jeffery Sconce discusses in his book Haunted Media: Electronic
Presence from Telepathy to Television (2000), in order to communicate directly to others without words
over distance. For this work, telepathy is considered desirable as a way to avoid the miscommunication
that can often occur through language. I assert herein that, if we could express our emotions, thoughts,
desires and sensations more directly to others by essentially duplicating or sending our embodied
experience to others’ minds and bodies, this might facilitate more empathy and understanding between
people. For this work however, I am using telepathy primarily as a metaphor, and am using the media
art project to simulate telepathy through a weave of technologies and activities. I also suggest that this
simulation of telepathic dream experience necessitates the understanding and facilitation of liveness
and presence of others to occur before the exchange is possible.
During this research process, a phenomenological perspective was taken to engage with the key
concepts of presence, liveness, embodiment, and telepathy in the practical context of performance, as
a means to assess my own assumptions about these more abstract notions. I also incorporated
discoveries made along the way. My underlying questions were:
1. Can a mobile, social interface simulate a telepathic exchange or a sense of presence at a distance?
2. How can artists understand previous notions of performance in relation to ever-new technologies and mediated activities, using both existing and developing approaches?
3. How is performance changing in response to networked technologies and wearable devices (mobile, biofeedback and other sensors, satellite/GPS, Internet)?
4. What is the relationship of 'real-time' computing or ‘streaming’ to liveness in mobile performance and games?
5. Is ‘performer’ another term for audience/viewer/visitor/participant of interactive mobile media performance work?
Discussed within this exegesis is the journey of developing this expressive, collaborative, participatory
telematic event series as a means of enacting mediated performance experiences over a mobile
network. It also explores the constructed virtual ‘space’ of a mobile, networked interaction and how, or
if, mobile participatory performance might be differently perceived from other forms of ‘live’
performance. I have also interpreted participants’ feedback on experiences during the mediated
participatory events. Another approach to participatory practice has been explored that facilitates
embodied experience of ‘liveness’ during a mobile, networked event. This research also hopes to
contribute an understanding of remote or distance perception and interaction, using mobile creativity
by enabling collaboration over the network. The theoretical research examines whether liveness can be
sensed within these mixed modalities of streamed, real-time, virtual, within in-person, live activities,
using mobile and wearable devices to interface the interaction between modalities.
16
I posit that mobile media phones act [5]: 1) as a conduit for non-literal or abstracted non-verbal
expression of experience and as an extension of the body/mind; and 2) as a vehicle to express inner
sensations between participants or with oneself. The unique contribution also includes my own
phenomenological perspective on the process of creating this ambitious project and the discovery of
the complexities involved with mobile networks. The goal has been to combine technologies not
otherwise used together to connect people remotely, allowing them to re-engage with each other
affectively and expressively in new non-verbal/textual ways, unlike current practice. Thus, the research
brings together divergent areas of new media research and digital performance practices, incorporating
physiological sensors that interface with mobile phone technologies, in unique ways.
For the BBC-funded dimension of the thesis project, the result is a greater, well documented
understanding, achieved through practice, of the possibilities and limits of technology as it can be
employed today to support a social networking application which enables participant interaction,
creativity and collaboration, in a more playful, visual, personal mode of non-linguistic communication.
The technical aspects of this project act as a prototype for such a tool, which would facilitate
alternative, direct, social and intimate communication and modes of expression between the body and
the digital device, with the user as the technology conduit in a mediated yet embodied dialogue.
This research aims to make a real-world impact as well as a theoretical contribution to the field of new
media with emerging mobile art, film, games design, locative media artists and designers, as well as to
the performing arts. It creates new modes and tools for performance expression and exploration. This
work may provide new parameters or a framework analysing this offshoot of the performance milieu
and media currently called ‘pervasive’ media or ‘mixed reality’ [6]. It could impact the Human Computer
Interaction (HCI) community, in terms of body-machine interfaces and modes of interaction,
contributing to new understandings of ‘interface’ and ‘platforms’ for intimate communication [7]. a note on the distributed literature survey
While it would be traditional to include a section containing analysis of all the key texts and authors
influencing this thesis in one section titled ‘The Literature Survey’, it is more appropriate to the structure
and subject matter of this thesis that such materials be distributed throughout the thesis, where they
can best demonstrate their own importance to the overall argument in relation to the diverse specific
theories and practices with which this thesis engages. For this reason, there is a set of analytical
summaries of key literature in relevant chapter sections in both parts of the thesis.
5 In conjunction with the appropriate software. 6 Defined and discussed further in Chapters 3 and 5.
7 Currently, many of the ‘mixed reality’ and ‘pervasive media’ projects come from the video-game and HCI communities, and very few from the performing arts. However, in 2010 both Blast Theory and a group called Theatre Sandbox are funding residencies and projects by theatre and performance professionals, in an effort to educate and encourage them to work more with technology in their performance practices.
17
1.2/ MINDtouch – summary of the major case study and its aims/outcomes This thesis engages with and analyses data gleaned from a larger R&D project (conducted by the
researcher in tandem with this thesis project, sponsored by the BBC) as its major case study. The
project, called MINDtouch , created a series of unique practice-based new media performance events
played out in real-time networked contexts. The MINDtouch events were framed as a means for
participants to simulate dream exchange or telepathic thought transfer using mobile phones and
biofeedback devices, linked to a bespoke video file protocol for archiving and sharing visual results.
While the issues (theoretical and technical) arising from MINDtouch as a project extend far beyond
the scope of this thesis, the first results as analysed here suggest that the mobile videophone could in
future become a means to make greater use of sensual semiosis and non-linguistic forms when
communicating from different physical locations. The corporeal, non-verbal forms of communication
and visual interaction observed when participants use such devices within performance events are
examined in this thesis.
The thesis benefits from access to the larger MINDtouch project and its original data, providing this
research with a set of process-based evidence files in both video and transcript form (contained in the
appendices, along with a full account of the project, performance events, user experience reports, and
biofeedback sensor design briefs). MINDtouch designed, developed and tested a new set of
biofeedback sensors and mobile phones, used together with a mobile network for video file sharing.
This set of tools was envisioned in the MINDtouch system as a possible emerging technology toolkit
to provide an alternative body–computer interface to trigger actions on mobile phones, in order to
discover what might be meaningful about technology-supported human interactions using ubiquitous
mobile devices. By analysing this unique data set and applying the theoretical contexts of kinaesthetic
philosophies where appropriate, the thesis demonstrates both the practical and the critical/contextual
effectiveness of the media facilitation process for the participants, and shares their senses of ‘liveness’
and ‘presence’ (of themselves and of others).
The thesis engages with the major MINDtouch case study and applies theories from fields of
kinaesthetics and phenomenology, including current theories of embodiment, in considering notions of
presence, liveness, ‘feltness’ and ‘telepathic exchange’ (all terms defined at length in this chapter). The
research brings together new thinking in the fields of media art theory, telecommunications,
neuroscience, performance and new media practices. The practice-based approach to exploring
theory has involved designing and applying bespoke biofeedback sensors to interface with mobile
phones, in the context of performance workshops incorporating theatre games and other creative
approaches to ‘being’ in both live and remote spaces. The thesis has also combined methodologies,
methods, practices and tools from Phenomenology, Ethnography and Practice-As-Research, and
Experience Design, bringing together the relevant aspects of these diverging areas of new media
research and media art/performance practices.
18
Thus, the MINDtouch project positions the physical body as the primary site of investigation from
which to embody technology. As such, the phrase ‘body as interface’, or ‘body interface’, can be used
in the context of this work. Body data and bodily processes, such as breath, blood pressure,
heartbeat, muscle electricity and sweat conductivity (as an indicator of stress), resulting from impulses
and responses, were used in tandem with custom mobile software to activate other technology in this
research. This data becomes the conduit for creative play and communication. The wireless
physiological sensors assist by connecting the embodied ‘interface’ and translating the data through
the mobile devices in a new form of non-linguistic expression.
MINDtouch harnesses the primary exteroceptive and proprioceptive senses through the sensing
devices, translating sensations and perceptions from participants’ bodies into a moving visual form,
from the ‘mind’s eye’ [8]. Instead of verbal or gestural language, the immediate surroundings of the
body, the architecture and the landscape within the frame, are employed with the visual or cinematic
language of the mobile video camera. Aided by technology, MINDtouch practically explores a form of
interpretation or conscious and synaesthetic translation of one’s internal world through the external
environment.
The project developed a deliberate, strategic methodology that served as the mediation between the
various disciplines and practices involved and discussed here. This hybrid methodology and overall
mode of working helped to achieve the goal of testing the qualities of liveness and presence in the
context of a performative technology-led project. This in turn lead into the specifics of the three project
phases below:
Briefly, the phases of MINDtouch project development were (see Appendix I for details):
1) phase one involved conducting mobile video workshops to collect the media.
Participants in the first phase video collection workshops were asked to explore and
visually represent their physical sensations, perceptions, emotions, thoughts and
impressions in a non-verbal, visual way, using the mobile video recording phones to later
share them with others;
2) phase two involved the technical development of the interface software for receiving the
physiological data on mobile phones, creating the video mixing software, as well as
assembling custom electronics and wearable sensors systems, then embedding these
electronics and sensors into suitable custom garments;
3) phase three was the staging of the live, participatory visualisation events, using the
sensor-embedded garments on participants, to trigger both the workshop video clips
and live streamed mobile video, from the event space. When these were mixed, a new
video collage streamed back to local and remote participants’ phones.
8 This is further developed in Chapter 5 on mobile video aesthetics, and in Appendix I on the workshops.
19
Phase 1 Phase 2 Phase 3
Figure 1 © 2007 C. Baker - early technical diagram of technical performance development phases and interactions
For MINDtouch , events were organised for groups to engage in live improvisation and
experimentation with mobile video, collaborative and generative VJing activities using the technology.
Within these events my goal was to see which activities and techniques worked with participants and
which did not, in terms of sensing others in the network. The data of participants’ experiences in this
primary series of mobile media performance events is collected and analysed in Appendix I. My
approach was to interview participants in order to gather their first-person impressions of their
performative experiences, in their own words. This data was then analysed to determine whether they
felt or experienced liveness and presence differently from during traditional performances. This analysis
would then inform the direction my research would take.
The primary media performance series for MINDtouch involved developing a database of archived
video clips from the phase 1 workshops alongside live-streamed video from participants’ mobile
phones during the events. These videos had visual effects added based on the body data of other
participants. A live, collaborative, visual performance montage of the effected participant videos was
simultaneously created. The remixed live cinema montage or installation was then restreamed back to
the remote and live participants’ phones and over the Internet.
20
Participants wearing the physiological sensors would be ‘performing’ the visuals through their body
data, creating the unique ‘mixes’, engaging in a conversation with the audience. The body data thus
controlled the video effects application and mixing, which were displayed in the live setting, while
accessed and contributed to both by live and remote participants. This video collage thus became a
collaborative, narrative and global mobile-cast, converging previously distinct technologies and
practices, bringing all the different virtual ‘presences’ together in the visual mix, simulating ‘collective
consciousness’.
During workshops and live events, participants become collaborators in the interactive, generative video
performances, streaming directly from their phones, using qik (proprietary third-party software), intended
to be in real-time [9] and from the media archive. This facilitated a type of ‘collective consciousness’ [10].
Each mobile media social event started with a short guided-improvisation activity, to inspire participants to
explore creatively. Some participants also used their bodies in a range of expressive, creative, non-
verbal/non-textual ways, which were then translated into the video mix.
1.3/ motivations, roles and voice 1.3.1/ motivations
I began creative practice with a relationship to words (as a poet/singer/songwriter/musician), then to
video and dance, moving to combinatorial practices of new media, performance video and mobile
media. Now, using all of these, my interest has moved to making the digital flesh, the electronic visceral
and the static alive through performance and social interaction.
Throughout my creative and academic life, I have been fascinated with the concept and experience of
telepathy. Definitions of telepathy often describe it as ‘presence at a distance’; others call it thought
communication (Radin, 1999, and Sheldrake, 2003). My definition is that it is the experience of sending
or receiving a feeling, sensation, experience, thought or idea from someone else (generally someone
close to you), non-verbally or through one’s mind, across distance, and most often during dreams
(Ullman, Krippner and Vaughan, 2002; Baker, 2004) [11].
9 Issues regarding and defining real-time will be elaborated on at the end of Chapter 8.
10 This is referring to the well-known Jungian sense of the term used in popular culture, defined here to mean minds coming together to a form of unity over distance. This will be discussed further in Part II theory and praxis, in Chapter 8.
11 In the past, I have reviewed many prominent scientists (biologists, physicists and neuroscientists, such as Fred Alan Wolf, Arnold Mindell, Dean Radin, Robert Jahn and Rupert Sheldrake) and their research on the existence and experience of telepathy, and have had the experience myself.
21
This background is relevant because MINDtouch focuses on simulating or emulating a telepathic
connection and a co-located sense of feltness, presence, liveness, co-presence and collaboration
within a mobile biosensing performance project. I have explored these experiences more deeply and
conceptually through the meta-theory level of the research. The premise has been that: in order to
discover whether liveness and presence is perceived by live and remote mobile participants during a
networked performance event, a visceral, sensory, emotional and philosophical grounding is key. This
project initially set out to explore the ‘body as interface’, but extended to create a visual, non-verbal
communication mode, possibly relevant to those who cannot, or who choose not to, speak.
In this and previous media research projects, I have worked with biometric medical devices as a means
to explore the use of the body as a conduit for media interaction, enabling users to explore and alter
their consciousness and experience a form of telepathy within an enclosed, mediated space. Using the
body essentially as a mouse, users engaged in a visceral way which, as intended, affected them very
deeply and personally. Finding ways to create in the body/mind of the participant directly from an
internal narrative, this past workpaired media supplied fragments intended to trigger mental imagery,
body experiences and sensations, memories, associations, fantasies and dreamscapes from the users’
past or current reality. This in turn triggered the media in an endless feedback loop meant to instil or
induce a meditative or pre-dream-like state.
For this previous work, the participatory media installation was an immersive, experiential media space,
incorporating biofeedback sensors, a variety of media, yoga and meditation to facilitate telepathic
exchanges. Ideally, this exploration was to lay the groundwork for more sophisticated ‘telepathic
portals’ or ‘telephone directory’, utilising mediated environments and systems. The impetus then was
also inspirational dream-like imagery, meant to induce a receptive state of creative consciousness in
participants, helping them to connect with others ‘non-locally’. My premise was that the body is the
integral site of the mind, so the experience had to take place within the body. I saw this as a return to
the body for interpersonal exchange, wisdom, transcendence, and exploration through body practices
and technology. This embodied, immersive experience was intended to attune people to their own
bodily sensations; a means to harness the ‘virtuality’ of sensation, imagination and a dreamlike
imaginative state. This work indirectly led me to the current project.
22
1.3.2/ key terms defined: 'liveness', 'feltness', 'embodiment', 'presence', 'co-presence', 'ephemerality', 'network', 'transference' liveness Liveness is the sense or experience of an event or person ‘in-person’, physically present, in the here-
and-now, being or taking place at this moment, in front of an audience in a traditional performance
context, or remotely (as with a live TV broadcast or online event). Liveness is intertwined with time
and/or space and physicality simultaneously, and entails an event or experience that is happening
concurrently with someone witnessing it. Yet it is not a priority that an audience is experiencing it in-
person or even in the same physical location as the event itself; or ‘in the flesh’ at the event location,
but the event must be experienced as taking place now and must be witnessed now , in real-time or
within seconds of the event. In a digital context, now-ness is critical to live events, but physical
presence is not: a computer avatar can stand in to substitute for actual bodily presence, so
representing a physical player is acceptable, and using voice-over-IP (Skype, etc) or video conferencing
to replace the corporeal presence can also be considered live.
feltness
Feltness is the sensation or perception of touch or the experience of being touched, either physically or
emotionally. It is a synonym for presence, yet has a more tactile quality and is concerned with the
‘feeling’ of others, viscerally, physically, or on an intuitive or emotional level. It is akin to how one feels
when speaking to a friend or loved one who is not actually ‘there’ with one in person, for example on
the telephone, or similar to the feeling of someone across the table from one, registered through the
qualities of the other person’s voice or their gestures. It is a physical, lived sensibility, like a vibration or
physical agitation. It is an exchange that is not one-sided, but a heightened perceptual sensation of
‘otherness’ and reciprocity, expressed between people or entities engaged in an embodied connection
(Massumi, 2002: 25) [12]. It is the feeling of others in localised real space or distant virtual space. embodiment Embodiment is the entanglement of mind and body as one, encased within the skin, enveloping the
bones and organs, blood, the brain and other organs, fluids and tissue. Yet it incorporates a presence
of a ‘being’, a ‘spirit’, of consciousness, or a ‘soul’ that seems to be separate from the corporeal or
fleshy encasement, but living within it (Varela et al., 1991: 27) [13]. Embodiment includes the elastic
ability of the mind to move beyond the boundaries of the skin and for one to send one’s presence
across distance (Idhe, 2002: 6). It is both inside and out, crossing boundaries of the senses, so that
they function separately, entirely outside of the mind (and vice versa), in an elastic sense, but not
detached entirely from the body:
12 This is argued and defended in Chapter 7 on presence.
13 This is discussed at length in Chapter 8.
23
Brain and skin form a resonating vessel. Stimulation turns inward, is folded into the body, except that there is no inside to be in, because the body is radically open, absorbing impulses quicker than they can be perceived, and because the entire vibratory event is unconscious, out of mind. (Massumi, 2002: 29)
Technological embodiment is a form of non-corporeality within which virtual connection can be
expressed through invisible exchanges of sentiment (Richardson, 2005: 6) or through a digital mode of
extended mind (Beauregard and O’Leary, 2007: 157), through projection or by sending the
consciousness through technology as a conduit. So embodiment can be 1) being in the body, and the
mind–body integration; and 2) being in the body, but also sensing and extending the mind beyond the
skin during dreams and in altered states, virtual technological spaces, and in other modes of
consciousness [14].
presence A sense of presence is a physical, lived experience akin to a vibration or agitation, felt both viscerally
and intuitively. Presence as defined here is a sense of ‘otherness’, a feltness that can be sensed in a
room, just as one feels a friend or loved one who is not actually ‘there’ with one in person. It is the
sense of feltness of the other. Presence can be experienced as meeting or sending one’s extended
body/consciousness or awareness of self or other [15] via invisible, non-corporeal embodiment. While
the flesh is not directly stretched over the distance, the mind and body are engaged by both parties
across distance, through extended embodiment. Presence is a togetherness without necessarily being
in the same physical space or place, but with an awareness of another.
co-presence
See presence above; co-presence is when one is aware that there are others feeling one’s own
presence. It is the simultaneous intuition of another person and their involvement in some mutual
activity or space (virtual or in person). It is felt when one is engaged emotionally, intellectually, playfully,
or otherwise taking part in an experience with others, and having the feeling of them also engaged in it.
It is the visceral knowledge that there is an ‘other’ out there somewhere, one who is alive and aware of
you, who is thinking of you, and you of them, either consciously or unconsciously; who senses you and
is reaching out to you consciously or unconsciously, emotionally or cognitively, physically or virtually,
who is concerned for you and who you are concerned for [16].
14 Here I am referring to virtual in the digital and online context.
15 See Rupert Sheldrake’s theory of the Extended Mind in A Sense of Being Stared At (2003).
16 This knowledge is based on prior relationship with that person as discussed in Chapter 7.
24
ephemerality
Ephemerality is the quality in something of being temporary or transient and intangible, which goes
beyond language, leaving ‘no visible trace’ (Phalen, 1993: 148, when describing Performance;
McKean, 2005–2010: online)
network
A network is a form like a spiderweb or rhizomatic structure, like a root system. Mitchell describes it this
way:
[…an] accumulation and habitation [of] sites, links, dynamic flow patterns, interdependencies, and control points (2003: 9).
Or Massumi says,
The network distributes. Interlinks. Relates. The network is the relationality [...] (2002: 86).
The New American Oxford Dictionary 2nd ed. (2005) defines it this way:
network |net-w•rk| noun 2 a group or system of interconnected people or things […] • a number of interconnected computers, machines, or operations. (McKean, online)
These are only two definitions, but the key term is ‘interconnected’. My work is about interconnection
between people, and between people and things (i.e. mobile and computer networks, systems and
operations).
My own conception of network connections is that they resemble: threads, sewing, netting, weaving,
knitting, fabric; enmeshed, entangled or tethered forms; knots or rigging; branches and roots; cellular
forms, veins; and tunnels or path systems. MINDtouch involves a network of bodies, connected to
sensors, connected to phones, connected to a database of video clips, manipulated by the body data,
connected to the world of networked possible interactors/participant performers online globally.
transference
For the purposes of this thesis, transference is the act of moving ideas and/or sensations from one
location to another: a thought, feeling or sensation moves from one person or body to another, over air
and distance. Sometimes transference is the act of extending embodiment through technology (see
above).
25
1.3.3/ voice
Throughout this document, and particularly in Part II (which explains the relationship between theory
and practice), a first-person voice periodically enters the text. At times, the voice takes on a deliberately
abstract or poetic register (primarily in the section on theory, Chapters 7 and 8). The use of italics is
intended to help the reader see the shift in register from the practical and academic viewpoint to the
personal and phenomenological. This is a stylistic and personal choice, evident when discussing my
conceptual and phenomenological experience framing my position on theories of presence and
embodiment. As mentioned above, I have different roles in this research, as performer/artist, as well as
the director planning it, and the curator/researcher discussing the works of others. As such, in this
exploration I have included some reflections and epiphanies (I often call them musings) or personal
insights on concepts discovered through the work or inspired by others. These reflections are
contextualised through others’ experiences, which aided me in finding other approaches and like minds
to demonstrate shared thinking.
Anita Ponton says in her thesis Vertigo: The Technophenomenological Body in Performance (2005):
While I do not address ‘voice’ as a specific issue in my thesis, throughout the text there are different registers of voice. Given reign to speak, they are symptomatic of my desire to write ‘through’ the practice, to follow the (il)logic of my practice, in order to take the text into the same zone (at the very least) as my practice. (Ponton, 2005: 23, my emphasis)
In a similar way, there are sections here that have a more first-person, poetic and introspective position
than the more theoretical sections. In these sections I have sometimes mused upon conceptual
explorations from my own experience and practice of presence in mobile and digital interactions. Other
sections are written in a more conventional academic, third-person voice. This is maintained for
academic rigour when discussing the ideas, theories, practice and works of others.
As this is the work of an artist, singer/songwriter, performer, poet, videographer and former dancer, it is
important that the theories and philosophy be read ‘through’ the practice and in the language,
metaphors and musings from my journey and the experience of it, explored and uncovered during the
research and framed at times personally. Taking a phenomenological approach has meant that in
practice and in writing, the work and process is described from a first-person, lived perspective, to
allow the reader to follow my journey within and during the project. I discuss this choice more deeply in
the methodology section in Chapter 2, and I defend the Art-as-Research and phenomenological
approaches as a critical means to explore and analyse the experiences of others more empathetically,
since they demand a sensitive understanding. As an artist, I can only ‘read through’ my own experience
of the practice and its staging, rather than pretending that any research is truly objective, especially as
a personal art project.
26
1.3.4/ roles
As the author of this thesis and also the creative director of the larger MINDtouch research project on
which the findings are based, I have played several roles in the research process, including: 1) media
artist and performer; 2) action researcher using the phenomenology of experience and experimental
mobile performance in order to explore abstract conceptual notions and qualities of experience; and 3)
performance director/facilitator of the case-study events. This last role is critical to my personal aim
(and my reason for choosing to undertake the PhD with the socially committed team at SMARTlab) of
practising my theory and applying all knowledge gained to the act of facilitating social experience, both
socially and artistically. My background leading to this point is critical, as it explains how I arrived at this
project and not any other. It demonstrates that my unique embedded, embodied and tacit knowledge
as a curator, performer, media arts researcher, and even employment counsellor/facilitator, from my
past lives, makes me uniquely qualified to carry out this research, in this way, as discussed further in
the methodology section in Chapter 2.
Below is a chart that expresses the various roles more clearly:
Researcher/Curator role Artist/Director role Facilitator role
• Exploring the concepts (liveness, presence, embodiment, telepathy, etc)
• Finding recognised theory to ground the concepts
• Finding relevant artists to ground the practice
• Surveys and analysis of feedback
• Devising performance activities
• Developing database categories from video content
• Working with technical collaborators on video effects, sensors/electronics and mobile software
• Devising the interfacing design of the technology
• Directing the staging of the events
• Facilitating workshop activities
• Facilitating performance activities
• Ensuring the technical collaborators were engaged and feeling valued for their work
• Guiding the guides for the performance events
• Setting up a virtual space for online participants
• Following-up with participants after workshops and events.
Figure 2 © 2007 C. Baker – chart of project roles
As a curator [17], I have been concerned with new media performance and all its permutations (live,
online, etc), as well as net art, interactive installation and technology-based performance. What excites
me is how current developments in media-based and new media performance practices are
addressing or investigating the presence or ‘aura’ of the performer, over time and through or across
space, and how this in turn affects audience members and their reception of the performance. I am
also concerned with the performance experience for the media artist and her/his audience, as well as
how the content of the work affects, or is experienced by, the audience. The performance staging and
17 I was Co-Executive of The Escape Artists Society, based in Vancouver, BC, Canada, from 2005 to 2007, which was an electronic and new media performance event organisation focusing on presenting site-specific works from interventionist artists.
27
nature of the actual experience for the performer are also worthy of investigation. Curatorial issues and
practices have been challenged by new modes of presentation and performance in the last decade,
especially with technology-based and participatory works, so this area warrants review as well [18].
In my practice as a video and digital artist, performer and curator, lecturer in film and video history,
media aesthetics and practices, various curiosities have surfaced, such as VJing and Live Cinema. It
has also long been a fascination of mine to use mobile devices to marry performance video mixing with
telematics (also known as networked performance, discussed in Chapter 3). Upon starting this
research, I had a vision of the work I wanted to create, based on my knowledge of the Internet, of other
digital art projects that used telematics, mobile and wearable devices, and on past art research, such
as with whisper[s] (Schiphorst and Kozel, 2003-2005). Experience as curator with the Escape Artists
Society (2005-2007) and the New Forms Festival (2002-2004) in Vancouver also aided. However, I
quickly became aware of the ever-changing landscape of mobile technology, as well as the latest in
medical, as well as DIY, physiological sensing technologies on the market. Here the curator is
transformed into more of a theatre director role, devising and ‘rehearsing’ the performance elements to
get the best possible performance and experience from and for the participants, or ‘actors’.
I bring embedded experience as a ‘life skills’ and employment counsellor from another part of my life
into my practice as a facilitator. When I returned to post-graduate studies in media art I left six years of
working in social services, aiding and facilitating people to find work or new career paths, better living
conditions or to at least get off the street. I have tried to carry the learning I gained from this former
career into my media art and performance practice, to find ways to continue to enrich people’s lives
through enabling them to have deeply felt experiences. This background makes my approach to
practice somewhat different from others’. Facilitation has a whole rich history and philosophy not
covered in this thesis. However, I carry my own interpretation of it as the enabling and shepherding of
others through a process or through learning. It necessitates checking regularly with participants that
they are safe and comfortable in the experience and that their needs are being met as well as possible,
to reach their own goals or expectations in the process, whatever process that might be. 1.4/ methodologies
The MINDtouch research project has combined methodologies, methods, practices and tools from
Phenomenology, Ethnography, Performance, Art-as-Research, Experience Design, bringing together
diverse areas of new media research and media art/performance practices. The project involved the
use of biofeedback sensors interfacing with mobile phone technologies in a unique way.
18 Yet sadly these are beyond the scope of this research.
28
The MINDtouch research project has been a means to innovate performatively and technologically,
within a relatively new performance and technology domain. However, this research is developed as a
media artwork within the context of an academic environment, and realised through a mode of practice
not possible through written scholarship alone. ‘Art-as-Research’ methods, more commonly used in
recent art and performance research, are primarily employed herein. Other methodologies and their
methods, practices and tools used in the research are complementary and layered together as part of
this Art-as-Research (AAR) methodology. In this research project, the mobile media performance series
became a case study, from which data on the experiences of the participants have been collected.
The hybrid practice-based art research methodology devised and composed for the project includes:
1) performance-based structured improvisation techniques from theatre and dance used in the
workshop devising activities, and as a means of helping to engage participants in the activities during
the events; 2) ethnography for collecting and analysing the interview data; 3) phenomenology for the
overall creative design approach and process analysis; 4) experience design for devising and
implementing the experience both of the activities and of the digital interfaces; 5) visual methods for
categorising and analysing the video imagery; and 6) technology design (empirical) methods used in the
technology development. These methodologies and their methods, tools and practices were
implemented to create this hybrid methodology, as a way to thoroughly address the work. I have also
borrowed an approach called ‘scratch performance’, used in London theatre and live performance,
which means ‘work-in-progress’ or iterative theatre, performance that asks for feedback from the
audience to develop a piece. This is similar to iterative design and other creative approaches that use
repetition, discussed more in Chapter 2 on methodologies.
Important to note here is that the work would not be realised by one of these methodologies alone, as
each addresses different aspects of project development for performance staging and technological
development:
• Phenomenology – for observation of self as artist/director/researcher in planning, performance
staging and experience analysis; to work from a first-person, lived-experience perspective in the
various roles I play in the research, and as a way to be a participant in the research as well as a
researcher, artist/director and facilitator; and as a way to understand my own inspiration and
choices as they surface in order to (in a way almost akin to Action Research) devise the research
and be ‘inside’ the research to better do so;
• Structured Improvisat ion – for devising and actively engaging participants in the
performance activities; to explore ways for the participants to interact with the technology and
each other and to discover how best to devise the activities;
• Ethnography – for observation of participants, note-taking and analysis of the participant
experience within the activities; to understand the experience from the perspective of the participant
but also to observe their behaviour from the outside, in order to analyse performances and
feedback from them;
29
• Experience Design – interwoven with and interdependent on the previous two; focused on
creating engaging activities with the technological devices, garments and software, as well as
the collaborative video collage they create together; taken from the graphics and web design
field as a means to craft the interaction design of the experience with the technology effectively
for participants;
• Visual Methods – for analysing the content of the visual imagery in order to devise
appropriate database categories for the software to trigger the images from body data received
by the sensors; as a way to understand the intended meaning and content of the visual
material created by participants, from a visual aesthetics and symbolic position;
• Technological Design Methods (empir ical ) – employed mainly by and with the technical
collaborators and software developers, in order to create, assemble, code, test, debug, iterate
and finally work with the technologies used; used to iterate and perfect the technology for use
in performances.
These are all discussed in much more detail in terms of why they were chosen and used in Part I,
Chapter 2 and their use in practice and the outcomes of their use in Appendix I.
1.5/ performance context (Literature Review Part 1)
I am not a scholar of the recent currents in the debates on Art or Practice-as-Research, especially as it
applies to artist-researchers doing PhDs in the UK versus Europe, North America or elsewhere. The
discussion here is based on research on artists’ experience and the use of their practice to create
knowledge. I am interested in what they have to say regarding their processes and methods and how
they have gleaned knowledge from these to contribute to the art world. The authors below were
selected in terms of how their ideas resonated with my own experience. I engage well with the artists
and voices presented here because they contextualised my use of practice in research. Therefore,
nuanced debates around the differences between Practice-as-Research, Art-as-Research or Practice-
based Research are not addressed within this discussion. There were no differences found during
searches. I am aware that Practice-led Research – research based on the work of others or ones’
previous practice, rather than a current project – is different from the above categories within
academia. I also know many practicing artists personally who were not aware of these distinctions until
entering academia, so they may be distinctions primarily imposed by the academy and not by
practicing artists. Further discussion on these debates is not within the scope of the thesis.
Key thinkers in performance theory have contextualised old and new directions in performance
practices, and were essential in identifying where new insights might be found. Sociologist Erving
Goffman first brought common behaviour of individuals to the fore (1959). He focused on the
30
performativity of everyday life – how we all play a ‘role’ in each social or personal interaction – and
defined ways in which we do so. Richard Schechner (1988 [2003]) has become a strong voice in
Performance Studies in the last 30 years. He broadened thinking and previously held views on
performance definitions within Theatre and Performance Studies. He did so by demonstrating that the
performativity in everyday life and practices, such as in play, sports, legal proceedings, rituals and pop
culture, was valid beyond the arts realm of theatre, dance and performance art. On the other hand,
Philip Auslander (1999) focused on how liveness is in opposition to what he calls ‘mediatisation’ in
performance, claiming that liveness did not exist outside the context of the mediatisation of the 20th
Century and that this aspect is key in understanding performance in the modern era. Meanwhile,
pragmatist philosopher Richard Shusterman (2000) explored the aesthetics of the experience of bodies
and performance for the audience, suggesting that conventional audiences and art viewers feel that the
art establishment has lost interest in satisfying their needs for affective, emotional, transformational or
aesthetic experiences [19]. He claims that people have gravitated to popular art forms and
entertainment to meet these needs. Boal’s (1992) participatory theatre work was highly influential,
challenging and working with actors and non-actors in order to engage (or reengage) them socially and
politically with the world.
As discussed in Chapter 5, performative video/live cinema/VJing is a form that uses my combined
interests in video, music and performance and thus the practice becomes relevant and exciting to
explore further in the project. Although there is still little written on it theoretically, this is changing.
Jaeger (2005) attempts a critical discourse on VJ practice and culture through a history of it, describing
what it is and is not artistically and practically. He then contextualises it in the larger art and cinematic
context, placing VJing within live and improvised performance practices. He discusses the practice of
‘jockeying’ and its behaviour and labour, as well as the role VJs play as artists, software designers and
performers, closely tied to film and digital art.
Youngblood’s (1970) Expanded Cinema is a forerunner of wider contemporary performance video
practice and is a surprisingly current and astute look at networked technologies of that day and the
new role that cinema could play with them. Youngblood looks at video-based artworks that used the
networked technology of the time (telephones, television, satellites, etc), comparing them to the human
brain, suggesting that they simulated a collective consciousness. This utopian work envisions how
cinematic and new technologies might extend human communication capabilities beyond imagination.
It also explores the possibilities of computer-based and holographic cinema, with visionary hopes for
cinematic development eventually leading to performative and live video creation. This work leads well
into the project examined here and its efforts in mobile ‘social VJing’ [20].
19 And, I would argue, some in the art establishment are too elitist for the populace.
20 This is what I was calling the project to draw participants to the events when advertising it as part of the social festivities of the parties it was a part of.
31
1.6/ mobiles, databases and network freedom 1.6.1/ mobile and database media
The discoveries made during the MINDtouch research on mobile screen aesthetics (Chapter 5) have
come primarily from observing mobile video capturing behaviour within the custom workshops and
participatory performances. Some of these include: the ease of expression of video capture using the
mobile; the intimacy [21] and affective [22] nature of mobile devices, the inherent gestural qualities
implied by the size and portability of the phones, as well as the ubiquity and accessibility of them.
The landscape I entered when I began the PhD study was varied and the research works in my areas
of interest all seemed unrelated to each other. Similarly, when I initially reviewed telematic and other
relevant art works using telepresent performance or mobile phones, databases and networks, they
were all varied and differed greatly from each other and from my focus. None exactly addressed my
interests. However, a few key voices stood out early on in the research, and they are described below:
Glesner (2002) presents telematic and distributed performances as emerging performance forms,
challenging the spatial relations between performer and viewer, by describing ‘telepresence’ and
‘liveness’ within Internet mediated environments as site-specific performances. This is shown in the
light of these performances’ interactivity, the audiences’ sense of presence of performers, and the
overall qualities of ‘liveness’. This is an intriguing concept if one thinks of the Internet as a place or ‘site’
‘where’ performances can occur, which then assumes an embodiment of or within the network
(discussed more in Part II). Dealing with presence slightly differently, Hauber et al (2005), come from a
Social Psychology position, and tackle the quality of ‘presence’ within the contexts of two-dimensional
and three-dimensional interfaces for videoconferencing. They define ‘social presence’ in terms of the
sense of ‘being together’ or ‘being there with others’, and base their claims on research of participants'
reported experiences of interacting with each other using different conferencing interfaces. Again this
assumes the embodiment of virtual space and of the network as a place or site. This in turn assumes
an extension of non-corporeal consciousness, discussed further in the embodiment theory in Chapter 8
in Part II.
Among the few intriguing voices on locative media is Gianni Corino in his thesis Spatial issues and
performative media in digital mobility: a network perspective (2006). He focused on the social, urban
and media aspects of performance in the use of mobile communication devices and how they enhance
people’s sense of place and connection. He examined the history of digital mobility, the development of 21 In the context of this project and my work overall, the term ‘intimacy’ is used in a generic, common, dictionary use of it, meaning: closeness, private, personal, familiar, in relationship to or on the body, or between two bodies/ people in relationship. 22 Affect and my use of it are discussed at length in the Literature Review sections, particularly looking at Massumi, Merleau-Ponty and Grosz’s interpretations of it. Briefly, I use Brian Massumi’s definition: Affect [...] as couched in its perceptions and cognitions [...] implying a participation of the senses in each other: the measure of a living thing's potential interactions is its ability to transform the effects of one sensory mode into those of another (tactility and vision being the most obvious but by no means the only examples; interoceptive senses, especially proprioception, are crucial) [...] Formed, qualified, situated perceptions and cognitions fulfilling functions of actual connection or blockage are the capture and closure of affect. Emotion is the most intense [...] expression of that capture [...] (Massumi, 2002: 35)
32
the cellular phone and social network theory. Corino also discussed recent activities in performative
media and artworks to expose the directions that future mobile communications may take based on
current explorations and applications. In Flash! Mobs in the Age of Mobile Connectivity (2005),
Nicholson focused on how mobile phones have changed the social practices of public performance
and activism. He looked at how the performative phenomena of Flash Mob events in 2003–2004 did
this. He demonstrated how mobile phones have transformed the way in which political and social
activist events are organised, then moved to how they paved the way for anti-political Flash Mob
events to take off. Nicholson details the incremental changes and precedents set that enabled these
activities to come about, and how they altered the way artists, and people in general, began to use
mobile phones – to instigate distributed social activity and performative events [23].
As a precursor to mobile phones and mobile art, there were also artists who explored communication
and distance through other forms. Neumark’s (2005) At A Distance: Precursors to Art and Activism on
the Internet addresses art that uses distance as the main constraint, influenced by geography,
technology, time, presence, space, connection between two points and the gaps in between. In
particular the chapter called ‘Introduction: Relays, Delays, and Distance Art/Activism’ focuses on text in
postal art and the precedent it set for current Internet art, specifically the concepts of relays, delays and
the anticipation of waiting. Here Neumark cites prominent theorists and their interpretations of time and
space, and how distance artists addressed these theories, both before and after Internet art
developed, exploring the past to understand current practices in bridging distance through art.
With respect to networks and network art, the article ‘Theses on Distributed Aesthetics’ (2005) by
Munster and Lovink develops a theory reaching beyond the Internet and television to now include multi-
player games and mobile media. The authors’ theory of distributed aesthetics which a network as a
theatre ‘black box’ of possibility for various media, activities, practices, users, artists and experiences.
They see this ‘black box’ as an amorphous, unstructured and unpredictable space, like an emergent
entity or complex ecology, not easily mapped or navigated. They also include a history of when
computer network mapping started in the early 1990s, alongside changes in definitions of aesthetics.
Several artists working with art and databases were featured in the book Making Art of Databases
(Brouwer, 2003). The artists included: Sher Doruff, Raphael Lozano-Hemmer with Brian Massumi, Lev
Manovich and Joel Ryan. In this series of essays based on four master classes, artists helped to
develop various systems to work with archives and databases to make interactive, dynamic and
responsive media works. Each artist wrote an essay on their own master class, as well as their practical
interpretation of ‘database’ in the context of their own work, and their conceptual view of database
structures and the complexity arising from their experiences with them. Lev Manovich’s work, as well
as some others here, will be explored in more detail in terms of database and generative video
aesthetics in Chapter 5. Artists that cover all the domains discussed above are delved into more deeply
in Part I.
23 Later I look at Boal’s work and how he could be considered a forerunner in instigating this type of social or theatrical activism.
33
1.6.2/ dreams of networked freedom The MINDtouch work collates many different popular visions or daydreams and science fiction desires
together, such as the sleeping dream recorder in Until the End of the World (1991) by filmmaker Wim
Wenders. These visions are introduced here to sketch and introduce the territory covered within this
exegesis. The work combines the vision of the video-capable mobile phone from so many sci-fi films,
including real-time mobile video interaction (which may well be developed by the end of 2010). As life
imitates art, art and science also imitate art. These films have not directly ignited my passion, but the
vision to make performances involving videophones has been my desire for almost a decade. I have
been waiting, and am still waiting, for the technology to catch up. Now I wish to find ways to push it
closer.
A seemingly simple goal of simulating embodied dream telepathy, researched and demonstrated as
possible for some during sleep (Ullman and Krippner, 2002) [24], has become a more complicated and
constrained activity in many ways. This is due to the complexity of current mobile networked systems,
not to mention private or proprietary systems that involve boundaries, passwords, firewalls, etc. One-
to-one connections in these networked contexts are generally reliable, however ‘one-to-many’ is still
problematic [25]. So many-to-many, in collaborative, real-time video interaction is still complex, difficult
and resource-intensive, as has been discovered during this research, especially as a lone
researcher/artist with a limited budget.
The history and expansion of mobile services has been more extensive and quicker in Europe (Greene
and Haddon, 2009: 22), and mobile cinema, art and participatory projects have sprung up and
expanded as a result. In addition, mobile phone use has been more ubiquitous and has grown quicker
in Europe (compared to Canada, my country of origin). Other mobile advancements are also superior to
what I was used to [26]. An exciting development occurred when Nokia and other mobile
manufacturers rolled out their phones with video recording capabilities to the market in North America
and Europe in the mid-2000s. As big international corporations, it would be expected that Nokia and
their competitors would have test marketed these products and negotiated with the partner service
providers to enable support for video and video streaming, anticipating its adoption and varied use [27].
Thus, one would expect that they would have negotiated more extensive and accessible network use
24 Radin (1997) has done metaresearch on all telepathy studies over the last couple of hundred years; whereas Ullman and Krippner (2002) have done much research on dream telepathy, especially how and when it is experienced, studies that inform my assertions herein, especially in Chapter 7 in Part II.
25 iChat claims it can support many video-chat windows simultaneously, yet in practice this is not successful with video fidelity and smooth, uninterrupted connection, unless one is on an extremely robust, fibre-optic broadband Internet connection (see this report online for more at http://www.dslreports.com/forum/remark,13288787 [accessed May 25th, 2010). Many of the multi-threaded chat sessions I have participated in were not able to support a medium to high-resolution audio-visual quality involving 2–3 participants, with only audio coming through effectively; 1-to-1 Skype video is now theoretically possible on a mobile phone, but no one I have spoken to who has tried it is able to get it to work effectively yet. 26 See news story on this issue online at http://www.cbc.ca/news/background/tech/cellphones/economy.html [Accessed May 25th, 2010].
27 They had, but in a limited way. This is slowly improving.
34
with these telecom service providers, to ensure wide adoption and ease of use of these services, for all
types of possible activities that the phone models enable. However, this research has found that this
was not exactly the case.
Security has become an obsession in society, in the digital as well as the 'real' world, since the
September 11th, 2001, World Trade Centre terrorist attacks. Many other world incidents of high-level
hacking (such as the hacking of Google by the Chinese in 2010) have also underlined its importance.
The media add to this fear. And yet, we are never truly safe in the ‘real’, everyday world. We are always
vulnerable (physically or otherwise) and that is, for me, what makes living interesting. Yes, people die
everyday, every minute – they are attacked, raped, bombed, etc, and experience loss, pain and
torment. This is life. The obsession with security, privacy, pass-codes, 'secure connections', firewalls,
virus checkers, censorship, etc, takes the individual responsibility for good behaviour and citizenship
away from individuals and communities, placing it in the hands of governments, institutions and
corporations. By buying their products, our ability to govern our neighbourhoods and ourselves is
removed, which is more frightening. This makes open-source and online creative commons in the
coding and digital creative worlds an inspiring but subversive act, or at least a political decision.
However, openness and the commons have not fully infiltrated the more controlled mobile world,
similar to the Internet. This might be due to the nature of the market-driven mobile system, the
proprietary, dependant and closed nature of mobile operating systems to date, which are managed by
corporate stake-holders. Some may say that open-source is just a corporate strategy to keep
innovation high while reducing research-and-development costs, by outsourcing to a wider network of
coders. This may be true, but if so it still allows more open access for users and developers to explore
software design, giving more options overall [28]. Many of my peers who are artists and technologists
are beginning to develop useful open-source tools and projects. These projects will require more open
networks to stream media more easily, without continually encountering economic and security barriers
[29] as I have encountered when trying to stream video from university campuses. This is discussed
further in Appendix I (see also the testimonials from technical collaborators on these issues in the
appendices).
This project was initiated from experience in working with mobile technology and wearable sensing
devices [30]. My expectation was that the intervening few years of development would have improved
these devices’ interoperability, especially in light of the last decade’s exponential speed of technological
28 An open-access summit was discovered while writing this section, taking place at the same time, discussing these very issues, and a ‘new mobile ecosystem’. It can be referred to online at http://www.openmobilesummit.com/aboutus.aspx (accessed on May 25, 2010).
29 More on markets and content delivery models will be discussed later on. Feijóo et al. (2007), in their paper Why YouTube Cannot Exist on a European Mobile: The European Regulatory Strategy on Mobile Content, discuss the different models that Europe and the world have taken on and why open access is not attractive to them from a monetary standpoint, even if it is what users and content developers might want. I was able to get the proceedings to the conference late 2008, and only ready in the project timeline, to address this in July 2009, but only encountered severe difficulty in after that.
30 I started working on MINDtouch having come from working on my Masters project and on the whisper[s] project for 2 years (see project online at http://whisper.iat.sfu.ca/ and discussed later in this Part I), which lacked the level of funding and engineering expertise of MINDtouch.
35
advancement. As an artist, I had to find a technical collaborator who knew the intricacies of medical
sensor devices and electronics, mobile phone programming and network systems, someone who was
interested in the challenges of working on a performance art project. This was a difficult task. After
several false starts and a year into my research, a few different people were found to work on the
project. In all the planning discussions with these technical experts, and others throughout the duration
of the project, nobody expressed much concern about any substantial mobile network barriers. As the
only constant person on the project and not a network systems expert, I became alarmed when these
barriers were encountered near the end of the project, during the performance events when network
connectivity was most needed.
At the onset of this research project, the indications were that the level of mobile networking that we
required would be possible online using mobile phones. Using advanced smartphone models, such as
the video-enabled Nokia N-Series phones, the systems and services initially appeared to be in place to
support complete mobility, including the ability to stream live video, but in fact only now, at the
conclusion of the research in 2010, is this becoming realistic. However, it is still not easy to do, or
reliable, without an accommodating service plan, phone, stable connectivity or company/service
provider, access codes, etc. Hopefully this will change in the very near future to enable a next version
of this project.
From the beginning of the project, 'unlocked' phones were chosen (meaning not ‘locked’ to a contract
or service plan). This would not have been easily done in North America in 2007; even though one
could purchase the phones, they would not work in North America. In early 2007, one could rarely find
pay-as-you-go SIM cards in North America, as a different system was in use (although this changed
soon after). However, by the end of 2007 in the UK, one could buy 'throw away' SIM cards, which
were purchased for the research phones for workshops and performances. My programmer
collaborators aided in 'hacking' into the medical sensing systems for the project – initially with the help,
knowledge and blessing of the manufacturer – to interface with the chosen phones. This began to
backfire as there were difficulties eliciting assistance from the sensor manufacturer in order to create
the custom software for our phones, [31] and their source code was too complex (and too poorly
documented) for my programmers to work with or for use with the phones’ operating system.
It was then decided that I continue down the DIY path, as artists often do (partly because it is less
costly). We began to work instead with more off-the-shelf sensors and electronics, which Michael
Market, my eventual collaborator, assembled and programmed. We also employed free, open-source
coding options. This meant sacrificing some quality and fidelity in the sensor data (less critical to artists
than medical doctors), as we had to ‘daisy-chain’ the technologies and find workaround options (at
least to the extent of the original vision), but it lent us more freedom. Michael and other technical
experts advised taking this route since the technology was not quite ready at the time for how I had
31 This was partly because they also produced their own sensing Software Development Kit (SDK), and perhaps did not want or have time to support our endeavour.
36
envisioned the project to work. I was optimistic that we could still create a version of my original vision.
What was not openly acknowledged was that the artistic decisions we made (partly due to limited
resources) would cause the project to collide head-on with the normal, condoned practice when using
mobile phones. While streaming video from a mobile phone is completely feasible using mobile
software applications such as qik , l ivecast and ustream (qik now ships with all Nokia N-series
smartphones since N-97), in practice delays and network barriers have consistently been encountered
at every organised event. This was especially true when trying to stream to and from university
campuses in the UK, as well as other firewalled networks. The reason given generally is that this
activity, no matter how low the numbers of streams, opens their secure ports to malicious hackers and
is thus a security risk.
So we had to avoid standard phone systems, instead repurposing the technology for our own uses.
This created added difficulties and challenges to achieving the original goals. Although none of what
was envisioned is impossible – according to most technical experts I have spoken to – it is harder with
limited resources and an ad hoc technical team.
1.7/ philosophical landscape (Literature Review Part 2) Theories and critical texts relevant to the MINDtouch project are expanded upon in Part II. Theories
were investigated through practice directly or indirectly within the series of iterative events featuring
collaboratively performed visuals meant to simulate dream and embodied mixing and exchange. Herein
is a review of relevant reading that was engaged in during this practice-based research. A few key
theoretical concepts that ground the practical work include: presence and perception, consciousness,
embodiment, the senses, telepathy, dream state, participatory performance practices, mobile media
aesthetics, and live cinema. Various theorists are examined herein, though these by no means cover all
the territory explored in the research process. However, below are discussed those who were most
relevant to the work and/or were key influences on my thinking, development and practice.
For thousands of years, philosophers have asked questions and postulated answers regarding the
nature of life, the universe, the nature of human experience, etc. Prominent thinkers who are still
studied include the Greeks such as Plato, Socrates and Aristotle. Building on their work in the last
several hundred years, prominent philosophers have included Kant, Kierkegaard, Hegel, Heidegger,
William James, Sartre, Bergson, Derrida, Foucault and Wittgenstein. Early in my academic studies,
stimulated by a personal tragedy, I was drawn to Existentialism and its focus on individual and social
daily existence and its expression, […] which attempts to man in his immediate, his original,
relationships to the universe, in all his concrete plentitude – and problematic ambiguity (Spanos, 1976:
2).
37
In masters-level study with Dr Susan Kozel, the field of Phenomenology was first brought to my attention as
a relevant framing methodology and critical context for my work. According to Spanos, ‘Phenomenology’ is
critically related to ‘Existentialism’. I paid close scrutiny to the relationship between those concepts, and
application of those ideas to the work of other phenomenologists including Edmund Husserl and his student
Martin Heidegger, followed by Merleau-Ponty and many others.
The Stanford Dictionary of Philosophy defines Phenomenology as a philosophy of existence and states that it is:
[…] the study of structures of consciousness as experienced from the first-person point of view. The central structure of an experience is its intentionality, its being directed toward something, as it is an experience of or about some object (2003: online).
It also states that,
For Heidegger, we and our activities are always ‘in the world’, our being is being-in-the-world, so we do not study our activities by bracketing the world, rather we interpret our activities and the meaning things have for us by looking to our contextual relations to things in the world […] We must distinguish beings from their being, and we begin our investigation of the meaning of being in our own case, examining our own existence in the activity of ‘Dasein’ (2003: online) [32].
As a prominent phenomenologist who is used extensively herein, Merleau-Ponty describes Phenomenology as:
[…] the study of essences; and according to it, all problems amount to finding definitions of essences: the essence of perception, or the essence of consciousness, for example [… it is] also a philosophy which puts essences back into existence, and does not expect to arrive at an understanding of man and the world from any starting point other than that of their ‘facticity’ (Merleau-Ponty, 2002: vii).
Merleau-Ponty’s Phenomenology of Perception (2002), has provided seminal theories for many within
the media arts. Not an exception, I too will look at his ideas. However, I will not reference the other
usual suspects – Deleuze, Bergson, Benjamin and McLuhan. Instead, I delve into Kozel, Varela, Grosz,
Massumi, Idhe and others who explore presence, embodiment, the senses and technology.
I also have an interest in the ideas of a few pragmatist philosophers, such as Richard Shusterman,
John Dewey and Mikhail Bakhtin, who influenced McCarthy and Wright. In their Technology as
Experience (2004), McCarthy and Wright deal with the nature of human experience, the body and
technology’s effect on people in various working scenarios. They argue that designers need to consider
the emotional, intellectual and sensory dimensions of a user’s interactions with technology, arguing that
people do not just use technology, they live it. Shusterman is compelling as he deals with aesthetics,
performance and liveness in his book Performing Live: Aesthetic Alternatives for the Ends of Art (2000).
He positions performing live more in terms of the reception, experience and aesthetics for the
32 The Stanford Dictionary of Philosophy (First published Sun Nov 16, 2003; substantive revision Mon Jul 28, 2008), available at: http://plato.stanford.edu/entries/phenomenology/#4 (Accessed May 4th, 2010).
38
audience, rather than against the mediated (Auslander, 1999) or the supremacy of the ephemeral
(Phelan, 1993). He also tackles the body and somatics in his recent Body Consciousness: A Philosophy
of Mindfulness and Somaesthetics (2008), from the point of view of body image or body consciousness in
modern society. To do so, he looks at the philosophies of Dewey, Merleau-Ponty, de Beauvoir, Wittgenstein and Foucault, and critiques each, making a new case for his own soma-aesthetics approach.
He posits that we need a more reflective sense of body-awareness, which can result from self-cultivation,
focused on experiential embodiment, as opposed to the current societal ‘body beautiful’ obsession.
Other philosophers reviewed include Keith Ansell Pearson and Pierre Lévy on virtuality and the virtual.
Ansell Pearson, in Philosophy and the Adventures of the Virtual (2002), focuses on Bergson’s theories
contrasted with Deleuze’s interpretations of them, and his own philosophy on the virtual, time,
perception, memory and being in modern contexts, in which he argues for the importance of rethinking
these concepts in light of modern concerns. In Becoming Virtual: Reality in the Digital Age (1998), Pierre
Lévy makes a philosophical reinterpretation of ‘virtuality’, and addresses concepts of the ‘real’ versus
the ‘virtual’, the ‘actual’ versus the ‘possible’, and how the body, text, economics and culture are
virtualised. Lévy also explains what his version of the virtual means philosophically and practically to
modern society in terms of the Internet and networked technology. He puts emphasis on this ‘collective
intelligence’ and its positive potential for freeing social activity, by enhancing knowledge production and
dissemination. Idhe was a fantastic revelation for rethinking many of these aforementioned
philosophers, while contextualising them in terms of theorising the body and embodiment within current
technology use. He focuses on the effects of cyberspace exploration on our lived experience, on what
it means to be embodied when using technology, and on the role of our bodies in virtual reality
contexts. This will be discussed further in Chapter 8 on the topic of technological embodiment. 1.8/ ‘connective tissue’
In placing my professional work, I have always been torn between three domains: the intellectual and
conceptual, the social, and the field of art/performance. To reconcile these, I have tended to create
work that incorporates all three; MINDtouch is no exception. The glue that adheres these three areas
in MINDtouch is the exploration of intellectual concepts through a practical, collaborative performance
video project. To connect the different domains covered in this research, there need to be connective
‘threads’. MINDtouch research has been about connection: connecting ideas, experiences, the body
and mind, technology and art, contrasting concepts, databases with digital and mobile networks, video
with performance, performance with mobile phones, body data with streaming video, dreams with
telepathy, telepathy with networks, etc, but most of all, people with each other in new and unique
ways.
39
Presence, liveness and telepathy are connected and interwoven concepts (see mind-maps of the
concepts in Part II in the theory section), as they are each about sensing others: in the room or across
distance, in near-to-real-time. The concern is with sensing at a distance and how it can be done
differently through our technologies to help us exchange more familiar experiences like vivid dreams
and the preconscious, emotive or visceral sensations. Such experiences are hard to otherwise express
or share in their true form, except by some who experience genuinely telepathic exchanges. In Part II, I
unpack these conceptually and philosophically at more length and show the connections I see. The
diagram below shows the relationships between the topic areas covered in this exegesis and how I see
them interrelated.
Figure 3 © 2010 C. Baker – Diagram of thesis knowledge domains
Participatory performance is one approach used by artists as a method of engaging the public in
exploring both technology and the ‘real world’ simultaneously, in a practice called ‘mixed reality’,
‘locative media’ or ‘pervasive gaming’ [33]. Participatory performance is used here as a way to enable
people to explore embodiment in technology, and to assist participants to sense presence over
distance differently, through the technology. So this is a means to explore high-level meta-concepts
through practice. The connection is: embodying specific instances of mobile and wearable technology.
Why mobile technology? Partly due to its powerful new image and video capabilities, which inspire
more creative and playful potential, enabling participation and mobility in a way that a computer cannot.
But primarily its potential is in its closeness to the bodies of users, and its ability to go anywhere with
them. So, the effort of learning of the new technology is diminished, unlike that of other software or
hardware options for social networked interaction, such as Second Life. As a performer and video
artist, I am fully sold on technology’s power and have been anxiously waiting to work with mobile
videophones in performance since the mid-1990s. I have worked with people for years, in numerous
capacities, and been engaged in trying to empower people in various ways. This project seemed a
natural approach to engage people in performative and creative activities, using their most closely held
technological objects to play, collaborate and sense each other over distance.
33 These practices will be explained more in Chapter 6.
presence, liveness and telepathy communication over distance
participatory performance
activities
mobile and wearable
technologies (+networks)
mobile and
communication over distance
mobile and
technologies participatory
40
Wearable biosensing devices were incorporated, partly due to the funding imperative for the research,
based on prior experience with using them (my masters project and whisper[s] in 2002–2005).
Therefore, it was an intriguing opportunity to connect participants in additional embodied ways, turning
the live connection into a more physicalised connection. It was also an imperative to connect remote
audience/participants and attempt to enable them each to sense or ‘feel’ the other live participants in a
more tangible way. The concept was that they would sense this by sending their breath data, etc,
through the network to ‘touch’ and ‘breathe’ life into the mobile video clips, expressing the feeling of
breathing through the visual. This would then create a full feedback loop of body-to-body connection
sent through digital translation, with networks as invisible threads connecting people – a digitally
embodied touch and exchange. The video is connected as a way to translate bodily and emotional
sensations visually, but it also acts as a simulation of abstract, embodied dreams, sent and
intermingled with others’ dreams. This simulation is meant as a mechanism to express and explore
what it would be like to share and play with your own and another’s dream sensations –
communicating this emotional and visual state [34] over distance. The three domains of the digital, the
bodily and ephemeral sub-consciousness (dream state) are connected by threads of philosophical
concepts, manifested through the practical exploration and activities of the project. This is based on
some initial assumptions, as well as discoveries made along the way. These discoveries neither prove
nor disprove my assumptions, but imply new and different possibilities and outcomes that could not
have been known without this exploratory process.
1.8.1/ exegesis/thesis structure
This introductory chapter sets the stage for the remainder of this dissertation, giving an overview of the
project, the methods, the theory, the issues and concerns encountered, and the approach taken to
achieving the original contributions to knowledge that the thesis as a whole seeks to make.
Part I : Pract ice, provides an overview of methodological, philosophical and actual approaches to the
practice. This is followed by an overview of the artistic modalities used in the major MINDtouch case
study, including modalities of: participatory performance, live cinema, mobile media, telematics,
generative and database cinema and network art. This section discusses key inspirational projects and
the work of exceptional artists from the domain of media practice, who engage in art works and
performances that involve digital media or emerging technology, including mobile media, wearable
technology, interactive video installation, VJing/ live cinema, generative cinema, participatory media
performance, etc. The works and artists selected and referenced here can obviously only provide a 34 The ‘dream state’ as defined here refers to a state of consciousness: […] that judging by the overall electrical activity of the brain, dreams are a kind of conscious state […] (Baars, 1997: 108). In an extended search of relevant sources in reference material and psychology literature review sections, there emerged very little of value in the way of a set of specific definitions of ‘dreams’ except a semi-aware state of consciousness featuring a series of images, thoughts and sensations in one’s mind (New Oxford American Dictionary, 2005: online www.oup.com/us/noad) and rapid eye movement.
41
sample of the larger field, since there is not space within the scope of one thesis to analyse the work of
the full range of artists in these large and overlapping fields.
Part I I : Theory into Praxis, dives into a deeper discussion of the meta-level conceptual thinking
behind the research, focusing on theories on presence, embodiment, liveness, and telepathy over
distance. The theories and definitions of these key terms are extended beyond those outlined in Part I.
Their relationship to each other and to the practices of grounding experiences, from and within the
practical work, are also discussed with reference to the MINDtouch case study. This part makes
extensive reference to Appendix I, where the MINDtouch project is included in summary form. This
section is also where the case-study data shows an active relationship between the technical and
performative processes in the development, planning and performance, as well as detailing the
outcomes based on participant feedback about their experiences of the workshops and performance
events in which they were involved.
Part I I I : Conclusions & references for further study, offers a summary of findings and a list of
suggested areas for further research beyond the scope of the PhD.
Part IV: Appendices
(not for examination, but for elucidation and background information for the examiners)
42
Part I: mobile media practice in critical context
43
2/ hybrid methodologies of practice
As a hybrid performance project, MINDtouch has drawn heavily from the full range of media
disciplines and practices in which I have engaged as an action researcher: video, media art, modern
dance, music and media curation/production. Personal experience, creative practices, professional
work, studies and training, combined with current activities and research, have been integrated for this
PhD project. MINDtouch project development and research has straddled and combined
methodologies such as phenomenological techniques using lived experience observations and the
writing of journals (or ‘journaling’) with more objective ethnographic tools, like audience surveys and
participant interviews. However, the overarching approach throughout the research has been an
intuitive, performance-based strategy seen as a form of Art-as-Research.
This chapter describes the philosophical and methodological approach taken within the MINDtouch
research project as the main exemplar of the praxis for this thesis, as manifested in a mobile media
performance event. It combines methods to create the practical participatory activities, in order to
study presence and liveness that is embodied in the technology through the network.
2.1/ art-as-research
Throughout the process of writing this thesis, the methodological engagement has evolved and shifted
focus to some extent, steering away from HCI (Human–Computer Interfaces) or Design Theory
approaches [35] over time, and beginning to draw towards dance and performance methods and
techniques. Art-as-Research (AAR) is the most encompassing, flexible, and therefore appropriate
research approach for this work and is thus one of the primary methodological frameworks employed,
despite being controversial [36] in that its status as a philosophically consistent approach has yet to
gain full acceptance within the academy (in part because it combines methods from so many different
disciplines). AAR [37] is used and considered by artists as a methodology, although some more
staunch academic methodology purists might disagree. The AAR method as it has been understood
and applied to this thesis is outlined in more detail below. Briefly here, it can best be defined as an
iterative approach that defines itself as it evolves in relation to a specific transdisciplinary practice, and
that can be applied as a methodology for research in the sense that it provides a philosophy of itself as
a research method within its application. In other words, AAR can be seen as the methodology of the
artistic practice of philosophy (and the philosophy of practice), which incorporates, or works as an 35 Which tend to use approaches like empirical methods for computing design or use-case scenarios for products in product design, discussed more later in the chapter.
36 Supervisors have suggested this.
37 The debates around the different terms for artists using practice as the source of their research is addressed in the introduction chapter and will not be further addressed here. I focus my attention here more on the reasons for using it instead.
44
umbrella, borrowing and employing multiple methods, tools and techniques from other methodologies.
In terms of AAR, practice can be defined as the act of making art, performance or other practical,
hands-on, embodied creative work (coding, dance, architecture, music, design, etc), i.e. making things
(digital or physical). The research aspect involves reading, studying, exploring, examining, observing
and analysing others’ artistic approaches and then synthesising – writing about them while applying the
knowledge to one’s own practice and/or thinking – while developing one’s own manifestations of
practice.
In the book Practice As Research: Approaches to Creative Enquiry, editors Barrett and Bolt:
[…] propose that artistic practice be viewed as the production of knowledge or philosophy in action [...] exploration of artistic research demonstrates that knowledge is derived from doing and the senses [... and] generative enquiry that draws on subjective, interdisciplinary and emergent methodologies. (Barrett and Bolt, 2007: 1)
Common to art and performance practitioners and researchers, the AAR process can be summarised as
follows: search for inspiration, brainstorm, create a prototype, improvise, repeat, improve, practise,
gather more skills, collaborate, practise, demonstrate or perform. AAR is often a personal, implicit,
inferred, experiential and largely unconscious process, deriving from training, practice and a natural
creative sensitivity combined. This step-by-step approach often also includes collaboration, feedback,
presentation, review, and discussion similar to a standard design methodology. (Burgess, 2002: 4)
The processes of art and performance practices often involve trial and error. Barrett and Bolt would
add to this by saying:
As a result of this reflexive process, methodologies in artistic research are necessarily emergent and subject to repeated adjustment, rather than remaining fixed throughout the process of enquiry. (Barrett and Bolt, 2007: 6)
The diverse creative practices and explorations employed over the years in my artistic work has
increasingly pushed me to be inquisitive, develop more skills and do research into new ways of working
within new media. Barrett and Bolt have deeply influenced my thinking, with their notion that:
[…] innovation is derived from methods that cannot always be pre-determined, and outcomes of artistic research are necessarily unpredictable. (Barrett and Bolt, 2007: 3, my emphasis)
Estelle Barrett’s chapter Foucault's 'What is an Author': Towards a Critical Discourse of Practice as
Research, goes further in defining the artistic process for knowledge production:
In the creative arts, the outcomes that emerge from an alternative logic of practice are not always easy to articulate and it can be difficult to discuss the work objectively given the intrinsically emotional and subjective dimensions of the artistic process. (Barrett and Bolt, 2007: 135)
45
Barrett and Bolt suggest that the innovation and critical potential of the research is within the artist, in
both the personally situated knowledge they bring to the work, and the techniques chosen to
externalise that knowledge in their own unique practice, within philosophical, social and cultural
contexts for wider application and engagement. Carter adds to this in the chapter ‘Material Thinking’:
[…] a dialogic relationship between studio practice and the writing of the creative arts exegesis is crucial to articulating and harnessing studio methodologies for further application beyond the field of creative arts […] is validated alongside other more traditional forms for research derived essentially from the scientific method […] Embodied vision […] links experience, practice and theory to produce situated knowledge, knowledge that operates in relation to established knowledge and thus has the capacity to extend or alter what is known. (Carter in Barrett, 2004: 143–145, my emphasis)
This last point highlights this process of engaging the theory through the practice. My process in linking
the embodied vision with ‘experience, practice and theory to produce situated knowledge’, has involved
taking the experience of facilitation and organising people in previous work and my social life, in tandem with
my training in yoga and meditation, into the devising of the performative activities. Intuitive and embedded
knowledge won from experience was accessed in this project while designing activities that would ensure
participants would have a meaningful experience, deeply connecting with their inner world, all the while
feeling safe to do so. Carter’s point and many other artists herein have resonance for this research and their
ideas and experiences bulwark much of my thought processes and aspects of my practice. To punctuate
this, Dianne Reid also adds from her chapter ‘Cutting Choreography: Back and Forth Between 12
Stages and 27 Seconds’:
[…] practice makes tangible the theoretical [; …] practice as a response to lived experience, the temporal, the personal, and the collaborative – reveal[s] how new subject matter requires new forms of expression and representation. (Reid, in Barrett and Bolt, 2007: 10)
Carter’s focus on artistic research being conducted like scientific experimentation, especially in media
art or technologically manifested art, directly relates to my experience. I found that working with the
technical aspects in the work, as well as the performances themselves, which involved making or
planning an element, then testing it and iterating it, can be said to be borrowed from the scientific
process of experimentation. Marilyn Burgess, formerly of the Media Arts division of the Canada Council
for the Arts, which promotes media artists in Canada, stated that media artists usually work beyond
traditional art practice methods, and incorporate ‘lateral thinking, re-purposing and invention’,
borrowing from sciences to measure their outcomes:
[…] such as ‘collaborativeness, algorithmic thinking, interface innovations, philosophical commentary, adaptability or robustness, humour, critical commentary and scalability’, which point to the specificity and radically different artistic paradigm of new media arts … The artist's role is to challenge, critique and investigate existing conventions. (Burgess, 2002: 2)
It would appear from Burgess and others here that a hybrid practice-based approach is standard in
media arts and performance practices. Thus, the AAR approach was ‘intuitively’ developed in
46
MINDtouch : not always conscious, but working according to the needs and explorations emanating
from the work-in-progress, and acted upon as the need arises, rather than necessarily according to a
step-by-step path.
When working with materials, Barrett suggests:
Materials methods and theoretical ideas and paradigms may be viewed as the apparatuses, or procedures of production from which the research design emerges [...] forged in relation to established or antecedent methods and ideas. (2007: 137–138)
Barrett additionally asserts that this approach,
[…] explores the complex interrelationship that exists between artistic research and other research and scholarly paradigms [...] Acknowledging the emergent and subjective dimensions of artistic research, Stewart describes this method as a process of continuous discovery, correspondence contradictions, intuition, surprise and serendipity. (Barrett on Stewart, 2007: 12)
AAR theorists and artists have been studying and developing more standardised and rigorous methods
as more undertake PhD studies (in the UK, Europe and North America). Many do this to validate their
work within an academic arena, and to complement their work in the art world. One researcher, Anne
Douglas, in a collaborative article with Karen Scopa and Carole Gray, outlines three different Art-as-
Research methods she has encountered within the Fine Arts: 1) personal research, 2) research as
critical practice, and 3) formal research (Douglas et al., 2000: 4). They say that personal research is
generally intimate, private, and often unpublished, work that an individual artist develops in a piece or
project. It is research that involves professional practices of sketching, storyboarding, creating,
exhibiting and publishing interviews. Research as critical practice, however, involves artists finding
innovative ways of working, attracting new audiences, and challenging conventions of exhibiting and
approaches to creativity. It […] is both embodied in individual examples of practice and carried across
professional platforms for debate around practice, such as artist’s talks, discussion platforms and
publications […] (2000: 4). Finally, ‘formal research’ is conducted within academic environments,
helping artists to gain research skills, contributing to knowledge in the field through standard academic
processes that are formalised, taught and defended, as well as promoting an understanding of how
and why things are done in art practice (2000: 5).
MINDtouch ’s creative approach lies somewhere between research as critical practice and formal
research. The project aims to innovate within a relatively new performance domain, using
interdisciplinary methods to understand and negotiate a relationship with the ‘audience’, as participant
performers, within and outside the conventions of exhibiting and performing. So, within Douglas’
definition of formal research, this work is a relative, heuristic research. The media performance
contributes to a shared body of knowledge, through a recognised process towards a degree, which
provides additional academic skills and credentials, but also toward further practice, and for the benefit
of other artist researchers. Sullivan adds, in Art Practice As Research: Inquiry in the Visual Arts, that
47
visual arts knowingly situates the imaginative and intellectual processes as artists make use of an […]
ongoing dialogue between, within, and around the self, artworks, viewers, and settings, where each is
used to help create new understandings. (Sullivan, 2005: 191)
Evaluating the approaches of many of these artists and practitioners has helped to clarify my own
practice, approach to the work, thinking and research, by validating my use of an embodied and
embedded knowledge to guide the process – as opposed to following a formulaic methodology for the
sake of it – and letting the work and experimentation with it guide next steps. This has meant being
sensitive and open to taking new directions from the work and improvising to the next level. As such,
even though it started with phenomenological and ethnographic guidance, tools from other
methodologies have been additionally applied to help balance it out. The participatory and
improvisational nature of activities within the events echoes the process of the project evolving
organically to its outcome, and this openness was necessary to deal with all the different
interdependent pieces of the work. If one aspect did not work I had to improvise and trial the next
solution to make sure all the other interlocking aspects would continue to work as well. Therefore, one
rigid, predetermined method would not have been effective. The process has been a balance of
working with technology, people and performances, each with their own systems, ecologies and
needs.
In his chapter ‘Correspondences Between Practices’, Stephen Goddard describes how to approach
research, writing: Rather than relying only on the written component of an exegesis to demonstrate a
reflective process, it can also be reflexively performed with the practice itself […] (2007: 117). While it
could have been intriguing to ‘write’ some of the MINDtouch analysis using the mobile phone video
itself, as a self-reflexive mode of further exploring theory, there was already enough going on in this
practice, and this idea can be saved for a future project. Goddard further discusses the writing process
in art-based research:
As a methodological strategy, it was useful to integrate the narrative of the research process into both the practice and the exegesis [...] The overall narrative of the research process includes the story of the practice–exegesis relationship, and the ways in which both the practice and the exegesis reflect upon the chronology of the research process. (Goddard in Barrett and Bolt, 2007: 118, my emphasis)
I have thought of the ‘narrative’ as the phenomenology of process. This narrative is critical to the work as it
unfolds; it is the research as much as the more practical, hands-on aspects. I have been writing throughout
the research and the writing has responded to the practice and been the practice as much as the work
itself. The writing, especially through journaling (discussed more in the next phenomenology section), has
helped to clarify the practice throughout this research.
Intriguingly, Goddard explores the creative practice as part of the methodology (2007: 119). He muses on
his process of finding his method within the practice and the exegesis itself as it evolved:
[…] both the video production and the written exegesis were two parts of the same practice. I
48
was attempting to trace the ways in which writing and video technologies mediated and recorded my memories, stories, annotations and analysis. (Goddard in Barrett, 2007: 118)
Goddard’s point that video and writing worked to mediate his memories, annotation and analysis made
me realise that this had been my experience as well. The approach taken for MINDtouch has likewise
been very subjective and has allowed for a change in approach in cases where the process appears
unintuitive or untrue to the work, or a mere exercise in objectivity for the sake of it. Yet the work is not
only in the writing about it, but exists in the experience, and is embodied in the artefacts from it by
participants. These video artefacts hold the experience of not only what is within the frame but also for
me they hold some essence of the participant and my memory of them. They tell me a little about each
of them and help me to understand their experience from my own phenomenological perspective. So,
unlike Goddard, the video in my research did not record my memories or stories, but instead the
experiences and sensations of the participants. Since they conjure up memories and stories from the
workshops and events and what is not in the frame through the viewing and analysis of them, I am able
to write and annotate in a similar fashion to Goddard.
The viewpoint taken here is that one method an artist has is to create their own philosophical or
conceptual expression of methodology, directly informed and/or led by her/his practice. In this way, the
more an artist uses their work as their research, with the exegesis to support it, the more the artwork,
artefact or performance, in its intrinsic artistic form alone, is the contribution to knowledge. Thus, the
methods and modes of artistic enquiry used to make the work become accepted in the academy.
Since my programme promotes practice-based research for the artists and performers, as do many art
and design programmes in the UK, this is happening more and more. However, my programme as a
media institute also sits within a computing school, as do many, so a tension might be perceived. As
such, convincing the academy that within interdisciplinary programmes such as this, practice is a valid
and critical mode of knowledge creation can still be complex. Yet new media art has been one catalyst
among other interdisciplinary art-science research modes, from which artists have wished to pursue
doctoral research and contribute their artistic knowledge, as research, to the scientific community and
the world at large in meaningful ways. However, there continues to be a question as to why it is not yet
accepted in the same way as the discipline of music, or computing for that matter, which allows for the
examination of students through the work itself. Within other forms of art and performance, this form is
not always an option, and theory alone is the mode of enquiry. What this issue indicates is that the
artist-researcher may wish to create their own version of a musical ‘score’, code or map of the work,
available for their peers, examiners and the academic community to experience and to recreate the
work as it is intended [38]. Also, artists must still have the written exegesis available to accompany the
work to further tease out their thinking, the conceptual research, as well as the project development
processes, to ground the work as knowledge creation. Yet, it should also be acknowledged that
conducting research is a practice itself; as such, artists need to contribute to the foundations of the
examination design and process for their higher academic designation. The more we do so (even if
38 I hope to have the opportunity to stage a performance for my examiners, to practice this.
49
logistically painful to organise at times), the more artistic knowledge can become integrated into the
academy, on its own merits, in the same way that a practical engineering, business, law or medical
practice is. There is a slow shift in this direction in western countries, and potentially elsewhere.
A first-person voice is often used in art-based research. The immediate, intuitive approach to writing by
artists commonly rejects the use of a removed, third-person voice characteristic of traditional academic
contexts, since the work studied is lived by the artist in the first person. In the same way, artists often
choose a phenomenological approach in their academic art research. The entrenched academic
traditions and constraints of disassociation and so-called ‘objectivity’ imposed on research practice, to
lend scientific credibility and reproducibility of experiments, feel foreign and antithetical to art practice,
where ideas originate from the artist and there is little desire for reproducibility of their art. There is an
evident paradox or tension here between scientific and artistic cultures of knowledge.
Resistance can also surface when artists are asked to contextualise their practice, ideas and
experience of process within those of other thinkers in the discourse within the research community,
especially when asked to make an original contribution to knowledge. No matter how similar or widely
discussed a more recognised and knowledgeable artist’s or theorist’s ideas are, an artist needs to feel
that they are true to the practice and their own experience first. To be clear, this is not a resistance to
explore others’ ideas or learn from them – on the contrary, since these theorists do inspire, influence
and motivate personal notions and approaches to the work. However, it can feel disingenuous when,
upon having a thought, feeling, sensation or experience in life or in practice – which can often become
the motivation or inspiration for the work or theory – one has to credit someone else for that
experience, thought or sensation, as if they have copyright on the experience. I try to experience my
own encounters with ideas through lived experience first, as pure sensation and expression as it
surfaces, and then express this in the writing and in the work. It is then possible to contextualise with
others who have had a similar sensation, thought or experience. I argue that an individual experience in
one’s own life and art practice still has its own dimensions, lived insight and nuance differing to that of
others’ work and ideas, which can be added to and contextualised amongst those other, more
prominent ideas. Regarding this practice of crediting others, Barrett makes a similar argument:
An innovative dimension of this subjective approach to research lies in its capacity to bring into view, particularities of lived experience that reflect alternative realities that are either marginalised or not yet recognised in established theory and practice. (2007: 143)
Individual and contextualised academic and artistic ideas are explored in this research process through
reflective ‘musings’, as mentioned in the introduction. Sometimes these are based on new connections
or realisations from the practice itself, and sometimes they are inspired by others. The process of
contextualising my own ideas, creative visions and unique thoughts through others does indeed help to
find other approaches or like minds to demonstrate that my thinking is shared. We cannot help but be
influenced; yet artistic work is often about expressing one’s own unique experience or interpretation of
the world, to reflect back to others and to contribute and share with others. Finding the right mix is a
50
fine balance. Artists and performers undertake research in order to explore their own work and specific
concerns, while adding to their own process, understanding and skills, and finding their place in the
field and discourse. Learning is primarily gained through the experience of other creatives and thinkers
in their work and thought, and synthesised, written about and applied as relevant to one’s own practice
and/or thinking. Much of my research has been inspired by the insights of other artists and theorists,
found by reading, studying, exploring, examining, witnessing and analysing in order to find a path of my
own.
During this research process, my personal life and experiences have also greatly influenced my
thinking, experience and understanding of researching ‘mobile presence’, influencing the choices and
approaches to making the work. In this vein, Reid confessed that her approach was even more
personal in her dance work, resulting in a blur between personal and academic. She reveals:
I have drawn on autobiographical incidents in which the inner self and the outer body collide, where a point on or movement of my body references and reveals a specific emotional experience or intellectual realisation. [...] extrapolated from stories connected to specific scars or points on my body associated with accidents or trauma. (Reid in Barrett and Bolt, 2007: 58)
Through my journaling I was reflecting and revealing specific emotional experiences as Reid did,
triggered from my own mobile communication exchanges and play with the device, as intellectual
realisations bubbled through, connecting the personal to the conceptual, as can be seen in Part II. This
was similar to Goddard’s experience, as he has noted on his own work:
My research was situated within and across the overlapping fields of autobiographical writing and subjective video practices. (Goddard in Barrett, 2007: 117)
This briefly inspired me to consider writing sections of this exegesis as a diary. However, this has been
abandoned in favour of including the personal/poetic observations of lived experience and musings of
the mobile phone, virtual and actual presence, embodiment and telepathy. Many entries in my research
journal and online wiki were called ‘MUSINGS’ and have been included in this writing. These are
integrated as an example of my phenomenological practice, within the theory exposition in Chapters 8
and 9.
Within her book Barrett charts many artistic approaches as a guide for artist researchers. One section
states:
The researcher traces the genesis of ideas in his/her own works as well as the works/ideas of others; compares them and maps the way they inter-relate; examines how earlier work has influenced development of current work; identifies gap/contribution to knowledge/discourse made in the works. (Barrett, 2007: 139, my emphasis) [39]
39 I have been advised not to do this but I agree here that this is a critical aspect to understanding or contextualising the genesis of current work.
51
As we can see, there are a great number of instances where artists allow personal experience to filter
into the research process, to inform and shape the work (Barrett, 2007: 68). This approach appears to
be essentially a phenomenological methodology, since one might argue that all research is influenced
by the researcher’s personal perspective, experience and embodied skills and knowledge; this is where
artists start. It could be argued that it is more honest to admit that this personal embedded experience
is key to the work’s authenticity. Barrett adds in her ‘Guide to Art as Research Methods’ chart:
The researcher discuss[es] the work in relation to: lived experience, other works; application of results obtained; contribution to discourse; new/transgressive possibilities; obstacles encountered and the remaining problems to be addressed in future research. (2007: 139)
I have tried to follow this guide directly and indirectly and found it quite helpful in this process. Barrett’s
final point has informed how I have approached the obstacles and remaining pieces of the work: to be
addressed in future research.
To conclude this section, Kolb suggests that AAR is a problem-based learning approach, relevant to
participatory learning as it focuses on knowledge generated through action and reflection. Thus, the
experiential approach put forth by Kolb (2004) […] starts from one’s own lived experiences and personal
experiences […] (Barrett, 2007: 5) through action and then reflection on the action. What is relevant
here is that AAR can also be seen as a form of Action-based Research (ABR), used in education and
other forms of community research. In ABR activities and approaches to teaching – or making changes
and improvements in, for and with communities – are planned, experimented with in the classroom or in
the intended community setting to be studied. The observations and outcomes are then analysed and
evaluated, and the activities are modified, refined and reworked for trial again. This participatory
approach works toward finding the best method to inspire learning by students or for the betterment of
the group or community (Reason and Bradbury, 2002). In some ways this is not dissimilar to
programming or HCI design, or even a scientific approach in its experimental ‘trial-and-error’ approach.
This is the approach I have taken in MINDtouch .
While each of the voices here have slightly different nuances to their interpretations of Art-as-Research
as a methodological approach, they all essentially state that valuable knowledge can be derived from
practice. Ranging from Australian, Canadian and UK academic contexts, all have similar approaches
and are influential to my own experience of using practice as a source of knowledge creation.
2.2/ phenomenology
This research process has been explored through a phenomenological lens, not only methodologically,
but as a practice and process of living and thinking; putting theory into daily practice. As such,
phenomenology as a method and approach has been explored as both an initial guiding approach and
a grounding philosophy for practice. This approach is appropriate since media performance can be
very intimate in its creation processes. The body-centred approach used in this project, specifically the
52
use of biofeedback devices and the facilitation of self-attention with participants, has required a more
intimate, personal, first-person treatment (as discussed in Appendix I). It is also a good fit with a
Merleau-Pontian [40] perspective on perception and lived experience. Thus phenomenology, as a first-
person methodology, is encompassed by the Art-as-Research approach discussed above.
In addition to the definitions provided in the introduction, Francisco Varela and Jonathon Shear state
that phenomenology is the study of, and the method of studying, lived, first-person experience that is
not about ‘things-out-there’, but rather one’s mental ‘contents-in-here’. They state that:
[…] the level of the user of one’s own cognitions, of intentions and doings, in everyday practices … the realm of experience is essential for human activity and life involving the use of one’s own mind […] the experiential domain can be explored […] (1999: 3)
This has been a good rationale for using a first-person perspective in my conceptual explorations
during the research. Varela and Shear’s phenomenological approach and tools of phenomenology
include: introspection; attention during defined tasks using verbal accounts; meditation or sustained
attention, uncontrived awareness and suspension of mental activity (1999: 5). I have employed these
tools and approaches when thinking through the design and application of conceptual notions of
presence and technological embodiment within the participatory activities in the workshops and events.
Many of the concepts and realisations employed in the project come from the lived experience of using
the mobile and digital communication technology with friends and family, as mentioned above. So I
was introspective and paid attention during these activities, meditating on or musing on these
experiences to draw further from them for the work.
Martha Ladly has another interpretation of Heidegger’s phenomenological methods in […] the self
showing in itself […] (Heidegger, 1996 in Ladly, 2007:142) and she states that,
[…] it is the active process we undertake when we assign meaning to our experiences [which] cannot be separated from reality; it is an active process and as such, a creative act. Communication – one to self; one to one; one to many; and many to many. (2007: 142)
This notion of assigning meaning to our experiences is one employed in my research on multiple levels:
in understanding the overall creative processes, but also in the specific activities when working with the
devices, the technical collaborators, and with participants. These positions manifested in my work in
terms of: one to self, in my phenomenological process of creating and reflecting the work throughout it;
one to one, in how I communicated with distance family and friends through mobile and digital means
daily, as well as in the more formal communication with each of the technical collaborators; one to
many, in how I related to the participants in the workshops; and many to many, in how the work was
made so that many live and remote participants could communicate.
40 I will work from Merleau-Ponty more directly in upcoming writing and the final methodology chapter to be able to refer to his theories more directly.
53
Ladly also reinterprets Herbert Speigelberg’s ‘elaborate seven-step method’ from The
Phenomenological, stating that his steps are to:
[…] formulate a phenomenological description using phenomenological intuition, […] make a phenomenological reduction […] determine […] which parts of the description are essential. […] isolate the object of consciousness, the thing, situation, emotion or person that constitutes the experience […] based directly on the experience, rather than on a conception of what the experience may be like […] produce a phenomenological interpretation, an attempt to signify meaning, using hermeneutic analysis. (2007: 142)
Some of these, and Susan Kozel’s step-by-step methods, were employed intuitively in MINDtouch .
Kozel’s methods of phenomenology were later discovered through her book Closer, which is a
rethinking (reconsideration) of Merleau-Ponty’s phenomenology. In it she provides a set of
phenomenological methods for body-based practices, performance, and for technology art practices,
relevant to my practice. This set of Merleau-Ponty-influenced tools is constructed and interpreted
through her performance and technology practice, and her practice as a philosopher. In it she reveals
that one of the objectives in her reinterpretation or framing of phenomenology is to devise a means for
creative practitioners and performers to use phenomenology to reflect and write on their practices.
Kozel’s list of processes or tools [41] to guide practitioners include that we:
[...] loosen our rationalist structures of meaning sufficiently to permit qualities associated with the pre-rational, such as ambiguity of meaning, fluidity of existential and conceptual structures, scope for entirely new thought, perceptions, including contradictions, reversals of meaning, or paradoxes. (2007: 18–19)
This perspective gives the artistic researcher more freedom to allow uncensored perceptions and
encounters with the work, and intuitive responses to it, express themselves, either directly or after
further reflection.
Kozel’s guidance as a supervisor during much of this research must be acknowledged: her influence is
very palpable. Closer exposed and provided a deeper examination of her poetic exploration of her
phenomenological, practice-based approach. While I have my own approach from personal and
professional experience and background, Kozel’s perspective has guided the refinement of my thinking
and approach to the processes of practice. Directly working with Kozel on other projects also
influenced my practice, even before reading Closer. This work with Kozel has unearthed an intuitive
phenomenological approach or sense in establishing a methodology from the felt experience and
process of making this work. Thus, phenomenology became a foundational methodology [42]. Working
through the practice and with Kozel’s encouragement, a hybrid approach was developed and realised
in the work. Therefore, her Merleau-Pontian-infused phenomenology flavours the thinking, intuition and
writing in my own process, as can be seen in this chapter and in Part II on embodiment.
41 I have quoted the bulk of these in my methods essay and chapter, so I will not repeat them here. 42 Closer was not yet published when I started the PhD, but was purchased much later in the process and has been used to help me articulate my process through Kozel’s philosophical clarity and breadth of experience.
54
Mindfulness meditation techniques are indirectly incorporated into the work from prior studies of them.
Dance, yoga practice and meditation training have also inspired my approach to facilitating participants
in the workshops and performances. Kozel’s methodology and methods are akin to Buddhist
Mindfulness practices. [43] An example of her steps is quoted here:
Take your attention into this very moment. Suspend the main flow of thought. Call your attention to your body and what it is experiencing (are you short of breath, is your back hurting, are you hungry?). Witness what you see, hear, and touch, how space feels, and temperature, and how the inside of your body feels in relation to the outside. What thoughts enter your mind once you suspend the main rational thrust? Register any seemingly trivial anxieties or thoughts but do not try to delve into their deeper significance at this moment. Let your mind wander and notice lateral associations. You sense-data retrieval depends on your context. Do what seems appropriate. Spend time getting in touch with your senses. (Kozel, 2007: 53–55) [44]
A similar guiding script was used in the mind-quieting exercises conducted with participants in the
workshops and shorter performance events for MINDtouch. Participants were asked to become
aware of and focus on their sensations and translate them into visual ‘low-focus’ [45] note-taking (see
Appendices for the script). To create a record of their sense retrieval and exploration, the activities for
participants were designed phenomenologically by revisiting, reiterating, revising and repeating each
activity for each participant. This process of reiterating, revising, and repeating each activity also
included observation of participations, an Action Research technique [46]. To uncover the most
effective way to shape and facilitate the activities and experiences for others, this iterative approach
was used. Next, the observations and experiences were compared with participants’ actual accounts.
However, participants' accounts only partly influenced how changes were made to the activities. Unlike
in Participatory Design [47], what is key here is the experience of observing participant responses to the
activities and process, listening to them, seeing them interact with the devices and interpret the
activities given to them, sensing the flow of the process, and seeing how well participants grasped the
instructions. These all determined how activities would be changed for each iteration. Each
performance staging was also based on the experience of the technological preparation, the ‘set up’
and ‘tear down’ of previous versions, making a significant difference in successive stagings or iterations
of the performance experience. In this way, my role as director/facilitator was responsive, first-person
and reflexive, but also leading development. So while each workshop and the overall process was
designed, the complete synthesis of how the methods and strategies framing the phenomenological
43 Pema Chodron, a prominent Buddhist teacher and writer, as well as others, has many books and teachings on Mindfulness that contain practical methods similar to Kozel’s that she uses with her students. Start Where You Are: A Guide to Compassionate Living (1994) is one Buddhist book that teaches Mindfulness meditation.
44 This very truncated version of the method gives the reader the thrust of it as is relevant to my work and perspective.
45 David Gelernter’s phrase, discovered through Kozel (2007: 52).
46 Action Research is not explored more deeply here, as it was not used as primary research methodology, but aspects of it resonate in this study and it is referenced, since there are many parallels to it in the MINDtouch methodological approach. As a teacher / lecturer I perhaps also intuitively use Action Research in all my participant-based work.
47 Participatory Design approaches were explored earlier in the process and found to be greatly different from mine. The project does not use PD, because in the design sense I am not designing an object or artefact. This is discussed further in the workshopping section in Appendix I.
55
perspective were evolving was only clear after reflection and analysis of a few workshops.
The impact of Kozel is also evident in the MINDtouch methodology through the performance
development process gleaned from lived experience; since collective bodies, networked art and other
embodied practices are used in both our work. Kozel begins by clarifying that she seeks:
[…] more than an integration of the body-machine duality, more than a creation of a continuum or spectrum with the body at one end and mind at the other, but […] an immanentist approach to each of the dualities [...] as enfoldings or entwinements. (2007: xvii)
This exegesis similarly argues for mind–body extension through embodiment in technology and
explores this through the phenomenological lens both of myself as artist and of the participants in the
project. The hybrid methodology of MINDtouch was first experienced intuitively, and first surfaced
through the development of the goals, expression and possible activities for the project. The intuitive
approach developed through a vision of how others might experience the interaction, as well as how
they would be encouraged to record their experiences during the events. Following this vision, a sketch
of the experience design for participants was created, and then enacted as a design process for
‘walking through’ the activities. The first step in this process was to note down these ideas like a script.
They were then tested, both in workshops and on my own. It was an AAR process of ‘trial and error’
(as Burgess suggested above) that was subsequently enacted by the participants, each with individual
and different responses to the work. I did not attempt to predetermine or influence their outcomes or
experiences, since each person comes with their own history, perspective and relationship with their
own and others' bodies, as well as expectations and interpretations of what might happen. They would
also explore the activities and use the device to interpret their mind's vision or visualisation of their
embodied experience of their sensations in varied and unique ways. This discussion frames the intuitive
process in the context of these methods and steps, rather than trying to suggest what they were
adhered to closely from the start.
Regarding an intuitive approach to writing from a phenomenological perspective of practice, Kozel
muses:
Writing from lived experience often amounts to writing without a clear methodological mandate, or demands the courage to assert that the methods are fluid and subjective. Paradigms are scraped together (defiantly, guilefully, playfully, intuitively) from philosophy, literature, the social sciences, physics. This bricolage or hybridisation is done in part to find a voice in the academy, but more important, to help the writer herself understand what it is that she is experiencing and to communicate these experiences. It is done to give voice to the shifting terrain of the writer's experience and to find others with whom this might resonate, and to help her understand her own experiences and express them in such a way that others might find it meaningful. This sometimes precarious process is sustained by the realization that reflection is not only a secondary process or a commentary on experience, but also the process of thinking that transforms the doing. (Kozel, 2007: 9, my emphasis)
56
This perspective gets to the heart of how this research has evolved, validating the hybrid approach to
the development of the practice. Kozel addresses phenomenological voice in writing of lived experience
and practice (research), by pointing out that Varela and Shear compare first-, second- and third-person
methodologies, stating that the content revealed is not different, but rather […] how the researcher is
'inserted into the network of social exchanges […] (2007: 57) and is validated makes the second-
person researcher an ‘empathic resonator’. This is because the researcher becomes sensitive to the
other person’s phrasing, body-language and expression. This is the role of a facilitator as well as of a
director and experience designer (discussed further in this chapter). In these roles, I became an
‘empathic resonator’ in responding to the participants.
In Closer, Kozel emphasises Merleau-Ponty’s insistence that a pre-reflective state is the starting point
to accessing and evaluating experience. In a phenomenological approach, prior to reflection (especially
when writing on lived experience as it is encountered), pre-reflection is necessary. In reference to the
pre-reflective state, Kozel clarifies Merleau-Ponty’s terms, useful in this research:
This [pre-reflective] belongingness with the world that preexists our conceptual engagement with it, but does not exist apart from it […] not just primordial but as mysterious: the body is 'a mysterious and expressive mode of belonging to the world through our perceptions, gestures, sexuality, and speech. It is through our bodies as living centres of intentionality ... that we choose the world and the world chooses us.' (Merleau-Ponty, 1986: 74, quoted by Kozel, 2007: 17, her emphasis)
This phenomenological intentionality of connecting to and engaging with the world through the body
and through reflection, shapes most encounters, whether understood and experienced as primordial,
primitive or mysterious. Nevertheless, Kozel, through her performance lens, reconfigures Merleau-
Ponty’s position in such a way that repositions my experiences in life, art and performance more
concisely and how I too try to engage through the body with intentionality and through reflection,
cultivating pre-reflection though meditation and in art practice.
This discussion illuminates another question to make a reflective practitioner pause and think: perhaps
it is not possible not to ‘violate’ or alter the pre-reflective state in the process of translation. One could
argue that culture, gender and language, along with other factors, alter and colour or filter pre-reflection
to enable us to work with and understand it in lived, daily contexts that we find ourselves in, as Kozel
and others suggest. As artists, we can find ways through the practice to manifest nearer to the original
pre-reflective state. When Kozel speaks of ‘opening to aesthetic experience’ in her introduction, she is
speaking of pre-reflection and hyper-reflection within phenomenological art practice. She maintains that
this is important as a means of working with bodies and technology – listening to the body pre-
reflectively and:
As we assess the artistic, social, and corporeal implications of developments in computational devices and ambient systems, we need to have a method and an intent to receive information from our own bodies and from the bodies of others. In giving a voice to diverse corporeal states we can overcome naive distinctions between matter and spirit, between
57
body and mind. The second reason for crafting a role for the pre-reflective is that it can open a way for understanding the deep entanglement between reflection and experience, between thinking and making, which is so important to many arts and sciences. (2007: 22–23, my emphasis)
This perspective clarifies and reconciles my own conceptual conflicts encountered regarding the
dualities of consciousness and corporeality that previously confounded my thinking and pre-reflection
on daily, lived experience. It also clarifies the concerns and philosophical notions explored in the work
as well as how to facilitate others in their experiences within technologically enabled performance
activities. Kozel makes a case for exploring the elastic nature of embodiment and consciousness, to
enable this pre-reflection, and to understand its paradoxical relationship with reflection:
[...] this philosophical impulse is like reaching beyond ourselves conceptually, perceptually, and existentially, but then returning to ourselves. (2007: 19)
She also proposes that a way to present an interpretation of another’s experience within the context of
one’s own practice is as a heterophenomenologist:
[…] based on two realizations: the first is that phenomenological philosophy provides the conceptual framework and methodological sketch for interpreting the experiences of others; the second is that sensibility and interpretive power come from the physical experience of the phenomenologist […] or heterophenomenologist [... which] precisely relies upon shared experience, or some degree of empathy. (2007: 58, my emphasis)
This concept of the heterophenomenologist interpretation helped frame the task of reviewing
participants’ post-workshop questionnaires and video interviews, and the video documentation of the
workshops. This focus was particularly important when reviewing another’s experiences, recognising
my role as researcher and my perspective with my observations. Thus what emerges is that
MINDtouch is semi-phenomenological. When determining how to analyse and interpret participant
data and observations to an ethnographic or heterophenomenological approach, Kozel suggests it:
[...] amounts to a slice of a shared knowledge, a morph of the lived experience of two people because the originator of the experience is trans-substantiated through the one who produces the phenomenological document [… and] that in an attempt to try to understand the experience of others, the fabric of one's perceptible, intelligent world comes through. (2007: 58–59, Kozel’s emphasis and my emphasis)
To understand the experience of others has been an important consideration in how to process and
interpret participants’ feedback and how to iterate the performance staging and activities, to
understand the effectiveness of the technological system, and to determine whether it enables the
experience of feltness, presence and liveness in the mobile network. Yet, as the researcher crossing
roles over to the director mode in the process, the approach to this work is then not entirely
phenomenological or even hetero-phenomenological, as I do not pretend to interpret or ‘trans-
substantiate’ (as Kozel calls it, 2007: 59) participants' experiences. Instead, my approach is more
analogous to how Kozel describes Rouhainen:
58
[…] she confesses that she never completely grasped what her interviewees meant. She writes of understanding the contents of the interview material only in a manner that is meaningful to her [...] (2007: 59)
This is a more authentic perspective for MINDtouch . The improvisational style that is used originates
from dance studies and is also commonly used in theatre. It involves creating and producing or
staging the events and workshops with a ‘trial-and-error’ or experimental approach, involving the
participants in creating the work.
Ladly contributes a contrasting point about phenomenologically interpreting others’
experiences:
[…] one must temporarily remove oneself from an ongoing synthesis of experience, in order to reflect, to create definition, and interpretation … the interpreter goes back and forth between experiencing an event or situation, and assigning meaning to the event […] (Ladly, 2007: 142)
This synthesis of experience by the interpreter as phenomenological art researcher was also the
approach taken in the analysis of participant feedback. Effort was made to be reflective using a first-
person process to evaluate participants’ feedback appropriately.
The phenomenological tools for the MINDtouch research have been:
• the first-person accounts of the experience of the research process and project development,
including the workshops, the technical aspects and the mobile performance events;
• hetero-phenomenological experiences and observations of the participants and their videophone
expressions/recordings.
First-person accounts and theorisations of my own mobile and other personal experiences of virtual
presence and liveness in daily life were recorded, as well as the performance design process.
2.2.1/ journaling phenomenologically One method common to practice-based research, phenomenology and ethnography is keeping a
journal and taking copious notes. For MINDtouch an online wiki was used, based on a paper-bound
journal, in addition to video recordings of and by the participants for the project. Journaling also
involved notes of the conceptual and design processes and outcomes. Typically, journaling in my
practice of it involves noting any ideas that surface, usually during mundane activities or while reading
related works, especially when trapped on a bus or train or sitting in outdoor cafes. These musings
have always been a rich source for planning, philosophising and structuring writing and research
activities. During the preparation, construction and experimentation of this project, much has been
documented through journaling, taking photos, as well as videophone recording. All documentation of
the video workshops, events and performances, VJ mixing and any other work-in-process is to be
59
sorted, processed, compiled and then designed into a multimedia format such as a Web/DVD. What is
sometimes blurry is the distinction between a phenomenological account and documentation, as the
latter implies a third-person ‘removal’, or ‘objectivity’, which is not my experience of journaling – usually
a more non-linear, first-person, lived account or ‘brain-dump’ of the work.
The journaling of research processes and thinking helps to clarify experiences, concepts, design
planning, tests and activities. Sketching ideas out, keeping track of advisory discussions and other
reference names/directions, helps to concretise the next steps, and create an overall repository [48].
Journaling every day is not necessary, but then again may occur twice a day: mainly whenever critical
ideas surface, meetings take place, or activities need to be explored and developed. Sometimes these
notes are not reviewed again for months. Writing in a physical, portable, bound, paper book is part of
the process, as it is the most comfortable and usable format for my process, since it can be done
anywhere, at any time. Salient notes are later transferred and reformulated, or ideas cemented in the
online wiki. The journal notes are often used as fodder for the final written work, as well as for journal or
conference papers. Two large, bound, paper journals were used for MINDtouch , starting from
October 2006. Thus, journaling is a key phenomenological approach used here.
2.3/ ethnography Ethnographic methodologies are generally used in social sciences to study human populations
(Marshall, 1998). They often include several parallel techniques to maintain objectivity, while performing
qualitative analysis of group behaviour. Ethnographic qualitative research techniques are sometimes
employed individually, but frequently they are performed in combination, to acquire the most accurate
and efficient accounts of group activities and structure. The ethnographic models employed here derive
from my Sociology/Anthropology degree and interdisciplinary postgraduate background [49].
Ethnographic techniques can include: interview-based studies, participant observation, textual analysis,
and conversation analysis. The ethnographic techniques used in this research include: workshops,
questionnaire analysis, observation of video clips created by participants in the workshops (also part of
Visual Methodologies discussed further on in this chapter), as well as:
• the analysis of interviews with participants after the performance events; and
• questionnaires, observation and interviews conducted with the workshop and event participants.
48 The research wiki was kept online but locked from the public (due to this wiki being spammed, it is here http://cami.chni.ca/wiki/index.php?title=Main_Page&chni_camiwiki_session=a7c81204fcb41f9dbfe3eec2ba5f947e but will be moved and made available elsewhere for examination purposes). 49 This understanding of Ethnography comes from extensive undergraduate studies in qualitative and quantitative methods, particularly ethnographic techniques that were used for embedded study of a Wiccan group. These skills and practices were updated through masters research and this present research.
60
More specifically, after each video collection workshop, interviews on the participants’ experiences
were collected to develop a sense of their perception and engagement with the activities. Techniques
also include: pre- and post-reflective experience questions, observational notes, videophone interviews,
and observational/documentary video recordings. Also, each iterative performance or ‘scratch’ event
involved post-performance surveys and a few video interviews with participants, in addition to
observational/documentary video recording. For the workshops, it was more disarming for participants
to interview each other. Customised structured improvisation techniques were developed for the video-
collecting workshops and participatory performance events, adapted from previous performance
training [50] in order to most effectively explore and develop the participatory aspects.
An ethnographic approach in my research involved observing both the participants in video-collection
workshops and performance events, and those involved remotely through the network [51]. Multiple
manifestations of sensing presence were facilitated during workshops and performance events, and
observed as possible evidence or forms of the experience of presence in the real and virtual interaction:
• engrossed attention during mobile phone use;
• physical presence of participants and their body data;
• in-between invisible space of the data being sent to the phone; followed by
• in-between of the phone accessing the database of video clips;
• in-between space of the network connection;
• captured emotional, aesthetic presences of the participants within their video clips;
• physical co-presences of the participants in the space of the events; and
• virtual multi- or co-presences of the remote participants and global collaborators.
Some of these are presented conceptually in Chapter 8, which will discuss presence, and the results in
Part III. 2.3.1/ interview process
For the video-collection workshops (and also for performances), participants were recruited via an email call,
or through word-of-mouth. Video interviews were conducted with almost all of the workshop participants
after each workshop. The interviews were semi-structured, with the same questions asked of each
participant (see appendices for the questions). Interviews were usually of short duration, depending on how
much the interviewee remembered, cared to divulge, or wished to speak of their experience. For privacy
reasons, interviewees were not required to reveal their faces on camera in these interviews. Participants
provided personal accounts of their experiences of documenting their videophone expressions/recordings
during these exit questionnaires and interviews. Certain topics were explored further, depending on an
50 Similar to Augusto Boal’s techniques, discussed more in the performance section in Chapter 3.
51 Technical delays in the development of the networked connectivity made some of this difficult to study, see more on this in Appendix I.
61
interviewee’s responses. Some questions were omitted if interviewees had already answered said question
elsewhere in the interview. Each interview was reviewed and notes were made on each. Analysis details are
discussed in the Outcomes chapter in Appendix I. I also looked for any patterns that emerged, any common
experiences of interviewees, from questionnaires and video interviews, as well as the overall analysis of
workshops and performances.
2.3.2/ performance ethnography
After several recommendations, it became clear that Denizen’s approach was relevant as a possible
methodological guide to contrast with and contextualise my own performance methodology. Differing
from the ethnography described above, in his book Performance Ethnography: A Critical Pedagogy
and the Politics of Culture, Denizen discusses his approach to methods and methodology using an
ethnographic approach for performance works:
Performance is an act of intervention, a method of resistance, a form of criticism, and a way of revealing agency [...] As fluid on-going events, performances mark and bend identities, remake time and adorn and reshape the body, tell stories and allow people to play with behavior that is restored [...] The way a performance is enacted describes performance behavior [...] (Denizen, 2003: 9)
What is key here is that Denizen, as with Boal, Phelan, McKenzie and others (discussed further in the
next chapter), confronts not only methods of conducting performance research, but the political
dimensions of performance work and its role in society. When carving a space for performance
ethnography in contrast to autoethnography, he states:
The performance paradigm privileges an "experiential, participatory epistemology" (Conquergood, 1998: 27). It values intimacy and involvement as forms of understanding. This stance allows the self to be vulnerable to its own experiences as we are to the experiences of the other (Behar, 1996: 3). In this interactionist epistemology, context replaces text, verbs replace nouns, structures become process. The emphasis is on change, contingency, locality, motion, improvisation, struggle, situationally specific practices and articulations, the performance of con/texts (2003: 16, quoting Pollack, 1998: 38, my emphasis)
This is a critical point when examining my own approach, which is about action, process, improvisation
and interactions, and ‘allowing the self to be vulnerable to its own experiences’. In a direct sense this is
employed in the making of the performative activities for participants but also within the workshops and
events themselves in MINDtouch. In the director role, I guided the whole performative process, from
the devising to the enacting features – ‘change, contingency, locality, motion, improvisation, struggle,
situationally specific practices and articulations’ – that Denizen puts forth as performance ethnography.
This role might also be considered reflexive performance or art research, or considered Kozel’s
heterophenomenology, or all of those rolled into one – with myself in different ‘roles’ in the research as
62
set out here. This is also where the ‘scratch’ performance concept or iterative theatre can work, as it is
responsive to the ‘audience’ or, in this case, the participant co-creating the work.
Denizen changes the focus to state:
The performance text-as-an-event is situated in a complex system of discourse where traditional and avant-garde meanings – of the real, the hyperreal, mimesis and transgression, audiences, performances, performance video, and performance art pedagogy – all circulate and inform each other. (2003: 36)
This perspective contextualises the mode of performance I use as differing from the traditional: it may
even be considered post avant-garde in its use of technology and removal of the audience–performer
dichotomy. I will discuss and unpack this, and concepts of the real, liveness, performance, and
performance video, further in coming chapters.
2.4/ experience & sensorial design With a background in performance, media art curation and event production, I have developed
production sense in creating experiences for others. This involves reviewing what has worked (for me
and the performers, technicians and the audience – one and the same in MINDtouch ) and what has
not worked in the past, what could be finessed and what needs to be cut. The London-style ‘scratch’
approach of staging performances was adopted – the style of audience interaction that involves
iterating the performance based on audience feedback – which was compelling after exposure to it at
several alternative, small, London theatre and technology-based productions [52]. A focus-group-style
method of receiving direct feedback on what was experienced from the audience and how to improve
the production, it appeared an effective way to include the audience in the development, staging and
iteration of the work.
Previous art-based research was revisited, in order to repurpose and reuse methods and philosophy of
Experience Design. Experience Design is often used by Interaction Designers in web design, game
design and in HCI for software design. Research on sensorial design [53] exposed that a framework
informed by performing arts was needed to properly assess the effectiveness of sensorial media art
design with users. Although focused on online education, Meredith Davis’ article, ‘A Curriculum
Statement: Designing Experiences, Not Objects’, is relevant to Experience Design researchers. I
modified her list for use regarding sensorial media design such as art and performance installation. She
52 One of those shows, called ‘Robot Show’ was by robots, at the Battersea Arts Centre, February 2007. The actors gave out surveys at the end of the show and then had subsequent shows that implemented changes suggested by the audience. There were others but I do not have their information.
53 Sensorial Design is a term coined by Nathan Shedroff (1994), mostly used when discussing interactive web advertisement design.
63
suggested that the Experience Designer should:
1) observe and understand the difference between users’ responses to objects versus experiences;
2) analyse and synthesise what is meaningful and enjoyable in human experience and interactions
with the world and other people in it, including: the physical, cognitive/intellectual, emotional,
spiritual, social and cultural dimensions of our daily lives;
3) evaluate current and past forms of technological mediation on sensorial experience and the
impact on people (positive and negative) in their daily lives;
4) assess users’ patterns of social interaction (ex. human conversation, interpersonal exchange) and
use these patterns when designing technologically mediated experiences and environments;
5) understand how technologically mediated experience alters experience expectations and in turn
changes human behaviour;
6) evaluate the mechanisms and tools used to create the experiences, by attempting to
represent/simulate/visualise/transform perceptions, events and ideas in an artificial environment,
eliminating all aspects of the non-artificial and potentially distracting or disengaging elements;
7) evaluate the appropriateness of the way in which the experience is received by particular
users/audiences, while understanding how it shapes the understanding of intended meaning of the
messages within the environment – or an appropriate ‘fit’ of form versus context, the very nature of
sensorial experience design, and how it will change over time;
8) evaluate how these experiences impact upon users’ perceptions of the world, potentially changing
them permanently;
9) evaluate how these experiences compare to other successful interactive, sensorial experiences,
as well as to other sensual experiences, understanding the conventions and expectations created by
users’ previous experiences, keeping in mind potential impact on both the user and society in the
larger context – such as altering human relationships, values and decision-making abilities;
10) evaluate the social characteristics of these technologically mediated sensorial experiences in
terms of how they meet users’ emotional, physical and cognitive needs, such as how they respond,
clarify, or provide feedback to users;
11) evaluate the way in which human attitudes toward, critical assessments and perceptions of each
other in terms of an experience designers’ credibility, reliability, and authority, as well as their own
role in self-determination and choice or agency in navigating an experience or the structure of
persuasive arguments or puzzles, which then stimulates critical thinking and utilises common skill
sets and values. (paraphrased and modified from Davis, 2000)
This framework is relevant and has indirectly influenced MINDtouch . Interactive and media artists can
call themselves sensorial designers by contributing to the field of design, although it is differently
motivated. The current research follows this framework in some ways, including:
64
• observation of participants’ needs in a variety of experience contexts;
• analysis of participants’ needs, goals and behaviour in general, but particularly within the
context of the experience provided; and,
• delving deeply into phenomenological and psychological domains of the participants
involved (from surveys, interviews and observations) so as to understand what motivates
them within or during these experiences and performances, in order to then ask why they
engage in these experiences.
For MINDtouch events I constructed a questionnaire, based on Davis’ and a similar framework I
developed for a previous interactive project (also based on the workshop participants’ feedback, and
curatorial audience surveys – see Appendices). The purpose of these questions is to assess the
experience of each participant, as in Davis’ framework above. From each workshop and performance
event the purpose is to understand what they experienced and then try to iterate each performance or
workshop from there. An example would be that if participants felt that one activity using the mobile
phone was too complex or they did not enjoy it, I would then evaluate that and decide to modify or cut
that activity if it was not crucial to their overall experience. In the facilitator role this is a critical function:
making sure the participant is having a good experience and then, based on the overall goals of the
project, modifying it for each iteration to ensure that an enriching experience is being had. See further
analysis of the feedback in Appendix I for specifics on the implications of the questions and what they
were trying to elicit, and the implications of the answers.
2.5/ embodied design / ‘workshopping’ Due to the complexity of the research, several methodological approaches were layered together to
make a hybrid methodology borrowing tools and methods from those discussed in this chapter, as
mentioned in the introduction. Here we look at what I am calling ‘workshopping’ (it could also be called
the workshop method, bodystorming or embodied design), used in HCI and other forms of product
design. Experience as a research assistant for the whisper[s] project [54] exposed me to the
embodied-design approach through its workshops. The intention of these workshops was to test user
interactions with the technological solutions, without the technology. This solution, also called
bodystorming [55], is a form of usability testing, from a performance and body practices perspective.
Thecla Schiphorst’s PhD practice was formalised in her thesis, The Varieties of User Experience:
Bridging Embodied Methodologies from Somatics and Performance to Human Computer Interaction,
which used the whisper[s] project as its main case study. Schiphorst’s approach and thinking
influenced my own artistic practice, although we each have our own research workshop design
approaches and contrasting methods. Schiphorst’s methodological approach, in her own words, is 54 See more on the project itself further on in this section.
55 Bodystorming is discussed by Oulasvirta, A. et al., Understanding Contexts by Being There (2003).
65
using:
[…] the notion of ‘paying attention’ to one’s self, and using this sense of self or body-state to connect and exchange with another. This requires an experience of ‘inner’ space or intimacy with oneself and an ability to recognize and transfer this ‘sense of self’ to another person [… so] we turned to performance and Somatics methodologies [… and] techniques for extending our bodily awareness through attention to breath and movement […] common to performance methodologies found in theatre, dance and body-based disciplines. The techniques within these domains build both intra-body and inter-body experience and knowledge through technical exercises that focus on our perception of our own physical data. (2007: 111, my emphasis)
Schiphorst explains how workshop techniques used in her work were borrowed from performance
practices as means to ‘act out’ or ‘act through’ aspects of the project design and planning process,
and that this was especially key when using or working with people and their bodies. She explains:
Within performance processes, workshops have a specific set of functions, which include methods of exploration and discovery that are directed toward the development of new material, or bringing new life to repertory. Richard Schechner, a Performance Studies practitioner and scholar has written: A workshop is the active research phase of the performance process [...] Probably the most prevalent kind of workshop is used to “open people up” to new experiences, helping them recognize and develop their own possibilities. (quoting Schechner, 2002: 199, Schiphorst, 2007: page 117)
She points out that the model in whisper[s] of using intention to test, develop and iterate is a theatrical
model, utilised as a user interaction model to explore […] experience concepts such as intention,
gesture, direction of attention, relationship, rhythm, body-state, and attitude to space […]. She explains
that this process can inform the physical and technological design and assist in the evaluation and
analysis of the ‘relational elements’ in constructing experience (2007: 117) Her thesis outlines her
project process and her discoveries in embodied interaction design to create and develop the
whisper[s] wearable installation art project.
What was learned from whisper[s] was the process of developing and conducting performance research
workshops, enabling people to engage with themselves, as a means to understand how to use technology
in this context. But it also aided in learning through someone else’s approach to the design of body-based
technology systems how to allow people to ‘listen’ to their bodies while engaging with others.
Although whisper[s] and MINDtouch share some similarities, their approaches differ in intent, outcome
and process. whisper[s] workshops engaged in ‘experience modeling’, as Schiphorst calls it, which were
short exercises to mimic technology use without the technologies themselves, using prototype or stand-in
tools. The intent was to design new devices and technology systems based on participant responses and
behaviour with ‘placebo’ versions of them. In contrast, the intentions of the ‘workshop’ approach used in my
research was instead to test new, embodied, performative and artistic uses for current technology. The
making of new tools or software was only implemented where needed to bridge technologies in new ways
or in new uses. My workshops were also intended to be a means of generating a media database for a
66
future performance installation. MINDtouch workshops and performance events were planned as
experiences to assist people in exploring body states and sensations, similar to whisper[s] , but also to
assist participants in ‘capturing’ and transforming those experiences into new, synaesthetically enhanced
and ever-changing experiences for the performance collage/social VJ mixes. My goal was not to further
design technology with the end artefact. Although technology and system design did take place, this was
not as a result of the workshops, but to facilitate those workshops and performances. Technology was
merely a facilitator and means to simulate and stimulate specific and unique forms of technological
embodiment.
Yet, the approach of whisper[s] has been hugely influential in clarifying the differentiation of the two
approaches to workshopping. Each project uses the technique as a way to try out something with
participants to be used later, in the context of technological experience. Here Schiphorst elucidates her use
of ‘workshopping’ as a technique:
The whisper experience workshops were born from the desire to explore how people engage in the act of ‘paying attention’ to themselves: their senses, their inner state and their ‘world’. The initial intention was to explore whether such an activity could be meaningful: could it have instrumental value in an interactive technologically mediated ‘experience’? (Schiphorst, 2007: 115)
Image 1 © 2007 C. Baker – still from Dublin workshop, October 2007
For the MINDtouch project, the workshops have been essential in helping to distil methods in the
creation of the participatory experience design. They were initially used as a means toward designing
the collaborative performative events and live video artefact. Other intentions included: to help people
explore the sensations of their bodies and their internal perceptions; to facilitate mental visualisations of
these internal perceptions and sensations; and to assist participants to explore the act of externalising
and representing these mental images (Image 1 from one of the workshops, shows participants during
the guided mind-quieting exercise). They were also an experiment in facilitating non-verbal and gestural
communication that could have other non-artistic, social or assistive outcomes after the project ended.
Another goal was to create a media archive for the end performative video collage. So, these
67
workshops were intended to:
1) facilitate internal focus, attention and mental visualisation (i.e. help to ‘capture’ visual imagery)
in participants;
2) facilitate participants’ exploration in representing their mental imagery and impressions of their
bodies’ sensations, using colours, shapes, patterns, objects, etc., from the external world;
3) facilitate the simulation of thought transfer and/or non-linguistic communication with others in
the workshop;
4) enable participants to use the mobile video clips cameras to ‘capture’ with the intention of
‘sending’ these visual expressions of body perceptions to someone else (not in the room);
5) have participants collaborate in creating a database of video clips, to be used in the future
performance events.
The details and results of the workshops, and how they informed and shaped the performances, are
discussed at length in Chapters 10 and 11. 2.6/ structured improvisation
Structured improvisation, borrowed from theatre, dance and telematic performance, has been most
influential in the ‘design’ and methodological approaches of MINDtouch. Training in improvisation
techniques from choreography and other Modern Dance techniques have assisted in the design
development of the participant activities. In dance and theatre experience, especially in Contact
Improvisation (a dance form that involves physical contact and communication between dancers), dancers
are ‘warmed up’ in their bodies and then given small movement activities, either alone or with a partner [56].
These activities may be based on a visual or metaphorical element used to inspire the dancer to move in
particular ways, or it may be based on sound or music played or used to explore certain forms of physical
trust, weight, gravity and balancing activities with a partner.
For the whisper[s] project, in the design of their experiential workshops, Schiphorst and Kozel used a
specific approach to create the structured activities for participants to explore within, during the workshops
discussed above. In a similar way, for MINDtouch participants were given tasks or activities [57] and were
then observed as to how they encountered these. Participants were then asked how they
phenomenologically experienced performing the tasks, as well as what they perceived or sensed in their
bodies when using the technology during the activities.
56 For more on Contact Improvisation see the online Contact Newsletter available at http://www.contactquarterly.com/cq/webtext/resource.html (accessed May 27th, 2010). I have participated in these warm-up activities and contact classes.
57 More on the techniques and their use is discussed in Appendix I. See the Appendices also for the scripts used to guide participants through the exercises, using these techniques.
68
Madeleine Lord (2001) states from her research that there are five basic teaching strategies used to
engage students to generate movement spontaneously: 1) setting up the situation; 2) presenting the
exercise; 3) giving students a transition to execution of movement; 4) guiding the task execution; and 5)
revisiting the situation. In her paper, Fostering the Growth of Beginners' Improvisational Skills: a study
of dance teaching practices in the high school setting, Lord studied teaching strategies in secondary-
school dance programs, and these forms of improvisation techniques used in dance to inspire students
to move creatively emerged from her research. Similarly, in my workshops I would start sessions with a
mind quieting exercise (set up); present participants with the activity; assist them if them had trouble
trying to engage with the activity and start using the video (transition to execution); guide them in the
task execution (if needed); and then when they completed that activity I would bring them together to
start again and revisit what had just been done.
Improvisation techniques for theatre were also developed my Augusto Boal. His work and these
techniques (as they apply, and were referred to in this project research) will be discussed in the next
chapter on Performance.
2.7/ visual methodologies
As mentioned in the introduction and discussed further in Chapter 5, visual methods were employed in
this project in order to evaluate and work with the video clips generated by participants in the
workshops and events. These were mainly for analysing the content of the visual imagery in order to
devise appropriate database categories. These categories were necessary for the custom software to
appropriately use the body data received from the sensors, as a way to understand the intended
content of the visual material created by participants. The methods used for the selecting, meta-
tagging, categorising and analysing the video clips for the database and video mixing for the
performances were borrowed from Gillian Rose’s book Visual Methodologies: An Introduction to the
Interpretation of Visual Materials (2007). Rose’s book influenced how to analyse each video’s visual
content as intended by participants. Her work on art and design research considers different ways to
look at images and visual materials from compositional analysis [58], including: 1) formal analysis, which
focuses on the form of the object in question – elements of form analysed include: line, shape, colour,
composition, rhythm, etc. At its simplest, such an analysis relies heavily on the researcher’s ability to
think critically and visually; and 2) stylistic analysis, which focuses on the particular combination of
formal elements into a coherent style (Rose, 2007: 57). This stylistic analysis makes reference to
58 I have focused on these as the most relevant to this project since other visual methods she discusses are for analysing museums and their collections, film, TV and other media for analysis not related to this work. She also discusses Content Analysis, which I chose not to employ with all the other methods involved.
69
movements or trends in art and design as a means of drawing out the impact and import of a particular
object.
Rose suggests that the way to go about compositional interpretation of image content involves looking
at and reading images by using the contextual information about the artist, their style and the imagery
they were depicting and inspired by:
1) looking at the image itself for what it is or says, rather than what it does or how it is used;
2) looking at the content and form of images, rather than ways in which they were produced or social contexts/practices;
3) focusing on the technological modality of making the image – how it was made technically and the materials used to make it, including:
a. colour: i.e. hue, saturation, value;
b. spatial organization: atmospheric perspective, geometrical perspective, eye-level, logic of figuration (inside and outside the image), focalisers;
c. light;
d. expressive content;
e. mise-en-scene/camera for moving images: screen ratio, screen frame, screen planes, multiple images, superimpositions, shot distance, focus, angle, point of view, pan, tilt, roll, tracking, crane, zoom;
f. montage for moving images: editing, continuity editing, cut; and
g. sound for moving images: environmental, speech and music – diegetic, non-diagetic, narration. (Rose, 2007: 35–58)
A modified version of compositional analysis was used here, focusing on participants’ (as artists in
Rose’s work) videos from the workshops, since those were the most collected and plentiful. The
content primarily featured the environment and spatial context of where the workshops were held, and
included elements inspired by the activity participants were engaged in, and the available visual imagery
they were depicting or capturing around them. This content is what Rose would call ‘materials’ in #3
above, which were used to make the videos. These materials mostly contained imagery of architectural
(interior and exterior) or structural elements that featured patterns. The materials also contained
movement and other ‘expressive’ content, such as extreme close-ups of a person’s eye or the
translucent piece of skin between the thumb and index finger.
I chose to do a fairly cursory analysis, as the goal was not to judge or interpret the intentions or
symbolism of these expressions by participants. Rather, this review and compositional analysis was
performed to determine how to organise, categorise and meta-tag the clips for the visual mixing
archive. There was also no intention of judging the quality of materials, as the participants were
recruited from the general public, with no necessity for visual or performative background, so any
expertise or talent in creating pleasing material was incidental. The aesthetics of the material is
discussed in more detail in Chapter 5. Details of the content patterns found in the participant videos
70
can be found in Appendix I. The selection criteria for the video database is discussed further in Chapter
6. 2.8/ technology and software design methods
As I am a media artist and not a software developer or coder, I am not able to fully comment on the
details of the methodology ascribed to by the technical collaborators working on the project. However,
I can give an overview of how they developed the biosensing technology for the phones, the computer
VJ mixing software to work with the video archive and the assembly of the electronics. In our meetings
and interaction, it was reported that the main methods for working with these systems came from an
empirical methodology, a process that moved from concept development and discussion to materials or
coding exploration, technology development, testing, debugging, retesting and finally system implementation. There were many working sessions where we would start from a diagram or concept I had
prepared and then work through and develop an action plan; these technical experts would then go away
and implement this action plan, while I would follow up from time to time to make sure all was on track in the
timeline. When we worked with the electronics, research on the appropriate sensor systems was done
together, and then decisions would be made as to which to purchase. These were ordered through the
university and then my collaborator Michael Markert would assemble them, try the mobile software out on
his own phones in his home workshop in Germany, then send the sensors and the software to me to test on
the research phones. We would retest until it worked with all phones, which were all different N-Series
models (since there would be different models of participants’ phones). The software did not easily work on
all systems right away and modifications had to be made so that it could. Each technical collaborator would
train me in what they had created, and, each in a different style, made notes and created a ‘user manual’ for
their software after the creation as well, in line with good practice.
This chapter has attempted to show how the differing methods and methodological frameworks have been integrated and implemented into this research to create a new hybrid Art-as-Research
methodological form to shape the research. One can see that these mixed methods were used to complement each other for the sake of thoroughness, to address specific aspects of each phase of the
project design and implementation, and to create an engaging and personal participatory technology-based experience, with the ultimate goal of implementing systems to enable the sensing of liveness and
presence of others over the mobile network. However, each methodological dimension also has had a distinct role to play in uncovering different aspects of the work.
71
3/ participatory performance
This chapter focuses on the performance aspect of the project, on prominent voices in the
performance studies, and on how different approaches have been implemented in the research. It
discusses participatory performance practices and their use within the project.
The form of performance applied in MINDtouch borrows from the participatory or public performance
art of the Fluxus or ‘Happenings’ or street theatre experimentation of the 1960s. This form often
involves a structured improvisation with available, untrained participants, guided (or coaxed) by trained
performers in semi-structured activities (Boal, 1992). For MINDtouch , participatory performance is
used as a way to explore the mobile video and wearable devices as the means to re-engage people in
performance and new media art. It is more aligned with new media and fine and performing arts than it
is with mobile or locative games. It also serves as a way to (re-)engage people with the physical world
through the media tool. This decision came from experience in performing arts and video, as well as a
discovery that many current locative media theorists and creators come from a computing HCI
(Human–Computer Interaction) or video-game background. In this way, this work contributes to
interactive arts, video art, performance and live art more than to the video gaming, mobile
entertainment, interface design, HCI or computing worlds, even though I use emerging technologies
and computing to achieve similar ends.
For modern dance influences and borrowings in my practice, key figures include Isadora Duncan,
Martha Graham, Doris Humphrey and Rudolf von Laban. From music, as a consumer and musician,
experiences are drawn from popular music and rock from the 1970s onward, which are many and
varied. The non-musical ‘performative’ experience and history is highly influenced by yoga, Tai Chi, and
other body practices I have studied. As a performer, my notion and experience of modalities of
performance have evolved and shifted: starting with a traditional understanding of it within music and
dance, as well as an audience member, to also now include media and performance curation. It now
includes a more interventionist, site-specific, live art [59] and networked performance approach, with
some ‘mixed-reality’ games (another name for locative media and psycho-geographic works,
discussed in the next chapter) being introduced more recently.
Gregory Sholette, as editor of the book Interventionist: User’s Manual for Creative Disruptions of
Everyday Life, defines Interventionist Art as:
[…] the liquidation of artistic detachment by staging a fresh assault upon the tenuous boundary between art and life […] understands this conflict as a site for critical, artistic engagement within the public sphere […] the contemporary, so-called interventionist reveals a definite congruence with the historic avant-garde program […] (Sholette, 2004: 133–134)
59 This was the mandate of The Escape Artists Society, of which I was an executive director. Tate Glossary is available online for interventionist art at http://www.tate.org.uk/collections/glossary/definition.jsp?entryId=276 (accessed March 20th, 2010); Performance and Live Art online at http://www.tate.org.uk/collections/glossary/definition.jsp?entryId=155 (accessed March 20th, 2010) and http://www.tate.org.uk/collections/glossary/definition.jsp?entryId=218 (Accessed March 20th, 2010).
72
Sholette explains how interventionists make socially useful art in the vein of the Soviet avant-garde of
the 1920s. Of course, ‘socially useful’ might be a debatable term, but the form has definitely been seen
to confront and find practical solutions for social issues, as can be seen from the work profiles in
Sholette’s book. The Tate Glossary has defined the term ‘site-specific’ as: […] a work of art designed
specifically for a particular location and that has an interrelationship with the location. If removed from
the location it would lose all or a substantial part of its meaning (2010, online). This project has not
been site-specific in this sense, but if we are to return to the earlier mentioned authors in the
Introduction, in terms of telematics and virtual performance, some practitioners such as Glesner (2003)
definitely consider the network to be a site. The Tate defines Performance and Live Art as referring to:
[…] Performance art and Action art and their immediate precursor Happenings, together with the
developments of Performance since the 1960s [… and being] Art in which the medium is the artist's
own body and the artwork takes the form of actions performed by the artist (2010: online). From these
definitions, we can see how this project can be correlated with these performance forms, borrowing
from the interventionist approach by repurposing the everyday mobile phone in order to socially engage
and challenge people to communicate differently, while using the ‘site’ of the network and the artist – in
this case the participant – as artist or co-artist/collaborator, as part of the artwork.
It is important to define the space this project occupies and why it is classified as performance rather
than a mobile video or pervasive game (defined below). While MINDtouch might also find itself
categorised with decidedly technological projects, it can be distinguished in many ways from these,
because: a) location and geography is not the focus of the work; b) it borrows from performance and
media practices, aesthetics and philosophy; c) many of the locative media and psycho-geography
projects – apart from the locative sound work of Janet Cardiff – have originated from a video gaming
tradition [60]; and d) this project has a more introspective, emotive, ‘touchy-feely’ focus, rather than
being an action-based, competitive treasure hunt, history walk or game, which many mobile projects
are at present.
3.1/ participation as performance
To understand the design of the performance activities in MINDtouch, one should understand how
the project began working with participants. In the first phase of the project (as described in the
Introduction), video collection workshops were organised and participants were asked to share their
internal images, feelings, thoughts and impressions, using the phone in a non-verbal, non-textual visual
way, with the video recording features sharing these videos with others. To help participants to engage
in the experience and activities, they were informed that their videos were part of the larger,
collaborative video, and this collection of their personal clips has since been used in the live, networked
60 Which is traditionally male-dominated – and many locative projects also demonstrate this fact – see the name listing of locative media creators at the Mixed Reality Lab in Nottingham, for example: http://www.mrl.nott.ac.uk/people.html.
73
performance experiments/events. As you can see from the image below, the performative engagement
of the participants made them express themselves with the video camera to others within the
workshops in a very engaged and gestural way.
Image 2 © 2007 C. Baker – participants in video collection workshops
On the surface, MINDtouch might fall into the locative media or ‘mixed reality’ category [61], related
to psycho-geography [62], connecting place and location to emotional or personal experience through
sound and media. However, this work is deliberately situated within the avant-garde, live art, theatre
games, or live cinema and video art contexts from which I come.
To understand performance and participatory performance more deeply we can start with David Saltz’s
stance on performance, audience and interactive arts, as explained by Kozel:
[… i]n his essay 'The Art of Interaction' (1997) [...] hinges on performance being constituted by actions performed before an audience: If we accept ‘performing for an audience’ as the distinguishing characteristic of performance, it follows that all staged interactions are performances, and all participatory interactions are not. […] if people see their actions as aesthetically significant, they are performing. (Kozel, 2007: 68, quoting Saltz, 1997, my emphasis)
On this point, one could argue that aesthetic significance or even an awareness of aesthetic
significance is not necessary. An audience member or participant can have an engaged experience
that contributes to the aesthetic significance of the work, without realising this significance. Rather,
just having any emotional, existential, corporeal or thoughtful transformation or response to the
activities engaged in within a participatory performance is in itself of significance. It can be argued
that most art is about shifting perceptions and trying to communicate with an audience or spectator.
61 i.e. using mobile devices in the real world, alongside computer game contexts and interfaces, as Blast Theory and Steven Benford use the term.
62 Janet Cardiff and other sound artists and musicians.
74
Awareness of one’s role in the event or piece, or some sense of contribution to the work, is indeed
performance, as it is a sense of dialogue that the participant/audience/spectator has with the artist,
yet it is more than that. Erving Goffman (1959) [63] might say that we all play a role or perform at any
given time, that just being part of a participatory performance places one in a different role in one’s
life – with or without awareness of its aesthetic significance, but with an aesthetic experience. Thus,
it automatically has aesthetic significance since it is transformative. This does not mean participants
become artists/performers themselves, in the same way as the artist who created it, but they
become ‘actors’ within the work and the artist/creator of the work becomes the choreographer or
director (or ‘facilitator’): each is important for the work and each is aesthetic in their role.
In a search of other discussions on the current discourse in Participatory Performance, I came across
two practitioners in dialogue on its nature. David Goldenberg and Patricia Reed define what constitutes
participatory practice and how it manifests practically through this exchange:
Reed: [...] the project is initiated, which means there is a condition of response inherent to it – you ‘play’ within, around, against that initiation, so in that sense it’s not infinitely open [… so] ‘partial unpredictability’ [....] the maintenance of ‘joint attention’ in a conversation – the perpetuation of partial novelty. Basically there are enough unwritten rules in a conversation that we know how to perform it [...] Goldenberg: By ‘setup’ I mean whatever location, site, or context where a participatory practice takes shape – without acknowledging the restricted notion of place where art occurs, whether in a gallery, museum, or public space […] so the possibility exists for participants to actively take over the running of a project […] There is a sense if you are involved in a participatory practice experiment that a level of success is achieved if the participants take control of the project on an equal footing. (Goldenberg and Reed, 2007, online)
This is a definition of Structured Improvisation: providing guiding elements for participants to ‘play’
within, with a setup and ‘gravitational force’, or activity containment factors, so that the activity is not
‘anything that goes’ and doesn’t become totally unclear to participants. It also has partial
unpredictability or a wide range of possibilities of expression, exploration and experimentation. To
reiterate in another way, Steve Dixon says that the:
[…] four types of interactive art and performance we discern are ranked in ascending order in relation to the openness of the system and the consequent level and depth of user interaction: 1. Navigation, 2. Participation, 3. Conversation and 4. Collaboration. (2007: 563)
and that,
[…] there are more genuinely collaborative models in interactive arts where the user’s input is freer, more open and more significantly changes what happens. (2007: 565)
63 Goffman has been studied during this research, but his work is not central to the work. His discussion on selfhood as an everyday series of roles we play, rather than performance itself, is intriguing, especially when working with non-actors. But in this regard, Boal is more relevant.
75
This project would fall into the ‘collaborative’ category in this scheme, although it uses a form of
interaction that is more of a conversation: between participants, participants’ bodies, and their own
interpretations of their bodily sensations mediated through the video imagery and technology. Saltz’
view is relevant however, as it contrasts with my approach of challenging the conventional notion of
‘performance’ – that is staged ‘in front of’ an audience – and instead allows the audience to be
anywhere in space and time, and to become performers with access to the final live ‘cinema’
performance event, ‘in front’ of them (via their devices).
3.2/ politics of participation
During the research planning and initial video workshops, a realisation surfaced that in order to draw
people to the events, an important step would be to make every event a ‘non-performance’ – in other
words, to drop the word ‘performance’ from all calls for participation. The word carries baggage and
connotations of power relations between the audience and performer, and could be intimidating for
potential participants. A sense of fear or inadequacy had initially been witnessed in a few participants,
some admitting that they felt they may not ‘perform properly’, as if there was a ‘correct’ way to
experience their own sensations or to do the activities. The word 'performance' is loaded and can
intimidate, suggesting the participants are performers and not that they simply perform. Some may feel
that they must try to be 'good' or 'trained', which could have an immobilising effect. So the words
'performance' and 'performer' were removed when recruiting participants [64], and in later events I
chose to remove the ‘performance’ language when guiding participants as well. Instead, it was framed
as a more inclusive social event, more like a party or participatory entertainment event, with the hope
that this would decrease pressure on participants to think they needed to be individual centres of
attention, performing for others [65]. Removing the term had the added advantage that people with
performance backgrounds would be as likely (as opposed to more likely) participants as those less
interested in such things. So, in choosing these more casual social contexts, the ‘performance’
activities could become more personalised for participants, yet playful within the larger social context
where participants would feel they had full control over when and what to share. Through iterations of
the work, it was discovered that when the workshops or events were called ‘activities’ rather than
‘performance’, more participation came from the non-creatives and non-performers. Thus, if the goal is
to reach and engage the average ‘Joe’ or ‘Jane’ on the street – non-traditional, untrained ‘performers’
– it is key to find a new language or means to make them feel comfortable to engage, thus less
intimidated and less inhibited. It is also important to make the activity meaningful to the participant as a
64 Participant recruitment methods will be covered in Appendix I on the workshop details. Briefly, they were found through word-of-mouth, email-outs, posters and Facebook.
65 Keep in mind that in this work there is no ‘audience’ per se – all participants are meant to be active and even those remotely engaging on their mobile phones or online watching the live mixed collaboratively ‘performed’ video collage are considered participants. Those in the party setting who are not part of the work might be the closest to an audience. However, if they are not aware of what’s going on around them they are not an audience either.
76
part of the project and not to create a sense of ‘right’ or ‘wrong’ in their expressivity [66]. So this was
also an effort to change the views of the participants, particularly on how they conceptualise what
constitutes a ‘performance’, as well as to change their relationship and involvement with technology, as
well as to encourage them to use technology in unique ways to express themselves, to empower them
to communicate creatively. So again [...] if people see their actions as aesthetically significant, they are
performing (Kozel, 2007: 68, in response to Saltz) [67].
Since the Fluxus movement and the age of Happenings, participatory performance has developed
considerably; Bishop as well as Bourriaud suggest that this is a growing area of art practice. Combining
participatory performance with interactive mobile media and video games, the group Blast Theory has
pioneered the creation of compelling participatory projects. Blast Theory has a talent for making their
projects intimate and personal, yet playful and collaborative for participants at the same time. They
combine theatre, video game play and technology with live interaction, in-the-streets engagement and
exploration. Thus, they have become the signature and poster-child group for locative mobile media
arts performance. Although there are others now, Blast Theory remain ahead of the pack and have
thus influenced my project a great deal.
Kate Adams explains the political aspects of Blast Theory’s project Uncle Roy All Around You [68] in
the context of participatory theatre:
Participatory theatre is associated with the political because of the possibility of creating a sense of community and empowerment as the audience interacts directly with the performers […] they are active in constructing meaning and active in relations with others […] which in turn requires the negotiation of boundaries within the relationship between performers and participants. (Adams, 2003: 7)
Images 3 and 4 © 2003 Blast Theory – Uncle Roy All Around You
What was most intriguing about ‘Uncle Roy All Around You’ was that it was a participatory
performance as well as a mobile and Internet game. It engaged participants in unique, theatrical
activities, involving the participants/gamers in the city using portable devices and phones rather than
66 This is based on observations made and interactions with participants in the workshops and events.
67 See page 97 in Chapter 5 for a definition of aesthetics for this research.
68 This project can be seen on their website, available online at http://www.blasttheory.co.uk/
(accessed March 20th, 2010),
77
only avatars on computers. This is inspiring since MINDtouch is also about creating unique
embodied, media experiences that move participants away from the desk indoors and engage them
more in the world and their own bodies. It is political in the sense indicated by Adams, because it
creates an active engagement with both others and self by negotiating the performer/participant
boundaries and relationship.
Blast Theory have had several other exciting projects, such as Riders’ Spoke, which involved individuals
exploring London city centre on a bicycle and with a PDA. The PDA had a customised, playful digital
application, which both guided participants to certain spots, and enabled them to add their own
experiences and stories about other spots. A friend participated in Riders’ Spoke and showed me the
interface, which had a very comic-book-style aesthetic, untypical for a digital interface (see Image 6
below), which had the effect of instilling a certain trust and intimacy in the project for this friend.
Participants were guided on their own journeys through the city through the device attached to the
handlebars of the bicycle (see Image 5 below), by previous riders. This created a collaborative,
interactive narrative. In a similar way, participants in my project have been contributing to a larger
database; in my case a database of visual experiences that add to the collective work.
Images 5 and 6 © 2007 Blast Theory, Riders’ Spoke
A prominent and original voice in political participatory theatre is Augusto Boal and his Games for
Actors and Non-Actors. What is crucial about Boal’s guide is that it is filled with practical advice and
exercises to prepare participants for theatre games, and actors or guides for participatory performance
and intervention. It also contains many examples of these activities in practice. The term ‘theatre game’
was coined by Boal and is used carefully here since my project is definitely not a game.
When devising the improvisation activities in my workshops and preparing the first version of the
performance staging, Boal became a guide to theatre games. His ideas helped to clarify how to
develop ways to motivate participants and facilitate engagement in the process and the theatrical
activities of the performance production. It also helped to clarify how to create a dialogue between the
participants as performers and the trained performance guides assisting them. Boal is extremely
relevant in this approach and in contextualising the ‘directorial’ or choreographic/activity design
78
process, because I consulted his games and techniques when devising my activities. His philosophy
was most deeply considered when working with the bodies of actors and non-actors, firstly regarding
specific exercises that I developed and was already using, which were similar to his techniques, in my
mind-quieting and body-focusing ‘warm ups’ [69]. His methodologies, while discovered after I created
my activities, validated my approach. He explains his performance philosophy:
A bodily movement ‘is’ a thought and a thought expresses itself in a corporeal form […] the idea of eating can induce salivation, the idea of making love can produce erection […] The phenomenon is less obvious when it relates to a particular way of walking, sitting, eating, drinking, speaking. And yet all ideas, all mental images, all emotions reveal themselves physically […] Bodily activities are activities of the whole body. We breathe with our whole body, with our arms, our legs, our feet, etc. (Boal, 1992: 61)
This is critical to consider when working with people to connect with their bodies, especially when
asking them to tune in and interpret their own perceptions and sensations as creative and performative
acts. When reviewing the exercises and games he developed from his Theatre of the Oppressed, Boal
says,
The games […] deal with the expressivity of the body as an emitter and receiver of messages. The games are a dialogue, they require an interlocutor. They are an extroversion. (1992: 60)
Some participants revealed in the video workshops that they never think about their bodies or listen
closely to them, and that the activities they were asked to do helped them to do so. This was a
revelation and exciting because: a) it was one of the intents of the project; but also b) it revealed that
there may be a need to help reconnect people to themselves and engage or listen to their bodies in
new and engaging ways, other that calisthenics and sports. Technology, specifically mobile phones,
may be able to help in this way. Boal informs this when he says:
In the body’s battle with the world, the senses suffer. And we start to feel very little of what we touch, to listen to very little of what we hear, and to see very little of what we look at. We feel, listen and see according to our specialty; the body adapts itself to the job it has to do. This adaptation is at one and the same time atrophy and hyperatrophy. (Boal, 1992: 61)
Boal is often credited as being responsible for developing the political approach to public intervention,
empowerment and change, as are other performance artists of the 1950s and 1960s, such as Marina
Abramovic. They were instigators of happenings and public interventions [70] and responsible for
sowing the seeds of modern participatory theatre and live art. My work is not apolitical in general, and it
must be acknowledged that most art and performance has a role to play, politically and culturally, in
69 However, since I worked with and was heavily influenced by Thecla Schiphorst who studied Boal’s work closely, one could say I was indirectly influenced by Boal.
70 Public interventionists are the topic of Claire Bishop’s Participation (2006), not discussed here, but this use here is primarily in reference to the book’s excerpt from Nicolas Bourriaud’s Relational Aesthetics (2001) and his discussion on the change in direction of current art practices, leaning more toward socially engaged, relational works, some also calling it the ‘performative turn’.
79
confronting society with itself, especially in historically critical moments. However, this work is more
about challenging society to take a look at how it uses its communication tools and technologies, from
a more philosophical than political perspective. It can be seen as interventionist in that it confronts
people, but it has a more emotional and expressive approach, focusing on a different mode of
empowerment; this mode of empowerment involves an understanding of self, and uses alternate forms
of communication to do so.
As mentioned earlier, the word ‘game’ or the game rhetoric of other locative media or 'mixed reality'
projects has been avoided in the MINDtouch project, since these projects have roots in video games
[71] (apart from Blast Theory, who are a theatre group), computing, video games, interaction design or
HCI. This research is a direct attempt to steer away and separate from other mobile and ‘mixed reality’
games, opting more for this theatre game or participatory theatre approach and perspective. This
choice stems from personal training, but also from my feeling that performance is not properly
acknowledged or understood within the field of mixed reality and pervasive games [72], apart from the
work of Blast Theory [73] and the theories of Brenda Laurel. Thus, aligning with Boal’s work and
approach to using participatory theatre to confront people with themselves is decisive.
In a broader sense, Richard Schechner’s key text Performance Theory provides a more cultural and
historical perspective on performance, in its many varieties and permutations. As a definitive voice in
performance studies since 1988, challenging traditionally held views in the field, Schechner
demonstrated the existence of performance in everyday life, across cultures and history, in play, sports,
legal proceedings, rituals, and in pop culture, in addition to the more traditionally accepted theatre,
music and dance. He has provided the foundation for performance studies by examining cultures and
history, covering all contexts in which humans perform. He borrows much from Sociological,
Anthropological and Archaeological studies, to create an overarching view, still referred to widely today.
While Schechner’s ideas might be obvious to many today, they were not always.
Schechner’s work grounds current participatory practices by expanding the parameters of
performance, which tends not to be acknowledged in most participatory and locative performance
projects. Yet performance practitioners are now crossing boundaries or definitions of performance
through participatory performance and ‘street’ theatre, not to mention video games or ‘mixed reality’
and pervasive media [74]. All of these forms are perhaps not new, but have been reconfigured, using
new and emerging technology.
71 The term ‘mixed reality’ meaning a mix of locative media game in the streets like a mobile treasure hunt with video or computer elements played by different participants interacting with the street counterparts.
72 Plenty within the gaming and interaction design fields use Laurel as their only performance or theatre reference or expert, ignoring much of performance studies, as noticed in numerous articles on locative media (i.e. Steve Benford et al. at the Mixed Reality Lab in Nottingham).
73 Again see more specifically on the project Uncle Roy All Around You by Blast Theory at http://www.blasttheory.co.uk/bt/work_uncleroy.html or read Kate Adams’ piece on the project in the Body, Space Technology.
74 See also the article ‘The Players' Realm: Studies on the Culture of Video Games and Gaming’ (J. Patrick Williams and Jonas Heide Smith, 2008).
80
Schechner has his own categories of the different types of performance that exist:
1. aesthetic = the audience is transformed by the performer;
2. ritual = a subject (or person) is transformed by an official performing a ceremony;
3. social drama = dramatic life events occur right in front of people, transforming them; they are
real but out of reach (for example, watching the planes hitting the World Trade Centre in New
York – on television), so the audience is transformed but helpless, and thus becomes
indifferent.
Then he suggests that participatory performance has another form of transformation:
Some people react by making and/or enjoying art that’s more ‘real’, introducing into aesthetics the interventions and feedback eliminated from ordinary life. Thus, it is no longer strange in theater or performance art to involve the audience directly in the story, to stage actual encounters among people […] (Schechner, 1988, 2003: 194, my emphasis)
Schechner defines what he means by ‘real’ (in performance theory), as,
The actualization of art – the existence of theater combining the social with the aesthetic. (1988, 2003: 195)
Thus, in these two passages, he is suggesting that participatory performance can give participants the
opportunity to give ‘feedback’ or somehow intervene in art more directly, feeling they have more of an
impact and perhaps even empower them when other aspects of their lives might be disempowering or
leaving them feel helpless. The ‘real’ here is then bringing art ‘actually’ into the social lives of the
audience, rather than the power dynamics of ‘us’ and ‘them’ in the traditional staged context. Then,
Schechner goes further on how the audience experiences and receives or is transformed by a
performance: The function of aesthetic drama is to do for the consciousness of the audience what social drama does for its participants: providing a place for, and means of, transformation […] Aesthetic drama compels a transformation of the spectator’s view of the world by rubbing their senses against enactments of extreme events, much more extreme than they would usually witness. (1988, 2003: 193, his emphasis)
This is a compelling statement on the function of the audience within a fictional drama. He suggests it is
like a ‘test run’ for the ‘real thing’, in their minds but not bodies. What I find fascinating about this idea
is that it puts the participant into the action – it becomes real for them, more deeply than participatory
performance. It suggests the fictional dramatisation but puts the ‘social drama’ into the consciousness
in a much more powerful way, so that the real and the performance are blurred. In recent participatory
theatre, groups like Blast Theory have gone as far as staging fake ‘social dramas’, such as kidnappings
[75] to further blur the lines and push the boundaries between the ‘real’ and the fictional. The
75 People actually paid to be kidnapped, see Blast Theory on the 1998 project Kidnap (online), available at:
81
‘performance’ was so real that the actor no longer even knew what the difference was between this
kidnapping and a ‘real’ one, since they were going through all the same actions, except that the victim
volunteered for it. This is a more extreme version of ‘Big Brother’ and other ‘reality’ television shows,
which create social dramas and turn them into a blurred ‘staging’ between ‘real’ and ‘fictional’. We will
continue on the real, the actual and presence next.
3.3/ presence in performance
As a performer, the experience of performing comes in several forms. The two most prominent are: 1)
the common, informal sense of storytelling and engaging with friends, such as describing a setting,
feeling, experience or event to the fullest, most visual and embodied extent, using vocal tone, inflection
and flourishes, gesturing with eye contact; 2) the more formal aesthetic form – performing music,
dance, etc., or teaching or demonstrating in front of a group or audience. This second form usually
takes a great deal of mental and creative preparation, planning, and nervous energy. Understanding
performance in these contexts, and in preparation for investigation of how the sense of presence is
experienced for non-performers, I have noted this from my experience:
Getting up in front of people sometimes causes great panic at first, seeing blank, expectant or judgmental
faces – it is a scary thing, like waiting to be beaten or insulted. Having had experiences of this terror of
negative response, one becomes braver to do it again and again, but the fear returns each time. This ‘stage
fright’ often vanishes after a few minutes or lessens once the performance commences, when a new
sensation often takes over – difficult to describe – a sensation like an out-of-body experience. I lose focus on
the specifics of what I am doing or presenting, or stop thinking and space and time slow down. I just allow
the performance to take over my body and actions. I am ‘in’ the performance, but I am floating in the mix of
watching the audience, feeling the music or words, or my body (in dance), or whatever I am doing, takes
over. It is almost as if I have also become an audience member, witnessing the overall experience and not
fully in my body. It is like a split consciousness – not thinking or dwelling on specifics, just being within it. I
occasionally even disconnect from the activity and place, filled with thoughts or observations that appear
unrelated. It is like I am two beings/people: one in the performance actively (like a puppet), on beat,
message, place, engaged with the performance, and another watching the audience watching me, studying
their faces, clothing, drinks, interaction with each other, the room, etc, with an inner voice making comments
and notes about the moment and what is happening. My body feels split also, as though I am there in front
of them, but also in a trance-like state, subtly removed from it as well. My body takes over and knows what
to do, taking over while my mind is multi-tasking. When this happens the audience can see or shares it too –
it is magic, or a spell, or a charisma that takes over the performer, like my consciousness is actually
‘touching’ the audience emotionally, triggering the intensity.
http://www.blasttheory.co.uk/bt/work_kidnap.html (accessed March 31, 2010). See also this 2005 Guardian article on another artist, Brock Enright, who charges clients $1,500 to abduct them (online), available at: http://www.guardian.co.uk/artanddesign/2005/sep/20/art (accessed March 31, 2010).
82
So am I ‘present’ in that state, place or time? It feels like a heightened state of presence that involves
intermittent absences to reengage and maintain constant presence, with the double-awareness: one in the
body ‘acting’ or performing and responding to the other musicians, dancers, the music, or the computer, or
students, or questions, or whatever else is happening in the performance/presentation space. I am also at
some meta-level of consciousness, observing it all and feeling a sensation of removal or separation from my
body. Strangely, often the more removed I am the better the performance is. So it begs the question of what
it means to be ‘in your body’ or out of it during performance? And what is presence here?
This subjective perspective amplifies the argument that the extended body/mind is a key instance of
presence, or of non-corporeal embodiment. Schechner addresses slightly different aspects of presence
and liveness in performance, considering the gap between live experience of ‘there and then’ in live
theatre on the one hand, and in the non-live event or asynchronous event on the other hand (whether
or not it is recorded and thus preserved as a replayable event):
Theatre, to be effective, must maintain its double or incomplete presence, as a here-and-now performance of there-and-then events. The gap between ‘here and now’ and ‘there and then’ allows an audience to contemplate the action, and to entertain alternatives. (Schechner, 1988, 2003: 190, his emphasis)
In this way, he is referring to differing timeframes. The audience engages and understands the
disconnect between live and non-live events.
Not totally convinced by the argument in Liveness, I went back to Auslander’s earlier work, Presence
and Resistance (1992), to investigate his views on performance and presence, and found it more
relevant. On presence and the audience, he explains,
[...] presence usually refers either to the relationship between actor and audience […] the actor's psychophysical attractiveness to the audience, a concept related to that of charisma. Concepts of presence are grounded in notions of actorly representation; the actor's presence is often thought to derive from her embodiment of, or even possession by, the character defined in a play text [... in] the experimental theater of the 1960s [...] the presence of the actor [was] as one living human being before others […] spiritually and psychologically liberating in itself […] (1992: 37)
Auslander’s definition of presence relates to intensity or the spell cast between the audience and the
performer, mentioned in my experience, and perhaps is what captivates audiences or temporarily
transforms them in the way Schechner describes. To focus on this relationship between the audience
and performer and the implicit power dynamics, Auslander adds, quoting Vera Frenkel, that it could be
defined in a psychoanalytical sense:
[...] charisma as projection ... by which we attribute to others, especially a leader, entertainer or artist the secret images within ourselves. (1992: 43)
He adds that we are moving away from:
83
[...] postmodernist performance [which] seeks to escape presence by working against mutual projection between audience and performer and toward 'a non-charismatic understanding’ […] (Auslander, 1992: 43)
Auslander continues that performers no longer wish to be seen as ‘other’ or put in the implied power
relation with the audience (1992: 43). Following this, the goal of MINDtouch was to break down
barriers and the power dynamics between the trained actor or ‘expert’ with her charismatic ‘otherness’
and the participants; to empower participants as creative collaborators, as co-creators, and not
intimidate them. One might also argue that all people are potential performers and creative beings so
these distinctions need to be blurred.
To segue into a discussion on presence in performance, Auslander made an early attempt to
contextualise technology in performance art. He states that by incorporating technology into her
performance, Laurie Anderson was also incorporating the epistemology of film. In this way, the
performance ‘escaped’ presence through itself in a paradoxical relation to mediation, because the
mediation was both a phenomenon of and a reaction to the performance. (1992: 49) Compelling as it
suggests that the presence of the performer was then missing or removed by escaping. However, it has
become a very common feature in modern performance to have ‘film’ or video projection in
performance and I do not think the audience would say that the performers’ presence is missing or has
escaped through the projection. Auslander developed this position further although less tentatively, in
Liveness, where he added liveness to the mix, although the concept of presence is ‘escaping’ now
through his contentious position on mediation. On this he states:
I have also suggested that performance art's most defining characteristic, its presence or 'liveness' […] is on the verge of being subsumed within a cultural flow that renders the distinction between live and mediated performance virtually irrelevant [… and] the relation of any individual to the medium itself is one of alterity, regardless of whether or not she or he agrees with the message being conveyed. (1992: 79)
Here, Auslander highlights that the disconnect is between the audience and the medium/performance
itself, whereas in later discussions he argues that the liveness aspect of presence versus mediation is
the issue. So, as with the complexity and intertwining of the perceived duality of mind and body, there
is a complexity of duality or distinction, as Auslander states, between liveness and the medium of
performance. MINDtouch has tried to integrate the two and addresses this distinction by integrating
the medium/performance with the participant as audience.
From a feminist perspective, Peggy Phelan’s critical work Unmarked has continually been used as a
contrast to Auslander’s view on the ephemerality of the live versus the mediated in performance.
Phelan emphatically states that […] performance cannot be saved, recorded, documented, or
otherwise participate in the circulation of representations of representations: once it does so it
becomes something other than performance (1993: 146). And yet, she cheapens the experience of
viewing recorded versions of performances, negating the emotional, sensory reception felt by viewers
84
who identify with the performance emotionally, whether live or recorded. If performance has the power
to trigger emotional intensity, as Massumi posits in Parables for the Virtual: Movement, Affect,
Sensation, then does it matter if it is stimulated by a ‘live’ or recorded version (consider how a voicemail
message can equally trigger an emotional response)? It can be said that a live conversation on the
phone or the viewing of a film, can trigger emotional intensity as easily as a live
conversation/play/dance or a live art piece. However, in-person experience allows for a more multi-level
exchange that is not possible in the mediated form (i.e. no eye contact or in a film, no exchange at all).
This exchange is the missing ephemeral piece that Phelan does not even mention, yet this intensity and
emotional experience is key for most interactive and performance art, as well as for effective
communication. Phelan’s intent is to politically empower the visible and real performance, and in her
opening chapter she states:
By locating a subject in what cannot be reproduced within the ideology of the visible, I am attempting to revalue a belief in subjectivity and identity when it is not visibly representable. (1993: 1)
Or put another way, since she sees performance as ephemeral, it is living only in the memory of the
audience, because it can never be truly reproduced and nor can the experience of it be completely
captured or reembodied by media. So she wants us to revalue the ‘real’ of that moment.
Phelan affirms Judith Butler’s point that it is the ‘before’ and ‘after’ that define the ‘real’, and that,
[‘…] representation becomes a moment of the reproduction and consolidation of the real.’ ([Butler]: 106) The real is read through the representation, and representation is read through the real […] (Phelan, 1993: 2)
So, in opposition to more common associations of the real as being something tangible, to Phelan the
real is only this exact moment. She also speaks of the impossibility of materiality and the act of
vanishing, loss and even corporeal death. She tries to convince us, by interpreting science and Lacan,
that there is a disconnect between the seer and the seen, insisting that the real is immaterial or only in
our consciousness or memory. She is adamant that representation in any form is not real, nor can it
ever be. The real is invisible (though this appears to be an oxymoron). Phelan adds that, Performance is
the art form which most fully understands the generative possibilities of disappearance, poised forever
on the threshold of the present, performance enacts the productive appeal of the nonreproductive
(1993: 27). She says that we are never ‘complete’ or ‘here’, so perhaps that is why the ‘real’ is invisible
or non-existent. The real can only be experienced in the moment and is elusive to capture or record.
Thus, the performer and the viewer experience differently, and there is no such quality as ‘real’. This
brings to the discussion the concepts of ‘becoming’ and ‘potential’, delved into more deeply in Part II.
The heart of Phelan’s argument is the supremacy of performance and the impossibility of recording or
documenting it, since it lives only in the experience and memory of the audience:
[…] the real through the presence of living bodies […] there are no left-overs, the gazing
85
spectator must try to take everything in. Without a copy, live performance plunges into visibility – in a maniacally charged present – and disappears into memory, into the realm of invisibility and the unconscious where it eludes regulation and control. (1993: 148)
In relation to performance liveness and becoming, Phelan says:
Performance’s only life is in the present […] Performance’s only being […] becomes itself through disappearance […] Performance occurs over a time which will not be repeated. It can be performed again, but this repetition itself marks it as ‘different.’ The document of a performance is only a spur to memory, an encouragement of memory to become present. (1993: 146)
So these points speak to the performance imperative, ‘You had to be there.’ A realisation from my
research [76], as with other aspects of performance, relationships and communication, is that the
unique exchange of the physicality and temporality of humanness can only take place once, in its own
particular way. All of the unexpected, improvised nuances between two (or more) individuals, each with
their distinctive identities, life history, gender and culturally inscribed characteristics, can only be
experienced in the time and manner that they are delivered. So ideally you have to be there, in-body,
in-person, to experience all these unique, ephemeral and fleeting qualities [77]. However, one could
argue that if the performance is telematic or networked, then ‘being-there’ changes to ‘being online’.
As such, this state of being is the liveness. If the time and modality of presence requirements are
‘there’, in-body or embodying the technology (i.e. using the technology with the body), then one is
extending the body through the network, and this is ‘being there’ and therefore being present.
From my experience as a performer and as a viewer, it can be personally affirmed that the subjectivity
of the viewer – their experience, moods, expectations, physical comfort (or lack of same) – at the
moment of the performance, not to mention that of the performer(s), all play into the irreproducibility of
the performance. This performance research has focused on creating a more personal and private
performance, without an actual spectator – only with interactors. Yet each ‘performance’ is still subject
to so many factors that it can never be duplicated. This ephemerality, ‘now-ness’, also segues into the
discussion of liveness and presence, with renewed focus. I would still argue that this effect or quality of
disappearance can be transmitted virtually, if not ‘reproduced’ through any form or media. And this is
what is compelling about trying to embody technology – the effort of trying to communicate this
essence of performance or presence through a network. If this is possible, perhaps it becomes a third
modality, between live and mediated.
Phelan’s overriding point is that performance eludes mass production, technological mediation and
economic consumption, and that in its ephemerality goes beyond language, designed only for a
defined audience within a confined space and time, leaving ‘no visible trace’ (1993: 148). In a related
sense, Schechner would concur that ‘aesthetic’ performance (and other performance types) has a
transformative effect. Phelan would argue that this effect is invisible, but I would suggest it is not. I 76 As will be discussed further in Part II on presence.
77 This is discussed in more detail in the theory section in Part II.
86
would suggest that performance, if truly transformative (effective and affective), changes a person
inside, and potentially externally (visibly) in terms of their direction or choices in life and views – perhaps
as only a transformative event can, even if infinitesimally. As this discussion proposes, the superiority of the live, ‘here-body’ [78], in-person corporeality for
performance, as well as in other human relationships or communication contexts, is convincing and is
supported by neurological studies (discussed herein) and personal experience. This art research can be
seen as a means to simulate, stand in, temporarily substitute, or half fulfil this ultimate fleshy interaction.
It can be seen as a hybrid interface for people when apart, over distance, during separation, to ‘mind
the gap’ or space between. It has become increasingly necessary to investigate alternatives to face-to-
face interaction, in the late capitalist world, due to Climate Change, volcanic ash and travel limitations
or regulations that force us to stay home and travel less. To maintain global business interactions, as
well as community and personal relations, we need better substitutes and nearly ‘real’ facsimiles.
Alternatively, we should radically restructure the way we live and work – to travel less, to live closer to
where we work, to be closer to our families and friends and/or communities of interest. Both of these
solutions are sure to emerge in coming years. A political stance might be to state that this and future
work are efforts to provide ‘more real’ distance connections. To do this, techniques and tools become
part of the solution – participatory art can act as a means both to encourage a reduction in travel and
to facilitate more felt interaction across distance.
This chapter has been an overview of participatory performance and the issues around performing
presence and liveness in performance, as well as key voices in the discussion, and how they are
relevant to the research. This can be seen as the foundation practice for the other media art practices
layered into this project.
78 A term Don Idhe and Merleau-Ponty use, discussed in Part II.
87
4/ video art + live cinema performance
This chapter discusses the background and rationale that has led to combining the different media
practices and technologies in this project. It covers my video art background that led to this research
and the definition and relationship of live cinema to the work. It demonstrates the connections in my
mind, and the relationship of each practice and technology to each other in the external world,
stemming from earlier practice and heading into this research. It is important to contextualise the
journey that led to this point, especially in video art and performance video practice, and particularly
within the current performance research project. 4.1/ a video art journey
I have been an informal student of cinema from pre-teenage years, watching late-night art-house films
on TV, and going to the last drive-in cinemas in the 1970s/early 1980s (with the speaker box on a
blanket on the ground in a car space). Later, not able to afford to attend film school, I sat in on film
history classes and took summer video production classes, attending every film festival during and after
undergraduate studies in Sociology and Dance. I had further video production and new media training,
and dreamed of making films on video. Still later, during Masters studies, an opportunity to be a
teaching assistant for a film history class came up, so I zealously seized on it and studied for it.
Video production and video art appealed to me, as a cheaper and more accessible option than film,
and the video art philosophy seeped into my thinking on media production, politically and artistically.
More akin to and aligned with performance art [79], video art in Vancouver started as a means of
documenting performance art and became firmly embedded in the gay and lesbian scene, especially
through the works of Vancouver video artist Paul Wong. Wong was a founder of the video movement in
Vancouver and the artist-run Video In Studios production centre (now called VIVO [80]). Eventually,
video art evolved into its own performative form. There has always been a deeply political association
between video production and poverty in Vancouver, which has been the politic of Video In since the
1970s – the mandate of Video In was to enable artists to make video productions comparatively
inexpensively, in exchange for volunteer hours. Video In had a decidedly political position of
empowering those experiencing poverty and a sense of ‘otherness’ (particularly feminists [81], the
gay/lesbian/transgender community, and various ethnic minorities) by involving them in alternative
media art. Video In became a Mecca for those who wanted their stories made but could not break into
79 UK Video Art History is documented online at http://ukvideoart.tripod.com/ (Accessed on May 28th, 2010) and a complete video art history in the western world is discussed in Rees’ A History of Experimental Film and Video (1999).
80 More on VIVO can be found on their website available online at http://videoinstudios.com/aboutus.php (accessed on May 28th, 2010).
81 More on feminism, film and video in Vancouver in the 1960s and 1970s is available online at http://front.bc.ca/research/texts/8 (accessed on May 28th, 2010).
88
the film industry. This is where my media practice started [82] and where I explored and reflected these
politics in my work, either directly or indirectly, as a way to address that marginalisation. Film was for
white, well off, mostly male creatives, and video for everyone else who wanted to make their work.
Eventually I had the desire to be a documentary filmmaker and tackle the competitive world of film and
TV. However, after some effort, numerous marketing sessions and the 1999 Banff Television Festival, I
discovered that searching for major financing through the pitching process could go on fruitlessly for
years. This was not how I wanted to explore the medium, or media and my creativity in general. Just as
I abandoned dancing after I envisioned a life of poverty in low-paid jobs, getting nowhere except
exhausted, I felt this too was not my path. I longed for more immediate access to the resources to
enable production and creative process, with a direct connection to ‘audiences’. Yet, without the
finances (and investing in a music career simultaneously), I instead explored the new frontier of the
Internet. This was a time (circa 2000) when online video streaming was just beginning. Producing
videos online and learning more about the online creative world looked like a path that could eliminate
waiting for financing, and losing control of the creative process. The potential of the web, over and
above video, for communication and creativity appeared fantastic. However, this was a bit too soon (or
late depending on one’s perspective), as neither web audience nor bandwidth were as robust for video
streaming as they are today. Just as I was diving in, the ‘dotcom bomb’ hit North America at the end of
2000 and all of the intelligent, unique, creative Internet projects and entertainment initiatives [83] almost
entirely died within the year (2000). All the risky investment into inventive ideas and creative work went
elsewhere, since many could not find adequate business models to support their creative production.
Thus, online creativity died down for a time, and so I returned to study digital media production, and
continued on to postgraduate studies to dive more deeply into digital media and interactive art. This
enabled me to use my video skills in new ways to communicate and explore innovative ideas with
media and moving images.
4.2/ live cinema
One of my many fascinations, since learning to produce video in the mid-1990s, has been exploring live
performance video – both as an artist/producer and as a curator. This interest was piqued when I held
the position of lead curator for the New Forms Festival (2002 to 2004) [84], featuring electronic and
interactive art and performance. This was an exciting opportunity to research the field first-hand and
82 What is written here is from my own experience. However, a history of Video Art in Canada and Vancouver to support this experience is available online at http://www.thecanadianencyclopedia.com/index.cfm?PgNm=TCE&Params=A1ARTA0008366 (Accessed March 20, 2010).
83 See more on the 2000 ‘dotcom bomb’ in Casey Kait and Stephen Weiss’ 2001 book Digital Hustlers: Living Large and Falling Hard in Silicon Alley, NYC, New York: HarperBusiness; and the movie ‘Startup.com’, available online at http://www.imdb.com/title/tt0256408/ (Accessed March 20, 2010).
84 The New Forms Festival is also in Vancouver, BC, Canada. More information is available online at http://www.newformsfestival.com (Accessed March 20, 2010).
89
become exposed to, among other media arts, the developing art practice of the live performance
video. Since then, while not becoming a VJ or Live Cinema practitioner, I have explored the software
and hardware tools of the practice, while learning about the form from other artists and curators. There
is now a great range and variety of technology options, such as Module8, ArKaos, Resolume Avenue,
FlowMotion, Union, CoGe, C[]NTR[]L, Isadora. Some artists make their own and sell them – as is the
case with Module8 – in visual programming environments like Max/MSP/Jitter or PureData, Quartz
Composer(CoGe) and OpenFrameworks [85].
Live cinema or VJing as a form of video art, has its roots in the rave party culture of the late 1980s and
early 1990s. It integrates video, computer programming and generative, new media tools for sound,
images and performance. VJing, also called Audio/visual and Live Cinema performance bridges:
electronic music, media installation, animation, motion graphics, experimental film and video art in a live
context. It is now slowly becoming recognised as a media art form and performance practice, outside
of the party context, and is moving into the streets, galleries and performance venues, adding site-
specificity to the live component of audio/video performance. RES magazine [86] featured VJ artists in
the January/February 2006 issue, focusing on how they were moving further into the art sphere,
gaining better exposure, recognition and respect, with art institutions now seeing that their work is an
extension of installation and video art. So the art establishment is noticing them more as they expand
their range of works, tools and practices beyond performance and into other digital art manifestations.
This article also discussed how Live Cinema artists are reinventing themselves, exploring new ways to
present themselves to the world beyond the late-night scene (although some still make the most
money and get the most exposure there).
The performance video duo NomIg Collective, or Stephanie McKay and Ed Jordon, curated an event at
the New Forms Festival in 2003, and it really inspired my own deeper exploration into Live Cinema as a
refined visual form (their excellent selection of performing artists was eye-opening). They state their
definition and philosophy of live cinema:
[... Audio/visual performance is] the creation of works where the audio and video components are composed and performed together with an awareness of each other's inherent compositional characteristics. [... Live cinema] requires that neither the audio nor the video serve the role of accompaniment, but that they work together to form a synergy of audio and video […] (McKay and Jordan, 2006, online)
85 I now have an extensive collection of video clips and visual material for my Live Cinema practice.
86 In the Jan/Feb 2006 issue of RES magazine www.res.com – no longer available online (accessed July 2007)
90
Image 7 © 2004 - still from PDX_01 by NomIg (Stephanie McKay and Ed Jordan) live performance [87]
The acronym ‘VJ’ or term ‘VJing’ has become a less attractive label for practitioners, as it carries the
baggage of associations with drug-infused parties, distracting from the art form itself. Thus, many
artists now call it ‘live cinema’, ‘audio-visual performance’ or ‘performance video’. There have been
some stunning works from the Montreal artist team skoltz_kolgen (Herman Kolgen and Dominique
Skoltz) [88], Vancouver filmmaker Velcro Ripper, the NomIg Collective and French Canadian film artist
Jean Piché, as well as US artist Sue Costible, not to mention a great many fantastic British artists such
as Hexstatic, and other global artists/performers, increasing in numbers and artistry yearly. The works I
have seen by these artists were primarily while in my role with the New Forms Festival, although I have
seen several other similar events in Vancouver, Montreal and London before and since.
87 PDX_01, NomIg Collective (2004). Find their work online at www.nomig.net and at http://www.freewaves.org/artists/nomig/ their site says of their work: NomIg.’s artistic interests lie in applying compositional techniques, usually reserved for music, to digital video. […] Together the duo are committed to approaching audiovisual art in terms of what can be learned from the intersections between musical and cinematic points of view on both theoretical and practical levels. (NomIg, 2004, online)
88 Find skoltz_kolgen’s work online at http://www.skoltzkolgen.com/ and http://www.youtube.com/watch?v=isX7IvevM4Y&feature=related. They say of their work, skoltz_kolgen is a plurimedia work cell based in montreal, comprising dominique [t] skoltz and herman w kolgen. Rigorous and raucous creators, their artistic pursuits plumb the integral linkages between sound and image. liberated by digital media they simultaneously assume numerous positions, inhabiting a space between film, photography, audio art, and installation. Architects of worlds, skoltz_kolgen penetrates the ephemeral skin between solid matter and the unsubstantiated, the intimate and the objective, their work conjures bewitched worlds that gestate betwixt accident and intent. (Skoltz and Kolgen, 2008: online)
91
Image 8 © 2004–2008, still from audiovisual performance [SILENT ROOM] film-poème, skoltz_kolgen
According to the Transmediale Festival press release and website in 2005:
[...] the term [...] Live Cinema ‘today stands for the simultaneous creation of sound and image in real time by sonic and visual artists who collaborate on equal terms and with elaborate concepts [... presenting] a much broader conception of cinematographic space [...] The term 'Cinema' is [...] embracing all forms of configuring moving images, beginning with the animation of painted or synthetic images.’ (Makela, 2006: 22)
This definition clearly carves out Live Cinema as an artform and not merely party ‘eye-candy’, as some
still see it. While the audio aspect of VJing is decidedly not used in MINDtouch , I nonetheless
recognise how intrinsic music and sound are in the live performance context.
There are still very few academic voices on this art practice. The few that there are, such as Jaeger
(2005), DFUSE (Faulkner as M/D-Fuse, 2006) and America (2005/7), have not been academically
published. Jaeger attempts a critical discourse about VJ practice and culture, providing a history of
VJing and video performance, describing what the practice involves and what it is not, while placing it
in the larger art and cinematic context. In this way, he creates an initial framework to critique video
performance art. Yet, from discussions with live cinema artists, it has been suggested that they are still
not respected or considered a legitimate artistic force within interactive and media arts, not to mention
the mainstream art world. Attempting to change this are a British academic artist group called VJ
Theory, who generated a collection of articles from contributors defining and discussing Live Cinema
practice and aesthetics [89]. The founders of VJ Theory have collected these primarily through their
website, editing and self-publishing a book that features interviews with prominent global Live Cinema
artists and VJs, as well as recent research on ‘visuals’ performance in the field. These articles have
begun to shed light on the art process and practice, as well as the dynamic nature of this visual
performance form, both for artists and for audiences [90]. Also acting as an entry point to expose and
89 VJ Theory is based in Falmouth, England. More information is available online at www.vjtheory.org (Accessed January, 2009).
90 It should be noted that D-Fuse and ondotzero representatives were recently speakers and performers at the prominent digital design/interactive art exhibition and conference DECODE, at the Victoria and Albert Museum in London. This exhibition ran from December 2009 to April 2010 (conference in February 2010). This inclusion of live cinema artists proved to legitimise their practice and the work of other such artists, as part of digital art (a relatively new form to be legitimised in the art world itself), and within the high art context, for nearly the first time.
92
focus on this form more seriously are the arts organisations Rhizome [91] (US), the British Film Institute
[92] (UK), and the Canadian online journal Vague Terrain.
Well-known Finnish artist Mia Makela is more precise in distinguishing VJing from Live Cinema, as do
others in Vague Terrain’s issue 9, ‘Rise of the VJ’ (2008). She points out that VJing is the act of mixing
material pre-created by others, though sometimes by the VJ themselves. Live Cinema, on the other
hand, is the act of creating visual material and experiences live, in real-time, somewhat spontaneously,
and using custom software or specialised tools, materials and hardware to do so (such as Sue Costible
[93], who uses film projectors, film strips and other materials to create something ‘on-the-fly’).
Images 9 and 10 © 2007 Sue Costabile still from audiovisual performance
In her thesis on Live Cinema, Makela explains:
LIVE CINEMA describes work which is in essence artistic, to make a separation from VJing, which is basically visual DJing. DJs don’t produce their own material, they mix music, the same way as VJs mix already existing material. This does not mean that VJs would not also create their video-clips, but there are many who consider that producing material is not necessary for a VJ […] content is not as important as its usability in a mix. The act of mixing and selecting becomes the work of a VJ. (2006: 23)
So in this context and while it may be obvious to some, the term ‘artist’ is meant to define a person
who creates his/her own visual and performance content and constructs new works from them, in a
unique way to express his/her own creative voice. This contrasts with the VJ, who presents other
people’s material or creates light, background entertainment. However, artists borrow and use other
people’s material at times, and in the age of remixing and appropriation it is a borrower’s unique
91 From the editorial on the Montreal digital music festival Mutek 2009, where a great number of VJ’s and Live Cinema artists perform each year. More information is available online at http://rhizome.org/editorial/2668#more (Accessed January, 2009).
92 The BFI in 2009 and 2010 has been in collaboration with the showcasing organisation onedotzero to host Live Cinema performance events called Dark Fibre with performance group Hexstatic http://www.onedotzero.com/ and https://www.bfi.org.uk/whatson/bfi_southbank/events/dark_fibre_0 (Accessed January, 2009).
93 Sue Costabile’s work available online at http://www.sue-c.net/video.html and http://www.23five.org/archives/suecostabile.html (Accessed March 20, 2010). Her site says. Employing a variety of digital tools to create an experimental animation "instrument," Costabile synthesizes cinema from photographs, drawings, watercolors, hand-made papers, fabrics and miniature interactive lighting effects. Dark, moody, textural, and physical, her live films inherit equally from the kinetic languages of Stan Brakhage's abstract cinema and Nicolas Schöffer's lumodynamic sculptures. (Costabile, 2007)
93
approach and style that often defines a work as art. This can open up a whole other debate on ‘what is
art’, ‘who can be defined as an artist’, etc., which will not be discussed here, as it is outside of the
scope of this thesis.
Images 11 and 12 © 2005-2006 stills from audiovisual performance by Sara Kolster and Derek Holzer [94]
Makela frames a Live Cinema performance staging and set up by stating:
What is needed in order to perform visuals in realtime? ... SPACE, TIME, PERFORMANCE, PUBLIC and PROJECTION, as these elements are present in all live cinema performances. (2006: 6, her emphasis)
She further discusses each of these elements of SPACE, TIME, PERFORMANCE, PUBLIC and
PROJECTION in detail, but she also adds to the discussion on the process of the performance design,
by stating that the construction uses such building blocks as video clips or:
[…] in the case of abstract imagery, the structure can be thought of as variations of rhythms, movements, colors and shapes in order to arouse (emotional) responses in the spectators. Live cinema has more "open architecture" than cinema, so there are more possibilities to build the performance. (Makela, 2006: 54)
There is a relationship between Live Cinema and MINDtouch in terms of liveness and presence in
performance and this is discussed further in Part II. However, important here is addressing the
performance in terms of liveness and presence and involves the ‘realness’ of the experience for the
audience, Makela says:
When we see "Live from New York" flashing on the TV screen, we know that the image is 'real', this is what is really happening. Normally the effect of 'realness' or 'liveness' is enforced by certain "reality effects", like a hand-held camera or even technical problems which makes us recognize the output as more "real" than the carefully chosen, edited and manipulated image
94 ResonanCity 2005-2006 was a Sara Kolster and Derek Holzer performance. Their work is available online at http://www.sarako.net/performance.html (Accessed January, 2009). Their site says of their work: Both artists find inspiration in the history of experimental cinema and electroacoustic music, as well as in contemporary video and microsound practices, and a variety of live sources such as photographic film and found objects are used to generate the visions and sounds […] (Kolster and Holzer, 2006: online).
94
material normally transmitted […] Anything can happen in a live situation and that is what keeps it interesting. Live is connected to real life, it is something happening in front of us in natural time. (2006: 36, my emphasis)
This is crucial to put into perspective my experiences and frustrations in setting up technology in
preparation for a ‘live’ networked or telematic performance for MINDtouch. Similarly, from my
performance experience, much of the sense and expectation from the audience of ‘realness’ or
liveness is because the live performance features the unexpected. The liveness, as Phelan might
suggest, when a performer forgets their move or line or musical notes, or decides spontaneously to
vary the piece, makes it a unique event. Makela points out that in live cinema or telematics, the
technology can fail or aspects of it might not deliver as desired by the artist/performer. However,
audiences want to see the process or mechanics of performance, with all its flaws, and this adds to
their experience. This foreshadows a discussion in the next chapter on the imperfections of mobile
video, in terms of resolution and pixilation, which suggests it is often the very imperfections which give
the performance (or image) its life, a humanness, a ‘realness’ and liveness that audiences find special
and seek out.
Overall, what is compelling is the accessibility and DIY nature of the Live Cinema community:
developing their own software tools along, not regulated by the same tastemakers and gatekeepers of
other forms of visual art (unless on the gallery or academic circuit). However, what has been most of
value in exploring the Live Cinema form, is witnessing its way of ‘speaking’ visually. It is how performers
use images, often spontaneously like a conversation, responding to the audience, unlike traditional
cinema. Live Cinema is like a play of images that dance in your mind when in a semi-awake,
hypnogogic state, mixing with the imagery from half-remembered dreams, while surfacing conscious
thoughts poke through to the waking mind. This notion is what motivates the MINDtouch
performance project. I have very vivid dreams, full of colour and passion, very visceral and full of design
and creativity, often featuring wildly designed shoes, clothes or architecture. So these dreams can be
seen as embodied emotional expressions that I often wish could be shared directly with the world or
recorded, as in Until the End of the World (Wenders, 1990), mentioned earlier. This is how this
participatory performance project was born and inspired. To wrap up, this chapter has looked at my path to video art and live cinema, and how both came into
my practice to help shape this research process. This is important, as it shows how this work could not
be created in the same way by others, due to my embedded or tacit knowledge, making me uniquely
qualified to develop the approach put forth here, as my unique contribution to knowledge.
95
Part II: theory into praxis
96
5/ mobile video aesthetics, non-verbal / non-linguistic
As discussed in the last chapter, the main concept that MINDtouch is based around is a vision of
sharing dream imagery and embodied sensations telepathically, and it attempts to simulate that
experience through mobile media, wearable devices and participatory, collaborative, networked
activities [95]. The aim of this chapter is to discuss: 1) a developing theory for aesthetics of mobile
media; 2) how video can be used as a new form of mobile communication; and 3) how database and
generative video informs and is used in this project. These are grouped together as they deal both with
the making and using of mobile video in this performance work.
Within the context of this chapter and thesis, the term ‘aesthetics’ refers to a visual or performative
expression or artwork that seeks to instill an affective, emotional, sensory, transformational or
pleasurable experience, or creates some form of meaning for the participant and/or viewer. It does not
necessarily subscribe to a specific sense, use or direct history of ‘beauty’ in its use, but rather suggests
that an audience, participant or viewer has a meaningful (to them) response to the work. This use of
‘aesthetics’ is focused especially within the context of video art and performance discussed here, and
influenced by theorists discussed within the introduction and elsewhere, such as Schechner,
Shusterman, Christiansen and Bourriaud [96].
5.1/ the emergent aesthetics of mobile video Theoretically, and in practice, works and research in the field of mobile performance media have been
few but growing, many being locative and ‘mixed reality’ [97] media games. The approach taken by
many of these locative media projects is to focus on urban space, on the environmental and ecological,
as well as on geographical activities using GPS or RFID in tag games. As mentioned in the introduction
and in Chapter 3, most of these projects do not reference or ground their work in traditional
performance practices, studies or history, but mainly come from a gaming or HCI perspective. The
focus for this research has been the emotive, affective perception or experience, and individualised,
visceral experiences of participants, rather than the psycho-geographical or situationist approach
focused on the urban condition or social geography, so popular in recent years.
95 MINDtouch workshop video edits are available online at www.swampgirl67.net in Appendix I in the workshop section.
96 Schechner (1988, 2003: 194); Shusterman (2000: 15-34); Christiansen (2006: 203); Bourriaud (1998, 2002: 84-102); and Benjamin (1929). There are hundreds of books and articles on art and aesthetics, so the definition, philosophy or history of aesthetics will not be discussed here, and while video and film aesthetics are most relevant here, this is also not the thrust of this argument, so it has only been explored on a cursory level and comes more from training in video and new media. Shusterman in particular has been referenced from his theory of the Philosophy of Aesthetics, Performance and Somatics.
97 ‘Locative’ game projects now call themselves ‘mixed reality’, mixing digital gaming interfaces with live performative or ‘treasure-hunting’ type tasks and game play, using mobile devices.
97
Over the years of electronic, digital and networked art development and practice, new media
performance practices using them have evolved into four areas, as identified by various media artists
and theorists [98], some in combination [99]. They are as follows, with the first three addressed in some
way in this performance project (as can be read throughout this thesis):
(1) Telematic performance and events: telematics connect people to people or people to objects
through a network, such as live video steams of performances or collaborations (or even
performance tele-conferencing) across geographic distances (the focus of Turbulence arts
organisation’s Networked Performance blog);
(2) Locative media or psycho-geography: locative media provides a location-aware connection
through GPS or mobile phones (such as Blast Theory’s work, discussed earlier);
(3) Wearable projects: wearables extend the body's senses through technological sensor devices
(such as the whisper[s] and Philips projects mentioned herein);
(4) Active objects and responsive environments: smart environments enable spaces and objects to
respond to environmental changes of state, generated by people within them [100].
The current notion of mobile media involves images, sound or videos created by, existing on, or
received through mobile phones or other portable devices [101].
In MINDtouch, low quality and low-resolution video imagery on mobile phones has been valued for its
immediacy, poor image quality, and pixilated imperfection – its own unique aesthetic. The mobile
phone is also valued in this project for its innate encouragement of spontaneity, like the speed of
thought in the mind, that enables the rewriting, superimposing and remixing of ideas, flashes and
clashes of images and emotion, layering of meaning and stream of consciousness, as well as a
simulation of telepathy and collective, if chaotic, intelligence. For those of us using this new moving-
image medium, the aesthetic of mobile video, as with the early video cameras, is one of imperfection of
image quality. However, in exchange for imperfection we are given an immediacy and a form of
personal empowerment through the simplicity of the device – the pixilated resolution holds its own
beauty and power. Due to its portability and close relationship to the body, the mobile videophone has
an inherent embodied expressivity.
The PhD dissertation As We May Feel: Interpreting The Cultural of Emerging Personal Affective Mobile
Media (2006) by Martin Sonderlev Christensen, is one of the few, if not the only, theoretical discussion
98 See the listserve topic: ‘networked performance’ by various authors (2005a), from soft_skinned_space -empyre- (Online).
99 See the turbulance.org blog topic: ‘network-enabled performance’ by various authors (2005b), ‘networked performance blog’ (Online).
100 An example of this can be found at the website of the company Experia, who claim to be: […] designers, manufacturers and installers of sensory equipment, sensory rooms and multi-sensory rooms. Available online at http://www.experia-innovations.co.uk/ (Accessed May30, 2010).
101 This includes mp3 players, portable digital assistants (Blackberry, Palm Pilot), iPods, and other such devices, sometimes using GPS or RFID components as well.
98
thoroughly exploring the affective qualities of mobile phones through their multimedia capabilities. He
examines the intimate, social uses that have been applied to them and the embodied, personal
expression possible when using mobile devices. For Christensen, affective expression has been
integrated into the device design because of mobile media devices and how users initially used them
affectively, influencing their design, but they:
[…] need in greater extent [sic] to encompass the notion of affect as a design motif. This 'culture of affect' has arrived not because technologies have catered for this in a user-friendly or hassle-free manner, but because users actively and critically have engaged with these technologies, while carrying these into the very centre of their personal everyday life activities. (2006: 10, his emphasis)
Christensen traces the transition of computing devices from static scientific, academic and business
machines to personal, affective, dynamic accessories. He claims that with the emergence of portable
devices, initially in the form of the Sony Walkman, the ‘culture of affect’ began. His position is unique
because he collates many ideas and phenomena from various, often disconnected domains, into one
place. He specifically focuses on the relevance of the multimedia capabilities of recent phones and their
significance in changing from being merely a telephone, to becoming a new media creation and
affective sharing device. He says that in the emergence and adoption of the mobile phone from a
business device to a personal device used in everyday life, it has become part of our lived experience
and personal identity. He adds that,
[…] mobile image technologies are productive in sustaining an embodied relation to the world through capturing in local contexts and sharing online experiences of mundane everyday life relations. (2006: 10)
Christensen ties many elements together to bolster the argument that mobile devices are affective and
social, and one of the more important new media tools and technological developments today,
claiming:
[...] the mobile image device, which allows me to take pictures and send them to others elicits a particular way of being in the world […] to experience the mundane world differently through the device and the activities [...] People take pictures of their everyday encounters […] (2006: 6-7)
and further:
[…] mobile device technologies are reinserting the real world experiences into new media context [...] offer[ing] locative and corporeal embodiment of technology […] that promotes at the same time upholds different relations such as an embodied ‘being-in-the-world’ and hermeneutical ‘reading-the-world’ in one […] (2006: 11)
In these passages, Christensen highlights the view that people are becoming more engaged or re-
engaged with the world and others affectively; they are looking at the world around them more closely,
with the intent of sharing their embodied experience of it. We are seeing the mundane more closely and
richly, with or through the ‘lens’ of our mobile phones (pun intended). I too have found myself engaging
99
with the phone, asking myself: what can I capture (looking more carefully), what can I do with this
moment/image? Who shall I send it to or where shall I put it, on Flickr or Facebook? Who am I sharing it
with? Am I doing this for me or to find a way to connect with others? Christensen articulates my own
experiences, investigations, and observations of participants using these behaviours.
Thus, mobile images are often experienced as personal, intimate and private; once sent, the immediacy
of them feels like a giving or blowing a kiss to another through the network or sharing pieces of yourself
(depending on the content). This is also my experience. So if a text message is like thought transfer,
then an image or video is like sending your sight, your visual experiences, your thoughts and feelings,
as well as your unique expression and perspective.
The final section of Christensen’s thesis was inspiring in conceptualising the mobile phone as a window
or microscope on life around us - helping us to see ourselves in the world more clearly and
communicate/connect this insight outward to others: to find the world and ourselves in it.
Affect then was seen as a way to understand what transpires as technologies become a more active part of the life-world of humans, the central concept being movement, bodily and culturally. (Christensen, 2006: 204, his emphasis)
I mused on the back page of the print copy of his thesis: mobile phone as window or more like
microscope to life around us, helps us to see ourselves in the world better and communicate/connect
this insight outward to others: to find ourselves in the world.
Another mobile video experimenter with similar objectives, Dean Terry, says about his work, a mobile
video project from 2005 called mo.vid.1 :
[… It] is centered on the idea of remapping private spaces into public ones, of reversing scale, of inverting and rejecting the consumerist idea of ‘quality’ and its technological expression in ever higher resolutions by exploiting the limits of the devices. (The videos are really pretty poor with all the compression artifacts […]) (2005).
This reflection echoes my experience of working with the participants in my video collection workshops,
enabling ordinary, non-artists and non-performers to become artists [102] by inverting what is
considered ‘quality’ image construction or filmmaking, using the limitations of the tool and pixilated
resolution as an asset, rather than as a hindrance to image-making. It is the very messiness or ‘glitchy’
quality of the medium that makes it visceral: the tension of imperfection created by the compression
and image artefacts lends rawness to the medium and gives it a feeling of ‘realness’ or ‘liveness’,
making it special. This echoes the way that a live music concert becomes special when the musicians
play the ‘wrong’ key or have audio issues in front of the audience – it is not edited and perfected but
102 This whole project is about bringing art and performance to the non-artist and engaging the public in collaboration and creative interaction, as is discussed throughout the thesis.
100
heard as it comes out. This can add to a concert’s uniqueness and ephemeral quality, distinguishing it
from any other and making it feel special to the observer who has co-experienced that moment. This
again refers to the argument in Chapter 3, to Phelan’s points on the ‘real’ and to Makela’s arguments
on imperfection lending a sense of liveness. I would extend this to mobile video, especially factoring in
its affordability and innate performativity, discussed next.
Key aspects of mobile video making observed within this research process [103], both with participants
and in my personal explorations include these four features:
1. innate performativity, or movement or gestural qualities encouraged by the device size;
2. the portability factor (the ability to watch or shoot anywhere), with the mundane becoming fascinating and easy to capture;
3. the phenomenon that users view the world through the camera ‘vision’ of the phone screen, rather than their own eyes alone, adding novelty, but also a re-engagement with the world;
4. the innate intimacy of expressivity fostered by the device and predilection for doing close-ups – framing only that which is intended to be seen.
To exemplify the performativity and intimacy aspects of the mobile recording practice, Terry (see his
work later in the chapter) states that the phone encourages one:
[…] to project [the] very private space immediately surrounding the body into meta-space. Many of the videos show objects little more than a few centimetres beyond the tiny lens, often some body part, like hands or forearms that obscure an unknown, overexposed background space. Other pieces are gestural performances, recording the movements required when following a line, or when trying to create shapes by moving the camera in certain ways (2005, my emphasis).
Exactly in this way, the participants in my workshops tended to capture their experiences and explore
the ‘meta-space’ of the body as the main focus of the mobile medium. It appears to be a common and
instinctive approach one takes to mobile video capture. The device inherently encourages movement,
resulting in blurry, abstracted patterns, and a ‘splattering’ effect through gesture. This intimate
approach to mobile video production is facilitated or afforded by the device size itself. Unfortunately,
the manufacturers will inevitably be adding more image stabilisation and anti-shake technology to
compensate for this, removing much of what makes these media creations special. When participants
in the workshops used and worked with the phones, it appeared that their immediate impulse was to
wave the device around, as if it was part of their hand, blurring the images intentionally to see the
patterns that resulted. Thus, it inspires a playful, gestural or performative exploration, not suggested,
but observed. This is intriguing since it is to be expected when using bigger, more professional video
cameras (even consumer home-movie camcorders), that shake stabilisation is used and effort made
NOT to blur the image. Yet, a new shooting aesthetic may be unfolding, resulting from the very size of
the device and how it lends itself to movement and gesture. The performance aspect of this project –
combining video and mobile phone and working with untrained participants – benefits from this
103 See observations in Appendix I in the workshop section for more on this.
101
inherent aspect of the device.
From the perspective of aesthetics and intimacy with the device, during the FILMOBILE conference
[104] in April 2008, video artist Steve Hawley presented his mobile cinema artwork and discussed
how, with the advent of HD High Resolution (and now 3D) video cameras and televisions, the world is
now shown to us in a larger than life ‘reality’ and quality, creating the illusion that we are somehow
getting closer to the world, more intimately close to places or things we could never otherwise get
close to or see clearly. However, he pointed out that in this way, HD and 3D present more of a
spectacle of the world, a world which most of us cannot get close enough to except through these
professionals. In this way, it keeps the medium and its presentation firmly in the hands of the ‘expert’
filmmaker or videographer who have more access and training in the medium. The images and work of
the high-resolution medium are usually in the National Geographic style of close-up, telephoto shots of
creatures in the Amazon forest or of huge vistas in a faraway, inaccessible landscape. Meanwhile,
Hawley suggests that the mobile phone, with its poor, imperfect resolution, pixilated/compressed
quality and small aspect ratio, is in some ways, more ‘real’ [105], in that everyone has access to the
medium and can learn it easily, resulting in more imperfect, but more personally meaningful, images.
More importantly, it captures ‘what I am doing now, in this moment’, and has a personal immediacy
that the imperfection of the image actually authenticates, through its lack of production values, or high
resolution, clean editing, and large aspect ratios. It is a more intimate and everyday life document,
which the ordinary person can relate to, and hence, feels more ‘real’ to them as it is imbued with
memory and emotion. Through its messiness, mobile video at once encourages a more personal, non-
expert, ‘every-person’, frank expression, not possible with the HD or 3D video image.
What has become evident in this research using mobile phone media features, is that the portability
factor enables users to explore the world around them differently – as Christensen suggests, with ‘new’
eyes – or to see differently and observe the details of the world more closely. With the ability to watch
and shoot anywhere, the mundane becomes compelling: a way of viewing the world through the
camera vision or through the ‘lens’ of the phone. Thus, we are becoming more engaged or re-engaged
with the environment, looking at the world around us more closely, and sharing our experience of it. I
have witnessed people recording video or taking images of the world spontaneously, as they encounter
it, almost randomly, and perhaps seeing differently. The phone becomes a ‘window on the world’ that
has users returning to truly observe the world in order to capture it, and then finding details that they
might not have otherwise noticed along the way (Christensen, 2006: 6-7). In this way, not only are they
learning how to be amateur digital videographers, but they are also re-engaging and interacting with
their surroundings more deeply, in a way they may not otherwise do if merely experiencing a mediated, 104 FILMOBILE is a network project developed at the University of Westminster, bringing together the mobile phone industry, filmmakers and artists working with mobile devices. In April and May 2008, FILMOBILE organised a major international event consisting of a gallery exhibition, cinema screenings and an international conference. This event explored the cultural and economic impact of new mobile technologies and initiated debate between artists, the media and the new mobile industry. The emphasis of the exhibition was on mobile cinema, but there were audio and photographic pieces, as well as MMS and SMS, and animated projects in the exhibition, of which I was a part. Information is available online at http://www.filmobile.net (Accessed March 20, 2010).
105 This sense of ‘real’ refers to a feeling of being close or true to everyday lived experience.
102
2D sense of the world.
The portability factor also encourages a high degree of intimacy – one can take one’s phone to bed, or
to the toilet, or to places where cameras often are not allowed, communicating much more personal,
private or hidden imagery than standard video cameras enable, due to the size and ease of
concealment of a phone [106]. This encourages complexity and a freedom and uninhibited quality in
what one can represent to others, with an immediacy and accessibility that enables participation and
intimacy unlike other media devices. This can be seen in the video that participants made, with close-
ups of areas in the rooms into which other cameras might not fit, or footage of the insides of mouths
and such. So here, when ‘intimacy’ is used it is with reference to a closeness to the body, and a sense
of personal and private engagement or exchange [107].
In the MINDtouch workshops, participants were asked to try to find imagery that represented, in an
abstract or literal way, how they were feeling, perceiving or sensing their own bodies, in a way that
could be communicated to others. This enabled more freedom, and perhaps gave permission to
people to explore and express themselves in less censored or guarded ways, and their visuals became
highly intimate, interpretive and subjective. What resulted, since most had already been exposed to a
vast amount of media daily, was that even without a cinematic language or training, they inherently
knew how to express themselves and create imagery, with instinctive framing and composition, using
the device [108].
Below are some examples of other projects. While not all come from the same motivation, all have
embraced the low-resolution, small-screen aesthetic fully. The work below by Steve Hawley was
striking because he both explores the small frame and plays with the frame in new forms of
composition. He also examines the aspect of ‘being here now’ and the possibilities of surreptitious use
of the phone. He also takes advantage of the low-resolution pixilation as a painterly nature of the
medium. He says the work is:
A 3 minute video, shot on a mobile phone. The stuttering low-fi images, collaged together, look like the very earliest television. (2004:online)
106 To demonstrate this point, at the FILMOBILE Exhibition in 2008 in London (see footnote 104), one of the artists had a set of mobile images and videos that she had recorded in prisons, which would not have been possible to obtain with a regular video or photo camera.
107 This is discussed more deeply in relation to Schiphorst’s performance methods and the whisper[s] project (in Chapter 2 of this thesis).
108 With the exception of course of those videos that were just pure movement, but even those more often than not were capturing patterns.
103
Image 13 © 2004 Steve Hawley – still from Speech Marks [109]
The work of video artist Mark Amerika surfaced numerous times during the research. His approach to media
art creation can be seen to parallel that employed in this research project, as he has translated the mode of
his practice from performance to video to mobile video. Amerika’s multi-form digital art piece Mobile Phone
Video Art Classics (MPVAC) [110] mixes and reconstitutes various art personas and artworks to create a
new narrative, using mobile phone video scenes in an attempt to bring forth ‘spirits of the past as well as
hauntological actors of the present’. This piece engages in an effort to re-edit art history using mobile phone
low-resolution video as its primary mode, and edited using the iMovie tool (Amerika, 2008: online).
Image 14 © 2008 Mark Amerika – still from Mobile Phone Video Art Classics
109 Steve Hawley, Speech Marks (2004), available online at http://www.artdes.mmu.ac.uk/profile/shawley/image/10349 (Accessed March 20, 2010).
110 Mark Amerika, Mobile Phone Video Art Classics (2008), available online at http://www.eaf.asn.au/2008/amerika.html. Artist's website and blog available at www.markamerika.com/ (Accessed May 6, 2010).
104
Mark Amerika’s work is important to the medium and genre of video art because of his approach as a
prominent VJ. His methods of creating performance video on mobile phones is relevant because he
takes advantage of and uses the low-resolution, pixilated quality of mobile video and explores how it
can expose and push the painterly qualities of the video image aesthetic to the forefront. He recently
produced a mobile phone movie project callled Immobi l i té. He promotes the project, proclaiming it
is,
The world's first feature-length mobile phone art film […] Amerika's Immobilité mashes up the language of ‘foreign films’ with landscape painting and literary metafiction. The work was composed using an unscripted, improvisational method of acting and the mobile phone images are intentionally shot in an amateurish or DIY [do-it-yourself] style. (2009: online)
Image 15 © 2009 Mark Amerika Immobilité
An arrogant claim, since I saw a feature-length mobile art film at FILMOBILE in 2008. However, it is
exciting that Amerika uses improvisational approaches with mobile video, using a similarly low-tech
approach with the remix [111] to my own. His marketing text adds:
Mobile Remixes […] Immobilité director Mark Amerika introduces the concept of remixology. In Amerika's version of art-life, we are all born to remix. What are dreams and active memories if not personally rendered remixes of multi-media source material? (2009: online)
Dean Terry’s database approach [112] is one that was adopted by my project initially as a means to
assist in storage, retrieval and mixing of mobile videos. It is curious that many of us working with mobile
111 Although I thought of this in 2006, before starting this project.
112 Dean Terry, mo.vid.1 (2005) video paintings, from a camera phone, are available online at www.100lies.com and http://www.deanterry.com/blog/index.php/2005/10/ (Accessed May 6, 2010).
105
art have similar approaches to working with this medium in a mobile video creation/display form. He
says of the work:
mo.vid.1 is a mobile video project that updates every time a video is sent from my mobile phone to the gallery, often several times a day. The display changes as new videos are sent from the phone, through wireless networks, and to a computer in the gallery. (Terry, 2005: online)
Image 16 © 2005 Dean Terry
and:
The project is centered on the idea of remapping private spaces into public ones, of reversing scale, of inverting and rejecting the consumerist idea of “quality” and its technological expression in ever higher resolutions by exploiting the limits of the devices. (2005: online, my emphasis)
This point is key when working with video with participants, as I learned in the video collection
workshops, as it allows ordinary people to become artists by inverting what is considered ‘quality’ and
using the limitations and pixilation as an asset or tool, rather than a hindrance. What is fascinating
about Terry’s project is that even though it is a few years old, he was observing the performative
aspects mentioned earlier in this chapter. Very few have written about this gestural element or the
innate performative features that were either an afterthought or accidentally afforded by the very nature
of the mobile device, but are perfect features for creating mobile video art. He adds on his work:
[…] This project is atomistic and performative. The bits (individual 15 second videos) pile up over time, and the composite, the themes, are the paintings – collections of idea streams, are captured in development, in process. This is a real time art piece, and so the themes are being worked out over the course of two months. (2005: online)
106
Max Schleser uses mobile filmmaking [113] as an alternative approach to the film and documentary
practice. He claims to provide a new definition of documentary while creating a new aesthetic
approach to feature-length, non-narrative documentary filmmaking.
Images 17 and 18 © 2006 Max Schleser – Max With a Kaitei
In his own words Schleser describes one of his projects:
The city film Max With a Kaitei, which was produced entirely on a mobile phone in Japan in 2006, depicts the new emerging pixel aesthetics […] The experimental feature documentary is juxtaposing the advanced Japanese mobile video format, which is based on the mpeg 4 codec, with the former 3gp compression format. (2009: 2)
His new form of documentary is not only shot on mobile phone, it is more abstract than most common
documentary and specifically uses the digital aesthetic and pixilated resolution as a feature of the
medium.
Giselle Beiguelman’s project sometimes always, sometimes never [114] was the first piece of the
mobile art form I encountered. It made a huge impression on me and inspired me to make
MINDtouch in some ways, especially the collaging/remix aspect (see images below), even though it is
totally different in outcomes. She says on her website on the project:
Sometimes Always, where the audience also sh[o]ots images with mobile phones to be deconstructed by the same algorithmic process [… and] consists of an interactive projection, based on generative systems, which allow the audience to sh[o]ots images with cell phones with video cameras and send them via Bluetooth to big screens. On mouse over, the videos fragments in frames can be reorganized by the interactors, following the movements and draws they do with the mouse. (Beiguelman, 2005: online)
113 Max Schleser, Max With a Kaitei (2006), available online at http://www.filmobile.co.uk/ (Accessed May 6, 2010).
114 Giselle Beiguelman’s sometimes always, sometimes never (2005), São Paulo, is available online at http://www.desvirtual.com/sometimes/never/index.htm and sometimes always video http://www.desvirtual.com/sometimes/always/video.htm and http://www.desvirtual.com/sometimes/never/video.htm (Accessed May 6, 2010).
107
Image 19 © 2005 Giselle Beiguelman, screen shot from sometimes never
And:
Sometimes Never is a (de)generative video which is decomposed through the inputs of its interactors. Videographic images, shot with mobile phones in the exhibition space, can be manipulated by keyboards and mouse and the audience edits, in real time, the order of its original frames, their position on the screen as well as introducing colored filters on the new images. When someone leaves the mouse, the original film restarts over the layers built by the interactors. (2005: online)
These companion pieces were a revelation to me and helped me to visualise the output of the mobile
video collage/installation in the live collaborative video performance space, on a larger screen.
Sometimes Never helped to conceptualise and explain to others the potential visual performative
installation in a live setting.
Images 20 and 21 © 2005 Giselle Beiguelman, screen shot from sometimes never
108
When I started the MINDtouch research, using body knowledge and impulses to interface with mobile
video technology as a means of alternative communication, I was aware of others concurrently working
in similar domains. Tina Gonsalves is an artist whose work is similar in intention, nature and outcomes
to mine. Her site says:
[… Gonsalves] has used the fluid and malleable medium of video and interactivity to explore complex emotional landscapes. Rich, painterly video abstractions evoke intimate associations with personal space. […] she attempts to enrich the public understanding of the hidden emotional language of the body. Using embodied interactive experience, she attempts to explore new ways of experiencing the internal body and the external environments. (2006: online, my emphasis) [115]
She infers that video abstractions are capable of evoking intimate associations in viewers in a way that
is akin to dream visual material and experiences. Video art is a perfect medium to explore the hard-to-
define-and-describe emotional content of mind–body sensations and perceptual experiences. It is
important to find ways to explore the “hidden emotional language of the body and continue to
investigate new ways of experiencing the internal body through external environments. The mobile
phone, in its affective, immediate, intimate and portable nature, has the built-in capacity to externalise
the internal in very personal and individualised ways for each user.
[… FEEL.TRACE] is a psychophysiologically responsive video installation synthesizing art [… It] explores new, more embodied languages of interactive and emotional communication, investigating the inter-relationship of the internal body and the external world […] Sensors begin to diagnose patterns of internal arousal states of the body, picking up how the body is responding to the imagery. (2006: online)
Image 22 © 2006 Tina Gonsalves – FEEL.TRACE
115 Her work can be seen online at http://www.tinagonsalves.com/ (Accessed March 20, 2010).
109
In FEEL.TRACE, Gonsalves’ focus is on exploring the mind/body as a much ignored means of
communication and expression, and the idea of our embodied and emotional responses speaking via
visual imagery:
Bodily reactions of the viewer continue to trigger changes in the projected video and audio content. The video footage constantly adjusts in response to the viewer’s internal state (creating a bio feedback dialogue between the image and the participant). The software reads the responses of the body, making a choice to either calm the viewer or further stress them out. (2006: online, my emphasis)
Gonsalves’ approach of having the video content adapt to the bodies of viewers and their internal
states is inspiring.
MEDULLA INTIMATA is a necklace that contains a video screen and biometric sensors. The sensors […] monitor the wearer's emotions to guide real-time video-generation that evokes a sense of seeing beneath the surface of the skin, exposing the emotional and physical inner body. (2004: online)
Image 23 © 2004 Tina Gonsalves with Tom Donaldson – MEDULLA INTIMATA
The imagery displayed is an emotional portrait, an exploration into the secret life of the emotional, physical, spiritual and psychic body of the wearer. (2004: online)
It is curious how the jewelry managed to “expose internal thoughts and feelings” of the wearer. There
are compelling parallels between Gonsalves’ philosophies and approach to her work and my own. She
uses body data to explore video imagery and trigger emotionally charged content, to affect the
visitor/participant in particular ways. She is equally concerned with the connection between visual
images and how the body responds, and vice versa in a feedback loop. She also employs wearables
and biosensing technologies to tap into this relationship between video and the emotional or
110
physiological. She also uses wireless, portable devices to unearth new methods for artistic discovery in
participant interactors. She integrates the spiritual and consciousness dimensions, exploring various
approaches to interweaving these through her projects. While I discovered her work well after starting
my own and was not influenced by it, it was rewarding to find a parallel researcher with similar
concerns.
5.2/ non-linguistic mobile video communication
My interest in non-verbal, visual communication for this research was increased when I realised, during
mind-quieting activities in the video collection workshops, that participants were repurposing the
phones. This was evident when conducting the body sensation visualisation and telepathy-simulation
activities with participants. When they were asked to express their perceptions with videophones,
instead of using the dominant modes of use of text and voice, what resulted were ingenious ways of
communicating through the imagery. Thus, the focus shifted from just making embodied video
expressions for the performance video to helping people to communicate non-verbally in order to
express body perceptions visually, and using their senses rather than relying solely on words. Video is
the chosen form, but now the mobile phone video camera allows one to repurpose the phone to
visually express emotions and perceptions experienced internally, rather than it being merely a device
for documentation or an entertainment gadget.
Murtagh noticed non-verbal behaviour in his research on mobile phone use, and saw that:
[...] the focus on bodily gesture and eye contact emerged from the data when it was noticed that activities with and responses to mobile phone use were almost invariably non-vocal in nature […] Yet, so much of mobile phone use in public is organised through non-verbal action and interaction. It is suggested that these non-verbal aspects of phone use display the 'unwritten rules' of usage behavior in public. (2001: 82)
Non-verbal communication normally means using eye-contact, physical gestures, body language, tone
of voice and other physical indicators to express certain information to another person, to underline a
conversational exchange and make a point more clear [116]. Sometimes it is a mode of communicating
the unspeakable, before verbal communication and introductions have been established, or as a
means of initiating contact, or to express something intimately without the need to verbalise the
sentiment. We all use non-verbal modes of expression and learn them at a very young age (possibly
before we are able to speak). In this work, since participants were asked to use videophones and
imagery to speak for them, they had to draw upon the visual material in their environment, as well as
from their own sensations, perceptions, thoughts and emotions, as their vocabulary. It also aided them
in using their prior knowledge or experience of visual language to find ways to create a language or 116 Based on 'non-verbal’ communication research and techniques (Argyle, 1975: 243–260).
111
visual vocabulary from the raw visual and sensory materials, and to construct video ‘sentences’ or
‘utterances’ into something communicable. This falls outside physical or body language and is more
akin to a cinematic language [117] of video art, abstract cinema or Live Cinema (previously discussed).
Thus, participants desirable for the workshops would be people untrained as ‘artists’ or ‘performers’
(some participants did have creative lives, but usually in a different form). The only training participants
were expected to come with was that of the regular exposure we all have to a daily stream of
advertising, television, Internet, films, games and entertainment media. This exposure is considerable
for most, enough to establish an equal starting point for engagement in the project activities. Even if
some came with a more intuitive talent for visual composition, or a natural (or trained) eye for
aesthetics, all were able to create something exceptionally poetic. My principle has always been that all
expressions are valid, since each is someone’s personal expression, from their own unique
perceptions, represented in their own unique ‘voice’, just as language allows.
This exploration, however, was not about teaching or developing a new or specific vocabulary for
participants to express within, except: within the limitations of the device; their comfort and knowledge
in manipulating the device; the limitations of the available environmental visual material; the participants’
ability to create or use the material to represent their emotions, perceptions and sensations or
experiences. AND, within the structured activities they were guided through to find ways to express or
record patterns and other visual elements. In this way, certain vocabularies or syntax of representations
of internal experience could emerge. If more workshops were conducted, with more variety of
structured improvisation activities for different types of expression, a symbolic language of non-
linguistic video expression could be developed, but that was not the original aim of the project,
although it could become the aim of future projects.
Thus, non-verbal communication emerged here, through gestures and the physicality of participants
and in their exploration of representing their internal experiences through the external world visually,
using movement afforded by the size and portability of the mobile phone. This is not the definition of
non-verbal communication generally used, and does not require in-person presence or liveness in other
ways. In fact, that is the point: this is a new, digital and remote, visual mode of non-verbal
communication (I posit) and through the absence of the other (in-person), the concept of presence
comes back to the foreground. This mode of non-verbal expression is like a mobile, democratised form
of live cinema, more of an exchange or conversation than a performance by one artist or expert for a
passive audience: it is an active, emotive interchange.
What is unclear, and also an area for future exploration, is how the receiver experiences and interprets
what is communicated. Does direct, literal understanding matter? Yes it might, if we are to make a new
semantic language. Nevertheless, it can be argued that, like in a dream or an artwork, the
receiver/viewer/dreamer has the freedom to interpret a message any way they like. In some cases, the
117 It is worth mentioning ‘non-verbal films’, like Baraka or Koyaanisqatsi, which are visual essays communicating their messages purely through imagery or cinematography.
112
imagery is clear enough that the communication is evident and obvious, as with any form (see Image
24 below [118]): the way the communicator constructs the message will either help or hinder its
interpretation and how it is decoded. Future research projects could look at making a more specific
visual ‘alphabet’ and grammar, but that is not what was attempted here. The aim was merely to give
people the tools and a loose structure, to help them to create a language of their own. This could be
developed further into a more structured language (for future uses with the device), and could prove
useful for people with real physical, verbal or linguistic limitations. This study, however, is an exploration
of the possibilities of expression of personal experiences using the mobile phone, allowing the average,
non-artistic user to be playful, creative and expressive in an intimate and novel way [119].
Image 24 © 2007 C. Baker – still from participant video during Dublin workshop, October 2007
The next few pages feature technological art works that employ non-verbal, wearable sensing and/or
mobile or other virtual connection technologies to communicate across distance.
As mentioned in the methodologies section in Chapter 2, I was involved in the whisper[s] project
[120] from 2003–2005 as a researcher/coordinator, working with artists/researchers Thecla Schiphorst
and Susan Kozel. My role involved research in performance, wearables, experience design, biosensor
testing, prototype garment construction, movement development with wearable devices, participant
118 This image is an example in which one of the participants’ videos was very clearly communicating her frustration of being stuck in her wheelchair after previously being an able-bodied dancer – of course only I know this (or those in the workshop), since I know her and knew she couldn’t walk down the stairs. And so, perhaps this was/is only obvious to me as the researcher, and anyone else I may have shown it to (maybe it was a message to me).
119 Johansson explains Serres’ theory of ‘The Five Senses: Philosophy of Mingled Bodies’, which says that tactility is ‘the most fundamental sense of the human soul and its experience of itself and its environment’ (2003: 143, paraphrasing Serres), and that this can be extended to how we experience images. Images are tactile if taken in one’s hands in a literal sense. However, Serres continues that tactility encompasses all senses and that photography and cinematography are physical impressions of the referential world or a ‘true image of reality’ presented on a ‘membrane’ or skin (2003: 144). Thus, […] a tactile image being a medium of (physical) impressions and emotions, of impressions of the physical world and their implications for sensation. (2003: 144–145)
120 This project is available online at http://whisper.iat.sfu.ca/ (Accessed March 20, 2010).
113
workshop development and assistance, as well as website design and maintenance, videography and
video editing.
The project’s main aim was to explore how embedded, wireless devices in clothing might create a
‘body area network’, in which bodies and people could exchange their own body data – that data
could then be seen or felt by others within the clothing, through various actuators or outputs, or vice
versa. Below is a better definition by Thecla Schiphorst:
whisper[s]: wearable, handheld, intimate, sensory, personal, expressive, responsive system. whisper[s] is a collaborative project involving artists (dance, sculpture, music), designers (of visuals, objects & textiles), computer scientists and hardware/software engineers. We are developing technology and communications metaphors that enable networked wearable devices to communicate affective states in a continuous manner. (Schiphorst, 2005: online)
Image 25 © 2005 – whisper[s] project screenshot from a video I shot at SIGGRAPH conference
whisper[s] is about finding new modes of embodied communication and exchange and a playful
means of incorporating fashion, gesture and movement, media installation, expressive devices and art
in new and compelling combinations.
Image 26 © 2005 – whisper[s] project, showing the electronics, conductive fabric and designs
114
The whisper[s] project had significant support through Canadian arts and technology funders due to the
innovative nature of the work, and it presented at prestigious technology exhibitions (Siggraph 2005), and
conferences (several CHI conferences), and iterations of clothing and technology. Beautiful garments were
created for it and many participants played with it. When I left the project in 2005, the technology had just
begun to ‘work’ in terms of body data consistently being exchanged between participants through rfid tags,
and represented both on led lights on the clothing and through triggering mobile phone sounds, as well as
other visual representations and installations.
Image 27 © Phillips Design, Skin Probes
Skint i le: Electronic Sensing Jewelry has been conceived alongside a European project, STELLA [121],
developing stretchable, flexible electronic substrates that integrate energy supply, sensors, actuators and
display. [122]
Skintile the Electronic Sensing Jewelry further explores emotional and physiological sensing [… using] wireless, stick-on body sensors that re-define traditional body adornment […] It explores a range of functionalities in new product forms that are playful, sensual, mood affected, bio activity stimulated, and arousal enhancing. It is a semi disposable, bio compatible, non-allergenic, breathable, mass customizable, self-contained body worn accessory. (Philips, 2008: online)
121 More on STELLA is available at www.stella-project.de.
122 See the Design Probes website, available at http://www.design.philips.com/probes/projects/electronic_sensing_jewelry/index.page (Accessed May 6, 2010).
Like whisper[s], Skin Probes focus on
finding new, embodied, playful or sensual
means to communicate emotions and
sensations non-verbally to others. An
interesting aspect of this project is that it has
been created by a large corporation. The
website states:
As part of SKIN, we have developed two ‘Soft Technology’ outfits to identify the future for high tech materials and Electronic Textile Development in the areas of skin and emotional sensing. The dresses show emotive technology and how the body and the near environment can use pattern and color change to interact and predict the emotional state. (Philips, 2008: online)
The various manifestations of the projects are
very intriguing:
115
Image 28 © Philips Design – Skintile
What is exciting in this research is the cable-free and aesthetic way in which these devices or ‘jewelry’
pieces sit on the body. This project is merely a conceptual (proof of concept) prototype. Yet, if they were
functional, that would be an exciting achievement.
Bare Conductive [123] ink was recently awarded a prize at the prestigious Ars Electronica electronic art
festival for 2010; it uses another exciting form of non-verbal communication technology, drawing
people back to the body as the primary mode of exchange and interaction.
Char Davies (1995) Ephemere
To use biometric medical devices with mobile phones is not new, as can be seen by Gonsalves and
whisper[s] above. However, the intention is to work with the ‘body as interface’ from an embodied
paradigm for media interaction, as well as communication exchange in a non-linguistic and visual form.
Char Davies is one of the most interesting artists to use biometric devices and new media in her
famous VR piece Ephemere (1995) [124], in which participants could control their movement with their
breathing within the virtual space. Davies had participants explore within a beautiful, abstract immersive
3D virtual world, using the body as a mouse. To experience the artwork she painstakingly crafted,
participants navigated through it engaging the whole body in a fully visceral way. The work affected
participants very deeply, personally and intimately. MINDtouch similarly facilitates the participant to
create and share their own perceptual narrative, triggering media created by them, through their body
experiences, sensations, memories, associations and imagination.
123 More on Bare Conductive is available online at http://www.bareconductive.com/ (Accessed May 30, 2010).
124 See more on Char Davies’ work online at http://www.immersence.com/ephemere/index.php (Accessed May 30, 2010).
116
Image 29 © 1995 Char Davies, Ephemere
Kathleen Rogers (1995) Psi Net
Terms such as virtuality and wireless communication can be applied to telepathic communication, and
incorporated into new practices and theories of communication and culture. This was a project I
discovered and I found unique and compelling during previous research.
Kathleen Rogers’ piece Psi Net (1995) [125] piqued my attention because her work was the first, and
one of few, that addressed concepts of telepresence and cyberception. These notions were coupled
with medical and military technologies within the context of a parapsychology experiment in her
performance piece. Psi Net compared telepathy and electronic virtuality, mediumship and
electromagnetic technology with wireless networking, by studying and recording participant responses.
Rogers created a pseudo-science experiment, from concepts and discoveries in physics, psychology
and parapsychology, delving into the unknown and controversial for artistic and theoretical ends.
Rogers tapped into similar issues of embodied, ‘virtual’ abilities or innate ‘personal energy’ (as she calls
it), by exploring ways in which technology can enhance or facilitate channelling of this energy, in a
clearer manner.
125 Kathleen Rogers, ‘THE PSI NET PROJECT: Psi-Phenomena - Telepresence – Virtual’ (1995). Available online at http://web.ukonline.co.uk/Members/kathleen.rogers/index.html (Accessed May 30, 2010).
117
Image 30 © 1995 Kathleen Rogers, Psi Net
My work continues to be focused on aspects of telepathic communication and its nature as a model for
other types of communication. Rogers had these same pursuits and explored new ways to use
emerging technologies to assist in the exchange between humans, in embodied (or, in Rogers’ words,
‘telepresent’ communication) and natural ‘networks’. In her project Psi Net, she recorded the ‘personal
energies’ of the participants, by recording three days of three-hour-long, live, remote viewing and
mediumship experiments, using highly sensitive thermographic optical scanning technology,
biofeedback heart monitors, and voice recordings. She then transmitted them live in the nearby area,
via a microwave link.
Rogers was less concerned about the participants’ experience and more concerned with the
telepresent images that were produced from this experiment – a fundamental difference in our
concerns. For my practice it is vital to have both in tandem, informing each other. Her work, however,
speaks to my interest in dream studies, parapsychology, neuroscience, studies of brain functioning and
consciousness.
5.3/ generative and database video
In Making Art of Databases (2003), mentioned in the introduction, Anne Nigten introduces the artists
and the master classes discussed in it. She explains that there are two forms of database and archive
art project: 1) projects with predefined and domain-specific applications, with content familiar to the
maker, and outcomes representing ways to configure and select the content; and 2) dynamic
environments and applications that are created through participant interaction, based on rule sets or
learning-based decision-making, using a wide range of parameters (2003: 7). Nigten indicates that a
combined approach, connecting the two different types, could be interesting to consider for projects
where the interpretation of a wide range of parameters and non-linguistic inputs needs to lead to some
118
sort of coherent environment (2003: 7–8). MINDtouch partially has this combinatorial approach,
although this is not in order to produce a ‘coherent’ environment. On the contrary, abstraction is very
desirable in this project as an emulation of dream experience.
Lev Manovich’s early experiments at generative narrative using a non-linear video structure have been
intriguing, particularly conceptually. His work has informed the creation of a database of mobile videos
from my workshops, which are the basis of the performance collage, where the videos are remixed live
over the network. His generative film project Soft Cinema (2002–2004) is relevant to live video
performance, with its focus on the non-linear, non-narrative, ambient or generative video constructions,
as well as software and database systems developed for them. Many artists and filmmakers are finding
new ways to generate narrative or non-narrative cinema online, using computer databases and
programming to display random film clips (as in video paintings) and audio. His ideas are conceptually
(if not artistically) relevant here in terms of the live and generative cinema aesthetic and database-driven
aspect.
Similarly, a database was used in MINDtouch to store archived footage from the workshops. This
database was also necessary for the temporary caching (temporary storage) of the live-streamed media
clips, as they made their way to the server during the live events. This generative aspect of the custom
software is similar to processes used by many Live Cinema/VJ performing artists, who make
specialised VJ software in programming environments like MAX, MSP, JITTER or Pure Data. In this
way, the live performative, the programmed and the semi-generative elements are intertwined. The
generative aspect for my project comes from the body data sent to the server, which accessed and
mixed archived and live footage, with specific visual effects added. Originally, it was not meant to be in
any way generative. Everything was to be controlled through participant interaction [126]. Conceptual
aspects of the database, along with visual methods [127] for tagging and categorising for network use
have become part of the work as well. For Manovich, generative, database-driven video is always
creating something new and ‘live’. He states:
Soft Cinema consists from [a] large media database and custom software. The software edits movies in real time by choosing the elements from the database using the systems of rules. The software decides what appears on the screen, where, and in which sequence; it also chooses music tracks. In short, Soft Cinema can be thought of as a semi-automatic VJ (Video Jockey) – or more precisely, an FJ (Film Jockey). (Manovich, 2004: online)
126 Being developed for future iterations.
127 I cover visual methodologies in the Gillian Rose sense in the methods chapter.
119
Image 31 © 2002 Lev Manovich, Soft Cinema [128]
In MINDtouch , narrative is not a key element. Any narrative connections are either those of the
participants making the video clips or in the minds of the viewers of the final layered collage of the
different videos. One can say, just as a dream may be meaningful only to the dreamer, here the
narrative and meaning is made in the mind of the viewer. There are connections between the video
content and the sensor and visual effects applied to the videos, but these are more emotional and
sensory connections I have applied, not narrative ones [129]. In contrast, Manovich propounds his view
on the relationship between narrative and the database, stating that a,
[…] database supports both the database form and its opposite – narrative, a composite organization of an image on the material level supports two opposing visual languages. One is modernist-MTV montage – two-dimensional juxtaposition of visual elements designed to shock due to its impossibility in reality. (2001: 228)
Watching the Soft Cinema video combinations, the argument for narrative as he explains it seems a bit
tenuous. I would argue that it is not a narrative that is constructed from these algorithmic combinations,
but more a painting or photographic journey. Definitively cinematic, montage-like meaning constructions
can be made from the different combinations. ‘Shock’ is a dramatic term, however, for the visual
content he presents. Manovich might consider MINDtouch also to have a modernist approach [130],
but it is not for shock value. He also claims he is exploring four ideas [131]:
128 See this project online at http://www.softcinema.net/ (Accessed May 29, 2010).
129 Unless we called personal interpretive visuals narratives of experience.
130 I imagine he means in contrast to a more traditional Russian early cinema, Eisenstein definition of Montage, intended for intellectual and political confrontation and agitation.
131 More on Manovich is available online at www.manovich.net (Accessed on March 20, 2010).
120
First is "Algorithmic Cinema": which is the writing and implementing of a script and a system of rules, that are defined by the creators or “authors” of the cinema clips, the software then controls what the screen layout is, including the number of windows and their content. Then the author/videographer can choose to have minimal control letting most of choices to be made by the software, or they can determine which clips the viewer will see at specific points in time. However, the actual editing is done in real time by the system and the video clips and sound can run continuously without repeating the same edits. (2004: online)
During the development and design of the MINDtouch mixing software, it was not my intention to
make it an algorithmic process between the data coming from the biofeedback sensors on the body,
the mobile phones and the computer database/streaming server. I wanted to give control over the
mixing to other live or remotely connected participants. The algorithmic approach has, however, been
employed in the current version of the system, due to the complexity of the technical work needed to
make the project work given the current state of network technology development and the resources
available [132]. Yet the aim is to move away from an algorithmically driven, generative system to a more
multi-participant, ‘live’ and participant-controlled system. Manovich continues:
Then there’s "Macro-cinema": where the computer user uses the various windows sizes and dimensions within the larger frame. Next is "Multimedia cinema": where the video is meant as only one type of representation, with others being 2D animation, motion graphics, 3D scenes, diagrams, maps, etc. (2004: online)
Finally, he explores the concept of value here is that of Database Cinema:
[…] where the media elements are randomly selected from a database of clips and sound, to create virtually limitless (limited only by the number of clips and media elements themselves) combination number of video elements or different versions of the same film. (2004: online)
This is an approach to using a database as a new representational form in and of itself, in order to find
new ways to display the videos in the database. Thus, it is an exploration of database versus narrative
structure or recombinations for various narrative effects, and this is what is also attempted in the
MINDtouch project. However, I wanted the recombinations and choices to be made by the
participants as performers, more in the manner of Live Cinema performances, rather than by generative
means. Perhaps I will explore this further in future versions.
What is compelling about Manovich’s thoughts on the narrative potential of the database is his
suggestion that the narrative exists in the minds of the interactors or viewers in a cinematic database
context:
The elements on a syntagmatic dimension are related in praesentia, while the elements on a paradigmatic dimension are related in absentia […] while the paradigmatic sets […] the elements which make it […] are present in reality, while pieces […] which could have been present instead – different […] only exist in the viewer's imagination. Thus, syntagm is explicit
132 This is discussed more in the outcomes and technical development chapter.
121
and paradigm is implicit; one is real and the other is imagined. Literary and cinematic narratives work in the same way. Particular words, sentences, shots, scenes which make up a narrative have a material existence; other elements which form an imaginary world of an author or a particular literary or cinematic style and which could have appeared instead exist only virtually. Put differently, the database of choices from which narrative is constructed (the paradigm) is implicit; while the actual narrative (the syntagm) is explicit. (Manovich, 2001: 30–31)
I have adopted this approach to the narrativity of the collaborative, database-supported, live cinema
performance enabled in MINDtouch : the narrativity is being made with the abstraction in the
participants’ minds, as ‘body mixers’, as video creators and as viewers. This is akin to making dream
visuals, with implicit meaning, while embodied through the biofeedback sensors, mobile video
capturing and streaming.
The various types of nonlinear, ‘live’, generated and performed video Manovich discusses in his books
and projects are compelling in terms of ways to ‘perform with video’, and important to this research
since they incorporate elements of live cinema or performance video and database triggering.
Combining VJing and live cinema or video performance mixing for my project is a means to emulate the
sharing of embodied dream imagery. This is a key aspect of my practice – using currently available
technology to simulate a telepathic exchange between two or more bodies, lucid dreaming visually
together. While not a Manovich follower, I find his ideas conceptually compelling and artistically
inspiring. Even though the outcomes of the Soft Cinema project are not that exciting in my view, they
have been a partial inspiration to aspects of the MINDtouch database, inasmuch as an archive of
mobile videos created in the workshops constitutes the building blocks of the work.
Fundamentally, the goal of this project has been to enable completely live, mobile video VJing, without
a central computer (even a server) intervening, but rather with a distributed system, across all users’
mobile phones – this is the ultimate mobile, social, expressive collaboration tool. However, that is for a
future version, MINDtouch2.
In summary, this chapter has proposed a new form of video aesthetic using the mobile phone. It has
also provided a rationale for using generative systems for database cinema in this work, and compared
it to other similar works in video art.
122
6/ networks, telepresence + connectivity This chapter reflects on telepresence and telematic performance, the connective threads of digital
networks in practice, and the access issues that can challenge mobile telematic performance. It
contextualises the network dimensions both performatively and in terms of the technical aspects and
constraints encountered in the project, alluded to earlier at the end of Chapter 1.
6.1/ telepresence and telematic performance Telepresence is that sense of presence of another person that can be felt using technology and/or
telecommunications devices and systems, the presence felt in a telephone call to a friend. The
MINDtouch project explores this sense of presence in performance and uses telepresence to explore
ideas on telepathy and extended mind (discussed at length in Part II). Steve Dixon quotes Inke Arns’
definition of telepresence:
Telepresence allows the viewer parallel experiences in three spaces at once: 1. The ‘real’ space in which the viewer’s body is physically located; 2. per tele-perception in the ‘virtual’, simulated visual space reproducing a fictional or real, remote visual sphere; and 3. per tele-action at the physical location of the ‘data work or even of a robot controllable over one’s movements or equipped with a sensory apparatus over which one can find one’s bearings’. (Dixon, 2007: 419, quoting Arns, 2004: 336)
Edwina Bartlem describes telepresent art and its development history through theorists and artists,
such as Roy Ascott and Paul Sermon, in her article ‘Reshaping Spectatorship: Immersive and
Distributed Aesthetics’ in the 2005 Distributed Aesthetics issue of the online journal Fibreculture:
Telepresent, networked installations share some concerns with telematic art in terms of linking participants from distinct locations, foregrounding the concept of a networked community, stressing process-orientated art practice and enticing multiple users into participatory relationships with art […] The operator extends their body through hardware and software technologies. A technological device becomes an extension of the operator’s body, continuing human presence beyond the corporeal body through information networks and into a mechanical form. (Bartlem, 2005)
Yet it seems that telematics involves more than a connection, but a performance in one or each
location. Dixon succinctly defines it in his chapter heading:
Telematics: Conjoining Remote Performance Spaces (Dixon, 2007: 419)
I was first introduced to telematics and network performance during my masters studies and through
the New Forms Festival. In July 2004 the US arts organisation Turbulence started the
123
Networked_Performance blog [133], which I began following. The works that were listed on the blog
were (and still are) varied and there are some truly amazing projects.
Susan Kozel led a course on telematic performance, at an undergraduate and graduate level, where I
was doing my masters studies. After completing the course, I later became her teaching assistant for
the undergraduate course. She says:
Telematics permits for a play across absence and presence, as well as a range of dynamic impulses, both human and nonhuman. The suffix ‘matics’ […] is infused with enough ambiguity for it to be open to diverse unspecified interactions […] performances and installations […] revealing how the human senses of intimacy and physical connection are dependent on playing across what is revealed and what is concealed, rather than simply on what is visually displayed. (Kozel, 2007: 86)
These experiences aided in the curatorial process for the New Forms Festival, and ultimately led me to
create the MINDtouch project, a mobile video telematic work. I witnessed a striking performance by
‘cyborg’ artist Stelarc at the Western Front Artist-Run centre in Vancouver in February 2002, of his
famous Ping Body, developed in 1996 [134]. This performance involved online participants sending
commands to a remote computer to trigger electrode sensors attached to the skin of Stelarc’s arm,
sending electrical pulses to shock him. This was a very visceral interaction and the audience present
empathetically experienced the shocks with him, watching the responses on his face. This performance
demonstrated the power of the remote networked participant and the effect on the audience of such a
performance. Ladly addresses this sense of empathy and the blurring of the boundaries between the
‘real’ and the virtual, exemplified so well in this Stelarc performance:
It would seem that the telematic performance of reality has become reality indeed, and that the boundaries between performance of the real and the virtual have dissolved to the extent that they have become one act, at least in the minds of most young users of the technology. (Ladly, 2007: 143)
I also had the pleasure of participating in a seminar/workshop with Paul Sermon, at Goldsmiths
University’s Intimacy conference in London, in December 2007. Sermon is a pioneer in telematic
performance. His performance installation Telematic Dreaming, from the early 1990s, featured Kozel as
the performer, connected to the audience in another room in the exhibition space through video
cameras only [135]. While this piece was a local network only, it represented the performative issues
and the qualities and experiences of digital presence and absence, as well as public interaction and
responses within such a construction. It demonstrated the power of networked interrelations and the
affective experience of those interrelations, bringing questions around the ‘real’, actual and virtual, as 133 This blog is moderated and entered by Jo-Anne Green and Helen Thorington, of Turbulence.org, and Michelle Riel, and was started as a way for them to record […] network-enabled practice, to obtain a wide-range of perspectives on issues and to uncover commonalities in the work (2005). Available online at http://turbulence.org/blog/about/ (Accessed May 30, 2010).
134 See a review of his work online at http://stage.itp.nyu.edu/history/timeline/pingbody.html (Accessed May 30, 2010).
135 There has been a great deal written on this work. Bartlem (2005) discusses it at length, as does Kozel, Sermon himself and many others.
124
well as liveness, into the foreground. All of these issues, and current ones that Sermon delves into in his
recent work in Second Life, were discussed and explored in the seminar. This workshop with Sermon
was another part of the journey to MINDtouch , inspiring me to explore physiological sensing as
another way to embody the experience of presence in participants during the performance. 6.2/ electronic threads between us
Connecting people through the ‘invisible threads’ [136] or ephemeral connections of mobile
telecommunications networks has always been the crux of the MINDtouch telepresence aspect
(defined further in Chapter 7). Complexity theory [137] is beyond the scope of this research, but it
should be acknowledged that involving network systems brings in elements of complexity, conceptually
and practically. It has been manifested through the network of bodies, connected to sensors,
connected to phones, connected to a database of video clips, manipulated by the body data, through
the output to the world of networked and possible interactors/participant performers online globally.
We could say that this is a complex set of body/mind interactions, connected through ‘the ether’ to
other people/bodies in a ‘real intuitive’ (telepathic) and ‘simulated virtual’ (mobile phones) form.
What is a network? It has been described as a spiderweb-like or rhizomatic structure, or like a root
system. However, Mitchell describes it this way:
The archetypal structure of the network, with its accumulation and habitation sites, links, dynamic flow patterns, interdependencies, and control points, is now repeated at every scale from that of neural networks (neurons, axons, synapses) and digital circuitry (registers, electron pathways, switches) to that of global transportation networks (warehouses, shipping and air routes, ports of entry). (Mitchell, 2003: 9)
This represents just a couple of forms of network [138], but the key word is ‘interconnected’ and this is
what my work is about: interconnection – between people, and between people and things (i.e. mobile
and computer networks, systems and operations).
136 This is discussed conceptually in terms of notions of presence in Part II.
137 Sher Doruff’s thesis does a fantastic job in looking at complexity, network and databases and the wider related philosophical concepts. Her PhD dissertation, The Translocal Event and the Polyrhythmic Diagram (2006), is based on her own experience working on projects related to my own, developing software and systems for collaborative networked performance.
138 See also the New American Oxford Dictionary definition in Chapter 1.
125
My own play on the notions of network connections arrived like this:
links = chain
web = net
threads / sewing / netting / weaving/ interwoven/ knitting / fabric
connective tissue (Susan's)
networking = social connection
mesh / enmeshed / entangled / tethered / tied / knots / rigging
rhizome / nodes
branches / roots
cellular / veins / tunnels / paths
inter- / intra-
As mentioned earlier, the initial concept was for the project to be completely free of computers, free of
wires, and non-nuclear. The intention was to move away from the immobile, computer-based video-
conferencing setup to a completely mobile system, a more untethered, true sense of ‘telematics’. Over
time, through much frustration and many conversations about the vision for the project with different
collaborators, who tried to steer me toward pragmatism, I discovered:
a) that the smartphones used (there are more sophisticated ones as I write, iPhones [139], etc.)
were not yet sophisticated enough to replace a computer server system;
b) that video is still too computationally complex for mobile phones to mix, process and stream at an
acceptable frame rate and resolution for satisfying viewing; and,
c) that researching and developing garments with wireless, cable-free biofeedback sensors,
conductive threads, etc, would constitute a separate project, and a resource-intensive one. For
MINDtouch , we had to work with DIY, cumbersome sensors and a lot of cabling, which had to be
hidden within the garments.
At the start of the project, I was unaware of the corporate and political, behind-the-scenes issues at
play regarding mobile networks. From my North American perspective, where mobile use is less
139 At the time of writing this, WIRED magazine has announced news of Skype for the iPhone. There is no mention of video, but claims of 3G capability suggest video would be possible. See news available online at http://www.wired.com/epicenter/2010/05/skype-over-3g-comes-to-the-iphone-its-not-all-good-news/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+wired%2Findex+%28Wired%3A+Index+3+%28Top+Stories+2%29%29 (Accessed May 30, 2010).
old definitions
126
ubiquitous than in Europe, the European networks had the appearance of being similar to the Internet:
more freely networked and more interconnected. I discovered that economic issues, as well as political
and corporate agendas, require mobile manufacturers and services providers to create network
security systems, firewalls and one-to-one-only Bluetooth connectivity on mobile devices (although
Bluetooth can be enabled for one-to-many broadcast). I became aware (late in the project, however) of
how extensively controlled mobile networks are when I ran into several barriers challenging the ideal of
truly untethered, unblocked or unrestricted mobility. New questions arose, such as: why can one make
a phone call from anywhere, but not stream video from anywhere, using the same phone in the same
place? How are the different negotiations between big corporate providers made? Why are such
negotiations regarding coverage or what is included in one’s service plan not transparent or readily
divulged to customers/users until they are prevented from doing something. It was discovered that,
due to the differing historical, technological and market development of the mobile phone versus the
Internet, there are different levels of accessibility. The Internet, developed by the US military in the
1980s and called the ARPANET, was initially to be used to provide free, open and easy accessibility
between researchers at universities [140]. Services in the corporate telecom universe were provided
based on competition for territory and customers. The carving up of territories has resulted in more
controlled and restricted phone access, tethered by where you live, which provider you subscribe with,
which plan you have, and which model of phone you have on your contract. This economic model of
mobile phone services is similar to that of fixed phone lines – severely limiting mobility and what can be
done using a mobile phone [141].
The ideal of untethered and unrestricted mobile connections appears increasingly distant in this highly
controlled, monitored and restricted mobile world (Mitchell, 2003: 59-60). This artistic exploration
stumbled into and uncovered a quagmire of contradictions about networked connectivity. By contrast,
telepathy is (so far) still free. The lack of unanimous acceptance of its existence may be a positive. One
could say that telepathy has its own levels of security, in that it is said only to normally occur between
people who have established a level of intimacy or trust, such as between mother and child, siblings,
lovers, and close friends [142]. However, the connection is not always obvious or solid and signals can
be obscure. Perhaps if it were accepted, it too would be harnessed, controlled and infiltrated by
government, corporations and marketers.
In fact, mobile communication networks were more free in 2007 at the start of the project, but through
ongoing development – technically, politically and economically – they have become more controlled
and restrictive. This increasing control is manifested in the ever-increasing implementation of locked
systems, passwords and firewalls (Greene and Haddon, 2009: 23–24). Initial experiments, made using
140 This is not to be confused with the development of the World Wide Web that Tim Berners-Lee is credited for through the European Organization for Nuclear Research (CERN) in 1989. Sourced on the Computer History Museum web archive, available online at http://www.computerhistory.org/internet_history/internet_history_80s.html (Accessed May 30, 2010).
141 More can be read on the development and history of the mobile phone economically, politically and socially, as well as mobile communications evolution, in Greene and Haddon (2009) Mobile Communications: An Introduction to New Media.
142 Ullman, Krippner and Vaughan did numerous studies on this and it will be discussed in Part II on forms of presence.
127
the service Fring [143] in London early in the research, demonstrated that many wireless hubs were
open to piggyback on and join without a password. This changed quickly as people and businesses
became more security aware. My interest in using an open, ‘roaming’ WLAN (wireless internet) system
was not in order to be a freeloader, but to minimise mobile costs for participants and experimentation
costs for the project when streaming video [144]. Other barriers began to crop up when conducting
workshops and performance experiments, trying to send sensor data over the Internet, trying to
connect remotely to the streaming server, or from any university campus I went to (even though IT staff
always assured me it would be no problem). Overall, there was a general lack of network accessibility,
interoperability of technology systems, cooperation from providers and institutional IT support.
Frustration developed as I exposed the myth of open networks, free roaming and ease of connection
sold to the public by providers, and my optimism soon faded. Mitchell calls this […] fluctuating
conditions of freedom and constraint […] (2003: 9) and addresses it in this way:
With the proliferation of networks and our increasing dependence upon them, there has been a gradual inversion of the relationship between barriers and links […] Extension and entanglement trump enclosure and autonomy. (Mitchell 2003: 10)
However, this state of affairs was not enough to deter me. I gradually learned that the open connectivity
I envisioned is possible, but only with the right access codes, political power, money and influence. This
does nothing for the common person, researcher or artist trying to use the system to reach out in non-
standard ways.
Mitchell equates the vascular system of the human body with the larger social/political body of twenty-
first-century connectivity. In essence this equates to the emotional, affective, consciousness
connectivity of people, animals and the planet (Mitchell, 2003: 22). However, this particular body has
some major artery clogs, in my view. Perhaps my vision is utopian, but as has been discussed by
others here and elsewhere, when given the tools people want to reach out and connect and will find
creative ways to do so. In recent history this has manifested through technology in the telegraph,
Morse code, the landline telephone, the fax machine, the Internet, mobile phones and, recently, social
networking tools. These all demonstrate our desire to make connections. The somewhat sinister side of
this is uncovering the extent to which governments and corporations wish to intervene, control and
143 A service and application for the mobile that searches for WiFi or WiLan hubs nearby while you move about. This can access the Internet for free and much faster and easier than the mobile network. This was quite easy in Vancouver in 2007, when very few people locked their wireless hubs with WEP passwords since it was only becoming widespread. The city is also small and a business hub compared to London. As I write this, Fring have just announced an application for iPhones that supports Skype voice-over IP and suggests on its webpage that this should include video, available online at http://www.fring.com/ (Accessed May 30, 2010). This is free for now. See also a story on this in WIRED magazine’s affiliated site Webmonkey, available online at http://www.webmonkey.com/2008/10/fring_turns_your_iphone_into_a_free_skype_phone/ (Accessed May 30, 2010). This development does not change the fact that most WiFi hubs are locked or have to be paid for, so easy roaming is not possible, but perhaps that will change; these things are in flux.
144 There have been movements toward free WiFi zones in city centres like London, but these have been slow or inconsistent (see this story online from 2007: http://www.zdnet.co.uk/news/networking/2007/07/13/london-gets-free-wi-fi-39288023/. I will discuss the European open model conflict. There is free WiFi in the downtown core of Toronto since 2007, see info online at http://wirelesstoronto.ca/ (Accessed May 30, 2010) and other cities and places in the world are moving toward this. Organisations like Wi-Fi-FreeSpot (see online at http://www.wififreespot.com) are mapping free hotspots elsewhere, so things could change for the better (Accessed May 30, 2010).
128
monitor citizens in how we connect and interact with each other. In some limited cases, this is with
good reason, but often it is simply in order to capitalise on human desires and loneliness. I may have a
naïve perspective. However, I see myself as an optimistic artist who wishes to use these systems to
enrich people’s lives for free. Through this project, I am now more aware of the structures of these
complex systems and am armed with more tools and strategies to achieve my aims within and around
them.
6.3/ access / protocols / privileges
Due to the issues mentioned above, the full network connection envisioned (see Figure 10, page 234)
would have required more knowledge, time and resources – and collaborators [145] – to access the
right Internet protocols, gain permissions and privileges to cross borders (digital/analogue or Hertzian),
or gain entry into independent networks. Big corporations have access through business negotiations
and contracts, but independent artists and coders require other means and ingenuity to circumvent or
negotiate access. This process can be complex.
As Mitchell states, [...] access capabilities and privileges are more important than traditional forms of
ownership and control of property (2003: 59). I discovered this the hard way. The simple idea of
connecting people via their phones using video becomes a complex set of ‘workarounds’ and ‘band-
aid’ solutions to enable linkages and connections that should, to my mind, be straightforward. To
enable the data and video to move freely across invisible borders, negotiating invisible transactions and
exchanges to connect, requires a daisy-chain of connections. This delays the movement of data
tremendously as it needs to be translated through various programming languages, codes and
protocols on its way to delivery. In a digital world, the possibilities of losing original content and
meaning are increasingly diminished compared to older, analogue systems. But the inevitability of
delay, or loss of real-time (and sometimes quality of image), is increased in the daisy-chain system that
had to be employed for my performance events. This then becomes entirely about coding languages,
translation and negotiation. What was envisioned for this project was the same idealist vision Mitchell
had: of the 'wireless nomad' (2003: 60). Yet the dream of untethered, freely collaborating minds and
bodies creating together, as they might telepathically over space, is not yet as easy as it could be. In
fact, it may be getting harder. Networking issues are shifting continually in several directions, so we will
see what changes unfold in the coming years. All programmers and network specialists who were
briefed on my vision said that it can be done, and that it is a nice vision, but that it is tricky [146] and
complicated. Why is it tricky? Why does it need to be? My view is that the reasons are fear,
profiteering, control, ‘security’, and a desire for privacy and law. This idea of ubiquitous mobile network
145 Such collaborators have been found for future work. 146 See testimonials by my collaborators and other mobile or network experts in the Appendices.
129
access is described by Mitchell (2003: 60) as an idealistic vision. Access privileges are becoming more
tightly controlled all the time, at least in Europe and North America [147].
Barriers to free interconnection of devices began manifesting partway through the project when a
collaborator, Michael Markert, informed me that we could not use the mobile operating system alone
for all the computing – we would have to use the Internet and a computer server to enable the project.
This was my first concession or compromise in the project. Because the mobile phone operating
system was not capable of performing the intensive processing required (perhaps iPhone is able to
now), my laptop had to be added to the chain, receiving the sensor data and sending it over the
Internet to the server to visually mix the mobile video clips. Also, during the events, we had to choose
whether to connect the sensors to the phones or to the computer in order to mix the video: Bluetooth
allows only a limited one-to-one connection, which would not allow both. Finally, we had to use a
streaming server to store the video clips, mix them and stream the processed video back out. The
result was that the original vision had to be modified continually to accommodate what was, in fact,
possible.
Related to the access issues, I became curious as to whether I was alone or if there was an open-
source/open-access mobile phone community, as there is with the Creative Commons online
community. On first search what was discovered was a consortium of corporations trying to tackle the
issue of freer roaming, interoperable services, mobile operating systems, applications and network
connectivity [148]. Discussions on the lack of options are also quite active now, between mobile users,
artists and a small number of service and product developers, with communities having arisen around
specific phone models [149]. In light of this, there is certainly hope for future projects.
This concludes this chapter on practice. This chapter contextualised the networked aspect of this
performance research in terms of the medium, and the issues and constraints experienced for this
work. The next two chapters focus more deeply on the meta-theory behind and investigated in the
work, both conceptually and practically.
147 This situation is totally different in Africa and ‘third-world’ countries, as Sadie Plant showed in her closing keynote presentation at the International Symposium of Electronica Art (ISEA), 2009. She expounded on how the mobile phone is transforming Africa and India positively, empowering many lives and enabling small business. There is also a more open model in Asia, as Feijoo, Gomez-Barroso and Marin show in their article ‘Why Youtube Cannot Exist on a European Mobile: The European Regulatory Strategy on Mobile Content Access’ (2007), pp. 263–275.
148 Original articles on this were lost, but I did find an active group at the end of this research. Created in October 2008 at the end of the MobileActive.org conference in Johannesburg, the Open Mobile Consortium was investigating the issues in South Africa. See http://www.open-mobile.org/technologies/technologies (Accessed May 30, 2010). Their mission is similar to my desires, found online here http://www.open-mobile.org/about-omc/why-open-mobile-consortium-needed (Accessed May 30, 2010).
149 The relatively new (2009) Nokia N900 has a Linux-based operating system – Maemo 5 – and so it has opened the doors to open-source application development, which has flooded in too late for this research, but perfect for exploration in future versions. This, of course, needs to grow beyond one phone. See information online at http://danlynch.org/blog/2010/01/n900-review/ (Accessed May 30, 2010). Nokia also began looking for projects to fund on this model (see http://blogs.nokia.com/pushn900/), almost as a way to outsource their research and development (a positive but perhaps a calculating approach as well).
130
7/ presence: permutations
This chapter focuses on the philosophical and conceptual notions of presence, liveness, embodiment
and telepathy, the meta-level concepts that underpin the MINDtouch practice-based media art
research. These are the crux of the high-level thinking in this thesis, primarily the intertwining of
presence, embodiment, mind/body interconnection, extended mind-body and telepathy. There will also
be a discussion on the connected concepts and perceptions of consciousness, love, intimacy,
absence, the space between and other related experiences.
A predominant question emerging from this practice is: how can or do embodiment and consciousness
exist within and outside the flesh of the body, simultaneously? While this dualistic conundrum is not
solved here, it is teased out in an effort to reconcile the issue for the research, to show how both can
be the case in an intermingled fashion, rather than in conflict. Through this investigation, I outline how I
have managed to balance the dualities as they inform and wrap around the practice-based
performance developments.
The mind map below was created to help visualise the connecting concepts and how they all relate
back to presence:
Figure 4 © 2009 Camille Baker
This diagram demonstrates the various non-linear facets, permutations and expressions of presence
and how I am working with them. Most of these threads are unpacked and brought together hereafter,
but starting with presence at the centre of the investigation, then followed by explorations of
embodiment, liveness and affect as they relate back to presence and how presence is essential to
each. It will be shown that many of these concept threads are interconnected and overlapping, and
131
that the linear discussion here is not a true reflection of how they are woven together.
7.1/ presence: feltness sensing
The starting position on presence is that a sense of presence is a physical, phenomenological
sensibility, almost akin to a vibration or physical agitation, felt viscerally and intuitively. Presence is the
feeling or sensation of another person and their engagement in some activity, alone or with another. A
sense of presence is experienced when one is engaged actively, emotionally, intellectually, playfully or
otherwise in an activity with others and can feel their presence within the experience. Philip Auslander in
his book Liveness states that presence has an emotional impact, is ephemeral and evanescent, ‘[... an]
intangible expression [that] only exists in the transitory present moment’ (1999: 132). Conversely,
presence for philosopher Shusterman (2000) requires one to be physically present in person, in bodily
attendance and in a physical space. Hans Gumbrecht states: ‘What is ‘present’ to us (very much in the
sense of the Latin from prae-esse) is in front of us, in reach of and tangible for our bodies’ (2004: 17). It
is an embodied engagement, a felt or intuitive sense of otherness, emotional, physical or virtual,
elsewhere in time and/or space.
The term ‘Mobile Presence’, coined by mobile phone manufacturers and software developers, means
that the phone literally has the capability for the user to ‘see’ when and how their friends are available
to be contacted, based on the Internet-based SMS feature of Skype and other online chat software
(Borden: 2007). These companies consider ‘presence’ to be one’s availability-to-connect status, either
set by them or the phone system. Such availability statuses include: ‘available to talk’; ‘available for
SMS only’; or ‘not available’. Therefore, the various modes of being reachable via the mobile include
text, voice, photo and video messages. This is in order that friends can remotely monitor (or keep tabs
on) each other. This already happens with SMS/chat software, but can now be done on mobile
phones anywhere, at any time (providing more potential for stalking). This is an attempt to fill in the
perceived gap between liveness and virtuality through mobile presence, what Ingrid Richardson calls
the mobile technosoma, or the incorporation of mobile devices into our evolving corporeality
(Richardson 2000: 6).
With so much emphasis in digital media circles over the last decade on digital presence within 3D
immersive worlds and virtual-reality environments, my contrasting experience of presence has been more of
a non-technological one. I often experience a form of non-digital virtuality [150], a very embodied,
immersive, almost liminal, space in the pre-waking/sleep state [151]. My understanding and experience of
150 This is in reference to the ontology of virtuality and is a pre-computer definition of ‘virtual’, meaning in contrast to ‘actual’, yet still ‘real’. This will be discussed further in Chapter 9 on Liveness and also references earlier discussion in Chapter 3 on performance and ‘the real’. Pierre Levy is introduced on this topic later and has another definition of virtual that connects this older version with a new sense that incorporates technology, networks and interconnections between people. 151 Farthing (1991) says this is the state of hypnogogic (pre-sleep) or hypnapagogia (pre-waking) in his well-known Psychology of Consciousness, pp. 269–270.
132
presence informing my practice and daily life is a sense of otherness, a feltness that can be physically
sensed in a room. It is concerned with the feeling of others viscerally, physically and on an intuitive or
emotional level, just as one feels when speaking to a friend or loved one who is not actually there with
one in person. It is akin to feeling someone while speaking to them on the telephone or across the
table, through the qualities of their voice or their gestures [152]. This notion of presence also
encompasses a physical, conceptual, lived sensibility, like a vibration or physical agitation. It is a
feltness that is not one-sided but is instead a heightened physical sensation of otherness and
reciprocity, expressed through an exchange between people or entities in an embodied connection
[153]. It is the simultaneous intuition of another person and their involvement in some mutual activity or
space (virtual or in-person). It is felt when one is engaged emotionally, intellectually, playfully or
otherwise taking part in an experience with others, having the feeling of others also engaged in it with
one. This connection is key and it is experienced as an emotional one – even if those in the exchange
are not in a close relationship (i.e. with an uninvolved audience member and an actor on stage).
This engagement and felt or intuitive sense of others in time and/or space (physical or virtual)
transcends physical attendance, including time but not necessarily space [154]. This assumes that
individuals have an awareness or sense of themselves in their bodies in space and time and what
Merleau-Ponty calls ‘body-image’ (2002/1962: 100–102) as a reference to gauge ‘otherness’ outside
themselves such as seeing the outline of one’s body next to someone else’s or catching a glimpse of
oneself in the mirror next to others. When a memory or idea surfaces in our minds, our emotions can
be physically felt as a ‘lump in the throat’, a flushed face or other bodily sensations, which might be
thought of as inward sensing.
In this context, presence can be experienced as extended body/consciousness or awareness [155], as
an ephemeral non-corporeal embodiment; while the flesh is not directly stretched over the distance;
the mind and body are engaged by both parties. It is like the concept of telepathy, but when
transmitted by technology, such as through a telephone line or digital network, it is as if someone’s
consciousness ‘reaches’ through a camera or videoconferencing apparatus. Or it can be expressed by
one and felt by another, with the aid of actuators or mechanisms in a head-mounted apparatus, or
visually expressed by the actions of an avatar within a virtual environment [156]. These technological
152 This is the connection between emotions and intensity, which Massumi (2002: 25) describes, and emotions in terms of the neurological necessity (and physical) connection Lewis et al. focus on (Lewis et al., 2000: 74–75), also discussed in Chapter 8 on embodiment.
153 Others study this from the perspective of VR and CAVE environments, such as the presence project out of Stanford University, the University of Exeter and University College, London. See http://presence.stanford.edu/ for more on that project.
154 Such as a text message or video Skype exchange, taking place in different time zones but at the same moment – if the interaction or conversation is very personal in content, presence is felt through the vehicle of emotion.
155 In reference to Rupert Sheldrake’s theory of the Extended Mind in his book A Sense of Being Stared At. 156 I attended the Performing Presence Conference at the University of Exeter in March 2009, a critical conference for my work on presence and performance using technology. I realised others were coming to similar conclusions on these concepts as I have, and that in-person still is the gold standard of presence for the best ‘reception’ of performance, and most other communication. For subsequent conference proceedings, information is available online at http://projects.exeter.ac.uk/performing-presence/con.ann.php.
133
versions are the most recently studied instantiations of the qualities of presence [157] as they are
commonly manifested within digital performance, virtual reality or other socially networked contexts
and visually represented technological experiences [158]. This presence represents a togetherness
without necessarily actually being in the same physical space or place. It is merely the awareness of
another. But what is this awareness? How do we become aware? How is this awareness related to
the extended mind-body, and how might they be connected to virtual reality?
The proposal here is that there is an emotional ‘investment’ [159], even in a performance context
(through attended expectation and anticipation), that binds the interactors together, if only temporarily.
This enables the exchange to occur and facilitates the sense of feltness of the other. Perhaps this is
why it is difficult to describe presence and liveness with language. We can say that this presence is the
visceral sensation and knowledge of an ‘other’, out there somewhere, one who is alive and aware of
you, who is thinking of you and you them, either consciously or unconsciously, and who senses you
and is reaching out to you, consciously or unconsciously, emotionally or cognitively, physically or
virtually, and who is concerned for you and you for them [160]. This experience is also akin to the
expression of, and experience of, love. It is the invisible ‘connective tissue’ that Kozel speaks of, or the
‘threads’ in my analogy.
7.2/ emotional presence
In A General Theory of Love, psychiatrists Thomas Lewis, Fari Amini and Richard Lannon add to and
support the original behavioural psychology theory of Attachment, made famous by Bowlby [161] and
others since the 1950s. Attachment theory (Bowlby, 1988: 135) states that throughout our lives we
need physical affection and emotional bonding (Lewis et al., 2000: 69–70). Attachment theory is
important to this discourse on presence and emotional connection, as it asserts that the human brain is
neurologically formed by physical interaction, affection and attachment (69–70), and that mammals
evolved attachment as a survival mechanism. Lewis et al. focus on the evolutionary and neuroscientific
157 Ralph Schroeder did research on presence experienced in shared virtual environments and discusses it in his paper Being There Together and the Future of Connected Presence (2006). Friedman, Brogni, Guger, Antly, Steed and Slater did a meta-analysis of other research on presence in virtual environments and state that: We regard presence as a successful substitution of real sensory data by artificially generated sensory data (2006). I see this as a limited and narrow definition, completely computer-centric, and hence not part of my argument.
158 See more on this through the Performing Presence: From the Live to the Simulated research project (completed in 2009), managed by Nick Kaye (Exeter, UK), Gabriella Giannachi (Exeter, UK), Mel Slater (University College, London) and Michael Shanks (Stanford, USA), (Online) available at http://presence.stanford.edu/.
159 Some might be uncomfortable with this word due to its financial connotations, yet I cannot find a more appropriate word to emphasise the commitment involved – the time and energy devoted to the activity.
160 This knowledge is based on prior relationship with that person and what they have expressed of course – see reference to Lewis et al. and Sheldrake later in the chapter. This experience would be mere projection if this prior knowledge was not established in advance. 161 See his books A Secure Base (1988) and Attachment and Loss, Volume 1: Attachment (1986), in which he discusses the theory, how he developed it and its initial poor reception in the Freudian-focussed era of the 1950s.
134
development of the limbic system and its role in attachment. They provide in-depth analysis of the
neuro-cognitive make-up of the brain in mammals, including humans, explaining the functioning of the
limbic system in the brain and how in humans, unlike other mammals, the limbic system, central to the
control of emotional responses, is not self-regulating, but open. This fundamental developmental
survival process starts with the intimate bond with our mothers (74–75) and subsequently transfers
onto romantic partners (206–208), enabling our continued neurological development throughout life.
They also show that improperly nurtured children, and later adults, may have limbic impairments,
disabling them from healthily engaging in relationships. This indicates that we need continual intimate
interaction with others through physical contact in order for the brain to be properly and continuously
regulated. To emphasise this they say:
The science of our day is allowing us to understand what interdependence is for […] We are attached to keep our brains on track, in a process that begins before birth and sustains until it end […] attachment changes a young mammal forever, as limbic regulation carves enduring patterns of knowledge in the developing circuits of the mind. (2000: 98–99)
The argument is that babies are born with open limbic systems that are not regulated entirely on their
own. They need their mothers physically close to them to slowly regulate their limbic system (2007: 74).
This is explained as a two-way exchange, as the mother also needs the limbic connection and is in turn
regulated by the baby:
As the nervous system matures, a baby reclaims some regulatory processes and performs them autonomously. Even after a peak parenting experience, children never transition to a fully self-tuning physiology. Adults remain social animals: they continue to require a source of stabilization outside themselves. That open loop design means that in some important ways, people cannot be stable on their own – not should or shouldn't be, but can't be [...] Stability means finding people who regulate you well and staying near them [...] We recognize instinctively that healthy humans are not loners. (2000: 86)
According to Attachment Theory, not only is in-person, physical presence essential for clear
communication and interaction, but essential for survival and the ongoing physical health of individuals
[162]. The authors point out that this basic form of presence is primary to the brain’s ongoing
development:
Ongoing exposure to one person’s Attractors does not merely activate neural patterns in another – it strengthens it … In a relationship, one mind revises another [… they] remodel the emotional parts of the people we love, as our Attractors activate certain limbic pathways, and the brain’s inexorable memory mechanism reinforces them. (2000: 144)
Following on from this line of thinking, Lewis et al. might suggest that there is no point in trying to
connect remotely if there is no solid relationship in person as well. Friends have claimed that they can
162 Lewis et al. cite numerous medical studies and reports on how social contact and relationships affect health, such as Dean Ornish’s ‘Love and Survival: 8 Pathways to Intimacy and Health’ (1999: 80), which surveyed medical literature on the relationships between isolation and human mortality. There are many others cited throughout the book.
135
feel a sensation of ‘transmitting’ or receiving warmth from each other through email, texts messages
and Skype calls. Yet only a fraction of this experience can be sent or portrayed; just as a photo cannot
capture the experience of a natural vista – or just as text message content or a voice conversation can
only convey a fraction of the totality of the communication exchange.
Lewis et al. address this:
Relatedness is a physiological process that, like digestion or bone growth, admits no plausible acceleration […] Advances in communication technology foster a false fantasy of togetherness by transmitting the impression of contact – phone calls, faxes, email – without its substance. (2000: 205)
Thus, the authors suggest that to maintain relationships and communicate effectively (not only in
conflict, but in any type of communicative process), distance must be reduced as much as possible,
since technological substitutions are unable to fill the gap of physically felt absence (in-person,
corporeal presence). With this evidence, why would one attempt to replace a physical presence? The
answer is that we have set up our society to be more and more globalised, with people living further
and further from loved ones or friends for the purposes of work, school or pleasure, and technology
offers options that did not exist before. As a result, we have become more dependent on digital forms
of connection.
Lewis et al.’s interpretation of Bowlby’s Attachment Theory is crucial to my argument, as it bolsters
claims that emotional and physical attachment, and even anticipation (which might be the case for a
performance) of interaction are required for presence to be experienced. Lewis et al. are not merely
saying that in-person interaction is an imperative because it is more immediate or enjoyable, but also
that:
Because loving is a reciprocal physiologic influence […] Limbic regulation affords lovers the ability to modulate each other’s emotions, neurophysiology, hormonal status, immune function, sleep rhythms, and stability. If one leaves on a trip, the other may suffer insomnia, a delayed menstrual cycle, a cold that would have been fought off in the fortified state of togetherness. (Lewis et al., 2000: 207–208)
They point out that this basic form of presence is key to the brain’s ongoing development, and:
Because limbic resonance and regulation join human minds together in a continuous exchange of influential signals, every brain is part of a local network that shares information […] Limbic Attractors thus exert a distorting force not only within the brain that produces them but also on the limbic networks of others – calling forth compatible memories, emotional states, and styles of relatedness to them […] Each relationship is a binary star, a burning flux of exchanged force fields, the deep and ancient influences emanating and felt, felt and emanating. (Lewis et al., 2000: 142, my emphasis)
This expressive notion of ‘emanating and felt, felt and emanating’ feeds well into Sheldrake’s theory of
the extended mind and the common experience of feeling the presence of a distant loved one through
136
an awareness that they exist and breathe somewhere, even if they have not been in recent contact
[163].
There is clearly a case for an interweaving of the concepts, theory and phenomena of Attachment
Theory (Lewis et al., 2000) with that of the Extended Mind (Sheldrake, 2002), with respect to
understanding how and why telepathy may (or may not) be experienced more widely by people who
are emotionally intimate (children with parents, lovers, siblings, best friends, etc) and have established a
physical connection. It could be argued, based on these theorists’ work, that we evolved telepathically
to stay attached to each other for survival. Perhaps, through Morphic Resonance (2002) [164], as
Sheldrake argues, we have evolved to be able to sustain connection and interaction through this
embodied presence due to our past physical interactions (combining these two evolutionary theories)
between our loved ones and ourselves when apart.
7.3/ distance + intimacy: mobile digital presence + connection A key question for this research has been, what does it mean to FEEL someone through the phone, the
Internet, artwork, or digital media? How does that translate into a mobile art/performance project? This
research has intertwined and overlapped with emotional and philosophical experiences of daily life, yet
the art investigations are also intertwined with my personal life and vice versa [165], so through this
project attempts have been made to make sense of both. This section is a reflection on that as it
relates to presence in real and digital interactions.
Musings on remote and virtual presence from past experiences with a distant loved one revealed that
the sense of presence exists due to, and is interconnected with, intimacy, emotional attachment and
love. With these notions in mind, some poetic threads surfaced along the way:
presence vibrates in a sensation, an intuition,
an indescribable touch or caress of another,
a knowledge of you, a connection,
the invisible thread in the ether,
an emotional bond that links me to you
it is now, but anywhere,
163 This speaks to the notion that mothers whose children have gone missing can ‘feel’ that their child is still alive somewhere. This is the topic of the recent Hollywood film Changeling, based upon a true case in the US in the 1920s, called the Wineville Chicken Coop Murders. More is available online at http://www.riversideca.gov/library/history_aids_northcutt.asp (Accessed May 6, 2010).
164 According to Sheldrake, Morphic Resonance is simply an inherent memory or a form of DNA ‘learning’ that crosses generations. It […] occurs between patterns of activity in self-organizing systems on the basis of similarity, irrespective of their distance apart. Morphic Resonance works across space and across time, from the past to the present. Through morphic resonance, each member of a species both draws upon and contributes to a collective memory of the species (2003: 278).
165 As other art researchers quoted in Chapter 2 also have found. In a research methods class early on in my PhD studies, we were told that one’s personal life inevitably shapes and influences one’s research, and to take note of that.
137
it has no material quality or manifestation,
no shape, no specific language,
it is part of the non-quantifiable realm,
it is only felt and known to the intended receiver/sender.
presence is you, here, beside me and the heat of your body, your touch, your breath on my face, your voice
in low tones telling me all your thoughts, opinions, feelings.
it is holding my hand as we walk down the street and pulling me close when we stop.
it is you caressing my arm absent-mindedly as we ride the train, just to stay connected.
it is electricity between us when we are together, and the regular emails and texts exuding love and affection when we are not.
presence is the emotional thread, that invisible, sticky, spider-like web that holds us to each other when we are both together and apart.
it is the way I feel when I hear your voice on the phone, instantly feeling a warm overwhelming joy, love,
and longing to see and touch you.
it is the chemical explosion when our lips touch and my heart beats faster and all my body tingles with love
and desire. [166]
An awareness of the sense of desperate isolation and disconnection that many people experience
[167], along with a deep empathy to help others, is behind the drive to find ways to communicate in
more felt ways, to connect more deeply. Facebook [168] and other social networking tools are
poignant examples of a collective urge to connect, share, express and experience collectively – to have
some sort of feeling of connection to others in the vacuum we may feel in our lives, work and
experiences. A friend once told me that she was very unhappy with the 'poke' or 'x-me' applications in
Facebook. I sympathised, since Facebook seems to me to be a fun but superficial means to experience
a digital connection or show affection to distant friends and family – it is not a touch or a physical hug,
but only our outer self being shared. Our fears and trials are not shared, merely our 'best face' is
posted. How can we find a way to share what we feel physically, viscerally, emotionally and deeply
within these social digital realms when we are distant from our friends and loved ones? While we all sit
alone in front of our computers in our offices at home or at work? To whom, when and how can we
166 This may appear to imply a connection with a person already known. This model of telepathy can rarely be experienced in the same way with someone new or a stranger since, as Sheldrake and Ullman, Krippner and Vaughan show, most telepathy occurs between people who are intimate, like family.
167 While writing this, a new report on loneliness from the UK Mental Health Foundation was released to show that 1 in 10 people feel lonely today and half are getting lonelier. This report is available online at http://www.mentalhealth.org.uk/media/news-releases/news-releases-2010/25-may-2010/ (Accessed June 1st, 2010). Also the popularity of people flocking to social networking sites and tool should be an indicator of people feeling the need to connect. A piece of research on the effects of social networking on depression has already been done, see the article ‘Potential Benefits and Harms of a Peer Support Social Network Service on the Internet for People with Depressive Tendencies: Qualitative content analysis and social network analysis’ (2009), by Takahashi Y, Uchida C, Miyaki K, Sakai M, Shimbo T, Nakayama T. Available at http://www.ncbi.nlm.nih.gov/pubmed/19632979 (Accessed May 31, 2010).
168 Statistics for social network popularity and penetration in the UK from December 2007 to Dec 2009 can be found online at http://www.clickymedia.co.uk/2010/02/social-media-statistics-february-2010/ (Accessed May 31, 2010). Sherry Turkle studied online behaviour in depth, in her book Life on screen: Identity in the Age of the Internet (1997), dealing with online identity and interaction, pre-social networking and Internet dating tools.
138
span the vast expanses? How do we truly share our presence, liveness, touch? These questions have
driven me to continue to explore and express, and seek to enable others to share through digital
technologies. Will these technologies solve the question of who ‘out there’ will share? No, not likely. Will
the feeling of isolation diminish if we have more tools with which to share the deeper parts of ourselves
more viscerally? Perhaps.
What does it mean to be connected emotionally, socially, or even in work or business contexts? There
must be a more personal way to connect than telephone, Internet, and other media. Such a connection
has to be more directly felt emotionally and physically, in the cells of our corporeal being or flesh. It
should be experienced though gut sensation, from real interaction and the experience of being
'touched', either by an emotional closeness to another or an actual physical embrace, or both. The
connection must be experienced as real.
As discussed previously, we know that babies become severely developmentally delayed, and can die,
if not touched and cuddled regularly. Premature babies still need regular touching when in their isolated
incubators in order to regulate their bodies. As adults, we need this too, but as we live more densely
together in growing metropolises, we are actually becoming more isolated, losing 'touch' more and
more with family, friends and tangible relationships that are present in the physical sense. We work
more and are reverting to or reaching for our devices and technologies more often in order to find a
connection, a network, or as Kozel calls it, the 'connective tissue' of friends and family (2007: 28–31). A
friend once told me that his best friend was a guy in New York, whom he had only met online and
played video games with daily until he went in person to the friend’s wedding. The friend had also
found his wife through online dating. One of my closest friends lives in Denmark and we Skype
regularly. I have a video Skype call with my parents once or twice a week, so that they can ‘see’ how I
am from halfway across the world, even if they cannot actually hug me. I find myself not accepting
‘friends’ on Facebook if I have not met them in person. Yet I still want in-person, live 'face-time’ or
experience, and to know someone ‘in the flesh’. Most people still want in-person interaction, not only in
personal relationships, but also in business, to assess people instinctively. So, is it possible to create
technologies that can exude this fleshiness, this touch? The porn industry has tried to exploit this type
of virtual tactility – not with emotional intent, merely for superficial sexual entertainment value and
commercial purposes. What I am focused on here is how we can share our thoughts and emotions
and/or experience each other more deeply and viscerally, when we are apart or at a distance for work,
school or other reasons – both for everyday life and in art and performance.
139
7.4/ presence and telepathy
As a controversial biologist, Rupert Sheldrake’s theory of telepathy posited in The Sense of Being
Stared At (2003) is based on his own and others’ multitude of double-blind human, biological and
animal studies on extrasensory experiences. In his book, Sheldrake discusses how telepathy might be
a sense evolved to assist with survival in the wild and over distance [169]. He puts forward his theory of
Morphic Resonance – also called Morphic Fields – which says that cellular memory exists over
generations, allowing animals to perform better at tasks with each generation. The former theory is
more relevant to my work (2003: 17–111), suggesting that telepathy is a form of embodied and virtual
presence, influenced by emotional interconnection, which I try to emulate (not create) through digital,
networked interaction. Sheldrake’s work, and that of other scientists’ studies (Radin, 1997; and Wolf,
1995), have been covered extensively in previous art research (Baker, 2004), so will not be revisited
here in great detail. Sheldrake’s theories are echoed in my own experience and conception of how
telepathy works and can be understood.
My fascination with telepathic experience here, and desire to engage or enable others to exchange
telepathically, is predicated on actual exchanges of feelings of love with distant friends and family. The
term telepathy [170] is used here to mean ‘sending’ empathically, and in an embodied way, thoughts,
feelings, experiences and sensations to an intended other. Within close relationships, I have
experienced a ‘knowing’, an intuition or precognition about loved ones in the form of an uncanny
intuition or ‘foresight’ of something about to happen. This would manifest as something like: ‘X is calling
me now, even though my ringer is off,’ or a knowing that something is about to happen to them, only to
discover later that it had. Such was the case many years ago, when I had a dream that my mother was
about to have her wallet stolen, then the next evening she reported that it had been. Similarly, even
though she had no terminal illness in 1988, I knew she would die within a year. This could be
considered inconsequential; however, Radin (1997), with Sheldrake among others, demonstrated in his
meta-research on all previous research on telepathy and other forms of extrasensory perception over
the last 150 years or so that this experience is common, even if not accepted by mainstream,
conventional science. Sheldrake has also found that most people have experienced the sensation of
being watched, which he calls the seventh sense (2003: 4–16). He has found numerous cases of
people whose jobs depended on honing this sense, such as private investigators, martial artists and
soldiers. He has also discovered that the vast majority of cases of telepathy that featured any great
frequency and accuracy occurred between people who were emotionally involved, like family, lovers
and friends (2003: 52).
In my previous exploration of telepathy through interactive media, I was also concerned with the
concept of the extended mind, noting that,
169 Sheldrake says it is […] a kind of ESP, or extrasensory perception – a form of perceptions beyond the known senses [… that] contradict the assumption that the mind is confined to the brain (2003: 19).
170 Sheldrake says the Greeks define it as ‘distant feeling’ (2002: 19); coined in 1882 by W.H. Myers, to describe ‘feeling at a distance’ (Ullman, Krippner and Vaughan, 2002: 10).
140
Sheldrake questions how we form conscious images inside our heads, within the standard neurophysiological framework of vision. His theory of vision and optics states that, our eyes not only receive images and light, but they also “reach out to” or “extend outward” into the world to “collect” images and information, and “touch” the outer environment, like a two-way projection system […] (Baker, 2004: 22)
Telepathy is a technique, sense or skill that is not widely understood or ‘practiced’ actively. Evidence
has demonstrated its existence [171] as a method to traverse distance, discreetly and invisibly with
intimate others. It is tied to emotional connection or personal bonds and is the ultimate instance of
remote presence. Radin, Sheldrake, Ullman et al. and many other scientists have shown that the key
precondition for telepathy to work is an intimacy or close relationship [172], a deeply felt and engaged
emotional bond between siblings, parent-child, lovers, etc., originally established during in-person
exchanges, through close interaction, proximity or intimate relations. Thus, for the phenomenon of
presence and feltness of others to be experienced ephemerally, it must originate from a pre-established
physical connection, a mutually felt emotional exchange. In addition, there needs to be an open-
mindedness on the part of participants, an attitude of acceptance, a trust to receive and experience it
and to be confident that there exists an authenticity of and belief in the experience; a suspension of
disbelief, if you will. This is the same with any bodily or emotional sensation experienced in daily life.
Perhaps all communication involves this trust and openness in order for the intended recipient to fully
understand its content. Sheldrake believes that before all modern forms of communication and
technology, body language, gestures and other forms of physical, non-verbal communication were all
that was possible between humans when in proximity to each other (2003: 67, 76–78). But as a bridge
over vast distances he also hypothesises that telepathy evolved in all animals and humans (2003: 111–
113). He also notes that telepathic occurrences increase during altered states of consciousness, such
as meditation or dream states (2003: 78).
In my experience, confounding the problem with telepathic exchange is that one cannot always know if
others are actively ‘sending’ messages; unless they have confirmation from the other person involved,
this is not always possible if the first person did not consciously send the second person their
message, which is very common. Convincing evidence shows that most people (and animals according
to Sheldrake) have the potential to ‘send’ and receive telepathically with others, as much as we have
the capacity to sing. Yet it is not clear how much control we have over this sense or capacity,
conscious or unconsciously, since few practice while others fake it for profit. I am not trying to prove
telepathy, as I am not a scientist; rather, I am using it as a conceptual model for my practice.
Motivated by my own conviction of the existence of telepathy as a human and animal ability, I practice
171 See Radin (1997); Ullman, Krippner and Vaughan (2002); Sheldrake (2003) and others. In each you can see their exhaustive studies and meta-studies of others’ experiments and research in this area, proving that telepathy exists (whether sceptics believe all the thousands of studies is something else), where they look at the many manifestations and occurrences, in a vast number of human and animal populations, where it occurs.
172 See previous research: Baker, C. (2004), Internal Networks: Telepathy Meets Technology in the DreamPod. Sheldrake discusses it (2002: 49-53) and Arnold Mindell, mentions it in his book Quantum Mind: The Edge Between Physics and Psychology (2000: 239).
141
sending my thoughts and love to people I care about (this is different from wishes or prayers [173]). I do
this by visualising them while sending thoughts and feelings that I would like to express to them when
they are not around, or when words are inadequate. Sometimes it seems that words cannot express
the flutter that one feels when one sees a photo of someone one loves, or the visceral sensation of
longing to wish them well, or a desire for them to visit or call. Sometimes one wishes not to have to ask
for those things overtly. How tremendous it might be if we could just put our hands on each others’
heads or hearts and empathically or intentionally send and receive memories and emotions that we are
experiencing to each other, without having to explain or badly translate these feelings and sensations
into words. If we could, we might communicate with, understand and therefore treat each other better.
How empowering it might be (or disempowering for some perhaps) to have a world where we could
selectively ‘read’ each other’s minds and bodies, in a more direct and honest way. This direct reading
does not bypass the body to the mind, but assumes that the body and mind are one and that
embodiment can be stretched beyond the confines of the skin. This vision of mind-reading incorporates
the knowledge from the body necessary for such and complete reading to be possible, including non-
verbal and gestural modes of communication.
Dream space is a state of consciousness that can act as a conduit, or portal, for telepathy [174], as
can near-death experiences (NDEs) (Beauregard, 2007: 158) [175]. This dream space is also where the
consciousness is freer and more open (Wolf, 1994: 178, 180–187; Sheldrake, 2003: 78; Ullman et al.,
2002). Presence experienced in the alternate state of consciousness of dream or pre-sleep is elusive
and hard to define [176]. My personal experience of it is as a permeable, thick viscosity of fog, where
images, sounds and people just appear in seemingly disconnected ways that somehow make sense in
context to the dreamer. Here the presence of others can often be felt, even while one’s awareness is
able to transition easily between wakefulness, the hypnogogic state and sleep.
Below is a journal entry, musing about dreaming: Dreaming is a state of exploration, to explore your current physical and emotional state, yet to find yourself in familiar places, but not, such as a shoe or clothing stores, with the most exotic and beautiful designs I could never otherwise think of; or in strange and horrible places such as a torture taking place in huge old house or hotel; or hallways with closed doors that seem familiar, but not; or with faceless lovers. But there's always a desire not to wake, to lay there and continue to explore the most recent dream, continue it, add to it, share it somehow, climb within it, keep playing the role I was in when I woke, lingering, hoping to fall asleep again and get carried away again but to stay aware, lucid (almost never happens). Dreaming is like being pulled into a movie that you sort of, or totally, know all the characters and can feel
173 The difference is that with a wish or prayer you want something to occur, whereas with telepathy it is an act of direct (if not always intentional) communication, not a hope, want or desire for something to happen. It is like calling someone versus hoping they will call you. In ‘distant healing’, groups of people actively wish and pray for someone’s recovery. This is sending a desire for change, not a message. See more from the Noetic Sciences Institute on distant healing online at http://www.noetic.org/research/dh/explore.html and http://www.noetic.org/research/overview/ and http://www.annals.org/content/132/11/903.abstract.
174 Again Ullman et al. discuss their research on this and it has also been found by Sheldrake and Radin in their studies.
175 Beauregard discusses this in the context of NDE’s and encounters with deceased loves ones.
176 Farthing calls this transitional state between sleeping and waking hypnogogic (1992: 205–209).
142
what they feel, but this movie has twists and turns you don't know are coming, yet feel compelled to experience for good or ill, and often the dream goes in such a strange direction that you wouldn't believe possible.
Throughout this research one goal, coming from long-held interests, was to encourage participants’
sensory awareness and mind-networking abilities in more controlled yet creative and playful ways [177].
This could be done in a way much like a visual phone call, to enable people to explore quietness and
intimacy as a means to connect ‘outwardly’ with others remotely. I explore this concept, using
biosensor and media interfaces, to reach out through the virtual, or invisible, in-between space to find
others to share and exchange with, in an embodied and affective experience.
7.5/ collective consciousness, spiritual presence and out-of-body-experiences
This artistic research attempts a small-scale version of Lévy’s (1998) ‘collective consciousness’ [178],
crossing boundaries of conceptual exploration and practical implementation. It does so by remixing or
re-forming mental images and sensations into new collective or universal narratives of living, being,
experiencing, emoting, expressing, feeling, performing and communicating via mobile media
performance. Collective communication is facilitated during the collaborative remixing of the media
elements of the performance, and the use of a multi-interaction approach.
This investigation can also be seen to cross the boundaries of mind/body philosophies within the
integrated individual body approach, because I posit that while the body is the fundamental site of the
mind, it has been shown that the embodied mind can detach from the body, to witness and do things
the body cannot otherwise physically do [179]. Shusterman refers to how Merleau-Ponty thinks we are
limited because we cannot see outside of ourselves (Shusterman, 2008: 65). Yet doctors and other
scientists have studied numerous patients who have had NDE’s (Beauregard, 2007: 153-166),
reporting out-of-body detachment, seeing their own bodies from high above themselves. I have had
this experience as well. Others have researched how many people have had the experience, in a
moment of crisis, of feeling and allowing themselves to be guided by a ‘third man’ [180]. This is
reported as occurring when one is under great threat of permanent detachment from one’s body,
through death. In the CBC Radio documentary ‘The Third Man Factor’ (January 2009), it was found
that many people had experienced this presence of a ‘third man’, guiding them to safety in times of
extreme distress. One of the examples given was of a woman who was diving in the deep ocean and
got lost in an underwater cave just as she was running extremely low on oxygen. She stated that she 177 My previous art research project sought to assist people to make a telepathic interchange.
178 Also in the Jungian sense, but more referencing the studies on modern physicists’ concepts of ‘collective consciousness’, such as Fred Alan Wolf, mentioned briefly herein.
179 I also have personally experienced this, and as this research comes from phenomenologically driven lived experience, my own experience is as valid as any research.
180 CBC Radio (January 27, 2009), The Current, documentary report ‘The Third Man Factor’, Toronto, Canada: The Canadian Broadcasting Corporation.
143
suddenly lost her fear and felt like she had her dead husband guiding her to the opening just before
her tank ran out of air (CBC, 2009). What is astounding is that NDE accounts often cite that people
have a review of their lives in the form of a visual experience, and during that review, not only do they
‘know’ their own thoughts from their own viewpoints while seeing certain events of their lives again,
but they also intuitively know the thoughts and feelings of everyone involved in the event, including
knowing others’ points of view for the first time (Beauregard, 2007: 157–158).
So how does this spiritual sense of presence fit in? The Spiritual Brain: A Neuroscientist’s Case for the
Existence of the Soul (2007) by Canadian neuroscientist Mario Beauregard with journalist Denise
O’Leary, discusses this and inspired aspects of my research. This discovery was similar to a revelation
that I had several years ago upon reading the research of astrophysicists, physicists, Sheldrake and
parapsychologists – namely, that there are many scientists who are ready for a non-materialist
paradigm shift. There are maverick scientists who have done numerous, credible studies to show that
traditionally accepted materialist science does not have all the answers. These scientists believe,
through their own investigations, that there are forces we cannot see or measure and that many
mysteries of the universe cannot be understood by conventional science alone. These scientists are
often shunned for sharing their discoveries (though less and less in recent years) [181]. But just
because their ideas are unconventional does not mean they are religious fanatics, or Intelligent
Designers. Nor does it mean that the universe beyond our consciousness or that the boundaries of our
bodies present a duality. Nor is the overall debate a binary one. Nor does one have to take sides –
either atheist, materialist sceptic or religious, dogmatic zealot – in order to explore these mysteries.
That is why is it so refreshing to discover these voices. This work is about pushing the boundaries of
communication through art and technology.
In The Spiritual Brain, Beauregard presents reports describing bodies’ states first revealed by physician
Van Lommel:
Van Lommel classified the experience his patients reported by type: Out-of-body experience (OBE). This is an experience of floating outside one’s own body, while retaining one’s identity and very clear consciousness. Most patients report looking down from above. As we have seen, in some cases, patients have reported information that was later verified.
Holographic life review. In the popular phrase, ‘I felt my whole life passing before my eyes.’ As Lommel describes it: […] ’All that has been done and thought seems to be significant and stored. Insight is obtained about whether love was given or on the contrary withheld. Because one is connected with the memories, emotions, and consciousness of another person, you experience the consequences of your own thoughts, words, and actions to that other person at the very moment in the past that they occurred.’
Encounter with deceased relations or friends. Deceased persons are recognised by their remembered appearance, but communications appears to be through direct transfer of thoughts.
181 Many have joined the Center for Consciousness in Tucson, Arizona, which studies quantum physics and the mind, neuroscience and philosophy of consciousness, and focuses on the concerns presented here. More on the centre is available online at www.consciousness.arizona.edu/ (Accessed May 30, 2010).
144
Return to the body. Some patients learn, through wordless communication with a Being of Light or deceased relative that they ought to return to life [...] (Beauregard, 2007: 157–158, my emphasis)
It should be noted that Beauregard as a neuroscientist is more conservative in his claims than some
other scientists mentioned herein, and goes to great lengths to qualify his statements. However, he
demonstrates that the mind does not exist in only one part of the brain or in the body, but that it can
be distributed beyond the corporeal flesh. He shows how mystics report feeling the experience of
profound, full body sensations during transcendental or spiritual visitations of other presences:
Typically, a very intense mystical experience occurs only once or twice in a lifetime of contemplation. It can also include a number of other elements, such as the sense of having touched the ultimate ground of reality, the sense of the incommunicability of the experience, the sense of unity, the experience of timelessness and spacelessness, and the sense of union with humankind and the universe as well as feelings of positive affect, peace, joy, and unconditional love. (Beauregard, 2007: 262)
When trying to understand these experiences scientifically, he says:
[…] some have argued that RSMEs [religious, spiritual, mystical experiences] are simply emotional experiences and nothing more. Distinguishing in a definitive way between two kinds of states and experiences would be useful. No it won’t tell us whether God exists, but it may help us determine whether people who have mystical experiences enter a state of altered consciousness that is mostly related to emotion. (Beauregard, 2007: 268)
Even if these mystics’ sense of other presences cannot be proven in a materially tangible way [182], it
cannot be denied that these people, who are not diagnosed as mentally ill, can and do report
experiencing many profound, embodied sensations resulting from what they see as detached forms or
manifestations of presence. This supports other research and claims revealing that there are a great
many other ephemeral and experiential phenomena that we cannot explain, but cannot simply deny or
easily dismiss. Even though these inexplicable phenomena might be rejected as the machinations of
‘crazy people’ (or the possessed, in past times), such sceptical claims have since been disproved.
Should we explore these experiences more seriously, as Sheldrake suggests in his talks [183], we
might get further in solving some major world mysteries and problems of the world [184]. Similarly,
Beauregard says,
[…] once we understand the universe itself as a product of consciousness. We might expect living beings to evolve toward consciousness if consciousness underlies the universe. Consciousness is an irreducible quality […] But it will be stymied if the only purpose is to reduce consciousness to something it is not or to demonstrate that it is an illusion. (Beauregard, 2007: 277)
182 Although he took fMRI scans of Carmelite nuns (as discussed on the next page) to discover that it affects many different parts of their bodies with no two nuns having the same locations in the body light up in the scanner. (Beauregard, 2008: 266)
183 Sheldrake podcasts (online).
184 I suspect this, and new discoveries can lead to better understanding and solutions to problems.
145
Many physicists and other scientists mentioned here, as well as others in the yogic and eastern
traditions [185], believe that the mind has the power to change and influence reality, and that our
individual and collective consciousnesses can ‘change’ reality. Whilst this is a controversial position to
take, the idea can be seen to add weight to arguments for telepathy and other forms of embodied,
non-corporeal presence, and in that sense can inform this thesis as a set of hypothetical and
peripheral framing ideas. I do not cite this idea in order to make an extreme claim about the possible
lack of material reality outside our own minds, but rather to suggest that – as the placebo effect and
numerous other phenomena seem to reveal – our conscious minds may have the power, as shown in
experimental physics, to influence reality [186].
The Spiritual Brain discusses and debunks much of the materialist science of the last few decades,
which has sought to solve these mysteries by seeking the ‘location’ of spirituality in the brain and body.
Materialist science also tries to compartmentalise and create a disease, ailment, neurosis or genetic
dysfunction of religious faith or spiritual practice – to then ‘cure’ it (much in the way that homosexuality
was once regarded). Beauregard argues that science needs to shift how it thinks about mystical
experience, that the scientific community, and indeed the general public, need to make a paradigm
shift in how the unseen, invisible and immeasurable are viewed. Beauregard argues that materialism is
limited and that proponents are as ‘religious’ in their fervour as those who practice religion. He
proposes that there must be something in between; that we must move away from this absolutist
mentality to embrace unseen mysteries, seeking to understand and accept them, rather than pigeon-
holing them, filing the belief in their studies into the Diagnostic and Statistical Manual of Mental
Disorders [187].
The Spiritual Brain is not an argument for the existence of God, but an argument for examining mystical
experience more closely, and a validation of it as a real, felt, embodied, human experience; whether
one is religious or not. Along with quantum mechanics and quantum physics, these invisible
experiences might suggest that there are other options for how we explore and understand the world,
beyond the science-versus-religion debate. Beauregard explains his own research, performing MRI
scans on Carmelite nuns’ bodies and brains during their transcendental or relived transcendental
experiences. He discovers and discusses these experiences as similar in magnitude and kind to those
of other visceral experiences, such as falling in love, achieving orgasm or post-natal hormonal
experiences, which take over the mind and body in a wash of chemical and electrical intensity. He says
that mystical rapture is experienced with possibly more intensity than all of these combined.
Beauregard’s work is highly relevant here, as it helps frame my own views on the different forms and 185 See Pema Chodron (1994) and other Buddhist texts.
186 The ‘Pauli Effect’ a well documented parapsychological influence, which was named after Wolfgang Pauli, who was rumoured to have had an effect on technical equipment, causing phenomena such as laboratory bottles suddenly exploding for no reason in his presence. He was therefore not allowed in certain spaces when experiments were being performed. It is also similar to the ‘observer effect’, or how the act of observing in quantum mechanics can change the state of an atom from a wave to a particle. (Mindell, 2000: 508–509).
187 This text provides a touchstone for paradigm shifts over time, as a frequently updated but always influential ‘bible’ of the psychiatric profession. The changing interpretations of the phenomena discussed in the text can be seen as a literal textual transition over time from belief in one set of controversial ideas or dogma to another.
146
experiences of presence and embodiment, both in-body and virtually. However, it must be restated
that this is not intended to prove or disprove telepathy or other mystical experiences of ethereal
presence. Rather, these ideas are put forward as conceptual explorations of felt experiences of
presence; they are instances of physical presence at a distance.
The logical trajectory from Beauregard leads to a discussion of instantiations of presence or other
‘presences’, including themes of detached consciousnesses – phenomena such as ghosts, god,
extended minds, or other versions or dimensions of being (Wolf; Sheldrake; CBC Report). The
physicist Fred Allan Wolf (1994) believes that there are various dimensional planes of reality that we
transcend to after death, which are not known to us in this plane of reality but which some have
glimpsed during dreams states, during higher level meditation and NDEs, in quantum consciousness
and in quantum states of reality, which our ordinary sense modalities cannot grasp (Wolf, 1994: 340).
Wolf’s proposition about the substantiality or physicality of ‘presence’ bears mention here:
Quantum physics and relativity have indicated that much of reality is hidden and mysterious, and this would be directly apparent if we could directly experience quantum reality. The universe is bizarre, and mind and meaning are as important as matter and energy. The mind appears to be present in matter at many levels. Not only does the mind appear to be present in living complex organisms such as humans and animals, but also at the level of cells and even at the level of molecules and subatomic matter. (Wolf, 1994: 340-341).
This suggests that our minds can impact other aspects of reality. Other non-western cultures and
religions call this dimension transcendence, pure consciousness, Godhead or sometimes
reincarnation. Wolf’s position suggests that once our bodies die, we are able to see this other level of
quantum conscious reality, without flesh: a pure consciousness. So important are these ideas to my
ongoing research, I have cited Wolf as a major influence on my thinking in previous research as well:
[Wolf] explains that consciousness is a non-physical or quantum part of the brain/body and each person’s consciousness is part of a greater universal whole. This quantum consciousness facilitates non-local communication between people across space, time and matter, consequently, we all are connected to a main ‘cloud’ of consciousness. This ‘cloud’ is not necessarily a god-like entity, but understood in quantum physics as an energy field that contains actions of the imaginal, actual and real within space and time, which dematerializes and rematerializes in various locations as it is created and perceived. His concept is of an invisible network of people, connected through “cords” to a “mainframe” of one consciousness, which connects all our collective lives. (Baker, 2004: 28)
This point brings us back to a concept of collective consciousness, referred to in Chapter 7.1.
Embodiment for Glen Mazis in his book Earthbodies: Rediscovering Our Planetary Senses (2002)
involves more than one individual body, as he claims we are part of a much bigger body comprising the
planet and the universe. He says:
[...] it is important to see that there can only be a 'space' of cyberspace because it is a place we can inhabit with our bodies in this fluid, energetic, and circulatory sense of earth body. (2002:
147
126)
The question of presence, which necessarily informs this thesis, leads to a number of psychological
and phenomenological areas of study that extend, in full, far beyond the scope of this thesis. While
these areas do not compose the body of ideas for ‘examination’ in this thesis, they do bear mention
here as framing principles or questions that beg at the edges of this more academic discourse.
Such questions, for this researcher, necessarily entangle some element of the experienced, embodied,
personalised ‘presence’ into the more theoretical and critical edge of analysis (and here I must admit
that, like many who are interested in these subjects, I am informed by a personal sense of presence
replacing loss, after the death of a central influence and loved one whose once-living presence has
taken other forms of reassuring instantiation in ideas and experiences over the years).
To digress for a moment, just far enough to allow such important influences to weave themselves
more visibly into this narrative, it is important to at least mention or highlight some of the human
questions that often go unspoken, such as:
• Is it possible that we are all part of the more immense quantum body of the earth – as Wolf
and Mazis believe – with elastic minds (Sheldrake) possessing the ability to transit into and out
of our individual encased (and assigned) bodies, roaming out of them when need be?
• Is the larger concept and ‘reality’ of time not more elastic than now and in this moment and
tomorrow and yesterday – as others’ presence can be always with us, in ever-mutating,
changing and transmuting forms of space and time? I posit, based on personal experience of
sharing such personal information with others over the years, that many people who have
experienced the loss of loved ones describe these feelings, even long after their losses. In
terms of this thesis, I argue that in analysing this sense of loss alongside the concept of
‘presence’, it is important to at least consider (though of course one can not ‘prove’ such) the
phenomenological experiences of the ubiquitous, as well as the more rarefied experiences
described by physicists and philosophers.
In this context, with just this much personal experience woven into the fabric of my argument as a
highlighting thread, perhaps the more distanced theoretical approaches to ‘presence’ can better be
explored with reference to the personal within the phenomenological. Perhaps, then, it is possible to
find personal touchstones as well as critical understandings when it is said that ‘sensing presence’ is
an evolutionary response to keep us from feeling alone, and to that extent functions as a tool for
psychological survival (Lewis et al., 2001; CBC, 2009; Sheldrake, 2002). As such, a sense of presence
of others over distance may also help us as animals in times of crisis or in extreme conditions, and in
some cases may guide us to safety. Sheldrake suggests that it is a survival sense, just like our other
senses (2002).
148
Beauregard (2008) discusses the mind’s ability to leave the body and still be ‘conscious’ enough to
connect with other dimensions (as in NDEs) and to heal or harm the body (as in the placebo and
nocebo effects in medicine), against the imperative for materialist scientists to ‘locate’ it within the
brain or body. A greater sense of ‘mind’ is often experienced during transcendental experiences, one
of a supreme presence of otherness or of an intelligent consciousness, often interpreted as ‘God’.
Transcendence is experienced throughout the body, yet no two people experience ultimate presence
within the same parts of the body and brain. This shows, as Sheldrake and others do [188], that there
is an elastic nature or relationship between the mind, consciousness, the body and other
consciousnesses, bodies and presences. This is relevant to the enquiry into liveness and presence
because it connects to the exploration of virtual presence and extended consciousness through the
network, and this discussion demonstrates variations on extended, ‘virtual’ and extended presence.
This chapter has covered the concept of presence in a myriad of permutations and manifestations as
relevant to this research. While it does not discuss all forms and meanings of presence – impossible
within the scope of one PhD – it attempts to articulate and connect those that guide the thinking and
development process through the duration of the MINDtouch performance project, which is the main
case study for this thesis.
188 As a media artist working with complex and abstract concepts, I find important corollaries and research references here which add a certain substance or weight to the explorations of my work. I cite these references in this context as framing references, but do not claim that the thesis can in itself begin to validate or prove the theories or studies of the controversial ideas offered by these scientists.
149
8/ embodiment and affect This chapter unpacks the notion of embodiment, and particularly embodiment of or through
technology, a concept that is key to the activities in this performance research project. This chapter
does not attempt to cover all instantiations of uses of the word ‘embodiment’, but covers those
relevant to the work.
8.1/ embodiment – towards a working definition
Presence is often associated with the notion and experience of embodiment. Through explorations of
the concepts previously discussed on extended mind, morphic resonance (Sheldrake, 2003) and
‘embodied’ out-of-body experiences, it is apparent that the nature of the mind/body relationship is still
greatly unknown. This makes the notion of embodiment more elastic. Conceptual explorations and
research on embodiment, here and in the past, have led me to the conclusion that the mind and body
are entangled as one, encased within the skin and enveloping the bones and organs, blood, the brain
and other organs, fluids and tissue. Yet embodiment also incorporates the presence of a ‘being’ or a
‘spirit’, a consciousness or ‘soul’ that seems to be separate from the corporeal or fleshy encasement of
the body, but living within it [189]. This is especially true in conjunction with the concept of virtual
embodiment. The separation between mind/body and soul/spirit is evident when one has had the
experience of seeing a dead body at an open-casket funeral – the body that is left cannot be seen as
an embodied being any longer. Along these lines, some also believe that embodiment is consciousness
itself (Rettie, 2005; Beauregard, 2007) or an ephemeral life force, rather than a physical quality.
As with other major themes of philosophy, such as ‘Why do we exist?’; ‘What is the meaning of life?’;
and ‘What happens after death?’, the nature of the mind and the role of the body have long been
pondered. Philosophers have previously thought that reason, logic and the mind are separate from the
body. Especially held high are the ideas of Descartes, otherwise known as Cartesian split theory (Lakoff
and Johnson, 1999). Before neuroscientists had a chance to examine the brain and nervous system
more closely, debunking some of these notions, philosophers like Heidegger, Bergson, Merleau-Ponty
and others were seriously questioning the validity of Cartesian split theory. In Philosophy of the Flesh:
The Embodied Mind and Its Challenge to Western Thought, Lakoff and Johnson use modern
neuroscience to explore philosophy past and present in order to more forcefully challenge the Cartesian
paradigm point by point (1999). Their views might at first appear to contradict my own and those
herein, as they take a more materialist approach to embodiment and the mind. However, I see that
both perspectives can co-exist and compliment each other. My position is not at all in disagreement
with science as a whole; in fact, neuroscience, physics and medicine together can answer many of our
long-pondered philosophical and spiritual questions, while affirming some previously held speculations.
189 Various religions and other spiritual practices like yoga go much more deeply into this, but this history and exploration is beyond the scope of this thesis.
150
Francisco J Varela, with Evan Thompson and Eleanor Rosch, delve deeply into an examination of
embodiment, evident in the title of their book The Embodied Mind: Cognitive Science and Human
Experience (1991). They focus on embodiment in terms of consciousness, perception and cognition,
reviewing western and eastern traditions of understanding the self and the world. They look at
phenomenology and Buddhist philosophy from a science or neuroscience perspective, infused with a
western European philosophical perspective. Their contemplation of embodiment and perception has
shaped my understanding of presence in performance in the network, and in view of telepathy with the
following terms:
By embodied, we mean reflection in which body and mind have been brought together [...] reflection is not just on experience, but reflection is a form of experience itself [...] (Varela et al., 1991: 27)
The concerted effort to counter and knit the previous concepts of Cartesian split discourse is also
evident here,
It is a matter of simple experience that our mind and body can be dissociated, that the mind can wander, that we can be unaware of where we are and what our body or mind are doing. But this situation, the habit of mindlessness, can be changed. Body and mind can be brought together. We can develop habits in which body and mind are fully coordinated [through meditation and mindfulness]. (1991: 28, my emphasis)
What is intriguing about this is the acknowledgement of a tendency or inclination of the mind to
‘wander’ and leave the body. Instead of trying to state this as a disconnect, as Merleau-Ponty and
Kozel do, the authors suggest that ancient eastern meditation techniques keep the body and mind
integrated.
By using the term embodied we mean to highlight two points: first, that cognition depends upon the kinds of experience that come from having a body with various sensorimotor capacities, and second, that these individual sensorimotor capacities are themselves embedded in a more encompassing biological, psychological, and cultural context [… as such] sensory and motor processes, perception and action, are fundamentally inseparable in lived cognition […] they have also evolved together. (1991: 172–173)
Thus, Varela et al. demonstrate that consciousness and cognition are dependant on the body in its
embedded contexts and vice versa – even if the mind wanders, it is pulled back to the body, to
continue to contextualise the experience. Recalling the Near-Death-Experience (NDE), Out-of-Body-
Experience (OBE) studies and accounts of people’s consciousnesses independently roaming in the last
chapter, one is led to wonder how long this wandering can last, and if it is only short in duration, how is
the mind reintegrated into the body? If many eastern philosophies [190] and religions are correct,
190 Fred Allan Wolf is one of those scientists who believes this is what happens, and that consciousness goes to another plane of reality, free from materiality, very similarly to reincarnation concepts of many religions. See his book The Dreaming Universe (1995).
151
perhaps upon death the mind is released more fully formed, from a lifetime of embodied, lived
experience, to roam other realms or dimensions of consciousness. As a non-religious but non-
materialist spiritual agnostic, this notion is worth entertaining, as many more respected non-materialist
scientists, physicists and neuroscientists are also doing [191]. As we are cut off from this invisible
realm, we can only speculate from what we do know. This notion is tangential so, while curious, it will
not be discussed further here.
When looking at embodiment from cognitive science, phenomenology, and Buddhism, Varela et al.
state early on:
[…] we will focus on one such tradition, that which derives from the Buddhist method of examining experience called mindfulness meditation. We believe that the Buddhist doctrines of no-self and nondualism that grew out this method have a significant contribution to make in a dialogue with cognitive science […] (1991: 21, their emphasis)
This third position of a no-self or non-dualism, of integration through meditation, is a very progressive
approach, validating not only the mind/body connection but also a broader focus on phenomena inside
the body and outside in the world. This focus might also include non-materialist phenomena such as
telepathic and extrasensory experience, which are still not well understood.
Varela et al. describe how meditation is used to unify the mind and body and how, with practice, one
can train oneself to be more in control. This in turn helps the practitioner to tame her/his awareness
and the mind’s restlessness in order to become a more integrated body and being (1991: 26). Varela et
al. add to this discussion on the integration of cognitive science, phenomenology and
mindfulness/meditation, saying:
What we are suggesting is a change in the nature of reflection from an abstract, disembodied activity to an embodied (mindful), open-ended reflection. By embodied, we mean reflection in which body and mind have been brought together. What this formulation intends to convey is that the reflection is not just on experience, but reflection is a form of experience itself – and that reflective form of experience can be performed with mindfulness/awareness. […] In our usual training and practice as Western scientists and philosophers we obviously proceed differently. We ask, ‘What is mind?’ ‘What is body?’ and proceed to reflect theoretically and to investigate scientifically. […] By not including ourselves in the reflection, we pursue only a partial reflection, and our question becomes disembodied […] theoretically confined, preconceptually entrapped somewhere. (1991: 27, their emphasis)
One can also see that Kozel’s phenomenological method is remarkably similar to the mindfulness
practices of Buddhism. One might even argue that it is a form of mindfulness, designed for the western
philosopher or theorist who is not interested in other aspects or practices of eastern religion. In
reference to non-dualism, Kozel emphatically states that there is no mind without the body and the 191 I am aware of many popular sceptics like Daniel Dennett, Susan Blackmore and others who refute these ideas, but since this is art not science practice I am not concerned with these arguments, except to acknowledge that they exist. The Spiritual Brian contains an in-depth debunking of the sceptical, materialist science and media perspective, but there is not scope to address these sceptical claims here.
152
Cartesian split is a simplistic distinction. She proposes:
[…] we need to have a method and an intent to receive information from our own bodies and from the bodies of others. In giving voice to diverse corporeal states we can overcome naïve distinctions between matter and spirit, between body and mind. (2007: 23)
So Kozel has taken this task on and created a method for performance media practitioners (and
others). She explains that this is needed because the forms of embodiment of mind, spirit and flesh in
performance are intertwined, as such:
The body is a weave of different materialities, the body is a dynamic process, the body navigates the world at the intersection of a cluster of languages (verbal, physical, archetypal, mnemonic, unconscious). It is electric, biological, and cultural. (2007: 33)
Kozel stresses that the complexity of this is the corporeal, conscious and embodied experience of
humanness. In a similar vein, Massumi states that,
Mind and body are seen as two levels recapitulating the same image/expression event in different but parallel ways, ascending by degrees from the concrete to the incorporeal, holding to the same absent center of a now spectral – and potentialized – encounter. (2002: 32)
Elizabeth Grosz in Volatile Bodies (1994) adds her voice to the debate, emphasising the intertwinement
of dualities rather than disjointed separation, easing my own concerns. She also explores Merleau-
Ponty’s notions of the corporeal and flesh:
The concept of flesh is [...] not the union or compound of two substances, but ‘thinkable by itself’ [...] While it does not displace perception as the thematic object of investigation [... it is] the condition of both seeing and being seen, of touching and being touched, and of their intermingling and possible integration, a commonness in which both subject and object participate, a single ‘thing’ folded back on itself. (Grosz, 1994: 95)
This curiously points to the possibility of being both subjective and objective, as well as of the body
having the ability to have two simultaneous perceptual experiences, yet still be able to distinguish the
differences. The concept of embodiment is compelling here, as it suggests a process which takes
place both inside and out, crossing boundaries of the senses. Therefore, we must consider that the
perception and the perceiver are one. The notion of the permeability and flexibility of consciousness
brings the discussion full-circle, as a return to and embracing of the idea of an elastic embodiment,
which speaks to that of an extended mind as well (Grosz, 1994: 96). The concept of permeable
boundaries of perception and the body suggests that the body can separately function entirely outside
of the mind (and vice versa), but in an elastic sense, not detached entirely from the body but, as Varela
et al. Suggest, informed by it. On this Massumi comes in to address the boundaries of the body: Brain and skin form a resonating vessel. Stimulation turns inward, is folded into the body, except that there is no inside to be in, because the body is radically open, absorbing impulses
153
quicker than they can be perceived, and because the entire vibratory event is unconscious, out of mind. (2002: 29) [192]
Massumi’s focus is on the unconscious as a means to move out of mind (body) [193]. This speaks to
the Sheldrake and Ullman et al. findings, namely that most telepathy and other experiences of ESP
happen during sleeping and dreaming (see pp. 141 and 142 in the previous chapter on this).
To summarise then: within MINDtouch as well as previous work, my enquiry into how embodied
experiences can be embedded into media art practices aims to uncover how such mediated work can
be experienced by the body to access memory and alter consciousness. I surmise that controlled
telepathic interactions can begin at a sensory level, facilitated below conscious awareness. One
approach was to create an experience that facilitated re-embodiment through altered states of
consciousness, enabling participants’ receptivity to new types of input. The intention was to tap into
deeper levels of sensory experience and physical and emotional responses or states in the mediated
space, then to enhance sensory awareness and mind-networking abilities in that space. This was
intended to enable people to make a telepathic interchange, like a telephone call, assisting them
through solitude, quietness, intimacy and privacy using biosensor and media interfaces. Participants
were guided to connect ‘outward’ to others remotely through the virtual, or invisible, space (Baker,
2004). This was not done in the current work, which continues from there, but I step away from trying
to stimulate telepathic exchange, assuming it happens for some, and instead make a technological
simulation of the process, focussed on the content of the simulated exchange; perhaps for those who
do not naturally experience it, to join the experience through a mediated means.
8.2/ embodiment of space / absence and the space between In this section, it is important to address and define/refine the concepts of ‘absence’ and ‘the space
between’ in relation to an emerging working definition of ‘embodiment’ for the purposes of this thesis.
The focus here is on practical engagements between bodies in space and conceptual bodies, with a
focus on the notion of extending the body beyond our physical boundaries, through engagement with
our innate telepathic abilities. The body can be engaged in practice by facilitating a digital, artificial
simulation of telepathy, in order to extend a real and actual embodied presence. This can be achieved
using simple ubiquitous technology tools such as mobile phones to bridge the space between. To
contextualise these concepts, especially in relation to embodying technology in order to traverse space
and to minimise the experience of absence and physical distance, a definition of space that can be
used comes from Paul Dourish’s Where the Action Is: The Foundations of Embodied Interaction:
192 This is an important point repeated from page 23, but used with more emphasis and context here.
193 To clarify: when I speak of consciousness I include the unconscious as part of the range of states of consciousness, just as there is a wide range of states of corporeality.
154
“Space” is largely concerned with physical properties (or metaphorical physical properties). It concerns how people and artifacts are configured in a setting; how far apart they are, how they interfere with lines of sight, how actions fall off at a distance, and so on. (2001: 89)
Personal musings and felt experiences of this idea of space between have exposed a space that is not
always or necessarily an absence or void. It is felt as a transition or transmission; a conduit, medium,
or channel to move from one place to another, much like an alley or passage. This feltness includes
telepathic exchanges, memes [194] and electromagnetic vibrations, and also the Internet, SMS/Chat,
mobile and other telecommunications more commonly experienced within virtual spaces. Similarly, the
space between can resonate as a memory or the moment of struggling to remember something; a
pause, hesitancy or tension. These are all elements that extend our bodies beyond themselves, both
as human natural abilities or traits, as well as artificial creations or simulations. They are fused together
or intermingling, becoming a space-between moments, objects or people. It may also be understood
as breathing space, thought space, meditation, imagination, dream space, being ‘in limbo’, and
‘negative space’ (although not necessarily negativity). The space between is a sense of sustain,
floating, suspension, or nowhere-ness; the ringing before answering, or waiting before arriving. It is
falling asleep or trying to wake, the process rather than the attainment, the longing for an experience
rather than the experiencing itself.
Brian Massumi calls the in-between space ‘potential’ or ‘possibility’ (2002: 95). He offers notions that
the body is in an ever ‘becoming’ state of potential, a state of being which never arrives, and is always
virtually becoming – originally a Bergson concept, picked up by Merleau-Ponty and others: The field of
potential is the effect of the contingent intermixing [...] (2002: 76)
Related to embodiment, space and the understanding the body’s experience of presence, we need to
understand the space it occupies, and how it senses and defines itself in order to position otherness or
presence of others in a physical and virtual, intangible or ephemeral space. Merleau-Ponty addresses
this issue:
Bodily space can be distinguished from external space and envelop its parts instead of spreading them out [... and] the body image is [...] a way of stating that my body is in the world [...] By considering the body in movement, we can see better how it inhabits space (and possibly time) because movement is not limited to submitting passively to space and time, it actively assumes them, it takes them up in their basic significance […] (2002/1962: 100–102)
Merleau-Ponty’s discussion here references his concept of the ‘here-body’, which becomes a recurring
motif for him and others, like Don Idhe. Idhe, in Bodies in Technology (2002), uses it in terms of placing
the integrated conscious flesh within space and time, as against the notion of the untethered, yet
embodied ‘there-body’ or extended mind-body, which leaves the confines of the material flesh encased
by skin. Significant also is Merleau-Ponty’s notion of body-image, as well as ‘body space’ or ‘body in
194 The scientist Richard Dawkins coined the word ‘meme’ in The Selfish Gene (1976) as a concept for discussion of evolutionary principles in explaining the spread of ideas and cultural phenomena (1986: 192).
155
space’ and the space between [bodies]. Idhe categorises the ‘here-body’ and ‘now-me’ as the
physicalised version, as opposed to out-of-body and virtual body states. He says:
If I am right about the secret norm of a here-body in action, it should be noted that such a body experience is one that is not simply coextensive with a body outline or one’s skin. The intentionality of bodily action goes beyond one’s bodily limits – but only within a regional, limited range […] One’s ‘skin’ is at best polymorphically ambiguous, and, even without material extension, the sense of the here-body exceeds its physical bounds. (Idhe, 2002: 6)
Grosz also explores the concept of bodies in space:
Our body is not in space like things; it inhabits or haunts space. It applies itself to space like a hand to an instrument, and when we wish to move about we do not move the body as we move an object. We transport it without instruments ... since it is ours and because, through it we have access to space […] (Merleau-Ponty 2002/1962: 5) In other words, we do not grasp space directly or through our senses but though our bodily situation. Space is not understood as a series of relations between different objectively located points, points of equal value; for one thing, this flattens and neutralises the positive contribution we ourselves make in the perceptions of objects. Rather, space is understood by us as a relation between these points and a central or organizing perspective which regulates perceptions so that they can occupy the same perceptual field. This perspective has no other location than that given by the body ... (Grosz, 1994: 90)
This perspective is key to understanding the notion of space between, because it shows how the body
fills this space, and points to a notion of presence and embodiment existing without flesh. It is from our
knowledge of space and ourselves within it that we see the basis from which to orient the body and
from which it can move, as well as how our consciousness can and does wander.
By looking at Human-Computer Interaction (HCI), Paul Dourish shows how a person can embody the
technology they use. He also applies philosophy to computer science theory to demonstrate
technological embodiment within certain contexts and in various manifestations. Dourish’s theories are
relevant here as he focuses on ephemeral ‘beingness’ or (tele)presence of being, without a physical
skin encasement which can manifest when users engage in various technological activities, such as
when using flight simulators for pilot training or video gaming (2001: 100). Richardson also addresses
technological embodiment by positing a form of non-corporeality within virtual contexts. She suggests
this form of embodiment exists during invisible exchanges of sentiment, such as during online chatting
and text messaging (Richardson, 2006: 6). These examples attempt to reconcile the non-Cartesian
model of embodiment by correlating a technological form of extended mind with that of NDEs, out-of-
body and related experiences, as discussed by Beauregard in The Spiritual Brain (2007: 157).
Beauregard suggests that the mind cannot be constrained to the flesh, or to matter or the body as an
epiphenomenon (2007: 110) of the brain emerging through evolution, as is commonly held in the
materialist scientific community. Rather, he is suggesting that neither scientists nor philosophers have
resolved the issue of qualia (104 and 108) or how the mind can heal and ‘control’ the body (such as
with the placebo and nocebo effects found in medicine: 145), or can leave the body altogether (as in
out-of-body and near-death experiences: 153–154). Thus, the theorists who advocate a form of
156
consciousness or mind extension through technology are inadvertently advocating a sort of telepathy
as a form of projection by sending the consciousness through technology.
Kozel delves into concepts of the visible and invisible related to notions of presence and space
between, as well as virtual and telepathic connection. She says:
The invisible does not make sense on its own, but gives depth and texture to what we see [...] the invisible can be seen as a dimension that cannot be isolated from others but permits the existing ones to intersect differently [...] (Kozel, 2007: 40)
Expanding on this, she says:
[...] the invisible is not simply nonvisible [...] Just as connective tissue does not appear on scans or x-rays but is integral to our structures and our physical communication of pain and pleasure, the invisible is that which is possibly visible; it spans temporality by being what has been or what in the future might be seen, and it is transsubjective [...] it is an absence that matters. Literally, it is an absence with a sort of materiality. (Kozel, 2007:41, quoting Merleau-Ponty, 1962a: 187)
This is an important argument that reconciles the dualities of perception and experience. Yet in light of
numerous studies and accounts of near-death experiences, out-of-body experiences and other related
extended-mind occurrences, which show the existence of consciousness beyond physical body and
death, questions still linger. These arguments make it evident that all of these states can still be part of
embodied consciousness; what is unclear is how or whether the consciousness can continue beyond
the life of the perceptual, corporeal body.
Kozel reflects further on the body, focusing on connective tissue and communication:
Connective tissue is an embodied way of understanding not only the pre-reflective within us but also the means of communication within one body and between bodies. Those patterns of tension communicate physicality and affect, pain, and peace, distortion and degeneration. They hold memory over time. Unpredictable, reshapeable, multilayered, affecting brain and body functioning, connective tissue is a fundamentally dynamic material and network. Language and the channels between people are material. They are connective tissue-tensile fibers, material and immaterial, imaginary and real, connecting and separating people, and connecting and separating meaning from itself. Beginning to view our bodies and the spaces between bodies differently is the condition for reworking in a Merleau-Pontian way our understanding of thought, the communication of ideas between people, and the relation between physicality and ideas, between self and world. (Kozel, 2007: 30, my emphasis)
This perspective on communication and spaces between bodies clearly places the body in the position
of interpreter of all thought, emotional and unconscious states, as well as the site of physical
expression. Kozel’s term ‘connective tissue’ is similar in meaning to my conception of ‘invisible
threads’. For this work, ‘invisible threads’ translates as the invisible connection between people, framed
around emotional connections and/or ‘investment’ between people, knitting them together. Yet Kozel’s
use argues that the body’s own fibres provide a more embodied understanding of human interaction,
157
thus implicating a more elastic, regenerative and alive system of connection than the word ‘thread’
affords. I see these connections or threads as the stitching or weaving of ideas, emotions, and people
together to create a new if ever-changing fabric. Both can co-exist as fabric-like tissue.
The discussion and analysis regarding human connective tissue contextualises the visible and the
invisible and makes the case for electro-magnetic, radio and other ‘invisible’ waveforms and similar
forms that are not visible to the eye. It also emboldens the argument that there are many elements in
our environment and daily experience that are not easily seen, but exist and connect us. These include
conscious, embodied extensions of our selves beyond our physical bodies. For the purposes of this
discussion, embodiment and its various states are described as: 1) being in the body, and the
mind/body integration; and 2) being in the body, but also sensing and extending the mind, during near-
death experiences, dreams and in altered states, VR space, and virtual worlds and consciousness.
Idhe speaks of the embodied, in-person aspects of liveness and how our bodies connect with
technologies of modern living. His notion of ‘here-body’ describes the ‘multi-sensory perspective’ or
‘body in action’/‘myself-as-body’ that is […] not simply coextensive with a body outline or one’s skin
[…]. He reiterates that the ‘here-body’ is not just the one encased within the skin and surrounding the
bones, but in the case of out-of-body experiences, the here-body is […] extended beyond the outline
of the wearers’ body […] (2002: 5–7). Indirectly referencing Merleau-Ponty on ‘body-image’, Idhe adds,
[…] image-body – is identified with 'my body', but under the perspective of not being the 'now-me' that is implicitly identified with the floating perspective [...] now-me and the here-body [...] multisensory sense of the here-body would locate this in the floating location. (2002: 5)
Here, the discussion enters the domain of the body image, but also Idhe tries to place the ‘floating’ or
‘out of body’ into some sort of framework of here-body, now-body, which also brings in the time
element, important here in terms of technology and the multisensory body in action and the experience
of liveness. He continues: […] here-body in action that provides the centered norm of myself-as-body
[…] (Idhe, 2002: 6) He also categorises ‘here-body’ ‘now-me’ as the physicalised version versus out-
of-body and virtual bodies states. In a related way, Kozel speaks briefly of ‘leaving ourselves’
existentially and conceptually (corporeally also), in that our consciousness has the ability to roam or
extend literally outside the boundaries of our skin and bodies (consciously, rather than in Massumi’s
unconscious sense). It could also be argued that we leave our bodies temporarily while we are thinking
and daydreaming, or while our minds are simultaneously multi-tasking with numerous ideas, plans,
worries, emotions and preoccupations.
Unlike psychogeography or locative media projects that focus on the external environment and
locativeness or situatedness in the geography out in the world. MINDtouch always returns to the
body as the primary location of exploration, with the goal of situating and locating individuals back
within themselves phenomenologically, engaging or listening to their bodies. Referencing body
situatedness when embodying technology, Don Idhe states:
158
Bodies are necessarily situated, which is not to say that one cannot take account of interactions between (my) body and materiality [...] bodily situatedness. (2002: xix)
On technological embodiment through extended consciousness, he adds:
[...] machine muscle replac[es] human muscle […] Neither muscle nor mind has reached out into the open world except in human-technology symbiotic forms [...] the standard form of replacement worry [...] (2002: 3)
This is main thrust of Idhe’s argument that technology and the body meet and interface to extend
beyond the skin (also suggesting this as the only way to extend the mind beyond the skin), and our
engagement with technology in the process. I differ on this point, but see its connection to my work
with the mobile phone and biofeedback sensors. Idhe’s argument focuses on efforts to connect the
body to technology, very directly on the skin, as a means to reach out into the open world through a
very symbiotic form. Idhe suggests a superficial outward reaching, through words as ‘visual’ and
‘spectacle’, in contrast to ‘embodied’:
The full, multidimensional experience gestalts in the here-body of the embodied perspective, whereas the visual objectification out there is spectacle-like. (2002: 4)
This is a problematic position since it suggests that the external experience is merely visual and
spectacle-like, as if the visual cannot be embodied. Additionally, it seems to suggest that this spectacle
cannot induce a visceral or felt sense even if experienced outside of the body [195]. I would suggest
that the extension of the conscious experience is sent back to the corporeal body of sensations and
perceptions, even if perceiving beyond the skin (as in the case of out-of-body or near-death
experiences). In this sense, therefore, out-of-body experiences, ‘out there’ are felt and still embodied ‘in
here’ in the fleshy body.
Idhe continues on the technological context of embodiment and in the realm of the ‘in and out’ of
bodily experiences:
[...] nontechnologized virtual body experience [...] some floating perspective [...] (2002: 5)
This position addresses the conundrum of the dualities of embodiment and then brings in the
technological dimension, relating it to my work and exploration of embodiment in the virtual, mobile
network. It also refers to the experience of embodying technology through virtual systems, like games
or the Internet, as a form of out-of-body-experience or extended mind. This gets into the cyborg ‘brain 195 Laura U. Marks, in her book The Skin of the Film: Intercultural Cinema, Embodiment, and the Senses (2000), might disagree, since she argues that film breathes, thinks and embodies the cultures and people it portrays. She argues that film contains tactile visual representations through sensual images evoking viewers’ memories of sensory experiences (such as water or the outdoors), or through film characters experiencing heightened states of sensory experience in the diegesis (such as smelling, tasting, hearing, etc.), or in the way that film cameras capture or pan across extreme body close-ups. This also relates to the symbolic order representation concept in Lacanian psychology used vastly in feminist film theory, which there is not space to discuss here.
159
in a vat’ ideas so popular in the 1990s, inspired by science fiction books and films of the time. This was
when computer enthusiasts dreamed of a time when we would no longer need our bodies and could
be pure mind [196], connected to a computer or streaming through a network or inhabiting a robotic
form. However, the more we learn about the wisdom and distributed intelligence of the body and mind
through neuroscience and biology, the less popular these fantasies become.
Idhe brings in the notions of permeability of the body outline or skin boundaries, which help to support
a view of the extended body:
[...] the sense of her here-body [... is] extended beyond the outline of the wearer's body. (2002: 7)
This is relevant not only to consciousness, but also in terms of digital virtuality and virtual connection
through technological facilitation of body-mind extension, beyond the boundaries of flesh, but also
necessitated by corporeality.
To continue musing on the concept of ‘extended self’, one proposal is that when we are extending
outward we send and receive threads of ourselves; sometimes they are not received or are ignored, or
even received but not responded to. If a person sends their thoughts, love and emotional pain to
someone else through their thoughts, sometimes the sender will know these are felt by the intended
other, but sometimes the thread hits a wall. Digital connections, through text messages especially,
seem like more tangible manifestations of those threads, and when the messages no longer come,
their imprint is still left behind. The text itself still exists, until the phone crashes and erases them all, or a
new phone is bought [197]. More resonant is the invisible emotional imprint felt inside, the felt absence
of texts no longer incoming, the silence where the buzzing and beeping once lived. What does absence
feel like? It has more nuances, essence and sensation than the expected model of a void, lack or
emptiness. These are the threads between us – when there is historical digital connection through
technology (i.e. in the form of text-message or email exchanges), or a shared memory, shared
perceptions and experiences, then these emotional ephemeral links intertwine people, and can be felt
beyond the digital messages, beyond the physical body and physical/tangible space and time. How
does one untether from that extended self, entwined with tattered, frayed sub-threads that are matted,
enmeshed and entangled? Especially when the threads have endured, stretched, loosened and
tightened over the years? How does one explain the excitement of getting those regular beeping,
buzzing and blinking physicalised messages, especially when so many come causing a knee-jerk,
Pavlovian need-to-answer response in a flirtation and in a frenzy – when it is the only way to smile at
another far away, to hug them, to touch them, to kiss and to laugh with them. When that ends, all that
remains is the visceral imprint in the body of those sensations, sounds, memories and emotions.
196 Katherine Hayles critiques this in How We Became Posthuman: virtual bodies in cybernetics, literature, and informatics (1999). Donna Haraway’s Cyborg Manifesto (1984) also covers this territory.
197 This is a common experience with the technology, which can have the benefit of allowing one to move on more easily from the emotional baggage and memories such messages held.
160
Finally, this digital archive can be difficult to review, as it enacts the cellular memory and emotional
intensity that can result in a temporary paralysis from reliving the sensations and moment again, leaving
a painful longing, as one tries to recall why the interaction stopped. This leads to the question of how to
distinguish between the reality and the fantasy created by these emotionally laden threads, spun by the
text messages, which become imbued with expectation and desire.
I have reflected on the experience of digital absence in another way (modified from a research journal
entry):
The sense of digital absence can be explained from lived experience. My first cognisant experience of absence
of virtual contact came a few years ago. Suddenly, following a very active, almost daily, exchange with a friend
for several months. The abrupt change in the interaction was immediately disconcerting, leaving an
immediate, affective sense of lack, void or silence, invisible but felt viscerally. This disconnect was like a tearing
away of flesh. It was very frustrating since there had been a very close connection. What was most upsetting
was the ability to ‘see’ this friend online or available to chat often making it hard to resist attempts to
reconnect. This example, the sense of another’s presence can be felt when they are visibly still online and can
be seen in one’s chat contact list. This is similar to working in the same room with them. In this case, the
experience of digital presence created a real sense of loss or being rebuffed. Unlike when someone is in close
proximity, or geographically local, where you can visit their home or run into them in commonly visited places,
or even ask their friends how they are, when there is a digital absence one has no recourse, you are left
dangling in the digital abyss, feeling the void affectively and emotionally.
This reflection speaks to Borden’s earlier point on ‘Social Presence’ for mobile chat applications, and
also points to the negative ramifications of ‘available to chat’ status as a means to increase emotional
space and absence through digital means. However, the psychology of digital relationships is not an
area that can be delved into more deeply here. I add it to show how technology can be embodied and
have a physical or visceral impact on the body, in the sense that Massumi speaks of intensity and the
triggering of emotions. Even if digitally manifested, these are actual and ‘real’ experiences, felt in the
flesh.
Returning to the notion of absence, Merleau-Ponty explores presence through absence:
[...] an object[’s ...] presence is such that it entails a possible absence. Now the permanence of my body is different in kind ... It defies exploration and is always presented to me from the same angle. Its permanence is the permanence in the world, a permanence from my point of view [...] Insofar as it sees or touches the world, my body can therefore be neither seen nor touched. What prevents it ever being an object, being "completely constituted," is that it is that by which there are objects. It is neither tangible nor visible insofar as it is that which sees and touches. (2002/1962, 2003: 90–92)
In phenomenological experience of virtual absences, when a friendship or romantic relationship involves
great geographical distance, followed by intermittent, proximal, then in-person interaction, interspersed
with long spells of merely virtual interaction, even ‘live’ text, voice or video chat cannot bridge or fix the
fragile communication issues or misunderstandings in the same way that live interaction might. If the
161
person on the other end does not respond, they can vanish without a trace. This is one of several
instances where it could be argued that there is a superiority of in-person liveness and presence, in
terms of conflict resolution and problem solving approaches to interaction.
To reiterate an earlier point: to maintain relationships and communicate effectively, distance must be
reduced. Technological substitutions can be used to fill the gap, but in-person, corporeal presence is
the most effective for complete human communication (Lewis et al., 2000: 205) [198]. This might
appear to refute the argument that we embody space through our virtual consciousness. However, it
supports the point that complete communication requires ‘the real thing’, since much is expressed that
cannot be translated digitally.
In summary, space, space between, and absence are interwoven as part of a continuum between
physical space and virtual space. The MINDtouch project has been about traversing physical space
or distance and embodying it with the aid of technology, though not replacing physical presence. I have
attempted to exemplify how I have conceived and worked with space and space between, with
physical bodies in physical locations connected through mobile phones.
8.3/ experience senses, perception and affect
To study participant experiences of feltness and presence using the body, we need to look at the
senses and perception. Merleau-Ponty starts this discussion by pointing out how we think about the
senses and perception:
We think we know perfectly well what 'seeing', 'hearing', sensing are, because perception has long provided us with objects which are coloured or which emit sounds. When we try to analyse it, we transpose these objects into consciousness [...] what we know to be in things themselves we immediately take as being in our consciousness of them. We make perception out of things perceived. (2002/1962: 5, my emphasis)
Therefore, although we think of our experience of objects as within us, we are actually reacting to these
objects through our senses, which transpose or interpret the objects outside into perceptions inside
our bodies and consciousness. This is similar to Sheldrake’s extended mind theory (2003: 122). He
postulates that our minds reach out to the world outside of us, and might agree that the body
processes these perceptions in order for us to respond, as in the case of sensing imminent danger
[199]. In philosophy, Merleau-Ponty explores the contrasting notion that we cannot understand our
198 To support Lewis et al, Cummings, Butler and Kraut did a study that shows that […] in one-to-one comparisons, an email message is not as good as a phone call or a face-to-face meeting for developing and sustaining social relationships […] Relationships sustained primarily over the Internet are not as close as those sustained by other means (2002: 19).
199 Sheldrake discusses studies with animals that show that telepathy is likely to be an evolutionary trait that keeps animals safe (2002: 111–112).
162
senses and perceptions, because our main articulations of them are through the different
interpretations of them through the disciplines of physiology and psychology. Yet our intellectual
constructions fail to frame our experience of ourselves and our perception adequately. He says:
There is no physiological definition of sensation, and more generally there is no physiological psychology which is autonomous, because the physiological event itself obeys biological and psychological laws [...] 'sense-experience is a vital process, no less than procreation, breathing or growth' (Weizsäcker, quoted by Stein, J, 1928: 354) Psychology and physiology are no longer, then, two parallel sciences, but two accounts of behaviour, the first concrete, the second abstract. (2002/1962: 11)
To connect these two points, while there are different ways to articulate our sense experience and
perceptions, our bodies have evolved to understand, respond and in some cases communicate them
to others, all below our conscious awareness. Merleau-Ponty shows how psychologists interpret
perception with respect to understanding the senses through the sciences:
[...] sensation is not experienced, and consciousness is always conscious of an object. We arrive at sensation when we think about perceptions and try to make it clear [...] Pure sensation, defined as the action of stimuli on our body, is the 'last effect' of knowledge, particularly of scientific knowledge, and it’s an illusion (a not unnatural one, moreover) that causes us to put it at the beginning and to believe that it precedes knowledge. (2002/1962: 42–43, his emphasis)
Therefore, one can infer that our definitions of sensations are superficial and external to our experience
of them, and are only the beginning of a true understanding. He points out that we need to live, feel,
listen to, experience and describe them for ourselves to get the full picture of them. To do that, he says:
The function of the organism in receiving stimuli is, so to speak, to 'conceive' a certain form of excitation [...] exteroceptivity demands that stimuli be given a shape, the consciousness of the body invades the body, the soul spreads over all its parts, and behaviour overspills it is at the end of a chain of physical and physiological events which alone can be ascribed to the 'real body'[...] Cannot I find in the body message-wires sent by the internal organs to the brain, which are installed by nature to provide the soul with the opportunity of feeling its body? Consciousness of the body, and the soul, are thus repressed. (2002/1962: 86–87) [200]
Massumi adds to this through his reflection and deconstruction of various theories on perception,
neurobiology and physiology, phenomenology, and arts and entertainment. His interpretation of
neuroscience discoveries and other studies is of particular interest, especially when he explains how
perception and sensation work:
[…] the feeling of having a feeling is what Leibniz called the ‘perception of perception’. (Massumi, 2002: 14)
and:
200 This is fascinating and fundamental, since he uses the word ‘soul’ as a connecting or interpreting force for consciousness and body, which adds contention in the current climate of skepticism of spirituality, especially within the science community and mainstream media. This relates to my earlier discussion on spirituality.
163
[…] the vast majority of the world's sensations are certainly non-conscious. (2002: 16) [201]
He later adds,
Sensation is the mode in which potential is present in the perceiving body […] Perception is never only impression [...] Studding each impression are shards of intentions and conscious memories, most presently bearing on pregame strategy – shimmers of reflection and language [...] Toward that effect, in that sensation, a heterogeneity of levels contract into the body from which they reissue in an action [...] The play is the event-dimension doubling the empirical event-space in which the substantial terms in play physically intermix. (2002: 75)
Massumi refers to physically playing football, which relates to the approach used in my work in
assisting people to use their physical and other sensations and perceptions as a new form of non-
verbal expression, responding to and acknowledging their bodies through a different modality or
interface. These are the sensations to be harnessed in this research, through the wearable
biofeedback sensors and using mobile media to translate them into visual expressions.
Kozel continues on the experience of perception, the pre-reflective and the body’s way of
communicating sensation and experience to itself. She says that the abstraction of the ‘being of
sensation’ (referring to Rajchman):
[…] tends toward an ephemeral immateriality, yet its challenge to language and logic makes the pre-reflective seem to be the domain of corporeality. Can the pre-reflective be abstract and corporeal at the same time? […] The voice of physical experience spans registers of language: words, gestures, sound, colors, textures, smells. There is never any question of entrapping corporeality in the domain of the pre-reflective, as if it were forever mute, brute and material; bodies are at home at the fringes of language and have one foot squarely in the fertile ground from which new knowledge enters into existing conceptual paradigms, but they are also highly articulate. (2007: 21, my emphasis)
This sense of the physical spanning registers is a powerful concept explored in small degrees with
participants in my workshops and performances. Kozel later adds:
Once the aesthetic experience is freed from these [shared structures of thought and judgement], what happens? Does it remain blissfully unfettered by all things rational and cognitive? No, after the escape from thought, further thought emerges from the experience, even concepts, frameworks, structures of sorts, in a move that seems to be contrary to the fluid being of sensation, contrary to art, contrary to phenomenology. It is impossible to prevent thought from attempting to pin down experience just as it is impossible to prevent experience from blowing holes in structures that attempt to codify it. (2007: 22, my emphasis)
This rings true even if the thought is in another form of expression from language such as gesture or
drawing. What is freeing about this statement is that it clarifies the idea that experience and thought are
201 Again I want to stress that even though Massumi uses the word ‘non-conscious’, I define the non-conscious and unconscious as all part of the continuum of the mind/body consciousness – it is not a case of literal awareness or acknowledgement at all times, but of the overall system of consciousness.
164
ever-evolving and shifting, forcing us continually to adapt and be open to wherever they take us
conceptually, experientially and existentially. This, if one is open to a variety of perceptions and
experiences, can expand one’s approach to each moment of lived experience. This in turn can enrich
and challenge new expressions, art practices, or further thought and philosophical exploration, each
feeding into the next in exciting ways.
The senses work together automatically in the body, so this involves the different dimensions of
experience, each adding to the other to provide a larger picture of experience to the body and
consciousness. Merleau-Ponty addresses this:
[...] the intelligence ensure no more than an apparent communication between different experiences, and the synthesis of visual and tactile worlds [...] the constitution of and intersensory world must be effected in the domain of sense itself, the community of significance between the two experiences being inadequate to ensure their union in one single experience. The senses are distinct from each other and distinct from intellect in so far as each one of them brings with is a structure of being which can never be exactly transposed. (2002/1962: 261)
and:
The senses translate each other without any need of an interpreter, and are mutual comprehensible without the intervention of any idea [...] With the notion of the bodily schema we find […] the unity of the senses and of the object [sensed]. My body is the seat or rather the very actuality of the phenomenon of expression (Ausdruck), and there the visual and auditory experiences […] and their expressive value is the ground of the antepredicative unity of the perceived world, and, through it, of verbal expression (Darstellung) […] My body is the fabric into which all objects are woven […] in relation to the perceived world, the general instrument of my comprehension. It is my body which gives significance […] (2002/1962: 273, his emphasis)
Not only is Merleau-Ponty focussing on the unification and intelligence of the body and our individual
senses, but also upon our ability to translate sensation into the visual, audible and intellectual symbols
of language and comprehend them in multiple dimensions of perception. Grosz adds:
[...] senses cannot be so readily separated off from each other in the attempt to understand their specific modalities and properties, for in lived experience the senses interact, form a union, and yield access to a singular world. Sight and touch are able to communicate with each other, to provide confirmations (or contradictions) of each other, because they are the sense of one and the same subject operating simultaneously, within one and the same world. (1994: 99)
and suggests,
Insofar as the body can no longer be seen as a unified and unifying organism, an organism centered either biologically or psychically, organised in terms of an overarching consciousness or unconscious, cohesive through its intentionality or its capacity for reflection and self-reflection [...] (1994: 168)
This underlines the greatly coordinated process of perception communication taking place within the
body, and how the senses operate to report as much subjective experience from the objective world as
165
possible. Yet one could argue that there is no objective world; only our subjective experience of objects
and of being in the world, which our senses engage with to help us make sense of the world. This does
not mean that there is no objective reality out there, but that we only have our own senses, bodies and
perceptions with which to experience it. While I am proposing that we can extend our minds and
bodies, we cannot climb into another mind or experience the world through another body’s
perceptions to confirm our own. We can only empathise, communicate and share experiences with
other people and form a collective and shared understanding of objective reality, based on our
individual, subjectively perceived, embodied experiences. This is what is explored in this project: shared
subjective experience.
Grosz’ perspective suggests that the body is a collective intelligence, modelled on notions of the
collective consciousness (of humans and animals). She also highlights that phenomenological, reflective
reporting and other forms of expression of lived experience may be essential in order to understand
ourselves and each other, since science can only give us a part of the story. It reinforces the position
that, as there are so many things we do not understand about our bodies, consciousness and the
universe, a materialist view is no longer adequate.
These passages address different aspects of an exploration of the senses that have been relevant to
my work with participants, and efforts to aid them in tapping into their own inner sensations and
perceptions. Biofeedback sensing devices were used in the performances to pick up on various body
sensations and translate them into data, controlling the representations of the video interpretations of
the inner sensations from the video group. These theorists’ discussions of how the body works have
been salient in understanding and developing an approach to tapping into and exploring them more
meaningfully.
Massumi also posits a theory on the affective, sensory aspects and issues relevant to dealing with
sensors and the embodied exploration of art and performance. He focuses much attention on
perception itself, perception of sensation, and the concept that the body is always in a state of motion
and potential. His analysis has been grounding and important for work on the body, simulated
telepathy and mobile media performance, not to mention the visceral/embodied experience of
performance and liveness. His definition of ‘affect’, based on the theories of Deleuze and Spinoza, is a
rich area of thought and useful in framing this research:
Affect is autonomous to the degree to which it escapes confinement in the particular body whose vitality, or potential for interaction, it is. Formed, qualified, situated perceptions and cognitions fulfilling functions of actual connection or blockage are the capture and closure of affect. Emotion is the most intense [...] expression of that capture [...] (Massumi, 2002: 35, my emphasis)
This is how Massumi explains how the body understands and translates its own sensory experience,
while he translates the biology, physiology and neuroscientific explanations of these activities into a
166
more philosophical version of how affect, emotion and perception work. This transition of science into a
phenomenological and philosophical argument is at times also very vivid and poetic in its interpretations
of experience and perception. He continues:
One's 'sense of aliveness' is a continuous, nonconscious self-perception (unconscious self-reflections or lived self-referentiality). (2002: 36)
Then, working with Merleau-Ponty’s idea of body image, Massumi adds:
[...] the body without an image is the involution of subject-object relations into the body of the observer and of that body into itself. (2002: 57)
This conception of body image, mentioned first by Merleau-Ponty, also comes up in other body and
somatic, as well as neuroscientific and psychological, discussions. It is core to have a sense or image
of self, like a map to one’s body and perceptions. This body image enables us to move within the world
and differentiate, not only one experience from another, but also ourselves and our bodies from other
people, creatures, entities and objects encountered on a daily basis. In terms of presence – corporeal
or virtual – and otherness, we must have a map, a sense and an image of self in order to differentiate
ourselves from other presences. This is especially true within the less visible spaces of virtually
networked, mobile and online environments, not to mention within the telepathic or non-tangible space
between of emotional connection, over distance – and any other ‘threads’ that connect us less
tangibly.
Massumi introduces his theory on proprioception and other inner body senses relevant to biosensing
devices in this research:
[...] there are other modes of perception involved [...] proprioception, defined as the sensibility proper to the muscles and ligaments as opposed to tactile sensibility [...] and visceral sensibility. Tactility is the sensibility of the skin as a surface of contact between the perceiving subject and the perceived object [...] enveloping the skin's contact with the external world [...] (2002: 58, my emphasis)
Then he delves into the felt and visceral:
Proprioception translates the exertions and ease of the body's encounters with objects into a muscular memory of rationality. (2002: 59)
and continues:
The immediacy of visceral perception is so radical that it can be said without exaggeration to precede the exteroceptive sense perception. (2002: 60)
What is crucial here is the notion of the senses experienced through translation of perceptions and
physical response or further translation. Like a relay of experiences that eventually are conscious and
felt both physically and emotionally.
167
Viscerality for me is an internally felt response in the ‘gut’, a ‘gut feeling/reaction’; a ‘lump in the throat’;
‘butterflies in the stomach’. These are the proprioceptive sensations of the internal body that involve
hormones, pheromones, fluids and/or nervous reactions and responses triggered by emotions and
thoughts, affect or situations encountered. Massumi says that […] visceral sensibility immediately
registers excitations gathered by the five 'exteroceptive' senses even before they are fully processed by
the brain […] (2002: 60–61). A case also can be made when using biofeedback devices to measure
senses, that these visceral, proprioceptive sensations and perceptions are the truest, most
unadulterated or non-violated, pre-reflective experiences before they are translated to the mind. This
also refers to how an emotional state can be critical for sending and receiving presence, in both live
(performance or intimate) and virtual contexts, because it engages the individual in the flesh, which also
creates an ‘investment’ in engaging. Massumi adds,
[...] viscerality subtracts quality as such from excitation. It registers intensity […] The dimension of viscerality is adjacent to that of proprioception, but they do not overlap. The dimension of proprioception lies midway between stimulus and response [...] viscerality is the perception of suspense [...] the space of passion [...] degrees of intensity. (2002: 61, my emphasis)
Here Massumi is making the case that the emotional is physical, possibly more so than it is conscious
or of the mind, due to translations and transmutations. He states that physical intensity is triggered by
emotional stimulus, which serves to integrate the body and consciousness in a tangible, visceral way.
As such, he points out,
[…] an emotional qualification breaks narrative continuity for a moment to register a state – actually to re-register an already felt state, for the skin is faster than the word. (2002: 25, my emphasis)
Intensity is then qualifiable as an emotional state. Further, on emotion as intensity, he explains:
An emotion is a subjective content […] the quality of an experience which is […] personal. Emotion is qualified intensity […] semantically and semiotically formed progressions, into narrativizable action-reaction circuits, into function and meaning. (2002: 28)
So emotion is not purely experienced, but instead intensity is the experience of emotion; it is translated
by the body so that felt intensities are understandable to our consciousness. Emotion narrativises
intensity, with the mind telling the body a story of what it has just experienced or transmutes the
sensory to the conceptual space of the mind [202]. Such a complex series of processes can essentially
help us get through our daily lives. He continues:
The body doesn't just absorb pulses or discrete stimulations; it infolds contexts, it infolds volitions and cognitions that are nothing if not situated. Intensity is asocial, but not presocial – it
202 Benjamin Libet did studies to prove that we respond to situations with the body seconds before the mind catches up, but that when it does respond, the mind then ‘back dates’ the brain to time stamp it and tell the memory what was registered and experienced when it was happening (Massumi, 2002: 30). See more on his recent work and findings on this in Mind Time: The Temporal Factor in Consciousness (2004: 75).
168
includes social elements but mixes them with elements belonging to other levels of functioning and combines them according to different logic. (Massumi 2002: 30, my emphasis)
The idea that the body includes the social and its response to the world is relevant to thoughts on
intimacy and relationships brought forward by Lewis et al. On a related point on the experience of
intensity, explaining the source and nature of it in relation to perception, he says:
The disconnection between form/content and intensity/effect is not just negative: it enables a different connectivity, a different difference, in parallel [...] Intensity is embodied in purely autonomic reactions most directly manifested in the skin – at the surface of the body, at its interface with things. Depth reactions belong more to the form/content (qualifications) level, even though they also involve automatic functions such as heartbeat and breathing. The reason may be that they are associated with expectation [...] Modulations of heartbeat and breathing mark a reflux of consciousness into the autonomic depths [...] coterminous with rise of the autonomic into consciousness. They are a conscious-autonomic mix, a measure of their participation in one another. (Massumi, 2002: 25, my emphasis)
Further, on how intensity is experienced as emotion in conscious awareness and expresses viscerally,
he says,
Passion, then, is best understood less as an abstract space than as the time-stuff of spatial abstraction [...] An emotion or feeling is a recognized affect, an identified intensity as reinjected into stimulus-response paths, into action-reaction circuits of infolding and externalization – in short, into subject–object relations. (2002: 61, my emphasis)
This is a curious, if clinical, description of the experience of passion, but comprises a breakdown
necessary to understand it more clearly and objectively. Massumi adds:
Mesoperception is the synesthetic sensibility: it is the medium where inputs from all five senses meet, across subsensate excitation, and become flesh together, tense and quivering […] Mesoperception can be called sensation. (Massumi, 2002: 62, my emphasis)
This point argues for flesh as the ultimate interface for perception, and for communication with others.
In light of this, all synthetic and virtual substitutes pale – so live physicality is still superior. Idhe adds to
the debate of the senses and embodiment within the technological context by stating:
The ultimate goal of virtual embodiment is to become the perfect simulacrum of full, multisensory bodily action. (2002: 7)
This is contentious. Can we not already shift our perception and extend our senses? I would argue that
people do so in dream telepathy, albeit not with the same language of the simulacrum. Idhe is correct
to say that a ‘perfect simulacrum’ seems an ultimate goal, one that humanity has continually envisioned
attaining in order to shift our embodied, sensing selves. He claims this ‘perfect simulacrum’ could arrive
digitally through technology, whereas I would argue that it already exists telepathically, but needs to be
honed and enhanced. One could say that the MINDtouch project is an attempt to create a form of
simulacra, yet in a more abstracted way – externalising inner experienced senses and then translating
them using external visual material, rather than creating a perfect mirror or copy of how they literally
169
appear in the world. In contrast to a focus on virtual and visual construction and representation, we
might instead develop an external mirror view or ‘body image map’ to represent us. This would then
enable us to extend ourselves outside our internalised body’s image. So an internal, emotional and
sensory approach has taken this research toward the visual representation of self, as opposed to an
externalised visual representation of the extended body [203].
To follow from these arguments then, one could say that we experience sensations within the body
when we are connected to others emotionally, whether they are in the room, on the phone or across
the world – we can still ‘feel’ them, as we feel these intense sensations through regular contact with
them. In the last ten years or more, emotional connections are often maintained through additional
virtual presence. So where do we experience the connection? Inside of us, as the rest of the world is
experienced, through our bodies and sensations. This discussion shows how emotional and sensory
intensity affect the body’s basic automatic responses. For MINDtouch participants, this involved
wearing the biofeedback sensors during live events, to induce and capture certain inner sensations and
perceptions. This sense of intensity is registered as emotion in the awareness of the participants, then
explored and transformed visually as a form of communication.
Figure 5 © 2010 Camille Baker – key ideas and theorists on embodiment
This discussion has outlined the various theories on embodiment as it relates to sensation and
perception, which are important to this research, since it involves both sensing and the body directly
using technology.
203 I use this term from the point of view of Sheldrake’s extended mind, framed with the argument herein that the mind and body are one, but recognising that embodiment does not always include the corporeal flesh that extends our consciousness and thus our body.
170
8.4/ liveness – when is now?
The Oxford Dictionary definition of ‘live’ is: […] of a performance, heard or watched at the time of its
occurrence, as distinguished from one recorded on film, tape, etc. […] (Oxford Dictionary: 1989, 1047)
Here I shall unpack a few different understandings of liveness in human interaction and with special
focus towards liveness and real-time in performance. The intention of the practical performance aspect
of MINDtouch has been to explore how liveness and presence factor into a form of mobile ‘here’ and
‘now’. As such, the project has been an enquiry into whether one feels presence viscerally (as
discussed previously) during mobile performance, and whether people can be enabled to feel it too and
exchange it with others.
The mind map below was an initial exploration of all the aspects of the quality and experience of
liveness that are relevant to this research.
Figure 6 © 2009 Camille Baker
Any discussion and research on presence and liveness in performance must refer back to Philip
Auslander’s work, and his book Liveness, specifically on the quality of ‘liveness’ in a performance
setting. He puts forth that liveness contrasts with a mediatised performance on media technologies,
and that the idea of liveness did not even come to be considered before the use of audio, film and
video recording technology. For Auslander, it is important that liveness involves a certain tangibility and
tactility within the context of performance. These qualities are considered the feature or selling point of
liveness (Auslander, 1999: 55), allowing audiences the intimacy of witnessing from a particular
performer’s point of view, in order to convince them that they are somehow physically interacting with
the performer. Many performing arts patrons desire physical proximity with the performer; they tend to
171
report a sense of an immediate, intense, sensory and sensual reality. Auslander states that, […]
liveness is ideological [... the] belief that live confrontation can somehow give rise to the truth in ways
that recorded representation cannot […] (Auslander 1999: 128–129). This sensory, sensual proximity
during live, staged performances offers viewers smell, touch and biochemical reactions or exchanges
between performers and audience, or amongst audience members, which provide a quality of
authenticity in the performance.
Auslander makes the assertion that a mediatised performance engages all senses in the body
differently not only in magnitude, but also in kind. However, he fails to explain how or in what way. It is
not evident that a mediatised event can be experienced via the olfactory, tactile, somatic and
kinaesthetic or proprioceptive senses, although that does not imply that the viewer of a mediatised
performance is not experiencing the event on a deeply emotional and visceral level that can influence
the other senses. I would add that emotions are at the centre of a sense or understanding of presence,
and to some extent liveness. In addition, sensory information from an event, from its visual and audio
expression and reception, cannot have a profound affect on the body of the viewer unless they
experience emotional engagement or have been affected with intensity, in the sense Massumi
proposes. This in turn can trigger other senses to respond, and then the body can feel the event as
intensity, and not just witness it (such as with sports fans and sporting events) (2002: 73). It could be
argued that this triggered intensity can be missing during an in-person performance if the audience is
not emotionally engaged; and on the other hand can be profoundly experienced during a mediatised
performance if intensity is triggered, even if all other senses are not engaged during the experience.
Proximity offers a certain quality of emotion and immediacy – in hearing small changes in the tone or
quality of a performer’s voice, in subtle body shifts, even in something as subtle as a glance or making
eye contact – a viscerality like a touch (Auslander 1999: 85, Notes #35). Much communication happens
non-verbally – through gestures and subtle body language [204]. Thus, much is communicated through
proximity that is not possible in a mediated, non-live format. Yet the aim of this research is to get closer
to the experience of proximity through differently-used communication and performance forms,
especially in these times when we are less locally connected. As we are more engaged in distance
relationships and global business, as global nomads, we are also trying to find alternate means of travel
and connection to get away from our carbon-fuelled, planet-harming methods of being present with
each other.
There are many other factors not discussed by Auslander [205] regarding the experience of the viewer,
such as context, the emotional and physical state of the viewer or people with whom the viewer
experiences, or whether they are alone. Factors such as the nature of the event, the relationship
between the viewer and the performer, the occasion, and other factors like alcohol consumption should
204 As discussed earlier in Chapter 7, Lewis et al. show us that we need physical interaction to continue to develop our brains.
205 However, Auslander is relevant in terms of this discussion, even as a counterpoint to exploring and understanding liveness and performance.
172
also be considered. These factors compound the complexity of the physical and emotional experience
that accompanies the reception of a performance, whether live or mediatised, which Schechner and
Boal address more deeply.
The sense or experience of ‘liveness’ that I have come to know originates from previous dance and
music performances, but also from the use of digital technology. Liveness for me is being intertwined
simultaneously in time and/or space and physicality. The concept is enmeshed with presence, but is
not the same quality. Liveness for this project has also been about real-time. Connotations of liveness
involve being physically present in person, in the here-and-now, at this moment, in front of an audience
in a traditional performance context, live or remotely (as with live TV broadcast or an online event).
Liveness often entails an event or experience that is taking place at a certain point in time, at an exact
moment, at some location, somewhere, with someone witnessing it in person. For my definition, it is
not a priority that people are experiencing it in person or even in the same physical locale, as long as
the liveness is happening simultaneously for all wherever they are, through the medium of technology.
Not everyone must be in person, ‘in the flesh’ at the event location, but the event is experienced as
taking place now and must be witnessed now, in real-time or within seconds. In a digital context, now-
ness is critical to live events, but being physically present is not: one can use a computer with an avatar
as a stand-in to substitute for actual bodily presence, or use voice-over IP or video conferencing to
replace the corporeal presence, depending on the context–level of granularity, and so forth.
As a researcher, getting to a true understanding and identifying the effect, reception and experience(s)
of liveness, aura, presence of the performer and the audience, to reveal their intrinsic embodiment
within the network, can only be achieved phenomenologically through participants’ experience. Then,
through my observation and experience of participants’ experience, a hetero-phenomenological lens
can be applied. In the MINDtouch project, a goal has been to deliver this mixed liveness with ‘real-
time’ precision simultaneously – or as close as is possible within technological and logistical limitations.
This visceral liveness has been difficult to maintain in practice within the mobile network, but a form of
viscerally felt embodiment has been experienced through the videophone. Thus, virtual presence can
be created by emotionally and affectively experienced responses and sensations in the body, triggered
by the activities during the performance event (cf. Massumi). In a related sense, Richardson speaks of
technosoma, which she describes as follows:
[…] in the context of mobile media, I suggest that both tool and body are covalent participants – and coalesce as various technosoma – in the making of meaning and environment. It is our openness to the spatially non-coincident flesh of the world — not only bodily flesh but the 'stuff' of our environment — that allows us to incorporate technologies and equipment into our own corporeal organisation […] Each –soma blending can work to describe a way of being rather than a what of being; any soma is not an entity – as 'the body' might sometimes be understood – but a process/network or ontological schematic. This terminology aims to effectively combine the concepts of intercorporeality and medium specificity, and also to grasp the way in which individual bodies digest the collective embodiments or shared habits of the wider cultural milieu. (2005: 5)
173
Is there such a state of liveness or is it possible to be present now, in this moment? Does 'now' exist?
Perhaps we are never really just being ‘in the now’. Pondering the expression ‘be here now’ [206]
exposes the underlying elements of the spatiality of ‘here’ and the temporality of ‘now’ [207], ‘at this
moment’, and this one, and this one … So how can we ‘be here now’, and where is here? Does it
need to be a physical or tangible space/place or can it be the space/place of one’s own body? Or can
it be in transit in the ephemeral space between bodies and sensors and phones? In the telepathic or
dream space between bodies, can others feel your ‘being’ now? Can you play and perform with this?
And when is now? How can we savour now and be present and live or in a specific space? Being in a
specific place, in person, no longer matters during a virtual or mobile space/experience, but does time?
How can liveness and presence factor into the form of mobile ‘here’ and the never really ‘now’? Can
we feel it in our bodies and can we help others to feel it too? Can we share it with others? Can we be
‘here’ in our bodies now, as now slips away into future, as we are always becoming? These are
questions that are taken from daily life into my media practice and this research.
What makes a networked performance ‘live’ in time, and not merely space? Is it the time specificity of
an event, the fact that it occurs at a precisely agreed point on the clock? This is debatable if the event
is online, since the people who join the event are in different time zones. Therefore, the number of the
hour is different, but the minute and seconds are the same globally (as Mitchell explores – see below).
However, if the event takes place for a given length of time, starting and ending at a certain point, and
is somehow transmitted in a manner that means that others need to ‘tune in’ as it is happening so as
not to miss the event, then the now-ness of temporality is vital. The key is that it takes place at a given
time, for a given duration, and needs to be participated in or witnessed (by whatever means possible),
as it is taking place ‘on’ the net and in a ‘real’ or actual place (whether digitally over the network or in a
physical location, for example, streamed from a live concert in Wembley Stadium) [208].
Why is being live during performance, in real-time, important? How is it critical conceptually,
technologically, physically, virtually or otherwise? Does it make the experience more authentic? Where
does authenticity of a performance or experience come in? Is there such a thing? Is it even possible to
actually be live? In an attempt to partially answers these questions, Merleau-Ponty discusses aspects
of the body in time relevant to a conceptualisation of liveness and the importance of temporality to
networked events:
The spatial synthesis and the synthesis of the object are based on this unfolding of time. In every focusing movement my body unites present, past and future, it secretes time, rather it becomes that location in nature where, for the first time, events […] present a double horizon of past and future [...] My body takes possession of time; it brings into existence a past and a
206 Alpert (1971) coined this expression with his book Be Here Now.
207 Which Lakoff and Johnson addresses more in terms of how we speak and think of them, which is culturally constructed (Lakoff and Johnson, 1999: 151). 208 During the Performing Presence conference (Exeter, 2009), such questions and issues arose and attendees presented their own unique interpretations. No conclusions were really drawn; only more explorations and ways to think about and address the issues.
174
future for a present [...] but creates time instead of submitting to it. But every act of focusing must be renewed, otherwise it falls into unconsciousness. (2002/1962: 278–279)
Massumi adds:
Technologically assisted channeling of event-transitivity constitutes a qualitatively different mode of power than either the regulating codifications of the Static or the regularizing codings of the ‘social’ or ‘cultural’ [...] The network distributes. Interlinks. Relates. The network is the relationality [...] It is the being of collective becoming. (2002: 86)
This makes the imperative of liveness in a performance critical. It is also a perfect elucidation of the
activity of facilitating virtual presence and transmitting it over a network, because it highlights the
interpersonal connection through these electronic threads. In this sense, Massumi makes the idea of
liveness less relevant – since ‘collective becoming’ is ongoing, so the body and such virtual events are
ever arriving, always in the process of becoming. He continues:
Each event transmitter is sustained and delivered by a dedicated collective apparatus deploying at least one technology of channelling that gives it body in the interval, where it disappears into its own immanence [...] (Massumi, 2002: 87)
For my work, a reasonable measure of precision and faithfulness to an exact moment or attention to
the timeliness of creating a precise moment of liveness or ‘nowness’ is critical during a scheduled
event. It needs to happen within fifteen seconds of the event’s start or transmission commencement (if
online or mobile and much less in-person) – this is considered an imperative during my performance
events. It is important if exploring this experience conceptually within the practical context, as I have
been. However, like many practitioners of networked, electronic and new media, I recognise the
technological limitations, lags and delays that are inherent in such ‘live’ media performance events.
Potential and becoming are the ‘virtual’ for Massumi [209], since we are never static, never actualised
fully in the now and always in the state of becoming something or someone else, never arriving
completely formed and always having potential to become. This argument for virtuality connects to
liveness and the question of time – ‘When is now?’ – and how an event can be live, as liveness pertains
to a media transmission. To this he says:
[...] the event-space of the home must be seen as one characterized by a very loose regime of passage. As a regime of openness to sign circulation – to the delivery, absorption, and relay of sounds, words, and visions – home is a node in a circulatory network of many dimensions (each corresponding to a technology of transmission). (2002: 85)
Yoga and meditation focus strongly on ‘being here now’, with attention focused on the internal
experience of the body, on the chanting of a mantra or on breathing while keeping the body still in a
specific posture, trying to void the mind of thought, but being attentive to whatever sensations or
209 Though it is beyond the scope of this thesis to discuss Bergson’s time and duration, it is worth mentioning that much of what Massumi says about being, becoming, time and actualisation originates from his work.
175
perceptions happen to arise in the moment. This relates to the discussion on Merleau-Ponty’s and
Idhe’s ‘here-body’ [210], although it manifests differently, as mentally focussing intently on clearing the
mind of all thought.
Pierre Lévy (1998) addresses liveness and becoming through the network by contributing his concept
of virtuality, with real versus virtual, actual versus possible, in the context of how the body, text,
economics and culture are virtualised. He explains what virtuality means philosophically and practically
to modern society. He posits that, based on the Greek definition of virtualis, current philosophy adopts
the virtual as potential rather than as actualization. He also believes that ‘virtual’ has come to mean
absences of existence rather than tangibility, but also much more than that (1998: 23). He puts a great
emphasis on distinguishing between ‘virtual senses’ and ‘real senses’, which may not be so divergent,
since we engage the physical and affective when experiencing both virtual and real stimuli. Lévy’s
emphasis is on how virtual collective intelligence on the Internet has the positive potential for freeing
social activity, as well as enhancing knowledge production and dissemination. This aspect of collective
intelligence relates well to MINDtouch ’s multi-interaction, collective/collaborative approach, and
partially inspired the practice as a modality to bring people together to create and experience
collectively.
A personal interpretation and experience of ‘virtuality’ reveals that it can be accessed through the
imagination or the dreaming body, and conveyed from within an altered state, expanded sensory
awareness, perception, imagination and daydream imagination, or extrasensory perception. It also
includes embodied perceptions of reality, which create our natural, invisible, inner worlds and
consciousness, the real experiences within an altered dimension, or dreams, psychic experiences,
meditation, mysticism, near-death experiences, intuition, or vision quests involving physical pain (Baker,
2004: 33). Virtuality is about emotional connection with others not spatially or temporally linked.
The concept of working in real-time must be addressed, as it is a goal of this project. Real-time in the
networked digital world is a variable concept, and Mitchell addresses this by saying:
[…] there is more to the construction of time than the increasingly precise subdivision of the day. As clocks multiply and distribute themselves spatially, the relationships among them begins to matter. (2003: 12)
and,
As transportation and telecommunication capacities have increased, we have entered the era of globalized network time – of GMT, time zones, and sleep cycles decoupled form the solar day. (2003: 13)
The effect of this is that networked, digital ‘real-time’ activities are not predicated on where something
is happening, or on what time it is at any one location; it is more important that wherever and whenever
you are on the planet, you are experiencing the event or activity on the network now, as the action is
210 Being physically in body, rather than the mind wandering from it.
176
unfolding. Mitchell adds to this […] if you take advantage of fast machines to compress process, you
can elide the distinctions between simultaneity and sequence. (2003: 13) In this way, through mobile
and networked media, we can be in ‘real-time’, anywhere, virtually, even if we are not ‘there’, where the
event is physically produced or taking place.
Mitchell speaks of time in terms of ‘quantum time’ (2003: 14) and the state of limbo [211] that is
experienced as information transfers through the network, or as someone travels through the air in a
plane. He describes it as a sense of nowhere, a no-time space or a state where time and space is
traversed, but not defined. In this respect, quantum time is very much like ‘becoming’ in the Massumi
sense. Addressing this state or space, Mitchell says: [...] my sense of continuity and belonging derives
from being electronically networked to the widely scattered people and places I care about (2003: 17).
Looking at time in terms of the digital realm of network delay, lag, or stops and starts in streaming,
Kozel describes her experiences in creating and performing telematic and networked, interactive
works. She tells of issues of delay and lag in the system or network, during a ‘live’ interaction, as
opportunities or even as features of the experience in this form of performance. She reveals at one
point:
Performance using low-bandwidth Internet video conference links call attention to the never-ceasing flow of interaction among bodies, spaces and equipment by means of consistent interruption. There is a shifting and juddering unpredictability to the movement of the image, and to the regular crashes or adjustments required for maintaining a link. Nothing is inert. The fragility of interaction across distances echoes the vulnerability of proximity […] All interaction can be seen in terms of the negotiation of flow […] (2007: 108–109)
This reflection refers to a 1997 piece she performed in. However, using mobile video ten years later, I
have encountered the same issues with telecommunications technology and the network for mobile
video interaction. Kozel’s point is that it is important to view the pauses, delays and failures in
transmission and interaction as metaphors for more immediate physical forms of human
communication or interaction, which are as easily blighted with the same pauses, delays and failures in
understanding. Thus, we continue to encounter such challenges with both technological and human
interaction, and use these as opportunities to prompt solution-seeking.
In summary, this chapter explores conceptual and practical issues around the notion of liveness, and
its related quality of real time, or the more generalised shift in how time is perceived and played with
when using new networked technologies for performance.
211 This is another type of space between. There is not space here to discuss it, as the term has a Christian origin, regarding the status of one’s soul in oblivion. In common language, limbo means waiting in an indeterminate state, lacking resolution or certainty. It is a state between decisions.
177
Part III: conclusions & references for further study
178
9/ thesis conclusion:
This conclusion sets out to achieve three final aims:
1) to present a strategic summary analysis of participant feedback that has informed both the
theory and practice of this thesis;
2) to demonstrate the original contributions to scholarship achieved by the thesis project itself;
and
3) to outline a number of new areas for future research arising from this work, potentially of
interest to future scholars and practitioners alike.
9.1/ implications and new directions
In the thesis as a whole, I have gathered and analysed individual experiences described by participants
in the project’s mediated participatory events and explained the event concept within the virtual ‘space’
of a mobile. Networked performance has been analysed, interpreted and correlated to the technology
used to enable the participatory experience. This includes the practice of facilitating the sensing of
embodiment and liveness, within an in-person and remote, mobile participatory event. The contribution
to knowledge includes the explorations and attempts to create and study distance perception and
interaction of the sensation of liveness and felt presence of others (or lack of same), within real-time, in-
person and telematic events, using mobile and wearable devices and interfaces. I posit that mobile
media phones can become a medium or conduit for expression of non-verbal experience, and an
extension of the body/mind in non-literal or abstracted ways: a vehicle to express embodied
consciousness with others, over distance.
With this research, I have explored and straddled various artistic and academic disciplines,
communities, practices and methodologies: the new era of mobile arts, interactive, Internet
performance and installation art, wearable technology and fashion, body practices and philosophies, as
well as telecommunications technology. The unique knowledge contributed also includes a
phenomenological perspective on the experience of the process of creating a mobile media, networked
performance project and the complexities encountered when working with mobile networks as against
the Internet.
This exegesis has described the practical aspects of the media art PhD project MINDtouch , as
manifested in the initial participant video collection workshops, conducted 2007–2008. The material
179
from these workshops was used during a few live events, staged from July 2009 to early 2010, with
participants wearing biofeedback sensors, which interacted with mobile devices. Observations,
discoveries and outcomes from all phases of the project, as well as the technological challenges faced,
have been outlined.
Theoretical notions on liveness, presence, embodiment, non-verbal communication, telepathy,
participatory performance and mobile media, as well as live cinema, database cinema and networked
performance were explored herein. These theoretical ideas guided my approach to the practical
implementation and development, including: the workshops, iterative mobile and VJ software
development, and the performance events. The ideas guided how participants were facilitated in the
participatory activities and what would be focused on, as well as how the software was developed, and
how the participatory events were staged overall. The intricacies of interweaving the theory and
practice have been discussed here as well.
Due to time and resource pressures it became difficult to complete the work of studying participants’
experiences of feltness or exchange of presence using mobile video, through the mobile network
during participatory performative activities, as originally envisioned. However, phenomenological
observations of my artistic and facilitation process, as well as the ethnographic observations and
analysis of responses of the participants during the activities, have been greatly instructional. The actual
video material created, as well as participant feedback on other significant outcomes, has been
discussed.
I learned that the embodiment of technology, the extension of presence, and consciously directed
emotional and interpersonal connection through the mobile devices did occur for some through the
activities described, but differently than anticipated. Through intent and desire to connect, participants
were able to send and transform their presence through the device as one does through Internet
engagement. I now see that through these qualities of intention and desire, we embody or send our
presence through our technologies and over distance. I experienced a sense of liveness, of in-person-
ness, during the last event, when preparing participants to start their video streaming. This was evident
when, while dressing a participant in a biofeedback garment, several text messages suddenly showed
up on my mobile, instantly, in real time, from the remote group who were waiting online on the qik
website. These remote participants were asking, in real time, when we would be starting the live
exchange; my response was to urge them to keep waiting a bit longer for us. These online participants
did seem to experience some liveness of the stream of video, in real time, though they were unaware
that the footage that they were seeing mixed was not live video upstreaming from the event. What they
saw was a live mix of the archived videos, which they thought was live. These still account for
manifestations of liveness that were created and experienced. This experiment continues, and is closer
to manifesting whole connection, completing the technological loop of a body-to-video to body-
through-video interaction, even though for the purposes of the PhD the technological development had
to be paused to complete the written part of the research. More still needs to be done to investigate
180
the nature of presence and liveness experienced by participants in a mobile performance context: in
future versions within various social contexts and with various different groups.
The goal has been to interface technologies to aid in connecting remote groups, enabling them to re-
engage with each other, affectively and expressively in new non-verbal/textual, creative ways. Thus,
this project brings together divergent areas of new media research and media art/performance
practices, and contributes to additional ways of working with wearable devices, mobile phones and
video in performance and participatory activities. It also contributes new ways of thinking about
presence and embodiment in the digital media context with respect to ‘interfacing’ the various
technologies, practices and ideas. Knowledge contributed here includes a nuanced perspective of
creating body-situated, participatory, mobile media performance work. This work could also have
implications in the health and healing industries, education, affective computing/sensing domains, new
forms of mobile media entertainment, and social and mobile computing fields. In addition, it provides
recommendations for live, collaborative, locative and networked performance circles, in tandem with
wearable tech art and other areas of interest. It could also have further applications in VJng/Live
Cinema and database cinema practices. Yet its most practical contribution is the proposal of a new
mobile social networking platform/product to engage and entertain people socially.
After careful consideration of the concepts, similar art and research projects and MINDtouch , there
remain questions for future consideration and research: is technological mediation as an extension of
embodied presence attainable or fully desirable? Will 3D surround projection TV or other new visual and
sensory technologies bring us closer to real, in-person presence? Do we want such a form of digital
representation, or should we return to and continue to cultivate more personal approaches?
My own answer to these questions, informed by this research and process, as well from conversation
with people about digital versus in-person interaction, is that most people do not want the digital to
replace in-person presence in life or in performance. As discussed herein, corporeal, fleshy physicality
is the ultimate experience and condition for optimal interaction, communication and indeed survival,
which we evolved to experience (Lewis et al., 2001). It has been made clear by A General Theory of
Love [212], earlier in Bowlby’s work, and other related relationship studies, that humans need in-person
contact. This interaction establishes and maintains intimate bonds that continually develop neural
pathways in the brain. As I study, think, feel and live, I am convinced that liveness and bodily, in-
person, fleshy presence is the only the only ‘real’ or authentic experience, the only way to clearly
convey emotion and to have sustained and meaningful relationships with others.
However, with the ever-increasing destruction and damage to our life-sustaining planet through
excessive carbon-spewing travel, for business, food distribution, pleasure and trade, it is clear that we
now need to find and use other solutions. Instead of face-to-face interaction and immediacy all the
time, we need to continue to develop alternatives to fleshy contact. We need alternatives that feel real
212 See earlier discussion on this.
181
and authentic, and convey trustworthiness, both emotionally and physically. In my view, the only valid
rational justification for substituting the fully embodied experience of in-person liveness and presence
would be in order to save money, resources, lives and carbon emissions. This could prevent other
destructive activities that threaten life under our current socio-economic, political and environmental
frameworks. This is my conclusion and one that was echoed in one form or another by many artists
and researchers who have discussed it during my research [213].
Where does this perspective leave the performance aspect of this research? This aspect remains under
investigation since the work is still in progress. Cultivating mobile presence in the live and mobile
performance context is still to be explored. I hope to continue this work and develop the technology
further, creating more participatory performance activities and events, studying more participants and
their perceptions, before making final assessments on liveness and presence in these contexts. But
from the phenomenological discoveries regarding the mobile phone and digital communication and
presence, I find myself still wondering, but with new insight to better prepare me for these continued
studies.
Throughout the PhD research process and the move to the UK, I have learned and experienced much
that has underlined my concern for liveness and presence over distance. It is important within human
relationships, communication and performance. This awareness has progressively become clear and
pronounced. Coming to the UK from Canada would not seem a drastic move, but it has made me
appreciate my former life and view it with a new perspective. This process has become very personal,
as well as academic and artistic. All of my experiences during this time have become relevant and
applicable to research into traversing distance and exuding presence over a digital and mobile network.
In addition, during the period of my research, the world has gone through some radical transitions and
shifts, highlighting the fact that globalisation has had a profound and sometimes problematic impact,
causing people to be ever more distant from each other. I wonder, if we could learn to ‘be present’ with
each other, feeling and validating that presence, and try to communicate and empathise with one
another better, perhaps then we could improve our world, as well as our personal experiences of life as
we are living it, phenomenologically, and hopefully evolve further as a species and society. My
motivation in this respect, and in my practice, is therefore one of social obligation, to make a
contribution that has some social meaning and value, to affect and assist people, as well as to help
them connect with each other more deeply.
This work continues.
213 Such as at the Performing Presence Conference in Exeter, England, March 2009.
182
9.2/ in conclusion This thesis benefits from access to the larger MINDtouch project and its original data set of process-
based evidence files, in both video and transcript form (contained in the thesis Appendices). By
analysing this unique data set and applying the theoretical contexts of kinaesthetic philosophies where
appropriate, the thesis demonstrates both the practical and the critical/contextual effectiveness of
media facilitation processes for the participants, and shares their senses of ‘liveness’ and ‘presence’ (of
themselves and of others) when using technology to externalise visual expressions of internalised
experiences.
The thesis engages with the major MINDtouch case study and applies theories from fields of
kinaesthetics and phenomenology, including current theories of embodiment, in considering notions of
presence, liveness, ‘feltness’ and ‘telepathic exchange’ (all terms defined at length in these pages). The
research brings together new thinking in the fields of media art theory, telecommunications,
neuroscience, performance and new media practices. The practice-based approach to exploring
theory through practice has involved designing and applying bespoke biofeedback sensors to interface
with mobile phones, in the context of performance workshops incorporating theatre games and other
creative approaches to ‘being’ in both live and remote spaces. The thesis has combined
methodologies, methods, practices and tools from Phenomenology, Ethnography, Practice-As-
Research and Experience Design, bringing together relevant aspects of these divergent areas of new
media research and media art/performance practices.
This thesis makes an original contribution to scholarship in the fields of performance and new media,
with cognate contributions to the fields of philosophy and technology, and locates its arguments at the
locus of the fields of Performance Art, Mobile Performance/Locative Media, and Philosophies of the
body and Communications. The research (supported by original data gleaned from major case studies)
demonstrates that there is a need for new technology tools to express viscerally felt emotion and to
communicate more directly. The findings show that when intimate ‘dream’ states are to be shared by
participants, it is often the case that written and spoken language can block clear creative expression.
On the other hand, more intimate media tools can help to enhance a sense of liveness and presence
by supporting both the sharing and receiving of ‘emotional’ messages via ubiquitous technologies
linked by fairly simple online network protocols. It is hoped that this study will be of use to scholars in
the arts and technology, and that it may help to demonstrate a way to communicate rich emotion
through felt and embodied interactions with others across vast distances (thus supporting political
movements aimed at reducing global travel in the age of global warming).
183
bibliography Adams, K. (2003), ‘The Threshold of the real: A Site for Participatory Resistance in Blast Theory’s Uncle
Roy All Around You’ In Body, Space & Technology Journal, 6(1) [Online] Available at http://people.brunel.ac.uk/bst/vol0601/kateadams/home.html (Accessed Jan 2010) London: Department of Performing Arts, Brunel University.
Alpert, R. aka Baba Ram Dass (1971), Be Here Now, San Cristobal, New Mexico: Lama Foundation.
Amerika, M. (2005/7), ‘Excerpts From “Portrait Of The VJ”’, Fibreculture Journal [Online] Available at http://journal.fibreculture.org/issue7/issue7_amerika.html (Accessed 12 July 2006).
Amerika, M. (2009), Immobilité. [Online] Available at http://www.immobilite.com/ (Accessed 12 July 2010).
Ansell Pearson, K. (2002), Philosophy and the Adventures of the Virtual, London and New York: Routledge.
Argyle, M. (1975), ‘Non-Verbal Communication in Human Social Interaction’ In Non-Verbal Communication, Hinde, R (ed), London and New York: Cambridge University Press.
Auslander, P. (1999), Liveness: Performance in a Mediatized Culture, London and New York: Routledge.
Auslander, P. (1997), ‘Ontology vs. History: Making Distinctions Between the Live and the Mediatized’, In Georgia Institute of Technology: The 3rd Annual Performance Studies Conference, Conference Archives, [Online] Available at http://webcast.gatech.edu/papers/arch/Auslander.html (Accessed 11 July 2006), Atlanta, Georgia: The School of Literature, Communication & Culture.
Baars, B.J. (1997), ‘In the Theatre of Consciousness: Global Workspace Theory, A Rigorous Scientific Theory of Consciousness’, In Journal of Consciousness Studies Vol 4(4), Exeter, UK: Imprint Academic, pp 292-309.
Baker, C., Schleser, M. and Molga, K. (2009), ‘Aesthetics of Mobile Media Art’ , In International Journal of Media Practice, Volume 10 (2), to be printed July 2009 [Online] Available at http://www.intellectbooks.co.uk/journals/view-Article,id=8520/ (Accessed March 21, 2010), Bristol, UK: Intellect Press.
Baker, C (2008), ‘Liveness and Presence Bio-Networked Mobile Performance Practices: Emerging Perspectives’, In The International Journal of Performance Arts and Digital Media, Volume 4 (2), [Online] Available at http://www.intellectbooks.co.uk/journals/view-Article,id=4976/ (Accessed March 21, 2010), Bristol, UK: Intellect Press, pp 117-136.
Baker, C. (2004), Internal Networks: Telepathy Meets Technology in the DreamPod, Masters of Applied Science, Interactive Arts Thesis, Surrey, Canada: Simon Fraser University, pp 33.
Baker, C. (2003), ‘Sensorial Design for Interactive Art’, unpublished paper / project proposal.
Barker, C. (1977), Theatre Games: A New Approach to Drama Training, London: Methuen London Ltd.
Barrett, E. and Bolt, B. (eds.) (2007), Practice As Research: Approaches to Creative Enquiry, London and New York: I.B. Tauris.
Bartlem, E. (2005), ‘Reshaping Spectatorship: Immersive and Distributed Aesthetics’, In Fibreculture Journal: Distributed Aesthetics, Vol 7 [Online] Available at http://journal.fibreculture.org/issue7/issue7_bartlem_print.html (Accessed 20 March 2010) Melbourne, Australia: University of Melbourne.
184
Beauregard, M. and O’Leary, D. (2007), The Spiritual Brain: A Neuroscientist’s Case for the Existence of the Soul, New York, NY: HarperOne, division of HarperCollins.
Blast Theory (2003), Uncle Roy All Around You, performance group from Brighton, UK [Online] Available at http://www.blasttheory.co.uk/bt/work_uncleroy.html (Accessed July 2006).
Blomberg, J., Giacomi, J., Mosher, A., and Swenton-Wall, P. (2003), ‘Ethnographic Field Methods and Their Relation to Design’, In D. Schuler and A. Namioka (eds.) Participatory Design: Perspectives on Systems Design. Hillsdale, New Jersey: Lawrence Erlbaum Associates, Inc. pp 123 – 154.
Bishop, C. (2006), Participation, London: Whitechapel Press and Cambridge, Mass: MIT Press.
Boal, A., Jackson, A. (trans.) (1992), Games for Actors and Non-Actors, London and New York: Routledge.
Borden, N. (2007), ‘Research Insights: Social Networking and Mobile Communities’, In Idlemode: Voices for Better Mobile Experience [Online] Available at http://idlemode.com/category/mobile-presence/ (Accessed May 21, 2007).
Bourriaud, N. (1998, 2002), Relational Aesthetics, Dijon, France: Les Presses du Réel pp. 84-102.
Bowlby, J. (1986), Attachment and Loss, Volume 1: Attachment, New York, NY: Basic Books.
Bowlby, J. (1988), A Secure Base: Clinical Applications of Attachment Theory, Oxon, England: Routledge.
Brandt, E. and Grunnet, C. (2000), ‘Evoking the Future: Drama And Props In User Centered Design’, In Proceedings of the Participatory Design Conference, New York: CPSR Conference Proceedings, pp 11-20.
Bresheeth, H. (2007), ‘Elasticity of Narrative Time: Time Travel in Science Fiction Cinema’, In Journal of European Psychoanalysis, Vol 23 (2) [Online] Available at http://www.psychomedia.it/jep/pages/number23.htm (Accessed October 7, 2010), Rome, Italy pp. 59-84.
Brouwer, J., Mulder, A. and Charlton, S. (eds.) (2003), Making Art of Databases, Rotterdam, The Netherlands: V2_Publishing / NAi Publishers.
Burgess, M. (2002), ‘Support For New Media Arts In Canada: What Place For Artists’, In Sidestreet Art and Ideas, Online Journal, Fall 2002(2) pp 2. [no longer available online – pdf copy available on request].
Byrne, B. and Carvalho, A. (eds.) (2006-2008), ‘VJ Theory.net’, journal and forum [Online] Available at http://www.vjtheory.org (Accessed October 2007), Falmouth, England: self- published.
Carter, P (2004), Material Thinking, Melbourne University Press: Carlton, Victoria, Australia.
Chodron, P (1994), Start Where You Are: A Guide to Compassionate Living, Shambhala: Boston.
Christensen, M. S. (2006), As We May Feel: Interpreting The Cultural of Emerging Personal Affective Mobile Media, PhD Dissertation, Denmark: Department of Innovative Communication at the IT University of Copenhagen.
Conway, F. (2005), Art of Decision, interactive multimedia exhibition [Online] Available at http://www.artofdecision.net/ (Accessed October 7, 2010)
Costible, S. portfolio [Online] Available at http://www.sue-c.net/video.html (Accessed Jan 2010)
Corino, G. (2006), Spatial issues and performative media in digital mobility: a network perspective, Digital Futures, MSc thesis, Institute of Digital Art and Technology [Online] Available at !http://x.i-
185
dat.org/~gc2/digital_mobility.pdf (Accessed 13 July 2006), Plymouth, UK: University of Plymouth.
Cummings, J.N., Butler, B. and Kraut, R. (2002), ‘The quality of online social relationships’, In How the Virtual Inspires The Real, Vol 45(7), Communications of the ACM Journal, New York, NY: ACM, pp103-108.
Damasio, A. (1999), The Feeling of What Happens: Body and Emotion in the Making of Consciousness. New York, San Diego, London: Harcourt Brace & Company.
Dark Fibre, performances (2009-2010), British Film Institute and onedotzero [Online] Available at http://www.onedotzero.com/ & https://www.bfi.org.uk/whatson/bfi_southbank/events/dark_fibre_0 (Accessed Jan 2010).
Davis, M. (2000) ‘A Curriculum Statement: Designing Experiences, Not Object’, In Designing experiences, LOOP (1), American Institute of Graphic Arts, Virginia, USA: Virginia Commonwealth University Center for Design Studies.
Dawkins, R. (2 ed.) (1989), The Selfish Gene, Oxford, England: Oxford University Press, pp 192.
De Medeiros, M. B. (2003), ‘Performance art and digital bodies, In Body, Space & Technology Journal, 3(2) Department of Performing Arts, Brunel University [Online] Available at http://people.brunel.ac.uk/bst/vol0302/mariabeatriz.html (Accessed 11 July 2006).
Denizen, N.K. (2003), Performance Ethnography: A Critical Pedagogy and the Politics of Culture, Thousands Oaks, California, London, New Delhi: Sage Publications.
Dixon, S. (2007), Digital Performance: A History of New Media in Theatre, Dance, Performance Art, and Installation, Cambridge, Massachusetts: MIT Press.
Douglas, A. Scopa, K and Gray, C. (2003), ‘Research Through Practice: Positioning the Practitioner as Researcher’, In Working Papers In Art And Design, Vol 1[Online]. Available at http://www.herts.ac.uk/artdes1/research/papers/wpades/vol1/douglas2.html (Accessed 22 September 2006), Hertfordshire, UK: University of Hertfordshire, pp 5-6.
Doruff, S. (2006), The Translocal Event and the Polyrhythmic Diagram, PhD dissertation, SMARTlab Programme in Performative New Media Arts, Central Saint Martins College of Art & Design, London: University of the Arts.
Dourish, P. (2001), Where the Action Is: The Foundations of Embodied Interaction, Cambridge, Massachusetts: The MIT Press.
Faulkner, M/D-Fuse (2006), VJ: Audio-Visual Art and VJ Culture, London: Laurence King.
Farthing, G.W. (1992), The Psychology of Consciousness, Englewood Cliffs, New Jersey: Prentice Hall Press.
Feijóo, C., Gómez-Barroso, J. and Marín, A.A. (2007), ‘Why YouTube Cannot Exist on a European Mobile: The European Regulatory Strategy on Mobile Content’, In Mobile Media 2007: Proceedings of an International Conference on Social and Cultural Aspects of Mobile Phones, Media and Wireless Technologies, July 2-4, 2007, Goggin, G. and Hjorth, L. (eds.), Sydney, Australia: University of Sydney, pp 263-275.
FILMobile (March 2008), Conference and Exhibition [Online]. Available at http://www.filmobile.net (Accessed Sept 2009), London, UK: University of Westminster.
Freidman, D., Brogni, A., Guger, C., Antley, A., Steed, A., and Slater, M. (2006), ‘Sharing and Analyzing data from Presence Experiments’, In Presence Vol 15(5), Cambridge, Massachusetts: Massachusetts Institute of Technology.
186
Front, online magazine [Online] Vol 8, Available at http://front.bc.ca/research/texts/8 (Accessed Jan 2010)
Gackenbach, J. (1998), ‘From Sleep Consciousness to Pure Consciousness’, In Presidential address at the annual meeting of the Association for the Study of Dreams in 1988, London, England. [Online] Available at http://www.sawka.com/spiritwatch/from.htm (Accessed 10 March 2003). See also A Continuum of Consciousness in Sleep [Online] Available at http://www.sawka.com/spiritwatch/gotoa.htm (Accessed 10 March 2003).
Gates, C. (ed.) (2008), ‘Rise of the VJ’, In Vague Terrain Vol 9 [Online] Available at http://vagueterrain.net/journal09 (Accessed Jan 2010)
Glaser, B with Holtor, J. (2004), ‘Remodeling Grounded Theory’, In Grounded Theory Review, Vol 4(1) [Online] Available at http://www.groundedtheoryreview.com/files/gtreviewvol4no1.pdf (Accessed 20 June 2007), Mill Valley, California: Sociology Press.
Glesner, J. (2002), ‘Internet Performances as Site-Specific Art’, In Body, Space & Technology Journal, Vol 3(1) Department of Performing Arts [Online] Available at http://people.brunel.ac.uk/bst/documents/juliaglesner.doc (Accessed 10 July 2006), London, UK: Brunel University.
Goldenberg, D. and Reed, P. (2008), ‘What is Participatory Practice?’, In Fillip Magazine Vol 8 [Online] Available at http://fillip.ca/content/what-is-a-participatory-practice (Accessed 23 January 2010), Vancouver, Canada: Projectile Publishing Society.
Goffman, E. (1959), The Presentation of Self in Everyday Life, New York: Bantam Dell Pub Group.
Green, N. and Haddon, L. (2009), Mobile Communications: An introduction to New Media, Berg New Media Series, Oxford and New York: Berg Publishers, a Division of Oxford International Publishers Ltd.
Gumbrecht, H. U. (2004), Production of Presence: What Meaning Cannot Convey, Stanford, California: Stanford University Press.
Hauber, J., Regenbrecht, H., Hills, A., Cockburn, A. and Billinghurst, M. (September 2005), ‘Social Presence in Two- and Three-dimensional Videoconferencing’, In International Society for Presence Research, Annual Conference Proceedings, [Online] Available at http://www.temple.edu/ispr/prev_conferences/proceedings/2005/presence2005.pdf (Accessed 7 July 2006), London, England: University College London, pp 189-198.
Hegedus, C and Noujaim, J. (dirs) (2001), Startup.com, film [Online] Available at http://www.imdb.com/title/tt0256408/ (Accessed Jan 2010).
Idhe, D. (2002), Bodies in Technology, Electronic Mediations, Vol5, Minneapolis, Minnesota: University of Minnesota Press.
Jaeger, T. (2005), live cinema unraveled: Handbook For Live Visual Performance, San Diego, California: self-published, with funds from UCSD Russell Grant.
Jamieson, H. V. (2005), ‘Mixing the Web’, In Rhizome, an online media arts portal [Online] Available at http://rhizome.org/news/story.php?timestamp=20051121 (Accessed April 2006), New York: New Museum. Also see the WJ-S project [Online] Available at http://www.wj-s.org/ (Accessed April 2006).
Johansson, T.D (2003), ‘The Tactile Image: A Veronican Approach to Pictoriality in Michel Serres’s Aesthetics of the Senses’, In Sense and Senses in Aesthetics, Backstrom, P. and Johansson, T.D (eds.) Helsinki, Finland: NSU Press.
187
Kait, C. and Weiss, S. (2001), Digital Hustlers: Living Large and Falling Hard in Silicon Alley, NYC, New York: HarperBusiness.
Kaye, N., Giannachi, G., Slater, M., and Shanks, M. (2009), ‘Performing Presence: From the Live to the Simulated’, research project [Online] Available at http://presence.stanford.edu/. Also see more on that project [Online] Available at the Performing Presence Conference at the University of Exeter in March 2009 http://spa.exeter.ac.uk/drama/research/intermedia/conference.shtml (Accessed March 2009).
Kolb, D. (1984), Experiential Learning, Englewood Cliffs, New Jersey: Prentice-Hall.
Kozel, S. (2007), Closer: Performance, Technologies, Phenomenology, Cambridge, Massachusetts: The MIT Press.
LaBerge, S. and Rheingold, H. (1990), Exploring the World of Lucid Dreaming. New York: Ballantine Books, a division of Random House, Inc.
Ladly, M. (2007), ‘Being there: Heidegger and The Phenomenon Of Presence In Telematic Performance’, In International Journal of Performance Arts and Digital Media, Vol 3 (2&3) [Online] Available at http://www.intellectbooks.co.uk/journals/view-Article,id=4955/ (Accessed March 2010) Bristol, UK: Intellect Press, pp. 139–150.
Lévy, P. (1998), Becoming Virtual: Reality in the Digital Age, New York: Plenum Press.
Lewis, T. MD, Amini, F. MD, Lannon, R. MD (2001), A General Theory of Love, New York: Vintage Books division of Random House.
Libet, B. (2004), Mind time: The Temporal Factor In Consciousness, Cambridge, Massachusetts: Harvard University Press.
Lord, M (2001), ‘Fostering the Growth of Beginners' Improvisational Skills: A Study Of Dance Teaching Practices In The High School Setting’, In Research in Dance Education, Vol (2)1 April 2001, London and New York: Routledge, pp 19 – 40.
Makela, M. (2006), Live Cinema: Language and Elements, MA thesis, Media Lab, Helsinki University of Art and Design, Helsinki, Finland.
Manovich, L. (2005), Soft-Cinema, interactive art project [Online] Available at www.manovich.net and http://www.softcinema.net/movies.htm or http://www.manovich.net/cinema_future/toc.htm (Accessed August 2009).
Manovich, L. (2001), The Language of New Media, Cambridge, Massachusetts: MIT Press.
Marks, L.U. (2000), The Skin of the Film: Intercultural Cinema, Embodiment, and the Senses, Durham and London: Duke University Press.
Marshall, G. and Barthel, D. (eds.) (1994,1998), The Oxford Dictionary of Sociology, Oxford, New York: Oxford University Press.
Massumi, B. (2002), Parables for the Virtual: Movement, Affect, Sensation, Durham and London: Duke University Press.
Mazis, G (2002), Earthbodies: Rediscovering Our Planetary Senses, Albany, New York: SUNY Press, division of State University of New York Press.
McCarthy, J. and Wright, P. (2004), Technology as Experience, Cambridge, Massachusetts: MIT Press.
McKay, S and Jordon, J. (2006), portfolio [Online] Available at http://www.nomig.net/ (Accessed July 2006).
188
McKean, E. (2005), The New Oxford American Dictionary, 2nd Ed., [Online] Available at http://www.oup.com/us/noad (Accessed May 25, 2010), Oxford: Oxford University Press.
Merleau-Ponty, M. (1962, reprinted 2002), Phenomenology of Perception, London and New York: Routledge.
Mindell, A. (2000), Quantum Mind: The Edge Between Physics and Psychology, Portland, Oregon: Lao Tse Press.
Mitchell, W (2003), Me++: The Cyborg Self and the Networked City, Cambridge, Massachusetts: MIT Press.
Mixed Reality Lab, [Online] Available at http://www.mrl.nott.ac.uk/people.html (Accessed January 2010), Nottingham, UK: University of Nottingham.
Munster, A. and Lovink, G. (2005), ‘Theses on Distributed Aesthetics’. Or, What a Network is Not, In Fibreculture Journal 2005(7), Fibreculture Publications [Online] Available at http://journal.fibreculture.org/issue7/issue7_munster_lovink.html (Accessed 12 July 2006).
Murtagh, G.M. (2001), ‘Seeing the 'Rules': Preliminary Observations of Action, Interaction and Mobile Phone Use.’ In Wireless World. Social and Interactional Aspects of the Mobile Age, Barry Brown, Nicola Green and Richard Harper, (eds.), London and Berlin: Springer-Verlag UK, pgs 81-91.
networked performance, online blog (2005b), turbulance.org, an open online forum, topic: network-enabled performance [Online] Available at http://www.turbulence.org/blog/ (Accessed July 2005).
Neumark, N. (2005). ‘Introduction: Relays, Delays, and Distance Art/Activism’, In At A Distance: Precursors to Art and Activism on the Internet, Chandler, A. and Neumark, N. (eds.) Cambridge, Massachusetts: MIT Press, pp 2-31.
Nicholson, J.A. (2005), ‘Flash! Mobs in the Age of Mobile Connectivity’, In Fibreculture Journal 2005(6), Fibreculture Publications [Online] Available at http://journal.fibreculture.org/issue6/issue6_nicholson.html (Accessed 12 July 2006).
Nowak, P. (November, 20, 2007) CBC News online [Online] Available at http://www.cbc.ca/news/background/tech/cellphones/economy.html (Accessed May 25, 2010).
Oulasvirta A. Kurvinen, E., Kankainen, T. (2003), Understanding contexts by being there: case studies in bodystorming, Vol 7, Pers Ubiquit Comput, Springer-Verlag London, pp 125–134
Pearson, C. (1992), ‘Precognitive Dreaming’, In A Public Experiment in Precognitive Dreaming, (1993) The Annual Conference of the Association for the Study of Dreams (1994) [Online] Available at http://www.nauticom.net/www/netcadet/npcg.htm (Accessed 10 March 2003), Pittsburgh, PA: The Pittsburgh Post-Gazette Dream Experiment.
Phelan, P. (1993), Unmarked: The Politics Of Performance, London and New York: Routledge.
Philips (2007), Design Probe Program: Skin, interactive presentation of research [Online] Available at http://www.design.philips.com/sites/philipsdesign/about/design/designnews/pressreleases/skin_reddot2007.page and http://www.design.philips.com/probes/projects/dresses/index.page http://live.philips.com/index.php/nl_nl/video/design-probe-skin-dresses/25262423001 (Accessed September 2009).
Ponton, A. (2005), Vertigo: The Technophenomenological Body in Performance, Fine Art PhD Dissertation, London, UK: Goldsmiths College, University of London.
Presence Project (2006-2009), research project, [Online] Available at http://presence.stanford.edu/ (Accessed July 2009), Stanford University, Palo Alto, Ca; Exeter, UK: University of Exeter and London: University College of London.
189
Radin, D. (1997), The Conscious Universe: The Scientific Truth of Psychic Phenomena. New York: HarperCollins Publishers Inc.
Reason, P. and Bradbury, H. (eds.) (2001), Handbook of Action Research: Participative Inquiry and Practice, Sage: Thousand Oaks, CA.
Rees, A. L. (1999), A History of Experimental Film and Video, London: British Film Institute.
Rettie, R. M. (2005), Presence and Embodiment in Mobile Phone Communication, PsychNology Journal 2005, Vol 3(1), [Online] Available at http://www.psychnology.org/File/PSYCHNOLOGY_JOURNAL_3_1_RETTIE.pdf (Accessed 10 June 2007) pp 16-34.
Richardson, I. (2005), ‘Mobile Technosoma: Some Phenomenological Reflections On Iterant Media Devices’, In Fibreculture Journal 2005, Mobility, New Social Intensities and the Coordinates of Digital Networks, Issue 6 [Online] Available at http://www.journal.fibreculture.org/issue6/issue6_richarson.html (Accessed 10 June 2007).
Rogers, K. (1995), THE PSI NET PROJECT Psi-Phenomena - Telepresence - Virtual Reality, documentation on multimedia project, [Online] Available at http://web.ukonline.co.uk/Members/kathleen.rogers/psinetweb/psimain.html (Accessed 01/15/2003)
Rose, G. (2007), Visual Methodologies: An Introduction to the Interpretation of Visual Materials, 2nd Ed, London, UK; Thousand Oaks, California, and New Delhi, India: Sage Publications.
Sadler, S. (1998), The Situationist City, Cambridge, Massachusetts: The MIT Press.
Schechner, R. (1988, 2003), Performance Theory, London and New York: Routledge.
Schleser, M., Baker, C. and Molga, K. (2009) ‘Aesthetics of Mobile Media Art’, In Intellect: Journal of Media Practice, Vol 10 (2&3), [Online] Available at http://www.intellectbooks.co.uk/journals/view-Article,id=8520/ (Accessed May 2010), Bristol, UK: Intellect Press, pp. 101–122.
Schiphorst, T and Anderson, K. (2004), Between Bodies: using Experience Modeling to Create Gestural Protocols for Physiological Data Transfer, International Conference For Human-Computer Interaction Systems, Vienna, Austria: Conference Proceedings for CHI 2004.
Schiphorst, T. (2007), The Varieties of User Experience: Bridging Embodied Methodologies from Somatics and Performance to Human Computer Interaction, PhD Dissertation, School of Computing, Plymouth, United Kingdom: CAiiA Star at the University of Plymouth.
Schroeder, R. (2006), ‘Being There Together and the Future of Connected Presence’, In Presence Journal, Vol 15 (4), August 2006, Cambridge, Massachusetts: Massachusetts Institute of Technology, pp 438-454.
Schuler, D., and Namioka, A. (eds.) (1993), Participatory Design: Principles And Practices, Hillsdale, New Jersey: L. Erlbaum Associates.
Sconce, J. (2000), Haunted Media: Electronic Presence from Telepathy to Television, Durham and London: Duke University Press.
Shedroff, N. (1994) ‘Information Interaction Design: A Unified Field Theory of Design’, In Design, ed. Bob Jacobson. Cambridge, Massachusetts: MIT Press.
Sheldrake, R. (2003), The Sense of Being Stared At: And Other Aspects of the Extended Mind, New York, NY: Crown Publishers.
Sheldrake R., podcasts. [Online] Available at
190
http://www.sheldrake.org/B&R/audiostream/index.html (Accessed June, 2009).
Shusterman, R. (2008), Body Consciousness: A Philosophy of Mindfulness and Somaesthetics, New York, NY: Cambridge University Press.
Shusterman, R. (2000), Performing Live: Aesthetic Alternatives for the Ends of Art, Ithaca and London: Cornell University Press.
Simpson, J. and Weiner, E. (eds.) (1989), Oxford English Dictionary, 2nd Edition, Oxford: Clarendon Press.
Skoltz_Kolgen, portfolio [Online] Available at http://www.skoltzkolgen.com/ (Accessed Jan 2010).
Smith, G.J. (2009) ‘Variable Frame Rate: Multimedia Performance at MUTEK 2009’, In Rhizome, online artists portal [Online] Available at http://rhizome.org/editorial/2668#more (Accessed Jan 2010) affiliated with New Museum: New York, NY.
Smith, D.W. (2009) ‘Phenomenology’, In The Stanford Encyclopedia of Philosophy (Summer 2009 Ed.), E. N. Zalta (ed.), [Online] Available at: http://plato.stanford.edu/entries/phenomenology/#4 (Accessed May 4th, 2010).
Spanos, W. V. (ed.) (1976), A Casebook on Existentialism 2, New York and Toronto: Harper & Row, Publishers, Inc.
Sullivan, G (2005), Art Practice As Research: Inquiry In The Visual Arts, Thousand Oaks, California and London, UK: Sage Publications.
Takahashi, Y., Uchida, C., Miyaki K., Sakai M., Shimbo, T., and Nakayama, T. (2009), ‘Potential Benefits And Harms Of A Peer Support Social Network Service On The Internet For People With Depressive Tendencies: Qualitative Content Analysis And Social Network Analysis’, In J Med Internet Res. 11(3):e29, National Institutes of Health, [Online] Available at http://www.ncbi.nlm.nih.gov/pubmed/19632979 (Accessed May 31, 2010) Bethesda, Maryland: National Library of Medicine.
Tate Collections, online gallery and library [Online] Available at http://www.tate.org.uk/collections/glossary/definition.jsp?entryId=276 and http://www.tate.org.uk/collections/glossary/definition.jsp?entryId=155 and http://www.tate.org.uk/collections/glossary/definition.jsp?entryId=218 (Accessed January 2010)
Terry, D. (2005), mo.vid.1, film [Online] http://www.deanterry.com/blog/index.php/2005/10/ [Accessed Dec 2008].
The Third Man Factor, The Current Part 3 (podcast) (27 Jan 2009), CBC Radio [Online] Available at http://www.cbc.ca/thecurrent/2009/200901/20090127.html or http://podcast.cbc.ca/mp3/current_20090127_11346.mp3 (Accessed March 10, 2009).
Thompson, N. and Sholette, G. (eds.) (2004), Interventionist: User’s Manual for Creative Disruptions of Everyday Life, North Adams, Massachusetts: Massachusetts Museum of Contemporary Art.
Thorington, H. and Riel, M. (moderators) (2005a), soft_skinned_space -empyre-, listserve, topic: ‘networked performance’ [Online] Available at http://www.subtle.net/empyre/ (Accessed 10 June 2007).
Turkle, S. (1997), Life on Screen: Identity in the Age of the Internet, New York, NY: Simon & Schuster Inc.
UK Video Art History, online archive [Online] Available at http://ukvideoart.tripod.com/ (Accessed January 2010).
191
Ullman, M. and Krippner, S. (1973, 2002), Dream Telepathy: Experiments in Nocturnal Extrasensory Perception. Charlottesville, Virginia: Hampton Roads Publishing Company, Inc.
Varela, F. and Shear, J., (eds.) (1999), ‘First-person Methodologies: What, Why and How’, In The View from Within: First-person Approaches to the Study of Consciousness. Exeter, UK: Imprint Academic, University Press.
Varela, F., Thompson, E., Rosch, E. (1991), The Embodied Mind: Cognitive Science and Human Experience, Cambridge, Massachusetts: MIT Press.
Video Art in Canada, online archive and encyclopaedia [Online] Available at http://www.thecanadianencyclopedia.com/index.cfm?PgNm=TCE&Params=A1ARTA0008366 (Accessed Jan 2010).
VIVO (Video In/Video Out studios), organisation website [Online] Available at http://videoinstudios.com/aboutus.php (Accessed Jan 2010).
VJ Theory, online forum and journal [Online] Available at http://www.vjtheory.org (Assessed April 2008), Falmouth, UK: self-published.
Wenders, W. (1991), Until the End of the World, film, Germany / France /Australia co-production: Argos Films.
whisper(s) project (2002-2006), research project, lead by Thecla Schiphorst, School of Interactive Arts and Technology, Simon Fraser University, Surrey, Canada.
Williams, J. P. and Smith, J.P. (2008), ‘The Players' Realm: Studies On The Culture Of Video Games And Gaming’, In Margin to the Center ++, New Media a Critical Introduction, Martin Lister, Jon Dovey, Seth Giddings, Iain Grant, Kieran Kelly (eds), London and New York: Routledge.
WJ-S project, research project [Online] Available at http://www.wj-s.org/ (Accessed July 2008).
Wolf, F.A. (1995), The Dreaming Universe, New York, NY: Simon & Schuster.
Youngblood, G. (1970), Expanded Cinema, New York: P. Dutton and Co., Inc.
192
Part IV: Appendices
(not for examination, but for elucidation and background information for the examiners)
193
Appendix I/ the MINDtouch project (BBC R&D sponsored)
194
A.1/ - technical appendix and background source material on mobile performance, practice and process
This Appendix is intended to provide a detailed background account of the major case study project
MINDtouch upon which many of the thesis conclusions are based. This section describes the
practical aspects of devising the MINDtouch Project, manifested through the initial video collection
workshops, conducted from July 2007 – Feb 2008, as well as the initial live experimental participatory
events from July 2009 to February 2010. These workshops and events involved performers wearing
biofeedback sensors while interacting with mobile devices. Also discussed are the technical issues of
developing custom software for mobile phones, and how the mobile and wearable technologies were
interfaced with each other technically, performatively and otherwise, as well as the design process of
embedding the sensors in clothing.
The practical project processes included developing techniques to encourage people to connect to
remotely to each other, in order to re-engage with each other and with the world affectively. This was
done through non-verbal/textual, non-linguistic activities in mobile, networked activities. As such,
participants were asked to use their bodies as well as a range of expressive, creative, non-textual
means to communicate visually using mobile videophones. This involved short activities during
workshops and in organised social events, which were mainly improvised by participants through
guided structured improvisation parameters, devised to assist participants perform the activities.
The interaction between the mobile participants and those wearing the sensors was intended as a
means to emulate the telematic sharing of dream imagery or other pre-linguistic communication. A
creative, technological way to simulate and enable participants to imagine a telepathic exchange
between themselves and other people was emphasised, much like lucid dreaming visually and
affectively together (LaBerge and Rheingold: 1990),. This is something I hope we will all be able to do
one day with or without technology.
A.1.1/ mobile video collection workshops This section is dedicated to details of the phase one workshop process (see REF. Figure 1 on page
18). MINDtouch workshops were initially conducted as a preparation for creating the performative
video events and collaborative artefacts. During the video collection process, participants in this first
phase were asked to explore their internal images, emotional and affective senses and states, internal
physical sensations, impressions of their immediate surroundings and body surfaces; inner thoughts,
consciousness, pondering, internal conversations; reflections and impressions, in non-verbal, visual
ways using the video capabilities of their phones. They were then encouraged to share these visual
sensation explorations with others and myself. The archive of personal clips was used later during the
195
collective collage in the VJ performance events (see the script of activities in the Appendix II).
Images 33, 34, and 35 © 2007 C. Baker – still images by participants in video collection workshops
Exploring the more emotive, sensory experience of presence has been the focus of this exploration, so
I have endeavoured to tap into visceral, bodily expressions that might surface during body practices as
a primary source of non-verbal, pre-conscious communication.
Five video clip collection workshops were conducted from June 2007 to February 2008, each
consisting of four to eight participants [214]. They were conducted in Vancouver, Canada (twice),
Dublin, Ireland, (twice) and London (once). Participants of the video collection workshops were asked
to express themselves using the video recording feature of each mobile phone, either using their own
videophones or ones supplied. Briefly [215], the workshop procedure involved several people
participating in a set of specific guided mind-quieting activities [216] before being guided through
several video capturing exercises. These exercises asked them to visualise their internal experiences,
thoughts, sensations, emotions and perceptions in a non-linguistic way, with the available surrounding
and visual fodder captured on their video mobiles (or mine).
In one activity in the workshops, participants were asked to introduce themselves to each other non-
verbally, using the videophones. The participants each decided to approach the activity differently.
Some decided to record parts of themselves, then show the recordings to each other as a way of
introduction. In the last activity, participants were asked to work with a partner non-verbally, in order to
create something that they might send to someone else, telepathically or through the mobile phone.
This was intended to be something that would express their inner perceptions and experiences. As a
result, several participants often went outside, each pair developing a different method to carry out the
activity. Each outcome was fascinating. Another activity was introduced in later workshops, involving
partners sitting back to back, trying to telepathically send an image or sensation they had to one
another. After this they were asked to visually capture what they had ‘sent’. Each would then guess
214 Performances stared in July 2009, so the gap in the timeframe between conducting the workshops and conducting the performance events was due to the time spent in phase two, developing biosensing electronic devices and garments, as well as all the VJing / mixing software and network development.
215 There is a longer description on the workshops and analysis later in this Appendix. This is more for an overview understanding of how they were designed and conducted.
216 By this I mean a guided relaxation and visualisation exercise.
196
what the other person had sent, and then they would share their videos with each other to see if all
three versions (the telepathic, the videophone and the guess) were similar. The activities were followed
by showing the participants how to upload the videos online [217] to share with others in the group and
myself, followed by an exit questionnaire and finally by an on-videophone interviews. More on the
outcomes of these workshops is detailed below.
A.1.1a/ biosensors and electronics
The biosensing system research commenced in November 2006, with acquisition and development of
the biofeedback hardware, software and mobile phones. The biofeedback sensors initially used were
very expensive but robust. They contained a professional medical grade system that monitored: EMG
(electromyography: measuring muscle electricity), GSR (galvanic skin response), respiration and (initially)
BVP (blood volume or pressure). This last was later changed to a Temperature sensor. These specific
sensors were evaluated as the most responsive and easiest to work with, as those which could
generate usable data, and ones which were least uncomfortable for the person wearing them [218].
The EMG measures muscle electricity to determine the muscle tension of larger muscles, like arms and
legs. GSR response measures the electrical conductance or resistance of the skin that can change
when people change their stress levels, usually worn on the fingers [219]. Respiration sensors are worn
around the ribcage and monitor the inhalation and exhalation depth and frequency of the abdominal or
chest expansion and contraction. BVP monitors the relative blood flow in the fingertips, using near
infrared light. The heart rate can then be determined from the pulse or blood pressure. Although the
first set included BVP, we changed to using temperature sensors, which read the body heat of an
individual. These sensors were attached to the body via sticky electrode pads or Velcro attachments.
Using wireless sensors was most desirable in order allow for ease of movement, but at the time they
could only be acquired as wireless (in terms of not being connected by wires directly to the computer)
using a Bluetooth hub (or Zigbee) carried or worn on the body, which sends the signals wirelessly to
the computer. In this case, the data is sent to mobile phone or the computer to receive, process, and
visualise the data. However they were ultimately rejected as unsuitable since as the images below
demonstrate, there is still a great deal of cabling from the sensors themselves to the hub.
217 Initially we used a free public online video editing website eyespot.com, but it went defunct last year with no venue model, and other easily assessable online upload and video sharing tools have replaced it. More on the technology and technical development will be discussed in future papers.
218 Based on research and prior experience working with the whisper project.
219 A video game was made in 2002 called Journey to the Wild Divine (still available at http://www.wilddivine.com/wilddivine-demonstration-page.html) was made using the GSR; the lower one gets their stress levels, the higher the players score. The aim is to help kids or other player learn how to control their stress levels and their body’s responses.
197
Image 36 © 2007 Mind Media – marketing images of the Nexus 4 Bluetooth Sensor transmitter
Images 37, 38, 39, 40 and 41 2007 Mind Media: electrode cable for EMG, sensor ground cable, Galvanic Skin Response,
Respiration Sensor, Blood Volume Sensor (image source from Mind Media technologies) [220]
While these expensive medical grade sensors themselves were high quality, the short, thick, heavier
and more restrictive cabling on the body encumbered movement and became unsightly for
performance unless hidden within clothing. This first system came with simplistic visualisation software
used for clinical stress reduction, not appropriate for MINDtouch in terms of how the body data [221].
In turn, this system was abandoned in favour of creating cheaper, more customisable, DIY sensors,
and data visualisation software, which changed the approach to working with the sensors in the
process.
Images 42, 43, and 44 © 2008 C. Baker: Michael Markert developing the custom biofeedback sensors & electronics.
220 Mind Media sensing systems [Online} Available at http://www.mindmedia.nl/english/sensors.php
221 We had to return this system for this reason and due to the time lost in miscommunication with the company’s programmers and representatives when trying to get access to the SDK, which they had previously agreed to provide so we could develop our own custom data visualisation software for the mobile phones.
198
German electronic musician/creative electronics artist/ Apple, Java and mobile programmer Michael
Markert joined me as a collaborator in Spring 2008, assembling and creating the DIY biofeedback
sensors with a Bluetooth Arduino system. I then found other help to embed the sensors within
customised clothing from two fashion designer colleagues. Effort was made to create garments that
allowed for more flexible, unencumbered movement, and that were more aesthetic, ubiquitous and
unseen on the body, creating a hidden ‘body area network’ [222] or sensing system for performance
contexts [223].
Image 45 © 2008 C. Baker: custom biofeedback visualisation software.
The intention was to develop subsequent mobile software for the live mixing and streaming of the
videos and delivery to the server database for real-time collaborative video collage. This software was
to allow people to share and interact with the visuals through their phones, in real-time. The concept
was that the custom mobile VJ software would receive the data from the body, made in Mobile
Processing [224] or another similar visual programming language or environment. This was envisioned
to work somewhat similarly to eyespot.com (a very simple and elegant web-based, mobile video editing
and sharing software, now sadly defunct) [225]. It seemed a realisable task, based on witnessing
whisper [s ]’ integration of biofeedback sensors with PDAs in 2005. With the software that Michael
Markert developed to operate on the phones working well with the sensors, I then proceeded to work
with another programmer to make the mobile VJ software, to work in tandem with the biosensing
receiving software. However, to make VJ software of the sophistication I wanted, was not and is still is
not possible to run on the operating systems of current mobile phone models on the market.
Theoretically, it will be possible for the iPhone or others in the very near future.
222 This term, while used by many others was picked up from the whisper project, is used in medical and physiological sensing contexts.
223 Although this proved more difficult than expected and ultimately to have no wires, using conductive threads and materials would be the best approach, for another project as it was outside the time constraints and resources of the MINDtouch project.
224 Processing is a visual programming language designed by John Maeda at MIT as an easy language for artists to learn to programme to make Internet visual art and visualisations and has a mobile version available.
225 It is my impression that eyespot.com went offline and its technology bought out because they did not have a sustainable business model and could not maintain the costs that its huge popularity brought it, Sadly, it seemed that they did not manage to get a sponsor. Unfortunately, I have no screenshots of the web software interface, which was amazing. They went under suddenly and I did not even have time to rescue all the videos I and participants loaded there, but I was able to get the videos from some of the participants after the fact.
199
Image 46 © 2007 Brady Marks with permission (for a spin-off project):
vision of the mobile interface for social VJing
Near the end of the development period, Manjit Bedi joined the technical team, specifically to create the
VJ software system using Apple’s Quartz Composer programming environment. The software Manjit
created uses the body data sent from the sensors within the lightweight garments to mix and stream
the video into the database of clips, then adds live visual effects to the video clips. The software
accesses and mixes these video clips or lives streams from four folders on the server into one mixed
video. Lastly, QuickTime Streaming Server streams the video out the mixed video back out (ideally), in
real-time to the web and to RealPlayer or web-enabled phones. This allows the remote audience
members to share and interact with the resulting generative visual collage of other people’s video.
Image 47 © 2009 C. Baker: Quartz Composer early software patch view
200
A.1.1b/ garment design
Fashion designers Tara Mooney and Rachel Lasebikan started working on the MINDtouch project as
part of the SMARTlab wearable research group in July 2008, continuing through to the autumn of
2009. Their aim was to design lightweight, flexible and fun garments in which to embed the sensors.
[226] What resulted in the first iteration of the MINDtouch garments, due to the limited time and
resources available for this aspect of the research, was a set of attractive pieces of clothing that
successfully hid the electronics but that were not, unfortunately, easy to self-fasten. As a result the
power, batteries and Arduino connections were not easy to access by participants wishing to start the
system. Due to extra wear and tear imposed by the dressing and undressing procedure, the
electronics were gradually compromised and were not suitable for general use after a limited number of
experimental events. [227]
The second garment system was designed in such a way that it tightened around the chest of
participants so that the breath sensor could measure the rise and fall of the ribcage with inhalation and
exhalation. This was a key requirement, however it was constructed in such a way so as to pull too hard
on the fragile sensor, so it was damaged. Due to scheduling conflicts, closer collaboration with the
designers was not possible, so I was unable to attend to the fragility of the electronics. Due to the nature
of the sensor components, the electronics (Arduino and the DIY sensor kits combined) were inexpensive
and more fragile, especially when exposed to wear and tear, and were too easily damaged. In addition,
while less expensive and more ‘programmable’, they still did not provide for easy movement, flexibility,
durability and performativity necessary to truly work for the project. All of these factors would have to be
taken into account in the construction of the garments for future designs, and I have since discovered that
they are common stumbling block for artists and designers working with similar DIY sensors and
electronics.
226 Both of the versions of the MINDtouch garments were designed with the inclusion of electronics in mind, but additional time for the user research would have enabled a more effective placement of the sensors for longer and more durable use.
227 The first set of garments with embedded systems has to date only been worn three times, and was not used for the last event as it was being repaired.
201
Image 48 © 2009 C. Baker: Alpha / Garment 1 with embedded sensors
It would, of course, have been an option to use other prefabricated garments with embedded systems (as
such do exist in the open market, primarily for sports and health analysis), but most such existing
garments are expensive and tied to proprietary software tools and exclusive research IP arrangements.
Thus, none of the existing garments would have been suitable for this project without significant refitting
and legal negotation, and after the difficulty with the first biofeedback sensor provider, this was not of
interest or financially viable. In hindsight, working with an already researched, designed, and tested
garment might have saved time but would have added significantly to the project’s cost and would also
have had an impact on the originality of the research outputs. This last factor in particular was to me, a
considerable issue for purposes of use of the results in the original contribution of this thesis.
Images 49, 50, and 51 © 2009 C. Baker: Beta / Garment 2 with embedded sensors
202
A.1.1c/ performance events This section details the performance events’ design and staging. As explained, phase three
performance events sought to create a social, abstract, non-linear narrative/montage or ‘remix’
collaboration, to be streamed back to audience participants’ phones and to the Internet. These events
attempted real-time video mixing and streaming to phones, although in practice there is significant lag
and delay given the inherent network and security issues discussed earlier. With this in mind, there
were multiple manifestations of presence intended to be experienced and witnessed by participants in
the workshops and performances, such as:
• the engrossed attention during a mobile phone exchange and the in-between space of the
network connection;
• the physical presence of participants and the in-between space as their data was sent to
the phone or computer, followed by the in-between space of the computer and phone
communicating with the database of video clips;
• the captured emotional, aesthetic presences embodied within the video expressions of the
clips;
• the physical co-presences of the participants in the space of the staged network events
• the virtual multi- or co-presences of the remote participants and the global co-presences
of these collaborators.
Figure 5 © 2009 C. Baker - early technical diagram of the networked performance project
203
The diagram above shows the three groups and technical interactions envisioned. In practice, Group 2
became the remote participants.
During the initial event, three different groups were invited to enter the ‘party-like’ environment in the
space. Each group was greeted by a guide who walked them through various activities:
1) Group 1 participants wore the biofeedback sensor garments and were guided through various
movement activities, to activate the sensors in various ways, triggering the remote system’s visual effects
to be applied to the live and archived video in the database. Simultaneously, the effected video was
displayed on screens around the room. There were only two garments and two sets of sensors, [228] so
this group was small, with only one or two participants each event [229]. The participatory activities for
this group involved:
a) movement activities, and
b) theatre games (see guiding activities scripts in the appendix);
Figure 6 © 2009 C. Baker – showing the staging and organisation of the performing groups in the live context.
228 As in the diagram, I had wanted more than or envisioned at least 4-6 in this group but with limited funding and resources was only able to acquire a limited number of sensor sets and make a limited number of garments. Also due to the Bluetooth pairing requirements each limited a garment / sensor set could only be paired with one computer or one phone at a time, an I also had a limited number of phones, which often were used for the video streaming aspect, since many participants did not have compatibles phones for streaming (and I did not want to prevent participation and tried to accommodate everyone.
229 In future iterations, I still hope to have more.
204
Images 52 and 53 © 2009 C. Baker – stills from July 2009 first performance
2) Group 2: This group was guided in video streaming activities using on their mobile phones – like earlier
workshops, but with fewer, shorter activities and simplified ways of ‘speaking’ by visualising internal
sensations. These were streamed using qik software on the phones to the qik.com website, to then be
transferred [230] with a custom php script to the server, and then to mix and the stream-ready database
folders;
Images 54 and 55 © 2009 C. Baker
3) Group 3: This initially was to be the group who mixed the video from the database via custom software
on their phone (downloaded as part of the activity). They would then interact with the biosensor
participants (choosing live or archived video, created by Group 2). Their choices would be mixed and with
visual effects added by the body data from Group 1. This activity would be guided so the participants
could easily use the videophone software and interact quickly and decisively, making a body narrative in
their own way.
In practice this event had to be changed. I discovered that the phone models being used and their
230 This did not entirely work due to a part conflict on the server but has been rectified for future events.
205
operating systems were not yet capable of implementing/operating this type of custom software (see
vision interface above on page 179). Instead, I adapted the event so that Group 3 became the remote
group who waited and watched for the streams to arrive on the project’s qik page or through the provided
video streaming link for QuickTime, RealPlayer (or Internet browser with the player plugin). The streams
could then be received on the Internet on a computer or people’s mobile phones (ideally), using Real Time
Streaming Protocol (RTSP) from the streaming server. This group followed ‘guiding’ instructions on the
Facebook event invite or on the research web page, which described how to make their own videos to
stream via q ik. Alternatively, they could just watch. I discovered that most of the remote participants had
more voyeuristic behaviour, as they chose mostly to watch, with only a couple remote participants
contributing their own video streams in the last event. From this observation and the lack of remote video
contributions on qik , it became evident that this group required more attention and guiding through an
online moderator, which I did not have or consider until after the fact.
Images 56, 57 and 58 © 2009 C. Baker
This was meant to be ideally a two-hour event, but I discovered that it can be made shorter depending on
the social event it is part of and the attention span, enthusiasm and creativity of the participants. The key
focus was the performance, not on the technology, so the technology was meant only as an experience
facilitator. The events were designed to be mainly about exchange, non-linguistic visual play, collaboration,
participation and interaction, exploring movement and spontaneity. The exact details of the technical set
up and preparation for each event is located in the Appendices. A.1.1d/ participant expectations
From the London, February 14th (2010) performance, I discovered that there was an expectation by
some participants that the videos streamed as part of the project would be 'high quality'. These
assumptions were made by a few of the remote viewers who had no training in video production
processes [231]. It appeared that they were expecting that the video mix would be HDTV quality due to
the cutting edge technology used in the project [232]. The one of the viewers, who commented on the
231 This is not an assumption, I discovered this directly from the participant whom I refer to.
232 Perhaps too much exposure to Sci-Fi and so much highly produced media exposure, such as TV shows like CSI.
206
videos uploaded from previous in-person event participants, said that what they saw was “amateurish”.
Compared to what they are used to on TV or in the movies [233], which uses the highest quality,
produced video––it may seem amateurism. What many without video training do not seem to realise is
that TV and film are highly produced and polished, with a high number of video editors and post-
production experts, on a high salary on big budget productions which takes months and sometimes
years to create.
As a new tool, mobile video is a new way of working with and understanding video production. As
stated in Chapter 5, the ‘blurry’ and ‘low quality’ mobile aesthetics include compression, network
delays, low bandwidth at times, poor image quality, as they are created by the device that way to start
with––I argue this has its own appeal and authenticity. It can show participants (and others) that
something meaningful can come from this technology. In this respect it is still early days in terms of
quality and possibility. The education of audiences on the value of lower quality resolution as an
aesthetic [234] is also important, I have discovered. Thus, audiences need to be shown that TV is a
specific product highly produced to look as super slick as it does. Conversely, the low-fi phone video
goes through many compression processes to make it small enough to move easily through the
Internet and mobile networks. It is not the same product. Something can be gained, however, from
mobile video through its immediacy and everyday personal imagery, versus other pre-produced, highly
edited, and somewhat elitist artefacts. For the non-technological types, I would hope that this project
has the potential to change their expectations from the removed polished, non-interactive media,
towards getting involved in creating something themselves. Another important point is that the video
may in fact be “amateurish”, since the goal was to have non-artists/non-professional videographers to
make the video expressions. The inclusive nature of the facilitation and the event attempted to
encourage other participants to refrain from judgment, which would result in further inhibiting
participation. This last substantiation is thereforehow I responded to the one person who commented
on the “amateurish” video.
A.1.2/ outcomes, analysis + interpretation: performances events This outcomes section is dedicated to details of phase 1 of the MINDtouch workshops and
performance events, observations and data analysis. It delves deeper into the discoveries of the
practical research. It especially focuses ethnographically on participant feedback, as well as upon my
phenomenological experience as creator of such a project.
233 Even compressed, originally high-definition video on YouTube looks better because it starts off much higher resolution before it is compressed for the web than mobile video.
234 While the manufacturers work hard to make it better see specifications on the newest video phone and all its enhanced video recording features available online at http://europe.nokia.com/find-products/devices/nokia-n8/specifications# Video and HD compatible hi-def resolution http://europe.nokia.com/find-products/devices/nokia-n8/features
207
A.1.2a/ workshop outcomes and observations
As already discussed, participants of the phase 1 video collection workshops, conducted between
June 2007 and January 2008, were asked to express themselves using the multimedia tools of mobile
phones – either using their own videophones or ones supplied.
Image 59 and 60 © 2007 C. Baker – still images of participants in video collection workshops
Five video clip collection workshops were conducted in Vancouver (twice), Dublin (twice) and London
(once). Each workshop involved meeting the participants, having them fill out consent for the research,
a personal profile, and release forms for use of their video clips, an explanation of the research,
conducting mind-quieting and body-tuning visualisations, and then four activities to capture video: two
individual and two in pairs. The workshop procedure involved having participants experience a set of
specific guided mind-quieting exercises to help them visualise their internal experiences, thoughts,
sensations, emotions and perceptions, in a non-linguistic way, before being guided through several
video capturing exercises to externally express these internal experiences by using the available
surroundings and visual material. This approach or technique was used in a condensed way in the
phase three performance event stage, but shortened and to a less immersive extent.
The first workshop in Vancouver (16/06/07) was set in a big centre with a huge gallery space, many
workshop rooms, a big black box theatre, and a gymnasium downtown next to the water. The initial
mind-quieting activities took place in one of the workshop rooms and then after two individual video-
collecting activities of five to ten minutes each, of which two were conducted in pairs. In one activity,
participants were asked to ‘introduce themselves’ non-verbally, using the videophones, to each other.
208
Images 61 and 62 © 2007 C. Baker - from Vancouver workshop June 2007
The participants each decided to approach the activity differently, some recording parts of themselves
and then showing the recordings to each other, as a way to introduce themselves. In the last activity,
participants were asked to work with their partner non-verbally in order to create something to send to
someone else, through the mobile phone, something that would express their inner perceptions and
experiences. As a result, several participants went outside. Each pair developed a different method to
carry out these activities. The activities were followed by showing the participants how to upload the
videos online [235] to share with others in the group and myself, followed by an exit questionnaire and
finally by an on-videophone interview.
Image 63 and 64 © 2007 C. Baker - stills from Vancouver workshop June 2007
The second workshop took place in Dublin at the Beckett Theatre in Trinity College (22/07/07). The
space for this second workshop was in a dance studio. The participants had more room to move
around, and since a couple were dancers they tended to move more, which inspired the others to do
so as well. The same four activities were facilitated (as in Vancouver), but in this instance the work was
supported by a video documenter, which freed me as the researcher/workshop leader observe the
participants and to assist when they were having trouble with the phones or needed more instruction.
This group was, in general, more exploratory with their video expressions. There was also one
235 Initially we used a free public online video editing website, but it went defunct last year with no venue model, and other easily assessable online upload and video sharing tools have replaced it.
209
participant with Cerebral Palsy who had two assistants to facilitate his engagement in the workshop
[236]. Participants were asked to interview each other with the videophones, rather than having one
interviewer interviewing each of them individually, this was done partially to save time and to enable
them to leave earlier after a long session. Even though participants did not know each other
beforehand, and had usually come from different backgrounds, this proved to be successful. It might
have also been the case since they had started to develop a rapport with each other in the workshop,
sharing similar experiences, and this allowed them t become more open to discuss the experiences
with each other.
Images 65 and 66 © 2007 C. Baker workshop still Dublin July 2007
The third workshop took place in Vancouver (21/08/07). Participants met in a waterfront park nearby
on a beautiful hot, sunny day, which appeared to add to their enjoyment of the workshop. It was a
much smaller group of four, thus, the first two individual activities were somewhat more difficult to
conduct in such a public space, causing participants to seem somewhat self-conscious performing the
‘internal’ and mind-quieting activities, with others in the park were throwing Frisbees and dogs were
running about around them. One woman brought her child with her, so her focus was reduced, making
it harder for her to relax completely and engage fully in the activities. Due to the bright sunshine and
inspiring surroundings, the video imagery from this workshop was quite aesthetically pleasing.
236 Although assessing his experience was more challenging due to the additional factors involved in his experience of the workshop as conducted with an able-bodied partner, and although we had a follow-up discussion on Skype, it remains uncertain whether the workshop was totally effective and productive for this participant. To pursue this subject further, however, would have taken the study beyond its core focus. In future, a follow-on study might usefully focus specifically on use of the MINDtouch system by people with severe disabilities and/or communication needs.
210
Images 68 and 69 © 2007 C. Baker - stills from participants’ videos from Vancouver workshop August 2007
The fourth workshop was held once again at the Beckett Theatre in Trinity College Dublin in October
(26/10/07). Like the previous workshops, these involved a mix of new participants from various
backgrounds who attended out of interest. Some participants were quite imaginative and had unique
approaches to the activities many looking for patterns from the structural environment, away from the
group at times. Participants had access to several spaces, so the individual exercises were done in one
studio and the pair exercises in a large black box performance space and outside, as it was an
unseasonably warm autumn day.
Images 70, 71 and 72 © 2007 C. Baker - stills from participants’ videos from Dublin workshop October 2007
The images from this group stood out as very aesthetically stunning and exceptionally creative in the
way in which the participants used the camera’s special effects to invert the image, play with extreme
close-ups and colours, as well as to further abstract the body parts and other object being recorded.
The workshop process was quicker and many of the activities were improvised. As a result, a new
activity was introduced involving partners sitting back to back, trying to ‘telepathically’ send an image
or sensation they had to one another, they then attempted to visually capture what they ‘sent’. Each
would then guess what the other person had sent, and then they would share their videos with each
other to see if all three versions (the telepathic, the videophone and the guess) were the same. This
activity appeared to be enjoyed by most as it garnered more captivating visual explorations and
creativity from participants, and so it was used again in the follow-up workshops.
The surroundings became a large actor in the content that participants created expressively in their
explorations within even though psycho-geography or locative media approaches were deliberately not
extensively explored for this project. This was possibly both due to how comfortable participants were,
211
and what visual material they could work with in the vicinity to externalise and represent their inner
explorations. This became a more significant factor than intended and has implications on how one can
externalise with the videophone and be influenced by the surroundings. This is because it is difficult to
replicate our inner eye’s view, especially if the immediate environment has contrasting or unique visual
fodder unlike our own internal experience and imagination. Using the available surroundings may have
felt limiting and uncontrollable for participants, as they sometimes found it difficult to make the
translation, yet it proved also to be a stimulus for intriguing visual explorations and experiments, as
were the case with many of the video expressions.
The process of conducting the workshops was very fruitful, and clarified the aims of the design/guiding
process through observation, exploration and iteration. After studying the responses from participants
and their interpretation of the activities and video expression/collection process, I was also able to
better iterate subsequent workshops. My ability to guide participants effectively developed greatly,
meaning that I could enable participants to have meaningful experiences while creating intriguing
personal media expressions. The workshops became an end in themselves, and several groups asked
for further practical engagement with these personal media empowerment tools (often for less
technologically literate groups and individuals), in the form of future workshops. It appears that because
mobile media is still a relatively new format, this hands-on engagement, with a human touch and an
intimate form of communication, affected participants emotionally, intellectually and creatively. Thus,
this stage of the project garnered unexpected results and served more than just to create clips for the
visual database.
A.1.2b/ performance discoveries, results + analysis
The researcher discussing the work in relation to: lived experience, other works; application of results obtained; contribution to discourse; new / transgressive possibilities; obstacles encountered and the remaining problems to be addressed in future research. (Barrett, 2007:139 - From CHART: "Foucault's Author Function: Application in Practice as Research)
This section discusses the data collected from participants involved in a limited number of scratch
performance events. The analysis here is partly ethnographic and used both qualitative and qualitative,
with observational analysis crossing ethnography and heterophenomenology. It is based on Kozel’s
definition mentioned in Chapter 2.
There were 5 scratch events in total:
1. July 19th 2009 – in the Matrix East Studios at UEL, with peers during SMARTlab seminar (20+ participants but only a 2/3rds filled in a survey - several left early - one documenter);
2. October 5th, 2009 – Leeds Trinity College, in Leeds, with undergraduate students at with a
student group as an Open Day activity (4 participants, and one semi-documenter);
212
3. December 4th, 2009 – 4M with peers and guests during SMARTlab Christmas party (4
participants and one documenter); 4. December 18th, 2009 – Christmas Student Exhibition with a group of students at the
Limkokwing University, London, during their student mobile project exhibition and Christmas party. (3 participants, no documenters);
5. February 13th, 2010 – Valentine’s Apartment Dinner Party at a friend's dinner party with new
acquaintances and unknown participants (to me) (6 participants, no documenters).
The performance events were mostly conducted with student groups, peers or acquaintances, since
these groups were most easily accessible. Each time, events or parties were compatible to combine
with or piggyback the MINDtouch activities upon. These groups were most open to participating or
allowing me to try out performative activities with them. However they may not have been necessarily
representative of the general public, which I would have ultimately preferred to work with. If there were
more time to conduct more of these events, there would have been a wider sample.
A.1.2c/ demographics There was a good split of gender overall, however there seemed to be more men in the performance
events compared to the workshops, where there were more women. Workshops drew participants
through email, while the performance events were part of other previously organised events. Therefore
it could have been that women were more likely to voluntarily get involved or be drawn to these
personal, creative activities more than men, but this is pure speculation. It would take further study to
be sure of the motivations for involvement for each gender, but this was not considered relevant, since
all participation was voluntary overall anyway. Similarly, there was a good range of ages, although it
would be preferable in future to have more youth involved. Thus, in future versions of the project, effort
will be made to seek out a wider age range of participants and more varied groups with which to work.
The education level of participants was high and represents the fact that many of the participants were
either PhD student peers or friends and acquaintances with similar backgrounds or education to my
own. Again, in future it would be preferable to explore communities with wider educational range. Most
performance participants were white British, some were mixed ethnicities raised in Britain (which I have
counted as British), with a few other foreign citizens in the mix.
The occupations were no surprise, since the participants from the first performance were all PhD
student peers. Therefore, it seemed not unusual that so many were also lecturers and academics, with
some participants also being the students of these peers. One concern was that this project was
intended for non-trained performers (though not excluding trained performers either) and the general
213
public. Thus, with so many artists and performers in the group (esp. the first performance), results
might be skewed and perhaps those participants had expectations much higher for such work. An
alternative reading is that they might be more open–minded or empathetic to participate in such work
than other public groups, since it is close to their own practices. That being said, many untrained, non-
artists participated in the last event, including a broader mix of backgrounds, occupations and training,
as well as ethnicities, and it was thus more representative of the target audience/participant group.
There was a range in academic backgrounds of the participants in the events overall, especially from
the events after July 2009. However, due to the methods used to draw people to the events (unlike the
workshops which were more random and self-selected), there was still a bias toward disciplines of
study related to my own, such as design, fashion, digital media, film and TV, drama and dance,.
Curiously, many had studied different disciplines than those related to their current job, showing that
people's paths can sometimes take them far away from where they started.
Since all but one of the events were conducted in London, it is not surprising that most participants
came from or lived in London, unlike the workshops that were conducted in three different cities, which
perhaps provided more differences. There also was wide range of incomes represented, since many
participants were students and/or underemployed artists. However, half of those who completed the
questionnaire chose not to disclose their income, so it is hard to entirely assess this range. Most of
those who chose not to answer were either from my peer group or of the twenty-something,
undergraduate students, who may not actually have had an income, but instead were possibly
supported through loans, bursaries/grants, scholarships/studentships and/or family or other means.
These events were definitely more of a prototype nature for a more broad mix in future.
A.1.2d / event survey feedback
Overall the survey feedback was very positive. Most participants stated that they enjoy being involved in
the participatory activities and several stated they most enjoyed the pre-activity, mind-quieting at the
beginning. Curiously, some of the things some really liked or thought worked best were the very
activities or aspects that others disliked or though needed improvement. In some cases, this was due
to the fact that they were in the different events and locations. For example, when I had people
assisting participants with the activities as guides, participants found that this guiding aspect helped
them and was done well. While at other events when I was without the guides, participants felt the
event was lacking because the activities needed more explanation. But this difference and this specific
example was not a one-to-one correlation. Sometimes these contradictions in participant experience
took place during the same event and can only be attributed to the differences in the participants
themselves. The overall themes or patterns from the answers people provided can be seen in the
appendices.
214
Most participants reported they had enjoyed being creative, but a few did not, especially during the
main video activity (which I thought curious). It was fascinating that many enjoyed watching others but
they did not like being observed. This was more significant in the first event where there was an
observer group due to the large size of the group and fewer participants moving and movement
activities in subsequent events. Most stated they enjoyed participating but not being observed, so this
activity was cut in the other events. Most also felt that what worked best was the staging, organisation
and facilitation of the event, particularly in the two events where I was able to find assistance in guiding
the participants. Overall people stated that they thought the activities, especially the mind-quieting and
video activities, worked well for them. Some participants felt the activities needed more explanation.
This was a fair point as these comments came from were those events where I was managing it all on
my own, including the technology set-up and other staging aspects, so my attention was split in
running the event without guides to assist.
Many participants from all events, stated that they wanted to see the project ‘results’ more directly or
immediately. I interpreted this to mean that they wanted to see the online group and the software
receiving their body data trigger the videos, rather than just the outcome collage, which all could see.
But most also stated that they felt this project was more interactive and participatory than most
performances they had attended, with many not even regarding it as a performance at all.
Unfortunately, due to the limited number of phones available, and most participants not having
compatible handsets to use the biofeedback data visualisation software, the body data could not
always be shown separately from the mix on the phones, since the video participants were usually
using all the research phones. On one occasion the mobile software suddenly stopped working and
had to be shown on computer instead. Some participants had trouble comprehending that their body
data was influencing the visuals, which were the ‘output’ or ‘results’; even when shown the software.
Many only wanted to see their own data as they moved, having trouble grasping the relationship
between the visualisation of the data itself, their bodies and the media collage displayed in the space
mixed by it. However, it was not the intent or interest to show just the biofeedback data alone, since
medical software and hardware is easily available to do that––the project is not about just visualising
physiological data. The project is more abstract, but this aspect likely needed more explanation and
emphasis, so that the live ‘results’ in front of participants, made more sense to them. It seemed that
participants also wanted more immediate feedback on their videos, to see them in the collage more
quickly in real-time, rather than have their videos in the ‘queue’ or in the archive; this is something that
was intended, but due to network set-backs and delays, was not possible at this time.
Most participants misunderstood the following question: ‘Did you feel the same sense of presence or
interaction between performer – audience in this event as you might in a conventional performance
event? And How?’ Since the networked aspect was not fully connected or explicit to participants, it
seems that people did not see the performance activities as performance or fully register that there
were remote participants or audience. They answered in such as way that they experienced
themselves and others in the space, involved in the activities in a personal, intimate and individual
215
sense. However, more stated that they were comfortable with this type of performance activity,
although a few were unsure.
The optional experience inventory received answers of a more individual nature, unique to participants
about their experiences, although many either left this section blank or did not elaborate in great detail,
in contrast to those in the workshops (see appendices for details). When asked: How did you perceive
your body’s sensations, perceptions, thoughts and other internal experiences during the activities
tonight?’ some answers were: 1) about the body––pulse, spiky crampy, lungs, head, heated, tired, ill
health; or 2) about mood or mental––calm, focused, racing thoughts, abstract, positive, heightened
awareness. Most were fine with or enjoyed expressing themselves and sharing their sensations through
the activities and similarly enjoyed participating in the activities using the mobile or wearable devices.
A.1.2e/ event analysis: production report 1) July 19th, 2009 with peers during the SMARTlab seminar:
This first performance was held in a 'black box' performance space on the university campus, in a
state-of-the-art performance video studio [237]. However, I wanted the first performance experiment to
be in a controlled space. I had a somewhat captive group with my PhD peers, who were attending our
thrice-yearly seminar, with several of us showing our work. It seemed an ideal group to receive
feedback from and a safe group to start the process with.
I worked with the production technical staff and performance studio technician, receiving helpful
support, accordingly I set up the equipment and tested it in advance. There were more than ample
numbers of screens: one for the body data, and two mirroring screens for the output mixed collage of
the mobile media. After consulting with the campus IT department and testing it all on campus only, it
seemed that all of the networked side was working properly and I was given my own IP address for the
server. However, on the day testing, it turned out that we could not get the network to connect, and
just two hours before the event the IT technicians told me that, actually, they would not allow streaming
off-campus. Even more frustratingly, they had also not managed to properly set up the performance
space for the Internet as originally intended. This was very disheartening, since we had had several
meetings and email exchanges on the project network requirements, and it was acknowledged that the
performance event involved streaming video. However, it was later claimed that they did not know that
it was necessary to stream videos on and OFF campus the, which would create a security hole that
they could not allow.
However, the show had to go on, even though there was no Internet connection, and video streaming
237 Contrary to the overall intention of these events, which was to be outside of conventional performance spaces.
216
to network was not possible that day. After all the preparation and testing, we could only stream from
qik on the phones in the space and work with mixed video from the archive. Ultimately this is what
took place.
Another issue arose because even though the group had been asked to bring their mobile phones and
to download the mobile software onto their phones in advance, most either did not have a compatible
phone, or had not had time to prepare in advance [238]. Therefore, the only live streaming was done
from the one phone available for the project (the other two were working with the biosensors), and one
other that one participant with a compatible phone used. Due to the poor mobile reception on campus,
the video streams by participants did not upload properly to the qik website and were only stored on
the phone memory itself. Instead, later video recordings from the phones had to be collected and used
after the event to contribute to the archive for future events.
Additionally, only one phone was able to show the body data, and another phone was given to
participants just to watch their body signals while they moved, the second set of sensors sent data to
another screen. Understandably, all the participant feedback on the experience of presence was more
about the experience of others in the space and the blurred lines between performing and being an
audience, rather than what I was looking for (there was some misinterpretation of the question; see
more on this further on). This again was due to the network circumstances, rather than the sensing the
remote others not present in the room through network interaction, as was intended.
Another distinctive facet of this first event was that second garment design, prototype 2, was not ready
for the event, no matter how hard the designers worked to do so. Since there was only one garment
and phone pairing with physiological data, the other paired set involved having the exposed sensors
hanging off participants and shown on the larger screens [239].
Another unique feature of this event was that there was having the three guides or assistants to
manage a large group of participants: 1) one guide working with two sensor garment-wearing
participants, helping them to move actively enough to trigger body data changes; 2) one guide working
with the video participants, and 3) a third guide with a placebo group (another unique feature used as a
base or control group). This last guide also helped prepare the participants to pay attention to their
bodies for the video activities and get used to moving for the sensors. This third guide also gave
participants additional things to do while the other two groups were busy. Since there were many more
participants than anticipated, I created a fourth group to observe the other groups in action and to note
down (in a stream-of-consciousness way) their experiences of the overall event. I improvised with these
last two groups, especially the last one, with the wish to include all who showed up, to keep it social
and participatory. This also illuminated what to do in future if a similar situation arose.
238 Which understandably IS a pain for them to do and it would be preferable if all phones had some similar multimedia software and operating system on them so this was not an issue, such as QuickTime (which has editing capabilities as well), but it is also not a priority for some people and an extra cost.
239 As mentioned in the last chapter, each set of sensors needs to be paired with only one phone or computer, due to the limits of the BlueTooth one-to-one pairing requirements.
217
As a result of the network limitations, the collaborative video mix was partially a ‘fake’ of the interaction
based on the archive of clips from the earlier workshops then mixed and projected in real-time as the
visual source. The body data from the sensors was live. Therefore, all the working technology was
used, but there was no external, networked interaction. What also emerged from this was the discovery
that there were some glitches with the mixing software, which had not been evident previously but
would need to be rectified before the next event. In addition, a qik-to-server 'patch' needed to be
made to bridge the gap. This ‘faking it’ is common in interactive and new media projects, so I was
prepared for its possibility. There is still always a great deal to be learned from such an event and that
was learned in this iterative type of work.
Event Summary:
L iveness (real-time) = people in the space, live body data, live mixing of archived video and one live streaming qik video camera (not received by qik) Presence (of others live and remote) = people in the space only ____________________________
2) October 5th, 2009 with undergraduate students at Trinity College, Leeds; a peer's student group as an Open Day activity.
From August to October a few events that were lined up fell through, in addition to other constraints
taking up time, much to my disappointment. When I was invited to go to Leeds to hold an event, I
jumped at the opportunity. However, this turned into an unsuccessful event, but since all of them were
'scratch'/iterative/ work-in-progress, performances, there was an understanding that each would be a
learning opportunity and each would build on each other.
The 5th of October was part of an Open Day and my host had just recruited some of her students to
help her. She wanted to give them something fun to do as a break in their day. Unfortunately, again,
the IT department of this college did not understand the project’s Internet needs. Even though they
were provided with a detailed Tech Rider (technical specification), they did not seem to realise that
streaming on and off campus was necessary, and so they only provided Ethernet cables to access the
Internet, but not the ability to stream data from the sensors over the Internet to the server. On site, the
mobile service reception was very weak and it took up to thirty minutes to upstream to the qik site.
Again the event was NOT real-time or live and it was problematic to mix the videos out to the web.
Thus, there was only the ‘live’ (in-person) group, who participated in two aspects of the workshop; as
with the previous workshops, this resulted in only about three to four participants making the video.
At this event, none of the participants felt comfortable wearing the garment. The Internet connection
issue would have prevented this data being sent over the Internet in order to trigger the mixing on the
server anyway. As a result, the remote audience who were trying to view the live stream of real-time
218
mixed video saw only a live mix of the archived workshop video and in 'demo-mode' to ‘fake’ receiving
the body data over the network.
Another unique aspect of this event was that, there was a remote helper at the location where the
server was hosted, which was not necessary at the first event. He made sure that the stream was
working for the remote audience while I was in Leeds. We kept in touch through mobile phone,
although the mobile reception was so poor that this was difficult. Another constraint was that his help
was only available for a thirty-minute duration, so the online aspect had to be short. During the in-
person aspect I had two potential helpers, but with so few participants they were not really needed and
one had other responsibilities hosting the open day, so the other became a participant.
Event Summary: L ive (real-time) = people in the space making recorded video, remote audience watching live video mix from archived footage online and waiting for live q ik video (that took 30 minutes), and one live streaming qik video camera. Presence (of others live and remote) = people in the space, people remote - but not connected or able to interact ____________________________
3) December 4th, 2009 with peers and guests during the SMARTlab Christmas party (4M Studios, London) This event was planned in advance as a celebration that concluded the performance series informing
the thesis. In most respects, the event was highly successful [240]. It was also more controllable since
the streaming server and the laptop receiving and sending body were both in the same space and
were accessible to me for constant monitoring.
What was distinctive about this event was that it was actually in a party context, as intended, and so
unlike the other events it was not quiet or contrived; it was more spontaneous, unselfconscious and
people were there to be social. Some attendees had no idea that others were participating and were
curious when they saw others running about with cameras and doing 'odd' things in the space. There
was one LCD monitor displaying the live mixes of video from the body data in a prominent place, and
many assumed it was just visual 'eye candy' for the party; they did not appear to realise (or ask) that it
was being mixed live.
The unique attribute at this event was that both garments were available. However, the sensors of the
beta garment had become damaged from wear and tear after the first event in July and were not
usable, so we only used the newer version. There was one guide to help out and one participant
wearing the sensor garment, and there were video participants. I conducted the meditative introduction
activity and guided the video participants through the video 'assignments' and gave the sensor 240 My measure of success is all the of the technology working as intended, all of the participants having a good experience and some of them having a sense of presence of the remote participants.
219
participant some guided movement activities. This was the first party environment I had held the
performance at, so participants were guided do the tasks, but could interact with the party attendees
as they wished. And indeed, participants all reported that they really enjoyed it.
However, there were still some glitches with the technology since the php fix script between qik and
the server was not working properly on the server. It turned out there was a new conflict with the
streaming ports. Again the live mix was only from the archived footage [241]. Also, none of the
participants had the required phone or compatible software, so a research phone was again provided,
but this meant that only one phone was streaming through qik . The participant using this phone was
successful in the upstream of her videos, but most of the streams did not arrive to the qik and
MINDtouch page until the next day. As this aspect was out of my hands it was a little annoying, and
as a result there was no ‘live’ video available for the server to stream. There must be something that
slows down the qik servers: perhaps too many users streaming at once causes a bottleneck. This
would validate the need for custom software to stream directly to research server, which there was no
time or additional resources to develop.
Event Summary: L ive (real-time) = people in the space making recorded video, live video mix in the space triggered from live body data, remote audience watching live video mix from archived footage online and waiting for live qik video (that never came), participants contributing video on qik remotely, and one live streaming qik video camera... Presence (of others live and remote) = people in the space, people remote - again only one way participation for online and in the space - still no interaction ____________________________
4) December 18th, 2009 with a group of students at the Limkokwing University, London, organised by a collaborator on other projects, during their student mobile project exhibition and Christmas party (3 participants, no documenters).
This event was similar to the Leeds situation in that, despite best efforts of my host and myself [242],
the IT department did not set up the Internet connection for the event in a way that was useful and then
went away for holidays just before the event. Therefore, the Internet connection again did not work.
Student participants were supposed to be organised in advance; however there were none available
when I arrived; instead they were participating in other activities at the event. At first it was quite
disappointing to be invited to show the project, then given little support from the host organisation. But
again, the show must go on, and as such, an LCD screen was eventually organised and set up for my
laptop to mix the video collage in 'demo-mode' as background ‘visuals’ for the event. Once that was
set up, students became more interested and three were recruited as participants: two making mobile
video and one wearing the garment with the sensors. However, the students making video confessed
241 Although I had been adding footage into the archive manually from each event for the one to follow.
242 See the tech rider/ specification I provided them in Appendix II.
220
that they were unclear how to do the activity and focus with all the other festivities going on to distract
them. Therefore, only a couple of videos were recorded and the participants could only engage with
the activity for twenty minutes. It was evident that they wanted less to participate and more to be given
a demonstration on the technology behind the project, and then get back to the party. The participant
in the garment was more engaged and enthusiastic about it, putting in more effort to make her body
trigger the video output and see the data on the phone interface. However, the mobile software began
to crash the phone and not start up properly. Meanwhile, the computer version of body day software
also was not receiving the data properly. As a result, this event was fraught with communication and
technological issues. So without technical assistance or participant guides, I was not able to split my
attention to stop engaging with the participants to fix or at least jerry-rig the technology. Since the
students’ attention spans were short, it was a priority to keep them engaged as long as possible.
Event Summary:
L ive (real-time) = people in the space, recording video, live video mixing in the space in 'demo-mode', the remote audience waiting for anything, participants contributing video on qik remotely via one live streaming video camera...
Presence (of others live and remote) = people in the space - only one way participation for those in the space
____________________________
5) February 13th, 2010 at a friend's dinner party for Valentine's, with new acquaintances and unknown participants (to me) (6 participants, no documenters)
In an effort to collect more and varied data before having to close the research (due to time and
financial constraints), and while the server was still available, I asked a friend if I could bring all the
technology to his large Valentine’s Day dinner party, to see how the performance activities would fare in
a more spontaneous environment. This was part of the effort to try different contexts for the event, and
it was thought to be best suited for parties, concerts and festivals, so this was another opportunity to
try that idea. However, one realisation surfaced in hindsight, which was that some help was needed
even in this context. A remote operator with the server would have been useful, since the streaming
still required manual operation until the automated or remote control functionality requiring networked
operation was working smoothly.
This was a fresh group, all whom were unfamiliar to me, and all from more varied backgrounds,
experience and ethnicities than previously worked with, which could make a difference in results or
feedback. Before the dinner party was fully under way, willing participants were recruited. One woman
wore the garment and was an ideal candidate, with very little experience with technology or
art/performance practices, and with a career in business consulting. The other participants included a
cook, an osteopath, psychologist, and an IT specialist,. All were of different ethnicities: an Egyptian
American, one Israeli French, a Cuban, a German. It was a great mix and none of them had done
anything like this. However, one drawback was that it was more difficult to convey the project concepts
221
or explain the activities quickly (important before all other party-goers arrived), and to get participants
engaged. The difference was that for example, during the first event, which had more artists,
performers and digital media specialists (peers), it was easier to explain the project since they
understood the language and domain of the performing arts and digital media. This difference was
good practice for me however, and as with the event prior to this one, it made me realise that the way it
was explained was too complex for the ‘public‘. More work needed to be done to simply the
explanation in order to really engage participants in future versions of this project. This underlines the
value of guiding.
There was only one attendee who had a compatible phone (the IT specialist), thus participants had to
use the research phones again, and so there were none remaining for the sensor participant to see her
own data, with only the laptop to show the live mixing as visual feedback of her body data. This was a
bit too abstract for this participant and she found it hard to conceptually connect the idea that her body
data was triggering the visuals and just wanted to see her body data alone. This was a side result that I
had to contend with, and while I had intended to show people more direct visual representations of
their body data to help them understand what was happening, it was not the goal of the project, so it
was sacrificed for the other elements to operate effectively. Instead effort was made to help participants
to see that their bodies were acting as a VJ or live cinema performer, and this was eventually
understood.
Again, without an assistant it was difficult to control the remote streaming, which was having some
difficulties. However, the remote participants left me messages on the qik site, which in turn, were sent
to my phone as text messages before we got started in the in-person space. This was an unexpected,
but excellent and welcome addition to the live interaction, even though only I was aware of these
messages within the local space. The remote participants were asking when we would get started,
since they were waiting online and they were making comments on the videos from a previous event
that were upstreamed and archived on qik . Had there been a moderator or online guide to work with
the remote group, or a remote server operator who also moderated the remote group, it would have
been easier to facilitate more interaction between each group. Again, the video from qik did not upload
properly in real-time, taking over an hour (improved from the last time, but certainly not live), and the
'video pull' php script was not properly working to transfer the video from qik to the streaming server
archive again either. Event Summary:
L ive (real-time) = people in the space making recorded video, live video mix in the space triggered from live body data, remote audience waiting for live qik video (that never came) and the mix did not work either, live participants contributing video on qik remotely with two live streaming qik video cameras, two remote participants making and contributing (new). Presence (of others live and remote) = people in the space, people remote watching and making video - again only one way participation for online and in the space - only networked interaction was
222
between me and the remote audience / participants via qik / mobile messaging.
post performance observations These notes came after the December 4th event but seemed applicable to all of them as an overall
assessment of the organisation of each event, discovered through iterations of them in practice:
1) Each was a manageable size when there was so much going on around at the party already;
2) Participants said they loved the activity of recording and streaming video––a private activity that
is only theirs within the social environment––and reported that it felt like it was just for them;
3) It was not clear if the participant who was streaming the video via qik completely understood
that she was connected to others online remotely (even though it was explained);
4) It became very clear that guiding is essential in this type of participatory event, but that I cannot
conduct these events entirely alone, since one person must put the garment on the participant
while someone needs to attend to the others, the technology, server and remote group, and
each group needs guiding. Guiding needs time: time to orient participants to the experience
they are diving into and explain what is happening overall (without dictating or influencing their
experience). The guide's role is to orient the participants, to reassure them, to give tips or
thoughts on how they can focus on the activity to illicit the video recording and how to can
translate what they are feeling outwardly, into a visual manifestation. Guiding is more difficult
with someone in the garment, because they have no direct feedback to watch their data, other
than the visualisation waveforms from the custom software on the phone and the resulting
video mixes. It was intended that there be a phone to do this each time, but we ran out of
phones almost at every event due to people not bringing compatible models and not having
enough research phones. It would have helped people wearing the garment, not otherwise able
to make the mental connection to their bodies, what was happening with the sensors without
this aid;
5) The online group needs more guiding in advance in future events. A message will be recorded
for them, with more instructions to prepare them before the next one, so that they can see
what is going on online more easily. A documenter who is streaming live from the event is also
necessary for the remote group as well (intended this time but the documenter’s phone battery
expired part way through). This should be on a different page on the qik site or custom
software be made so that the files do not get mixed up with the files going to the server;
6) The visual display worked well and people at the party who were not directly involved in the
activity enjoyed the display, especially when it was explained to them. Some became excited
by what was taking place around them with video makers. It was as if there were invisible VJ's
in the room which made it seem more compelling;
223
7) The documenter’s role is key to the documentation and they would be more thoroughly briefed
and monitored to interview others not part of the activities for their reaction or impressions of
on the event during and after. This would assist in discerning the external, incidental ‘audience’
response to the work.
8) One major concern was the time needed to dress the person in the garments, and the delicacy
of the sensors [243]. Effort had to be made to try not to further damage them. The way the
beta garment was made it fragile and easily damaged (since the breath sensor is used to
tighten it). However, discoveries like these were part of the research process. In future, more
durable and flexible wearable electronics would be desired. It would need to be easier to put on
and less cumbersome for the wearer: hiding the electronics is not the goal, rather working with
conductive threads and fabrics in an easy to work with manner is (for another research project).
Another major challenge was all of the network and security issues and passwords; complexity
of the connectivity. Guides and Assistants Necessary for Events include:
• A guide for the videophone group
• A guide for the person wearing the garment
• A documenter
• A guide for the online group is needed add info to my site and to qik / streaming
In future, my role would be to become the monitor of the whole event, more like a film or theatre
director, moveing around the space to explain the activities and event to those not involved, as well as
to explain how the visuals are created, especially if participants are not able to access the video directly
on their phones (as intended).
A.1.2f/ participant response analysis After reviewing the comments from all participants, conflicting comments arose from the same aspects
of the event. One example was when undertaking the 'telepathic' exercise, when many reported that
they really enjoyed (as did those in the workshops), and yet two really did not like the idea (though did
not say they did not like the activity itself). There were other conflicting responses to the video exercises
from a couple participants in the moving activities during the first event [244]. As such, different
activities worked for different people and not others, an expected variation. However, based on the
243 The alpha garment’s sensors were damaged and returned to the electronics collaborator Michael Markert to be fixed, so only one could be used for the performance. 244 See the appendices for the specific responses in conflict here to contextualise more.
224
written responses most participants enjoyed the events overall, yet shyness or self-consciousness
could have interfered with the enjoyment for a few. The guiding process worked very well in the first
and third performances because I had more assistance to manage all the groups and the technology.
By contrast, in the second, forth and fifth performances, the participants reported that the activities and
the project needed more explanation. This outcome is understandable, since recruiting assistants was
difficult, so I was left managing too many things on my own, leaving participants on their own more.
Overall, the network connection and interaction was more abstract or obscure in the minds of
participants [245] as mentioned earlier. However, it would not have been technically possible to have
displayed both the live video mixing from the laptop and while showing the activities of the online
remote participants, simultaneously during the smaller events without additional computers. Nor would
I wish participants to split their attention to do both, because they then would not be as fully engaged
in either activity. Therefore, this aspect involving the engagement with others remotely over the
network, evidently was not clear or felt by most, if at all unfortunately. In addition, most participants
misinterpreted the question in the survey on presence, which in hindsight may have been obscurely
written, asking them if they felt […] the same sense of presence or interaction between performer–
audience in this event as you might in a conventional performance event […][246]. There could be two
reasons why this was misinterpreted, in addition to the wording, they are: 1) interactions between the
live and remote audiences were not obvious (in the first event there was no remote audience, as
discussed above), therefore, the only sense of presence felt would have been of each other in the
physical space, and within the separate groups performing the activities; 2) the participants did not see
themselves as performers (which was what I tried to foster in order to breakdown inhibitions, also
mentioned earlier in Chapter 3), and the guides helped participants to feel comfortable and less
intimidated. This was also the case whenever I was the only guide. During those events, participants
expressed that they thought of presence in terms of live presence in the space and of those in the
room physically, live in-person. Participants who knew about the networked aspect [247] or
understood what was supposed to happen, reported that they did not feel the sense presence of the
external interactors/audience and felt disconnected from them. However this would make sense, since
those responses came from the first event where there was no network connection. Only in the last
event, when text messages were sent to my personal phone from the remote audience, did the live
audience have an awareness of the remote interactors, and only because I told them. Even then, due
to the delays in qik 's streaming service and due to the lack of additional equipment or support,
participants were not able to see these remote messages themselves and we were not able to set up a
second computer and connection on site (and it was inappropriate at the time) in the party context to
show them.
245 It was 90% working this time, but still one piece of the exchange/ interaction loop was not connecting – my programmer failed to get php qik script to work on the server even though it worked on all other machines we work working on and we did not have a network specialist with us to determine how to solve the port conflict where the server was housed. So this examining piece of the puzzle was never solved however I will change servers for upcoming events and hope to solve this part of the problem or make a custom tool to stream from the phone to my own server.
246 The primary question dealing with this sensation during the event. All the questions can be found in Appendix II.
247 This was explained to everyone, yet many may have forgot, or as seen in the responses, did not understand it.
225
Many intriguing discoveries were made when reading through the performance and workshop
responses (and drawings), even though there were inconsistent network connections and there was
never a complete connection loop, completely linked. So unfortunately, the full interaction was not
possible to experience presence during a mobile, networked performance or to study. Yet one great
discovery was that people really engaged with the technology in solo and pair activities, and in making
very intimate and personal expressions of their inner experience. They did so within the social
environments and contexts, and many found the experience very relaxing, some said 'therapeutic'. Like
the workshops, the unexpected outcome was the willingness of participants to engage in something
quite different for them, especially using mobile phones, and involving a fairly personal exploration
within a social context. They did express a wish to be apart of a larger artistic endeavour. Still, if more
of an interaction with the external/remote audiences was more obvious (and fully operational), there
would likely be a whole new level of interaction between participants, which had been the aim. As it
was, a fun yet personal, creative experience was enabled without the network, which still provided a
participatory, expressive, interaction using mobile phones and wearable devices. Therefore, a majority
of the aims were achieved. The social aspect was there, the engagement and a sense of interactivity
were there, and a sense of liveness and real-time was definitely experienced (as noted after each event
above). Missing was the live/real-time and collaborative networked interaction. Although one could
argue that the project was still a collaboration between myself and the participants, as well as the
participants with each other, since all visual material was sourced from previous workshops and events
and was used in the live context. However, there remain missing elements I was aiming for in terms of
closing the technological and experiential loop: real-time, networked presence to be felt and
experienced.
Overall, participants reported a definite experience of some sense of presence and embodiment within
and through the mobile device, as well as the media artefacts they created––this succeeded very well.
Participants gave themselves completely over to the experience, almost like a therapy, which seemed
enjoyable for most. It is not clear what influence I, as the researcher/ facilitator, had on them, since
some of them were my peers and friends of friends. It is possible (and desirable) that my facilitation and
that of the guides fostered a sense of emotional and participatory investment, trust or eagerness to
engage. Since I am unable to assess this, one might assume that I had some effect on the experience,
not only in the activities but also in the delivery, reception and experience for participants. This also
affirms the aims to improve my facilitation skills.
The differences that emerged between these five events became essential to the 'iterative' nature of the
research. While each event featured many similar activities and processes, successive events were
changed based on the circumstances of the host environment/context and the learning from each
previous event. For example, the 'placebo' movement activities of the first event were dropped in future
events, based on feedback and observations. In addition, the participant 'observer' role was also
dropped, but this role did garner interesting results. For example, many participants were instructed to
take notes of anything that came to their minds while watching the others. This resulted in many
226
expressing themselves through free-association. This appeared liberating for some, as it allowed them
to use the familiar and comfortable modality of words and language, not to be used in the other
activities. By contrast, a few participants did not like moving or being observed. Unfortunately,
participants did not end up moving much during the physiological sensing activity, which the movement
group was meant to prepare them for, without guidance and on-going encouragement. In addition,
their movement did not result in any grand or energetic gestures that would register any significant
fluctuations in the representative data visualisation. This was because most participants merely paced
around the space and occasionally waved their arms around. As such, the sensor software had to be
recalibrated to quite a low threshold to register any differences or changes to trigger the visual media
mixing software and resulting visual output [248].
During the December 4th event, the lack of movement was very unfortunate and the mixing software
had to be changed to demo-mode because the participant was not moving enough, and did not
respond to guidance to do so to change the visuals (she ignored it or felt shy and inhibited with her
workmates there). While it was intriguing to people conceptually that they could watch their body
responses and signals, seeming to fascinate them that their bodies were triggering the visual effects
and mixes, it became less engaging and too abstract for participants later in the event. Yet other
participants reported that they would have liked to have had more information on their body data. The
video activities and the idea that their own videos would show up in the mix at some point, as well as
the intimate and personal experience of expressing their bodily and emotional sensations, primarily
seemed to excite people and casued them to state that they had had a good experience. Not to
mention that the relaxation element facilitated participants to connect with their own bodies, the
activities, and to help them feel less self-conscious.
In summary, as a personal, creative perception activity, the video expression exercises were quite
successful and reportedly enjoyed by most participants, even though the networked interaction was
missing (many did not notice). Yet even if the network connection completely functioned as expected,
perhaps if participants in the social, live context would still be more absorbed in the video activity and
personal expression aspect and more into their own creations than they would be in the interaction and
collaborative creation aspects with an abstracted, remote audience. In addition, even if all had worked
fully, the remote participants appeared to have been a voyeuristic. Only one online participant
completed the online survey and only two contributed their own video to the database. Still others
reported to have only watched the live streamed video, feeling unsure if was the totality of the
experience/event (even though they were briefed in advance), yet they did not contribute. Perhaps
having this aspect, with a more 'live' moderator, with a means to 'log in' [249] and see the live group in
action (through live streaming documenter), could assist in motivating the remote group to participate
more actively. In addition, having a more simply stated set of instructions and guidance might help
248 Originally there had been an intention to involve Tai Chi practitioners to perform their movements, though this did not end up being included, it might have resulted in more varied visual collages. 249 This was intended and an attempt to implement was started, but then it became more complex with time constraints to be completed and implemented online.
227
(although effort was made to make it easy). It could be inferred that a simpler or more guided
facilitation, with a more engaging and simple online and mobile interface, which tracks users and allows
for some moderation and motivation could improve interaction.
From the overall feedback received when there were guides to assist, the experience was better for
participants. In addition, during the three events where I was unaided, the participants felt that they
needed more assistance to do the activities; also the issue with the remote group. Therefore, it could
be said that these "scratch"/iterative performances were like user testing events in preparation for a
more successful MINDtouch 2.0 series of participatory performances. A.1.2g/ overall performance assessment
Below are some overall observations from the staging of these initial iterative events that can be taken
into future versions of this project.
performance contexts From this research, using the various contexts for the events, it can be seen that each have provided a
different motivating element: when events were organised in a contrived and purpose-specific way
similar to the workshops, it was difficult to recruit many participants. Based on people spoken to in the
recruitment process, many were a bit inhibited by the sound of the activities: fear of not being creative,
fear of technology, fear of looking foolish, fear of being ‘in’ the video [250]. When, by contrast, in a
more controlled group like a class (still contrived), these fears may still have been there, but there was
more complicity. In the party context or small group, people were more willing to try something new
and even if they do not know each other, they become more open and social. In this party setting,
people were coming to relax and socialise, but not expecting to do anything more challenging that
drink, eat and converse, maybe dance, possibly sing, even play games. Either way, in these two
contexts, the participants were more actively seeking new and challenging experiences. However, both
times when caught unawares, there were people still ready to try something a bit foreign for the sake of
potential fun. Of those I worked with, the younger participants were more cautious (to my surprise),
more self-conscious (more expected) and less open-minded to the experience than the older
counterparts.
body data Not only because it was part of the BBC funded work, but the physiological sensor use was added as
a mechanism to change the video and add more 'embodied' effects and attributes to the presence
experience, especially in the live context. But it was discovered that the sensor-embedded garments
250 Although that was not the point or even necessary, but people had to be reassured of this.
228
were also effective in engaging people with the concept of visual body mixing and the interactive
aspect of the work. However, they quickly lost interest in their body data when they could not ‘see’ it
more concretely. One would assume that participants would feel more engaged with the work if all they
needed to do was move and 'be', although that was not witnessed here.
When guiding participants through the participatory activities it was important to explain what the
sensors represented in the body and how the data was used in the performance. Some people were
asked to wear the garments with the embedded devices measuring breath, temperature, stress,
muscle activity, etc., so naturally they wanted to understand the data more. These participants were
asked to move and express their sensations and perceptions, and were guided through the activities
with their bodies, embodying the devices through their movement. While the other videophone
participants did the same through the video imagery.
Participants did wish to know more about the sensing aspect more directly and specifically, and which
body activities were effecting the video/visuals, not a feature built into the current project. This was not
something the programmer was asked do, and it would have added even more complexity to the
current complex programming and weave of technologies and interactions. However, the sensors can
add to the experience and should continue to be implemented, and perhaps more simply. Ideally I
would use prefabricated garments (or drop the wearable aspect for simplicity), giving participants more
direct and obvious feedback on their body actions.
MINDtouch 2.0 What was observed during the staged events was that people have trouble literally seeing a connection
between the remote interactors and what they are doing within the live setting. To redesign this piece
further, efforts and method would need to be implemented to demonstrate and visualise this
connection to participants. This might include an additional interface to show both sides of the
interaction for each group [251]. Therefore, a feedback mechanism is needed for each group to see all
of the activities, such as an online and mobile application, with a much more powerful server behind it,
and the aid of (hired) operators to manage the incoming and outgoing activities, live and remotely. As
such, a proposal for a new project is arising from and informed by this research. Such a project
includes guides and moderators for each group: live groups and remote mobile participants (on
phones). Also necessary would be an ability to tag videos captured (as qik enables but for a custom
software), to better enable the videos to stream into the correct folders. This would also guide them
into appropriate (to their emotions, sensations and perceptions) or participant-made categories,
including comments for the online interface (with moderation). In addition, this might involve the online
guide requesting that lurkers and non-participants report what they see. Then how participants
experience it could be compared to those in the live or mobile video groups.
251 This also was part of the original video but also had to be abandoned due to time and resource limitations.
229
230
A.1.3/ visual mixing software + streaming development process It has not been revealed yet as to how decisions were made with the video-clip database or footage
folders in the development process of MINDtouch . This process was iterative and in collaboration
with the creative programming experts/artists on the project. Below is a desktop screen shot showing
the interface of the Quartz Composer programming environment and the specific patch or application
that was made by Manjit Bedi, using another earlier created application to receive the sensor data
made by Michael Markert.
The process by which the visual mixing software was developed using the body data from the sensors,
was as follows:
1) The first decision necessary was how to define how to work with, differentiate and categorise
the video content collected from the workshops.
• This necessitated separating the videos into a small number of folders located on the server,
each corresponding to the number of sensors used in the garment, worn on the body
sending the body data to the custom mobile application;
Image 73 © 2010 C. Baker – screen shot of the Quartz Composer patch for video mixing
2) From there, a decision was made about the process by which this video mixing software
application transforms the body data and adds visual effects to clips,
• Metaphorical associations were made related to qualities of life-force and kinesthetics of a
human body as correlated to each sensor, followed by how to visualise these life-force and
kinesthetic qualities. The following were the choices made (figure 7):
231
sensor life-force / kinesthetic quality action = visual effect to apply
breath life-force, internal winds (yoga) +
well being
Initial: slow or fast forward or back or change speed
up according inhalations or exhalation speed
Final: swells / an effect to stretch and contract the image
temperature fire, passion or ice, fear intensity -
emotions hot or cold
Initial: colour changes or colour intensity or contrast
Final: contrast (darkens and lightens)
GSR mental / nerves/ stress/ fear or
calm
Initial: jittery or smoothness of playback, sharpness
or blurring
Final: from blurring to sharpness EKG life-force, blood, emotions, rhythm Initial: swells of opacity / mix overlay + transparency
Final: colour changes or saturation of colour
EMG muscles, strength, movement/
gesture
Initial: drawing + animation, mobile interaction/
changing the video or adding text or image
Final: (same as EKG) colour changes or saturation of
colour
Figure 7 © 2010 C. Baker – physiological sensors mapped to life-force qualities then to visual effects
3) Next we implemented the visual effects necessary to emulate or visually communicate the
corresponding life-force metaphor (see chart above) within the possibilities of the
software/programming environment of Quartz Composer.
• I brainstormed ideas, including personal assumptions of these qualities. To meet time
constraints of project event delivery dates, Manjit implemented these ideas.
• He then interpreted these concepts within his programmatic decision-making and creativity,
within the visual possibilities of Quartz Composer (see Image 73 below, which shows these
implemented into the patch).
4) The videos were selected and categorised as they corresponded to each sensor (Appendix II
for this spreadsheet). Each video was reviewed and a spreadsheet made of the visual content
and qualities observed within each video, and were placed into the most appropriate of the four
folders.
• The folders were labeled based on the predominant activities conducted with the
participants in each workshop.
One challenging aspect of this decision process was that this categorisation was required for the
archived videos from the workshops, as well as to be applicable for the live video streams during each
event, where the live video would stream directly into the folders on the server. Initially, the intention
was to create an additional custom mobile software application that would tag each live streamed
video (qik has this functionality, but it is only available for a few smartphone models. Tagging would
232
make it easier for people to add code that would direct the incoming streams of video into the
appropriate folders, in order to then be mixed according to the body data and then streamed back out
to participants [252].
Future iterations of the technology system will include tagging and folder interaction for the database.
For past events, participants involved tended not have streaming/qik compatible phones, therefore the
research phones were the only ones used and qik software available late in the research, thus
participants were not instructed to tag. Making sure the associated content gets into the corresponding
folder to be mixed according to the sensors and activity is crucial, but has been tricky in practice.
Hence, archived clips from workshops have been mostly used for the first performances. More
facilitation with participants will ensure that tagging takes place and that the content itself matches the
participant's intent for the output video the database to work corresponding to the sensors in future
events. This will complete the loop of: body data––> sensation/life-force quality––>visual effect––
>montage remix of body and body sensation video expression/dialogue ––>abstract visual
collaboration (only 90% achieved for the events staged).
Image 73 © 2010 C. Baker – zoomed in screen shot of patch for video mixing from sensor data
252 The software was in development all the way up to the last moments of the allowed timeframe for the PhD part of the project, so we did not yet complete this part, nor managed to get qik tagging function used in performances.
233
Another issue worth mentioning is the daisychain of tools, scripts and applications (see Figure 8 below),
necessary and implemented to bring the vision some form of fruition.
Figure 8 © 2009 C. Baker – demonstrating the technology connections
Even though the concept seemed simple, direct and beautiful to me: a simulated, embodied, mobile
global, collaborative live cinema performance. In reality, ended up consisting of (see Figure 9 below to
clarify):
1. body to sensor device to Arduino hub;
2. Arduino sending data to mobile phone or to my laptop computer [253], with only one-to-
one BlueTooth connection at a time (not multiple garments and sensor performers as
desired)––so one or the other but not both––I chose the computer to send the data to the
server and had the second garment go to a second phone to visual the data for
participants;
3. data then sent from the laptop through Quartz, through the Internet to the streaming
server;
253 When using Quartz Composer, the body data ended up having to be sent to computer, since the mobile phones are not yet sophisticated to deal with complex applications like Quartz (also not compatible yet with Apple software that we all chose to work with for visuals); soon this will change and be possible with the iPhone.
234
4. on the server-side, the data received is processed by the Quartz Composer and another
software simultaneously (Quartz Crystal) to encode the mix into one mini-clip to be
‘streamable’;
5. then it is taken by Apple’s QuickTime Broadcaster, in conjunction with the Apple Streaming
Server software, and streamed to the Internet and phones (ideally) with the (see figure 9
below).
On the streaming server there were the folders of movie clips receiving the incoming streams (as long
as no firewall prevented the streams getting to the Internet and server, on both computers). Ideally, this
would be with all web ports open on both local and remote computers, with an Ethernet connection
and static IP address on both ends, to omplete the loop shown in Figure 10 below. However, we then
needed to use XML on the server side, in order to prevent the information from being intercepted by a
firewall and allow the stream to be received by any Internet enabled computer or Real Time Streaming
Protocol enabled movie player, on any computer, or mobile phone, anywhere in the world. Screen
sharing other intermediary tools also had to be implemented. A sketch of the server-side processes is
below in Figure 9.
Figure 9 © 2010 C. Baker – demonstrating the server-side processing
Additionally, the participants who were live streaming video with qik from their own phones had to
send their videos to my qik page, otherwise I would not have had access to them and it would have
235
been harder to access them and they would not be sent to the streaming server for the mixing. It was
necessary to have participants all log into qik using my account, where all the streams were collected
and stored after streaming. There was a custom script written in php to pull those videos off the qik
server and then convert them from .flv (flash video files) into QuickTime .mov files and put them in the
right folders for the streaming server to process for streaming.
Figure 10 © 2009 Evan Raskob (collaborator) – sketch of networked interactions
This figure demonstrates how the technological and networked processes interacted during these
performance experiments. Naturally, there was a lag created by all this processing, thus, a delay of
real-time or liveness occurred in actual events and qik alone could take up to thirty minutes to process
each video in some cases. Also, the software Quartz Crystal encoded and rendered the mixed video
from the data sent to the Quartz Composer patch to create a streamable video clip, which could take
twice as long as the desired clip to render; the longest clip was two minutes. Smaller streams were
preferable to ensure quicker, more live transmission, but this meant a technician was need to monitor
and enable the rendering process mix by mix on the server-side for each performance. Since this was
the only solution found to create a streamable piece, it again added to the overall processing time and
again was not enabling a real-time experience, and was more like making a podcast. So at first, the
stream was live, with hinted (preloaded) video, and then the process begins to slow down in the
process of compression, rendering, then delays and lags in the streaming took over and these delays
started to lengthen the whole stream.
While in-person Live Cinema or VJing IS real-time, since all the technological production elements take
place all in the same space and in-person, live, with all the network complications of this type of project
236
the real-time aspect is NOT quite there , not without delay. STILL, after all the years of online
teleconferencing and videoconferencing, it can be unbearably slow. Tim Berners-Lee stated in a
GuardianTech podcast in January 2010 [254] that the new version of the Internet is coming and will
improve such issues, so very soon mobile video-Skyping and other such mobile Internet application
should be lag free. However for this project, using a multiplicity of technologies to study a technological
embodied, visual collaborative dream mixing: mobile phones, sensors, etc. with telematic and
networked performance using video, has been very complex, more complex than intended or
expected. In retrospect, it is clear that perhaps too many layers of complex interconnection were
necessary to achieve the fully live-streamed video, VJ mixing and sensor technology integration that the
project set out to achieve. In this way, the original aims were over ambitious. However, the attempt to
achieve these ‘big picture’ aims pushed the project to reach beyond the possible and to invent and test
new boundaries for live interaction using real time networks. While the goal of achieving a real-time
telepathic simulation using mobile media remains a strong motivator for further research, it would
necessarily extend beyond the scope of this thesis. Technical collaborators on the MINDtouch Project
from the BBC and external experts all confirm (see testimonials, in Appendix II) that what the project
seeks to achieve, while exciting and innovative is also too extensive and ambitious to be completed
within the few years of a fixed research project or PhD. This would, if extended require much more
support and technical expertise on a greater scale. All the technical consultants have confirmed the
success of the MINDtouch Project in its own terms as having made a significant contribution to
knowledge, to be explored further by much larger teams in major industry bodies over years to come.
A.1.4/ video content categories + analysis
After reviewing all the video clips made by participants in the workshops, the main visual themes found
were:
1. close-up and extreme close-up shots focusing on but abstracting the face and body so as to not be easily recognizable;
2. movement of camera either surveying the space or more vigorous movement like shaking,
and fast movement of the camera and/ or the body to create blurr ing and streaking (walking feet very common), and other visual patterns resulting from the movement;
3. patterns of all varieties, especially l ines and inter ior architectural features, and using
movement of the camera to accentuate these visual patterns (lines very common): such as moving elevator doors, wires, radiator coiling, spiraling staircases, etc
254 This was heard in a podcast but an archive could not be found again but the story is available online to read at: http://www.fastcompany.com/1593583/tim-berners-lee-to-head-up-institute-of-web-science-in-uk-as-britain-goes-for-broke-on-the-w and http://www.ted.com/talks/tim_berners_lee_on_the_next_web.html
237
4. natural and outdoor elements: trees, water, clouds etc and any patterns to be captured
from them.
Below is a chart (Figure 11) that was made to connect the various pieces of the workshops to imagery
created and then to the sensors of the body and database folder for the performance collage output. I
wanted every aspect of the workshop and performance activities to have correspondence and a
relationship to the sensor and video manipulation in the final performance context. Therefore, since
there were probably more refined categories that could be made from the visual image content,
resulting in further categories and folders in the database, this would be too complex for the Quartz
Composer and other software developed. The four themes focused on were those corresponding to
the four main activities in the workshops (although there were actually five in the workshops. I narrowed
them to four categories for performances, for the four sensors that trigger the videos, with four streams
of video from four folders in the database on the server.
main content/ camera
action
related activity corresponding
sensor
folder
placement
close-ups and extreme close-
ups of face or body
use only yourself or environment in
close proximity (representing an
internal experience); also the
‘introduce yourself’ exercise with a
partner
temp warmth/
emotion = 1
folder 02
moving camera survey room
or space, gestural / blurring,
feet moving
near-by space, beyond the body
(representing an internal
experience)
GSR nerves = sensor 0
folder 01
abstraction of architecture +
interior objects and decor,
moving objects: elevators,
wheelchairs, water
with a partner, make something
that you might send to others on
your internal experience
(collaborate without speaking)
EKG/ EMG -
blood = 2
folder 03
long shot of exter ior /
outdoors, close-up of trees,
clouds, grass or natural
world, outdoor shadow play
some from above activity and
some from the telepathic exchange
exercise (see workshop
documentation)
breath = 3 folder 04
Figure 11 © C. Baker – chart of the activities, resulting content and connection to the database and sensors
A deliberate effort was made to connect the content themes found in the participant video imagery,
regardless of the workshop location, to the metaphorical qualities assigned to the sensors and mapped
to the visual effects to mix them with the software. This connection correlated to an emotion or physical
238
attribute to an activity in the workshop [255] and it was very clear that certain activities elicited certain
types of visual content creation and visual imagery capture. This outcome directly addresses the earlier
discussion on non-verbal communication through visual material, because it became evident that
certain activities could be developed (in future) to help create a specific visual communication language,
or grammar to represent certain emotions or internal experiences, but using external material as
symbolic, semiotic or metaphorical representations of those internal experiences; for another research
project. Below is a sample of the spreadsheet used to categorise and analyse the video content in
order to appropriately assign it to a database folder on the streaming server.
Image 74 © 2010 C. Baker – screen shot of one of the spreadsheet pages (see appendices for full spreadsheet)
A.1.5/ mobile/ networks and connectivity During the last stages of phase three, concessions and modifications were made all along to the
original vision of the project in order to get it to the performance stage. These occurred while organising
the last ‘scratch’ events before the project officially ended, and while reflecting on the whole process I
realised that these were necessary in order to facilitate a networked connection through the mobile
phone, This happens in most projects and in developing performance and other productions; it is part
of the evolution of the work. Some of those concessions and modifications included:
1) having to give into the possibility of foregoing a mobile only interaction for all technical operations of
the project as:
255 This was NOT based on participant feedback, as they were not (and were purposely) asked the literal connection between their visual expressions in the videos and their actual emotional, physical or perceptual experience that they were trying to express – this was though to be too invasive and participants were reassured that this personal information collection was not part of the work, but merely for them to express abstractly. Therefore this connection was made was based on my own interpretation of the material and its correspondence to workshop activities it came from.
239
a) we could not close the loop from qik to the streaming server [256] before development time
limit expired, due to unresolved server port conflicts. A network specialist was needed and not
available to address this problem, which would need doing before future events are staged, but
there was not the time or resources to do so for the purposes of the PhD);
b) loading and playing the mixed videos on mobile phones using RealPlayer (the only player
available for mobile phones currently) on the Nokia models used for the project [257] was
problematic it turned out. Nokia online forums indicate that this has been a common problem for
many of the Nokia N-Series models, including streams from corporate news servers, such as the
BBC [258]. Thus, I decided that for these scratch events we would focus on encoding for the
web, rather than waiting for the technology to be debugged allowing people to see the streamed
the mixes on the Internet; and
c) based on Michael's advice, it was decided early on that using the Internet for streaming would
be a necessary part of the network equation, even though I had originally wanted all aspects of
the interaction to operate only on the phones, except for the video database, including: all body
data processing, all VJ mixing and all the streaming to and from the phones to the database.
However, with current mobile operating systems and networks this technically is still not quite
possible, or was not when we started and needed it. In addition, even though the custom
software was developed to receive the body data for the phones, due to the nature of the pairing
system of Bluetooth connection, only one connection to sensors could be paired to one system
at a time, not the phone and the computer simultaneously. So this only made multiple
connections impossible and together these issues made it an imperative to rely heavily on
computers and that it was not possible for every aspect to be mobile. Mobile companies may be
developing new technologies in their research labs that will change these limitations, but not in
time for this project, however. So a big early concession was: MINDtouch is not all mobile.
2) the network issues are similar for mobile phones as Internet issues, since phones can use the
Internet connectivity.
A significant discovery was that presence might not be explored in the way envisioned––except in
one-to-one sense (i.e.voice/sms/video/text), since the networks for the internet and mobile systems
are set up differently, therefore delays and severe lags are still experienced on mobiles with video
streaming. Presence in this context may be felt in a video-conferencing/video telephone sense at
one point soon, BUT at present, with the limitations of resources, time and literal barriers of firewalls
and limited access to seamless connections (all mentioned earlier), not to mention the economic
256 This was the script that pulled the videos onto from qik to the MINDtouch server while it streams outward to the Internet from the
257 iPhones have not yet been tested.
258 See screenshots from the forums in the appendices to show that is a common problem using Real Player on Nokia smartphones.
240
and corporate agendas of service providers, tariffs, plans, services etc., government/political
boundaries––the project envisioned is not yet possible without major delays and lags and without
numerous translations and 'interchanges' (Mitchell 2003:60) or daisy-chained workarounds, which
all add to the delay. So liveness, which necessitates a sense of felt presence, are diminished at the
moment, but can continue to be explored in future versions of this project. These are part of the
discoveries from this research.
241
Appendix II/ workshop materials + technical elements
242
A.2.1/ technical timeline for technical development
• November 2006–January 2007– research and decision-making on equipment suppliers
• January–March 2007– acquisition of equipment and beginning to use and work with it
• January–March 2007 – search for qualified programmer to work with
• April–June 2007– Li Zhang begins to work with the sensors and mobile phones to start the
programming
• March–Sept 2007 – troubles getting access to SDK/ API from Mind Media to do the
programming on the phones to receive the bio feedback data
• July 2007 – Li Zhang cannot continue to work on project due to new job opportunities and
relocation
• July 2007 – Dr. Price and I spend some time working with biofeedback sensors and Mind
Media software systems just to get a sensing of the movement and stillness activities and
vocabulary for further Tai Chi explorations, until the phones are ready to sue with the sensors.
• August–September 2007 – Jeremi Sudol works on the project and make great head way
programming the phones, but is delayed due to communication lags between Mind Media
programmers and himself, but finally start to make progress but leaves unfinished due to his
own PHD commitments and deadlines – says there’s two weeks full-time work left before there
is communication between sensors and phones, but the project and hardware is more
sophisticated than he originally though and the end seems always to be further way.
• September 2007- February 2008 – new search for another programmer
• November 2007 – return of one set of the sensors and Nexus 4 transmitter unit – this was done
in order to try another product (not available) and see if we have less trouble with manufacturer
and in the interface programming between phones and sensing hardware
• January 2008 – found possible new programmer /collaborator in Michael Markert from
Germany, who is planning to join the PhD programme in July 2008, but start on the
programming in March
• February 2008 – ordering new sensor system from Infusion Systems
243
A.2.2 / workshop participant consent form
Participant Consent Form
The SMARTlab Digital Media Institute Practice-based PhD Programme, UNIVERSITY OF EAST LONDON
Workshop Consent Form - Confidential data Title: MINDtouch : Biosensors, Liveness and Networked Performance with Mobile Devices - Emerging Perspectives in Embodiment Theory with User Interface Devices Investigator/Workshop Leader Name: Camille Baker, MASc. Interactive Arts I understand that my participation in this project will involve being observed in a participant-centred workshop, conducted by Camille Baker as investigator, who will record the workshop on video, and ask me to fill in a survey afterward, to determine the effectiveness of participatory performance approach, as a part of a larger research project on mobile media performance and the experience on presence on the network as part of the practice-based PhD. I understand that participation in this workshop is entirely voluntary and that I can withdraw from the workshop at any time without giving a reason. I am free to withdraw my participation at any time or to discuss my concerns with the workshop leader or her supervisor. I understand that I am free to ask any questions at any time. I am 19 years of age or older. I understand that at the end of the workshop I will be provided with additional information and feedback about the purpose of the research project, should I request it. I also understand that the activities during this workshop may involve minimal physical and personal risks to me, such as tripping or falling- depending on how active I choose to be in the workshop. I shall undertake to move safely about the space, with this in mind. By involvement in this workshop, I understand that I am giving my informed consent for information I provide therein to be included for my video contribution to be broadcast worldwide, and for the outcomes of the event be published widely, and in future studies.
I understand that the information provided by me will be held confidentially, such that only the Investigator (workshop leader) can trace this information back to me individually. The information will be retained for up to 10 years, when it will be deleted/destroyed.
I understand that I can ask for the information I provide to be deleted/destroyed at any time and, in accordance with the Data Protection Act, I can have access to the information at any time.
I also understand that I may register any complaint with University Of East London, University Research Ethics Committee. I, ___________________________________(NAME) consent to participate in the workshop conducted by Camille Baker, SMARTlab Digital Media Institute PhD Programme, UNIVERSITY OF EAST LONDON, under the supervision of Professor Lizbeth Goodman. Signed: Date: Further information can be obtained from Research Ethics Committee, University Of East London, Docklands Campus, 4-6 University Way, London, UK, E16 2RD, or contact Debbie Dada at +144208 223 2976 or [email protected]. I may obtain copies of the results of this event, upon its completion by contacting:
244
Camille Baker Flat 14, 58 West End Lane, West Hampstead, London England NW6 2NE UK
+44(0) 78 4768 1127 Notes on participation:Your signature on this form will signify that you have received a document which describes the procedures, possible risks, and benefits of this research project, that you have received an adequate opportunity to consider the information in the documents describing the project or experiment, and that you voluntarily agree to participate in the project or experiment.
Any information that is obtained during this study will be kept confidential to the full extent permitted by the law. Knowledge of your identity is not required. You will not be required to write your name on any other identifying information on research materials. Materials will be maintained in a secure location. Risks to the participant, third parties or society: none known. Procedures: The project is intended to gather video clips for use in a larger collective interactive visual collage, using mobile phones and biofeedback sensor during a live performance. We will collect specific information regarding the experience of the participant performer while collecting video impressions. Exit interviews will be conducted with as many participants as is possible or participants asked to take a list of questions and record their own answers on the video phones. Participants will be sent an online survey to answer any outstanding questions as well – if necessary – especially for those unable to do on-camera interview. Participants in the video workshops or performance events will create the video archive/database using the recording feature of the mobile phones. They are invited to participate and to express themselves using of mobile media tools – either using their own video phones or ones supplied by the researchers. Each person using one of our phones will be asked to leave all their identification, contact information and belongings with the researcher for security against theft of the phones Participants may either use one of the camera phones supplied or may use their own equipment. All participants will be given a theme to use to help them to collect video that explores:
the senses (5 external + internal) - attend to them and try express them non-verbally with video words or concepts (without talking) emotions, feelings, worries (non-verbally) intimate thoughts or ideas (no talking) shapes + patterns people, places, environment emotions/expressions thoughts/confessions motion (abstracted?), sounds
Then the participants will be shown how to “phone” or transfer them into the database, initially using www.comvu.com, www.jumpcut.com or www.shozu.com and or our customised database for the project. If this proves too complicated for the participants, all the media will be collected via Bluetooth and/ or USB directly into the researcher’s computer for later upload to the server.
Benefits of study to the development of new knowledge: This media art research involves creating one major project to investigate 'liveness' and 'presence' in mobile performance. This research uses biofeedback sensors in tandem with the moving meditation, movement and play, in order to uncover any new understandings that may have emerged from the use of networked and wireless technologies. Under investigation are concepts such as: telepathic performance, performing presence, live versus simulated/virtual, simulated telepathy in performance. The phenomenological perspective of the experience of creating a mobile, networked performance, as well as the production of a curated mobile, networked performance event of other artists will be my contribution to new media performance studies. The intent is also to study the process, reception, and participation in the experience(s) of “liveness”, “aura”, “presence” during
mobile performance events and activities constructed. The research will also study the performance experience of participant /
performers and the audience reaction, in terms of embodiment of liveness and presence, within the network and staged live.
245
246
A.2.3 / workshop participation and non-exclusive licensing agreement By this agreement, you understand that Camille Baker is producing and presenting a SMARTlab PhD research project, involving mobile media collection, leading to a networked mobile performance involving, for some participants, wearing biofeedback sensors on the body. This research project and workshops have already taken place take place since June 2007 in Vancouver, Canada, Dublin, Ireland, in London, UK, Sao Paulo, Brazil and will be completed in spring 2010. This document sets forth the terms of your agreement to participate in a SMARTlab PhD mobile media research project. This agreement is between Camille Baker and you as a participating Participant (the “Participant”) in a SMARTlab PhD research project. This agreement will set out the terms governing the licensing of your video clips created/recorded for this SMARTlab PhD research project (“Work”) to Camille Baker. 1. Subject to the terms and conditions of this agreement, you the Performer/ Participant hereby grant to Camille Baker a royalty free, non-exclusive, license to exercise the rights to the Work for the full term of the project and thereafter, including any and all extensions, as follows: a. All rights necessary for Camille Baker to produce, publish, advertise and perform live the Work for admission sales, or give-away print and online catalogues and/or books relating to a SMARTlab PhD research project. i. The rights necessarily granted thus include the right to reproduce, publish, distribute, display, transmit, transcribe, record on video, audio, digital or electronic media or otherwise use your Work, in all forms, formats and media whether now known, including without limitation, in print, digital and electronic form, or hereafter developed, throughout the world; ii. Insofar as any such catalogue or book may be deemed a “Collective Work”, because it may be comprised of a number of separate and independent works, you also grant to Camille Baker the right to incorporate the Work into Collective Works and to exercise the same rights in your Work as incorporated in that Collective Work, as granted in the preceding section, although your name can be listed as a participant if you so wish(a)(i). 2. The Participant warrants and represents the following: a. That the Work is original and that the Participant is the sole author and sole owner of the copyright in the Work. If the Work includes materials of others, the Participant has obtained the permission of the owners of the copyright in all such materials to enable them to grant the license herein and shall provide copies of such permissions to Camille Baker; If the Work is previously published, the Participant has all necessary rights, permissions, licenses, and/or other authorizations necessary to allow them to present their Work publicly should it be used in this SMARTlab PhD research project; b. That the presentation or performance of the Work for the SMARTlab PhD research project, and any use of the Work as permitted by the license herein, will not be defamatory, violate any right of privacy or publicity, infringe on any intellectual property rights, or otherwise violate the laws of rights of any third party; c. The Participant signing this document has the full right, power and authority to present the Work publicly as well as for this SMARTlab PhD research project and grants Camille Baker all rights granted by this agreement. 3. The Participant understands and agrees that the inclusion of their Work in the SMARTlab PhD research
247
project is at the sole discretion of Camille Baker and that agreeing to the terms of this agreement in no way guarantees participation. The Participant further understands and agrees that Camille Baker reserves the right, at any time up to and including the scheduled date of their participation in a SMARTlab PhD research project, and in its sole discretion, to cancel the Performer/Participant’s participation in a SMARTlab PhD research project. 4. The Participant agrees to indemnify Camille Baker against any and all losses, liability, judgments, costs, including without limitation, reasonable attorneys fees arising out of any breach of the above representations and warranties. 5. The Participant agrees that to the extent their Work for a SMARTlab PhD research project requires approval of any law enforcement or administrative body, including but not limited to the acquisition of permits for projects taking place in public spaces, that acquiring such approval is solely the responsibility of the Camille Baker. Participant will not be responsible for obtaining such approvals and makes no guarantee as to whether such approvals are or are not necessary. 6. The Participant further agrees to indemnify and hold harmless Camille Baker for any and all liabilities, fines, claims, errors, omissions, losses, accidents, and expenses, including attorney’s fees and court costs in any manner caused by, arising out of, or in connection with, either directly or indirectly, the Performer/Participant’s participation in a SMARTlab PhD research project, the Work, the license granted herein. 8. This agreement will be shall be treated as though executed, delivered and performed within London, England, without regard to conflicts of laws rules, and so governed by the laws of London, England; any disagreement regarding it must be heard by a tribunal in London, England, unless we both agree to a different location. This is our entire agreement and supersedes any and all oral understanding or agreements and may not be modified except by another written document signed by both parties. IN WITNESS WHEREOF, this Agreement has been executed as of the day and year first above written.
By: (Performer/Participant Date (Camille Baker) Date
248
A.2.4/ Participant Profile Information
Introduction: As part of a larger research project conducted in these media collection workshops, you are asked to express your inner perceptions and sensations, non-verbally, through your mobile videophone. This is meant to act as a simulation of telepathic communication, and help you to do so by visually expressing your emotions, thoughts, daydreaming, physical sensations, and perceptions. The project then plans to use these video representations to create a performance installation simulating collective consciousness and creative exchange.
Instructions: Please answer the following questions as honestly and clearly as possible to help us
understand your user profile. Attitude Inventory: How do you perceive your body’s sensations, as well as your thoughts, dreams and other internal experiences and perceptions? How do you feel about expressing yourself and sharing these sensations and experiences with others?
What are your attitudes about participating in a performative exploration through mobile or wearable electronic devices?
How do you feel about sharing these media expressions and performative experiences in a larger performance installation?
What are your expectations about this project and the workshop activities?
Demographic Information:
Age: Gender:
Highest Level of Education: Ethnicity/Nationality:
Academic Major/Stream: Occupation:
Annual Salary: City of Residence:
249
A.2.5/ step-by-step videophone expressions workshop procedure This procedure uses the practice-as-research, and ethnographic methods primarily.
Individual performers/ participants, from untrained and trained backgrounds have an introduction to the research
aims again, and then each is given a consent form and release form to sign.
1. Each participant performer is given consent forms and participant information surveys to complete, which
takes 20-30 minutes, including the explanation of why they are needed.
2. Participants in a group are led through a 10 minute meditation/ guided visualisation/ exploration of senses as
preparation,
3. Then each participants takes a video/ camera phone or use their own, and are given 3-4 separate activities to
record expressive videos, each with a theme or goal to collect certain types of internal perceptions and
sensations. Themes to use to help participants to collect video include exploring:
the senses (5 external + internal) - attend to them and try express them non-verbally with video
words or concepts (without talking) emotions, feelings, worries (non-verbally)
intimate thoughts or ideas (no talking)
shapes + patterns
people
places/environment emotions/expressions
thoughts/confessions
motion (abstracted?)
sounds
4. There are two activities that are done individually and two that are done in pairs, to help facilitate the act of
communication with someone.
5. After each video collection activity, participants are led through another mind-quieting, guided imagery
visualisation.
6. Then the participants are be shown how to “videophone” the clips they have created into the database,
initially using www.comvu.com, www.eyespot.com or www.shozu.com and eventually a customised database
for the project. When this proves too complicated for the participants, all the media is collected via Bluetooth and/ or USB directly into the researcher’s computer for later upload to the server.
7. Participants complete exit questionnaires and they are asked to take a list of exit interview questions and
interview their partners or record their own answers on the videophones (for the most honest, uninhibited and possibly performative answers), or the researcher conduct these video interview.
8. Some participants might be sent an online post-survey, to answer any outstanding questions – if necessary –
especially for those unable to do on-camera interview.
250
A.2.6/ workshop mind quieting script
Find a place in the space; lie down if you like, to be more comfortable
[pause]
Close your eyes and try to relax and tune into your body.
[pause]
Then, take several deep breaths down into ribs and throughout your body, be silent, and settle into
your body.
[pause]
Begin to pay attention to your breath, heartbeat, sensations in your body and the images or colours
behind your eyes.
[pause]
Try to listen to/feel the sensations inside your body, such as: hot or coldness, tingling, twitching, pain,
gurgling in your stomach, the movement of muscles in your organs or limbs, or your blood flow or your
thoughts, emotions, and other perceptions.
[pause]
Now focus on one area of your body that draws your attention, try to stay with that sensation and take
a mental snapshot of it, as if to record it for future reference.
[pause]
Keep focusing your attention on that part of the body.
[pause]
Experience thus inner sensation for as long as you can.
[pause]
Think now how you might you express this sensation non-verbally physically and visually to someone
else?
[pause]
Pay attention to the images or thoughts or emotions you are experiencing and how you might express
them to someone else.
[pause]
How would you transmit these image sensations to them? As a feeling or image? As a thought or some
other inner form?
[pause]
If you could “send” or transmit these sensations, images or emotions to this person, either telepathically
or using the videophones, think again of your snapshot and what you would send them now
[pause]
Think now of who you might want to share them with near or far (non-verbally), doctor, colleague,
friend, family or lover.
[pause]
251
Hold these images and sensations with you now – linger with them.
[pause]
When you are ready, open your eyes and try to take these sensations or images or emotions and
experiences with you and use the camera/video phones now to express them, starting very close to
your own body or space around you on the floor.
Do this for 10 minutes.
252
A.2.7/ post-workshop questionnaire A.2.7/ post workshop questionnaire
253
254
255
A.2.8/ post-workshop interview questions These questions are intended help me, as researcher, understand each participant’s experience and
find out if the activity was successful in initiating a embodied experience
1) Describe what you just experienced?
2) What did your body feel like when you were performing or in the audience?
3) What did you think about during this experience?
4) What sensations did you perceive during this experience?
5) Did you sense or perceive anything of particular of interest or out of the ordinary?
6) Did you think about or feel anything of particular of interest or out of the ordinary?
7) Was there a sense of the other performers or audience during this experience, yes or no – if
yes what did it feel like?
8) What did performing feel like ? What did you sense, perceive or think while performing, and
after?
9) Is there anything else you would like to say abut your experience?
256
A.2.9/ workshop video file content database
Date Workshop Location
Participant Name
Video#/ Name Length in Time
Content Type / Imagery
16-Jun-2007 Vancouver Jin 180620073gp 00:10 eyes, close-up on face, slow motion
16-Jun-2007 Vancouver Jin 180620070013gp 00:10 close-up red, undefined object, slow
16-Jun-2007 Vancouver Jin 180620070023gp 00:09 long-shot, upward, interior structural, gestural waving
16-Jun-2007 Vancouver Jin 180620070033gp 00:09 long-shot, upward, interior structural, gestural waving
16-Jun-2007 Vancouver Jin 180620070043gp 00:07 close-up, red fabric, turning
16-Jun-2007 Vancouver Jin 180620070053gp 00:09 medium-shot, upward ceiling pipes, dark gestural
16-Jun-2007 Vancouver Jin 180620070063gp 00:09 medium-shot, upward ceiling pipes, dark gestural
16-Jun-2007 Vancouver Jim MOV000023gp 00:14 close-up on mouth, minimal movement
16-Jun-2007 Vancouver Jim MOV000043gp 00:22 close-up on mouth smoking, minimal movement
16-Jun-2007 Vancouver Jim MOV000053gp 00:12 extreme close-up of eye, from black
16-Jun-2007 Vancouver Jim MOV000063gp 00:16 extreme close-up of ear, from black
16-Jun-2007 Vancouver Jim MOV000073gp 00:11 close-up of TV with figure making gestures
16-Jun-2007 Vancouver Jim MOV000093gp 00:11 extreme close-up of ear with headphone
16-Jun-2007 Vancouver Jim MOV0000103gp 00:31 extreme close-up of blurry text on book?
16-Jun-2007 Vancouver Jim MOV0000113gp 00:09 side view of himself walking
16-Jun-2007 Vancouver Jim MOV0000123gp 00:09 different views of himself walking
16-Jun-2007 Vancouver Nels 06182007029mp4 00:32 zooming in and out to ceiling light
16-Jun-2007 Vancouver Nels 06182007030mp4 00:31 zooming in and out to door stop
16-Jun-2007 Vancouver Nels 06182007031mp4 00:28 moving up and down from wall to light
16-Jun-2007 Vancouver Nels 06182007032mp4 00:24 moving side to side on wall to firebell
16-Jun-2007 Vancouver Nels 06182007033mp4 00:27 moving up and down from light to interior ceiling structure
16-Jun-2007 Vancouver Nels 06182007034mp4 00:27 scan around the interior building space
16-Jun-2007 Vancouver Nels 06182007035mp4 00:23 extreme close-up of metal wall bump
16-Jun-2007 Vancouver Nels 06182007036mp4 00:30 moving up and down from light to interior ceiling structure
16-Jun-2007 Vancouver Nels 06182007037mp4 00:20 extreme close-up of dark space/ wall
16-Jun-2007 Vancouver Nels 06182007038mp4 00:17 downward view of legs and feet
16-Jun-2007 Vancouver Nels 06182007040mp4 00:13 close-up of arm tattoo
16-Jun-2007 Vancouver Tiina 18062007095 00:09 extreme close-up of bare feet
257
16-Jun-2007 Vancouver Tiina 18062007096 00:01 extreme close-up of bare feet
16-Jun-2007 Vancouver Tiina 18062007097 00:08 extreme close-up of bare feet
16-Jun-2007 Vancouver Tiina 18062007098 00:11 extreme close-up, upside down slow pan along carpeted floor
16-Jun-2007 Vancouver Tiina 18062007100 00:02 extreme close-up, upside down slow pan along carpeted floor
16-Jun-2007 Vancouver Tiina 18062007101 00:07 extreme close-up, upside down slow pan along carpeted floor
16-Jun-2007 Vancouver Tiina 18062007102 00:28 mid-shot, slow pan along wall and ceiling molding
16-Jun-2007 Vancouver Tiina 18062007103 00:13 black + white close-up of metal wall bolts
16-Jun-2007 Vancouver Tiina 18062007107 00:13 blurry close-up of interior structures
16-Jun-2007 Vancouver Tiina 18062007108 00:05 extreme, very dark close-up of wall bolts
16-Jun-2007 Vancouver Tiina 18062007109 00:01 extreme, very dark close-up of finger
16-Jun-2007 Vancouver Tiina 18062007110 00:01 dark close-up of arm on a wooden stair rail
16-Jun-2007 Vancouver Tiina 18062007111 00:13 very dark, black + white shot of floor, hand and legs
16-Jun-2007 Vancouver Tiina 18062007112 00:41 very dark upside down shot of floor and legs
16-Jun-2007 Vancouver Tiina 18062007113 00:41 outdoor pavement close-up
16-Jun-2007 Vancouver Tiina 18062007115 00:02 outdoor pavement mid-range shot
16-Jun-2007 Vancouver Tiina 18062007116 00:08 outdoor downward shot of feet walking
16-Jun-2007 Vancouver Tiina 18062007117 01:12 outdoor upward + zoom shot of trees + sky
16-Jun-2007 Vancouver Rina c-1.mov 00:10 moving camera focussed on grass + a tree
16-Jun-2007 Vancouver Rina c-2.mov 00:47 moving camera focussed on pavement
16-Jun-2007 Vancouver Rina c-3.mov 00:22 moving camera focussed on piping grid
16-Jun-2007 Vancouver Rina c-4.mov 00:41 extreme close-up of a tree
16-Jun-2007 Vancouver Rina c-5.mov 00:20 extreme close-up of participant's hand
16-Jun-2007 Vancouver Rina c-6.mov 00:15 extreme close-up of gravel and leaves
16-Jun-2007 Vancouver Rina c-7.mov 00:27 extreme close-up of concrete object
16-Jun-2007 Vancouver Rina c-8.mov 00:23 extreme close-up of pavement designs
16-Jun-2007 Vancouver Rina c-9.mov 00:11 extreme close-up of blurry ground
16-Jun-2007 Vancouver Rina c-10.mov 00:24 extreme close-up of blurry grass and then pull out to long show of
trees and lawn 16-Jun-2007 Vancouver Rina c-11.mov 00:42 mid-shot of trees and leaves, park
grounds 16-Jun-2007 Vancouver Rina c-12.mov 00:17 zoomed in shot of trellis rooftop
16-Jun-2007 Vancouver Rina c-13.mov 00:22 extreme close-up of blurry doors and surroundings
258
16-Jun-2007 Vancouver Rina c-14.mov 00:32 close-up of pavement, doors and surroundings
16-Jun-2007 Vancouver Rina c-15.mov 01:35 extreme close-up of copper sculpture
16-Jun-2007 Vancouver Rina c-16.mov 00:36 close-up of pavement features
16-Jun-2007 Vancouver Rina c-17.mov 01:07 extreme close-up of pavement and rail line metal
16-Jun-2007 Vancouver Rina c-18.mov 00:29 ground level shot of pavement
22-Jul-2007 Dublin Bobby 22072007149.mp4 00:12 moving, turning, shot of roof
22-Jul-2007 Dublin Bobby 22072007150.mp4 00:41 moving, turning, running shot of wooden floor
22-Jul-2007 Dublin Bobby 22072007151.mp4 00:24 slow moving shot of corduroy trousers
22-Jul-2007 Dublin Bobby 22072007152.mp4 00:20 moving, turning, twirling shot of room, upside down at times
22-Jul-2007 Dublin Bobby 22072007153.mp4 01:34 extreme close-up of a slow pan of metallic furnace or heating vents,
wall moldings and features
22-Jul-2007 Dublin Bobby 22072007154.mp4 01:18 close-ups and extreme close up of a woman and her digital
camera image, moving around the space
22-Jul-2007 Dublin Bobby 22072007155.mp4 00:11 running shot around the room
22-Jul-2007 Dublin Bobby 22072007156.mp4 00:11 shots of a woman jumping with her digital camera
22-Jul-2007 Dublin Bobby 22072007157.mp4 00:48 turning of phone of woman and her camera
22-Jul-2007 Dublin Bobby 22072007158.mp4 00:52 upside down turning, close-up shot of woman
22-Jul-2007 Dublin Bobby 22072007159.mp4 00:54 extreme close-up of a dead fly on a ledge
22-Jul-2007 Dublin Bobby 22072007160.mp4 00:18 sideways shot of woman with her camera
22-Jul-2007 Dublin Bobby 22072007161.mp4 00:37 sideways follow shot of a pigeon
22-Jul-2007 Dublin Jo Lee MVI_0139.AVI 00:24 zooming in and out to roof /ceiling structures
22-Jul-2007 Dublin Jo Lee MVI_0140.AVI 00:22 zooming + panning in + out of darkened roof skylight
22-Jul-2007 Dublin Jo Lee MVI_0141.AVI 00:16 erratic shaking, zoomed in, darkened roof skylight
22-Jul-2007 Dublin Jo Lee MVI_0142.AVI 00:19 slow swinging + panning roof skylight + ceiling structures
22-Jul-2007 Dublin Jo Lee MVI_0143.AVI 00:23 extreme close-up of white metal wall
22-Jul-2007 Dublin Jo Lee MVI_0144.AVI 00:33 close-up of white concrete wall and moldings
22-Jul-2007 Dublin Jo Lee MVI_0145.AVI 00:20 extreme close-up of white electrical wall plug
22-Jul-2007 Dublin Jo Lee MVI_0147.AVI 00:15 side turning close-up of white water heater
259
22-Jul-2007 Dublin Jo Lee MVI_0148.AVI 00:25 close-up of hand crawling on a white concrete wall
22-Jul-2007 Dublin Jo Lee MVI_0149.AVI 00:17 close-up of port hole window and extreme close up of light
22-Jul-2007 Dublin Jo Lee MVI_0150.AVI 00:15 extreme close-up of cabling
22-Jul-2007 Dublin Jo Lee MVI_0151.AVI 00:17 shot of port hole window & woman through it
22-Jul-2007 Dublin Jo Lee MVI_0152.AVI 00:30 shot of opening and closing elevator doors
22-Jul-2007 Dublin John Video070.mp4 00:14 extreme close-up of black dot on white surface
22-Jul-2007 Dublin John Video071.mp4 00:20 close-up of wood floor
22-Jul-2007 Dublin John Video072.mp4 00:22 close-up of opening and closing of hand
22-Jul-2007 Dublin John Video073.mp4 00:26 slow pan from wood floor to close up of legs
22-Jul-2007 Dublin John Video074.mp4 00:13 side-ways shot of woman and others in the room
22-Jul-2007 Dublin John Video075.mp4 00:28 close up of wood floor
22-Jul-2007 Dublin John Video076.mp4 00:34 long shot of sky to quick pan to close up of wall
22-Jul-2007 Dublin John Video077.mp4 00:36 close up of legs to pan of room back to legs
22-Jul-2007 Dublin John Video078.mp4 00:31 blurry white/ grey indefinable image
22-Jul-2007 Dublin John Video079.mp4 00:28 blurry indefinable blue/grey image
22-Jul-2007 Dublin John Video080.mp4 00:34 extreme close-up of fabric/ jeans
22-Jul-2007 Dublin John Video081.mp4 00:05 close-up of face
22-Jul-2007 Dublin John Video082.mp4 00:25 close-up pan of clothing and body
22-Jul-2007 Dublin John Video083.mp4 00:05 mid-range shot of wood floor
22-Jul-2007 Dublin John Video084.mp4 00:27 sideways shot of sofa chair pan to window, zoomed in outdoors
22-Jul-2007 Dublin John Video085.mp4 00:22 zoomed in shot of pigeon, pan to trees, back to pigeon
22-Jul-2007 Dublin John Video086.mp4 00:24 outdoor shot of grounds
22-Jul-2007 Dublin John Video087.mp4 00:10 zoomed in shot of pigeon
22-Jul-2007 Dublin John Video088.mp4 00:22 extreme close up of floor, pan around the room, of woman taking
photos 22-Jul-2007 Dublin John Video089.mp4 00:12 close-up, zooming in and out,
panning of porthole window
22-Jul-2007 Dublin John Video090.mp4 00:10 downward shot of piano pedals and feet
22-Jul-2007 Dublin John Video091.mp4 00:03 extreme close up of indefinable blue transparent object
22-Jul-2007 Dublin John Video092.mp4 01:09 close up pan around a bicycle
22-Jul-2007 Dublin John Video093.mp4 00:05 close up pan around a bicycle
22-Jul-2007 Dublin John Video095.mp4 00:25 pan around a parking lot
22-Jul-2007 Dublin John Video096.mp4 00:34 side shot of yard, tree and crypt-like object
22-Jul-2007 Dublin John Video097.mp4 00:24 close up of mossy ground
260
22-Jul-2007 Dublin John Video098.mp4 00:12 close up of woman's slow walking feet along a car
22-Jul-2007 Dublin John Video099.mp4 00:10 close-up of porthole window slowly turning
22-Jul-2007 Dublin John Video100.mp4 00:07 shaking shot of fast dancing feet
22-Jul-2007 Dublin John Video101.mp4 00:27 roving shot of metal bucket, old wood floor and antique sewing
machine 22-Jul-2007 Dublin John Video102.mp4 00:07 mid-shot of flowery trees
22-Jul-2007 Dublin Lizbeth SSPX0019.mov 00:08 opening with upside-down face, then moving around room
22-Jul-2007 Dublin Lizbeth SSPX0020.3g2 00:14 close-up moving along the body and wood floor
22-Jul-2007 Dublin Lizbeth SSPX0021.3g2 00:30 phone waving vigorously around the room
22-Jul-2007 Dublin Lizbeth SSPX0022.3g2 00:25 phone waving slowly around the room and along the body
22-Jul-2007 Dublin Lizbeth SSPX0023.3g2 00:09 phone waving slowly around the room
22-Jul-2007 Dublin Lizbeth SSPX0024.3g2 00:14 phone waving slowly around the room upside-down
22-Jul-2007 Dublin Lizbeth SSPX0025.3g2 00:30 shot moving from close-up of face to long shot of outdoor view
22-Jul-2007 Dublin Lizbeth SSPX0026.3g2 00:30 zooming into close-up of friend in wheelchair
22-Jul-2007 Dublin Lizbeth SSPX0027.3g2 00:30 shot moving from close-up of face to view around the room
22-Jul-2007 Dublin Lizbeth SSPX0028.3g2 00:30 shot moving from close-up of foot and face to view outside
22-Jul-2007 Dublin Lizbeth SSPX0029.3g2 00:30 shot moving from close-up of leg and face to view outside and
room 22-Jul-2007 Dublin Lizbeth SSPX0030.3g2 00:30 shot moving from close-up of leg
and body to view of the room in background
22-Jul-2007 Dublin Lizbeth SSPX0031.3g2 00:30 shot of reflection on the wood floor and leg waving through
22-Jul-2007 Dublin Lizbeth SSPX0032.3g2 00:30 shot of friend in wheelchair and around the room
22-Jul-2007 Dublin Lizbeth SSPX0033.3g2 00:30 shot of reflection on the wood floor
22-Jul-2007 Dublin Lizbeth SSPX0034.3g2 00:30 shot of another person shooting, scan to person in wheelchair
22-Jul-2007 Dublin Lizbeth SSPX0035.3g2 00:30 shot of person in wheelchair and his computer screen
22-Jul-2007 Dublin Lizbeth SSPX0036.3g2 00:30 close-up of person in wheelchair and back to her own face and
body 22-Jul-2007 Dublin Lizbeth SSPX0037.3g2 00:30 close-up of another phone video,
the person in wheelchair and extreme close-up of hand
22-Jul-2007 Dublin Lizbeth SSPX0038.3g2 00:30 mid-shot the person in wheelchair
22-Jul-2007 Dublin Lizbeth SSPX0039.3g2 00:30 close-up of the person in wheelchair and his computer
261
22-Jul-2007 Dublin Lizbeth SSPX0040.3g2 00:30 close-up of the computer and the person in his wheelchair
22-Jul-2007 Dublin Lizbeth SSPX0041.3g2 00:30 close-up of the person in wheelchair, close-ups of face,
hands eyes 22-Jul-2007 Dublin James/
Chris ChrisjamesMINDto
uch.mov 00:24 mid shot of the person in
wheelchair and others around him
22-Jul-2007 Dublin James/ Chris
james_video.mov 00:07 long shot of outdoors, garden, lawn
22-Jul-2007 Dublin James/ Chris
Jamescloseup2.mov
00:24 zooming into close-up of friend in wheelchair
22-Jul-2007 Dublin James/ Chris
Jamesintro.mov 00:16 zooming into close-up of friend in wheelchair and extreme close up
of hand 22-Jul-2007 Dublin James/
Chris Jameslizbethtrees.
mov 00:24 mid shot of the person in
wheelchair woman with him
22-Jul-2007 Dublin James/ Chris
JamesMINDtouch3.mov
00:24 shot of woman recording in front and moving toward screen
22-Jul-2007 Dublin James/ Chris
JamesMINDtouch4.mov
00:24 mid shot of the person in wheelchair and woman with him
22-Jul-2007 Dublin James/ Chris
Jamesoutsidetrinity.mov
00:24 mid shot of outdoors, garden, hedge
17-Aug-2007 Vancouver Amalia Video109.mp4 01:06 zoom out of shadow on the grass
17-Aug-2007 Vancouver Amalia Video110.mov 00:04 zoom in of a branch
17-Aug-2007 Vancouver Amalia Video111.mov 00:11 mid shot of shadow on pavement
17-Aug-2007 Vancouver Amalia Video112.mov 00:08 shot of people on a blanket and mid shot of legs
17-Aug-2007 Vancouver Amalia Video113.mov 00:31 slow zoom out from extreme close-up of glasses and eye to
face and face 17-Aug-2007 Vancouver Amalia Video114.mov 00:18 long shot of shadows on the grass
17-Aug-2007 Vancouver Amalia Video115.mov 02:08 extreme close up shot of blanket and grass
17-Aug-2007 Vancouver Amalia Video116.mov 01:13 extreme close up shot of girl's face/head and activity, zoom out
to a woman's face/head 17-Aug-2007 Vancouver Lori SSPX0000.3g2 00:30 close up of grass, flower and hand
17-Aug-2007 Vancouver Lori SSPX0001.3g2 00:30 close up of dandelion fluff
17-Aug-2007 Vancouver Lori SSPX0002.3g2 00:30 long shot of shadow figure and tree on grass
17-Aug-2007 Vancouver Lori SSPX0003.3g2 00:30 long shot of shadow figures on grass
17-Aug-2007 Vancouver Lori SSPX0004.3g2 00:30 long shot of shadow figures dancing on grass and sideways
17-Aug-2007 Vancouver Lori SSPX0005.3g2 00:30 mid shot of top of woman's head panning around
26-Oct-2007 Dublin Blair 20071026013.mp4 00:21 close up of a circular window or object
262
26-Oct-2007 Dublin Blair 20071026014.mov 01:40 downward shot of stairwell in a circular pan
26-Oct-2007 Dublin Blair 20071026015.mov 00:25 moving close up shot of wall moldings to extreme close up of
boot 26-Oct-2007 Dublin Blair 20071026016.mp4 00:25 moving extreme close up shot of
boot, another person, the room and a wheelchair
26-Oct-2007 Dublin Blair 20071026017.mp4 05:50 moving shots of outdoors and other women in the leaves, in
parking lot 26-Oct-2007 Dublin Blair 20071026018.mov 00:58 close ups of objects and words
around the space, ending on piano keys
26-Oct-2007 Dublin Dermot Video118.mp4 00:06 zoomed in close up of finger tip
26-Oct-2007 Dublin Dermot Video119.mp4 00:26 moving shot of wooden floor
26-Oct-2007 Dublin Dermot Video120.mp4 13:53 moving shot of wooden floor, people in the room, close ups of
jeans and hand 26-Oct-2007 Dublin Dermot Video121.mp4 00:12 moving shot of wooden floor, a
chair and a hand
26-Oct-2007 Dublin Dermot Video122.mp4 00:03 moving shot of wooden floor and hand
26-Oct-2007 Dublin Dermot Video123.mp4 00:09 moving shot of wooden floor, around the room and the roof
window 26-Oct-2007 Dublin Dermot Video124.mp4 00:41 moving shot around the room and
the roof window
26-Oct-2007 Dublin Dermot Video125.mp4 00:26 pan of outdoor parking space /alley
26-Oct-2007 Dublin Dermot Video126.mp4 00:29 sideways shot of tree, straighten to lawn
26-Oct-2007 Dublin Dermot Video127.mp4 00:58 downward moving shot of legs walking and reflection on
pavement 26-Oct-2007 Dublin Dermot Video128.mp4 00:18 sideways moving shot of building
26-Oct-2007 Dublin Participant F
27102007210.mp4 00:10 extreme close-up of fingers
26-Oct-2007 Dublin Participant F
27102007211.mp4 00:26 extreme close-up of finger tips
26-Oct-2007 Dublin Participant F
27102007212.mp4 00:35 extreme inverted colour close-up of fingers and hand
26-Oct-2007 Dublin Participant F
27102007213.mp4 00:30 extreme close-up of unknown object
26-Oct-2007 Dublin Participant F
27102007214.mp4 00:25 extreme inverted colour close-up of unknown object
26-Oct-2007 Dublin Participant F
27102007215.mp4 00:12 blurry view through a window
26-Oct-2007 Dublin Participant F
27102007219.mp4 00:10 extreme close-up of lips, mouth and cheek
26-Oct-2007 Dublin Participant F
27102007220.mp4 00:23 extreme close-up of white/grey wall
263
26-Oct-2007 Dublin Participant F
27102007221.mp4 00:15 extreme inverted colour close-up of lips, mouth and cheek
26-Oct-2007 Dublin Participant F
27102007222.mp4 00:25 extreme sepia tone close-up of hands
26-Oct-2007 Dublin Participant F
27102007223.mp4 00:11 mid-long shot through a window of sky and building
26-Oct-2007 Dublin Participant F
27102007224.mp4 00:12 long shot through a window of sky and cars below
26-Oct-2007 Dublin Participant F
27102007225.mp4 00:15 mid shot of reflection on wood floor
26-Oct-2007 Dublin Participant F
27102007226.mp4 00:02 close up of head shot outside with sun reflecting
26-Oct-2007 Dublin Participant F
27102007227.mp4 00:25 moving close up of pavement
26-Oct-2007 Dublin Marcus Experiment1.mov 01:46 zooming in and out of radiator coils
26-Oct-2007 Dublin Marcus Experiment1b.mov 00:56 through window shot of sky/ clouds and outside buildings
26-Oct-2007 Dublin Marcus Experiment1c.3gp 01:01 close up of wall writing; zooming in and out of shoes
26-Oct-2007 Dublin Marcus Experiment2.3gp 00:44 zooming in and out of porthole window
26-Oct-2007 Dublin Marcus Experiment3.3gp 00:44 close-up of hand playing piano
26-Oct-2007 Dublin Marcus Experiment4.3gp 01:10 interior close-up on wall and long shot of room
26-Oct-2007 Dublin Marcus Experiment5.3gp 01:38 moving shot of kitchen
26-Jan-2008 London Babak 20080128022.mp4 01:32 mix of images: ceiling, lights, computer screen, moving dot
reflection, trees outside, close up of patterned fabric and
screensaver 26-Jan-2008 London Babak 20080128023.mov 00:18 close ups of heads and hair
26-Jan-2008 London Babak 20080128024.mp4 00:20 moving shot around the room
26-Jan-2008 London Babak 20080128025.mp4 00:51 shot of turning on water, sink, running water
26-Jan-2008 London Babak 20080128026.mp4 00:39 shot of others videoing each other, close-ups of faces and
cameras 26-Jan-2008 London Babak 20080128026.mp4 00:22 moving shot of floor, legs, and
hand 26-Jan-2008 London Vesna Video135.mp4 00:08 moving shot legs
26-Jan-2008 London Vesna Video136.mp4 00:21 close up on fingers and hand
26-Jan-2008 London Vesna Video137.mp4 00:17 downward view of moving legs
26-Jan-2008 London Vesna Video138.mp4 00:12 close up of teeth and mouths
26-Jan-2008 London Vesna Video140.mp4 00:49 extreme close-up of eyes
26-Jan-2008 London Vesna Video141.mp4 00:16 extreme close-up of words on a paper
264
26-Jan-2008 London Zoe 29012008239.mp4 00:15 medium shot of yellow and black line on floor
26-Jan-2008 London Zoe 29012008240.mp4 00:13 medium shot of feet moving around on floor
26-Jan-2008 London Zoe 29012008241.mp4 00:22 medium shot of feet moving very little
26-Jan-2008 London Zoe 29012008242.mp4 00:07 medium shot of feet and the room
26-Jan-2008 London Zoe 29012008243.mp4 01:30 medium shot of water filling a container from tap and moving the
container and hands playing around and in the water, making
patterns on the counter 26-Jan-2008 London Zoe 29012008244.mp4 00:36 close up of fingers - to make it
seem like they are walking on the ground and around the room
26-Jan-2008 London Zoe 29012008245.mp4 00:54 medium shot of hands moving and animating against the floor as
background 26-Jan-2008 London Zoe 29012008246.mp4 00:44 follow shot of someone walking
down the hall from neck down, then finger obscuring
20-Mar-2009 London Alec ar-Smart-3.mov 00:39 medium shot of torso, making
various gestures, then turning upside down, plus hand gestures
20-Mar-2009 London Ivan Video000.3gp 00:01 shot of floor
20-Mar-2009 London Ivan Video001.3gp 00:08 medium shot to zoom in of reflection recording
20-Mar-2009 London Ivan Video002.3gp 00:08 zoom out to medium shot of reflection recording
20-Mar-2009 London Ivan Video003.mov 00:07 blurred moving shot of floor pattern
20-Mar-2009 London Ivan Video004.3gp 00:08 moving upward angled headshot with background changing
20-Mar-2009 London Ivan Video005.3gp 00:08 moving downward headshot with background changing
20-Mar-2009 London Ivan Video006.3gp 00:08 moving sideways headshot surroundings changing
20-Mar-2009 London Ivan Video007.3gp 00:08 moving mid shot of woman with coke
20-Mar-2009 London Ivan Video008.3gp 00:08 long shot of canal water and plane taking off
20-Mar-2009 London Ivan Video009.3gp 00:08 zoom in shot of woman looking at the view
20-Mar-2009 London Ivan Video010.3gp 00:08 moving shot of pointed finger and woman's feet
20-Mar-2009 London Ivan Video011.3gp 00:08 moving camera through a door to encounter a woman video
recording 20-Mar-2009 London Ivan Video012.3gp 00:08 long shot of hallway with a woman
video recording
265
20-Mar-2009 London Ivan Video013.3gp 00:08 side shot of hallway with a woman in the foreground video recording
20-Mar-2009 London Ivan Video014.3gp 00:08 shot of wall zooming into a headshot of a woman smiling
20-Mar-2009 London Ivan Video015.3gp 00:08 zoom in a circular graphic object
20-Mar-2009 London Ivan Video016.3gp 00:08 zoom out and fast moving shot of circular graphic object
20-Mar-2009 London JeeHee 20090320035.mp4 00:07 shot of feet and carpet
266
A.2.10/ technical updates and software guides A.2.10a/ Arduino and mobile software guides After a meeting end of February 2008, the current project status is as followed:
The following restrictions apply for J2ME applications: - no video streaming via Bluetooth (but with WiFi) -no pixel-manipulations on videos, due to missing codec implementation(but on photos), see http://java.sun.com/javame/reference/apis/jsr135/
-access from and to video-database only via WiFi - no multiple Bluetooth connections from (multiple) sensor-boards to one phone -jar filesize-restriction for application-embedded videos (but videos stored on the mobile or just recorded) -normally no video-out on any phone / handheld (-> „audience“?) - no Video-Mixing (Overlays, Blends) possible with phones; neither J2ME apps (incl. those on the available Nokia devices) - even the iPhone has no support for live-video-mixing (in terms of smooth blending) and/or Quartz Patches andthis is a very suitable, brand-new mobile device with a noticeable processing power!
Mobile Phone Visualisation invoking a Video Database:
Conclusion: The processing powers of mobile handheld devices are simply too low to enable pixel-based video manipulations.However, 2D Animations are possible (Pixel / Flash Lite / Text / 3D), as well as simple consecutive video playback and/orvideo with graphic overlays. For more extreme manipulations and smooth transitions, a solution invoking a desktopcomputer (I definitely recommend a Macintosh with Quartz Composer) is required.I can imagine these possible solutions:
1) Video Streaming via WiFi & browser, I recommend data processing with Quartz Composer (& Video DB Access) and streaming QT Web-Server: SENSOR <-Bluetooth -> Mac / Quartz Composer -> Network -> QT-Server -> WiFi/GPSR/UMTS/EDGE -> Phones the sensor device sends it‘s values by bluetooth to a computer which takes these values and uses them to mix visualsand videos from the database, then streams the results to a Quicktime Streaming Server that feeds this stream onto theweb, where it can be streamed to WiFi-enabled mobile devices.
2) Videos are stored on the phone / inside the .jar and accessed by the J2ME application: SENSOR <-Bluetooth -> Phone If the device is able to access the web, more videos could be downloaded from a database and used randomly in abackground process. However, I think the application should produce results even when it‘s not connected to the web. I‘d strongly recommend adapting the premises to the available technology, the other way is full of obstacles!
3) Videos are recorded live by the performer (eg. 4 small clips), then live-rendered and triggered by biofeedback. This could constantly produce new visual output. The visuals can be merged by in-between-graphic animations, 2D-graphic overlays etc… If the performer uses a WiFi enabled device, a background upload into a database should be possible.
267
-What body function is controlling what visual reaction? Body states such as pulse, breath and skin resistance aren‘t actively controllable events and their change is usually slow and almost impalpably.
-When there‘s an „audience“ mentioned: what does the audience do: does it watch streamed videos? does it actively participate, send videos? mobile phones don‘t have no Video-Out, so you cannot beam the screen.
Things to remember:
A mobile phone (even PDAs or gaming devices like the Nokia N-Gage or the new iPhone) has low processing power:In contrast to 32 or 64 bit desktop computers with averagely 1 GB RAM, weʻre talking of 8 to 16 bit microcontrollers with 16 kB to 8 MB RAM A SPP (serial connection) via Bluetooth is only possible between two „paired“ devices!
Web connections arenʻt realtime anymore, a delay has to be calculated. more downsides to a web-based application is slow timing and expensive costs for GPSR devices that have no built-in WiFi module. The gain of J2ME-universal access is lost by the restriction of WiFi.
Michael [email protected]
268
After working through the code-sources, getting information on the Nexus-System and trying to compile the currentproject J2ME compatible, these are my current conclusions:
Mobile Phone Application: -Code currently uses Java5 syntax (e.g. use of generics, which makes sense) - Code is based on super-new MIDP2.1, which is extremely seldom (e.g. Android -> no hardware yet!) -Java5 syntax requires additional tools like J2ME Polish (for example) -J2ME Polish cannot (or not easily) work with MIDP2.1, just MIDP2.0 -The targeted Nokia N80 (and others) uses Symbian OS S60, which provides only support for MIDP2.0 -S60 supports C and C++ programs (access to security related hardware features requires „Symbian Signed“authorisation ($$)
Mobile Phone Visualisation: Video Streaming via Bluetooth is far over my programming powers Video Streaming via WiFi & browser easily accomplishable Mobile Phones have low processing power, video playback wonʻt be realtime and has to be low-resolution and/or efficiency-optimised 2D Animations possible (Pixel / Flash Lite / maybe 3D)
Nexus Sensor Device: Complicated signal decoding, poor API documentation due to proprietary code protection (itʻs a security and trust-demanding medical device that needs to make sure no lives can be endangered directly or indirectly). Proprietary sensory device that can be paired with only one Bluetooth receiver at a time. The huge advantage of working with mobile phones is that they are available in masses without any cost (everyone has one). By coupling oneexpensive & maybe-not-even-yet-available phone to the Nexus and communicating with difficult encoded messages,the mobile application has to be tightly bundled to this and only this Nexus model. Besides the risk of total dependenceon the manufacturer to continue sales and support for this product, I donʻt see any technical reason why it has to be arestricted J2ME phone and not an open device running Linux Embedded (like Openmokoʻs Neo1973).
Conclusion: I agree 100% with Jeremi‘s final comment about going for an alternative approach with a custom sensor board.My proposals:
1. Sensors -> Arduino BT -> Mobile Device (with Processing J2ME) See arduino.cc and libelium.com for further information
Arduino BT Specs: 8 AIN (USB: 6 AIN), DIN/DOUT: 14 (PWM:6) It‘ s cheap: 1 Arduino BT assembled costs 100,- €, multiple devices possible!It‘s easy: no complex protocol decoding, standard protocols, open libraries available => new features instead of re-inventing the wheel!It‘s open: and therefore potentially popular => {Arduino+MyPhone=100$} OR {Nexus+HiTechPhone=$5000}? Arduino is commonly used: plenty of information and code available Processing seems very straightforward, huge community as well, BT lib available The application will run on many phones => user/audience feedback can be implemented
2. Sensors -> ACSensorizer -> MIDI -> Mac -> QuartzComposer -> Web-Broadcasting incl. possible Bluetooth Feedback from local devices sending in custom media and/or dataSee audiocommander.de and midibox.org for further information ACSensorizer Specs: 8 AIN (expandable to 32), DIN:64, DOUT:64 (no PWM)
It‘ s even cheaper: 1 ACSensorizer unassembled costs between 25,- to 250,- € (with or without HUI)It‘s easy: no host software needed, direct signal feed into QC (intuitive realtime VJ application)It‘ s open, DIY, non-commercial (due to MIOS licence) Reliable: I spent 2+ years developing. Signal enhancement & fine control avbl. at the lowest processing levelExpandable with hardware modules and software (custom QC Plug-ins) By using the World Wide Web many-to-many and local-to-distance relations could widen the project scope
Michael [email protected]
269
This is a bit of a summary of where we are with the MindMedia project (alreadyspoke with Camille about this). Essentially, I kind of felt like Alice inwonderland at some point, because the deeper I needed to dig to get this towork, the more opened up that needed to be implemented due to the complexityof their underlying design.
What we have at the moment is an understanding of the communications protocolenough to communicate with the device, get status and device information,initiate and stop the streaming from all the sensors attached. While that'sgood it's not good enough, because the raw data is encoded using a variablelength encoding scheme of their proprietary design (more complex data gets tohave more bits, very smart and efficient design). But to get that requires alittle bit more work -- I have outlined the next steps in the fullassessment/readme file (which is the one attached). In my estimate it wouldtake about two weeks full time work to get that sorted. If there's someoneelse that will pick this up after me, I would gladly answer any questions tobring them up to speed. The overall assessment is that it's a) definitelyworkable, and b) 85% there. The reason for it taking so long, is that it'sreally a high end medical device and they had to from the getgo use a complexdesign to handle a very large amount of data streams over very limitedchannels (bluetooth is fairly low bandwidth and their new devices using thesame protocol handle more than 10 channels at insane sample rates--it's prettyhot). And since we're doing this custom for portable devices we can't usetheir DLL -- which takes care of these details for us.
Unfortunately with the amount of work I have on my hands at the moment I can'tcontinue working on this -- even though I'd really like to see this through tothe end -- especially since it's so close. So instead of holding up theprocess the sensors went back with Jacki on Wednesday. The box also containsthe N80 cell phone, charger, the BioTrace CD, and the CD with all associatedresources and source codes and documentation (a copy is also attached in thezip in this email).
I'm around to answer any and all questions, technical or otherwise.
As always -- all the best greetings,
___jeremi
270
This is an asynchronous call and response system communicating over the serial port over a bluetooth connection. The bluetooth library takes care of all the connection details (as long as the particular device is present). I included the code (although it's not being used) for searching for other BT devices.
First all general information about the device (Nexus, also referred to in their documentation as the Front End) needs to be received by the PC (or the cellphone). These are the getFrontEndInfo, getVLDataInfo, and getIDData.
The frontEnd information is the general information about the device. the VLdatainfo contains information that will be helpful in decoding the variable length data-- the raw sensor readings(sampling rates,and accuracy) IDDataInfo -- contains the critical information for each channel -- it's type, units, the gain value,
For debugging purposes I have written a set of functions for easy conversion between string hex representation and actual hex values -- so it's easy to just send a message like "AAAA 2202 0010 0008" + checksum.
The API indicates that some of the messages are unidirectional and some are bidirectional. This is how to communicate with the system. take the bidirectional messages, change the bits you want, and send them back --the return should be an acknowdledgment of that command. i din't get put in a robust system for checking that but you can see the simple examples in getting front end info for instance. also the PortiSerial code has a great suggestion on how to implement a generic ack system.
Some of the datastructures seem weird -- and they are sometimes. trial and error and a lot of patience helps to pick this protocol apart. At this moment I am in the middle of parsing the IDData. important note, this is not yet implemented but needs to be done immediately -- iddata is too long to potentially fit in manageable messages. so it contains a mechanism where you specify in the request the starting address of the full structure and a number of words, and then the will recieve some of those pieces. you then need to assemble all the received messages and when the datastructure is complete, then parse it for all the critical data --that data is KEY to decoding the sensor readings. unfortunatley.
see the extracted code in the cpp folder -- that is code from the portiserial project extracted to only deal with the decoding part -- but since i was missing the IDData information the decoding does not work.
A lot of the functions in the API are not even implemented in the hardware here --it's not a problem, they're not really necessary.
Take a look at the API (comm.pdf) and search for the related bytecodes in the source -- or search for the functionnames -- i tried to keep those nearly the same.
Once the connection is established two threads are spawned, a reader and a writer that will handle all the low level communications. Java uses network byte order, while C++ the original design language here doesn't so at the lowest level all the bytes have to be reversed (see this in both the reader and the writer) -- this is critical, but after that conversion all byte operations are almost the same (special cases due to unsigned vs signed conflicts of java vs c++).
ACNexus 2008 Michael Markert http://www.audiocommander.de http://www.smartlab.uk.com
271
3. CODE STRUCTURE
The code is aimed to maximize parallelism between doing development on the pc and having code that is simulatanously functional on a cellphone. That's why the code is shared and there are only two different wrapper classes one for each platform --Bluefin for the PC and TunaApp for the cellphone (although for debugging purposes I started adding a lot of System.out.println statements which don't work on the cellphone, so for real depolyment they will have to be removed, and some of that structure would have to be cleaned up).
So the main app is Bluefin / or TunaApp Then the main logical control over the device is the SensorManager, it deals with high level operations like initialize, start/stop dataflow, and disconnect.In turn it contains a BTClient which is responsible for the actual transactions, manages the sender and receiver threads. BTClient extends the Nexus class, which contains all the nexus specific information, like message parsing, the necessary variables about the state of the system. This is where a lot of the nitty gritty happens.
util.Hex provides the helpful functionality in conversions and managing bit operations. There's been a lot of experimentation in there and so multiple approaches to the same issue may be visible.
Temp class --that what i use for quick prototyping and development and testing of new functions. it used to be in an outsider package, but due to scope issues i brought it in to air.
VERSIONING I used darcs versioning while developing this system. so if you want to go back and see the incremental stages of development it's all there. darcs allows you to hold the repository locally --yey. (info and tutorials at darcs.net) -- note there may be a few stages before i added a lot of the "PortiSerial" c++ code ported over to java in the attempts to decode the variable length data. prior to those massive functions it was way cleaner and more manageable. it would be great if that code could be streamlined before adding it to the project -- but that's why the side version of the cpp experiments.
MIDLET development to deploy the code (a jar ) onto the phone you will need the Sun JAva Wireless toolkit 2.5.1+ Sun Java Wireless Toolkit 2.5.1 (download free, requires Java 5.0+)
the projects sit in C:\WTK2.5.1\apps (or where the WTK is installed)
required project settings for the wtk
Target platform JTWI Configuration CLDC 1.1 Mobile media JSR 135 Bluetooth OBEX JSR 82
when making a wtk project make sure you specify the package name for the class that will be the main midlet (ie. air.TunaApp )
ECLIPSE I have been using eclipse to develop this project. it's been really great. i highly recommened it. eclipse.org the .project and the .classpath files are for eclipse
CURRENT STATUS if you run the Bluefin application right now it should connect to the nexus, get the frontend info and the vldatainfo, and for 5 secods pull data from all available channels and then terminate. you can save that output and use those logs for quicker development than directly with the device everytime.
4. OTHER THINGS
Curious and randomg things I've come across during this project. -- sometimes when you're getting incorrect or weird responses from the nexus the batteries may be dead -- check those once in a while, as the LED light does not indicate low power.
-- you can assign your own com ports to suit you better if you need in
windows (and even if the system says they're in use, you can just override it and use them anyways)
-- use the default microsoft stack driver for the bluetooth. it's more reliable
-- take a look at proce55ing for mobiles at processing.org -- good for easy nice and quick visualization
-- java does not have unsigned ints or shorts. just remember that when you're doing byte conversions and something's not right
--in iDData parser i just started adding stuff, and there are casts to floats -- those are definitely wrong. just wanted to get something going -- there is a mindMedia forum on yahoo. it's mostly useful to the general users though, not so much low level developers. but potentially worth a try. i made us an account: user: mindmedia password: smartlab -- the chief engineer Pieter Ermers Email: [email protected] Phone: +31 (0)475 41 01 23 Fax: +31 (0)475 33 06 02
www.mindmedia.nl
--next in the developing sequence for me would be: -- finish requesting iDDataInfo --this means multi message assembly with multiple requests, starting at various addresses until the whole large structure is received
--parse that big structure -- use the information from iDDataInfo to seed the demo c++ VLData parser in the /cpp folder. get that data stream decoded properly
-- try to streamline that code (their spaghetti mess is really less than appealing and very bloated)
-- bring it in to the java project
-- decode the variable length data. -- make sure the correct units also go right with the correct sensors
--- this would be a great place to be at. -- then make a simple visualization -- create simple simulator with data (this could be done now already from
data from bioTrace ) for visualization
5. OTHER THINGS
This is an awesome device --very capable. It's just maybe a bit complex for very quick prototying innitially, especially when we have to redevelop the low level communications protocol (the generic windows developers get the DLL and don't need to worry about any of that) --so for that kind of development this would be easier. However with a bit more patience this will be done. For some encouragement just hook yourself up to the sensors and see your body vital signs on the screen. pretty amazing -now on the cell phone.
The EEG sensors would really make this the bom-diggity.
multi threading is key, and keeping the jar file size down. also important for good runtimes on the cellphone -- less division and costly operations. more bitshifts.
Another alternative approach -- buying and wiring up cheaper sensors tocustom board with bluetooth. Much cheaper solution with full flexibilityto design a simple and reasonable communications protocol. instead ofhaving to deal with such an intense system. advantages include widerappeal, potential existing open source solutions (modules, both in termsof software and hardware).
272
Preface
System Preferences:
Java enabled device with the following implemented protocols:
-MIDP2.0 / CLDC 1.1 for Floating Point Math
JSR 082 for Bluetooth Connections JSR 135 1.2 and above for MIDI Sound ACSpecs can be used to determine if and which versions of these JSRs are available.
273
I. Main Menu
Module Overview
ACPlotter Graph plotting values
ACSensorizer MIDI synth sonifies values
ACConsole Serial Terminal to connected devices Find & connect to ACNexusBT devices Presets Load and Save Patches (& Settings)
Specs Show Information about the phone Incl. MIDP, JSRs, Multimedia Features & Bluetooth
About Information & Weblinks
II. General Control
All main modules (see I.) have a unified menu control.
Use the cursor keys:UP/DOWN to switch between menu itemsLEFT/RIGHT to toggle the selected parameter
ACPlotter Screen Menu -Units & Color
-Enhancement Method -Samplesize
Scale (From / To)
274
275
-Invert -Release Detect
ACSensorizer Screen Menu
-Harmony (Base / Scale) -Beats per Minute - Note Length Value -Enhancement Method -Samplesize Restrict (Ignore Below / Above) Scale (From / To)
-Invert -Release Detect -MIDI Channel -Program (Instrument) - Channel Volume
Numeric Keys
1,2,3,4,5,6 : Select Sensor (& toggle on/off if selected) 7: Reserved for further use (could have debug settings)8: Panic! (ACSensorizer), Start/Pause Sensorizer (ACPlotter)9: Scale value to fit screen maximum 0 : Toggle Screen Info */# : Previous/Next Type (Units & Color)
Special Keys Back: Return to Main Menu Button 1: Return to Main Menu Button 2: Calibrate selected sensor „C“ (Clear): Reset (! Resets **ALL** customized settings for this sensor immediately !)
III. Calibration Settings
ACEnhancer is a subclass of ACSensor that outputs ACUnits as well. The Unit-conversion is not fully implemented yet!
Calibration If you press the softbutton „Calibrate“ you can select two options:
1. Calibrate: gives you 4 seconds to define a minimum and maximum value operating thecurrently selected sensor. After these 4 seconds, the values are scaled between these max and mins. Thought the saving and restoring of these calibrations are prepared in the firmware, theyʻre not yet fully implemented in ACNexus. That means this setting is volatile, once the ACNexusBT is switched off.
2. Reset: resets previously occured calibrations for the selected sensor.
Pressing the Clear-Button „C“ results in an immediate reset of all parameters to defaultvalues (eg., scale from 0 to 127...).Additionally, the sensorʻs calibration is being reset.
276
IV. Enhancement Types
The Images in the top row show Raw / Average / Smooth / Relative,the second row shows all Drop Enhancement with different parameters
If you open up ACSensorizer or ACPlotter you are being asked if you want to connect to the ACNexusBT Bluetooth device:
„NO“:the application goes into DEMO-mode, where all values are randomly generated.„YES“:the application connects to the default BT-ID Hardware Address
You can get the default BT-ID Hardware Address if you open ACConsole, select the „ACNexusBT ID:“-Field and choose „More -> Search“. Be patient, it sometimes takes up to1 minute until the search process is being shown. If completed and devices found, pleasechoose a device out of the shown list. The Hardware-ID of the selected device will be stored with your patch (and autorestored next launch).
If you are in DEMO-mode and wish to connect, open ACConsole, select the „ACNexusBTID:“ - Field and choose „More -> Connect“.The connection indicator now shows weather the connection is okay or not.
277
V. Screenshots
Various Screenshots from different modes:
278
Clear (Softkey) Clear ScreenLEFT/UP remove last sensor RIGHT /DOWN add next sensor FIRE Auto-Calibrate (In Sensor-Mode ONLY!)
* Search new Arduino Bluetooth Device # Edit Arduino Bluetooth Hardware ID directly
Connect(Softkey) Connect to Arduino BT with shown Hardware ID Exit (Softkey) Quit Application
* show numeric info # toggle Sensor Mode / Enhancer Mode
@ Sensor Mode: 0 show all sensors 1 show only sensor #12 show only sensor #23show only sensor #34show only sensor #45show only sensor #56show only sensor #6
@ Enhancer Mode: 0 RAW (raw signal)1 AVG (Average, nice & smooth curve) 2 REL(Relative, only up or down)
Enhancer Modes can be set for all or for
AUTO-CALIBRATION: @ Sensor Mode: -if not already in sensorMode („S#0“): select sensorMode (#) select sensor to calibrate (1-6)
press FIRE -Now you have 4 seconds to create a minimum & maximumvalue. These values get stored and will be interpolated. Values below the minimum & above the maximum will be ignored.
-Press the ‚Cʻ-Button to clear all previously calibrations
279
ACNEXUSMOBILE
(J2ME MOBILE APP) _0.2.2
J2ME Applications for the Mobile Phones Tested for SonyEricsson & Nokia Phones
System Requirements -CLDC 1.1 / MIDP 2.0 -Bluetooth & JSR-082 -MMAPI & JSR-135 Version 1.2
( for ACSensorizer module! note that the N80 only has version 1.1 !)
Description: This is the main software that runs on phones! It is able to connect to ACNexus devices and receive sensor values. ACNexusMobile has currently two main modes: a Graph Plotter (ACPlotter) and a musical representation (ACSensorizer). The sensors can be configured and calibrated in both modes. ACConsole is there to search for new devices and transmit various debug strings.
Installation: Send ACNexus.jar to your mobile phone and install. Some phones require addionally the .jad file
ACNexus Suite v0.2 · Quickstart Overview · Michael Markert · http://www.audiocommander.de · 2009-03-17
ACNEXUSQC (QUARTZCOMPOSER PLUGIN) _0.4.0
Quartz Composer Plugins incl. Examples (signal & video)
System Requirements: - Mac OS X 10.5.x - Installed Developer Tools
Description: QuartzComposer is a realtime OpenGL application, a so called "visual programming environment". Without the need to program texts, dierent objects ("patches") can be connected by wires.ACNexusQC is a custom developed plugin ("patch") that enables the communication with an ACNexus device (ArduinoBT) and makes it as simple to use as for example the Mouse-Coordinate Source-Patch.
If installed properly, a new Plugin "ACNexusQC" can be dragged from the Patch Creator (Menu: Edit > Show Patch Creator / or press RETURN).
The general aim for this plugin is that it can read videos from a folder on the computer and manipulate these videos by filters and eects which are in return controlled by the values received from the ACNexus sensors. The output is a realtime video that can be projected.
Outlook for later versions: When this runs ona Mac OS X Server, the generated video can be transcoded to a rtsp-webstream and broadcasted to the web, which can then be received by phones that support this streaming video protocol. The broadcast will be near-realtime only.
281
Installation: Install & Run these plugins as described in readme.txt: - ACNexusQC.plugin -ACProcessorSuite.plugin (contains ACSignalEnhancer)
Installation and Usage is a bit complex, see readme.txt and various pages with video-tutorials introducing QuartzComposer. Once the plugins are installed successfully, you can check out the included Examples by simple double-clicking:
"Examples/ACNexusBT_06.qtz" Shows the six sensor values as bars (and numeric values)
"Examples/ACNexusBT_Video_01.qtz" playback of the video "27102007211.mp4" while it's hue and exposure are controlled by the sensor values of the connected ACNexusBT device
"Examples/ACNexusBT_Video_02.qtz" like the above example, but a variation and other (more realistic video).
ACNexus Suite v0.2 · Quickstart Overview · Michael Markert · http://www.audiocommander.de · 2009-03-17
282
ACNEXUSX (MAC OS MIDI APP) _0.1.0
Mac OS X Stand-Alone Application that connects to ACNexusBT devices and transmits the sensor values as MIDI (CH1, CC#20-#25)
v0.1 lacks support of an Enhancement module and MIDI configurations settings (planned for v0.2)
Description: ACNexusX is a Mac OS X 10.5. stand-alone application. Once opened, it can communicate with any ACNexusBT device and receive the current sensor data. This data is then converted to MIDI-Messages and sent to the selected out port. Any application on the Mac (or on the network) or connected MIDI devices can then receive these MIDI messages and interpret them, that means: use these values to manipulate and control something (normally music or video).
The general usage for this program is to enable inter-application communication of the sensor values. For example: the MIDI signals can be received by any VJ application that supports MIDI. It is not required to write a plugin for these applications, because nearly any VJ-app takes MIDI controller messages.
Installation: Copy ACNexusX to the /Applications folder and double-click. To actually see anything, a correct MIDI-setup is required! In other words: if no application or synthesizer or VJ-tool is actually receiving the MIDI messages, the sent values are simply invisible!
Camille BAKER, PhD thesis final – September 2010 SMARTlab, CITE- University of East London
A.2.10b/ QuartzComposer patch guide
Overview Summary
This document describes the function, installation and use of the Quartz Compositions created for Project
Mindtouch. The compositions have been tested on a Mac Power PC running OS 10.5 and on a Intel Mac
running OS 10.6. The compositions run on Quartz Composer version 3.1 and up.
Composition Name Description
MoviePlayer_and_publish_body_sensor_ data_using_OSC.qtz
This composition communicates with the Arduino using Bluetooth, plays back Quicktime movies filtered using the sensor readings and publishes the sensor readings to a web server using open sound control (OSC)
MovieLoader_using_XML_feed.qtz This composition recreates the movie ‘performance’ at a remote computer by downloading the sensor data in an XML file from a web server and using the data to apply the same visual treatments to the Quicktime movies.
284
General Notes
Note: the directory scanner patch used in the plug-in only works with Quicktime movies ending with the .mov
extension.
285
MoviePlayer_and_publish_body_sensor_data_using_OSC.qtz
http://sourceforge.net/projects/acnexus/
The download will contain instructions on how to install the plugin.
It is possible to test the video composition by simulating sensors using random data. To do so, activate
the Random @ Offline setting checkbox in the composition’s patch parameters. This and other settings
in the patch parameters are described below.
Show Numeric Info – when activated, this setting displays the raw sensor data, or the randomly generated test data. Random @ Offline – when activated, random numbers are generated to simulate sensor data. This should be switched off for performance. Quartz Compositions for Project Mindtouch
286
Scan Signal – tells the composition to scan given directories for movie files; you should not need to change this.
Location 1 to Location 4 – tells the composer where to search for Quicktime files on the local machine. If multiple Quicktime files are in a directory, the composition will play through all of them. Important note: the file path needs to end with a forward slash ‘/’.
Width and Height - this controls the size of the final video window after compositing and applying effects.
Shuffle – when activated, the composition will play the movies in each folder at random. Show random
debug - this shows the random order in which movies will be played.
ACNexus BT patch
The current set-up supports four body sensor datas. In theory, it appears that up to six sensors can be
supported.
OSC
OSC stands for Open Sound Control, a protocol used for conveying data. OSC is being used to send the body sensor data from the local composition to a web server. This is shown in the diagram near the beginning of this document. The OSC data is picked up by an application called OSC2HTML which takes the sensor feeds and uploads them to a web-site.
The OSC patches setting should not need to be changed.
The sensor data gets published a 4 floats with the OSC
format of /sensor{sensor number} {value}.
The OSC2HTML application expects the data on port 7003.
287
Image Processing Effects in Quartz Composer
The following screen grab shows the part of the composition that applies data from the four body sensors to the
Quicktime movies.
The image processing system applies different brightness, color saturation, and Gaussian blurs to the Quicktime movies according to the Arduino sensor data, then overlays all four movies to a single screen.
MovieLoader_using_XML_feed.qtz
Patch
Parameters
]]
288
Scan Signal – tells the composition to scan given directories for movie files; you should not need to change this. Location 1 to Location 4 – tells the composer where to search for Quicktime files on the local machine. If multiple me
Quicktime files are in a directory, the composition will play through all of them. Important note: the file path needs to end with a forward slash ‘/’. Width and Height - this controls the size of the final video window after compositing and applying effects. Random Shuffle – when activated, the composition will play the movies in each folder at random. Show random debug - this shows the random order in which movies will be played. Show Numeric Info – when activated, this setting displays the raw sensor data, or the randomly generated test data. XML Location - this is the location where the XML containing sensor readings are downloaded from. Period - this controls how frequently the composition checks for new data from the server. The lower the period, the more frequent the updates are.
Troubleshooting Movie playback is slow
The compositions are applying real-time effects while 4 Quicktime movies are being played and
composited, and this requires significant processing power. Make sure all unnecessary applications
are closed while running composition.
Check that the ACNexus patches are installed. Check that Bluetooth is enabled on the performance computer.
Sensor data doesn’t appear to match performance
Make sure that the ʻRandom@Offlineʼ option in the Patch Preferences is deactivated.
No Movies Playing, Only blue screen showing
Make sure that directories are properly formatted and notated in the ‘Location 1-4’ text boxes. Make sure
that Quicktime movies are used, and that they have an .mov extension.
Sensor data is not being received by the server
No sensor data coming through and compositions is reporting errors
Check the app OSC2HTML is running. Check the internet connection.
289
290
A.2.10c/ XML solution for webserver
Evan's Mindtouch Guide
Overview: This document describes how to use the OSC app and webserver (see diagram) to connect the two Quartz Composer sketches, one running on the performance laptop and one running on the server.
OSC2HTML.app About:
This application, running on OS X 10.5 and above, receives messages from the performance laptop's Quartz Composer composition and sends them to the webserver, where they are stored in a MySQL database and picked up later by the Quartz Composer composition running on the server.
To Run: On the performance laptop, simply double-click the application to run it anytime before you run the Quartz Composer composition on the performance laptop. When finished, quit the app. There is no visual feedback.
Technical Details:
The app expects OSC data on port 7003 of the form:
/sensor[INDEX] [float value] For example, to send some data
(3.1415) to the app from the 1st
sensor: /sensor1 3.1415
Note: Sensor indices start at 0.
Web Server
The Quartz Composer sketch running on the server will automatically retrieve data from the webserver.
Note: Data is never erased from the webserver database, and will need to be
periodically cleaned out, or exported and saved for future use. Technical Details:
291
To retrieve data from the database via a web link is easy:
http://swampgirl67.net/mtxml/putSensorData.php?p=m1m1nd&mode=display
This will retrieve an XML file with the last 20 sensor data entries in it.
All other information can be found in the Php source code for the single file that handles the saving and display of data:
PutSensorData.php
<?php
/***** db fields: +--------+--------------+------+-----+-------------------+----------------+ | Field | Type | Null | Key | Default | Extra | +--------+--------------+------+-----+-------------------+----------------+
+--------+--------------+------+-----+-------------------+----------------+
******/
// test password! if (isset($_REQUEST["p"]) && $_REQUEST["p"] == "m1m1nd") {
$dsn = array( 'phptype' => 'mysqli', 'username' => 'swamp_mtouchw', 'password' => 'm1ndxna', 'hostspec' => 'localhost', 'database' => 'swamp_mindtouch',
);
$db = mysql_connect($dsn["hostspec"], $dsn["username"], $dsn["password"] ); if (!$db) {
die(mysql_error()); } $db_selected = mysql_select_db ($dsn["database"], $db);
if (!$db_selected) { die ('Can\'t use ' . $dsn['database'] . ': ' . mysql_error()); }
// set default mode for page $mode = "save"; //display mode
if (isset($_REQUEST["mode"])) { $mode = strip_tags($_REQUEST["mode"]); }
// do we display results? if ($mode == "display") {
echo '<?xml version="1.0" encoding="ISO-8859-1"?>';
| id | int(11) | NO | PRI | NULL | auto_increment | | data
| varchar(255) | YES | | NULL | |
| time
| timestamp | NO | | CURRENT_TIMESTAMP | |
292
echo "\n";
$limit = 20; $result =& mysql_query ('SELECT * FROM `sensordata` ORDER BY `time` DESC LIMIT 0,' . $limit, $db);
if (!$result) { die('Could not query:' . mysql_error()); } /* while ($obj =& $result->fetchRow (DB_FETCHMODE_OBJECT))
printf ("%s, %s\n", $obj->name, $obj->category); */
/* print("<p>SAVED MSGS:</p> <p>\n");
while ($row =& mysql_fetch_array ($result)) { printf ("id: %s<br> data: %s<br> time: %s<br>", $row["id"], $row["data"], $row["time"]);
print("\n</p>\n"); } */
print("<sensordata>\n");
while ($row =& mysql_fetch_array ($result)) { printf (" <entry id=\"%s\" time=\"%s\">
<data>%s</data> </entry>", $row["id"], $row["time"], $row["data"]); }
print("</sensordata>\n");
mysql_close($db); } else {
/* $_POST["d"] // data $_POST["ts"] // timestamp */
$data = strip_tags($_REQUEST["data"]); $ts = strip_tags($_REQUEST["ts"]);
$result =& mysql_query ("INSERT INTO sensordata (time, data) VALUES ('$ts', '$data')", $db);
if (!$result) { die("error inserting: " . mysql_error()); } mysql_close($db); // create timestamp, insert into table msgs in db txtmsgs (text, #,
timestamp) // create other page to display last 10 txts
} } ?>
293
Appendix III/ performance event materials
294
A.3.1/ streaming server set-up and operational notes
Logging into the server username admin password SMARTlab (note capitalisation) Serv ices Running (already setup) Web Quick Time Streaming Web Publ ishing to put a document in the right place to be served via the web server - use the alias on the desktop (Documents for Web Server) on the desktop - drop the file on this and it will be served immediately Quickt ime Publ ishing to put a document in the right place to be served via the web server - use the alias on the desktop (Documents for Quicktime Server) on the desktop - drop the file on this and it will be served immediately IP Address At this research phase, the server does not have a name or fixed IP address - to find the IP address, click System Preferences on the Dock, select Network, Select Ethernet 2 and the IP address will be shown in the pane on the right To check a file is being served via the web server Go to Safari (on a Mac, especially if you are serving QC patches) type http://<IP Address>/filename so if your IP address is 161.76.254.33 and the file is quartz.qtz the URL would be http://161.76.254.33/solar.qtz To check a file is being served via the Quick Time Streaming server (not live) Go to Quicktime Player - Click File > Open URL type rtsp://<IP Address>/filename so if your IP address is 161.76.254.33 and the file is sample_300kbit.mov the URL would be rtsp://161.76.254.33/sample_300kbit.mov Quartz Composer Notes If your patch requires access to any audio/video/Bluetooth, then it will not work via a web browser. Podcast Producer, please note: The Podcast Producer service is not currently viable for the MINDtouch project as it does not offer any service beyond that already supplied by the streaming server and web server. Quickt ime Streaming Server Set up as service – there is no need to change this. Quickt ime Broadcaster - L ive stream
295
To stream the contents of the web cam of the server (or other live streams from cameras) - click on Quicktime Broadcaster in the dock - rather than go through all the options - if you want to broadcast the isight camera live, check the monitor (on the left) is showing the output from the isight - click broadcast - this can then be seen on any Mac in the lab via QuickTime player Click File> Open URL type rtsp://<IP Address>/untitled2.sdp this is currently a 25 frames per second 500kbit/sec stream click stop at the end or the broadcast will continue for ever ! Backup The main hard drive of the machine is backed up to one of the internal drives - the other drives aren't at present (but this is made clear in the name of the disc)
296
A.3.2/ technical workplan (December 2009): Evan : - set up MySQL on Camille's (hosted) server yesterday (to store the sensor data for a php script to retrieve), the database is swamp_MINDtouch , username is mtouchw and pass is m1ndxna. 1) Write a Php script that runs on Apache on your hosted server called "handleSensorData.php" that will receive an XML string as input (sensor data, in a format to be worked out between Pawel and Manjit) and store it in a MySQL database (he will have to work out how, such as what database tables to use. I'd suggest unique id, timestamp, sensor name, sensor data) - to make a brief specification for me on the data format from the Quartz Composition that gets uploaded to the web server for SOAP for Manjit. 2) Write another script (Php on Apache) called something like "retrieveSensorData.php" to retrieve it in XML form for Quartz Composer (Manjit) to handle. 3) Then, there needs to be an application running on the streaming server (or a script, regularly triggered by something such as Quartz or a scheduled Cron job) that rips the Qik movies from their website and converts them to a Quicktime format that Quartz can readily use. There are commandline-tools to do this that can be scripted, such as ffmpeg. Manj i t : Modifying the send / receive patches in QC to handle the translate the sensor data into XML code: 1) using a SOAP patch in the quartz composition that runs on Camille's laptop. This sends data to the web server. He needs a brief from on what the XML looks like. 2) I will be changing the quartz composition that resides on the server to handle XML via RSS. This is with respect to sending and receiving the body sensor data.
297
A.3.3/ performance preparation checklist - contact external participants with task list - create remote "task activity list + promo - contact past participants to watch and participate - put software (QC patches and phone) and videos on appropriate machines - Instructions on phone software out to participants to load - charge phones --N80, N93i - find a documenter - print consent forms - post-event surveys printed - have extra batteries available - have script created (PHP) for video pulling - put new videos in folders - make .movs
298
A.3.4/ performance preparation outline
⇒ Tech checklist – phones etc
⇒ Testing procedures:
o Computers : server and laptop / software and networks on both
o Garment electronics / batteries etc
⇒ Venue (make a timeline of a day of performance)
⇒ Networks – Ethernet and network set up – general connection and streaming server set up
and configurations
⇒ People
o Questionnaires and consent forms
o Promotion: Info and invites on Facebook and my own website (get screen shots)
o Prepping remote group
o Documentation person
o Prepping Guides
The Event Process
⇒ Double-check technology
⇒ Introductions to guests
⇒ Working with Guides / guides working with participants
⇒ Dressing the person/ people in the electronic garment
⇒ Sending video group off
⇒ Post activity discussion and / interviews
⇒ Questionnaires
⇒ Journaling (my observations)
⇒ Collection of phones and media
Post Event
⇒ Send notes of appreciation for participation via email
⇒ View of media and documentation
⇒ ‘Journaling’
⇒ Post mortem Review of what worked and what did not
⇒ Reading of the questionnaire
299
A.3.5/ event technical preparation
1) taking biofeedback data collected by the phones previously and video mix them using custom
software on a local computer in Quartz Composer's programming environment and send it out to the
web server for others to download via web or WiFi or WLan;
2) to have a web vide streaming server set up to use RTSP (Real Time Streaming Protocol) to serve the
live and recorded video from the web, then back to the phone web browser;
3) embed the RTSP streams into a website that is a) be viewable / displayed and interacted with or
selected them on a phone (using qik or l ivecast application) and; b) be able to select videos inside
QC, mix them and send them back to web/ then to the phones.
video documentation The digital documentation and journaling includes an online and DVD presentation, intended to
give readers various, accessible ways to understand the project process. It will include a website
overview of the thesis research and ongoing processes of:
• conducting research, sourcing and costing sensors and phone equipment used in the
research, other tools and people for the development of the media, events and
technologies explored and software created;
• on-going audio/video, visual and textual documentation;
• meditation and movement research experiments, practices, processes and experiences;
• Initial planning concepts, sketches and images of the initial building steps;
• programming and exploration of the phones with the biofeedback sensors,
• work-in-progress planning and staging of performance events and activities; and
• video and text transcriptions of interviews with media collection performers, Tai Chi
practitioners, VJ mixing performers/ participants and event audiences.
http://www.smartlab.uk.com/2projects/MINDtouch.htm
http://www.youtube.com/v/2qUSp1yl7T4&color1=0x6699&color2=0x54abd6&hl=en&feature=play
er_embedded&fs=1
There has already been an exhibit of the mobile video edit taken from workshops and performances
from the research project, as abstract mobile video art work on the phones they were created on
installed in a gallery, which was shown in London, April 4, 2008 FILMobile.
These videos can stand alone as artwork / output from the project on its own, as apart of the
practice-based performance media research for the PhD. http://www.swampgirl67.net/filmobile.html
300
A.3.6/ additional logs of facebook, website and online interaction
301
302
303
304
305
306
307
308
309
310
311
A.3.7/ first performance: introductions script This is the first of a series of experimental participatory performances and it will be mostly wobbly, because much of the key networking/ streaming are just getting to work in the last few days and not enough time to test much to find any issues yet. There were to be two identical garments to house the biofeedback sensors however we have to two different versions instead. So all the pieces of the complex puzzle I’ve constructed are having their maiden voyage here today with you :) I want to acknowledge and thank profusely, all the great people who have been working on this with or assisting me, mostly for free or for my adoration alone - in no particular order: 4MGroup and Rama Përparim for hosting my server, Fashion Designers : Rachel Lesebikan, Tara Mooney, Networking and Quartz Composer Specialists: Manjit Bedi, and Huw Williams, the great Michael Market: electronics and mobile programmer, and others over the last two years previous programmers: Li Zhang and Jeremi Sudol, as well as at one point Clilly, not to mention the support and guidances of my supervisors Lizbeth, Susan Kozel, and Marc Price. And tonight I thank my lovely host and guide: Chrissie Poulter and Trinity Leeds. So before we get started I will say that will put you into 2 groups of active participants and one remote online group for this iteration - no is allowed to be totally inactive or passive throughout. Only two people at a time can use the biofeedback sensors sets within the gorgeous garments that Tara and Rachel have created. Please be gentle with them!!!! One group will stream / record via qik during their guided activities with me - these videos will be used in a live interaction with everything tonight. Anyone without the qik software can video or pictorially document it for me. We might need an observer group to be involved with at least one or two of these groups (I’m hoping people might share phones if not enough - I have one extra one) and maybe all will get to be observers as well. The observer group is asked to just take notes and not analyse - be recorders not critics -- not just of what you see but what you experience as well. ONE RULE: NO TALKING at all throughout -- laughing and giggling is permitted and only the guides voices and maybe mine will be allowed - this is meant to explore using technology to assist in non-verbal and embodied ways of communicating, so try use the tools and options available to you here other than your words and voice. The video group will be coming in and out but everyone else stays in the room please and video group try be quiet coming in and out. One other note: This is intended to be fun and playful but also reflective, expressive and exploratory. Yet no one is expected to be a professional performer, even if some in the room are -- this is meant to be for the public, for anyone and all - so please have no fear and try not to self-exclude if you have no performance experience - that's the whole point / idea for it to be for non-performers. And I’d like you to all stick around for ten minutes after to fill in a short post-performance survey afterward, but to save your tech or other project question for my process/ practice presentation tomorrow.
312
A.3.8/ performance mind-quieting script [in a soft voice]: Find a place in the space - standing or sitting. Close your eyes and be very still. Then, take several deep breaths, be silent, and settle into your body. Begin to pay attention to your body and the images or colours behind your eyes. Go inside your body and listen to/feel the sensations, such as: your breath, your heart beat, areas of heat or coolness inside your body, electrical energy, your thoughts or brainwaves, your sensations, the movement of muscles in your organs or limbs, or your blood flow. [pause] Focus on one area of your body or sensation that draws your attention now. [pause] Keep focusing your attention on that part of the body or sensation. [pause] Experience the inner sensations for a few breaths. [pause] Feel the blood flow, the expansion and contraction of your breath in the body, and any other sensations or perceptions like stomach gurgling or tickling toes, heat or cold, or any physical manifestations of emotions or thoughts.... [pause] Pay attention to any images or thoughts or emotions you are experiencing with these sensations or perceptions [pause] Think of how you might visualise these sensations or perceptions, thoughts or emotions [pause] Hold these mental images in your minds' eye now – linger with them.
313
[pause] Now think how you might express them to share them with others near or far --non-verbally but visually? [pause] When you are ready, open your eyes ....and try to take these sensations or images or emotions and experiences with you and now and use the camera/video phones to express them, starting very close to your own body or space around you on the floor. Do this for 5 mins.
314
A.3.9/ group guiding information Performance Guide Info - Group 1 (biofeedback garment) Participatory Performance Aspects There will be three different groups/ activities in a social / game-like environment / participatory performance - so designing the staging the event and the logistics are currently in development. Guides need to prepare themselves physically, aesthetically, and emotionally to be a nurturing, empathetic and aware of participants and create a ritualized, performative approach when working with participants. As guides, we need to keep our intention on creating a social, intimate, safe and playful experience for participants. Guides will assist a small group of participants through the activities: group 1 - Activity Overview Participants will be wearing the biofeedback sensors for this activity and guided through various movement and emotional/ sensory activities = playful both calming and vigorous or "stressful" - to activate the sensors in various ways - which trigger video from the database and the effected video is displayed on the big screens around the room; Your job is to help the two (until we have more sensors sets) participants put on the garment: 1) invite a couple or people in the group to put the garment - in a playful way; 2) explain to them that they will not be asked to do anything they are not comfortable to do and they can stop at anytime and take the garment off; 3) help them put on the garment; 4) connect the battery to the Arduino (make sure they are fresh batteries); 5) start the same as the video group -- ask them to close their eyes and do a short guided imagery / mental visualisation - 5 mins max 6) guide them through the telepathic activity - sending and receiving - 5 mins -- each (telepathic activity is: partners sitting back-to-back, try to ‘telepathically’ send an image or sensation in their mind to one the partner, each afterward can guess what the other person sent, and share what was sent were the similar to what was received) Paper and pens
315
7) guide participants to do some tensing and releasing of muscles on the spot - 2-3 mins max 8) then guide them to do a slow walking / movement activity individually (see the Augusto Boal book) - 2-3 mins max 9) ask them then to do a weight sharing activity back to back (see Laban exercise) - 5 mins max 10) guide them through faster movement (maybe some running or jumping or something more strenuous) through the space -- weaving through the other participants, interacting with others in the space if they want as long as they don't stop too long, don't speak and then keep moving. 3 mins max
MINDtouch: Embodied Transference/ Transcendence – mobile performance event July 2009
316
Performance Guide Info - Group 2 (no tech) Participatory Performance Aspects There will be three different groups/ activities in a social / game-like environment / participatory performance - so designing the staging the event and the logistics are currently in development. Guides need to prepare themselves physically, aesthetically, and emotionally to be a nurturing, empathetic and aware of participants and create a ritualized, performative approach when working with participants. As guides, we need to keep our intention on creating a social, intimate, safe and playful experience for participants. Guides will assist a small group of participants through the activities: group 2 - Activity Overview Participants will be doing something similar to group 1+ 3, but without any technology and paying more attention to the screen imagery. Your job is to: Start the same as the video group-- ask them to close their eyes and do a short guided imagery / mental visualisation (5 mins max) This activity will be broken down into guided 4 sections of short explorations:
1) close range movement – slow walking / movement, expressions/ gestures of internal sensation/ feelings or body itself, etc. (3 mins max)
2) in partners, participants practice expressing themselves non-verbally / gesturally: they are asked to ‘introduce themselves’ non-verbally without speaking - using body language and gestures -- no speaking allowed !!! (5 mins max);
3) collaboration of experiences:
partners sitting back-to-back, try to ‘telepathically’ send an image or sensation in their mind to one the partner, then guess what they saw in their mind and was sent from the other - taking turns (3 mins each person). Afterward each can tell what the other person sent, sharing their mental images with each other to see if the exchange was successful (they can talk in this part) 4) Observation + interaction exercise : guide them to observe the videos mix and bodydata visualization made by the biosensor group and then try to NON-VERBALLY find ways to interact or send mental messages to those participants (non-verbally) to see if it changes the data outputs from them (5 + mins) -- guide them to move freely around the space [Tell them: see script at the end of group 3 ideas]
317
MINDtouch: Embodied Transference/ Transcendence – mobile performance event July 2009
318
Performance Guide Info - Group 3 (video capture) Participatory Performance Aspects There will be three different groups/ activities in a social / game-like environment / participatory performance - so designing the staging the event and the logistics are currently in development. Guides need to prepare themselves physically, aesthetically, and emotionally to be a nurturing, empathetic and aware of participants and create a ritualized, performative approach when working with participants. As guides, we need to keep our intention on creating a social, intimate, safe and playful experience for participants. Guides will assist a small group of participants through the activities: group 3 - Activity Overview Start the same as the other groups -- ask them to close their eyes and do a short guided imagery / mental visualisation (5 mins max) This activity will be broken down into guided 4 sections of short explorations:
1) close range movement – expressions/ explorations of internal sensation/ feelings or body itself, etc. (5 mins max)
2) then go further: expressions/ explorations of external experience or what they see outside of themselves and their own body;
3) in partners, participants explore exchanging with each other and the themes below, exploring body sensations, internal feelings (keep all non-verbal / visual); external elements of the body and surrounding it.
exercise :
participants are asked to ‘introduce themselves’ non-verbally using the videophone without speaking -- no speaking allowed -- one example: recording parts of themselves to show each other 4) collaboration of experiences – make something together – 5 minutes each activity (time it) to prepare and make. Options (choose one per group)
exercise 1:
participants are asked to work with their partner, also non-verbally, to create something to send to someone else not in the room, telepathically or through the mobile phone, something that would express their combined inner perceptions and experiences. exercise 2:
partners sitting back-to-back, try to ‘telepathically’ send an image or sensation in their mind to one the partner, then attempt to visually capture what they saw and sent from their mind
319
- taking turns, 3mins each. Each afterward can guess what the other person sent, and share their videos with each other to see if all three versions (the telepathic, the videophone and the guess) were the similar. ____________ Tell them: Consider yourself a performer not just a documenting observer in this event. Try to use a different sense of seeing, reading through your own body in order to discern and select moments to capture … but try maintain enough outside perspective to be aware of the world while you explore this sensory tool. Be sensitive and aware of yourself, others and the world. [see workshop script for the creative framing suggestions] MINDtouch: Embodied Transference/ Transcendence – mobile performance event July 2009
320
A.3.10/ performance event participant consent form
The SMARTlab Digital Media Institute Practice-based PhD Programme, UNIVERSITY OF EAST LONDON
Consent Form - Confidential data I understand that my participation in this project will involve being observed in a participant-centred event, conducted by Camille Baker as investigator, who will record the event on video, and ask me to fill in a survey afterward, to determine the effectiveness of participatory performance approach, as a part of a larger research project on mobile media performance and the experience on presence on the network as part of the practice-based PhD. I understand that participation in this event is entirely voluntary and that I can withdraw from the event at any time without giving a reason. I am free to withdraw my participation at any time or to discuss my concerns with the event leader or her supervisor. I understand that I am free to ask any questions at any time. I am 19 years of age or older. I understand that at the end of the event I will be provided with additional information and feedback about the purpose of the research project, should I request it. I also understand that the activities during this event may involve minimal physical and personal risks to me, such as tripping or falling- depending on how active I choose to be in the event. I shall undertake to move safely about the space, with this in mind. By involvement in this event, I understand that I am giving my informed consent for information I provide therein to be included for my video contribution to be broadcast worldwide, and for the outcomes of the be published widely, and in future studies.
I understand that the information provided by me will be held confidentially, such that only the Investigator (event leader) can trace this information back to me individually. The information will be retained for up to 10 years, when it will be deleted/destroyed.
I understand that I can ask for the information I provide to be deleted/destroyed at any time and, in accordance with the Data Protection Act, I can have access to the information at any time.
I also understand that I may register any complaint with University Of East London, University Research Ethics Committee. I, ___________________________________(NAME) consent to participate in the event conducted by Camille Baker, SMARTlab Digital Media Institute PhD Programme, UNIVERSITY OF EAST LONDON, under the supervision of Professor Lizbeth Goodman. Signed: Date: Further information can be obtained from Research Ethics Committee, University Of East London, Docklands Campus, 4-6 University Way, London, UK, E16 2RD, or contact Debbie Dada at +144208 223 2976 or [email protected]. I may obtain copies of the results of this event, upon its completion by contacting: Camille Baker Flat 14, 58 West End Lane, West Hampstead, London England NW6 2NE UK
+44(0) 78 4768 1127
321
A.3.11/ post performance audience survey
Please tell us about yourself! Answer the following questions as honestly and clearly as
possible, in order to help us understand your experience better. Information gathered from this
survey will assist in planning future events. Your responses will remain confidential, and will be
used for statistical purposes only; all questions are optional of course.
Demographic Information:
Age: Gender:
Highest Level of Education: Ethnicity/Nationality:
Academic Major/Stream: Occupation:
Annual Household Income: City of Residence:
Event Experience Feedback:
What was the most intriguing part of the event /
activity? Why?
What was the least intriguing part of the event /
activity? Why?
What aspect of this event / activity do you think
worked well?
What aspect of this event / activity do you think
could be improved and how?
How is the experience of this type of performance
different than a more conventional performance
(stage dance, music, theatre, etc)?
Did you feel the same sense of presence or
interaction between performer and audience in this
event / activity, as you might in a conventional
performance event? If so, how?
How did you feel about being a performer, rather
than an audience?
Other comments about the event / activity, as a
whole?
322
Experience Inventory (optional):
answer as briefly or extensively as you like.
If/when you were making your video. . .
- How did you perceive your body’s sensations, perceptions, thoughts and other internal
experiences during the activities tonight?
- How did you feel about expressing yourself and sharing these sensations and experiences
with others through these activities tonight?
- What did you think about participating in these activities using mobile devices?
- How did/ do you feel about sharing these media expressions and experiences in the larger live
video installation?
- Other experience comments:
Thank you for your participation!
323
A.3.12/ event participant experience feedback themes What was your favourite part of the event tonight? Why? (themes) • participating • relaxing • being creative • watching others moving • making images What was your least favourite part of the event tonight? Why? (themes) • didn't understand (2) • moving (2) • expressing / filming self - self-conscious (3) • being observed by others (3) What aspect(s) of this event do you think worked well? (themes) • guiding / facilitation / organisation (3) • staging / set up / space (2) • relaxation / meditation (4) • exercises (different ones for different responses) (6) • making videos / visuals (6) What aspect of this event do you think could be improved and how? (themes) • more explanation (5) • want to see the project results (5) • more info on the body sensors and what is happening with the body data (2) How is the experience of this type of performance different than a more conventional performance (stage dance, music, theatre, etc)? (themes) • participative / participatory (7) • more interactive (3)
324
• more personal (3) • more improvised (3) Did you feel the same sense of presence or interaction between performer – audience in this event as you might in a conventional performance event? And How? (themes) * • didn't perceive difference between performer and audience (4) • not a performance (2) • more confident / less self-conscious (3) *most misunderstood the question as intended, since the networked aspect wasn't working fully nor explicit. people didn't connect with the performance activities as performance or that there were remote participants or audience ––> they just experienced themselves and others in the space, involved in the activities in a personal, intimate and individual sense. How did you feel about being a performer rather than an audience? (themes) • yes/ ok/ happily/ enjoyed (even if not usually performer) (16) • unsure (2) • uncomfortable (4) • didn't realise or feel it was performing/ they were a performer (4) Experience Inventory (optional): answer as briefly or extensively as you like. How did you perceive your body’s sensations, perceptions, thoughts and other internal experiences during the activities tonight? senses noted(themes): • bodily - pulse, spiky crampy, lungs, head, heated, tired, ill health • mood or mental - calm, focussed, racing thoughts, abstract, positive, heightened awareness How did you feel about expressing yourself and sharing these sensations and experiences with others through these activities tonight? (themes) • good/ fine/ enjoyed (7) • nervous / uncomfortable (4) • other = neutral, low-key, safe (3) What did you think about participating in performance activities using mobile or wearable electronic
325
devices? (themes) • fun/ cool/ enjoyed it (7) • interesting / new (3) • no problem but... (3) How did/ do you feel about sharing these media expressions and performative experiences in the larger performance installation ?
(themes) • fine/ sure/ good (8) • unsure (2) • need more info / or to see results first (3) Other comments about the event as a whole? Other experience comments: (themes) • good / enjoyed it / great work / interesting (12) • problems or focus on gadgets (2) • interested in the screens, video, media, or live feeds (5)
326
A.3.13/ performance survey feedback
327
328
329
330
331
332
333
334
Appendix IV/ collaborator and technical feedback
335
A.4.1/ testimonials on network issues
336
A.4.2/ screenshots of forum discussions on Nokia Realplayer issues
337
338
Appendix V/ DVD Support Materials: Table of Contents
339
A.5/ Notes There are five media DVDs to support the research exegesis/ thesis:
1) Participant workshop video clips*
2) Workshop documentation videos and stills
3) Performance videos and final ‘VJ’ mixes
4) Promotional materials
5) Technical materials
These media support materials were split into 5 parts due to the great amount of video and media
generated from each phase of this project.
A.5.1/ Disc 1 - participant workshop video clips This disc is the only one that does not have a designed interface and interactive menu presentation
of the videos created by participants in workshops from 2007-2009. After many months and many
attempts was unable to get the software to encode these mobile phone videos in any DVD
creation software without crashing. This is due to the differing compression put on the video clips
for each different phone model that is incompatible with the DVD Encoding protocols. However,
videos can be viewed individually in Apple’s QuickTime video player software.
N ote: that while there were many more participants in these workshops, the video here is all the was shared or useable. Disc Table of Contents Main Menu
1. Participant workshop videos
i. June 2007 Vancouver
a. Participant 1
b. Participant 2
c. Participant 3
d. Participant 4
e. Participant 5
ii. July 2007 Dublin
a. Participant 1
b. Participant 2
c. Participant 3
340
d. Participant 4
e. Participant 5
iii. August 2007 Vancouver
a. Participant 1
b. Participant 2
iv. October 2007 Dublin
a. Participant 1
b. Participant 2
c. Participant 3
d. Participant 4
v. January 2008 London
a. Participant 1
b. Participant 2
c. Participant 3
vi. November 2008 London - Dana Centre demonstration
a. Participant 1
b. Participant 2
vii. March 2009 London
a. Participant 1
b. Participant 2
c. Participant 3
d. Participant 4
e. Participant 5
f. Participant 6
A.5.2/ Disc 2 - Workshop documentation videos and stills This disc and the remaining 3 discs were easily encoded with the original layout and interactive menus designed for all. Disc Table of Contents Main Menu
1. Workshop documentation video
i. Workshop documentation June 2007
ii. Workshop participant interviews July 2007
iii. Workshop documentation August 2007
iv. Workshop participant interviews October 2007
341
v. Workshop participant interviews January 2008
2. Workshop images
i. Workshop still images
ii. Participant 1 workshop stills March 2008
iii. Participant 2 workshop stills March 2008
A.5.3/ Disc 3 - Performance videos and final ‘VJ’ mixes This disc and the remaining 2 discs were easily encoded with the original layout and interactive menus designed for all. Note: that while there were many more participants in these performances, the video here is all the was shared or useable. Disc Table of Contents Main Menu
1. Performance participant video
i. July 2009 (first performance) Matrix, UEL campus, London
a. Participant 1
b. Participant 2
c. Participant 3
d. Participant 4
ii. October 2009 Leeds Trinity College, Leeds
a. Participant 1
b. Participant 2
iii. December 2009 – 4M Studios, London
a. Participant 1
b. Participant 2
c. Participant 3
iv. December 18, 2009, Limkokwing University, London
a. Participant 1
v. February 12, 2010 – Latitudinal Cuisine Dinner, London
a. Participant 1
2. Final performance mixed collages
i. Collage 1
ii. Collage 2
342
iii. Collage 3
iv. Collage 4
v. Collage 5
3. First performance documentation video (July 2009)
A.5.4/ Disc 4 - Promotional materials This disc and the remaining disc were easily encoded with the original layout and interactive menus designed for all. Disc Table of Contents Main Menu
1. Promotional video and posters
i. Promotional video 2007
ii. Mindtouch Promotion 2008 (long version)
iii. SMARTlabPhD Plumbline 2009
iv. Ars Electronica Video Application 2010
2. FILMoble
i. Filmobile edit: workshops July 07
ii. Filmobile edit: workshops June 07
iii. Filmobile edit: workshops August 07
iv. Filmobile edit: workshops Oct 07
3. MobileFest 2008
i. MobileFest08 - test VJ mix
ii. Art video edit from workshops 2008 (long)
iii. Art video edit from workshops 2008 (small)
4. Promotional posters
343
A.5.5/ Disc 5 – Technical materials This disc was easily encoded with the original layout and interactive menus designed for it. Much of the technical development is documented in the main body of the thesis text. This disc just contains additions items relevant to the technical description. Disc Table of Contents Main Menu
1. Technical process
i. Garment design and development
a. Garment design image slideshow
ii. Biosensors and electronics development
a. Biofeedback sensor testing video
iii. Software design
a. Electronics development image slideshow
344
Appendix VI/ published papers relevant to the thesis
345
A.6.1/ ‘Liveness’ and ‘presence’ in bio-networked mobile media performance practices: emerging perspectives (Dec 2008)
The International Journal of Performance Arts and Digital Media, Volume 4 (2), Intellect Press
‘Liveness’ and ‘presence’ in bio-networked mobile media
performance practices: emerging perspectives
Camille Baker University of East London
‘Liveness’ and ‘presence’ in bio-networked mobile media performance practices: emerging
perspectives Abstract
If you could share and exchange your dream imagery, feelings and sensations with your friends and loved ones, how would you do it? If could not only share and exchange, but remix and collage them, what would it look like or feel like? How would it work? Explored through the new media performance project, MindTouch, uses biofeedback sensors and mobile media phones in live, staged events. MindTouch is a BBC R+D sponsored PHD research, to connect people remotely through the mobile phone, allowing them to re-engage with each other and with the world, affectively and expressively in new creative, non-verbal/non-textual ways through mobile video performance events. The aim of this article is to discuss current practice-based PhD research underway at the SMARTlab Digital Media Institute’s UK base at the University of East London, under the direction of Professor Lizbeth Goodman. The MindTouch project involves the pursuit of a better understanding and facilitation of the individual’s mode of expression in relation to the collective experiences of liveness and presence, as observable and ‘capturable’ within the virtual ‘space’ of a live mobile performance event. This article reflects on meta-concepts of this research and on past video collection workshops completed in the first phase of the project, including observations from these workshops, and the on-going practice, development and technical issues involved. It discusses practical processes and concerns (such as methods for encouraging people to connect remotely, that are allowed to re-engage each other and the world affectively using the mobile phone), both in and through participants’ bodies, and through a range of expressive, creative, non-verbal/non-textual means, through networked performance activities.
Keywords liveness and presence
mobile media
body-computer interfaces
interactive performance practices
346
Introduction The aim of this meta-research project is to uncover new understandings of the sensations of ‘liveness’1 and ‘presence’ that may emerge from the use of mobile technologies and wearable devices in performance contexts. A secondary exploration, funded by BBC R&D (2007-present), involves finding new ways to interface between the body/mind and technology. Through the use of biofeedback sensors on the bodies of performers and moving meditators, sending data to mobile phones, I intend to find meaningful ways to communicate non-verbally and through the body directly to technology.
Critical to this work is to enable the perception and embodied sense of liveness and
presence within the virtual, non-space of mobile networks. A visceral, sensory, emotional and philosophical understanding is key in this research as to whether or not liveness and presence are perceived by mobile performers, participants and audiences, in a networked performance event. Also of interest is whether this perception has changed for them or is experienced differently during such an event, compared to traditional performances. To explore these concepts a practice-based investigation has been undertaken, using paradigms of participatory performance and other live visual experiments and improvizations.
This research set out initially to explore the ‘body as interface’ and has both a focused and extended concern through practical testing, to look at the specific conditions of silent communication between people who cannot speak, or who choose not to speak. Thus, the findings consider new ways to simulate, emulate, or even facilitate, a non-verbal or telepathic connection and the sense of feltness, presence or liveness, co-presence or collaboration, within a mobile performance – in this case through the added use of biofeedback sensors to increase the embodied interaction.
Metaphorical underpinnings
The desire to gain a visceral, sensory, emotional and philosophical understanding of how participant performers and audiences perceive liveness and presence1 within real-time, in-person and virtual domains, during mobile performance settings by participants is central. The impetus for much of my work are the inspirational sensations and imagery taken from my dreams, intended to be induced in others, helping them to experience a receptive state of creative consciousness to connect with others ‘non-locally’.
The notion of non-verbal, embodied communication or telepathy, is often felt through dreams is integral to the metaphor that shapes and guides this research. Definitions of telepathy often describe it as ‘presence at a distance’; others call it ‘thought communication’.1 It is the experience of sending or receiving a feeling, sensation, experience, thought, idea, news or traumatic event from someone else (generally someone close or intimate to you), non-verbally or through one’s mind, across distance, and often during dreams (Baker 2004).1 Consciousness Researcher, Dean Radin has done extensive research on telepathy and defines ‘non-locality,’ as ‘physical objects (or people) that may seem unrelated or separate but are connected, transcending the limitation of space and time
347
or taking place simultaneously, either through space and time, or in space or time’ (Radin 1997: 277).
Mobile media phones are used in the MindTouch research as the medium or conduit for self-expression, especially to convey non-verbal experiences and sensations as an extension of the body/mind1 in both non-literal or abstracted ways, as well as more direct representations. Phones could be seen as the vehicle to express embodied consciousness with others, through movement, or creative processes between participants or with self.
This embodied mixing exchange or interaction between the mobile users and the body of the person wearing the sensors can then be seen as a means to emulate the sharing of dream imagery or other non-verbal, pre-language, telepathic communication or conversation between the unspeaking body and the mobile phone users.1 It is surmised here that controlled telepathic interactions could begin at a sensory level, facilitated through dreams, below conscious awareness.
Thus, simulation of telepathy is a key aspect of my performance exploration: employing a creative, technological way to help others imagine a telepathic exchange between two or more bodies who are lucid dreaming (LaBerge and Rheingold 1990) visually and physically, together. In doing so, current available technology is used to illustrate the concept that we can all do it one day, with or without technology.
The intent is to represent a small-scale collaborative consciousness or presence, which can be accessed and ‘remixed’ or re-performed into new collective or universal narratives of living, being, experiencing, emoting, expressing, feeling, performing, and communicating via mobile media performance.
Project description From 2007-present SMARTlab was commissioned to undertake a project on somatics and the body as interface, now called MindTouch, in collaboration with Dr Marc Price of BBC R&D. Camille Baker is the researcher, Dr. Susan Kozel is an additional supervisor, and the technology team includes Michael Markert, Jeremi Sudol, Jana Riedel and Dr. Li Zhang. Performance collaborators at SMARTlab include: Bobby Byrne, James Brosnan and Chrissie Poulter. The project in progress brings together diverging areas of new media research and media art/performance practices, in terms of use of biofeedback sensors interfacing with mobile phone technologies in a unique way.
Embodied, non-verbal communication and interaction is explored, using wearable biosensing devices as interfaces to mobile media experiences, within participatory performance events. In addition, a small, curated exhibition event is being produced for the Fall 2008, including works from other media performance artists, collecting responses on their experiences and those of their audiences, as well as similar performance media projects to date, for a contrasting sources of data, to compare and contrast them with my own project. The effectiveness of my own facilitation or enablement of sensing liveness and presence in these contexts is also studied.
348
Participants in the first phase video collection workshops are asked to share internal images, feelings, thoughts, and impressions using the phone in a non-verbal, visual way, using the mobile video recording cameras and online video editing tools such as www.eyespot.com, then share these with others. This is collection of personal clips is then to be used later in a collective collage/performance of this experience. During the video collection process of the project, participants are asked to explore their own: consciousness, dream states and relaxation; non-verbal emotional and affective senses and states; internal sensations and embodiment; inner feelings, pondering, internal conversations expressed through abstractions; reflections, either literal mirror images or more intangible manifestations; representations of experiences of their immediate surroundings; external (spatial), eternal (temporal) and psychic (non-tangible) worlds, as well as mundane routines versus profound realizations or epiphanies.
The media performance aspect of the project involves staging various mobile media event(s) that utilize a database of archived of these streamed and/or archived video clips, created by the video recording mobile phones, then retrieved, streamed and remixed during live visuals performance(s). The phones in this project act as controllers, or essentially as the computer processor, as well as a receiver and visual performance mixer taking the biofeedback data from the body via the biofeedback sensors.
Participants are invited to wear the biosensors and perform or video jockey (VJ) the visuals live, creating the unique ‘mixes’ from a video archive. Various groups are invited to participate, from party-goers, to Tai Chi practitioners and meditators, to groups with various types of communication disabilities, from deaf/dumb to cerebral palsy and locked-in syndrome, to children’s groups, to the general public – to see what types of communication exchanges and collaboration experiences they might have.
Figure 1: © 2006 C. Baker – Early technical diagram of the networked performance project.
VJing mixing techniques, using software on the phones, instigate a mobile means to collage the video expressions in a live performance setting, then the video clips are mixed and manipulated live (or as near to live as we can), accessing a remote database. This software reads body data (breath flow, heart rate, skin response), from the biofeedback
349
sensors on a performer or participant in the audience. The live modified video/visuals are sent back to the phones of the in-person, or remote audience, like a conversation between the body/performer and the audience.
Thus, the body data controls the creative interactions of video mixing, displayed in the live setting, while remotely accessed on phones or Internet. This video collage is meant to become a collaborative, non-linear narrative or ‘remix’, ultimately to be globally streamed via mobile-casting, converging these previously distinct technologies and practices, and by bringing all the different presences together to collaborate on the visual mix, simulating ‘collective consciousness’.
This live video performance collage is the main performance event, combining the technological and experiential context of mobile videophone screens and possibly larger screen, with the body-data responses taken from the biofeedback sensors – in a continuous loop or conversation between the body(ies) moving in the live space and the local and remote audience/interactors.
Body states, body interface, affect and embodiment
In my conceptual explorations and research of ‘embodiment’, the mind and body are seen as one, encased within the skin, containing bones and organs, blood, the brain and other organs, fluids and tissue. Rettie (2005) believes that embodiment is consciousness itself1 or an ephemeral essence, rather than physical quality. My work is an enquiry into how embodied experiences could be embedded into media art practices, as well as how they could be experienced in the body, to access memory and altered consciousness. The assumption for me is always that the body is the integral site of the mind, so the experience must take place within the body. This research intended to return to the body for interpersonal exchange, wisdom, transcendence, and exploration, to immersively attune people to their own bodily sensations to harness the ‘virtuality’ of sensation, imagination and a dreamlike imaginative state.
In aim is to understand how people can find alternate, embodied ways to ‘connect to each other’, wireless biofeedback sensor technology is used to assist as the embodied ‘interface’ with mobile devices. From there, it might be determined whether such an interface can simulate or facilitate the telepathic exchange and what we can learn from the body-as-interface in the exchange, possibly serving as a means to interface with technology or as a conduit for unique social interactions.
Body data, such as breath, blood pressure, heartbeat, muscle electricity, stress and sweat
and other bodily functions, impulses and behaviours are used to interact with mobile software, then be interpreted, or interacted with, as the conduit for creative play and expression.
Massumi’s Parables for the Virtual: Movement, Affect, Sensation (2002) is relevant in
terms of presenting theory on affective, sensory aspects and issues, when dealing with sensors and the embodied exploration of art and performance. He focuses much attention on
350
perception itself, perception of sensation, and the concept that the body is always in a state of motion and potential.1 His analysis, in reference to other prominent thinkers and scientists, is grounding and fundamental for work on the body, simulated telepathy and mobile media performance, not to mention the visceral/embodied experience of performance and liveness. Massumi’s definition of ‘affect’, based on the theories of Deleuze and Spinoza, is a rich area of thought. It states that:
Affect... as couched in its perceptions and cognitions... implying a participation of the senses in each other: the… ability to transform the effects of one sensory mode into those of another (tactility and vision… interoceptive senses, especially proprioception, are crucial)…Formed, qualified, situated perceptions and cognitions fulfilling functions of actual connection or blockage are the capture and closure of affect. Emotion is the most intense... expression of that capture. (Massumi 2002: 35)
Massumi’s ideas of sensation, potential, becoming and movement are critical to embodied mobile media and sensor artwork. His notion of virtuality will be explored more in future writing as well. Presence perspectives Philip Auslander suggests that presence has an emotional impact, but is ephemeral and evanescent an ‘intangible expression’ that ‘only exists in the transitory present moment’ (Auslander 1999: 132). While agreeing with this statement, my notion of presence is also a physical, conceptual, phenomenological sensibility almost akin to a vibration or physical agitation. Presence can be felt, viscerally and intuitively. It is the feeling or sensation of another person and their engagement in some activity or with someone. It is when one has engaged actively, emotionally, intellectually, playfully, or is otherwise involved in an activity or experience with others and can feel them also engaged in the experience.
According to Shusterman (2000), presence can also mean, to be physically present, in-person, in bodily attendance, in a physical space. Yet, for me, it is about an embodied engagement and the felt or intuitive sense of others, in time and/or space (physical or virtual), it has an emotional quality that transcends physical attendance, including time but not necessarily space.1 Presence for this research is concerned with embodiment and of feeling others on an intuitive level, as well as emotionally and physically, but not necessarily in-person, like one feels another’s presence when speaking to them on the telephone or across the table.1
The sense of presence is critical in my attempt to tap into the visceral, bodily expressions
that surface during body practices; Tai Chi, yoga, dance and meditation for example, as a primary source of non-verbal, pre-conscious communication. Physical presences and sensations then are intended to be transformed or transduced into a digital form and used to ‘touch’ and ‘play’ with others, remotely or non-locally, using their mobile media devices to represent their presence in a live networked context.
351
Presence experienced in the alternate state of consciousness of dream or pre-sleep is rather elusive to define, but could be described as the permeable, thick viscosity of a fog, where images, sounds and people just appear and whose presence can often be felt, while awareness is able to transition easily between wakefulness or sleep.1
Pierre Lévy’s1 views on the concept of virtuality are compelling in terms of ‘real’ versus ‘virtual’, actual versus ‘possible’, how the body, text, economics, and culture are virtualized and what that means philosophically and practically to modern society. He posits that, based on the Greek definition of virtualis that current philosophy adopts, the virtual is about potential rather than actualization. Yet he believes that ‘virtual’ has come to mean absences of existence rather than tangibility (Lévy 1998: 23). He focuses on how virtual collective intelligence on the Internet has positive potential for freeing social activity, as well as enhancing knowledge production and dissemination. This aspect of collective intelligence drives the concept of mixing of the mobile media in a live performance setting, using a multi-interaction approach. Lévy also puts a great emphasis on distinguishing between ‘virtual senses’ and ‘real senses’; however, it is my view that these may not be so divergent, as we engage the physical and affective when experiencing both virtual and real senses. My experience of virtual presence is less technological. My interpretation and experience of ‘virtuality’ is that it can be accessed through the imagination or the dreaming body, conveyed from within an altered state, expanded sensory awareness, perception, imagination and daydream imagination, or extrasensory perception. This is the very embodied, ‘immersive’, almost liminal space of the pre-waking/sleep state. It also includes embodied perceptions of reality, which create our natural, invisible, inner worlds and consciousness, the real experiences within an altered dimension, or dreams, psychic experiences, mediation, mysticism, near-death experiences, intuition or vision quests involving physical pain (Baker 2004: 33).
When discussing presence, it is valuable to also address the concepts of ‘absence’ and ‘the
space-between’, in order to clarify the concept and experience of presence and its evolving definition. This can only explained this sense of virtual absence in my own experience of it, which includes sudden absences of virtual contact, where once there were previously very active, almost daily, exchanges. This kind of change in the interaction can be disconcerting and create an affective sense of lack, void or silence, which was experienced emotionally and viscerally as a disconnect. Hence, unlike when someone is in close physical proximity and geographically local, and you can go to their home, run into them in commonly visited places, or ask their friends how they are, with digital absence, one has no recourse: one is left dangling in the digital abyss, still feeling the void affectively. When a relationship involves great geographical distance – followed by intermittent, proximal, then in-person interaction, and often interspersed with long spells of merely virtual interaction – even ‘live’ text, voice or video chat cannot bridge or fix the fragile communication issues or misunderstandings in the same way that live interaction might. If the person on the other end does not respond, they can vanish without a trace. This is where it could be argued that there is a superiority of in-person liveness, in terms of conflict resolution and problem solving approaches to virtual interaction, which is worth exploring further but not at this time.
352
With these ideas in mind, there are multiple manifestations of feeling the presence explored, facilitated and observed of participants in performance research workshops, such as:
• The engrossed attention during a mobile phone exchange and the in-between space of the network connection.
• The physical presence of performers and their body data and the in-between space of the data being sent to the phone, followed by the in-between of the phone accessing the database of video clips.
• The possible telepathic presences during the moving meditation. • The captured emotional, aesthetic presences of the video performers within their
clips. • The physical co-presences of the audience members in the space of the stage events. • The virtual multi- or co-presences of the remote audience performers and the global
co-presences of these collaborators.
This conceptual exploration is critical to gain insight into what presence is and is not, in order to facilitate it in practical ways in my experimentation and performance events.
Liveness perspectives The sense or experience of ‘liveness’, originates from previous dance and music performances, and also from using digital technology. This liveness is intertwined with time and/or space, and physicality, or all simultaneously. Connotations of liveness1 are with literally being ‘in-person’, physically present, in the here-and-now, happening at this moment, in front of an audience in a traditional performance context, live or remotely (as with live TV broadcast or online event). In a digital context, ‘now-ness’ is critical, but being physically present is not: for example, one can use a computer with an avatar as a stand-in to substitute for actual physicality and representing a physical player, or use voice-over IP, or video conferencing to replace the physical presence, depending on the context–level of granularity, and so forth.
For many it is important that liveness involve a certain tangibility and tactility, especially
in the context of performance. Tactility and physicality are considered a feature or selling point of liveness according to Auslander, allowing audiences the intimacy of witnessing from a particular performer’s point of view, convincing them that they are somehow physically interacting with the performer (Auslander 1999). Many performing arts patrons desire physical proximity with the performer; they tend to report a sense of an immediate, intense, sensory and sensual reality. Auslander states that, ‘liveness is ideological...[the] belief that live confrontation can somehow give rise to the truth in ways that recorded representation cannot’ (Auslander 1999: 128, 129).
In this way, proximity during live, staged performances offers viewers smell, touch,
biochemical reactions or exchanges between performers and audience, or audience members amongst themselves. Proximity offers a ‘sense’ of emotion (not only visual), such as hearing small changes in the tone or quality of a performer’s voice, or subtle body shifts, conveying
353
emotion and immediate feedback, sometimes as subtle as a glance making eye contact – a viscerality (Auslander 1999: 85, Notes #35), like a touch.
Viscerality for me is an internally felt response in the ‘gut’, that is ‘a gut feeling/reaction’,
‘lump in the throat’, ‘butterflies in the stomach’. These are proprioceptive sensations of the internal body that include: hormones, pheromones, fluids and/or nervous reactions and responses triggered by emotional or other thoughts, affect or situations encountered by a person. Brian Massumi says that,
…visceral sensibility immediately registers excitations gathered by the five 'exteroceptive' senses even before they are fully processed by the brain…The immediacy of visceral perception is so radical that it can be said without exaggeration to precede the exteroceptive sense perception... viscerality subtracts quality as such from excitation. It registers intensity. (Massumi 2002: 60-61)
This liveness has viscerality that might be difficult to cultivate within the mobile network, but at least a viscerally felt embodiment might be possible,1 since it seems to take place during telepathic experiences,1 and as discussed, virtual presence can cause emotionally and affectively experienced responses and sensations in the body.
Understanding and identifying the effect, reception and experience(s) of liveness, aura, presence of the performer and the audience, in order to reveal their intrinsic embodiment within the network, within the construction of the events and activities, can only be accessed phenomenologically. Thus, only by receiving feedback from audiences and participant performers can one learn of their first-person impressions, and then analysis of these experiences can determine whether they felt or experienced liveness differently from traditional performances. 1
Participatory performance
Richard Shusterman1 explores the pragmatic, aesthetic experience of performance, suggesting that traditional audiences and art viewers feel that the art establishment has lost interest in trying to satisfy viewers’ need for affective, emotional, transformational or aesthetic experiences, and so they have gravitated to popular art and entertainment to meet these needs. All of these are important voices to ground investigations on mobile and networked performance practice.
Performance in my projects is more in the tradition of public participatory art or
performance art, whereby the performance is like a structured improvization, involving available, untrained participants, who are guided by trained performers in semi-structured activities. This form is used and intended as a way to explore the mobile phone and mobile media, as a means to re-engage people in performance and new media art, is aligned with mobile games as much as it is with new media and fine and performing arts.
My own notion and experience of performance has evolved from a traditional
understanding as a performer in music and dance, a media and performance curator, and as
354
audience member, to a more interventionist (Thompson and Sholette, eds. 2004), site-specific, performance art and now ‘mixed-reality’ or pervasive games1 and networked performance. My path is forged by fascination with new domains and experiences that help people to explore and share, which I am always seeking in my own life. This is a fertile, timely area for both new media and performance explorations.
Mobile media
The current notion of mobile media involves images, sound or videos created by, existing on, or received through mobile phones or other portable devices.1 In networked new media performance practices, four areas of practice have been identified by various media artists,1 some explored in various combinations.1 They are as follows and incorporating most of these practices into the work: 1. Telematic performance and events: telematics connect people to people or people to
objects through a network, such as live-video steams of performances or collaborations (or even performance tele-conferencing) across geographic distances.
2. Locative media or psycho-geography: locative media provides a location aware connection through GPS or mobile phones.
3. Wearable projects: wearables extend the body's senses through technological sensor devices.
4. Active objects and responsive environments: smart environments enable spaces and objects to respond to environmental changes of state, generated by people with within them.
Martin Sonderlev Christensen’s recent PhD thesis, As We May Feel (Christensen 2006),
thoroughly explores the affective qualities of mobile phones in the context of their recent multimedia creation properties, and the intimate, social uses that have been applied to them. This perspective is fundamental in its exploration of embodied, personal expression possible using mobile, wearable devices, rather than in merely geographical and locative ways. His ideas really ground mine in terms of the intimate nature of mobile phones and their appropriateness for intimate, embodied, participatory performance activities with the public.
Otherwise, theoretically and in practice, current works and research in the field of mobile
performance and locative media are few but growing, and many are locative or involving ‘mixed reality’1 or pervasive games. The approach taken by many locative media projects is to focus on urban space, environmental, ecological and geographical activities and games, and most do not reference traditional performance practices or studies, or performance history, but often come from a gaming or HCI perspective.
Although this project uses mobile phones, it is not as concerned with the space, location,
and geographic or psycho-geographic, mapping aspect of mobile devices in locative games and other similar projects1. The fascination here is more with the emotive, affective perception or embodied experience possibilities of mobile media, the ‘situatedness’1 (Idhe
355
2002: 68) of the person and their experience within and attentiveness to themselves. Also curious is participants’ individualized, visceral experience of the space, or locativeness within their own bodies, as opposed to situating them geographically, for example, the mapping of their own body parts or an abstracted ‘mapping’ of their internal thoughts and emotions, or their physical sensations. Participants are encouraged to share the intimate nature of a person describing their own process/practice of living or being. Alternatively, they might describe the experience of performing their lives, activities and thoughts, share them with others, and collect and collage this experience.1
Performance video/vjing The practice of VJing or video jockeying has been chosen as the methods of mixing video in a live context. Live performance video has its roots in the rave culture of the late 80s and 90s, bridging electronic music, new media installation, and experimental film and video, audio/visual or VJ performance is coming into its own as a media art form and practice.1
There are very few academic voices in this domain presently; however, a group called VJTheory have recently shed light on the art process and practice, as well as the dynamic nature of this visual performance. Stunning live performance works have come from the artists worldwide including Canadian artists nomIg, skoltz-kogan, yet many artists are now calling their practice ‘live cinema’, ‘audio-visual performance’ or ‘performance video’ to disassociate from the perceived negative connotations of VJs as just party visuals, (see Image 1 below).
Image 1: Still from nomIg live performance (permission granted by Stephanie McKay and Ed Jordan). 1
Another interesting development is that of Web Jockying: While the DJs and VJs of the world remain tethered to the remix of sound and image, WJs (web jockeys) have at their fingertips on an infinite and diverse pool of material--sound, image, text, code, web cams, blogs, and more--all constantly changing and expanding. A new software development, WJs, offers artists a tool to create live multimedia performances from this digital soup ... the world of artistic algorithms, and computer art in a live performance...1
This has tremendous potential to be used with mobile platforms as well.
Related to live video performance is non-linear, non-narrative, ambient or generative video, as well as software and database systems developed for them. Many artists and filmmakers are finding new ways to generate narrative or non-narrative cinema online, using
356
computer databases and programming to display random film clips and audio, online, for ambient home backgrounds, and in installation artworks. In this way, they are always creating something new and ‘live’.
These two types of nonlinear, ‘live’, generated and performed video forms are compelling
ways to ‘perform with video’, which is what motivates my ideas around using the style and form of VJing practices in my project but involving the mobile phone. The MindTouch project does not adhere strictly to the form of VJing more commonly practiced in clubs and parties and does not incorporate music or sound at all to drive the video mixing, which is fundamental to the art of VJing. Instead it uses the body ‘rhythms’ and data to drive the mixing. Still, it is important to acknowledge VJ practice as an influence, since this project appropriates its form and some of its style or visual language to construct the collaborative collages.
Combining VJing or live video performance mixing, within the technological and
experiential context of mobile phone screens, triggered by body data, is the project means to emulate the sharing of dream imagery. This is a key aspect of the work and a way one can imagine a telepathic exchange of two or more bodies, lucid dreaming visually together, while using the current available technology to simulate the concept and at the same time create a live and collaborative experience that puts the VJing in the hands of the participants.
Participatory mobile media artifact collection Participants of the video collection workshops that have been conducted are asked to express themselves using the multimedia tools of the mobile phones – either using their own videophones or ones supplied.
Low-grade video/media on mobile phones are valued for their immediacy, poor image quality and pixilated imperfectness with its own aesthetic quality, but also used for the innate encouragement of spontaneity, speed of thought, including: rewriting, superimposing, remixing of ideas, flashes, clashes of images and emotion, layering of meaning and stream of consciousness; or simulation of telepathy/collective, if chaotic, intelligence.
Five video clip collection workshops have been conducted to date, each of four to eight participants. They were conducted in Vancouver, Canada (twice), Dublin, Ireland (twice) and London, England. These workshops have been successful, if in smaller groups than anticipated, but this has proved useful. Each workshop to date has been different due to the number of participants showing up, the available space and date to conduct them, and the environment each has offered in terms of visual materials or fodder for participants to work with.
The first workshop in Vancouver (16/06/07) was set in a big centre with a huge gallery space, many workshop rooms, a big black box theatre, a gymnasium downtown next to the water. The initial mind-quieting activities took place in one of the workshop rooms and then after two individual video-collecting activities of 5 to 10 minutes each, and two were conducted in pairs. In one activity, participants were asked to ‘introduce themselves’ non-verbally, using the videophones, to each other. The image below (Image 2) presents how participants decided to approach the activity, recording parts of themselves and then
357
showing the recordings to each other, as a way to introduce themselves. In the last activity, participants were asked to work with their partner, non-verbally, to create something to send to someone else, telepathically or through the mobile phone, something which would express their inner perceptions and experiences. As a result, several participants went outside and each pair developed a different method to carry out these pair activities and each outcome was fascinating; one of the results can be seen in below.
Image 2 and 3: © 2007 C. Baker – Stills from participants’ videos from Vancouver workshop June 2007.
The second workshop took place in Dublin at Trinity College (22/07/07). The space for this second one was in a dance studio and the participants had more room to move around, and, since a couple was dancers they tended to move more, which inspired the others to do so as well. The same four activities as in Vancouver were facilitated, but were video recorded and I was able to observe the participants and assist when they were having trouble with the phones or needed more instruction on the activity. This group was more exploratory with their video expressions and also had one participant with cerebral palsy and two helpers/participants to work with him.1 Participants were asked to interview each other with the videophones, rather than having one interviewer for each of them individually to save time. This proved to be successful, as they seemed to develop a rapport with each other, shared similar experiences, and thus, seemed were more open to discuss them with each other them.
358
Image 4: © 2007 C. Baker workshop still Dublin July 2007.
The third workshop took place in Vancouver (21/08/07). Participants met in a waterfront park nearby on a beautiful sunny, hot day, which added to their enjoyment of the workshop. It was a much smaller group of four, thus the first two individual activities were somewhat more difficult to conduct in such a public space, causing participants to seem somewhat self-conscious performing the ‘internal’ and mind-quieting activities, with others in the park were throwing Frisbees and dogs were running about around them. One woman brought her child with her, as such her focus was reduced, making it harder for her to completely relax. Yet she attempted to do the activities with her child as to the best of her ability in this circumstance. The bright sunshine and inspiring surroundings resulted in the video imagery from this workshop being quite lovely.
Image 5 and 6: © 2007 C. Baker – stills from participants’ videos from Vancouver workshop August 2007.
The fourth workshop was again in Dublin, at Trinity Collage again in October (26/10/07). Like the other workshops before it, there was mix of new people from various backgrounds, attending out of interest. Some participants were quite imaginative and had unique approaches to the activities the ways they used the space and so forth, many looking for patterns from the structural environment, away from the group at times. Participants had access to several spaces, thus the individual exercises were done in one studio and pair exercises in a large black box performance space and outside, as it was an unseasonably warm fall day.
359
The images from this group were stunning and exceptionally creative in the way in which
the participants used the camera special effects to invert the image, extreme close-ups and colours to further abstract the body parts being recorded. The process was quicker and the activities were improvized somewhat, thus a new activity was introduced involving partners sitting back to back, trying to ‘telepathically’ send an image or sensation they had to one another, then attempting to visually capture what they ‘sent’. Each would then guess what the other person had sent, and then they would share their videos with each other to see if all three versions (the telepathic, the videophone and the guess) were the same. This activity was enjoyed by most, to be used again in future workshops, as it garnered more captivating visual explorations and creativity from participants.
Image 7, 8 and 9: © 2007 C. Baker - stills from participants’ videos from Dublin workshop October 2007.
Each of the workshops involved different types of participant (not just because of the country location, but that might factor in) and different types of environments. The surroundings became a large actor in the content that participants created expressively in their explorations within. This was possibly both because of the comfort level of participants, and what types of visual material they could work with in the vicinity to externalize and represent their inner explorations. This became a more significant factor than intended and has implications upon how one can externalize with the videophone and was influenced by the surroundings, since its difficult to replicate our inner eye’s view if the immediate environment provides contrasting or unique visual fodder unlike our own internal experience and imagination – more limiting and uncontrollable, yet a stimulus for intriguing explorations.
The process of conducting the workshops has been very fruitful, in clarifying my aims of the process, through observation, exploration and iteration, and studying the responses from participants in their interpretation of the activities and video expression/collection process. My ability to guide participants effectively has also developed greatly, in order to assist participants to have meaningful experiences in creating intriguing personal media expressions.
The workshops have become an end in themselves, as many groups have asked for practical engagement with these personal media empowerment tools (often for less technologically literate groups and individuals). Since mobile media is still a relatively new format, this hands-on workshop engagement, with a human touch and an intimate form of communication, has touched participants both intellectually and creatively. Thus, this stage of the project has garnered unexpected results.
360
System design and software development The research commenced in November 2006, with acquisition and work with the biofeedback hardware, software and the mobile phones as appropriate for the research.
The biofeedback sensors acquired for use in the research are be EMG (electromagnetic), GSR (galvanic skin response), respiration and BVP (blood volume or pressure), as they are the most responsive, easiest to work with in terms of generating usable data, and not uncomfortable to the person wearing them. Electromyogram measures muscle electricity to determine muscle tension of any larger muscle, like arms and legs. Galvanic skin response is work on the fingers and measures the electrical conductance or resistance of the skin that can change when people change their stress levels. Respiration sensors are worn around the ribcage and monitor the inhalation and exhalation depth and frequency based on the abdominal or chest expansion or contraction. Blood volume monitors the relative blood flow in the fingertips, using near infrared light, and from the pulse or blood pressure the heart rate can be determined. These sensors are attached to the body via sticky electrode pads or Velcro attachments (for image source of sensors, see http://www.mindmedia.nl/english/sensors.php).
Wireless sensors are desired in order allow for ease of movement. Starting the project we
were unable to get sensors from the preferred company, since they were unable to complete their line of biofeedback sensors by the start of the research. So much more expensive and more sophisticated medical grade sensors, than needed, were purchased, with their heavier and more restrictive cabling on the body. However, after some frustration in their use, we more recently abandoned them in favour of creating our own cheaper and more customizable sensors.
The sensors initially used were high quality, but the short, thick cables restrict movement
and are visually unsightly for performance purposes, unless hidden in customized clothing. The system also came with simplistic software for clinical stress reduction, inappropriate for this project in terms of the visualization of the body data. Thus, because of the time lost in miscommunication with the manufacturer, and in order to develop our own customer software for the mobile phones, we have changed our approach to working with sensors.
Most recently, the German electronic musician/creative electronics/java programmer
Michael Markert has been working as a collaborator on the project, assembling and creating biofeedback sensors to work with a wireless Arduino system, to embed them into some customized clothing in order to be flexible for unencumbered movement for dance and Tai Chi, as well as to be ubiquitous or unseen on the body, so that is less obvious the ‘body area network’ sensing system in performance contexts.
361
Image 10: custom biofeedback visualisation software.
We are now creating additional mobile live VJ software for streaming video, accessing the database of clips, for editing, mixing and add live effects to this video in real time and then allow audience members to share and interact using these visuals as a communication exploration, as well as working on embedding the custom sensors into lightweight, adaptable garments. Conclusion This article has described the practical aspects of the media art PhD work, new media performance project called MindTouch, as manifested in the initial participant performance video collection workshops conducted last year, intended to be used with performers wearing biofeedback sensors that interact with mobile devices during live events later this year. Some of the workshops, initial observations and outcomes, as well as some of the technological difficulties of working with biofeedback sensors have been outlined.
The theoretical notions on liveness, presence, non-verbal communication, telepathy, performance and mobile media explored herein are integrated into the more practical implementation of the workshops, custom mobile software and performance events, in terms of what is important to ask participants to focus on how the software is developed and what its meant to do, and how the participatory performances are guided and staged. The intricacies of the interweaving of theory and practice will be further explored at later stages of completion.
Once the research and the video software is completed, the sensors garments are made and
a few scratch performances are staged in coming months, more will be known of nature of the presence and liveness being experienced by participants. In addition, more will be known in terms of what is meaningful in the body data from further work with meditators and various groups of ‘performers’, and its application for performance media events.
The goal is that interfacing technologies will assist in connecting people remotely and
allow them to re-engage with each other affectively and expressively in new non-verbal/textual ways. Thus, this project brings together diverging areas of new media research and media art/performance practices, in terms of use of biofeedback sensors interfacing with mobile phone technologies in a unique way.
362
The unique knowledge contributed by this research is a body-situated, theoretical
perspective of the experience of creating a participatory mobile media performance. There will be a contribution to the understanding of individual and collective experience of liveness in performance within the virtual ‘space’ of a mobile, networked performance. More will be known about the perception and experience, both participatory and observatory, of liveness and sensed perception (or not), within these performed, real-time, in-person and virtual domains, using mobile and wearable devices and interfaces.
References Auslander, P. (1999), Liveness: Performance in a Mediatized Culture, London and New York: Routledge.
Baker, C. (2004), Internal Networks: Telepathy Meets Technology in the DreamPod, Masters Thesis, Surrey, BC, Canada: Simon Fraser University, pp. 33.
Byrne, B and Carvalho, A (eds.) (2006-2008), ‘VJ Theory.net’, Falmouth, England. Online at www.vjtheory.org Accessed October 2007.
Christensen, M.S. (2006), As We May Feel: Interpreting The Cultural of Emerging Personal Affective Mobile Media, PhD Dissertation, Denmark: Department of Innovative Communication at the IT University of Copenhagen. Corino, G. (2006), Spatial Issues and Performative Media in Digital Mobility: A Network Perspective, Digital Futures, MSc thesis, Institute of Digital Art and Technology, University of Plymouth [Online], http://x.i-dat.org/~gc2/digital_mobility.pdf. Accessed 13 July 2006. Damasio, A. (1999), The Feeling of What Happens: Body and Emotion in the Making of Consciousness, New York, San Diego, London: Harcourt Brace and Company, pps. 113-125.
Idhe, D. (2002), Bodies in Technology, Electronic Mediations, Volume 5, Minneapolis, Minnesota: University of Minnesota Press. Jaeger, T. (2005), Live Cinema Unraveled: Handbook For Live Visual Performance, San Diego, California: Self-published, with funds from UCSD Russell Grant. Jamieson, H. Varley. (2006), online at http://rhizome.org/news/story.php?timestamp=20051121, accessed April 2006. Kozel, S. (2007), Closer, Cambridge, Massachusetts: The MIT Press.
LaBerge, S. and Rheingold, H. (1990), Exploring the World of Lucid Dreaming. New York: Ballantine Books.
Lévy, P. (1998), Becoming Virtual: Reality in the Digital Age, New York: Plenum Press. Massumi, B. (2002), Parables for the Virtual: Movement, Affect, Sensation, Durham and London: Duke University Press. McKay, S and Jordon, J. (2006), http://www.nomig.net/. Accessed July 2006.
363
Pearson, C. (1992), ‘Precognitive Dreaming’, in A Public Experiment in Precognitive Dreaming, Presented at the annual conference of the Association for the Study of Dreams (1993) and The Pittsburgh Post-Gazette Dream Experiment (1994), http://www.nauticom.net/www/netcadet/npcg.htm. Accessed 10 March 2003.
Radin, D. (1997), The Conscious Universe: The Scientific Truth of Psychic Phenomena, New York: HarperCollins Publishers.
Rettie, R.M. (2005), ‘Presence and Embodiment in Mobile Phone Communication’, PsychNology Journal, 3: 1, pp. 16–34, http://www.psychnology.org/File/PSYCHNOLOGY_JOURNAL_3_1_RETTIE.pdf. Accessed 10 June 2007. Richardson, I. (2005), ‘Mobile Technosoma: Some Phenomenological Reflections on Iterant Media Devices’, Fibreculture Journal, Issue 6: Mobility, New Social Intensities and the Coordinates of Digital Networks, http://www.journal.fibreculture.org/issue6/issue6_richarson.html. Accessed 10 June 2007. Sadler, S. (1998), The Situationist City, Cambridge, Massachusetts: The MIT Press.
Schechner, R. (1988, 2003), Performance Theory, London and New York: Routledge. Shusterman, R. (2000), Performing Live: Aesthetic Alternatives for the Ends of Art, Ithaca and London: Cornell University Press. Simpson, J. and Weiner, E. (1989), Oxford English Dictionary, 2nd edn., Oxford: Clarendon Press. Thompson, N and Sholette, G. et al (eds.) (2004), The Interventionists: Users Manual for the Creative Disruption of Everyday Life, Cambridge, Massachusetts and London, England: The MIT Press.
Ullman, M. and Krippner, S. (1973, 2002), Dream Telepathy: Experiments in Nocturnal Extrasensory Perception, Charlottesville, Virginia: Hampton Roads Publishing Company.
Various authors (2005a), Thorington, H. and Riel, M. (moderators), from soft_skinned_space -empyre- listserve topic: ‘Networked Performance’, http://www.subtle.net/empyre/. Accessed 10 June 2007. WJ-S project, online at http://www.wj-s.org/. Accessed July 22, 2006.
——(2005b), ‘networked performance blog’, An Open Online Forum, Topic: Network-Enabled Performance, http://www.turbulence.org/blog/. Accessed July 2005.
364
Contributor details
Camille Baker is a Ph.D. Candidate at the SMARTlab, University of East London, conducting research on Networked Performance Media, with funding provided by BBC R&D. Her research interests include: mobile devices, video art, live cinema, performance/interactive media, responsive environments and installation, telematics, new media curating and networked communities. Contact: Camille Baker, MASc. Interactive Arts, the SMARTlab Digital Media Institute, University of East London, Docklands Campus, 4–6 University Way, London. E16 2RD, UK. E-mail: [email protected]
365
A.6.2/ aesthetics of mobile media art (July 2009)
INTELLECT: Journal of Media Practice http://www.intellectbooks.co.uk/journals/view-Article,id=8520/
Aesthetics of Mobile Media Art Camille Baker, University of East London, Max Schleser, University of Westminster and
Kasia Molga, University of East London The current state of mobile media involves images, sound or videos created by, existing on, or received through mobile phones or other portable and telephonic devices, and includes mp3 players, portable digital assistants (Blackberry, Palm Pilot, etc), iPods and other such devices, sometimes using GPS or RFID components as well. This article argues for the existence of aesthetics unique to the mobile media. To examine what those aesthetics are, the mobile media’s specific qualities of immediacy and intimacy, afforded by mobile devices, are explored and expressed by three media artists herein, via mobile documentary, mobile performance video and visual fine arts. The following reflects each artist’s experience and reflections in turn, starting with Max Schleser, then Camille Baker, then Kasia Molga.
MOBILE SCREEN AND DOCUMENTARY FILM AESTHETICS The mobile phone can be seen as a viewing device for micro-movies [i]. In the age of High Definition, the mobile phone has introduced a new standard at the other end of the cinematic spectrum. Lisa Gye writes in Picture This: The Impact of Mobile Camera Phones on Personal Photographic Practices:
Camera phones are not, however, just another kind of camera. Located as they are in a device that is not only connected to the telecommunications grid but that is usually carried with us wherever we go, camera phones are both extending existing personal imaging practices and allowing for the evolution of new kinds of imaging practices (Gye, 2007: 279).
Although mobile media can be translated between different screen formats and viewing experiences, this translation should not be considered as a one-fits-all approach, but rather as having distinctive parameters according to each medium’s specificity.
Mobile phone moving image productions can be situated in the field of experimental documentary filmmaking and can, therefore, be categorized as ‘mobile-mentary’ productions. This new category is in its early stages of development and further technological progress will continue to shape this mobile-mentary category. The category can be defined through the characteristics of an original aesthetic signified by pixilated video images. Thus, it is the mobile phone’s limitations that are the defining pattern for the establishment of this new format. Writing in High Technê, Rutsky outlines the ‘aesthetic’ dimension within the contemporary definition of technology:
366
It is not only the conception of technology that has changed, but also the notion of aesthetics. The aesthetic can no longer be figured in traditional terms of aura and wholeness, nor in the modernist terms of instrumentality or functionality. Like technology, it too comes to be seen as an unsettling, generative process, which continually breaks free of their previous context and recombines them in different ways (Rutsky, 1999: 5).
Aesthetics are here understood and analyzed as the mediation of an experience. Within this experience, which creates an original aesthetic, the significance of location is manifest in the mobile-mentary practice and encouraged by the mobility of the device. This notion can be exemplified through one of the most common phrases in mobile-phone conversations ‘Where are you?’ or ‘I am at…’ These phrases emphasize the importance of location within voice communication, which transcends into the field of filmic communication.
This London based (mobile phone) filmmaker refers to the new emerging mobile screen aesthetic as Keitai aesthetic. Keitai is Japanese meaning ‘hand-carry, small and portable, carrying something, form – shape or figure, mobile phone’ (Flaherty, 2007). The Keitai aesthetic is related to the mobile experience in three ways. First, on the visual level, it is expressed through the digital pixel compositions. In mobile phone filmmaking, the period between 2005 and 2008 is characterized by the advancement from the 3gp mobile phone video file format to the mpeg4 compression format. The city film Max with a Keitaiii, which was produced entirely on a mobile phone in Japan in 2006, depicts the new emerging pixel aesthetics. The experimental feature documentary explores the Japanese megapolis of the Taiheiyō Belt [iii] through the lens of a mobile phone. It juxtaposes the advanced Japanese mobile video format, which is based on the mpeg 4 codec, with the former 3gp compression format. The result is visually distinctive as can be seen from the images below.
Max’s images (max 1, max 2, max 3) Second, the Keitai aesthetic is expressed in the way mobile phones have impacted
body language, and hence the way the body experience is incorporated in the screening and viewing process using mobile devices. The mobile phone ‘has shifted electronic eyes from desktops to the palms of our hands’ (Mitchell, 2006: 175). Writing in New Philosophy for New Media, Hansen (2004) correlates the aesthetics of new media with a strong theory of embodiment. This experience can be understood as a form of mobile presence, which Ingrid Richardson (2000: 5) calls the technosoma of mobile phones, or the incorporation of the mobile devices into our evolving corporeality, both through the nature of the device itself, as well as its uses.iv As Hansen argues, ‘[t]he image can no longer be restricted to the level of surface appearance, but must be extended to encompass the entire process by which information is made perceivable through embodied experience. This is what I propose to call the digital image’ (2004: 10). He states that contemporary media art has effected what amounts to a paradigm shift in the very basis of aesthetic culture: a shift from a dominant ‘ocularcentric’ aesthetic to a ‘haptic’ aesthetic rooted in embodied affectivity (Hansen, 2004: 11). Through this new sensorium, new media experiences emerge, witnessing ‘the capacity
367
of the body to experience itself as “more than itself” and thus to deploy its sensorimotor power to create the unpredictable, the experimental, the new’ (Hansen, 2004: 6).
Whether one can generalize this account to the extent argued by Hansen is one question, however I would like to adapt ‘the unpredictable, the experimental and the new’ characteristic to the works produced for and by mobile devices. The mobile device incorporates the haptic notion as one touches the device actively to record/playback an audio-visual media file: As Greek etymology tells us, haptic means ‘able to come into contact with’. As a
function of the skin, then, the haptic – the sense of touch – constitutes the reciprocal contact between the environment and us. It is by the way of touch that we apprehend space, turning contact into communicative interface (Bruno, 2008: 23).
This physicality is a characteristic of the new media in general. Manovich writes that the new media
…change our concept of what an image is – because they turn a viewer into an active user. As a result, an illusionist image is no longer something a subject simply looks at, comparing it with memories of represented reality to judge its reality effect. The new media image is something the user actively goes into, zooming in or clicking on individual parts with the assumption that they contain hyperlinks…(2001: 180).
However, with the mobile device, one is now able to (hyper)link any location to any (new media) image. The mobile device works against the tendency ‘of digital imaging to detach the viewer from an embodied, haptic sense of physical location and “being-there”’ (Lenoir in Hansen, 2004: 2). This allows the audience/viewer to identify with the location. In this sense, Keitai can be seen as characterized by
…its colonization of the small and seemingly inconsequential in-between temporalities and spaces of everyday life. … keitai connectivity is a membrane between the real and the virtual, here and elsewhere, rather than a portal of high fidelity connectivity that demands full and sustained engagement (Ito, 2005: 14).
Finally, the Keitai aesthetic is connected to the qualities of a state of ‘in-
betweenness’. In-betweenness here refers to how mobile media operate in-between photography, video and the internet, while simultaneously establishing new links [v]. The mobile phone merges communication and lens-based media. It can connect different online environments, agents and avatars [vi] in any location, so that ‘the space in which communication happens is no longer physical or virtual; it is a hybrid’ (de Souza e Silva, 2006: 30):
Hybrid spaces merge the physical and the digital in a social environment created by the mobility of users connected via mobile technological
368
devices…Mobile and portable interfaces are embedded in physical space, promoting the blurring of boundaries between physical and digital spaces (de Souza e Silva, 2006: 19).
The mobile device is a case of media converging in a spectacle of information/data, where the division between the media forms seems blurred, but where the location of consumption emphasizes the original experience. Mobile devices push the media experience into a new domain.
MOBILE VIDEO AND PERFORMANCE AESTHETICS As a video artist performer and new media curator, who also teaches film and video
history, aesthetics and practices, with an interest in VJing and Live cinema, I have been long interested in trying to use mobile devices to marry performance and video. The discoveries I have made of mobile screen aesthetics come from research specifically examining mobile video in a performance or participatory performance context, and revolve around the ease of expression in capture, the intimacy and affective nature of the devices, highlighted by Christensen (2004), the gestural qualities implied by the size and mobility of the device, and the accessibility of mobile phones for all. The project discussed below is a sponsored project called MindTouch.[vii]
Theoretically, and in practice, current works and research in the field of mobile media and performance activities [viii] are few but growing, and many are locative media, ‘mixed reality’ [ix] and now called pervasive games. However, of interest to me is the emotive, affective perception or experience of performers and participants of these projects, as well as their individualized, visceral experiences, rather than the psycho-geographical (Corino, 2006), or Situationist nature (Sadler, 1998) or exploring the urban condition, or those of a social geography which has been so popular in recent years.
The fascination with mobile phones in the MindTouch project is more with the embodied possibilities of mobile media. What is of interest to me is the ‘situatedness’ [x] of the user in their bodies and their experience within and attentiveness to themselves, rather than the landscape or the world. As such, in addition to liveness and presence, also of interest is the individuals’ experience of the space or state of being located within their own bodies. In this sense they become more aware of and attuned to their own embodiment and can perceive and locate their own sensations. For example, how people map their own body parts or an abstracted ‘mapping’ of their internal thoughts and emotions, or their physical sensations. Alternatively, non-verbally illustrating the experience of performing their lives, activities and thoughts, then sharing them with others visually, through the collection and collage of this experience.
Performance in the MindTouch project is more in the tradition of public participatory art or performance art, whereby the performance is more a structured improvisation involving available, untrained participants, who are guided by trained performers in semi-structured activities. This form is used and intended as a way to explore the mobile phone
369
and mobile media as a means to re-engage people in performance and new media art, and is aligned with mobile games as much as it is with new media and fine and performing arts. My perspective is also that it is necessary to explore key thinkers in performance theory in order to understand the basis for new directions in performance practices, and to discover whether there are any new insights to be found. In this way, Richard Schechner (1988 [2003]) is a definitive voice on performance, as he challenges previously held views in this field, demonstrating the existence of performance in everyday life and practices, such as in play, sports, legal proceedings, rituals, and pop culture. Philip Auslander (1999), on the other hand, focuses on how liveness is in opposition to mediatization in performance, claiming that liveness does not exist outside the context of mediation. While philosopher Richard Shusterman (2000) explores the more pragmatic aesthetics of the experience of performance, suggesting that traditional audiences and art viewers feel that the art establishment has lost interest in satisfying viewers’ need for affective, emotional, transformational or aesthetic experiences, and so, he claims, they have gravitated to popular art and entertainment to meet these needs. However, my own notion and experience of performance is evolving, moving from a more traditional understanding as a performer in music and dance, a media and performance curator, and as audience member, to a more interventionist, site-specific, performance art and now ‘mixed-reality’ games (another name for locative media and psycho-geographic and networked performance). The mobile media phones used in the MindTouch research are the medium or conduit for self-expression, especially to convey non-verbal experiences and sensations as an extension of the body/mind [xi] in both non-literal or abstracted ways, as well as more direct representations. Phones are explored as vehicles or methods to express thoughts, perceptions and emotions to friends and between participants, simulating telepathic exchanges, like exploring and sharing a dream. Martin Sonderlev Christensen’s recent PhD thesis, As We May Feel, thoroughly explores the affective qualities of mobile phones in the context of their recent multimedia creation properties, and the intimate, social uses that have been applied to them. This is an exploration of embodied, personal expression made possible using mobile, wearable devices. For Christensen,
[a]ffect is expressed through human actions and constructed in the threshold between ‘our body’ and everybody that is the others’ bodies... Looking more specifically at affect, we might see it as… primal … a psychical and biologically given part of how we function as a physical being. Affect is in the body -- it is our readiness for the world, or readiness to act and it is on the body as expression or actions (2004: 12).
It is easy to find oneself engaging in this practice as well: ‘What can I capture?’, (looking more carefully), ‘What can I do with this moment/image?’, ‘Who shall I send it to?’ or ‘Where shall I put it, on Flickr or Facebook?’, ‘Who am I sharing it with?’, ‘Am I doing this for me or to finding a way to connect with others?’ Christensen provokes a compelling point, articulating something I was feeling and experiencing.
370
Mobile images are often experienced as personal, intimate, private, and once sent, the immediacy of them feels like a giving or blowing a kiss to another through the network or sharing pieces of yourself (depending on the content) (Christensen, 2004). This experience seems common. As with a text message, they are like thought transfers. However, with video, it is like sending your sight, your experiences, thoughts, feelings visually, but also your unique expression, insight and perspective. MindTouch explores embodied, non-verbal communication and interaction, using wearable biosensing devices as interfaces to mobile media experiences within the future planned performance events. In turn, MindTouch studies the effectiveness of the facilitation or enablement of sensing liveness and presence in these contexts.
Image 1
© 2007 C. Baker – still images by participants in video collection workshops
Participants in the first phase video collection workshops are asked to share internal images, feelings, thoughts, and impressions using the phone in a non-verbal, visual way, using the mobile video recording cameras, then share these with others. This collection of personal clips is to be used later in a collective collage/performance of this experience.
Image 2, 3 and4
© 2007 C. Baker - stills from participants’ videos from Dublin workshop, October
2007
Low-grade video on mobile phones is valued for its immediacy, poor image quality,
and pixilated imperfectness -- its own aesthetic quality -- but it is also used for the innate encouragement of spontaneity, speed of thought, including rewriting, superimposing, remixing of ideas, flashes, clashes of images and emotion, layering of meaning and stream of consciousness, and simulation of telepathy and collective, if chaotic, intelligence.
The key aspects that have been noticed in this research, through both observing participants and own use of the mobile media phone, are four key features:
5. innate performativity or movement/gestural qualities of device 6. portability factor (watching/shooting anywhere)/mundane is interesting 7. viewing world through camera/camera vision of the phone 8. intimacy/expressivity of device/predilection for close-ups
371
When participants are using and working with the phones, it seems that their immediate impulse is to wave them around in their hand and make the images blurred to see the patterns that result from this blurring, so it inspires a playful, gestural or performative exploration that I do not suggest but see people engaging in (I am inclined to do the same). Coming from a video art background this intrigues me, since using the older, bigger video cameras for more professional purposes, and even consumer home-movie camcorders, demanded shake stabilization and an effort NOT to blur the image. Yet a new shooting aesthetic may be unfolding from the mere size of the device and how it lends itself to movement, gesture and exploring the blurring effect. This is of obvious interest to me, since it combines the performance aspect with video and the mobile phone. Portability also encourages intimacy, since one can take their phone to bed with them, to the toilet, or to places where cameras are not allowed, and can communicate much more personal images and videos than standard video cameras enable due to their size, and thus encourage complexity in what they represent to the world. In my workshops, participants are asked to try to find imagery that represents, in an abstract way, how they are feeling or sensing, in a way that they could communicate to others. Dean Terry says about his work, mo.vid.1, a mobile video project from 2005:
The project is centered on the idea of remapping private spaces into public ones, of reversing scale, of inverting and rejecting the consumerist idea of ‘quality’ and its technological expression in ever higher resolutions by exploiting the limits of the devices. (The videos are really pretty poor with all the compression artifacts, and these limitations are exposed by using two Apple Mac G5’s to drive the system and two high resolution projectors to display it) (Terry, 2005).
This is key to how I work with video from my video collection workshops, allowing ordinary people to become artists by inverting what is considered ‘quality’ and using the limitations and pixilation as an asset or tool, rather than a hindrance. He also states:
The project also examines what it means to project very private space immediately surrounding the body into meta-space. Many of the videos show objects little more than a few centimetres beyond the tiny lens, often some body part, like hands or forearms that obscure an unknown, overexposed background space. Other pieces are gestural performances, recording the movements required when following a line, or when trying to create shapes by moving the camera in certain ways (Terry, 2005, my emphasis).
This seems a common approach to mobile video and what I have found myself creating. The device inherently encourages movement, and thus blurry, abstracted patterns and a ‘splattering’ effect. It affirms my theory that this intimate approach to mobile video
372
production is facilitated or afforded by the device itself. It will be a shame as the manufacturers inevitably are putting in more image stabilization and anti-shake technology to compensate for this. What has also become evident in my work is that the portability factor, which enables users to explore the world around them -- as Christensen suggests, with new eyes, as well as with the ability to watch and shoot anywhere, the mundane – becomes compelling: a way of viewing world through the camera vision or through the ‘lens’ of the phone. Christensen inspires one to think of the mobile phone ‘…as a window or microscope to life around us -- helping us to see ourselves in the world more clearly and communicate/connect this insight outward to others: to find the world and ourselves in it’ (Christensen, 2004: 11). Thus, we are becoming more engaged or reengaged with the environment, looking at the world around us more closely, and sharing our experience of it. People find themselves, as do I, shooting video or taking images of their world as they encounter it, and perhaps, seeing differently. The phone becomes a ‘window on the world’ that has users seeing it differently, or returning to truly observing the world in order to capture it, and then finding details that they might not have otherwise noticed (Christensen, 2004: 6-7). In this way, not only are they learning how to be amateur digital videographers, but they are also reengaging with and interacting with the world again, in a way they may not otherwise do. Christensen ties many elements together and bolsters the argument for mobile devices as affective, social and as important new new media tools and technological developments: ‘… mobile device technologies are reinserting the real world experiences into new media context ... offer[ing] locative and corporeal embodiment of technology...’ (Christensen, 2004: 11). During the FILMOBILE conference [xii] in April 2008, Steve Hawley presented his own work and discussed how, with the advent of HD and High Resolution video cameras and televisions, the world is now shown to us in larger than life ‘reality’ and quality, creating the illusion of somehow getting closer to the world, close to places or things we could never get close to or see clearly. However, he pointed out that, in this way, HD presents more of a spectacle of the world and still keeps this presentation firmly in the hands of the expert filmmaker or videographer, with access and training in this medium. The images and work using this high-resolution medium are usually the close up of creatures in the Amazon forest, in a National Geographic sense, or of huge vistas in the desert. Meanwhile, Steve Hawley implicated that, the mobile phone, with its poor, imperfection of resolution and pixilated quality and small aspect ratio is in some ways more ‘real’, in that everyone has access to the medium and can learn it easily, but more importantly it captures ‘what I am doing now, in this moment’, and has that personal immediacy that the imperfection of the image actually authenticates through its lack of production values--of high resolution, clean editing, and large aspect ratio. It is a more intimate and everyday life document that the everyday person can relate to and hence feels more ‘real’. Through its messiness, it at once encourages a more personal, non-expert/every-person, frank expression not possible with the HD video image.
373
MOBILE VISUAL ART AESTHETICS In an article written in 2006 and published in the proceedings of the EVA
conference, I concluded that one ‘could not (however) find a sufficient way of creating visual art forms exclusively for the mobile phone handsets’ (Molga, 2007: 16). This was mainly because, during the time that the article was written, even though mobile devices were advanced enough to carry visual content, MMS [xiii] as the most appropriate carrier of this sort of content was not popular enough among users. Moreover, handsets produced by different manufacturers were often incompatible, and sending or exchanging larger sized data files than a simple text message incurred much higher costs. The other important factor was the size and quality of resolution of the mobile phone screen.
However, during last two years, the mobile phone market, and the way people use it, have changed. Compatibility and convergence are the top marketed concepts: ‘No technology ever, not electricity, not the automobile, not the personal computer, not the internet, not credit cards, has had such a fast cannibalization of an existing industry’ (Ahohen 2004: 3). Small screen resolution is higher, nearly HD in dimension and format, although still tiny, and is expanding. Data sent and received can be larger and is cheaper to exchange. Thanks to Bluetooth, data can be exchanged free of charge with phones or other Bluetooth-enabled devices (computers, printers). ‘Illuminations’ is a body of work consisting of a number of art installations concerned with the various ways of adopting mobile phones in artistic expression as well as art experience. Dealing exclusively with visual content, informed by fine art practice, this work treats the mobile phone as a ‘portal’ between the audience, the artwork and the artist. In installations such as ‘Mirror of Infinity’ or ‘Breaking News’,[xiv] the mobile phone is an avenue of communication between a spectator and a visual art piece, exhibited in a semi-conventional way as an image on the wall of the venue.[xv] In ‘Little Heavens’, however I employed a mobile phone as the main platform of exposition of visual artworks, delivering a unique and intimate experience to the user. ‘LITTLE HEAVENS’
A couple of decades ago in Poland, children had a peculiar game called ‘Little Heavens’. The concept was to create beautiful collages from flower petals and pieces of colorful candy wrappers and place them under a piece of glass in a hole in the ground. The hole would be then covered by soil, leaves and grass to make it look like it was untouched. A child who made the collage would remember this location and then share it with their closest friends. Children with whom the secret was shared were let into the ‘circle of trust’. There were no winners in this game, however, children would compete with each other in the creation of the most beautiful collages. Sometime collages disappeared, dug out and destroyed by kids who belonged to more than one ‘circle of trust’ or by those making ‘Little Heavens’ on their own who wanted to get rid of the competition for the most beautiful creation.
In the art installation ‘Little Heavens’, I attempted to convey the meaning of this game, from the intimate hole in the ground to the intimate screen of one’s mobile phone.
374
Spectators will be able to send and receive fragments of text or images which then combined with my drawings will be send back in a form of the picture as an intimate token of trust. Before embarking onto the creation of this art installation, I had to examine the aesthetics of a mobile phone screen to ensure the best quality of art experience. THE MOBILE PHONE SCREEN AS A MEDIUM OF DISPLAY AND EXPERIENCE
The digital screen as a mean of display of works of fine art created with traditional techniques has not been met with too much enthusiasm among artists to date. Paul Zelensky, an artist and writer living in Los Angeles noted:
…basic problems endemic to viewing art on screen. For the artwork: the limitations of size and scale, the elimination of texture, materiality, shadow, reflected light; the absence of other images in the visible environment with which to measure and compare size, scale, texture, and so forth. For the viewer: the denial of the body’s and the eye’s immediate perception, coordination, and assimilation of all of the above (Zelevansky, 1997).
The article by Zelensky was written more than 10 years ago. His opinion of art viewing on a screen is still valid; however, since then, not only has the display technology improved, but also the spectators’ gaze has become more sophisticated in understanding digital presentation. Through ‘Little Heavens’ I sought to exploit those limitations and turn them into advantages for creation and viewing of mobile art. I attempted to establish this on three levels:
1. The mobile phone screen display attributes and its position within the framework of art history, i.e. light as a one of the most apparent features and the effort of the artists from the past to achieve the effect of light and glow;
2. The mobile phone screen surface attributes and its influence on formal qualities and technique of picture making, i.e. size in pixels and number of colours.
3. Mobile phone screen display and the spectator’s interaction as part of a creative and
emotional experience, i.e. the way spectators operate their phones to have a look at the picture.
Both of my art installations for this project aimed to reintroduce the picture as a
‘window to another world’, finding their roots in the history of art, when ‘images were first made to conjure up the appearances of something that was absent’ (Berger 1972: 10) and to convey the concept of the Sublime and the Beautiful through the Renaissance focus on ‘flat, rectangular surface… intended for frontal viewing’ (Manovich 2001: 95).
The small size of the screen surface draws an analogy to miniature paintings of the Middle Ages – medieval illuminated manuscripts. Decorated with gold or silver leafing,
375
those tiny illustrations meant to ‘illuminate’ – lighten up the page. An analogy to ‘illuminated manuscripts’ can be seen not only in a small size of the surface of the screen but also an internal light appearing when the phone is active. Light has another function. The colours of a painting are visible, thanks to the light reflected from the painted surface. The light rays having various lengths, make the human eye see different colours. The digital screen of a mobile phone has a light ‘within’, creating a very characteristic aspect of ‘glow’ to this medium. The image glows with its lights. The intensity of colours is deeper. Paul Zelensky noticed the benefits of such presentation: ‘the clean, concentrated colour and light of the screen… buttresses the technological and authorial power of the message’ (Paul Zelevansky 1997: 46). When creating a picture and then digitizing it for the use of the miniature screen, the importance of the technique becomes apparent. In order to achieve the ‘glow’ effect and to use the feature of internal light of the small screen, simplicity is the key. Over-complicated drawings or paintings done on a textured surface or with thick brush-stroked paint might not work. The texture on the surface of paper or canvas would almost be lost, looking rather like dirt; brush strokes would be invisible or create shadows; too many fine lines would merge into one or become entirely invisible. Spectators answering the questionnaire about this project, after being subjected to a week-long experiment, seemed to prefer the ‘smooth’, figurative, colour pictures, which were either made entirely using vector graphic applications or high resolution scans of paintings or drawing on smooth surface, and using either very decisive lines and spots of colours or polished paint. Image KM_02
Image KM_03
Image KM_01
Art experiment# 1 Technique: black ink pen, watercolor pens on watercolor paper
Art Experiment # 2 Technique: black ink pen, watercolor pens on watercolor paper
Art Experiment # 3 Technique: vector graphic and digital collage made using Photoshop
Interestingly, while seeking the effect of ‘Renaissance depth’, the mixed technique pictures, combining both traditional and vector/digital media, seemed to convey the effect best. An important aspect of visual mobile phone art making is the way in which the picture is digitized. If it is done with traditional techniques, it can be scanned and then transformed to the right size and right format for a mobile phone. However, the more obvious method would be to use the phone’s own built-in digital photo camera. The latest digital camera phones are of a decent quality (above 5 Mg px) and have various settings. This approach enables immediacy and is more personal when the picture is photographed and sent, rather than scanned and manipulated and then sent. Although most of the spectators taking part in the experiment claimed that they could not see any difference between those two methods, when asked about ‘glow’ and ‘light’ they pointed to paintings photographed or images made entirely with vector graphic applications.
376
Image KM_04
Image KM_05
Image KM_)6
Little Heavens # 4 Technique: acrylic ink on tracing paper
Little Heavens # 5 Technique: acrylic ink on tracing paper
Little Heavens # 6 Technique: ink on tracing paper
The emotional attitude of viewers to the personal aspect of the mobile phone is a
very important characteristic in both experience and creation of visual art forms for a mobile phone screen. The experience of receiving messages of the ‘art piece’ in one’s private space on the private window of a mobile phone makes a strong statement about this concept, which might make daily contemplation on art a seamless experience. This discovery also can help create a new set of rituals of experience of this type of art form – one of most important factors in the appreciation of work of art as noted by Walter Benjamin: ‘The uniqueness of a work of art is inseparable from its being imbedded in the fabric of tradition… It is significant that the existence of the work of art… has its basis in ritual, the location of its original use value’ (1936 [2000]: 677). In his article ‘Interaction as an Aesthetic Event’, Lev Manovich examines the visual qualities of mobile devices and their effect on a user:
Have you realized that the phone that you own – assuming it is a model that came out in the last couple of years – constantly plays games with you? It seduces you with its animated icons and sounds, the shape and surface finishes, the feel of its buttons and every other detail of its material and media definitions (Manovich 2007: 1, my emphasis).
According to Manovich, ‘the modernist design formula “form follows function” came to be replaced by new formulas such as “form follows emotion”’ (ibid). Christensen (2004) also noted in his thesis the intimate and affective nature of this device. Both statements made by Manovich and Christensen show an analogy to another aspect of miniature paintings from the previous centuries – namely miniature portraits intimately exchanged by lovers. It is obvious that people always have had a need for emotional connection with others and a shared secret or an intimate object is a token of their connection. The way miniature portraits were treated is very similar to the ‘Little Heavens’ game and the way we swap text messages with people we are close to. Direct interaction with the art piece is another crucial element in such an art practice. It creates an opportunity for a deeper, memorable experience and a better understanding of content. The unique interaction with the visual art piece can enhance and reach further than just static ‘art experience’ within the space of the art gallery. The mobile phone picture here – the visual artwork – not only can substitute something absent as stated by Berger; it also makes a spectator directly and intimately connected to an artist, even if anonymous. There is clearly a felt state of ‘connected presence’ – a term coined by Christian Licopp, especially if
377
pictures are sent to spectators every day during a certain period of time. The ‘connectedness’ through the private communication device makes spectators feel special – a very important and unique aspect of this type of art experience. With the mobile phone, spectators have to actively engage in ‘un-packing’ the ‘parcel’ in order to see the artwork. First they hear the sound or feel the vibration of the phone notifying them that someone or something has sent a message. Usually during the same time or when a user takes the phone into their hands and presses a button to look at the message, the phone lights up. Then there is a journey through the interface to retrieve the message. People taking part in the experiment noted that the picture – an artwork – was a very positive surprise and most of them saved it to their phones’ galleries to look at it on a neutral background, with no interruption from other elements of the interface. If in a space with other people – friends – they quite often shared the excitement and the picture with others. As mentioned previously, mobile media enable a new mobile experience, which can be linked to the embodied sensorium of performance works and early cinema. Why is it important to utilize a mobile phone as a carrier for visual arts? Because nowadays the disappearing experience of ritual as something special and the ubiquity of visual ‘art’ imagery in advertising, make such messages epidemic. Visual imagery has become too mundane – thus, people stop noticing it. So perhaps to use a mundane object to deliver an art piece can help us to remember that
[t]he visual arts have always existed within a certain preserve; originally this preserve was magical or sacred. But it was also physical: it was the place, the cave, the building, in which, or for which, the work was made. The experience of art, which at first was the experience of ritual, was set apart from the rest of life – precisely in order to be able to exercise power over it (Berger, 1972: 32).
CONCLUSION In summary, in our individual practices we have made converging discoveries to begin to define the new mobile screen aesthetics. These discoveries include the following. In order to understand mobile video projects, it is important to evaluate them in a category of their own, which this article suggests is within the new emerging mobile-mentary (mobile – documentary) and performance video categories. Mobile devices position us as agents not in cyberspace, but in the everyday environment. The new emerging pixel aesthetics include low-resolution digital images and video, digital pixel compositions, constrained and condensed screen dimensions and file size. This aesthetic emphasizes the importance of location, space and also non-space – being here, being anywhere, but here is where the phone is and not any fixed place, via immersion. The individuals’ experience of the space, or state of being is located within their own bodies. This is expressed as a haptic aesthetic, rooted in an embodied affectivity, manifested through gesture and movement of the device while documenting or ‘videoing’ innate performativity or movement, which is
378
afforded by the phones themselves. Through the emergence of mobile devices as a tool for creative production, an alternative space for the creation of artwork has emerged. This article has outlined the potential of mobile media for the fields of mobile performance video, visual fine arts, and mobile documentary filmmaking. This environment provides choice for creatives (artists and mobile phone users alike) to express themselves beyond the realm of the media industry. The new aesthetic drives innovation and illustrates the potential of mobile media within the evolving mediascape. Mobile media can be accessed and utilized by a large number of people everyday. There is an unintended intimacy and affective nature of the device. Expressivity with the device constitutes a predilection for close-ups and a sense of immediacy – instant, realtime, ‘being here now’. The portability factor allows for the watching and shooting anywhere, with the intimacy of enabling one to take their phone to private and personal spaces. Mobile devices make the mundane interesting, the everyday confronted, providing a new lens for viewing the world through a new camera vision. The mediazation of the world through the phone magnifies and brings the everyday world back into focus, having an amplified ‘reality’ and quality, creating expectations of somehow getting closer to the world. The everyday relation with this device creates a new set of rituals which can be used in appreciation of delivered content – whether it is a piece of visual art or a picture sent by a friend. The sound or a vibration made by this device creates ‘a sense of expectation, even urgency’ (McLuhan 2004: 289) – a crucial part of a ‘ritual’ followed by manual ‘unpacking’ of a message. Visual display is made available thanks to a small LCD screen; its resolution is constantly improving allowing the exhibition of consistently more complex imagery. What is also important is the use of light – light within the screen produced when the phone is activated – a vital component of the represented image. The sense of intimate connectedness to the message, its subject and the author, has emotional implications in everyday art experience – making one feel special, important and inspired. As this article has illustrated, the mobile media device is a new medium for expression and experience, with its own unique aesthetics.
379
References Ahonen, T.T. (2004), ‘From Interesting Ideas to Compelling Mobile Services’, Vodafone
Receiver, Issue 12 [Online] http://www.vodafone.com/flash/receiver/12/articles/index04.html Accessed September 11, 2008
Auslander, P. (1999), Liveness: Performance in a Mediatized Culture, London and New York: Routledge.
Benjamin, W (1936 [2000]) ‘The Work of Art in the Age of Mechanical Reproduction’, in McKeon, M (ed.), Theory of the Novel: A Historical Approach, Baltimore: The Johns Hopkins University Press, pp. 673-695.
Berger, J (1972) Ways of Seeing, London and Suffolk: British Broadcasting Corporation and Penguin Books Ltd
Bruno, G (2008) Atlas of Emotion: Journeys in Art, Architecture, and Film, London: Verso. Christensen, M S (2004), As We May Feel: Interpreting the Culture of Emerging Personal
Affective Mobile Media, unpublished PhD thesis, Copenhagen: Department of Innovative Communication, IT University of Copenhagen.
Corino, G (2006), Spatial Issues and Performative Media in Digital Mobility: A Network Perspective, Digital Futures MSc dissertation, Institute of Digital Art and Technology, University of Plymouth [Online] http://x.i-dat.org/~gc2/digital_mobility.pdf Accessed 13 July 2006
Davenport G, R Evans and M Halliday (1993), Orchestrating Digital Micromovies, Leonardo, 26: 4, pp. 283-288.
De Souza e Silva A (2006) ‘Interfaces of Hybrid Spaces’, in Kavoori, A P and N. Arceneaux, (Eds.), The Cell Phone Reader: Essays in Social Transformation, New York: Peter Lang Publishing, pp. 19-44.
Gye, L (2007) Picture This: The Impact of Mobile Camera Phones on Personal Photographic Practices. Continuum: Journal of Media & Cultural Studies, 21: 2 (June), pp. 279–288.
Flaherty, S. (2007) Keitai culture [online] http://www.keitaiculture.com Accessed August 07, 2007.
Hansen, M (2004) New Philosophy for New Media, Cambridge, MA: The MIT Press. Idhe, D (2002) Bodies in Technology, (Electronic Mediations, Volume 5), Minneapolis,
Minnesota: University of Minnesota Press. Ito, M (2005). ‘Introduction’, in Ito, M., D Okabe and M Matsuda, (Eds.), Personal, Portable,
Pedestrian: Mobile Phones in Japanese Life, Cambridge: MIT Press, pp. 1-16. Manovich L (2001) The Language of New Media, Cambridge, MA: The MIT Press Manovich, L. (2007), ‘Interaction as an Aesthetic Event’, Vodafone Receiver, Issue 17
[Online] http://www.vodafone.com/flash/receiver/12/articles/index04.html Accessed August 02, 2008
Mitchell, W (2006) ‘Networked Eyes’, in Jones, C A. (ed.), Sensorium Embodied
380
Experience, Technology and Contemporary Art, Cambridge, MA: MIT Press, pp. 174-9. McLuhan, M (2005) Understanding Media, London and New York: Routledge. Molga, Kasia (2007) ‘Wireless Art: the Possibilities of Application of Mobile
Communication Media to a Modern, Visual Art Practice and as a Medium of Expression?’, Proceedings, EVA Conference, July 2007, pp. 16.1-16.6.
Richardson, I. (2005), ‘Mobile Technosoma: some phenomenological reflections on iterant media devices’, Fibreculture Journal 2005, Issue 6: Mobility, New Social Intensities and the Coordinates of Digital Networks. [Online] http://www.journal.fibreculture.org/issue6/issue6_richarson.html. Accessed 10 June 2007.
Rutsky, R L (1999) High Technē: Art and Technology from the Machine Aesthetic to the Posthuman, Minnesota: University of Minnesota Press.
Sadler, S (1998) The Situationist City, Cambridge, MA: The MIT Press. Schechner, R. (1988 [2003]), Performance Theory, London and New York: Routledge. Shusterman, R. (2000), Performing Live: Aesthetic Alternatives for the Ends of Art, Ithaca
and London: Cornell University Press. Terry, D (2005) mo.vid.1. [Online] www.100lies.com or
http://www.deanterry.com/blog/index.php/2005/10/ Accessed September 12, 2008 Wolf, R W (2009) Micromovies – Pocket Short, Part 1. Das Kurzfilmmagazin [Online], Available: http://www.shortfilm.de/index.php?id=414&L=2&print=1. Accessed 20 March 2009. Zelevansky, P (1997) ‘Shopping on the Home Image Network’, Art Journal, 56: 3, p. 64
[Online] http://www.questia.com/read/5000561018 Accessed September 04, 2008 Endnotes i According to Reinhard W. Wolf, ‘The term “micro-movie” was coined in the 1980s in Nicholas Negroponte’s Architecture Machine Group at MIT and expanded in 1993 by Glorianna Davenport (Interactive Cinema Group at MIT Media Lab) in the context of interactive video databases – which were introduced in 1997 at the International Short Film Festival Oberhausen. (Wolf, 2009). In Orchestrating Digital Micromovies, Davenport, Evans and Halliday (1993) describe a micromovie as a short piece of video with descriptive information attached to it. ii Max with a Keitai (2008) Japan/UK, 3gp and mpeg4 video. iii The megapolis Taiheiyō Belt, the Tokaido corridor, includes the Ibaraki Prefecture in the north of Japan and reaches to the Fukuoka Prefecture in the south in Japan. This ‘Pacific belt’ includes the major cities of Tokyo, Osaka, Kyoto, Kobe, Himeji and Hiroshima amongst others iv Such as in the gestural work of MindTouch, discussed below.
381
v An example of this is semacodes, a mobile barcode technology which allows mobile phone users to connect the online environments through taking a picture of the specific mobile barcode. vi ‘An avatar is one’s representation in space in which one is not’ (de Souza e Silva, 2006: 29) vii MindTouch workshop video edits are available online at www.swampgirl67.net in the Media Research section viii Such as projects like Uncle Roy All Around You from the performance group Blast Theory, http://www.blasttheory.co.uk/bt/work_uncleroy.html. ix I have noticed many ‘locative’ game projects are now calling themselves ‘mixed reality’, since they mix digital gaming interfaces with live performative tasks and game play, using mobile devices. x In Don Idhe’s sense that ‘…to be situated entails that the knower is always embodied, located, is a body…’ (Idhe, 2002: 68) xi This use of body/mind acknowledges the studies and theories of Brian Massumi,
neurobiologist Antonio Damasio, Benjamin Libet and other philosophers, showing that the mind not only exists in the brain but also in parts of the body, especially the senses.
xii FILMOBILE is run by a networked team of professionals working in telematic film, leading a major project developed at the University of Westminster. The Project brings together the work of many international artists, filmmakers and professionals working in the mobile phone industry. In April and May 2008, FILMOBILE organized a major international event consisting of a gallery exhibition, cinema screenings and an international conference. This event explored the cultural and economic impact brought about by new mobile technologies and initiate debates between artists, the media and the new mobile industry (www.filmobile.net).
xiii Multimedia Messaging Service xiv Art installations engaging the mobile phone as an avenue of experience, developed within the last 3 years. Mirror of Infinity and Breaking News interactive art installations – pictures available online at www.kasiamolga.net in the Interactive Installation section xv The visual manifestation of this installation is either in a form of a projection or on LCD display.
