As it is: PEGASYS and Presence

Sarah Moss

Introduction

This chapter discusses an interactive art system produced in the context of the

author’s research into the nature of presence and designing for creative engagement.

The artwork is known as PEGASYS, an acronym for PresencE-Generating Art

SYStem. PEGASYS provides a time-based interactive cinematic experience that

utilises an eye tracking device to access video-on-demand. The research explores the

implementation of design features identified as necessary for enabling optimal

presence engagement for participants. The features include designing a system that

stimulates participants creative engagement, produces the sensation of spatial

presence and enhances the experience of ecological validity. PEGASYS also explores

the use of interactive narratives and addresses the use of colour, motion and audio as

key engagement factors. The art system was designed to consider the participant’s

personal interaction with, and reaction to, the system’s response time.

Researchers of presence aim to support the development of a comprehensive

definition of presence with tangible measures that assist in determining ample levels

of presence for the achievement of goals. ‘Media Presence’ and ‘Inner Presence’ are

two components of presence that are constantly under evaluation in the search for a

standard reliable and valid measure (Coelho et al., 2006). A study was conducted to

evaluate and compare participant’s experience of an art system designed to adhere to

these two aspects of the experience of presence. The art system is composed of two

main parts: a technological component (Media Presence) and a psychological

experience (Inner Presence). Combining research into current presence theories with

research about interactive art systems I then drew upon two models of creativity: a

Model of Creative Engagement (Edmonds et al., 2006) and a Model of Engagement

(Bilda et al., 2008). These models and how they were implemented into the design

framework are discussed in more detail later in the chapter.

PEGASYS: a Presence based Art System

PEGASYS consists of ninety minutes of high definition video (SONY HDV 1080i).

Shot from fourteen fields of view, with audio recorded using a binaural sound capture

device, the art system facilitates a 360 degree interactive cinematic experience for

participants (Moss and Edmonds, 2009). The biotechnological component of the art

system is an eye tracking device (Tobii X120-203-73700036). Networked into the

system by Oleg Sakharov, it provides on demand access to the audio and video data

clips. As the participant navigates the interface via eye-tracking, they pan left and

right choosing to interact between various levels of stimuli or ‘stages’ as they are

referred to. A dynamic database, designed by Alastair Weakley, contains interlinking

video clips that enable participants to traverse through a location named, Crystal

Pool, in Sydney’s Royal National Park, Australia.

Figure 1. PEGASYS: Stage One, Stage Two and Stage Three (Sarah Moss, 2009)

During evaluation, participants were observed to become immersed in the narratives

employed within PEGASYS. These narratives provided a variety of alternative

scenarios for participants to explore. The interview data revealed that the narratives

produced three very diverse phases of interaction. Stage One (five minute duration)

presents the natural landscape with natural sounds, Stage Two (eight minute

duration) introduces an avatar and various audio components, and Stage Three (ten

minute duration) provides features that require a higher level of participation and

engagement (see Figure 1).

The camera techniques employed, such as ‘direct address’I, assisted participants to

develop relationships with the screen-based characters or avatars, influencing the

experience of ‘being together with another’ a signifier of a presence-based

engagement. Participants expressed interest in sharing the digital environment with

others, indicating that mutal understanding and awareness created moments of

intimacy and sensory awareness. Three stages of interaction are accessible: the

participants made creative choices about where they wished to be located within the

system and defined for themselves the content of the engagement through individual

narrative selection. As participants move between the stages the stimuli changes the

experience as additional interactive elements emerge.

Participants were engaged in the experience as they observed and interacted with

elements such water, wind and shadow moving throughout the natural location. In

order to achieve the best effects, the video reproduction of the environment had to

best represent that environment, and to that end, a significant emphasis was placed

on this aspect during post-production. In order that participants could experience the

verisilimitude of a ‘natural’ environment, the cinematic footage used in PEGASYS

was shot in real-time and played back on demand. The art system aimed to provide

not only entertaining experiences, but a place for relaxation, personal reflection and

exploration of the artist’s concepts of enabling experiences of sensory illusion. These

experiences were observed during a month long evaluation period in Beta_Space at

the Powerhouse Museum, Sydney (Beta_Space, 2011).

Design Through Theory

The PEGASYS art system was designed in order to test participants’ experiences of

the degree of immersion. Three issues were considered from a physiological or a

psychological perspective (Media and Inner Presence): content in respect of Media

Presence, form in respect of Media Presence and participant engagement as a

component of the experience of Inner Presence. Riva’s three layers of presence (Riva

et al., 2004) were also considered in the design of PEGASYS as a theory-based

prototype. The layers include: self versus non-self (Proto Presence), self versus

present external world (Core Presence) and self relative to present external world

(Extended Presence).

Existing research on presence predicts that participants can be drawn into a state of

relaxation and restoration through a presence-based experience (Sponselee et al.,

2004, Witmer and Singer, 1998). The experience of PEGASYS itself was, therefore,

designed to relax the participant and facilitate an engagement with an art system that

accesses their conscious and subconscious minds. In this way, PEGASYS was

designed, through attention to form, content and participant, to create a peaceful,

serene and playful experience in an otherwise over-crowded noisy day. In the

evaluation study, the art system was found to produce a meditative, explorative and

creative environment that drew participants into a state of happiness via play and

engagement. The immersive environment explores possibilities for developing

interactive systems that bring the ‘peace of mind’ sensations usually generated by

exposure to natural habitats. In the city, PEGASYS provides a lunch break without

concrete. PEGASYS was thus designed to be experienced as a de-stress technique,

reducing anxiety and enhancing tranquillity in a relaxed yet playfully challenging

engagement. This research therefore extended conclusions about obtaining and

maintaining a state of harmony in a low stress mediated environment: the full extent

of the outcomes may be found in Moss (2011). Keeping all this in mind, PEGASYS

was then designed to adhere to a range of presence signifiers that stem from previous

research into presence.

The Art Systems Seven Design Features

The art system design features described below are ranked in relation to the

practitioner’s process. This involved devising a taxonomy of interactive art systems,

identifying previous research into presence-based systems and allowing for the

practitioner’s twenty years of experience working on large-scale projects in the

creative industries: TV, Theatre and Film. Seven features were identified for

consideration. These features include designing a system to: stimulate participants’

creative engagement, produce the sensation of spatial presence, enhance the

experience of ecological validity, explore the use of narratives, examine the use of

colour and motion, consider the audio, and account for the nature of the participants’

interaction with the system response time.

‘Engagement’ is the first feature to consider in respect of participants’ experiencing

psychological immersion. Engagement includes the participants’ ability to get

involved, facilitating a psychological immersion that can be a two-way

communication with another person or with an artificial entity e.g. computer system.

Engagement is an ‘Inner Presence’ component: however, it is also associated with

participants’ experiences of both the content and the form of the system. The creative

engagement experience differs from participant to participant. This is because

engagement with an interactive art system involves not only the act of looking, but

necessarily involves the participant taking an active role and engaging with the

system physically. Creative engagement invites the spectator to become a participant

within the system, facilitating a unique engagement involving human and computer.

Subjective personal human qualities, such as motivation, emotions and the ability to

suspend disbelief become intermingled with concepts of beauty, fun and pleasure.

Therefore, aesthetic appreciation plays a key role as does the cognitive state of the

participant. In order to engage the participant, two engagement models were

employed in the implementation of the art system design.

A Model of Engagement (Bilda et al., 2008) addresses engagement from an audience

perspective, thus providing guidelines for artists who wish to create interactive

systems with the most engaging possible outcomes.

The Creative Engagement Model proposes a method for designing engagement for

interactive art systems in terms of three classifications: attracters which draw the

participant into the engagement, sustainers that maintain the interest and hold the

participant’s attention, and relaters, which work to captivate and engage the long-

term interest of the participant ( Edmonds et al., 2006). Table 1 illustrates how the

design of PEGASYS combined The Model of Engagement interaction modes and

phases with The Creative Engagement Model’s attractors, sustainers and relaters. A

creative engagement was thus designed in respect of the art system’s six other design

features.

The second design feature is the experience of ‘spatial presence’. Spatial presence is

commonly referred to as physical presence and/or perceptual immersion (ISPR,

2000). Spatial presence transmits a sense of physical space, transporting the

participant into the landscape producing a ‘sense of being there’. i.e.; virtual reality

systems (Schubert et al., 1999). Lessiter states that spatial presence is also referred to

as ‘physical space’ which is proposed to closely connect engagement and naturalness

as key binding elements. Together these factors work with the individual’s personal

evaluation based on appeal and believability of both the displayed environment and

its content (Lessiter et al., 2001). It has been suggested that a central determining

factor for creating a sense of presence lies within the place itself, i.e. the virtual

environment or other location. Previous research into presence suggests that these

places are different from physical, and even mental spaces, as the space of presence is

viewed not as a component but as a defining dimension of presence itself (Spagnolli

and Gamberini, 2004). Spatial presence refers to the physical bodily orientation in

the world and requires appropriate updating of displays through tracking devices.

In presence terms, ‘ecological validity’ is akin to sensory presence. Sensory presence

is experienced through the system’s ability to employ the participant in a tactile

engagement and is gained through the systems ability to produce a sense of

naturalness. To work best, it requires high quality images and an efficent playback

system (Smyth et al., 2006). The pictorial realism and the visual depth are important

factors in the ability of a displayed environment to create a sense of depth. By

addressing ecological validity as a design feature it was possible to create and

maintain a sense of ‘naturalness’ within the art system’s content. Participants in this

system should feel that the environment is natural, believable and part of the real

world. Auditory cues are also associated with an increase in sensory presence

engagement.

The participants’ experiences of the ‘narratives’ as a design feature, has been

discussed in respect of the three stages of interaction (see Table 1). These stages

increase in media content or activity, each stage providing a different view of the

same location with additional interactivity embedded. In this way, each stage was

measured for presence engagement attributes independently through an interview

process that addressed participants’ experiences of both the visual and aural content.

By providing the participant with a contrast of media, comparisons were drawn about

participants’ experiences of the art system. During the evaluation process, a majority

of participants indicated that they enjoyed Stage Two the most. Study results

confirmed that Stage Two, containing one character who dominates the narrative

with a blend of natural sounds, voice and music combined, contained more emotional

dialogue and built in narratives, thereby enhancing the experience of the art system

for many (see Figure 3).

In designing the art system, it was important to consider the impact on the

participant of the use of colour. Since the system was built using real-time video clips,

the colour balance of each clip (fourteen clips in total) was edited to enhance the

natural environment. Where video ‘stitching’ occurs, the colour differences become

more apparent. Hence, it was important to capture the correct daylight colours with

the cameras using the built-in white balance feature.

Motion is also a key factor in the interaction process as participants navigate the

video clips through the eye tracking device by gazing left to right and vice versa. Thus,

the system’s content, through the system’s form, is in constant motion. Negative

effects such as feeling dizzy, disorientation and motion sickness can hinder the

participant’s experience and have an impact upon their engagement; this means that

testing the developing system with users regularly was essential to determining the

ultimate speed of the system.

In keeping with the theme of producing a relaxed experience for participants, the

‘audio’ plays an extended purpose, a duality for creating a heightened awareness of

oneself as well as sustaining the environment in which the film shoot took place. It

has been noted that a very specific form of music with a particular rhythm can induce

a relaxed state in the body; these types of music have been shown to create a music-

induced relaxation but nevertheless, leave the mind alert and able to concentrate.

Physiologists found the rhythms of the body: heartbeat, brain waves, and so forth,

tend to synchronise themselves to the beat of music. Classical music with a very slow,

stately, restful rhythm induces a calm and restful bodily experience (Ostrander and

Schroeder, 1979). The use of a continum of sound facilitates an environment

conducive to reduce stress and anxiety, assists in creating for the listener profound

states of deep mediation and relaxation, increases self awareness and has been

proven to improve mental and emotional well being. These are qualities which were

sought after in the design of PEGASYS. Therapists, new age healing centres and

health practitioners implement this type of musical score precisely to encourage long-

term improvement in mental and emotional well being (Mcleish, 1985). Composer

Marianthe Loucataris produced the score for PEGASYS. Features designed to

enhance the audio experience included atmospheric sounds, music, voice and sound

effects. Some of these sounds were embedded within the media and others were

designed to be retrieved as interactive components. During the interview process

positive comments were made about the use of sounds as a system feature to enhance

the experience of presence engagement. The implementation of sound cues as

attractors, sustainers and relaters increased the possibilities for engagement and

complimented the exhibition environment through a surround-sound score, while

the participant wore headphones (see Table 1).

The system response time is the time taken between the participant’s action and the

perceived effects of those actions upon the displayed environment (Floridi, 2005).

Latency of visual feedback has been seen to detract from presence engagement when

significant intervals of time pass between the action and the re-action. Therefore, the

form of the technology should facilitate a natural flow between the participant and

the interface. The PEGASYS participants experienced a hands-free engagement

because the eye tracking device activated the video-on-demand. Thus, they were

much more likely to become ‘located’ within the content and therefore, less likely to

be distracted by other worldly events; in other words, the state of presence was more

likely to occur. Furthermore, this art system was designed in order to replicate a slow

engagement. This means that the system’s interactivity was slowed down

deliberately, but dynamically, in order to create an experience which forced

participants to examine the details of each image as they were experiencing it. In this

way, the art system facilitated the original intention of the work, which was, to draw

the participant into a reflective ‘place’ that encouraged immersion. PEGASYS

facilitated rest and relaxation as participants engaged with a natural bushland

environment.

Overall, the participants ability to pay attention was found to be vitally important. If

participants were equally involved in the content (23/25 people agreed that the

content appealed to them) and the form of the system (21/25 people agreed that they

could interact with the art system) then it is an indicator that participants were

paying attention. During evaluation participants indicated that the systems features

were found to assist in the generation and experience of presence-based engagement.

A Model of Engagement PEGASYS The Creative Engagement Model

Interaction

Modes

Interaction

Phases

Attractors Sustainers Relaters

Unintended Adaptation STEP 1

Calibration

of the eye

tracker

Introduction to

the operating

system

Interface

design, simple

and efffective

Deliberate Anticipation

Learning

STEP 2

Stage One

Attempts are

made to

operate the eye

tracking

interface

360 degree

view

Diagetic sound

score

Intended /

In control

Anticipation

Deeper

Understanding

Learning

STEP 3

Stage Two

New content

alters the pace

of interaction

The sounds of a

girl humming

Introduction of

a musical score

Guided

interaction

Intended /

Uncertain

Adaptation

Deeper

Understanding

STEP 4

Stage Three

13 avatars

increase the

interaction

engagement

Additional fast

paced visual

media

More choices of

stimuli

Dynamic

sound content

introduced

Unexpected Adaptation STEP FIVE

Stage Three

Interactivity

increases as

participants

adapt their

behaviour

Multi viewing

options

Dynamic

sound content

explored as a

feature

Table 1 PEGASYS as a Dynamic-Interactive Art System

Figure 2. View of PEGASYS under evaluation in Beta_Space (2009)

Exhibiting and Evaluating PEGASYS

Evaluation of PEGASYS was conducted with fifty participants at the Creativity and

Cognition Studios, Interaction Lab and in Beta_Space at the Powerhouse Museum in

Sydney. In Beta_Space, twenty-five participants’ interactive experiences were

evaluated for presence attributes. Each participant was seated in the space as in

Figure 2. Here you can see the eye-tracking device is situated approximately 2.6

metres away from the display environment positioned between the screen and the

chair and directly in front of the seated participant. In order to establish contact with

the built in infrared camera lens embedded within the device, the participant had to

manually adjust the height and positioning of the chair to meet their own personal

requirements. Two tables and a chair were arranged diagonally in the bottom left

corner of the room with access to power. The artist researcher was able to observe the

participants with discretion. Video documentation was captured from a camera

situated directly behind the participant. Two external sound systems were in

operation. The first sound system created the atmospheric sounds and operated

through a DVD player positioned backstage, relaying into two speakers downstage of

the display environment. The DVD player looped a ten-minute audio track

throughout the day. The second sound system was operated via a CD player and mini

amplifier. The CD player was accessible from the operating desk allowing the

evaluator the opportunity to raise and lower the volume at will. Three small speakers

were rigged on the rear wall of the space directly behind the chair. Each participant

wore headphones.

Figure 3. PEGASYS: Stage Two, Clip Three. (Sarah Moss 2009)

PEGASYS and Presence

As we have seen, I designed PEGASYS in accordance with (a) seven presence-

orientated design features, (b) two models for creative engagement, (c) consideration

of participants’ experiences of low-stress environments as healing environments and

(d) elements of human consciousness requiring hightened attention, creativity,

mindfullness and engagement which are required in order to enhance the sensation

of a presence-based engagement.

Designed to provide deep immersive engagement PEGASYS initiated new engaging

experiences for participants with digital networked systems.

“Like a form of visual poetry. At times I was able to feel part of it or in shared control

but mostly I let it wash over me I think. Very tricky combination of sitting still –

feeling a little strapped in but more or less a sense of ‘letting go’ was the joy of the

experience”. Participant 16

An analysis of participant experiences, based on comments made during the

interview, observation and the results of the questionnaire (ITC-SOPI) indicated that

a majority of the participants were very engaged in their experience. Eye tracking

technology had a significant impact on participants’ experiences of the art system.

Participants interacted with the system through a series of repeated patterns or

random movements according to individual choice. The majority of participants

(21/25) agreed they enjoyed themselves, and 18/25 felt involved. 19/25 participants

agreed that they felt they weren’t just watching something and felt as though they

were participating in the displayed environment.

The results also indicated that the degree of interactivity made possible through eye

tracking was a successful design feature. The sensation of immersion was made

possible to some degree, as most of the participants (23/25) expressed that they had

a strong sense of involvement and felt themselves drawn into the experience. 21/25

completely lost track of time, the average duration being 35 minutes. Overall, there

were adequate presence signifiers acknowledged to ascertain that participants

experienced the sensation of presence engagement whilst interacting with the art

system. Participants experienced a new form of communication, and learnt how to

navigate, control and interact with, an audio-visual system through the use of their

eye movements only.

Most participants engaged with PEGASYS in a fluid playful manner; however, as

previously mentioned, a slow system response time was implemented as a system

feature and was observed to hinder some participants’ experiences. What I witnessed

occurring then was ‘absence’, as the participant tried to gain control and failed to do

so, they were drawn to engage with the technology in a new way. By observing the

participants’ eye movements using a track status window, it became apparent that

there had to be a genuine willingness on the part of the participant to interact and

engage with eye tracking technology, particulary when it is programmed to behave

dynamically.

“Yes, parts of it were relaxing. Parts of it I think I was trying really hard to get it to

work, and trying hard not to move and I think that… that I tend to hold everything

from the shoulders up to the chin. I do this (demonstrates). But that’s just me. But yes

it was actually really nice, and I didn’t mind that it wasn’t working… it didn’t frustrate

me. I just went oh well I just have to look at what’s actually happening now rather

than move it, ...because you do tend to go, oh this is really exciting, I can move it with

my eyes and you just want to keep moving around. So when it wasn’t working…I

actually… it didn’t frustrate me at all. It was actually… it meant that I saw things that I

might not have seen.” Participant #17

PEGASYS provides participants with a near realistic sense that they are somewhere

else, located within the media content, transported by the system’s form. Through

practice-based research it was possible for me to develop an interactive art system

that contributes new knowledge about aspects of presence engagement and

interactivity. This information can now inform designers and creative practitioners

about designing art systems through a process that combined theory with creative

practice. By uniting current theories on presence, interactive art systems and models

for creative engagement I produced, exhibited and evaluated a presence-generating

art system that utilises eye tracking in an interactive cinematic experience.

I ‘direct address’ : a technique where the actor on the screen addresses the viewer by looking directly into the camera.