View
227
Download
0
Tags:
Embed Size (px)
Citation preview
Enaction and the Concept of Presence
Enactive/06
Structure• Introduction (10 minutes)
– Presentation of the Symposium (6 minutes)– Presentation of the panelists (1 minute for each panelist)
• Discussion sessions around 4 questions– Answer of each panelist (2 minutes for each panelist)– Questions from public and answers from panelists (8
minutes)
• Conclusions– Summing up of the relevant points emerged (10
minutes)
Introduction
Aim• The aim of this special session at Enactive/06 is to explore
relationships between Enaction and Presence. • The invited panellists are members drawn from the
Enactive Network of Excellence and the Peach FET Coordination Action in Presence.
• They will debate fundamental concepts, philosophical roots and practical examples of enactive interfaces and presence.
Basic concepts: Enaction & Presence
• Enactive knowledge is constructed on motor skills and doing is the mean for learning in an enactive context (Bruner, 1966). Enactive interaction is direct, natural and intuitive and enactive interfaces deal with the role played by action in the shaping of the perceptual content, the role of active exploration and the role of perception in the guidance of action. The close coupling of the perception-action loop is hence a key characteristic of enactive interfaces.
• Research into the concept of presence also draws upon similar notions. Presence has been defined as ‘the illusion of non-mediation’ (Lombard and Ditton, 1997). It is concerned with developing technologies that make it possible for people to feel as if they are somewhere else, or to feel that distant others are in fact close by. It is concerned with tele-presence and haptic interaction, with new places for interaction, with gesture, full body and other forms of multimodal interactions.
• Many issues concerning Enaction and Presence are still under debate and will addressed by 4 main questions.
Questions1. Believability is a notion that has been explored in the domains
of literature, theatre, film, and animated characters or agents. Believability has been conceived of as the illusion of life or of reality, the illusion that even the odd, caricatured, unrealistic characters are alive or real.
But the mental state of illusion of reality is far for being conceptually and psychologically well characterized. It can be understood as an illusion of non-mediation, as transparency. But it is not evident that users and film or other media audience mistakenly take a fake world for real nor that believability require this error. People confronted with VR worlds know they are experiencing something that is not real, yet they may react as if it were real.
Can we consider Believability as one condition that makes users and spectators act and react as if the virtual world was real? In this case, which actions and reactions should be taken into account as manifestations of believability?
Questions2. The notion of realism as simulation of the real world is problematic
too.
The Japanese roboticist Mori has introduced the notion of the Uncanny Valley: the sense of familiarity increases in association to realistic robotic artefacts (human-like robots) up to a certain point and then comes to a valley of negative familiarity and strangeness. Out of the valley there is another, higher peak of positive familiarity associated with perfectly human-like artefacts and real humans. The fall in the valley of uncanny is especially experienced in presence of discrepancies between visual cues and cues from other sensory modalities and between the realism of the physical aspect and a lower level of realism for behavioural aspects.
Is photorealism enough to faithfully simulate the physical characteristics of the real world or should we be concerned with other sensory modalities? Is realism as simulation of the properties of the real world a necessary condition for producing believable worlds and agents? Which is the role of coherence?
Questions
3. Feelings of connectedness come through presence allowing interfaces that provide a high degree of presence to extend a person’s influence.A surgeon performing a tele-operation wants to feel as if he or she is working on a body immediately to hand. The surgeon wants to make movements and gestures as they would in the real world.
What meanings can gestures have at the technological scale and at the human scale?
Questions
4. The sense of immersion is also important to a sense of presence.
But in mixed realities the human is effectively immersed in both the real and the digital condition.
How can realism, consistency and believability be maintained as people move between representations?How can we design to make interactions with digital artefacts as natural and intuitive as interactions with the physical world with which we are so familiar?
Panelists
David Benyon: Background
Framework 5 Presence project, Benogo (Being there, no need to go)– we were concerned with reproducing real places through
using photo-realistic images. – Issues of fidelity of representation were dealt with– wanted to capture the sense of place experienced– design for presence as a sense of place.
• Good interaction is unmediated interaction– Presence as the illusion of non-mediation– Enaction as the tight coupling of perception and action– Interacting through rather than interacting with
• Embodied Interaction– Paul Dourish ‘Where the action is’– Malcolm McCullogh ‘Digital Ground’– Marshall McLuhan Media as the ‘extensions of man’
• Mixing Realities• Designing for interaction at a human scale
Position Statement
Wijnand IJsselsteijn, Eindhoven University of Technology
Enactive /06Montpellier, 20 nov 2006
Imagine the year 12,006....
What would a team of Martian paleantologists and archeologists deduce about our species when encountering the remnants of our information society, i.e., a desktop computer?
Homo Desktopus
© Koert van Mensvoort
‘El Cine del Futuro’ – Morton Heilig (1955)
Embodiment?
“The biggest challenge to developing telepresence is achieving that sense of “being there.” Can telepresence be a true substitute for the real thing? Will we be able to couple our artificial devices naturally and comfortably to work together with the sensory mechanisms of human organisms?” – Marvin Minsky, 1980
Teleoperation
Fluent telemanipulation
When we interact with virtual or remote environments using intuitive interaction devices, isomorphic to our sensorimotor abilities, the real-time, reliable and persistent chain of user action and system feedback allows the integration of technology as a phenomenal extension of the self and the perception of technology-mediated environments as distal, but actionable environments in relation to one’s own active probing.
On the road to transparancy….
George Papagiannakis
• Position Statement. Mixed Realities (MR) and their concept of cyber-real space interplay invoke such interactive ’touching’ experiences that promote new patterns of believability and presence. Believability is a term used to measure the realism of interaction in the MR environments. Case studies are represented by the reproduction of ancient frescos-paintings through the real-time revival of their fauna and flora, featuring groups of virtual animated characters with artificial life dramaturgical behaviours in an immersive, fully mobile Augmented Reality environment. The main goal is to push the limits of current Augmented Reality (AR) and virtual storytelling technologies and exploring the processes of mixed narrative design of fictional spaces (e.g. frescos-paintings) where visitors can experience a high degree of realistic immersion. We argue that in order to deliver ‘real’ experiences and to create believable interactive applications for mobile MR, future steps in mobile MR enabling technologies should cater for enhanced social awareness of the virtual humans to the real world and new channels for interactivity between the real users and virtual actors. Only then the believability factor of virtuality structures will be enhanced and allow for compelling real experiences through mobile virtual environments.
Thomas A. Stoffregen
• Position statement. There seems to be a powerful desire to equate the quality of simulation of virtual environment systems with their ability or tendency to induce certain subjective experiences in users, such as the feeling of “being there”. Many concepts used in the study and design of virtual environment systems are not rigorously defined, and have no widely accepted definition (e.g., presence, immersion, fidelity). Weak definitions make it difficult to use such concepts for the formulation or testing of empirical hypotheses about user performance. Weak concepts also make it difficult to determine what factors truly are important in evaluating the quality of simulation and virtual environment systems. In some applications (e.g., entertainment), the subjective experience of users is the chief criterion for system quality. However, in many other applications, the subjective experience of users is irrelevant, or of only indirect relevance to the quality of the system. In many applications, it is the user’s behavioral interaction with the system that matters most, and the success of this behavioral interaction often can be addressed through direct measures of behavior, rather than though indirect measures of subjective experience.
Answers
Question 1 Believability
• Answer sequence: – George – David– Thomas– Wijnand
George Papagiannakis: Main elements of believability
Elements of BelievabilitySensory Level
Believability
PerceptualLevel
BelievabilityVirtual World
Participants
Interface
Immersion
Presentation
Interaction
EmotionEmotion
PersonalityPersonality
Physical Info.Physical Info.
EmotionEmotion
PersonalityPersonality
Physical Info.Physical Info.
Elements of BelievabilitySensory Level
Believability
PerceptualLevel
Believability
Sensory Level
Believability
PerceptualLevel
BelievabilityVirtual World
Participants
Interface
Immersion
Presentation
Interaction
Immersion
Presentation
Interaction
EmotionEmotion
PersonalityPersonality
Physical Info.Physical Info.
EmotionEmotion
PersonalityPersonality
Physical Info.Physical Info.
EmotionEmotion
PersonalityPersonality
Physical Info.Physical Info.
EmotionEmotion
PersonalityPersonality
Physical Info.Physical Info.
George Papagiannakis• Personality and
Emotion for VHs• Facial animation
with emotion • Body animation
with emotion: – Idle motions– Gestures– Reflex motions
• Integration and blending
• Face & body are both important for perceiving emotions and for realism of motion
David Benyon: Believability
• Scenarios are very powerful in terms of believability
• E.g. In security guard scenario people ‘heard’ glass breaking when there was none.
• In an auralisation of a large hall people ‘heard’ footsteps
• Many famous illusions rely on a false scenario
David Benyon: Believability
• Believability is about the combination of media, narrative, affect, activities
• Presence enables us to fill in the gaps of experience; to create believable experiences.
• It is about what is not said, or not shown that makes us believe
• In literature, the gap in the text
Thomas Stoffregen: Belief in what?
• Two types of subjective experience: Subjective Realism versus Subjective Reality
• Subjective Realism– Degree to which a virtual environment resembles the
corresponding real environment– User correctly believes that they are interacting with a
simulation• “That looks realistic”, or “that looks believable” implies “I know it is not
real”
• Subjective Reality– The virtual environment cannot be distinguished from the
corresponding real environment– User incorrectly believes that they are in the real
environment• These are mutually exclusive
Thomas Stoffregen: Design goals
• When do users need Subjective Realism?– When is it useful for users to know that they
are interacting with a simulation?
• When do users need Subjective Reality?– When do we want users to believe that they
are truly in the depicted environment?
• In most cases, we want subjective realism, not subjective reality
• We need to be clear about which experience users need
Question 2 Realism
• Answer sequence: – Thomas– Wijnand– George– David
Thomas Stoffregen: Realism and Believability
• Can we have a strong definition of realism or believability?
• My claim: Realism/believability is context-specific, and is influenced by experience– When linear perspective was invented (ca. 1400), it
was called “realistic”– When photography was invented (1839), it was called
“realistic” (and linear perspective was called “unrealistic”)
– When cinema was invented (ca. 1890), it was called “realistic” (and photography was called “unrealistic”)
– When closed-loop displays were invented (ca. 1970), they were called “realistic” (and cinema was called “unrealistic”)
Thomas Stoffregen: Obsolescence
• A technology that is “realistic” or “believable” today may be “primitive” or “unrealistic” tomorrow
• There is no fixed meaning, no fixed standard or criterion for realism, or for believability
• Conclusion: Realism and believability should not be general criteria for design
vs
George Papagiannakis:Believable virtual characters
• Virtual Characters composited via offline Special Effects (SFX) in movie industry
George Papagiannakis Illumination registration for believable virtual characters
A. Egges, G. Papagiannakis, N. Magnenat-Thalmann. An Interactive Mixed Reality Framework for Virtual Humans. Cyberworlds 2006, EPFL, Switzerland, November 28-29, IEEE Computer Society. September 2006
David Benyon: Realism• In a comparison of photo-realistic and graphical
virtual worlds, there was little difference
David Benyon: Realism
• Comparing real and photo-realistic renderings of places, responses to real places were more detailed, more exaggerated
David Benyon: Comparing Real and Virtual
• Comparing real and virtual
Very Quite Neither Quite Very
Attractive 23 3 1 1 1 Ugly
Big 7 12 6 2 1 Small
Colorful 5 12 8 4 Colorless
Noisy 4 8 5 8 4 Quiet
Temporary 1 7 6 8 7 Permanent
Available 4 11 10 4 Unavailable
Versatile 2 11 8 7 1 Limited
Interactive 5 8 5 6 5 Passive
Pleasant 23 5 2 Unpleasant
Interesting 19 6 2 1 1 Boring
Stressful 1 1 3 4 20 Relaxing
Very Quite Neither Quite Very
Attractive 7 15 7 Ugly
Big 2 11 13 2 1 Small
Colorful 14 5 9 1 Colorless
Noisy 3 7 9 6 4 Quiet
Temporary 3 6 8 8 4 Permanent
Available 1 9 11 5 2 Unavailable
Versatile 2 6 12 7 2 Limited
Interactive 1 7 6 10 5 Passive
Pleasant 4 16 5 4 Unpleasant
Interesting 8 11 5 4 Boring
Stressful 5 5 11 8 Relaxing
David Benyon: Realism
• Stereo effects were very important to realism
• The uncanny valley exists in places as well as characters– A slightly wrong
perspective– A strange field of view
• Non-realism can be exploited for interesting experiences
• Presence of the non-realistic - the sense of ‘being there’ in au unreal place - can be powerful.
Question 3 Connectedness
• Answer sequence: – David– Thomas– Wijnand– George
David Benyon: Connectedness
• Presence is not a cognitive phenomena; it is about embodied interaction
• Wearing a head mounted display (HMD) and you lose your hands and feet!
• Current VR is inherently disembodied - need for more sensors and full body interaction.
David Benyon: Connectedness
• As remote control and extensions of physicality.– The blind person’s stick
• Need to increase influence of haptics in interactions
• Connectedness is also about connecting people with people: social presence– Phones, connected rings/earrings, glowing
picture frame
Thomas Stoffregen: Tele-Surgery
• Do we want the surgeon to feel as if they are working on a real body?
• Or, do we want the surgeon to complete the surgery successfully?
• These are not necessarily the same• If we can get “successful performance”, then “it
feels real” may be irrelevant
Thomas Stoffregen: Believability and action
• Two concepts of believability, or fidelity• 1. Experiential Fidelity
– What subjective experiences are associated with an interface or virtual environment
• Does the system “feel real”?
• 2. Action Fidelity– Do users’ actions in a virtual environment
resemble their actions in the corresponding real environment?
• Was the operation a success?
Thomas Stoffregen: Empirical question
• Experiential Fidelity may not imply action fidelity• Action Fidelity may not imply experiential fidelity• Research is needed to understand how they are
related– Does experiential fidelity provide a significant
improvement in performance?
Thomas Stoffregen: Specific applications
• If Action Fidelity and Experiential Fidelity are different, then we need to ask– In what applications should we try to design for
Experiential Fidelity (when do we want a virtual environment to “feel real”?)
• Example: Entertainment applications (e.g., video games)– In what applications should we try to design for
Action Fidelity (when do we want users to move in a virtual environment as the move in the real world?
• Example: Process control (e.g., control rooms)– In what applications do we want both types of
fidelity?• Example: Vehicle simulators?• It is possible to have both Experiential and Action fidelity
Figure 1: Real and virtual interactive characters in MR
Figure 3: Close up of AR HMD & camera h/w components
Figure 2: Overview of believable animation process for improved presence and interaction in MR
George Papagiannakis: Connectedness
Question 4 Immersion
• Answer sequence: – Wijnand– George– David– Thomas
Panorma Mesdag – View of Scheveningen, 1881
Presence experiment at GeorgiaTech
M. Botvinick (2004) Science, Vol 305, Issue 5685, 782-783
(2) This hand is my hand…
George Papagiannakis: Believability and Immersion in
MR• Mobile Mixed Reality Presence Environments
– Dramaturgical Storytelling Virtual Human Simulation in AR (LIFEPLUS EU Project: http://www.miralab.unige.ch/subpages/lifeplus)
• Personalized learning and training environments– Training-Interactivity in MR populated Industrial
Environments (STAR EU Project: http://review.realviz.com/)
- [Papagiannakis05b] G.Papagiannakis, S. Schertenleib, B. O’Kennedy , M. Poizat, N.Magnenat-Thalmann, A. Stoddart, D.Thalmann, "Mixing Virtual and Real scenes in the site of ancient Pompeii", Computer Animation and Virtual Worlds, p 11-24, Volume 16, Issue 1, February 2005
- [Papagiannakis05a] G. Papagiannakis, H. Kim, N. Magnenat-Thalmann, "Believability and Presence in Mobile Mixed Reality Environments", IEEE VR2005 Workshop on Virtuality Structures, February 2005
David Benyon: Immersion
• Breaks in presence and Fragments of presence
• Immersion for how long?
• More important than believability and realism.
• The Book ‘Problem’ in Presence research
David Benyon: Immersion• Attention and divided attention• The mixed reality of car driving.• Maintaining immersion (attention?) when moving
from one experience to another.
Thomas Stoffregen: Mixed-Reality
• All interfaces and virtual environments involve mixed reality
• This is especially clear in the context of multisensory perception
• Stimulation of the vestibular, haptic, and kinesthetic systems always is influenced by the “real world”
Conclusions