Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
The Influence of Rendering Styles on Participant Responses in Immersive
Virtual Environments
Mel SlaterInstitució Catalana de Recerca i Estudis Avançats (ICREA)
@ Universitat Politècnica de Catalunya&
Department of Computer Science, UCLhttp://www.lsi.upc.edu/~melslaterhttp://www.lsi.upc.edu/~moving
www.icrea.es
Global Illumination – Who Cares?
• Why spend effort and resources on ‘global illumination’ (eg, ray tracing, radiosity, photon mapping,…)?
• Scientific and practical question?• What is its impact on people who are in
immersive virtual environments?• Brings together my two strands of research
• Presence in virtual environments• Real-time global illumination
Outline
• Introduction • Immersive VEs and Presence
• Real-time Ray Tracing and Presence• An experiment
• Virtual Light Field • Real-time global illumination for the Cave
• Conclusions
Immersive Virtual Environment• Displays in all sensory systems• Fully encloses participant in the displays• Tracks the head, limbs, body• determines the optical, auditory... sensory
data as a function of (head) tracking
Immersive Virtual Environment
• ‘Cave’ – immersive projection system
Immersion: the technology• Displays
• Field of view (visual), stereo, resolution• Frame rate• Multiple sensory modalities (visual, auditory, haptics,
olfactory)• Content, level of realism
• Interaction• Head-tracked (body tracking), capability for interaction• latency
• General• Correlation between sensor data and overall proprioception• Transparency of devices (weight, cables, etc)
Reported Presence…• ‘The sense of “being there”’ (Held & Durlach,
Sheridan, Zeltzer: premier issue of PRESENCE, 1992)
• ‘A perceptual illusion of nonmediation’ (Lombard and Ditton, 1997)
• ‘A mental state in which a user feels physically present within the computer-mediated environment’ (Draper & Kaber, 1998)
• ‘The subjective experience of being in one place or environment, even when one is physically situated in another’ (Witmer & Singer, 1998)
Presence• Successful substitution of real sense
data by computer generated sense data• ‘Successful’ – response is as if the
sense data were from a real source• ‘Response’ –
• Low level physiological → high levelcognitive and emotional
• Includes verbal responses about ‘being there’
Presence• The evidence that presence occurs is strong• People tend to respond automatically in a realistic
manner• In spite of knowing for sure that what they are
experiencing is unreal
The ‘Pit Room’ ExperimentsLong history of usingthis environment for experimentation: eg
•Slater, Usoh, Steed (1999)
•Meehan et al (2002)
•Zimmons and Panter (2003)
Fear of Heights• Meehan/Insko (2002)
exposed 10, 52 and 33 subjects in 3 different studies.
• Heart rate increasewhen in the pit room.
• Static haptics further significantly increased heart rate.
• Heart rate correlatedwith subjective self-report of presence level.
Visual Realism
• In a between-groups study utilising the ‘pit room’Zimmons et al. (2003) used 5 levels of rendering:
• From wire frame to radiosity (global illumination)• All subjects showed increased heart rate when approach
the precipice in all conditions• No significant differences between the
conditions in heart rate or reported presence.• So, from the point of view of presence
is visual realism not important?
Outline
• Introduction • Immersive VEs and Presence
• Real-time Ray Tracing and Presence• An experiment
• Virtual Light Field • Real-time global illumination for the Cave
• Conclusions
Does Ray Tracing Help?• What do we get with ray tracing?
• Shadows (umbras only)• Reflections
• Real-time ray-tracing (parallelism + GPU) offers chance to investigate impact of dynamic changes (shadows+reflections) on presence
Experimental Design• 33 people• Displayed with V8 HMD + polhemus
tracking• Pit room rendered in two ways
• Real-time Ray Tracing (RT) with reflections and shadows of crude virtual body
• Ray Casting (RC) ~ OpenGL shading (no reflections/shadows)
• Both between 13-15 fps
Experimental Design• Participants divided into two groups
• 16 experienced first RT and then RC• 17 experienced first RC and then RT• Both groups had relaxation periods for ‘baseline’
readings
• This design is both between groups• Consider 1st results only
• Within groups• Consider comparisons between the two results• (with high danger of interference between the two
experiences!)
Scenario:
VIDEO 1:12, 2:08
Measured Variables
• Standard presence questionnaire• Skin conductance• Heart rate and respiration• Demographic variables – age, gender• Background
• Programming knowledge, game playing, prior experience of VR, etc
Between Group Analysis
• Reported presence is significantly associated with:
• Ray tracing (higher than for RC)
• Prior VR experience
Within Group Analysis
Physiological Responses
• Participants experienced 3 parts to the experiment• A baseline period – just standing looking
around (2 minutes)• First exposure (RT or RC) (3 minutes)• Second exposure (RC or RT) (3 minutes)
• Baseline allows incorporation of natural differences between people
• Recorded electrodermal activity & ECG
Electrodermal Activity
• A change in electrical properties of the skin, associated with sweat, which typically indicates arousal.
• Arousal in response to events leads to skin conductance responses (SCR) (which also occur spontaneously)
• The number of SCRs increases with the level of arousal
• EDA sampled at 32Hz
Skin Conductance Responses
Example from one participant in the baseline period
SCR Between Groups Analysis
• The number of SCRs per unit time should follow a Poisson distribution (random events in time)
• If there is no difference in arousal between the 2 conditions then the means should be approx the same
• Can use log-linear regression to regress the number of SCRs on Condition + other ‘nuisance’variables
• Baseline number of SCRs• Gender, age, VR experience, game playing, etc..
Result• The analysis showed that the following variables were
significantly associated with number of SCRs• Baseline number of SCRs (+ve)• Age (+ve)• Gender (higher for females)• Prior exposure to VR (-ve)• Programming knowledge (+ve)• Subjective report of physiological responses (+ve)• Condition (RT higher than RC)
• Also there was a strong positive association with the questionnaire based presence score
• n = 29 (some Skin Conductance data not usable)
Interpretation
• Taking into account other factors• The evidence suggests that the RT condition led
to greater arousal than the RC condition• From this alone we cannot determine the
associated valence
ECG Wave Form
• ECG waveform during first few seconds of baseline for one participant
Heart Rate and HRV
• ECG recordings give the raw waveforms sampled at 256Hz
• Extract the QRS complexes from the raw signal
• Determine the RR intervals• Derive HR (beats per minute)• HR variability (various measures)
Heart Rate and HRV
• Interpretation – under conditions of stress• HRV should be lower• HR should be higher
• Simple overall measure is to take response as• Y = HR / SD(HR)• Higher values indicate lower HRV and/or
higher HR• We should see a difference between RT and RC
HR and HRV Analysis
• As before regress Y on • Baseline-Y• Condition (RT / RC)• Other ‘nuisance’ variables
• age, gender, previous VR, etc
Results HR and HRV
• The analysis showed that Y was associated with• Baseline-Y (+ve)• Gender (-ve)• Status (+ve for postgrads)• Condition (higher for RT compared to RC)
Overall Interpretation• Taking into account other factors
• Greater sympathetic nervous system arousal in RT condition
• RT condition had signs that would be associated with higher stress than RC condition
• RT associated with higher subjective presence score than RC
• Unknown whether this is the effect of dynamic shadows and reflections or the impact of overall greater realism
Outline
• Introduction • Immersive VEs and Presence
• Real-time Ray Tracing and Presence• An experiment
• Virtual Light Field • Real-time global illumination for the Cave
• Conclusions
Virtual Light Field
• The Virtual Light Field (VLF) is based on a discretisation of ray-space for representation and walkthrough of a virtual scene. It also uses this ray-parameterisation for propagation of radiance
• Global illumination solution for diffuse + specularsurfaces
• Easy extension for BRDFs
• Uses a partition of a hemisphere over a scene• Each ray direction spawns a field of parallel rays (PSF)
Parallel Subfields
• For any discretiseddirection, consider the set of rays parallel to it
• Parallel subfield of rays (PSF)
• Rays originate from a 2D grid perpendicular to the direction
Discretising Ray-space :Placing rays in space - Tiling
• Subdivide the grid of rays into Tiles
• Each Tile contains a number of Cells
• Rays (=Cells) in a Tile have coherence with respect to the objects they intersect
Discretising Ray-space :Ray-Polygon intersections
• Cells in a Tile represent parallel rays that intersect polygons in the scene
• These intersections for the Tile can be found by OpenGL rasterisation
• Tiles store a list of intersected polygons
• Neighbouring rays are likely to intersect the same polygons
Propagation and Rendering
• Starting from light sources energy is propagated through the scene
• At rendering time each ray segment has an associated radiance
• Can be rendered using much the same method as traditional light fields
• But now global illumination for virtual scenes
A Faster VLF
• The CPU implementation of the VLF could spend days propagating simple scenes (100s polygons)
• Due to clipping operations in tile-tile transfers• Also because of the O(n2) complexity using
OpenGL visibility determination
Point sampling
• The new approaches uses point sampling• Adaptively chosen sampling density
proportional to the projected area of a face onto the PSF plane
• This yields a speed-up of ~5x
Incremental visibility
• By pre-sorting polygons in the PSF directions, visibility can be pre-calculated
• The pre-sorting step employs a BSP subdivision step to optimise sorting
• For scenes >10,000 polygons the sorting step takes less time than a propagation iteration
• Incremental visibility yields a speed-up in excess of 10x (for a total of ~50x)
• Propagation scaling becomes O(n)
VLF-GPU
• Incremental visibility and adaptive point sampling have been ported to a GPU
• Uses a sparse tiled data structure suitable for the GPU
• Exploits fp16 textures and FBOs (framebufferobjects) for efficient render-to-texture functionality
• GPU offers a speed-up of ~10x (total: ~500x)
Scaling of propagation time with scene complexity
0
50
100
150
200
250
300
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000
Polygons
Tim
e(s
ecs)
Propagation times
GPU-VLF• Propagation times with >10K polygons
are minutes• Similar running times to Coombe’s* GPU
radiosity, but with specular reflections too… another example
• An XVR renderer is in the works for use with HMD and CAVETM
* Coombe, G.; Harris, M.J. & Lastra, A. “Radiosity on graphics hardware”GI '04: Proceedings of the 2004 conference on Graphics interface, Canadian Human-Computer Communications Society, 2004, 161-168
XVR• XVR is a platform independent virtual reality
system developed at the PERCRO lab in Pisa (PRESENCCIA Project)
• VLF being ported to run under XVR• Next experiments will be in the Cave with
capability of global illumination beyond ray tracing
• Unravel the issues about whether it is simply greater dynamism (shadows and reflections)
• Or overall more realism• www.vrmedia.it
XVR Demo
• Grotto demo• Bugatti
Outline
• Introduction • Immersive VEs and Presence
• Real-time Ray Tracing and Presence• An experiment
• Virtual Light Field • Real-time global illumination for the Cave
• Conclusions
Conclusion• Computer graphics is for … people• There is the aim for greater realism, faster speed,
better animations – the technical and engineering side.
• There is also the ‘so what?’ side – this is the scientific side
• What impact does what we do have on the people who use it.
• ‘Presence’ is one such issue (appropriate to immersive systems)
The Virtual Light Field Team
Pankaj Khanna Jesper Mortensen Insu Yu
Department of Computer Science, [email protected]
Acknowledgements
• UK EPSRC Project “Presence in the Virtual Light Field” – for funding
• FET PRESENCCIA Integrated Project• Christoph Guger, Gtec, Austria, for help with
physiological signals analysis.