Designing sonic spaces

8/17/2019 Designing sonic spaces

1/7

Chueng, P., Marsden, P., Designing Auditory Spaces to Support Sense of Place: The Role of Expectation. Position paper for The Role of

Place in On-line Communities Workshop, CSCW2002, New Orleans, November 2002. http://scom.hud.ac.uk/scompc2/research.htm

Designing Auditory Spaces to Support Sense of Place:

The Role of Expectation

Priscilla Chueng & Phil MarsdenInteractive Technology Research Group

School of Computing and Engineering

University of Huddersfield

Huddersfield, West Yorkshire, UK

{p.chueng; p.h.marsden @hud.ac.uk}

ABSTRACT

This paper reviews current approaches to designing virtual

environments and investigates aspects of influence in

designing auditory spaces to support novel forms ofinteraction in virtual places. Initial research on human

imagined sounds from places has identified ‘expectation’ as

an important psychological construct, which must be

considered when designing sounds for virtual places. The

research work continues to provide evidence that there are

differences between sounds people expect to hear in places

and sounds recorded in real life places. Instead of designing

realistic virtual spaces, the paper suggests a user’s sense of

presence as a measure of the user’s experience in virtual

environment. The results indicate that using highly

expected sounds increases users’ sense of presence. As

such, it is to propose that designing auditory spaces using

expectations as perceived affordance is perhaps a minimalway to design auditory spaces that support sense of place,

hence provoke the emergence of virtual communities.

Future work of the project is discussed.

Keywords

Audio, Auditory Display, Sound, Place, VirtualEnvironments, Presence

INTRODUCTION

Modern communication technologies, like virtual

environments, aim to create virtual spaces in which people

can exchange ideas. These virtual environments support

social interaction in many different forms through thesupporting of the emergence of virtual places.

There are various types of virtual environments deriving

from such diverse origins of technological platforms and

design of applications. Collaborative virtual environments

such as DIVE [15] and MASSIVE [25], support group work

through spatial virtual worlds which allow embodied

interaction. Workspaces can be enhanced through media-

spaces, which aim to support peripheral awareness whilst

maintaining working relationships between physically

remote colleagues through audio video communication

facilities [6]. While tele-presence systems try to imitate

reality for users to experience a remote physical space

through an immersive display, augmented reality systems

integrate virtual and physical space for a combination of

digital and physical information.Despite their deviated natures of design, these virtual

environments mainly 1) aim to create virtual, hybrid spaces

or space-less systems [27] in order to support people’s

daily lives and work through novel forms of social

interaction among virtual communities, 2) allow the use of

multimedia communication through human sensory

channels such as vision, auditory, haptic, etc.

USER’S EXPERIENCE IN VIRTUAL SPACES

Humans have a tendency to imitate reality. Early designs of

simulators and tele-operations mainly focus on emulating

real task. These historical influences have resulted in

current research focusing on the design of virtual systems,which centralize around spatial models of interaction [3] and

real life metaphors, in particularly designing affordances for

virtual space. The design involves implementation of

spatial topographical data and is heavily dependent upon

environmental stimuli and users’ perception and imagery. In

these environments, recognition and interpretation of

stimuli input that registers our senses are taken as important

aspects to optimise the virtual experience.

However, perceptual realism attained through accurate

perspective projection may not always be the best approach

to effectively convey information. For example, intentional

distortion is employed in cartography to exaggerate features

[30] and artistic dis tortion to represent a realistic‘appearance’ [24]. Furthermore, Gaver suggested that

collaborative systems using audio-visual technologies have

different properties for perception and interaction [19]. The

nature of collaboration can be very different from those

happening in our everyday world, which is rich in

perceptual information from objects and events. In such,

virtual environments can be designed with challenges and

opportunities this technology has to offer.

“After all, it is humans who are interacting in the virtual

environment”. Dourish’s notion of ‘place’ [27] puts forward


2/7



the idea that the properties of space have little bearing on

shaping virtual communities, but only on our behaviour and

actions within them. As he states: “Space is the

opportunity; place is the understood reality”. We design

space by mean of appropriate behavioural framing” [27] for

action and behaviour to happen in the space.

We can argue that designing virtual environments, which

mimic spatial orientation of the real world, is perhaps too

simplistic a technological solution. A deviation from

realistic patterns of information representation may allow

virtual experience to be more effective than reality itself. In

such, the concept of ‘presence’ is introduced as a minimal

approach to designing virtual environments, taking away

the realism goal of conventional virtual design.

The concept of presence [29] has becoming well accepted

as the key concept to redefine virtual environments.

‘Presence’ is defined as the perceptual illusion of "being

there" in a mediated environment [29,34] for an engagingexperience. An illusion of non-mediation occurs when user

fails to perceive or acknowledge the existence of a medium

in the environment and responds as if the medium were not

there.

Indeed, we need not denied one or another of the above

approaches. Aimed at combining useful elements from each

of the approaches mentioned for an engaging virtual

experience, this project explores an alternative way of using

auditory space to support sense of place, hence provoke

the emergence of virtual communities. It is important to

understand the users’ perceptual responses to auditory

cues, when they are combined with the understanding ofcontext awareness for interaction in virtual spaces.

AUDITORY SPACES

Although vision tends to be the dominant sensory channel,

it is reported that auditory cues are important to the

establishment of a ‘full’ sense of presence in virtual

environments. Research on suddenly deafened adults

reported that subjects feel a sense of disconnection from

their surrounding [23]. Sound conveys a wide range of

information about events and surroundings, providing

timely information and constant awareness of people and

background events [32,11]. The "Cocktail Party Effect"

noted that our auditory channel monitors several audio

streams simultaneously, selectively focusing on any one,

and placing the rest into the background of our awareness

[26]. Not only that but sound is temporal, it is perceived

spatially in relation to the listener’s location [36]. Absence

of sound in the environment can result in inaccurate spatial

orientation [21]. This unique characteristic of sound helps

users’ navigation in a virtual system [13] and also assists in

identifying the directions and locations of rooms, objects or

even events in virtual spaces [2].

Work done by acoustics research, outlined above is further

compliment by the notion that auditory space is a truly

shared space, which can be all pervasive [27]. For example,

people may attend to different visual presentation but at the

same time, share the same auditory space. Furthermore

sound from one space can reach out to another space in a

way that visual stimulus cannot.

The ecological approach to auditory perception

The ecological approach to auditory perception [16,17]

notes that although we understand scientific descriptions

of auditory perception, phenomenally we don’t ‘hear’

acoustic signals or sound waves but instead we hear events

[16,17]: the sounds of people and things moving, changing,

beginning and ending, forever interdependent of the

dynamics of the present moment. We ‘hear’ the sound of

silence for example. We hear the semantics of sound

producing objects, events and the environment. This is

what we call everyday listening. Ecological approach is the

central approach of this research project.

Acknowledging the importance of auditory channels, there

has been an increasing activity in the area of auditorydisplay research. This research project focuses on the

sound generated from surroundings, which has been termed

ecological sound, environment sound, sound-scapes,

ambient sound or everyday listening sounds, respectively.

EMPIRICAL WORK

The research aims to understand the users’ perceptual

responses to auditory cues, when they are combined with

the understanding of context awareness for interaction in

virtual spaces, and how it can be used to support a sense of

place in virtual spaces.

The main contribution to knowledge made by the research

will be to provide virtual environment designers withguidelines for designing sound for virtual spaces that will

provoke a shared sense of place amongst virtual

communities.

A range of user studies and experiments has been carried

out aiming to investigate guidelines to design minimal

sound in virtual spaces that support sense of place among

virtual communities.

INITIAL STUDY: Sounds we imagine to hear in places

The research work started off by studying user’s

interpretations of experiencing real life auditory spaces. An

open interview was carried out to investigate the sounds

which people imagine they can hear in four different real life places, namely: a pub, a supermarket, a high street and a

park.

Results from the initial study [8] showed participants

tended to report sounds according to the context of the

place from sound producing objects and events, such as,

‘people talking’, ‘people shouting’, the’ wind’ etc. [16, 17].

From these results, it is to propose that – expectation and

discrimination [8] can be the important perceived

affordances of sound producing objects that contribute to

sense of place for users and contribute to a feeling of


3/7



presence. Expectation defines the extent to which a person

will expect to hear a sound in a particular place;

Discrimination is the extent to which a sound will help to

uniquely identify a particular place. As such, the most

useful sounds are those that elicit either high expectation or

high discrimination from the listener.

ONLINE SURVEY: How much do we expect and

discriminate sounds in places?

We further confirmed this outcome with an online survey

with aims at gathering quantitatively ratings of the range of

highly expected and highly discriminated sound producing

objects and events reported from the initial study. The place

chosen for this study was the ‘Pub’.

Method

For each sound listed, 20 participants of age 18 to 40 years

from the UK rated their expectation and discrimination by

answering two questions related to the Pub. A sample of

the questionnaire [10] can be seen at the web site ~

http://www.supersurvey.com/cgi-bin/surveys/s10492.pl. No

time restriction was placed on participant before stating

their answers.

Question for level of expectation: How frequently do you

expect to hear each of these sounds? (Rating: High,

moderately high, moderate, moderately low, low, not at all)

Question for level of discrimination: Using your judgement,

to what extent do you think each of these sounds is unique

to a pub environment (Rating: High, moderately high,

moderate, moderately low, low, not at all) .

Data analysisData gathered from the initial study and from the online

survey on the pub were analysed using quantitative and

qualitative techniques. Figure 1 shows the comparison of

expectation scores from the two studies.

Sounds Expectation Discrimination

initial

study

online

survey

initial

study

online

survey

people talking 4.75 4.55 1.00 2.55

music 3.75 3.75 1.00 2.95

foot steps 3.5 1.75 2.00 1.05

glasses clink 3.5 3.05 5.00 3.7

tills open & close 2 2.3 3.00 2.05

chair & table move 1.75 2.6 4.00 2.55

people laughing 2.25 3.85 3.00 3.2

gambling machine 2 2.5 4.00 2.65

traffic 0.75 1.15 3.00 0.7

freezer humming 0.5 0.35 4.00 0.6

Figure 1 Level of expectation from online survey.

Expectation scores from initial study is generated by

transformed mean of total count reported from participants

on the particular sound into five points scale. Expectation

scores from online survey shown are the means of ratings

collected from participants.

Discrimination scores of initial study are taken from

qualitatively comparing count of the particular sound

reported across four places and transformed into five pointsscale. Discrimination of online survey is the means of

ratings collected from participants.

Results

Overall results show low differences between expectation

scores between the two studies but higher differences

between discrimination scores between two studies. T-tests

shows that there are no significance differences between 1)

expectation ratings from initial study and online survey (t=-

0.395, df=9, p= 0.702); 2) discrimination ratings from initial

study and online survey (t=1.545, df=9, p=0.157). As such,

the online survey has proven the validity of the proposed

expectation and discrimination as important roles on people

sounds imagined in places from initial study.

COMPARISON STUDY: Real life auditory spaces vs.

expected auditory spaces.

Analysis from the above user studies [8,9] suggests that

what people expect to hear in real life places can be

significantly different from what they actually hear in real

life places. Expectation as a dimension for sound design is

not new for film sound designers [7]. For example, the

dinosaur’s foot steps in the movie “Jurassic Park” was

created by recording the sound of a heavy weight being

dropping from a great height. There is however no empirical

work to date carried out on this area. Thus, this study aims

to investigate the differences between sounds recordedfrom real life places and sounds reported from people’s

expectation. The pub is picked as one of the places for this

study for its variety of sound producing objects in a closure

environment. We reported the qualitative analysis of this

study on Pub.

Method

Four places are chosen to analyse: Pub, Supermarket. High

Street and Train Station. To reproduce realistic sound,

binaural recording [5,22] methods were used for recording.

A pair of mini microphones made by AKG model 417pp was

attached to both ears of the sound engineer who stands in

the middle of the places to imitate real life hearing of participants. Sound clips with length of ten minutes were

digitised to 44khz and 16 bit from a DAT tape to a Cd -ROM.

Sound clips were played back using headphones make by

Sennheiser model eH2270.

Data analysis and Result

Analysis on recorded sound is done in three fold. We

investigates 1) sounds that happened continuously

through out not less then two third of the duration of

recordings. 2) forefront sounds which catches the most

attention 3) sounds happened in serendipity. We then

compare the data to the sounds reported from initial study.


4/7



Preliminary result from qualitative study by listening to

recorded pub shown some differences between real life

sound and people’s expectation. Qualitative analysis on

sounds recorded from real life places have shown many

layers of information depending on the sound producing

objects in the environment. Sounds that people expect to

hear are consists of 1) sounds profoundly heard from a

place which generated from typical activity that

characterised the place and 2) sound heard continuously,

generated from a sound producing object. The major

differences are drawn by the difference between 1) the

hearing dominance of a sound-producing object in real life

in comparison to people’s expectation and 2) the degree of

contextual information relevant to the places.

EXPERIMENT: The effect of expectation on people’s

sense of presence.

From previous studies, expectation is identified as an

important construct to be considered in sound design. Theaim of this experiment was to investigate if the expectation

of sounds in places, influence people’s sense of the

presence. During the experiment, participants’ mental

models are prompted by still image of places while listening

to audio clip recorded from real life places. Initially, four

places from the initial study were chosen for this

experiment. : Pub, Supermarket, High Street and Park.

However we encountered difficulty in recording clarity of

sound from a park. It was therefore decided that train

station was chosen to replace the Park and the Park is used

for familiarisation of participants at the beginning of each

experiment.

Hypotheses

The main hypothesis for this experiment is that a person’s

sense of presence will increase with high expectation. This

is achieved by displaying matching visual images together

with matching audio clips. Secondly, correct responses of

people’s memory task collected from places-related

questions of both sounds and visuals are expected to be

higher with high expectation. Thirdly, low expectation

(mismatched audio-visual) will result people’s irritation and

hence participants’ parasympathetic reading will be higher.

Method

The experiment uses a two factorial within subjects design.

The dependent variables are (1) total weight collected fromsense of presence questionnaire (rank with the scale of

five). (2) Number of correct responses collected from places -

related questions of sounds and visuals. 3) Electro dermal

reading.

To reproduce realistic sound, binaural recording [5,22]

methods were used for recording. A pair of mini

microphones made by AKG model 417pp was attached to

both ears of the sound engineer who stands in the middle of

the places to imitate real life hearing of participants. Sound

clips with length of 60 seconds were digitised to 44khz and

16 bit from a DAT tape to a PC computer. Images sized 800 x

600 pixels were captured by digital camera.

During the experiment, the facilitator uses a Multimedia

Pentium PC computer to play back sequence of audio and

visual images through Macromedia Director. Images were

projected to a white screen in a darken room using a digital

projector whilst sound clips were played back using

headphones make by Sennheiser model eH2270.

Twenty participants took part in this experiment. They were

a mixture of students and staff with a variant age of 22 to 50

from schools of computing, engineering and music, at the

University of Huddersfield. None had hearing problems andall either had normal vision or wore prescribed corrective

lenses. Participants are exposed to matching or

mismatching visual stimuli while listening to sound clips

recorded from real life places.

There are four variations of audio-visual pairs combination.

Five participants were chosen randomly for each variation.

Every variation consists of two pairs of matching audio-

visuals and two pairs mismatching audio-visuals, non-

repeated in random order. They are asked to 1) record their sense of presence by

answering a sound related presence questionnaire, 2)

answer place-related questions from each sound listened to

and each image viewed. During every audio-visual display,

their readings of parasympathetic responses of arousal are

recorded using psycho-physiological sensors.

Each participant takes approximately twenty minutes to

finish viewing five audio-visual displays. Audio-visual

stimuli of the park were used for familiarisation purpose.

Following each display, they answer nine sounds-related

presence questionnaire taken from SVUP [28] with five-

scale rating, and places-related questions (three taken from

information of sound clip and three taken from information

of visual displayed) using paper and pen. Parasympathetic

Pub Supermarket

High Street Train Station

Figure 2 Visual images of real life places.


5/7



responses of Galvanic Skin Resistance (GSR) and Blood

Volume Pulse (BVP) are captured using Datalab 2000 and

software Biobench.

Results

The overall mean of presence score is higher for matchingaudio-visual than mismatching audio-visual. T-test (t=5.990,

df=78, p


6/7



completion of user’s navigation task, b) user’s sense of

presence c) the ability of user to identify a virtual place.

2) Non-verbal sound generated by users in multi-user

virtual environments such as footsteps and rustling of

clothes is the key element to investigate in this work.The study aims to identify minimum sounds needed to

provoke users co-presence.

An evaluation study will conclude the research project by

building virtual auditory spaces in virtual environments

based on the outcomes gathered from our previous studies

and experiments. The research will provide guidelines for

future virtual environments developers with regard to

designing minimum auditory space to support virtual places

for an effective user experience.

ACKNOWLEDGMENTS

This project is funded by the DRIVE project grant,

University of Huddersfield held by Janet Finlay and Phil

Marsden. I am grateful to Jacqueline Brodie from Brunel

University for valuable discussions about place and virtual

communities and added readability of the initial submission

of this paper. I also thank Andrew Monk from the

University of York for valuable comments on the paper.

REFERENCES

1. Ark, W. and Selker, T. 1999. A Look at Human

Interaction with Pervasive Computers. IBM Systems

Journal , 38(4), 504-507.

2. Begault, D. R. (1994). 3-D sound for virtual reality.

New York: Academic Press.

3. Benford, S. and Fahlen, L. “A Spatial Model ofInteraction in Large Virtual Environments”, Proc. Third

European Conference on Computer-Supported

Cooperative Work ECSCW’93, Milano, Italy, September

13–17, 1993.

4. Biocca, F., & Levy, M. R. (1995). Communication in the

age of virtual reality . Hillsdale, NJ: Lawrence Erlbaum

Associates.

5. Blauert J., Spatial Hearing, The Psychophysics of

Human Sound Localization, MIT Press, Cambridge, MA

1997.

6. Bly, S. A., Harrison, S. R., Irwin, S., Media Spaces:

Video, Audio, and Computing, Communications of theACM, No.1 Vol.36, January 1993, pp. 28-47.

7. Bordwell, D. and Thompson, K. Film Art: An

Introduction. McGraw-Hill, August 5, 1996

8. Chueng, P., Designing sound canvas: The role of

expectation and discrimination, in extended proceedings

of CHI'012 Conf . on Human Factors in Computing

Systems. 2002

9. Chueng, P., Minimal ecological sound design: the issues

of presence in virtual environments, in extended

proceedings of British HCI 2002, London. 2002

10. Chueng, P. Online questionnaire of sounds in pub.

http://www.supersurvey.com/cgi-bin/surveys/s10492.pl.

2002

11. Cohen, J. Monitoring background activities. In Kramer

G. (ed), Auditory Display, SFI Studies in the Sciences ofComplexity, Proc. Vol. XVIII, Addison-Wesley, 1994.

12. Curtis, P and Nichols, D “MUDs Grow Up: Social Virtual

Reality in the Real World”, Proc. IEEE Computer

Conference, pp. 139–200, January 1994.

13. Darken, R. Sibert, J. “A toolset for navigation in virtual

environments”. UIST’93, ACM Now 3-5, 1993.

14. Dourish, P. 2001. Where the Action Is: The Foundations

of Embodied Interaction. Cambridge: MIT Press.

15. Fahlén, L. E., Brown C. G., Stahl, O., Carlsson, C., A

Space Based Model for User Interaction in Shared

Synthetic Environments, in Proc. InterCHI’93,

Amsterdam, 1993, ACM Press. Proc, CHI’97, pp 234-241,ACM Press, 1997.

16. Gaver, W. How do we hear in the world? explorations in

ecological acoustics. Ecological Psychology,

5(4):285313, 1993.

17. Gaver, W. What in the world do we hear? an ecological

approach to auditory event perception. Ecological

Psychology, 5(1):129, 1993.

18. Gaver, W.W. (1991). Technology affordances. In CHI

‘91, Human factors in computing systems conference

proceedings on Reaching through technology (pp. 79 –

84). New Orleans, LA USA: ACM Press.

19. Gaver, W. The Affordances of Media Spaces for

Collaboration. In Proceedings of ACM CSCW'92

Conference on Computer Supported Cooperative Work,

Toronto, Ontario, pages 17-24. ACM, New York,

November 1992.

20. Gibson, J.J. (1979) The Ecological Approach to Visual

Perception, Boston, MA: Houghton Mifflin.

21. Gillingham, K.K., (1992), The spatial disorientation

problem in the United States Air Force, Journal of

Vertibular Research, vol. 2, pp. 297-306.

22. Gilkey, R. H. and Anderson, T. R., "Binaural and spatial

hearing in real and virtual environments." Mahwah, NJ:Lawrence Erlbaum Associates, 1997.

23. Gilkey, R.H. and Weisenberger, J.M. (1995) The sense of

presence for the suddenly deafened adult, Presence, Vol.

4, No. 4, 1995, pp. 357-363

24. Gombrich, E. H. 1977, Art and Illussion: A study in the

Psychology if Pictorial Representation, 5th edition,

Oxford: Phaidon Press.

25. Greenhalgh, C. and Benford, S., MASSIVE: A Virtual

Reality System for Tele-conferencing, ACM


7/7



Transactions on Computer Human Interaction (TOCHI),

ACM Press, 1995.

26. Handel, S. Listening: An Introduction to the Perception

of Auditory Events. MIT Press, 1989.

27. Harrison, S. and Dourish, P., Re-Place-ing Space: TheRoles of Place and Space in Collaborative Systems.

Proceedings of ACM Conference on CSCW , Boston,

USA November 1996: 67-76, 1996.

28. Larsson P., Västfjäll D., & Kleiner M. The actor-observer

effect in virtual reality presentations. CyberPsychology

& Behavior, 4(2), 239-246 (2001).

29. Lombard, M., & Ditton, T. At the heart of it all: The

concept of presence. Journal of Computer Mediated-

Communication [On-line], 3(2) (1997).

http://www.ascusc.org/jcmc/vol3/issue2/lombard.html

30. Monmonier, M., 1991, How to lie with Maps, Chicago:

University of Chicago Press.

31. Moran, T. and Dourish, P. 2001. Context-Aware

Computing: A Special Issue of Human Computer

Interaction, 16(2-4).32. Mynatt, E.D., Back, M., Want, R. and Frederick, R.

"Audio Aura: Light-Weight Audio Augmented Reality".

Proceedings of UIST '97 User Interface Software and

Technology Symposium, Banff, Canada, Oct 15-17, 1997.

33. Slater M., Usoh M., & Steed A. Depth of presence in

virtual environments. Presence: Teleoperators and

Virtual Environments, 1, 221-233 (1994)

34. Active Worlds, http://www.activeworlds.com

Documents

Designing sonic spaces