Games with Sounds: Games with Sounds:

the Blind Navigation the Blind Navigation

and Target Acquisition and Target Acquisition

Antti Nyman

Department of Computer Sciences

University of Tampere Finland

Antti.Nyman@uta.fi

March, 2005

AAFG 2005

A. Nyman p 01_16 21.03.2005

Games with Sounds

Blind users do not prefer mouse (lack of feedback on movement &

position).

Making use of spatial sound has been suggested in [1, 4, 5].

Problems include the difficulty of recognizing simultaneously playing

sounds and mixing up of directions (forward <-> backward) [3].

Earlier studies on sound grids have shown that easily recognized

sound in the center of the grid reduces the navigation time and that a

“landmark” technique used by cane-travelers could be used to assist

blind persons in identifying different areas [5].

Commercial games could be modified and new game concepts

created to better support blind navigation within the game field [2].

The task is to optimize/minimize the number of sounds (3 or 6)

employed to mark 25 positions in the virtual sound grid (5 x 5) and to

estimate navigation performance with the use of conventional mouse

input in blind conditions.

A. Nyman p 02_16 21.03.2005

Demo 1

center – 1

angles – 4

middle1 – 4

middle2 – 4

middle3 – 6

middle4 – 6

= 25

Games with Sounds

Typographic Semantics of Web

Pages, P. Gorny, 2000 [3].

A. Nyman p 03_16 21.03.2005

Games with Sounds

Four subjects (two females, two males) with normal hearing and seeing

abilities participated in the tests.

During the test the subjects were blindfolded and used headphones to

minimize the effect of distractions.

Each subject played the game twice: with 3-sound grid and with 6-

sound grid. Each game consisted of 25 tasks presented in a random

order.

Application stored the track of mouse movement, the passed time, the

traveled distance and the number of the passed spots for each task.

The original application was modified to use animal sound .wav-files to

make sounds easier to recognize and to make game instructions

easier.

A. Nyman p 04_16 21.03.2005

Games with Sounds

Sound 1

Sound 1

Klikkaa eläintä vasemmassa ylänurkassa.

Klikkaa eläintä joka on ylärivissä toinen vasemmalta.

Klikkaa eläintä ylärivin keskellä.

Klikkaa eläintä joka on ylärivissä toinen oikealta.

Klikkaa eläintä oikeassa ylänurkassa.

Klikkaa eläintä joka on toiseksi ylimmän rivin vasemmanpuoleisin.

Klikkaa eläintä joka on toiseksi ylimmässä rivissä toinen vasemmalta

Klikkaa eläintä toisen rivin keskellä.

Klikkaa eläintä joka on toiseksi ylimmässä rivissä toinen oikealta.

Klikkaa eläintä joka on toiseksi ylimmän rivin oikeanpuoleisin.

Klikkaa eläintä joka on keskimmäisen rivin vasemmanpuoleisin.

Klikkaa eläintä joka on keskimmäisessä rivissä toinen vasemmalta.

Klikkaa eläintä keskimmäisen rivin keskellä.

Klikkaa eläintä joka on keskimmäisessä rivissä toinen oikealta.

Klikkaa eläintä joka on keskimmäisen rivin oikeanpuoleisin…..

3 1 2 1 3

1 2 1 2 1

2 1 3 1 2

1 2 1 2 1

3 1 2 1 3

1 2 3 2 1

4 5 4 5 4

3 2 6 2 3

4 5 4 5 4

1 2 3 2 1

Sound 2

Sound 2

Sound 3

Sound 3

Sound 4

Sound 5

Sound 6

A. Nyman p 05_16 21.03.2005

Games with Sounds

0

5

10

15

20

25

30

35

40

1 2 3 4 5 6 7 8 9 1011 121314 1516 171819 2021 222324 25

Task number

Time, s

0

5

10

15

20

25

Spots, nTime Passed spots

the target acquisition time and the number of spots passed at 3

sounds used for marking 25 positions

corr. (time to n spots) = 0.96

A. Nyman p 06_16 21.03.2005

Games with Sounds

the target acquisition time and the distance to target at 3 sounds

used for marking 25 positions

corr. (time to distance) = 0.347

0

5

10

15

20

25

30

35

40

1 2 3 4 5 6 7 8 9 10 111213 141516 171819 2021 222324 25

Task number

Time, s

0

100

200

300

400

500

600

700

800

Distance, pxlTime Distance

A. Nyman p 07_16 21.03.2005

Games with Sounds

the target acquisition time and the shortest paths to target at 3

sounds used for marking 25 positions

corr. (time to path) = 0.233

0

5

10

15

20

25

30

35

40

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

Task number

Time, s

0

2

4

6

8

10

Spots, nTime Shortest path

A. Nyman p 08_16 21.03.2005

Games with Sounds

Correlation between the target

acquisition time and the distance to

target for 6- and 3-sound matrices

in 4 subjects

t = - .271, df = 3, p < 0.01

Correlation between the target

acquisition time and the number of

passed spots (targets) for 6- and 3-

sound matrices in 4 subjects

t = - .611, df = 3, p < 0.01

0.8

0.85

0.9

0.95

1

1 2 3 4

Subjects

Corr. 3 sounds 6 sounds

0

0.2

0.4

0.6

0.8

1

1 2 3 4Subjects

Corr. 3 sounds 6 sounds

A. Nyman p 09_16 21.03.2005

Games with Sounds

the overall target acquisition time for

6- and 3-sound matrices in 4

subjects

t = 2.022, df = 3, p < 0.01

the overall averaged error rates in a

target acquisition task with the use of

6- and 3-sound matrices in 4 subjects

t = 1.578, df = 3, p < 0.01

010203040506070

1 2 3 4

Subject

Time, s 6 sounds 3 sounds

02468

10121416

1 2 3 4Subject

Errors, n 6 sounds 3 sounds

A. Nyman p 10_16 21.03.2005

Games with Sounds

overall data in a target acquisition task with the use of 6- and 3-sound matrices

A. Nyman p 11_16 21.03.2005

Games with Sounds

Behavioral patterns recorded when subjects inspected game field by

moving cursor down unintentionally while trying to move it vertically to

right.

A. Nyman p 12_16 21.03.2005

Games with Sounds

Behavioral patterns recorded during an inspection game field by using

the corners as navigational “landmarks” to make target acquisition

easier.

target

target

A. Nyman p 13_16 21.03.2005

Games with Sounds

The number of passed spots has more effect on target acquisition time

than the distance between starting and ending points Other factors

influence the difficulty of the task, they are e.g. the position of the

target in the grid and the distinctiveness of the sound in the target.

Average number of errors committed and average target acquisition

time both were notably better with 6-sound grid 6 different sounds

gives more support for navigation and target acquisition.

Least errors and shortest acquisition times were related with the tasks

to point out the target in corners of the grid Corners of the grid could

be used as “landmarks” to support navigation.

A. Nyman p 14_16 21.03.2005

Games with Sounds

The sense of position could be lost totally when lifting the mouse.

The different spots with the same sounds were easily mixed. This

happened especially after getting lost and was more evident in the

3-sound grid.

In future work we could further investigate the effect of different

sound combinations in the 6-sound grid. Also the effect of subjects

selecting the sounds themselves could be investigated since it has

been noted that sounds in audio interfaces should be adaptable to

personal preferences [3].

References

[1] Bölke, L., Gorny, P. Auditory Direct Manipulation of Acoustical Objects by Blind Computer Users.

Proc. International Conference on Auditory Display 1994, 287-288.

[2] Friberg, J., Gärdenfors, D. Audio Games: New perspectives on game audio. Proc. ACE

Conference 2004.

[3] Gorny, P. Typographic Semantics of Webpages Accessible for Visually Impaired Users: Mapping

Layout and Interaction Objects to an Auditory Interaction Space. Proc. Int’l Conf. on Computers

Helping People with Special Needs 2000, 251-257.

[4] Kamel, H.M., Roth, P., Sinha, R.R. Graphics and User’s Exploration via Simple Sonics (GUESS):

Providing Interrelational Representation of Objects in a Non-visual Environment. Proc. International

Conference on Auditory Display 2001, 261-266.

[5] Parente, P., Bishop, G. BATS: The Blind Audio Tactile Mapping System.

A. Nyman p 15_16 21.03.2005

Games with Sounds

A. Nyman p 16_16 21.03.2005

Games with Sounds

Click to go the first slide