Design Process for Auditory Interfaces

1

Design Processfor Auditory Interfaces

Sylvain Daudé and Laurence Nigay

HCI team of the CLIPS-IMAG Lab.(Grenoble, France)

2

Purpose

Auditory display = interlinking location

Unified design framework for auditory interfaces

HCI Psycho-acoustics

DSP

Information visualization Computer music

Sonification

3

Motivations

Design framework existing for data visualization... ... but not for data sonification

Double interest of a design framework classification of existing results and

systems help to the auditory interfaces designer

4

Visualization process[Chi 1998]

DataBienvenue à

IHM2002Bienvenue à

IHM2002Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002Bienvenue à

IHM2002Bienvenue à

IHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

Documents set

Analytical abstraction :Data view

Data transformation

Documentslist

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

Visualization abstraction :Abstract visual space

Visualization transformation

Visual realization transformation

View : Space view

5

Visualization process [Chi 1998]

Data


Data transformation

Documentslist

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002



2D space

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

View : Space view


6

Visualization process [Chi 1998]

Data


Data transformation



2D space

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002

Bienvenue àIHM2002


Deformed viewView : Space view

7

Sonification process

Data Picture


Data transformation

Pixels columns

Sonic rendering

Aud. display transformation

Sonification transformation

Sonic abstraction :Abstract sound space

. . .

8

. . .


Data


Data transformation

Sonic rendering


Sonification transformationGray level = volumeOrdinate = pitch


Sounds sequence. . .

9


Data


Data transformation

Sonic rendering




Sounds sequence. . .

10

Data transformation

Common with visualization

Dependent on the data the data semantics the user task ex : AROMA

Data



Sonic rendering

Data transformation



11

Sonification transformation :abstract sound space

Data



Sonic rendering

Data transformation



Abstract sound space : = euclidean space = 1 point if no spatialization abstract sound =

duration during which acoustical cues vary

Two sonification levels 1 data element -> 1 abstract

sound Coordination of the abstract

sounds

12

Sonification transformation :abstract sound space

Data



Sonic rendering

Data transformation



Abstract sound space : = euclidean space centered on user = 1 point if no spatialization abstract sound =

duration acoustical cues varying in time

Two sonification levels 1 data element -> 1 abstract sound Coordination of the abstract sounds

13

Sonification transformation : representing a single element

3 independent mappings

Data element Abstract sound

- semantics- structure- value


123

14

Sonification transformation : representing a single element

3 independent mappings Special cue : spatial position Semantics = independent

from semantics of sound Analogical spatialization ?

Analogy with a phenomenon :

• Sonic : cathedral• Non-sonic : Diary in the Sky

Appointment at 3 pm

Data element Abstract sound



123

15

Sonification transformation : representing a set of data

Two sonification levels 1 single data element 1 sound 1 set of data 1 set of sounds

Coordination of the sounds of the sound set in time in space

Same place

Different places

Simultaneous Alternate

Pixelscolumn

Stereosound

Diary in theSky

Speechsynthesis

Space

Time

16

Auditory display transformation:physical device

4 steps transformation :(i) chosing a view point(ii) sound space -> device

parameters(iii) device parameters -> sound

signal(iv) sound signal -> physical sound

Data



Sonic rendering

Data transformation



17

Auditory display transformation:physical device

(iv) Physical device : sound precision degree of spatiality geometrical distribution of the

loudspeakers degree of sharing liberty of movement degree of immersion

Ex : SoundbeamNeckset

Data



Sonic rendering

Data transformation



18

Contributions a. classification

Example : the « earcons category » :

19

Contributions b. help for the designer

Identification of design steps

Identification of design parameters for each step :

guide for the design

foundation for design heuristics

context-aware sonifications

20

Perspectives

Design heuristics

Links with a software architecture

Platform for sonification design based on this process

21

Thank you for your attention…

… Any question ?

22

Param.System

Gram. Data tosonify

Soundnature

Simult./alternate

Phone bell(caller-dep.)

No X is calling … …

Phone bell(caller-indep.)

No Someone iscalling

… …

Blattner’searcons

Yes … Synthetic Alternate

Brewster’sserial earcons

Yes … Musical Alternate

Brewster’sparallel earcons

Yes … Musical Simultan.