REMOTE 3D-AUDIO PERFORMANCE WITH SPATIALIZED DISTRIBUTION

Winfried Ritsch

Thomas Musil Alois Sontacchi Johannes Zmlnig Franz Zotter

ritsch@iem.at musil@iem.at Institute of Electronic Music

and Acoustics Graz, Austria

sontacchi@iem.at

zmoelnig@iem.at zotter@iem.at

ABSTRACT

Today's electro-acoustics concert halls are usually equipped with a multi speaker setting. Some of these environments have the ability to play music on an spatialised 3D space including virtual acoustics. For remote performances, a concert in a source place is transmitted to a remote place where the audience is located. The concert's audio signature is replicated in the destination. The needed streams over the network should include an abstract description of the sound environment to be interpreted correctly to a different speaker setting in another place. Implementing this scenario we realized the lack of a common streaming technique to realize with a limited number of audio streams and control data to be transmitted. Here we suggest using an Ambisonics approach to stream complex audio environments including extra channels for monitoring, and special audio signals in a time accurate way. The description of the audio signals streamed with metadata use different containers, also storing them in common file formats as PCM-WAV or SDIF. As a proof of concept we implement this on our general purpose production software CUBEMixer, originally designed to be used for performances as spatialisation-system, a 3D-Audio mastering application including additional virtual acoustics in a concert-room. The entire system can be controlled with OSC1[10].

Keywords : Spatialisation, mixer, virtual room acoustics streaming, post-production.

1. INTRODUCTION

In electro-acoustic concerts in the western tradition we think of sound as a spatial, sculptural phenomenon. Pre-recorded audio signals are rendered in an 3D-Audio-Environment, mixed with live signals. Since playback situation differ from each other and concert halls have different acoustics, we need to transport more or less dry signals of the sound environment to be rendered in

1 Open Sound Control Protocol

another, possibly virtual, acoustic environment, from big halls, via small rooms to binaural production systems. To achieve this goal, a 3D-Mixer is needed that has the possibility to produce an adequate spatialized stream. This could be done with audio and parallel time aligned control data like OSC or to stream the mix for a given speaker setting in the target room. The latter cannot do complex spatial environments and is inflexible for different performance places without adaptation. The main target is to use an higher order Ambisonics system as an abstract base of spatialisation. The 3D-spatialized data can be interpreted in different targets with their own decoding master sections for multi speaker systems, or special Ambisonics decoders to render the mix for example in 5.1 surround or even plain stereo for headphones. As a first implementation and proof of concept, The CUBEMixer[13] is adapted. The goal is to find a common performance standard for streaming audio, mixed in the 3D-domain.

2. STREAMING FORMAT

Since there is no commonly used format for Ambisonics streams in 3D, their is the need to define one for the exchange of Ambisonics art works and streams. In the Ambisonics XChange meeting in Graz specialists for Ambisonics works in Europe met to find one general solution where full or reduced sets of Ambisonics channels can be mixed with additional audio channels for eg. click tracks, sub woofer, monitoring, ... .

2.1. Audio Channels

The Ambisonics channels represent the Spherical Harmonics of a sound field. The Ambisonics order defines the number of channels needed. On 2D audio it is 2n+1, on 3D (n+1)^2 and on mixed order there are different solutions.

Figure 1. Spherical harmonics for 3D Ambisonics

2.1.1. Naming

The name space for description of the channels are traditionally WXYZRSTUVKLMNOPQ... which is not sufficient for higher order Ambisonics. Therefore a common naming convention should be defined.

2.1.2. Number of Channels Needed

Not all channels are needed for most situations because of symmetries or other partially covered areas of sound distribution and some of them can be calculated from a linear combination of others.

2.1.3. Monitor Channels

Monitor channels can also be calculated from Ambisonics signals which filters audio from a certain position in the 3D signal, like a magnifying glass on an audio scene.

2.1.4. Combining Ambisonic Material

For mixing Ambisonics streams sometimes only one channel is needed to be spatialized from a certain direction and covering area distribution. Therefore needed Ambisonics channels can be calculated.

2.1.5. Special Purpose Channels

These are needed for click tracks, special effects, etc and should be added to the multi-channel stream without being part of the Ambisonics encoded soundfield.

2.1.6. Container

The audio channels and additional streamable audio data have to be transmitted in container. One container could be Broadcast WAV or for better streaming behaviour SDIF[13], which can also include some control streams like OSC.

2.2. Tools

The mixer is split split into a configurable number of input sections assignable to individual input signals, with different optional encoders and a master section with extensions like different decoder and subwoofer-systems, effects, 3D-reverbs, sound file players and recorders, a binaural rendering stage and other tools. At

the time of writing an 3D Ambisonics up to 5th order[1,2,3] is used. In the Ambisonics domain, the 25 Ambisonics signals can be used for storage and as a stream source for complex spatialized audio.

Figure 2. Mixer Organisation

3. CONCLUSION

The need for a common format is shown above. With the proposal of the IEM Ambisonics XChange Format we hope to have made a step forward solve this problem. At the end of this process a rough consensus for the used format should be found and as proof of concept, streaming actions should be done. This will help in the preservation of Ambisonics artworks as well as raise the standard of 3D-Audio environments to be shared.

4. REFERENCES

[1] Gerzon, M. A. (1985). Ambisonics in Multichannel Broadcasting and Video, J. Audio Eng. Soc., 33(11), pp. 859-871.

[2] Malham, D. G. (1992). Experience with Large Area 3D Ambisonic Sound Systems. In Proc. Inst. Acoust.

[3] Musil, T. and J. Zmlnig, M. Noisternig, A. Sontacchi, Robert Hldrich: AMBISONIC 3D-Beschallungssystem 5.Ordnung fr PD, IEM Report 15/2003

[4] Musil, T.: IEMLIB fr PD, IEM Report 12/2003.

[5] Noisternig N and A. Sontacchi, T. Musil, R. Hldrich: A 3D Ambisonic based Binaural Sound Reproduction System" AES 24th

International Conference, 26-28 June 2003, Banff, Canada.

[6] Puckette, M. 1996. "Pure Data: another integrated computer music environment." Proceedings, Second Intercollege Computer Music Concerts, Tachikawa, Japan, pp. 37-41.

[7] Ritsch W. and R. Hldrich, C. Frauenberger: Internet Archive for Electronic Music IAEM-iARS internet Audio Rendering System, AES 24th International Conference, 26-28 June 2003, Banff, Canada.

[8] Ritsch W. and Musil T., Zmlnig J., Hldrich R. (1995):Implementation eines kombinierten Ambisonic- und Bus-Mixers fr den Einsatz in 3D Audio Environments, IEM Report 28/05.

[9] Sontacchi, A.: Neue Anstze der Schallfeldreproduktion, Dissertation der TU Graz 2001.

[10] Wright, M. and A. Freed 1997. Open Sound Control: A New Protocol for Communicating with Sound Synthesizers. Proceedings of the International Computer Music Conference, Thessaloniki, Hellas, pp. 101-104.

[11] Zmlnig, J and W. Ritsch, A. Sontacchi: "Der IEM CUBE ein periphones (Re) Produktions system", 22.TMT,Produktforum, Jahrestagung des Vereins Deutscher Tonmeister, Hannover, November 2002.

[12] Zmlnig J. and W. Ritsch, A. Sontacchi: "The IEM CUBE", ICAD - International Conference on Auditory Display, July 7-9, 2003, Boston University, Boston, MA.

[13] M. Wright, A. Chaudhary, A. Freed, D. Wessel, and X. Rodet. New Applications of the Sound Description Interchange Format. In Proceedings of the International Computer Music Conference, pages 276279, Ann Arbor, US, 1998.

IndexICMC 2008 HomeConference InfoWelcome from the ICMA PresidentICMA OfficersWelcome from the ICMC 2008 Organising CommitteeICMC 2008Previous ICMCsICMC 2008 Paper Panel & Music CuratorsICMC 2008 ReviewersICMC 2008 Best Paper Award

SessionsMonday, 25 August 2008Languages and Environments 1Interaction and Improvisation 1Sound SynthesisComputational Modeling of MusicDemos 1Posters 1Interaction and Improvisation 2Aesthetics, History, and Philosophy 1

Tuesday, 26 August 2008MiscellaneousAlgorithmic Composition Tools 1Network PerformanceComputational Music Analysis 1Panel 1: Reinventing Audio and Music Computation fo ...Panel 2: Towards an Interchange Format for Spatial ...

Wednesday, 27 August 2008Studio Reports 1Mobile Computer Ensemble PlayDemos 2Posters 2Algorithmic Composition Tools 2Interface, Gesture, and Control 1

Thursday, 28 August 2008Interface, Gesture, and Control 2Languages and Environments 2Spatialization 1Computational Music Analysis 2Panel 3: Network PerformanceDemos 3Posters 3

Friday, 29 August 2008Sound ProcessingAesthetics, History, and Philosophy 2Interface, Gesture, and Control 3Spatialization 2Algorithmic Composition Tools 3Studio Reports 2

AuthorsAll authorsABCDEFGHIJKLMNOPQRSTUVWYZ

PapersAll papersPapers by Sessions

Topicscritical theory/philosophy of technology, postmodern cy ...sociology/anthropology of everyday sounds, situated per ...history of computer music, women and gender studies, ed ...philosophy/culture/psychology, music information retrie ...electroacoustic music composition, aesthetics of music, ...singing analysis/synthesis, music analysis/synthesis, v ...interactive and real-time systems and languages, music ...human-computer interaction, sound synthesis/analysis, i ...interaction design, computer music, performance art, el ...physical interface design, performance systems, gesture ...language/education/history/sociology of computer music, ...composition systems and techniques, languages for compu ...programming languages/systems, audio synthesis/analysis ...composition, music cognition, music informatics, human- ...music information retrieval, audio signal processing, p ...computational musicology, music cognition, music and AI ...music cognition, rhythm/meter/timing/tempo, computation ...music information retrieval, audio content analysis, to ...spatial audio, audio signal processing, auditory percep ...physical modelling, spatial audio, room acoustics, aura ...sonic interaction design, physics-based sound synthesis ...audio signal processing, sound synthesis, acoustics of ...audio signal processing, acoustics, software systemsphysics-based sound synthesis, virtual room acousticscomposition, music analysis, software for pedagogyPANEL: Towards an Interchange Format for Spatial audio ...PANEL: Network PerformancePANEL: Reinventing Audio and Music Computation for Many ...

SearchHelpBrowsing the Conference ContentThe Search FunctionalityAcrobat Query LanguageUsing the Acrobat ReaderConfiguration and Limitations

AboutCurrent paperPresentation sessionAbstractAuthorsWinfried Ritsch