Written Review 2 Granulation Effects

DESC 9115 Semester 1 2014 Laurence Williams SID: 308137647

Abstract

Granulating sampled audio is a sound transformation technique which operates by redistributing micro­temporal elements of an input signal. A variety of new sonic textures can result from such manipulation, from subtle to extreme modifications of the original. Earlier assignments submitted for this unit of study presented a theoretical and historical overview of granular analysis and synthesis methods (Written Review 1), and described the implementation of a prototype granulation effect in MATLAB (Lab Report 2). It is within the scope of this second written review to condense and synthesise the material covered in the previous two assignments, as well to present a broader and more comprehensive summary of granular techniques.

1. Overview While granulation is a method of sound transformation, rather than synthesis as such, it is generally considered a variety of granular synthesis where the grain waveform is an arbitrary digital audio signal. Parameters governing the distribution of grains are similar whether the waveform is sampled or synthesised. Indeed, granulation is the most commonly encountered implementation of granular synthesis (Roads, 2004 p.117). Musical applications of granular techniques found their genesis in the theories of physicist Dennis Gabor, specifically his representation of complex signals as agglomerates of units of acoustical quanta. Gabor’s research, concerned with increasing the efficiency of information transmission, proposed an alternative to the ‘idealisations’ of Fourier analysis and time­series representation (which each provide rigorous definition in one domain to the exclusion of the other). Sceptical of describing real­world sounds and signals as collections of harmonically related sinusoids of infinite duration, Gabor defined an elementary unit of acoustic information (corresponding to perceptual thresholds of the human auditory system) as a sinusoidal oscillation modulated by a Gaussian envelope, where the resolution in time/ frequency is determined by the duration of the envelope. A matrix of these elementary units, scaled/ shifted on the time­frequency plane and weighted by an amplitude coefficient, is calculated to represent an arbitrary signal (Gabor, 1947 p.429;

Roads, 2004 p.59). Gabor’s conception of this kind of quantum of sonic materials inspired Iannis Xenakis, a Greek composer interested in probabilistic and stochastic processes. Xenakis’ 1959 piece Analogique B can be considered the first implementation of pure granular synthesis, realised by splicing and rearranging magnetic tape to which an electronic signal had been recorded. Initial pioneering experiments with digital granular synthesis were studies, executed laboriously with the available technology, by Curtis Roads (Klang­1, 1974 and Prototype, 1975). Barry Truax contributed further to the early development of granular processing in music, performing real­time granular synthesis (Riverrun, 1986) and composing the first piece to feature granulation of sampled sound (Wings of Nike, 1987) (Truax, 1986; Roads, 2004 p.311). Since this time, granulation and granular synthesis platforms have been used increasingly. The next section reviews some of the environments where granular techniques can be implemented, including the MATLAB example provide in Lab Report 2.

2. Implementations of Granulation Effects Granular systems are now commonly available to the general consumer in a variety of forms (such as plug­ins, software synthesisers, standalone applications) thanks largely to massive advances in the capabilities of computer processors. The number of different applications prevents a comprehensive discussion from taking place here; a good starting point for a list of available software can be viewed at Opie (1999­2009). The effect submitted for the previous assignment will be discussed alongside other MATLAB examples, before some different programming approaches are outlined and some brief examples given. The effect created in MATLAB for this unit of study expanded upon De Poli’s source code included in Zolzer (2011). Designed to produce useful meso­structural compositional materials for electronic/ electroacoustic music, an attempt was made to emphasise ease of usability while directly engaging the user with the programming environment (ie, without using a GUI). This was intended to encourage people to become familiar with and start developing the code. Hanna (2013) also created a MATLAB granulation effect derived from De Poli’s original routine, however there were some significant differences in the approach taken and the options provided to the user (such as selection/ distribution types and amplitude coefficients). In the current implementation, the user sets values for all the relevant parameters within a high level script. Once an input signal is loaded and a sampling frequency determined, the user selects the length of the output signal, the number of grains for each channel, the maximum and minimum grain durations for each channel, the selection type (linear, backwards or random), the size of the window applied to each grain and the amplitude over the output (randomised grain by grain, Gaussian envelope

over output, increasing amplitude through output or decreasing amplitude through output). Once evaluated, the script calls two functions. The first function interprets the input parameters and calls the second function within a for loop, selecting the grains and performing the synthesis. Although the parameters described above represent the requisite input data for a relatively simple granulation program, the effect is nonetheless capable of a large range of powerful transformations. Considering the stated intention of creating an easily comprehensible starting point for further development, the effect was regarded by the author as successful. Figure 1a shows a spectrogram of an unprocessed guitar signal, while figure 1b displays the processed version (note: only one channel of each stereo signal is shown). On comparison, sharp temporal and spectral discontinuities can be observed in the signal to which the effect has been applied. In this example, grains of relatively short duration were extracted randomly from the input and a Gaussian envelope applied over the output (also seen in figure 1b, amplitude decreases at the extremes of the plot). These and other audio examples can be heard in Lab Report 2.

A second spectrogram comparison of two different audio examples illustrates another kind of radical alteration (Figures 2a and 2b). Here a short spoken vocal recording of just under 1.5 seconds duration is stretched over 40 seconds, introducing severe artefacts and destroying intelligibility. The grains were selected linearly through the input file and a random amplitude coefficient assigned to each. Readily noticeable is the smearing in time of the high frequency energy from the single sibilant sound in the original spoken phrase, as well other parts of the signal, retaining only a remote resemblance to the original structure.

Examining the average magnitude spectra corresponding to figures 2 a) and b), it can be seen that dramatic divergence occurs mainly above approximately 400Hz (figures 2c and 2d):

A final spectrographic example shows an unprocessed recording of a violin playing an ascending glissando­type sound (figure 3a) compared with a processed version where the file is read backwards and amplitude is decreasing throughout. Aside from being longer than the original, it is clear that the contours in figure 3b are the inverse of those in 3a, indicating that the glissando is now descending:

Since the completion and submission of the prototype effect, two other accounts of having implemented granular analysis/ synthesis within MATLAB have been viewed by the author (Lee et al. 2013; Picard et al. 2009). These two reports are directly relevant to material discussed section 3, and will be discussed further. Three commonly used programming environments for granular synthesis applications are Max/MSP, SuperCollider and Csound. All of these languages include powerful built­in and user contributed granular synthesis applications and support real­time operations, unlike MATLAB, which is an offline language. Csound, a text­based and also traditionally offline environment, performs in real­time with less ease than Max/MSP or SuperCollider. Max/MSP is a graphical language, which many users find a more intuitive interface than text, especially in live performance contexts. The SuperCollider family is essentially text­based, featuring some graphic interface controls (Bates, 2004 p.75; Roads, 2004 p.115). Bates (2004) experimented with granulation in max/MSP, employing both synchronous and asynchronous techniques within her composition “And the Sun Fought the Clouds”. While there was nothing exceptionally remarkable about the granulation

methods, she experimented with different stereo and quadraphonic panning algorithms. This interest evidently continued and in Bates & Furlong (2009), more sophisticated synthesis and spatialisation techniques were explored through a combination of Max/MSP and Csound. In this instance, the primary processing and synthesis was performed offline in Csound, while a Max patch generated a text file that could be read by Csound dictating the spatial position and trajectory of each grain as corresponding to an individual agent of a ‘flocking’ algorithm (ie. emulating the behaviour of a group of birds in flight). The introduction to their convention paper provides a compelling rationale for the use of a flocking algorithm in spatialising grains (Bates & Furlong, 2009). Roads (1991), along with Alberto de Campo, utilised SuperCollider 2 in order to further explore the microsonic technique of pulsar synthesis, where textures are built from units consisting of a cycle of a waveform followed by a period of silence.

3. Discussion and Conclusion While the focus of this exploration of granulation and granular synthesis has been on application to music composition, much of the recent literature surrounding these techniques is concerned with providing an aesthetically effective and computationally viable way of creating environmental sound for immersive/ interactive virtual worlds, most predominantly sound for video games. Granulation is an obvious option for this application, as it is a simple way to avoid sounds becoming gratingly repetitive without overloading memory with sampled sounds. Picard et. al. (2009), present a comprehensive approach to this, separating impulsive signals from continuous ones prior to segmentation. Spectral flux is calculated as an onset detection system to govern grain extraction in all parts of the signal. A ‘dictionary’ of grains is then compiled and a correlation criteria established to determine the distribution of grains based on movement within the virtual environment. This loosely describes a form of concatenative synthesis, a process where a signal is segmented, as in granulation, but the segments are retained in a ‘corpus’ (comparable to the aforementioned ‘dictionary’) and are output based on their close approximation of a ‘target’ signal. Concatenative synthesis is often used in speech synthesis, but also in free sound synthesis not dissimilar from granulation. Paul (2011) and Lee et. al. (2013) appear heavily influenced by Picard’s approach. From pure theory to painstaking experiments with tape and computers, granulation and granular synthesis have now become valuable and widely used techniques in areas such as music composition, sound design and digital signal processing. Perhaps an important awareness to keep is that the ‘elementary units of acoustical quanta’ are not ‘elementary particles’ in a physical sense, there is no fundamental atomic element of sound. Rather, they correspond to durational thresholds of discrimination within the human hearing apparatus. This is perfectly useful from both a creative and technical perspective. Whitelaw

(2003), bemoans the indiscriminate conflation within contemporary electronic/ ambient music criticism of these various ideas, as well as criticising Roads (2004) for labouring the particulate metaphor in the technical community. While Roads does make it clear (2004 p.300) that there are no elementary acoustic units as such, perhaps some amount of fancy could be forsaken in favour of accuracy in general when describing this area of sound synthesis and transformation.

References

Bates, Enda. 2004. Composing, Perceiving and Developing Real­Time Granulation in Surround Sound (Masters Thesis) School of Music, Trinity College, Dublin. Bates, Edna; Furlong, Dermot. 2009. Score File Generator for Boids Based Granular Synthesis in Csound. Audio Engineering Society 126th Convention, Munich. Accessed via: http://www.aes.org.ezproxy2.library.usyd.edu.au/tmpFiles/elib/20140605/14957.pdf Gabor, Dennis. 1946. Theory Of Communication. Journal of the Institution Of Electrical Engineers, 93(3):429­457 Hanna, Arne D. 2013. An Exploration of Granular Synthesis. USYD e­learning Repository: http://ses.library.usyd.edu.au/browse?type=author&value=Hanna%2C+Arne+Jr Lee, Jung­Suk; Thibault, Francois; Depalle, Phillipe; Scavone, Gary P. 2013. Granular Analysis/ Synthesis for Simple and Robust Transformations of Complex Sounds. Audio Engineering Society 49th International Conference, London, UK. Viewed via: http://www.aes.org.ezproxy2.library.usyd.edu.au/tmpFiles/elib/20140604/16647.pdf Opie, Timothy. 1999­2009. Granular Synthesis Resource Website: Software Online Resource. Viewed: http://www.granularsynthesis.com/software.php

Paul, Leonard. 2001. Granulation of Sound in Video Games Audio Engineering Society 41st International Conference, London, UK Picard, Cecile; Tsingos, Nicolas; Faure, Francois. 2009. Retargeting Example Sounds to Interactive Physics­Driven Animations. Audio Engineering Society 35th International Conference. London, UK. Viewed via: http://www.aes.org.ezproxy2.library.usyd.edu.au/tmpFiles/elib/20140604/16647.pdf Roads, Curtis. 2001. Sound Composition with Pulsars. Journal of the Audio Engineering Society, Vol.49 no.3, 135­147 Roads, Curtis. 2004. Microsound. The MIT Press. Cambridge, Massachusetts Truax, Barry 1986. Riverrun, for four computer­synthesized soundtracks http://www.sfu.ca/~truax/river.html Whitelaw, Mitchell. Sound Particles and Microsonic Materialism Contemporary Music Review 22(4) 93­100. Zolzer, Udo (ed)2011. DAFX: Digital Audio Effects (2nd ed), Wiley & Sons. West Sussex, UK.