Digital Sound Representation & Introduction to Software Synthesis Electronic Music Studio TU Berlin Institute of Communications Research

Digital Sound Representation & Introduction to Software Synthesis

Electronic Music Studio TU Berlin

Institute of Communications Research

http://www.kgw.tu-berlin.de/

Prof Eduardo R MirandaVarèse-Gastprofessor

[email protected]

Topics:

Digital representation of sound

Software sound synthesis fundamentals

Sound synthesis programming with Csound: Introduction to Csound

Digital Representation of Sound

What is a sound?

• From a physical point of view, it is the result of a mechanical disturbance of some object in a physical medium, such as air.

• This mechanical disturbance generates vibrations that can be represented as electrical signals by means of a device (for example, a microphone), that converts these vibrations into a time-varying voltage.

Analog vs. Digital

• An analog sound signal is the result of measuring the voltage that represents the sound.

• These kinds of signals are continuous in the sense that they consist of a continuum of constantly changing values.

• A digital sound is the result of counting all these values many times per second for a certain defined length/time.

• Each measurement value is called a sample, and this kind of process is called sampling.

• In order to process sounds on the computer, the analog sound must be converted into a digital format understandable by computer; that is, binary numbers.

The process of converting a sound (e.g., coming from an instrument such as the violin) into a digital representation:

a) a microphone or a line-level source converts sound into voltage

b) an analog-to-digital converter then converts voltage into binary numbers

c) a computer stores the resulting sound

Analog sound into a digital sound

• 16-bit conversion suffices for home-quality recording

• Professional quality recording often uses 32-bit and 64-bit words

• Analog-to-digital converter (ADC) are devices used to convert the waveform into a succession of binary numbers, each representing the voltage level at a given instant.

• Digital-to-analog converter (DAC), converts binary numbers (i.e., digital sound representation) into analog signals in order to play the sound.

Computers are normally configured to function based upon strings of bits (binary numbers) of fixed size, called words.

For example, a computer configured for processing 4-bit words would represent the decimal numbers 0,1,2 and 3 as the binary numbers

0000 = 0 0001 = 1 0010 = 2 0011 = 3 0100 = 5 0101 = 6 etc.

Sampling theorem

• The analog-to-digital conversion process is called sampling.

• The frequency of a sound is equal to the number of cycles which occur every second ("cycles per second", abbreviated "cps" or "Hz").

• In order to convert an analog sound signal into digital representation one needs to sample the signal many times per second.

• The frequency of this sampling process is called sampling frequency or sampling rate, and it is measured in Hertz (Hz).

• The sampling theorem states that in order to accurately represent a sound digitally, the sampling rate must be higher than at least twice the value of the highest frequency contained in the signal.

• The average upper limit of human hearing is approximately 18 kHz (18000 Hz), which implies a minimum sampling rate of 36 kHz (36000 Hz).

• The sampling rate frequently used in computer sound design systems is 44.1 kHz (44100 Hz).

Aliaising distortion

• Nyquist frequency is the name of the highest frequency that can theoretically be represented in a digital audio system.

• It is calculated as half of the value of the sampling rate. Passing this level causes aliasing distortion, or foldover.

• A word, for instance, could be 4-bits long or 16-bits long etc. Unsatisfactory lower resolutions are prone to cause a damaging loss of sound quality, referred to as quantization noise.

Sampling quality

• The level of resolution, or quantization of a sampler: depends upon the size of the word used to represent the amplitude of a sampling sound and is determined by the resolution of the ADC and DAC.

Sound examples

• The following four sound examples were sampled from a mono sound, 4027 milliseconds long, sampled at 44100 Hz, 11025 Hz, 2756 Hz and 2000 Hz, respectively.

• The last file uses only 17 kB to store the sound, but it is considerably distorted when compared to the first example, which uses 367 kB.

a) [Example 1] 44100 samples per second results in a sound file of 367 kB b) [Example 2] 11025 samples per second results is a sound file of 93 kB

c) [Example 3] 2756 samples per second results in a sound file of 23 kB

d) [Example 4] 2000 samples per second results in a sound file of 17 kB

• The ability to adjust the sample rate affects the length of a sample for a given number of samples.

• This will affect the size of the digital sound file in a storage system.

• There is a penalty to be paid for the lower rate: a narrower band of frequencies will be recorded.

• The problem is that acoustic sounds are in fact composed of many sound components called partials. I

• In the majority of cases, most of these partials are of very high frequencies. If the sampling frequency is too low, the computer will not capture these high frequency components and this causes distortion.

Comments

Software Sound Synthesis

Unit generators

• Synthesis programming languages provide a number of synthesis subroutines, usually referred to as unit generators. (Inspired by the modules of an analog synthesiser).

• Synthesis instruments are designed and programmed by connecting a number of unit generators.

• Most basic unit generator of a synthesis programming language or systems is the oscillator.

Digital oscillator

• The speed at which the lookup table is scanned defines the frequency of the sound.

• The lookup table contains a list of samples for one cycle of the waveform, which does nor necessarily need to be a sinusoid.

• The specification of an oscillator on a computer typically involves:

- Frequency- Amplitude- Waveform

Other synthesis units

• Function generators: create lookup tables for the oscillators and for sound transformation and control (“GEN routines” in Csound).

• GEN routines fill data tables with values produced with specified procedures or mathematical formulae such as trigonometric functions and polynomials (the length is usually specified as a power of two (29 = 512, 210 = 1024, etc.)

• Signal modifiers: filters, envelopes, etc. Envelops add time-varying control to synthesis instruments.

Introduction to Csound• Has its origins in the early 1960’s at Bell Telephone Research Lab (New Jersey, USA) by Max Matthews – first computer music language called Music 3.

• Early 1970’s Barry Vercoe created a version called Music 11 (for PDP 11 computer) at MIT.

• Later Berry Vercoe re-made Music 11 in the C language, thereby creating Csound.

• Separate control rate and audio rate processing.

• Audio rate is the same as the sampling rate.

• Control rate is a slower rate used to shape certain features of the sound.

• It is widely portable under C and UNIX.

• It is structured in a way that allows for further developments; e.g., in 1992 real-time control under MIDI was introduced.

• User can add their own functions.

Main features

The structure of the language

• The composer needs to specify two types of files: orchestra and score files

• The orchestra file has 2 sections: headers and instrument definition

• Header: specifies global issues of the synthesis algorithm: sampling rate, number of channels, etc. (Note: anything after “;” is regarded as a comment and is not considered when the program runs).

sr = 22050 ; the sampling rate is set to 22050 samples per secondkr = 1470 ; there will be 1470 control point per secondksmps = 15 ; there will be a control point every 15 sound samples sr/krnchnls = 1 ; mono sound

eg1

• Instrument definition: defines the synthesis algorithm.

• The following example shows a simple instrument composed of a single sinewave oscillator

instr 1 ; start the definition of instrument 1a1 oscil 10000, 440, 1 ; make an audio rate oscillator with an

; amplitude of 10000, at a frequency of 440 Hz, ; using wavetable number 1out a1 ; output audio signalendin ; end of instrument

eg1

• The complete orchestra file looks like this:

;-----------------------------sr = 22050kr = 1470ksmps = 15nchnls = 1 ;-----------------------------instr 1a1 oscil 10000, 440, 1out a1endin ;-----------------------------

10000 440 Hz

GEN 1

eg1

• Score: defines what the instrument is going to do. The score comprises:

1. Definition of the wavetable(s) used in the instrument2. Timed event triggers; e.g., note sequences

• Wavetable definition: function table (hence the use of an “f”) at the beginning of the line.

• Example: wavetable 1, comes to take effect from time 0, with 4096 elements, using GEN no. 10 (GEN = generator function), with only one harmonic (which in this case defines a sinewave)

;name start size GEN relative strengthf1 0 4096 10 1

• The second part of the score defines a note event for the instrument, starting at time 0 and lasting for 4 secs

;instr start durationi1 0 4e

• In this case, frequency (pitch) and amplitude are state in the orc file

• The complete orchestra and score files look like this:

;-----------------------------sr = 22050kr = 1470ksamps = 15nchanls = 1 ;-----------------------------instr 1a1 oscil 10000, 440, 1out a1endin ;-----------------------------

;-----------------------------f1 0 4096 10 1;-----------------------------i1 0 4e

10000 440 Hz

GEN 1

eg1

• pfields (p = score parameters):

;-----------------------------sr = 22050kr = 1470ksmps = 15nchnls = 1 ;-----------------------------instr 1a1 oscil p4, p5, 1out a1endin ;-----------------------------

;-----------------------------f1 0 4096 10 1;-----------------------------;p1 p2 p3 p4 p5;-----------------------------i1 0 4 10000 440e

p4 p5

GEN 1

• p1 = instrument• p2 = start time• p3 = duration• p4 = amplitude• p5 = frequency

eg2

• Examples of scores for the simple oscil instrument: (by Andy Hunt, Uni of York, from CDP tutorial)

;--------------------------------------------------; 3 note melody;--------------------------------------------------f1 0 4096 10 1;--------------------------------------------------;p1 p2 p3 p4 p5;--------------------------------------------------i1 0 4 10000 440i1 4 2 10000 493i1 6 4 10000 440e

;-----------------------------sr = 22050kr = 1470ksmps = 15nchnls = 1 ;-----------------------------instr 1a1 oscil p4, p5, 1out a1endin;-----------------------------

eg3


;--------------------------------------------------; Overlapping notes;--------------------------------------------------f1 0 4096 10 1;--------------------------------------------------;p1 p2 p3 p4 p5;--------------------------------------------------i1 0 12 3000 220i1 2 12 3000 440i1 4 12 3000 660i1 6 12 3000 880i1 8 12 3000 1100i1 10 12 3000 1320e


eg4


;--------------------------------------------------; Overlapping with different amplitudes;--------------------------------------------------f1 0 4096 10 1;--------------------------------------------------;p1 p2 p3 p4 p5;--------------------------------------------------i1 0 12 3000 220i1 2 12 1500 440i1 4 12 700 660i1 6 12 350 880i1 8 12 175 1100i1 10 12 85 1320e


eg5


;--------------------------------------------------; Simultaneous starts;--------------------------------------------------f1 0 4096 10 1;--------------------------------------------------;p1 p2 p3 p4 p5;--------------------------------------------------i1 0 12 3000 220i1 0 12 1500 440i1 0 12 700 660i1 0 12 350 880i1 0 12 175 1100i1 0 12 85 1320e


eg6

Amplitude envelopes (or envelops)

• May synthesisers feature enveloping functions called ADSR:

Attack = time from the start to the peak of the soundDecay = loss of volume from the initial peakSustain = the continuation level of the sound until releaseRelease = speed at which the sound dies away

linen

• A Csound unit commonly used for making an envelope.

• Specified with the following parameters:

Amplitude = peak amplitude of the soundRise = attack time Duration = total duration of the soundDecay = period at which the sound dies away

Example

;ORC ----------------------sr = 22050kr = 1470ksmps = 15nchnls = 1 ;-----------------------------instr 1k1 linen p4, p6, p3, p7 a1 oscil k1, p5, 1out a1endin ;-----------------------------

;OSC ---------------------------------------f1 0 4096 10 1;----------------------------------------------i1 0 1 10000 440 0.05 0.7i1 1.5 1 10000 440 0.9 0.1i1 3 1 5000 880 0.02 0.99i1 4.5 1 5000 880 0.7 0.01i1 6 1 20000 220 0.5 0.5e

• p1 = instrument• p2 = start time• p3 = duration• p4 = amplitude• p5 = frequency• p6 = rise• p7 = decay

eg7

Perceptual role of attack

• As the attack gets shorter and shorter the whole character of the sound alters

;OSC ----------------------------------------------------------f1 0 4096 10 1;-----------------------------------------------------------------i1 0 1.5 10000 440 0.9 0.5i1 2 1.5 10000 440 0.7 0.5i1 4 1.5 10000 440 0.5 0.5i1 6 1.5 10000 440 0.3 0.5i1 8 1.5 10000 440 0.15 0.5i1 10 1.5 10000 440 0.07 0.5i1 12 1.5 10000 440 0.04 0.5i1 14 1.5 10000 440 0.02 0.5i1 16 1.5 10000 440 0.015 0.5i1 18 1.5 10000 440 0.01 0.5i1 20 1.5 10000 440 0.005 0.5i1 22 1.5 10000 440 0.004 0.5i1 24 1.5 10000 440 0.003 0.5i1 26 1.5 10000 440 0.002 0.5i1 28 1.5 10000 440 0.001 0.5i1 30 1.5 10000 440 0.0005 0.5e

eg8

Pitch notation

• N.00, ‘N’ = the octave and ’00’ the pitch

• middle C is octave N=8• .00 = varies from 00 = C to 11 = B

• Examples

Csound Note MIDI Frequency8.00 C4 60 261.60 Hz8.01 C4# 61 277.18 Hz8.02 D4 62 293.67 Hz8.03 D4# 63 311.13 Hz8.04 E4 64 329.63 Hz8.05 F4 65 349.23 Hz8.06 F4# 66 369.99 Hz8.07 G4 67 392.00 Hz8.08 G4# 68 415.30 Hz8.09 A4 69 440.00 Hz8.10 A4# 70 466.16 Hz8.11 B4 71 493.88 Hz

Example

;ORC ----------------------sr = 22050kr = 1470ksmps = 15nchnls = 1 ;-----------------------------instr 1inote=cpspch(p5)k1 linen p4, p6, p3, p7 a1 oscil k1, inote, 1out a1endin ;-----------------------------

;OSC ---------------------------------------f1 0 4096 10 1;----------------------------------------------i1 0 1 10000 8.09 0.05 0.7i1 1.5 1 10000 8.09 0.9 0.1i1 3 1 5000 9.09 0.02 0.99i1 4.5 1 5000 9.09 0.7 0.01i1 6 1 20000 7.09 0.5 0.5e

• p1 = instrument• p2 = start time• p3 = duration• p4 = amplitude• p5 = octave.pitch notation• p6 = rise• p7 = decay

eg9

Example

;ORC ----------------------sr = 22050kr = 1470ksmps = 15nchnls = 1 ;-----------------------------instr 1inote=cpspch(p5)iamp=ampdb(p4)k1 linen iamp, p6, p3, p7 a1 oscil k1, inote, 1out a1endin ;-----------------------------

;OSC ---------------------------------------f1 0 4096 10 1;----------------------------------------------i1 0 1 80 8.09 0.05 0.7i1 1.5 1 80 8.09 0.9 0.1e

• p1 = instrument• p2 = start time• p3 = duration• p4 = amplitude in dB• p5 = octave.pitch notation• p6 = rise• p7 = decay

eg10

Chorusing effect

• Technique to enrich or make the sound fuller.

• Done by creating beat frequencies; that is by playing copies of the note at slightly different frequencies.

;ORC CHORUS ----------------------sr = 22050kr = 1470ksmps = 15nchnls = 1 ;-------------------------------------------instr 1inote=cpspch(p5)iamp=ampdb(p4)

k1 linen iamp, p6, p3, p7 a1 oscil k1, inote, 1a2 oscil k1, inote*0.99, 1a3 oscil k1, inote*1.01,1aout=(a1+a2+a3)/3out aoutendin ;----------------------------------------------

;OSC CHORUS -------------------------f1 0 4096 10 1;----------------------------------------------i1 0 4 80 8.09 0.05 0.7i1 4 4 80 8.09 0.9 0.1e

eg11

Timbre-migrating instrument

;ORC MIGRATING -----------------------sr = 22050kr = 1470ksmps = 15nchnls = 1 ;------------------------------------------------instr 1inote=cpspch(p5)

;amplitude envelope

k1 linen p4, p6, p3, p7

;transition controlk2 linseg 1, p3*0.2, 1, p3*0.6, 0, p3*0.2, 0a1 oscil k1, inote, 1a2 oscil k1, inote, 2out (a1*k2)+(a2*(1-k2))endin

;SCO MIGRATING ---------------------------------f1 0 1024 10 1 0.5 0.3f2 0 1024 10 81 0 27 0 9 0 3 0 1 0 0.3 0 0.1;----------------------------------------------------------i1 0 6 15000 6.00 0.5 0.5i1 0 1 12000 10.0 0.1 0.2i1 1 1 12000 9.11 0.1 0.2i1 2 1 12000 9.07 0.1 0.2i1 3 1 12000 9.06 0.1 0.2i1 4 1 12000 9.04 0.2 0.5eg12

Defined pitch-space filled with glissando texture(from CDP Csound Tutorial Instrument Library)

eg13

Defined pitch-space filled with glissando texture

Further reading:

• Getting Started with Csound, by Andy Hunt (PDF file tutorial1.pdf)

• CDP’s Csound Tutorial Instrument Library (PDF file tutorial2.pdf)

Documents

Digital Sound Representation & Introduction to Software Synthesis Electronic Music Studio TU Berlin Institute of Communications Research