R Programming & Digital Audio Donald Byrd rev. 14 Jan. 2008 Copyright © 2006-08, Donald Byrd

R Programming & Digital Audio

Donald Byrdrev. 14 Jan. 2008

Copyright © 2006-08, Donald Byrd

rev. 7 Feb. 2007 2

Elements of Digital Audio (1)

• Requirements of discrete time sampling– Pohlmann’s “video of ride over bumpy road” analogy

• Sampling rate determines maximum frequency– Human hearing goes up to ca. 15-20 KHz– Sampling Theorem (Nyquist, Shannon, etc.): need 2

samples per cycle• Less than 2 samples/cycle => aliasing• Visual equivalent: wheels “going backwards”

– Video & movies have low sampling (frame) rates

• Practically, need more than 2• CD sampling rate = 44,100 = 20 KHz * 2.205

rev. 6 Feb. 2007 3

Elements of Digital Audio (2)

• Sample depth (bits per sample) determines freedom from quantization noise– Also called bit depth, sample width, or bit width– SQNR (Signal-to-Quantization Noise Ratio) = about 6

dB per bit– For digital audio, almost always 8, 16, or 24 bits– Usually (CDs, etc.) 16 bits => ca. 96 dB SQNR

• Exceed maximum sample => clipping– A nasty type of distortion– Very different from overdriving analog media

rev. 3 Feb. 2007 4

Elements of Digital Audio (3)

• A simple example– Input: sound waves => microphone => Analog to

Digital Converter (ADC) => computer, etc.– ADC needs low-pass filter to avoid aliasing– Output: computer, etc. => Digital to Analog

Converter (DAC) => loudspeaker => sound waves– DAC needs low-pass filter to avoid imaging (related

to & often confused w/ aliasing)• NB: theoretically, should apply “sinc” function

instead of low-pass filter, but that’s impractical (equiv. to ideal low-pass filter?)

• Process introduces noise & distortion

15 Sep. 2006 5

Audio in R: the tuneR Package

• tuneR is an “add-on” library for R• Adds functions to create, work with, &

analyze Wave (.wav audio) files• Installation: type “install.packages()”, or

(with the R GUI) use menu command Packages>Install Packages

• For more information, see “tuneR” under Packages at http://www.r-project.org/

rev. 20 Jan. 2007 6

Structure of the tuneR Wave Object

• left: vector containing samples for left channel

• right: vector containing samples for right channel (NULL if mono)

• stereo: a boolean to say if stereo or mono

• samp.rate: sampling rate(e.g., 44,100 = 44,100 samples per sec. for CD quality)

• bit: sample depth, in bits: controls quantization (usually 16, e.g., for CDs; can be 8 for low quality)

left

samp.rate

right

stereo

bit

An object in R has slots. The Wave object has 5 slots.

rev. 3 Apr. 07 7

Creating a Wave Object from Scratch• install.packages() # do only once after installing R on a computer• library(tuneR) # do every time you run R & need tuneR• setWavPlayer("/Library/Audio/playRWave") # do every time you run R

& need to play sounds (OS X only)• #• # Create & play 2.5-sec. sine wave with pitch about middle C (262 Hz)• wavs <- sine(262, duration=2.5, samp.rate=16000, bit=16,

xunit="time")• play(wavs)

rev. 22 Oct. 07 8

Creating a Wave Object from a File

• install.packages() # do only once after installing R on a computer• library(tuneR) # do every time you need tuneR• setWavPlayer("/Library/Audio/playRWave") # do every time you run R

& need to play sounds (OS X only)• #• # Set the working directory to the correct path for your computer.• setwd("work")

• # Read Wave (samples plus sampling rate, depth, etc.) from file; display and play it.

• wav <- readWave("Piano.mf1st5secMono.A4.wav")• print(wav)• #plot(wav@left) # slow if it's a long sound!• play(wav)

rev. 27 Jan. 2007 9

What Do We Have?• play(wav)

– Uses whole Wave object• plot(wav, nr=1000)

– Uses whole Wave object• plot(wav@left) --------->• plot(wav@left[1:5000])

– Uses just vector of samples• wav

– Shows the Wave’s 5 slots:

Wave Object Number of Samples: 198562 Duration (seconds): 4.5 Sampling rate (Hertz): 44100 Channels (Mono/Stereo): Mono Bit (8/16): 16

rev. 22 Oct. 07 10

Wave Manipulation Example #1

• # Assumes “Creating a Wave Object” already done

• samData <- wav@left• samData1 <- samData*3• plot(samData1)• wav1 <- Wave

(left=samData1, samp.rate=wav@samp.rate, bit=wav@bit)

• play(wav1)

rev. 3 Apr. 07 11

Wave Manipulation Example #2

• R code– # Assumes “Creating a Wave Object” already done– samData <- wav@left– samRate <- wav@samp.rate– samDepth <- wav@bit– newSamRate <- samRate/2^(6/12)– wav2 <- Wave(left=samData,

samp.rate=newSamRate, bit= samDepth)– play(wav2)

• Effect: pitch is 6 semitones = tritone lower

rev. 3 Apr. 07 12

Wave Manipulation: More Techniques in R

• Not Wave-specific, just standard R– See “An Introduction to R” (R-intro.pdf)

• Under Manuals, at http://www.r-project.org/

• 1. Extract every nth element– samData3 <- samData[seq(3, len, by=3)]

• 2. Make two sounds overlap– # Append 0’s to samData3, or there would

be NA which causes error later– samData3[len:round(0.5*samRate)] <- 0– samData3 <- samData3+samData

27 Nov. 07 13

Programming in General (1)• Details are often vital (& errors are costly)

– A great many details really are. Commonly:• Quote marks, including single vs. double• Capitalization

– “Wav” & “wav” are different– TIP: “steal” as much as possible!

• Via Copy & Paste is ideal: avoids typos

• Programs tend to be very hard to understand– TIP: include useful, readable comments– TIP: choose variable names for clarity

• “wavdata” isn’t good; how about “samples”?– TIP: consistency helps clarity and correctness

• Don’t mix “v = expr”, “v <- expr”, and “expr -> v”• Use the same variable name for something in every prog.

• Program defensively

rev. 3 Apr. 07 14

Programming in General (2)

• Comments– Classic example of a bad comment

• x <- x+1 # add 1 to x– Doesn’t explain anything!

• Good commenting style (thanks to Ed Wolf)# Using the Add Sines Demo, create and play a wave at G3,# then do the same for a wave at 5/4 this frequency. Finally, # normalize the sum of the two waves and listen to result.…

# create and play first sound waveswave1 <- sine(f, duration=secs, samp.rate=sr, bit=16,

xunit="time")play(swave1)…

3 April 2007 15

Programming in General (3)• Block comments (w/ overall description) more

important than comments on single stmts• Ideal: say just the right things: Not too much

or too little– Basic principle of all human communication– …including this slide show & music notations (CMN,

tablature, etc.)– …and comments in a program

• Other aspects of formatting & style– Variable names

• Choose variable names for clarity• camelCase is helpful

– Space around operators– “v <- f(expr)”, not “v<-f(expr)”

rev. 20 Jan. 2007 16

Programming in R (1)• R offers to save workspace when you quit

– Are you sure it’s what you want?– TIP: Just say no.

• Can restore original with ‘load(".Rdata")’ (?)

– TIP: Use a text editor & files to save work• If real text editor (not word processor) file, can run with

R “source” command• Regardless, can Copy & Paste, even just part of file

• setwd() to correct path for your computer– Depends on where you have files– Can be tricky, esp. in Windows

• Typical Windows ex.: setwd("C:/Documents and Settings/donbyrd.ADS/Teaching/N560")

• Easier: use R GUI “Change Working Directory” menu command!

rev. 14 Jan. 08 17

Programming in R (2)

• Many useful built-in functions– Many of them handle vectors (no loop needed)

• diff(v): vector of consecutive differences• sum(v): sum of vector elements

– Random numbers with various distributions: runif (uniform), rnorm (normal), etc.

– read.table, table (and related functions)– fft– tuneR adds sine, square, noise, bind, mono, etc.

• R (and tuneR) have excellent on-line help– Type either ‘help(sine)’ (e.g.) or ‘?sine’

• …but NB: sometimes need ‘help("sine")’– TIP: Copy & Paste from help window!– Caveat: terminology is statistics oriented

rev. 3 Apr. 07 18

Programming in R (3)

• Introducing loops– Loops are hard for many beginners– A very simple (though pointless) example

• mnnV <- 1:6 # make mnV a 6-place vector• mnnV # see what mnnV is before loop

• for (n in 1:6) {• mnnV[n] <- n+59• }• mnnV # ...and after

– Instead of “in 1:6”, can use any vector!– C, Perl, etc. users can put the vector in the “for”

• for (n in seq(1, 6)) { – Loop is a type of control statement

27 Nov. 07 19

Software Engineering & Debugging (1)

• Experience: all complex programs have bugs– Judge in Florida e-voting case: claim that voting

machine software was buggy is speculation– True, but… !

• Disclaimer: I don’t know any hard evidence

• Expect bugs & program defensively• True stories

– The program that failed only on Wednesdays! Why?• Hint: “Wednesday” has 9 characters

– Weeks of debugging to find a “1” that should have been “i”

27 Nov. 07 20

Software Engineering & Debugging (2)

• Good engineering (design, coding, comments, etc.) => less debugging & more robust (reliable & flexible) programs

• Don’t underengineer• …but don’t overengineer, either!• Underengineering is much bigger danger for

inexperienced programmers• Main factors

– Complexity of problem– Is program or code it includes likely to be used for

very long?– If so, how expert future programmers are likely to be

27 Nov. 07 21

Software Engineering & Debugging (3)• Standard technique: zero in on problem code• Debug on short/simple cases, not long/complex

ones– Makes it practical to look at results of several print

statements– Reduces or eliminates long delays to see results– “short/simple” often means simply not much data– Can easily reduce days of debugging to hours

• Usually easy to do by turning lots of data into a little data– Real situation: nThemes <- 3500, or 20 sec. audio file– For testing: use nThemes <- 4 (say), or 1 sec. audio– Caveat! the “little” data may not show the bug– …and if bug results from a conceptual problem, fixing

it may be very hard

8 Sept. 07 22

Debugging in R• Basic technique: zero in on bug with print (or cat) & plot

– E.g., before & after doing something questionable• print(c("max before scaling=", max(wNotes@left)))• wNotes <- wNotes*2.5• cat("max after scaling=", max(wNotes@left), “\n”)

– cat merges its arguments, gets rid of the extra parens– …but doesn’t end the line => do it yourself with “\n”

– If you use “source” (& inside loops?), just naming variable doesn’t work; must use print or cat

• A good debugger can be very helpful– Especially w/ complex programs, or…– Learning a new language– R has a debugger; one student tried & liked it

3 Sep. 2007 23

Dangers of R (1)• More danger of nasty bugs in R than many

programming languages & environments– No explicit types => doesn’t warn of questionable

usage– No variable declarations => doesn’t catch typos

(only a problem in old versions of R?)– Both above like Perl (e.g.), but Java (e.g.) is great on

both => Java programmers likely to be careless!

• Defensive programming– E.g., add “sanity checks” as you work, use

conventions for variable names, etc.– always important: a subtle bug can waste a huge

amount of time• Weeks of debugging to find a “1” that should have been “i”

– …but especially in dangerous environments like R

rev. 14 Dec. 08 24

Dangers of R & tuneR (2)• “Gotchas” in R (all from real life)

– Operator precedence, esp. in “for” statement• In sets, need parentheses to get addition before “:”• E.g., “start:(start+5)”, not “start:start+5” !

– Undocumented requirements (tuneR): some functions fail (with no error message) if frequency isn’t an integer

– “;” is usually ignored, but not always– Line break sometimes starts a new statement, not always

• Other real-life examples from Don’s classes– Undeclared variable: “allNotes” vs. “allnotes” (only a problem

in old versions of R?)– Call function that returns a value but ignore value– Nonexistent named params. sometimes give error, not

always• Danger much worse because R often gives lousy feedback

for errors or likely errors– Apparent exception: play w/ unnormalized values– …but that’s tuneR, not R

rev. 27 Jan. 2007 25

Programming in R with tuneR

• On OS X (and LINUX): play() problem– Must say what program to use to play Waves

• Either setWavPlayer once, or add 2nd param. to each play()

– OS X can use QuickTime Player• It’s on every OS X machine, & it works, but…

– Usually gives scary error messages; must hit the escape key to get R to continue; leaves open more & more QuickTime Players

– OS X alternative: playRWave• Works fine, but…• Not pre-installed; you must get & install it.• Instructions at:

– http://www.informatics.indiana.edu/donbyrd/tuneRPlaybackOnMacs.txt