Studying Music Performance: A Probabilistic Approach

David Temperley Eastman School of Music University of Rochester

Four goals of music performance:

1. Render the notes (and other score information) correctly2. Convey the emotional content of the piece3. Clarify the structure of the piece4. Modulate the flow of information

Four goals of music performance:

1. Render the notes (and other score information) correctly2. Convey the emotional content of the piece3. Clarify the structure of the piece4. Modulate the flow of information The topic of today’s talk

Part I: Clarification of structure

• Three examples• The circularity problem (performance perception or

perception performance?)• Ambiguity avoidance

Clarification of structure in music performance

Many authors have observed that one goal of a good performance is to convey the structure of the piece (Sloboda 1985, Clarke 1988, Gabrielson 1999).

(“Structure” might include many things—meter, harmony, phrase structure, form, motivic connections, etc.)

Sometimes this point is made indirectly—e.g. when it is said simply that a good performer must be aware of thestructure of the piece. (Why is it beneficial to be aware of this? Presumably so that it can be conveyed to the listener.)

Clarification of structure – Example 1: Accentuation of metrically strong events

The perception of meter is a complex process, involving a variety of different cues or “phenomenal accents” (Lerdahl & Jackendoff, 1983): length, loudness, articulation, harmony, parallelism...

Performers can facilitate this process by aligning phenomenal accents with strong beats—that is, by emphasizing metrically strong events.

Clarification of meter, cont’d.

The same notes (pitches and rhythms) in different metrical contexts will be played quite differently: metrically strong notes are louder, longer, more legato (Sloboda 1983, 1985).

…And listeners can identify which melody is being performed.

Clarification of meter: Dynamics

Performers tend to play metrically strong notes somewhat louder than others, though the difference is relatively small.

Data from Drake & Palmer (1993) for piano performance, showing average “hammer velocity” values for notes on beats 1-4 of a 4/4 measure. (Note: range for entire piece was 30 to 600 units!)

Clarification of structure – Example 2: Phrase-final lengthening

Performers tend to decrease tempo at phrase boundaries, and more markedly at larger sectional boundaries (Todd 1985; Palmer 1996; Windsor & Clarke 1997).

Data from Windsor & Clarke 1997

phrase boundaries

(smooth curve)(jagged line)

Clarification of structure – Example 3: Individuation of melodic lines

The perception of polyphonic music requires the identification of individual lines—voice separation or “contrapuntal analysis” (Temperley 2001).

Performance teachers (of polyphonic instruments) often urge students to individuate different lines of the texture in some way—for example, by emphasizing the melody.

A specific case in point is melodic lead—the tendency of the melody notes to anticipate other notes (Palmer 1996). This could be an strategy to individuate melodic notes by making them asynchronous with other notes.

According to the “clarification of structure” view, thesephenomena (accentuation of metrically strong events,phrase-final lengthening, melodic lead) arose, at least in part, as ways of clarifying musical structure for the listener.

A possible circularity in this argument:

therefore

Listeners expect metrically Performers play metricallystrong notes to be louder / longer strong notes louder / longer

A possible circularity in this argument:

therefore

Listeners expect metrically Performers play metricallystrong notes to be louder / longer strong notes louder / longer

therefore

How to address the circularity problem? Three responses...

1. Perhaps the process really is circular. Performance tendenciesgave rise to perceptual tendencies, but were then reinforcedby them.

2. Perceptual tendencies may have come first and may haveoriginated outside music, e.g. in language:

- Stressed syllables are normally longer and louder (Fry 1955).

- Phrase-final lengthening is a cue to syntactic boundariesin speech (Lehiste 1973).

3. The case for perception influencing performance is strengthened if it can be shown that the performance tendencies are strongest in cases that would otherwise be ambiguous perceptually.

3. The case for perception influencing performance is strengthened if it can be shown that the performance tendencies are strongest in cases that would otherwise be ambiguous perceptually.

Ambiguity avoidance is well-documented in language: For example, speakers exaggerate prosodic marking of phrase structure (e.g. lengthening at syntactic boundaries) when a sentence is ambiguous in context (Snedeker & Trueswell 2003).

Ambiguity Avoidance in Music – Example 1: Organ performance

On instruments where dynamic accentuation is not possible, do performers increase other kinds of accentuation on metrically strong notes?

In organ pedagogy, performers are taught to emphasize metrically strong notes through duration (i.e. with longer inter-onset intervals to the following note—[A]) or articulation (playing them slightly more legato— [B]) (e.g. Hurford 1988).

: . : .AB

Ambiguity Avoidance – Example 2: Phrase-final lengtheningat ambiguous points

Do performers exaggerate phrase-final lengthening in caseswhere the phrase boundary would otherwise be unclear orambiguous?

Brahms, Intermezzo Op. 116 No. 6, mm. 1-8

Question: Why does Brahms mark sostenuto and a fermata at the phrase ending in mm. 7-8 – when the performer would normally slow down at a phrase boundary anyway?

Answer: Given the unusual harmonic context (the sudden move to C# major), the boundary might not easily be noticed here unless especially reinforced by the performer.

Rubinstein’sperformance(1959)

Schumann, “Von fremden Ländern und Menschen” from Kinderszenen

Question: Why does Schumann mark ritardano before the return of theopening theme?

Answer: The return happens at an unexpected place in the phrase structure—after only a six-measure phrase—and with no clear change of harmony; thus the structural boundary might be missed if it were not especially marked by phrase-final lengthening.

Ambiguity Avoidance – Example 3: The rise of “swingtempo” in jazz

Swing tempo: The lengthening of the first half of the beatin relation to the second.

Maple Leaf Rag, as performed by Scott Joplin (1918) and Jelly Roll Morton (1938) (my transcriptions)Excerpt 1

(Joplin)

(Morton)

“Maple Leaf Rag”, Excerpt 2

(Joplin)

(Morton)

Morton’s performance features a great deal more syncopation than Joplin’s—especially left-hand events on weak eighth-note beats; this potentially obscures the quarter-note level of the meter. But it also features a swingtempo, which clarifies the quarter-note level of the meter (by making strong-beat notes longer than weak-beat ones).

And in general, the transition from ragtime to jazz features...

• a shift from the “even-note” rhythms of ragtime to the “swing” feel of jazz (Schuller, 1967)

• an increase in the degree and complexity of syncopation (Waterman, 1974; Sales, 1984)

In each of these cases of ambiguity avoidance...

- Accentuation of metrically strong notes through length and articulation in organ performance

- “Disambiguating” ritards marked at potentially ambiguous phrase boundaries

- The rise of swing tempo in jazz

...we see increased use of “clarifying” performance cues in situations that would otherwise be ambiguous—supporting the idea that these cues arose, in part, as a way of clarifying structure.

Part II: Modulation of Information Flow

Information is the negative log of probability. So in a sequence of events, the lower the probability of an event, the more information it conveys. (Also known as surprisal...)

6543210

1.0

Probability

Information

The Uniform Information Density Hypothesis (Levy & Jaeger, 2007): A certain rate of information flow is optimal inlanguage perception, and language production tends to beadjusted to maintain this rate.

Uniform Information Density in Language (Levy & Jaeger, 2007)

In prosody, words that are more probable are pronounced more quickly:

- More frequent words are shortened (Jurafsky et al., 2001)

- Words that are more probable in local context (e.g. P(wn | wn–1)) are shortened (Jurafsky et al. 2001; Aylett & Turk, 2004)

- Words that are more probable given larger context (e.g. that have already occurred in the discourse) are shortened (Fowler & Jousum, 1987)

Modulation of information flow: What does this predict formusic performance?

One prediction: Less probable events should be performed more slowly.

Modulation of information flow: What does this predict formusic performance?

One prediction: Less probable events should be performed more slowly.

What about unexpected harmonies, e.g. chromaticharmonies?

Sundberg’s (1988) rules for good expressive performance include the “Harmonic Charge Rule”: Slow down on chords whose pitches are far from the tonic on the circle of fifths. (Never really tested with regard to performance.)

Question: Why does Brahms mark sostenuto and a fermata at the phrase ending in mm. 7-8?

Possible Answer: Given the modulation to a distant key and the rapid harmonic rhythm, this passage is high in information content; slowing down smooths out the information flow.

Expressive performance as modulation of information flow: Anecdotal evidence

Brahms, Intermezzo Op. 116 No. 6, mm. 1-8 (again)

How does harmony affect expressive timing? Do performersslow down on unexpected harmonies?

Chris Bartlette’s dissertation (Eastman, 2007) examined thisexperimentally.

Skilled pianists were given pairs of passages to practice and perform. Within each pair, the passages were identical except for one chord, which was diatonic (within the key) in one case and chromatic (outside the key) in the other case.

Effect of harmony on performance expression was examined(timing, dynamics, asynchrony).

Examples (Barlette, 2007)

The second half of a diatonic passage (above) and a chromatic passage (below). (MIDI files generated from experimental trial, reflecting actual timing and dynamics of performance.)

&

?

####

####

86

86

.œ œœœ

œœœ.œ

œ

Jœ œ

œœ

œœœ .œ

œœœœœœ

œœœ

œœœ

œ

Jœ œœœ

œœ

œ .œ

&

?

####

####

œœ

œœœœ

.œ .œ

œ œœœœœ

.œ.œ

œœœ œœ œ

.œ .œ

.˙

œœ œ .œ

[ T i t l e ]

[ T i t l e ]

&

?

####

####

86

86

.œ œœœ

œœœ.œ

œ

Jœ œ

œœ

œœœ .œ

œœœœœœ

œœ

œ

œœœ

œ

Jœ œœœ

œœ

œ .œ

&

?

####

####

œœ

œœœœ

.œ .œ

œ œ œ œ œnœn

.œ.œ

œœœœœœ

.œ .œ

.˙

œœ œ .œ

[ T i t l e ]

[ T i t l e ]

E: ii6

E: bII6

Result: Both the “chord of interest” and the previous chord arelengthened when the chord of interest is chromatic.

Cn-1 Cn Cn+1

(“chord of interest”)

Why? To allow more time for the chord to be processed...“smoothing the information flow.”

Various sources of evidence...

- Sundberg’s rules- Bartlette’s dissertation- anecdotal evidence from score markings

...suggest a preference in performance to slow down at unexpected harmonies. This can be seen as an example ofmodulation of information flow.

Part III. The Syncopation-Rubato Trade-Off

Syncopation - non-alignment between beats and phenomenal accents

Rubato - irregularity or fluctuation in tempo for expressive purposes

The Syncopation-Rubato Trade-Off: Music with moresyncopation seems to permit less rubato.

Rhythm in traditional sub-Saharan African music (compared to Western classical music)

- Greater use of syncopation (Jones 1959, Chernoff 1977, other authors)

- Greater strictness of tempo (less rubato):

“When we Europeans imagine we are beating strict time, the African will merely smile at the ‘roughness’ of our beating.” (Jones 1959, p. 38)

Also: rock and jazz - a high degree of syncopation (compared toclassical Western music); very little tempo fluctuation.

An Experimental Study of the Syncopation-Rubato Trade-Off (with Kelly Francis)

Subjects: Doctoral students in piano, Eastman School of Music

Procedure: Subjects were given short passages and asked to perform them (after practicing). The variance in tempo within each performance was then analyzed.

Materials: Four pairs of musical phrases, with the same pitches in the same approximate positions; one pattern in each pair is unsyncopated (1A), the other is highly syncopated (1B). (Deadpan MIDI performances)

Results

Pianists played unsyncopated excerpts with significantly morevariance in tempo than syncopated excerpts (F(1, 9)=8.17, p=.02).

The effect varied greatly across excerpt pairs:

0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

0.045

1 2 3 4

Excerpt pair

Tempo variance

UnsyncopatedSyncopated

THE BIG QUESTION:

The syncopation-rubato trade-off—is it clarification of structure, OR modulation of information flow?

THE BIG QUESTION:

The syncopation-rubato trade-off—is it clarification of structure, OR modulation of information flow?

Clarification of structure: Could we argue that the combination of rubato and syncopation obscures the metrical structure more than either one alone?

A. A perfectly regular pattern

B. Syncopation: one event occurs slightly before the beat

C. Rubato: intervals between beats are slightly irregular

D. Syncopation and rubato combined: the last beat is slightly late, the last event slightly anticipates it (now the onset pattern is the same as in A!)

The syncopation rubato trade-off as clarification of structure

600 ms 600 ms 600 ms

600 ms 600 ms 600 ms

600 ms 600 ms 800 ms

600 ms 600 ms 800 ms

450 ms

600 ms

intended rhythmbeatsnote onsets

A. A perfectly regular pattern

B. Syncopation: one event occurs slightly before the beat

C. Rubato: intervals between beats are slightly irregular

D. Syncopation and rubato combined: the last beat is slightly late, the last event slightly anticipates it (now the onset pattern is the same as in A!)

The syncopation rubato trade-off as clarification of structure

600 ms 600 ms 600 ms

600 ms 600 ms 600 ms

600 ms 600 ms 800 ms

600 ms 600 ms 800 ms

450 ms

600 ms

intended rhythmbeatsnote onsets

So...syncopation and rubato in combination obscures themeter.

While there are cases where the combination of syncopation and rubato obscures the meter, it is difficult to make a general argument to this effect.

What about modulation of information flow?

q q

The Syncopation-Rubato Trade-Off as Modulation of Information FlowImagine a one-second segment of a melody. The listener must predict when the next note will occur.

P

q q

The Syncopation-Rubato Trade-Off as Modulation of Information FlowImagine a one-second segment of a melody. The listener must predict when the next note will occur.

P

P

P

High information—all locations areequally likely

Low information—one location ismuch more likely than all others

In a style with no syncopation or rubato, the location ofeach note is highly predictable. (Low information)

q q

In a style with rubato, the beat location itself is unpredictableso the note location is as well. (Moderate information)

P

q q

In a style with syncopation, note locations become less predictable because a note may occur on a weak beat. (Moderate information)

P

q q

In a style with rubato and syncopation, the possibility of notes on weak beats combined with the variation in tempo creates high uncertainty. (High information)

P

q q

In a style with rubato and syncopation, the possibility of notes on weak beats combined with the variation in tempo creates high uncertainty. (High information)

Perhaps the combination of syncopation and rubato is avoided to keep information flow at a moderate level.

P

q q

Conclusions

Two goals of music performance:

- Clarification of structure

- Modulation of information flow

The syncopation-rubato trade-off:

Could be clarification of structure, could be modulation of information flow; more likely the latter.

Thank you!

References Aylett, M. and Turk, A. 2004. The Smooth Signal Redundancy Hypothesis: A functional explanation for relationships between redundancy, prosodic prominence, and duration in spontaneous speech. Language and Speech, 47(1):31–56. Bartlette, C. 2007. "A Study of Harmonic Distance and Its Role in Musical Performance." Ph.D dissertation, Eastman School of Music, University of Rochester. Chernoff, J. M. 1979. African Rhythm and African Sensibility. Chicago: Chicago University Press.Clarke, E. 1988. Generative principles in music performance. In J. A. Sloboda, (Ed.), Generative processes in music: The psychology of performance, improvisation, and composition (Oxford: Clarendon), pp. 1–26. Drake, C., & Palmer, C. 1993. Accent structures in music performance. Music Perception 10, 343-78. Fowler, C., & Housum, J. 1987. Talkers’ signaling of ‘new’ and ‘old’ words in speech and listeners’ perception and use of the distinction. Journal of Memory and Language, 26, 489-504. Fry, D. 1955. Duration and intensity as physical correlates of linguistic stress. Journal of the Acoustical Society of America 27, 765-768. Gabrielsson, A. 1999. “The Performance of Music”, in D. Deutsch, Ed., The Psychology of Music (San Diego: Academic Press), pp. 501-602. Hurford, P. 1988. Making music on the organ. Oxford: Oxford University Press, 1988. Jones, A. M. 1959. Studies in African Music. London: Oxford University Press. Jurafsky, D., Bell, A., Gregory, M., & Raymond, W. 2001. Probabilistic relations between words: Evidence from reduction in lexical production. In J. Bybee & P. Hopper (Eds.), Frequency and the Emergence of Linguistic Structure (Amsterdam: John Benjamins), pp. 229-54. Lehiste, I. 1973. Phonetic disambiguation of syntactic ambiguity. Glossa, 7, 107-21. Lerdahl, F., & Jackendoff, R. 1983. A Generative Theory of Tonal Music. Cambridge, MA: MIT Press.

(cont’d on next slide) 

References (cont’d.) Levy, R., & Jaeger, F. 2007. Speakers optimize information density through syntactic reduction. Proceedings of the Twentieth Annual Conference on Neural Information Processing Systems. Palmer, C. 1996. On the assignment of structure in music performance. Music Perception 14, 21-54. Repp, B. 1996. Patterns of note onset asynchronies in expressive piano performance. Journal of the Acoustical Society of America, 100, 3917-3932. Roederer, J. 1975. Introduction to the physics and psychophysics of music. New York: Springer. Sales, G. 1984. Jazz: America's Classical Music. Englewood Cliffs, N.J.: Prentice-Hall.Schuller, G. 1968. Early Jazz. New York: Oxford University Press.Sloboda, J. 1983. The communication of musical metre in piano performance. Quarterly Journal of Experimental Psychology 35, 377-96. Sloboda, J. 1985. The Musical Mind. Oxford: Clarendon Press. Snedeker, J., & Trueswell, J. 2003. Using prosody to avoid ambiguity: Effect of speaker awareness and referential context. Journal of Memory and Language 48, 103-30. Sundberg, J. 1988. Computer synthesis of music performance. In J.A. Sloboda (Ed.), Generative Processes in Music, 52–69. Oxford: Clarendon Press. Temperley, D. 2001. The Cognition of Basic Musical Structures. Cambridge, MA: MIT Press. Temperley, D. 2007. Music and Probability. Cambridge, MA: MIT Press. Todd, N. P. M. 1985. A model of expressive timing intonal music. Music Perception 3, 33-58. Waterman, G. 1974. Ragtime. In N. Hentoff and A. J. McCarthy (eds.), Jazz: New Perspectives on the History of Jazz by Twelve of the World’s Foremost Jazz Critics and Scholars, pp. 43–57. New York: Da Capo Press. Windsor, L., & Clarke, E. 1997. Expressive timing and dynamics in real and artificial musical performances: Using an algorithm as an analytical tool. Music Perception, 15, 127-152.