Polyphonic Transcription

Bruno Angeles

McGill University - Schulich School of Music

MUMT-621

Fall 2009

1/14

Outline

Polyphonic vs Monophonic The Human Method Issues Methods Evaluation of Transcription Methods

2/14

Polyphonic vs Monotonic Transcription

Audio signal with musical content

symbolic format Monophonic: a single instrument playing individual notes Polyphonic transcription:

A single instrument playing several concurrent notes An instrument playing individual notes with other

instruments An instrument playing several concurrent notes with other

instruments Several instruments playing concurrent notes

3/14

The Human Method

1. Initial sketch Sections and key phrases

2. Chord scheme or bass line

3. Melody and counter–melodies

Instrument playback Musical knowledge Beat tracking Style detection Instrument identification

Hainsworth and Macleod (2004)4/14

Issues with Polyphonic Transcription

Restrictions:instrument, genre, maximum polyphony

Transcription:pitch + timing + attack + release

Overlapping harmonics difficult frequency–based analysis

Solution: better understanding and reproduction of the human method?

5/14

A Few Definitions

Musical note: “a discrete note pitch with a specific onset and an offset time”

Melodies: “consecutive note sequences with organized and recognizable shape”

Chords: “combinations of simultaneously sounding notes”

Ryynänen and Klapuri (2005)

6/14

Methods Bottom–up methods:

no high–level analysis

Blackboard systems:develop hypotheses at various levels

Model–based algorithms:high– and low–level analysis + parameter extraction

Common method:

1. Preprocessing (e.g., lowpass filtering)

2. Event extraction

3. Classification: Support Vector Machines, Hidden Markov Models, Neural Networks, Gaussian Mixture Models, etc.

4. Postprocessing (e.g., remove outliers)

Other methods: probabilistic note-based, matrix factorization

7/14

Evaluation of Transcription Methods

NC: number of correct events detected

ND: total number of events detected

N: actual number of events

Precision: P = NC / ND

Recall: R = NC / N

F–Measure: F = 2RP/(R+P) = 2 NC / (N + ND)

8/14

Evaluation of Transcription Methods

Gillet and Richard (2005) 2 drummers Different musical

sequences

Celtic & Groove 5/4:Many ghost notes

Table 4 of Gillet and Richard (2005)9/14

Evaluation of Transcription Methods

Ryynänen and Klapuri (2005):R ≈ P ≈ 40%Multiple instrumentsMultiple concurrent notes

Hainsworth and Macleod (2005):P = 78.7%Bass guitarPolyphonic context

Gillet and Richard (2005):R ≈ P ≈ 84%Drums

10/14Ryynänen and Klapuri (2005):

Conclusion

Analogy with speech recognition The problem has not yet been solved Still easier for humans to do – although complex Combine with source separation?

11/14

Thank you!

12/14

ReferencesBruno, I., S. L. Monni, and P. Nesi. 2003. Automatic music transcription supporting different instruments. In

Proceedings of the Third International Conference on Web Delivering of Music. Leeds, UK. 37–44.

Cemgil, A. T., B. Kappen, and D. Barber. 2003. Generative model based polyphonic music transcription. In Proceedings of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics. New Paltz, NY, USA. 181–4.

Gillet, O. K., and G. Richard. 2005. Drum Track Transcription of Polyphonic Music Using Noise Subspace Projection. In Proceedings of the International Conference on Music Information Retrieval. London, UK. 92–99.

Hainsworth, S. W. 2001. Analysis of musical audio for polyphonic transcription. In First Year Ph.D. Report.

Hainsworth, S. W., and M. D. Macleod. 2001. Automatic bass line transcription from polyphonic music. In Proceedings of the International Computer Music Conference. Havana, Cuba.

Hainsworth, S. W., and M. D. Macleod. 2004. The automated music transcription problem. In Cambridge University Engineering Department.

Klapuri, A. 2004. Signal processing methods for the automatic transcription of music. In Ph.D. Dissertation.

13/14

ReferencesLidy, T., A. Rauber, A. Pertusa, and J. M. Iñesta. 2007. Improving genre classification by combination of audio and

symbolic descriptors using a transcription system. In Proceedings of the International Conference on Music Information Retrieval. Vienna, Austria. 61–6.

Marolt, M. 2001. SONIC: Transcription of polyphonic piano music with neural networks. In Proceedings of the Workshop on Current Research Directions in Computer Music. Barcelona, Spain. 217–24.

Nichols, E., and C. Raphael. 2007. Automatic transcription of music audio through continuous parameter tracking. In Proceedings of the International Conference on Music Information Retrieval. Vienna, Austria. 387–92.

Niedermayer, B. 2008. Non-negative matrix division for the automatic transcription of polyphonic music. In Proceedings of the International Conference on Music Information Retrieval. Philadelphia, PA, USA. 544–9.

Paulus, J. 2006. Acoustic modelling of drum sounds with Hidden Markov Models for music transcription. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. Toulouse, France.

Pertusa, A., and J. M. Iñesta. 2003. Polyphonic music transcription through dynamic networks and spectral pattern identification. In IAPR International Workshop on Artificial Neural Networks in Pattern Recognition. 19–25.

Ryynänen, M., and A. Klapuri. 2006. Transcription of the singing melody in polyphonic music. In Proceedings of the International Conference on Music Information Retrieval. Victoria, BC, Canada.

14/14