Rhythm related MIR tasks

Ajay Srinivasamurthy1, André Holzapfel1

1MTG, Universitat Pompeu Fabra, Barcelona, Spain

10 July, 2012

Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 1 / 23

1 Rhythm

2 Onset detection

3 Tempo Estimation and Beat Tracking

4 Meter and Time Signature Recognition

5 Rhythmic Similarity/Classification

6 Structural Analysis

Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 2 / 23

Outline

1 Rhythm

2 Onset detection

3 Tempo Estimation and Beat Tracking

4 Meter and Time Signature Recognition

5 Rhythmic Similarity/Classification

6 Structural Analysis

Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 3 / 23

Rhythm

Clayton (1996)Cooper and Meyer (1960)Rhythm: the way one or moreunaccented events are grouped inrelation to an accented one.

London (2001)Rhythm: pattern of durations that isphenomenally present in the music.

There is rhythm without meter, periodicity, or even without pulse!Examples: Turkish taksim, Beijing opera, Indian alapMIR research concentrated on rhythm in music with meter (mainlyWestern music, typically 4/4)It is not only “us”: Clayton (1996) reports the same tendency formusicology.

Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 4 / 23

Outline

1 Rhythm

2 Onset detection

3 Tempo Estimation and Beat Tracking

4 Meter and Time Signature Recognition

5 Rhythmic Similarity/Classification

6 Structural Analysis

Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 5 / 23

Onset detection

Bello (2005) The figure depicts theflowchart of a standard onsetdetection algorithm.Evaluation happens usingeither manually annotatedonsets, or onsets derived frominstruments with MIDI outputs.

(a) Cello Spectr.Magn.

time/s

freq

/kH

z

0.15 0.29 0.44 0.58 0.73 0.87 1.02 1.16

2.75

1.38

4.1

(b) Guitar Spectr.Magn.

Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 6 / 23

Onset Detection

Pre-processingMultiband processingSeparation of percussive/harmonic content

ReductionUsing temporal, spectral magnitude, phase, or F0 information.Feature fusion was proposed to improve performance.Probabilistic models and neural networks were also applied forreduction.

Rhythmic structure was used to improve onset detection recently.

Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 7 / 23

Outline

1 Rhythm

2 Onset detection

3 Tempo Estimation and Beat Tracking

4 Meter and Time Signature Recognition

5 Rhythmic Similarity/Classification

6 Structural Analysis

Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 8 / 23

What is Beat Tracking ?

“Tapping one’s foot in time to music" [Davies-07]

Extract a sequence of beat instants and the correspondinginter-beat intervals given an audio filePerceptually accurate beat timesLocally constant inter-beat intervals (IBI)Causal v/s non-causal beat trackingBeat Tracking on Symbolic v/s audio data

Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 9 / 23

What is Beat Tracking ? (contd...)

ChallengesTempo variationOn-beat and off-beatGenre variationDifferent time signatures

Before we begin - Fundamental QuestionsIs the problem well defined? - Metrical levelsIs it an intuitive and an easy task for humans ?Ground truth for evaluation ?

“All (most) beats occur at onsets, but not all onsets are beats"

Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 10 / 23

Components of a Beat Tracking System

Rhythmic Feature ExtractionExtract relevant rhythm featuresOnset detectionOnset Salience estimation

Tempo InductionDetermine the basic tempo/tempo hypothesesTempo definition for Indian music ?

Beat TrackingEstimate Beat timesBeat phase ?

Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 11 / 23

Approaches to Beat Tracking

Tempo InductionPulse selection methods e.g BeatRoot [Dixon-06]Periodicity functions e.g. [Klapuri-06, Davies-07, Ellis-07]

Beat TrackingMultiple agent architecture e.g BeatRootStatistical Model [Klapuri-06]Dynamic Programming [Ellis-07]Context Dependent Tracking [Davies-07]

Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 12 / 23

Beat Tracking Systems

Davies (2007)Dixon (2006) - BeatRoot

Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 13 / 23

Examples of Beat Tracking

Money (Pink Floyd)

Clip Beats

Charleston Dance piece

Clip Beats

MahaganapatimNaata raga, Chaturashra Eka taala

Clip Beats

Light Indian classicalShivaranjani raga, Jhap ‘like’ taal (pentuple meter)

Clip Beats

Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 14 / 23

Outline

1 Rhythm

2 Onset detection

3 Tempo Estimation and Beat Tracking

4 Meter and Time Signature Recognition

5 Rhythmic Similarity/Classification

6 Structural Analysis

Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 15 / 23

Meter and Time Signature Recognition

Long-term, high level rhythm descriptionBeat similarity based approachOnset detection, Tempo estimation and Beat Tracking trackingnecessaryBeat Similarity matrix based approach: [Gainza-09]Recent work in Indian Music: [Gulati-11], [Miron-11]

Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 16 / 23

Outline

1 Rhythm

2 Onset detection

3 Tempo Estimation and Beat Tracking

4 Meter and Time Signature Recognition

5 Rhythmic Similarity/Classification

6 Structural Analysis

Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 17 / 23

Rhythm Similarity

Figure: Example for periodicitydescriptors (J.H.Jensen)

Periodicity descriptorsCan be considered state-of-the artin MIR.No beat estimation necessary →can be applied to all types ofsignalsTempo robustness can beachieved in various degreesHowever, phase information is lost.Usually evaluated on dance music,or in music similarity tasks.

Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 18 / 23

Rhythm Similarity: Sequential Description

General PropertiesNo information loss because not based on transform magnitudes.Disadvantage: Music must have a pulse which can be estimated.

Paulus and Klapuri (2002)Using spectral centroid and loudness to derive pattern description.Dynamic Time Warping is used to compare patterns.

Whiteley et al.(2007)

Probabilistic framework is proposedto infer tempo and rhythmic patternsOnly applied to MIDI signals in thispaper, rhythmic patterns are given.

Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 19 / 23

Outline

1 Rhythm

2 Onset detection

3 Tempo Estimation and Beat Tracking

4 Meter and Time Signature Recognition

5 Rhythmic Similarity/Classification

6 Structural Analysis

Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 20 / 23

Structural Analysis

Chorus detectionMain goal: Detect repetitions of parts of a songMost common: Self-similarity matrix analysis

Cover song detectionIdentify if two songs are different interpretations of the samecompositionFeatures: e.g. chroma features, chord estimations.Similarity measures e.g. dynamic programming, string matching.

Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 21 / 23

Rhythm

Structural segmentationMain goal: Obtain a musical meaningful segmentation of a songinto large time-scale structures.Again, self similarities play a big role.Also, HMM were applied for labeling states at beat level, and thenfind similarities in the state distributions to obtain segments (Levyand Sandler (2008)).

Figure: Compare structure of a query in form of a score to the structure ofaudio (Martin et al. 2009)

Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 22 / 23

References

Gouyon-05 Gouyon, F. A Computational Approach to Rhythm Description. PhD Thesis, Pompeu FabraUniversity, Barcelona, 2005

Dixon-06 Dixon, S. Evaluation of The Audio Beat Tracking System Beatroot. Journal of New MusicResearch 36 (1), pp. 39-50, 2006

Davies-07 Davies, M. and Plumbley, M. Context-Dependent Beat Tracking of Musical Audio. IEEETransactions on Audio, Speech, and Language Processing 15 (3), pp. 1009-1020, 2007.

Ellis-07 Ellis, D. P. W. Beat Tracking by Dynamic Programming. Journal of New Music Research,36(1), pp. 51-60, 2007.

Gainza-09 Gainza M., Automatic musical meter detection, in Proc. ICASSP 2009, pp. 329-332,Taipei, Taiwan

MIREX-06 http://www.music-ir.org/mirex/wiki/2006:Audio_Beat_Tracking

Uhle-03 Christian Uhle, Juergen Herre, Estimation of Tempo, Micro Time and Time Signature fromPercussive Music, in Proc. of 6th Int. Conference on Digital Audio Effects (DAFX-03),London, UK, September 8-11, 2003

Gulati-11 Sankalp Gulati, Vishweshwara Rao and Preeti Rao, Meter Detection from Audio for IndianMusic, CMMR/FRSM 2011, Bhubhaneswar, March 2011

Miron-11 M. Miron, Automatic Detection of Hindustani Talas. Master’s thesis, Universitat PompeuFabra, Barcelona, Spain, 2011.

Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 23 / 23