38
Fundamentals of Music Processing Chapter 6: Tempo and Beat Tracking Meinard Müller International Audio Laboratories Erlangen www.music-processing.de

Fundamentals of Music Processing

  • Upload
    others

  • View
    9

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Fundamentals of Music Processing

Fundamentals of Music Processing

Chapter 6: Tempo and Beat Tracking

Meinard MüllerInternational Audio Laboratories Erlangen

www.music-processing.de

Page 2: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 2

Book: Fundamentals of Music Processing

Meinard MüllerFundamentals of Music ProcessingAudio, Analysis, Algorithms, Applications483 p., 249 illus., 30 illus. in color, hardcoverISBN: 978-3-319-21944-8Springer, 2015

Accompanying website: www.music-processing.de

Page 3: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 3

Chapter 6: Tempo and Beat Tracking

6.1 Onset Detection6.2 Tempo Analysis6.3 Beat and Pulse Tracking6.4 Further Notes

Tempo and beat are further fundamental properties of music. In Chapter 6, we introduce the basic ideas on how to extract tempo-related information from audio recordings. In this scenario, a first challenge is to locate note onset information—a task that requires methods for detecting changes in energy and spectral content. To derive tempo and beat information, note onset candidates are then analyzed with regard to quasiperiodic patterns. This leads us to the study of general methods for local periodicity analysis of time series.

Page 4: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 4

6 Tempo and Beat TrackingTeaser

Page 5: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 5

6 Tempo and Beat TrackingFig. 6.1

Time (seconds)

Page 6: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 6

6 Tempo and Beat TrackingFig. 6.1

Page 7: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 7

6 Tempo and Beat TrackingFig. 6.1

Page 8: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 8

6.1 Onset DetectionFig. 6.2

Page 9: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 9

6.1 Onset DetectionFig. 6.3

Local energy function

Discrete derivative

Local energy functionLocal energy functionLocal energy function

Novelty function (half-wave rectification)

Novelty function (logarithmic energy)

Annotated note onsets

Page 10: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 10

6.1 Onset DetectionFig. 6.3

Time (seconds)

Local energy function

Annotated note onsets

Page 11: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 11

6.1 Onset DetectionFig. 6.3

Time (seconds)

Annotated note onsets

Discrete derivative

Page 12: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 12

6.1 Onset DetectionFig. 6.3

Time (seconds)

Novelty function (half-wave rectification)

Annotated note onsets

Page 13: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 13

6.1 Onset DetectionFig. 6.3

Time (seconds)

Novelty function (logarithmic energy)

Annotated note onsets

Page 14: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 14

6.1 Onset DetectionFig. 6.4

Piano Violin Flute

Waveform and energy-based novelty function of the note C4 (261.6 Hz)

Page 15: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 15

6.1 Onset DetectionFig. 6.5

Magnitude spectrogram Compressed spectrogram Gamma = 1

Compressed spectrogram Gamma = 1000

Page 16: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 19

6.1 Onset DetectionFig. 6.6

Compressed spectrogram

Novelty function Δ Spectral and local average function

Annotated note onsets

Novelty function subtracting the local average

Page 17: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 20

6.1 Onset DetectionFig. 6.7

Energy-based novelty function

Spectral-based novelty function

Annotated note onsets

Page 18: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 21

6.1 Onset DetectionFig. 6.8

Page 19: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 22

6.1 Onset DetectionFig. 6.9

Wrapped phase

Unwrapped phase

Page 20: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 23

6.1 Onset DetectionFig. 6.10 Jointly consider phase and magnitude

Page 21: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 24

6.1 Tempo AnalysisFig. 6.11

Tempogram with harmonics

Tempogram with sub-harmonics

Page 22: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 25

6.2 Tempo AnalysisFig. 6.12

Assumptions:- beat positions occur at note onset positions- beat positions are more or less equally spaced

Page 23: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 26

6.2 Tempo AnalysisFig. 6.13

Fourier Tempogram

Page 24: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 30

6.2 Tempo AnalysisFig. 6.13

Autocorrelation Tempogram

Page 25: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 39

6.3 Beat and Pulse TrackingFig. 6.18

Predominant Local Pulse (PLP)

Novelty function

Tempogram (with frame-wise tempo maxima)

Optimal windowed sinusoidscorresponding to the maxima

Accumulation of all sinusoids (with overlap-add)

PLP function obtainedafter half-wave rectification

Page 26: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 40

Time (seconds)

6.3 Beat and Pulse TrackingFig. 6.18

Tem

po

(B

PM

)

Page 27: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 41

Time (seconds)

6.3 Beat and Pulse TrackingFig. 6.18

Tem

po

(B

PM

)

Page 28: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 42

Time (seconds)

6.3 Beat and Pulse TrackingFig. 6.18

Tem

po

(B

PM

)

Page 29: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 43

Time (seconds)

6.3 Beat and Pulse TrackingFig. 6.18

Tem

po

(B

PM

)

Page 30: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 44

Time (seconds)

6.3 Beat and Pulse TrackingFig. 6.18

Tem

po

(B

PM

)

Page 31: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 45

6.3 Beat and Pulse TrackingFig. 6.19

Page 32: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 51

6.3 Beat and Pulse TrackingFig. 6.21

Penalty function measuring the deviation of agiven beat period from the ideal beat period

Since tempo deviations are relative in nature (doubling the tempo should be penalized to the same degree as halving the tempo), the penalty function is defined to be symmetric on a logarithmic axis

Page 33: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 52

6.3 Beat and Pulse TrackingFig. 6.22

Beat sequence

Score function

Page 34: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 53

6.3 Beat and Pulse TrackingTable 6.1 Accumulated score

Page 35: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 54

6.3 Beat and Pulse TrackingFig. 6.23

Page 36: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 55

6.3 Beat and Pulse TrackingFig. 6.23

Time (seconds)

Page 37: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 56

6.3 Beat and Pulse TrackingFig. 6.23

Time (seconds)

Page 38: Fundamentals of Music Processing

Meinard Müller: Fundamentals of Music Processing© Springer International Publishing Switzerland, 2015

Chapter 6: Tempo and Beat TrackingSlide 57

6.4 Further NotesFig. 6.24