Singer Similarity Doug Van Nort MUMT 611. Goal Determine Singer / Vocalist based on extracted...

Singer Similarity

Doug Van Nort

MUMT 611

Goal

Determine Singer / Vocalist based on extracted features of audio signal

Classify audio files based on singer Storage and retrieval

Introduction

Identification of singer fairly easy task for humans regardless of musical context

Not so easy to find parameters for automatic identification

More file sharing and databases leads to increased demand

Introduction

Much work done in speech recognition, performs poorly for singer ID Systems trained on speech data, with no

background noise

The vocal problem has some fundamental differences Vocals exist in variety of background noise Voiced/unvoiced content

Singer recognition similar problem to solo instrument identification

The Players

Kim and Whitman 2002

Liu and Huang 2002

Kim and Whitman

From MIT Media Lab

Singer identification which Assumes strong harmonicity from

vocals Assumes pop music

Instrumentation/levels within critical frequency range

Two step process

Untrained algorithm for automatic segmentation

Classification with training based on vocal segments

Detection of Vocal Regions Filter frequencies outside of vocal

range of 200-2,000 Hz Chebychev IIR digital filter

Detect harmonicity

Filter Frequency Response

Filtering alone not enough Bass and cymbals gone, but Other instruments fall within range

Need to extract features within vocal range to find voice

Harmonic detection

Band limited output sent through bank of inverse comb filters Delay varied

Most attenuated signal represents strongest harmonic content

Harmonicity measure calculated by ratio of signal energy to maximally attenuated signal Allows for establishment of

threshold

Singer Identification

Linear Predictive Coding (LPC) used to extract location and magnitude of formants

One of two classifiers used to identify singer based on formant information

Feature Extraction

A 12-pole linear predictor used to find formants using autocorrelation method

Standard LPC treats frequencies linearly, but human sensitivity is more logarithmic Warp function maps frequencies to

approximation of Bark scale Further beneficial in finding fundamental

Classification Techniques 2 established pattern recognition

algorithms used:

Gaussian Mixture Model (GMM)

Support Vector Machine (SVM)

GMM

Uses multiple weighted Gaussians to capture behavior of each class Each vector assumed to arise from mixture of

gaussian dists.

Parameters for Gaussians found via Expectation Maximization (EM) Mean and variance

Prior to EM, Principal Component Analysis (PCA) taken of data Normalizes variances, avoids highly irregular

scalings which EM can produce

SVM

Computes optimal hyperplane to linearly separate two classes of data

Does not depend on probability estimation

Determined by a small number of data points (support vectors)

Experiments & Results Testbed of 200 songs by 17

different artists/vocalists

Tracks downsampled to 11.025 Khz Vocal range still well below Nyquist

Half of database used for training, half for testing

Two experiments: LPC features taken from entire

song LPC features taken from vocal

segments

1024 frame analysis with hop size of 2

LP analysis used both linear and warped freq scales

Results

Results

Results better than chance (~6%) but fall short of expected human performance

Linear freq alone outperforms warped freq

Oddly, using only vocal segments decreases performance for SVM

Liu and Huang

Based on MP3 database

Particularly high demand for such an approach, given widespread use of Mpeg 1, layer 3

Algorithm works directly on MP3 decoder algorithm

Process

Coefficients of polyphase filter taken from MP3 decoding process

File segmented into phonemes based on said coefficients

Feature vector constructed for each phoneme, and stored along with artist name in database

Classifier trained on database, used to identify unknown MP3 files

Flowchart for singer similarity System of Liu/Huang

Phoneme Segmentation MP3 decoding provides polyphase

coefficients

Energy intensity of each subband is sum of squares of subband coefficients

Frame energy calculated from polyphase coefficients

Energy gap exists between two phonemes

Segmentation looks to automatically identify this gap

Waveform of two phonemes

Frame energy of two phonemes

Phoneme Feature Extraction Phoneme features computed

directly from MDCT coefficients 576 dimensional feature vector for

each frame

Phoneme feature vector of n frames

Classification : setup

Create database of phoneme feature vectors Becomes training set

Discriminating Radius: measure of uniqueness by min Euclidean distance between dissimilar vectors

Good vs. Bad discriminators

Number of similar phonemes within discriminating radius also cosidered

Number of phonemes within radius = wf = frequency of phoneme f

Discriminating ability of each phoneme depends on frequency and distance

Classification: in action Unknown MP3 segmented into

phonemes Only first N used for efficiency

kNN used as classifier K neighbors compared to N

phonemes and weighted by discriminating function

K*N weighted “votes” clustered by singer, and the winner is one with largest score

Experiments/Results

10 Male, 10 Female singers

30 songs apiece 10 phoneme database 10 training (discriminator weights) 10 test set

Free parameters

User defined parameters: k value Discrimination threshold Number of singers in a class

Varying threshold

Varying k

Varying number of singers

Results for all Singers

Conclusion

Not much work yet strictly on singer

Tough because of time and background variances

Quite useful as many people identify artists with singer

Initial results promising, short of human performance

See also: Minnowmatch [Whitman, Flake, Lawrence]