16
Lyric alignment in popular songs Luong Minh Thang

Lyric alignment in popular songs Luong Minh Thang

Embed Size (px)

Citation preview

Page 1: Lyric alignment in popular songs Luong Minh Thang

Lyric alignment in popular songs

Luong Minh Thang

Page 2: Lyric alignment in popular songs Luong Minh Thang

Outline

Project description & techniquesSome important knowledgeBase-line system

Page 3: Lyric alignment in popular songs Luong Minh Thang

Outline

Project description & techniques Some important knowledgeBase-line system

Page 4: Lyric alignment in popular songs Luong Minh Thang

Project description

Given: textual transcription of lyrics

acoustic musical signal of a song

Purpose: find the timestamps for the beginning and ending points for each line of the song.

Page 5: Lyric alignment in popular songs Luong Minh Thang

Techniques to be investigated

A repetition based technique for detecting self-similarities in both the audio and the text

Dynamic programming (also known as dynamic time warping) will be employed to align the repetition analysis of both media together to produce an alignment

Page 6: Lyric alignment in popular songs Luong Minh Thang

Outline

Project description & techniques Some important knowledgeBase-line system

Page 7: Lyric alignment in popular songs Luong Minh Thang

Chroma vectors

1 octaves with 12 semitones

12-dimensional chroma vector

C C# D … B C C# D…B C C# D…B C C# D…B

Tone heights1 octaves with 12 semitones

0.30.50.1

0.7

Page 8: Lyric alignment in popular songs Luong Minh Thang

Dynamic time warping algorithm

Actually, dynamic programming

Page 9: Lyric alignment in popular songs Luong Minh Thang

Word similarity using phoneme dictionary

Using CMU phoneme dictionaryEach word is decomposed into phoneme

sequence ALIGNMENT: AH0 L AY1 N M AH0 N T ALGORITHMS: AE1 L G ER0 IH2 DH AH0 M Z

Similarity of 2 words is the similarity of the 2 phoneme sequences

Page 10: Lyric alignment in popular songs Luong Minh Thang

Outline

Project description & techniques Some important knowledgeBase-line system

Page 11: Lyric alignment in popular songs Luong Minh Thang

Base-line system: overview

Musical signal input Text input

Chroma-vector calculations

SimplificationText processing

Music notation sequence Word sequence

Word self-similarity matrix

Mapping

Aligning

Phoneme dictionary

Symbol Self-similarity matrix

Page 12: Lyric alignment in popular songs Luong Minh Thang

Base-line system: overview

Musical signal input Text input

Chroma-vector calculations

SimplificationText processing

Music notation sequence Word sequence

Word self-similarity matrix

Mapping

Aligning

Phoneme dictionary

Symbol Self-similarity matrix

Page 13: Lyric alignment in popular songs Luong Minh Thang

Base-line system: overview

Musical signal input Text input

Chroma-vector calculations

SimplificationText processing

Music notation sequence Word sequence

Word self-similarity matrix

Mapping

Aligning

Phoneme dictionary

Symbol Self-similarity matrix

Page 14: Lyric alignment in popular songs Luong Minh Thang

Base-line system: overview

Musical signal input Text input

Chroma-vector calculations

SimplificationText processing

Music notation sequence Word sequence

Symbol Self-similarity matrix

Mapping

Aligning

Phoneme dictionary

Word self-similarity matrix

Page 15: Lyric alignment in popular songs Luong Minh Thang

Base-line system: overview

Musical signal input Text input

Chroma-vector calculations

SimplificationText processing

Music notation sequence Word sequence

Symbol Self-similarity matrix

Mapping

Aligning

Phoneme dictionary

Word self-similarity matrix

Page 16: Lyric alignment in popular songs Luong Minh Thang

Thank you !