Automatic Generation of Karaoke-Game Stages from Audio Files

Natthawut KulnirundornProfessor Boonserm Kijsirikul

Department of Computer EngineeringChulalongkorn University

1/9/2009

1

Project Background• Karaoke Game

▫ User try to sing at correct pitch for each note

▫ Can analyze how well you sing

• People enjoy playing karaoke game

▫ Existing commercial products: SingStar, Rockband

1/9/2009

2

How to play karaoke game?

54100

1/9/2009

3

Problems with current karaoke games• Limited number of stages

▫ Usually pre-packed with game

▫ Cannot add new song

• Available on limited platforms▫ Singstar on PS3▫ Rockband on XBOX360

Need to buy new game to play more songs!

1/9/2009

4

Proposed Solution

1/9/2009

5

Project Objectives

1/9/2009

6

Vocal Pitch Detection in Polyphonic Audio• Still an open research

problem

• Cannot simply apply the same algorithms as pitch detection in single source

▫ Autocorrelation

▫ Harmonic Product Spectrum (HPS)

• Current approaches based on many different techniques

▫ Machine Learning

▫ Independent Component Analysis (ICA)

Detected frequency from HPS

Freq. from Human voice highlighted in red

1/9/2009

7

timefreq

Demonstration – Stage Generation

1/9/2009

8

Demonstration – Karaoke Game

1/9/2009

9

Method Proposed

1/9/2009

10

Spectrum Subtraction

1/9/2009

11

normal

voice-removed

result

time

freq

*Human singing frequency

Conclusion

•Enable users to sing a song in native languages.

•The method proposed is easy to implement and can be applied to classes of application

•The prototypes can be developed further to be an entertainment application

1/9/2009

12

Questions ?

1/9/2009

13

References

1/9/2009

14

Appendix1/9/2009

15

Autocorrelation

1/9/2009

16

Harmonic Product Spectrum

1/9/2009

17