Articulatory Synthesis of Singing

Peter Birkholz

Institute for Computer Science, University of Rostock

Singing Synthesis Challenge 2007 at the Interspeech‘07, Antwerp

28.08.2007 Articulatory Synthesis of Singing 2

Introduction

• Our synthesis of singing is based on a comprehensive articulatory speech synthesizer that was developed at the University of Rostock since about 2001 (Birkholz et al. 2004-2007).

• The synthesizer comprises a geometrical model of the vocal tract, and modules for articulatory control and the aero-acoustic simulation.

• The input for the synthesizer is a gestural score.• For the synthesis of singing, the synthesizer has been extended

by...– a rule-based creation of gestural scores from musical scores– an implementation of pitch-dependent supraglottal articulation– the incorporation of pitch effects like vibrato and overshoot

28.08.2007 Articulatory Synthesis of Singing 3

The articulatory speech synthesizer (1)

28.08.2007 Articulatory Synthesis of Singing 4

The articulatory speech synthesizer (2)

28.08.2007 Articulatory Synthesis of Singing 5

From musical scores to gestural scores

• An inherent duration is assigned to each speech sound

• Reduced consonant duration in consonant clusters

• Pitch transitions and supraglottal transitions towars the first phone in a syllable start at the same time

• „Phase rules“ determine the coordination of glottal and supraglottal gestures

28.08.2007 Articulatory Synthesis of Singing 6

Pitch-dependent supraglottal articulation (1)

• Why? Look at the transfer function and the harmonics of the voice source for the vowel /i:/ at 110 Hz and 440 Hz:

28.08.2007 Articulatory Synthesis of Singing 7

Pitch-dependent supraglottal articulation (2)

• As soon as F0 exceeds F1, the articulation is adapted in such a way that F1 follows approximately F0 (Sundberg, 1975)

28.08.2007 Articulatory Synthesis of Singing 8

Pitch effects

• The pitch contour is defined in terms of pitch targets according to Xu et al. (2006, 2007)

• Approximation of these targets is realized using 3rd order dynamical systems

• Pitch effects: Vibrato, overshoot, preparation (Saitou et al., 2004)

28.08.2007 Articulatory Synthesis of Singing 9

And now: The Synthesizer Song !

• „Dona nobis pacem“:

• The Synthesizer Song: