15
A Parametric Piano Synthesizer Jukka Rauhala, Mikael Laurson, Vesa V ¨ alim ¨ aki, Heidi-Maria Lehtonen, and Vesa Norilo Department of Signal Processing and Acoustics Helsinki University of Technology P.O. Box 3000, FI-02015 TKK, Espoo, Finland www.acoustics.hut.fi [email protected], {vesa.valimaki, heidi-maria.lehtonen} @tkk.fi Centre for Music and Technology Sibelius Academy P.O. Box 86, FI-00251, Helsinki, Finland cmt.siba.fi {laurson, vnorilo}@siba.fi Sound synthesis of the piano is a great challenge. The piano sound is difficult to synthesize mainly be- cause it has a rich, somewhat inharmonic spectrum, and the decay characteristics of its partials vary sig- nificantly over time. The piano is also a well-known musical instrument, so many listeners will notice any deficiencies in the sound quality of a synthetic piano. Nevertheless, the grand piano is particularly interesting for sound synthesis, owing to its large size and high cost. Digital pianos imitating grand pianos are currently among the most popular electronic musical instruments. Piano-synthesis products are usually based on sampling techniques. In extreme cases, the tones of all the keys of the instrument are sampled at several velocity levels to cover the whole dynamic range, and these samples are as long as necessary, even about a minute each for low piano tones. Excellent sound quality can be achieved, but it is dependent on the size of the sample memory. Limitations in memory reduce the obtainable quality, because samples must be shortened or their bit rate must be compressed. In this work, we examine parametric piano sound synthesis, which attempts to generate natural- sounding tones algorithmically without using a large sample database. It should provide high- Computer Music Journal, 32:4, pp. 17–30, Winter 2008 c 2008 Massachusetts Institute of Technology. quality piano sound synthesis to systems that cannot afford a large memory, such as mobile phones and portable electronic games. Moreover, it contributes additional physical realism to the sound synthesis, and it allows the timbre of the instrument to be adjusted parametrically. As the features of the piano tone are coded into a large set of parameters, it allows the synthesis model to be turned into a specific piano simply by setting the parameters into those obtained from the specific actual piano. Furthermore, the highly parametric nature of the model enables exploration of parameter values that are difficult to achieve in real pianos, owing for example to physical constraints. Recently, a commercial product based on physical modeling was introduced independently by Pianoteq (www.pianoteq.com). We describe a physics-based piano synthesizer based on digital waveguide mod- eling (Smith 1992; Smith 2006; V ¨ alim ¨ aki et al. 2006). An early attempt to design a waveguide piano synthesizer was reported by Garnett (1987). Later, Van Duyne and Smith developed a digital piano syn- thesizer based on commuted waveguide synthesis (Smith and Van Duyne 1995; Van Duyne and Smith 1995). Borin, Rocchesso, and Scalcon (1997) and Bank (2000) worked on a full physical piano model includ- ing hammers and their interaction with strings. In this article, we first discuss the structure of the synthesis model and its various components. Then, Rauhala et al. 17

A Parametric Piano Synthesizer

  • Upload
    vesa

  • View
    216

  • Download
    0

Embed Size (px)

Citation preview

Page 1: A Parametric Piano Synthesizer

A Parametric PianoSynthesizer

Jukka Rauhala,∗ Mikael Laurson,†

Vesa Valimaki,∗Heidi-Maria Lehtonen,∗ andVesa Norilo†

∗Department of Signal Processingand AcousticsHelsinki University of TechnologyP.O. Box 3000, FI-02015 TKK,Espoo, [email protected],{vesa.valimaki, heidi-maria.lehtonen}@tkk.fi†Centre for Music and TechnologySibelius AcademyP.O. Box 86, FI-00251,Helsinki, Finlandcmt.siba.fi{laurson, vnorilo}@siba.fi

Sound synthesis of the piano is a great challenge.The piano sound is difficult to synthesize mainly be-cause it has a rich, somewhat inharmonic spectrum,and the decay characteristics of its partials vary sig-nificantly over time. The piano is also a well-knownmusical instrument, so many listeners will noticeany deficiencies in the sound quality of a syntheticpiano. Nevertheless, the grand piano is particularlyinteresting for sound synthesis, owing to its largesize and high cost. Digital pianos imitating grandpianos are currently among the most popularelectronic musical instruments. Piano-synthesisproducts are usually based on sampling techniques.In extreme cases, the tones of all the keys of theinstrument are sampled at several velocity levels tocover the whole dynamic range, and these samplesare as long as necessary, even about a minute eachfor low piano tones. Excellent sound quality canbe achieved, but it is dependent on the size of thesample memory. Limitations in memory reducethe obtainable quality, because samples must beshortened or their bit rate must be compressed.

In this work, we examine parametric piano soundsynthesis, which attempts to generate natural-sounding tones algorithmically without using alarge sample database. It should provide high-

Computer Music Journal, 32:4, pp. 17–30, Winter 2008c© 2008 Massachusetts Institute of Technology.

quality piano sound synthesis to systems thatcannot afford a large memory, such as mobilephones and portable electronic games. Moreover, itcontributes additional physical realism to the soundsynthesis, and it allows the timbre of the instrumentto be adjusted parametrically. As the features of thepiano tone are coded into a large set of parameters,it allows the synthesis model to be turned intoa specific piano simply by setting the parametersinto those obtained from the specific actual piano.Furthermore, the highly parametric nature of themodel enables exploration of parameter values thatare difficult to achieve in real pianos, owing forexample to physical constraints.

Recently, a commercial product based on physicalmodeling was introduced independently by Pianoteq(www.pianoteq.com). We describe a physics-basedpiano synthesizer based on digital waveguide mod-eling (Smith 1992; Smith 2006; Valimaki et al.2006). An early attempt to design a waveguide pianosynthesizer was reported by Garnett (1987). Later,Van Duyne and Smith developed a digital piano syn-thesizer based on commuted waveguide synthesis(Smith and Van Duyne 1995; Van Duyne and Smith1995). Borin, Rocchesso, and Scalcon (1997) and Bank(2000) worked on a full physical piano model includ-ing hammers and their interaction with strings.

In this article, we first discuss the structure of thesynthesis model and its various components. Then,

Rauhala et al. 17

Page 2: A Parametric Piano Synthesizer

we present the implementation of the synthesizerusing PWGL software.

Piano Synthesis Model

A general overview of the waveguide piano synthesismodel used in this work is shown in Figure 1. Eachkey is connected to two string models Sj,1 andSj,2, which are slightly detuned by approximatelyone cent, and to a hammer simulation block Hj,where j is the key index. It is possible to usemore string models than just two if desired, but itwould increase the computational load and modelcomplexity. Figure 2 shows a block diagram of asingle-string model, which consists of six blocks:excitation model, loss filter, delay line, tuning filter,dispersion filter, and beating model.

When the player presses a single key, the corre-sponding string models and the hammer simulationblock receive a trigger signal. Then, the triggersignal is passed on to the excitation model, whichsimulates how the hammer strike excites the stringto vibrate, and to the loss filter, which simulates thelifting of the damper. Similarly, when the pressedkey is released, a trigger signal is again sent to thestring models. In addition, when the sustain pedalis pressed, the loss filters in all active string modelswill receive a trigger signal to simulate the lifting ofall dampers.

A more detailed explanation of the excitationmodel, the loss filter, the dispersion filter, and thebeating model blocks is given in the following.The tuning filter used in the synthesis model isa first-order allpass filter (Jaffe and Smith 1983;Laakso et al. 1996). In addition, a new approach forimplementing sympathetic resonances is presented.

String Model Excitation and Hammer Simulation

As the player presses a key on a grand piano,a hammer strikes a set of strings. This can beseen as a two-fold event. First, the hammer strikepasses energy to the strings causing them to beginvibrating. Second, the hammer strike produces anaudible knocking sound. These two parts can be alsodescribed as a harmonic component and a broadband

Figure 1. General view ofthe piano-synthesis model.Sj,k are the string models,and Hj are thehammer-simulationblocks.

component of the piano tone (Keane 2007; Lehtonenet al. 2007). This is taken into account in themodel by separating the hammer simulation intotwo blocks: an excitation model inside the stringmodel simulating the energy that is passed on tothe strings, and a hammer simulation block thatsimulates the additional knocking sound.

In this work, an additive synthesis-based approachis used in the excitation model (Rauhala andValimaki 2006a). The basic idea of the approachis to use additive synthesis to set the amplitudelevels of important partials and to use filteredwhite noise to compensate for brightness. High-passed noise is used for keys 1–49 with cutofffrequencies ranging from 2,450 Hz to 7,350 Hz. Theblock diagram of the excitation model is shown inFigure 3. The model consists of five blocks: additivesynthesis block, noise generator block, equalizingfilter block (EQ), one-pole filter block (LP), anda windowing function block. The purpose of thefilters is to provide velocity-controlled dynamicsfilters simulating the nonlinear behavior of thehammer elasticity to the produced tone, while thewindowing function, which is a Hamming windowsplit into half and applied to the beginning andto the end of the signal, prevents any undesirablespectral components caused by sharp edges in thewaveform of the excitation signal. Other researchershave proposed alternative ways for simulating thehammer-string interaction, such as the use of a

18 Computer Music Journal

Page 3: A Parametric Piano Synthesizer

Figure 2. Block diagram ofa single string model.

Figure 2

Figure 3. Block diagram ofthe excitation model(adapted from Rauhalaand Valimaki 2006a).

Figure 3

nonlinear hammer model (Bank and Sujbert 2002),or by considering tension modulation with largeamplitudes (Bilbao 2004).

The hammer-tone block uses an approach basedon modal synthesis, as this technique is able toproduce impact sounds. A block diagram of thehammer-tone block is shown in Figure 4. First,a source signal is produced by low-pass filteringband-limited white noise (the bandwidth of thenoise is 0–4 kHz). For key indices less than 40,the low-pass filter has a constant cutoff frequency(2 kHz), whereas for the rest of the key indices,the cutoff frequency depends on the f0 (the cutofffrequency is taken to be 2 f0). The second-orderlowpass filter is designed with the Chebyshev filter-design method using a multi-rate technique. Thefilter is initially designed for a frequency band from0 to fs/3, and then it is upsampled to cover the fullfrequency band. Filter parameters are pre-calculatedand stored into memory. To produce modes ofthe signal, the signal is filtered with a cascade ofsecond-order equalizing filters. A single equalizingfilter amplifies the frequencies close to the f0, while

the rest of the filters simulate single peaks in thesignal. Then, the signal is shaped with a logarithmicenvelope generator whose output signal is shownin Figure 5. Finally, the signal is filtered with thesame velocity-dependent equalizing filter as in theexcitation model. Table 1 shows the parametersfor the six equalizing filters that were obtained byanalyzing recorded piano tones and by manuallyselecting dominant spectral peaks.

Simulation of Losses

The losses in the stringed instruments can bemodeled with a low-pass filter because, in general,high frequencies tend to decay faster than the lowfrequencies (Jaffe and Smith 1983; Valimaki et al.1996). In the piano, however, the decay processes oftones are not that straightforward. Some of the low-frequency partials can decay quickly; moreover, thedecay times may vary significantly, even betweenadjacent partials. As a consequence, a simple low-pass filter is not capable of producing enoughvariation to the magnitude response and, thus, tothe decay process of a tone.

Specific design algorithms have been proposed(e.g., Smith 1983; Bank and Valimaki 2003), forsimulating the losses in vibrating strings. Rauhala,Lehtonen, and Valimaki (2005) presented a multi-ripple filter design method that extends the ideaof matching the decay time of one harmonicwith an inverse comb filter, proposed by Valimakiet al. (2004), by matching the decay times ofseveral partials accurately. Lehtonen, Rauhala, andValimaki (2005) introduced a similar filter structure

Rauhala et al. 19

Page 4: A Parametric Piano Synthesizer

Table 1. Parameters for the Equalizing Filters in the Hammer-Tone Simulation Block

f0 < 465 Hz f0 ≥ 465 Hz

Filter fh (Hz) fhbw (Hz) gh (dB) fh (Hz) fhbw (Hz) gh (dB)

EQ1 f0 2 f0 38 f0 2 f0 1EQ2 1.034 f0 10 45 0.964 f0 10 9EQ3 3.103 f0 10 48 0.949 f0 10 33EQ4 4.828 f0 10 49 0.939 f0 10 21EQ5 13.62 f0 10 48 0.336 f0 10 35EQ6 26.21 f0 10 44 0.217 f0 10 45

Parameter fh is the center frequency of the notch, fhbw is the bandwidth of the notch, and gh is the peak gain at thenotch.

Figure 4. Block diagram ofthe hammer-tonegenerator.

Figure 4

that consists of three FIR subfilters. The idea in themulti-ripple design method is to cascade a one-poleloop filter matching the overall decay behavior witha sparse FIR filter designed using the frequency-sampling method. To obtain a good fit even withlow filter orders, the data is smoothed to facilitatethe design problem. The block diagram of the lossfilter by Rauhala, Lehtonen, and Valimaki (2005)inserted into a waveguide string model is presentedin Figure 6. The transfer function of the multi-rippleloss filter and the delay line can be written as

HMR(z) = bz−L + ∑Nb

n=1 rnz−Rn

1 + aopz−1

where b and aop are the gain factor and the parameterof the one-pole filter, respectively; rn and Rn are theripple gain and the ripple delay length, respectively;and Nb is the number of feedforward paths (branches).

An example fit for the piano tone B0 (correspond-ing to key index 3) is given in Figure 7. The resultsare given in the T60-domain, which is a perceptually

Figure 5. Envelope for thehammer tone.

Figure 5

meaningful domain for designing loss filters (T60 isthe time that it takes for a partial to decay 60 dBfrom its peak). In the present work, the loss filterproposed by Rauhala, Lehtonen, and Valimaki (2005)with five feedforward paths (Nb = 5) is used.

Simulation of Dispersion

Piano strings are highly dispersive, making theproduced tones inharmonic. It has been suggestedthat inharmonicity is linked to the perceivedwarmth in piano tones (Fletcher, Blackham, andStratton 1962). In digital waveguide synthesis,dispersion is simulated by inserting an allpass filter

20 Computer Music Journal

Page 5: A Parametric Piano Synthesizer

Figure 6. Block diagram ofthe multi-ripple loss filterwith Nb branches proposedby Rauhala, Lehtonen, andValimaki (2005) combinedwith the delay line. Theparameters aop and b are

the one-pole filterparameter and the gainparameter, respectively; r1,. . . , rNb are the ripple-gainparameters; and R1, . . . ,RN are the ripple delay-linelengths.

Figure 6

into the feedback loop. Inharmonicity can be viewedas a frequency-dependent phase delay of the feedbackloop that sets the target for the allpass filter design(Van Duyne and Smith 1994; Rocchesso and Scalcon1996).

Rauhala and Valimaki (2006b) proposed a tunabledispersion filter design method, which is used inthis piano synthesizer. It is based on the Thiranfilter-design method, which is commonly used fordesigning fractional-delay allpass filters. Rauhalaand Valimaki observed that the phase-delay re-sponse of an allpass filter designed with the Thiranmethod nicely approximates the desired phase delayresponse at various fundamental frequency valuesand inharmonicity coefficient values with a simpleparameterization. A cascade of four second-orderallpass filters for the low fundamental frequenciesand a single second-order allpass filter for the highfundamental frequencies are used.

Simulation of Beating

Beating is another significant phenomenon occur-ring in piano tones. It is caused by the coupling ofstrings. Moreover, it can occur in keys that haveonly one string attached (Capleton 2004). Beatingcan be easily detected in low piano tones a fewseconds after the attack.

In this work, a beating model proposed byRauhala, Lehtonen, and Valimaki (2007) is used.Another approach is to use two string models

Figure 7. Target response(dots) and the response ofthe multi-ripple loss filter(solid line) with fivefeedforward paths in (a)dB-scale and (b) inT60-domain for key B0(f0 = 30.9 Hz).

Figure 7

(Aramaki et al. 2001), which does not offer directcontrol over the phenomenon. Another methoduses parallel resonators (Bank 2000), which requires

Rauhala et al. 21

Page 6: A Parametric Piano Synthesizer

Figure 8. Working principleof the beating model.

accurate values of the partial frequencies, becausefrequency modulation is sensitive to changes inthe frequencies of the modulated signals. As thedispersion filter may produce slight deviations fromthe partial frequencies, parallel resonators cannotbe used in applications where inharmonicity andfundamental frequency parameters must be tuned inreal-time. On the other hand, the model suggested byRauhala, Lehtonen, and Valimaki (2007) is suitablefor real-time applications, as it produces the beatingeffect with amplitude modulation, and, hence, asmall bias in the estimation of partial frequenciesdoes not affect the beating effect significantly. In thismethod, parallel beating blocks are inserted to thesystem, each producing the beating effect for a singlepartial. A block diagram of the higher-level beatingmodel in the piano-synthesis model is shown inFigure 8, and a block diagram of a partial-beatingblock is presented in Figure 9. Each beating blockconsist of a bandpass filter, which separates thetarget partial from the rest of the signal, and a low-frequency oscillator (LFO) modulator that modulatesthe target partial signal (Figure 9). Finally, the depthof the beating effect is controlled via gain parametergc.

Simulation of Sympathetic Resonances

Sympathetic resonance (Le Carrou et al. 2005)means that energy is leaked from vibrating stringsto other non-damped strings via the bridge or viaair radiation, and other strings begin ringing as aresult. This is one of the phenomena that cannot beproduced with the sampling technique. Sympathetic

Figure 9. Partial-beatingmodel.

resonances can be perceived, for example, when agroup of keys are pressed down silently, thus liftingup the dampers, and forte staccato notes are played.As a result, the vibrations of the non-damped stringscan be heard. Another case, where sympatheticresonances affect the piano sound strongly, is whenthe sustain pedal is used (Lehtonen et al. 2007).

Jaffe and Smith (1983) suggested using a setof external string models to produce sympatheticvibrations. Other similar approaches have beenpublished by Smith (1993) and Valimaki et al.(1996). Karjalainen, Valimaki, and Tolonen (1998)proposed a model for acoustic guitar synthesisin which the phenomenon is produced with theexisting string models. The structure included twostring models per string for horizontal and verticalpolarizations. The sympathetic resonances wereexcited by using the output from the horizontalstring model and it was fed to the vertical stringmodels of all string models, thus making the systemstable, because there is no feedback actually. Thisapproach has been applied also to synthesis of theclavichord (Valimaki, Laurson, and Erkut 2003) andthe piano (Bensa, Jensen, and Kronland-Martinet2004), where the phenomenon is simulated betweenstrings attached to a single key. Borin, Rocchesso,and Scalcon (1997) have introduced a more advancedsimulation method in which the outputs of allthe active piano string models are added and fedto a bridge-admittance filter. Then, the differencebetween the filtered signal and the output of acertain string model is transmitted back to thestring model.

The goal for the sympathetic-resonance simula-tion is to enable signal propagation from all vibratingstring blocks to other non-damped strings while pre-serving stability. In this work, a simulation methodis proposed extending the previous approaches. Theblock diagram of the proposed method is shown

22 Computer Music Journal

Page 7: A Parametric Piano Synthesizer

Figure 10. Block diagramof the proposedsympathetic-resonancesimulation method. Keys 1to Ns are the active keys atthe moment.

in Figure 10. The main idea is to have two stringblocks (denoted here as primary and secondarystring blocks), where the energy is flowing upward(from the first key to the last key) in the primarystring blocks and downward in the secondary stringblocks. Moreover, the energy between the stringblocks within a single key is defined to flow onlyfrom the secondary string block to the primary stringblock. Hence, the energy is flowing inside a singlekey and between all keys without making the modelunstable, because there is no closed loop within themodel. The major difference between this methodand the method suggested by Borin, Rocchesso, andScalcon (1997) is that with this method it is possibleto control how much the sympathetic resonance af-fects a single string, whereas in the method by Borin,Rocchesso, and Scalcon, practically the same sym-pathetic resonance signal is fed to all string models.

Usually, synthesizers are implemented in sucha way that the number of simultaneous notes islimited to 32 or 64, for example. In the case of thepiano, the number of active string blocks can belimited. Hence, information about which stringblocks are active is available and can be used inthe sympathetic-resonance simulation to apply itonly to the active string models, which reducesthe computational load. Coefficients ki, li, and micontrol the amount of energy leaking from the stringblock to another, where i = 1, 2, . . . , Ns, and Ns is thenumber of active strings at a specific moment. Thecomputational load per sample in this approach is2(Ns – 1) +Ns multiplications and 2(Ns – 2) additions.

Figure 11 shows a block diagram of the stringblocks within a single key and their relations tothe sympathetic-resonance simulation. The signalfrom the sympathetic resonance blocks is added to

the excitation signal before feeding it to the stringmodel. When these additions are taken into accountin the computational load, then the number ofadditions per sample is increased to 2(3Ns – 2), whilethe number of multiplications remains the same.

In the real-time implementation presented in thiswork, the method is further simplified by settingmi = 0 and ki = li = 0.005 for all i = 1, 2, . . . , Ns. Inother words, a constant value is used for all atten-uation parameters. The signal propagation from thesecondary strings to the primary strings is disabled,because it would affect the beating phenomenonthat is simulated in the beating block. In addition,all string models that are initialized accordingto the available polyphony are included in thesympathetic-resonance simulation. Also, the stringmodels are not in a specific order in the simulation.

Implementation and Calibration

Piano Model Implementation

The synthesis, control, and calibration parts of thepiano synthesizer are realized using a visual softwaresynthesis package called PWGLSynth (Laurson,Norilo, and Kuuskankare 2005). PWGLSynth, inturn, is part of a larger visual programming environ-ment called PWGL (Laurson and Kuuskankare 2006).The control information is normally generated usinga music notation package called ENP (ExpressiveNotation Package; Kuuskankare and Laurson 2006),or recently also from a MIDI controller (Laurson andNorilo 2006). Previous work in designing computersimulations of musical instruments has resultedin several applications, such as the classical guitar

Rauhala et al. 23

Page 8: A Parametric Piano Synthesizer

Figure 11. Block diagramof the two stringscorresponding to a singlekey with thesympathetic-resonancesimulation.

(Laurson et al. 2001), the Renaissance lute andthe Turkish ud (Erkut et al. 2001), the clavichord(Valimaki, Laurson, and Erkut 2003), the harpsi-chord (Valimaki et al. 2004), and the Chinese guqin(Penttinen et al. 2006).

The piano model tackled in the current studyis challenging and complex owing to its enormousrange and large number of parameters. This kind ofwork is partly based on analysis results and partlyon experimental testing. Thus, the synthesis modelmust be refined by interactive listening, and for thiswe need a system that is capable of making fastand efficient prototypes. To achieve these goals, wedeveloped an environment where the same visualinstrument definition can be controlled either froma high-level musical score or from a MIDI controller.The heart of the instrument definition consists ofa special loop constructor box, called copy-synth-patch, that allows the duplication of arbitrarysubpatches. The system generates automaticallysymbolic pathnames to the required control entrypoints to parameterize the synthesizer. Theseimplementation problems are discussed in moredetail in Laurson, Norilo, and Kuuskankare (2005).

Figure 12 gives an overview of our system. Thispatch supports both real-time and non-real-timemodes: MIDI mode (MIDI) and score mode (score).The current mode can be selected by using a masterswitch box labeled MSW in the lower-right corner

of the box. The master switch box can have one orseveral slave switch boxes that will follow the stateof the master switch. Both the master and slaveswitch boxes share the same box-string, which is inour case equal to MIDI/score.

The left part of the figure gives the top-leveldefinition of a piano synthesizer prototype. Itconsists of a string-model abstraction (the boxlabeled string) that is duplicated by the copy-synth-patch box Np times, where Np is eitherthe maximum number of voices needed for thecurrent musical score (get-max-voice-count) inscore mode, or the maximum number of voicesreserved for MIDI control in real-time mode. Theright part of Figure 12, in turn, shows an examplescore (Score-Editor), which has been prepared inadvance by the user. In addition to basic musicalinformation, such as pitches and rhythms, it cancontain special instrument-specific expressions thatallow the indication of various playing styles andperformance practices that are needed to producemusically acceptable performances.

Piano Model Calibration

Although automatic analysis of parameters may pro-duce good results (Bensa, Gipouloux, and Kronland-Martinet 2005), an instrument model typically still

24 Computer Music Journal

Page 9: A Parametric Piano Synthesizer

Figure 12. Top-level patchdefinition of our pianomodel prototype. Thismodel supports bothscore-based and real-timecontrol. The current

control mode (either MIDIor score) can be selectedusing the master switchbox (MSW) found in theright part of the figure.

needs a finalizing phase in which the user fine-tunesvarious parameters of the model by ear. To facili-tate this phase for the piano model, we designed ageneric matrix editor box in PWGL that maps allpitch classes (from C to B) as columns, and all octaves(from 0 to 8) of the instrument as rows. The octavecontaining middle C (C4) is shown in darker colorthan the other octaves. Figure 13 gives two exampleswhere we attempt to fine-tune the instrument byhand with the help of scalars (thus the names of the

matrices pno-f0-sc and pno-B-sc). These valuesare multiplied with the f0 parameters (fundamentalfrequencies) and B parameters (which control theinharmonicity of the strings) found in the databasederived from the automatic analysis, where thedefault values are obtained from recorded tones byanalyzing them automatically and then fine-tuningthem manually. By default, all matrix values are1.0; thus, the model uses database values withoutchange. The user can now tune individual keys with

Rauhala et al. 25

Page 10: A Parametric Piano Synthesizer

Figure 13. Two visualinstrument-calibrationtools that allow tuning thepiano model withhigh-level controlparameters forfundamental frequencyand inharmonicity.

the mouse either below or above 1.0 by ear whilesimultaneously using a MIDI keyboard. This schemeallows the user to adjust f0 and B values indepen-dently, and these parameter changes will automati-cally affect other low-level parameters—such as thelength of the delay line—that depend on either f0 orB values.

A similar approach can also be used for other crit-ical parameters, for instance to find the correct bal-ance between the hammer noise and excitation, or tocalibrate attack and release times for the envelopes.

Conclusion

In this article, a piano synthesizer implementedwithout any sampled sounds is presented. Thephysics-based synthesis model, which uses the dig-

ital waveguide synthesis technique, includes a lossfilter, a dispersion filter, a simulation of the beatingeffect, a simulation of the sympathetic resonanceeffect, and a block that produces the knockingtone simulating the hammer strike. The modelis developed using methods that enable real-timecontrol of the parameters, such as the inharmonicitycoefficient and the fundamental frequency.

The major differences between the proposedpiano model and the previous models are in theexcitation process and in the real-time controlover the inharmonicity. The previous models(Borin, Rocchesso, and Scalcon 1997; Bank et al.2003) use physics-based hammer models to excitethe string that do not allow tuning of individualpartial amplitudes, whereas in the proposed model,individual partial amplitudes can be tuned easily.Moreover, in the previous models, the inharmonicity

26 Computer Music Journal

Page 11: A Parametric Piano Synthesizer

of individual strings must be determined in advanceto compute the dispersion filter parameters offline,whereas in the proposed model, the dispersion filtercan be controlled in real time.

The proposed model is implemented usingPWGL software, where the synthesis model can becontrolled with a MIDI keyboard or a notation tool.Moreover, the synthesizer has a visual instrumentcalibration tool that can be used for fine-tuningthe model parameters in real time. Examples ofparameter values that were determined semi-automatically from recorded tones used in this workare presented in the Appendix. Sound examples canbe found on the Computer Music Journal Sound andVideo Anthology DVD accompanying this issue.

As the proposed model is highly parametric, itis an excellent tool for perceptual experiments inwhich the perception of specific parameter values isinvestigated; future work includes performing theseperceptual experiments. Moreover, the results fromthese experiments can provide information on howto further develop the model to produce perceptually“good” piano tones.

Acknowledgments

This work was financially supported by theAcademy of Finland (Projects No. 104934, 105557,114116, 122815, and 126310). Jukka Rauhalawas supported by the Nokia Foundation, andHeidi-Maria Lehtonen was supported by the GETAgraduate School, Tekniikan edistamissaatio, theFinnish Cultural Foundation, the Nokia Foundation,and the Emil Aaltonen Foundation.

References

Aramaki, M., et al. 2001. “Resynthesis of Coupled PianoString Vibrations Based on Physical Modeling.” Journalof New Music Research 30(3):213–226.

Bank, B. 2000. “Physics-Based Sound Synthesis ofthe Piano.” Report No. 54, Helsinki Universityof Technology, Laboratory of Acoustics and AudioSignal Processing, Espoo, Finland. Available online atwww.acoustics.hut.fi/publications.

Bank, B., and L. Sujbert. 2002. “On the NonlinearCommuted Synthesis of the Piano.” Proceedings ofthe Intenational Conference on Digital Audio Effects.Hamburg, Germany: University of the Federal ArmedForces, pp. 175–180.

Bank, B., and V. Valimaki. 2003. “Robust Loss FilterDesign for Digital Waveguide Synthesis.”IEEE SignalProcessing Letters 10(1):18–20.

Bank, B., et al. 2003. “Physically Informed Signal Process-ing Methods for Piano Sound Synthesis.” EURASIPJournal on Applied Signal Processing 2003(10):941–952.

Bensa, J., O. Gipouloux, and R. Kronland-Martinet.2005. “Parameter Fitting for Piano Sound Synthesis byPhysical Modeling.” Journal of the Acoustical Societyof America 118(1):495–504.

Bensa, J., K. Jensen, and R. Kronland-Martinet. 2004.“A Hybrid Resynthesis Model for Hammer-StringInteraction of Piano Tones.” EURASIP Jour-nal on Applied Signal Processing 2004(7):1021–1035.

Bilbao, S. 2004. “Energy-Conserving Finite DifferenceSchemes for Tension-Modulated Strings.” Proceedingsof the IEEE International Conference on Acoustics,Speech, and Signal Processing. Piscataway, New Jersey:Institute of Electrical and Electronics Engineers, pp.285–288.

Borin, G., D. Rocchesso, and F. Scalcon. 1997. “A PhysicalPiano Model for Music Performance.” Proceedings ofthe 1997 International Computer Music Conference.San Francisco, California: International ComputerMusic Association, pp. 350–353.

Capleton, B. 2004. “False Beats in Coupled Piano StringUnisons.” Journal of the Acoustic Society of America115(2):885–892.

Erkut, C., et al. 2001. “Model-Based Synthesis of theUd and the Renaissance Lute.” Proceedings of the2001 International Computer Music Conference. SanFrancisco, California: International Computer MusicAssociation, pp. 119–122.

Fletcher, H., E. D. Blackham, and R. Stratton. 1962. “TheQuality of Piano Tones.” Journal of the AcousticalSociety of America 34(6):749–761.

Garnett, G. E. 1987. “Modeling Piano Sound UsingWaveguide Digital Filtering Techniques.” Proceedingsof the 1987 International Computer Music Conference.San Francisco, California: International ComputerMusic Association, pp. 89–95.

Jaffe, D. A., and J. O. Smith III. 1983. “Extensions of theKarplus-Strong Plucked-String Algorithm.” ComputerMusic Journal 7(2):76–87.

Rauhala et al. 27

Page 12: A Parametric Piano Synthesizer

Karjalainen, M., V. Valimaki, and T. Tolonen.1998. “Plucked-String Models: From the Karplus–Strong Algorithm to Digital Waveguides andBeyond.” Computer Music Journal 22(3):17–32.

Keane, M. 2007. “Separation of Piano Keyboard Vibrationsinto Tonal and Broadband Components.” AppliedAcoustics 68(10):1104–1117.

Kuuskankare, M., and M. Laurson. 2006. “ExpressiveNotation Package.” Computer Music Journal 30(4):67–79.

Laakso, T. I., et al. 1996. “Splitting the Unit Delay—Toolsfor Fractional Delay Filter Design.” IEEE SignalProcessing Magazine 13(1):30–60.

Laurson, M., and M. Kuuskankare. 2006. “Recent Trendsin PWGL.” Proceedings of the 2006 InternationalComputer Music Conference. San Francisco, California:International Computer Music Association, pp. 258–261.

Laurson, M., and V. Norilo. 2006. “From Score-BasedApproach Towards Real-Time Control in PWGLSynth.”Proceedings of the 2006 International Computer MusicConference. San Francisco, California: InternationalComputer Music Association, pp. 29–32.

Laurson, M., V. Norilo, and M. Kuuskankare. 2005.“PWGLSynth: A Visual Synthesis Language for VirtualInstrument Design and Control.” Computer MusicJournal 29(3):29–41.

Laurson, M., et al. 2001. “Methods for Modeling RealisticPlaying in Acoustic Guitar Synthesis.” ComputerMusic Journal 25(3):38–49.

Le Carrou, J.-L., et al. 2005. “Modelling of SympatheticString Vibrations.” Acta Acustica United withAcustica 91(2):277–288.

Lehtonen, H.-M., J. Rauhala, and V. Valimaki. 2005.“Sparse Multi-Stage Loss Filter Design for WaveguidePiano Synthesis.” Proceedings of the 2005 IEEEWorkshop on Applications of Signal Processing toAudio and Acoustics. Piscataway, New Jersey: Instituteof Electrical and Electronics Engineers, pp. 331–334.

Lehtonen, H.-M., et al. 2007. “Analysis and Modeling ofPiano Sustain-Pedal Effects.” Journal of the AcousticalSociety of America 122(3):1787–1797.

Penttinen, H., et al. 2006. “Model-Based Sound Synthesisof the Guqin.”Journal of the Acoustical Society ofAmerica 120(6):4052–4063.

Rauhala, J., H.-M. Lehtonen, and V. Valimaki. 2005.“Multi-Ripple Loss Filter for Waveguide PianoSynthesis.” Proceedings of the 2005 International

Computer Music Conference. San Francisco, California:International Computer Music Association, pp. 729–732.

Rauhala, J., H.-M. Lehtonen, and V. Valimaki. 2007. “To-ward Next-generation Digital Keyboard Instruments.”IEEE Signal Processing Magazine 24(2):12–20.

Rauhala, J., and V. Valimaki. 2006a. “ParametricExcitation Model for Waveguide Piano Synthesis.” Pro-ceedings of the 2006 IEEE International Conference onAcoustics, Speech, and Signal Processing. Piscataway,New Jersey: Institute of Electrical and ElectronicsEngineers, pp. 157–160.

Rauhala, J., and V. Valimaki. 2006b. “Tunable DispersionFilter Design for Piano Synthesis.” IEEE SignalProcessing Letters 13(5):253–256.

Rocchesso, D., and F. Scalcon. 1996. “Accurate DispersionSimulation for Piano Strings.” Proceedings of the 1996Nordic Acoustical Meeting. Espoo, Finland: AcousticalSociety of Finland, pp. 407–414.

Smith, J. O. 1983. “Techniques for Digital Filter Designand System Identification with Application to theViolin.” Ph.D. thesis, Department of Music, StanfordUniversity.

Smith, J. O. 1992. “Physical Modeling Using DigitalWaveguides.” Computer Music Journal 16(4):74–91.

Smith, J. O. 1993. “Efficient Synthesis of Stringed MusicalInstruments.” Proceedings of the 1993 InternationalComputer Music Conference. San Francisco, California:International Computer Music Association, pp. 64–71.

Smith, J. O. 2006. Physical Audio Signal Processing:For Virtual Musical Instruments and Audio Effects.Available on-line at ccrma.stanford.edu/∼jos/pasp.

Smith, J. O., and S. A. Van Duyne. 1995. “Commuted PianoSynthesis.” Proceedings of the 1995 International Com-puter Music Conference. San Francisco, California: In-ternational Computer Music Association, pp. 335–342.

Valimaki, V., M. Laurson, and C. Erkut. 2003. “CommutedWaveguide Synthesis of the Clavichord.” ComputerMusic Journal 27(1):71–82.

Valimaki, V., et al. 1996. “Physical Modeling of PluckedString Instruments with Application to Real-TimeSound Synthesis.” Journal of the Audio EngineeringSociety 44(5):331–353.

Valimaki, V., H. et al. 2004. “Sound Synthesis of theHarpsichord Using a Computationally EfficientPhysical Model.” EURASIP Journal on Applied SignalProcessing 2004(7):934–948.

Valimaki, V., et al. 2006. “Discrete-Time Modelling ofMusical Instruments.” Reports on Progress in Physics69(1):1–78.

28 Computer Music Journal

Page 13: A Parametric Piano Synthesizer

Van Duyne, S. A., and J. O. Smith. 1994. “A SimplifiedApproach to Modeling Dispersion Caused by Stiffnessin Strings and Plates.” Proceedings of the 1994International Computer Music Conference. SanFrancisco, California: International Computer MusicAssociation, pp. 407–410.

Van Duyne, S. A., and J. O. Smith. 1995. “Developmentsfor the Commuted Piano.” Proceedings of the 1995International Computer Music Conference. SanFrancisco, California: International Computer MusicAssociation, pp. 319–326.

Appendix

The parameter values used in the proposed modelfor all C notes are shown in Figures 14–16 and inTable 2.

Figure 14. Partialamplitude values for Cnotes on a linear scale. Thevalues are scaled between0 and 1.

Figure 14

Figure 15. Multi-ripplefilter rrate values for notesC1–C5. Values for notesC6–C8 are zero. Parameterrrate,1 is equivalent to R1/L.

Figure 15

Figure 16. Multi-ripplefilter rn values for notesC1–C5. Values for notesC6–C8 are zero.

Figure 16

Rauhala et al. 29

Page 14: A Parametric Piano Synthesizer

Table 2. Parameter Values for C Notes

C1 C2 C3 C4 C5 C6 C7 C8

f0 32.7 65.4 130.8 261.6 523.3 1046.5 2093.0 4186.0B 0.00026 0.00015 0.00013 0.00030 0.00072 0.0017 0.00416 0.01aop −0.444 −0.012 −0.012 −0.001 −0.003 −0.04 −0.04 −0.04b 0.550 0.976 0.981 0.995 0.994 0.958 0.959 0.960hg 1.92 2.49 2.58 2.83 36 40 26 5Gmin −72 −70 −65 −62 −62 −62 −62 −62gn −12.8 −9.3 −19.1 −24.3 −14.3 −29.2 −31.9 −30gm −42.7 −39.2 −31.7 −13 −46.3 0 0 0w0 703 703 703 800 800 800 800 800β 2160 2160 2160 4050 4050 4050 4050 4050

Parameter f0 is the fundamental frequency, B is the inharmonicity coefficient value, aop is the one-pole filtercoefficient of the multi-ripple loss filter, b is the loop gain, and hg is the hammer tone gain. The remaining parametersare excitation method parameters: Gmin is the one-pole filter block gain in dB, gn is the notch parameter of theequalizing filter in dB, gm is the overall equalizing filter gain in dB, w0 is the notch bandwidth in Hz, and β is thenotch center frequency in Hz.

30 Computer Music Journal

Page 15: A Parametric Piano Synthesizer

This article has been cited by:

1. Leonardo Gabrielli, Vesa Välimäki, Henri Penttinen, Stefano Squartini, Stefan Bilbao. 2013. A digital waveguide-based approachfor Clavinet modeling and synthesis. EURASIP Journal on Advances in Signal Processing 2013:1, 103. [CrossRef]

2. Jacques SteynUniversal Information Architecture of Acoustic Music Instruments 193-240. [CrossRef]3. Nelson Lee, Julius O Smith, Vesa Valimaki. 2010. Analysis and Synthesis of Coupled Vibrating Strings Using a Hybrid Modal-

Waveguide Synthesis Model. IEEE Transactions on Audio, Speech, and Language Processing 18:4, 833-842. [CrossRef]4. Balázs Bank, Stefano Zambon, Federico Fontana. 2010. A Modal-Based Real-Time Piano Synthesizer. IEEE Transactions on

Audio, Speech, and Language Processing 18:4, 809-821. [CrossRef]5. J.S. Abel, V. Valimaki, J.O. Smith. 2010. Robust, Efficient Design of Allpass Filters for Dispersive String Sound Synthesis. IEEE

Signal Processing Letters 17:4, 406-409. [CrossRef]