Morphing Musical Instrument Sounds

8/12/2019 Morphing Musical Instrument Sounds

1/324


2/324


3/324


4/324


5/324


6/324


7/324


8/324


9/324


10/324


11/324


12/324


13/324


14/324


15/324


16/324


17/324


18/324


19/324


20/324


21/324


22/324


23/324


24/324


25/324


26/324


27/324


28/324


29/324
http://www.composersdesktop.com/http://www.cerlsoundgroup.org/Lemur/http://www.cerlsoundgroup.org/Lemur/http://www.cerlsoundgroup.org/Loris/http://www.symbolicsound.com/cgi-bin/bin/view/Company/WebHomehttp://www.bantusound.com/SoundMorphing/SoundMorphingPage.html


30/324


31/324

= 0.5


32/324


33/324


34/324


35/324

y (t)

ys(t)

yr(t)

y (t) =ys(t) + yr(t)

yr(t) ys(t)

y (t)

yr(t) = y (t) ys(t) ys(t) yr(t)


36/324


37/324


38/324


39/324


40/324


41/324


42/324


43/324


44/324


45/324


46/324


47/324


48/324

r rM = MN M

M

F N

rM = 0.7

M

P N =P = [1, 1, , 1] [p1, p2, , pN]

T

N

P M F

P

M,F

M = MP = [1, 0, , 1] [p1, p2, , pN]

T

P

r

rM,F =P

M,FP =

PM,FN


49/324

[0, 1]

M = [0.25, 1, 0.5, 0.75]


50/324


51/324


52/324


53/324

AC


54/324

A

p, q,r B

p, q, , n

1p+ 2q+ ... + nn


55/324

i i 0 1+ 2+ ... + n = 1

1p+ 2q =p+ (1 ) q = p,q

n

1 + 2= 1

n

N n

=

123

N

p

+ (1 ) q

=

p1p2p3

pN

+ (1 )

q1q2q3

qN

=

p1p2p3

pN

+

(1 ) q1(1 ) q2(1 ) q3

(1 ) qN

=p,q

j

j =

123

N

= 1 p = 0 q

= 0.5


56/324

p q r

A

p q r B p p p p


57/324


58/324


59/324


60/324

p p
http://i40.tinypic.com/11tqy52.jpg


61/324


62/324


63/324


64/324


65/324


66/324


67/324


68/324


69/324


70/324


71/324


72/324

x


73/324

x

x

N

1N1

N


74/324

x

x

x

x


75/324


76/324


77/324


78/324

x (t) =K

k=1

Akcos k(t)

Ak kth

k K

x (t) t

(t) =

t

0 () d

f

2

f= 1

2

d

dt


79/324

x (t) =Kk=1

Akcos (2fkt + k)

x (t)

Ak fk

kth t k

Ak k = 2fkt+k

x (n)

N

ck(n) =

n+N1r=n

x (r)cos(rk0)

dk(n) =n+N1r=n

x (r)sin(rk0)

0 = 2/N

ak(n) k(n)

ak(n) =

c2k(n) + d2k(n)

k(n) = arctan

dk(n)

ck(n)

k(n)

x (t)

X(f) =

x (t)exp(j2f t) dt

X(f)

x (t)

t

x (n) X(k) N


80/324

N x0...xN1 N

X0...XN1

X(k) N1n=0

x (n) ej2N

kn

k= 0...N 1

x (n) 1

N

N1n=0

X(k) ej2N kn

n= 0...N 1

X(k, n) =

m=w (n m) x (m)exp

j

2

N

km

, k= 0, 1, , N 1

X(k, n)

kth

n w (m) M

x (m)

N/2+ 1

N

X(k, n)

yn(m) = x (m) w (n m)


81/324

X(k, n) k

n

n yn(m) x w

yn x w

w

X(k, n) n k

w (m)

w


82/324

X(k, n)

x (n)exp j 2N km w (n) X(k, n) =

x (n)exp

j

2

N

km

w (n)

w (n)

x (n)exp

j2N

km

x (n) exp

j2N

km

x (n) k = 2k/N x (n) k

w (n)

n m= l

X(k, n) = ejn

lw (l) x (n l) e

jl

=ejn x (n) w (n) ejn

w (n) ejn

x (n)

ejn

x (n) =K

k=1

ak(n) ejk(n)

ak(n) k(n) ck

dk

K

ak(n)

k(n)

= ck dk dkck

c2k+ d2k


83/324

2

2

w


84/324


85/324


86/324

N

1 a1 f1 12 a2 f2 2

N aN fN N

an nth

fn

n


87/324

x (t) =K

k=1

Ak(t)cos k(t) + e (t)

Ak(t) k(t) kth

e (t)


88/324


89/324


90/324


91/324


92/324

S(, t)

Ss(, t)

S Ss Sp(, t)


93/324


94/324


95/324


96/324


97/324


98/324

(N 1)

N k k


99/324


100/324


101/324


102/324


103/324

=

tta(t)Pta(t)

t

a (t)

t

lat at1

at2

lat= log (at2 at1)


104/324

k

p (k) = |X(k)|

k |X(k)|

m= E[(p (k))m

] =k

kmp (k)

p (k)

p (k)

=k

kp (k)

p (k)

p (k)

2 =k

(k )2p (k)

3=

k(k )

3p (k)

3


105/324

p (k)

4=

k(k )

4p (k)

4


106/324

fm = flfbC

fl < fb

fm= C1 + log flfb fl > fb

fm fl fb

C

fb C

C=N/2

1 + logSRN2

2fb

SR

N

0 1000 2000 3000 4000 50000

20

40

60

80

100

120

Bass ClarinetLinear Spectrum

Linear Frequency (Hz)

LinearAmplitude

Linear Spectrum

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

x 104

30

25

20

15

10

5

0

Mid Ear Filter

Log Frequency (Hz)

LogAmplitude(dB)

Mid Ear Filter

0 5000 10000 15000100

80

60

40

20

0

20

Bass ClarinetPerceptual Spectrum

Mel Frequency (Hz)

LogAmplitude(dB)

Perceptual Spectrum


107/324


108/324


109/324


110/324


111/324


112/324

e (t) E()

v (t)

V ()

e (t) v (t)

s (t) = e (t) v (t)

S() = E() V ()


113/324

1/

1/


114/324

|V ()|


115/324

e (t)

Sc

x (t)

e (t)

Sc x (t)

Wc

x (t)

Wc

Wc


116/324

s (t)

x (t)

Wc

Wc

Sc

e (t)

Sc

Sc x (t)

Sc

x (t)

s (t)

x (t)

Wc s (t)

ss(t) sr(t)

s (t) = ss(t) + sr(t)

sr(t)

ss(t) s (t) sr(t) = s (t) ss(t) ss(t) sr(t)


117/324

s (t)

Wc x (t)

x (t) = xs(t) + xr(t)

Wc

s (t) = x (t)Wc(t) = [xs(t) + xr(t)]Wc(t) =xs(t)Wc(t)+xr(t)Wc(t) =ss(t)+sr(t)

xs(t)

Wc hs(t)

ys(t) Wc yr(t) xr(t)


118/324


119/324

mn

2nt2

+ nQn

nt

+ 2nn

= F(t)

mn p

12

mn(n/t)2p , n = 2fn

Qn

F(t)

p


120/324


121/324

fk(t) kf0(t) |H(f, t) | Ak(t) k |H[fk(t)] | k(t) k k(t) s[fk(t)] k

Ak(t)

ss(t) sr(t)

xs(t) Wc xs(t) hs(t)

fk(t) hs(t)

Hs()

Hs()


122/324

sr(t) Wc

xr(t) xr(t)

sr(t)

H(f, t)


123/324


124/324

S() =H()

S H() |H()|

H()

S1 = S1 (H()) |H()| S


125/324

1 2

S1 S2 T 1 2

2= T(1)

|H()| 1

S1(1) = H()

2 H() S12 S

12 (H()) = 2


126/324


127/324

s (n)

S(z)

n=s (n) zn

H(z)

u (n)

H(z)

a (k) = 0 1 k p

b (l) = 0 1 l q

s (n)

u (n)

s (n) =

pk=1

a (k) s (n k) + Gu (n)

H(z)

H(z) = G

1 +p

k=1 a (k) zk

s (n)

a (k)

s (n)

s (n)

u (n)

s (n)

s (n) s (n)

s (n) =

pk=1

a (k) s (n k)


128/324

s (n) s (n)

e (n) = s (n) s (n) =s (n) +p

k=1

a (k) s (n k)

e (n)

a (k)

s (n)

s (n)

E

E=n

e2 (n) =n

s (n) +

pk=1

a (k) s (n k)

2

s (n)

E E

E

a (i)= 0

1 i p

p

k=1

a (k)n s (n k) s (n i) = n s (n) s (n i)

1 i p

s (n) p p {a (k) , 1 k p} E

Ep

Ep =n

s2 (n) +

pk=1

a (k)n

s (n) s (n k)

n

< n <

pk=1

a (k) R (i k) = R (i)


129/324

1 i p

Ep = R (0) +

pk=1

a (k) R (k)

R (i) =

n=s (n) s (n + i)

s (n)

R (i)

R (i) = R (i)

R (i k)

s (n)

s (n)

w (n) s (n) 0 n N 1

s (n) =

s (n) w (n) , 0 n N 1

0, .

R (i) =N1i

n=0s (n) s (n + i)

i 0

w (n)

E

0 n N 1

pk=1

a (k) (k, i) = (0, i) , 1 i p

Ep = (0, 0) +

p

k=1 a (k) (0, k)

(i, k) =N1n=0

s (n i) s (n k)

s (n)

(k, i)

(i, k)


130/324

(i, k) = (k, i)

(i + 1, k+ 1) = (i, k) + s (i 1) s (k 1) s (N 1 i) s (N 1 k)

s p n N 1 p+N

n

sn e (n)

E=

e2 (n)

=

s (n) +

pk=1

a (k) s (n k)

2

pk=1

a (k) ( (n k) s (n i)) = (s (n) s (n i)) , 1 i p

Ep =

s2 (n)

+p

k=1

a (k) (s (n) s (n k))

s (n)

s (n)

(s (n k) s (n i)) = R (i k)

R (i)

R (i)

s (n)

a (k)

s (n)

(s (n k) s (n i)) = R (n k, n i)

R (t, t)

t

t

R (n k, n i)

n

a (k)

n= 0


131/324

pk=1

a (k) R (k, i) = R (0, i)

Ep = R (0, 0) +p

k=1

a (k) R (0, k)

s (n)

R (k, i) (i, k)

a (k)

R (t, t) = R (t t)

G

s (n) =

p

k=1 a (k) s (n k) + e (n)

u (n)

s (n) Gu (n) = e (n)

u (n)

H(z)

s (n)

u (n)

s (n)

H(z)

Gu (n) Ep

a (k) , 1 k p p p

p3/3+ Op2

p2

p3/6+ Op2

p2/2


132/324

R0 R1 R2 Rp1R1 R0 R1 Rp2R2 R1 R0 Rp3

Rp1 Rp2 Rp3 R0

a1a2a3

ap

=

R1R2R3

Rp

p p

2p

p2 + O (p)

E0= R (0)

k (i) = [R(i)+Pi1

j=1ai1(j)R(ij)]/Ei1

ai (i) = k (i)

ai (j) = ai1 (j) + k (i) ai1 (i j) , 1 j i 1

Ei =

1 k2 (i)

Ei1

i = 1, 2, . . . , p

a (j) = ap (j) , 1 j p

p

p

pN N p

R (i) R (0)

r (i)


133/324

r (i) = R (i)

R (0)

|r (i)| 1

Ei Ei

Ei

0 Ei Ei1, E0= R (0)

Ei R (0) Vi

Vi = EiR (0)

= 1 +

ik=1

a (k) r (k)

0 Vi 1, i 0

Vp

Vp =

pi=1

1 k2i

ki 1 i p

ki

sn sn+i sn+1 sn+i1

ki

Zi Zi+1 ki

ki = Zi+1 ZiZi+1+ Zi

H(z)

Zi+1Zi

=1 + ki1 ki

, 1 i p

p

ki


134/324

p

p

p p Vp

p0 Vp = Vp0 , p p0

kp = 0, p > p0 p > p0 p

p0

p > p0

p > p0

1 Vp+1

Vp<


135/324

p

m

ELP =

1

N

N

m=1P(m)

P(m)

P() s (n) P()

P(m) P(m) P() P() m m

ELP

P(m)

ELP

P()

P(m)

P(m)

P() P() P()

P(m) P(m) P(m)

P() Ro

P() =

l=Ro(l) e

jl

R

R (i) = 1

N

Nm=1

l=

Ro(l) ejm(li), i


136/324

R Ro

m m= 2(m1)/N

R (i) =

l=

Ro(i lN) , i

ELP RLP

P() R (i) , P(m)

RLP(i) =R (i) =

l=

Ro(i lN) =Ro, 0 i p

Ro P(m)

P()

x (n) = s (n) s (n )


137/324

|X(k)|

2

= |S(k)|

2 1 + 2 + 2 cos (2k)

X(k) = log |X(k)|2 = log |S(k)| + log

1 + 2 + 2 cos(2k)

X(k)

L [x (n)] = L [x1(n) + x2(n)] = L [x1(n)] + L [x2(n)] =y1(n) + y2(n) =y (n)

L [ax (n)] =aL [x (n)] =ay (n)

L a

y (n) =

k=h (n k) x (k) = h (n) x (n)

H [x (n)] = H [x1(n) x2(n)] = H [x1(n)] H [x2(n)] = y1(n) y2(n) = y (n)


138/324

y1(n) = (n)

D []

D [x (n)] =D [x1(n) x2(n)] = D [x1(n)] + D [x2(n)] =

x1(n) +

x2(n) =

x (n)

D1

D1 [y (n)] = D1 [y1(n) + y2(n)] = D

1 [y1(n)] D

1 [y2(n)] = y1(n) y2(n) = y (n)

x (n) = x1(n) x2(n)

X(z) = X1(z) X2(z)

X(z) = log [X(z)] = log [X1(z) X2(z)] = log [X1(z)] + log [X2(z)]


139/324

z= ej Xej= log Xej+j arg Xej

log(z)

(, ]


140/324

x (nT) =

Z1

log |X(z)|22

=

1

2j

C

log |X(z)| zn1dz

2

X(z) x (nT)


141/324

x (nT) = 1

2j

C

log [X(z)] zn1dz

x (0) = log [x (0)]

X(z)

x (nT)

X(z) log X(z) x (nT)

X(z)

X(z)

X(z) = log X(z)

x1 x2

x2 x x= x1 x1 x1 x2

2

log[X(z)]

x (nT)


142/324

log[X(z)]

2

/T

x (nT)

log[X(z)]

C(k) 2

P(k)

C(k)

C(0) = 0

C(k) =

C(k 1) 2, P(k) P(k 1)>

C(k 1) + 2, P(k 1) P(k)>

C(k 1) ,

/2

2

x (nT) = 0 n 0

n= 0

x (n) =

Z1 [log (X(z) X (z))]

2

=

Z1 [log (X(z)) + log (X (z))]

2

x (n) X (z) =X

z1

x (n) =

12jC

log |X(z)| zn1dz+ 1

2j

C

logXz1 zn1dz

2

z= z1

x (n) =

1

2j

C

log |X(z)| zn1dz+ 1

2j

C

log |X(z)| zn1dz

2


143/324

x (n) = [x (n) + x (n)]2

x (n)

x (n) = 1

2

X

ej

ejnd= 1

2

logXej ejnd+ j

2

arg

X

ej

ejnd

c (n) = 1

2 log Xej ejnd

x (n)

x (n)

c (n) = 1

N

N1k=0

log |X(k)| ej2Nkn

x (n) |X()| arg [X()] x (n) X() x (n)


144/324

|X(k)|

C(k) =N1n=0

w (n) c (n)exp

j2kn

N

C(k)

w (n)

w (n) =

1, |n| < nc

0.5, |n| =nc

0, |n| > nc

nc

C(k)


145/324

0 0.5 1 1.5 2

x 104

5

4

3

2

1

0

1

2

Cepstral Smoothing

Frequency (Hz)

Amplitude(dB)

Original Spectrum

Smoothed Spectrum

X()

S()

d (Y, Z)

C P() X() S() P()

pm P() C

d (X,SC) S() = 1, d (Y, Z)

C

P()

P() =L1k=0

epkcosk

log |P()| =L1k=0

pkcos (k) =

L1k=0

(2 k0) ckcos (k) = c0+ 2L1k=1

ckcos(2f k)

S()

X()

= {n, n= 1 . . . N }

n sn xn

S() =N

n=1

sn( n)


146/324

X() =

Nn=1

xn( n)

()

hn

(c) =N

n=1

hn[log |P(n)| log(xn)]2 =

Nn=1

hn

L1k=0

(2 k0) ckcos (k) log (xn)

2

|H1()| |H2()| c1 c2

X() X()

dX() , X()= 12

X()X()

logX()X()

1 d

d

X() , X()

=d

X() , X()

c

(c)

ck=

Nn=1

hn

L1k=0

(2 k0) ckcos (k)

cos(kk) = 0

Ac= a

Aij =N1k=0

hk(2 k0)cos(ik)cos(jk)

P

B

aij =N1k=0

hklog (xk)cos(ik)

A

r


147/324

ri = 12

N1

k=0

hkcos (ik)

aij =ri+j rij

L

L

(i, xi)

P rn(, x) sn = hn = 1, n

(c) =

N1n=0

P rn(, x) [log |P(n)| log xn]

2 ddx

P rn(, x)

P rn(, x) (n, xn) (k, xk) hk = P rk(k, xk) P rn(, x)

AC = B


148/324

X(k) K x (n) Ci(k)

i

Ai(k)

Ci1(k)

Ai(k) = max (log |X(k)| , Ci1(k))

Ai(k) Ci(k)

A0(k) = log |X(k)| C0(k) i

log |X(k)| Ci(k) Ai(k)

F0

FS FS = 2F0

F

O= FS2F

=FSF0

, = 0.5


149/324

H(z)

A (z)

A (z)

a (k) , 1 k p

p + 1

a (k)

(i) , 0 i p

A (z)

i = (0) + 2

pj=1 (j)cos

2ij2p+1

, 0 i p (i)

i { (i)}

A (z)

H(z)

A (z)

z (k) , 1 k p {z (k)}

s

z (k) = es(k)T

s (k) = (k) +j (k)

s

T z (k) = zr(k) +jzi(k) zr(k) zi(k)

z (k)

(k) = 1/Tarctan (zi(k)/zr(k))

(k) = 1/2Tlog

z2r(k) + z2i (k)

ki, 1 i p

p+ 1

p

G

p + 1

z


150/324


151/324

A (z)

A1 (z)

Mth

A (z) = 1 +

M

m=1

amzm

As(z) M

{as(m)}

As(z) = zMAs

z1

Aa(z) {as(m)}

Aa(z) = zMAa

z1

(M+ 1)th A (z) P(z) Q (z)

(M+ 1)th

P(z) = A (z) + z(M+1)A

z1

Q (z) =A (z) z(M+1)A

z1

k

A (z)

A (z) = P(z) + Q (z)

2

z


152/324


153/324

z= zm Fk(z) Fk(zm) = 0 |zm| = 1

F1(z) F2(z)

z = 1

N z = 1 F1(z) F2(z)

z(j)i = exp

j2

(m)k

Fk(z)


154/324

k

k

k =k+ (k+1 k)

k+1= k+1 (k+1 k)

k

k = k+ k( 1) ( k)

p


155/324

M

{am}

1/A(z)

log

1

A (z)

=

n=1

cnzn

am cn

z1

z1

cn = an 1

n

n1k=1

kckank


156/324

an = cn+1

n

n1k=1

kckank

A(z)/A(z)

cn = 1

n(1)n

a1

1 0 02a2 a1 1 0 . . . 0

nan an1 a1

ln

1 +

Mm=1

amzm

= n=1

cnzn

ln (1 + x)

k=0

1k M

m=1

amzmk =

n=1

cnzn

k=0

1

kk!

n=k

zn(a1)m1 (aM)mM

m1! mM! =

n=1

cnzn

m1+ 2m2+ + MmM=n

m1+ m2+ + mM =k

k

k

m1+ m2+ + mM

z1

cn =(m1+ m2+ + mM 1)!

m1! mM! (a1)

m1 (aM)mM

mr

n= 4


157/324

M 4 mr

m1 m2 m3 m4

mr r >4

mr r mr

r

n

n

P(n)

n

M P(n)

Mn=1

(1 xn)1

P(n)

p (n)

P(n) = p (n) nM1

i=0

p (i)

p (0)

n

M, P(n) =p (n) m= M+ 1, P(n) = p (n) 1

Mm=0

amzm = exp

k=1

ckzk

Mm=0

amzm =

n=0

1

n!

k=1

ckzkn


158/324

nth

Mm=0

amzm =

n=0

m=n

(c1)k1 (cM)kMk1! kM!

k1+ 2k2+ + nkn = m

k1+ k2+ + kn = n

n

z1

an =(c1)k1 (cn)kn

k1! kn!

kr

Sxx()

Sxx() = F {rxx()} =

rxx() ej2fd

rxx() = [x (t) x (t )]

Sxx()

x (t)

x (t)

Sxx() =

k=rxx[k]e

j2kf

rxx[k] =

x [n] x [n k]

Sxx(f) x (n)

x (n)

h (n)


159/324

y (n) =

n=

x (m) y (n m) = x (n) h (n)

x (n)

F {rxx()} = F {x (n) x (n)} X() X() = |X()|2 =Sxx()

{an}

H(z)

a1, a2, ap a1 a2

H(z)

H(z)


160/324

0 0.5 1 1.5 2

x 104

5

4

3

2

1

0

1

2

True Envelope at iteration 1

Frequency (Hz)

Amplitude(dB)

Original SpectrumSmoothed Spectrum

0 0.5 1 1.5 2

x 104

5

4

3

2

1

0

1

2

Magnitude Spectrum at iteration 1

Frequency (Hz)

Amplitude(dB)

Smoothed Magnitude Spectrum

0 0.5 1 1.5 2

x 104

5

4

3

2

1

0

1

2


Frequency (Hz)

Amplitude(dB)


0 0.5 1 1.5 2

x 104

4

3

2

1

0

1

2


Frequency (Hz)

Amplitude(dB)


0 0.5 1 1.5 2

x 104

5

4

3

2

1

0

1

2


Frequency (Hz)

Amplitude(dB)


0 0.5 1 1.5 2

x 104

4

3

2

1

0

1

2


Frequency (Hz)

Amplitude(dB)


0 0.5 1 1.5 2

x 104

5

4

3

2

1

0

1

2


Frequency (Hz)

Amplitude(dB)


0 0.5 1 1.5 2

x 104

4

3

2

1

0

1

2


Frequency (Hz)

Amplitude(dB)


0 0.5 1 1.5 2

x 104

5

4

3

2

1

0

1

2


Frequency (Hz)

Amplitude(dB)


0 0.5 1 1.5 2

x 104

4

3

2

1

0

1

2


Frequency (Hz)

Amplitude(dB)


i

log |X(k)| Ci(k) Ai(k)


161/324


162/324


163/324


164/324


165/324


166/324


167/324

AE(x) =v0+ (v1 v0) (1 (1 x)n

)1/n

v0 v1

x

tstart x = 0 tend x = 1 n n

n

n nopt


168/324

N

M

M= 3


169/324

C(t) =

Mb=1 fb(t) ab(t)M

b=1 ab(t)

C(t) fb(t) ab(t)

b

Mth


170/324


171/324


172/324

x (t)

x2 (t)

x (t)

RMS(t) =

1T

Ti=1

x2i (t)

xi(t) ith

t t

T

p = 2

x2 (t)

x (t)

t


173/324


175/324

x (t)

X()

x (t) = 1

2

X() eitd

x (t)

x (t) = 1

2

0

X() eitd

U()

U() =

U() = 0, 0

U(0) = 1/2

x (t) = 1

U() X() eitd

u ()

X()

X() x (t) u (t)

U()

x (t) = 2

x () u (t ) d

U()

u (t) =1

2(t) +

i

2

1

t

x (t) = 2

x ()

1

2(t ) +

i

2

1

(t )

d=x (t) + iH {x (t)}

H {x (t)} x (t)


176/324

H {x (t)} = 1

x ()(t )

d=x (t) 1t

x (t)

x (t) = x (t) +jH {x (t)} =E(t)exp[j (t)] =

x2 (t) + [H {x (t)}]2 exp

j arctan

H {x (t)}

x (t)

E(t)

x (t)

E(t)

(t)

x (t)

x (t)

E2 (t)

x2 (t)

E2 (t) = x2 (t) + [H {x (t)}]2 = 2x2 (t)


177/324

X(k) =N1n=0

x (n) cos

N

n +

1

2

k

, k= 0,...,N 1

X(k) = 1

2x (0) +

N1n=1

x (n) cos

Nn

k+

1

2

, k= 0,...,N 1

f0

T

f


178/324

x

T T0

O= Fs2T

=FsT0

, = 0.5

T0

= 1 |x (t)|

x (t) |x (t)|

x (t)


179/324


180/324


181/324


182/324


183/324
http://www.vsl.co.at/en/65/71/84/1349.vslhttp://www.vsl.co.at/en/65/71/84/1349.vslhttp://www.music.mcgill.ca/resources/mums/html/index.htmhttp://www.music.mcgill.ca/resources/mums/html/index.htmhttp://www.zikinf.com/news/ircam-solo-instruments-218http://www.zikinf.com/news/ircam-solo-instruments-218http://theremin.music.uiowa.edu/http://theremin.music.uiowa.edu/http://staff.aist.go.jp/m.goto/RWC-MDB/http://staff.aist.go.jp/m.goto/RWC-MDB/


184/324


185/324


186/324

0 0.5 1 1.5 2

0.5

0

0.5

Original Bass Clarinet

Time (s)

Amplitude

Time (s)

Frequency(Hz)

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

0.5

1

1.5

2

x 104

0 0.5 1 1.5 2 2.50.8

0.6

0.4

0.2

0

0.2

0.4

Original Bassoon

Time (s)

Amplitude

Time (s)

Frequency(Hz)

0.5 1 1.5 2 2.50

0.5

1

1.5

2

x 104

0 0.5 1 1.5

0.5

0

0.5

Original Clarinet

Time (s)

Amplitude

Time (s)

Frequency(Hz)

0.2 0.4 0.6 0.8 1 1.2 1.40

0.5

1

1.5

2

x 104

0 0.5 1 1.5

0.4

0.2

0

0.2

0.4

0.6

Original English Horn

Time (s)

Amplitude

Time (s)

Frequency(Hz)

0.2 0.4 0.6 0.8 1 1.2 1.40

0.5

1

1.5

2

x 104

0 0.5 1 1.5 2 2.5

0.6

0.4

0.2

0

0.2

0.4

Original Flute

Time (s)

Amplitude

Time (s)

Frequency(Hz)

0.5 1 1.5 2 2.50

0.5

1

1.5

2

x 104

0 0.2 0.4 0.6 0.8 1 1.2 1.40.5

0

0.5

Original Oboe

Time (s)

Amplitude

Time (s)

Frequency(Hz)

0.2 0.4 0.6 0.8 1 1.2 1.40

0.5

1

1.5

2

x 104


187/324

0 0.5 1 1.5 2 2.50.4

0.2

0

0.2

0.4

0.6

Original Bass Trombone

Time (s)

Amplitude

Time (s)

Frequency(Hz)

0.5 1 1.5 2 2.50

0.5

1

1.5

2

x 104

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.60.4

0.2

0

0.2

0.4

0.6

0.8

Original Bass Trumpet

Time (s)

Amplitude

Time (s)

Frequency(Hz)

0.2 0.4 0.6 0.8 1 1.2 1.4 1.60

0.5

1

1.5

2

x 104

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6

0.5

0

0.5

Original Cimbasso

Time (s)

Amplitude

Time (s)

Frequency(Hz)

0.2 0.4 0.6 0.8 1 1.2 1.4 1.60

0.5

1

1.5

2

x 104

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8

0.5

0

0.5

Original Contrabass Tuba

Time (s)

Amplitude

Time (s)

Frequency(Hz)

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.80

0.5

1

1.5

2

x 104

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.60.5

0

0.5

Original French Horn

Time (s)

Amplitude

Time (s)

Frequency(Hz)

0.2 0.4 0.6 0.8 1 1.2 1.40

0.5

1

1.5

2

x 104

0 0.5 1 1.5 2 2.5 3 3.5

0.5

0

0.5

Original Tenor Trombone

Time (s)

Amplitude

Time (s)

Frequency(Hz)

0.5 1 1.5 2 2.5 3 3.50

0.5

1

1.5

2

x 104

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8

0.2

0

0.2

0.4

0.60.8

Original Trumpet

Time (s)

Amplitude

Time (s)

Frequency(Hz)

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.80

0.5

1

1.5

2

x 104

0 0.5 1 1.5 20.4

0.2

0

0.2

0.4

0.60.8

Original Tuba

Time (s)

Amplitude

Time (s)

Frequency(Hz)

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

0.5

1

1.5

2

x 104


188/324

0 0.5 1 1.5 2

0.4

0.2

0

0.2

0.4

Original Double Bass

Time (s)

Amplitude

Time (s)

Frequency(Hz)

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

0.5

1

1.5

2

x 104

0 0.5 1 1.5 2 2.5 3

0.5

0

0.5

Original Cello

Time (s)

Amplitude

Time (s)

Frequency(Hz)

0.5 1 1.5 2 2.5 30

0.5

1

1.5

2

x 104

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6

0.5

0

0.5

Original Viola

Time (s)

Amplitude

Time (s)

Frequency(Hz)

0.2 0.4 0.6 0.8 1 1.2 1.4 1.60

0.5

1

1.5

2

x 104

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6

0.6

0.4

0.2

0

0.2

0.4

Original Violin

Time (s)

Amplitude

Time (s)

Frequency(Hz)

0.2 0.4 0.6 0.8 1 1.2 1.4 1.60

0.5

1

1.5

2

x 104


189/324


190/324


191/324


192/324


193/324

N

1 a1 f1 12 a2 f2 2

N aN fN N

an n

th

fn n


194/324

ss(t)

ss(t) =

K(t)k=0

Ak(t)sin(2fkt + k)

Ak k

th

fk

Hs() Ak fk Hs()

K(t)k=0

Ak(t) = Hs

K(t)k=0

2fkt + k

ss(t)

ss(t) =

K(t)k=0

sk(t) Hs(2fkt + k)

sk(t) = sin(2fkt + k) fk


195/324


196/324

f11 f12

= 1200log2

fn1fn2

fn1 n

th

fn2

nth

fn1 fn2 f


197/324

fn = fn12

1200 =fn12 log2

fn2f

n1

fn nth

fn[1]

[1 ] fn fn1 fn2 n

th

N1 N2 Nth1

N1< N2 Nth2

N1 < N2

n > N1

fn1

f11 n fn1 nf11


198/324


199/324

0 1000 2000 3000 4000 50000

20

40

60

80

Original Bassoon Spectral Representation

Frequency (Hz)

Amplitude

spectrum

partials

Frequency (Hz)

Amplitude

0 1000 2000 3000 4000 5000

0

20

40

60

80 spectral envelope

partials

0 1000 2000 3000 4000 50000

20

40

60

Original Trumpet Spectral Representation

Frequency (Hz)

Amplitude

spectrum

partials

Frequency (Hz)

Amplitude

0 1000 2000 3000 4000 5000

0

20

40

60

80

spectral envelope

partials

0 1000 2000 3000 4000 50000

50

100

Original Viola Spectral Representation

Frequency (Hz)

Amplitude

spectrum

partials

Frequency (Hz)

Amplitude

0 1000 2000 3000 4000 5000

0

50

100


partials

0 1000 2000 3000 4000 50000

50

100

Original Bass Trumpet Spectral Representatio

Frequency (Hz)

Amp

litude

spectrum

partials

Frequency (Hz)

Amplitude

0 1000 2000 3000 4000 5000

0

100


partials

0 1000 2000 3000 4000 50000

20

40

60

80

Original Cello Spectral Representation

Frequency (Hz)

Amp

litude

spectrum

partials

Frequency (Hz)

Amplitude

0 1000 2000 3000 4000 5000

0

20

40

60


partials

0 1000 2000 3000 4000 50000

10

20

riginal Bass Trombone Spectral Representati

Frequency (Hz)

Amp

litude

spectrum

partials

Frequency (Hz)

Amplitude

0 1000 2000 3000 4000 5000

0

10


partials

0 1000 2000 3000 4000 50000

20

40

60

Original Cimbasso Spectral Representation

Frequency (Hz)

Amplitude

spectrum

partials

Frequency (Hz)

Amplitude

0 1000 2000 3000 4000 5000

0

20

40

60spectral envelope

partials

0 1000 2000 3000 4000 50000

50

100

Original Viola Spectral Representation

Frequency (Hz)

Amplitude

spectrum

partials

Frequency (Hz)

Amplitude

0 1000 2000 3000 4000 5000

0

50

100


partials

0 1000 2000 3000 4000 50000

20

40

60

80

Original Double Bass Spectral Representation

Frequency (Hz)

Amplitude

spectrum

partials

Frequency (Hz)

Amplitude

0 1000 2000 3000 4000 5000

0

20

40

60

80

spectral envelope

partials

0 1000 2000 3000 4000 50000

20

40

60

80

Original Oboe Spectral Representation

Frequency (Hz)

Amplitude

spectrum

partials

Frequency (Hz)

Amplitude

0 1000 2000 3000 4000 5000

0

20

40

60

80spectral envelope

partials

0 1000 2000 3000 4000 50000

20

40

Original French Horn Spectral Representation

Frequency (Hz)

Amplitude

spectrum

partials

Frequency (Hz)

Amplitude

0 1000 2000 3000 4000 5000

0

20

40

60

spectral envelope

partials

0 1000 2000 3000 4000 50000

20

40

60

80

Original Tuba Spectral Representation

Frequency (Hz)

Amplitude

spectrum

partials

Frequency (Hz)

Amplitude

0 1000 2000 3000 4000 5000

0

20

40

60

80spectral envelope

partials

0 1000 2000 3000 4000 50000

20

40

riginal Tenor Trombone Spectral Representati

Frequency (Hz)

Amplitude

spectrum

partials

Frequency (Hz)

Amplitude

0 1000 2000 3000 4000 5000

0

20


partials

0 1000 2000 3000 4000 50000

20

40

60

Original Violin Spectral Representation

Frequency (Hz)

Amplitude

spectrum

partials

Frequency (Hz)

Amplitude

0 1000 2000 3000 4000 5000

0

20

40

60

80

spectral envelope

partials

0 1000 2000 3000 4000 50000

20

40

60

riginal Contrabass Tuba Spectral Representati

Frequency (Hz)

Amplitude

spectrum

partials

Frequency (Hz)

Amplitude

0 1000 2000 3000 4000 5000

0

20

40

60spectral envelope

partials


200/324

0 0.5 1 1.5 2 2.50.8

0.6

0.4

0.2

0

0.2

0.4

Original Bassoon Sinusoidal Component

Time (s)

Amplitude

Time (s)

Frequency(Hz)

0.5 1 1.5 2 2.50

0.5

1

1.5

2

x 104

0 0.5 1 1.5 2 2.50

2000

4000

Original Bassoon SourceFilter Representatio

Time (s)

Frequency(H

z)

Time (s)

Frequency(Hz)

0 0.5 1 1.5 2 2.50

2000

4000

0 0.5 1 1.5 2 2.50.4

0.2

0

0.2

0.4

0.6

Original Bass Trombone Sinusoidal Compone

Time (s)

Amp

litude

Time (s)

Frequency(Hz)

0.5 1 1.5 2 2.50

0.5

1

1.5

2

x 104

0 0.5 1 1.5 2 2.50

2000

4000

6000

iginal Bass Trombone SourceFilter Represent

Time (s)

Freque

ncy(Hz)

Time (s)

Frequency(Hz)

0 0.5 1 1.5 2 2.50

2000

4000

6000

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6

0.5

0

0.5

Original Viola Sinusoidal Component

Time (s)

Amplitude

Time (s)

Frequency(Hz)

0.2 0.4 0.6 0.8 1 1.2 1.4 1.60

0.5

1

1.5

2

x 104

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.60

5000

10000

Original Viola SourceFilter Representation

Time (s)

F

requency(Hz)

Time (s)

Frequency(Hz)

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.60

5000

10000

0 0.5 1 1.5 2 2.5

0.6

0.4

0.2

0

0.2

0.4

Original Flute Sinusoidal Component

Time (s)

Amplitude

Time (s)

Frequency(Hz)

0.5 1 1.5 2 2.50

0.5

1

1.5

2

x 104

0 0.5 1 1.5 2 2.50

5000

10000

Original Flute SourceFilter Representation

Time (s)

Frequency(Hz)

Time (s)

Frequency(Hz)

0 0.5 1 1.5 2 2.50

5000

10000


201/324

0 0.5 1 1.5 2

1

0.5

0

0.5

1

1.5

Original Bass Clarinet Spectral Shape Feature

Time (s)

NormalizedDescriptorValues

waveform

centroid

spread

skewness

kurtosis

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6

0.5

0

0.5

1

1.5

Original Bass Trumpet Spectral Shape Feature

Time (s)


waveform

centroid

spread

skewness

kurtosis

0 0.5 1 1.5 2 2.5

0.5

0

0.5

1

1.5

Original Bassoon Spectral Shape Features

Time (s)


waveform

centroid

spread

skewness

Documents

Morphing Musical Instrument Sounds