Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
stand B-7
Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music GenerationHao-Wen Dong and Yi-Hsuan YangResearch Center of IT Innovation, Academia Sinica
Gz
noise
True/Fake
Synth Pad
x1 or x1
x2 or x2
xM or xM
Drums
Piano
…
~
~
~
D
Generator DiscriminatorDrums
Piano
Guitar
Bass
Ensemble
Reed
Synth Lead
Synth Pad
Target outputs - Multi-track piano-rolls Convolutional Generative Adversarial Networks
CNNs + GANsmulti-trackpolyphonic
stand B-7Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generation Hao-Wen Dong and Yi-Hsuan Yang
Drums Piano Guitar Bass Strings
stand B-7Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generation Hao-Wen Dong and Yi-Hsuan Yang
bass linechords
drum patterns
Drums Piano Guitar Bass Strings
stand B-7Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generation Hao-Wen Dong and Yi-Hsuan Yang
Gz
noise
D True/Fake
Synth Pad
x1 or x1
x2 or x2
xM or xM
Drums
Piano
…
~
~
~
Generator Discriminator
real-valued
stand B-7Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generation Hao-Wen Dong and Yi-Hsuan Yang
Generator Refiner Discriminator
Gz
noise
x1
x2
xM
…
R
R
R
1
2
M
…
ˆ
ˆ
ˆ
D True/Fake
Synth Pad
x1 or x1
x2 or x2
xM or xM
Drums
Piano
…~
~
~
real-valued binary-valued
stand B-7Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generation Hao-Wen Dong and Yi-Hsuan Yang
Generator Refiner Discriminator
Gz
noise
x1
x2
xM
…
R
R
R
1
2
M
…
ˆ
ˆ
ˆ
D True/Fake
Synth Pad
x1 or x1
x2 or x2
xM or xM
Drums
Piano
…
~
~
~
real-valued binary-valued
Generator’s outputs Real data
MuseGAN real-valued binary-valued
BinaryMuseGAN binary-valued binary-valued
stand B-7Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generation Hao-Wen Dong and Yi-Hsuan Yang
Generator Refiner Discriminator
Gz
noise
x1
x2
xM
…
R
R
R
1
2
M
…
ˆ
ˆ
ˆ
D True/Fake
Synth Pad
x1 or x1
x2 or x2
xM or xM
Drums
Piano
…
~
~
~
Deterministic Binary Neurons (DBNs)𝐷𝐷𝐷𝐷𝐷𝐷 𝑥𝑥 = 𝑢𝑢 𝜎𝜎 𝑥𝑥 − 0.5
Stochastic Binary Neurons (SBNs)𝑆𝑆𝐷𝐷𝐷𝐷 𝑥𝑥 = 𝑢𝑢 𝜎𝜎 𝑥𝑥 − 𝑣𝑣 , 𝑣𝑣~𝑈𝑈 0, 1
stand B-7Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generation Hao-Wen Dong and Yi-Hsuan Yang
shared/privatedesign
shared/private design
Generator Refiner Discriminator
onset/offset stream
chroma stream
Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generation Hao-Wen Dong and Yi-Hsuan Yang stand B-7
MuseGAN’s output
(real-valued)
hardthresholding
Bernoullisampling
many overly-fragmented notes
Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generation Hao-Wen Dong and Yi-Hsuan Yang stand B-7
BinaryMuseGAN(+DBNs)MuseGAN’s output
(real-valued)
hardthresholding
Bernoullisampling
many overly-fragmented notes
less overly-fragmented notes
stand B-7Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generation Hao-Wen Dong and Yi-Hsuan Yang
MuseGAN BinaryMuseGAN
Come to learn more and listen to the demos!
(different training strategies)