Normalizing Flow ModelsPart 2

Dmytro Urukov24 Mar 2020

Recap: Normalizing Flows

Recap: Change of variables rule

Recap: triangular matrix

Invertible transformations

NICE: Non-Linear Independent Components Estimation

NICE: Non-Linear Independent Components Estimation (Dinh et al., 2014)

NICE: Additive coupling layers

NICE: Additive coupling layers

Volume preserving transformation

NICE: Scaling layers

NICE: Scaling layers

NICE: Scaling layers

NICE: Generated samples

Real-NVP: Non-volume preserving extension of NICE

Density estimation using Real NVP (Dinh et al., 2017)

Real-NVP: Examples

http://www.youtube.com/watch?v=E66_EB79BKQ&t=9

Autoregressive models as flow models

Masked Autoregressive Flow (MAF)

Masked Autoregressive Flow for Density Estimation (Papamakarios et al., 2017)

Masked Autoregressive Flow (MAF)

Inverse Autoregressive Flow (IAF)

Improving Variational Inference with Inverse Autoregressive Flow (Kingma et al., 2017)

IAF is inverse of MAF

Parallel WaveNet

Parallel WaveNet

References● Normalizing Flow Models

● Primer on Normalizing Flows

● What are normalizing flows?

● CS 480/680 Lecture 23: “Normalizing Flows”

● Flow-based Deep Generative Models

Thank you!