Embed Size (px)
Citation preview
Bayesian Inference and Latent Variable Models in
Machine LearningDmitry P. Vetrov
Head of Bayesian methods research group
http://bayesgroup.ru,
Faculty of Computer Science, HSE
Skoltech
Outline
Today
• Probabilistic modeling in Machine Learning
• Exponential class of distributions
• Learning with latent variables
• EM-algorithm
Next time
• Examples of models with discrete and continuous latent variables
• Extensions of EM-algorithm
• Stochastic optimization in EM framework
What is machine learning?
Simple example
Conditional and marginal distributions
Bayesian Framework
Frequentist vs. Bayesian frameworks
Bayesian Learning and Inference
Combining models
Maximal a posteriori (MAP) learning
Exponential class of distributions
Log-concavity of exponential class
Log-concavity of exponential class
Example: Gaussian distribution
Incomplete likelihood
Variational lower bound
EM-algorithm
EM-algorithm
EM-algorithm
EM-algorithm
EM-algorithm
EM-algorithm
EM-algorithm
Discrete T
Mixture of gaussians
Mixture of gaussians
Mixture of gaussians
Mixture of gaussians
Mixture of gaussians
Mixture of gaussians
Mixture of gaussians: formal description
EM-algorithm for mixture of gaussians
Continuous T
Example: PCA model
Advantages of EM PCA
Mixture of PCA
Example: Latent Dirichlet Allocation
LDA: formal description
General nature of EM-framework
Extending E-step
Examples of conjugate distributions
Crisp E-step
Variational E-step
Stochastic optimization
Stochastic EM
Summary: extensions of basic EM
Conclusion
Challenge
For those who’s interested
• Help Nick Carter to find the criminal who kidnapped lady Thun’s dog http://cmp.felk.cvut.cz/cmp/courses/recognition/Labs/em/index_en.html
