Lecture 6: Hidden Variables and Expectation-Maximization

Machine  Learning  for  Language  Technology    Lecture  6:  Hidden  Variables  and  Expecta6on  Maximiza6on  (EM)  

Marina  San6ni  Department  of  Linguis6cs  and  Philology  Uppsala  University,  Uppsala,  Sweden  

 Autumn  2014  

 Acknowledgement:  Thanks  to  Prof.  Joakim  Nivre  for  course  design  and  materials  

Repe66on:  Baysian  Approach  to  Classifica6on  

MLE  

Hidden  and  Latent  Variables  

Expecta6on-­‐Maximaza6on  (1)  

Expecta6on-­‐Maximaza6on  (2)  

Expecta6on-­‐Maximiza6on  (3)  

Expecta6on-­‐Maximiza6on  (4)  

Naive  Bayes  Revised    

•  Equa6ons  are  ok  

EM  for  Naive  Bayes  

A  Simple  Example  

Supervised  Learning  

Unsupervised  Learning  

First  Guess  

First  E-­‐Step  

First  M-­‐Step  

Second  E-­‐Step  

Second  M-­‐Step  

Third  E-­‐Step  

Third  M-­‐Step  

Fourth  E-­‐Step  

Fourth  M-­‐Step  -­‐  Convergence  

Uses  of  EM  

The  end