Click here to load reader
View
33
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Natural Language Processing. Spring 2007 V. “Juggy” Jagannathan. Course Book. Foundations of Statistical Natural Language Processing. By Christopher Manning & Hinrich Schutze. Chapter 9. Markov Models March 5, 2007. Markov models. Markov assumption - PowerPoint PPT Presentation
Natural Language ProcessingSpring 2007V. Juggy Jagannathan
Foundations of Statistical Natural Language ProcessingByChristopher Manning & Hinrich SchutzeCourse Book
Chapter 9Markov ModelsMarch 5, 2007
Markov modelsMarkov assumptionSuppose X = (X1, , XT) is a sequence of random variables taking values in some finite set S = {s1,,sN}, Markov properties are:Limited HorizonP(Xt+1 = sk|X1,,Xt) = P(Xt+1 = sk|Xt)i.e. the t+1 value only depends on t valueTime invariant (stationary)Stochastic Transition matrix A:aij = P(Xt+1 = sj|Xt=si) where
Markov model example
Probability: {lem,ice-t} giventhe machine starts in CP?
0.3x0.7x0.1+0.3x0.3x0.7=0.021+0.063 = 0.084Hidden Markov Model Example
Why use HMMs?Underlying events generating surface observable eventsEg. Predicting weather based on dampness of seaweedshttp://www.comp.leeds.ac.uk/roger/HiddenMarkovModels/html_dev/main.htmlLinear Interpolation in n-gram models:
Look at Notes from David Meir Blei [UC Berkley]http://www-nlp.stanford.edu/fsnlp/hmm-chap/blei-hmm-ch9.pptSlides 1-13
(Observed states)
Forward Procedure
Initialization:Induction:Total computation:Forward Procedure
Initialization:Induction:Total computation:Backward Procedure
Combining both forward and backward
Finding the best state sequenceTo determine the state sequence that best explains observationsLet:Individually the most likely state is:This approach, however, does not correctly estimate the most likely state sequence.
Finding the best state sequenceViterbi algorithmStore the most probable path that leads to a given nodeInitializationInductionStore Backtrace
Parameter Estimation
Parameter EstimationProbability of traversing an arc at time t given observation sequence O:
Parameter Estimation