Embed Size (px)
Citation preview
Introduction
Transition Matrix : A matrix in which all entries are nonnegative and all columns sum to 1.Thus, an n x n Transition matrix gives the probability of movement from any one of n states to any of the other n states.
Probability Vector : A vector whose coordinates are nonnegative and sum to 1. Thus, an n-dimensional probability vector gives the chances of occurrence of each one of n events.
Markov Chain If p0 is a probability vector and A is a transition matrix, then the sequence, p0,
p1, p2, p3, ... , where pk = (A^k)*p0 for k = 1, 2, 3, ..., is called a Markov chain.
If A*p = p for some probability vector, then p is called a steady-state vector.
A Stochastic model that describes the probabilities of transition among the states of a system.
Change of state depends probabilistically only on the current state of the system.
Independent of the past given that the present state is known.
The behavior depends on the structure of the transition matrix P.
Problem of Television Viewers Suppose there are two regional news shows in the local
television viewing area, and we have conducted a survey of viewers to determine which channel the viewers have been watching. The first survey revealed that 40% of the viewers watched station X and 60% watched station Y. Subsequent surveys revealed that each week 15% of the X viewers switched to station Y and 5% of the Y viewers switched to station X.
We will use Transition matrices and Markov chains to make a prediction about the future television market from this information.
We assume that the 15%-5% switching trends will continue indefinitely.
Let pk be a two-dimensional column vector whose entries give the proportion of people who watch station X and the proportion who watch station Y, in that order, during week k. Thus, p0 = [0.4,0.6]t.
From the assumptions above, we see that the proportion of people watching station X in week k = 1 will be 85% of the X viewers in week k = 0 (which is 85% of 40%, or 34%) plus 5% of the Y viewers in week k = 0 (which is 5% of 60%, or 3%) for a total of 37%:
0.85(0.4) + 0.05(0.6) = 0.37.
Similarly, 0.15(0.4) + 0.95(0.6) = 0.63,
is the proportion of people watching station Y in week k = 1. So, we have computed p1 = [0.37,0.63]t.
Similarly, we could compute the sequence of vectors p2, p3, ... , but this process would be quite tedious if we continued in the manner above.
Andrei Markov (1856-1922) described a much more efficient way of handling such a problem.
First, we think of the movement of viewers as being described by the following array, which gives the weekly proportion of viewers who change from one station to another:
From:
The matrix A of entries in the table is called the transition matrix for this problem.
Looking carefully at the definition of matrix multiplication, we see that p1=A*p0. Indeed, we see that, for k = 0, 1, 2, ..., and so on, pk+1=A*pk.
Each pk is called a probability vector, and the sequence, p0, p1, p2, p3, ..., is called a Markov chain.
Because matrix multiplication is associative, we find
p2 = A*p1 = A(A*p0) = (A^2)*p0,
p3 = A*p2 = A((A^2)*p0) = (A^3)*p0,
and, in general, pk = (A^k)*p0, for k = 1, 2, ...
Compute A^5. Then compute p5 directly from A^5 without computing p1, ... , p4.
Suppose p is a probability vector with the property that A*p = p.If this p describes the current viewers .
We can solve the equation A*p = p by rewriting it as A*p - p = 0 and factoring out p. We get the matrix equation (A - I)p = 0, where I is the 2x2 identity matrix. Using our given transition matrix, solve this matrix equation for p.
From the solutions pick the one which is a probability vector (i.e., its entries must sum to 1).
This vector is called the steady-state vector.
The fact that p is the steady-state vector does not imply that the Markov chain will actually reach that state from p0.
References
• Statistics & Probability by Sheldon M Ross• Linear Algebra by David C Lay
