Upload
derek-bruff
View
1.103
Download
4
Embed Size (px)
DESCRIPTION
An introduction to Markov chains via Monopoly
Citation preview
How to Win at Monopoly: Markov Chains for Fun and Profit
Derek Bruff, PhDDirector, Center for TeachingSenior Lecturer, Mathematics
Which properties are landed on most
often?
Which properties are landed on most
frequently?
Which properties are most profitable to own?
Monopoly Rules
• 40 spaces (Go through Boardwalk)• Roll two six-sided dice to move.• “Go to Jail” sends you to Jail.• Rolling three doubles in a row sends you to Jail.• Get out of jail by…– Paying $50,– Using a “Get out of Jail, Free” card,– Rolling doubles, or– Spending three turns in Jail.
• Chance and Community Chest cards have various effects.
Monopoly Rules
• 40 spaces (Go through Boardwalk)• Roll two six-sided dice to move.• “Go to Jail” sends you to Jail.• Rolling three doubles in a row sends you to Jail.• Get out of jail by…– Paying $50,– Using a “Get out of Jail, Free” card,– Rolling doubles, or– Spending three turns in Jail.
• Chance and Community Chest cards have various effects.
Monopoly Rules
• 40 spaces (Go through Boardwalk)• Roll two six-sided dice to move.• “Go to Jail” sends you to Jail.• Rolling three doubles in a row sends you to Jail.• Get out of jail by…– Paying $50,– Using a “Get out of Jail, Free” card,– Rolling doubles, or– Spending three turns in Jail.
• Chance and Community Chest cards have various effects.
Monopoly Rules
• 40 spaces (Go through Boardwalk)• Roll two six-sided dice to move.• “Go to Jail” sends you to Jail.• Rolling three doubles in a row sends you to Jail.• Get out of jail by…– Paying $50,– Using a “Get out of Jail, Free” card,– Rolling doubles, or– Spending three turns in Jail.
• Chance and Community Chest cards have various effects.
Monopoly Rules
• 40 spaces (Go through Boardwalk)• Roll two six-sided dice to move.• “Go to Jail” sends you to Jail.• Rolling three doubles in a row sends you to Jail.• Get out of jail by…– Paying $50,– Using a “Get out of Jail, Free” card,– Rolling doubles, or– Spending three turns in Jail.
• Chance and Community Chest cards have various effects.
Monopoly Rules
• 40 spaces (Go through Boardwalk)• Roll two six-sided dice to move.• “Go to Jail” sends you to Jail.• Rolling three doubles in a row sends you to Jail.• Get out of jail by…– Paying $50,– Using a “Get out of Jail, Free” card,– Rolling doubles, or– Spending three turns in Jail.
• Chance and Community Chest cards have various effects.
Monopoly Rules
• 40 spaces (Go through Boardwalk)• Roll two six-sided dice to move.• “Go to Jail” sends you to Jail.• Rolling three doubles in a row sends you to Jail.• Get out of jail by…– Paying $50,– Using a “Get out of Jail, Free” card,– Rolling doubles, or– Spending three turns in Jail.
• Chance and Community Chest cards have various effects.
Monopoly Rules
• 40 4 spaces (Go through Boardwalk)• Roll two six-sided dice Flip a coin to move.• “Go to Jail” sends you to Jail.• Rolling three doubles in a row sends you to Jail.• Get out of jail by…– Paying $50,– Using a “Get out of Jail, Free” card,– Rolling doubles, or– Spending three turns in Jail.
• Chance and Community Chest cards have various effects.
Suppose we only have four spaces (A, B, C, and D) and that a move consists of flipping a coin.• Heads = Move two
spaces• Tails = Move one space
TERMINALLY BORING
Monopoly: Terminally Boring Edition
x0 = x1 =
What is x2?
Monopoly: Terminally Boring Edition
x0 = x1 =
x2 =
If x2=Px1, then what is P?
If x2=Px1, then what is P?
=
x2 x1P
Monopoly: Terminally Boring Edition
Model: xk+1=Pxk
P =
Monopoly: Terminally Boring Edition
Model: xk+1=Pxk
x0 = (1, 0, 0, 0)
Monopoly: Terminally Boring Edition
Model: xk+1=Pxk
x0 = (1, 0, 0, 0)
x1 = (0, .5, .5, 0)
Monopoly: Terminally Boring Edition
Model: xk+1=Pxk
x0 = (1, 0, 0, 0)
x1 = (0, .5, .5, 0)
x2 = (.25, 0, .25, .5)
Monopoly: Terminally Boring Edition
Model: xk+1=Pxk
x0 = (1, 0, 0, 0)
x1 = (0, .5, .5, 0)
x2 = (.25, 0, .25, .5)
x3 = (.375, .375, .125, .125)
Monopoly: Terminally Boring Edition
Model: xk+1=Pxk
Monopoly: Terminally Boring Edition
• 40 4 spaces (Go through Boardwalk)• Roll two six-sided dice Flip a coin to move.• “Go to Jail” sends you to Jail.• Rolling three doubles in a row sends you to Jail.• Get out of jail by…– Paying $50,– Using a “Get out of Jail, Free” card,– Rolling doubles, or– Spending three turns in Jail.
• Chance and Community Chest cards have various effects.
Monopoly: Simple Model
• 40 spaces (Go through Boardwalk)• Roll two six-sided dice to move.• “Go to Jail” sends you to Jail.• Rolling three doubles in a row sends you to Jail.• Get out of jail by…– Paying $50,– Using a “Get out of Jail, Free” card,– Rolling doubles, or– Spending three turns in Jail.
• Chance and Community Chest cards have various effects.
Rolling Two Six-Sided Dice
Spaces Moved Probability
2 1/363 2/364 3/365 4/366 5/367 6/368 5/369 4/36
10 3/3611 2/3612 1/36
P =
Monopoly: Simple Model
Monopoly: Simple Model
Monopoly: Simple Model
Monopoly: Simple Model
Monopoly: Simple Model
Monopoly: Simple Model
Monopoly: Simple Model
Monopoly: Simple Model
Monopoly: Simple Model
Monopoly: Simple Model
Monopoly: Simple Model
Monopoly: Simple Model
Monopoly: Simple Model
Markov Chains
Definition: A vector with the property that the sum of its entries is 1 is called a probability vector.
Definition: A square matrix with the property that the sum of the entries in each of its columns is 1 is called a stochastic matrix.
Andrey Markov, 1856 – 1922
Markov Chains
Definition: A Markov chain is a dynamical system for which
• the probability vector xk describes the state of the system at time k and
• successive state vectors are related by the following equation, where P is a stochastic matrix called the transition matrix for the system.
xk+1=Pxk
Markov Chains
Theorem: If P is the transition matrix for a Markov chain (and P is regular), then…
• There is a unique probability vector q such that Pq=q.
• For any initial state vector x0, xk q as k
Finding q means solving the equation
Pq=q
Monopoly: Simple Model
Finding q means solving the equation
Pq=q
Monopoly: Model #2
• 40 spaces (Go through Boardwalk)• Roll two six-sided dice to move.• “Go to Jail” sends you to Jail.• Rolling three doubles in a row sends you to Jail.• Get out of jail by…– Paying $50,– Using a “Get out of Jail, Free” card,– Rolling doubles, or– Spending three turns in Jail.
• Chance and Community Chest cards have various effects.
P =
Monopoly: Model #2
Finding q means solving the equation
Pq=q
Monopoly: Model #3
• 40 spaces (Go through Boardwalk)• Roll two six-sided dice to move.• “Go to Jail” sends you to Jail.• Rolling three doubles in a row sends you to Jail.• Get out of jail by…– Paying $50,– Using a “Get out of Jail, Free” card,– Rolling doubles, or– Spending three turns in Jail.
• Chance and Community Chest cards have various effects.
Monopoly: Model #3
Finding q means solving the equation
Pq=q
Monopoly: Model #4
• 40 spaces (Go through Boardwalk)• Roll two six-sided dice to move.• “Go to Jail” sends you to Jail.• Rolling three doubles in a row sends you to Jail.• Get out of jail by…– Paying $50,– Using a “Get out of Jail, Free” card,– Rolling doubles, or– Spending three turns in Jail.
• Chance and Community Chest cards have various effects.
Monopoly: Model #4
Finding q means solving the equation
Pq=q
What’s Left?
• Rolling three doubles in a row sends you to Jail.
• Chance and Community Chest cards have various effects.
You still have two underlying models—leave jail quickly or stay as long as you can.
Probabilities by Truman Collins
Short Jail Stay
Probabilities by Truman Collins
Short Jail Stay
Probabilities by Truman Collins
Short Jail Stay
Probabilities by Truman Collins
Short Jail Stay
Probabilities by Truman Collins
Long Jail Stay
Probabilities by Truman Collins
Long Jail Stay
Probabilities by Truman Collins
Long Jail Stay
Probabilities by Truman Collins
Long Jail Stay
OTHER APPLICATIONS OF MARKOV CHAINS
RISK—and other board games
Baseball, tennis, jai alai,…
Migration Models
Google’s PageRank Algorithm
Flickr Credits• “monopoly,” foreverdigital• “Black Dice,” Mariano Kamp• “Monopoly,” unloveablesteve• “last man standing,” Robert Terrell• “Racing for Home,” Scott Ableman• “Nomads (brog pa) crossing Lha chu at Kailash
Kora,” reurinkjan