-
8/3/2019 Lecture01_StochasticProcesses
1/3
1
MS 302Stochastic Models inOperations Research
Stochastic Processes
September 26, 2011
2
Stochastic Processes
In this course, we will primarily studyprocesses that evolve
over time in a
probabilisticmanner. Stock price of a firm over time
Market share of a firm over time
Inventory level of a product over time
Weather over time
3
Stochastic Processes
Two important types of such processes are
Markov chains
Queueing models
Let Xtbe the value of a systemcharacteristic at time t. If Xt is
not knownwith certainty before time t, then Xtcan berepresented as
a random variable r.v.
The random variable Xt is referred to as the
state of the systemat time t.4
Discrete TimeStochastic Processes
If we observe Xtat discrete points in time,then the collection
of random variables {X0,X1, X2,..} is called a discrete
timestochasticprocess.
Stock price of a firm over time
If the stock price is observed at the start of eachtrading
session, then we have a discrete timestochastic process.
Weather
If the weather temperature is measured each morning,then we have
a discrete time stochastic process.
5
Continuous TimeStochastic Processes
If we may observe Xtat any point in time,then {Xt, t 0} is
called a continuous time
stochastic process.Stock price of a firm over time
If the stock price is observed continuouslythroughout trading,
then we have acontinuous time stochastic process.
Weather If the weather temperature is measured
continuously, then we have a continuoustime stochastic process.
6
State Space
Finite state space If Xtmay only assume a finite number of
values, then the stochastic process is said to
have a finite state space. Inventory level of a product assuming
that there is
a storage capacity.
Infinite state space If Xtmay take an infinite number of
different
values, then the stochastic process is said tohave an infinite
state space. Stock prices.
Weather temperature.
-
8/3/2019 Lecture01_StochasticProcesses
2/3
7
Focus of the Course
We will primarily discuss discrete time, finitestate spaceMarkov
chains.
In queuing theory, we will also considercontinuous time,
infinite state space
stochastic processes.
We are interested in the mathematicalmodeling of stochastic
processes whichrequires us to describe the relationshipsbetween the
r.v.s X0, X1, X2, or {Xt, t0}.
8
A Weather Example
Weather in Orhanli changes quickly fromday to day.
If the weather is dry today, then it will bedry tomorrow with
probability 0.8.
If it is raining today, then the weather willbe dry tomorrow
with probability 0.6.
Starting on some initial day (labeled aszero), the weather is
observed on eachday t=0,1,
9
A Weather Example
Discrete or continuous time?
Discrete
The state of the system
State 0 = day is dry
State 1 = it is raining
State space
Finite
10
A Weather Example
Thus, for t=0,1,2,, we have
The stochastic process {X0, X1, X2, }describes how the weather
in Orhanlichanges over time.
11
Gamblers Ruin Problem
At t= 0, we have $2.
At times t= 1, 2, we play a game in
which we bet $1.
With probability pwe win the game, andwith probability 1-pwe
lose the game.
We quit the game if our capital isincreased to $4 or reduced to
$0.
12
Gamblers Ruin Problem
What kind of a stochastic process?
Discrete time, finite state space.
What if we keep playing unless we lose allof our capital?
Discrete time, infinite state space.
-
8/3/2019 Lecture01_StochasticProcesses
3/3
13
Gamblers Ruin Problem
State of the system
Xt= capital on-hand at time t.
What is known about the state of the system?
X0 = 2 is a known constantXt is a r.v. for all t 1.
There is a probabilistic relationship between XtandXt+1 t 0. For
instance,
P(X1=1 /X0=2) = 1- p
P(X1=3 /X0=2) = p
P(Xt+1=1 /Xt= 4) = 0
The stochastic process {X0, X1, X2, } describeshow our capital
changes over time.
14
An Inventory Example
A camera store stocks a particular model to beordered
weekly.
Let Dtbe the demand in week t. Note that Dtincludes lost sales
if there is no camera in stock.
Inventory management policy
Each Saturday the owner places an order of 3 units ifno camera
is left in stock. Otherwise, no order isplaced. Orders arrive on
Monday morning before thestore is opened.
We are interested in modeling the stock level ofthe store over
time.
15
An Inventory Example
What kind of a stochastic process?
Discrete time, finite state space.
State of the system
Xt= inventory on-hand at the end of weekt=0,1,
What are the possible states?
0, 1, 2, 3 cameras on hand.
16
An Inventory Example
What is the relationship between Xt, Xt+1and Dt+1?
The stochastic process {X0, X1, X2, }describes how the inventory
level of thestore changes over time.
