Embed Size (px)
Citation preview
1
2
No lecture on Wed February 8th
Thursday 9th Feb14:15 - 17:00
3
The Maximum Principle:
. .s t
y = Q y t , z t ,t
G y t , z t ,t 0
max t
T
0ty ,zF y t ,z t ,t dt
. .
For each : maximizes
the Hamiltonian
s t
t z t
H y t , z t ,π t ,t
G y t , z t
1
,t
.
0
*y2 π = -H y. t ,π t ,t
*π y t = H3. y t , t z t ,
max s.t. *
zH y t ,π t ,t = H y t , z,π t ,t G y t , z,t 0
A Reminde
r
4
Example
k = F k - δk - c
F 0 = 0, F' 0 = , F'' k < 0, F' = 0
k
F(k)
capital
consumption
production function
depreciation rate
5
k t ,c t
Example
k = F k - δk - c
max -ρt
0c eU
U' > 0,U'' < 0
The Hamiltonian :
-ρtH = U c e + π F k - δk - c
k t ,c t
Choose to maximize
-ρt
U' c e
c H :
= π
H
π = - = -π F' k δk
Solve for c
Two differential equations in k,π
6
Example
k = F k - δk - c
-ρtU' c e = π
π = -π F' k δ
Another way:
differentiate
-ρtπ = U'' c c - ρU' c e
-π F' k δπ =
-ρtU'' c c - ρU' c= eπ
-ρtU'' c c - ρU' c e -π F' k δ
-ρt -ρtU'' c c - ρU' c e -U' c e F' k δ
-ρtU'' c c - ρU' c e F' k δπ-
-ρtU' c e = π
7
Example k = F k - δk - c
-ρtU' c e = π
π = -π F' k δ
Another way:
-ρt -ρtU'' c c - ρU' c e -U' c e F' k δ
U'' c- c F' k δ - ρ
U' c
X X
U'' c c- F' k δ - ρ
c
U c' c
elasticity of the marginal utility
U'' c c -
U' c
U' c
= η c > 0
F' k - δ - ρc
c η c
8
Example
k = F k - δk - c Another way:
F' k - δ - ρc
c η c
Given , find the direction in which it
moves with time
k,c
k
c
No t !!!!!!
9
Example k = F k - δk - c Another way:
F' k - δ - ρc
c η c
k
c
> 0 ????
F k - δk > c
F k - δk = c F' k' = δ
k’
c = F k - δk
10
Example k = F k - δk - c Another way:
F' k - δ - ρc
c η c
k
c
> 0 ????
k’
F' k' = δ
F' k > δ + ρ
F' k* = δ + ρ > δ = F k'k* < k'
k*
11
Example k = F k - δk - c Another way:
F' k - δ - ρc
c η c
k
c
k*
12
Example k = F k - δk - c Another way:
F' k - δ - ρc
c η c
k
c
k*
13
Example k = F k - δk - c Another way:
F' k - δ - ρc
c η c
k
c
k*
14
Example k = F k - δk - c Another way:
F' k - δ - ρc
c η c
k
c
k*
15
Example k = F k - δk - c Another way:
F' k - δ - ρc
c η c
k
c
k*
Stationary point
= 0
= 0
16
Example k = F k - δk - c Another way:
F' k - δ - ρc
c η c
k
c
k*
17
Example k = F k - δk - c Another way:
F' k - δ - ρc
c η c
k
c
k*
18
Example k = F k - δk - c
F' k - δ - ρc
c η c
k
c
k*
k(0),c(0) ????k(0), is given
k(0)
c(0), is chosen
c → 0
k → 0
19
Example k = F k - δk - c
F' k - δ - ρc
c η c
k
c
k*
k(0),c(0) ????k(0), is given
k(0)
c(0), is chosen
c → 0
k → 0
20
Richard E. Bellman
1920-1983
Another approach to dynamic programming
. .s t
t+1 t t t
t t
y - y = Q y ,z ,t
G y ,z ,t 0 max
T
t tt =0t t
y ,zF y ,z ,t
21
Another approach to dynamic programming
. .s t
t+1 t t t
t t
y - y = Q y ,z ,t
G y ,z ,t 0 max
T
t tt =0t t
y ,zF y ,z ,t
For a given time τ < Tdefine the problem:
. .s t
t+1 t t t
t t
y - y = Q y ,z ,t
G y ,z ,t 0 max
T
t tt
t=ty ,z
F y ,z ,tτ
maxT
t tt t=ty ,z τ
F y ,z ,t
Denote the solution by : τV y ,τ
max t t t t+1tz
V y ,t F y ,z ,t +V y ,t + 1
22
Another approach to dynamic programming
max t t t t+1tz
V y ,t F y ,z ,t +V y ,t + 1
. .s t
t+1 t t t
t t
y - y = Q y ,z ,t
G y ,z ,t 0 max
T
t tt =0t t
y ,zF y ,z ,t
.s t
max
t t
t t
t t ty + Q y ,zF y ,z ,t +V ,t + 1
G y ,z
,
,t
t
0
z t t y t+1 z t t t z t tF y ,z ,t +V y ,t + 1 Q y ,z ,t λ G y ,z ,t = 0
Lagrange: (equating the derivative w.r.t. zt to 0 )
But: t+1= π ????? ?
23
Another approach to dynamic programming
. .s t
t+1 t t t
t t
y - y = Q y ,z ,t
G y ,z ,t 0 max
T
t tt =0t t
y ,zF y ,z ,t
z t t y t+1 z t t t z t tF y ,z ,t +V y ,t + 1 Q y ,z ,t λ G y ,z ,t = 0
But: t+1= π
{
}
T
t tt=0
t+1 t t t t+1 t t t
F y ,z ,t
+ π y + Q y ,z ,t - y - λ G y ,z ,t
L
????? ?
The Lagrangian of the original problem:
is the marginal value of increasing t+1t+1 yπ That is : t+1y ,t + 1yV
24
Another approach to dynamic programming
. .s t
t+1 t t t
t t
y - y = Q y ,z ,t
G y ,z ,t 0 max
T
t tt =0t t
y ,zF y ,z ,t
z t t y t+1 z t t t z t tF y ,z ,t +V y ,t + 1 Q y ,z ,t λ G y ,z ,t = 0
But: t+1= π
{
}
T
t tt=0
t+1 t t t t+1 t t t
F y ,z ,t
+ π y + Q y ,z ,t - y - λ G y ,z ,t
L
????? ?
The Lagrangian of the original problem:
is the marginal value of increasing t+1t+1 yπ That is : t+1y ,t + 1yV
z t t t+1 z t t t z t tF y ,z ,t + π Q y ,z ,t λ G y ,z ,t = 0
25
Another approach to dynamic programming
. .s t
t+1 t t t
t t
y - y = Q y ,z ,t
G y ,z ,t 0 max
T
t tt =0t t
y ,zF y ,z ,t
z t t y t+1 z t t t z t tF y ,z ,t +V y ,t + 1 Q y ,z ,t λ G y ,z ,t = 0
z t t t+1 z t t t z t tF y ,z ,t + π Q y ,z ,t λ G y ,z ,t = 0
but this is the (first order) condition for maximizing the Hamiltonian
26
Another approach to dynamic programming
. .s t
t+1 t t t
t t
y - y = Q y ,z ,t
G y ,z ,t 0 max
T
t tt =0t t
y ,zF y ,z ,t
max t t t t+1tz
V y ,t F y ,z ,t +V y ,t + 1
. .s t
t+1 t t t
t t
y - y = Q y ,z ,t
G y ,z ,t 0
Calculating the Bellman value functions is equivalent to the maximum principle (Hamiltonian)
27
Another approach to dynamic programming
. .s t
t+1 t t t
t t
y - y = Q y ,z ,t
G y ,z ,t 0 max
T
t tt =0t t
y ,zF y ,z ,t
max
s.t.
T T TT
T +1 T T T T T
zV y ,T F y ,z ,T
y = y + Q y ,z ,T , G y ,z ,T 0
max
s.t.
T -1 T -1 T -1 TT -1
T T -1 T -1 T -1 T -1 T -1
zV y ,T - 1 F y ,z ,T - 1 V y ,T
y = y + Q y ,z ,T - 1 , G y ,z ,T - 1 0
, , ,......, T T -1 T -2 t+1V y ,T V y ,T - 1 V y ,T - 2 V y ,t + 1
28
Another approach to dynamic programming
. .s t
t+1 t t t
t t
y - y = Q y ,z ,t
G y ,z ,t 0 max
T
t tt =0t t
y ,zF y ,z ,t
max t t t t+1tz
V y ,t F y ,z ,t +V y ,t + 1
. .s t
t+1 t t t
t t
y - y = Q y ,z ,t
G y ,z ,t 0
, , ,......, T T -1 T -2 t+1V y ,T V y ,T - 1 V y ,T - 2 V y ,t + 1
0V y ,0
Backwards Induction
