Contentsnc2371/teaching/R3.pdf · 2013. 10. 3. · G6215.001 - Recitation 3: Ramsey Growth model with technological progress; discrete time dy-namic programming and applications 1

G6215.001 - Recitation 3: Ramsey Growth model with technological progress; discrete time dy-namic programming and applications

Contents

1 The Ramsey growth model with technological progress 21.1 A useful lemma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2 Household . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.2.1 Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.2.2 Characterization of the optimal path for consumption and asset holdings . . . . . . . . . . . . 61.2.3 Euler equation and transversality condition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.3 Firms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.4 Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.5 The model equations in intensive form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2 Dynamic programming and applications 132.1 A first look at discrete time deterministic problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.1.1 Sequential problem and functional equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.1.2 The principle of optimality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.1.3 Solving the functional equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

1


1 The Ramsey growth model with technological progress

Spoiler: the differences with the Ramsey growth model seen in class are very minimal.In class, we saw that the competitive equilibrium of the Ramsey economy was the unique pair of determin-

isitic processes {c(t), k(t)}t≥0 solving the following system of differential equations:

c(t)

c(t)=

1

εu(c(t))(f ′(k(t)− δ − ρ)

k(t) = f(k(t))− (n+ δ)k(t)− c(t)

for the initial and boundary conditions:

limT→∞

[k(T ) exp

(−∫ T

0

(f ′(k(s))− δ − n)ds

)]= 0

k(0) > 0 given

(c(t), k(t)) were consumption per capita and capital per capita, respectively.Now, assume we add in exogneously growing Harrod-Neutral technology:

Y (t) = F (K(t), A(t)L(t))

A(t)

A(t)= g

In this recitation, we will show that the equilibrium of the newly obtained Ramsey economy is the uniquesequence {c(t), k(t)}t≥0 such that:

˙c(t)

c(t)=

1

σ

(f ′(k(t)− δ − ρ− σg

)˙k(t) = f(k(t))− (n+ δ + g)k(t)− c(t)


limT→∞

[k(T ) exp

(−∫ T

0

(f ′(k(s))− δ − n− g)ds

)]= 0

2


k(0) > 0 given

where (k(t), c(t)) are now consumption and capital per unit of effective labour, i.e.

k(t) ≡ K(t)

A(t)L(t)

c(t) ≡ C(t)

A(t)L(t)

Notice the subtle difference: εu(c(t)) ≡ −u′′(c(t))c(t)u′(c(t))

in the first definition has been replaced by the σ in thesecond definition. This is because when there is technological progress, we will show that the only form ofinstantaneous utility consistent with our Kaldor facts from the first recitation are CRRA utility functions.

Otherwise, the introduction of technological progress will merely mean that per capita variables now growat the constant rate g in the steady-state. In the differential equations governing the system, effective rates ofdepreciation have to be modified accordingly. However:

• The household’s optimization problem remains completely unchanged, and in particular, one may rewritethe new transversality condition in term of capital per capita (that is, in terms of k not k) to see that it isidentical to the transversality condition in the first problem.

• The analysis of existence and stability of the steady-state will be very similar to that made in class. Thesystem is saddle-path stable, and transitional dynamics are as described in class.

Since the analysis of the model is very similar to what was done in class, we will only emphasize importantpoints.

1.1 A useful lemma

We use this repeatedly in the analysis of the Ramsey economy.

Result 1 (First-Order Differential Equation with variable coefficients). Given two continuous functions r(t) andY (t), the linear differential equation:

X(t) = r(t)X(t) + Y (t)

with initial condition X(0) has unique solution:

X(T ) = X(0) exp

(∫ T

0

r(s)ds

)+ exp

(∫ T

0

r(s)ds

)∫ T

0

exp

(−∫ t

0

r(s)ds

)Y (t)dt

Proof. In class.

3


Application In the Ramsey economy, assets of the representative household (ie, aggregate assets) follow:

A(t) = r(t)A(t) + w(t)L(t)− C(t)

This is simply the budget constraint of the household. Using our lemma, along with the value of initial assetsA(0) we can solve for A(T ), as:

A(T ) = A(0) exp

(∫ T

0

r(s)ds

)+ exp

(∫ T

0

r(s)ds

)∫ T

0

exp

(−∫ t

0

r(s)ds

)(w(t)L(t)− C(t))dt

We can rewrite this as:∫ T

0

exp

(−∫ t

0

r(s)ds

)w(t)L(t)dt =

∫ T

0

exp

(−∫ t

0

r(s)ds

)C(t)dt+ exp

(∫ T

0

−r(s)ds)A(T )− A(0)

This is the budget constraint of the household (with population L(t), which can easily be rewritten in per capitaterms. On the left hand side is the present discounted value of total income from time 0 to time T , discountedfrom the point of view of period 0. The discount factor up to any time t ≤ T is: exp

(−∫ t

0r(s)ds

). On the right hand

side is the present discounted value of everything I purchase between time 0 and time T : that is, consumptionplus the net change in my stock of capital.

1.2 Household

1.2.1 Setup

At time t, the infinitely lived household is populated by L(t) = ent individuals who all derive the instantaneousutility from identical amount of consumption c(t) = C(t)/L(t), u(c(t)). Instantaneous utility is assumed to betwice continuously differentiable, strictly concave, and to satisfy the Inada-type condition: limc→0+ u′(c) = +∞.The lifetime utility of the household is given by:

U(c) =

∫ ∞0

u(c(t)) exp(−(ρ− n)t)dt

We always make the following assumption

Assumption 1.1 (Discounting). The discount rate always exceeds the rate of population growth, that is: ρ > n

4


Intuitively, why do we need to make this restriction? If population where to grow faster than the rateat which we discount the future, the population-adjusted discounted value of the utility of the guys living inperiod t, from the point of view of period 0, L(t)e−ρtu(c(t)), would tend towards infinity in the very long run, if u(ct)remains bounded. The households’ problem would cease to be well defined: any plan is optimal if discountedconsumption grows toward infinity as time increases.

Asset accumulation by the household follows:

A(t) = r(t)A(t) + w(t)L(t)− C(t)

or in per capita terms:a(t) = (r(t)− n)a(t) + w(t)− l(t)

The household seeks to pick a path (c(t), a(t)) that maximizes U(c) subject to the asset accumulation constraint.As discussed in class, if the household were only solving this problem, then it would prefer to choose a path

of A(t) that diverges toward −∞. Looking at the integrated budget constraint in the previous section, we seethat this would allow the household to pursue a path where consumption diverges (in the sense that the presentdiscounted value of consumption could be infinite).

Thus, in order to force the household to have a well-defined optimization problem, we need to impose anadditional constraint. There are different possible choices for this additional constraint; for a discussion of this(and in particular for why natural debt constraints are not sufficient here), see Acemoglu, 8.1.2. The constraintthat we choose to impose is the No-Ponzi scheme condition:

limT→+∞

A(T ) exp

[−∫ T

0

r(s)ds

]≥ 0

Or, in per capita terms:

limT→+∞

a(T ) exp

[−∫ T

0

(r(s)− n)ds

]≥ 0

Note that under the No-Ponzi scheme condition, taking limits in the integrated budget constraint as T → +∞,we obtain:

A(0) +

∫ T

0

exp

(−∫ t

0

r(s)ds

)w(t)L(t)dt ≥

∫ T

0

exp

(−∫ t

0

r(s)ds

)C(t)dt

This is a "lifetime" budget constraint (for the infinitely lived household): the present discounted value of totalearnings cannot be lower than the present discounted value of total consumption. However, this constraintdoes not, a priori, need to hold with equality.

5


1.2.2 Characterization of the optimal path for consumption and asset holdings

The household maximizes lifetime utility subject to the asset accumulation constraint and the No-Ponzi schemecondition. To solve this problem, we set up its associated Jacobian:

J(t) = u(c(t)) exp(−(ρ− n)t) + λ(t) (w(t) + (r(t)− n)a(t)− c(t))

Here, λ(t) represents the value, in utils at time 0, of an additional unit of income at time t, sometimes called theshadow value of assets at time t.

Under the No-ponzi scheme condition and the initial condition a(0) = 0, and because of the properties ofu, theorems 7.13 and 7.14 in Acemoglu tell us that the sequence of asset holdings and consumption that theconstrained maximization problem of the household is characterized by the following equations:

∂J(t)

∂c(t)= 0

∂J(t)

∂a(t)= −λ(t)

limT→∞

λ(T )a(T ) = 0

From the second equation and our initial lemma, the shadow value of income follows:

λ(t) = λ(0) exp

[−∫ t

0

(r(s)− n)ds

]= u′(c(0)) exp

[−∫ t

0

(r(s)− n)ds

]Eliminating out the shadow value of income, we can rewrite the sufficient conditions as:

c(t)

c(t)=

1

εu(c(t))(r(t)− ρ)

limT→∞

exp

[−∫ t

0

(r(s)− n)ds

]a(T ) = 0

The second condition is known as the transversality condition.

6


1.2.3 Euler equation and transversality condition

We will discuss two important intuitions about this solution during the recitation:

• First, the Euler equation. The term εu(c(t)) (the inverse of the intertemporal elasticity of substitution)is key here. It is always positive, and it governs the intensity of the response of consumption growthto the difference between interest and discount rates. Informally, when r(t) > ρ, the household wantsconsume less today in order to save and consume more tomorrow. However, when it does this, it also"suffers" because its consumption profile becomes steeper. The resulting choice of consumption path is acompromise between the two effects. To see why the elasticity of marginal utility governs the second effect,we can show that1

εu(c(t)) = lims→t−d ln(u′(c(s))/u′(c(t)))

d ln(c(s)/c(t))

To interpret the term inside the limit, think of s > t, cs > ct, and think of the effect of consuming slightlyless at time t and slightly more at time s, that is, increasing slightly the ratio cs/ct. At time t, you loseu′(ct) utils, but at time s, you gain u′(cs). Because u is concave, u′(ct) > u′(cs), so that small move alwaysmakes the agent worse off. By how much worse off depends on by how much u′(ct) increases relative tou′(cs), that is, it depends on the elasticity in the limit above. The bigger that elasticity, the worse thistrade-off is for you. In the limit, the larger the εu(c(t)), the worse it is for the household to increase theslope of its consumption profile. From the Euler equation, one sees that this effect dampens the responseof consumption to changes in interest rates.

• Second, the optimality condition:

limT→∞

exp

[−∫ t

0

(r(s)− n)ds

]a(T ) = 0

1Here is a dodgy proof. Let kt,s ≡ ln(

c(s)c(t)

), then up to first order:

u′(c(s))u′(c(t))

= 1 +u′′(c(t))c(t)

u′(c(t))kt,s + o(kt,s)

Thus,

ln(u′(c(s))u′(c(t))

) ≈ u′′(c(t))c(t)u′(c(t))

kt,s

As kt,s ≡ ln(

c(s)c(t)

), this roughly justifies the limiting equality.

7


is actually the No-Ponzi scheme condition, holding with equality. Be careful not to get this the wrong way.We need to impose the No-Ponzi scheme condition (inequality) in order to ensure that the optimizationproblem of the household is well defined. Once that is ensured, it turns out that the optimal plan involvesa path for assets that makes the No-Ponzi scheme condition bind. The pedestrian intuition for this is thatthere should be no money left on the table by the household: rouhgly, if the optimal solution involvedhousehold "dying" with a strictly positive amount of wealth, one can think of deviating from the optimumby saving slightly over less the life-cycle (in a way that the No-Ponzi condition still doesn’t bind) andincreasing the present discounted value of its consumption with the extra available funds. Because utilityis increasing in the PDV of lifetime consumption, this would violate optimality of the initial solution.

Finally, note that consumption per capita follows:

c(t)

c(t)=

1

εu(c(t))(r(t)− ρ)

Remember the Kaldor facts: in the long-run, consumption per capita has a constant growth rate, and r(t) isconstant. Therefore, the only type of preferences allowing for long-run growth and a constant interest rate arethose such that:

εu(c(t)) = constant ≡ σ

It can be shown that the this reduces us to the class of CRRA utility functions:

u(c) =c1−σ

1− σ

Our interpretation of the coefficient εu(c(t)) still holds; a small σ means that the utility cost of choosing asteeper consumption profile is relatively small (so that households will not care too much about variability ofconsumption over their life-cycle), while a large σ means the opposite (households prefer smooth consumptionprofiles).

As in class, we can use the transversality condition and our initial remark about the integrated budgetconstraint to show that:

c(0) = µ(0)(a(0) + w(0))

where:

µ(0)−1 =

∫ +∞

0

exp

(1− σσ

∫ t

0

r(s)ds− ρt/σ + nt

)dt

and w(0) is the discounted value of lifetime earnings.

8


We can use this expression to think about the effects of higher lifetime interest rates (an increase in∫ t

0r(s)ds)

on period 0 consumption. On the one hand, it increases our lifetime wealth (because all future savings will earna higher return); therefore, consumption should increase in all periods, and in particular c(0) should increase.But on the other hand, higher interest rates also mean the household gains from postponing consumption tothe future, and so c(0) should decrease. Which of the two effects dominates for c(0) is again a function of muchthe household dislikes steep consumption profiles. We know from our previous discussion that if σ is high, thenthe household dislikes steep consumption profiles, and so the second effect is probably small with respect tothe first effect; therefore we should expect consumption to rise. Indeed, if σ > 1, we see in the expression abovethat µ(0)−1 decreases with the increase in interest rate, and so c(0) rises. With a small σ the increase in interestrates would have the opposite effect: the substitution effect (towards future consumption) would dominate, andlead to a fall in c(0). When θ = 1, both effects exactly cancel out, and consumption does not change. (This isonly approximately true, since w(0) also changes with the interest rate).

1.3 Firms

Firms in the Ramsey model are exactly as in the Solow model. Namely, there is a representative firm withproduction technology F (K(t), A(t)L(t)) that, taking input prices R(t) and w(t) as given, minimizes its costs. Thefirst order conditions of the cost minimization problem are:

R(t) = F1(K(t), A(t)L(t))

w(t) = A(t)F2(K(t), A(t)L(t))

Since we wrote the consumer’s problem in per capita terms, we can also write the optimality conditions of thefirm in per capita terms using homogeneity of degree 0 of F1 and F2:

R(t) = F1(k(t), A(t))

w(t) = A(t)F2(k(t), A(t))

That’s all.

1.4 Equilibrium

This definition is written in per capita terms, so that we don’t need to make reference to the growth of thepopulation. Alternatively, we could write it in aggregate terms or (as we will do in the next section) in the formthat will allow to study the steady-state, that is, in intensive units.

9


Definition 1.1 (Competitive equilibrium in the Ramsey economy). A competitive equilibrium of the Ramsey econ-omy is a per capita allocation {c(t), k(t), a(t)}t≥0 and prices {r(t), R(t), w(t)}t≥0 such that given an exogenous processfor labour-augmenting technology {A(t)}t≥0:

1. Taking prices as given, households maximize lifetime utility subject to the capital accumulation constraint andthe No-Ponzi scheme condition, and given initial per capita assets a(0). This esults in a path for consumptionand assets fully characterized by the following conditions:

c(t)

c(t)=

1

σ(r(t)− ρ)

limT→∞

exp

[−∫ t

0

(r(s)− n)ds

]a(T ) = 0

a(t) = (r(t)− n)a(t) + w(t)− c(t)

2. Taking prices as given, firms minimize cost, resulting in the optimality conditions:

R(t) = F1(k(t), A(t))

w(t) = A(t)F2(k(t), A(t))

3. Asset (and implicitly labour) markets clear:R(t) = r(t) + δ

a(t) = k(t)

The only tricky part of this equilibrium definition is the clearing condition for the capital markets. It mayseem like there are two different pices for capital, R(t) and r(t). If there is no depreciation, then the twocoincide. If δ > 0, however, when renting out capital to the firm (through its asset purchases), the householdis also wearing it off; every unit rented out loses a share δ of its value each period. Thus, the effective returnto renting each unit of capital is R(t) − δ, the rental price offered by the firm, minus the depreciation rate ofcapital. I tend to think of δ in this model not as a depreciation but as a transaction cost: in transforming thehousehold’s assets into usable capital, the household incurs a loss of δ per unit of capital lended - possiblybecause some unspecified financial intermediary took it.

10


With this definition in hand, we can fully characterize the dynamics of the economy in terms of {c(t), k(t)}t≥0

by the following system of differential equations:

c(t)

c(t)=

1

σ(F1(k(t), A(t))− δ − ρ)

k(t) = (F1(k(t), A(t))− δ − n)k(t) + A(t)F2(k(t), A(t))− c(t)


limT→∞

exp

[−∫ t

0

(F1(k(s), A(s))− δ − n)ds

]k(T ) = 0

k(0) > 0 given

1.5 The model equations in intensive form

The system above is hard to study graphically, because it is not a system with constant coefficients: there isan extra variable A(t) which, although exogenous, affects the dynamics of the per capita variables. In order tostudy the system graphically, we need to find stationary variables. As usual, the stationary variables in thiseconomy will be consumption and capital per units of effective labour:

c(t) ≡ c(t)

A(t)

k(t) ≡ k(t)

A(t)

Let us define:f(k(t)) ≡ F (k(t), 1)

which, as in the previous recitation, implies that:

F1(k(t), A(t)) = f ′(k(t)

F2(k(t), A(t)) = f(k(t))− k(t)f ′k(t)

11


Then, using the fact that:c(t)

c(t)=

˙c(t)

c(t)+ g

k(t)

k(t)=

˙k(t)

k(t)+ g

we can rewrite the system of differential equations above as:

˙c(t)

c(t)=

1

σ

(f ′(k(t)− δ − ρ− σg

)˙k(t) = f(k(t))− (n+ δ + g)k(t)− c(t)

The initial condition k(0) > 0 is unchanged, since the level of technology is normalized to 1 at time 0. To seehow the boundary condition is modified, simply replace k(T ) = A(T )k(T ) = egT k(T ):

exp

[−∫ t

0

(F1(k(s), A(s))− δ − n)ds

]k(T ) = exp

[−∫ t

0

(f ′(k(t)− δ − n)ds

]egT k(T )

= exp

[−∫ t

0

(f ′(k(s))− δ − n− g)ds

]k(T )

so that the boundary condition becomes:

limT→∞

[k(T ) exp

(−∫ T

0

(f ′(k(s))− δ − n− g)ds

)]= 0

The two differential equations and the initial and boundary conditions are now fully written in terms ofendogenous variables and constants, and we can apply our usual steady-state and graphical analysis.

We’ll draw the new (and almost unmodified) phase diagram in class.A key element in the analysis of the existence of a steady-state of this system is that we must impose the

following restriction on parameters:

Assumption 1.2 (Discounting with technological progress). The discount rate must always exceed the populationgrowth rate, plus the (adjusted) growth rate of technological progress:

ρ > n+ (1− σ)g

12


Analytically, we must impose this constraint because if there is to be a steady-state for capital in the longrun, the the transversality can only hold if the steady-state discount factor f ′(k∗) − δ − n − g is strictly positive.Using the Euler equation in steady-state, we find that this restriction is equivalent to ρ > n+ (1− σ)g.

More intuitively, one can notice that since the steady-state interest rate is r∗ = ρ + σg, this assumption isequivalent to r∗ > n + g. To understand the meaning of this condition, remember that n + g is the growth rateof capital and output in the steady-state of this economy. If n + g > r∗, roughly speaking, the household couldborrow on unit of output today for future repayment r∗ tomorrow; because the economy is growing at rate (n+g),the asset would be worth (n+ g) tomorrow, and the household would be able to repay its debt and still obtain apositive profit, doing nothing. Doing this with not one but infinitely many assets, the household would be ableto obtain infinite utility.

Why does σ come into play in this condition? Notice that given our initial assumption about discounting, theadditional restriction ρ > n + (1 − σ)g is relevant when σ > 1, that is, when the household really dislikes steepconsumption profiles. However, when it does not really care about steep consumption profiles, that is, whenσ < 1, this assumption is stronger than the initial discounting assumption ρ > n. In effect, the higher the σ, thehiger the interest rate r∗: households that dislike steep consumption profiles need to be compensated more forthe consuming less today, saving, and consuming more tomorrow.

Thus, there are two effects driving the real interest rate in the steady-state: it must be high enough that no"arbitrage" of the products of exogenous technological progress and population growth is possible (the conditionr∗ > n + g; on the other hand, it must be lower, the less the household dislikes steep profiles. The conditionρ > n+ (1− σ)g reflects both these effects.

2 Dynamic programming and applications

For lack of time, we won’t study the (serious) theory of dynamic programming. What I will give you below shouldbe interpreted are unrigorous guidelines to how you should approach a general discrete time deterministicproblem using the tools of dynamic programming.

The serious theory of discrete time, deterministic and stochastic dynamic programming is detailed in "Re-cursive Methods in Economic Dynamics" by Stockey, Lucas and Prescott (1982).

13


2.1 A first look at discrete time deterministic problems

2.1.1 Sequential problem and functional equation

Consider a general discrete time, infinite horizon deterministic optimization problem. The ingredients of theproblem are:

• Discount rate β ∈ ]0, 1[

• State variable xt in X, subset of RL

• A non-empty correspondence Γ : X → X, such that: xt+1 ∈ Γ(xt), which describes the constraints of theproblem.

• Instantaneous utility function F (xt, xt+1), with values in R, and defined on {(x, y) ∈ X2 : y ∈ Γ(x)}.An example of such of problem is given in page 20-21.Given an initial value for the state, x0, a feasible plan is a sequence xt such that xt+1 ∈ Γ(xt) for all t ≥ 0.What we want to do is find a feasible plan that maximizes lifetime utility:

V (x0) =+∞∑t=0

βtF (xt, xt+1)

This is often called a "sequential problem" (SP):

V (x0) = max{xt}t≥0

+∞∑t=0

βtF (xt, xt+1)

s.t. xt+1 ∈ Γ(xt) (1)x0 given

The sequence that attains this optimum (if it exists) is called an "optimal plan"; I will denote it by {xt}t≥0.Now, as we saw in class, the solutions (V , x) to this problem (for a given initial solution) have some kind of

special relationship with the solutions to the problem:

J(x) = maxy ∈Γ(x)

F (x, y) + βJ(y) (2)

(3)

We will call this problem the Functional Equation (FE) problem, or also (as in class) the Bellman equation.

14


2.1.2 The principle of optimality

The main insight of dynamic programming is that value functions solving 1 also solve 2. This is known as the"principle of optimality".

Result 2 (Principle of optimality). Suppose that V is well defined on X. Then, V satisfies the functional equation2, that is:

∀x ∈ X, V (x) = maxy∈Γ(x)

F (x, y) + βV (y)

Proof. Take some x0 ∈ X, and let {xt} be a corresponding optimal plan for (SP). By optimality of the plan, for anyother feasible plan {xt} starting at x0,

V (x0) ≥ F (x0, x1) + β+∞∑t=1

βt−1F (xt, xt+1)

By definition,

V (x0) = F (x0, x1) + β+∞∑t=1

βt−1F (xt, xt+1)

so that:

F (x0, x1) + β+∞∑t=1

βt−1F (xt, xt+1) ≥ F (x0, x1) + β+∞∑t=1

βt−1F (xt, xt+1)

Fix a particular x1 ∈ Γ(x0), and let { ˆxt}t≥1 be an optimal plan from x1, then by definition of V :

V (x1) =+∞∑t=1

βt−1F (ˆxt, ˆxt+1)

so that:V (x0) ≥ F (x0, x1) + βV (x1)

Now, remember this holds for any x1 ∈ Γ(x0). Picking x1 = x1, that is, the second state for the optimal plan westarted with, we get:

β+∞∑t=1

βt−1F (xt, xt+1 ≥ V (x1)

15


But optimality of V (x1) over feasible plans starting at x∗1, we have that:

V (x1) ≥ β+∞∑t=1

βt−1F (xt, xt+1

So that:

V (x1) = β+∞∑t=1

βt−1F (xt, xt+1)

Now we can replace this into the initial relation to get:

V (x0) = F (x0, x1) + βV (x1)

≥ F (x0, x1) + β+∞∑t=1

βt−1F (xt, xt+1)

= F (x0, x1) + βV (x1)

This holds for any x1 ∈ Γ(x0), so V solves (FE).

We can use the computations in this proof to gain some insight into the optimal plan {x}t≥0.

Result 3 (Optimal plans are generated by the policy correspondence). . Suppose V is well defined on X. Fixx0 ∈ X and let {xt}t≥0 be a corresponding optimal plan. Then, the optimal plan satisfies:

∀t, xt+1 ∈ G(xt)

where G is defined as:G(x) = arg max

x∈Γ(y)F (x, y) + βV (y)

Proof. The previous computations showed that:

V (x0) = F (x0, x1) + βV (x1)

Because V (x0) solves the Bellman equation, this implies that:

x1 ∈ G(x0)

. Moreover, we showed that the plan {xt}t≥1 is optimal starting from x1. The desired result follows from induction.

16


The function G is called the policy correspondence. This theorem tells you that if you happen to know thesolution to (SP), then you may derive a large set of sequences from the policy correspondence (sequences sothat xt+1 ∈ G(xt), among which you will find our optimal plan.

Why do we care ? As you guessed from the formulation of the theorems, unless we add some additionalrestrictions to the problen, there will be solutions to the FE that are not solutions to the SP. Similarly, theremay be plans that are generated by the policy correspondence, but that are not the optimal plans. (We will giveone such example in class).

However, there are partial converses to the theorem: that is, restrictions on the solutions to FE that ensurethey are also solutions to SP.

Result 4 (Optimality of plans generated by the policy correspondence). Let {xt}t≥0 be generated by the policycorrespondence. It is an optimal plan for (SP) if and only if :

limt→+∞

βtV (xt) = 0

Result 5 (Partial converse to the principle of optimality). Suppose V solves (FE). Furthermore, assume that, foreach x0, there is a plan {xt} such that:

V (xt) = F (xt, xt+1) + βV (xt+1)

and such that:limt→+∞

βtV (xt) = 0

Finally, suppose that:lim sup

t→∞βtV (xt) ≥ 0

for all feasible plans. Then, V = V .

Given all this, our strategy to tackle a problem of the type (SP) will be the following:

1. Write down the corresponding problem (FE).

2. If you can solve this problem, solve it. The next section describes how to. If the problem has not solutions,then you’re done: the principle of optimality then also tells you there are no solutions to the (SP) problem.

17


3. Once you have your solutions to the (FE) problem, apply the converse theorem: check that some solutionV to FE, you can generate feasible plans that satisfy the conditions of the previous theorem; and checkthat the solution you picked, V , itself satisfies the limsup condition.

4. If you find such a V among the solutions to (FE), you’re done: it’s your solution to (SP). Moreover, then youcan apply the converse theorem for the policy correspondence to generate a plan that is optimal for (SP).

In practice, what people really do is use some restrictions on X, F and Γ that ensure the existence andunicity of a solution to (FE) that satisfies the conditions of the partial converse.

Some other restrictions, which we will not cover here, ensure that V is strictly concave, differentiable andthat the policy function is a single-valued correspondence.

2.1.3 Solving the functional equation

So how do we solve the functional equation? There are two ways. One is numerical: under some (very restrictive)on the primitives of the problem, we can show that the solution to (FE) is unique and moreover, it is the limit ofa sequence of functions that are easy to approximate numerically. I will show a short example of Matlab codeon how to do this. The other method is to characterize the solution by using optimality conditions.

By applying the contraction mapping theorem Make the following restrictions:

• X is a convex set

• F (x, y) is continuous and bounded

• Γ is continuous and compact-valued

The key is to think of the Bellman equation as a mapping. Define the mapping:

Tf(x) = maxy∈Γ(x)

F (x, y) + βf(y)

We define T over the space of continuous and bounded functions over X, with the sup norm. Because Fis continuous and Γ is compact-valued, the theorem of the maximum (see SLP, chapter 3) tells us that thisdefinition makes sense.

Notice that if we can find an f such that:Tf = f

18


then we are done: this f , by construction, solves (FE). Remember that to get back to (SP), we would still need tocheck that it verifies some additional transitional conditions. In this case, because our solution is in the spaceof continous and bounded functions, it is very simple to check that the conditions for the converse theoremactually hold. Therefore, if we show that this fixed point is unique, we are done: it is the solution to the (SP).

So how do we show that the function equation Tf = f has a unique solution, and how do we find thissolution? The idea is to prove that T is a contraction.

Definition 2.1 (Contraction mapping). Let C be a complete metric space for the norm ‖.‖ and let T be a mappingfrom C to C. Let β ∈ ]0, 1[. T is a contraction mapping, if and only if, for all f, g in C,

‖Tf − Tg‖ ≤ β ‖f − g‖

This definition is fairly general; but since the space of continuous and bounded functions over X is a completemetric space for the sup norm, we may hope that our own T is also a contraction.

But why do we want T to be a contraction? Because of the following theorem:

Result 6 (Contraction mapping theorem). Let C be a complete metric space for the norm ‖.‖, and let T be acontraction on (C, ‖.‖). Then:

• T has a unique fixed point, f , inside C.

• For any f0 ∈ C, define the sequence (fn)n≥0 by fn+1 = Tfn. Then,

limn→+∞

fn = f

The proof of this theorem is in SLP, chapter 3.This theorem is interesting to us because it tells us that if our correspondence is a contraction, then we have

existence and unicity of the solution to our problem, and furthermore we know how to find it.In order to check that the our mapping is indeed a contraction, we need to check that Blackwell’s sufficiency

conditions hold. For a proof that these conditions are sufficient, see SLP, chapter 3.They are as follows:

Result 7 (Blackwell’s sufficiency conditions). Suppose the mapping T , defined on the space of continuous boundedfunctions on X endowed with the sup norm, has the following two properties:

• T is monotone: if f, g are such that f(x) ≥ g(x) for all x ∈ X, then Tf(x) ≥ Tg(x) for all x ∈ X.

19


• T satisfies a discounting inequality: there is 0 < δ < 1 such that for any a ≥ 0, and any f ∈ X, the functionf(x) = g(x) + a) satisfies:

T (f − g)(x) ≤ δa

Then, T is a contraction mapping over C.

An example of applying the contraction mapping theorem to solvle numerically an FE We will look at thefollowing, very simple capital accumulation problem under the SP form:

maxct,bt+1

+∞∑t=0

βtln(ct)

ct + bt+1 ≤ Rbαt

The only constraints here are:ct > 0

bt+1 > 0

We can put this in our canonical form with:X = R+

xt = bt

Γ(x) = {y ∈ X : Rxα − y > 0}

F (x, y) = ln(Rxα − y)

We can check that, except for the fact that F is not bounded, all other conditions that we reviewed in theprevious section apply. However, the fact that F is bounded is only useful because it serves to apply themaximum theorem in order to obtain that our mapping T is well defined. Here, it is easy to check that themappin is well defined even if F is not bounded.

The mapping in question is:Tf(x) = max

y∈Γ(x)ln(Rxα − y) + βf(y)

As previously mentioned, this mapping is well defined on the space of continuous and bounded functionsfor the sup norm (the space for f , not F !). Furthermore, checking Blackwell’s sufficiency condition is trivial. We

20


can therefore apply the contraction mapping theorem: the solution to (SP ) exists, is unique, and can be foundby constructing the sequence:

fn+1 = Tfn

for some initial f0.How do we do this, in practice? Here is some bad pseudo-code that one can follow to solve our problem

using, for example:

1. Choose some compact subset of X, for example Y = [0.1; 15].

2. Create a vector of n points, equally spaced between the boundaries of Y . This is your "grid".

3. Pick an initial function f0; for example, f0 = 1 (f is constant). Note that this choice of f0 ensures thatf0 is bounded and continuous, so we start within C, the space of continuous bounded functions. Thisfunction is represented by a vector of length n inside the computer: each element corresponds of thisvector corresponds to the value of f0 at a point of the "discretized" grid of Y .

4. Apply repeatedly T to generate the sequence fn, until the fn almost don’t change, that is, until maxx∈Y (fn+1(x)−fn(x)) < ε for some very small epsilon. In the computer, one needs to do the following:

• Given your current value for f , For each value x ∈ Y , compute all possible values of ln(Rxα− y) + βf(y)for all the y ∈ X. Note that you compute all values of the objective function for y ∈ X, not y ∈ Γ(x). Ifnecessary, just throw out the values of y that are not in Γ(x).

• Among those values for which you computed the objective function, pick the value of y which maxi-mizes ln(Rxα − y) + βf(y). This value is your new Tf(x).

• Once you finished doing this for all x ∈ Y , you can compute the maximum of the distance betweenTf(x) and f(x). This is the sup norm. If it is smaller than ε, exit; if not, repeat the steps above

The contraction mapping theorem ensures that this process will eventually converge, and when it does, wewill have retained an approximation to the value function on our subset Y .

By Euler equations and transversality conditions This has already bee done in class: assuming that thevalue function is continuously differentiable and concave, and that the constraint set is convex, solutions to(FE) are characterized by:

DV (x) = F1(x, y)

21


Fy(x, y) = −βDV (y)

and y ∈ Γ(x). As you may remember from our initial discussion, we need to impose additional conditions inorder to ensure that this solution corresponds to the solution to our initial problem, (SP).

It turns out that, for instantaneous value functions F that are continuously differentiable, increasing inx and strictly concave, it is easy to characterize some optimal plans directly (without resorting to the (FE)).Indeed, in this case, any feasible plan such that xt ∈ int Γ(xt) and such that:

F2(xt, xt+1) + βF1(xt+1, xt+2) = 0

limt→+∞

βtF1(xt, xt+1) = 0

is an optimal plan. When seeking to characterize the solutions to (SP) problems through Euler equations, it isthis theorem that we are often using.

22

Documents

Contentsnc2371/teaching/R3.pdf · 2013. 10. 3. · G6215.001 - Recitation 3: Ramsey Growth model with technological progress; discrete time dy-namic programming and applications 1