Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Foundations ofDynamic Macroeconomic Analysis

Ping Wang, Washington University in St. Louis/NBER

January 2016

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Introduction

This set of lecture notes provides foundations of macroanalysis, focusing particularly on the following issues:

1 Fundamentals of Dynamic General Equilibrium2 Optimal Growth in Discrete Time3 Optimal Growth in Continuous Time

Basic references:

1 Acemoglu (2009): chs. 5-72 Aghion and Howitt (1998): chs. 1-23 Barro and Sala-i-Martin: ch. 2, secs. 4.1-4.34 Ljungqvist and Sargent (2000): chs. 3 and 115 Stokey and Lucas with Prescott (1989): chs. 3-56 Wang (2012), “Endogenous Growth Theory,” Lecture

Notes, Washington University-St. Louis.

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Representative Household I

Representative household is valid when the optimizationof individual households can be represented as if therewere a single household making the aggregate decisionsusing a representative preference subject to aggregateconstraints.Consider a particular preferences representation:

max∞

∑t=0

βtu (c (t))

Let the excess demand of the economy be x(p).Key: whether this excess demand function x(p) can beobtained as a solution to the single household optimizationproblem.

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Representative Household II

Answer at the first glance: it cannot be so obtained ingeneral as the weak axiom of revealed preferences forindividuals need not hold for the aggregate.

Theorem H1 (Debreu-Mantel-Sonnenschein Theorem) Letε > 0 and N 2 N. Consider a set of pricesPε = fp 2 RN

+ : pj/pj0 ε for all j and j0g and any continuous

function x : Pε ! RN+ that satisfies Walras’s Law and is

homogeneous of degree 0. Then there exists an exchangeeconomy with N commodities and H < ∞ households, wherethe aggregate excess demand is given by x (p) over the set Pε.

To yield a positive answer ensuring the excess demandfunction x(p) to be obtained as a solution to the singlehousehold optimization problem, we need to imposefurther restrictions, in particular, to remove strong incomeeffects.

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Representative Household III

A special but useful case for such a representation is tohave linear value (indirect utility) function:

Theorem H2 (Gorman’s Aggregation Theorem) Consider andeconomy with N < ∞ commodities and a set H of households.Suppose that the preferences of each household h 2 H can berepresented by an indirect utility function of the form

υh

p, wh= ah (p) + b (p)wh

and that each household h 2 H has a positive demand for eachcommodity. Then these preferences can be aggregated andrepresented by those of a representative household, withindirect utility

υ (p, w) = a (p) + b (p)w,

where a (p) R

h2H ah (p) dh, and w R

h2H whdh is aggregateincome.

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Representative Household IV

The class of preferences described in Theorem H-2 isreferred to as “Gorman preferences” (1959 Econometrica).In this class, the Engel curve of each household for eachcommodity is linear and its slope is identical to allindividuals for the same commodity.By Roy’s Identity,

xhj

p, wh

= 1

b (p)∂ah (p)

∂pj 1

b (p)∂b (p)

∂pjwh

Therefore, for each household, a linear relationship existsbetween demand and income and the slope, 1

b(p)∂b(p)

∂pj, is

independent of the household’s identity (h).

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Representative Household V

Even under class of “Gorman preferences” a representativehousehold exists, typical macro models require furtherrestrictions on:

the abstract of distribution effects from the representativehousehold’s concern (strong representation);the use of the representative household’s preference as thewelfare function of the aggregate economy (normativerepresentation).

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Representative Household VI

Normative representation requires convexity, interiorityand price-invariant basic value (o.w., one can transfer εfrom low to high-valuation households for different p):

Theorem H3 (Normative Representation) Consider aneconomy with a finite number N < ∞ of commodities, a set Hof households, and a convex aggregate production possibilitiesset Y. Suppose that the preferences of each household h 2 H isrepresented by υh p, wh = ah (p) + b (p)wh withp = (p1, ..., pN) and that each household h 2 H has a positivedemand for each commodity.1. Then any feasible allocation that maximizes the utility of therepresentative household υ (p, w) = ∑h2H ah (p) + b (p)w, withw ∑h2H wh, is Pareto optimal.2. If ah (p) = ah for all p and all h 2 H (price-invariant basicvalue), then any Pareto optimal allocation maximizes the utilityof the representative household.

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Representative Firm I

Representative firm is valid when the optimization ofindividual firms can be represented as if there were asingle firm making the aggregate decisions using anaggregate production function subject to aggregateconstraints.Consider price and output vectors: p = (p1, ..., pN) andy = (y1, ..., yN), with p y = ∑N

j=1 pjyj.Let F be the countable set of firms in the economy and

Y (

∑f2F

yf : yf 2 Yf for each f 2 F)

be the aggregate production possibility set (PPS).

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Representative Firm II

When there is no price-dependent fixed cost (ensuredunder perfect competition in the absence of externalities),we have:

Theorem F (Representative Firm Theorem) Consider acompetitive production economy with N 2 N[ f+∞gcommodities and a countable set F of firms, each with aproduction possibilities set Yf RN. Let p 2 RN

+ be the pricevector in this economy and denote the set of profit-maximizingnet supplies of firm f 2 F by Yf (p) Yf so that for anyyf 2 Yf (p), we have p yf p yf for all yf 2 Yf . Then thereexists a representative firm with production possibilities setY RN and a set of profit-maximizing net supplies Y (p) suchthat for any p 2 RN

+, y 2 Y (p) if and only if y = ∑f2F yf forsome yf 2 Yf (p) for each f 2 F .

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Equilibrium

An economy E is described by preferences, endowments,production sets, consumption sets, and allocation ofshares, that is, E (H,F , U, ω, Y, X, θ).An allocation (x, y) in E , x 2 X, y 2 Y, is feasible if∑h2H xh

j ∑h2H ωhj +∑f2F yf

j for all j 2 N.

Definition E (Competitive Equilibrium) A competitiveequilibrium for economy E (H,F , U, ω, Y, X, θ) is given by afeasible allocation

x = fxhgh2H, y = fyfgf2F

and a price

system p such that1. (Firm optimization) For every firm f 2 F , yf maximizesprofits: p yf p yf for all yf 2 Yf .2. (Household optimization) For every household h 2 H, xh

maximizes utility: Uh xh Uh xh for all x such that xh 2 Xh

and p xh p

ωh +∑f2F θhf yf

.

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Pareto Optimum

The standard concept of optimality is Pareto optimum,though social optimum is often used in macroeconomics.

Definition O (Pareto Optimum) A feasible allocation (x, y) foreconomy E (H,F , U, ω, Y, X, θ) is Pareto optimal if thereexists no other feasible allocation (x0, y0) such that x0h 2 Xh forall h 2 H, y0f 2 Yf for all f 2 F , and Uh x0h Uh xh for all

h 2 H with Uh0

x0h0> Uh0

xh0

for at least one h0 2 H.

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Welfare Theorems I

Household h 2 H is locally nonsatiated if, at each xh 2 Xh,Uh xh is strictly increasing in at least one of its argumentsand Uh xh < ∞.

Theorem W1 (First Welfare Theorem I) Suppose that(x, y, p) is a competitive equilibrium of economyE (H,F , U, ω, Y, X, θ) with H finite. Assume continuouspreferences with all households locally nonsatiated. Then(x, y) is Pareto optimal (PO).Theorem W2 (First Welfare Theorem II) Suppose that(x, y, p) is a competitive equilibrium of economyE (H,F , U, ω, Y, X, θ) with H countably infinite. Assumecontinuous preferences with all households locally nonsatiatedand

p ω ∑h2H

∞

∑j=0

pj ωhj < ∞.

Then (x, y, p) is Pareto optimal.

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Welfare Theorems II

Theorem W3 (Second Welfare Theorem) Consider a Paretooptimal allocation (x, y) in an economyE (H,F , U, ω, Y, X, θ). Suppose that all consumption setsare convex, all production sets are convex cones, all utilityfunctions are continuous and quasi-concave, and interiorityconditions are met such that (i) there exists χ < ∞ such that∑h2H xh

j,t < χ for all j and t; (ii) 0¯2 Xh for each h; (iii) for any h

and xh, xh 2 Xh such that Uh xh > Uh xh, there existsT(h, xh, xh) such that Uh xh [T]

> Uh xh for all

T T(h, xh, xh); and (iv) for any f and yf 2 Yf , there existsT(f , yf ) such that yf [T] 2 Yf for all T T(f , yf ). Then thereexists a price vector p and endowment and share allocations(ω, θ) with ω = ∑h2H ωh, θ = ∑f2F θf, such that:1. for all f 2 F , p yf p yf for any yf 2 Yf ;2. for all h 2 H, if Uh xh > Uh xh for some xh 2 Xh thenp xh p wh, where wh ωh +∑f2F θfyf.

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Welfare Theorems III

The Second Welfare Theorem can be readily used to provethe existence of a competitive equilibrium based onBrouwer/Kakutani.

Theorem W3 (Existence of Competitive Equilibrium)Consider an economy E as described in Theorem W3, ifp wh > 0 for each h 2 H, then there exists a competitiveequilibrium (x, y, p).

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Dynamic General Equilibrium I

To generalize the standard Arrow-Debreu-McKenzeigeneral equilibrium (GE) analysis to an infinite-horizondynamic framework (DGE), one needs to impose furtherrestrictions, particularly in the following aspects:

finite value: bounded valuation of households/firmsinfinite dimensional space: Banach Space, Hilbert Space,Polish Space with weak or weak* topology rather thanstandard product topologyinfinite dimensional fixed point: Schauder and othersinteriority in infinite dimensioninformation and Arrow-Debreu trade.

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Dynamic General Equilibrium II

In the context of optimal growth, such DGE frameworksare given by

discrete time:

maxfct,ktg∞t=0

∑∞t=0 βtu (ct)

s.t. kt+1 = f (kt) + (1 δ) kt ctct, kt 0, k0 > 0

continuous time:

maxfc(t),k(t)g∞t=0

R ∞0 exp (ρt) u (c (t)) dt

s.t. k (t) = f (k (t)) c (t) δk (t)c (t) , k (t) 0, k (0) > 0.

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Dynamic Programming I

Discrete-time infinite-horizon optimization problem:

supfxt,ytg∞t=0

∑∞t=0 βtUt(t, xt, yt)

s.t. yt 2 G(t, xt) 8t 0xt+1 = f (t, xt, yt) 8t 0x0 > 0 given

Eliminate yt and rewrite the optimization problem:

V0 (x0) = supfxtg∞t=0

∑∞t=0 βtUt(xt, xt+1)

s.t. xt+1 2 Gt(t, xt) 8t 0x0 > 0 given.

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Dynamic Programming II

Under stationarity, Ut = U and Gt = G, yielding thefollowing recursive problem:

V(x0) = supfxtg∞t=0

∑∞t=0 βtU(xt, xt+1)

s.t. xt+1 2 G(xt) 8t 0x0 given

The functional problem is (Bellman equation):

V(x) = supy2G(x)

fU(x, y) + βV(y)g 8x 2 X.

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Dynamic Programming III

Feasible set:Φ(xt) = ffxsg∞

s=t : xs+1 2 G(xs), s = t, t+ 1, ...g.Let X be a compact subset of RK andXG = f(x, y) 2 XX : y 2 G(x)g.Assumption A1:

(a) G(x) 6= ∅ for all x 2 X;(b) G is compact-valued and continuous;(c) G is an increasing set s.t. x x0 ) G(x) G(x0).

Assumption A2:(a) 8 x0 2 X and x 2 Φ(x0),limn!∞ ∑n

t=0 βtU(xt, xt+1) = V∞ < ∞;(b) U : XG ! R is continuous;(c) U(x, y) is strictly increasing in the first K elements foreach y 2 X;(d) U(x, y) is strictly concave in (x, y);(e) U is continuously differentiable over intXG.

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Dynamic Programming IV

Applying sandwich theorem, we can show:

Theorem DP1 (Equivalence of Values) Under A1(a) and A2(a),the solution to the recursive problem V(x0) is equivalent to thesolution to the functional problem V(x), i.e., V(x) = V(x) forall x 2 X.

By recursive substitution, it is straightforward to obtain:

Theorem DP2 (Principle of Optimality) Under A1(a) andA2(a), consider a feasible plan x 2 Φ(x0) that attains V(x0) inthe recursive problem. Then

V(xt ) = U(xt , xt+1) + βV(xt+1)

for t = 0, 1, ... with x0 = x0. Moreover, if any x 2 Φ(x0) attainsV(x) in the functional problem, then it attains the optimal valuein the recursive problem.

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Dynamic Programming V

With continuity as well as compactness of the constraintset, we can apply Beige’s Theorem of Maximum andWeierstrass Theorem to obtain:

Theorem DP3 (Existence of Solutions) Under A1(a,b) andA2(a,b), there exists a unique continuous and boundedfunction V : X ! R that satisfies the Bellman equation.Moreover, for any x0 2 X, an optimal plan x 2 Φ(x0) exists.

With concavity and differentiability, we have:

Theorem DP4 (Differentiability of the Value Function) UnderA1(a,b) and A2(a,b,d,e), consider an optimal plan x and defineπ as the policy function satisfying xt+1 = π(xt ). Furtherassume that x 2 intX and π(x) 2 intG(x). Then V() isdifferentiable at x, with gradient DV(x) = DxU(x, π(x)).

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Dynamic Programming VI

Let (s, d) be a norm space and T : S ! S be an operatormapping S into itself. If for some β 2 (0, 1),

d(Tz1, Tz2) βd(z1, z2) 8z1, z2 2 S

Then T is a contraction mapping (with modulus β).

Theorem DP5 (Contraction Mapping Theorem) Let (s, d) be acomplete norm space and suppose that T : S ! S is acontraction. Then T has a unique fixed point, z, i.e., there existsa unique z 2 S such that Tz = z.

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Dynamic Programming VII

In practice, the following theorem provide sufficientconditions for a contraction map:

Theorem DP6 (Blackwell’s Sufficient Conditions for aContraction) Let X Rk, and B(X) be the space of boundedfunctions f : X ! R defined on X equipped with the sup normjj jj. Suppose that B0(X) B(X), and let T : B0(X)! B0(X) bean operator satisfying the following two conditions:1. (Monotonicity) For any f , g 2 B0(X), f (x) g(x) 8x 2 X,(Tf )(x) (Tg)(x) 8x 2 X;2. (Discounting) 9 β 2 (0, 1) s.t. [T(f + c)](x) (Tf )(x) + βc8f 2 B(X), c 0, and x 2 X.Then T is a contraction (with modulus β) on B0(X).

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Dynamic Programming VIII

Applying Contraction Mapping Theorem to the valuefunction V, we can establish its properties:

Theorem DP7 (Value Function Properties) Under A1(a,b,c)and A2(a,b,c,d), V(x) in the Bellman equation is strictlyincreasing in all of its arguments and strictly concave.Theorem DP8 (Necessity and Sufficiency) Under A1(a,b,c)and A2(a,b,c,d,e), a sequence fxt g∞

t=0 such that xt+1 2 intG(xt ),t = 0, 1..., is optimal for the recursive problem given x0 if andonly if it satisfies the following:1. (Euler Equations) DyU(x, π(x)) + βDxU(π(x), π(π(x))) = 02. (Transversality Conditions) limt!∞ βtDxU(x, xt+1)x

t = 0.

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Dynamic Programming IX

In standard optimal growth models, A1(a,b,c) andA2(a,b,c,d,e) are all met.Example: consider U = ln (ct) and G = kα

t ct, so theBellman equation is: V(x) = maxy0fln(xα y) + βV(y)g

(Euler) 1xαy = βV0(y)

(Benveniste-Scheinkman) V0(x) = αxα1

xαy(Policy function) kt+1 = αβkα

t , ct = (1 αβ)kαt .

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Optimal Control I

Consider a canonical continuous-time optimizationproblem (P):

maxfx(t),y(t),t1g W =R t1

0 f (t, x(t), y(t))dt

s.t.

x(t) = G(t, x(t), y(t)) 8tx(t) 2 X (t), y(t) 2 Y(t) 8t; x(0) = x0

Special case: finite time horizon t1 < ∞ with one state xand one control y with ˙x(t) = g(t, x(t), y(t)) where y (t) iscontinuous in t and hence x(t) continuously differentiableso that variational arguments can be applied:

f : [0, t1] R R ! R and g : [0, t1] R R ! R arecontinuously differentiable;X and Y are nonempty and convex;W(x(t), y(t)) < ∞ for any admissible pair (x(t), y(t));(x(t), y(t)) 2 intX Y (never involving discontinuities) )W(x(t), y(t)) W(x(t), y(t)).

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Optimal Control II

Calculus of variation arguments:

y(t, ε) = y(t) + εη(t) 2 intY , 8t 2 [0, t1], 8ε 2 [ε0η , ε0η ]

Feasible path of state:

x(t, ε) = g(t, x(t, ε), y(t, ε))8t 2 [0, t1] and x(0, ε) = x0

Optimization over ε for all variations:

W(ε) =Z t1

0[f (t, x(t, ε), y(t, ε))+λ(t)g(t, x(t, ε), y(t, ε))+λ(t)x(t, ε)]dt

- λ(t1)x(t1, ε) + λ(0)x0

W0(0)=Z t1

0[fx(t, x(t), y(t))+λ(t)gx(t, x(t), y(t))+λ(t)]xε(t, 0)dt

+Z t1

0[fy(t, x(t), y(t))+λ(t)gy(t, x(t), y(t))]η(t)dt-λ(t1)xε(t1, 0)=0.

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Optimal Control III

Theorem OC1 (Necessary Conditions) Consider the problem(P) with t1 < ∞, one state-one control and f and g continuouslydifferentiable. Suppose that (P) has an interior continuoussolution (x(t), y(t)) 2 intX Y . Then there exists acontinuously differentiable costate function λ() defined ont 2 [0, t1] such that:

(FOC) fy(t, x(t), y(t)) + λ(t)gy(t, x(t), y(t)) = 0(Euler) λ(t) = [fx(t, x(t, ε), y(t, ε)) + λ(t)gx(t, x(t, ε), y(t, ε))](Terminal) λ(t1) = 0

(Complementary slackness) λ(t1)(x(t1) x1) = 0 ifx(t1) x1, with λ(t1) > 0 if x(t1) = x1.If t1 is given (as is x(t1) = x1), then there is no need forλ(t1) = 0.

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Optimal Control IV

Hamiltonian (which is continuously differentiable as well):H(t, x(t), y(t), λ(t)) = f (t, x(t), y(t)) + λ(t)g(t, x(t), y(t)).

Theorem OC2 (Maximum Principle) Consider (P) given inTheorem OC1. Suppose that (P) has an interior continuoussolution (x(t), y(t)) 2 intX Y . Then there exists acontinuously differentiable function λ(t) such that the optimalcontrol y(t) and the corresponding path of the state variablex(t) satisfy the following necessary conditions (NC):

(FOC) Hy(t, x(t), y(t), λ(t)) = 0, 8t 2 [0, t1](Euler) λ(t) = Hx(t, x(t), y(t), λ(t)), 8t 2 [0, t1](State Evolution) x(t) = Hλ(t, x(t), y(t), λ(t)), 8t 2 [0, t1]

with x(0) = x0 and λ(t1) = 0. Moreover, H(t, x, y, λ) alsosatisfies the Maximum Principle:H(t, x(t), y(t), λ(t)) H(t, x(t), y, λ(t)), 8y 2 Y , 8t 2 [0, t1].

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Optimal Control V

For global (rather than local) maximum:

Theorem OC3 (Mangasarian Sufficiency Conditions) Supposean interior continuous pair (x(t), y(t)) 2 intX Y exists andsatisfies necessary conditions (NC). Suppose (i) X Y isconvex and (ii) given λ(t), H(t, x, y, λ) is jointly concave in(x, y) 2 X Y , 8t 2 [0, t1]. Then (x(t), y(t)) achieves the globalmaximum of (P) given in Theorem OC1. If H is strictly concavein (x, y), 8t 2 [0, t1], then such a solution is unique.Theorem OC4 (Arrow-Kurz Sufficiency Conditions) Suppose(x(t), y(t)) 2 intX Y exists and satisfies (NC) and defineM(t, x(t), λ(t)) = maxy2Y H(t, x(t), y, λ(t)) as the MaximizedHamiltonian. Suppose (i) X is convex and (ii) M(t, x, λ) isconcave in x 2 X , 8t 2 [0, t1]. Then (x(t), y(t)) achieves theglobal maximum of (P) given in Theorem OC1. If M(t, x, λ) isstrictly concave in x, 8t 2 [0, t1], then such a solution is unique.

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Optimal Control VI

We now generalize the theorems to the multivariate case:

Theorem OC5 (Maximum Principle) Consider (P) with t1 < ∞and f and g continuously differentiable. Suppose it has aninterior continuous solution (bx(t), by(t)) 2 intX Y and defineH(t, x, y, λ) = f (t, x(t), y(t)) + λ(t) G(t, x(t), y(t)), λ(t) 2 RKx .Then (bx(t), by(t)) satisfy the following necessary conditions:

(FOC) DyH(t,bx(t), by(t), λ(t)) = 0, 8t 2 [0, t1]

(Euler)λ(t) = DXH(t,bx(t), by(t), λ(t)), 8t 2 [0, t1]

(State Evolution)x(t) = DλH(t,bx(t), by(t), λ(t)), 8t 2 [0, t1]

(Terminal) λ(t1) = 0 (λ(t1) > 0 if t1 given)

If X is convex and M(t, x, λ) = maxy(t)2Y H(t, x(t), y, λ(t)) isconcave in x 2 X , 8t 2 [0, t1], then (bx(t), by(t)) achieves theglobal maximum of (P) given above. If M(t, x, λ) is strictlyconcave in x, 8t 2 [0, t1], then such a solution is unique.

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Optimal Control VII

We next consider the case of infinite horizon with onecontrol and one state (P0):

maxfx(t),y(t)g W =R ∞

t0f (t, x(t), y(t))dt

s.t. ˙x(t) = g(t, x(t), y(t)) 8tx(t) 2 X (t), y(t) 2 Y(t) 8t; x(t0) = x0

Problem (P0) can be rewritten in value function form:

V(t0, x(t0)) = sup(x(t),y(t))2X (t)Y(t)R ∞

t0f (t, x(t), y(t))dt

s.t. ˙x(t) = g(t, x(t), y(t)) 8t

(Principal of Optimality): If (x(t), y(t)) solves (P0), then8t t0, (V0):

V(t0, x(t0)) =Z t

t0

f (t0, x(t0), y(t0))dt0 +V(t, x(t))

This is analogous to the Bellman equation in dynamicprogramming.

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Optimal Control VIII

Theorem OC6 (Maximum Principle) Consider (P0) with f andg continuously differentiable. Suppose it has an interiorcontinuous solution (x(t), y(t)) 2 intX Y and defineH(t, x(t), y(t), λ(t)) = f (t, x(t), y(t)) + λ(t)g(t, x(t), y(t)). Then:

H(t, x(t), y(t), λ(t)) H(t, x(t), y(t), λ(t)), 8y(t) 2 Y 8t

Moreover, the following necessary conditions (NC0) are met:

(FOC) Hy(t, x(t), y(t), λ(t)) = 0 8t(Euler) λ(t) = Hx(t, x(t), y(t), λ(t)) 8t(State Evolution) x(t) = Hλ(t, x(t), y(t), λ(t)) 8t

(Hamilton-Jacobi-Bellman Equation) Differentiating thevalue function (V0) using Leibniz’s Rule yields:

f (t, x(t), y(t))+∂V(t, x(t))

∂t+

∂V(t, x(t))∂x

g(t, x(t), y(t)) = 0 8t.

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Optimal Control IX

Sufficiency conditions ensuring global maximum anduniqueness:

(i) X is a convex;(ii) M(t, x, λ) is strictly concave in x 2 X 8t;(iii) limt!∞ λ(t)(x(t) x(t)) 0 8x(t) associated with anadmissible control path y(t).

(Transversality Condition, TVC) Consider (P0) given inTheorem OC5. Suppose V(t, x(t)) is differentiable in x andt for t sufficiently large and that limt!∞

∂V(t,x(t))∂t = 0. Then

the pair (x(t), y(t)) satisfies the necessary conditions (NC0)and the transversality conditionlimt!∞ H(t, x(t), y(t), λ(t)) = 0.The above TVC is weaker than limt!∞ λ(t) = 0, which iscounterpart of λ(t1) = 0 in finite horizon.

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Optimal Control X

With discounting in infinite horizon, we modify (P0) to (P00)with W =

R ∞0 f (t, x(t), y(t))eρtdt.

Theorem OC7 (Maximum Principle) Consider (P00) with f andg continuously differentiable. Suppose it has an interiorcontinuous solution (x(t), y(t)) 2 intX Y and definecurrent-value Hamiltonian asH(t, x(t), y(t), λ(t)) = f (t, x(t), y(t)) + λ(t)g(t, x(t), y(t)). Then:

H(t, x(t), y(t), λ(t)) H(t, x(t), y(t), λ(t)), 8y(t) 2 Y 8t

Moreover, the following necessary and sufficiency conditionsare met:

(FOC) Hy(t, x(t), y(t), λ(t)) = 0 8t(Euler) λ(t) = ρλ(t)Hx(t, x(t), y(t), λ(t)) 8t(State Evolution) x(t) = Hλ(t, x(t), y(t), λ(t)) 8t(TVC) limt!∞ H(t, x(t), y(t), λ(t))eρt = 0.

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Optimal Control XI

Corollary OC7 (Stronger form of TVC) Consider (P00) given inTheorem OC7. Suppose in addition that(i) f is weakly monotone in (x, y) and g is weakly monotone in(t, x, y),(ii) 9m > 0 such that

gy (t, x(t), y(t)) m 8t and 8 admissible

(x(t), y(t)),(iii) 9n < ∞ such that

fy (x(t), y(t)) n 8 (x(t), y(t)).

Then, TVC can take a stronger form:

limt!∞

λ(t)x(t)eρt = 0

This stronger form can be readily generalized to themultivariate case (Kamihigashi 2001).

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Optimal Control XII

Q(t, x)=f(p, z) 2 R RKx jp f (t, x, y), z=G(t, x, y), y 2 Yg.Theorem OC8 (Existence of Solutions) Consider (P) witht1 ! ∞, satisfying:1. X ,Y 6= ∅, compact & closed-valued, and uhc;2. f and G are continuous;3. The sets of admissible (x(t), y(t)), Ω(0, x0), and Q(t, x) arenonempty for x0 2 X and 8 (t, x), and Q(t, x) is closed &convex-valued, and uhc 8 (t, x);4. For any [t1, t1+δ] and any ε > 0, 9 continuous functionΦt1δε

(t) s.t. (i) for any T 2 [0, ∞],R T

0 Φt1δε(t)dt Φ < ∞ and (ii)

kG(t, x, y)k Φt1δε(t)-εf (t, x, y) 8t 2 [t1, t1+δ], (x, y) 2 X Y ;

5. 9 a positive function φ(t) such that (i)R ∞

0 φ(t)dt φ < ∞and (ii) f (t, x, y) φ(t) 8t and 8(x, y) 2 X YThen, 9 (x(t), y(t)) 2 Ω(0, x0) that is a solution to (P) witht1 ! ∞ that W(x(t), y(t)) = W W(x0(t), y0(t)) for any(x0(t), y0(t)) 2 Ω(0, x0).

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Optimal Control XIII

In standard optimal growth models, the conditionsspecified in Theorem OC8 are all met.Example:

maxfk(t),c(t)g∞t=0

R ∞0 u(c(t))eρtdt

s.t. ˙k(t) = f (k(t)) δk(t) c(t)

(Current-value Hamiltonian)

H(k, c, λ) = u(c(t)) + λ(t)[f (k(t)) δk(t) c(t)]

(FOC)uc(c(t)) = λ(t)

(Euler)˙λ(t) = ρλ(t) λ(t)[fk(k(t)) δ]

(TVC)limt!∞

λ(t)k(t)eρt = 0.

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Concluding Remarks I

From deterministic (perfect foresight) to stochastic(rational expectations):

stochastic dynamic programming: straightforwardextension in particular with finite dimensional state spacefor shocksstochastic control: straightforward extension in particularwith finite dimensional state space shocks

From representative agent to heterogeneous agents:

heterogeneous households and inequalitiesheterogeneous firms and firm distributionaggregation validity and tractability (uniform, normal/lognormal, gamma, Pareto and Fréhet).

Introduction Fundamentals Dynamic Programming Optimal Control Concluding Remarks

Concluding Remarks II

From infinite horizon-infinite lifetime to infinitehorizon-finite lifetime overlapping generations (OG):

the prototypical setup is in discrete time (Allais 1947,Samuelson 1958), though there are continuous-time OGwith variational survival (Cass-Yarri 1966, Blanchard 1985)– generalized lifecycle modelswhile the framework is clean, tractable and useful formodeling heterogeneous agents, one must deal carefullywith the following problems:

economy with an infinity of agents (Wilson 1981) and aninfinite of dated goodsgeneric market incompleteness in dated goods trading (noArrow-Debreu securities)generic intergenerational externalities (via preferences ortechnologies)transfer from infinity (Gamow’s Hotel; classic economy;curvature) and lack of market clearing “at infinity” (Cantor)concept of optimality: forward-looking PO andsocial-discounted social welfare maximum.