Monotone Dynamic Programming:Director’s Cut

Martin Kaae Jensen∗

Brown University and DET∗∗

October 1st 2003

Abstract

By integrating dynamic programming with the theory of order pre-serving dynamical systems (Hirsch (1982), Smith (1995)) thispaper develops a new approach to stability analysis in dynamiceconomic models. The main results provide simple conditions onthe primitives of a model which imply that steady states are glob-ally stable for almost all initial conditions. If the steady state isunique or lattice methods are employed in their study; this com-pletely resolves all stability questions. The methods’ applicabilityis demonstrated through examples from investment theory, growththeory, and renewable resource economics.

Keywords: Dynamic Programming, Optimal Control Theory, Lattice Pro-gramming, Order Preserving Dynamical Systems, Monotone Semiflows,Differential Inclusions, Global Stability, Samuelson’s Correspondence Princi-ple.

JEL-classification: C61, C62, C65, D90.

Correspondence: Martin Kaae Jensen, Brown University, Department of Eco-

nomics, 64 Waterman Street, Providence, RI 02912, US. E-mail:

Martin Jensen@Brown.edu. Homepage: http://www.econ.brown.edu/fac/

Martin Jensen

∗ Thanks are due to Herakles Polemarchakis and Manuel Santos for helpful com-ments. The present version of this paper is the full working paper version. In

particular explicit constaints are included and complete proofs are included (thus

the paper’s excessive length!). This research was supported by a grant from the

Danish Social Research Council, which is gratefully acknowledged by the author.∗∗ Dynamic Economic Theory Group, University of Copenhagen.

0

1 Introduction

Dynamic optimization techniques are indispensable to those economic disci-plines which study intertemporal allocation; be it of capital, inventories, orchildren. Provided that the problem is sufficiently smooth, solutions can bedescribed by a system of differential equations and steady states are describedby the (static) system of equations which arrives when all time-derivativesare set equal to zero. Those steady states are often what the modeler isinterested in to begin with. In particular comparative dynamics - predictingsteady states’ response to changes in exogenous variables - is perhaps thesingle most important tool in macroeconomics, development economics, andother fields for which testable predictions and policy recommendations areprimary objectives.Unfortunately, analyzing a steady state and its immediate neighborhood

is meaningless unless the studied steady state is globally stable. And unlikecomputing and comparing steady states - for which the economist’s tool kitis very well equipped (cf. below) - global stability questions are not easy toresolve. More often than not to make even local stability analysis feasible,the modeler is forced to adopt a battery of simplifying assumptions whichare otherwise of no relevance for the analysis (i.e., assumptions which arenot related to stability but to the methods employed to detect stability). Inaddition, standard linearization methods do not work unless the system issufficiently well-behaved (smooth, convex, etc.). To top this off, local stabil-ity is insufficient in the first place: Even if there exists a unique and locallystable steady state this may coexist with other stable attractors such as cyclesor chaotic paths. Thus local stability does not imply global stability even inthe most well-behaved cases. As a consequence, the validity of comparativedynamics often stands and falls with a number of ad hoc assumptions: Theinitial condition must lie in a small neighborhood of the steady state, anyexogenous parameter change must be a continuous and ”sufficiently slow”function of time, any exogenous change must lead to a continuous reactionin the steady state, etc. So while methods to compare steady states are be-coming increasingly more sophisticated - and by now can handle any of theproblematic situations mentioned so far (cf. Milgrom and Roberts (1994) andfurther references below) - methods which guarantee that the conclusions arealso meaningfull in dynamic models are falling hopelessly behind.

1

This study presents an approach to stability analysis which is capable ofcompletely resolving the above discrepancy between the goal (comparingsteady states) and the way to obtain this goal (global stability analysis).The approach works in many important situations where even the most so-phisticated Lyapunov-function based methods fail (which, indeed, they oftendo even in simple macroeconomic models, e.g.). Aside from this, the ap-proach has several advantages over strategies which attempt to determine(global) stability by considering the Euler conditions, applying Pontryagin’smaximum principle, or studying the Bellman-Hamilton-Jakobi equation: (i)It is much easier to use (ii) It yields robust global conclusions, (iii) It requiresneither smoothness, interiority, or convexity assumptions, and (iv) It directlyexpresses conditions for stability in terms of the primitives of the model.For the main part of the paper, focus will be placed on a specific class

of dynamic programs marked by explicit assumptions. The notion of orderpreserving semiflows (Hirsch (1985), Smith (1995)) thus quickly leads ourattention to what we name monotone dynamic programming.1 It will beshown, roughly, that if a given problem falls within this class, then the set ofsteady states will be non-empty and globally attract solutions for almost allinitial conditions.2 In terms of applications this means that the modeler canproceed directly to the study of steady states since all other types of pathsare irrelevant from a robust long-run perspective. If there is a unique steadystate, this completely resolves all stability problems. If there are multiplesteady states, the approach in a very natural way integrates with ”monotonecomparative statics” (cf. Milgrom and Roberts (1994), Topkis (1998)) thusenabling the modeler to derive meaningful comparison results also in thissituation. Again this completely resolves the global stability problematic.In section 2 the basic problem is introduced along with various mathe-

matical concepts used throughout the paper. Section 3.1 describes the class

1Order preserving dynamical systems are also called monotone dynamical systems (infact this is the title of the monograph of Smith (1995)). The term monotone dynamicprogramming is meant to refer to this strand of literature while being consistent withthe term monotone map methods as developed by Amir et al (1991), Amir (1996), and anumber of other authors (see Becker and Boyd (1997) for further references and a survey).

2Generally, systems of the type marked by monotone programs will have periodic oreven chaotic paths coexisting with steady states. Yet, the latter are the only locally stableattractors. The probability that the initial condition is drawn such that convergence to asteady state does not occur is zero: The set of initial conditions whose solutions convergeto steady states is open and dense in the set of initial conditions.

2

of monotone dynamical programs and clarifies the assumptions. Section 3.2contains the first main result. This states that a monotone dynamic programleads to order preserving solution maps. Under a weak additional continu-ity assumption, this implies that solutions are order preserving semi-flows(Hirsch (1982), Smith (1988), and references above). Once the connectionwith the theory of order preserving semiflows has been made, a number ofremarkable results can be proved. These are contained in section 4, and themost important has already been mentioned: A monotone dynamic programwill (under some additional regularity assumptions) be generically globallystable. That is to say: For initial conditions in a dense and open set, theassociated solutions will converge to steady states. Section 5 contains vari-ous applications which supplement especially section 3.1. Finally, section 7comments on the relationship to existing literature and concludes. Proofs ofmain results are placed separately in section 6.The non-mathematically minded reader should not be put off by the

mathematical content of the paper. The methods are easily accessible andlittle expertise is necessary to understand and make use of the main results(cf. section 5). In fact most, if not all, more complex mathematical state-ments can simply be skipped in a first reading.

2 Preliminaries

2.1 The Basic Problem

Consider the standard continuous-time dynamic programming problem:

(DP)max ∞

0 e−ρtF (x(t), x(t))dt

s.t.(x(t), x(t)) ∈ T, a.e. t ∈ R+

x(0) ∈ X given

Here ρ > 0, X ⊆ RN is closed, connected, and locally convex,3 T ⊂ X × Y ,Y = {y : (x, y) ∈ T, x ∈ X}, and F : X × Y → R is a continuousfunction. It will be assumed that 0 ∈ Y (otherwise a steady state couldnever exist). In applications (DP) is sometimes specified directly (e.g., afirm’s problem), or it can arise as the reduced form of an optimal control

3A subset ofRN is locally convex if every neighborhood of any x ∈ X contains a convexneighborhood of x.

3

problem.4 For example the classical Ramsey-type optimal control problemreduces to the form (DP) when one takes: F (x(t), x(t)) = u(f(x(t))− x(t))and T = {(x, x) ∈ R+ ×R : x ≤ f(x)} (here u is the utility function and fthe production function).The section of T in a point x ∈ X defines a set-valued function, Γ : X →

2Y , where Γ(x) = {y : (x, y) ∈ T} ⊆ Y , x ∈ X. Γ thus defined will bereferred to as the section map. Clearly (x, y) ∈ T if and only if y ∈ Γ(x)(i.e., T = GraphΓ). It is often more convenient to express the constraint in(DP) in terms of Γ rather than T :

x(t) ∈ Γ(x(t)), a.e. t ≥ 0(1)

A feasible trajectory is an absolutely continuous function of time whichsatisfies the constraints in (DP); alternatively, a solution to the differentialinclusion (1). Note that one can always eliminate the explicit constaint from(DP) by considering in stead F (x, x) defined as being equal to F (x, x) when(x, x) ∈ T and equal to −∞ otherwise. In fact this way of rewriting (DP) iswell-known from growth theory where it follows from uniform boundary con-ditions on utility and technology which effectively eliminate the constraints.What is convenient about such a transformation of the problem is that nec-essary conditions for optimality may be expressed by the Euler conditions ofclassical variation calculus.5

A trajectory is optimal, denoted x∗(t, x(0)), if it is feasible and maximizesthe objective among all feasible trajectories starting from x(0). Note thatfor this to be meaningful, no feasible trajectory must be able to yield infinitevalue. The orbit in x(0) is the set O(x(0)) = {x∗(t, x(0)) : t ≥ 0} ⊂ X,

4 Consider the control constraint, x(t) = f(x(t), u(t)), u(t) ∈ U(x(t)). If one defines theset valued function Γ(x) = {f(x, u) : u ∈ U(x)}, and assumes that (i) f is continuous, (ii)U has closed, non-empty images and is continuous, then the set of admissible absolutelycontinuous state vectors coincides with the set of absolutely continuous solutions to thedifferential inclusion, x(t) ∈ Γ(x(t)), a.e. t (Frankowska (2001), theorem 2.6.). TakingT = GraphΓ and letting F be the indirect utility function, the form (DP) thus arises as anequivalent description of the original control problem. The converse, i.e. transformationof the differential inclusion x(t) ∈ Γ(x(t)), a.e. t, to an optimal control constraint (aso-called parameterization), is possible if Γ has closed and convex values (for this case seeFrankowska (2001), theorem 1.46). In particular, if solutions to a dynamic programmingproblem are interior, such a parameterization is feasible.

5See Clarke and Vinter (1985) for conditions ensuring that Euler conditions (or suitableClarkian generalizations if F is not differentiable) are valid necessary conditions.

4

where x∗(·, ·) may be any optimal trajectory (there may be more than one).The orbit is simply the union of all points which optimal trajectories at somepoint pass through. Throughout will be assumed that an optimal trajectoryexists for all x(0) ∈ X, and that O(x) is bounded in RN for all x ∈ X. Thelatter is not a serious restriction, since if one happens to study a situationwhere optimal paths are not bounded, one can transform the system to obtainboundedness (”detrend” the system).

2.2 Order Preservation

It is fruitful to view optimal trajectories as solution mappings, x∗ : R+×X →X, by letting not only t but also the initial vector x(0) vary. Thus x∗(t, x(0))denotes an optimal allocation at date t for initial condition x(0). Note thatthere will be more than one such function unless the solution to (DP) isunique for all x(0) ∈ X.Order preservation, as defined in a moment, refers to an order on RN

(and products and subsets and products of subsets of RN). The reader whowishes can think in terms of the usual order on RN (x ≥ y ⇔ x− y ∈ RN

+ ).More generally, we are allowed the flexibility of choosing an order, , withinthe class of orders which are generated by positive cones, H ⊂ RN , withnon-empty interior.6 For such orders one can meaningfully define ”strictlylarger than” by x y ⇔ x − y ∈ IntH. Note that in this paragraphand everywhere else in the main text of the paper, topological statementsrefer to the usual topology on RN . Orders generated by positive cones inRN will partially order any subset of RN . In particular (X, ) will be apartially ordered vector subspace. Moreover, the sets {y ∈ X : y x} and{y ∈ X : y x} will be closed, which implies that will be a continuousorder. All of the above will be used repeatedly in the proofs of the mainresults. The next definition is crucial:

Definition 1 An optimal solution map for (DP) is said to be order preserv-ing if x x implies that x∗(t, x) x∗(t, x ) for all t ≥ 0.The notion of order preservation will be discussed in details later on. Here

6H is a positive cone (also called a pointed cone) in RN , if H is closed, H +H ⊆ H,λH ⊆ H, all λ > 0, H ∩ −H = {0}, and H is closed. A positive cone defines an order,, on X by: x y if and only if x− y ∈ H. H has non-empty interior if IntH = ∅. The

usual order on RN is defined by the cone H = RN+ and it clearly has non-empty interior.

5

some related properties will be investigated. Due to the stationary form of(DP): x∗(t + s, x(0)) = x∗(s, x∗(t, x(0))), for all x ∈ X and all s, t ≥ 0.Moreover, x∗(0, x(0)) = x(0) and x∗(t, x(0)) is (absolutely) continuous in tby definition. From this follows that if x∗ : R+ × X → X is continuousin x(0), it will be a so-called semiflow.7 There exits a rich literature onsemiflows or semidynamical systems as such mappings are also called. Mostimportantly, the theory of cooperative differential equations (Kamke (1932),Hirsch (1982), and the concluding remarks) can be cast more abstractlyin terms of semiflows. As we shall see this is ultimately what allows itsintegration with dynamic optimization theory.

2.3 Lattice Programming

Various concepts from lattice programming are needed to define a monotoneprogram. For an exhaustive treatment of this topic the reader is referred toTopkis (1998).Recall that a partially ordered set X ⊂ RN is a lattice if it contains

the join and the meet of every two element subset, that is to say if for allx, x ∈ X: x ∨ x ≡ inf{z ∈ RN : z x and z x } ∈ X and x ∧ x ≡sup{z ∈ RN : z x and z x } ∈ X. A ⊂ X is a sublattice of X if themeet and join in X of every two elements from A are in A. An example isX = RN

+ with the usual order (which is a sublattice of RN ).

A partially ordered set X ⊂ RN is order convex if for all x, y ∈ X withx y: λx + (1 − λ)y ∈ X all λ ∈ [0, 1]. Clearly any convex set is orderconvex. X is order bounded from below (above) if there exists x0 ∈ X suchthat x x0 (x x0) for all x ∈ X.Let A : X → 2Y , where X and Y are lattices. A real valued function

G : X × Y → R is supermodular in y on A(x) if for all y, y ∈ A(x), x ∈ X:

G(x, y) +G(x, y ) ≤ G(x, y ∨ y ) +G(x, y ∧ y )(2)

If, as is often the case in applications, G(x, y) is supermodular in y on avector subspace of RN which contains X, this is sufficient. In particular,if G is twice differentiable in y on Y , it will be supermodular (in the usualorder on RN ) if and only if ∂2G

∂yi∂yj(x, y) ≥ 0, all i = j, all x ∈ X, y ∈ A(x).

7A function φ : R+ × X → X is a semi-flow if it is continuous, φ(0, x) = x, andφ(t+ s, x) = φ(s,φ(x, t)), all s, t ∈ R+, x ∈ X.

6

A function, G : X × A(X) → R, has increasing differences on (x, y) ∈X ×A(x), if for all y, y ∈ A[B], y y , B ⊂ X (A[B] = {y : x ∈ B and y ∈A(x)}), it holds that G(x, y)−G(x, y ) is a non-decreasing function of x ∈ B.If the inequality is strict whenever x x , G has strictly increasing dif-ferences. Again, differentiability of G in y simplifies matters, since then(strictly) increasing differences follows if Gyn(x, y) = Fyn(x, y − x) is (in-creasing) non-decreasing in x, n = 1, . . . , N .

3 Monotone Dynamic Programming

This section defines the concept of a monotone dynamical program. Thefirst subsection presents the assumptions and discusses these. Subsection3.2 then presents a central result which states that monotone dynamicalprograms yield order preserving solution maps. This, then, is used in thenext section to give a complete characterization of the monotone class.

3.1 Basic Assumptions and Discussion of These

Concrete examples of monotone dynamic programs can be found in section5, and the reader is encouraged to consult this if the following assumptionsshould appear too far removed from concrete applications. Note that ingrowth models x will be a vector of inputs and y, typically, investments. Thefirst assumption places basic lattice structure on (DP).

Assumption 1 Y is a lattice, X is an order convex lattice, and T is a closedsublattice of X × Y . The section map of T : Γ(x) = {y : (x, y) ∈ T} ⊆ Y ,x ∈ X, is lower hemi-continuous and compact valued.

Remark 1.1 If the section map is taken as primitive, assumption 1 is equiv-alent to the following: X and Y are lattices with X order convex and Γ iscontinuous and its values are non-empty, compact lattices.8

Assumption 1 will be satisfied in most situations of interest. Verifyingit is normally also a straight-forward affair (see section 5). More often thannot, the constraint in (DP), thus the map Γ or the set T will not bind in

8If T ⊂ X×Y is a sublattice ofX×Y , andX and Y are lattices, then Γ(x) is a sublatticeof Y (Topkis (1998), lemma 2.2.3.). Since T = GraphΓ, Γ is upper hemi-continuous if andonly if T is closed (Γ has compact values).

7

the sense that (DP) is a classical calculus of variations problem. In this caseassumption 1 only imposes a weak notion of ”periodwise compactness”. Ofcourse some such compactness is necessary in the first place to ensures oneof the overall assumptions, namely that O(x) must be bounded (cf. section2.1).The next three assumptions are those which place actual structure on the

problem.

Assumption 2 T is weakly monotone, i.e., if (x, y) ∈ T then (x , y) ∈ Twhenever x x. Moreover, T allows for free order disposal: If(x, y) ∈ T and y ∈ Y then (x, y ∧ y ) ∈ T .

The first part of assumption 2 is standard. The second part is weaker thanfree disposal in the usual sense. If is the usual order it says simply that ify is feasible from x (i.e., (x, y) ∈ T ) and y is feasible (from some x ∈ X),then the coordinatewise minimum of y and y should also be feasible from x.As will be used in the proof of theorem 1, assumption 2 has an importantimplication for the section map Γ:

Lemma 1 If T satisfies assumption 2 then the section map Γ is ascending.9

Proof: Take arbitrary x, x ∈ X, x x , and u ∈ Γ(x), v ∈ Γ(x ). By weakmonotonicity, v ∈ Γ(x ) implies v ∈ Γ(x). Since Γ(x) is a lattice, therefore,u ∧ v, u ∨ v ∈ Γ(x). Since (x , v) ∈ T and u ∈ Y it follows directly from freeorder disposability that u ∧ v ∈ Γ(x ). 2

Assumption 3 For all (x, y) ∈ T and ∆x 0, there exists ∆y 0 suchthat F (x, y) = F (x+∆x, y +∆y).

Assumption 3 expresses that F must be capable of counteracting any increasein x with an increase in y. Assume, for example, that F (x, y) is increasingin x (which is the most typical case). Consider F (x, y) ∈ R and increasex to x + ∆x. If F (x + ∆x, y) = F (x, y) then the assumption is trivially

9A set-valued map Γ : X → 2Y is ascending if for x x : Γ(x) Γ(x ), wheredenotes the strong set order (induced by ). is defined by: For U, V ∈ X, U V ifand only if for all u ∈ U , v ∈ V , u ∨ v ∈ U and u ∧ v ∈ V . The frequently used termnon-decreasing is deliberately avoided because it confuses the terminology of monotonesemiflows.

8

satisfied (pick ∆y = 0). Therefore assume that F (x + ∆x, y) > F (x, y).What the assumption then says is that it is possible to increase y in such away that the value again becomes F (x, y): If x is ”good”, y must become”bad” eventually. It takes a strong will to find an economically relevant(continuous) momentary objective function which violates this and at thesame time is compatible with existence of a solution to (DP).Let G(x, y) = F (x, y − x) and corresponding to this translation define

A : X → 2Z by: A(x) = {y : (x, y − x) ∈ T}, where Z = {y : (x, y − x) ∈T, x ∈ X}. Note that y ∈ Γ(x) if and only if y − x ∈ A(x).

Assumption 4 G(x, y) = F (x, y − x) has increasing differences on (x, y) ∈X ×A(x) and is supermodular in y ∈ A(x) for all x ∈ X.

It may be shown that G(x, y) = F (x, y−x) is supermodular in y on A(x)if and only if F (x, y) is supermodular in y on Γ(x) (see the proof of lemma3.4). As mentioned in section 2.3, if F is twice differentiable and is theusual order on RN the latter is in turn equivalent to ∂2F

∂yi∂yj(x, y) ≥ 0, all

i = j, all x ∈ X, y ∈ Y .

Definition 2 A dynamic programming problem which satisfies assumptions1-4 is called a monotone dynamic program. If, in addition, G(x, y) = F (x, y−x) has strictly increasing differences on (x, y) ∈ X×A(x), it is called a strictlymonotone dynamic program.

It should be noted that the assumptions above do not impose eithersmoothness, interiority, or convexity / concavity upon (DP). In fact, evenwithin the ”standard class” marked by such assumptions, the class of monotonedynamic programs is quite broad. The critical part is, of course, the requiredcomplementarity properties as expressed in assumption 4. For further discus-sion of the complementarity requirement involved in this type of assumptionsthe reader is referred to Topkis (1998). Another important reference is Amir(1996); one of whose results play an important role in the proof of theorem1 as explained below (and in the proof).

3.2 Monotone Programs are Order Preserving

It is now possible to state the first main theorem of this paper. Recall thata solution map, x∗ : R+ × X → X, is order preserving if it verifies the

9

following global comparative statics requirement: For all x(0), y(0) ∈ X,with x(0) y(0) it holds that: x∗(t, x(0)) x∗(t, y(0)) for all t ≥ 0 (cf.definition 1).

Theorem 1 A monotone dynamic program has at least one order preservingsolution map x∗ : R+ × X → X. If the program is strictly monotone thenevery solution map will be order preserving.

The complete proof is placed in section 6.1. There are three main steps.First, an indexed sequence, n = 1, 2, 3, . . ., of discrete-time approximationsto (DP) is constructed (cf. (8) and (9)). As n → ∞, the discrete-time ap-proximations’ solutions converge to those of (DP) (see lemmas 2.1, 2.3 and2.4 for a precise statement, in particular concerning the type of convergenceand the properties this leads to). The second and most important step inthe proof is an induction argument which extends the properties for n = 1to the finer and finer approximations, n = 2, 3, . . .. It is easily shown thatassumptions 1-4 ensure that the coarsest approximation (n = 1) has a super-modular value function. The reduction step then consists in showing that ifthe n’th discrete time approximation has a supermodular value function, sodoes the n + 1’th (in fact various other important properties of n are alsoshown to carry over, but supermodularity is the crucial one). Lemmas 3.2and 3.4 are central to this step; indeed they are the most important resultsin the entire proof. The final step finishes the proof by showing that if so-lutions to every discrete approximation are order preserving, then this mustalso hold in the limit, i.e., for solutions to (DP). It deserves mentioning thatfor the discrete time approximations, a result due to Amir (1996) plays animportant role. In fact a result similar to theorem 1 for discrete-time dy-namic programming problems is an immediate consequence of Amir (1996),theorem 1 (iii). For further remarks on Amir (1996) and related results, seethe concluding remarks.

4 Order Preserving Dynamics

To sum up, in section 2.2 was defined the notion of order preservationfor solution maps to (DP). Intuitively a solution map, x∗ : R+ × X → X,is order preserving if an increase in the initial condition does not lead to adecrease in the optimal allocation at any future date. Section 2 described

10

an important order preserving class, namely the class of monotone dynamicprograms.10 In this section we shall draw heavily on mathematical results byHirsch, Smith, Thieme and others. For an exhaustive treatment of the the-ory of order preserving / monotone dynamical systems the reader is referredto the monograph by Smith (1995). More precisely, we shall here use theabstract semiflow approach in this literature as it is described in e.g. Smith(1995), chapter 1.

A set Z ⊆ X is invariant for the solution map x∗ : R+ × X → X ifx(0) ∈ Z implies that x∗(t, x(0)) ∈ Z for all t ≥ 0. Z is order boundedfrom below (above) if there exists z0 ∈ Z such that z z0 (z z0) forall z ∈ Z (cf. section 2.3). Establishing that a dynamic program hasan invariant order bounded subset is a standard exercise in dynamic eco-nomics. Take as an example the Ramsey problem (see section 2.1) whereT = {(x, y) : x ≥ 0 and y ∈] −∞, f(x)]}. If there exists x ∈ R+ such thatf(x)− x < 0, all x ≥ x, then it is clear that x∗(t, x(0)) > x cannot hold butfor a finite time span for any x(0) ∈ X. From a long-run dynamic perspec-tive there is therefore no loss of generality in restricting X to Z = [0, x]. Zthen is an invariant subset of the Ramsey problem and it is order boundedfrom below as well as above. Our first result and its corollary provide a firstexpression of just how formidable a tool order preservation is.

Theorem 2 An order preserving solution map, x∗(t, x), has a steady stateon any compact, invariant subset Z ⊆ X which is order bounded either fromabove or from below.

Proof: A slightly more informative result will be shown first (this is stated asa corollary below). Since x∗(t+ s, x) = x∗(t, x∗(s, x)), x∗ will be a monotonemap if and only if x∗(t, x∗(s, x)) x∗(t, x), all s, t ≥ 0. If it is there-fore assumed that x∗(s, (x(0)) x(0), all s ≥ 0, order preservation im-plies that x∗(t, x∗(s, x(0))) x∗(t, x(0)) for all t ≥ 0 and s ≥ 0. Butthis means exactly that x∗(t, x(0)) is monotonically non-decreasing. A sim-ilar argument implies that if x∗(t, x(0)) x(0), all t, then x∗ must bemonotonically non-increasing. Since the orbit in x(0) has compact closure,

10For another approach see the concluding remarks on the case with a continuously dif-ferentiable policy function. For a survey of results on such cooperative differential equationssee Smith (1988).

11

every sequence has a convergent subsequence. If x∗ is monotonically non-decreasing or non-increasing (monotonic), then in fact every subsequencemust converge to the same limit point. Indeed, assume that this is not thecase, i.e., x∗(α(t), x(0)) → x1 and x∗(β(t), x(0)) → x2 as t → ∞, whereα, β : R+ → R+ and are injective. By monotonicity, for every t there existst such that x∗(α(t), x(0)) x∗(β(t ), x(0)). This clearly implies x1 x2.But the same argument implies x2 x1, and by antisymmetry of thenx1 = x2. That every subsequence converges to the same point implies thatthe sequence is Cauchy, hence it converges. That the convergence point is inX is trivial since O(x) ⊆ X for all x ∈ X and X is closed. Now assumethat there exists z ∈ Z such that z z for all z ∈ Z (a minimal element).Clearly then x∗(t, z) z for all t. Hence the above results immediately im-ply the conclusion: x∗(t, z) converges to a steady state which therefore, inparticular, exists. An analogous argument applies if Z contains a maximalelement. 2

It should be stated clearly that an order preserving optimal solution need notbemonotonic: It does not follow from order preservation that x∗(t+s, x(0))x∗(t, x(0)) for all s ≥ t ≥ 0 (the non-decreasing case) or 0 ≤ s ≤ t (the non-increasing case). It is often possible, however, to use order preservation toprove monotonicity. From an economic viewpoint, monotonicity is in itself aninteresting feature of a dynamic problem. From a global stability perspectiveno less so, since monotonic solutions always converge to steady states. Thefollowing corollary to theorem 2 establishes an explicit connection betweenmonotonic and order preserving mappings.

Corollary 2.1 Assume that a solution, x∗(t, x), is order preserving and thatfor x(0) ∈ X either x∗(t, x(0)) x(0), for all t ≥ 0; or x∗(t, x(0)) x(0), forall t ≥ 0. Then x∗(t, x(0)) is monotonic, and as a consequence x∗(t, x(0))→x as t→∞, where x is a steady state.

The corollary says, among other things, that for a monotone dynamic pro-gram monotonicity can be established simply by showing that the solutionfrom some x(0) ∈ X: x∗(t, x(0)), t ≥ 0, will ”take off” from x(0) in the sensethat it is contained in one of the sets {z : z x(0)} or {z : z x(0)}. Ifthis is so, it will be monotone, in particular it will converge to a steady state.Taking x(0) as the lower or upper bound of an invariant set, the intuition

12

behind theorem 2 should then also be clear.11

To progress, it is necessary to focus on solutions which have slightlystronger continuity properties (see section 2.2 for further discussion):

Assumption 5 The optimal solution map, x∗(t, x), is continuous in x, andconsequently x∗ : R+ ×X → X is a semiflow.

In most situations of interest, continuity is immediate. For example, if thereis a unique solution for all x(0), then one can show that the (unique) solutionmap will be continuous in x and therefore a semi-flow. A special case ofthis is when the unique solution from x(0) can be described by a systemof ”weak” differential equations: x(t) = g(x(t)), a.e. t ≥ 0, g continuous,x(0) given. g might in this case be called a ”weak” policy function, reservingpolicy function for the case where x(t) = g(x(t)), for all t ≥ 0. In either case,if g is Lipschitz continuous then both uniqueness and the semi-flow propertyfollows. For more remarks on continuity see section 5.1 (and for a variousgeneral results including the continuous selection results mentioned above,Aubin and Frankowska (1990)).A set O ⊂ X is said to be locally attracting if there exists an open set U ⊆

X containing O and for all x(0) ∈ U , dH(x∗(t, x(0)), O)→ 0 as t→∞ (heredH is the Hausdorff distance between the set O and the point x

∗(t, x(0))).

Theorem 3 An order preserving solution semiflow, x∗ : R+×X → X, can-not have a non-trivial stable periodic cycle O ⊂ X. In particular a monotonedynamic program which satisfies assumption 5 has no other locally attractingperiodic solutions than the steady states.12

Proof: Since O(x) has compact closure, all x ∈ X, Smith (1995), theorem2.2., implies the conclusion provided that the following can be shown: Forx ∈ X and any neighborhood V of x (not necessarily an open neighborhood),there exists a neighborhood U of x with U ⊆ V , such that either u x, forall u ∈ U ; or u x, for all u ∈ U . To prove that this is indeed satisfied,11In fact one can show much more by persuing this line of though. Among other things

the stable manifolds for unique steady states can be characterized.12x∗(t, q), q ∈ X is a cycle of prime period T if for all t ≥ 0: x∗(t+T, q) = x∗(t, q) (and

this is not true after any decrease in T ). A cycle of (prime) period 0 is a called trivial ora steady state.

13

note that is generated by a positive cone H in RN with non-empty inte-rior. Since X is locally convex and directed by H, this implies that for anyx ∈ X there exists a convex neighborhood, V , of x containing an element z,for which either z x or z ≺≺ x (if x is an interior point, both will hold).Since X is Hausdorff, disjoint neighborhoods around x and z exist, and it isthen clear that the above property follows since these neighborhoods can bechosen arbitrarily small (X is locally convex). 2

The next lemma is closely related to the previous result:

Lemma 2 If for x(0) ∈ X it holds that x∗(T, x(0)) x(0) for some T > 0,then x∗(t, x(0)) converges to a periodic cycle (possibly a trivial one). If forx(0) ∈ X it holds that x∗(t, x(0)) ≥ x(0) for t in an open interval in R+,then x∗(t, x(0)) converges to a steady state.

Proof: The results in the lemma are standard in the theory of order preserv-ing semiflows. For proofs see e.g. Smith (1995), chapter 1.2. 2

The previous results are often sufficient to prove that steady states are glob-ally stable. However, under an extra assumption, the stability problematiccan be completely resolved.

Assumption 6 If x∗(t, x) x∗(t, y) for all t ∈ R+ and x∗(t, x) x∗(t, y)

for infinitely many t ∈ R+, then x∗(T, x) x∗(T, y) for at least one T ∈

R+.

It can be shown that assumption 6 will be satisfied for interior solutions if(DP) is strictly monotone and F is twice differentiable. Considering assump-tion 6 it is clear, however, that these conditions are not necessary.13 Underassumption 6, order preserving semi-flows will be what is known as stronglyorder preserving semiflows. Without going into any further details, it isnoteworthy from a mathematical perspective that this stronger property isimplied essentially by smoothness for strictly monotone dynamic programs.

13Finding necessary and sufficient conditions would be very interesting; but unfortu-nately it is a difficult problematic which is beyond the scope of this paper. Loosely speak-ing assumption 6 expresses a robustness property of (DP): If two solutions are orderedfor some pair of initial conditions, then they should eventually be ordered also for initialconditions arbitrarily close to the original ones.

14

A dynamic program is said to be generically globally stable (GGS) if thereexists a solution map x∗ : R+ × X → X, such that for all x(0) ∈ Q ⊆ X,Q open and dense in X, the solution converges to a steady state. If anysolution map has this property, the dynamic program is said to be stronglygenerically globally stable (strongly GGS). Of course the definitions coincideif (DP) has a unique solution on X. The next result is framed in terms ofmonotone dynamic programs; but as is clear from the proof, it can easily beadapted to arbitrary strongly order preserving solution semiflows of (DP).

Theorem 4 (Global Generic Stability) Let assumptions 5-6 be satisfiedand assume in addition that the set of steady states, E, is at most countable.Then any monotone dynamic program is generically globally stable, and anystrictly monotone dynamic program is strongly generically globally stable.

Proof: Section 6.2. 2

It is of interest to ask what can and cannot happen if the conditions oftheorem 4 are satisfied except that the number of steady states is uncount-able. A situation which violates this is for systems which exhibit long-runhysteresis, i.e., where the steady state manifold is an open set. In this casethe following corollary to the proof or theorem 4 applies:

Corollary 4.1 Assume that the conditions of theorem 4 are met, except thatthe set of steady states, E, is uncountable (e.g. contains an open set). ThendH(x

∗(t, x(0)), D) → 0 as t → ∞, all x(0) ∈ Q, where D is a connectedsubset of E.

Proof: Take D = ω(x(0)), x(0) ∈ Q, in the proof of theorem 4. 2

The corollary says that convergence into a connected subset of the equi-librium manifold E is generic. This does not rule out the possibility thatoptimal trajectories display complex and even chaotic behavior at D. A par-allel (but very different) phenomenon is known from turnpike theory wherethe von Neumann facet may contain complex paths, and so convergence tothe von Neumann facet does not imply global stability (see Nishimura andYano (1995)).

15

5 Applications

In this section various examples of monotone dynamic programs will beconsidered. In every case below existence of a steady state will follow if”eventual decreasing returns” or similar assumptions are adopted (theorem2). Monotonicity, and in particular convergence to a steady state, may bestudied by use of corollary 2.1. Finally, generic global stability can be estab-lished by use of theorem 4. While the examples shall mainly be concernedwith establishing that a given program is (strictly) monotone in the sense ofdefinition 2; various comments on assumptions 5 and 6 which are helpful inapplications will be included.

5.1 One-Sector Models

Consider the following general one-sector growth problem:

max ∞0 e−ρtu(c(t), k(t))dt

s.t.c(t) ≤ g(k(t), k(t)), t ≥ 0k(0) ∈ R+ given

(3)

Here u : R2+ → R and g : R+ × R → R are continuous functions (al-

though we hasten to assume differentiability in order to simplify affairs). Ifg(k(t), k(t)) = f(k(t)) − k(t) this is a one-sector growth model with wealtheffects. The presense of wealth-effects is frequently encountered in renew-able resource economics; but they may of course arrive in many other sit-uations as well.14 If, in addition one specializes to, u(c(t), k(t)) = u(c(t))(no wealth effects), this yields the Ramsey / neo-classical one-sector growthmodel (f concave); and the non-classical growth model (cf. Skiba (1978))if f is convex-concave. If the absense of wealth effects is coupled with

g(k(t), k) = f(k(t))− k(t)−φ( k(t)k(t)) or g(k(t), k) = f(k(t))− k(t)−φ(k(t)) one

gets costs-of-adjustment models (see for example Hopenheyn and Prescott(1992) albeit in the uncertainty, discrete-time case).To be sure, even if convexity, differentiability, and interiority are assumed,

(3) may yield complex dynamic behavior. Thus in order for comparative

14For discussion and references see Becker and Boyd (1997), section 7.4.1. Becker andBoyd also contains a treatment of monotonicity/supermodularity very close to the onehere (albeit in discrete time). Two other imporant references are Amir et al (1991) andHopenheyn and Prescott (1992).

16

dynamics to be a meaningful exercise; a throughout stability analysis is ab-solutely mandatory.Assume that u is increasing in c, and consider the reduced form utility

function: F (k, k) = u(g(k, k), k). Assumptions 1-2 will be satisfied if gis continuous, increasing in k and decreasing in k. Assumption 3 will besatisfied if, for example, u(c, k) is nondecreasing in k and g is decreasing in k(there are various other possibilities; but the aforementioned would seem tobe the economically most reasonable). F will automatically be supermodularin k due to the single dimension. Let G(k, k) = F (k, k − k) (= u(g(k, k −k), k)). If u and g (and hence G) are twice differentiable it may be checkedthat sufficient conditions for (3) to be a monotone dynamic program are:D2cku,D

2ccu,D

2kkg ≤ 0, and D2

kkg ≥ 0. If either one of these inequalities is

strict, moreover, (3) will be strictly monotone. These conditions say in turnthe u must be concave in c, that g must be concave in k, that k and k mustbe complements in g (g supermodular in (k, k)), and finally, that c and kmust be substitutes in u (u submodular).In models with a single dimension, assumption 6 follows trivially from

assumption 5. Thus, any solution to (3) which varies continuously in theinitial condition k(0) will display generic global convergence to a steady stateby theorem 4 (assuming in addition that (3) has countably many steadystates, otherwise use corollary 4.1). This illustrates the usability of the mainresults of this paper.15

5.2 A Two-sector Example

Take N = 2, u : R2 → R, f1, f 2 : R2+ → R+, and consider the reduced form

utility function F (x, y) = F (x1, x2, y1, y2) = u(f1(x1, x2)−y1, f2(x1, x2)−y2).

Take F and f i, i = 1, 2, twice differentiable. Let be the usual order. F is

15For other types of results which can be derived from the monotone literature seethe references above, in particular Amir et al (1991) and, for a survey, Becker and Boyd(1997). Note also that Hopenhayn and Prescott (1992) contain global stability resultsunder a certain ”monotone mixing condition”. Since this condition implies in particularthat the steady state (or rather: invariant distribution) is unique (theorem 2), and since - atleast with non-convexities - uniqueness generally does not hold; a direct relationship withthe generic global stability results in this paper does not seem to exist. Since Hopenheynand Prescott (1992) do not employ the theory of order preserving dynamical systems; theexplanation for this would seem to be obvious.

17

supermodular in y iff D212u(·, ·) ≥ 0.16 Now consider F (x, y − x). This has

increasing differences if Fyixj(x, y−x) ≥ 0 all i, j (and x, y). Pick i, j ∈ {1, 2}.If u is concave in each coordinate and D2uij = 0 (additive case) this istrivially satisfied and the model will yield a monotone dynamic program.Consider therefore u coordinatewise concave andD2

iju > 0 (wlg. fromD2iju ≥

0 which is necessary). Then D2y1xjF (x, y − x) ≥ 0, j = 1, 2 if and only if:

−D211u(x, y − x)

D212u(x, y − x)

≥ max{ Dx1f2(x)

1 +Dx1f1(x)

,1 +Dx2f

2(x)

Dx2f1(x)

}(4)

while F 2y2xjF (x, y − x) ≥ 0, j = 1, 2 iff:

−D222u(x, y − x)

D212u(x, y − x)

≥ max{ Dx2f1(x)

1 +Dx2f2(x)

,1 +Dx1f

1(x)

Dx1f2(x)

}(5)

Note the symmetry such that if for example max{ Dx1f2(x)

1+Dx1f1(x),1+Dx2f

2(x)

Dx2f1(x)

} =Dx1f

2(x)

1+Dx1f1(x)

then max{ Dx2f1(x)

1+Dx2f2(x),1+Dx1f

1(x)

Dx1f2(x)

} = Dx2f1(x)

1+Dx2f2(x). Also u must clearly

be coordinatewise concave.As in the previous example, assumption 5 will now rule out the possibility

of stable periodic cycles and assumption 6 further lead to generic globalstability for this model.

5.3 Investment Theory

A true classic in applied microeconomics is the following optimal investmentproblem:

max ∞0 e−rt[pf(k(t))− N

n=1Cn(kn(t), kn(t))]dt

s.t.k(t) ≥ 0, t ≥ 0k(0) ∈ R+ given

(6)

Here r > 0 is the constant rate of interest, p ∈ RN++ a vector of present

value prices, f : RN+ → RN

+ a production function and Cn : R+ ×R → R+

are capital cost functions. The production function f should be interpretedas a reduced form ”output function” which has already selected an input oflabor and thus in particular has ”chosen” an output vector among different

16If D212u(·, ·) ≤ 0 everywhere; the order generated by the cone R+ ×R− can be used

which yields alternative conditions to the four below.

18

feasible output vectors given k. (6) is frequently studied in, for example,labor economics (of course with labor as an explicit entry!), real businesscycle theory, and - not surprisingly - managerial economics.17

The trivial case is when Cn((k(t), k(t)) = pk(t). This simply means thatthe firm purchases capital equipment at a complete and competitive mar-ket and that capital installment and adjustment costs are absent. Moreinteresting is when adjustment costs are present because this adds ”iner-tia” to the firm’s decisions. Lucas (1967) proposes the form: C(k(t), k(t)) =Nn=1Cn(kn(t)) (note that the price p is suppressed here as in the formulation

above). (6) is but a slight generalization of this. Generally C can be seenas a sum of purchase costs and installment/adjustment costs (Lucas (1967)suggests an alternative interpretation in terms of imperfect markets).Assumptions 1-3 are satisfied if, for example, f is increasing in each

coordinate, Cn is strictly increasing in kn and decreasing in kn. Theseare hardly controversial restrictions. As for assumption 4 let F (x, y) =pf(x) − N

n=1Cn(kn(t), kn(t)). F will be supermodular in y if and onlyif N

n=1Cn(xn, yn) is submodular in y. But this is automatically satis-fied due to the additive structure. The requirement that F (x, y − x) has(strictly) increasing differences is (also) completely independent of the pro-duction function f . Indeed, this will be the case if and only if Cn(xn, yn−xn)−Cn(xn, yn − xn) is (strictly) increasing in xn whenever yn > yn (note thathere we take as the usual order on RN). If Cn is twice differentiablethis will be satisfied if D2

22Cn(xn, zn) ≥ D221Cn(xn, zn), n,m = 1, . . . , N ,

x ∈ RN+ , zn = yn − xn ∈ RN . If Cn(kn(t), kn(t)) = Cn(kn(t)) + pnkn(t)

this is equivalent to the requirement that Cn is convex (and if it is strictlyconvex for all n, the program will be strictly monotone). If, on the other

hand, Cn(kn(t), kn(t)) = Cn(kn(t)kn(t)

) + pnkn (so installment costs depend on

the relative increase in capital, which would seem equally realistic); this willbe satisfied simply provided that the cost functions Cn(z) are increasing (asassumed above). If Cn, n = 1, . . . , N , are strictly increasing, the programwill be strictly monotone. None of the above depend on the form of f . Inparticular the firm may face fixed costs or other types of non-convexities inproduction.18

17For a discrete-time, uncertainty case treatment see (again!) Hopenhayn and Prescott(1992) (and references therein).18Whether a solution exists is, of course, a different question. Likewise some type of

19

Again, assumptions 5-6 will imply global generic stability of the steadystate(s). One can actually show that assumption 5 is sufficient for this con-clusion, although this will not be persued here.

6 Proofs

6.1 Proof of Theorem 1

Let T (x0) denote the set of absolutely continuous functions which satisfy theconstraint in (DP). Let H(x) = ∞

0 e−ρtF (x(t), x(t))dt, for x ∈ T (x0).

Lemma 2.1 For any x(0) ∈ X and any function x ∈ T (x0):

H(x) = limn→∞

i=0, 1n, 2n,...

1

ne−ρiF (x(i), n(x(i+ n−1)− x(i)))(7)

Proof: By Lebesgue’s majorant convergence theorem (e.g., Howes (1995),ch. 8): limn→∞ X fndµ = X fdµ, when lim fn(x) = f(x) for all x ∈ X, andthere exists a measurable function g on X with X | g | dµ <∞ and | fn(x) |≤g(x) for each x ∈ X. Let fn(i) = e−ρiF (x(i), n(x(i+ n−1)− x(i))), and notethat the summation on the r.h.s. of the claim is simply the Lebesgue measureof the simple function fn (

1nis the measure of any point i). A majorant, g,

exists because F may be assumed to be a bounded function without loss of gen-erality (see the proof of lemma 2.2 below). Finally, limn→∞ fn = f , since ineach coordinate, m say, limn→∞ n(xm(i+n−1)−x(i)) = limδ→0

xm(i+δ)−x(i))δ

=∂x(i)∂i. 2

For any n ∈ N consider the discrete-time problem:

maxi=0, 1

n, 2n,...

1

ne−ρiF (x(i), n(x(i+ n−1)− x(i)))

s.t.(x(i), n(x(i+ n−1)− x(i))) ∈ T, all ix(0) ∈ X given

(8)

Let Dn = {(x, y) : (x, n(y − x)) ∈ T} ⊆ X × Zn, where Zn = {y : (x, n(y −x)) ∈ T and x ∈ X}. Corresponding to this, define the section in a point”eventual decreasing returns” is needed for a steady state to exist (cf. the previous section).

20

x ∈ X, as An(x) = {y : (x, n(y − x)) ∈ T} ⊂ Zn. There exists a closed setYn ⊆ RN which is a lattice and contains both Zn and X. It is convenientto consider An as a set-valued mapping from Yn into subsets of Yn. SinceΓ(x) is compact valued and non-empty for all x ∈ X, it follows straight fromthe definition that An(x) will be compact valued and non-empty for all x ∈X. Any sequence (ym)

∞m=1 such that ym ∈ An(xm) for a sequence (xm)∞m=1,

xm ∈ X, maps one-to-one into a sequence (zm)∞m=1 such that zm ∈ Γ(xm) forall m. From this follows that An will be continuous since Γ is continuous byassumption 1. Finally, note that x(0) ∈ Yn by construction.Let Gn(x(i), x(i + n

−1)) = n−1F (x(i), n(x(i + n−1) − x(i))). By the bi-jection i→ in, (8) is isomorphic to:

max∞

j=0

(e−ρn )jGn(x(j), x(j + 1))

s.t.x(j + 1) ∈ An(x(j)) ⊆ Yn, j = 0, 1, 2, . . .x(0) ∈ Yn given

(9)

which is a standard discrete-time dynamic programming problem (note thatfor n = 1, 2, 3, . . .: e−

ρn < 1⇔ ρ > 0).

By the principle of dynamic programming, the value function to (9), vn :Yn → R, defines solutions through a policy correspondence, Πn : Yn → 2Yn:

Πn(x) = {y ∈ An(x) : vn(x) = Gn(x, y) + e−ρnvn(y)}(10)

A policy function is a selection from Πn, i.e., a function gn : Yn → Yn suchthat gn(x) ∈ Πn(x) for all x ∈ Yn. Now define:

fn(x) ≡ n[gn(x)− x](11)

And the correspondence induced from Πn, Πn : Y → 2Y :

Πn(x) = {z : z = n[y − x], y ∈ Πn(x)} ⊂ Y(12)

Lemma 2.2 vn is continuous and Πn is compact-valued and u.h.c.

Proof: It was shown that An : Yn → 2Yn is a non-empty, compact-valued, andcontinuous correspondence. Since the orbit O(x(0)) is assumed to be boundedand F is continuous, there is no loss of generality in assuming that F isbounded from above (simply restrict T to a compact set using that for O(x(0))

21

to be bounded, x(t) must lay in a compact subset of Y for almost every t).It is now standard to show that vn is continuous and that Πn is compact-valued and upper hemi-continuous. Clearly, Πn preserves compactness. Thelast claim then follows because a compact valued correspondence is u.h.c. ifand only if its graph is closed, and closedness again readily follows by thedefinition of Πn. 2

Lemma 2.3 Let (fn)∞n=1 be a sequence of selections from Πn. Then (fn)

∞n=1

has a subsequence which converges pointwise to a function f : RN → RN .

Proof: A selection from Πn is a function such that fn(x) ∈ Πn(x) for all x.Moreover, (x, fn(x)) ∈ T for all n and x by (11). As used on several occasionsabove, T may be restricted to a compact set since the orbit of x(0) is bounded.Without loss of generality, the statement of the lemma thus concerns the spaceof upper hemi-continuous, compact and non-empty set valued functions froma compact set X ⊂ RN into 2Y , Y ⊆ Q ⊂ RN , where Q is compact. Call thisset P. Equip Y with the usual topology, 2Y with the product topology (i.e.,Z ∈ 2Y is open if and only if it is coordinatewise open in Y ), and P withthe topology of pointwise convergence. A standard result in topology statesthat P is pointwise-compact if (and only if, since 2Y is Hausdorff) P is apointwise closed subset of the product space (2Y )X , and for all x ∈ X, theset P[x] ≡ {y : y = H(x), H ∈ P, x ∈ X} ⊆ 2Y has a compact closure (cf.Kelley (1955), chapter 7). The graph of any H ∈ P is closed by upper hemi-continuity of Πn. Therefore P is pointwise closed in the relativized producttopology (which is the same as the topology of pointwise convergence). By theTychonoff product theorem in order to show that P[x] has compact closure in2Y it suffices to show that P[x] is coordinatewise bounded in Y . This is thecase if there exists B > 0 such that for all x ∈ X, fn ∈ P: fn(x) ≤ B.But since fn(x) ∈ Y ⊂ Q and Q, w.l.g. is compact (see above), hencebounded, this is immediate. Now consider the sequence (Πn)

∞n=1 as defined

prior to the lemma. Since this rests in a pointwise-compact set it has anaccumulation point, i.e., there exists an injection, m : N → N, such that(Γm(n))

∞n=1 converges in the relativized product topology on (2

Y )X . From thisresult, the statement of the lemma is trivial because the sequence of functions,(fn)

∞n=1, is merely a selection from (Πn)

∞n=1. 2

22

Lemma 2.4 Any function f determined from lemma 2.3 is a ”weak” policyfunction for the problem (DP). That is to say, any absolutely continuoustrajectory satisfying:

x∗(t) = x(0) +t

0f(x(τ)) dτ, t ≥ 0(13)

will solve (DP).

Proof: By definition of fn: n−1 fn(x(j)) = gn(x(j)) − x(j), where gn is

a selection from Πn and j = 0, 1, 2, . . .. A solution to (9) is a sequencex(0), x(1), x(2), . . ., such that x(j+1) = gn(x(j)) for all j. Since (8) and (9)are isomorphic under the mapping j → ni, this may also be expressed as asolution to (8): (x(i))i∈In, In =

0n, 1n, 2n, . . ., (x(i+ 1

n) = gn(x(i)), all i). Take

T ∈ N and note that:

i= 0n, 1n,...,T

n

1

nfn(x(i)) =

i= 0n, 1n,...,T

n

[gn(x(i))− x(i)] = gT+1n (x(0))− x(0)(14)

where gT+1n (x(0)) = gTn (x(1n)) = . . ., i.e., the T + 1 times composite of x(0).

Next, extend gTn (x) to a continuous function in T ∈ R+, by for all a ∈ N0

letting: gTn (x) := (1−(T−a))gan(x)+(T−a)ga+1n (x), when T ∈ [a, a+1]. Also(this is in fact equivalent), extend gn(x(i)), i ∈ In, to i ∈ R+ by connectingeach pair of subsequent points with a straight line. Finally extend fn(x(i)) toi ∈ R+ by connecting subsequent point with line segments. These extensionswill be denoted by the original function name. It is now possible to consider(14) for a fixed endpoint, t ∈ R+:

i= 0n, 1n,...,t

1

nfn(x(i)) =

i= 0n, 1n,...,t

[gn(x(i))− x(i)] = gtn+1n (x(0))− x(0)(15)

Note that gtn+1n (x(0)) resides in a compact set for all t ∈ R+ since sodoes the original function for tn = 0, 1, 2, 3, . . .. From the sequence (15),n = 1, 2, 3, . . ., there consequently exists a subsequence such that the right-hand-side converges. Since by lemma 2.3, fn converges pointwise (for yetanother subsequence), Lebesgue’s majorant convergence theorem applies co-ordinatewise (the details are omitted, see the proof of lemma 2.1 for a similar

23

argument), leading to the following:

x(0) +t

0f(x(τ))dτ = lim

n→∞ gnt+1n (x(0))(16)

Denote the left-hand-side of (16) by x∗(t), t ∈ R+. Obviously this defines afunction, and in fact this is the solution to (DP). By the fundamental theoremof calculus for the Lebesgue integral, x∗(t) will be absolutely continuous on[0, b], b <∞, if and only if f ◦ x is Lebesgue integrable on [0, b]. That f ◦ xis in fact Lebesgue measurable follows from the fact that it is the limit of asequence of Lebesgue-measurable functions (for every n, the extension fn ◦ xis continuous in t). Thus x∗(t) is absolutely continuous and satisfies (13).Below it is referred to as the ”solution candidate”.By an argument similar to the previous one it may in addition be shown

that,f(x∗(t)) = lim

n→∞n(gnt+1n (x(0))− gntn (x(0))(17)

The maximal objective value of the problem (8) is:19

vn(x(0)) =i= 0

n, 1n,...

1

n(e−ρ)iF (gnin (x(0)), n(g

ni+1n (x(0))− gnin (x(0)))(18)

Note that this statement is independent of the particular selection from Πn.Also note that limn→∞ gnt+1n (x(0)) = limn→∞ gntn (x(0)) because gn(x)→ x asn→∞, all x ∈ X, i.e., gn converges pointwise to the identity map. Indeed,if this were not the case, fn(x) = n[gn(x)− x)] could not converge pointwisefor all x ∈ X, which contradicts lemma 2.3.Using the method of proof of lemma 2.1 together with the previous obser-

vations and the fact that F is continuous, we get that:

limn→∞ vn(x(0)) = H(x

∗)(19)

where x∗ = (x∗(t))t≥0 is the solution candidate determined above (note thatsince Πn is not necessarily a singleton there will generally be more than onesuch candidate). Note also that H was defined as an extended functional(i.e., taking values in the extended reals, R∗), so the previous statement isalways well-defined. However, it is not hard to show that there exists B > 0

19Note that this is also valid for the extension of gn.

24

such that | vn(x(0)) |≤ B for all n, and so limn→∞ vn(x(0)) is finite (usethat for Yn, n = 1, 2, 3, . . ., may be restricted to a compact set). Denote thevalue function of (DP) by v(x(0)). If v(x(0)) > limn→∞ vn(x(0)), lemma 2.1and continuity of F would imply that one could obtain an objective value of(8) which is strictly larger than vn(x(0)) for n sufficiently large. This is acontradiction. Nor can v(x(0)) < limn→∞ vn(x(0)) (this is trivial since x∗ isa feasible trajectory for (DP)). This finishes the proof of the lemma, since x∗

is then a solution to (DP). 2

A vector-valued function such as gn is said to be non-decreasing if x, x ,x x implies gn(x) gn(x ). The problem (8) is said to be monotoneif x(0), x (0) ∈ X, implies x(m

n) x (m

n), m = 1, 2, 3, . . . (here, of course,

x (·) denotes the solution associated with x (0)). If gn generates the solu-tion for x(0) ∈ X, then clearly the solution is monotone if and only if gnis non-decreasing. Let D = {(x, y) : (x, y − x) ∈ T} ⊆ X × Y , whereY = {y : (x, y − x) ∈ T and x ∈ X}. The projection in a point x ∈ X isdenoted A(x) = {y : (x, y − x) ∈ T} ⊂ Y . Clearly A : Y1 → 2Y1 without lossof generality.The next lemmas are devoted to proving the following:

Lemma 3 Assume that G(x, y) has increasing differences in (x, y) ∈ X ×A(x) and is supermodular in y on A(x), that A is monotonically non-decreasingin x, and that A(x) is a sublattice of X for each x ∈ X. Then for everyn ∈ N the minimum and maximum policy selections of (8), gn and gn, arenon-decreasing. If, in addition, G(x, y) has strictly increasing differencesthen every policy selection will be non-decreasing, n ∈ N.

The proof progresses through a number of sublemmas. For Z ⊂ RN let L(Z)denote the set of non-empty sublattices of Z. The first lemma shows thatAn : Yn → L(Yn) for all n ∈ N (An is a lattice correspondence).

Lemma 3.1 If T ∈ L(X ×X), then for n = 1, 2, 3, . . .: An(x) ∈ L(Yn), allx ∈ Yn.

Proof: Take y, y ∈ An(x). By definition there then exists x ∈ X suchthat (x, n(y − x)) ∈ T and (x, n(y − x)) ∈ T . Since T is a sublattice,(x, n(y−x))∨ (x, n(y −x)) ∈ T . The order on T is the product order, henceit is easily verified that (x, n(y − x)) ∨ (x, n(y − x)) = (x, n(y∗ − x)) where

25

y∗ = y ∨ y . But this means exactly that y ∨ y ∈ An(x). The proof for themeet is the same. That An is non-empty for all x ∈ Yn is shown below. 2

Lemma 3.2 If the function, G(x, y) = F (x, y − x) has increasing differ-ences on (x, y) ∈ X × A(x), then Gn(x, y) = F (x, n(y − x)) has increasingdifferences on (x, y) ∈ X × An(x) for all n = 1, 2, 3, . . ..Proof: The proof establishes the existence of a coordinate transformationfn : Dn → D, fn : (x, y)→ (x, y), such that Gn, fn, and G can be representedby an order preserving commutative diagram, i.e., such that (∗) Gn(x, y) =F (fxn (x, y), f

yn(x, y)− fxn (x, y)), all (x, y) ∈ Dn, and fn is order preserving.20

Since G has increasing differences in (x, y), this will imply that Gn has in-creasing differences in (x, y). fn is order preserving if and only if f

xn (x, y) x

and fyn(x, y) y, for all (x, y) ∈ Dn. The induction start: For n = 1,fx1 (x, y) = x and f

y1 (x, y) = y satisfy (∗) and are trivially order preserving.

The induction step: Assume that for n ∈ N, fn is order preserving and sat-isfies (∗). Take (x, y) ∈ Dn+1, i.e., such that (x, (n + 1)(y − x)) ∈ T . SinceGn and Gn+1 have the same range, (∗) implies the existence of (x, y) ∈ Dnsuch that Gn+1(x, y) = F (f

xn (x, y), f

yn(x, y)− fxn (x, y)). If, throughout Dn+1,

(x, y) ∈ Dn can be chosen such that x x, y y, this determines fn+1 asthe composition: (x, y) → (x, y) → (fxn (x, y), f

yn(x, y)). Since by the induc-

tion hypothesis fn is order preserving, fn+1 will clearly be order preserving.Equivalently the objective is to find δx, δy ∈ RN such that (†) F (x, n(y−x)+(y−x)) = F (x+δx, n(y−x)+n(δy−δx)), (x+δx, y+δy) ∈ Dn, and δx, δy ≥ 0.There always exists a solution with δx = 0, namely δ∗y = n−1(y − x) (it iseasily checked that this solution is in Dn). Hence if y − x 0, the proof iscomplete. If not, use δx = ∆δx + 0, ∆δx 0. By assumption 2 this choiceof δx is feasible and by assumption 3 there exists ∆δy ∆δx such that (†) issatisfied with δy = δ∗y +∆δy. But then to finish the proof, pick ∆δx so largethat δy = δ∗y +∆δy δ∗y +∆δx 0. 2

Lemma 3.3 If T satisfies assumption 2, then An : Yn → L(Yn) is ascending,for n = 1, 2, 3, . . ..

Proof: The proof consists in showing that Dn satisfies assumption 2 which,by lemma 1, implies the conclusion. (i) Weak monotonicity: Take (x, y) ∈ Dn20fxn and f

yn denote the first, resp. last, N coordinates of fn (corresponding to x and

y).

26

and x ∈ Yn such that x x. (x, y) ∈ Dn ⇔ (x, n(y − x)) ∈ T . Since Tis weakly monotone it follows that (x , n(y − x)) ∈ T . But then, since Tallows free order disposal and n(y − x) n(y − x ), hence n(y − x ) =n(y − x ) ∧ n(y − x): (x , n(y − x )) ∈ T ⇔ (x , y) ∈ Dn. (ii) Free orderdisposal: Take (x, y) ∈ Dn and y ∈ Yn. (x, y) ∈ Dn ⇔ (x, n(y − x)) ∈ T .By free disposal in T therefore (x, n(y − x) ∧ n(y − x)) ∈ T ⇔ (x, y ) ∈ Dn,where the biimplication follows from n(y − x) ∧ n(y − x) = y ∧ y − x whichis valid because is generated by a cone (Yn is a vector lattice, see Schaefer(1999), chapter 5). 2

Lemma 3.4 If F (x, y− x) is supermodular in y then Gn(x, y) is supermod-ular in y, n = 1, 2, 3, . . ..

Proof: Fix x ∈ X, and consider y, y ∈ An(x). It must be shown that:(∗) F (x, n(y−x))+F (x, n(y −x)) ≤ F (x, n(y∨y −x))+F (x, n(y∧y −x)).Consider the positive, affine coordinate transformation fn : y → n(y−x)−x.fn : An(x) → A(x) is order preserving and has an order preserving inverse.Moreover Gn(x, y) = G(x, fn(y)) (so, as in the previous proof, the map-pings commute in the associated diagram). For a coordinate transformationwith the above properties (an order homeomorphism), it is easily verified thatfn(y)∨fn(y ) = fn(y∨y ) (and likewise for the meet). Using this the proof istrivial: Simply note that (∗) is equivalent to: F (x, fn(y)− x) +F (x, fn(y )−x) ≤ F (x, fn(y) ∨ fn(y ) − x) + F (x, fn(y) ∧ fn(y ) − x). By the previousrelationship and supermodularity in y ∈ A(x) of F (x, y − x) the conclusionfollows. Note that by a similar argument, one reaches the same conclusionby assuming that F (x, y) is supermodular in y ∈ Γ(x). 2

Proof of lemma 3: Under the assumptions of lemma 3, the above sublem-mas imply the following: Yn is a lattice, An(x) is a sublattice of Yn for eachx ∈ Yn (lemma 3.1), Gn(x, y) has increasing differences in (x, y) ∈ X×An(x)(lemma 3.2), Gn(x, y) is supermodular in y on An(x), all x ∈ Yn (lemma 3.4),and finally, An(x) is ascending (lemma 3.3). Also, as explained immediatelyafter (8), An will be upper hemi-continuous and compact valued. Finally,since F is continuous, Gn will be continuous. As shown by Amir (1996) thefirst four of the above conclusions immediately imply that the value functionof (8) will be supermodular on Yn (By Topkis (1998), lemma 2.6.2., increas-ing differences and coordinatewise supermodularity implies supermodularity

27

when each coordinate space is a lattice and the dimension is finite. WhatAmir (1996) shows is that this can be extended to the value function by con-sidering a sequence of finite problems). The statement of the lemma nowfollows as a standard application of Topkis’ theorem (cf. Amir (1996), the-orem 1.1(iii)). If G(x, y) = F (x, y − x) has strictly increasing differences,the above proofs can be repeated in order to conclude that also Gn(x, y) hasstrictly increasing differences. Again by use of Topkis’ theorem this impliesthat every policy selection for (8) is non-decreasing (cf. Amir (1996), corol-lary 1.1). 2

To sum up, it has been shown that every sequence of policy selections,(gn)

∞n=1, has a subsequence such that fn, defined pointwise by fn(x) =

n(gn(x) − x), converges pointwise (lemma 2.3). The limit function, f :X → Y , defines a ”weak” policy function in the sense of lemma 2.4. Asexplained in the proof of that lemma, x∗(t), as generated from the ”weak”policy function will be a solution to (DP). In particular it is absolutely con-tinuous (which follows from just about the only thing we know about f ,namely that it will be Lebesgue measurable). To prove the theorem’s firstpart it remains to be shown that some (x ∗ (t))t≥0 generated by some f willbe order preserving. To this end pick either g

nor gn (the maximum and

minimum policy selections). By lemma 3, any of these generate a sequence(x(0), g1n(x(0)), g

2n(x(0)), . . .) which varies monotonically in x(0) (with respect

to the infinite product order induced by the order on X). For each n, thissequence uniquely defines a continuous, piecewise linear function, xn(t), intwo steps. First for t ∈ R+ ∩ Q, where Q is the set of rational numbers,let xn(t) = gtnn (x(0)). Second connect subsequent points, xn(t) and xn(t )by linear segments (here t is rational and t is the ”next” rational number- a statement with a self-evident and well-defined meaning since the set ofrational numbers is countable). Now focus on a subsequence generated byn = 20, 21, 22, . . .. Intuitively the continuous piecewise linear extension aboveis added ”midpoints” for every increment in n. Fix t ∈ R+. Again withreference to the proof of lemma 2.4, limn→∞ gntn (x(0)) = x∗(t). Intuitivelythis means that if we plot the finite approximations in a diagram with t atone axis and xn(t) at the other, then as n → ∞, we will see the finer andfiner piecewise linear curve approaches x∗(t), t ∈ R+ (pointwise). Now, sincexn(t) = gtnn (x(0)) is order preserving over t as a function of x(0), it is nothard to accept the conclusion that the limit x∗(t) (which is also a function

28

of x(0) being generate by f - or, alternatively, being the limit of gtnn (x(0)),will preserve the order. In fact this conclusion is a direct consequence ofpointwise compactness (pointwise compactness was verified generally in theproof of lemma 2.3): If we could pick x(0) x (0) and obtain x∗(t) x∗ (t)for just one t (where x∗ is the limit from gntn (x (0))) we could also find msufficiently large, such that gmtm (x(0)) gmtm (x (0)), contradicting lemma 3.The proof of the second claim of the theorem - that strictly increasing

differences implies order preservation of any semi-flow - is an immediateconsequence of lemma 3 which in this case says that any policy selection isnon-decreasing (strictly speaking, one will also have to show that all solutionsto (DP) are ”captured” by the above approximation procedure. This is true,due to the continuity assumptions placed on the problem but we shall notattemt to prove it here).

6.2 Proof of Theorem 4

The proof steers towards applying a celebrated result due to Hirsch (1985) on”generic global stability”. Define the omega set for the initial condition x ∈X: ω(x) = ∩t≥0∪s≥tφ(x, s). Since the orbit, O(x(0)) in x(0) is bounded, andX ⊂ RN , O(x) is compact. This implies that ω(x) is non-empty, connected,compact, and attracts x∗ (i.e., dist(ω(x), x∗(t, x(0)) → 0 as t → ∞) (seee.g. Coddington and Levinson (1955), chapter 16). But a connected subsetof RN is curve-connected, and so ω(x) is curve connected. Since E is acountable set, and any curve in RN is uncountable, ω(x) ⊂ E thereforeimplies that ω(x) consists of a single point from E (a curve-connected subsetof a countable subset of RN is a point). By theorem 1, x∗(t, x) x∗(t, y),all t ≥ 0, i.e., x∗ is order preserving. By regularity it is in fact a semiflow.Now take x, y ∈ X with x y. By continuity of x∗ in t, there existsT1 > 0 such that x∗(T1, x) x∗(T1, y), but then there exists T2 > T1 suchthat x∗(T2, x) x∗(T2, y), etc. Therefore regularity implies that there existsT > 0 such that x∗(T, x) x∗(T, y). By Smith (1995), proposition 1.1.,this implies that the semi-flow is strongly order preserving. For x ∈ X, onesays that x can be approximated from above (below) if there exists (xn)

∞n=1

in X, such that xn ≺ xn+1 (xn xn+1), all n, and the sequence converges tox. Either X consists of a single point, in which case the theorem is trivial.Therefore take y ∈ X, x = y. x ∨ y ∈ X since X is a lattice, and one easilysees that x∨y x. By order convexity therefore (xn)

∞n=1 approximates x from

29

below, where xn =1nx∨ y+(1− 1

n)x ∈ X, all n, and consequently xn ≺ xn+1

all n. One also easily shows that ∪n∈NO(xn) has compact closure. Theabove results now makes it possible to apply a result originally due to Hirsch(Hirsch (1985), see also Hirsch (1982)), but here used in the strengthenedversion of Smith and Thieme (Smith (1995), theorem 4.3.) to conclude thatω(x) ⊂ E for x ∈ Q where IntQ dense in X. By the above conclusions, thisimmediately leads to the statement of theorem 4.

7 Related Literature and Concluding

Remarks

To the best of knowledge, this paper is the first in the literature which at-tempts to integrate dynamic optimization with the remarkable results onmonotone/order preserving dynamical systems developed by M. Hirsch, H.Smith, and others. For the main part this integration has been carried bythe much celebrated lattice programming methodology as pioneered by Top-kis (1978) and in recent years refined and widely adopted by economists (cf.Milgrom and Shannon (1994), Milgrom and Roberts (1994)). In dynamicprogramming the most important references are Amir et al (1991) and Amir(1996) (cf. below). While none of these use order preserving dynamicalmethods in the sense of Hirsch-Smith, such methods have been used in gametheory. Thus Hofbauer and Sandholm (2002) is an important reference be-ing in a sense the ”game theory dual” of this paper (see also references inthis paper to related work by Hirsch (yes, the same Hirsch!), Benaim andothers). To be more specific on the contributions of this paper, a main pointis that in a dynamic optimization context supermodularity / monotonicityconditions in the sense of Topkis, Amir and others (op.cit) when suitablytransfered to continuous time (theorem 1), imply that solution maps fall un-der the domain of monotone dynamical systems (Hirsch, Smith, op.cit). Theresult then is an approach to global stability analysis since to be sure themain results of Hirsch, Smith, Thieme and others are exactly global stabilityresults. A similar integration in discrete time is most likely infeasible. Hence,the difficulties and extra assumptions relative to e.g. Amir (1996) involvedin proving theorem 1, are worth the while, for a direct parallel in discretetime would seem to be subject to failure (see below for further elaboration).

30

Although the mathematical content may have served to hide it, the ap-proach to stability analysis thus developed is targeted towards applications.It can answer global stability questions where no existing methods can do so.If the problem is low-dimensional (by which is meant one or two dimensional),checking the conditions is normally a trivial exercise; something which canhardly be said of for example a phase-diagram analysis - even when a batteryof simplifying assumptions has been adopted to make this tractable.It is well known that Samuelson’s correspondence principle can be suc-

cessfully underpinned in certain situations. Thus Magill and Scheinkman(1979) show that for symmetric variational problems, the requirements thatcomparative dynamics yield ”expected” outcomes and that steady states belocally stable, coincide. Upon reflection it is clear that the main conclusionsof this paper can be framed in a similar way.21 That a dynamic optimiza-tion problem yields order preserving solutions is indeed a (global) compara-tive static property: If the exogenous initial condition is raised; the solutionshould not fall at any future date in response. As proved in theorem 4 (undersome extra assumptions), this can be used to prove generic global stabilitywhich implies, among other things, that comparative dynamics is well-definedsince the only relevant long-run allocations are the steady states. Clearly,that theorem may be interpreted as the expression of a global correspondenceprinciple which links global comparative statics to global stability.

A related approach to order preservation should be elaborated upon in thisfinal section. Assume that (DP) has solution maps which can be describedby a policy function, g : X → X: x(t) = g(x(t)), t ≥ 0, x(0) ∈ X given. Thefollowing restatement of a standard result on cooperative differential equa-tions then applies (for a proof see Smith (1988), the result is originally dueto Hirsch (1982), (1985)):

Let g : X → X be a continuously differentiable policy function for (DP).Then solutions will be order preserving semiflows if and only if P TDg(x)Phas non-negative off-diagonal elements for all x ∈ X, where P ∈ RN is afixed vector of the type ((−1)m1 , (−1)m2 , . . . , (−1)mN ), mn ∈ {0, 1} (i.e., Pis an N dimensional vector with 1 or −1 in all coordinates). If, in addition,Dg(x) is an irreducible matrix for all x ∈ X, and the set of steady states is21I owe my thanks to Herakles Polemarchakis for pointing the following out.

31

finite, then solutions will converge to a steady state for initial conditions inan open and dense subset of X.22

Under the conditions above, g is called a cooperative and irreducible dif-ferential equation. Here it might be called a cooperative, irreducible policyfunction. Such a ”policy function approach” has the advantage of expressingthe order preservation property in terms of the policy function’s Jakobi ma-trix. As such it makes no mentioning of lattice programming what so ever.As such is underscores the important fact the it is the theory of order pre-serving / montone dynamical systems (cf. Smith (1995)) rather than latticeprogramming which is the ”backbone” of this paper. Its disadvantages are,however, obvious: To arrive at a policy function which is merely continu-ous, one must impose convexity, differentiability, and interiority assumptions(see Benveniste and Scheinkman (1979), Gota and Montruccio (1999), espe-cially the proof of corollary 2). While this does not rule out most standardproblems; assumptions which guarantee a continuously differentiable policyfunction most certainly do. In fact, known conditions for g to be merelyLipschitz continuous exclude virtually all growth models (sic).23 A secondobvious disadvantage of the policy function approach is that in order to applyit, g must be characterized up to first derivatives. This may or may not bepossible; but it would seem to be a rare occurrence that this could ever besimpler than checking whether (DP) is a monotone dynamical program.24

Amir (1996) (and in one dimension Amir et al (1991)) and Hopenhayn andPrescott (1992) study supermodular discrete-time dynamic programs in the

22A matrix is irreducible (or indecomposable) if it is impossible to rearrange rows andcolumns such as to divide it into four square submatrices one of which is a zero-matrix.23Indeed, strict concavity of the reduced form utility function is among the needed con-

ditions for Lipschitz continuity (cf. Gota and Montrucchio (1999)). In optimal growththeory this is violated whenever firms can produce the same vector of outputs by differentcombinations of inputs (thus in the standard non-joint case with neo-classical produc-tion functions). The reason is that even if the extensive form utility function is strictlyconcave, the reduced form utility surfaces will contain flat segments at all input-outputcombinations which result in the same consumption vector (cf. McKenzie (1976), section7).24In game theory (cf. Hofbauer and Sandholm (2002) and references there in) cooper-

ative and irreducible differential equations have, in contrast, proved highly applicable inthe study of dynamic games’ stability properties.

32

certainty and stochastic cases, respectively. All establish order preservationresults comparable to theorem 1. While Amir’s focus is on comparative stat-ics results, Hopenhayn and Prescott (1992) employ the fact that solutions areorder preserving to show existence of an invariant distribution by use of theKnaster-Tarski fixed point theorem for order preserving self-maps (a similarapplication of that theorem can be used to prove that the set of fixed pointsfor a monotone dynamic program is a complete lattice). In discrete-time,order preservation can be given a simple equivalent formulation in terms of agiven policy selection, g. Focusing on the certainty case for simplicity, it canbe seen that g yields order preserving solutions if and only if x x impliesg(x) g(x ), that is to say if and only if g is non-decreasing.25 Using this, itis not too difficult to prove the following result which resembles lemma 2 ofsection 4: Take x0 as given, assume that g is non-decreasing, and consider asolution given by xt+1 = g(xt), all t. If then xτ+T xτ for some T, τ ∈ N,T > 0, then the solution converges to a cycle of period smaller than or equalto T . If, in particular, T can be picked equal to 1 (i.e., if xτ+1 xτ , someτ ∈ N), then the solution converges to a steady state (the proof of these state-ments are available from the author upon request). If more is known aboutg these conditions can be given a sharper characterization. If, for example,g is differentiable (Santos (1991)), then the conditions can be expressed interms of the eigenvalues of the Jakobi-matrix Dg. Though transferring theresults of this paper to discrete-time models is an interesting topic for furtherresearch, it is important to be aware from the outset that the main results ofthis paper (in particular the main result, theorem 4) will not easily generalizeto discrete-time systems. Without going into details, the most one shouldhope for is a result which states that if g is strongly order preserving thensolutions will converge to a periodic cycle for almost all initial conditions.This somewhat disappointing statement follows from well-established resultsin the literature on monotone dynamical systems (which in the recent yearshas paid much attention to discrete-time systems).26 The disappointing as-

25In continuous time, order preservation does not lend itself to such a simple character-ization in terms of policy functions, in fact policy functions in the equivalent sense maynot even exist (see, however, the previous discussion of cooperative differential equationswhere order preservation can be expressed in terms of a policy function).26See e.g. Ji-Fa and Shu-Xiang (1996), especially the introduction where the authors

sum up what is known on the differences between discrete and continuous time orderpreserving systems.

33

pect of this fact is of course that generic convergence to cycles is not a resultwhich carries much application potential with it (but it does have at leastone interesting implication: It rules out chaos as a robust phenomenon for(strongly) order preserving discrete time systems).

As some readers have undoubtedly already noticed, the results in this paperbear some familiarity with turnpike theorems. It deserves clear mentioningthat the results of this paper do not relate to the turnpike literature in any di-rect way. Under the conditions of a turnpike theorem (which include uniforminteriority, convexity, and above all the condition that consumers weight fu-ture utility ”sufficiently” strongly) all optimal trajectories ”bunch” togetherin the far future as a direct consequence of the global Lyapunov functionwhich the assumptions place on the system (McKenzie (1976)). In partic-ular, steady states will get arbitrarily close to each other in phase-space asconsumers become increasingly patient; and for example cycles cannot existsave on the von Neumann facet (Nishimura and Yano (1995)). In fact onemight say that turnpike theorem’s implications for steady states is inciden-tal: A steady state need not even exist for turnpike theory to be meaningful- but if a steady state does exist it will be among the paths which bunchtogether, hence all paths will bunch together around it. This stands in sharpcontrast to the general situation in order preserving systems. Here (interior)steady states may be arbitrarily ”far apart” in phase-space and will generallyco-exist with cycles and even chaotic regions (Smith (1995)). In fact globalstability is only generic exactly of this reason (whereas turnpike theoremslead to ”strong” global stability, i.e., convergence from all initial conditions).An obvious consequence is that the results in this paper concern steady statesin the strict sense: If no steady state exists, the results become meaningless.Note conversely that the global stability results in this paper certainly donot generalize turnpike theory. Turnpike theory does not rest of complemen-tarity assumptions, but rather on assumptions related to the concavity ofthe instant reduced form utility function (see in particular McKenzie (1976)about this and its relationship to the ”high discount factor” assumption).Finally, two remarks about extensions. The preferred dynamic program-

ming problem has been the case with an additively stationary objective func-tion. However, as is clear from section 2.2, what is important for solutionmaps to define semiflows is only the stationarity part. There would seem tobe no difficulty what so ever in extending the methods to the general class

34

of stationary utility functions (e.g. Koopmans (1960)), including preferenceswith habit formation and other types of non-additive recursivity. Secondly,this paper has exclusively considered dynamic programming under certainty.It would of course be very interesting to extend the present results to thestochastic case (in order to show global generic stability of invariant distri-butions). While this will, probably, be mathematically demanding; the nec-essary mathematics does exist (even in the form of lecture notes: Chueshov(2002)) and there is therefore reason to conjecture that the results of this pa-per can be carried over to continuous-time stochastic optimal programming(note, as has been noted above, that an attempt in discrete time is likely tofail, at least as far as global generic stability of steady states is concerned).

References

Amir, R., L.J. Mirman and W.R. Perkins (1991): ”One-Sector NonclassicalOptimal Growth: Optimality Conditions and Comparative Dynamics”,International Economic Review 32, 625-644.

Amir, R. (1996): ”Sensitivity Analysis of Multisector Optimal EconomicDynamics”, Journal of Mathematical Economics 25, 123-141.

Aubin, J.P. and H. Frankowska (1990): ”Set Valued Analysis”, SpringerVerlag.

Becker, R.A. and J.H. Boyd (1997): ”Capital Theory, Equilibrium Analysisand Recursive Utility”, Blackwell Publishers, Oxford.

Benveniste, L.M. and J.A. Scheinkman (1979): ”On the Differentiability ofthe Value Function in Dynamic Models of Economics”, Econometrica47, 727-732.

Chueshov, I. (2002): ”Monotone Random Systems: Theory and Applica-tions”, Lecture Notes in Mathematics 1779, Springer Verlag, Heidel-berg.

Clarke, F.H. and R.B. Vinter (1985): ”Regularity Properties of Solutionsto the Basic Problem in the Calculus of Variations”, Transactions ofthe American Mathematical Society 289, 73-98.

35

Coddington, E.A. and N. Levinson (1955): ”Theory of Ordinary DifferentialEquations”, MCgraw Hill Co., Inc.

Frankowska, H. (2001): ”Value Function in Optimal Control”, ICTP Lec-ture Notes Series 8. Available at http://www.ictp.trieste.it/∼pub off/lectures

Gota, M.L. and L. Montrucchio (1999): ”On the Lipschitz Continuity ofPolicy Functions in Continuous-time Optimal Growth Problems”, Eco-nomic Theory 14, 479-488.

Hirsch, M.W. (1982): ”Systems of Differential Equations Which are Com-petitive or Cooperative I: Limit Sets”, SIAM Journal of Applied Math-ematics 13, 167-179.

Hirsch, M.W. (1985): ”Systems of Differential Equations Which are Com-petitive or Cooperative II: Convergence Almost Everywhere”, SIAMJournal of Mathematical Analysis 16, 432-439.

Hofbauer, J. and W.H. Sandholm (2002): ”On the Global Convergence ofStochastic Fictitious Play”, Econometrica 70, 2265-2294.

Hopenhayn, H.A. and Prescott, E.C. (1992): ”Stochastic Monotonicity andStationary Distributions for Dynamic Economies”, Econometrica 60,1387-1406.

Howes, N.R. (1995): ”Modern Analysis and Topology”, Springer-Verlag,New-York.

Ji-Fa, J. and Y. Shu-Xiang (1996): ”Stable Cycles for Attractors of StronglyMonotone Discrete-Time Dynamical Systems, Journal of MathematicalAnalysis and Applications 202, 349-362.

Kamke, E. (1932): ”Zur Theorie der Systeme Gewohnlicher Differentialgle-ichungen II”, Acta Math. 58, 57-85.

Kelley, J.L. (1955): ”General Topology”, Springer-Verlag, New-York.

Koopmans, T.C. (1960): ”Stationary Ordinal Utility and Impatience”, Econo-metrica 28, 287-309.

36

Lucas, R.E. (1967): ”Optimal Investment Policy and the Flexible Acceler-ator”, International Economic Review 8, 78-85.

Magill, M.J.P. and J.A. Scheinkman (1979): ”Stability of Regular Equilib-ria and the Correspondence Principle for Symmetric Variational Prob-lems”, International Economic Review 20, 297-315.

McKenzie, L.W. (1976): ”Turnpike Theory”, Econometrica 44, 841-865.

McKenzie, L.W. (1998): ”Turnpikes : Richard T. Ely Lecture”, AmericanEconomic Review 88, 1-14.

Milgrom, P. and J. Roberts (1994): ”Comparing Equilibria”, AmericanEconomic Review 84, 441-459.

Milgrom, P. and C. Shannon (1994): ”Monotone Comparative Statics”,Econometrica 62, 157-180.

Nishimura, K. and M. Yano (1995): ”Nonlinear Dynamics and Chaos inOptimal Growth: An Example”, Econometrica 63, 981-1001.

Santos, M. (1991): ”Smoothness of the Policy Function in Discrete TimeEconomic Models”, Econometrica 59, 1365-1382.

Schaefer, H.H., with M.P. Wolff (1999): ”Topological Vector Spaces”, 2nded., Springer-Verlag, New York.

Skiba, A.K. (1978): ”Optimal Growth with a Convex-Concave ProductionFunction”, Econometrica 46, 527-540.

Smith, H.L. (1988): ”Systems of Ordinary Differential Equations whichGenerate an Order Preserving Flow. A Survey of Results”, SIAM Re-view 30, 87-113.

Smith, H.L. (1995): ”Monotone Dynamical Systems: An Introduction tothe Theory of Competitive and Cooperative Systems”, MathematicalSurveys and Monographs 41, American Mathematical Society, Provi-dence, Rhode Island.

Topkis, D.M. (1978): ”Minimizing a Supermodular Function on a Lattice”,Operations Research, 26, 305-321.

37

Topkis, D.M. (1998): ”Supermodularity and Complementarity”, PrincetonUniversity Press, Princeton, New Jersey.

38