On the local stability of the solution to optimal control problems

Journal of Economic Dynamics & Control 28 (2004) 2475–2484www.elsevier.com/locate/econbase

On the local stability of the solution to optimalcontrol problems

Alvaro RodriguezDepartment of Economics, Rutgers University, 360 Dr. Martin Luter King Jr. Blvd,

Newark, NJ 07103, USA

Received 12 October 2001; accepted 22 December 2003

Abstract

New conditions for the local stability of optimal control problems are presented. The conditionsare an extension of the results of a previous study focusing on the solution to problems solvableusing calculus of variations. A comparison is made between the conditions introduced here andthose presented in the literature. It is shown that our condition is roughly as powerful as theone presented by Sorger (J. Math. Anal. Appl. 148 (1990) 191) but it is easier to check.? 2004 Elsevier B.V. All rights reserved.

JEL classi(cation: C62; 041

Keywords: Optimal control; Stability

1. Introduction

The study of the dynamic properties of the solution to optimal control problems hasproduced several elegant results (for a survey of the literature see Brock and Malliaris,1992). However, some of the conditions for stability discussed so far are so restrictivethat they cannot deal with simple examples such as the standard one-sector optimalgrowth model. Others such as the conditions introduced by Sorger (1989, 1990, 1992)are very di;cult to check. For the particular problem of local stability, this paper=lls the need for a more comprehensive approach by presenting a quite general set ofconditions. It extends the results of a previous study, Rodriguez (1996), which appliesonly to problems solvable using calculus of variations. Since there are many casesin economics where the number of control variables is smaller than the number ofstate variables, the current extension appears useful. Moreover, the new formulation

E-mail address: [email protected] (A. Rodriguez).

0165-1889/$ - see front matter ? 2004 Elsevier B.V. All rights reserved.doi:10.1016/j.jedc.2003.12.003

mailto:[email protected]

2476 A. Rodriguez / Journal of Economic Dynamics & Control 28 (2004) 2475–2484

can be more easily compared with the results obtained in previous studies that statetheir conditions as requirements on the Hamiltonian of optimal control problems.

The stability conditions are applied to a problem presented by Sorger (1990). It isshown that although our conditions are easier to apply they perform as well as thecondition introduced by him.

2. The problem

The standard problem encountered in intertemporal economic models is the maxi-mization of∫ ∞

oe−�tU (K; u) dt; (1)

where K is an n-dimensional vector of state variables and u is an m-dimensional vectorof control variables. The vector K is subject to

•Ki = gi(K; u); i = 1; 2; : : : ; n: (2)

Both the utility function and the functions gi are assumed to be concave in K and u.The intertemporal rate of discount, �, is greater than zero.

The Hamiltonian for this problem is given by

H (K; u; �) = U (K; u) +∑i

�igi: (3)

The optimal choice of u maximizes H ; moreover the vector of state variables K andthe vector of costate variables � must satisfy the following equations:

•K =

9H∗

9� (K; �);•� = −9H

∗

9K (K; �) + ��; (4)

where H∗ is the maximized value of the Hamiltonian (once u has been set at itsoptimal value).

The local stability analysis is based on the properties of the linear approximation tothe system around its steady state values

•K•�

=

[H∗

�K H∗��

−H∗KK −H∗

K� + �I

](K

�

): (5)

The system is locally stable in the saddle sense, when the 2n× 2n matrix in the aboveequation has n eigenvalues with negative real parts and the remainder n have positivereal parts.

3. Two simple stability tests

The connection with the previous work on variational problems can be easily es-tablished by reducing the system described in (5) to a set of second-order diEerential

A. Rodriguez / Journal of Economic Dynamics & Control 28 (2004) 2475–2484 2477

equations in K . From the =rst equality in (5) it follows that

H�� =•K − H�KK: (6)

To simplify the notation we suppressed the ∗ on top of H . DiEerentiating with respectto time one obtains

H��•� =

••K − H�K

•K: (7)

Substituting for•� from (5), using (6) to eliminate � and making some rearrangements

one obtains the following expression:

A••K + [B − BT − �A]

•K − [C + �B]K = 0; (8)

where

A = (−H��)−1;

B = (H��)−1H�K;

C = HKK + (H�K)T(−H��)−1(H�K):

The symbol T is used to denote the transpose of a matrix.For optimal control problems with a concave objective function Cass and Shell

(1976) show that the Hamiltonian is convex in � and concave in K . Thus, both A andC are negative semi-de=nite.

The problem has now been reduced to the study of the solutions that make thefollowing determinant equal to zero

‖D(u)‖ = ‖Au2 + (B − Bt − �A) u − (C + �B)‖ = 0: (9)

The above equation is a polynomial degree 2n on u. The 2n solutions are also theeigenvalues of the matrix in (5). When all the ui are distinct the solutions to (8) and(5) are linear combinations of terms Yieui(t).

For this type of problem, Rodriguez (1996) shows that a su;cient condition forsaddle path stability is that the matrix C + �(B + BT)=2 must be negative de=nite.Some of the arguments in the proof must be altered in an obvious way so they canbe applied in the present setting. 1 The result extends a previous =nding of Magill andSheinkman (1979), who proved that when the matrix B is symmetric a necessary andsu;cient condition for stability is that the matrix C + �B must be a negative-de=nitematrix.

Recalling (8) the stability condition can now be stated as follows:

Proposition I. A su6cient condition for the saddle point stability of the system in(5) is that the following matrix:

(H�K)T(−H��)−1(H�K) + HKK +�[(H��)−1H�K + (H�K)T(H��)−1]

2(10)

is negative de(nite.

1 The reader is warned about the presence of many typos in the derivation of the main result in the 1996paper.


It is possible to obtain an alternative condition by starting from system (5), eliminat-ing the vector K , and obtaining a set of second order diEerential equations in �. Thealternate system leads to a problem of the type stated in (9). The associated su;cientcondition for stability is the following:

Proposition II. A su6cient condition for the saddle point stability of the system in(5) is that the following matrix:

H�K [HKK ]−1HT�K − H�� + �

(H�K [ − HKK ]−1 + [ − HKK ]−1HT

�K

2

)(11)


Since these are su;cient conditions, one obtains stability if one or both of the twoconditions (10) and (11) is satis=ed. In cases where one of the matrixes HKK and H��

is singular only one of the conditions will be available.

4. Comparison with previous results

As mentioned in the introduction there is an extensive literature on su;cient condi-tions for saddle path stability. The most recent and comprehensive of those conditionsis due to Sorger (1990). He proves the following result.

Proposition III. A su6cient condition for saddle path stability is the existence of areal symmetric matrix Q such that the following matrix:

G(Q) = HKK + HK�Q + QH�K + QH��Q − �Q +�2

4(H��)−1 (12)


Sorger’s condition as well as ours can be used to derive other conditions discussedin the literature. For example make Q=(�=2)(H��)−1. According to Sorger’s condition,the stationary solution exhibits saddle path stability if

G[�2

(H��)−1]

= HKK + �[

(H��)−1H�K + (H�K)T(H��)−1

2

]is a negative-de=nite matrix. This result is a generalization of a weaker su;cientcondition discussed by Brock and Scheinkman (1977), Magill (1977) and more recentlyby Medio (1987). Their condition requires the matrix

�[

(H��)−1H�K + (H�K)T(H��)−1

2

]to be negative de=nite. Obviously our condition concerning the matrix in (10) alsocovers this result as a special case. The same applies to well-known facts concerninga zero rate of discount. In such a case G[0] is negative de=nite so Sorger’s su;ciencycondition is satis=ed as well as ours (see (10)).


Following Magill (1977), the literature has paid considerable attention to the casewhere the following symmetry condition holds:

H�KH�� = H��HK�: (13)

In that case Sorger (1990) has showed that his condition is necessary (except perhapssome hairline cases). A link to our stability condition (10) can be established asfollows.

Premultiplying (13) by H−1�� yields

H−1�� H�KH�� = HK�: (14)

Postmultiplying by H−1�� one obtains

(H��)−1H�K = HK�(H��)−1: (15)

Using this result it is easy to verify that

G[�2

(H��)−1 − (H��)−1H�K

]= HKK + (H�K)T(−H��)−1(H�K)

+�2

[(H��)−1H�K + (H�K)T(H��)−1]: (16)

The right-hand side of the matrix is de=ned in (10). Thus if our condition concerningthe matrix in (10) holds, Sorger’s condition is also satis=ed.

The above result opens the possibility that Sorger’s conditions can be more generalthan ours. However, the work of Magill and Scheinkman mentioned before implies thatthe condition in Proposition I is also necessary. This is so because when (15) holdsthe matrix B in (8) is symmetric. Magill and Scheinkman studied the system in (8)under the assumption that B is symmetric. They showed that in such a case there issaddle path stability if and only if C + �B is negative de=nite. Thus we have:

Proposition IV. When the symmetry condition (13) holds, Sorger’s condition and therestriction stated in Proposition I are both necessary and su6cient.

The following examples further illustrate the similarities between the two conditions.The =rst case is when there is a single state variable. This is a problem where symmetrycondition (13) is automatically satis=ed. The eigenvalues of (5) are the solution to

f(u) = u2 − �u + HKKH�� − (H�K)2 + �H�K = 0: (17)

The saddle point property holds if and only if

(H�K)2 − �H�K − H��HKK ¿ 0: (18)

It follows therefore that condition (10) is equivalent to the saddle point property if H��

is greater than zero and that condition (11) is equivalent to the saddle point propertyif HKK is less than zero.


On the other hand, Sorger’s condition requires that there must be a real numberu such that

G(u) = H��u2 + (2H�K − �)u +�2

4(H��)−1 + HKK ¡ 0: (19)

Assuming H�� ¿ 0 as Sorger does, the function achieves a minimum when

u =� − 2H�K

2H��: (20)

Therefore, Sorger’s condition is

G(� − 2H�K

2H��

)=�H�K − (H�K)2 + HKKH��

H��¡ 0 (21)

which is identical to (10). Thus when H�� is positive both conditions are equally goodas expected.

In contrast the condition requiring that the matrix [(H��)−1H�K +(H�K)T(H��)−1] benegative de=nite is more restrictive. That is also the case for most of the conditionspresented in the special issue of the Journal of Economic Theory (12 (1) 1976).

They focus on the matrix

CV =

H��

�In2

�In2

−HKK

; (22)

where In is the n-dimensional identity matrix. It is well known that local stability isensured if CV evaluated at the steady state value of the variables is positive de=nite.For the present case, this is satis=ed if

(H��)(−HKK)¿�2

4: (23)

Given that (18) can be written as(H�K − �

2

)2

− �2

4− H��HKK ¿ 0;

the inequality in (23) is unnecessarily restrictive.The second example considered here is due to Sorger (1990) and it does not satisfy

the symmetry condition. It assumes that once the controls are set to their optimalvalues, the maximized value of the Hamiltonian is given by

H = 12�

21 + 1

2�22 + 2K1�1 + 5K1�2 + K2�1 + 2K2�2: (24)

Here K1; K2 are the state variables and �1; �2 are the costate variables.It is easy to verify that regardless of the value of the discount rate �, the quartet

K1; K2; �1; �2 equal to zero is a steady state. For this problem, the matrix de=ned in(10) is given by[−29 + 2� −12 + 3�

−12 + 3� −5 + 2�

]: (25)


The =rst element is negative if �¡ 29=2. The determinant of the above matrix ispositive if �¡ 1. Thus in order to satisfy our stability condition the rate of discountshould not exceed one.

Sorger’s conditions are in general quite di;cult to check but his cleverly designedexample turns out to be tractable. We will =nd the bound it imposes on �. 2 Let thematrix Q be given by

Q =

[q r

r s

]: (26)

After lengthy computations one obtains the following expression:

G(Q) =

�2

4+ 4q − �q + q2 + 10r + r2 q + 4r − �r + qr + 5s + rs

q + 4r − �r + qr + 5s + rs�2

4+ 2r + r2 + 4s − �s + s2

: (27)

Let A= {[q; r]; | �2

4 + 4q− �q+ q2 + 10r + r2 = −25 + �2

4 + 4q− �q+ q2 + (r + 5)2 ¡ 0}and let

B ={q;∣∣∣∣−25 +

�2

4+ 4q − �q + q2 ¡ 0

}:

Clearly if [q; r] ∈A then q∈B.For the matrix G(Q) to be negative de=nite, the pair [q; r] must belong to A and

the determinant of G(Q), denoted by F[q; r; s] must be positive. The example has theconvenient feature that s does not matter when de=ning A. Thus we may want tochoose s as to maximize F .

Let s∗(q; r) be the solution to (9F[q; r; s])=9s = 0. The second derivative of F isgiven by

92F(q; r; s)9s2 = 2

[�2

4+ 4q − 9q + q2 − 25

]: (28)

Thus, given a pair [q; r] in A, making s equal to s∗(q; r) truly maximizes F .The function F[q; r; s∗(q; r)] is a quartic equation in r. It is not presented here

because it is rather long. However, it is possible to solve r as a function of q fromthe expression F[q; r; s∗(q; r)] = 0. There are four solutions. The =rst two are given by

r = −5 −√

25 − �2

4− 4q + �q − q2;

r = −5 +

√25 − �2

4− 4q + �q − q2: (29)

They are the boundary of the open set A and therefore not included in that set.

2 Sorger’s paper (1990) does not pursue this topic. He only shows that for a rate of discount as high asfor his condition cannot be satis=ed.


The other two solutions are given by

r =10 + 20� − 3�2 − 24q + 6�q − 2

√−1 − 4� + 5�2√

−(25 − �2

4 − 4q + �q − q2)

2(−29 + 2�);

r =10 + 20� − 3�2 − 24q + 6�q + 2

√−1 − 4� + 5�2√

−(25 − �2

4 − 4q + �q − q2)

2(−29 + 2�):

(30)

Consider the case when the discount rate � exceeds one. Then for any q∈B thetwo solutions to the above equation are complex numbers. Thus there is no point [q; r]in A such that F[q; r; s∗(q; r)] is equal to zero. Moreover, direct examination of thefunction F[q; r; s∗(q; r)] reveals that for all q∈B it is a continuous function. It is alsocontinuous on A. Since the set A is connected all the elements of the image of Aunder F must have the same sign. To show that Sorger’s condition cannot be ful=lledit is su;cient to =nd for each value of � a pair [q; r] ∈A such that F[q; r; s∗(q; r)] isnegative.

The function Y [q; r] = �2=4 + (4 − �)q + q2 + 10r + r2 is minimized when q = �−42

and r = −5. Since Y [(� − 4)=2;−5] is equal to 2� − 29, it follows that when � is lessthan 29

2 the set A is non-empty. 3 The function

F[� − 4

2;−5; s∗

(� − 4

2;−5

)]

equals −463 + 36� − 5�2, a quadratic expression that it is negative for all values of�. This fact implies that when F cannot change signs because � exceeds one, Sorger’scondition cannot hold.

Fig. 1 illustrates a case where � is less than one (it was drawn assuming � equalszero). The interior of the large circular curve is the set A. The boundary of the elongatedcurve is formed by the solutions to (30). In the interior of this curve, the functionF[q; r; s∗(q; r)] is positive so the matrix Q generated by points inside the two curvessatis=es the Sorger’s condition. As � increases the smaller curve becomes thinner andit completely disappears when � is made larger than one.

It is interesting to point out that the other stability conditions discussed here areuseless in this example. No positive discount rate can make either

�[

(H��)−1H�K +HK�(H�K)−1

2

]or Hkk +�

[(H��)−1H�K +(H�K)T(H��)−1

2

]

negative-de=nite matrixes. Similarly, there is no positive value of � such that the ma-trix CV de=ned in (24) is positive de=nite. In all these cases some of the matrixeseigenvalues are positive while others are negative: the eigenvalues of CV are equal to12 [1 ± √

1 + �2] repeated twice, and in the case of the other two matrixes their eigen-values are [ − 2�; 10�] (all the elements of the matrix HKK are equal to zero, so thematrix does not make a diEerence).

3 The same bound on � was obtained before when considering the condition that the =rst element of thematrix in (10) must be negative.


Value of q

Solutions to (30)

Boundary of A

Value of r

-6 -4 -2 2

-10

-8

-6

-4

-2

Fig. 1. The Sorger’s condition.

Two important topics for future research emerge from the examples considered here.One is the search for more general results than those stated in Proposition IV aboutthe equivalence of Sorger’s condition and ours. The existence of stronger results issuggested by the fact that in the second case studied here both conditions turned outto be identical.

A second topic is the search for more comprehensive conditions. Although the twoconditions are the best available, if symmetry condition (13) is not ful=lled they arenot necessary conditions. Sorger (1990) shows that his example has the saddle pathproperty for values of � as large as four. There is, obviously, room for improvement.

Acknowledgements

The author gratefully acknowledges the comments of an anonymous referee.David Goldbaum read the manuscript and made useful comments. Any remaining errorsare solely the author’s.

References

Brock, W.A., Malliaris, A.G., 1992. DiEerential Equations, Stability and Chaos in Dynamic Economics.North Holland, Amsterdam.


Brock, W., Scheinkman, J., 1977. The global asymptotic stability of optimal control with applications todynamic economic theory. In: Pitchford, J.D., Turnovsky, S.J. (Eds.), Applications of Control Theory toEconomic Analysis. North-Holland, Amsterdam, 1977.

Cass, D., Shell, K., 1976. The structure and stability of competitive dynamical systems. Journal of EconomicTheory 12, 31–70.

Magill, M.J.P., 1977. Some new results on the local stability of the process of capital accumulation. Journalof Economic Theory 15, 174–210.

Magill, M.J.P., Sheinkman, J., 1979. Stability of regular equilibria and the correspondence principle forsymmetric variational problems. International Economic Review 20, 297–315.

Medio, A., 1987. Oscillations in optimal growth models. Journal of Economic Behavior and Organization 8,413–427.

Rodriguez, A., 1996. On the stability of the stationary solution to variational problems. Journal of EconomicDynamics and Control 20, 415–431.

Sorger, G., 1989. On the optimality and stability of competitive paths in continuous time growth models.Journal of Economic Theory 48, 526–547.

Sorger, G., 1990. The saddle point property in Hamiltonian systems. Journal of Mathematical Analysis andApplications 148, 191–201.

Sorger, G., 1992. Local stability of stationary states in discounted optimal control systems. Journal ofOptimization Theory and Applications 72, 143–162.

Documents

On the local stability of the solution to optimal control problems