Timing replacement decisions under discontinuous technological change

Timing Replacement Decisions Under Discontinuous Technological Change*

Wallace J. Hopp Department of Industrial Engineering and Management Sciences,

Northwestern University, Evanston, Illinois, 60208

Suresh K. Nair Information Management Department School of Business,

University of Connecticut, Storrs, Connecticut, 06269

We consider the problem of deciding whether to keep a piece of equipment or replace it with a more advanced technology in an environment of technological change. Our model assumes that the costs associated with the presently available technology and future technologies are known, but that the appearance times of future technologies are uncertain. We develop a procedure for computing the optimal keep-or-replace decision that iteratively incorporates a technological forecast. For a certain class of situations, we show that our approach requires the minimum possible amount of forecasted data.

1. INTRODUCTION

Equipment replacement problems have long been studied in the industrial engineering and operations research literature (for surveys, see Barlow, Pros- chan, and Hunter [l], McCall [lo], Pierskalla and Voelker [ll], and Sherif and Smith [14]). The vast majority of these models assume that technology remains constant, so that old machines are replaced with identical models. However, in many important classes of equipment, such as computer hardwarelsoftware, robotics, automatic control, and other electronics technology, replacements are made explicitly to take advantage of improved technology. For these applications, the issue is not to determine the economic life of the equipment by comparing increasing maintenance costs with fixed replacement costs. Instead, the decision maker must compare the benefits of replacing the current equipment with the most attractive alternative currently on the market versus waiting for a better alternative to become available.

In order to evaluate the benefits of waiting for a technological improvement, we must incorporate information about future technological change. However, technological forecasting is difficult and fraught with uncertainties (Martino [9]).

"This work was supported in part by the National Science Foundation under Grant No. ECS-8619732

Naval Research Logistics, Vol. 38, pp. 203-220 (1991) Copyright 0 1991 by John Wiley & Sons, Inc. CCC 0028-1441/91/020203-18$04.00

204 Naval Research Logistics, Vol. 38 (1991)

The technology forecast can either be assumed to be stationary (Goldstein, Ladany, and Mehrez [6]), in which case the probability of appearance of future technology does not change with time, or more realistically, nonstationary . In either case, it is clear that the further into the future the forecast extends, the more uncertain it becomes. Therefore, it is desirable to make use of as little forecasted information as possible in making a keep-or-replace decision. The theory that enables us to do this is that of planning horizons. The planning horizon results of Sethi and Morton [12], Sethi and Chand [13], Chand and Sethi [5], Bean and Smith [3], Hopp, Bean and Smith [S] and Bbs and Sethi [4] show that in many problems changes in variables far into the future have little or no effect on the optimal decision in the first period. The key to using any planning horizon procedure is a method for determining when a finite horizon is sufficiently long to ensure that the resulting initial decision will be optimal for all longer time horizons. We make use of the terminology of Bbs and Sethi in calling such a horizon a solution horizon. We call the method for discovering it a stopping rule.

In this article, we model the replacement decision problem when discontinuous technological improvements occur at uncertain times in the future. We then use a planning horizon approach to develop an iterative procedure for incorporating a technological forecast. This procedure uses data about the near future first and only uses data further in the future if it is required to resolve the uncertainty concerning the current keep or replace decision. To test horizons to determine whether they are solution horizons, we develop a specialized stopping rule for the case where at most two technological improvements are allowed. This procedure has the advantage of being simple to implement and, when only a single technological improvement is allowed, is guaranteed to identify the shortest possible solution horizon. We also discuss the application of a stopping rule for nonhomogeneous Markov decision processes to the case where an arbitrary number of technological improvements are allowed. For this case, we indicate that the special structure of the replacement problem allows for a more efficient implementation of the stopping-rule test.

2. MODEL DESCRIPTION

We begin by describing a general model of equipment replacement under technological change. We consider the case of a firm that has a piece of equipment of a type that undergoes technological improvements over time. The firm has to decide, under these circumstances, what actions to take (i.e., keep the equipment or buy new equipment). This decision becomes complicated because the decision maker must consider not only the technologies presently available on the market, but also future improvements of the technology. We assume that the firm’s objective is to maximize expected discounted present revenue over the infinite horizon.

We model this problem as an infinite horizon, discounted nonhomogeneous Markov decision process (MDP) with finite state and action spaces. Our notation and approach are described below.

Hopp and Nair: Timing Replacements 205

Notation

We assume that at most n + 1 different technologies, including those currently available, can become available over time. We label these 0 through n and let S = {O,l;..,n} represent the set of all technologies. We label the technology currently in use by the firm as technology 0 and define a subset of S, denoted by X,, that contains 0 and any other technologies that are available on the market at the current time. We let x denote the set of all subsets of S contain- ing X,.

We define a “state” at any point in time by ( i , X ) where i represents the index of the technology in use by the firm and Xis the set of all technologies currently available on the market. We let A( i ,X) = {K,, R,; j E X} represent the feasible actions in state ( i , X ) , where K, indicates the action “keep technology i” and Ri represents the action “replace the current technology with technology j .” For notational convenience, we define

a = Ki , a = R,’ I(i,a> = (:.

which represents the technology in use by the firm after taking action a while owning technology i.

To formulate a Markov decision process, we suppose the firm makes keep- replace decisions at discrete evenly spaced intervals. We let r, represent the (expected) one-period revenue generated by technology i and c, represent the expected capital cost of purchasing technology j , and define

a = K, p(i,X,a) =

where p ( i , X , a ) represents the one-period revenue function resulting from action a when the current technology is i. We assume that r, and c, are bounded and hence so is p( i ,X ,a) for all i, j E S.

We represent the uncertainty in the problem by defining probabilities of the form P,(YlX), which represents the probability of technologies in Y C S being available at the beginning of period t given that the technologies in X C S are available at the beginning of period f - 1.

Letting P represent the one-period discount factor, with P < 1 and f:(i,X) represent the maximal expected return in periods t through T , discounted back to the beginning of period t , given the state at the beginning of period f is ( i , X ) , and L( i ,X ) represent the boundary condition for the T-period problem in state ( i , X ) , we can write the following dynamic programming recursion


for all i E S, X E x. We let nT(i,X) represent the actions that achieve the above maximizations (i.e., n f ( i , X ) is the optimal action in state ( i , X ) in period t of the T period problem). As long as T is finite, these actions can be computed using the backward recursion of dynamic programming. Typically, we would take L( i ,X ) = 0 for all i ,X . However, as we shall see later, using alternative values of L( i ,X ) can give useful results. In general, we assume that IL(i,X)l is bounded for all i ,X .

For the purposes of this article, we assume that the expected costs and revenues associated with all technologies are known, but the appearance times of the technologies are uncertain.

Solution Horizons

Because the P,(XIY) in our formulation depends on t , the above Markov decision process is nonhomogeneous in time. This means that we cannot write down, let alone compute, the optimal actions in the infinite horizon problem with a finite algorithm. For this reason, we make use of the theory of planning horizons to enable us to restrict attention to finite horizon problems.

The planning horizon results of Hopp, Bean, and Smith [8] and Hopp [7] show that, since p < 1 and Ip(i,X)l < m, f:(i,X) converges to a limiting value function f , ( i ,X) as T+ m. Furthermore, since A ( i , X ) is finite, their results imply that if the infinite horizon optimal decision for the first period, written no(i,X), is unique, then there exists a finite horizon T such that nr( i ,X) = no(i ,X) for all T 2 T . This will be true regardless of the terminal rewards, L( i ,X ) , as long as they are bounded. Such a T is called a solution horizon. The significance of a solution horizon is that if we can identify it, then we can solve a finite horizon problem and be assured that the optimal initial decision is optimal for any time horizon longer than the solution horizon.

Hopp [7] gives a means for identifying solution horizons in finite state and action space nonhomogeneous Markov decision processes. However, because of the generality of the problem, the method of Hopp can be inefficient (i.e., can result in overly long solution horizons). The method of B b and Sethi [3] also results in identifying solution horizons an order of magnitude or more larger than the method we propose for a restricted version of the problem, as we will see later. Because we must forecast data in the equipment replacement problem (i.e., the P,(YIX) values), it is vital that we find as short a solution horizon as possible. In this article we exploit the special structure of the equipment replacement problem to devise a new, more efficient way of identifying solution horizons for a restricted version of the problem. We also describe how the special structure can be used to improve on the test of the Hopp [7] stopping rule as implemented in Bean, Hopp, and Duenyas [2].

The existence of a solution horizon implies that data sufficiently far into the future have little or no effect on the current decision. In light of this, our assumption that the set of technologies that may appear, S, is finite is not strong from a practical modeling standpoint.

3. COMPUTING OPTIMAL ACTIONS

We now turn to the question of deciding whether to keep the current equipment of technology 0 for at least one more period or replace it with a new


technology in the set X,. Since we are assuming that equipment does not de- teriorate, the only motivation for replacing the equipment is the greater revenue earning potential of the new technology. Hence, it will be attractive to replace technology 0 provided that we are likely to keep it long enough to amortize its cost through the increased revenues. However, how long we will keep the technology will depend on the appearance dates of new technologies on the market. Thus, we will have to look some distance into the future and calculate additional optimal decisions to determine the initial optimal decision. The question we address now is how to test a given horizon T to determine whether it is long enough to guarantee the optimal decision in period 0.

For now, we suppose that the firm owns technology 0 and technology 1 is the only other technology on the market at the start of the problem that the firm would consider buying, i.e., X, = {O,l}. Technologies 2 and 3 represents the only breakthroughs that may occur in the future (i.e., n 4 3 ) . We discuss the case where more than two improvements can occur in Section 5 .

The approach we take below states two versions of dynamic programming recursion (1) by using the actual forecasted data for periods 0 through T and then defining two different sets of terminal rewards. The first of these problems is shown to make the “keep” decision in the first period as attractive as possible. The second makes the “replace” decision as attractive as possible. If the optimal decision is “replace” for the first problem then it is “replace” for all problems with time horizons longer than T, regardless of the forecast beyond T. Similarly, if the optimal initial decision in the second problem is “keep”, then it is “keep” for any problem with a time horizon longer than T. Thus, whenever the two problems yield the same optimal initial decision, this decision is optimal regardless of the forecast beyond period T and the decision maker can stop the forecast/optimization process. In this sense, the two finite horizon problems provide a stopping rule. We show that this stopping rule will eventually be satisfied for a finite T provided that the infinite horizon initial decision is unique. We also show that it is “perfectly efficient” in the sense that if it is not satisfied then there are some forecasts beyond Tfor which it is optimal to keep technology 0 and other forecasts for which it is optimal to replace with technology 1. Hence, choosing an optimal initial decision requires forecasted information beyond T.

To develop the above approach, we first note that from (1) we can write

where the letters to the left of each term in the maximizations represent the action taken to achieve the expected revenue represented by that term. Let


so that a sufficient condition for nT(O,XO) = R1 is AT > 0 and nT(O,Xo) = KO is A,? 5 0 (we break ties in favor of KO).

Clearly, if fT(1,Y) - f,?(O,Y) (and hence AT) is nondecreasing in T for all t then, from (3), nT(O,Xo) = R1 implies that nn;U(O,Xo) = R1, for all M 2 T, and since fT(-, .) 3 fl(- ,-) , as T + w, nr(O,Xo) = R1 also implies that no(O,Xo) = R1, where no(O,Xo) is the infinite horizon optimal initial action. Analogously, if f,?(l,Y) - f:(O,Y) (and hence AT) is nonincreasing in T for all t then, from (3), n;(O,Xo) = KO implies that nf(O,Xo) = KO, for all M 2 T , and hence

To develop a stopping rule, we must define terminal rewards under which f,?(l ,Y) - fT(0,Y) is nondecreasing (nonincreasing) in T. After two brief lemmas characterizing the optimal value functions, we present such conditions below as Lemmas 3 and 4.

no(0,Xo) = KO.

LEMMA 1: (a) If L ( j , X ) 2 L(i,X), then fT ( j ,X ) 2 f,?(i,X) (b) If L ( j , X ) - L( i ,X) 2 L(j,Y) - L(i,Y) then fT(j,X) - fT(i,X) 2

If j > i and X C Y for all X, Y E ,y then

fT(j,Y) - fT(i,Y) for all t < T.

PROOF: See the Appendix. 0

LEMMA2: If

= m,

then fT(i,S) = mi, i E S, t = 0,1, ..., T.

PROOF: See the appendix. 0

To evaluate (3) for AT we need an expression for fT+,(l,Y) - fT+l(O,Y) for all ,y 2 Y 1 X. If the condition of Lemma 1 is valid at the boundary condition, then from the claim in the proof of Lemma 1 (see the appendix), we know that if RI is optimal in fT(j,X) then RI is also optimal in f ,?( i ,X) where 1 > j > i. Recall that Xo = {O,l}, and S = {O,l, . . . , n}. For the case n = 3, we define Xl = {0,1,2}, X, = {0,1,3}, and Sy = {Y,S}, where Y = Xl or X,. From (l) , we can then write


for n = 2 and n = 3. For n = 3, Y E XI, X,, 2 E Sy, and k defined as

2, if Y = XI if Y = X, k = { 3,

we can also use (1) to write

We use expressions (3)-(6) to define two sets of boundary conditions, the first of which makes the “keep” action as attractive as possible; the second of which makes the “replace” action as attractive as possible.

LEMMA 3: For n I 3. if

L( i ,S ) = mi,

, for n = 3, [ [ + ( I 1 - r k )

L(1,Y) - L(0,Y) = max 0, min c1

L(k,Y) - L(1,Y) = ck, for It = 3

for each i E S , Y E {Xl,X,}, and k defined as above, then gT(1,W) - gT(0,W) is nondecreasing in T , for all W E x, where g:(i,Y) = fT(i,Y) under these particular boundary conditions.

PROOF: First note that gT(1,S) - gr(0,S) = ml - mo for all t , which is clearly nondecreasing in T , by Lemma 2.


Now we prove the result for n = 2. Since ,y = {X , ,S } for this case, we need only show that it holds for W = X,. To do this, note that

L ( 1 , S ) - L ( 0 , S ) = m, - rno

= max[0,min{c2 - ( r , - t l ) / ( l - p),

5 min{c,, ( r , - r o ) / ( l - p)}

Hence, by Lemma l(b) and Lemma 2, we have

for all t . Using this in (4) implies

Thus, for t + 1 = T ,

Supposing (7) to be true for general t + 1 , (4) implies it to be true for t . Hence, the result for n = 2 follows by induction.

Now we prove the result for n = 3. To do this, first note that it follows immediately from (6) that

for all t and Z E { X , , X , , S } . From Lemma l(a), since gT(1 ,Z) - gT(0,Z) 2 0, and substituting (8) into

( 5 ) yields

gT(1 ,Y) - gT(0 ,Y) 2 max 0,min

for Y E {Xo,Xl} and all t .


Finally, from Lemma l(a), since gf(1,Z) - gf(0,Z) 2 0, (4) implies that

We complete the proof by observing that for t + 1 = T we have shown that

for Y E { X l , X 2 , S } , W E x. Assuming (9) and (10) are true for a general t + 1 and substituting in (4)-(6) shows them to be true for t . Induction completes the proof. 0

LEMMA 4: For n 4 3. if

L(i,S) = mi,

L(1,Y) - L(0,Y) = c1,

for each i E S, j E fi{O,l}, Y E x\{X,,S}, and k E R{O,l}, then hf(l,X,) - hf(O,X,) is nonincreasing in T , where hf ( i ,Y) = ff(i,Y) under these particular boundary conditions.

PROOF: The proof is similar to that of Lemma 3. 0

Let Sy(O,X,) = n,( OJ,) under the boundary conditions of Lemma 3 and *lr(O,Xo) I 1 = nf(O,Xo) under the boundary conditions of Lemma 4. Then the optimal infinite horizon decision can be found using the following theorem.

THEOREM 1: (a) If Sf(O,X,) = R, then nf(O,Xo) = R1 for all M 2 T. (b) If yf(O,X(,) = K O , then n,T(O,Xo) = KO for all M 2 T.

PROOF: The result follows immediately from Lemmas 3 and 4 and the discussion following expression (3). 0

In practical terms, the above results mean that, by using the two boundary conditions and then solving two finite-horizon problems, we are effectively “bracketing” the infinite horizon problem with respect to the optimal decision. Hence, it is sensible to iteratively solve the two problems beginning with a short


horizon, say three periods, and then progressively increasing the horizon until the two problems give the same first decision. If this happens, then we are assured that the result is infinite horizon optimal (from Theorem 1). In many cases, this will occur for a relatively short horizon. This would mean that technology forecasts need be made for only a few periods into the future. However, if the initial decision in the two problems do not agree when the horizon is made as long as practical, then the decision maker can compute the maximum error resulting from the two possible decisions and choose accordingly. We give details of the epsilon optimality approach later.

Whether or not the two sets of boundary conditions will eventually yield the same optimal initial decision depends on whether the “keep” action is superior to the “replace” action, or vice versa, in the infinite horizon problem. If both actions are equally good then no solution horizon will exist. We can state this condition for the existence of a solution horizon formally as follows.

THEOREM2: If

is nonzero, then there exists a 7 such that Sf(0,XO) = yT(O,Xo) = a for all T 2 T and hence T = T is a solution horizon.

PROOF: Since gT(.;) and hT(.;) both converge to fT(.,.) as T+ w, for any E 2 0 it is possible to choose sufficiently large to make

for all T 2 T . Hence both

and

will have the same sign as d. If d < 0 then Sf(0,XO) = yf(0,Xo) = KO, and if d > 0 then SY(O,X,) = yf(O,X,) = R1, for all T 2 T .

If d is small, the required 7 may be large. If d is zero, then a finite T will not exist. In either case, we could like an epsilon optimality approach for bound- ing the error introduced by using either 6,T(0,Xo) or ~ , T ( O , X ~ ) in the infinite horizon problem. The following theorem gives such a bound.

0


THEOREM 3: (a) The maximum error from using S$(O,Xo) in the infinite horizon problem

is

if AtT > 0 if AtT 5 0‘ { t$T,

(b) The maximum error from using Y$(O,XO) inathe infinite horizon problem is

if A$T 5 0 if AiT > 0’ {07 - A t r ,

where AkT = A; when hT(.,.) = fT(.,.) and Af = A; when gf(*;) = ff(.;) in expression (3).

PROOF: Straightforward induction shows that hT(1,Y) - hf(0,Y) 2 ff(1,Y) - ff(0,Y) L gtT(l,Y) - gf(0,Y). It follows from (3) that hiT 2 A,T 2 hiT for all T.

If A. 5 0 then no(O,Xo) = KO and if A. > 0 then Ko(0,XO) = R1. Since Atr 2 A; L A{‘, if AfjT > 0 then S,T(O,Xo) = ni(O,XO) = yT(O,X0) = R 1 , and there is

y;(O,Xo) = KO, and there is no error in using y$(O,Xo). If A i r 5 0 but A, > 0, then S$(O,Xo) = KO, but no(O,Xo) = R1 and A. represents the error from using S$(O,Xo). AtT is an upper bound on this error. Thus, (a) is proved. If ACT > 0 but An 5 0, then yi(O,Xo) = R 1 , but no(O,Xo) = KO and - A 0 represents the

no error in using S,T(O,Xo). Likewise, if AtT 5 0 then Si(O,Xo) = no( 0, XO) =

error from using yoT(O,Xo). - AiT is an upper bound on this error.-This proves (b). 0

4. EFFICIENCY

We now show that the specialized approach for the problem with n = 2 can, under certain circumstances, identify the shortest possible solution horizon for a given problem. Even if the conditions for this “perfect efficiency” do not hold, we show that this method tends to result in substantially shorter solution horizons than other methods currently available both for the case n = 2 and n = 3.

We begin by stating conditions under which the stopping rule of Theorem 1 is guaranteed to find the shortest possible solution horizon for the case n = 2.

THEOREM 4: If n = 2 and c1 5 (rl - ro)/(l - 8) then T = min{M: S?(O,X,) = yf(0,XO)) is the shortest solution horizon consistent with the partial forecast {P,( YIX)}Tr,l.

PROOF: We prove the result by showing that under the condition of the theorem, the boundary conditions of Lemmas 3 and 4 are consistent with actual technological forecasts for periods T , T + 1, . . . . Hence, if S,M(O,X,) and y,M(O,Xo) differ, then there are forecasts beyond T that lead to different decisions and hence T is not a solution horizon. Recall that for n = 2, XO = {O,l}, S = (0 2 1 21.


First, we prove that the boundary conditions in Lemma 3 are consistent with any forecast that has PT(SIXO) = 1. Note that in this case fr(i,S) = mi for all t 2 T, where fl(.,.) represents the infinite horizon value function in period t. By substituting this along with the forecast, into the infinite horizon counterpart to (4) for period T , we get

which is exactly the boundary condition of Lemma 3. We now prove that the conditions in Lemma 4 are consistent with the forecast

P,(X,,IX,) = 1, t = T , T + 1, . . . . The condition of the theorem implies that

for all t 2 T. Substituting this along with the forecast into the infinite horizon counterpart to (4) we can write:

- - c,, from the condition of the theorem.

which is the boundary condition of Lemma 4. 0

We have shown that if one of the conditions of Theorem 1 is satisfied, then T is a solution horizon. If, in addition, the problem data satisfy the conditions of Theorem 4, then T is the minimum possible solution horizon. The condition of Theorem 4 implies that the savings accrued over the long run by replacing technology 0 with technology 1 are at least as large as the cost of replacing with technology 1. If this condition were not to hold, then the optimal decision would always be “keep technology 0,” as one would never recoup the expenditure purchasing technology 1. Hence, in all interesting problems with n = 2, the condition of Theorem 4 will be satisfied, and our specialized approach is guaranteed to find the shortest possible solution horizon.

To illustrate the magnitude of the efficiency improvements that can be achieved by the specialized approach, we compare it to an existing approach designed for general sequential decision problems by Bes and Sethi [4]. Their stopping rule can be stated for our problem as follows.

Stopping Rule (Bb and Sethi). If A: 2 (I -c l + rllpT)/(l - P) then nf(O,xo) = n;(O,Xo) for all M 2 T.

First, we consider a case where the conditions of Theorem 4 are satisfied. Suppose n = 2. Let r, = 50, r, = 100, r2 = 300, cI = 150, c2 = 175, p = 0.9, and T = 4. Note that here rn, = 2825, m, = 2825, m2 = 3000. From recursion (1) using the boundary conditions in Lemmas 3 and 4 we solve the problem for


the following technological forecasts for the appearance of technology 2. For notational convenience we use p , = P,(SlX,), where S = {0,1,2} is the set of technologies when technology 2 becomes available. Depending on the forecast, various cases can arise:

(a) If p , = 0, p: = 0, pi = 0.6, we can compute Aid = 0.5 by using the boundary conditions in Lemma 3, and hence Sd(O,X,) is R , . It follows from Theorem 1 that n,(O,X,,) will also be R , , and so the decision to replace the current equipment with that presently available in the market is optimal for any technological forecast in periods 4 and beyond.

(b) If pI = 0.2, p z = 0.7, and p3 = 0.4, then using the boundary conditions in Lemma 4 shows A? = -49, and hence yd(O,X,,) is K,,. It follows from Theorem 1 that no(O,Xo) =

K,, regardless of the forecast for periods 4 and beyond. Hence, one should keep the existing piece of equipment so as to replace it later when technology 2 appears.

(c) If p , = 0.2. p 2 = 0.0, and p , = 0.5, then using the boundary conditions in Lemma 3 we get 6Q(0,X0) = KO and using the boundary conditions in Lemma 4, ;$(O,Xo) = R , . Hence, the optimal decision depends on the forecast in periods 4 and beyond. The decision maker could then either increase the horizon and repeat the exercise or use the c-optimality result discussed in Theorem 3. In this problem A $ = - 17, and Aid = 8. By Theorem 3, the maximum error in choosing KO is 8, and the maximum error in choosing R, is 17. Hence, we could choose KO. that is choose to keep the current piece of equipment and be ensured that the maximum error we can incur is 8.

For case (a) above, the Bes and Sethi stopping rule is not satisfied until the time horizon exceeds 27 periods, which is substantially higher than the solution horizon of four periods arrived at using our method.

Now we consider a case where n = 3. Let ro = 50, rl = 135, r2 = 175, r3 = 250, c, = 150, c2 = 450, c3 = 600, and B = 0.9. Also, let P,(X,/&) = 0.3, P,(SIX,) = 0.15, P,(SIX,) = 0.15, P,(SIX2) = 0.15 for all i . From recursion 2.1 using the boundary conditions in Lemmas 3 and 4 for the case n = 3 we solve the problem for the following additional technological forecasts.

(a) If P,(X21Xl,) = 0.3 for all t , we can compute at T = 4 A $ = 8.08 by using the boundary conditions in Lemma 3, and hence Sd(O,X,) is R,. It follows from Theorem 1 that a,(O,X,,) will also be R , , and so the optimal infinite horizon optimal decision would be to replace technology 0 with technology 1.

(b) If P,(X,lX,,) = 0.37 for all t , then using the boundary conditions in Lemma 4 for T = 2 shows A$* = -31.03, and hence y~(O,X,) is KO. It follows from Theorem 1 that the infinite horizon optimal decision no(O,Xo) = KO regardless of the forecast for periods 4 and beyond.

(c) If Pt(X21Xo) = 0.36 for all t. then using the boundary conditions in Lemma 3 with T = 4, we get Sg(O,X,) = KO with Ai4 = - 1 . 1 and using the boundary conditions in Lemma 4, ~ d ( 0 . X ~ ) = R , with Aid = 1.5. In such a case, we know from Theorem 1 that the optimal decision depends on the forecast in periods 4 and beyond. The decision maker could then either increase the horizon by one period and repeat the exercise or use the c- optimality result discussed in Theorem 3. In this case the maximum error from choosing to replace is 1.1, which is the preferred decision. As it happens, if the decision maker had increased his horizon to T = 5 he would have obtained A$ = 0.96 and A P = 1.15, which demonstrates that the optimal infinite horizon decision is to replace technology 0 with technology 1.

As a comparison with the method of Bks and Sethi for case (a) above, their method would have required a horizon of 28 periods to assure infinite horizon results, whereas ours resulted in a solution horizon of four periods. The difference


becomes even more dramatic as (A( decreases. For example, in case (c), their method resulted in a solution horizon of 47 periods, whereas ours gave a solution horizon of five periods.

5. EXTENSION TO GENERAL CASE

If more than two technological improvements can appear in the future, then it is no longer possible to specify simple finite horizon problems that make “keep” or “replace” as attractive as possible. Thus, another approach is needed. One possibility is to adapt the stopping rule for nonhomogeneous Markov decision processes given in Hopp [7].

To do this, first define

p = max [p(i,X,u)l. i .X ,a

Now notice that no matter what sequence of actions is taken beyond period T,

for all i E S and all X techniques as in Hopp, we can similarly show that

x. Moreover, by using standard contraction mapping

for any i , j E S, X , Y x. The stopping rule in Hopp essentially says that if the same initial action is optimal for all T period problems with terminal rewards satisfying conditions (11) and (12), then it must be optimal for all time horizons M 2 T. Bean, Hopp, and Duenyas [2] have described an integer programming formulation for testing this stopping rule for a given horizon T.

Since it is straightforward to implement the stopping rule test of Hopp, Bean, and Duenyas for this problem, we omit the details. However, it is worthwhile to point out that the special structure of the replacement problem enables us to improve the efficiency of this test. Specifically, we can make use of the following restrictions on the optimal value functions.

PROOF: The proof follows immediately from Lemma 1 by setting L( i ,X) = 0 for all i E S, X E x, and observing that f,‘(i,X) + f i ( i ,X ) as T + CQ. c7

Since the above restrictions must hold for the optimal value functions for all t in the infinite horizon problem, they must hold for period T. Hence, we can append these to conditions (11) and (12) to further reduce the set of possible terminal rewards for the T period problem in the stopping rule test given in


Hopp. Algorithmically, this can be done by simply adding the inequalities of Lemma 5 as constraints to the integer program formulated by Bean, Hopp, and Duenyas. This addition will have two benefits. First, because the added constraints reduce the feasible region of the integer program, making it easier to solve. Second, because some terminal rewards that could prevent the stopping rule from being satisfied are ruled out, the added constraints tend to lead to shorter solution horizons. Further computational work is needed to ascertain the magnitude of these benefits.

6. CONCLUSIONS

We have described an approach for computing the optimal “keep” or “replace” decision on the basis of a finite technological forecast when (1) costs and revenues for technologies are known but the timing of appearance is uncertain, and (2) revenue generated by a technology is constant over time. We have also described a simpler and more efficient approach for the case where at most two additional technologies can appear in the future. These methods allow the decision maker to evaluate the initial “keep” or “replace” decision on the basis of a minimal amount of forecasted data.

Further work is needed to relax conditions (1) and (2) in the context of our model. Allowing uncertainty regarding costs and revenues in the model would allow decision makers to explicitly evaluate the importance of these additional sources of uncertainty in replacement decisions. Introducing nonconstant revenue functions would be an important extension, since it would allow us to introduce age deterioration as well as the decline in the value of a piece of equipment due to technological obsolescence.

Finally, further work is needed to address in a practical manner the realistic situation where technological change continues indefinitely. If the number of technological improvements allowed is large, the size of the integer program required to test the stopping rule can grow very large; the number of integer variables in the integer program for testing whether T is a solution horizon is 2T(zI, where 1x1 represents the cardinality of ,y (Bean, Hopp, and Duenyas [2]). Even using the specialized approach given in that reference, this could become prohibitive. Given this and the fact that forecasting multiple technological breakthroughs is likely to be very difficult, it may make more sense to restrict specific attention to only one or two improvements and represent further technological change by means of a technological growth parameter. Such a model might be able to retain the simplicity and computational advantages of our restricted model with only one or two technological breakthroughs, but still capture the essence of the replacement problem in the face of continuing technological change.

APPENDIX

LEMMA 1: I f j > i and X C Y for all X,Y E x, then (a) if L ( j , X ) 2 L(z,X) then f F ( j , X ) 2 f ,‘(i,X); (b) if L ( j , X ) - L( i ,X) 2 L ( j , Y ) - L(i ,Y) then fT(j,X) - f r ( z , X ) 2 f:(j,Y) - f,‘(i,Y) for all t < T.


PROOF: To prove (a), note that it is true by definition at t = T. Suppose it is true at t + 1. Then to show it is true at t , we need the following.

CLAIM: n:( j,X) = Rk 3 n:(i,X) = R, for all k > j > i.

PROOF OF CLAIM: n:(j,X) = Rk implies

for all m E X\{k,j}. Further, n:(j,X) = Rk also implies

Since j > i implies rl > r, , this plus the inductive hypothesis implies

-ck + rk + p C pt+l(ylX)fT+l(k,Y) / 2 Y X

Since -cl < 0, this also implies

Hence, we have shown that

for all m E X\{k}, so n:(i,X) = Rk, which completes the proof of the claim

Using the result of the claim we can write f:(j,X) - f:(i,X) as

which is clearly nonnegative, so ff(j ,X) 2 fT(i,X). Induction proof of (a).


To prove (b), notice that it is true by definition at t = T. Suppose it were true at t + 1. Then using the above expression for f;( j , X ) - f,'(i,X) and noting that the only differences between the expressions for f : ( j , X ) - f : ( i ,X) and fT(j,Y) - ff(i,Y) are (1) X i s replaced by Y and (2) the region over which the minimization is taken is increased. Hence, the inductive hypothesis implies f,T(j,Y) - fT(i,Y) I f T ( j , X ) - ff(i,X), and (b) follows by induction on t . 0

LEMMA2: If

max -c, + r, / ( l - p) L(z,S) = max P I

r J ( 1 - P) i - = mi,

then f,'(i,S) = mi, i E S, t = 0,1, . . . , T .

PROOF: The lemma is true by definition for t = T. To show it for t < T , we notice that if we define 1 and i as follows:

i = arg max { -c, + r J ( 1 - p)}, IES

then we can write

Now, making the inductive hypothesis that the lemma is true at t + 1, and using recursion (l), we can write

fT(i,S) = max max -c, + r, + p(-c; + c/(l - 8)) 1 r:+ p ( - q + c/(1- p))

= -c; + y/(l - b)

for i < i and

for i 2 i. Hence, the lemma is true at t and the result follows by induction. 0


REFERENCES

[l] Barlow, R., Proschan, F., and Hunter, L., Mathematical Theory of Reliability, Wiley, New York, 1965.

[2] Bean, J., Hopp, W., and Duenyas, I., “A Stopping Rule for Forecast Horizons in Nonhomogeneous Markov Decision Processes,” Technical Report No. 89-03, De- partment of Industrial Engineering and Management Sciences, Northwestern Uni- versity, Evanston, IL, .June 1989.

[3] Bean, J. and Smith, R., “Conditions for the Existence of Planning Horizons,” Mathematics of Operations Research, 9, 391-401 (1984).

[4] Bks, C. and Sethi, S. , “Concepts of Forecast and Decision Horizons: Applications to Dynamic Stochastic Optimization Problems,” Mathematics of Operations Re- search, 13, (1988).

[5] Chand, S. , and Sethi, S. , “Planning Horizon Procedures for Machine Replacement Models with Several Possible Replacement Alternatives,” Naval Research Logistics Quarterly, 29, 483-493 (1982).

[6] Goldstein, T., Ladany, S., and Mehrez, A., “A Discounted Machine-Replacement Model with an Expected Future Technological Breakthrough,” Naval Research Lo- gistics Quarterly, 35, 209-220 (1988).

[7] Hopp, W., “Identifying Forecast Horizons in Nonhomogeneous Markov Decision Processes,” Operations Research, 35, 339-343 (1987).

[8] Hopp, W., Bean, J., and Smith, R., “A New Optimality Criteria for Nonhomo- geneous Markov Decision Processes,” Operations Research, 35, 875-883 (1987).

[9] Martino, J., Technological Forecasting for Decision Making, North Holland, New York, 1983.

[lo] McCall, J., “Management Policies for Stochastically Failing Equipment: A Survey,” Management Science, 11, 493-524 (1965).

[ l l ] Pierskalla, W. and Voelker, J., “A Survey of Maintenance Models: The Control and Surveillance of Deteriorating Systems,” Naval Research Logistics Quarterly, 23,

[12] Sethi, S. and Morton, T., “A Mixed Optimization Technique for the Generalized Machine Replacement Problem,” Naval Research Logistics Quarterly, 19, 471-481 (1972).

[13] Sethi, S. and Chand, S. , “Planning Horizon Procedures for Machine Replacement Models,” Management Science, 25, 140-151 (1979).

[14] Sherif, Y. and Smith, M., “Optimal Maintenance Models for Systems Subject to Failure-A Review,” Naval Research Logistics Quarterly, 28, 47-74 (1981).

353-388 (1976).

Documents

Timing replacement decisions under discontinuous technological change