31
Unità di Perugia e di Roma “Tor Vergata” ncertain production systems: optimal feedback contr the single site and extension to the multi-site cas workshop Ottimizzazione e Controllo delle Supply Chain Siena, Certosa di Pontignano, 23-25 ottobre 2005 Francesco Martinelli Fabio Piedimonte Università di Roma “Tor Vergata” Mauro Boccadoro Paolo Valigi Università di Perugia

Unità di Perugia e di Roma “Tor Vergata” "Uncertain production systems: optimal feedback control of the single site and extension to the multi-site case"

Embed Size (px)

Citation preview

Unità di Perugia e di Roma “Tor Vergata”

"Uncertain production systems: optimal feedback controlof the single site and extension to the multi-site case"

workshop

Ottimizzazione e Controllo delle Supply Chain

Siena, Certosa di Pontignano, 23-25 ottobre 2005

Francesco MartinelliFabio Piedimonte

Università di Roma “Tor Vergata”

Mauro BoccadoroPaolo Valigi

Università di Perugia

Unità di Perugia e di Roma "Tor Vergata" 2/31

x(t)

(t)u(t)

],0[)( tu

backlog/inventory level at time t (fluid model)

x(t):

dtux )(

(t): The machine is failure prone, (t)=1 if the machine is up at time t, (t)= 0 if the machine is down, with failures and working times characterized by some deterministic or random law, depending on the production control

d

Unità di Perugia e di Roma "Tor Vergata" 3/31

Two main objectives:

In the literature, in the Markov case, it has been observed (mainly numerically) a relevant difference between the case the failure rate is a convex function of the production rate and the case it is concave [Hu Vakili Yu, 1994; Liberopoulos Caramanis, 1994]

Explore this analytically in the Markovian and in the non Markovian (deterministic) case

Several papers on single failure prone machines:

Explore the multi-site case where the production of each site may be increased by the production of the other, with some penalty (modeling for example transportation costs)

Unità di Perugia e di Roma "Tor Vergata" 4/31

Minimize

T

TdttxgE

TJ

0

)]([1

lim

u(t) 0

x

d

cp

cm

g(x)

BacklogInventory

Unità di Perugia e di Roma "Tor Vergata" 5/31

0 1

Machinedown

Machineup

qu

qd(u)

Markov

The site is modeled as a failure prone machine with a failure-repairprocess which can be:

Deterministic

Deterioration rate:

The machine is stopped for a repair/maintainance operation when z(t)=1

btuatz )()(

The single site case

Unità di Perugia e di Roma "Tor Vergata" 6/31

Optimal policy: hedging point policy (Kimemia and Gershwin, 1983; Bielecki and Kumar, 1988)

*

*

*

*

0

)(

zx

zxd

zx

xu

t

x(t)

z

otherwisez

p

mp

mp

m

c

cc

ccc

)1(log

01

*

))(()(

ud

udu

qqdqqdq

)(

)(ddqqdq udu

Single site, Markov: the homogeneous case (qd constant)

Unità di Perugia e di Roma "Tor Vergata" 7/31

],0(

],()(

2

1

Uuq

Uuquq

d

dd

u

qd(u)

U

qd1

qd2

d

Single site, Markov: a non homogeneous case (qd=qd(u))

Unità di Perugia e di Roma "Tor Vergata" 8/31

*

*

**

*

*

0

),()(

Zx

Zxd

ZXxU

Xx

xu

(OPT)

t

x(t)

Z

X

Single site, Markov: a non homogeneous case (qd=qd(u))

Unità di Perugia e di Roma "Tor Vergata" 9/31

Single site, Markov: a non homogeneous case (qd=qd(u))

Procedure followed for the proof and for the computation of the optimal thresholds X* and Z*

Take X Z and apply policy (OPT). At steady state the buffer level is a random variable with pdf:

Xx

ZXxxp

deK

dUUeK

XZ xXXZ

xZ

)(1)(20

)(20 ),(

)(

where:

22

)(2

1

)(2 1100

/1),(

d

ZXZX

qUee

dZXKK

and )( 11 udu qqdq )( 22 udu qqdUq

)(11

dd

)(22

dUd

Unità di Perugia e di Roma "Tor Vergata" 10/31

Single site, Markov: a non homogeneous case (qd=qd(u))

For the level x=Z, there is a point mass probability (X,Z):=K0(X,Z)d/qd2

Z X have to be properly selected to minimize:

)(),()()(),( ZgZXdxxpxgZXJZ

XZ

Once X* and Z* have been found and the optimal J* has been derived, compute the cost-to-go functions solving the HJB equations where the min operationhas been replaced by the (supposed) optimal policy u*(x):

)()()()]([))(( *01

*1* xgJxVxVxuqdx

dVdxu d

*01

0 )()()( JxgxVxVqdx

dVu

Unità di Perugia e di Roma "Tor Vergata" 11/31

Single site, Markov: a non homogeneous case (qd=qd(u))

Once the cost-to-go functions V0(x) and V1(x) have been computed, show thatthese functions, with the policy considered to compute them, satisfy the followingHJB equations:

)()()()()(min *01

1

],0[xgJxVxVuq

dx

dVdu d

u

*01

0 )()()( JxgxVxVqdx

dVu

If these equations are satisfied and the cost-to-go functions are C1 and boundedby a quadratic function, then the considered policy is optimal.

Unità di Perugia e di Roma "Tor Vergata" 12/31

Single site, Markov: a non homogeneous case (qd=qd(u))Computation of X* and Z*

Unità di Perugia e di Roma "Tor Vergata" 13/31

Single site, Markov: a non homogeneous case (qd=qd(u))

Unità di Perugia e di Roma "Tor Vergata" 14/31

Single site, Markov: a non homogeneous case (qd=qd(u))

Unità di Perugia e di Roma "Tor Vergata" 15/31

Single site, Markov: a non homogeneous case (qd=qd(u))

=30; U=22; qd1=0.06; d=20; cm=100; cp=1; qu=0.5Example

Unità di Perugia e di Roma "Tor Vergata" 16/31

Single site, Markov: a general heuristic approach for the non homogeneous case

In the general case we propose the following heuristic approach:

discretize qd(u) obtaining a multi-value failure rate function with production levels Ui and corresponding failure rates qdi

apply the results of the two level failure rate case to the multi-value case by considering each couple (Ui, Uj) and the corresponding qdi and qdj: this gives a threshold X*

ij, such that

select the longest sequence of all the X*

ij computed

*

**

*

* ),(

0

)(

ijj

ijiji

ij

XxU

ZXxU

Zx

xu *24X

*45X

Example: x2U

4U

5U

Unità di Perugia e di Roma "Tor Vergata" 17/31

For multi-value failure rate functions (as the ones obtained by discretizing qd(u) = a u+ b), Liberopoulos and Caramanis (IEEE TAC 1994) numerically found that:

if ≤1, the optimal feedback policy will operate the machine at maximum rate until a safety stock Z* is reached (i.e. it is a hedging point policy)

if >1, the optimal feedback policy will operate the machine progressively reducing the production rate from its maximum value as the inventory level increases

The heuristic proposed above confirms these findings.

Z*

x

u*(x)

x

u*(x)

Z*

Single site, Markov: a general heuristic approach for the non homogeneous case

Unità di Perugia e di Roma "Tor Vergata" 18/31

Single site, Markov: a general heuristic approach for the non homogeneous case

Example=50; d=1; cm=1000; cp=1; qu=0.5

Unità di Perugia e di Roma "Tor Vergata" 19/31

Single site, Markov: a general heuristic approach for the non homogeneous case

Example

For qd2=0.01 the points (Ui,qdi) lie on a line.

U1=50; U2=25; U3=5; qd1=0.02; qd3=0.002; d=1; cm=1000; cp=1; qu=0.5

Unità di Perugia e di Roma "Tor Vergata" 20/31

The discussion above seems in conflict with the results of Hu, Vakili and Yu (IEEE TAC, 1994) where hedging policy is proved optimal iff =0 or 1.

Remark.

This is not a conflict: if 0<<1 the optimal policy probably is a switched non-feedback policy, with the hedging point policy remaining optimal among feedback policies.

Single site, deterministic

To clarify this we have considered a deterministic system and approached it through the Maximum Principle.

g(x) =c x2

To simplify the analysis we have considered a symmetric system and a quadratic cost function

Deterioration rate:

The machine is stopped when z(t)=1. After each repair z=0.

btuatz )()(

The system is stable if and only if there exists a constant production rate (not larger than ) which is large enough to meet the demand

Unità di Perugia e di Roma "Tor Vergata" 21/31

The analysis of this case confirms the heuristic and the numerical results of the Markov system:

Single site, deterministic

If =0 or =1 (affine case) the optimal policy is -d- (similar to the hedging point policy)

x(t)

0

If 0<1, the optimal policy looks macroscopically like the -d- but an infinite number of switches between 0 and is performed to obtain a production rate equal to d

If >1, the optimal policyreduces the productionrate around 0

0

lim

0

x(t) x(t)

0

Unità di Perugia e di Roma "Tor Vergata" 22/31

Multi site, Markov, homogeneous

Each site is like the one considered by the classical paper of Bielecki and Kumar, for which the optimal policy is optimal.

x(t)

(t)u(t)

x(t)(t)

u(t)

u(t)

d

d

u(t)

A penalty cost (a) is incurred whenever a site receives items produced by the other site

A two site system

1221

2

1

),( uuaxcxcuxgi

imip

T

TdttutxgE

TJ

0

)](),([1

lim

Unità di Perugia e di Roma "Tor Vergata" 23/31

Multi site, Markov, homogeneous

Using a dynamical programming approach, in the s=(1,1) operational state, it is possible to expect the following regions, whose shape in the state space (x1,x2) is usually very complex to derive:

ii x

Vv

11

V11(x) being the cost-to-go function in the operational state (1,1)

Unità di Perugia e di Roma "Tor Vergata" 24/31

Multi site, Markov, homogeneous

Through a numerical integration of the HJB equations (for a finite inventorysystem with loss cost R, x=0.1), we have derived the following solutions, corresponding to the s=(1,1) state (arrows denote the production flow):

a=10

a=50

a=System parameters:=5, d=4, qu=1, qd=0.01, cm=50, cp=1, R=2500

Unità di Perugia e di Roma "Tor Vergata" 25/31

Multi site, Markov, homogeneous

In the case the operational state is s=(0,1) and a=50:

Unità di Perugia e di Roma "Tor Vergata" 26/31

Multi site, Markov, homogeneous

Single site theoretical values: z*=3.8, J*=7.73

Hedging point and total cost as a function of the cost parameter a:

Unità di Perugia e di Roma "Tor Vergata" 27/31

Multi site, Markov, homogeneous

Numerical solution through Hamilton Jacobi Bellman (HJB) equations

Performance index to minimizeOptimal value: J*

Js(k)(x) The minimum average expected cost on a time horizon kt, starting in

(s,x), hence it is 0 for k=0 for all s and x

Iterative equation(discretized space):

limk!1Js(k)(x) = J*

It gives the optimal minimum cost J* but not the optimal policy

Unità di Perugia e di Roma "Tor Vergata" 28/31

Multi site, Markov, homogeneous

Applying a stable stationary policy, let at steady state J=E[g(x,u)]

Then define a differential cost:

The total (not average) expected cost in [0,T] from x(0)=x and s(0)=s can be written as J T + Vs(x). For the optimal policy, J=J* and we have for its differential cost:

Vs(k)(x) The minimum expected differential cost on a time horizon kt, starting in

(s,x), hence it is 0 for k=0 for all s and x

Iterative equation(discretized space):

limk!1Vs(k)(x) = V*

s(x) From V*s(x) it is straightforward to get the optimal policy

Unità di Perugia e di Roma "Tor Vergata" 29/31

Multi site, Markov, homogeneous

Unità di Perugia e di Roma "Tor Vergata" 30/31

A single site and a multi site system have been considered in this research.

As for the single site problem:

A similar behavior has been observed in a deterministic scenario where the machine is characterized by a deterioration rate which is a deterministic function of the production rate

The optimal analytical solution for a non homogeneous Markov failure prone system has been completely derived

This solution has been used to investigate (through a heuristic approach) the property observed in the literature that a major difference arises when the failure rate of the machine is a concave or a convex function of the production rate

As for the multi site problem, a HJB approach has been used to analyze a Markov, homogeneous, two site system, and the optimal solution has been completely derived numerically for some examples

Unità di Perugia e di Roma "Tor Vergata" 31/31

The general Markov non homogeneous case could be better analyzed, improving the heuristic and studying its validity

The deterministic case should be generalized and possibly approached through a numerical algorithm to solve the maximum principle equations

As for the single site problem:

As for the multi site problem:

More general models to describe some typical dynamical phenomena of supply chains are under investigation