modelado ec.diferenciales

8/10/2019 modelado ec.diferenciales

1/35

The Sensitivity Analysis and Parameter Estimation ofMathematical Models Described by Differential Equations

Hossein ZivariPiran

[email protected]

Department of Computer Science

University of Toronto

(part of my PhD thesis under the supervision of professor Wayne Enright)

SONAD 2008 .1/21


2/35

Outline

Modeling with Differential Equations (IVPs, DDEs) Sensitivity Analysis of Models Described by DEs

Parameter Estimation of Models Described by DEs

SONAD 2008 .2/21


3/35

Modeling with DE - Formulation

An Initial Value Problem (IVP) for Ordinary Differential Equations (ODEs)

y(t) = f(t, y(t))

y(t0) = y0

Retarded Delay Differential Equations (RDDEs)

y(t) = f(t, y(t), y(t 1), , y(t )) for t0 t tF

y(t) = (t), for tt0

i =i(t, y(t))0delay(constant / time dependent / state dependent)

(t)historyfunction (constant / time dependent)

Neutral Delay Differential Equations (NDDEs)

y(t) = f(t, y(t), y(t 1), , y(t ),

y

(t +1), , y

(t +)) for t0ttF

y(t) = (t), y(t) =(t), for tt0,

SONAD 2008 .3/21


4/35

Modeling with DE - Some Areas of Application

Area Example

Ecology predator-prey

Epidemiology spread of infections

Immunology immune response models

HIV infection

Physiology human respiration system

Neural Networks

Cell Kinetics

Chemical Kinetics The Oregonator

Physics Ring Cavity Lasers , two-body problem of electrodynamics

SONAD 2008 .4/21


5/35

Modeling with DE - An Example

A neutral delay logistic Gause-type predator-prey system [Kuang 1991]

y1(t) = y1(t)(1 y1(t ) y

1(t )) y2(t)y1(t)

2

y1(t)2 + 1

y

2(t) = y2(t) y1(t)2

y1(t)2 + 1

where= 1/10,= 29/10and= 21/50, fortin[0, 30]. The history functions

are

1(t) = 33

100

1

10t

2(t) = 111

50

+ 1

10

t

fort0.

SONAD 2008 .5/21


6/35

Modeling with DE - An Example

The predator-prey model

0 5 10 15 20 25 300.31

0.32

0.33

0.34

0.35

0.36

0.37

t

y1

0 5 10 15 20 25 302.21

2.215

2.22

2.225

2.23

2.235

2.24

2.245

2.25

2.255

t

y2

SONAD 2008 .5/21


7/35

Modeling with DE - Numerical Simulation

Classical Theory of Step by Step Integration for IVPs.

Runge-Kutta (RK).

Linear Multistep (LM).

+

Continuous Solution using Polynomial Approximation.

Continuous Runge-Kutta (CRK). Linear Multistep methods have natural approximating polynomials.

DDEs: Combining an interpolation method (for evaluating delayed solution

values) with an ODE integration method (for solving the resulting ODE).

SONAD 2008 .6/21


8/35

Modeling with DE - Special Difficulties with DDEs

Derivative Discontinuities

In general

(t0)=f(t0, (t0), (t0 1), , (t0 ))

Due to the existence ofdelays, discontinuitiespropagatealong the

integration interval.

Solution issmoothedfor RDDEs but in general not for NDDEs.

The RK and LM methods fail in presence of discontinuities.

Treatment : Tracking Discontinuities and forcing them to be mesh points.

SONAD 2008 .7/21


9/35

Modeling and Sensitivity Analysis - Definitions

Parameterized Models

A parameterized IVP

y(t; p) = f(t, y(t; p); p)

y(t0) = y0(p)

For example p= []in the Van der Pol oscillator

y

1(t) = y2(t)y2(t) = (1 y

21)y2 y1

A simple parameterized DDE

y(t; p) = f(t, y(t; p), y(t (t; p)); p) for t0(p)t

y(t; p) = (t; p), for tt0(p)

SONAD 2008 .8/21


10/35

Modeling and Sensitivity Analysis - Definitions

Forward Sensitivity Analysis

The (first order) solution sensitivity with respect to the model parameterpiis

defined as the vector

si(t; p) ={

pi }y(t;p), (i= 1, . . . , L)

The second order solution sensitivity with respect to the model parameters

piandpjis defined as the vector

rij(t; p) ={

pj}si(t; p) ={

2

pjpi}y(t; p), (i, j = 1, . . . , L)

SONAD 2008 .8/21


11/35

Modeling and Sensitivity Analysis - Importance

Sensitivity information can be used to: Estimate which parameters are most influential in affecting the behavior of the

simulation. Such information is crucial for

Experimental Design Data Assimilation

Reduction of complex nonlinear models

Study of Dynamical Systems : Periodic orbits, the Lyapunov exponents, chaos

indicators, and bifurcation analysis are fundamental objects for the complete

study of a dynamical system, and they require computation of the sensitivities

with respect to the initial conditions of the problem. Evaluate optimization gradients and Jacobians in the setting of

Dynamic Optimization

Parameter Estimation

SONAD 2008 .9/21


12/35

Numerical Sensitivity Analysis of IVPs - Forward

Finite Difference Approach

{ pi

}y(t; p) y(t; p+ eipi) y(t; p)pi

Due to the rounding errors, the approximation is only

O(

T ol)with the best choice for

pi.

Internal Differentiation

y(t; p) =f(t, y(t; p); p), y(t0) =y0(p)

Differentiation + Chain Rule + Clairauts Theorem

si =

f

ysi+

f

pi, si(t0) =

y0(p)

pi, (i= 1, . . .L)

Taylor Series method using extended rules of AD [Barrio 2006].

second(or higher)-order sensitivities.SONAD 2008 .10/21


13/35

Numerical Sensitivity Analysis of IVPs - An Example

The Van der Pol oscillator

y1(t) = y2(t)

y2(t) = (1 y21)y2 y1

s1(1)(t)

s

1(2)(t)

=

0 1

2y1y2 1 (1 y2

1)

s1(1)

s1(2)

+

0

(1 y2

1)y2

s

1(1)(t) = s1(2)

s1(2)(t) = (2y1y2 1)s1(1)+(1 y21)s1(2)+ (1 y

21)y2

SONAD 2008 .11/21


14/35

Numerical Sensitivity Analysis of IVPs - An Example

The Van der Pol oscillator (p= 0.2). The decrease ofy1at t = 20can be explained as second

order sensitivities being dominant ( y1

(t= 20) 0, 2y12

(t= 20)


15/35

DDEs and Sensitivity Analysis - Governing Equations

y(t; p) = f(t, y(t; p), y((t, y; p); p), y((t, y; p); p); p)

y(t; p) = (t; p), for tt0(p)

Differentiation + Chain Rule + Clairauts Theorem

si(t)= f

y

si(t) + f

y(k)y(k)

y

si(t)+

p+si()

+ f

y()

y()

ysi(t)+

p

+si()

+

f

p

SONAD 2008 .12/21


16/35

DDEs and Sensitivity Analysis - Proper Handling of Jumps

Hybrid ODE systems [Tolsma & Barton]

Continuous transition att=,

y(+) =y()

y

pl(+) =

y

pl() +

y() y(+)

pl

Triggered by,

g(t,y,y; p) = 0

g

y

t

y

pl

+y

pl

+

g

y

y

pl+y

pl

+

g

pl+

g

t

pl= 0

SONAD 2008 .13/21


17/35


DDEs

Discontinuity points ofy(t; p),

(p) {1(p), 2(p), . . .}.

Identified by,

(r+1(p), y; p) =r(p) r= 1, 2, . . .

Can be viewed asr+1(p)being a solution of,

g(t, y; p) =(t, y; p) r(p) = 0.

Using the result for hybrid ODEs,

r+1(p)

pl=

y

ypl

+ pl

r(p)pl

y

y + t

forr1

1(p)pl

= t0(p)pl

SONAD 2008 .13/21


18/35


Integration (first order sensitivities)

1. Initialize (1=t0(p)).

2. r1.

3. Integrate the equations up to aC

1-discontinuity point (r+1

).

4. Update the state variables (sensitivities) using

y

pl

(+r+1) = y

pl

(r+1) + y(r+1) y(+r+1)r+1(p)

pl

5. rr+ 1and restart (3).

SONAD 2008 .13/21

S


19/35

DDEs and Sensitivity Analysis - Examples

The predator-prey model

y1(t) = y1(t)(1y1(t)y1(t))

y2(t)y1(t)2

y1(t)2 + 1

y2(t) = y2(t)

y1(t)

2

y1(t)2 + 1

where = 1/10, = 29/10and = 21/50, fortin[0, 30]. The history functions are

1

(t) = 33

100

1

10t

2(t) = 111

50 +

1

10t

fort 0.

Parameters are

p= [, , , a, b, c, d]

where

1(t) = a + b t2(t) = c + d t

SONAD 2008 .14/21

DDE d S i i i A l i E l


20/35


The predator-prey model (structure-related parameters, y1)

0 5 10 15 20 25 300.31

0.32

0.33

0.34

0.35

0.36

0.37

t

y1

0 5 10 15 20 25 302.5

2

1.5

1

0.5

0

0.5

1

1.5

2

2.5

t

y1

0 5 10 15 20 25 300.08

0.06

0.04

0.02

0

0.02

0.04

0.06

0.08

t

y1

0 5 10 15 20 25 300

0.5

1

1.5

2

2.5

t

y1

SONAD 2008 .14/21

DDE d S i i i A l i E l


21/35


The predator-prey model (structure-related parameters, y2)

0 5 10 15 20 25 302.21

2.215

2.22

2.225

2.23

2.235

2.24

2.245

2.25

2.255

t

y2

0 5 10 15 20 25 300.2

0.1

0

0.1

0.2

0.3

0.4

0.5

t

y2

0 5 10 15 20 25 300.005

0

0.005

0.01

0.015

0.02

0.025

0.03

t

y2

0 5 10 15 20 25 3018

16

14

12

10

8

6

4

2

0

t

y2

SONAD 2008 .14/21


22/35

DDE d S iti it A l i E l


23/35


The predator-prey model (history-related parameters, y2)

0 5 10 15 20 25 300.5

0

0.5

1

1.5

2

t

y2

a

0 5 10 15 20 25 300.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0

t

y2b

0 5 10 15 20 25 300.2

0

0.2

0.4

0.6

0.8

1

1.2

t

y2

c

0 5 10 15 20 25 300.45

0.4

0.35

0.3

0.25

0.2

0.15

0.1

0.05

0

0.05

t

y2

d

SONAD 2008 .14/21

M d li d P t E ti ti D fi iti


24/35

Modeling and Parameter Estimation - Definitions

Parameter Estimation Problem

A System of Parameterized IVP

y(t; p) = f(t, y(t; p); p)

y(t0) = y0(p)

or DDE

y

(t; p) = f(t, y(t; p), y(t (t; p)); p) for t0(p)ty(t; p) = (t; p), for tt0(p)

A Set of Data (Observations/Measurements)

{Y(i) y(i; p)}

Estimate p by minimizing an objective function.

e.g.W(p) =

i

[Y(i) y(i; p)]2

.

SONAD 2008 .15/21

M d li d P t E ti ti I t


25/35

Modeling and Parameter Estimation - Importance

Physical/Biological Phenomenon

Mathematical Model

Refined Mathematical Model

Practical Mathematical Model

Physical Laws / Empirical Rules

Sensitivity Analysis

Parameter Estimation

Observed Data

SONAD 2008 .16/21

M d li d P t E ti ti O ti i


26/35

Modeling and Parameter Estimation - Optimizers

Algorithms for Nonlinear Least-Squares Unconstrained

minp

W(p) =i

[Y(i)y(i;p)]2 .

Levenberg-Marquardt

Variations of Sequential Quadratic Programming (SQP)

Constrained

minp

W(p) =i

[Y(i)y(i;p)]2 ,

cj(p) = 0, j E,

cj(p)0, j I.

Sequential Quadratic Programming (SQP)

Thesmoothnessof functions involved in the problem, theobjective function and constraints, is anecessaryassumption.

SONAD 2008 .17/21

N merical Parameter Estimation of IVPs


27/35

Numerical Parameter Estimation of IVPs

Steps:

1. Choose an initial guess for the parameters

2. Solve model equations

3. Check optimality conditions, (if satisfied stop).4. Choose a better value for the parameters and continue with (2)

If the optimization method needs to compute thegradientor theHessianof the objective

function, W(p)

pl

= 2

i

[Y(i)y(i;p)]

y(i;p)

pl

2W(p)plpm

= 2

i

y(i;p)pl

y(i;p)pm

[Y(i)y(i;p)]

2y(i;p)plpm

the sensitivity equations are usually used to provide the required values. An alternative

approach is to use a divided-difference approximation.

SONAD 2008 .18/21

DDEs and Parameter Estimation


28/35


[ Paul , 1997 ] :

Assume that jumps in the derivative ofy(t; p)with respect totoccur at the points

(p) {1(p), 2(p), . . .}.

Such discontinuities, when arising from the initial pointt0(p)(and the initial function(t; p)),may propagate intoW(p)via the solution values {y(i; p)}.

The first and second order partial derivatives of the objective function are

W(p)

pl

= 2i

[Y(i)y(i;p)]

y(i;p)pl

2W(p)

plpm = 2i

y(i;p)

pl y(i;p)

pm [Y(i)y(i;p)]2y(i;p)

plpm

SONAD 2008 .19/21



29/35


Non-smooth optimization

Consider the continuous function

y(t) =

5(t ) + c, ift < 5(t

) + c, ift

with discontinuous derivative

y(t) = 5, ift < 5, ift

and observed value ofyat the discontinuity point = 4.

SONAD 2008 .19/21



30/35


0 1 2 3 4 5 6 70

5

10

15

20

25

t

y(t)

c

SONAD 2008 .19/21



31/35


3 3.5 4 4.5 560

40

20

0

20

40

60

W()

W()

SONAD 2008 .19/21



32/35


Try to find using MATLABs unconstrained minimization routine fminunc

31 iterations.

Try to use MATLABs constrained minimization routinefmincon

with the added constraint

4

2 iterations.

Try to use MATLABs constrained minimization routinefmincon

with the added constraint 4

2 iterations.

SONAD 2008 .19/21

DDEs and Parameter Estimation Safe Minimization


33/35

DDEs and Parameter Estimation - Safe Minimization

Appearance of Non-smoothness

Partial derivatives(gradient) of the objective function

W(p)

pl

= 2i

[Y(i) y(i;p)]

y(i;p)

pl

2W(p)

plpm

= 2i

y(i;p)

pl

y(i;p)

pm

[Y(i) y(i;p)]

2y(i;p)

plpm

Recall the jump equation for sensitivities

y

pl(+r+1) =

y

pl(r+1) +

y(r+1) y(+r+1)

r+1(p)

pl

The General Rule : A jump occurs inW(p)when

a discontinuity pointr+1(p)passesa data pointi

SONAD 2008 .20/21

DDEs and Parameter Estimation Safe Minimization


34/35

DDEs and Parameter Estimation - Safe Minimization

Avoiding The Non-smoothness

t

y

r(p) i r+1(p)

t

y

r(p)

ir+1(p)

true solution

continiously extended solution

discontinuity point

data point

Force the ordering by addingr(p) i r+1(p)to the set of constraints.

The partial derivatives (gradient) of the new constraints ,r

+1(p)

p ,can be computed recursively.

SONAD 2008 .20/21

DDEs and Parameter Estimation A Test Case


35/35

DDEs and Parameter Estimation - A Test Case

Estimatingfor the predator-prey model

y1(t) = y1(t)(1y1(t)y1(t))

y2(t)y1(t)2

y1(t)2 + 1

y

2(t) = y2(t) y1(t)2

y1(t)2 + 1

Start with up to10%random perturbation in original,

and up to3%randomly perturbedy(t; )as Data (Y).

Fors we choose10random points, one of which is a discontinuity point.

Run the parameter estimator3times.

Results

Estimator Choice FCN OBJ

Very Simple 72,2697 0.000142917

Using Sensitivities 26,942 0.000142917

Adding Constraints 9,916 0.000142917

Documents

modelado ec.diferenciales