Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
1 / 43
Week 5, Lecture 9 - Dynamical models
Aaron Meyer
2 / 43
Outline
I Administrative IssuesI Review ODE models
I Some common constructionsI Classic analysis:
I Stability analysisI Pseudo-steady-state
I Implementation:I Numerical integrationI Stiff systemsI Matrix exponentials
Slides partly adapted from those by Bruce Tidor.
3 / 43
Ordinary Differential Equations
I ODE models are typically most useful when we already havean idea of the system componentsI As opposed to data-driven approaches when we don’t know
how to connect the dataI Incredibly powerful for making specific predictions about how a
system worksI Limits of these approaches:
I Results can be extremely sensitive to missing components ormodel errors
I Can quickly explode in complexityI May rely on variables that are impossible to measure
4 / 43
Applications of ODE models: Molecular kinetics
Remember BE100!
Let’s say we have to ligands that dimerize, then this dimer binds toa receptor as one unit:
Lf + Lf ↔ LD
LD +Rf ↔ Rb
If we want to know about how these species interact, we can modeltheir behavior with the rate equations that describe this process.
5 / 43
Applications of ODE models: Pharmacokinetics
C1
V1
druginjection
ke
C2
V2
k1
k2
centralcompartment
peripheralcompartment
6 / 43
Applications of ODE models: Pharmacokinetics
I Central compartment corresponds to the plasma in the body.I V1 is the distribution volume of plasma in the body.I C1 is the concentration of drug in the plasma.
I Peripheral compartment represents a group of organs thatsignificantly take up the particular drug.I V2 is the volume of these group of organs.I C2 is the concentration of drug in the group of organs.
7 / 43
Applications of ODE models: Pharmacokinetics
I ke is the rate constant for clearance.I keC1V1 is the mass of drug/time that’s cleared.
I k1 is the rate constant for mass transfer from the central toperipheral compartment.I k1C1V1 is the mass of drug/time that transfers from the
central to peripheral compartment.I k2 is the rate constant for mass transfer from the peripheral
to central compartment.I k2C2V2 is the mass of drug/time that transfers from the
peripheral to the central compartment.
8 / 43
Applications of ODE models: Pharmacokinetics
I Have a bolus i.v. injectionI No drug in both compartments for t<0I D µg of drug administered at once at t=0I Drug distribution occurs instantaneously in the central
compartment.I Also get well-mixed instantaneously.I Concentration in central compartment at t = 0 is D µg/mL
I No chemical reactions in the compartment
9 / 43
Applications of ODE models: Population kinetics
Lotka-Volterra Equations
10 / 43
Note about difference from other models we’ve covered
I ODE models can be part of inference techniques just aselsewhereI If we have a symbolic integral, then fitting an ODE model to
data is just non-linear least squaresI But we often don’t have a symbolic expression of the answer
I Have to simulate the model every timeI Can only focus on the input-output we get from the black box
I In this respect, what we do with ODE models will be verysimilar to what you could do with any computationalsimulation
11 / 43
Analytic vs Numerical Modeling
I AnalyticI Wider range of parametersI Avoid numerical problemsI Physical intuition more directI Often must simplify model
I NumericalI Can handle complex modelsI Dependence on parameters & initial conditionsI Physical insight may be difficult to extractI Convergence, numerical stability
Reality often requires handling in between:
I Use analytic treatment to study entire parameter spaceI Use numerical treatment to study interesting regionsI Use both to handle complex behavior
12 / 43
Stability Analysis
I Can solve for steady-states of a system
δF
δt= 0
I Results of this can be both stable or unstable pointsI With stable points, slope of δFδt is negativeI In multivariate case, this means eigenvalues of Jacobian are
negativeI Steady-state points aren’t necessarily realistic or feasible!
I NNLSQ can solve for pointsI Only simulating system ensures they are accessible
13 / 43
Generalization
I Linear models are easier to simulate and understand thannon-linearI Linearity: If x1 and x2 are both solutions, then c1x1 + c2x2 is
also a solutionI Linear systems tend to be separable (effective decoupling)I Non-linear systems exhibit interesting properties
14 / 43
Linearity & Separability
2
• Linear models are easier to simulate and understand than non-linear
Generalization
understand than non linear– Linearity:
If x1 and x2 are both solutions, then c1x1 + c2x2 is also a solution
– Linear systems tend to be separable (effective decoupling)
¸·
¨§¸
ᬤ
¸·
¨§ c aa 14131211 NNNN
¸·
¨§¸
ᬤ
¸·
¨§
cc
EDO
ED 11 000
¸¸¸
¹
·
¨¨¨
©
§�
¸¸¸
¹¨¨¨
©
¸¸¸
¹
·
¨¨¨
©
§
ccc
dcb
dcb
44434241
34333231
24232221
NNNNNNNNNNNN
¸¸¸
¹¨¨¨
©
�¸¸¸
¹¨¨¨
©
¸¸¸
¹¨¨¨
© ccc
GJE
OO
O
GJE
44
33
22
000000000
• Non-linear systems exhibit interesting properties
Linearity & Separability
¸¸¸·
¨¨¨§
�¸¸¸·
¨¨¨§
¸¸¸·
¨¨¨§
ccc
cba
cba
24232221
14131211
NNNNNNNNNNNN
¸¸¸·
¨¨¨§
�¸¸¸·
¨¨¨§
¸¸¸·
¨¨¨§
ccc
JED
OO
O
JED
22
11
000000000
¸¸¹
¨¨©¸
¹¨©
¸¸¹
¨¨© c d
cdc
44434241
34333231NNNNNNNN ¸
¹¨©¸
¹¨©
¸¹
¨© c G
JO
OGJ
44
33000
000
ab D E
c d J G
15 / 43
Phase Portraits
3
Phase Portraits
)(),(
),(
2122
2111xfx
xxfx
xxfx &&�&
�
� ��� o�
°¿
°¾½
),( 2122 xxfx °¿
)(tx&1t
)( 1tx�&
• Non-linear systems– No general analytic approach to finding trajectory– So, goal is to understand qualitative trajectory behavior
Features in Phase Portraits
B
1. Fixed Points (A, B, C). Steady states
A
B
CD
2. Closed Orbits (D). Periodic solutions3. Flow patterns in trajectory
A & C are similar to each other, different from B
4. Stability of fixed points & closed orbitsA, B, & C are unstable, D is stable
Non-linear systems
I No general analytic approach to finding trajectoryI So, goal is to understand qualitative trajectory behavior
16 / 43
Features in Phase Portraits
3
Phase Portraits
)(),(
),(
2122
2111xfx
xxfx
xxfx &&�&
�
� ��� o�
°¿
°¾½
),( 2122 xxfx °¿
)(tx&1t
)( 1tx�&
• Non-linear systems– No general analytic approach to finding trajectory– So, goal is to understand qualitative trajectory behavior
B
A
B
CD
17 / 43
Solving a Set of Equations for Phase Portrait
I Numerical computationI i.e., Runge-Kutta integration
I QualitativeI Sufficient for some purposes
I AnalyticI Elegant, though not always tractable
18 / 43
Example – fixed points
4
Solving a Set of Equations for Phase Portrait
• Numerical computation – i e Runge-Kutta integration– i.e., Runge-Kutta integration
• Qualitative– Is often sufficient for most purposes
• Analytic– Elegant, though not always tractable
Example – fixed points
yyexx y
� � �
��
Nonlinear, becausecan’t be represented as
dycxybyaxx
� �
��
dycxy � Step 1: Find fixed points
Fixed points (also called stationary points) are those points where the time-derivative of each coordinate is zero. 0 and 0 yx ��
0010
� ��� ��
yyexex yy
)0,1(),(at point fixed one Thus,
00
�
��
yx
yy
19 / 43
Example – stability
Step 2: Determine stability of fixed pointsI If the systems moves slightly away from each fixed point, will
it return or will it move further away?I Another way to ask the same question is to ask whether, as
time approaches infinity, does the system tend toward or awayfrom a given stable point.
I Note y solution must be of form:I y = y0e
−t (because y = dydt = −y)
I So y → 0 for t→∞I Thus, x = x+ e−y becomes x→ x+ 1 for long times
I This has exponentially growing solutionsI Toward ∞ for x > −1 and −∞ for x < −1
Thus, overall solution grows exponentially in at least one di-mension, and so is unstable.
20 / 43
Example – nullclines
Step 3: Sketch nullclinesNullclines are the sets of points for which x = 0 or y = 0, so flow iseither horizontal or vertical.
5
Example – stability
Step 2: Determine stability of fixed pointsIf the systems moves slightly away from Another way to ask the same question each fixed point, will it return or will it move further away?
.for 0so),(because
formofbemust solution Note
0
foo
� �
tyy
dtdyyeyy
yt �
is to ask whether, as time approaches infinity, does the system tend toward or
away from a given stable point.
y
)1for and 1for (towardssolutions growinglly exponentia has This times.longfor 1becomes Thus,
��f��!f�
�o� �
xx
xxexx y ��
Thus, overall solution grows exponentially in at least one dimension, and so is unstable
y
yexx
xxxyyy
�� o
�!� � o�
00for verticalis Flow
1for right the tois flow so here, 1axis-at x horizontal is flow 00for 0
�
��
x
y
00
�!
yx��
00
��
yx��
–1
yex �� 0
00
!!
yx��
00
!�
yx��The nullclines partition the
space into classes of flow direction:
¯®¿¾½²¢ 0, yx ��
1
21 / 43
Example – computed
Step 4: Plot flow lines
6
Example – computed
yStep 4: Plot flow lines
x–1
Notice that the major vertical blue flow line does not coincide with the nullcline.
This is a non-linear version of a saddle point.
Can we identify saddle-like behavior in linearized
version of system?
Existence & Uniqueness
)(xfx &&�& and given initial conditionNon-linear
• Existence & uniqueness of solution guaranteed if is continuously differentiable
• Corollary: Trajectories do not intersect, because if they did, then there would be two solutions for the
f&
y ,same initial condition at the crossing point
22 / 43
Existence & Uniqueness
Non-linear x = f(x) and given an initial condition.
I Existence and uniqueness of solution guaranteed if f iscontinuously differentiable
I Corollary: Trajectories do not intersect, because if they did,then there would be two solutions for the same initialcondition at the crossing point
23 / 43
Linearization About Fixed Points
7
Linearization About Fixed Points
)()(0
),(point fixed with systemlinear -non a be ),(),(Let
****
**
f
yxyxgyyxfx¿¾½
¯®
��
),(),(0 **** yxgyxf
point fixed from deviations be Let *
*
¿¾½
¯®
� � yyvxxu
Change of variable
*
),,( Likewise,
expansion seriesTaylor ),,(),(,(constant) is (
22
22**
*)*
*
uvvuOygv
xguv
uvvuOyfv
xfuyxfyvxuf
xxu
�ww
�ww
�ww
�ww
�
��
�
��linear
0 0
Solving Linearized Systems
¸¹·¨
©§�
¸¸¸¸
¹
·
¨¨¨¨
©
§
ww
ww
ww
ww
¸¹·¨
©§
vu
yg
xg
yf
xf
vu��
¹© ww yx
0det
00)(
)(let
·§ �
¸¹·¨
©§
��
�
¸¹·¨
©§
OO
OO
OO OO
O
bavdc
bavIA
vAvveAve
vetxdcbaAxAx
tt
t
&
&(&(&&(&
&&(&(�&
If O1zO2, then v1 & v2
24
24
00))((
0det
2
2
2
1
2
'��
'��
'�� ���
¸¹·¨
©§
�
WWOWWO
WOOOO
Obcda
dc 1 2, 1 2are linearly independent and solutions of the following form are valid.
W=trace '=determinant
221121)( vecvectx tt &&& OO �
24 / 43
Solving Linearized Systems
7
Linearization About Fixed Points
)()(0
),(point fixed with systemlinear -non a be ),(),(Let
****
**
f
yxyxgyyxfx¿¾½
¯®
��
),(),(0 **** yxgyxf
point fixed from deviations be Let *
*
¿¾½
¯®
� � yyvxxu
Change of variable
*
),,( Likewise,
expansion seriesTaylor ),,(),(,(constant) is (
22
22**
*)*
*
uvvuOygv
xguv
uvvuOyfv
xfuyxfyvxuf
xxu
�ww
�ww
�ww
�ww
�
��
�
��linear
0 0
Solving Linearized Systems
¸¹·¨
©§�
¸¸¸¸
¹
·
¨¨¨¨
©
§
ww
ww
ww
ww
¸¹·¨
©§
vu
yg
xg
yf
xf
vu��
¹© ww yx
0det
00)(
)(let
·§ �
¸¹·¨
©§
��
�
¸¹·¨
©§
OO
OO
OO OO
O
bavdc
bavIA
vAvveAve
vetxdcbaAxAx
tt
t
&
&(&(&&(&
&&(&(�&
If O1zO2, then v1 & v2
24
24
00))((
0det
2
2
2
1
2
'��
'��
'�� ���
¸¹·¨
©§
�
WWOWWO
WOOOO
Obcda
dc 1 2, 1 2are linearly independent and solutions of the following form are valid.
W=trace '=determinant
221121)( vecvectx tt &&& OO �
25 / 43
Example
8
Example
4111
32
6
1
)32()(24
2
1
2
1
¸¹·¨
©§
¸¹·¨
©§
¯®
�
� 'o�
°¿
°¾
½
� �
�
v
v
yxyxy
yxx
&
&
�
�
OOW
cond. init. from 1with 41
11)(
4)3,2(),(
213
22
1
20
¸¹·¨
©§��¸
¹·¨
©§
¹©�¿
�
ccecectx
yx
tt
t
&
Can draw phase portrait directly from eigenvalues & eigenvectors:
y (unstable) 2,11
11 ¸¹·¨
©§ Ov&Saddle Point
x
¹©
(stable) 3,41
22 � ¸¹·¨
©§� Ov&
Saddle Point(one positive real and one negative real eigenvalue)
More ExamplesyStable Node
(two real negative eigenvalues)
ySpiral(two complex eigenvalues;x
yUnstable Node(t l iti
xeigenvalues;Can be growing or shrinking)
yCenter( l i i
x
(two real positive eigenvalues)
x
(purely imaginary eigenvalues)
Also, equal eigenvalues lead to stars & degenerate nodes
26 / 43
More Examples
8
Example
4111
32
6
1
)32()(24
2
1
2
1
¸¹·¨
©§
¸¹·¨
©§
¯®
�
� 'o�
°¿
°¾
½
� �
�
v
v
yxyxy
yxx
&
&
�
�
OOW
cond. init. from 1with 41
11)(
4)3,2(),(
213
22
1
20
¸¹·¨
©§��¸
¹·¨
©§
¹©�¿
�
ccecectx
yx
tt
t
&
Can draw phase portrait directly from eigenvalues & eigenvectors:
y (unstable) 2,11
11 ¸¹·¨
©§ Ov&Saddle Point
x
¹©
(stable) 3,41
22 � ¸¹·¨
©§� Ov&
Saddle Point(one positive real and one negative real eigenvalue)
More ExamplesyStable Node
(two real negative eigenvalues)
ySpiral(two complex eigenvalues;x
yUnstable Node(t l iti
xeigenvalues;Can be growing or shrinking)
yCenter( l i i
x
(two real positive eigenvalues)
x
(purely imaginary eigenvalues)
Also, equal eigenvalues lead to stars & degenerate nodes
27 / 43
Classification of Fixed Points
9
Classification of Fixed Points
WUnstable nodes
042 '�W
'
Unstable spirals
Stable spiralsSadd
le po
ints
Centers
Stable nodes
Non-isolatedfixed points Stars, degenerate nodes
Relevance for Nonlinear Dynamics
• So, we have said that we can find fixed points of nonlinear dynamics, linearize about each fixed point, and y pcharacterize the dynamics about each fixed point in the non-linear model by the corresponding linear model.
• Is this always true? Do the nonlinearities ever disturb this approach?
• A theorem can be proven which states – That all the regions on the previous slide are “robust” (nodes,That all the regions on the previous slide are robust (nodes,
spirals, saddles) and correspond between linear and nonlinear models.
– But that all the lines on the previous slide are “delicate” (centers, stars, degenerate nodes, non-isolated fixed points) and can have different behaviors in linear and non-linear models.
28 / 43
Relevance for Nonlinear Dynamics
I So, we have said that we can find fixed points of nonlineardynamics, linearize about each fixed point, and characterizethe dynamics about each fixed point in the non-linear modelby the corresponding linear model.
I Is this always true? Do the nonlinearities ever disturb thisapproach?
I A theorem can be proven which statesI That all the regions on the previous slide are “robust” (nodes,
spirals, saddles) and correspond between linear and nonlinearmodels.
I But that all the lines on the previous slide are “delicate”(centers, stars, degenerate nodes, non-isolated fixed points)and can have different behaviors in linear and non-linearmodels.
29 / 43
Bifurcations
I The phase portraits we have been looking at describe thetrajectory of the system for a given set of initial conditions.However, for “fixed” parameters (rate constants in eqns, forinstance).
I What we might like is a series of phase portraitscorresponding to different sets of parameters.
I Many will be qualitatively similar. The interesting ones willbe where a small change of parameters creates a qualitativechange in the phase portrait (bifurcations).
I What we will find is that fixed points & closed orbits can becreated/destroyed and stabilized/destabilized.
30 / 43
Saddle-Node Bifurcation
10
Bifurcations
• The phase portraits we have been looking at describe the trajectory of the system for a given set of initial conditions. j y y gHowever, for “fixed” parameters (rate constants in eqns, for instance).
• What we might like is a series of phase portraits corresponding to different sets of parameters.
• Many will be qualitatively similar. The interesting ones will be where a small change of parameters creates abe where a small change of parameters creates a qualitative change in the phase portrait (bifurcations).
• What we will find is that fixed points & closed orbits can be created/destroyed and stabilized/destabilized.
Saddle-Node Bifurcation
yyxx
� �
�� 2P
x
y
x
y
x
y
P�> 0 P�= 0 P�< 0
31 / 43
Genetic Control Network
Griffith (1971) model of genetic control:
I x = protein concentrationI y = mRNA concentration
11
Genetic Control Network
Griffith (1971) model of genetic controlx = protein concentrationy = mRNA concentrationy
byxx
y
yaxx
��
��
2
2
1�
� protein degrades and is synthesized from mRNA
mRNA degrades and is stimulated by protein dimer
DNA
mRNA (y)
protein (x)dimerization
a
b
Genetic Control Network
nullclines:
� �2
2
2
2
110
0
xbx
ybyxx
y
axyyaxx
� o�
�
o��
�
�
ychanging a with fixed bonly changes slope of blueline and can change numberof stationary points
x
32 / 43
Genetic Control NetworkBiochemical version of a bistable switch:
1. Only stable points are no protein and mRNA or a fixedcomposition
2. If degradation rates too great, only stable point is origin
12
Genetic Control Network
Biochemical version of a bistable switch:(1) only stable points are no protein and mRNA or a fixed composition(2) if degradation rates too great, only stable point is origin
y
x
33 / 43
Implementation - Testing
I Many properties one can testI Mass balanceI Changes upon parameter adjustment
I Good to test these before and after integration
34 / 43
Implementation
SciPy provides two interfaces for ODE solving:
I scipy.integrate.odeI scipy.integrate.odeint
Notes:
I Both can solve stiff and non-stiff equations.I ode has a number of different methods. Pay attention to the
“set_integrator” option.
35 / 43
Implementation - ExampleThe second order differential equation for the angle θ of a pendulumacted on by gravity with friction can be written:
θ′′(t) + b ∗ θ′(t) + c ∗ sin(θ(t)) = 0
where b and c are positive constants, and a prime (‘) denotes aderivative. To solve this equation with odeint, we must first convertit to a system of first order equations. By defining the angularvelocity ω(t) = θ′(t), we obtain the system:
θ′(t) = ω(t)
ω′(t) = −b ∗ ω(t)− c ∗ sin(θ(t))
36 / 43
Implementation - Example
Let y be the vector [θ, ω]. We implement this system in python as:
def pend(y, t, b, c):theta, omega = ydydt = [omega, -b*omega - c*np.sin(theta)]return dydt
We assume the constants are b = 0.25 and c = 5.0:
b, c = 0.25, 5.0
37 / 43
Implementation - Example
For initial conditions, we assume the pendulum is nearly vertical withθ(0) = π−0.1, and it initially at rest, so ω(0) = 0. Then the vectorof initial conditions is
y0 = [np.pi - 0.1, 0.0]
We generate a solution 101 evenly spaced samples in the interval0 ≤ t ≤ 10. So our array of times is:
t = np.linspace(0, 10, 101)
38 / 43
Implementation - Example
Call odeint to generate the solution. To pass the parameters b andc to pend, we give them to odeint using the args argument.
from scipy.integrate import odeintsol = odeint(pend, y0, t, args=(b, c))
The solution is an array with shape (101, 2). The first column is θ(t),and the second is ω(t). The following code plots both components.
39 / 43
Implementation - Example
import matplotlib.pyplot as pltplt.plot(t, sol[:, 0], 'b', label='theta(t)')plt.plot(t, sol[:, 1], 'g', label='omega(t)')plt.legend(loc='best')plt.xlabel('t')plt.grid()plt.show()
40 / 43
Implementation - Example
41 / 43
Implementation - Stiff Systems
I Very roughly, most ODE solvers take steps inverselyproportional to the rate at which the state is changing
I For systems where there are two processes operating ondiffering timescales, this can be problematicI If everything happens really fast, the system will come to
equilibrium quicklyI If everything is slow, you can take longer steps
I Stiff solvers additionally require the Jacobian matrixI This very roughly allows them to keep track of these
differences in timescalesI odeint can automatically find this for you
I Sometimes it’s faster/better to provide this as parameter Dfun
42 / 43
Implementation - Matrix Exponential
If J is the Jacobian matrix of an ODE model, y(t) = eJty0.
Matrix exponential is also implemented.
I scipy.linalg.expmI This method is numerically stable, but there are faster
implementations elsewhere.I A commonly used package is expokit
For linear systems, this can be >1000x faster.
43 / 43
Further Reading
I scipy.linalg.expmI scipy.integrate.odeintI Steven Strogatz, Nonlinear Dynamics and Chaos