Embed Size (px)
Citation preview
CS537
Numerical Analysis
Lecture
Numerical Solution of Ordinary Differential EquationsNumerical Solution of Ordinary Differential Equations
Professor Jun ZhangProfessor Jun Zhang
Department of Computer ScienceUniversity of Kentuckyy y
Lexington, KY 40206‐0046
April 15, 2010
What is ODE
An Ordinary Differential Equation (ODE) is an equation that involves one or more derivatives of an unknown function
A solution of a differential equation is a specific function that satisfies the equation
For the ODE
The solution is
texdtdx
=−The solution is
where c is a constant. Note that the solution is not unique
tt cetetx +=)(q
Also, the function has derivatives only respect to one variable
2
For derivatives with multiple variables, the equation is usually called partial differential equation (PDE)
An Example of ODE p
A ODE l ti f ti f th ti t
3
An ODE solution as a function of the time t
Initial‐Value Problem
A differential equation does not, in general, determine a unique solution f i Addi i l di i d d d i ifunction. Additional conditions are needed to determine a unique solution
The initial value problem has the following standard formThe initial‐value problem has the following standard form
⎩⎨⎧
==
saxxtfx
)(),('
Here x is a function of t, x(a) is the given initial value of the problem. We would like to determine the value of x at any time before or after a
⎩
Some examples of initial value problems and solutions
SolutionValue InitialEquationt
4436)1(16'10)0(1'
2 +−==−=−==+=ttxxtx
exxxx t
Numerical Solution
Analytical solutions in closed forms are only available for special differential equationsdifferential equations
Most differential equations encountered in practical applications do not have analytical solutions
Some analytical solutions may be too complicated to be useful in practice
A numerical solution of a differential equation is usually obtained in the form of a table of discrete values
Interpolation procedures can be used to obtain all values of theInterpolation procedures can be used to obtain all values of the approximate numerical solution within a given interval
A large amount of data may be displayed as a solution curve on a color
5
A large amount of data may be displayed as a solution curve on a color graphical monitor
ODE and Intergration
Consider the differential equation
⎪⎩
⎪⎨⎧ =
sax
xrfdrdx
)(
),(
We integrate it from t to t+h
⎪⎩ = sax )(
∫∫++
=htht
drrxrfdx ))((
We obtain
∫∫ =tt
drrxrfdx ))(,(
∫+
+=+ht
drrxrftxhtx ))(()()(Replacing the integral with one of the numerical integration rules we studied before, we obtain a formula for solving the differential equation
∫+=+t
drrxrftxhtx ))(,()()(
6
, g q
Numerical Rules
Using the left rectangular approximation formula, we have
∫+ht
which leads to the Euler’s method
∫+
≈ht
ttxthfdrrxrf ))(,())(,(
On the other hand, the trapezoid rule leads to
))(,()()( txthftxhtx +=+
and the implicit formula
))](,())(,([2
))(,( htxhtftxtfhdrrxrfht
t+++≈∫
+
and the implicit formula,
))](,())(,([2
)()( htxhtftxtfhtxhtx ++++=+
7
because x(t+h) appears on both sides
Taylor Series Methodsy
The Taylor series of a function
+++=+ )('"1)(''1)(')()( 32 txhtxhthxtxhtx
h l l f ( h) h h f
LL ++++
+++=+
)(!
1)(!4
1
)(!3
)(!2
)()()(
)()4(4 txhm
txh
txhtxhthxtxhtx
mm
gives the numerical solution of x(t+h) when we truncate the series after the (m+1)st terms
If h is small and if we knowIf h is small, and if we know
the Taylor series computes very accurate value of x(t+h)
)(,),("),('),( )( txtxtxtx mKthe Taylor series computes very accurate value of x(t+h).
If we truncate the series after the (m+1)st term, the method is called the Taylor series method of order m
8
y
The Taylor series method of order 1 is known as the Euler’s method
Euler’s Method
The initial value problem
⎧ ttf ))(('
over the interval [a,b], we use the first two terms of the Taylor series⎩⎨⎧
==
axaxtxtfx
)())(,('
over the interval [a,b], we use the first two terms of the Taylor series
This gives the formula of the Euler’s method
)(')()( thxtxhtx +≈+
The computation starts from t=a and stops at t=b with n steps of size
))(,()()( txthftxhtx +=+
nabh )( −
=
9
Euler’s method
Using Euler’s method to solve an ODE. The typical behavior of the computed solution is that it runs away from the exact solution as more steps are taken, due to the accumulation of errors at each step
10
Taylor Series Method of Higher Ordery g
Consider the initial‐value problem
1' 32 ++= txx
d ff h l
4)1(1
−=++=
xtxx
We differentiate the equation several times, as
3'2"1'
2
32 ++=
ttxx
6''2"2'"3'2"
)4(
2
++=+=
txxxxxtxxx
These terms can be applied in order in the Taylor series method and the computed solution will be more accurate. For example x(2)=4.235841
6"'6'"2)4( ++= xxxxx
11
p p ( )from the Euler’s method. x(2)=4.3712096 from the use of the 4th
derivative. The exact value to five significant figures is x(2)=4.3712
Type of Errorsyp
When we truncate the Taylor series, we introduce the local truncation error 1 55
The truncation error is of order h5 , or O(h5).
)(120
1 55 ξxh
The second type of error is the accumulation effects of all local truncation errors
The computed value of x(t+h) is in error because x(t) is already in error, due to the previous truncation error, and the current step involves another truncation erroranother truncation error
The roundoff error can also contribute to the accumulation of error
12
These errors may be magnified by succeeding steps
Runge‐Kutta MethodsThe high order Taylor method for solving the initial‐value problem
⎩⎨⎧
==
xaxxtfx
)(),('
needs the derivatives of x by differentiating the function f . This is not convenient. We need some highly accurate methods that make use of fdirectly
⎩ = axax )(
directly
The Runge‐Kutta methods are a class of methods using multiple evaluations of f, not its derivatives, to enhance computational accuracyevaluations of f, not its derivatives, to enhance computational accuracy
We will illustrate the procedure to derive the Runge‐Kutta method of order 2, and give the formula for Runge‐Kutta method of order 4, which g gis popularly used
We assume that f(t,x) can be computed for any (t,x), so that x(t), x(t+h),
13
x(t+2h), …, can be computed
Taylor Series for f(x,y)y f( ,y)
We can expand the Taylor series in two variables1
i∞ ⎞⎛ ∂∂
The first few right hand terms are
),(!1),(
0yxf
yk
xh
ikyhxf
i∑∞
=⎟⎟⎠
⎞⎜⎜⎝
⎛∂∂
+∂∂
=++
The first few right‐hand terms are
),(0
fyxfy
kx
h =⎟⎟⎠
⎞⎜⎜⎝
⎛∂∂
+∂∂
),(1
yfk
xfhyxf
yk
xh
yx
∂∂
+∂∂
=⎟⎟⎠
⎞⎜⎜⎝
⎛∂∂
+∂∂
⎟⎠
⎜⎝ ∂∂
2),( 2
22
2
2
22
2
yfk
yxfhk
xfhyxf
yk
xh
yxyx
∂∂
+∂∂
∂+
∂∂
=⎟⎟⎠
⎞⎜⎜⎝
⎛∂∂
+∂∂
∂∂⎠⎝ ∂∂
14
M yyy ⎠⎝
Taylor Series for f(x,y)y f( ,y)
The Taylor series can be truncated as
We now use subscripts to denote the partial derivatives
),(!
1),(!
1),(1
0yxf
yk
xh
nyxf
yk
xh
ikyhxf
nin
i⎟⎟⎠
⎞⎜⎜⎝
⎛∂∂
+∂∂
+⎟⎟⎠
⎞⎜⎜⎝
⎛∂∂
+∂∂
=++ ∑−
=
We now use subscripts to denote the partial derivatives
( )+++
++=++
21
)(),(
22yyxyxx
yx
fkhkffh
kfhffkyhxf
( )
( )++++
+++
33!3
1
2!2
3223yyyxyyxxyxxx
yyxyxx
fkfhkkfhfh
fkhkffh
L+ L++++=+ xxxxxx fhfhhffyhxf
!3!2),(
32
15L++++=+ yyyyyy fkfkkffkyxf
!3!2),(
32
Runge‐Kutta Method of Order 2Runge Kutta Method of Order
Define two function evaluations
⎧ )(hfK
We add the linear combination of these quantities to the value of x at t to⎩⎨⎧
++==
),(),(
12
1
KxhthfKxthfK
βαWe add the linear combination of these quantities to the value of x at t to obtain the value at t+h
or 2211)()( KwKwtxhtx ++=+
We want to determine the constants in the above expression so that it
)),(,(),()()( 21 xthfxhthfwxthfwtxhtx βα ++++=+
will be as accurate as possible
In other words, we want the above expression to match the Taylor series t t ibl
16
to as many terms as possible
Euler’s Method & Runge‐Kutta Methodg
The Euler’s method (left) requires one function evaluation at each step
17
The second‐order Runge‐Kutta method (right) requires two function evaluations at each step
Runge‐Kutta Method of Order 2g
Compare the intended formula
))(()()()( hfhhfhfh βwith the Taylor series
)),(,(),()()( 21 xthfxhthfwxthfwtxhtx βα ++++=+
)('"1)("1)(')()( 2hhh
If we set w1=1 and w2=0, the two expressions agree to the h term
L++++=+ )('"!3
)("!2
)(')()( 2 txtxhthxtxhtx
1 2
The formula obtained is
)()()( xthftxhtx +=+which is the Euler’s method, since
),()()( xthftxhtx +=+
)()(' xtftx =
18
),()( xtftx =
Runge‐Kutta Method of Order 2g
Revisit the two variable Taylor series
1 2⎞⎛ ∂∂
Using this formula in
),(21),(
2
yxfx
hft
ahhffhffhfxhtf xt ⎟⎠⎞
⎜⎝⎛
∂∂
+∂∂
+++=++ ββαβα
Using this formula in
We have
)),(,(),()()( 21 xthfxhthfwxthfwtxhtx βα ++++=+
Note that
)()()()( 322
2221 hOffhwfhwhfwwtxhtx xt +++++=+ βα
ffftx
xf
dtdt
tf
dtxtdf
dtdxx xt +=⎟
⎠⎞
⎜⎝⎛∂∂
⎟⎠⎞
⎜⎝⎛∂∂
+⎟⎠⎞
⎜⎝⎛⎟⎠⎞
⎜⎝⎛∂∂
===),('"
19
txdttdtdt ⎠⎝ ∂⎠⎝ ∂⎠⎝⎠⎝ ∂
Runge‐Kutta Method of Order 2g
The Taylor series equation11
becomes 11
L++++=+ )('"!3
1)("!2
1)(')()( 32 txhtxhthxtxhtx
becomes
Compare the two expressions of x(t+h), we have
)(21
21)( 322 hOffhfhhfxhtx xt ++++=+
One solution is21
21 1 2221 ===+ wwww βα
Oth l ti ibl b t th t ti ill b th
21
21 1 1 21 ==== wwβα
20
Other solutions are possible, but the truncation errors will be the same
Runge‐Kutta Method of Order 2g
So the Runge‐Kutta Method of Order 2 ishh
Or equivalently 1
)),(,(2
),(2
)()( xthfxhtfhxtfhtxhtx ++++=+
Or equivalently
where
)(21)()( 21 KKtxhtx ++=+
⎧ )(hf
⎩⎨⎧
++==
),(),(
12
1
KxhthfKxthfK
The solution function at t+h is computed at the expenses of two evaluations of the function f
Th t ti f th d d R K tt th d i O(h3)
21
The truncation error of the second‐order Runge‐Kutta method is O(h3)
Compare Euler‘s Method & Runge‐Kutta Methodp g
The initial‐value problem is
1)0(3)(2)(' 4− t
22
1)0( ,3)(2)(' 4 =+−= xetxtx t
Runge‐Kutta Method of Order 4g
The most popular Runge‐Kutta Method is that of order 4, as
( )1
where
( )432161)()( KKKKtxhtx ++++=+
⎪⎧ =1 ),( xthfK
⎪
⎪⎪⎪
⎨⎟⎞
⎜⎛
⎟⎠⎞
⎜⎝⎛ ++= 12
1121,
21
hhf
KxhthfK
( )⎪⎪⎪
⎩ ++=
⎟⎠⎞
⎜⎝⎛ ++=
34
23
,21,
21
KxhthfK
KxhthfK
This formula requires four function evaluations
The truncation error of the fourth‐order Runge‐Kutta method is O(h5)
23
g ( )
Fourth‐Order Runge‐Kutta Methodg
Illustration of Runge‐Kutta method of order 4. Calculations are made at the initial time, two at half of the stepsize beyond the initial time and at the final time These four calculations allow the use of larger overall stepsize with good
24
time. These four calculations allow the use of larger overall stepsize with good accuracy
Fourth‐Order Runge‐Kutta Methodg
The initial‐value problem is 1)0()(' =+= xxytx
25
The initial value problem is 1)0( ,)( + xxytx
Some Amazing Simulation Resultsg
26
Adaptive Runge‐Kutta Methodsp g
In practical computations, we need to know the magnitude of errors i l d i h iinvolved in the computation
Given a tolerance of error, we want to be assured that the computed numerical solution must not deviate from the true solution beyond thenumerical solution must not deviate from the true solution beyond the tolerance
If a method is selected, the error tolerance dictates the allowable step sizeIf a method is selected, the error tolerance dictates the allowable step size
A uniform step size may not be desirable
If the solution is smooth, a large step size may be used to reduce computational cost
27
It is preferable that the method can automatically adjust the step size during the computation
Compute the Errorsp
To advance the solution curve from t to t+h, we can take one step of size hi h f h d R K h dusing the fourth‐order Runge‐Kutta method
We can also take two steps of size h/2 to arrive at t+h
If there were no truncation errors, the values of the numerical solution x(t+h) computed from both procedures would be the same
The difference in the numerical results can be taken as an estimate of the local truncation error
If this error is within the prescribed tolerance, the current step size h is satisfactory. If the difference exceeds the tolerance, the step size is halved
28
This procedure is easily programmed but rather wasteful of computing time
Fehlberg Method of Order 4g
The formula is12197140825
where
5431 51
41042197
25651408
21625)()( KKKKtxhtx −+++=+
⎧where
⎪⎪⎪⎪⎧
⎟⎠⎞
⎜⎝⎛ ++=
=
12
1
1,1),(
KxhthfK
xthfK
⎪
⎪⎪⎪⎪
⎨ ⎟⎠⎞
⎜⎝⎛ +++=
⎟⎠
⎜⎝
213
12
329
323,
83
4,
4
KKxhthfK
f
⎪⎪⎪⎪⎪
⎞⎛
⎟⎠⎞
⎜⎝⎛ +−++=
⎠⎝
3214
845368043921977296
21977200
21971932,
1312 KKKxhthfK
29⎪⎪⎪
⎩⎟⎠⎞
⎜⎝⎛ −+−++= 43215 4104
845513
36808216439, KKKKxhthfK
Fehlberg Method of Order 4, RK45g ,
The Fehlberg method of order 4 requires one more function evaluation than the classical Runge Kutta method it is not very attractivethan the classical Runge‐Kutta method, it is not very attractive
However, with one more function evaluation, as
⎞⎛ 111859354481
We can obtain the fifth‐order Runge‐Kutta method, i.e.,
⎟⎠⎞
⎜⎝⎛ −+−+−+= 543216 40
1141041859
1282535442
278,
21 KKKKKxhthfK
65431 552
509
5643028561
128256656
13516)()( KKKKKtxhtx +−+++=+
The difference between the values of x(t+h) computed from the fourth and fifth order procedures is an estimate of the local truncation error in th f th d d S i f ti l ti i f th
30
the fourth‐order procedure. So six function evaluations give a fourth‐order approximation, together with an error estimate
Adaptive Process
1. Given a step size h and an initial value x(t), the RK45routine computes the value x(t+h) and an error estimate ε
2. If , then the step size h is not changed and the next step size is taken by replacing step 1 with initial value x(t+h)
f h h l d b h d d
maxmin εεε ≤≤
≤ |2| hh ≤3. If , then h is replaced by 2h, provided
4. If , then h is replaced by h/2, provided
minεε ≤ max|2| hh ≤
min|2/| hh ≥maxεε >
5. If , then the step size is repeated by returning to step 1 with x(t) and the new h value
maxmin || hhh ≤≤
Here are the maximum and minimum error tolerances allowedminmax,εε
hh
31
are the maximum and minimum step sizes allowedminmax,hh
Adams‐Bashforth‐Moulton Formulas
Let us consider the first‐order ordinary differential equation
))(,()(' txtftx =If we assume the values of the unknown function have been computed at several points to the left of t, i.e., t, t‐h, t‐2h, …, t‐(n‐1)h, we want to
( h) h h f l l h
))(,()( txtftx
compute x(t+h). Using the theorem of calculus, we have
)(')()( ∫+
+=+ht
tdssxtxhtx
)(
))(,()(
∑
∫+
+≈
+=n
jj
ht
t
fctx
dssxsftx
where the abbreviation fj is
)( 1
∑=
+j
jj fctx
)))1(()1(( hjthjtff
32
)))1((,)1(( hjtxhjtff j −−−−=
Adams‐Bashforth Formulas
The last step needs a rule of the form
∫ +++≈1
)1()1()0()( nFcFcFcdrrF L
We need to determine the n coefficients cj. We can insist on integrating each function exactly. The appropriate equation is
∫ −++−+≈0 21 )1()1()0()( nFcFcFcdrrF nL
12 ,,,,1 −nrrr L
A linear system of the form Ax=b of n equations can be solved for n
∫ ∑=
−− ≤≤−=1
01
11 )1( )1(n
j
ij
i nijcdtr
A linear system of the form Ax=b of n equations can be solved for nunknowns. The coefficients of the matrix A are
1)1( −−= iij jA
and the right‐hand side is bi =1/i
33
Adams‐Moulton Formulas
In Adams‐Moulton formulas, the quadrature is of the form
Use the change of variable from s to p given by s=hp t the new integral will
∫ ∑=
−≈1
01
)2()(n
jj jGCdrrG
Use the change of variable from s to p given by s=hp‐t, the new integral will be
∫ +1
0)( dpthpgh
We now have
[ ])3(9)2(37)(59)(55)( htFhtFhtFtFhdrrFht
−−−+−−≈∫+ [ ]
[ ])2(249)(5)(19)(924
)(
)3(9)2(37)(59)(5524
)(
htGhtGtGhtGhdrrG
htFhtFhtFtFdrrF
ht
t
t
−−−−++≈
+
∫
∫+
34
Stability Analysisy y
For some initial value problems, a small error in the initial value may d f l i d d h d l i iproduce errors of large magnitude, and the computed solution is
completely wrong
The truncation error in each step further deteriorates the computedThe truncation error in each step further deteriorates the computed solution
For other initial value problems, the situation is not that severeFor other initial value problems, the situation is not that severe
35
A Case of Diverging Solutionsg g
The initial values are close to each other. But over the time, the solutions diverge to different paths
36
A Case of Converging Solutionsg g
The initial values are quite different. But over the time, the l ti t thi l t h th
37
solutions converge to something very close to each other
Solution Stability y
For the general differential equation
⎧ = )xt(f'x
⎩⎨⎧
==
s)a(x)x,t(fx
If
0>> δδ somefor fx
the curves diverge. If
0>< δδ ff
then the curves converge.
0>−< δδ somefor fx
38
