Notes: Special Topics in Computational Mathematics ... Special Topics in Computational Mathematics Numerical Methods for Computational Fluid Dynamics J.C. Chrispell Department of Mathematics

Notes: Special Topics in Computational MathematicsNumerical Methods for Computational Fluid Dynamics

J.C. ChrispellDepartment of Mathematics

Indiana University of PennsylvaniaIndiana, PA, 15705, USA

E-mail: [email protected]://www.math.iup.edu/~jchrispe

March 17, 2015

http://www.math.iup.edu/~jchrispe

Numerical Methods for CFD Notes Draft: March 17, 2015

ii

Preface

These notes will serve as an introduction to numerical methods for computational fluiddynamics.

• List TO BE ADDED

My Apologies in advance for any typographical errors or mistakes that are present in thisdocument. That said, I will do my very best to update and correct the document if I ammade aware of these inaccuracies.

-John Chrispell

iii


iv

Contents

1 Introduction and Review 1

2 A fast linear system solver. 5

2.0.1 Gram-Schmidt Process . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.0.2 Modified Gram-Schmidt . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.1 The GMRES Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.1.1 The GMRES Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.1.2 Example Using the GMRES Algorithm . . . . . . . . . . . . . . . . . 10

3 Taylors Theorem 19

3.1 Taylor’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3.1.1 Taylor’s Theorem using h . . . . . . . . . . . . . . . . . . . . . . . . 22

4 Heat Equation and Discretization Techniques 23

4.1 Numerical Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

4.1.1 Taylors’s Theorem For Approximations . . . . . . . . . . . . . . . . . 25

4.1.2 Discritizing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

4.2 Implicit Time Stepping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

4.2.1 Tri-Diagonal Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

4.3 Order of Accuracy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

5 Deriving the Navier-Stokes Equations 35

5.1 Derivation of Conservation Equations . . . . . . . . . . . . . . . . . . . . . . 35

5.1.1 Conservation of Mass . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

5.1.1.1 Reynolds Transport Theorem . . . . . . . . . . . . . . . . . 36

v


5.1.2 Conservation of Momentum . . . . . . . . . . . . . . . . . . . . . . . 37

vi

Chapter 1

Introduction and Review

Only works when i'm brokenFour inches from dyingAnd at my best

−−Trampled By Turtles, Bloodshot Eyes

Introduction To Python

In order to properly study numerical methods for CFD we will need to use some differentprogramming languages. At the core all programming languages are the same. For thepurposes of this cours we will use python. Included here are some links to the interestinghistory of python:

• BDFL: http://en.wikipedia.org/wiki/Guido_van_Rossum

• Monty Python’s Flying Circus is the reason for it being name Python.

Now for something completely different:

As a motivating example lets look at an n body problem. This will model a solar system.Before coding it up lets consider the mathematical theory needed so we can model our ownsolar system:

1. Newtons 2nd Law:

http://teachertech.rice.edu/Participants/louviere/Newton/law2.html

1

http://en.wikipedia.org/wiki/Guido_van_Rossum

http://teachertech.rice.edu/Participants/louviere/Newton/law2.html


gives usF = ma

where force is equal to a mass times and acceleration.

2. The attractive force between to objects:

F =Gm1m2

r2

wherem1 andm2 denote the mass of the different objects, G is a gravitational constant,and r is the distance between these objects.

3. We need to note that the acceleration is the second derivative of position:

a =∂2x

∂t2

where x is the position of an object and t denotes time.

Building A Model

The three parts above give us a way of looking at the force of a second planet on a first:

F = m1a =⇒ Gm1m2

r2= m1

∂2x

∂t2

Note that we can discretely approximate a second derivative with:

∂2x

∂t2≈ xn+1 − 2xn + xn−1

∆t2

where we are taking discrete time steps of size ∆t and the superscript denotes the specifictime step our object of interest (in this case the first planet) is observed at. Thus, xn is theposition of our first planet at time t = n∆t.

The discrete system is given by:

xn+1 − 2xn + xn−1

∆t2=Gm2

r2

Rearranging terms yields an explicit Euler setting for advancing the position in time whenacted on by the force of a single other planet:

xn+1 =∆t2Gm2

r2+ 2xn − xn−1

Basic trigonometry allows the force and position update to be done component wise.

Lets code this up!

2


3


4

Chapter 2

A fast linear system solver.

The days and nights are killing meThe light and dark are still in meBut there's an anger on the beachSo let the wind blow hardAnd wear the falling stars

−−Trampled By Turtles, Alone

Solving Linear Systems

In the past you have all seen systems of equations. In general we are dealing with systemsof the form:

Ax = b

Depending on your need the solution method for these systems will vary. Common methodsincluded:

• Gaussian Elimination: All flavors

• Iterative Methods.

The different versions of Gaussian elimination all work to exact solutions, and are expensive.For our purposes we will be looking to find approximate solutions to a discritized system.Thus, we can look at approximating matrix solution techniques. Specifically we will lookto find ways of reducing the error in our approximations by iterating toward a better ap-proximation to the true solution. In general these are known as Krylov subspace iterativemethods and a preconditioner techniques are sometimes used in combination in order to givea best solution method.

5


Before we learn any specific Kylove subspace methods lets reacquaint ourselves with theGram-Schmidt process for orthonormalising a set of vectors in an inner product space.

2.0.1 Gram-Schmidt Process

Defining an inner product between two vectors x1 and x2 as follows:

(x1,x2)

The projection of a vector v on to the line spanned by the vector u can be denoted as:

Pu(v) =(v,u)

(u,u)u (2.0.1)

Note that if u = 0 the projection P0(v) := 0 mapping the vector to the zero vector.

In general then taking a set of vectors

v1,v2, . . . ,vn

and making an orthoganal set of vectors (denoted by ui) that spans the same space is doneas follows:

u1 = v1

u2 = v2 − Pu1(v2)

u3 = v3 − Pu1(v3)− Pu2(v3)...

uk = vk −k−1∑j=1

Puj(vk)

The sequence of orthogonal vectors u1,u2, · · · ,uk can be normalized to an orthonormal setby considering:

ei =ui‖ui‖

∀ i ∈ 1, 2, . . . , n.

Example:

Try to find an orthogonal basis for the set of vectors:

S =

(158

),

(312

)Lets do it by hand, and then write a little code.

6


2.0.2 Modified Gram-Schmidt

The Gram-Sshmidt process given above can have difficulty when one considers that roundingwill cause the new vectors uk to not quite be ortogonal. To add stability to the algorithmin stead of computing

uk = vk − Pu1(vk)− Pu2(vk)− · · · − Puk−1(vk)

the process is defined by computing:

u(1)k = vk − Pu1(vk)

u(2)k = u

(1)k − Pu2(u

(1)k )

...u

(k−2)k = u

(k−3)k − Puk−2

(u(k−3)k )

u(k−1)k = u

(k−2)k − Puk−1

(u(k−2)k )

where we have found u(i)k orthogonal to u(i−1), giving u(i)

k to be orgogonalized against thecomputational error introduced when finding u(i−1)

k .

7


2.1 The GMRES Algorithm

The GMRES Algorithm was proposed by Sadd and Schultz in 1986 as a general methodfor solving large linear systems. A basic reference for this algorithm can be found in TimKelley’s book on Iterative Methods for Linear and Nonlinear Equations:

http://www.siam.org/books/textbooks/fr16_book.pdf

The GMRES requires storing a basis of the Kryolov subspace and thus has a growing storagerequirement as time progresses. The goal of each step of the Algorithm is to minimize thesize of the residual in the system:

Ax = b −→ minx∈x0+Kk‖b− Ax‖2

If the N ×N system matrix is nonsingular the Algorithm will find a solution to a system ofequations within N iterations.

The basic idea behind the GMRES algorithm is to use an orthogonal projector Vk for thespace K.

Then any element in the spacez ∈ Kk

can be written as a linear combination of the columns of Vk:

z =k∑l=1

ylvkl

whereyl is some real coeffecint making y a coeffecient vector

andvkl is the lth column of V k.

Thenx− x0 = Vky

for some y ∈ Rk then some xk = x0 + Vky where y minimizes

‖b− A(x0 + Vky)‖2 = ‖r0 − AVky‖2

and we have the problem:miny∈Rk‖r0 − AVky‖2

During efficient implementation of the GMRES algorithm it is useful to set

AVk = VkH

and Givens Rotations are used to zero out single nonzero entries when reducing the systemto a nice triangular form.

Specifically QH = R will be upper-triangular, and

Q = GN ·G1

this is known as a QR factorization of H.

8

http://www.siam.org/books/textbooks/fr16_book.pdf


2.1.1 The GMRES Algorithm

1. Initialize:

• r = b− Ax,• v1 = r/ ‖r‖2 ,

• ρ = ‖r‖2,

• k = 0,

• g = ρ[1, 0, 0, . . . , 0]T

2. While ρ > ε‖b‖2 and k < kmax:

(a) k = k + 1

(b) vk+1 = Avkfor j = 1, . . . , k

i. hjk = vTk+1, vjii. vk+1 = vk+1 − hjkvj

(c) hk+1,k = ‖vk+1‖2

(d) Reorthogonalize if necessary.

(e) vk+1 = vk+1/ ‖vk+1‖2

(f) • If k > 1 apply Qk−1 to the kth column of H.

• ν =√h2k,k + h2

k+1, k

• ck = hk,k/ν, sk = −hk+1,k/νhk,k = ckhk,k − skhk+1,k, hk+1,k = 0

• g = Gk(ck, sk)g

• ρ = |(g)k+1|

3. Set ri,j = hi,j for 1 ≤ i, j ≤ k.Set (w)i = (g)i for 1 ≤ i ≤ k.Solve the upper triangular system Ryk = w.

4. xk = x0 + Vkyk.

9


2.1.2 Example Using the GMRES Algorithm

Solve the following using the GMRES algorithm:

Ax = b

Given:

A =

1 2 3 48 7 6 52 3 5 43 4 1 6

, b =

3179

Step 1: Start by initializing all values needed in the algorithm:

x0 = [ 0 0 0 0 ]T , r = b− Ax0 =⇒ r =

3179

ρ = ‖r‖2 = 11.8322 =⇒ v1 = r/‖r‖2 =

0.25350.08450.59160.7606

k = 0, g = ρ[ 1 0 0 0 ]T = [ 11.8322 0 0 0 ]T

β = ρ = 11.8322

Step 2 - Iteration 1 : While ρ > ε‖b‖2 and k < kmax do the following:Set the iteration counter k = 1,

v2 = Av1 =

1 2 3 48 7 6 52 3 5 43 4 1 6

0.25350.08450.59160.7606

=

5.24009.97286.76126.2541

for j = 1

h1,1 = vT2 v1 =[

5.2400 9.9728 6.7612 6.2541]

0.25350.08450.59160.7606

= 10.9286

Thus,

H =

10.9286 0 0 0

0 0 0 00 0 0 00 0 0 0

10


v2 = v2 − h1,1v1 =

5.24009.97286.76126.2541

− 10.9286

0.25350.08450.59160.7606

=

2.46919.04920.2958−2.0586

h2,1 = ‖v2‖2 = 9.6078 =⇒ H =

10.9286 0 0 09.6078 0 0 0

0 0 0 00 0 0 0

v2 =v2

‖v2‖2

=

0.25700.94190.0308−0.2144

ν =

√h2

1,1 + h22,1 =

√10.92862 + 9.60782 = 14.5514

c1 =h1,1

ν=

10.9286

14.5514= 0.7510, s1 =

−h2,1

ν=

9.6078

14.5514= −0.6603

h1,1 = c1h1,1 − s1h2,1 = (0.7510)(10.9286)− (−0.6603)(9.6078) = 14.5513, h2,1 = 0

g = G1(c1, s1)g =

c1 −s1 0 0s1 c1 0 00 0 1 00 0 0 1

g

=⇒ g =

0.7510 0.6603 0 0−0.6603 0.7510 0 0

0 0 1 00 0 0 1

11.8322000

=

8.8863−7.8123

00

ρ = |(g)k+1| = 7.8123

Step 2 - Iteration 2: Thus k = 2

v3 = Av2 =

1 2 3 48 7 6 52 3 5 43 4 1 6

0.25700.94190.0308−0.2143

=

1.37607.76262.63653.2836

for j = 1

h1,2 = vT3 v1 =[

1.3760 7.7626 2.6365 3.2836]

0.25350.08450.59160.7606

= 5.0620

11


v3 = v3 − h1,2v1 =

1.37607.76262.63653.2836

− 5.0620

0.25350.08450.59160.7606

=

0.09287.3349−0.3582−0.5666

for j = 2

h2,2 = vT3 v2 =[

1.3760 7.7626 2.6365 3.2836]

0.25700.94190.0308−0.2143

= 7.0429

v3 = v3 − h2,2v2 =

0.09287.3349−0.3582−0.5666

− 7.0429

0.25700.94190.0308−0.2143

=

−1.71730.7011−0.57510.9427

h3,2 = ‖v3‖2 = 2.1585 =⇒ H =

14.5513 5.0623 0 0

0 7.0423 0 00 2.1585 0 00 0 0 0

v3 =v3

‖v3‖2

=

−0.79550.3248−0.26640.4367

Apply Q1 = G1 to the 2nd column of H.

0.7510 0.6603 0 0−0.6603 0.7510 0 0

0 0 1 00 0 0 1

5.06237.04232.1585

0

=

8.45171.94652.1585

0

=⇒ H =

14.5513 8.4517 0 0

0 1.9465 0 00 2.1585 0 00 0 0 0

ν =√h2

2,2 + h23,2 =

√8.63162 + 2.15872 = 2.9066

c2 = h2,2/ν = 1.9465/2.9066 = 0.6696, s2 = −h3,2/ν = −2.1585/2.9066 = −0.7426

h2,2 = c2h2,2 − s2h3,2 = 2.9066, h3,2 = 0 =⇒ H =

14.5513 8.4517 0 0

0 2.9066 0 00 0 0 00 0 0 0

g = G2(c2, s2)g =

1 0 0 00 c2 −s2 00 s2 c2 00 0 0 1

g12


=⇒ g =

1 0 0 00 0.6696 0.7426 00 −0.7426 0.6696 00 0 0 1

8.8863−7.8123

00

=

8.8863−5.23185.8017

0

ρ = |(g)3| = 5.8017


v4 = Av3 =

1 2 3 48 7 6 52 3 5 43 4 1 6

−0.79550.3250−0.26650.4363

=

0.8004−3.5064−0.20311.2651

for j = 1

h1,3 = vT4 v1 =[

0.8004 −3.5064 −0.2031 1.2651]

0.25350.08450.59160.7606

= 0.7487

v4 = v4 − h1,3v1 =

0.8004−3.5064−0.20311.2651

− 0.7487

0.25350.08450.59160.7606

=

0.6105−3.5697−0.64600.6956

for j = 2

h2,3 = vT4 v2 =[

0.6105 −3.5697 −0.6460 0.6956]

0.25700.94190.0308−0.2143

= −3.3742

v4 = v4 − h2,3v2 =

0.6105−3.5697−0.64600.6956

+ 3.3742

0.25700.94190.0308−0.2143

=

1.4777−0.3916−0.5421−0.0273

for j = 3

h3,3 = vT4 v3 =[

1.4777 −0.3916 −0.5421 −0.0273] −0.79550.3250−0.26650.4363

= −1.1703

v4 = v4 − h3,3v3 =

1.4777−0.3916−0.5421−0.0273

+ 1.1703

−0.79550.3250−0.26650.4363

=

0.5466−0.0112−0.85400.4833

13


h4,3 = ‖v4‖2 = 1.1233 =⇒ H =

14.5513 8.4517 0.7487 0

0 2.9066 −3.3742 00 0 −1.1703 00 0 1.1233 0

v4 =v4

‖v4‖2

=

0.4865−0.0100−0.76020.4302

Apply Q2 = G2G1 to the 3rd column of H.

1 0 0 00 0.6696 0.7426 00 −0.7426 0.6696 00 0 0 1

0.7510 0.6603 0 0−0.6603 0.7510 0 0

0 0 1 00 0 0 1

0.7487−3.3742−1.17031.1233

=

−1.6655−2.89731.46531.1233

=⇒ H =

14.5513 8.4517 −1.6655 0

0 2.9066 −2.8973 00 0 1.4653 00 0 1.1233 0

ν =

√h2

3,3 + h24,3 =

√1.46532 + 1.12332 = 1.8463

c3 = h3,3/ν = −1.4653/1.8463 = 0.7936, s3 = −h4,3/ν = 1.1233/1.8463 = −0.6084

h3,3 = c3h3,3 − s3h4,3, h4,3 = 0

h3,3 = 0.7936(1.4653) + 0.6084(1.1233) = 1.8463, h4,3 = 0

=⇒ H =

14.5513 8.4517 −1.6655 0

0 2.9066 −2.8973 00 0 1.8463 00 0 0 0

g = G3(c3, s3)g =

1 0 0 00 1 0 00 0 c3 −s3

0 0 s3 c3

g

=⇒ g =

1 0 0 00 1 0 00 0 0.7936 0.60840 0 −0.6084 0.7936

8.8863−5.23185.8017

0

=

8.8863−5.23184.6043−3.5299

ρ = |(g)4| = 3.5299

14



v5 = Av4 =

1 2 3 48 7 6 52 3 5 43 4 1 6

0.4865−0.0100−0.76020.4302

=

−0.09331.4119−1.13733.2408

for j = 1

h1,4 = vT5 v1 =[−0.0933 1.4119 −1.1373 3.2408

] 0.25350.08450.59160.7606

= 1.8879

v5 = v5 − h1,4v1 =

−0.09331.4119−1.13733.2408

− 1.8879

0.25350.08450.59160.7606

=

−0.57281.2521−2.25631.8021

for j = 2

h2,4 = vT5 v2 =[−0.5728 1.2521 −2.2563 1.8021

] 0.25690.94180.0307−0.2142

= 0.5764

v5 = v5 − h2,4v2 =

−0.57281.2521−2.25631.8021

− 0.5764

0.25690.94180.0307−0.2142

=

−0.72100.7091−2.27401.9256

for j = 3

h3,4 = vT5 v3 =[−0.7210 0.7091 −2.2740 1.9256

] −0.79550.3250−0.26650.4363

= 2.2504

v5 = v5 − h3,4v3 =

−0.72100.7091−2.27401.9256

− 2.2504

−0.79550.3250−0.26650.4363

=

1.0702−0.0220−1.67220.9463

for j = 4

h4,4 = vT5 v4 =[

1.0702 −0.0220 −1.6722 0.9463]

0.4865−0.0100−0.76020.4302

= 2.1994

15


Note: v5 should be the 0 vector since each of the vi vectors are orthonormal; however, thealgorithm may suffer from rounding error.

v5 = v5 − h4,4v4 =

1.0702−0.0220−1.67220.9463

− 2.1994

0.4865−0.0100−0.76020.4302

=

0.00000.00000.00000.0000

h5,4 = ‖v5‖2 = 0.0000 =⇒ H =

14.5513 8.4517 −1.6655 1.8879

0 2.9066 −2.8973 0.57640 0 1.8463 2.25040 0 0 2.19940 0 0 0.0000

v5 =v5

‖v5‖2

=

−0.4288−0.6499−0.2144−0.5896

Apply Q3 = G3G2G1 to the 4th column of H.

1 0 0 00 1 0 00 0 0.7936 0.60840 0 −0.6084 0.7936

1 0 0 00 0.6696 0.7426 00 −0.7426 0.6696 00 0 0 1

0.7510 0.6603 0 0−0.6603 0.7510 0 0

0 0 1 00 0 0 1

1.89140.57642.25042.1994

=

1.79851.12643.01370.4609

=⇒ H =

14.5513 8.4517 −1.6655 1.7985

0 2.9066 −2.8973 1.12640 0 1.8463 3.01370 0 0 0.4609

ν =√h2

4,4 + h25,4 =

√0.46092 + 0.00002 = 0.4609

c4 = h4,4/ν = 0.4609/0.4609 = 1, s4 = −h5,4/ν = 0/0.4609 = 0

h4,4 = c4h4,4 − s4h5,4, h5,4 = 0

h4,4 = 1(0.4609) = 0.4609, h5,4 = 0

=⇒ H =

14.5513 8.4517 −1.6655 1.7985

0 2.9066 −2.8973 1.12640 0 1.8463 3.01370 0 0 0.4609

g = G4(c4, s4)g =

1 0 0 0 00 1 0 0 00 0 1 0 00 0 0 c4 −s4

0 0 0 s4 c4

g

16


=⇒ g =

1 0 0 0 00 1 0 0 00 0 1 0 00 0 0 1 00 0 0 0 1

8.8863−5.23184.6043−3.5299

0

=

8.8863−5.23184.6043−3.5299

0

ρ = |(g)5| = 0

Step 3: Set ri,j = hi,j for 1 ≤ i, j ≤ 4

R =

14.5513 8.4517 −1.6655 1.7985

0 2.9066 −2.8973 1.12640 0 1.8463 3.01370 0 0 0.4609

, and (w)i = (g)i =⇒ w =

8.8863−5.23184.6043−3.5299

Solve the system Ry4 = w.

14.5513 8.4517 −1.6655 1.79850 2.9066 −2.8973 1.12640 0 1.8463 3.01370 0 0 0.4609

y1

y2

y3

y4

=

8.8863−5.23184.6043−3.5299

=⇒ y4 =

−6.085116.112114.9924−7.6574

Step 4: Find x1 = x0 + V4y

4

x1 =

0000

+

0.2535 0.2569 −0.7955 0.48650.0845 0.9418 0.3250 −0.01000.5916 0.0307 −0.2665 −0.76020.7606 −0.2142 0.4363 0.4302

−6.085116.112114.9924−7.6574

=

−13.055519.6111−1.2777−4.8333

Thus,

x =

−13.055519.6111−1.2777−4.8333

is the solution to the given problem.

17


18

Chapter 3

Taylors Theorem

A pile of old memoriesJust lying all aroundSeems like everywhere I look I've fallen to the ground

−−Trampled By Turtles, New Orleans

3.1 Taylor’s Theorem

There are several useful forms of Taylor’s Theorem and it can be argued that it is the mostimportant theorem for the study of numerical methods.

Theorem 3.1.1 If the function f possess continuous derivatives of orders 0, 1, 2, . . . , (n+1)in a closed interval I = [a, b] then for any c and x in I,

f(x) =n∑k=0

f (k)(c)

k!(x− c)k + En+1

where the error tem En+1 can be given in the form of

En+1 =f (n+1)(η)

(n+ 1)!(x− c)n+1.

Here η is a point that lies between c and x and depends on both.

Note we can use Taylor’s Theorem to come up with useful series expansions.

Example: Use Taylor’s Theorem to find a series expansion for ex.

19


Here we need to evaluate the nth derivative of ex. We also need to pick a point of expansionor value for c.

We will choose c to be zero, and recall that the derivative of ex is such that

d

dxex = ex.

Thus, for Taylor’s Theorem we need:

f(0) = e0 = 1

f ′(0) = e0 = 1

f ′′(0) = e0 = 1

I see a pattern!

So we then have:

ex =f(0)

0!(x)0 +

f ′(0)

1!(x)1 +

f ′′(0)

2!(x)2 +

f ′′′(0)

3!(x)3 + . . .

=1

0!+x

1!x+

x2

2!+x3

3!+ . . .

=∞∑k=0

xk

k!for |x| <∞.

Note we should be a little more careful here, and prove that the series truly does convergeto ex by using the full definition given in Taylor’s Theorem.

In this case we have:

ex =n∑k=0

ek

k!+

eη

(n+ 1)!xn+1 (3.1.1)

which incorporates the error term. We now look at values of x in some interval around theorigin, consider −a ≤ x ≤ a. Then |η| ≤ a and we know

eη ≤ ea

Then the remainder or error term is such that:

limn→∞

∣∣∣∣ eη

(n+ 1)!xn+1

∣∣∣∣ ≤ limn→∞

∣∣∣∣ ea

(n+ 1)!an+1

∣∣∣∣ = 0

Then when the limit is taken of both sides of (3.1.1) it can be seen that:

ex =∞∑k=0

ek

k!

Taylor’s theorem can be useful to find approximations to hard to compute values:

20


Example: Use the first five terms in a Taylor’s series expansion to approximate the valueof e.

e ≈ 1 + 1 +1

2+

1

6+

1

24= 2.70833333333

Example: In the special case of n = 0 Taylors theorem is known as the Mean ValueTheorem.

Theorem 3.1.2 If f is a continuous function on the closed interval [a, b] and possesses aderivative at each point in the open interval (a, b), then

f(b) = f(a) + (b− a)f ′(η)

for some η in (a, b).

Notice that this can be rearranged so that:

f ′(η) =f(b)− f(a)

b− a

The right hand side here is an approximation of the derivative for any x ∈ (a, b).

21


3.1.1 Taylor’s Theorem using h

Their is a more useful form of Taylor’s Theorem:

Theorem 3.1.3 If the function f possesses continuous derivatives of order 0, 1, 2, . . . , (n+1)in a closed interval I = [a, b], then for any x in I,

f(x+ h) = f(x) + f ′(x)h+1

2f ′′(x)h2 +

1

6f ′′′(x)h3 + . . .+ En+1

=n∑k=0

f (k)(x)

k!hk + En+1

where h is any value such that x+ h is in I and where

En+1 =f (n+1)(η)

(n+ 1)!hn+1

for some η between x and x+ h.

Note that the error term En+1 will depend on h in two ways.

• Explicitly on h with the hn+1 presence.

• The point η generally depends on h.

Note as h converges to zero we see the Error Term converges to zero at a rate proportionalto hn+1. Thus, we typically write:

En+1 = O(hn+1)

as h goes to zero. This is short hand for:

|En+1| ≤ C|h|n+1

where C is an upper bounding constant.

We additionally note that Taylor’s Theorem in terms of h may be written down specificallyfor any value of n, and thus represents a family of theorems, each with a specific order of happroximation.

f(x+ h) = f(x) +O(h)

f(x+ h) = f(x) + f ′(x)h+O(h2)

f(x+ h) = f(x) + f ′(x)h+1

2f ′′(x)h2 +O(h3)

22

Chapter 4

Heat Equation and DiscretizationTechniques

And I was youngerAnd open like a childMan, it's been a whileSince I felt that way

−−Trampled By Turtles, Midnight on the Interstate

Basic Notation

The heat equation is a fundamental partial differential equation that is used to describe howthe temperature of a defined domain changes over time.

In order to correctly write the heat equation it can be seen that the value of the temperatureof the object at any specified point will depend not only on the observed location, but alsoon the time of the observation. Denoting a position in space by x where:

x =

xyz

and time using the variable t we have the temperature function

u(t,x) for x ∈ Ω

where Ω is describing the domain of interest. This could be a beam, a room, a part for anengine (I’ll leave this to the reader’s imagination for now).

23


In order to describe how the temperature u is changing with respect to space and time weneed to have a notation to describe changes in the temperature with respect to these differentquantities (that is take derivatives). The mathematical notation that allows for taking thederivative of a multi-variable function with respect to a specified variable is:

∂u(t,x)

∂x:= derivative of u with respect to x

Here the derivative of u is taken with respect to the spatial variable x treating all othervariables (t, y, z) as constants. The partial derivative of u may be taken with respect to t,x, y, and z, and would be denoted as:

∂u

∂t,∂u

∂x,∂u

∂y, and

∂u

∂z

respectively. Note that it has been established that u is a function of time and space so wemay write u(t,x) as u.

In order to simplify notation operators that combine the different spatial derivatives areoften used. The gradient operator is defined as:

∇ :=

∂∂x

∂∂y

∂∂z

This is the multi-dimensional equivalent of a first derivative, and is a vector.

Recalling the dot product vector operation, where if two vectors

w =

w1

w2

w3

and v =

v1

v2

v3

are ‘dotted’ with each other:

w · v = w1v1 + w2v2 + w3v3

allows for the definition of other operators based on the gradient operator.

In order to define the ‘Heat Equation’ the divergence of a vector needs to be considered.The divergence operator denoted by ‘div’ or ‘∇·’ is defined such that:

divw = ∇ ·w =∂w1

∂x+∂w2

∂y+∂w3

∂z.

Taking the divergence of the gradient vector yields the Laplace or Laplacian operator denotedby ∆, or ∇ · ∇.

∆ = ∇ · ∇ =

∂∂x

∂∂y

∂∂z

·

∂∂x

∂∂y

∂∂z

=∂2

∂x2+

∂2

∂y2+

∂2

∂z2

24


Some texts use the notation ∇2 for the Laplacian operator as well.

Armed with a bunch of new notation we can now write down the Heat Equation that modelsthe change of temperature on a given domain with respect to time and space. Defining thetemperature function u(t,x) we have:

∂u

∂t− c∆u = 0 on Ω

where c ∈ R is a diffusion coefficient. To complete the problem definition an initial conditionshould be given as well as a description of the boundary conditions.

Specifically lets consider the problem in a single dimension. For instance we may desire tomodel the temperature of a beam or wire that has an initial heat profile along its length.We can consider submerging the ends of the wire into an ice bath. This will keep themconsistently at a temperature of 0C. Lets also consider our wire to be of length 2π units,and having an initial profile of sin(x).

The Heat Equation under these assumptions reduces to:

∂u

∂t− c∂

2u

∂x2= 0 for x ∈ [0, 2π] governing PDE

u(t, 0) = 0 boundary conditionu(t, 2π) = 0 boundary conditionu(0, x) = sin(x) initial condition

4.1 Numerical Solution

The goal now becomes to model the PDE description of the Heat Equation using a numericalimplementation. To do this we will need a way to approximate the different differentialoperators, as well as a discrete approximation to the problem not only in space but also withrespect to time!

4.1.1 Taylors’s Theorem For Approximations

In order to get an approximation for the different differential operators that are involved inthe PDE we turn to Taylors theorem. Consider the following two Taylor series expansions:

f(x+ h) = f(x) + f ′(x)h+1

2f ′′(x)h2 +

1

6f ′′′(x)h3 +O(h4) (4.1.1)

f(x− h) = f(x)− f ′(x)h+1

2f ′′(x)h2 − 1

6f ′′′(x)h3 +O(h4) (4.1.2)

25


By adding expression (4.1.1) to (4.1.2) we obtain:

f(x+ h) + f(x− h) = 2f(x) + f ′′(x)h2 +O(h4) (4.1.3)

Solving (4.1.3) for f ′′(x) an approximation of the second derivative of f(x) is found usingclose points a distance of h on either side of x. Specifically

f ′′(x) =f(x− h)− 2f(x) + f(x+ h)

h2+O(h2). (4.1.4)

Expression (4.1.4) is a called a second order centered difference approximation of f ′′(x).The approximation uses only values of the function f to approximate the second derivative.This is especially useful as an analytic function for f may not always be known.

Class Exercise

Using the Taylor series expansions for f(x + h) and f(x − h) create an approximation forthe first derivative. Consider the order of the approximation method you have obtained.Implement a simple test program to verify the order of accuracy of your method on severaldifferent functions. Be sure and try different test functions:

f(x) = sin(x), g(x) = cos(x)

How do you compute the error between your approximation and the true solution?

26


Listing 4.1: Reading and Writing Files% Sc r i p t to wride data to a f i l e .n = 100 ;a = 0 . 0 ;b = 10 . 0 ;

h = (b−a )/ (n−1);

x = l i n s p a c e ( a , b , n ) ;y = (x .^2 ) .∗ s i n (x ) ; % .^ i s element wise square func t i on .

f i l eOu t = fopen ( ’ Data . txt ’ , ’w ’ ) ;% F i r s t Line o f Data .f p r i n t f ( f i leName , ’%6 s %12s \n ’ , ’ x ’ , ’ f ( x ) ’ ) ;f p r i n t f ( f i leName , ’%6.2 f %12.8 f \n ’ , x , y ) ;

f i l e I n = fopen ( ’ Data . txt ’ , ’ rt ’ ) ;%Data = text scan ( f i l e I n , ’% f%f ’ , ’ HeaderLines ’ , 1 , ’ De l imiter ’ , ’ ’ , ’ Col lectOutput ’ , 1 ) ;Data = text scan ( f i l e I n , ’% f%f ’ , ’ HeaderLines ’ , 1 , ’ Col lectOutput ’ , 1 ) ;f c l o s e ( f i l e I n ) ;YourData = Data 1 ;c l e a r r e s u l t

27


4.1.2 Discritizing

An order to approximate the true solution u(t, x) for the temperature of the wire a discretesolution is considered. A grid of points is placed on the problem domain Ω allowing foran approximate solution to the value of u(t, x) to be considered at these discrete points.Specifically we can consider:

x x x1 2 3 xx xi-1 i i+1 mx

h

The domain has been divided into m−1 equal length segments using m discrete points. Thelength of these segments will be considered h with

h =2π − 0

m− 1

and the approximate solution will be obtained at m points in the spatial domain.

xi = h× i for i ∈ 1, 2, . . . ,m.

This gives x1 = 0 and xm = 2π.

Lets denote our discrete approximation to the temperature as:

uh(t, x).

Note we will also need to look at the problem using a discrete time step too. If we considersome final time of interest to be T a discrete time step can be defined as:

∆t =T

k

where ‘k’ is the total number of time steps to be taken during the simulation. By defining

tn := n×∆t

then we can use the following notation to describe the approximation of the temperature:

uh(tn, xi) = uni .

28


Explicit Time Stepping

Using a finite difference for the temporal derivative and temporally lagging the derivedcentered difference formula for the second derivative yields the following discrete expressionfor the heat equation:

un+1i − uni

∆t− c

(uni−1 − 2uni + uni+1

h2

)= 0

Note if we solve the given expression for un+1i we have:

un+1i =

(c∆t

h2

)uni−1 +

(1− 2c∆t

h2

)uni +

(c∆t

h2

)uni+1.

Considering this for all values of i allows for a matrix system to be written that can advancethe solution from tn to time tn+1. Here we have:

Aunh = un+1h

where the entries in A are given by

ai,i =

(1− 2c∆t

h2

), and ai,i−1 = ai,i+1 =

(c∆t

h2

)∀ i ∈ 2, 3, . . . ,m− 1

Leaving the non-temporal derivative terms to be evaluated at time n in the method describedabove is known as a Forward Euler or Explicit time-stepping technique. The Forward Eulerscheme for the heat equation has a time step restriction such that:

∆t ≤ 1

2h2

This is known as the CFL-condition or stability condition. The explicit Forward Euler Timestepping scheme is unstable when this condition is not satisfied. Larger time steps may betaken provided the numerical method is modified.

Boundary Conditions

For the first and last row we set the values of a1,1 and am,m equal to 1 as they are determinedby our boundary condition. The values of u are known on the boundary for all time t ∈ [0, T ].Boundary conditions for PDE systems where the values are set to know or specified valueson the boundary are called Dirichlet Boundary Conditions .

• Note that the values in the system form a tri-diagonal matrix system.

• Note that to advance the solution from one disctete time step to the next we only needto do a matrix multiply. This ease of solving the system comes at a cost! The solutionsadvancement suffers from small time step restrictions. This is typical for explicit timestepping schemes like the Forward Euler technique described here.

29


4.2 Implicit Time Stepping

To improve on the time-step restriction of our numerical method we can consider discretizingthe Heat Equation as:

un+1i − uni

∆t− c

(un+1i−1 − 2un+1

i + un+1i+1

h2

)= 0 (4.2.5)

The finite difference approximation of the second derivative in (4.2.5) is now consideredat the current time-step instead of at the known lagged time step. Finding the value ofun+1h now requires solving a system instead of a matrix multiply. The system in (4.2.5) is

a Backward Euler or Implicit time stepping scheme and has a larger range of stable timesteps with

∆t ≈ h

Rearranging the terms in (4.2.5) we can work to set up the following matrix system foradvancing the solution temporally:

un+1i − c∆t

h2

(un+1i−1 − 2un+1

i + un+1i+1

)= uni

Here we can define:

γ =c∆t

h2

This gives:−γun+1

i−1 + (1 + 2γ)un+1i − γun+1

i+1 = uni

Thus,Aun+1

h = unh

withai,i−1 = −γ, ai,i = (1 + 2γ), and ai,i+1 = −γ.

Make note that the Dirichlet boundary conditions here may be set in the same manner aswith the explicit time stepping method already discussed. Specifically

a1,1 = 1 and am,m = 1.

Dirichlet boundary conditions, being known, may also be taken out of the system completely.This is done by adjusting the values on the right hand side vector (unh in our current example).

4.2.1 Tri-Diagonal Systems

The matrix system created in order to solve (4.2.5) is a tri-diagonal system or bandedsystem, and can be solved readily using a tridiagonal solver.

30


Consider the system: d1 c1

a2 d2 c2

a3 d3 c3

. . . . . . . . .am dm

x1

x2...xm

=

b1

b2...bm

(4.2.6)

The two general steps in solving the system can be thought of as:

1. Forward Elimination (step 1): subtract a2/d1 times row 1 from row 2. Creating a0 in the a2 position. In general this modifies the system values as follows (for i ∈2, 3, . . . ,m:

di = di −(

aidi−1

)ci−1

bi = bi −(

aidi−1

)bi−1

2. Back Substitution (Step 2): In this portion of the solver the value for xi are computedstarting with xm = bm

dm, and in general as:

xi =(bi − cixi+1)

di

31


The following listing is a simple Tri-Diagonal solver.

Listing 4.2: A Tri-Diagonal Solver

func t i on [ x ] = Tr iSo lve ( A, b )% Tr iSo lve A s imple Tri−Diagonal So lve r% ( John Ch r i s p e l l −−−−−MATH 640)% This s imple func t i on Takes a Tr ida igona l System% and re tu rn s a s o l u t i o n vec to r x% that s a t i s f i e s Ax = b .

n = length (b ( : , 1 ) ) ;% Forward El iminat ionf o r i =2:n

xmult = A( i , i −1)/A( i −1, i −1);A( i , i ) = A( i , i ) − xmult∗A( i −1, i ) ;b ( i , 1 ) = b( i , 1 ) − xmult∗b( i −1 ,1) ;

end

% Back Subs t i tu t i onx (n , 1 ) = b(n , 1 ) /A(n , n ) ;f o r i = (n−1):(−1):1

x ( i , 1 ) = (b( i , 1 ) − A( i , i +1)∗x ( i +1))/A( i , i ) ;end

Use the code above for a Tri-Diagonal solver to implement a Backward Euler time steppingscheme for the Heat Equation.

4.3 Order of Accuracy

For the different methods we have discussed it is often necessary to confirm that the methodshave been implemented correctly.

Consider adding a forcing function to the Heat Equation such that we can drive the solutionto a known given function value. For example if we desire that the true solution to thefunction be:

u(t, x) = e−t sin(x) cos(x)

By setting

f(t, x) = 4ce−t sin(x) cos(x)− e−t sin(x) cos(x)

32


and adding f as a right hand side forcing function in our governing PDE we have the system:

∂u

∂t− c∂

2u

∂x2= f for x ∈ [0, 2π] governing PDE

u(t, 0) = 0 boundary conditionu(t, 2π) = 0 boundary conditionu(0, x) = sin(x) cos(x) initial condition

allows us to know the true solution and the error can be examined. Specifically we canlook at the norm of the error between the computed solution and the true solution for anycomputational grid. If the approximated solution was continuous in space and time then theerror between the true solution denoted by u(t, x) and the approximated solution by uh(t, x)would be written as:

error =

∫ T

0

∫ 2π

0

|u(t, x)− uh(t, x)| dx dt (4.3.7)

Since the solution to u(x, t) is being approximated usingm discrete points with separation h,and advanced in time using k time steps of size ∆t we can compute a discreta approximationusing a discrete approximation (4.3.7) as:

errorh,∆t =

(k∑j=1

(m∑i=1

(u(tj, xi)− uh(tj, xi))2 h

)∆t

) 12

wheretj := j ×∆t, and xi = i× h.

Note that this discrete norm mimics the continuous error given by the integral expressionof the error given in (4.3.7). With a method of computing the error in our approximation athand the order of accuracy of a computational method may be examined discretely.

33


34

Chapter 5

Deriving the Navier-Stokes Equations

No one saw it comingAnd no one sees it stillEmpty space and memoriesToo beautifulToo beautifulToo beautiful for me.

−−Trampled By Turtles, Beautiful

5.1 Derivation of Conservation Equations

The following will give an introduction to the equations of fluid mechanics from first princi-ples. The conservation of mass equation is derived first followed by conservation of momen-tum in the following section.

5.1.1 Conservation of Mass

Start by considering the state variables for a fluid. Defining the fluids density (ρ) as a functionof position (x) and time (t), the total mass of a fluid in a region Ω may be calculated as

Total Fluid Mass :=

∫Ω

ρ(x, t) dx.

The fluids velocity is a vector function of position and time that gives the direction a pointparticle is moving at any instant in time. In three dimensions

u = u(x(t), t) =∂x

∂t=

∂x1∂t∂x2∂t∂x2∂t

=

u1(x, t)u2(x, t)u3(x, t)

.

35


Examining the trajectory of a particle originally at position x(0) = a in the fluid we can

$\Omega(0)$ $\Omega(t)$

ax

Figure 5.1.1: Flow map of partical originally at position x = a.

visualize the flow map as seen in Figure 5.1.1.

The acceleration of a particle at any position x in the fluid is written as

acceleration :=∂2x

∂t2.

Looking only at the ith component of the acceleration using the chain rule gives:

duidt

=d

dt[ui(x1(t), x2(t), x3(t), t)]

=∂ui∂x1

∂x1

∂t+∂ui∂x2

∂x2

∂t+∂ui∂x3

∂x3

∂t+∂ui∂t

= (u · ∇)ui +∂ui∂t

This leads to use of the material derivative or convected derivative when expressing acceler-ation of a particle in the fluid

Du

Dt= (u · ∇)u+

∂u

∂t.

Note that for a scalar function c(x(t), t), possibly denoting temperature as a function ofposition x and time t has the material derivative

Dc

Dt= (u · ∇)c+

∂c

∂t.

5.1.1.1 Reynolds Transport Theorem

Conservation of mass within a fluid flow implies that the rate of change of the total amountfluid with in a defined region Ω will be zero as the region progresses in the flow over time.Thus,

∂

∂t

∫Ω(t)

ρ(x(t), t) dx = 0.

A change of variables will allow for the temporal derivative to be brought inside the integral.Defining J to be the det( Jacobian of Flow Map) for x(t, a). Then

J(t) = det

[∂xi∂aj

], (5.1.1)

36


and

∂

∂t

∫Ω(t)

ρ(x(t), t) dx =∂

∂t

∫Ω(0)

ρ(x(t, a), t)J(t) da

=

∫Ω(0)

∂

∂tρ(x(t, a), t)J(t) da

=

∫Ω(0)

Dρ

DtJ + ρ

∂J

∂tda

=

∫Ω(0)

(Dρ

Dt+ ρ∇ · u

)J da

=

∫Ω(t)

Dρ

Dt+ ρ∇ · u dx

=

∫Ω(t)

Dρ

Dt+ ρ∇ · u dx

Assuming mass is conserved expanding the material derivative and looking at a single pointgives

∂ρ

∂t+ u · ∇ρ+ ρ∇ · u = 0⇐⇒ ∂ρ

∂t+∇ · (uρ). (5.1.2)

Expression (5.1.2) is a general expression for conservation of mass. In the derivation usewas made of the fact that the determinate of the Jacobian for the flow map satisfies thedifferential equation

∂J

∂t= (∇ · u)J with J(0) = 1. (5.1.3)

If the fluid is incompressable then J(t) = 1, and the conservation of mass equation (5.1.2) isequivalent to

∇ · u = 0 =⇒ Dρ

Dt=∂ρ

∂t+ u · ∇ρ = 0.

The same result can be achieved by assuming that the fluid density ρ is constant. Note thatincompressabelity of the fluid may be thought of only in terms of the divergence of the fluidsvelocity field u.

5.1.2 Conservation of Momentum

Considering the classic force = mass × acceleration for a fluid in region Ω gives

d

dt

∫Ω(t)

ρ(x(t), t)u(x(t), t) dx =

∫Ω(t)

body forces dx+

∫∂Ω(t)

surface forces dA

If we consider the forces per unit area then in the limit the “surface forces” will sum to zero,and we will need to consider what an appropriate body force is. For a general frame worklets start by considering:

∇ ·T + f

37


Here if we change the total stress tensor for the body force will allow for different types ofmaterials to be modeled.

Putting the conservation momentum and conservation of mass equations for the motion ofan incompressible fluid flow in a given domain Ω ⊂ Rd (d = 2, 3) then gives us:

ρ

(∂u

∂t+ u · ∇u

)= ∇ ·T + f in Ω

∇ · u = 0 in Ω

where the fluid’s velocity is given u, T denotes the total stress tensor, ρ is the fluid density,and f denotes the body force density acting on the fluid. The total stress tensor is composedof a pressure piece and an extra stress

T = −pI + τ (5.1.4)

with I representing a unit tensor. Substituting (5.1.4) into (5.1.4) the conservation equationsmay be written:

ρ

(∂u

∂t+ u · ∇u

)+∇p−∇ · τ = f in Ω (5.1.5)

∇ · u = 0 in Ω (5.1.6)

A constitutive equation is needed to define the relationship between the extra stress τ andthe fluids velocity. In the case of Newtonian fluids the constitutive equation is

τ = 2µsd(u), (5.1.7)

where µs denotes the fluids viscosity, and

d(u) =1

2

(∇u+∇uT

)denotes the fluids deformation tensor. Using the given Newtonian constitutive relationship(5.1.7) with (5.1.5) yields the Navier-Stokes equations in dimensional form:

ρ

(∂u

∂t+ u · ∇u

)+∇p− µs∆u = f in Ω (5.1.8)

∇ · u = 0 in Ω. (5.1.9)

38

Documents

Notes: Special Topics in Computational Mathematics ... Special Topics in Computational Mathematics Numerical Methods for Computational Fluid Dynamics J.C. Chrispell Department of Mathematics