Theorem 1 (Mean Value Theorem). If f ∈ C[a, b] and f is differentiable

on (a, b), then there exists c ∈ (a, b) such that f ′(c) =f(b)−f(a)

b−a

Theorem 2 (Weighted Mean Value Theorem). Suppose f ∈ C[a, b]. theRiemann Integral of g exists on [a, b], and g(x) does not change sign on[a, b]. Then there exists a number c in (a, b) with∫ b

af(x)g(x) dx = f(c)

∫ b

ag(x) dx

Theorem 3 (Generalised Rolle’s Theorem). Suppose f ∈ C[a, b] is ntimes differentiable on (a, b). If f(x0) = f(x1) = · · · = f(xn) for somen + 1 distinct numbers a ≤ x0 < x1 < · · · < xn ≤ bn, then there existsc ∈ (x0, xn) ⊂ (a, b) such that f (n)(c) = 0.

Theorem 4 (Taylor’s Theorem). Suppose f ∈ Cn[a, b], that f (n+1) existson [a, b] and x0 ∈ [a, b]. For every x ∈ [a, b], there exists a number ξ(x)between x0 and x (exclusively if x 6= x0) with

f(x) =

n∑k=0

f (k)(x0)

k!(x− x0)k +

f (n+1)(ξ(x))

(n+ 1)!(x− x0)n+1.

Benign cancelation occurs when subtracting exactly knwon quantities,catastrophic cancellation occurs when quantities to subtract are subject toround-off errors.

Definition 1 (Rate of Convergence). 1. Suppose {an}∞n=1 converges to

a number α. If a positive constant K exists with |an − α| ≤ K · 1hp

for large n, then we say that {an}∞n=1 converges to α at a rate of

convergence O( 1hp

), denoted by an = α+O( 1hp

)

2. Suppose limh→0+ F (h) = L. If a positive constant K exists with|F (h) − L| ≤ K · hp for sufficiently small h, then we say that F (h)converges to L with the rate of convergence of O(hp). It is indicatedby F (h) = L+O(hp).

Theorem 5 (Lagrange’s Interpolating Polynomial). Suppose x0, x1, · · ·xnare distinct numbers in the interval [a, b] and f ∈ Cn+1[a, b]. Then, foreach x in [a, b], a number ξ(x) between x0, x1, · · ·xn and hence in (a, b)exists with

f(x) = P (x) +f (n+1)(ξ(x))

(n+ 1)!

n∏k=0

(x− xk)

where P (x) is the interpolating polynomial given by

P (x) =

n∑k=0

f(xk)Ln,k(x), Ln,k =

n∏i=0,i 6=k

x− xixk − xi

Definition 2 (Divided Difference). The n − th divided difference of afunction with respect to x0, x1, · · · , xn is defined as

f [x0, x1, · · ·xn] =

{f(x0) if n = 0f [x1,x2,···xn]−f [x0,x1,···xn−1]

xn−x0if n > 0

Theorem 6 (Newton’s Divided Difference Interpolation Formula). Sup-pose that f ∈ Cn[a, b] and x0, x1, · · ·xn are n + 1 distinct points in [a, b]for some n ≥ 1, then the polynomial

Pn(x) = f [x0] +

n∑k=1

f [x0, x1, · · ·xk](x− x0)(x− x1) · · · (x− xk−1)

interpolates f at x0, x1, · · ·xn.

Theorem 7 (Hermite’s Interpolating Polynomial). If f ∈ C[a, b] andx0, x1, · · ·xn ∈ [a, b] are distinct, the unique polynomial of least degreeagreeing with f and f ′ at x0, x1, · · ·xn is the Hermite polynomial of degreeat most 2n+ 1 given by

H2n+1(x) =n∑

j=0

f(xj)Hn,j(x) +n∑

j=0

f ′(xj)Hn,j(x)

where, for Ln,j(x) denoting the jth Lagrange coefficient polynomial ofdegree n, we have

Hn,j = [1− 2(x− xj)L′n,j(xj)]L2n,j(x),

Hn,j = (x− xj)L2n,j(x)

Moreover, if f ∈ C2n+2[a, b], then

f(x) = H2n+1(x) +(x− x0)2 · · · (x− xn)2

(2n+ 2)!f (2n+2)(ξ(x))

Also, define a new sequence z0, z1, · · · zn by z2i = z2i+1 = xi for eachi = 1, 2, · · ·n. We have an alternate form of Hermite’s interpolating poly-nomial:

H2n+1 = f [z0] +

2n+1∑k=1

f [z0, z1, · · · zk](x− z0)(x− z1) · · · (x− zk−1)

Algorithm 1 (Constructing Natural Cubic Spline). Given n + 1 nodesx0, x1, · · ·xn and their corresponding function values, the natural cubicspline interpolant will be in the form of Sn(x) = an + bn(x−xn) + cn(x−xn)2 + dn(x− xn)3 for n = 0, 1, · · ·n− 1. To obtain the coefficients,

1. Compute hi = xi+1 − xi for all i = 0, 1, · · ·n− 1

2. Compute ai = f(xi) for all i = 0, 1, · · ·n

3. Solve the linear system Ax = b where

A =

1 0 0 . . . . . . 0

h0 2(h0 + h1) h1. . .

...

0 h1 2(h1 + h2) h2. . .

......

. . .. . .

. . .. . .

......

. . . hn−2 2(hn−2 hn−1

+hn−1)0 . . . . . . 0 0 1

,

x =

c0c1...cn

, b =

03h1

(a2 − a1)− 3h0

(a1 − a0)

...3

hn−1(an − an−1)− 3

hn−2(an−1 − an−2)

0

4. Compute bj for j = 0, 1, · · ·n− 1 as

bj =1

hj(aj+1 − aj)−

1

3hj(cj+1 + 2cj)

5. Compute dj for j = 0, 1, · · ·n− 1 as

dj =1

3hj(cj+1 − cj)

Theorem 8 (Forward-Difference Formula).

f ′(x0) =f(x0 + h)− f(x0)

h−h

2f ′′(ξ(x))

for some ξ(x) between x0 and x0 + h

Theorem 9 (Three-Point Midpoint Formula).

f ′(x0) =1

2h[f(x0 + h)− f(x0 − h)]−

h2

6f (3)(ξ(x))

for some ξ(x) between x0 − h and x0 + h

Theorem 10 (Three-Point Endpoint Formula).

f ′(x0) =1

2h[−3f(x0) + 4f(x0 + h)− f(x0 + 2h)] +

h2

3f (3)(ξ(x))

for some ξ(x) between x0 and x0 + 2h

Theorem 11 (Five-Point Midpoint Formula).

f ′(x0) =1

12h[f(x0−2h)−8f(x0−h)+8f(x0+h)−f(x0+2h)]+

h4

30f (5)(ξ(x))

for some ξ(x) between x0 − 2h and x0 + 2h

Theorem 12 (Five-Point Endpoint Formula).

f ′(x0) =1

12h[−25f(x0) + 48f(x0 + h)

− 36f(x0 + 2h) + 16f(x0 + 3h)− 3f(x0 + 4h)] +h4

5f (5)(ξ(x))

for some ξ(x) between x0 and x0 + 4h

Theorem 13 (Second Derivative Midpoint Formula).

f ′′(x0) =1

h2[f(x0 + h)− 2f(x0) + f(x0 − h)]−

h2

12f (4)(ξ(x))

for some ξ(x) between x0 − h and x0 + h

Theorem 14 (Midpoint Rule (Open n = 0, h = (b− a)/(n+ 2))).∫ b

af(x) dx = 2hf(x0) +

h3

3f ′′(ξ)

Theorem 15 (Trapezoidal Rule (Closed n = 1, h = (b− a)/n)).∫ b

af(x) dx =

h

2[f(xo) + f(x1)]−

h3

12f ′′(ξ)

for some ξ between x0 and x1.

Theorem 16 (Simpson’s Rule).∫ b

af(x) dx =

h

3[f(x0) + 4f(x1) + f(x2)]−

h5

90f (4)(ξ)

for some ξ between x0 and x2.

Theorem 17 (Composite Simpson’s Rule (h = (b− a)/n)).

∫ b

af(x) dx =

h

3

f(a) + 2

(n/2)−1∑j=1

f(x2j) + 4

n/2∑j=1

f(x2j−1) + f(b)

−b− a180

h4f (4)(µ)

Theorem 18 (Composite Trapezoidal Rule).∫ b

af(x) dx =

h

2

f(a) + 2

n−1∑j=1

f(xj) + f(b)

− b− a12

h2f ′′(µ)

Theorem 19 (Composite Midpoint Rule (h = (b− a)/(n+ 2))).∫ b

af(x) dx = 2h

n/2∑j=0

f(x2j) +b− a

6h2f ′′(µ)

for some µ between a and b.

Theorem 20 (Gaussian Quadrature Rules).∫ b

af(x) dx =

n∑i=1

cif(xi) +f (2n)(ξ)

(2n)!

∫ b

a

n∏i=1

(x− xi)2 dx

Theorem 21 (Gaussian Quadrature on Arbitrary Interval).∫ b

af(x) dx =

∫ 1

−1f

((b− a)t+ (b+ a)

2

)(b− a)

2dt

Theorem 22. Suppose A ∈ Rn,n. A factorization A = LDLt, where L isunit lower triangular and D is diagonal is called an LDLt factorization. IfA is nonsingular, then A has a LDLt factorization if and only if A = At

and Ak is nonsingular for k = 1, 2, 3, · · · , n− 1.

Definition 3 (Strict Diagonally Dominant Matrix). The n× n matrix Ais said to be strictly diagonally dominant matrix when

|aii >n∑

j=1,j 6=i

|aij

Gaussian elimination can be performed on strictly diagonally dominantmatrix without row/column interchanges.

Definition 4 (Positive Definite Matrix). A matrix A is positive definiteif it is symmetric and if xtAx > 0 for every x ∈ Rn/{0}.

Theorem 23. Let A be a symmetric matrix. The following are equivalent:

1. A is positive definite;

2. each of A’s leading principle submatrices has a positive determinant;

3. A = BtB for some B with full column rank.

Theorem 24. A symmetric matrix A is positive definite if and only ifGaussian elimination without row interchanges can be performed on thelinear system Ax = b with all the pivot elements positive.

Theorem 25 (Cholesky Factorisation). The matrix A is positive definiteif and only if A can be factorised in the form LLt where L is is lowertriangular with positive diagonal entries.

1. Set l11 =√a11 and lj1 = aj1/l11

2. Set lii =(aii −

∑i−1k=1 l

2ik

)1/2, lji =

(aji −

∑i−1k=1 ljklik

)/lii

Algorithm 2 (Crout’s Method for Solving Tridiagonal Matrices). .

1. Set l1 = α1 and u1 = γ1/l1.

2. For i = 2, · · · , n, set li−1 = βi−1, li = αi − li−1ui−1, ui = γi/li.

3. Set ln−1 = βn, ln = α− ln−1un−1

Theorem 26 (Trigonometric Interpolation). The interpolatory trigono-metric polynomial on 2n equidistant points is given by

S(n) =a0

2+

n−1∑k=1

(ak cos kx+ bk sin kx) +an

2cosnx

where

ak =1

n

2n−1∑j=0

yj cos kxj for each k = 0, 1, · · · , n

bk =1

n

2n−1∑j=0

yj sin kxj for each k = 0, 1, · · · , n− 1

Definition 5 (Inverse Discrete Fourier Transform). Given a vector

(f0, f1, · · · ˆfN−1)t of complex numbers, we define its DFT to be

fj =

N−1∑j=0

fkeijk2πN

for j = 0, 1, 2, · · ·N − 1.

Algorithm 3 (Fast Fourier Transform). .

1. Compute (g0, g1, · · · gN/2−1) = IDFT (f0, f2, · · · , ˆfN−2)

2. Compute (h0, h1, · · ·hN/2−1) = IDFT (f1, f3, · · · , ˆfN−1)

3. Set fj = gj + ζjNhj for j = 0, 1, 2, · · · , N/2− 1

4. Set fj+N/2 = gj − ζjNhj for j = 0, 1, 2, · · · , N/2− 1

ak =(−1)k

nRe(ck), bk =

(−1)k

nIm(ck)