Embed Size (px)
Citation preview
Analysis III
Lecturer: Prof. Oleg Zaboronski, Typesetting: David Williams
Term 1, 2015
Contents
I Integration 2
1 The Integral for Step Functions 21.1 Definition of the Integral for Step Functions . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Properties of the Integral for Step Functions . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2 The Integral for Regulated Functions 62.1 Definition of the Integral for Regulated Functions . . . . . . . . . . . . . . . . . . . . . . . 62.2 Properties of the Integral for Regulated Functions . . . . . . . . . . . . . . . . . . . . . . 8
3 The Indefinite Integral & Fundamental Theorem of Calculus 103.1 The Indefinite Integral . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.2 The Fundamental Theorem of Calculus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
4 Practical Methods of Integration 124.1 Integration by Parts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124.2 Integration by Substitution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134.3 Constructing Convergent Sequences of Step Functions . . . . . . . . . . . . . . . . . . . . 14
5 The Characterisation of Regulated Functions 155.1 Characterisation & Proof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
6 Improper & Riemann Integrals 176.1 Improper Integrals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176.2 The Riemann Integral . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
7 Uniform & Pointwise Convergence 217.1 Definitions, Properties and Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217.2 Uniform Convergence & Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257.3 Uniform Convergence & Differentiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
8 Functional Series 288.1 The Weierstrass M-Test & Other Useful Results . . . . . . . . . . . . . . . . . . . . . . . . 288.2 Taylor & Fourier Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
II Norms 33
9 Normed Vector Spaces 339.1 Definition & Basic Properties of a Norm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339.2 Equivalence & Banach Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
10 Continuous Maps 4010.1 Definition & Basic Properties of Linear Maps . . . . . . . . . . . . . . . . . . . . . . . . . 4010.2 Spaces of Bounded Linear Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
11 Open and Closed Sets of Normed Spaces 4311.1 Definition & Basic Properties of Open Sets . . . . . . . . . . . . . . . . . . . . . . . . . . 4311.2 Continuity, Closed Sets & Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4411.3 Lipschitz Continuity, Contraction Mappings & Other Remarks . . . . . . . . . . . . . . . 45
12 The Contraction Mapping Theorem & Applications 4612.1 Statement and Proof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4612.2 The Existence and Uniqueness of Solutions to ODEs . . . . . . . . . . . . . . . . . . . . . 4712.3 The Jacobi Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5012.4 The Newton-Raphson Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
1
Part I
Integration
1 The Integral for Step Functions
1.1 Definition of the Integral for Step Functions
Motivation: “Integrals = Areas Under Curves”
Consider constant functions of the form fc : x 7→ f0 ∈ R for x ∈ [a, b]. Let D be the area under the lineat f0 from a to b. Then it makes sense to define∫ b
a
fc(x) dx ..= sign(f0) ·Area(D)
= sign(f0) · (|f0| · (b− a))
= f0 · (b− a)
Now this definition will be extended to piecewise constant or step functions.
Definition 1: ϕ : [a, b] → R is called a step function if there exists a finite set P ⊂ [a, b] witha, b ∈ P such that ϕ is constant on the open sub-intervals of [a, b] \ P . P is called a partition of [a, b].More concretely, P = {a = p0, p1, p2, . . . , pk−1, pk = b}, with p0 < p1 < p2 < . . . < pk−1 < pk, and
ϕ∣∣∣(pi−1,pi)
= ci (an arbitrary constant), for i = 1, 2, . . . , k − 1, k.
Notes: If φ is a step function then there are many partitions of [a, b] such that φ is constant on theopen sub-intervals defined by the partition, call one P . There exists at least one other point greater thana in P (since both a and b belong to P by definition), call the first p. Then φ is also constant on theopen intervals of [a, b] \ (P ∪ {n}), where a < n < p. Let Q be a partition of [a, b], with Q ⊃ P . Then Qis called a refinement of P (so (P ∪ {n}) is a refinement of P ). The minimal common refinement of Pand Q is denoted by P ∪Q.
If φ is a step function on [a, b], then any partition P = {a = p0, p1, p2, . . . , pk−1, pk = b}, with p0 < p1 <
p2 < . . . < pk−1 < pk, such that φ∣∣∣(pi−1,pi)
= ci (an arbitrary constant), for i = 1, 2, . . . , k− 1, k, is called
compatible with, or adapted to, φ.
Proposition 1: Let S[a, b] be the space of step functions on [a, b]. If f, g ∈ S[a, b], then ∀µ, λ ∈ R,(λf + µg) ∈ S[a, b] (so S[a, b] is closed under taking finite linear combinations of its elements).
Proof: f ∈ S[a, b], so ∃P = {a = p0, p1, p2, . . . , pk−1, pk = b}, with p0 < p1 < p2 < . . . < pk−1 < pk,which is compatible with f , and g ∈ S[a, b] so ∃Q = {a = q0, q1, q2, . . . , ql−1, ql = b}, with q0 < q1 <q2 < . . . < ql−1 < ql, which is compatible with g. Consider P ∪ Q = {a = r0, r1, r2, . . . , rm−1, rm = b},with r0 < r1 < r2 < . . . < rm−1 < rm. Then this partition is compatible with (λf + µg). It remains to
show that (λf + µg)∣∣∣(ri−1,ri)
= ci (an arbitrary constant), for i = 1, 2, . . . ,m− 1,m.
P ⊂ (P ∪ Q), so (ri−1, ri) ⊂ (pa−1, pa) for some a, and Q ⊂ (P ∪ Q), so (ri−1, ri) ⊂ (qb−1, qb) for
some b. Then, since f∣∣∣(pa−1,pa)
is constant, f∣∣∣(ri−1,ri)
is constant, and since g∣∣∣(qb−1,qb)
is constant,
g∣∣∣(ri−1,ri)
is constant, so (λf + µg)∣∣∣(ri−1,ri)
is constant. Since the choice of i was arbitrary, this proves
the proposition. �
2
Notes: This demonstrates that S[a, b] is a vector space over R. Define C[a, b] as the space of continuousfunctions on [a, b], and define B[a, b] as the space of bounded functions on [a, b]. These are also vectorspaces. It was demonstrated in Analysis II that C[a, b] ⊂ B[a, b], and by definition S[a, b] ⊂ B[a, b].
Definition 2: For φ ∈ S[a, b], let P = {pi}ki=0 be a partition compatible with φ. Define (temporarily)
φi ..= φ∣∣∣(pi−1,pi)
. Then∫ ba
: S[a, b]→ R is defined as
∫ b
a
φ ..=
k∑i=1
φi(pi − pi−1)
Remark: If φ ≡ φ0 ∈ R, then P = {a, b}, so∫ baφ = φ0(b− a), which is precisely the area described in
the motivation.
Lemma 2: The value of∫ baφ does not depend on the choice of partition compatible with [a, b].
Proof: For any φ ∈ S[a, b], let P and Q be partitions compatible with [a, b]. Suppose there are k points
in Q that are not in P . Denote them n1, n2, . . . , nk. Temporarily let∫ baφ∣∣∣P
be the integral of φ from a
to b defined by the partition P which is compatible with [a, b].
Now, consider P ∪ {n1}. Since Q is compatible with [a, b], we know that n1 ∈ [a, b], so ∃i ∈ N such thatn1 ∈ (pi−1, pi) for some pi−1, pi ∈ P . φ is constant on (pi−1, pi), and so is also constant on (pi−1, n1)and (n1, pi), so P ∪ {n1} is compatible with φ. By definition∫ b
a
φ∣∣∣P∪{n1}
−∫ b
a
φ∣∣∣P
= φi(n1 − pi−1) + φi(pi − n1)− φi(pi − pi−1) = 0, so
∫ b
a
φ∣∣∣P∪{n1}
=
∫ b
a
φ∣∣∣P
The proof is completed by induction, since there are a finite number of points. This is left as anexercise. �
3
1.2 Properties of the Integral for Step Functions
1. Proposition 3 - Additivity: For any φ ∈ S[a, b], ∀v ∈ (a, b),∫ b
a
φ =
∫ v
a
φ+
∫ b
v
φ
2. Proposition 4 - Linearity: For any φ, ψ ∈ S[a, b], ∀λ, µ ∈ R,∫ b
a
λφ+ µψ = λ
∫ b
a
φ+ µ
∫ b
a
ψ
3. Proposition 5 - The Fundamental Theorem of Calculus: Let φ ∈ S[a, b], and let P be apartition of [a, b] compatible with φ. Consider I : [a, b]→ R, I : x 7→
∫ xaφ. Then
(I) I is continuous on [a, b].
(II) I is differentiable on⋃ki=1(pi−1, pi), and, ∀x ∈
⋃ki=1(pi−1, pi), I
′(x) = φ(x).
Proof:
1. Let P = {pi}ki=0 be a partition compatible with φ ∈ S[a, b]. Then there are two cases:
(i) If v ∈ P , then by definition∫ b
a
φ =
k∑i=1
φi(pi − pi−1) =
v∑i=1
φi(pi − pi−1) +
k∑i=v+1
φi(pi − pi−1) =
∫ v
a
φ+
∫ b
v
φ
(ii) If v /∈ P , then ∃i ∈ N, 1 ≤ i ≤ k, such that v ∈ (pi−1, pi). Then {a = p0, . . . , pi−1, v},{v, pi, . . . , pk = b} are partitions compatible with φ, so φ
∣∣∣(a,v)
∈ S[a, v], φ∣∣∣(v,b)
∈ S[v, b], so
their integrals are well-defined on those intervals. Then∫ b
a
φ =
k∑j=1
φj(pj − pj−1)
= φ1(p1 − p0) + φ2(p2 − p1) + . . .+ φi(pi − pi−1) + . . .+ φk(pk − pk−1)
= φ1(p1 − p0) + . . .+ φi(pi + v − v − pi−1) + . . .+ φk(pk − pk−1)
= φ1(p1 − p0) + . . .+ φi(v − pi−1) + φi(pi − v) + . . .+ φk(pk − pk−1)
=
i−1∑j=1
φj(pj − pj−1) + φi(v − pi−1)
]+
[φi(pi − v) +
k∑j=i+1
φj(pj − pj−1)
=
∫ v
a
φ+
∫ b
v
φ �
2. Let P be a partition compatible with φ ∈ S[a, b], and let Q be a partition compatible with ψ ∈S[a, b]. Then, as was proved in the previous section, P ∪ Q is compatible with both φ and ψ.Let P ∪ Q = {a = x0, x1, . . . , xk−1, xk = b}, with a = x0 < x1 < . . . < xk−1 < xk = b. Then
φ∣∣∣(xi−1,xi)
= φi and ψ∣∣∣(xi−1,xi)
= ψi, so (λφ+ µψ)∣∣∣(xi−1,xi)
= (λφ + µψ)i, so P ∪ Q is compatible
with λφ+ µψ. Then∫ b
a
λφ+ µψ =
k∑j=1
(λφ+ µψ)j(xj − xj−1) =
k∑j=1
(λφj + µψj)(xj − xj−1)
= λ
k∑j=1
φj(xj − xj−1) + µ
k∑j=1
ψj(xj − xj−1) = λ
∫ b
a
φ+ µ
∫ b
a
ψ �
4
3. Let P = {pi}ki=0 be a partition compatible with φ ∈ S[a, b]. Choose x ∈⋃kj=1(pj−1, pi), then
∃i ∈ N, 1 ≤ i ≤ k such that x ∈ (pi−1, pi). Then
I(x) =
∫ x
a
φ =
∫ pi−1
a
φ+
∫ x
pi−1
φ = I(pi−1) + φi(x− pi−1) = φix+ constant
so I is continuous on⋃kj=1(pj−1, pi) and I ′(x) = φi = φ(x). Furthermore, limx↓pi−1
I(x) = I(pi−1),
so I is right continuous on [a, b). Similarly, limx↑pi I(x) = I(pi−1) + φi(pi − pi−1) =∫ pi−1
aφ +
φi(pi− pi−1) =∫ pi−1
aφ+
∫ pipi−1
φ =∫ piaφ = I(pi), so I is left continuous on (a, b], so I is continuous
on [a, b]. �
Definition 3: Let f ∈ B[a, b]. Then the supremum norm (or sup norm) of f , denoted by ‖f‖∞, with‖·‖∞ : B[a, b]→ R is defined as
‖f‖∞ ..= supx∈[a,b]
∣∣f(x)∣∣
Proposition 6: Let φ ∈ S[a, b] be bounded between m,M ∈ R, so m ≤ φ(x) ≤M ∀x ∈ [a, b]. Then
m(b− a) ≤∫ b
a
φ ≤M(b− a), in particular
∣∣∣∣∣∫ b
a
φ
∣∣∣∣∣ ≤‖φ‖∞ (b− a).
Proof: For φ ∈ S[a, b], choose a compatible partition P . Now∫ b
a
φ =
k∑i=1
φi(pi − pi−1)
thenk∑i=1
m(pi − pi−1) ≤k∑i=1
φi(pi − pi−1) ≤k∑i=1
M(pi − pi−1)
so, due to telescoping sums on the upper and lower bounds,
m(b− a) ≤k∑i=1
φi(pi − pi−1) ≤M(b− a), so m(b− a) ≤∫ b
a
φ ≤M(b− a).
If ‖φ‖∞ is given, then, by definition, −‖φ‖∞ ≤ φ(x) ≤ ‖φ‖∞, so using the same argument as above,
−‖φ‖∞ (b− a) ≤∫ baφ ≤‖φ‖∞ (b− a), so
∣∣∣∫ ba φ∣∣∣ ≤‖φ‖∞ (b− a). �
Note: So far∫ baφ has been defined for b > a. Define, for a > b,
∫ baφ ..= −
∫ baφ. Then the bound proved
in the previous proposition becomes∣∣∣∫ ba φ∣∣∣ ≤ ‖φ‖∞|b− a|. It is a simple exercise to demonstrate that
such an extension preserves the three properties of the integral for step functions proved previously.
5
2 The Integral for Regulated Functions
2.1 Definition of the Integral for Regulated Functions
Definition 4: A function f on [a, b] is called a regulated function if, ∀ε > 0, ∃φ ∈ S[a, b] such that‖φ− f‖∞ < ε, or, alternatively, if, ∀ε > 0,
∣∣f(x)− φ(x)∣∣ < ε ∀x ∈ [a, b]. We denote the set of regulated
functions on [a, b] by R[a, b].
Definition 4.1: A sequence (φn)n≥1 ⊂ S[a, b] is said to converge uniformly to a function f : [a, b]→ Rif limn→∞‖f − φn‖∞ = 0.
Lemma : f : [a, b] → R is regulated iff ∃(φn)n≥1 ⊂ S[a, b] such that, as n → ∞, (φn) convergesuniformly to f .
Proof: The proof is routine and is left as an exercise. �
Definition 4.2: A function f : A ⊂ R → R is called uniformly continuous on A if, ∀ε > 0, ∃δ suchthat ∀x, y ∈ A, |x− y| < δ and
∣∣f(x)− f(y)∣∣ < ε.
Note: In general, uniform continuity implies continuity, but not the other way around. For example,f : (0, 1]→ R, f : x 7→ 1
x , is continuous, but not uniformly continuous. However, on closed intervals, thetwo are equivalent.
Lemma: For a function f : A→ R, if A = [a, b], continuity implies uniform continuity.
Proof: Suppose f is continuous on [a, b] but not uniformly continuous. Then ∃ε > 0 such that ∀δ > 0,∃x, y ∈ [a, b] such that |x− y| < δ but
∣∣f(x)− f(y)∣∣ ≥ ε. Taking δn = 1
n → 0 as n → ∞ yields twosequences (xn)n≥1, (yn)n≥1 ⊂ [a, b] with |xn − yn| < δn. Since (xn)n≥1 is bounded, it converges by theBolzano-Weierstrass Theorem, so ∃(xnk)k≥1 ⊂ (xn)n≥1 which converges in [a, b], since [a, b] is closed. Solimk→∞ xnk = u ∈ [a, b]. Choose (ynk)k≥1, which converges by a similar argument, and choosing δ = 1
nk
demonstrates that limk→∞∣∣xnk − ynk ∣∣ = 0, so limk→∞ ynk = u. But f is continuous, so limk→∞ f(xnk) =
f(u) = limk→∞ f(ynk), which contradicts the condition that |x− y| < δ but∣∣f(x)− f(y)
∣∣ ≥ ε, so f isuniformly continuous on [a, b]. �
Proposition 7: If a function f : [a, b]→ R is continuous on [a, b], then it is also regulated on [a, b].
Proof: For any ε > 0, let P = {pi}ki=0 be a partition of [a, b] such that |pi − pi−1| < δ, where δis such that |x− y| < δ =⇒
∣∣f(x)− f(y)∣∣ < ε. Such a δ exists because f is continuous and so
uniformly continuous on [a, b]. Let φ ∈ S[a, b] be such that
φ∣∣∣[pi−1,pi)
= f(pi−1) for i = 1, . . . , k
φ(b) = f(b). Now,
∀x ∈ [a, b), ∃j such that x ∈ [pj−1, pj), then∣∣f(x)− φ(x)
∣∣ =∣∣f(x)− f(pj−1)
∣∣ < ε since∣∣x− pj−1∣∣ < δ.
Also,∣∣f(b)− φ(b)
∣∣ = 0, so, ∀x ∈ [a, b],∣∣f(x)− φ(x)
∣∣ < ε, so f is regulated by definition. �
Exercise: Prove that any monotone function is regulated.
6
Proposition 8: For any f, g ∈ R[a, b] and λ, µ ∈ R, λf + µg ∈ R[a, b].
Proof: First prove that ∀f, g ∈ B[a, b], ∀λ ∈ R, ‖f + g‖∞ ≤‖f‖∞ +‖g‖∞ and ‖λf‖∞ = |λ|‖f‖∞. Theremainder of the proof is left as an exercise. �
Definition 5: Let f ∈ R[a, b]. Then∫ ba
: R[a, b]→ R is defined as∫ b
a
f ..= limn→∞
∫ b
a
φn
where (φn)n≥1 ⊂ S[a, b] converges to f uniformly as n→∞.
Proposition 9: limn→∞∫ baφn exists and is independent of the choice of (φn)n≥1 ⊂ S[a, b], which
converges to f ∈ R[a, b] uniformly as n→∞.
Proof: Let f ∈ R[a, b] and (φn)n≥1, (ψn)n≥1 ⊂ S[a, b] converge to f uniformly as n → ∞. (φn) con-verges to f uniformly, so for any ε > 0, eventually there will be an n such that ‖φn − f‖∞ < ε
2(b−a) , so
eventually there will be n andm, when sufficiently large, such that‖φn − φm‖∞ =∥∥(φn − f) + (f − φm)
∥∥∞
≤‖φn − f‖∞ +‖f − φm‖∞ ≤εb−a . Then
∣∣∣∫ ba φn − ∫ ba φm∣∣∣ =∣∣∣∫ ba (φn − φm)
∣∣∣ ≤‖φn − φm‖∞ (b− a) < ε, so
(∫ baφn)n≥1 is Cauchy, so the limit exists.
Let ωn = (φn, ψn)n−1 = (φ1, ψ1, φ2, ψ2, . . .) ⊂ S[a, b]. Then ωn converges to f uniformly as n → ∞,
the proof of this fact is routine and is left as an exercise. limn→∞∫ baωn exists, so the subsequences
∫ baφn
and∫ baψn must converge to the same limit
∫ baf , so
∫ baφn =
∫ baψn. �
7
2.2 Properties of the Integral for Regulated Functions
Proposition 10: For any f ∈ R[a, b], u ∈ [a, b],∫ b
a
f =
∫ u
a
f +
∫ b
u
f
Proof: f is regulated, so ∃(φn)n≥1 ⊂ S[a, b] which converges uniformly to f . Then (φn)∣∣∣[a,u]
converges
uniformly to f∣∣∣[a,u]
, since restrictions are regulated. The same holds for (φn)∣∣∣[u,b]
. So
∫ u
a
f +
∫ b
u
f = limn→∞
(∫ u
a
φn +
∫ b
u
φn
)= limn→∞
∫ b
a
φn =
∫ b
a
f
by the additivity of integrals of step functions. �
Proposition 11: The map I : R[a, b]→ R, I : f 7→∫ baf is linear.
Proof: The proof is similar to a combination of the last proof and the proof for the linearity of theintegral for step functions, and so is left as an exercise. �
Note: Not all functions are regulated.
Example: Consider f : [0, 1]→ R, with
f : x 7→
{1 x ∈ Q0 x /∈ Q
f is also denoted by 1Q - the indicator of Q. Let φ be any step function on [a, b], with a compatible
partition P , and let φ1 = φ∣∣∣(po,p1)
. Then ‖f − φ‖∞ ≥∥∥∥∥f ∣∣∣
(p0,p1)− φ1
∥∥∥∥∞
= max(|φ1| ,|φ1 − 1|) ≥ 12 , so
there cannot exist a sequence of step functions converging uniformly to f .
Proposition 12: Suppose that f ∈ R[a, b], and that, ∀x ∈ [a, b], m ≤ f(x) ≤ M for some m,M ∈ R.Then
m(b− a) ≤∫ b
a
f ≤M(b− a)
Proof: f ∈ R[a, b], so, ∀ε > 0, ∃φε ∈ S[a, b] such that‖f − φε‖∞ < ε, so ∀x ∈ [a, b], f(x)−ε ≤ φε(x) ≤f(x) + ε, so m− ε ≤ φε(x) ≤M + ε, now using the result proved for step functions
(m− ε)(b− a) ≤∫ b
a
φε ≤ (M − ε)(b− a)
Choose ε = 1n for n ∈ N, then (φn)n≥1 converges uniformly to f . Take the limit of the above with ε = 1
n ,then
m(b− a) ≤∫ b
a
f ≤M(b− a) �
Exercise: Prove that, for any f ∈ R[a, b],∣∣∣∣∣∫ b
a
f
∣∣∣∣∣ ≤‖f‖∞ (b− a)
8
Example: f(x) = x is uniformly continuous on R, since∣∣f(x)− f(y)
∣∣ = |x− y|, so when|x− y| < δ = ε,∣∣f(x)− f(y)∣∣ < ε.
Proposition 13: Let fα : [1,∞) → R, fα : x 7→ xα for some α ≥ 0. Then fα is uniformly continuousfor any α ≥ 0 but not uniformly continuous for α > 1.
Proof: For α > 1, fα(x+δ)−fα(x) = (x+δ)α−xα = xα((1+ δx )α−1) ≥ xα(1+α δx−1) = αδxα−1 →∞
as x→∞, which contradicts the condition for uniform continuity.
For α < 1, α 6= 0, 0 ≤ fα(x+ δ)− fα(x) = xα((1 + δx )α − 1) ≤ xα(1 + αδ
x − 1), (α− 1) < 0, so xα → 0as x→∞. For any ε > 0, choose δ = ε
2αδ , then xα is uniform continuous.
The case when α = 0 is trivial and is left as an exercise. �
Proposition 14: If f : [a, b]→ R is weakly increasing, then f ∈ R[a, b].
Proof: For f(b) > f(a), let d = f(b)−f(a)n . Let P be a partition with n + 1 elements such that
p0 = a, pj = supx∈[a,b]{x : f(x) ≤ f(a) + dj}, and pn+1 = b. By construction, since pj > pj−1, then
∀x ∈ (pj−1, pj), f(a)+d(j−1) ≤ f(x) ≤ f(a)+dj. Let ϕn ∈ S[a, b] such that ϕn
∣∣∣(pj−1,pj)
= f(a)+dj and
ϕn(pj) = f(pj). Now, let x ∈ (pj−1, pj). Then, using the previous inequality, −d ≤ f(x)−ϕn(x) ≤ 0, so
‖f − ϕn‖∞ ≤ d = f(b)−f(a)n → 0 as n→∞, so f is regulated. �
Exercise: Prove that if f : [a, b] → R is weakly decreasing, then f is regulated. This proves that allmonotonic functions on bounded intervals are regulated.
Example: Let Q ∩ [0, 1] = { 01 ,11 ,
12 ,
13 ,
23 ,
14 ,
34 ,
15 , , . . .} = {q1, q2, q3, . . .}, and let
1(x > qn) =
{1 x > qn
0 x ≤ qn
Let f : [0, 1]→ R be given by
f : x 7→∞∑n=1
2−n1(x > qn)
which exists ∀ x ∈ [0, 1]. Then f is non-decreasing, and is regulated, but jumps an infinitely many times,so is impossible to draw.
9
3 The Indefinite Integral & Fundamental Theorem of Calculus
3.1 The Indefinite Integral
Definition 6: Let f ∈ R[a, b]. Define F : [a, b]→ R by
F : x 7→∫ x
a
f
F is called the indefinite integral of f .
Lemma: R[a, b] ⊂ B[a, b]
Proof: f ∈ R[a, b], so ∃φ ∈ S[a, b] such that ‖φ− f‖∞ ≤ 1, so ∀ x ∈ [a, b], φ(x)− 1 ≤ f(x) ≤ φ(x) + 1.Since step functions are bounded, φ ∈ B[a, b], so f ∈ B[a, b]. �
Proposition 15: F is continuous on [a, b].
Proof: Let x, y ∈ [a, b]. Then
∣∣F (x)− F (y)∣∣ =
∣∣∣∣∫ x
a
f −∫ y
a
f
∣∣∣∣ =
∣∣∣∣∣∫ x
y
f
∣∣∣∣∣ ≤‖f‖∞|x− y| ,so by the sequential definition of continuity, f is continuous.
Note: Let f : [a, b] → R. Then, if ∃ L > 0 such that, ∀ x, y ∈ [a, b],∣∣f(x)− f(y)
∣∣ ≤ L|x− y|. Then fis called Lipschitz continuous or Lipschitz. Lipschitz continuity implies continuity, but the reverse is notgenerally true.
Example: f(x) = e−x is Lipschitz on [0, 1], but g(x) =√x is not Lipschitz on [0, 1].
10
3.2 The Fundamental Theorem of Calculus
Theorem 16 - FTC1: Let f ∈ R[a, b] such that f is continuous at c ∈ (a, b). Then F (x) =∫ xaf is
differentiable at c and F ′(c) = f(c).
Proof: f is continuous at c, so, ∀ε > 0, ∃δ > 0 such that, ∀x ∈ (c − δ, c + δ),∣∣f(x)− f(c)
∣∣ < ε, sof(c)− ε < f(x) < f(c) + ε. Choose 0 < h < δ. Then, by integrating the previous statement
h(f(c)− ε) <∫ c+h
c
f < h(f(c) + ε), so h(f(c)− ε) <∫ c+h
a
f −∫ c
a
f = F (c+ h)− F (c) < h(f(c) + ε)
So, for any h ∈ (0, δ), h(f(c)− ε) < F (c+ h)− F (c) < h(f(c) + ε), so
−ε < F (c+ h)− F (c)
h− f(c) < ε
So F is differentiable at c, and F ′(c) = f(c). The proof for the case where −δ < h < 0 is similar and sois left as an exercise. �
Corollary 17: If f ∈ C[a, b], then f is differentiable on (a, b) and F ′ = f .
Proof: f is continuous, so it is regulated. Applying FTC1 completes the proof. �
Theorem 18 - FTC2: Let f : [a, b] → R be continuous, and let g : [a, b] → R be differentiable, withg′(x) = f(x) ∀x ∈ [a, b]. Then ∫ b
a
f = g(b)− g(a)
This is the Newton-Leibniz Formula, and g is called the antiderivative of f . This is sometimes written∫ b
a
g′ = g(b)− g(a)
Proof: Let δ : [a, b]→ R, with δ(x) = g(x)−∫ xaf . Then δ is continuous on [a, b]. it is also differentiable
on (a, b) by FTC1. By the MVT, δ(b)−δ(a) = δ(ξ)(b−a) for some ξ ∈ (a, b). But δ′(ξ) = g′(ξ)−f(ξ) by
FTC1, so δ′(ξ) = f(ξ)− f(ξ) = 0, so δ(a) = δ(b), so g(a)−∫ aaf = g(b)−
∫ baf , so
∫ baf = g(b)− g(a). �
11
4 Practical Methods of Integration
4.1 Integration by Parts
Proposition 20: If f, g ∈ R[a, b], then f · g ∈ R[a, b].
Proof: ‖f + g‖∞ ≤‖f‖∞ +‖g‖∞. Then ∃(ϕn), (ψn) ∈ S[a, b] such that (ϕn)→ f, (ψn)→ g uniformly.0 ≤‖f · g − ϕnψn‖∞ =‖f · g − ϕng + ϕng − ϕnψn‖∞ ≤
∥∥(f − ϕn)g∥∥∞ +
∥∥ϕn(g − ψn)∥∥∞.
Since supx∈[a,b]∣∣a(x) · b(x)
∣∣ ≤ supx∈[a,b]∣∣a(x)
∣∣·supx∈[a,b]∣∣b(x)
∣∣, this is less than or equal to‖g‖∞‖f − ϕn‖∞+‖ϕn‖∞‖g − ψn‖∞. Now, ‖g‖∞ is regulated, and so is bounded. ‖f − ϕn‖∞ ,‖g − ψn‖∞ → 0 as n → ∞,and ‖ϕn‖∞ = ‖ϕn − f + f‖∞ ≤ ‖ϕn − f‖∞ +‖f‖∞. Also, ‖ϕn − f‖∞ → 0 as n → ∞, and ‖f‖∞ isregulated, and so is bounded. Taken together, this means that ‖f · g − ϕnψn‖∞ → 0 as n → ∞, so(ϕnψn) converges uniformly to f · g, so f · g is regulated. �
Corollary 19 - Integration by Parts: If F and G are differentiable such that F ′ and G′ are contin-uous, hence regulated, then ∫ b
a
F ′G = F (b)G(b)− F (a)G(a)−∫ b
a
FG′
Proof: (FG)′ = FG′ + F ′G. Then∫ b
a
(FG)′ =
∫ b
a
FG′ + F ′G =
∫ b
a
FG′ +
∫ b
a
F ′G
By FTC2
F (b)G(b)− F (a)G(a) =
∫ b
a
FG′ + F ′G =
∫ b
a
FG′ +
∫ b
a
F ′G
So ∫ b
a
F ′G = F (b)G(b)− F (a)G(a)−∫ b
a
FG′ �
Note: F (b)G(b)− F (a)G(a) can be denoted by FG|ba.
Example: The Gamma Function Γ : (0,∞)→ R is given by
Γ : x 7→∫ ∞0
tx−1e−t dt ..= limε↓0,L→∞
∫ L
ε
tx−1e−t dt
Then
Γ(x) = limε↓0,L→∞
∫ L
ε
(1
xtx)′e−t dt
=1
xlim
ε↓0,L→∞(txe−t)|Lε −
∫ L
ε
tx1(−e−t) dt
=1
x
∫ ∞0
txe−t dt
=1
xΓ(x+ 1)
So Γ(x+ 1) = xΓ(x), and Γ(1) = 1, so Γ(n+ 1) = n!
12
4.2 Integration by Substitution
Corollary 21 - Integration by Substitution: Let f ∈ C[a, b], and let g : [c, d] → [a, b] be differen-tiable with Im g ⊂ [a, b]. Then ∫ d
c
f(g(x)) · g′(x) dx =
∫ g(d)
g(c)
f(t) dt
Proof: Let F (x) =∫ xaf . Since f is continuous, F is differentiable and F ′(x) = f(x) by FTC1, then
by FTC2 ∫ g(d)
g(c)
f(t) dt = F (g(d))− F (g(c))
Also, ddxF (g(x)) = F ′(g(x)) · g′(x) = f(g(x)) · g′(x) which is the initial integrand. Now, by FTC2,
F (g(x))|dc = F (g(d))− F (g(c)), so∫ d
c
f(g(x)) · g′(x) dx =
∫ g(d)
g(c)
f(t) dt �
Note: g does not need to be injective or surjective.
Example: Let g(x) = x2. Then ∫ a
−af(x2)(x2)′ dx = F (x2)|a−a
Example: Let
G(a, j) ..=
∫ ∞−∞
e−ax2+2jx dx = lim
L1→∞,L2→∞
∫ L1
−L2
e−ax2+2jx dx
for some a > 0, j ∈ R. Then
G(a, j) =
∫ ∞−∞
e−ax2+2jx dx
=
∫ ∞−∞
e−a(x2+ 2j
a x) dx
=
∫ ∞−∞
e−a((x−ja )
2− j2
a2) dx
= ej2
a
∫ ∞−∞
e−a(x−ja )
2
dx
Now, substituting g(x) = x− ja ,
ej2
a
∫ ∞−∞
e−a(x−ja )
2
dx = ej2
a
∫ ∞−∞
e−at2
dt = ej2
a
∫ ∞−∞
e−(√at)2 dt
Now, substituting h(t) =√at,
ej2
a
∫ ∞−∞
e−(√at)2 dt = e
j2
a1√a
∫ ∞−∞
e−y2
dy = ej2
a
√π
a
So
G(a, j) = ej2
a
√π
a
13
4.3 Constructing Convergent Sequences of Step Functions
Theorem 22: Let f : [a, b]→ R be regulated. Then
limn→∞
n∑j=1
b− ah
f(aj) =
∫ b
a
f(x) dx, where aj = a+b− ah
j
Proof: First, consider φ ∈ S[a, b]. Let P be a partition compatible with φ, with |P | = k. Then
0 ≤
∣∣∣∣∣∣∫ b
a
φ−n∑j=1
b− ah
φ(aj)
∣∣∣∣∣∣ ≤ k b− an 2‖φ‖∞ → 0 as n→∞
since errors only occur and contribute to the difference at jumps on the step function, of which there arek, and the maximum error this can be is twice the maximum value of the step function across the wholesection. Then ∫ b
a
φ = limn→∞
n∑j=1
b− ah
φ(aj)
Now, for f ∈ R[a, b], let
R(n)f =
n∑j=1
b− ah
f(aj), where aj = a+b− ah
j
Since f is regulated, then ,∀ε > 0, ∃φ ∈ S[a, b] such that ‖f − φ‖∞ < ε3 . Then∣∣∣∣∣R(n)
f −∫ b
a
f
∣∣∣∣∣ =
∣∣∣∣∣R(n)f −R(n)
φ +R(n)φ −
∫ b
a
φ+
∫ b
a
φ−∫ b
a
f
∣∣∣∣∣≤∣∣∣R(n)
f −R(n)φ
∣∣∣+
∣∣∣∣∣R(n)φ −
∫ b
a
φ
∣∣∣∣∣+
∣∣∣∣∣∫ b
a
φ− f
∣∣∣∣∣Now, ∣∣∣R(n)
f −R(n)φ
∣∣∣ ≤‖φ− f‖∞ (b− a) <ε
3(b− a)
and ∣∣∣∣∣R(n)φ −
∫ b
a
φ
∣∣∣∣∣ < ε
3(b− a) eventually in n
and ∣∣∣∣∣∫ b
a
φ− f
∣∣∣∣∣ =
∣∣∣∣∣∣n∑j=1
(f(aj)− φ(aj))b− an
∣∣∣∣∣∣ ≤n∑j=1
∣∣f(aj)− φ(aj)∣∣ b− a
n<ε
3(b− a)
since∣∣f(aj)− φ(aj)
∣∣ < ε3 ,
so ∣∣∣∣∣R(n)f −
∫ b
a
f
∣∣∣∣∣ < ε(b− a) eventually in n
so
limn→∞
R(n)f =
∫ b
a
f �
14
5 The Characterisation of Regulated Functions
5.1 Characterisation & Proof
Proposition 23: A function f ∈ R[a, b] iff, ∀x ∈ (a, b), f(x±) both exist, and both f(a+) and f(b−)exist, where f(x+) = limy↓x f(y) (the right limit), and f(x−) = limy↑x f(y) (the left limit). Alternatively,this can be stated as “all one-sided limits exist”.
Note: This characterisation only requires the existence of these limits, not equality, f(x+) 6= f(x−) isacceptable so long as they both exist.
Proof =⇒ : f is regulated, so, ∀ε > 0, ∃φ ∈ S[a, b] such that ‖f − φ‖∞ < ε. Let x ∈ (a, b).There are now two cases, φ is continuous at x, or φ is discontinuous at x. In either case, ∃δ > 0
such that φ∣∣∣(x,x+δ)
= φ0, a constant. Let (xn)n≥1 ⊂ R be any sequence such that, ∀n ∈ N, xn > x,
and limn→∞ xn = x. Eventually xn ∈ (x, x + δ). Then, eventually in n and m,∣∣f(xn)− f(xm)
∣∣ =∣∣f(xn)− φ0 + φ0 − f(xm)∣∣ ≤ ∣∣f(xn)− φ0
∣∣ +∣∣f(xm)− φ0
∣∣ < 2ε, since∣∣f(xn)− φ0
∣∣ ,∣∣f(xm)− φ0∣∣ < ε,
so f(xn)n≥1 is Cauchy, and so converges. If yn is any other sequence with yn ↓ x as n → ∞, then(f(yn))n≥1 converges to the same limit. The proof of this fact is routine after considering (xn, yn)n≥1,and so is left as an exercise. It has been shown that for any (xn)n≥1 ⊂ R such that xn ↓ x as n → ∞,limn→∞ f(xn) = f(x+) exists and does not depend on xn. A similar argument can be used to provethis result for the left limit, and not much modification is required for x = a, b, so these proofs are leftas exercises.
Proof ⇐= : All one-sided limits exist. Now, fix any ε > 0. Let
Aε = {γ ∈ [a, b] : ∃ψ ∈ S[a, b] such that
∥∥∥∥(f − ψ)∣∣∣[a,γ]
∥∥∥∥∞< ε}
The proof shall follow from proving three statements:
(i) Aε 6= ∅: f(a+) exists, so, ∀ε > 0, ∃δ > 0 such that, ∀x ∈ (a, a + δ),∣∣f(x)− f(a+)
∣∣ < ε. Take
γ = δ2 . Then define
φ(x) =
{f(a) x = a
f(a+) x ∈ (a, a+ δ2 )
It is clear that φ(x) ∈ S[a, a+ δ2 ]. Since
∣∣f(x)− f(a+)∣∣ < ε and
∣∣f(a)− φ(a)∣∣ = 0,∥∥∥∥(f − φ)
∣∣∣[a,a+ δ
2 ]
∥∥∥∥∞< ε, so γ = δ
2 ∈ Aε, so Aε 6= ∅.
(ii) supAε = b: Assume that c ..= supAε < b. Then, ∀δ > 0, (c−δ) ∈ Aε, so ∃φ ∈ S[a, c−δ] such that∥∥∥∥(f − φ)∣∣∣[a,c−δ]
∥∥∥∥∞< ε. But f has limits f(c±) at c, so ∀ε > 0, ∃δ1, δ2 such that, ∀x ∈ (c, c+ δ2),∣∣f(x)− f(c+)
∣∣ < ε and, ∀x ∈ (c − δ1, c),∣∣f(x)− f(c−)
∣∣ < ε. Choose δ < δ1. Now, let ψ ∈ S[a, b]be given by
ψ(x) =
f(c−) x < c
f(c) x = c
f(c+) x > c
and let φ ∈ S[a, b] be given by
φ(x) =
{φ(x) x ∈ [a, c− δ]ψ(x) x ∈ (c− δ, c+ δ2
2 ]
Then (c+ δ22 ) ∈ Aε, with δ2
2 > 0, which contradicts the fact that c = supAε, so b = c.
15
(iii) b ∈ Aε: It has been shown that, ∀δ > 0, ∃φ ∈ S[a, b − δ] such that
∥∥∥∥(f − φ)∣∣∣[a,b−δ]
∥∥∥∥∞< ε. But
f(b−) exists, so ∃δ1 > 0 such that, ∀x ∈ (b − δ1, b),∣∣f(x)− f(b−)
∣∣ < ε. Choose δ < δ1. Now, letψ ∈ S[a, b] be given by
ψ(x) =
{f(b) x = b
f(b−) x < b
and let φ ∈ S[a, b] be given by
φ(x) =
{φ(x) x ∈ [a, b− δ]ψ(x) x ∈ (b− δ, b]
Now, by definition,∥∥∥f − φ∥∥∥
∞< ε, so b ∈ Aε.
So γ = b, so ∃ϕ ∈ S[a, b] such that ‖f − ϕ‖∞ < ε, so f ∈ R[a, b]. �
Exercise: Prove that if f ∈ R[a, b], then the set of discontinuities of f is countable. Hint: Check that,∀ε > 0, the set of jumps of size ε or greater is countable.
16
6 Improper & Riemann Integrals
6.1 Improper Integrals
Motivation: It is possible to generalise the regulated integral to unbounded functions and/or un-bounded domains.
Example: ∫ 1
0
x−12 dx = 2
because, although x−12 is not regulated on (0, 1], ∀ε > 0, x−
12 ∈ R[ε, 1]. Then∫ 1
ε
x−12 dx = 2x
12 |1ε = 2− 2ε
12 → 2 as ε→ 0
Example: ∫ ∞1
1
x2dx = 1
because, ∀L > 1, 1x2 ∈ R[1, L]. Then∫ L
1
1
x2dx = − 1
x
∣∣∣∣L1
= 1− 1
L→ 1 as L→∞
Example: ∫ 0
1
1
xdx = lim
ε↓0
∫ 1
ε
1
xdx = lim
ε↓0log x|1ε = lim
ε↓0log
1
ε
which does not exist, therefore the integral is not defined.
Definition 7.1: If, for any ε > 0, f : (a, b]→ R is regulated on [a+ ε, b] and
l ..= limε↓0
∫ b
a+ε
f
exists, then∫ baf converges and is equal to l. If the limit does not exist, then
∫ baf diverges.
Definition 7.2: If, ∀L > 0, f ∈ R[a, L] and
l ..= limL→∞
∫ L
a
f
exists, then∫∞af converges and is equal to l. If the limit does not exist, then
∫∞af diverges.
17
Note: Let f ∈ R[−L1, L2] for some L1, L2 > 0. Then∫∞−∞ f converges if, ∀c ∈ R,
∫ c−∞ f and
∫∞cf
converge. In this case,∫ c−∞ f +
∫∞cf =
∫∞−∞ f . As an exercise, show that if
∫ c−∞ f and
∫∞cf exist, then∫ c
−∞ f +∫∞cf does not depend on the choice of c.
Example: Let f(x) = x ∈ R[−L,L] ∀L > 0. Then∫ L
−Lx dx = 0, so lim
L→∞
∫ L
−Lx dx = 0
However, take any c ∈ R. Then
limL→∞
∫ L
c
x dx =∞, and limL→∞
∫ c
−Lx dx = −∞
so do not exist. Then ∫ ∞−∞
x dx
does not exist, since the limits must exist independently.
Notes: Suppose f : (a, b) → R, and, for any ε1, ε2 > 0, f ∈ R[a + ε1, b − ε2]. Then∫ baf exists if, for
any c ∈ (a, b),∫ caf and
∫ bcf both exist (converge). Then
∫ baf converges, and
∫ baf =
∫ caf +
∫ bcf .
Suppose f is defined on [a, b)∪ (b, c]. Then∫ caf exists (converges) if
∫ baf and
∫ cbf both exist (indepen-
dently), with∫ caf =
∫ baf +
∫ cbf .
Example: f(x) = 1√|x|
for x ∈ [−1, 0) ∪ (0, 1].
Note: Let f : (a, b)→ R such that, for any ε1, ε2 > 0, f ∈ [a+ ε1, b− ε2]. Then ∃c ∈ (a, b) such that
l1(c) = limε1↓0
∫ c
a+ε1
f and l2(c) = limε2↓0
∫ b−ε2
c
f
exist iff, ∀c′ ∈ (a, b), l1(c′) and l2(c′) exist. The first formulation is often the more convenient in practice
however. If this formulation is true of f , then∫ baf converges, and
∫ baf = l1(c) + l2(c). As an exercise,
check that l1(c) + l2(c)− l1(c′)− l2(c′) = 0 to demonstrate that this integral is independent of the choiceof c.
Definition 7.3: Let f ∈ R[−L1, L2] for some L1, L2 > 0. Then∫ ∞−∞
f ..= limL1→∞
∫ L1
c
f + limL2→∞
∫ c
−L2
f
Definition 7.4: If f : [a, b) ∪ (b, c]→ R, then∫ c
a
f ..= limε1↓0
∫ b−ε1
a
f + limε2↓0
∫ c
b+ε2
f
Note: All integrals in the limit must make sense as regulated or improper integrals.
Note: It is possible to define the integral of functions with domains such as
f :⋃j∈Z
(j, j + 1)→ R
Situations like this occur when studying gamma functions.
18
6.2 The Riemann Integral
Definition: Let f : [a, b]→ R. Then
Uf ..= infφ∈S[a,b]
{∫ b
a
φ : φ ≥ f}
(“upper sum”)
Lf ..= supφ∈S[a,b]
{∫ b
a
φ : φ ≤ f}
(“lower sum”)
Definition: f : [a, b]→ R is Riemann integrable if Uf = Lf . Then
R
∫ b
a
f ..= Uf = Lf
Proposition: If f ∈ R[a, b], then it is Riemann integrable, and
R
∫ b
a
f =
∫ b
a
f
Proof: f ∈ R[a, b], so, ∀ε > 0, ∃φ ∈ S[a, b] such that, ∀x ∈ [a, b], φ(x) − ε ≤ f(x) ≤ φ(x) + ε. Sinceφ(x)− ε, φ(x) + ε ∈ S[a, b], ∫ b
a
φ− ε(b− a) ≤ Lf ≤ Uf ≤∫ b
a
φ+ ε(b− a)
Taking ε ↓ 0,
Lf = Uf =
∫ b
a
f = R
∫ b
a
f �
Note: There are functions which are Riemann integrable but not regulated.
Example: Let f : [0, 1]→ R be given by
f : x 7→
{1 x = 2−n for n = 0, 1, 2, . . .
0 otherwise
Then f is not regulated since, ∀p > 0, ∃x ∈ (0, p) such that f(x) = 1 and ∃y ∈ (0, p) such that f(y) = 0,so a step function can get no closer than a half to both “sections” of the function on any interval, so∫ baf is not defined.
However, let
φN =
{1 x ≤ 2−N
f(x) x > 2−N
Now, φN ≥ f ≥ 0, so
0 ≤ Lf ≤ Uf ≤∫ 1
0
φN = limN→∞
∫ 1
2−NφN = 0
Then Lf = Uf = 0, so
R
∫ 1
0
f = 0
so the Riemann integral of f is defined.
19
Note: There are functions which are not Riemann integrable.
Example: Let
1Q(x) =
{1 x ∈ Q0 x ∈ R \Q
Then U1Q = 1 6= 0 = L1Q , so 1Q is not Riemann integrable. However, it is Lebesgue integrable, with
L
∫ b
a
1Q = 0
20
7 Uniform & Pointwise Convergence
7.1 Definitions, Properties and Examples
Example: Let (fn)n≥1 be a sequence of functions on [-1,1], with
fn : x 7→ x−1
2n−1 for n = 1, 2, . . .
Fix x and consider
limn→∞
fn(x) =
1 x > 0
0 x = 0
−1 x < 0
, so limn→∞
fn(x) = signx.
So the limit of (fn(x))n≥1 as n → ∞ exists ∀x ∈ [−1, 1], and fn is continuous ∀n ∈ N, but the limit isnot continuous at 0.
limn→∞
limx→0±
fn(x) = 0 6= ±1 = limx→0±
signx = limx→0±
limn→∞
fn(x)
This function has non-commutative limits and a sequence of continuous functions converges to a discon-tinuous function, both of which are undesirable properties of pointwise convergence.
Definition 8: Let (fn)n≥1 be a sequence of functions, and let A ⊂ R. Then fn → f pointwise asn→∞ if, ∀x ∈ A, limn→∞ fn(x) = f(x).
Note: Pointwise convergence is very “loose”, it does not preserve properties of the fns such as continuity
(as in the case of x−1
2n−1 → signx). Pointwise convergence can also be very non-uniform (the sameexample converges pointwise very slowly near 0).
Note: Let f and (fn)n≥1 be functions on A ⊂ R. If (fn)→ f pointwise as n→∞, then f can fail tobe continuous, even if all fns are. Alternatively, ∃x0 ∈ A such that
limn→∞
limx→x0
fn(x) 6= limx→x0
limn→∞
fn(x)
Example (The Witch’s Hat): Let fn : [0, 1]→ R, with fn(0) = 0 and
fn(x) =
2n2x x ∈ [0, 1
2n ]
n− 2n2(x− 12n ) x ∈ ( 1
2n ,1n ]
0 x > 1n
Then fn ∈ C[0, 1]. Fix x > 0, then limn→∞ fn(x) = 0. Now, as n → ∞, fn → f ≡ 0 on [0, 1], which is
continuous. However,∫ 1
0f = 0, but
∫ 1
0fn = n · 1
2n = 12 , so
1
2= limn→∞
∫ 1
0
fn 6=∫ 1
0
limn→∞
fn = 0
It is clear from examples such as this that pointwise convergence must be strengthened.
Definition 9: Let f, fn : A ⊂ R→ R for n = 1, 2, . . ., then (fn)n≥1 → f converges uniformly as n→∞if, ∀ε > 0, ∃Nε such that, ∀n > Nε,
∣∣fn(x)− f(x)∣∣ < ε ∀x ∈ A, or, equivalently, if limn→∞‖f − fn‖∞ = 0.
21
Note: For the remainder of this section, assume, unless specified otherwise, that f, fn : A ⊂ R → Rfor n = 1, 2, . . ..
Remark: Uniform convergence =⇒ pointwise convergence, but pointwise convergence 6=⇒ uniformconvergence.
Definition 9.1: A sequence (fn)n≥1 (of functions on A ⊂ R) is called uniformly Cauchy if, ∀ε > 0,∃Nε such that, ∀n,m > Nε,
∣∣fn(x)− fm(x)∣∣ < ε ∀x ∈ A.
Note: For a fixed x ∈ A, if (fn(x))n≥1 is Cauchy, then it is pointwise convergent (to a common limit f).
Lemma: A uniformly Cauchy sequence converges uniformly.
Proof: Let (fn)n≥1, (fm)m≥1 be uniformly Cauchy sequences converging to f . Then, ∀ε > 0, ∃Nεsuch that, ∀n,m > Nε, fm(x) − ε
2 ≤ fn(x) ≤ fm(x) + ε2 ∀x ∈ A. Take m → ∞, then fm → f , so
f(x)− ε2 ≤ fn(x) ≤ f(x) + ε
2 ∀x ∈ A, so∣∣fn(x)− f(x)
∣∣ < ε ∀x ∈ A, so (fn)n≥1 converges uniformly. �
Theorem 24: Let (fn)n≥1 ⊂ R[a, b]. Assume that fn → f uniformly. Then
(i) f ∈ R[a, b]
(ii) limn→∞∫ bafn =
∫ ba
limn→∞ fn =∫ baf
Proof:
(i) fn ∈ R[a, b], so, ∀ε > 0, ∃φn ∈ S[a, b] such that‖fn − φn‖∞ < ε2 . Now,‖f − φn‖∞ =‖f − fn + fn − φn‖∞ ≤
‖f − fn‖∞ +‖fn − φn‖∞. Assume that n is large enough such that ‖f − fn‖∞ < ε2 , which is guar-
anteed since fn → f , so ‖f − φn‖∞ < ε, so f ∈ R[a, b].
(ii) It was shown in (i) that f is regulated, so∫ baf is well defined and
0 ≤
∣∣∣∣∣∫ b
a
f −∫ b
a
fn
∣∣∣∣∣ ≤‖f − fn‖∞ (b− a)
‖f − fn‖∞ → 0 as n→∞, so
limn→∞
∫ b
a
fn =
∫ b
a
limn→∞
fn =
∫ b
a
f �
Theorem 25: Suppose that (fn)n≥1 ⊂ C[a, b], and fn → f uniformly as n → ∞. Then f ∈ C[a, b].This theorem also holds for some arbitrary A ⊂ R in place of [a, b], and shall be proven for this moregeneral case.
Proof: fn → f uniformly, so ∀ε > 0, ∃Nε such that, ∀n > Nε,∣∣fn(x)− f(x)
∣∣ < ε ∀x ∈ A, so fn is
continuous ∀n ∈ N. Then ∃δ′ε such that, ∀x ∈ (x0 − δ′ε, x0 + δ′ε) ∩A,∣∣fn(x)− fn(x0)
∣∣ < ε3
∀x ∈ (x0 − δ′ε, x0 + δ′ε) ∩ A. Set δε = δ′ε. Then, ∀x ∈ (x0 − δ′ε, x0 + δ′ε) ∩ A,∣∣f(x)− f(x0)
∣∣ =∣∣f(x)− fn(x) + fn(x)− fn(x0) + fn(x0)− f(x0)∣∣ ≤ ∣∣fn(x)− fn(x0)
∣∣+∣∣fn(x)− fn(x0)∣∣+∣∣fn(x0)− f(x0)
∣∣.fn → f uniformly as n → ∞, so the first and last of these terms are less than ε
3 , and the middle termwas shown to be less than ε
3 earlier, so∣∣f(x)− f(x0)
∣∣ < 3ε3 = ε, so f is continuous at x0. �
22
Example (Cantor’s Function or The Devil’s Staircase): Let (fn)n≥1 ⊂ C[0, 1] be such thatf0 ≡ 1 and
fn+1(x) =
12fn(3x) x ∈ [0, 13 )12 x ∈ [ 13 ,
23 ]
12 + 1
2fn(3x− 2) x ∈ ( 23 , 1]
Proposition: limn→∞ fn exists and is continuous. It is called Cantor’s function.
Proof: Clearly f0 ∈ C[0, 1]. Now, suppose that fn ∈ C[0, 1]. Then fn+1 is clearly continuous on ( 13 ,
23 ).
Now, fn is continuous on [0, 13 ) by assumption, so 12fn(3·) is continuous, so fn+1 is continuous on [0, 13 ).
Also, fn is continuous on ( 23 , 1] by assumption, so 1
2 + 12fn(3· − 2) is continuous, so fn+1 is continuous
on ( 23 , 1]. Now the only points remaining to investigate the continuity of are 1
3 and 23 .
fn+1(1) = 12 + 1
2fn(1), which is 1 if fn(1) = 1. f0(1) = 1, so fn(1) = 1 ∀n ∈ N by induction.
Concerning the continuity of 13 , fn+1( 1
3 ) = 12 , and fn+1( 1
3+) = 12 , so fn+1 is right-continuous at 1
3 , andfn+1( 1
3−) = 12fn(3( 1
3−)) = 12fn(1−) = 1
2fn(1) = 12 since fn is continuous at 1, so fn+1 is left-continuous
at 13 , so fn+1 is continuous at 1
3 . The proof for 23 is similar and is left as an exercise.
Now,
‖fn+1 − fn‖∞ = max
(∥∥∥∥(fn+1 − fn)∣∣∣x∈[0, 13 )
∥∥∥∥∞,
∥∥∥∥(fn+1 − fn)∣∣∣x∈[ 13 ,
23 ]
∥∥∥∥∞,
∥∥∥∥(fn+1 − fn)∣∣∣x∈( 2
3 ,1]
∥∥∥∥∞
)
= max
(∥∥∥∥(fn+1 − fn)∣∣∣x∈[0, 13 )
∥∥∥∥∞, 0,
∥∥∥∥(fn+1 − fn)∣∣∣x∈( 2
3 ,1]
∥∥∥∥∞
)
= max
∥∥∥∥∥∥(
1
2fn(3·)− 1
2fn−1(3·)
)∣∣∣∣∣x∈[0, 13 )
∥∥∥∥∥∥∞
,
∥∥∥∥∥∥(
1
2fn(3· − 2)− 1
2fn−1(3· − 2)
)∣∣∣∣∣x∈( 2
3 ,1]
∥∥∥∥∥∥∞
≤ 1
2‖fn − fn−1‖∞
So, for some c, d ∈ R,
‖fn−1 − fn‖∞ ≤1
2‖fn − fn−1‖∞ ≤
1
22‖fn−1 − fn−2‖∞ ≤ . . . ≤
c
2n+1‖f1 − f0‖∞ ≤
d
2n+1
The constants are introduced to correct for f0.
Now, fix n > m, then, for some D ∈ R,
0 ≤‖fn − fm‖∞ =‖fn − fn+1 + fn+1 − fn−2 + . . .+ fm−1 − fm‖∞≤‖fn − fn−1‖∞ +‖fn−1 − fn−2‖∞ + . . .+‖fm+1 − fm‖∞
≤ D(
1
2n+
1
2n−1+ . . .+
1
2m+1
)≤ D
2m+1
n−m−1∑k=0
1
2k
≤ D
2m+1
∞∑k=0
1
2k=
D
2m→ 0 as m→∞
Since ‖fn − fm‖∞ can be made arbitrarily small by taking n > m large enough, (fn) is uniformlyCauchy. Therefore, f = limn→∞ fn exists and is continuous, since a uniformly Cauchy sequence convergesuniformly and fn is continuous ∀n ∈ N. The limit is called Cantor’s function or the Devil’s Staircase.
23
Now, f ∈ C[0, 1], so f ∈ R[0, 1]. Taking the limit of fn(x), it is clear that
f(x) =
12f(3x) x ∈ [0, 13 )12 x ∈ [ 13 ,
23 ]
12 + 1
2f(3x− 2) x ∈ ( 23 , 1]
So ∫ 1
0
f =
∫ 13
0
f +
∫ 23
13
f +
∫ 1
23
f =
∫ 13
0
1
2f(3x) dx+
∫ 23
13
1
2dx+
∫ 1
23
1
2+
1
2f(3x− 2) dx
Let y = 3x, z = 3x− 2. Then∫ 1
0
f =1
6+
1
6
∫ 1
0
f(y) dy +1
6+
1
6
∫ 1
0
f(z) dz =1
3+
1
3
∫ 1
0
f , so2
3
∫ 1
0
f =1
3, so
∫ 1
0
f =1
2�
Remark: Assume that f is constant on E ⊂ [0, 1]. Then f is constant on ( 12 ,
23 ) ∪ 1
3E ∪ ( 23 + 1
3E), so|E| = 1
3 + 13 |E| +
13 |E|, so 1
3 |E| =13 , so |E| = 1, so f is locally constant on some E ⊂ [0, 1] of length 1,
but f : [0, 1]→ [0, 1] continuously.
Note: It is possible to map [0, 1] ⊂ R onto (surjectively) [0, 1] × [0, 1] ⊂ R2 continuously. If γ :[0, 1] → [0, 1] × [0, 1] is such a map, then, ∀x ∈ [0, 1] × [0, 1], ∃t ∈ [0, 1] such that γ(t) = x. Asurjective map is relatively simple to construct, such as τ : [0, 1]→ [0, 1]× [0, 1], with τ : 0.x1x2x3 . . . 7→(0.x1x3x5 . . . , 0.x2x4x6 . . .), but τ is not continuous. A continuous map satisfying these criteria is calleda space filling curve, and examples were constructed by Hilbert and Peano in the 1890s.
24
7.2 Uniform Convergence & Integration
Note: Throughout this section, let D = [a, b]× [c, d] ⊂ R2, and (x, t) ∈ D.
Definition 10: A function f : D → R is continuous at (x0, t0) ∈ D if, ∀ε > 0, ∃δε(x0, t0) > 0 such that,∀(x0, t0) ∈ D with |x− x0| < δε(x0, t0) and |t− t0| < δε(x0, t0),
∣∣f(x, t)− f(x0, t0)∣∣ < ε. An equivalent
definition, which shall be introduced later, uses Euclidean distance.
Definition 10.1: A function f : D → R is uniformly continuous on D if, ∀ε > 0, ∃δε such that,∀(x, t), (x0, t0) ∈ D with |x− x0| < δε and |t− t0| < δε,
∣∣f(x, t)− f(x0, t0)∣∣ < ε.
Lemma 26.2: If f is continuous on D, then it is also uniformly continuous if D is closed.
Proof: The proof is left as an exercise. Hint: Fix t ∈ [c, d], and consider f(·, t) : [a, b] → R withf : x 7→ f(x, t). �
Exercise: Prove that, if f is continuous on D, then f(·, t) is continuous on [a, b] (so f(·, t) ∈ C[a, b]).
Lemma 26.1: If a map I is such that
I : t ∈ [c, d] 7→∫ b
a
f(x, t) dx
then, if f ∈ C(D), I ∈ C[c, d] (I is often called the integral depending on a parameter).
Proof: Let (tn)n≥1 ⊂ [c, d] with limn→∞ tn = t0, and let fn : [a, b] → R with fn : x 7→ f(x, tn). Now,f is continuous on D, so f is uniformly continuous on D since D is closed, so, ∀ε > 0, ∃δε>0 such that,if |x− x0| < δε and |t− t0| < δε, then
∣∣f(x, t)− f(x0, t0)∣∣ < ε. Since (tn), (tn+m)→ t0 as n→∞, then,
eventually in n, |tn − tn+m| < δε, so∣∣fn(x)− fn+m(x)
∣∣ =∣∣f(x, tn)− f(x, tn+m)
∣∣ < ε eventually in n, so(fn) is uniformly Cauchy, so (fn) converges uniformly to f . Now,
limn→∞
I(tn) = limn→∞
∫ b
a
f(x, tn) dx = limn→∞
∫ b
a
fn(x) dx =
∫ b
a
limn→∞
fn(x) dx =
∫ b
a
f(x, t0) = I(t0)
So I is continuous at t0, and since t0 ∈ [c, d] was arbitrary, I is continuous on [c, d]. �
Proposition 27: If f, ∂f∂t are continuous on [a, b]× [c, d], then, ∀t ∈ (c, d),
∂
∂t
∫ b
a
f(x, t) dx =
∫ b
a
∂f
∂t(x, t) dx
Or, alternatively, ∀t ∈ (c, d),
F (t) =
∫ b
a
f(x, t) dx and G(t) =
∫ b
a
∂f
∂t(x, t) dx
both exist on (c, d), F is differentiable on (c, d), and F ′ = G.
25
Proof: Fix t ∈ (c, d). Then f(·, t), ∂f∂t (·, t) ∈ C[a, b], so F and G exist. Now, ∃[c1, d1] ⊂ [c, d] such that
t ∈ (c1, d1). But ∂f∂t is continuous on [a, b]× [c, d], and so is uniformly continuous on [a, b]× [c, d]. Then∣∣∣∣F (t+ h)− F (t)
h−G(t)
∣∣∣∣ =
∣∣∣∣∣∫ b
a
f(x, t+ h)− f(x, t)
h− ∂f
∂t(x, t) dx
∣∣∣∣∣MVT=
∣∣∣∣∣∫ b
a
∂f
∂τ(x, τ)− ∂f
∂t(x, t) dx
∣∣∣∣∣for some τ ∈ (t, t+h). Now, ∂f∂τ is uniformly continuous on [a, b]× [c, d], so ∀ε > 0, ∃δε > 0 such that ∀hwith |h| < δε, ∣∣∣∣∂f∂τ (x, τ)− ∂f
∂t(x, t)
∣∣∣∣ < ε, so
∣∣∣∣∣∫ b
a
∂f
∂τ(x, τ)− ∂f
∂t(x, t) dx
∣∣∣∣∣ <∫ b
a
ε = ε(b− a)
so
limh→0
∣∣∣∣F (t+ h)− F (t)
h
∣∣∣∣ = G(t)
so F ′(t) = G(t). �
Theorem 28 - Fubini’s Theorem: If f : D → R is continuous, then∫ b
a
(∫ d
c
f(x, y) dy
)dx =
∫ d
c
(∫ b
a
f(x, y) dx
)dy
Proof: Let
F (t) =
∫ t
a
(∫ d
c
f(x, y) dy
)dx−
∫ d
c
(∫ t
a
f(x, y) dx
)dy
for t ∈ [a, b]. Now, F (a) = 0, and
F ′(t)FTC=
∫ d
c
f(t, y) dy −∫ d
c
f(t, y) dy = 0
F is continuous on [a, b] and differentiable on (a, b), and F ′(t) = 0, so F (b)− F (a) = 0, so∫ b
a
(∫ d
c
f(x, y) dy
)dx =
∫ d
c
(∫ b
a
f(x, y) dx
)dy �
Note: f must be continuous on the whole of D, or counterexamples such as the following can arise:
Let D = [0, 2]× [0, 1], and let f(x, y) : D → R with
f : (x, y) 7→
xy(x2−y2)(x2+y2)3 (x, y) 6= (0, 0)
0 (x, y) = (0, 0)
Then ∫ 2
0
(∫ 1
0
f(x, y) dy
)dx =
1
56= 1
20=
∫ 1
0
(∫ 2
0
f(x, y) dx
)dy
26
7.3 Uniform Convergence & Differentiation
Note: Suppose that (fn)n≥1 is a sequence of functions on [a, b], and that (fn)→ f uniformly as n→∞.Then, even if every fn is differentiable on [a, b], f is not necessarily differentiable on [a, b]. And even iff is smooth, (f ′n) does not necessarily converge to f ′.
Example: Suppose that fn(x) = 1n cosnx on R. This function is smooth, and f ′n(x) = sinnx. Let
f : x 7→ 0. Then ‖fn − f‖∞ = ‖fn‖∞ = 1n‖cosn·‖∞ = 1
n → 0 as n → ∞, so (fn) → f uniformly asn→∞. All fns are smooth, and f is smooth, since f ′ ≡ 0. But (f ′n) 6→ 0 as n→∞, in fact, (f ′n) doesnot converge at all, except for at certain points such as x = kπ for k ∈ N.
Example: Let fn : R → R, with fn : x 7→√x2 + 1
n . Then (fn(x)) → |x| pointwise as n → ∞. Let
f(x) = |x|. Then∣∣fn(x)2 − f(x)2
∣∣ =∣∣x2 + 1
n − x2∣∣ = 1
n .∣∣fn(x)− f(x)
∣∣ =|fn(x)2−f(x)2||fn(x)+f(x)| ≤
1n
1√1n
= 1√n→
0 as n → ∞, so (fn) → f uniformly as n → ∞. Now, all fns are smooth, and (fn) → f uniformly asn→∞, but f is not differentiable at 0.
Note: If a function f : [a, b] → R is continuously differentiable, it is called a C1 function, or, alter-natively, f ∈ C1[a, b] is written, where C1[a, b] is the space of continuously differentiable functions on[a, b].
Theorem 29: Let (fn) be a sequence of C1 functions on [a, b], with (fn)→ f pointwise as n→∞ on[a, b], and suppose that (f ′n) converges uniformly. Then f ∈ C1[a, b] and limn→∞(f ′n) = f ′.
Proof: Let g = limn→∞(f ′n). Now, ∀n ∈ N, f ′n ∈ C[a, b], so g ∈ C[a, b] since (f ′n)→ g uniformly, so∫ x
a
g =
∫ x
a
limn→∞
(f ′n) = limn→∞
∫ x
a
f ′nFTC= lim
n→∞[fn(x)− fn(a)] = f(x)− f(a)
so, since
f(x) = f(a) +
∫ x
a
g∫ xag is continuous, so by the FTC, f ′ = g = limn→∞(fn), so(
limn→∞
(fn)
)′= f ′
FTC= lim
n→∞(f ′n) �
27
8 Functional Series
8.1 The Weierstrass M-Test & Other Useful Results
Note: Let (fn)n≥1 be a sequence of functions on A ⊂ R. Then thei functional series∑∞k=1 fk converges
pointwise (or uniformly) on A if the sequence of partial sums Sn = (∑nk=1 fk)n≥1 converges pointwise
(or uniformly).
Theorem 24′: Let (fn)n≥1 ⊂ R[a, b]. Suppose that Sn =∑nk=1 fk converges uniformly. Then∑∞
k=1 fk ∈ R[a, b] and ∫ b
a
∞∑k=1
fk =
∞∑k=1
∫ b
a
fk
Proof: If (fk) ∈ R[a, b], then Sn =∑nk=1 fk ∈ R[a, b] ∀n ∈ N, so
∫ ba
∑∞k=1 fk =
∑∞k=1
∫ bafk. �
Theorem 25′: Suppose that (fn)n≥1 ⊂ C[a, b] such that Sn =∑nk=1 fk converges uniformly. Then∑∞
k=1 fk ∈ C[a, b], and, ∀x ∈ [a, b],
limx→x0
∞∑k=1
fk(x) =
∞∑k=1
limx→x0
fk(x)
Proof: Sn ∈ C[a, b] ∀n ∈ N, so limx→x0
∑∞k=1 fk(x) =
∑∞k=1 limx→x0 fk(x). �
Theorem 29′: If (fn)n≥1 ⊂ C1[a, b] and S′n =∑nk=1 f
′k converges uniformly on [a, b], then
d
dx
∞∑k=1
fk(x) =
∞∑k=1
d
dxfk(x)
Proof: The proof is left as an exercise. �
Theorem 30 - Weierstrass M-Test: Let (fn)n≥1 be a sequence of functions on A ⊂ R. Then, if∃(Mk)k≥1 ⊂ R such that, ∀x ∈ A,
∣∣fn(x)∣∣ < Mn and
∑∞k=1Mk converges, then
∑∞n=1 fn converges
uniformly.
Proof:∑∞k=1Mk converges, so (
∑nk=1Mk)n≥1 is Cauchy, so, ∀εn > 0 ∃Nεn such that ∀n,m > Nεn ,
with n > m,∣∣∑n
k=1Mk −∑mk=1Mk
∣∣ < εn. Then∣∣∣∑n
k=m+1Mk
∣∣∣ < εn, so, since MK ≥ 0,∑nk=m+1Mk <
εn. Therefore ∣∣∣∣∣∣n∑k=1
fk(x)−m∑k=1
fk(x)
∣∣∣∣∣∣ =
∣∣∣∣∣∣n∑
k=m+1
fk(x)
∣∣∣∣∣∣ ≤n∑
k=m+1
∣∣fk(x)∣∣ ≤ n∑
k=m+1
Mk < εn
This holds ∀x ∈ A, so the sequence of partial sums of fk is uniformly Cauchy, so∑∞k=1 fk converges
uniformly to its pointwise limit. �
Corollary 30.1: If (fn)n≥1 is such that∑∞k=1‖fk‖∞ converges, then
∑∞k=1 fk converges uniformly.
28
8.2 Taylor & Fourier Series
Motivation: Taylor polynomials are of the form
Sn =
n∑k=0
[1
k!f (k)(a)(x− a)k
]and were studied in Analysis II.
Fourier series are of the form
1
2a0 +
n∑k=1
(ak cos kx+ bk sin kx), x ∈ [0, 2π]
Applications of functional series include solving PDEs, ODEs and time series.
Definition: For any f ∈ R[−π, π], the series
a02
+
∞∑k=1
(ak cos kx+ bk sin kx)
with coefficients given by
ak =1
π
∫ π
−πf(x) cos kx dx, k ≥ 0
bk =1
π
∫ π
−πf(x) sin kx dx, k > 0
is called the Fourier series generated by f .
Note: It is not immediately obvious for which classes of functions this series is convergent, or if itconverges to f .
Theorem 31: Let (ak)k≥0, (bk)k≥1 ⊂ R, with∑∞k=0|ak| and
∑∞k=1|bk| convergent. Then
• a02 +
∑∞k=1(ak cos kx+ bk sin kx) converges uniformly on R.
• The pointwise limit f : R→ R is continuous.
• f is 2π-periodic (f(x+ 2π) = f(x)).
Moreover,
ak =1
π
∫ π
−πf(x) cos kx dx, k ≥ 0
bk =1
π
∫ π
−πf(x) sin kx dx, k > 0
so the terms of the series can be recovered.
Proof (Convergence): ∀x ∈ R, |ak cos kx+ bk sin kx| ≤ |ak cos kx| + |bk sin kx| ≤ |ak| + |bk| ..= Mk.Then
∑∞k=1Mk =
∑∞k=1|ak|+|bk| =
∑∞k=1|ak|+
∑∞k=1|bk|, so
∑∞k=1Mk converges, so the Fourier series
converges by the M-test.
Proof (Continuity): Let f(x) ..= a02 +
∑∞k=1(ak cos kx+ bk sin kx). Then f ∈ C(R).
Proof (Periodicity): Let SN (y) ..= a02 +
∑nk=1(ak cos kx + bk sin kx). Then f(x + 2π) − f(x) =
limn→∞ Sn(x+2π)−limn→∞ Sn(x). Both of these limits exist, so f(x+2π)−f(x) = limn→∞ Sn(x + 2π)−Sn(x) = 0, since cos (k(x+ 2π)) = cos kx and sin (k(x+ 2π)) = sin kx.
29
Lemma: ∀m ∈ Z ∫ π
−πeimxdx = 2π1(m = 0)
where 1(m = 0) is the indicator function for m = 0.
Proof: For m ∈ Z \ {0}:∫ π
−πeimx dx =
∫ π
−πcosmx+ i sinmx dx =
∫ π
−πcosmx dx+ i
∫ π
−πsinmx dx =
2 sinπm
m= 0
For m = 0: ∫ π
−πeimx dx =
∫ π
−πe0 dx =
∫ π
−π1 dx = 2π
So ∫ π
−πeimx dx = 2π1(m = 0) �
Proof (Recovery):∫ π
−πf(x) dx =
∫ π
−π
a02
+
∞∑k=1
(ak cos kx+ bk sin kx) dx
=a02
∫ π
−π1 dx+
∞∑k=1
[ak
∫ π
−πcos kx dx+ bk
∫ π
−πsin kx dx
]= πa0
so
a0 =1
π
∫ π
−πf(x) dx =
1
π
∫ π
−πf(x) cos 0x dx
Now,∫ π
−πf(x) cosmx dx =
1
2a0
∫ π
−πcosmx dx+
∞∑k=1
[ak
∫ π
−πcos kx cosmx dx
]+
∞∑k=1
[bk
∫ π
−πsin kx cosmx dx
]and ∫ π
−πsin kx cosmx dx =
∫ π
−π
1
2i(eikx − e−ikx)
1
2(eimx − e−imx) dx
=1
4i
∫ π
−πei(k−m)x − ei(m−k)x + ei(k+m)x − ei(m+k)x dx
=1
4i(2π1(k −m = 0)− 2π1(m− k = 0)) = 0
and ∫ π
−πcos kx cosmx dx =
∫ π
−π
1
2(eikx + e−ikx)
1
2(eimx − e−imx) dx
=1
4
∫ π
−πei(k−m)x + ei(m−k)x + ei(k+m)x + ei(m+k)x dx
=1
42π((1(k −m = 0) + 1(m− k = 0)) = π1(k = m)
then ∫ π
−πf(x) cosmx dx =
∞∑k=1
(akπ1(k = m)) = πam
so
am =1
π
∫ π
−πf(x) cosmx dx
The proof for bk is similar and so is left as an exercise. �
30
Note: This proof relies on an application of Theorem 24, where the sum is multiplied by a boundedfunction. The proof that the result holds is left as an exercise.
Corollary 31.1: If f : [−π, π]→ R is a C2 function, ( or f ′ ∈ C[−π, π]), and f(π) = f(−π). Then theFourier series generated by f converges to f uniformly.
Note: It must be established that
(i) The Fourier series converges uniformly.
(ii) The limit is f .
However, (ii) is a fairly clunky argument using existing tools, and so shall be omitted and left as anexercise, whilst (i) shall be proved.
Proof:
bk =1
π
∫ π
−πf(x) sin kx dx =
1
πk
∫ π
−πf(x) cos′ kx dx =
1
πk
([f(x) cos kx
]π−π −
∫ π
−πf ′(x) cos kx dx
)
=1
πk
∫ π
−πf ′(x) cos kx dx =
1
πk2
∫ π
−πf ′(x) sin′ kx dx =
1
πk2
([f ′(x) sin kx
]π−π −
∫ π
−πf ′′(x) sin kx dx
)
= − 1
πk2
∫ π
−πf ′′(x) sin kx dx
So
|bk| =1
πk2
∣∣∣∣∣∫ π
−πf(x) sin kx dx
∣∣∣∣∣ ≤ 1
πk2∥∥f ′′ · sin k·∥∥∞ · 2π =
2
k2∥∥f ′′∥∥∞ ..=
cbk2
and cb is independent of k. A similar argument establishes that |ak| ≤ cak2 for k > 0, where ca is also
independent of k.∑∞k=1
1k2 converges, so the Fourier series converges by the M-test with Mk = ca+cb
k2
for k > 0. �
Example (Temperature Equilibration on a Non-Uniformly Heated Ring): Consider the unitcircle S1. Let any point on it be described by the angle ϕ ∈ [−π, π] it makes with the x-axis, and letthe initial temperature on the ring be given by f(ϕ) ∈ C2[−π, π] such that f(−π) = f(π). Then thetemperature T at time t at the point at ϕ, for t > 0, ϕ ∈ [−π, π], is given by the Fourier heat equation
∂T
∂t(t, ϕ)− ∂2T
∂ϕ2(t, ϕ) = 0
Since the point at π is the same as the point at −π, there are some boundary conditions to consider also:
T (t,−π) = T (t, π)
∂T
∂ϕ(t,−π) =
∂T
∂ϕ(t, π)
The initial condition is given byT (0, ϕ) = f(ϕ)
Throughout this example,
T ..=∂T
∂t, T ′ ..=
∂T
∂ϕ, T ′′ ..=
∂2T
∂ϕ2
31
Now, (cos kϕ)′ = −k sin kϕ, and (cos kϕ)′′ = −k2 cos kϕ, so A(t) cos kϕ seems a suitable ansatz tomake for a solution. Substituting this into the heat equation yields A cos kϕ + k2A cos kϕ = 0, soA(t) + k2A(t) = 0, which is an ODE. The boundary conditions are in fact satisfied by either thissolution, or one using the ansatz B(t) sin kϕ, but neither of these satisfy the initial conditions. Since afinite linear combination of any of these functions will have the same shortcoming, it seems sensible totake the next ansatz as being an infinite linear combination of them, such as
T (t, ϕ) =a0(t)
2+
∞∑k=1
(ak(t) cos kϕ+ bk(t) sin kϕ)
Assume that T , T , T ′ and T ′′ all converge uniformly for t > 0, ϕ ∈ [−π, π]. Then substituting T intothe heat equation and differentiating pointwise yields
a02
+
∞∑k=1
(ak cos kϕ+ ·bk sin kϕ) +
∞∑k=1
(akk2 cos kϕ+ bkk
2 sin kϕ) = 0
Soa02
+
∞∑k=1
[(ak + akk
2) cos kϕ+ (bk + bkk2) sin kϕ
]= 0
which is the Fourier series generated by 0, so
a0 = 0
ak + k2ak = 0
bk + k2bk = 0
for k > 0. This yields infinitely many ODEs, which require infinitely many initial conditions.
Expanding f(ϕ) ∈ C2[−π, π] by a Fourier series yields
f(ϕ) =a(f)0
2+
∞∑k=1
(a(f)k cos kϕ+ b
(f)k sin kϕ
)where n
(f)k denotes the coefficient nk dependent on the function f . This gives the required amount of
initial conditions. Then
a0(t) = a(f)0
ak(t) = a(f)k e−k
2t
bk(t) = b(f)k e−k
2t
So
T (t, ϕ) =a(f)0
2+
∞∑k=1
e−k2t(a(f)k cos kϕ+ b
(f)k sin kϕ
)Whilst this is a solution, its uniqueness must also be established, however that shall not be done here.
Exercise: Check that T converges uniformly for t ≥ 0, and that T , T ′, T ′′ do so also for t > 0.
Note: Over time, the temperature becomes distributed evenly across the whole ring, since
limt→∞
T (t, ϕ) =a(f)0
2=
1
2π
∫ π
−πf(ϕ) dϕ = Average Temperature.
The approach to this equilibrium is exponential.
32
Part II
Norms
9 Normed Vector Spaces
9.1 Definition & Basic Properties of a Norm
Definition 11: Let V be a vector space over R. Then a function ‖·‖ : V → R, with ‖·‖ : v ∈ V 7→‖v‖which satisfies the following:
(i) ∀v ∈ V , ‖v‖ ≥ 0 (positivity). Moreover, ‖v‖ = 0 ⇐⇒ v = 0 ∈ V (the ability to separate points).
(ii) ∀λ ∈ R, ∀v ∈ V , ‖λv‖ = |λ| ·‖v‖ (absolute homogeneity).
(iii) ∀u, v ∈ V , ‖u+ v‖ ≤‖u‖+‖v‖ (triangle inequality).
is called a norm on V . A pair (V,‖·‖) is called a normed vector space, or a normed space.
Remarks:
• (ii) and (iii) use the linear structure of V .
• Positivity follows from (ii) and (iii), since, ∀v ∈ V , 0 = 0 ·‖v‖ abs.=
hom.‖0 · v‖−‖v − v‖ ≤‖v‖+‖−v‖ =
‖v‖+|−1| ·‖v‖ =‖v‖+‖v‖ = 2‖v‖, so 0 ≤‖v‖.
Examples:
1. |·|, the absolute value, is a norm on R. The triangle inequality is satisfied since |x+ y| ≤ |x| +|y|.The proofs of the other properties are left as an exercise.
2. ‖·‖∞, the sup norm, is a norm on B[a, b].
(i) : ∀f ∈ B[a, b], if ‖f‖∞ = 0, then supx∈[a,b]∣∣f(x)
∣∣ = 0, so, ∀x ∈ [a, b], 0 ≤ f(x) ≤ 0, so f ≡ 0on [a, b], so ‖·‖∞ can separate points (‖f − g‖∞ = 0 ⇐⇒ f = g, ‖f − g‖∞ > 0 ⇐⇒ f 6= g).
(ii) : ∀λ ∈ R, ∀f ∈ B[a, b],‖λf‖∞ = supx∈[a,b]∣∣λf(x)
∣∣ = supx∈[a,b]|λ|∣∣f(x)
∣∣ = |λ| supx∈[a,b]∣∣f(x)
∣∣ =λ‖f‖∞, so ‖·‖∞ has absolute homogeneity.
(iii) ∀f, g ∈ B[a, b],‖f + g‖∞ = supx∈[a,b]∣∣f(x) + g(x)
∣∣ = supx∈[a,b](∣∣f(x)
∣∣+∣∣g(x)∣∣) ≤ supx∈[a,b]
∣∣f(x)∣∣+
supx∈[a,b]∣∣g(x)
∣∣ = ‖f‖∞ +‖g‖∞, so ‖f + g‖∞ ≤ ‖f‖∞ +‖g‖∞, so ‖·‖∞ satisfies the triangleequality.
Note: Norms generalise the notion of magnitude of vectors in Rn.
Proposition 34: Let x = (x1, . . . , xn) ∈ Rn (a vector). Then the following functions on Rn are norms:
1.
‖x‖1 =
n∑i=1
|xi| (the taxicab or Manhattan norm)
2.
‖x‖2 =
√√√√ n∑i=1
xi2 (the Euclidean norm or Euclidean distance)
3.‖x‖∞ = max
1≤i≤n|xi|
33
Proof: Triangle inequality:
1.
‖u+ v‖1 =
n∑i=1
|ui + vi| ≤n∑i=1
(|ui|+|vi|) =
n∑i=1
|ui|+n∑i=1
|vi| =‖u‖1 +‖v‖1
2. For any u, v ∈ Rn,
u · v =
n∑i=1
ui · vi =‖u‖n ·‖v‖2 · cos θ ≤‖u‖2 ·‖v‖2
for some θ ∈ [0, 2π]. This is known as the Cauchy-Schwarz inequality for Rn. Then
‖u+ v‖22 = (u+ v) · (u+ v)
=
n∑i=1
(ui + vi) · (ui + vi)
= u · (u+ v) + v · (u+ v)
Cauchy
≤Schwarz
‖u‖2 ·‖u+ v‖2 +‖v‖2 ·‖u+ v‖2
So‖u+ v‖22 ≤‖u‖2 ·‖u+ v‖2 +‖v‖2 ·‖u+ v‖2
The case where u+ v = 0 is trivial to check, otherwise
‖u+ v‖2 ≤‖u‖2 +‖v‖2
3.‖u+ v‖∞ = max
1≤i≤n|ui + vi| ≤ max
1≤i≤n(|ui|+|vi|) ≤ max
1≤i≤n|ui|+ max
1≤i≤n|vi| =‖u‖∞ +‖v‖∞
The verification of the other properties is left as an exercise. �
Remark: All of these norms are instances of the following family of norms:
‖x‖p =
n∑i=1
|xi|p 1
p
, p = 1, 2, . . .
In particular,‖x‖∞ = lim
p→∞‖x‖p
That this family of functions is a family of norms follows from the Minkowski inequality : n∑i=1
|xi + yi|p 1
p
≤
n∑i=1
|xi|p 1
p
+
n∑i=1
|yi|p 1
p
∀x, y ∈ Rn
Note : Rn can be thought of as the space of functions on Nn = {1, 2, . . . , n} by considering x : Nn → R,with x : j 7→ xj .
34
9.2 Equivalence & Banach Spaces
Definition 12: Norms ‖·‖a and ‖·‖b on V are called equivalent (or Lipschitz equivalent) if ∃k1, k2 ≥ 0such that, ∀v ∈ V , k1 ·‖v‖b ≤‖v‖a ≤ k2 ·‖v‖b. This equivalence is denoted by ‖·‖a ∼‖·‖b.
Remark: The equivalence of norms is an equivalence relation on the set of all norms on V . That is,
1. ‖·‖a ∼‖·‖a (choose k1 = k2 = 1).
2. ‖·‖a ∼‖·‖b ⇐⇒ ‖·‖b ∼‖·‖a.
3. If ‖·‖a ∼‖·‖b and ‖·‖b ∼‖·‖c, then ‖·‖a ∼‖·‖c.The proof is left as an exercise.
Lemma 35: ‖·‖1, ‖·‖2 and ‖·‖∞ are all equivalent on Rn.
Proof: Because equivalence is an equivalence relation, it is enough to prove that ‖·‖1 ∼ ‖·‖∞ and‖·‖2 ∼‖·‖∞. Now, ∀x ∈ Rn,
‖x‖∞ = max1≤i≤n
|xi| ≤‖x‖1 =n∑i=1
|xi| ≤n∑i=1
max1≤j≤n
∣∣xj∣∣ = n‖x‖∞
So ‖·‖1 ∼‖·‖∞ with k1 = 1, k2 = n.
There is a similar argument for the second part of the proof, although a much more general statementwill be proved shortly, and so this shall be left as an exercise.
Definition 13: Let (V,‖·‖) be a normed space. Then
(i) A sequence (σn)n≥1 ∈ V converges to σ ∈ V if limn→∞∥∥σn − σ∥∥ = 0.
(ii) A sequence (σn)n≥1 ∈ V is Cauchy if, ∀ε > 0, ∃Nε such that, ∀n,m > Nε,∥∥σn − σm∥∥ < ε.
(iii) If every Cauchy sequence in V converges, then (V,‖·‖) is called Banach (or complete). A completenormed space is also called a Banach space.
Remarks:
• Any Cauchy sequence on (R,|·|) converges, so (R,|·|) is Banach.
• Consider (S[a, b],‖·‖∞). Let (sn)n≥1 ⊂ S[a, b] be a uniform Cauchy sequence, so (sn) is Cauchywith respect to ‖·‖∞. Then sn → r ∈ R[a, b] as n → ∞, so, in general, (sn) doesn’t converge toa “point” in S[a, b] because not all regulated functions are step functions, so (S[a, b],‖·‖∞) is notBanach.
Note: All finite-dimensional normed spaces are complete. A proof for the case when the space is overR shall be given later.
Remarks:
1. If limn→∞ σn exists, then it is unique.
Suppose that σn → a ∈ V and σn → b ∈ V as n → ∞. Then 0 ≤‖a− b‖ =∥∥a− σn + σn − b
∥∥ ≤∥∥a− σn∥∥+∥∥σn − b∥∥→ 0 as n→∞, so ‖a− b‖ = 0, so a = b by separation of points.
2. (R[a, b],‖·‖∞) is complete, because a uniformly Cauchy sequence of regulated functions converges toa regulated function. Also, (C[a, b],‖·‖∞) and (B[a, b],‖·‖∞) are complete by results demonstratedearlier.
35
3. Suppose that‖·‖a ∼‖·‖b. Then, if limn→∞ σn = σ with respect to‖·‖a, then limn→∞ σn = σ withrespect to ‖·‖b. The proof is left as an (important) exercise.
4. If ‖·‖a ∼‖·‖b, then (V,‖·‖a) is Banach iff (V,‖·‖b) is Banach. This is essentially a consequence ofthe previous remark.
5. (V,‖·‖) is Banach iff any series∑∞n=1 σn which converges absolutely converges to an element in
V .∑∞n=1 σn converges absolutely if
∑∞n=1
∥∥σn∥∥ converges. The proof of this is left as an exercise,although the ⇐= direction is difficult.
Theorem 36: All norms on Rn are (Lipschitz) equivalent.
Proof: Let (e1, . . . , en) ⊂ Rn be the standard basis of Rn. Then, ∀x ∈ Rn, x =∑nk=1 xkek. Now,
‖x‖ =
∥∥∥∥∥∥n∑k=1
xkek
∥∥∥∥∥∥ ≤n∑k=1
∥∥xkek∥∥ =
n∑k=1
|xk|∥∥ek∥∥ ≤ max
1≤j≤n
∣∣xj∣∣ n∑k=1
∥∥ek∥∥ =‖x‖∞n∑k=1
∥∥ek∥∥Let k2 =
∑nk=1
∥∥ek∥∥. Then ∃k2 such that, ∀x ∈ Rn, ‖x‖ ≤ k2‖x‖∞.
Now, let
J ..= infx 6=0
‖x‖‖x‖∞
, A ..=
{‖x‖‖x‖∞
: x 6= 0
}, B ..=
{‖x‖ :‖x‖∞ = 1
}Clearly B ⊂ A by definition. If α ∈ A, then ∃u, v ∈ Rn such that
α =‖u‖‖u‖∞
=
∥∥∥∥∥ 1
‖u‖∞· u
∥∥∥∥∥ =‖v‖
Then
‖v‖∞ =
∥∥∥∥∥ u
‖u‖∞
∥∥∥∥∥∞
=1
‖u‖∞·‖u‖∞ = 1
so α ∈ B, so A ⊂ B, so B = A. Then J = inf A = inf B, and clearly J ≥ 0.
Suppose that J = 0. Then ∃x1, x2, x3, . . . ∈ Rn with∥∥xk∥∥ = 1
k (so limk→∞∥∥xk∥∥ = 0) and
∥∥xk∥∥∞ = 1 ∀k.
Now, it must be established that there is a convergent subsequence(xkp
)p≥1⊂ (xk)k≥1 such that(
xkp
)→ x = (x(1), x(2), . . . , x(n)) ∈ Rn as p→∞ with respect to ‖·‖∞. Let xk =
(x(1)k , x
(2)k , . . . , x
(n)k
).
Since∥∥xk∥∥∞ = 1,
∣∣∣x(1)k ∣∣∣ ≤ 1, so(x(1)k
)k≥1
is bounded, so by the Bolzano-Weierstrass theorem ∃(x(1)kp
)p≥1⊂(
x(1)k
)k≥1
which converges to x(1) ∈ R. Applying this same argument to each component of xk
yields(xkp
)p≥1
such that(x(j)kp
)p≥1
converges to x(j) ∈ R for any 1 ≤ j ≤ n, so(xkp
)p≥1→ x
“component-wise” as p → ∞. Since(x(j)kp
)p≥1→ x(j), then, ∀ε > 0, ∃Nε(j) such that, ∀n > Nε(j),∣∣∣x(j)kp − x(j)∣∣∣ < ε. Let Nε = max1≤j≤nNε(j). Then, ∀k > Nε,
∥∥∥xkp − x∥∥∥∞ = max1≤j≤n
∣∣∣x(j)kp − x(j)∣∣∣ < ε,
so limp→∞
∥∥∥xkp − x∥∥∥∞ = 0, so(xkp
)p≥1→ x as p→∞ with respect to ‖·‖∞.
Now, 1 =∥∥∥xkp∥∥∥∞ =
∥∥∥xkp − x+ x∥∥∥∞≤∥∥∥xkp − x∥∥∥∞ +‖x‖∞, so ‖x‖∞ ≥ 1 −
∥∥∥xkp − x∥∥∥∞ > 0 since∥∥xkp − x∥∥∞ → 0 as p→∞, so x 6= 0 by the separation of points property of ‖·‖∞.
Also, 0 ≤‖x‖ =∥∥∥x− xkp + xkp
∥∥∥ ≤∥∥xkp∥∥+∥∥∥x− xkp∥∥∥ ≤∥∥xkp∥∥+k2
∥∥∥x− xkp∥∥∥∞. (xkp) ⊂ (xk), so∥∥∥xkp∥∥∥→
0 as p → ∞, and it was established that∥∥∥x− xkp∥∥∥∞ → 0 as p → ∞, so
∥∥xkp∥∥ + k2
∥∥∥x− xkp∥∥∥∞ → 0 as
p→∞, so ‖x‖ = 0, so x = 0 by the separation of points property of ‖·‖.However, this is a contradiction, so J > 0, so let k1 = J . Then, from the definition of J , k1‖x‖∞ ≤‖x‖∀x ∈ Rn \ {0}, and if x = 0, then ‖x‖∞ = ‖x‖ = 0, so k1‖x‖∞ = ‖x‖, so k1‖x‖∞ ≤ ‖x‖ ∀x ∈ Rn, sok1‖x‖∞ ≤‖x‖ ≤ k2‖x‖∞ ∀x ∈ Rn, so ‖·‖ ∼‖·‖∞ on Rn, so all norms are equivalent on Rn. �
36
Proposition 37: The space (Rn,‖·‖) is Banach for any norm ‖·‖ on Rn.
Proof: It is enough to show that (Rn,‖·‖∞) is complete, since if ‖·‖a ∼‖·‖b, then (V,‖·‖a) is Banach⇐⇒ (V,‖·‖b) is Banach, and ‖·‖∞ ∼‖·‖ for all norms ‖·‖ on Rn as shown previously.
Consider any Cauchy subsequence (xk)k≥1 ⊂ Rn. Then, for any 1 ≤ j ≤ n, 0 ≤∣∣∣x(j)k+m − x(j)k ∣∣∣ ≤
max1≤j≤n
∣∣∣x(j)k+m − x(j)k ∣∣∣ =∥∥∥xk+m − xk∥∥∥
∞→ 0 as k → ∞ since (xk)k≥1 is Cauchy in Rn, so
(x(j)k
)k≥1
is Cauchy in R, so ∃x(j) ∈ R such that x(j)k → x(j) as k → ∞. Let x = (x(1), x(2), . . . , x(n)) ∈ Rn. It
was shown in the proof of the previous theorem that limk→∞∥∥xk − x∥∥∞ = 0, so (xk)k≥1 converges in
Rn with respect to ‖·‖∞, so every Cauchy sequence in Rn converges with respect to ‖·‖∞, so (Rn,‖·‖∞)is complete, so (Rn,‖·‖) is Banach for any norm ‖·‖ on Rn. �
Corollary: (V,‖·‖) is Banach for any norm ‖·‖ on V , provided dimV is finite. This is because anyn-dimensional vector space V over R is isomorphic to Rn, simply choose a basis in V .
Lemma (Cauchy-Schwarz Inequality for Integrals): ∀f, g ∈ C[a, b],
∫ b
a
fg ≤‖f‖2‖g‖2 , where ‖f‖2 =
√∫ b
a
f2
Proof: ∀λ ∈ R,
λ2∫ b
a
g2 + 2λ
∫ b
a
fg +
∫ b
a
f2 =
∫ b
a
(f + λg)2 ≥ 0 (1)
This is a polynomial in λ of degree 2 with at least one real root, so the discriminant
D = 4
(∫ b
a
fg
)2
− 4‖g‖22‖f‖22 ≤ 0, so
∫ b
a
fg ≤‖f‖2‖g‖2 �
Proposition 38:
1. The following functions are norms on C[a, b]:
(a)‖f‖∞ = sup
x∈[a,b]
∣∣f(x)∣∣ (the sup norm)
(b)
‖f‖1 =
∫ b
a
|f | (a generalisation of the taxicab norm)
(c)
‖f‖2 =
√∫ b
a
f2 (a generalisation of the Euclidean norm)
2. (C[a, b],‖·‖∞) is Banach, but (C[a, b],‖·‖1,2) is not Banach.
3. ∀f ∈ C[0, 1], ‖f‖1 ≤‖f‖2 ≤‖f‖∞.
4. No pair of norms (a), (b) or (c) is equivalent.
37
Proof:
1. (a) It was demonstrated that ‖·‖∞ is a norm on B[a, b], and C[a, b] ⊂ B[a, b], so ‖·‖∞ is a normon C[a, b].
(b) Separation of points: Assume that ∃f such that ‖f‖1 = 0. Then∫ ba|f | = 0. Assume that
f 6≡ 0. Then ∃x0 ∈ [a, b] such that∣∣f(x0)
∣∣ > 0. But f ∈ C[a, b], so ∃[t1, t2] ⊂ [a, b] such that∣∣∣∣f ∣∣∣[t1,t2]
∣∣∣∣ > 0. Now, 0 =‖f‖1 =∫ ba|f | ≥
∫ t2t1|f | > 0, so 0 > 0, which is clearly a contradiction,
so f ≡ 0.
Absolute homogeneity: ∀λ ∈ R, ‖λ · f‖1 =∫ ba|λ · f | =
∫ ba
(|λ| ·|f |) = |λ| ·∫ ba|f | = |λ| ·‖f‖1
Triangle inequality: ∀f, g ∈ C[a, b], ‖f + g‖1 =∫ ba|f + g| ≤
∫ ba
(|f | + |g|) =∫ ba|f | +
∫ ba|g| =
‖f‖1 +‖g‖1(c) The proofs of the first two properties are very similar to those in (b), and so they are left as
exercises.
Triangle inequality: Using the Cauchy-Schwarz inequality
‖f + g‖22 =
∫ b
a
(f + g)2 =
∫ b
a
f(f + g) +
∫ b
a
g(f + g) ≤‖f‖2‖f + g‖2 +‖g‖2‖f + g‖2
Then, assuming that (f + g) 6≡ 0,
‖f + g‖2 ≤‖f‖2 +‖g‖2If f + g ≡ 0 then the proof is trivial.
2. Consider the sequence of functions (fk)k≥1 ⊂ C[a, b] given by
fk(x) =
−1 a ≤ x < a+b
2 −1k
k(x− a+b2 ) a+b
2 −1k ≤ x ≤
a+b2 + 1
k
1 a+b2 + 1
k < x ≤ b
Then, for any k,m ∈ N,
‖fk − fk+m‖1 =
∫ b
a
|fk − fk+m| =∫ a+b
2 + 1k
a+b2 −
1k
|fk − fk+m|
since the functions are equal outside of these bounds. Now, fk is bounded by [−1, 1], so|fk − fk+m| ≤2, so
‖fk − fk+m‖1 ≤∫ a+b
2 + 1k
a+b2 −
1k
2 = 2 · 2
k=
4
k→ 0 as k →∞
so (fk)k≥1 is Cauchy with respect to ‖·‖1. Suppose that it has a limit f ∈ C[a, b]. Then, ∀ε > 0,
f(a+b2 − ε
)= −1 and f
(a+b2 + ε
)= 1, since otherwise, for any k > 1
ε ,∫ ba|fk − f | > 0 which
contradicts f being the limit of (fk)k≥1. But then f is not continuous, since
−1 = limx↑ a+b2
f(x) 6= limx↓ a+b2
f(x) = 1
This is a contradiction, so (fk)k≥1 does not have a limit in C[a, b] with respect to ‖·‖1, but it isCauchy with respect to ‖·‖1, so (C[a, b,‖·‖1) is not Banach.
3. For any f ∈ C[0, 1], and some x0 ∈ [0, 1]
‖f‖1 =
∫ 1
0
|f | ≤∫ 1
0
‖f‖∞ =‖f‖∞
‖f‖1 =
∫ 1
0
|f | · 1 ≤‖f‖2 ·‖1‖2 =‖f‖2
‖f‖2 =
(∫ 1
0
f2
) 12
≤(∥∥∥f2∥∥∥
∞
) 12
=
(supx∈[0,1]
f(x)2
) 12
=(f(x0)2
) 12
=∣∣f(x0)
∣∣ =‖f‖∞
38
4. This is mostly left as an exercise, however a proof that ‖·‖1 6∼ ‖·‖2 will be given. For simplicity,this shall be proven with respect to C[0, 1], the generalisation is left as an easy exercise.
Let, for any ε > 0,
fε(x) =
1√ε
0 ≤ x < ε1√x
ε ≤ x ≤ 1∈ C[0, 1]
Then
‖fε‖1 =
∫ 1
0
|fε| =∫ ε
0
1√εdx+
∫ 1
ε
1√xdx =
[√ε+ 2
√x]ε1
= 2−√ε ≤ 2
‖fε‖22 =
∫ 1
0
f2ε =
∫ ε
0
1
εdx+
∫ 1
ε
1
xdx = [1 + log x]
1ε = 1 + log
1
ε→∞ as ε ↓ 0
So @k > 0 such that ‖f‖2 ≤ k‖f‖1 ∀f ∈ C[0, 1] . �
Exercise: Using a proof similar to the one in 3, show that (C[a, b],‖·‖2) is not Banach.
39
10 Continuous Maps
10.1 Definition & Basic Properties of Linear Maps
Definition 14: Let f : (V,‖·‖V ) → (W,‖·‖W ) be a map from V to W . Then f is called (‖·‖V ,‖·‖W )continuous (or simply continuous when the spaces in question are clear) at u ∈ V if, ∀ε > 0, ∃δε > 0such that, ∀v ∈ V , ‖u− v‖V < δε =⇒
∥∥f(u)− f(v)∥∥W< ε.
Example: Let I : (C[a, b],‖·‖∞)→ (R,|·|) be given by
I : f ∈ C[a, b] 7→∫ b
a
f ∈ R
It was demonstrated in the first section that this map is linear.
Theorem 39: Let T : (V,‖·‖V ) → (W,‖·‖W ) be a linear map. Then the following conditions areequivalent:
(i) T is continuous at 0V ∈ V .
(ii) T is continuous on V (T is continuous at every point of V ).
(iii) The set{∥∥T (x)
∥∥W
: x ∈ V such that ‖x‖V ≤ 1}⊂ R is bounded (then T is called bounded).
Proof (i)⇐⇒ (ii): The ⇐= direction is clear, if T is continuous on V , it is continuous at 0V ∈ V .
For the =⇒ direction, suppose that T is continuous at 0V . T (0V ) = 0W , so, ∀ε > 0, ∃δε > 0 such that‖x‖V < δε =⇒
∥∥T (x)∥∥W< ε. Choose some u ∈ V . Then, for any v ∈ V , let x = u − v. Now, ∀ε > 0,
∃δε > 0 such that ‖u− v‖V < δε =⇒∥∥T (u)− T (v)
∥∥W
=∥∥T (u− v)
∥∥W< ε, so T is continuous at u.
Since the choice of u was arbitrary, T is continuous on V .
Proof (i) ⇐⇒ (iii): For the =⇒ direction, by continuity, ∀ε > 0, ∃δε > 0 such that, ∀x ∈ V ,
‖x‖V ≤ δε = δε2 =⇒
∥∥T (x)∥∥W< ε. Then
∥∥∥ xδε
∥∥∥V≤ 1 =⇒
∥∥∥∥T ( x
δε
)∥∥∥∥W
≤ εδε
, so ∀x ∈ V , ‖x‖V ≤ 1 =⇒∥∥T (x)∥∥W< ε
δε, so
{∥∥T (x)∥∥W
: x ∈ V such that ‖x‖V ≤ 1}
is bounded by εδε
, so it is bounded.
For the ⇐= direction, T is bounded, so ∃B > 0 such that ∀u, ‖u‖V ≤ 1 =⇒∥∥T (u)
∥∥W< B. Then,
for any ε > 0, multiplying this inequality by εB yields
∥∥ εuB
∥∥V≤ ε
B =⇒∥∥∥T ( εuB )∥∥∥
W< ε, so ∀v ∈ V ,
‖v‖V < δε = εB =⇒
∥∥T (v)∥∥W< ε, so T is continuous at 0V with δε = ε
B . �
Example: For any f ∈ C[a, b], ‖f‖∞ ≤ 1,∫ b
a
≤‖f‖∞ ·|b− a| ≤|b− a|
so I : C[a, b]→ R,
I : f ∈ C[a, b] 7→∫ b
a
f ∈ R
is bounded, so I is continuous on (C[a, b],‖·‖∞).
40
Proposition 40: Let V and W be vector spaces, with norms ‖·‖V1∼ ‖·‖V2
and ‖·‖W1∼ ‖·‖W2
respectively, and let T : V →W be a linear map. Then
(i) T is (‖·‖V1,‖·‖W1
) continuous iff T is (‖·‖V2,‖·‖W2
) continuous.
(ii) Any linear map (Rn,‖·‖1)→ (W,‖·‖W ) is continuous.
(iii) Any linear map (Rn,‖·‖)→ (W,‖·‖W ) is continuous, where ‖·‖ is an arbitrary norm.
(iv) Let F : (C[0, 1],‖·‖1,∞) → (R,|·|) be an evaluation map with F : f 7→ f(0) ∈ R. Then F is(‖·‖∞ ,|·|) continuous but not (‖·‖1 ,|·|) continuous.
Proof:
(i) T is (‖·‖V1,‖·‖W1
) continuous, so T is bounded with respect to (‖·‖V1,‖·‖W1
), so ∃B ≥ 0 such that
∀u ∈ V , ‖u‖V1≤ 1 =⇒
∥∥T (u)∥∥W1≤ B. Since ‖·‖V1
∼‖·‖V2, ∃K > 0 such that, ∀u ∈ V , ‖u‖V1
≤K‖u‖V2
. Consider all u ∈ V such that K‖u‖V2≤ 1. Then ‖u‖V1
≤ 1, so∥∥T (u)
∥∥W1≤ B. Now,
‖·‖W1∼‖·‖W2
, so ∃L > 0 such that, ∀v ∈ W , L‖v‖W2≤‖v‖W1
, so L∥∥T (u)
∥∥W2≤∥∥T (u)
∥∥W1≤ B.
Then, K‖u‖V2≤ 1 =⇒ L
∥∥T (u)∥∥W2≤ B. Let w = Ku. Then, ∀w ∈ V , ‖w‖V2
≤ 1 =⇒∥∥T (w)∥∥W2≤ KB
L = B′, so T is bounded with respect to (‖·‖V2,‖·‖W2
), so T is (‖·‖V2,‖·‖W2
)continuous.
(ii) For any x ∈ Rn, ∃x1, . . . , xn ∈ R such that, for some basis e1, . . . , en ∈ Rn,
x =
n∑i=1
xiei, so ‖x‖1 =
n∑i=1
|xi|
Now, consider any x ∈ Rn with ‖x‖1 ≤ 1. Then
∥∥T (x)∥∥W
=
∥∥∥∥∥∥T n∑i=1
xiei
∥∥∥∥∥∥W
=
∥∥∥∥∥∥n∑i=1
xiT (ei)
∥∥∥∥∥∥W
≤n∑i=1
|xi|∥∥T (ei)
∥∥W≤ max
1≤i≤n
∣∣T (ei)∣∣ n∑i=1
|xi| ≤ max1≤i≤n
∣∣T (ei)∣∣ ..= B
So ∃B > 0 such that, ∀x ∈ Rn with ‖x‖1 ≤ 1,∥∥T (x)
∥∥W≤ B, so T is bounded, so T is continuous.
(iii) This is a direct consequence of (i) and (ii), since all norms on Rn are equivalent.
(iv) For F : (C[0, 1],‖·‖∞) → (R,|·|), take any f ∈ C[0, 1] such that ‖f‖∞ ≤ 1. Then∣∣f(0)
∣∣ ≤ 1, so∣∣F (f)∣∣ ≤ 1, so F is bounded by 1, so F is continuous.
For F : (C[0, 1],‖·‖1)→ (R,|·|), let (fn)n≥1 ⊂ C[a, b] be the sequence given by fn : [0, 1]→ R, fn :x 7→ ( 1
n − n)x+ n. Then
‖fn‖1 =
∫ 1
0
fn =
∫ 1
0
(1
n− n
)x+ n dx =
1
n≤ 1 ∀n ∈ N
However,∣∣F (fn)
∣∣ =∣∣fn(0)
∣∣ = n → ∞ as n → ∞, so{∣∣F (f)
∣∣ :‖f‖1 ≤ 1}
is not bounded, so F is
not continuous. �
41
10.2 Spaces of Bounded Linear Maps
Theorem 41: Suppose (V,‖·‖V ) and (W,‖·‖W ) are normed spaces, let
L(V,W ) = {T : V →W : T is linear and continuous/bounded}
and let‖·‖ : T ∈ L(V,W ) 7→ sup
x∈V :‖x‖V ≤1
∥∥T (x)∥∥W
Then (L(V,W ),‖·‖) is a normed vector space.
Proof: For any λ, µ ∈ R, T,G ∈ L(V,W ), x ∈ V , (λT + µG)(x) = λT (x) + µG(x) - a map from Vto W . Take any α, β ∈ R, u, v ∈ V . Then (λT + µG)(αu + βv) = λT (αu + βv) + µG(αu + βv) =λ(αT (u) + βT (v)) + µ(αG(u) + βG(v)) = α(λT + µG)(u) + β(λT + µG)(v), so (λT + µG) is linear.Now, ‖λT + µG‖ = sup‖x‖V ≤1
∥∥λT (x) + µG(x)∥∥W≤ |λ| sup‖x‖V ≤1
∥∥T (x)∥∥W
+ |µ| sup‖x‖V ≤1∥∥G(x)
∥∥W
=
|λ|‖T‖+|µ|‖G‖. Then, since T and G are both bounded, λT +µG is bounded, so λT +µG is continuous,so L(V,W ) is closed under linear combinations of maps. Verification of the remaining axioms of a vectorspace is left as an exercise.
It remains to show that ‖·‖ is a norm. Taking λ = µ = 1 in the previous section of the proof yields‖T +G‖ ≤‖T‖+‖G‖ which proves the triangle inequality. The proof of absolute homogeneity is left as aneasy exercise. To prove separation of points, suppose that ‖T‖ = 0L(V,W ). Then sup‖u‖V ≤1
∥∥T (u)∥∥W
=
0L(V,W ), so∥∥T (u)
∥∥W
= 0L(V,W ) ∀u ∈ V such that‖u‖V ≤ 1. Since‖·‖W is a norm, T (x) = 0L(V,W ) ∀u ∈
V such that ‖u‖V ≤ 1. Now, take any v ∈ V such that ‖v‖V > 1. Then∥∥T (v)
∥∥W
=‖v‖V ·∥∥∥∥T ( v
‖v‖V
)∥∥∥∥W
= 0L(V,W ), so T (x) = 0L(V,W ) ∀x ∈ V , so T = 0L(V,W ), which completes the proof. �
Remarks:
1. The norm ‖·‖ is called an operator norm.
2. Consider L(Rm,Rn) - this is the space Rm×n of all m× n matrices.
3. Suppose then that m = n and that both of these Rns are equipped with the ‖·‖2 norm. Then forany T ∈ Rn×n, ‖T‖ =
√λmax, where λmax is the maximum eigenvalue of TT ′, with T ′ being the
transpose of T .
4. If (W,‖·‖W ) is Banach, then (L(V,W ),‖·‖) is Banach also.
Exercise: Prove 4 and that ‖T‖ =√λmax as given in 3 is a norm.
42
11 Open and Closed Sets of Normed Spaces
11.1 Definition & Basic Properties of Open Sets
Definition 15: Let (V,‖·‖) be a normed space, and let u ∈ V , δ > 0. Then the set
B(u, δ) = {v ∈ V :‖v − u‖ < δ} ⊂ (V,‖·‖)is called an open ball of radius δ centred at u. B(u, δ,‖·‖) denotes an open ball defined with respect to‖·‖, B(u, δ) is shorthand when the norm in question is obvious.
Examples:
1. For (R,|·|), B(x, δ) = (x− δ, x+ δ).
2. For (R2,‖·‖1), B1(0, 1) = {(x1, x2 ∈ R2 :∣∣x1∣∣+
∣∣x2∣∣ < 1)}.For (R2,‖·‖2), B2(0, 1) = {(x1, x2 ∈ R2 :
√x12 + x22 < 1)}.
For (R2,‖·‖∞), B∞(0, 1) = {(x1, x2 ∈ R2 : max(x1, x2) < 1)}.
3. For (C[a, b],‖·‖∞), B(g, ε) = {f ∈ C[a, b] : supx∈[a,b]∣∣f(x)− g(x)
∣∣ < ε}
Definition 15.1: Let (V,‖·‖) be a normed space. A subset U ⊂ V is called open if, ∀x ∈ V , ∃δx suchthat B(x, δx) ⊂ U .
Exercise: Prove that the empty set is open.
Lemma 42: Let (V,‖·‖) be a normed space. Then
(i) Open balls in V are open sets.
(ii) ∀u ∈ V, δ > 0, B(u, δ) = u+ δB(0, 1) ..= {v ∈ V : v = u+ δw,w ∈ B(0, 1)}.
Proof:
(i) v ∈ B(u, δ) ⇐⇒ ‖v − u‖ < δ. Let δ′ = δ −‖v − u‖ > 0. Consider B(v, δ′). Then, for anyw ∈ B(v, δ′), ‖w − v‖ < δ′. Then ‖u− w‖ =‖u− v + v − w‖ ≤‖u− v‖+‖v − w‖ <‖u− v‖+ δ′ =‖u− v‖+ δ −‖v − u‖ = δ, so B(v, δ′) ⊂ B(u, δ). Then, since the choice of v was arbitrary, B(u, δ)is open.
(ii) Take v ∈ B(u, δ), then ‖u− v‖ < δ, so∥∥∥u−vδ ∥∥∥ < 1. Let w = u−v
δ . Then w ∈ B(0, 1), so v = u+ δw
with w ∈ B(0, 1), so v ∈ u+ δB(0, 1), so B(u, δ) ⊆ u+ δB(0, 1).
Now, take any v ∈ u + δB(0, 1). Then ∃w ∈ B(0, 1) such that v = u + δw, so ‖u− v‖ = ‖δw‖ =δ‖w‖ < δ, so v ∈ B(u, δ). Then u+ δB(0, 1) ⊆ B(u, δ), so B(u, δ) = u+ δB(0, 1). �
Lemma 43: Let ‖·‖A ∼‖·‖B be equivalent norms on V . Then U ⊂ V is open with respect to ‖·‖A iffU ⊂ V is open with respect to ‖·‖B .
Proof: Assume without loss of generality that V is open with respect to ‖·‖A. Then, ∀u ∈ U , ∃δu > 0such that B(u, δu,‖·‖A) ⊂ U . Now, consider some δ′u > 0. Then v ∈ B(u, δ′u,‖·‖B) ⇐⇒ ‖v − u‖B < δ′u.
Since‖·‖A ∼‖·‖B , ∃K > 0 such that‖v − u‖A < K‖v − u‖B < Kδ′u. Let δ′u =δuK . Then B(u, δ′u,‖·‖B) ⊂
B(u, δu,‖·‖A), so V is open with respect to ‖·‖B . �
Example: If v ∈ Rn is open with respect to ‖·‖1, then V is open with respect to an arbitrary norm‖·‖ on Rn since all norms on Rn are equivalent.
43
11.2 Continuity, Closed Sets & Convergence
Definition 15.2: A function f : (V,‖·‖V )→ (W,‖·‖W ) is continuous at x ∈ V if, ∀ε > 0, ∃δ > 0 suchthat f(B(x, δ,‖·‖V )) ⊂ B(f(x), ε,‖·‖W ).
Exercise: Prove that this definition is equivalent to the previous definition of continuity.
Definition 15.3: A set U ⊂ (V,‖·‖) is bounded if it is contained in a ball of finite radius. Equivalently,U ⊂ V is bounded if ∃x ∈ V, δ > 0 such that U ⊂ B(x, δ).
Proposition 44: Let f : (V,‖·‖V ) → (W,‖·‖W ). Then f is continuous on V iff the preimage of anyopen subset U ⊂ W is open. Equivalently, f : V → W is continuous iff, ∀U ⊂ W such that U is open,f−1(U) ..= {x ∈ V : f(x) ∈ U} is open.
Proof =⇒ : Let U ⊂W be an open subset. Then, either f−1(U) = ∅ which is open, or f−1(U) 6= ∅.If f−1(U) 6= ∅, then let u ∈ f−1(U) ⊂ V be any point in the preimage. Let v = f(u) ∈ U . Then ∃ε > 0such that B(v, ε,‖·‖W ) ⊂ U . But f is continuous on V , so f is continuous at u, so ∃δε > 0 such thatf(B(u, δε,‖·‖V )) ⊂ B(v, ε,‖·‖W ) ⊂ U , so B(u, δε,‖·‖V ) ⊂ f−1(U), so f−1(U) is open.
Proof ⇐= : Take any u ∈ V , and let v = f(u). For any ε > 0, considerB(v, ε,‖·‖W ). f−1(B(v, ε,‖·‖W ))is open, and u ∈ f−1(B(v, ε,‖·‖W )). Since f−1(B(v, ε,‖·‖W )) is open, ∃δε such that B(u, δε,‖·‖V ) ⊂f−1(B(v, ε,‖·‖W )), so f(B(u, δε,‖·‖V )) ⊂ (B(v, ε,‖·‖W ), so f is continuous on V . �
Definition 16: A set U ⊂ (V,‖·‖) is called closed if V \ U is open.
Examples:
(i) [a, b] ⊂ R is closed since R \ [a, b] = (−∞, a) ∪ (b,∞) is open.
(ii) [a, b) ⊂ R is neither open nor closed (it is not open since B(a, ε) 6⊂ [a, b) for any ε > 0, and it isnot closed since R \ [a, b) = (−∞, a) ∪ [b,∞) is not open).
Theorem 45: Let (V,‖·‖) be a normed space. Then U ⊂ V is closed iff, for any sequence (xn) ⊂ Uwhich converges to x ∈ V , x ∈ U .
Proof =⇒ : Let U ⊂ V be a closed subset. Assume that U 6= ∅, the case when U = ∅ is trivial.Then let (xn) ⊂ U be a convergent sequence, with x = limn→∞ xn, so limn→∞
∥∥x− xn∥∥ = 0. Supposethat x /∈ U , then x ∈ V \ U . Since U is closed, V \ U is open, so ∃δ > 0 such that B(x, δ) ⊂ V \ U .However, since xn → x as n → ∞, ∃Nδ such that, ∀n > Nδ,
∥∥x− xn∥∥ < δ, so xn ∈ B(x, δ) ⊂ V \ U ,which is a contradiction since xn ∈ U , so x ∈ U .
Proof ⇐= : Now, any sequence in U which converges does so to a point in U . Suppose that U is notclosed. Then V \U is not open, so ∃x ∈ V \U such that, ∀ε > 0, B(x, ε) 6⊂ V \U , so B(x, ε)∩U 6= ∅. Let(εn) be any null sequence. Then, for any n ∈ N, B(x, εn)∩U 6= ∅, so ∃xn ∈ U such that
∥∥x− xn∥∥ < εn,
so limn→∞∥∥x− xn∥∥ = 0. Then (xn) ⊂ U is a sequence which converges to a point x /∈ U , which is a
contradiction, so U is closed. �
Notes: Consider the normed space (V,‖·‖). For any x ∈ V and any δ > 0, B(x, δ) ⊂ V , so V is open.However, any sequence in V which converges does so in V , so V is closed also. ∅ is also both closed andopen, since ∅ = V \ V . These sets are called clopen.
44
11.3 Lipschitz Continuity, Contraction Mappings & Other Remarks
Remarks: Let (V,‖·‖) be a normed space. Then
1. The map ‖·‖ : (V,‖·‖) → (R,|·|) is continuous. To see this, fix any u ∈ V . Then, for any v ∈ V ,∥∥‖u‖ −‖v‖∥∥ ≤ ‖u− v‖ by the triangle inequality. Now, if ‖u− v‖ < ε, then∥∥‖u‖ −‖v‖∥∥ < ε, so∥∥B(u, ε,‖·‖)∥∥ ⊂ B (‖u‖ , ε,|·|), so setting δε = ε demonstrates that ‖·‖ : V → R is continuous.
2. Suppose that, for a map f : (V,‖·‖)→ (V,‖·‖), ∃K ∈ (0, 1) such that, ∀u, v ∈ V ,∥∥f(u)− f(v)
∥∥ ≤K‖u− v‖. Then f is continuous on V . To see this, note that f(B(v, ε
2K )) ⊂ B(f(v), ε), the detailsof the proof are left as an exercise. Such a map is called a contraction mapping. In fact, if thiscondition holds for any k > 0, continuity is preserved, and the map is called Lipschitz continuous.
3. A closed ball of radius δ centred at u is defined as B(u, δ) ..= {v ∈ V :‖u− v‖ ≤ δ} ⊂ (V,‖·‖). Thisset is closed. To see this, let (un)n≥1 ⊂ B(u, δ) such that un → w ∈ V as n → ∞. Then, by the
continuity of ‖·‖ at u ∈ V , ‖w − u‖ = limn→∞∥∥un − u∥∥ ≤ δ, so w ∈ B(u, δ), so B(u, δ) is closed.
4. Let U ⊂ (V,‖·‖). Then w ∈ V is a boundary point of U if, ∀ε > 0, B(w, ε) ∩ U 6= ∅ andB(w, ε) ∩ V \ U 6= ∅. w ∈ V is a limit point of U if ∃(xn)n≥1 ⊂ U which converges to w.
45
12 The Contraction Mapping Theorem & Applications
12.1 Statement and Proof
Theorem 46 - The Contraction Mapping/Banach Fixed Point Theorem: Let (V,‖·‖) be aBanach space, U ⊂ V a closed subset of this space, and f : U → U a contraction mapping. Then∃!w ∈ U such that f(w) = w. Moreover, for any u ∈ U , the sequence (un)n≥0 ⊂ U converges to w asn→∞, where u0 = u and un+1 = f(un).
Proof: f is a contraction mapping, so ∃K ∈ (0, 1) such that, ∀u, v ∈ U ,∥∥f(u)− f(v)
∥∥ ≤ K‖u− v‖.Then
∥∥∥un+1 − un∥∥∥ =
∥∥∥f(un)− f(un−1)∥∥∥ ≤ K
∥∥∥un − un−1∥∥∥ ≤ K2∥∥∥un−1 − un−2∥∥∥ ≤ . . . ≤ Kn
∥∥u1 − u0∥∥.
Now, ∥∥∥un+m − un∥∥∥ =∥∥∥un+m − un+m−1 + un+m−1 − un+m−2 + un+m−2 − . . .+ un+1 − un
∥∥∥≤∥∥∥un+m − un+m−1∥∥∥+ . . .+
∥∥∥un+1 − un∥∥∥
≤ (Kn+m−1 +Kn+m−2 + . . .+Kn)∥∥u1 − u0∥∥
≤∞∑j=0
Kn+j∥∥u1 − u0∥∥
=Kn
1−K∥∥u1 − u0∥∥→ 0 as n→∞,
so (un)n≥0 is Cauchy. Since (V,‖·‖) is Banach and U is closed, (un) → w ∈ U as n → ∞. Now,consider un+1 = f(un). Then limn→∞ un+1 = limn→∞ f(un). limn→∞ un+1 = w, and limn→∞ f(un) =f(limn→∞ un) = f(w) since f is a contraction mapping and therefore Lipschitz continuous. Thenw = f(w), so w is a fixed point of f .
It remains to establish the uniqueness of w. Suppose that ∃v ∈ U such that v = f(v). Then ‖w − v‖ =∥∥f(w)− f(v)∥∥ ≤ K‖w − v‖, so (1−K)‖w − v‖ ≤ 0. Now, (1−K) > 0 and‖w − v‖ ≥ 0, so‖w − v‖ = 0,
so w = v since ‖·‖ separates points. �
Example: Let α ∈ (0, 1). Then, using the Contraction Mapping Theorem, it can be shown that theequation x = e−αx, x ∈ [0, 1], has a unique solution x∗ ∈ (0, 1).
Now, (R,|·|) is Banach, and U = [0, 1] is closed. It remains to show that f : [0, 1]→ [0, 1], f : x ∈ [0, 1] 7→e−αx, is a contraction mapping. ∀x, y ∈ [0, 1],
∣∣f(x)− f(y)∣∣ =
∣∣f ′(ξ)(x− y)∣∣ =
∣∣−αe−αξ(x− y)∣∣ ≤
α|x− y| for some ξ ∈ (x, y) by the Mean Value Theorem, which can be applied since f is continuouslydifferentiable. Then, since α ∈ (0, 1), f is a contraction mapping, so the Contraction Mapping Theoremcan be applied, so ∃!x∗ ∈ (0, 1) such that x∗ = f(x∗). Then x∗ solves the equation x = e−αx.
However, the Contraction Mapping Theorem not only guarantees the existence of x∗, it provides amethod for constructing it, or at least an approximation to it. Let α = 1
2 and take x0 = 12 . Then
x0 =1
2
x1 = e−14 ≈ 0.78
x2 ≈ 0.68
x3 ≈ 0.71
x4 ≈ 0.70
x5 ≈ 0.70
It happens that x∞ = x∗ = W ( 12 ), where W is Lambert’s W -function, with W ( 1
2 ) ≈ 0.70.
46
12.2 The Existence and Uniqueness of Solutions to ODEs
Definition: An ordinary differential equation (ODE) is of the form[dy
dt(t) =
]y(t) = F (t, y(t)), t ∈ A ⊂ R
y(t0) = y0, t0 ∈ A
with y(t) the unknown to be solved for subject to the conditions stated. The second equality is knownas the intial condition(s).
Remark: Applying∫ tt0
to the first equality in the ODE yields∫ t
t0
y = y(t)− y(t0) =
∫ t
t0
F (τ, y(τ)) dτ
so
y(t) = y(t0) +
∫ t
t0
F (τ, y(τ)) dτ
Suppose that F : R2 → R is continuous, and that y ∈ C[t0 − δ, t0 + δ] for some δ > 0. Then y(t0) +∫ tt0F (τ, y(τ)) dτ is well-defined and differentiable, and so y(t) solves the ODE.
Definition: The operator
T (f) ..= y0 +
∫ t
t0
F (τ, f(τ)) dτ
acting from C[t0 − δ, t0 + δ] to C[t0 − δ, t0 + δ] is called the Pickard operator. An operator is simply amapping from one vector space to another, and in many contexts they are linear, however the Pickardoperator is not.
Remark: A fixed point of T occurs when f = T (f), so
f(t) = y0 +
∫ t
t0
F (τ, f(τ)) dτ
so f solves the ODE.
Theorem 47 - The Picard-Lindelof Theorem: Suppose that F ∈ C[t0− a, t0 + a]× [y0− ε, y0 + ε]for some a, ε > 0, and that F is Lipschitz continuous on D = [t0− a, t0 + a]× [y0− ε, y0 + ε] with respectto y, that is, ∃L > 0 such that, ∀(t, y), (t, y′) ∈ D,
∣∣F (t, y)− F (t, y′)∣∣ ≤ L
∣∣y − y′∣∣. Then ∃δ ∈ (0, a] suchthat the ODE as given in the definition has a unique solution in C[t0 − δ, t0 + δ].
Remark: Suppose that∂F
∂y∈ C(D)
Then, ∀(t, y)(t, y′) ∈ D and for some ξ(t, y, y′) ∈ D,∣∣F (t, y)− F (t, y′)∣∣ MVT
=
∣∣∣∣∂F∂y (ξ(t, y, y′))∣∣y − y′∣∣∣∣∣∣ ≤∥∥∥∥∂F∂y
∥∥∥∥∞
∣∣y − y′∣∣so F is Lipschitz continuous with respect to y, with
L =
∥∥∥∥∂F∂y∥∥∥∥∞
47
Proof: The aim of the proof is to find some δ > 0 such that T acts on C[t0 − δ, t0 + δ] and is acontraction mapping for some closed U ⊂ C[t0 − δ, t0 + δ].
For any δ > 0, T : C[t0− δ, t0 + δ]→ C[t0− δ, t0 + δ]. Let y0 represent the constant function of value y0,and consider B(y0, ε) = {f ∈ C[t0 − δ, t0 + δ] :‖f − y0‖∞ ≤ ε} ⊂ C[t0 − δ, t0 + δ]. For any f ∈ B(y0, ε),
∥∥T (f)− y0∥∥∞ =
∥∥∥∥∥y0 +
∫ t
t0
F (τ, f(τ)) dτ − y0
∥∥∥∥∥∞
≤ sup(τ,y)∈D
|F | ·|t− t0| ≤ sup(τ,y)∈D
|F | · δ
So choose δ > 0 sufficiently small such that
sup(τ,y)∈D
|F | · δ ≤ ε
This can be done sincesup
(τ,y)∈D|F | <∞
because F is continuous on D and D is bounded. Then T : B(y0, ε)→ B(y0, ε).
Now, ∀f, g ∈ B(y0, ε),
∥∥T (f)− T (g)∥∥∞ =
∥∥∥∥∥∫ t
t0
[F (τ, f(τ))− F (τ, g(τ))] dτ
∥∥∥∥∥∞
≤ L
∥∥∥∥∥∫ t
t0
∣∣f(τ)− g(τ)∣∣ dτ∥∥∥∥∥
∞
≤ L‖f − g‖∞‖t− t0‖∞≤ Lδ‖f − g‖∞
So choose δ > 0 sufficiently small such that Lδ < 1 also. Then
1. (C[t0 − δ, t0 + δ],‖·‖∞) is Banach.
2. B(y0, ε) ⊂ C[t0 − δ, t0 + δ] is closed.
3. T : B(y0, ε)→ B(y0, ε) is a contraction mapping.
Then, by the Contraction Mapping Theorem, T has a unique fixed point y ∈ C[t0 − δ, t0 + δ] such that
y(t) = y0 +
∫ t
t0
F (τ, y(τ)) dτ
which solves the ODE as given in the definition. �
Note: Because of the Contraction Mapping Theorem, it is possible to construct this solution iteratively.This process is known as Picard’s iteration. The iteration is given by y(n+1)(t) = T (y(n)(t)).
Example: Consider the ODE
y(t) =√y(t)
y(0) = 0
Then y1 ≡ 0 is a solution. However, y2(t) = t2
4 is also a solution. This can occur because F is notLipschitz in this case,
√y is much greater than y close to 0, so there is no L which bounds the difference
between these functions.
48
Example: Consider the ODE
y(t) = 2ty(t)
y(0) = 1
with D = [−1, 1]× [0, 2]. Then
F (t, y) = 2ty
(t0, y0) = (0, 1)
This satisfies the conditions of the Picard-Lindelof Theorem, so a unique solution exists, and can beconstructed iteratively using Picard’s iteration, in this case
y(n+1) = T (y(n)) = y0 +
∫ t
0
2τy(n)(τ) dτ
Now, taking y(0)(t) = y0 = 1 the initial approximation to the solution,
y(0) = 1
y(1) = 1 +
∫ t
0
2τy(0)(τ) dτ = 1 +
∫ t
0
2τ dτ = 1 + t2
y(2) = 1 +
∫ t
0
2τy(1)(τ) dτ = 1 +
∫ t
0
2τ(1 + τ2) dτ = 1 +
∫ t
0
2τ + 2τ3 dτ = 1 + t2 +t4
2
...
Using an inductive argument, it can be shown that
y(n)(t) = 1 + t2 +t4
2+ . . .+
t2n
n!
=
n∑k=0
t2k
k!
since y(0) is of this form, and, assuming the statement holds for y(n),
y(n+1)(t) = 1 +
∫ t
0
2τy(n)(τ) dτ
= 1 +
∫ t
0
2τ
(1 + τ2 +
τ4
2+ . . .+
τ2n
n!
)dτ
= 1 +
∫ t
0
2τ + 2τ3 + τ5 + . . .+2τ2n
n!dτ
= 1 + t2 +t4
2+ . . .+
t2n
n!+
t2(n+1)
(n+ 1)!
which is of the correct form. Then, with respect to ‖·‖∞,
y(t) = limn→∞
y(n)(t) =
∞∑k=0
t2k
k!= et
2
which satisfies the ODE.
Exercise: Check the existence and uniqueness of a solution to this ODE on C[−δ, δ], where δ ∈ (0, 14 ).
49
12.3 The Jacobi Algorithm
Motivation: Let A ∈ Rn×n be an n × n real-valued matrix, and let b ∈ Rn be a real-valued n-dimensional vector. Then, if detA 6= 0, the equation Ax = b has the unique solution x = A−1b for somex ∈ Rn. However, inverting A is computationally expensive, typically O(n3). Whilst not a problem forsmall n, if, for example, n = 108, this quickly becomes impractical to compute.
However, if A = D+R for some D,R ∈ Rn×n, where D is diagonal and non-degenerate (detD 6= 0) andR is “small”, so
A =
d11 0 . . . . . . 00 d22 0 . . . 0... 0
. . .. . .
......
.... . .
. . . 00 0 . . . 0 dnn
+
0 r12 . . . . . . r1nr21 0 r23 . . . r2n... r32
. . .. . .
......
.... . .
. . . rn−1nrn1 rn2 . . . rnn−1 0
then Dx + Rx = b, so Dx = b − Rx. If R is “small”, then Dx ≈ b. Let x(n) ⊂ Rn be a sequence ofapproximations to x. Then it seems reasonable to set x(0) = D−1b, and let
x(1)=D−1(b−Rx(0))x(2)=D−1(b−Rx(1))
...
x(n+1)=D−1(b−Rx(n))
This is called the Jacobi iteration. Using the Contraction Mapping Theorem, under certain conditionsit can be shown that this sequence converges to x. Since diagonal inversion is only of order O(n), thisis very fast to compute, so these approximations are extremely useful in situations where inverting A isnot practical. Additionally, the sequence converges to x exponentially.
Proposition: Suppose that ∥∥∥D−1R∥∥∥ ..= supw 6=0
∥∥D−1Rw∥∥2
‖w‖2= λ < 1
Then the Jacobi iteration as defined above converges to x, the solution to Ax = b.
Proof: Let T : Rn → Rn, T : x 7→ D−1(b − Rx). T acts on (Rn,‖·‖2), which is a Banach space, andRn ⊆ Rn is closed. Now, ∀u, v ∈ Rn,
‖Tu− Tv‖2 =∥∥∥(D−1b−D−1Ru)− (D−1b−D−1Rv)
∥∥∥2
=∥∥∥D−1R(u− v)
∥∥∥2
≤∥∥∥D−1R∥∥∥‖u− v‖2 = λ‖u− v‖2
Then, since λ < 1, T is a contraction mapping on Rn, so, by the Contraction Mapping Theorem, T hasa unique fixed point h ∈ Rn. h = T (h) = D−1(b−Rh), so Dh = b−Rh, so (D +R)(h) = b, so Ah = b,so the Jacobi iteration constructs a solution to Ax = b. Moreover∥∥∥h− x(n)∥∥∥
2=
∥∥∥∥T (h)− T(x(n−1)
)∥∥∥∥2
≤ λ∥∥∥h− x(n−1)∥∥∥
2≤ . . . ≤ λn
∥∥∥h− x(0)∥∥∥2
= e−n log 1λ
∥∥∥h−D−1b∥∥∥2
so the sequence converges exponentially, and the number of iterations needed does not depend on thedimension of Rn. �
50
12.4 The Newton-Raphson Algorithm
Motivation: For some continuously differentiable f : R→ R, finding an x ∈ R satisfying the equationf(x) = 0 is non-trivial, and a closed-form expression for x cannot always be found, quintic polynomials area simple example of this. It would be useful then to construct a series approximating x, and convergingto this solution.
Initially, choosing some x(0) “close” to the root, then taking the Taylor expansion about this point yieldsg(x) ≈ g(x(0)) + g′(x(0))(x− x(0)). Now, for some x(1) ∈ R, 0 ≈ g(x(1)) = g(x(0)) + g′(x(0))(x(1) − x(0)),so it seems reasonable to set
x(1) − x(0) = − g(x(0))
g′(x(0))
then
x(1) = x(0) − g(x(0))
g′(x(0))
Now, let
x(n+1) = N(x(n)) ..= x(n) − g(x(n))
g′(x(n))
This is called the Newton-Raphson method. N : (R,|·|) → (R,|·|), and is called the Newton-Raphsonoperator. (x(n))n≥0 ⊂ R is a sequence of approximations to x, and, using the Contraction MappingTheorem, under certain conditions it can be shown that this sequence converges to x.
Note: This sequence does not always converge, especially if the initial guess is poor. For example, iff is concave on the right and convex on the left, then the sequence can diverge if the initial guess isparticularly bad.
Proposition: For any continuously differentiable f : R→ R, if∥∥∥∥f · f ′′(f ′)2
∥∥∥∥∞
= λ < 1
then the Newton-Raphson method as defined above converges to x, the solution to f(x) = 0.
Proof: ∀x, y ∈ R and for some ξ ∈ (x, y),
∣∣N(x)−N(y)∣∣ MVT
=∣∣N ′(ξ)|x− y|∣∣ =
∣∣∣∣∣(
1− f ′(ξ)
f ′(ξ)+f(ξ) · f ′′(ξ)f ′(ξ)2
)|x− y|
∣∣∣∣∣ =
∣∣∣∣f(ξ) · f ′′(ξ)f ′(ξ)2
∣∣∣∣|x− y|≤∥∥∥∥f · f ′′(f ′)2
∥∥∥∥∞|x− y| = λ|x− y|
Then, since λ < 1, N is a contraction mapping on R, so, by the Contraction Mapping Theorem, N hasa unique fixed point h ∈ R.
h = N(h) = h− f(h)
f ′(h)
so
h− h = − f(h)
f ′(h)
so f(h) = 0. �
51
