MA40043: Real and Abstract Analysis1

E P Ryan

Department of Mathematical SciencesUniversity of BathBath BA2 7AY

Email: masepr@bath.ac.ukURL: http://people.bath.ac.uk/∼masepr

Timetable

Monday 11.15 in 3E2.4Monday 15.15 in CB3.5Tuesday 17.15 in CB3.1

December 3, 2013

1Unit Catalogue www.bath.ac.uk/catalogues/2013-2014/ma/MA40043.htm

Preface

The aims of the course are:

• to fill in gaps left in the Year 2 basic analysis courses - the main gap concerns uniformapproximation of continuous functions by polynomials or of continuous periodic functions bytrigonometric polynomials;

• to augment the treatment of metric spaces in MA30041;

• to introduce elements of functional analysis (which will be built on in MA40057) – “functionalanalysis” is a standardized way of dealing with functions (identify the function space, estab-lish whether it is a metric, normed, or inner product space, establish whether it is complete,and apply appropriate general theorems.

There are numerous books in the library which, collectively, cover the material in the course. Belowis a sample list.

• Kolmogorov, A N & Fomin, S V, Introductory Real Analysis, Dover.

• Kreyszig, E, Introductory Functional Analysis with Applications, Wiley.

• Protter, M H & Morrey, C B, A First Course in Real Analysis, Springer-Verlag.

• Simmons, G F, Introduction to Topology and Modern Analysis, McGraw-Hill.

• Stoll, M, Introduction to Real Analysis, Addison Wesley.

• Sutherland, W A, Introduction to Metric and Topological Spaces, OUP.

1

Chapter 1

Uniform continuity, uniformconvergence and uniformapproximation

1.1 Uniform continuity

Until further notice, D will denote a non-empty subset of R, not necessarily an interval.

Definition 1.1. A function f : D → R is continuous at a point d ∈ D if, for all ε > 0, there existsδ > 0 such that

x ∈ D & |x− d| < δ =⇒ |f(x)− f(d)| < ε.

Note that, in the above definition, δ depends on ε and d in general. The following is an equivalent“sequential” definition of continuity.

Definition 1.2. A function f : D → R is continuous at a point d ∈ D if, for any sequence(xn) ⊂ D such that xn → d as n→ ∞, we have f(xn) → f(d) as n→ ∞.

Definition 1.3. A function f : D → R is continuous if it is continuous at every point of D.

Definition 1.4. A function f : D → R is uniformly continuous if, for all ε > 0, there exists δ > 0such that

x, y ∈ D & |x− y| < δ =⇒ |f(x)− f(y)| < ε.

The crucial point to note here is that δ depends only on ε.

It is clear thatuniform continuity =⇒ continuity,

butcontinuity =⇒ uniform continuity.

Example. Define f : R → R, x 7→ f(x) := sin(x2). Then f is continuous but not uniformlycontinuous. [You are asked to prove this in Exercise Sheet 1, Question 1.]

Theorem 1.1. If D is closed and bounded and f : D → R is continuous, then f is uniformlycontinuous.

Proof. Let D be closed and bounded and assume that f : D → R is continuous. Seeking acontradiction, suppose that f is not uniformly continuous. Then there exist ε > 0 and sequences(xn), (yn) in D such that

|xn − yn| <1

nand |f(xn)− f(yn)| ≥ ε ∀ n ∈ N.

2

By boundedness of D and the Bolzano-Weierstrass Theorem1, there exists a subsequence (xnk) of

the sequence (xn) in D with xnk→ x as k → ∞. Clearly, we also have ynk

→ x as k → ∞. Byclosedness of D, we may infer that x ∈ D. By continuity of f , f(xnk

) → f(x) and f(ynk) → f(x)

as k → ∞. Choose k sufficiently large so that |f(xnk) − f(x)| < ε/2 and |f(ynk

) − f(x)| < ε/2.We now arrive at a contradiction

ε ≤ |f(xnk)− f(ynk

)| ≤ |f(xnk)− f(x)|+ |f(ynk

)− f(x)| < ε.

Therefore, f is uniformly continuous. 2

1.2 Uniform convergence

Definition 1.5. A sequence of functions (fn : D → R) converges to f : D → R

(i) pointwise if, for all x ∈ D and all ε > 0, there exists N ∈ N such that

n > N =⇒ |fn(x)− f(x)| < ε ,

(ii) uniformly if, for all ε > 0, there exists N ∈ N such that

x ∈ D & n > N =⇒ |fn(x)− f(x)| < ε.

Note that, in (ii), N depends only on ε. In (i), f is the pointwise limit of (fn); in (ii), f is theuniform limit of (fn).

The above definition is equivalent to the following.

Definition 1.6. A sequence of functions (fn : D → R) converges to f : D → R

(i) pointwise if, for each fixed x ∈ D, we have fn(x) → f(x) as n→ ∞,

(ii) uniformly if supx∈D

|fn(x)− f(x)| → 0 as n→ ∞.

Clearly,uniform convergence =⇒ pointwise convergence,

butpointwise convergence =⇒ uniform convergence.

Example. Consider the sequence (fn) of functions x 7→ fn(x) = xn defined on D = [0, 1]. Thepointwise limit of this sequence is the (discontinuous) function f : D → R given by

f(x) :=

{0 if x ∈ [0, 1),

1 if x = 1.

However, the convergence is not uniform because supx∈D

|fn(x)− f(x)| = 1 for all n ∈ N.

fn(x)

x

n = 1

2

3

1

10

b

bc

f(x)

x

1

10

converges pointwise

n→ ∞

1Recall the Bolzano-Weierstrass Theorem: every bounded sequence in R has a convergent subsequence.

3

1.2.1 Uniform Cauchy Principle

Recall that a sequence (xn) of real numbers is a Cauchy sequence if, for each ε > 0, there existsN ∈ N such that

m,n > N =⇒ |xn − xm| < ε.

The standard Cauchy Principle states that (xn) is a Cauchy sequence if, and only if, it is convergent.Our next goal is to extend these ideas to sequences of functions.

Augustin Louis Cauchy (1789-1857)

Definition 1.7. A sequence of functions (fn : D → R) is a uniform Cauchy sequence if, for eachε > 0, there exists N ∈ N such that

x ∈ D & m,n > N =⇒ |fn(x)− fm(x)| < ε.

Note that, in the above definition, N depends only on ε.

The following is the Uniform Cauchy Principle.

Theorem 1.2. A sequence of functions (fn : D → R) is a uniform Cauchy sequence if, and onlyif, it is uniformly convergent.

Proof.Necessity. Assume that (fn) is a uniform Cauchy sequence of functions D → R. Then, for eachx ∈ D, (fn(x)) is a Cauchy sequence of real numbers and so, by the standard Cauchy Principle, isconvergent: we denote its limit by f(x). Thus, with each x ∈ D, we may associate a unique realnumber f(x). This defines (pointwise) a function f : D → R. Let ε > 0 be arbitrary. Since (fn) isa uniform Cauchy sequence, there exists N ∈ N such that

|fn(x)− fm(x)| < ε/2 ∀ m,n > N, ∀ x ∈ D,

whence|fn(x)− f(x)| = lim

m→∞|fn(x)− fm(x)| ≤ ε/2 < ε ∀ n > N, ∀ x ∈ D,

and so (fn) is uniformly convergent.

Sufficiency. Assume that (fn) is uniformly convergent, with limit f . Let ε > 0 be arbitrary.Then, there exists N ∈ N such that |fn(x)− f(x)| < ε/2 for all n > N and all x ∈ D. Therefore,

|fn(x)− fm(x)| ≤ |fn(x)− f(x)|+ |fm(x)− f(x)| < ε ∀ m,n > N, ∀ x ∈ D

and so (fn) is a uniform Cauchy sequence. 2

Theorem 1.3. Let (fn) be a uniformly convergent sequence of continuous functions D → R withuniform limit f . Then f is continuous.

4

Proof. Let d ∈ D be arbitrary. It suffices to prove that f is continuous at d. Let ε > 0. Byuniform convergence of (fn), we may choose n ∈ N sufficiently large so that |fn(x) − f(x)| < ε/3for all x ∈ D. By continuity of fn, there exists δ > 0 such that

x ∈ D & |x− d| < δ =⇒ |fn(x)− fn(d)| < ε/3.

Therefore,

x ∈ D & |x− d| < δ =⇒

|f(x)− f(d)| ≤ |f(x)− fn(x)|+ |fn(x)− fn(d)|+ |fn(d)− f(d)| < ε

3+ε

3+ε

3= ε

and so f is continuous at d. 2

Some notation. The space of continuous functions over the domain D is denoted by C(D).We remark that, equipped with the usual notions of addition and scalar multiplication, C(D) is avector space. WhenD is an interval, we will use abbreviated notation of the sort C[a, b] := C([a, b]),C(a, b) := C((a, b)), C[a, b) := C([a, b)), etc..

The space C[a, b] (note the domain is compact, that is, closed and bounded) will be the focus ofconsiderable attention. On this space, we can introduce the function f 7→ ∥f∥, C[a, b] → R, definedby

∥f∥ := supx∈[a,b]

|f(x)| = maxx∈[a,b]

|f(x)|. (1.1)

It is readily verified that the axioms of a norm hold:

• ∥f∥ ≥ 0;

• ∥f∥ = 0 if, and only if, f = 0;

• ∥αf∥ = |α| ∥f∥ ∀α ∈ R;

• ∥f + g∥ ≤ ∥f∥+ ∥g∥.

Therefore, equipped with this norm, C[a, b] is a normed vector space. By Theorem 1.3, a (uni-formly) convergent sequence (fn) in C[a, b] has its limit in C[a, b]. Thus, the concept of a (uni-formly) convergent sequence (fn) in C[a, b], with uniform limit f , can be characterized by theproperty

∥fn − f∥ → 0 as n→ ∞.

By Theorems 1.2, we may infer that every (uniform) Cauchy sequence in C[a, b] is (uniformly)convergent to an element of C[a, b]: this is the notion of completeness of the space C[a, b]. Therefore,equipped with the norm given by (1.1), C[a, b] is a complete normed vector space: such a space isreferred to as a Banach space. These abstract notions (normed space, Banach space, etc) will berevisited later in the course: they are mentioned here to give a feel of the abstract setting.

1.3 Uniform approximation

1.3.1 Approximation of continuous functions by polynomials

First consider the uniform approximation problem for continuous functions. Given a functionf ∈ C[a, b] (with a < b), can it be “nicely” approximated by a “nice” function, say, a polynomialp ∈ P [a, b] (the vector space of polynomials with domain [a, b] and with real coefficients)?

There are two (equivalent) ways of capturing the concept of uniform approximation of f ∈ C[a, b]by polynomials:

(i) for all ε > 0, there exists p ∈ P [a, b] such that ∥f − p∥ < ε;

(ii) there exists a sequence (pn) in P [a, b] such that ∥f − pn∥ → 0 as n→ ∞.

5

1.3.2 Approximation of continuous periodic functions by trigonometricpolynomials

Next, consider the problem of approximation of continuous periodic functions by trigonometricpolynomials.

Definition 1.8. The space C2π is the vector space of continuous 2π-periodic functions

C2π := {f ∈ C(R)| f(t+ 2π) = f(t), ∀t ∈ R}

equipped with the norm∥f∥C2π := max

t∈[−π,π]|f(t)|.

A trigonometric polynomial of degree n is a function of the form

t 7→ c0 +n∑

k=1

(ak cos kt+ bk sin kt).

Here n ∈ N0 := N ∪ {0}, the convention being that one drops the sum in the above formula forn = 0. The vector space of trigonometric polynomials on R is denoted by TP. Clearly, TP ⊂ C2π.

There are two (equivalent) ways of capturing the concept of uniform approximation of a functionin f ∈ C2π by trigonometric polynomials:

(i) for all ε > 0, there exists p ∈ TP such that ∥f − p∥C2π < ε;

(ii) there exists a sequence (pn) in TP such that ∥f − pn∥C2π → 0 as n→ ∞.

1.3.3 Weierstrass Approximation Theorems

Karl Theodor Wilhelm Weierstrass (1815-1897)

Theorem 1.4. Every function f ∈ C[a, b] can be uniformly approximated by polynomials.

Theorem 1.5. Every function f ∈ C2π can be uniformly approximated by trigonometric polyno-mials.

An alternative way of stating Theorem 1.4 (respectively, Theorem 1.5) is “P [a, b] is dense in C[a, b]”(respectively, “TP is dense in C2π”).

The notion of “denseness” will be revisited in the next chapter.

We will postpone the proofs of these theorems to later in the course: in particular, it will turn outthat the two theorems are simple corollaries of a more general result known as the Stone-WeierstrassTheorem.

6

Chapter 2

Metric spaces

2.1 Basic concepts and definitions

Definition 2.1. A metric space (X, d) is a non-empty set X equipped with a function d : X×X →R satisfying the following axioms:

• d(x, y) ≥ 0 ∀ x, y ∈ X, with d(x, y) = 0 iff x = y;

• d(x, y) = d(y, x) ∀ x, y ∈ X;

• d(x, y) ≤ d(x, z) + d(z, y) ∀ x, y, z ∈ X.

Any such function d is called a metric on X.

When the metric d is clear from context, we write X in place of (X, d).Notation. The function d : (x, y) 7→ |x − y| defines a metric on the set of real numbers:henceforth, unless otherwise stated, the symbol R should be interpreted as the metric space (R, d).

Definition 2.2. A real normed space is a real vector space X equipped with a function ∥ · ∥ : X → Rsatisfying the following axioms:

• ∥x∥ ≥ 0 ∀ x ∈ X, with ∥x∥ = 0 iff x = 0;

• ∥αx∥ = |α| ∥x∥ ∀ x ∈ X ∀ α ∈ R;

• ∥x+ y∥ ≤ ∥x∥+ ∥y∥ ∀ x, y ∈ X.

A normed space is a special case of a metric space: use d(x, y) := ∥x− y∥ as the metric.

Notation. The function ∥ · ∥ : x 7→√x21 + · · ·+ x2n defines a norm on the space Rn of n-tuples

(x1, ..., xn) = x of real numbers: henceforth, unless otherwise stated, the symbol Rn should beinterpreted as normed space (Rn, ∥ · ∥).

Definition 2.3. Let X be a metric space with metric d. A sequence (xn) in X is said to convergeto x ∈ X if d(xn, x) → 0 as n→ ∞.

Definition 2.4. Let X be a metric space with metric d. A sequence (xn) in X is a Cauchysequence if, for all ε > 0, there exists N ∈ N such that

m,n > N =⇒ d(xm, xn) < ε.

Definition 2.5. A metric space X is complete if every Cauchy sequence in X converges to anelement of X.

Complete normed spaces are called Banach spaces. Simple examples of Banach spaces include Rand C[a, b]. As we have already seen, the fact that C[a, b] is complete follows from Theorems 1.2and 1.3. In Section 2.1.2 below, we will generalize the latter.

7

Stefan Banach (1892-1945)

2.1.1 Uniform continuity and uniform convergence - revisited

Let X and Y be metric spaces, with respective metrics dX and dY . We wish to consider propertiesof functions or maps f : X → Y .

Definition 2.6. A function f : X → Y is continuous at a point p ∈ X if, for all ε > 0, thereexists δ > 0 such that

x ∈ X, dX(x, p) < δ =⇒ dY (f(x), f(p)) < ε.

equivalently, f : X → Y is continuous at a point p ∈ X if, for any sequence (xn) in X convergingto p, the sequence (f(xn)) in Y is convergent to f(p) ∈ Y .A function f : X → Y is continuous if it is continuous at every point of X.

Definition 2.7. A function f : X → Y is uniformly continuous if, for all ε > 0, there exists δ > 0such that

x1, x2 ∈ X, dX(x1, x2) < δ =⇒ dY (f(x1), f(x2)) < ε.

Definition 2.8. A sequence (fn) of functions fn : X → Y is uniformly convergent with limitf : X → Y if, for all ε > 0, there exists N ∈ N such that

x ∈ X & n > N =⇒ dY (fn(x), f(x)) < ε.

The following is a generalization, to a metric space setting, of Theorem 1.3.

Theorem 2.1. The limit of a uniformly convergent sequence of continuous functions X → Y iscontinuous.

You are asked to prove this in Question 3 of Exercise Sheet 2.

2.1.2 The Banach space C(X) of bounded continuous functions X → RLetX be a metric space with metric d and consider the space C(X) of bounded continuous functionsX → R. Equipped with the obvious notions of addition (for f, g ∈ C(X), f + g ∈ C(X) is thefunction x 7→ f(x) + g(x)) and scalar multiplication (for f ∈ C(X) and α ∈ R, αf ∈ C(X) is thefunction x 7→ αf(x)), C(X) is a real vector space. Defining the function ∥ · ∥ : C(X) → R by

∥f∥ := supx∈X

|f(x)|,

it is readily verified that ∥ · ∥ is a norm on C(X) and, so equipped, C(X) is a normed space. Toconclude that C(X) is a Banach space, it remains to show that, as a metric space with metric(f, g) 7→ ∥f − g∥, it is complete. This you are asked to do in Question 4 Exercise Sheet 2.

8

2.1.3 Denseness and the Baire category theorem

Let y be an element of a metric space X (metric d), and let r > 0. Then

Br(y) := {x ∈ X| d(x, y) < r}

is the open ball of radius r centred at y. A set Y ⊂ X is said to be open if, for all y ∈ Y , thereexists r > 0 such that Br(y) ⊂ Y . A set Z ⊂ X is said to be closed if its complement, Y = X \Z,is open1. Equivalently, a set Z ⊂ X is said to be closed if it contains all its limit points (recall thatx ∈ X is a limit point of Z if every open set containing x also contains some point z ∈ Z, z = x).The closure, Y , of a set Y ⊂ X is the union of Y and all its limit points.

The interior of Y ⊂ X is the union of all open sets contained in Y and is denoted by◦Y . Equiv-

alently,◦Y is the set of all interior points of Y (a point y is an interior point of Y if there exists

ε > 0 such that Bε(y) ⊂ Y ).

Definition 2.9. Let X be a metric space, and let Y ⊂ X. Then Y is (everywhere) dense in X if,for all x ∈ X and all ε > 0, there exists y ∈ Y such that d(x, y) < ε.

In other words, Y ⊂ X is dense in X if every open ball (no matter how small) in X contains atleast one point of Y . The prototype example is the set Q of rational numbers which forms a densesubset of the reals R.

Lemma 2.1. Let X be a metric space (with metric d), and let Y ⊂ X. The following statementsare equivalent:

(a) Y is (everywhere) dense in X;

(b) for all x ∈ X, there exists a sequence (yn) in Y such that yn → x as n→ ∞;

(c) Y = X.

Proof.(a) =⇒ (b): Assume that Y is (everywhere) dense in X. Let x ∈ X be arbitrary. By densenessof Y , for each n ∈ N there exists yn ∈ Y such that d(x, yn) < 1/n. The sequence (yn) in Y , sodefined, is such that yn → x as n→ ∞. Therefore, (b) holds.

(b) =⇒ (c): Assume that (b) holds. Let x ∈ X be arbitrary. To prove that (c) holds, it sufficesto show that x ∈ Y . By (b), there exists a sequence (yn) in Y such that yn → x as n→ ∞. If thesequence has only finitely many distinct terms, then x = yn for some n and so x ∈ Y ⊂ Y . If thesequence has infinitely many distinct terms, then x is a limit point of the set {yn| n ∈ N} and sois a limit point of the set Y : therefore, x ∈ Y .

(c) =⇒ (a): Assume that Y = X. Then, for all x ∈ X and ε > 0, there exists y ∈ Y such thatd(x, y) < ε and so (a) holds. 2

Definition 2.10. Let X be a metric space. A set Y ⊂ X is nowhere dense if◦Y = ∅.

Exercise: show that a set Y is nowhere dense in the metric space X if, and only if, X \Y is dense.

Proof.

Y nowhere dense in X ⇐⇒◦Y = ∅

⇐⇒ every non-empty open set in X contains a point of X \ Y⇐⇒ X \ Y is dense.

2

1Note that the notions of open and closed are well defined without the assumption of the metric space beingcomplete. Note also that the empty set ∅ and the whole space X are assumed to be open and closed simultaneously;this is a standard convention.

9

We now proceed to introduce an important theorem, the Baire Category Theorem, relating tocomplete metric spaces. This theorem underpins three of the cornerstones of functional analysis:the uniform boundedness principle, the open mapping theorem and the closed graph theorem.

Rene-Louis Baire (1874-1932)

Theorem 2.2. Let X be a complete metric space. If (Gn) is a sequence of dense open subsets of

X, then∞∩

n=1Gn is dense in X.

Proof. Let x0 ∈ X and δ0 > 0 be arbitrary. It suffices to show that

Bδ0(x0) ∩( ∞

∩n=1

Gn

)= ∅. (2.1)

As G1 is dense, there exists x1 ∈ G1∩Bδ0(x0). Note that G1∩Bδ0(x0) is open (the intersection of afinite number of open sets is open). Therefore, there exists δ1 > 0 such that B2δ1(x1) ⊂ G1∩Bδ0(x0)and δ1 <

δ02 .

As G2 is dense, there exists x2 ∈ G2 ∩ Bδ1(x1). The set G2 ∩ Bδ1(x1) is open and so there existsδ2 > 0 such that B2δ2(x2) ⊂ G2 ∩Bδ1(x1) and δ2 <

δ12 < δ0

22 .

Proceeding inductively, we obtain sequences (xn) in X and (δn) in (0,∞) such that B2δn(xn) ⊂Gn ∩Bδn−1(xn−1) and δn <

δn−1

2 < δ02n .

We now have

B2δn(xn) ⊂ Bδn−1(xn−1) ⊂ B2δn−1(xn−1) ⊂ Bδn−2(xn−2) ⊂ . . . ⊂ B2δ1(x1) ⊂ Bδ0(x0),

and som,n ∈ N, m > n =⇒ B2δm(xm) ⊂ Bδn(xn) ⊂ Bδ0(x0).

This implies, in particular, that

m,n ∈ N, m > n =⇒ d(xm, xn) < δn <δ02n. (2.2)

Therefore, (xn) is a Cauchy sequence and, by completeness of X, it follows that (xn) converges tosome x ∈ X.

By the triangle inequality, we have

d(x, xn) ≤ d(x, xm) + d(xm, xn) ∀ m,n ∈ N. (2.3)

Now fix n and let m→ ∞ (and so d(x, xm) → 0 as m→ ∞). By (2.2) and (2.3), we have

d(x, xn) ≤ δn .

Consequently,x ∈ B2δn(xn) ⊂ Gn ∩Bδn−1

(xn−1) ⊂ Gn ∩Bδ0(x0).

10

As n is arbitrary, we may conclude that

x ∈∞∩

n=1(Gn ∩Bδ0(x0)) = Bδ0(x0) ∩

( ∞∩

n=1Gn

)which establishes (2.1). 2

Intuitively, a nowhere dense subset of a metric space may be thought of as a set that does notcontain very much of the space. The following reformulation of the Baire Category Theorem assertsthat a complete metric space cannot be covered by a sequence of such sets.

Theorem 2.3. The complement of a countable union of nowhere dense subsets of a completemetric space is dense. A complete metric space is not a countable union of nowhere dense subsets.

Proof Let (Yn) be a sequence of nowhere dense subsets of a complete metric space X and, foreach n, define Gn := X \ Y n. For each n, the set Gn is dense and open. Moreover, we have

X \∞∪

n=1Yn ⊃ X \

∞∪

n=1Y n =

∞∩

n=1

(X \ Y n

)=

∞∩

n=1Gn.

But, by Theorem 2.2,∞∩

n=1Gn is dense. Therefore, X\

∞∪

n=1Yn contains a dense set and so is itself

dense which, in turn, implies that∞∪

n=1Yn = X. 2

The title of Baire’s theorem arises from his terminology for sets.

Definition 2.11. A set Y ⊂ X is of first category or meagre in X if it is a countable union ofnowhere dense sets, otherwise, Y is of second category.

Thus, Baire’s theorem asserts the following.

Corollary 2.1. A complete metric space is of second category in itself.

Corollary 2.2. R is uncountable.

Proof. With metric d(x, y) = |x− y|, R is a complete metric space. For any x ∈ R, the set {x} isclosed and has empty interior. Thus, if R were countable, it would be a countable union of nowheredense sets – which is impossible in view of Theorem 2.3. 2

The following result asserts not only that there exist functions f ∈ C[a, b] that are nowheredifferentiable on (a, b) but also that such functions are dense in C[a, b].

Corollary 2.3. Let

L := {f ∈ C[a, b] | there exists t ∈ (a, b) such that f is differentiable at t}.

Then the set C[a, b]\L of continuous functions on [a, b] that are nowhere differentiable on (a, b) isdense in C[a, b].

Proof. For each N ∈ N, define

LN := {f ∈ C[a, b] | ∃t ∈ [a, b] s.t. |f(s)− f(t)| ≤ N |s− t|, ∀s ∈ [a, b]}.

Intuitive picture: for all s ∈ [a, b], the modulus of the slope of the line connecting (t, f(t)) and(s, f(s)) is less than or equal to N .

b

bb

b

(a, f(a))

(b, f(b))

(s, f(s))

(t, f(t)) modulus of slope less than N

11

Our strategy is as follows:

• We show that LN is closed for each N ∈ N.

• We then show that C[a, b]\LN is dense, and so LN is nowhere dense. Therefore, ∪NLN ismeagre.

• By Theorem 2.3, C[a, b]\ ∪N LN is dense.

• Finally, we show that L ⊂ ∪NLN whence C[a, b]\L ⊃ C[a, b]\ ∪N LN and so C[a, b]\L isdense.

First, we will prove that, for all N , LN is closed. Let N ∈ N and let (fn) be a sequence in LN besuch that fn → f ∈ C[a, b] uniformly on [a, b]. We will show that f ∈ LN and so LN is a closedset. There exists a sequence (tn) in [a, b] such that

|fn(s)− fn(tn)| ≤ N |s− tn|, ∀s ∈ [a, b].

By the Bolzano–Weierstrass Theorem, there exists a subsequence (tnk) such that tnk

→ t ∈ [a, b]as k → ∞. Therefore,

|f(s)− f(t)|≤ |f(s)− fnk

(s)|+ |fnk(s)− fnk

(tnk)|+ |fnk

(tnk)− fnk

(t)|+ |fnk(t)− f(t)|

≤ |f(s)− fnk(s)|+N |s− tnk

|+N |tnk− t|+ |fnk

(t)− f(t)| ∀ s ∈ [a, b].

Fixing s ∈ [a, b] arbitrarily and letting k → ∞ we have

|f(s)− f(t)| ≤ N |s− t|.

Thus, f ∈ LN and so LN is closed.

Next, we prove that C[a, b] \LN is dense (and so LN is nowhere dense) for all N ∈ N. Let N ∈ N,ε > 0 and f ∈ C[a, b] be arbitrary. We will show that there exists g ∈ C[a, b] \ LN such that∥f − g∥ < ε. Since P [a, b] is dense in C[a, b], there exists p ∈ P [a, b] such that ∥f − p∥ < ε/2. Tocomplete the argument, it suffices to show that ∥p − g∥ < ϵ/2 for some g ∈ C[a, b] \ LN . Withreference to the figure below, let g be any continuous piecewise linear function [a, b] → R such that(a) its graph lies between the (smooth) graphs of the functions p(·) + (ε/2) and p(·) − (ε/2), and(b) the slopes of the linear segments of the graph are greater in modulus than N . Every such ghas the requisite properties: g ∈ C[a, b] \ LN and ∥p− g∥ < ε/2.

p(·)− (ε/2)

p(·) + (ε/2)

p(·)g(·)

We briefly digress to give an explicit construction of one such g. Clearly, p is continuously differ-entiable with derivative p′. Choose M ∈ N such that

εM

2(b− a)> ∥p′∥+N.

Define ∆ := (b − a)/M and ti := a + i∆, i = 0, 1, ...,M . Now define the continuous, piecewiselinear function g : [a, b] → R by the property that, for each i = 0, 1, ...,M ,

g(t) =1

∆

[(p(ti) + (−1)iε/4

)(ti+1 − t) +

(p(ti+1) + (−1)i+1ε/4

)(t− ti)

]∀ t ∈ [ti, ti+1].

12

Then, ∥p− g∥ = ε/4 and, for each i = 0, 1, ...,M , the modulus of the slope Si of the linear segmenton [ti, ti+1] satisfies

|Si| =1

∆|p(ti+1)− p(ti) + (−1)i+1ε/2| ≥ ε

2∆− |p(ti+1)− p(ti)|

∆≥ εM

2(b− a)− ∥p′∥ > N.

In particular, we have g ∈ C[a, b] \ LN with ∥p− g∥ < ε/2 as required.

Returning to the main argument, we may now infer that, for each N ∈ N, C[a, b]\LN is dense andso LN is nowhere dense. By Theorem 2.3, it follows that C[a, b]\ ∪N∈N LN is dense.

Next, we show that L ⊂ ∪N∈N

LN . Assume f ∈ L, and so f ′(t) exists for some t ∈ (a, b). Then there

exists δ > 0 such that

0 < |s− t| < δ & s ∈ [a, b] =⇒∣∣∣∣f(s)− f(t)

s− t

∣∣∣∣ < |f ′(t)|+ 1 .

Also, we have

|s− t| ≥ δ & s ∈ [a, b] =⇒∣∣∣∣f(s)− f(t)

s− t

∣∣∣∣ ≤ |f(s)− f(t)|δ

≤ 2∥f∥δ

.

Therefore,

s ∈ [a, b] =⇒ |f(s)− f(t)| ≤(|f ′(t)|+ 1 +

2∥f∥δ

)|s− t|.

Now, choose

N ∈ N with N ≥(|f ′(t)|+ 1 +

2∥f∥δ

),

in which case we have|f(s)− f(t)| ≤ N |s− t|, ∀s ∈ [a, b].

Thus, every f ∈ L is also in LN for some N ∈ N. Therefore, L ⊂ ∪N∈N

LN and so

C[a, b]\L ⊃ C[a, b]\ ∪N∈N LN .

Therefore, C[a, b]\L contains a dense set and so is itself dense. 2

2.2 Separable spaces

Loosely speaking, a metric space with the useful property that it has an at most countable subsetfrom within which every element of the space can be approached is said to be separable (theprototypical example being the approximation of real numbers by rationals). More precisely:

Definition 2.12. A metric space is separable if it contains an at most countable dense set.

(Recall: a set S countable if there exists a bijection S → N.)

Example 2.1. R is separable because it contains the dense countable subset Q.

Example 2.2. Polynomials with rational coefficients are dense in C[a, b] and are countable (seeQuestion 5, Exercise Sheet 3). This implies that C[a, b] is separable.

Definition 2.13. Let X be a metric space, and let Y ⊂ X. A collection {Zα, α ∈ I} of sets(respectively, open sets) in X such that Y ⊂ ∪α∈IZα is said to be a cover (respectively, an opencover) for Y .

Theorem 2.4. Let X = (X, d) be a separable metric space. Every open cover of X has a countablesubcover.

13

Proof. Since X is separable, it has an at most countable dense subset S. Let B denote thecollection of open balls with centres in S and rational radii, that is,

B := {U | U = Bq(s), s ∈ S, q ∈ Q, q > 0}.

This collection is countable.

Claim. For every open ball Br(x) in X, there is an open ball Ux in B such that x ∈ Ux ⊂ Br(x).

To establish this claim, we argue as follows. Let Br(x) be an open ball in X. Since S is dense inX, there exists s ∈ S such that d(x, s) < r/4. Now choose q ∈ Q such that r/4 < q < r/2. Itfollows that x ∈ Bq(s). Moreover, if y ∈ Bq(s), then

d(x, y) ≤ d(x, s) + d(s, y) <r

4+ q <

r

4+r

2< r

and so Bq(s) ⊂ Br(x). Setting Ux := Bq(s) establishes the claim.

We now return to the proof of the theorem. Let {Zα, α ∈ I} be an open cover of X. Let x ∈ Xbe arbitrary. Then, x ∈ Zα for some α ∈ I and, by the claim, we may infer that x ∈ Ux ⊂ Zα forsome Ux ∈ B. Since B is countable, it follows that the set {Ux| x ∈ X} ⊂ B is also countable, andso there exist a sequences (xn) in X and (αn) in I such that

{Ux| x ∈ X} = {Uxn | n ∈ N} ⊂ B and Uxn ⊂ Zαn ∀n ∈ N.

Therefore,

X ⊂ ∪x∈X

Ux =∞∪

n=1Uxn

⊂∞∪

n=1Zαn

.

This completes the proof. 2

2.3 Compactness

2.3.1 Compact sets

Definition 2.14. A set S in a metric space is said to be compact if every open cover of S containsa finite subcover.

Definition 2.15. A set S in a metric space is said to be sequentially compact if every sequencein S has a subsequence convergent to an element of S.

Theorem 2.5. A set in a metric space is compact if, and only if, it is sequentially compact.

Proof. In the course Metric Spaces. 2

Lemma 2.2. A compact set in a metric space is bounded and closed.

Proof. Let S be a compact subset of a metric space (X, d). Let x be an element of the closureS of S. Then there is a sequence (xn) in S such that d(xn, x) → 0 as n → ∞. By compactnessof S, it follows that x ∈ S and so S is closed. For contradiction, suppose that S is unbounded.Then there exists a sequence (xn) in S such that d(xn, x) > n for all n ∈ N, where x is somefixed element of S. Clearly, every subsequence of (xn) is unbounded and so cannot be convergent,contradicting compactness of S. Therefore, S is bounded. 2

Remark 2.1. The converse of the above result does not hold: a bounded and closed subset of ametric space is not necessarily compact.

For example, consider the metric space X of square-summable real sequences x = (xn) (with∑n∈N x2n < ∞), with metric d(x, y) :=

√∑n∈N(xn − yn)2. For each m ∈ N, let x(m) denote the

sequence which has m-th term 1, all other terms being 0 (and so d(x(m), 0) = 1 and d(x(m), x(n)) =√2 for all m,n ∈ N, n = m. Then S = (x(m)) is a bounded subset of X; S is also closed since it

has no limit points and, for the same reason, S is not compact.

We proceed to identify a class of metric spaces for which the converse does hold.

14

Lemma 2.3. Let X be a metric space with the property that every bounded sequence in X has aconvergent subsequence. A set in X is compact if, and only if, it is bounded and closed.

Proof.Necessity. This follows by Lemma 2.2.

Sufficiency. Let S be a closed and bounded set in X. Let (xn) be a sequence in S. Since S isbounded, the sequence (xn) is bounded and so, by hypothesis, has a convergent subsequence (xnk

);we denote the limit of the subsequence by x. Since S is closed, we have x ∈ S. Therefore, S issequentially compact and so, by Theorem 2.5, is compact. 2

We know that, by Bolzano-Weierstrass, the metric space X = Rn has the property that everybounded sequence in X has a convergent subsequence. Therefore, we have the following.

Corollary 2.4. A set in Rn is compact if, and only if, it is bounded and closed.

2.3.2 Relatively compact sets

Definition 2.16. A set S in a metric space is said to be relatively compact if its closure S iscompact.

Lemma 2.4. In a metric space a set S is relatively compact if, and only if, every sequence in Scontains a convergent subsequence. [The limit of this subsequence may not be in S.]

Proof See Exercise Sheet 5. 2

Definition 2.17. A set S in a metric space is said to be totally bounded if, for each ε > 0, itcan be covered by finitely many balls of radius ε. These balls are said to form a finite ε-cover ofS, and their centres to form a finite ε-net for S.

It is clear thattotal boundedness =⇒ boundedness

butboundedness =⇒ total boundedness.

To confirm the latter, consider again the metric space X of square-summable real sequences x =(xn) with metric

d(x, y) :=

√∑n∈N

(xn − yn)2 .

For each m ∈ N, let x(m) denote the sequence which has m-th term 1, all other terms being 0 (andso d(x(m), 0) = 1). Then S = (x(m)) is a bounded subset of X: however, it is not totally boundedsince d(x(m), x(n)) =

√2 for n = m and so any ball of radius ε = 1/

√2 cannot contain more than

one element of the sequence.

A finite-dimensional setting can simplify the situation: it is easy to see that a set in the Euclideanspace Rn is bounded if, and only if, it is totally bounded.

Theorem 2.6.(i) Any totally bounded subset of a metric space is separable (as a metric space in its own right).

(ii) Any relatively compact subset of a metric space is totally bounded.

(iii) Any totally bounded subset of a complete metric space is relatively compact.

Proof.(i) Let S be a totally bounded subset of a metric space (X, d). Then, for each m ∈ N, S has a finite2−m-net which we denote by {cm,1, . . . , cm,Km}: we may assume that B2−m(cm,n)∩S = ∅ for n =1, ...,Km. For each m ∈ N and each n ∈ {1, . . . ,Km}, choose an element sm,n ∈ B2−m(cm,n) ∩ S.Clearly, the set Σ =

{sm,n| m ∈ N, n ∈ {1, ...,KM}

}of all these elements is countable. Moreover,

15

Σ is dense in S: to see this, let ε > 0 and s ∈ S be arbitrary; choose m ∈ N such that 2−m < ε/2;s ∈ B2−m(cm,n) for some n ∈ {1, ...,Km} and so d(s, sm,n) < ε.

(ii) Let S be a relatively compact subset of a metric space (X, d). Let ε > 0 be arbitrary. We willconstruct a finite ε-net for S. If S = ∅, then there is nothing to prove. Assume S = ∅ and letx1 ∈ S be arbitrary. If x ∈ Bε(x1) for all x ∈ S, then {x1} is an ε-net for S and we are done:otherwise, proceed to step 2, that is, choose x2 ∈ S such that d(x1, x2) ≥ ε. If x ∈ Bε(x1)∪Bε(x2)for all x ∈ S, then {x1, x2} is an ε-net for S: otherwise, proceed to step 3, that is, choose x3 ∈ Ssuch that d(x2, x3) ≥ ε and d(x1, x3) ≥ ε. If x ∈ ∪j=1,2,3Bε(xj) for all x ∈ S, then {x1, x2, x3}is an ε-net for S: otherwise proceed to the next step. After finitely many such steps we mustarrive at an ε-net for S: otherwise, the construction would yield a sequence (xn) in S with theproperty that d(xn, xm) ≥ ε for all n,m ∈ N with n = m; this sequence cannot have a convergentsubsequence, which contradicts relative compactness of S.

(iii) Let S be a totally bounded subset of a complete metric space (X, d). Let (xn) be a sequencein S. By total boundedness of S, it has a finite 1-net and so there exists an element c1 of the 1-netsuch that the ball B1(c1) contains a subsequence (xk1(n)) of (xn). Similarly, S has a finite (1/2)-netand there exists an element c2 thereof such that the ball B1/2(c2) contains a subsequence (xk2(n))of (xk1(n)) and so (xk2(n)) is contained in ∩2

j=1B1/j(cj). Proceeding inductively, we generate asequence (cm) and a sequence of subsequences

(xn) ⊃ (xk1(n)) ⊃ (xk2(n)) ⊃ · · · (xkm(n)) · · ·

with the property that, for all m ∈ N, (xkm(n)) is a sequence in ∩mj=1B1/j(cj). Define the sequence

(ym) = (xkm(m)) = (xk1(1), xk2(2), xk3(3), ...).

Let ε > 0 be arbitrary and fix N ∈ N such that 1/N < ε/2. Then

ym = xkm(m) ∈ B1/N (cN ) ⊂ Bε/2(cN ) ∀ m > N

and so d(ym, yn) < ε for all n,m > N . Therefore, (ym) ⊂ S is a Cauchy sequence in the completemetric space X. We may now conclude that (ym) is a convergent subsequence of the sequence (xn)in S. Therefore, S is relatively compact. 2

2.3.3 Arzela-Ascoli theorem

Cesare Arzela (1847-1912)(Giulio Ascoli (1843-1896): portrait unavailable)

Definition 2.18. Let (X, d) be a metric space and let D ⊂ X. A set S of functions f : D → R isequicontinuous if, for all ε > 0, there exists δ > 0 such that

x, y ∈ D & d(x, y) < δ & f ∈ S =⇒ |f(x)− f(y)| < ε.

Of course, if the set S contains only one function the above definition coincides with the definitionof uniform continuity.

Denote by C(D) the Banach space of bounded continuous functions f : D → R with ∥f∥ :=supx∈D |f(x)|.

16

Theorem 2.7. Let D be a compact subset of a metric space (X, d). A set of functions S ⊂ C(D)is relatively compact if, and only if, it is bounded (that is, supf∈S ∥f∥ <∞) and equicontinuous.

Proof of Theorem 2.7Necessity . Let S be relatively compact. Then, by Theorem 2.6, S is totally bounded and, hence,is bounded.

It remains to prove equicontinuity of S. Let ε > 0 be arbitrary. By total boundedness of S,there exists a finite (ε/3)-net for S, which we denote by {g1, ..., gN}. By compactness of D, eachgn ∈ C(D) is uniformly continuous (Question 1 on Exercise Sheet 5). It follows that there existsδ > 0 such that

x, y ∈ D & d(x, y) < δ =⇒ |gn(x)− gn(y)| < ε/3 for n = 1, ..., N .

Let f ∈ S be arbitrary. Then there exists n ∈ {1, ..., N} such that ∥f − gn∥ < ε/3 and so

x, y ∈ D & d(x, y) < δ =⇒

|f(x)− f(y)| ≤ |f(x)− gn(x)|+ |gn(x)− gn(y)|+ |gn(y)− f(y)| < ε

3+ε

3+ε

3= ε.

Therefore, S is equicontinuous.

Sufficiency. Let S be bounded and equicontinuous, and let (fn) be any sequence in S. We seek aCauchy subsequence. Let (xp) be a countable dense set in D. Such a set exists since, by Theorem2.6,

D compact =⇒ D totally bounded =⇒ D separable.

The sequence (fn(x1)) is bounded in R and, hence, has a convergent subsequence, which we denoteby (fk1(n)(x1)).

Similarly, (fk1(n)(x2)) is a bounded sequence in R and, hence, has a convergent subsequence(fk2(n)(x2)). Note that (fk2(x)) converges for x = x1 and x = x2.

Proceeding inductively, we generate a sequence of subsequences of (fn):

(fn) ⊃ (fk1(n)) ⊃ (fk2(n)) ⊃ · · · ⊃ (fkm(n)) ⊃ · · ·

with the property that, for each m ∈ N, (fkm(n)(x)) converges at x = x1, . . . , xm.

Extract the “diagonal” sequence (gm) := (fkm(m)) = (fk1(1), fk2(2), ...). Then, for each p ∈ N,(gm)∞m=p is a subsequence of (fkp(n)) and so (gm(xp)) converges for each p ∈ N.

Since (gm) is a subsequence of the sequence (fn) in the Banach space C(D), the proof is completeif we can show that (gm) is a Cauchy sequence: this we proceed to do. Let ε > 0 be arbitrary. Byequicontinuity of S, there exists δ > 0 such that

p ∈ N & x ∈ D & d(x, xp) < δ =⇒ |gm(x)− gm(xp)| <ε

4∀ m ∈ N . (2.4)

By compactness of D, there exists a finite (δ/2)-cover: D ⊂ ∪Jj=1Bδ/2(cj) with D ∩ Bδ/2(cj) = ∅,

j = 1, ..., J . For j = 1, ..., J , choose xpj ∈ (xp) such that xpj ∈ Bδ/2(cj) (such a choice is possiblebecause (xp) is dense in D).

Recalling that (gm(xp)) converges for each p ∈ N, we may infer the existence of N ∈ N such that

m,n ∈ N m,n > N =⇒ |gm(xpj )− gn(xpj )| <ε

4for j = 1, ..., J. (2.5)

Let x ∈ D be arbitrary. Then, for some j ∈ {1, ..., J}, we have d(x, xpj ) < δ which, together with(2.4) and (2.5), gives

|gm(x)− gn(x)| ≤ |gm(x)− gm(xpj )|+ |gm(xpj )− gn(xpj )|+ |gn(xpj )− gn(x)|

<ε

4+ε

4+ε

4=

3ε

4∀ m,n > N .

17

Thus, we have shown that, for each ε > 0, there exists N ∈ N such that

m,n > N =⇒ |gm(x)− gn(x)| <3ε

4∀ x ∈ D =⇒ ∥gm − gn∥ ≤ 3ε

4< ε.

Therefore, (gm) is a Cauchy sequence in the Banach space C(D) and is a subsequence of (fn).Therefore, S is relatively compact. 2

2.3.4 The Stone-Weierstrass theorem

We are now in a position to prove an elegant and powerful generalization of the classical Weierstrassapproximation theorem (Theorem 1.4), due to Marshall H Stone.

Marshall Harvey Stone (1903-1989)

Let D be a non-empty compact subset of a metric space X = (X, d). We regard D as a metricspace in its own right (and so a set Y ⊂ D is open if, for each y ∈ Y , there exists r > 0 such thatthe set {x ∈ D| d(x, y) < r} is contained in Y ). Consider the space C(D) of continuous functionsD → R. By continuity of f and compactness of D, it follows that f is bounded and so, equippedwith the uniform norm (∥f∥ := supx∈D |f(x)|), C(D) is a Banach space. Let S be a non-emptysubset of C(D). Recall the following equivalent definitions of uniform approximation of functionsin C(D) by functions in S.

Definition 2.19 (Uniform Approximation). Functions in C(D) are said to be uniformly approx-imated by functions in S if, for each each f ∈ C(D) and all ε > 0, there exists f0 ∈ S such thatf0 ∈ Bε(f).

The following is readily verified.

Proposition 2.1. The following statements are equivalent:

1. functions in C(D) are uniformly approximated by functions in S;

2. for each f ∈ C(D), there exists a sequence (fn) in S such that ∥f − fn∥ → 0 as n→ ∞;

3. S is dense in C(D);

4. the closure of S is C(D).

Lattices and the Stone Approximation Theorem

For (f, g) ∈ C(D)× C(D), define the functions f ∧ g and f ∨ g by

(f ∧ g)(x) := min{f(x), g(x)}, (f ∨ g)(x) := max{f(x), g(x)}.

Observe that, for f, g ∈ C(D), the map D → R2, x 7→ (f(x), g(x)) is continuous; moreover, themaps R2 → R, (a, b) → max{a, b} and R2 → R, (a, b) → min{a, b} are also continuous. Therefore,the functions f ∧ g and f ∨ g are continuous, that is, f ∧ g ∈ C(D) and f ∨ g ∈ C(D). We referto ∧ and ∨ as the lattice operations. We record that,as a map C(D)×C(D) → C(D), each latticeoperation is continuous (you are asked to prove this in Exercise Sheet 6).

A set S ⊂ C(D) is said to be a lattice if it is closed under the lattice operations, that is,

f, g ∈ S =⇒ f ∧ g, f ∨ g ∈ S.

18

Remark 2.2. Since the lattice operations are continuous, it follows that, if S is a lattice, then sois its closure S.

Theorem 2.8. Let S ⊂ C(D) be a lattice. Then a function f ∈ C(D) is in the closure S of S if,and only if, for all x, y ∈ D and ε > 0, there exists a function fx,y ∈ S such that

|f(x)− fx,y(x)| < ε, & |f(y)− fx,y(y)| < ε .

Proof. Necessity of the stated condition is obvious. We proceed to prove sufficiency. Let f ∈C(D) and ε > 0 be arbitrary. We will construct a function f0 ∈ S such that ∥f−f0∥ < ε. For eachpair of points x, y ∈ D, there exists fx,y ∈ S such that |f(x)−fx,y(x)| < ε and |f(y)−fx,y(y)| < ε.For each fixed x ∈ D, Nx(y) := {z ∈ D| fx,y(z) > f(z) − ε} is an open set containing y. Thefamily of these sets {

Nx(y)| y ∈ D}

covers D and so, by compactness of D, has a finite subcover which we denote by{Nx(yk(x))| k = 1, ...,K

}.

Definehx :=

∨Kk=1fx,yk(x), z 7→ hx(z) = max{fx,y1(x)(z), ..., fx,yK(x)(z)}.

Since S is a lattice and fx,y ∈ S for all y ∈ D, it follows that hx ∈ S. Observe that, for each z ∈ D,there exists k ∈ {1, ...,K} such that z ∈ Nx(yk(x)) and so

hx(z) ≥ fx,yk(x)(z) > f(z)− ε.

Therefore,hx(z) > f(z)− ε ∀ z ∈ D.

Moreover, since |f(x)− fx,yk(x)(x)| < ε, k = 1, ...,K, we have fx,yk(x)(x) < f(x) + ε, k = 1, ...,K.Therefore,

hx(x) < f(x) + ε.

We have now shown that

∀x ∈ D ∃hx ∈ S : hx(z) > f(z)− ε ∀ z ∈ D & hx(x) < f(x) + ε. (2.6)

For each x ∈ D, M(x) := {z ∈ D| hx(z) < f(z) + ε} is an open set containing x. The family ofthese sets {

M(x)| x ∈ D}

covers D and so has a finite subcover, which we denote by{M(xj)| j = 1, ..., J

}.

Definef0 :=

∧Jj=1hxj , z 7→ f0(z) = min{hx1(z), ..., hxJ (z)}.

Since S is a lattice and hxj ∈ S, j = 1, ..., J , we have f0 ∈ S. Let z ∈ D. Then z ∈ M(xj) forsome j ∈ {1, ..., J} and so

f0(z) ≤ hxj (z) < f(z) + ε .

By (2.6), we also havef0(z) > f(z)− ε

and so we may conclude that

|f(z)− f0(z)| < ε ∀ z ∈ D, whence ∥f − f0∥ < ε .

2

19

A set S ⊂ C(D) is said to separate points in D if

x, y ∈ D, x = y =⇒ ∃ f ∈ S : f(x) = f(y).

A set S ⊂ C(D) is said to separate points in D and R if

a, b ∈ R, x, y ∈ D, x = y =⇒ ∃ f ∈ S : f(x) = a, f(y) = b.

An immediate consequence of the above theorem is the Stone Approximation Theorem.

Theorem 2.9 (Stone Approximation Theorem). Let S ⊂ C(D) be a lattice that separates pointsin D and R. Then S is dense in C(D).

Proof. Let f ∈ C(D) be arbitrary. For each pair of points x, y ∈ D, there exists fx,y ∈ S suchthat fx,y(x) = f(x) and fx,y(y) = f(y) (including the possibility that x = y). By Theorem 2.8, itfollows that f ∈ S. Therefore, S = C(D). 2

Algebras and the Stone-Weierstrass Theorem

A set S ⊂ C(D) is said to be an algebra if S is a real vector subspace of C(D) and

f, g ∈ S =⇒ fg ∈ S.

Example 2.3. Let D = I = [a, b] be a compact interval in R. Then the space of polynomialfunctions S = P (I) ⊂ C(I) is an algebra and S separates points in I and R.

Remark 2.3. Since the algebra operation (that is, (f, g) 7→ fg) is continuous (as a map C(D)×C(D) → C(D)), it follows that, if S is an algebra, then so is its closure S; moreover, if S separatespoints in D and R, then so does S.

Theorem 2.10 (Stone-Weierstrass). Let S ⊂ C(D) be an algebra that separates points in D andassume that the constant function x 7→ 1 is in S. Then S is dense in C(D).

We preface the proof of the Stone-Weierstrass Theorem with a proposition and a lemma, whichcontribute to the proof.

Proposition 2.2. For n ∈ N, define un ∈ C[−1, 1] by

un(t) :=

( 12

n

)tn, where

( 12

n

):=

12 (

12 − 1) · · · ( 12 − (n− 1))

n!.

The series 1 +∑∞

n=1 un converges uniformly to the function [−1, 1] → R, t 7→√1 + t.

Proof. See Exercise Sheet 6.

Lemma 2.5. Let a > 0 and write I := [−a, a]. The absolute value function v ∈ C(I) givenby v(t) := |t| can by uniformly approximated by polynomials from P (I) (the space of polynomialfunctions on I with real coefficients).

Proof. Let ε > 0 be arbitrary. We will show that there exists a polynomial p ∈ P (I) such that∣∣|t| − p(t)∣∣ < ε for all t ∈ I. By Proposition 2.2, we know that there exists N ∈ N such that the

polynomial q ∈ P [−1, 1], given by

q(s) = 1 +N∑

n=1

(12

n

)sn,

has the property that ∣∣√1 + s− q(s)∣∣ < ε

a∀ s ∈ [−1, 1].

20

Let p ∈ P (I) be the polynomial given by p(t) = a q((t/a)2 − 1). Let t ∈ I be arbitrary and writes := (t/a)2 − 1 ∈ [−1, 1]. Then∣∣|t| − p(t)

∣∣ = ∣∣a√1 + (t/a)2 − 1− a q((t/a)2 − 1)∣∣ = a

∣∣√1 + s− q(s)∣∣ < ε.

This completes the proof. 2

We are now in a position to prove the Stone-Weierstrass Theorem.

Proof of the Stone-Weierstrass Theorem. First, we show that S separates points in D andR. Let a, b ∈ R and x, y ∈ D with x = y. Since S separates points in D, there exists f ∈ S suchthat f(x) = f(y). We seek α, β ∈ R such that the function g = α 1 + βf ∈ S (where 1 denotesthe constant function z 7→ 1) has the requisite properties g(x) = a and g(y) = b. In particular, weseek α, β ∈ R such that α+βf(x) = a and α+βf(y) = b which, since f(x)− f(y) = 0, has uniquesolution

α =bf(x)− af(y)

f(x)− f(y), β =

a− b

f(x)− f(y).

Therefore, S separates points in D and R. Recalling Remark 2.3, the closure S of the algebra S isalso an algebra and separates points in D and R. If it can also be established that S is a lattice,then an application of Theorem 2.9 completes the proof. We proceed to show that S is a lattice.We claim that

f ∈ S =⇒ |f | ∈ S, (2.7)

where |f | denotes the function x 7→ |f(x)|. If f = 0 (the zero function), then the claim is evidentlytrue. Assume f = 0, define a := ∥f∥ > 0 and I := [−a, a]. Let ε > 0 be arbitrary. By Lemma 2.5,there is a polynomial p ∈ P (I) such that

∣∣|t| − p(t)∣∣ < ε for all t ∈ I. Therefore,

|(p ◦ f)(x)− |f(x)|∣∣ < ε ∀ x ∈ D

and, since S is an algebra, p ◦ f ∈ S. We have now shown that, for every ε > 0, there exists f0 ∈ Ssuch that

∥∥f0 − |f |∥∥ < ε and so, since S is closed, it follows that |f | ∈ S.

Noting that

f ∧ g =1

2

((f + g)− |f − g|

), f ∨ g =

1

2

((f + g) + |f − g|

),

we may now conclude thatf, g ∈ S =⇒ f ∧ g, f ∨ g ∈ S

and so S is a lattice. By Theorem 2.9, S is dense in C(D) and so S = C(D). 2

2.3.5 The classical Weierstrass approximation theorems

In this section, we show that the two classical approximation theorems (on approximation of contin-uous functions by polynomials and approximation of continuous periodic functions by trigonometricpolynomials), due to Weierstrass, are simple consequences of the Stone-Weierstrass Theorem.

Theorem 2.11. Let I = [a, b] be a compact interval in R. The space S = P (I) of polynomialfunctions on I with real coefficients is dense in C(D) = C(I).

Proof. Clearly, P (I) is a real vector space and

p, q ∈ S =⇒ pq ∈ S.

Therefore, S is an algebra. The constant function x 7→ 1 is in S. Moreover, the identity functionx 7→ x is in S and so S separates points. By the Stone-Weierstrass Theorem, we conclude that Sis dense in C(D). 2

The proof of the above result applies, mutatis mutandis, to establish the following.

21

Theorem 2.12. Let D be a compact subset of Rn. The space S = P (D) of polynomial functionson D with real coefficients is dense in C(D).

Let C2π denote the Banach space of continuous 2π-periodic functions R → R:

C2π := {f ∈ C(R)| f(t+ 2π) = f(t) ∀ t ∈ R}

equipped with the norm ∥f∥ := maxt∈[−π,π] |f(t)| = maxt∈(−π,π] |f(t)|. Let TP ⊂ C2π de-note the space trigonometric polynomials, that is, the space of functions of the form t 7→ a0 +∑n

k=1

(ak cos kt+ bk sin kt

), with real coefficients a0, ..., an, b1, ..., bn.

Theorem 2.13. The space TP is dense in C2π.

Proof. Let f ∈ C2π and ε > 0 be arbitrary. We will establish the existence of f0 ∈ TP such that∥f − f0∥ < ε. Let D be the unit circle in R2, that is, D = {x = (x1, x2) ∈ R2| x21 + x22 = 1}. LetS ⊂ C(D) be the space of polynomial functions on D with real coefficients. By Theorem 2.12, Sis dense in C(D). Let h denote the continuous bijection (−π, π] → D, t 7→ (cos t , sin t) and defineφ ∈ C(D) by φ(x) = φ(x1, x2) := f(h−1(x1, x2)) = (f ◦ h−1)(x). Observe that

∥φ∥ := max(x1,x2)∈D

|φ(x1, x2)| = maxt∈(−π,π]

|f(t)|

and so ∥f∥ = ∥φ∥ (of course, the former is the norm on C2π and the latter is the uniform normon C(D)). Since S is dense in C(D), there exists φ0 ∈ S such that ∥φ − φ0∥ < ε. Now definef0 ∈ C(−π, π] by f0(t) := φ0(cos t , sin t) = (φ0 ◦ h)(t) and extend this function to f0 ∈ C2π byimposing periodicity: f0(t+2π) = f0(t) for all t ∈ R. Since φ0 is a polynomial function, it followsthat f0 ∈ TP : moreover,

∥f − f0∥ = ∥φ− φ0∥ < ε .

This completes the proof. 2

2.4 Completion of metric spaces

Can a metric space fail to be complete?

Example 2.4. (Q, d), with metric d(p, q) = |p− q|, is incomplete.

To see this, simply note that, for each p ∈ Q, the set {p} is closed and has empty interior and so,since Q is countable, it is a countable union of nowhere dense sets. By Theorem 2.3, it follows thatQ cannot be complete.

Example 2.5. The space preLp[a, b] is incomplete.

For p ≥ 1, preLp[a, b] denotes the space C[a, b] equipped with the norm ∥ · ∥p given by

∥f∥p :=

(∫ b

a

|f(x)|pdx

)1/p

.

To see that this space is incomplete, we construct a Cauchy sequence (fn) in preLp[−1, 1] whichfails to converge to an element of preLp[−1, 1]. This is done in Exercise Sheet 4, Question 2.

Our next objective is to show that every incomplete metric space has a “completion”.

Definition 2.20. A map φ : (X, dX) → (Y, dY ) is an isometry if

dY (φ(x1), φ(x2)) = dX(x1, x2), ∀x1, x2 ∈ X.

It is clear that an isometry is injective and continuous.

If there exists a bijective isometry φ : X → Y we say that X and Y are isometric or congruent.

22

Definition 2.21. Let (X, dX) be a metric space. A completion of (X, dX) is a complete metricspace (X, dX) with a dense subset X0 isometric to X.

(It is usual to identify X0 with X and view X as a subset of X.)

Example 2.6. R is a completion of Q.

Before stating the main theorem, we assemble some definitions and basic facts.

Lemma 2.6. Let X be a metric space, with metric dX .

|dX(p, q)− dX(x, y)| ≤ dX(p, x) + dX(q, y) ∀ p, q, x, y ∈ X.

Proof of Lemma 2.6 – See Exercise Sheet 4, Question 4.

Lemma 2.6 implies, in particular, that the metric dX is continuous as a map X × X → R (seeExercise Sheet 4, Question 4).

Corollary 2.5. Let X be a metric space with metric dX . If (xn) and (yn) are convergent sequencesin X with respective limits p ∈ X and q ∈ X then dX(xn, yn) → dX(p, q) as n→ ∞.

Lemma 2.7. Let X1 and X2 be metric spaces and assume that X2 is complete. Let X01 be a

dense subset of X1 and X02 a dense subset of X2. Any isometry X0

1 → X02 extends uniquely to an

isometry X1 → X2.

Proof of Lemma 2.7. Let d1 and d2 be the metrics on X1 and X2, respectively, and let φ0 :X0

1 → X02 be an isometry. Let x ∈ X1 be arbitrary. By denseness, there exists a sequence (xn)

in X01 converging to x. Since φ0 is an isometry, it follows that

(φ0(xn)

)is a Cauchy sequence in

the complete space X2 and so is convergent: denote its limit by φ(x) ∈ X2. If (yn) ⊂ X01 is also

a sequence converging to x, then d1(xn, yn) → 0 as n → ∞ and, since φ0 is an isometry, we haved2(φ

0(xn), φ0(yn)) → 0 as n → ∞. Therefore, φ0(yn) → φ(x) as n → ∞. Therefore, with each

x ∈ X1 we associate a unique φ(x) ∈ X2: in other words, the map X1 → X2, x 7→ φ(x) is welldefined and is clearly an extension of φ0 (that is, φ|X0

1= φ0). We proceed to show that φ is an

isometry. Let x, y ∈ X1 be arbitrary and let (xn), (yn) be sequences in X01 converging to x, y,

respectively. Invoking Corollary 2.5, we have

d2(φ(x), φ(y)) = limn→∞

d2(φ0(xn), φ

0(yn)) = limn→∞

d1(xn, yn) = d1(x, y)

and so φ is an isometry. It remains only to establish uniqueness. Let φ : X1 → X2 be an isometryextending φ0. Let x ∈ X1 be arbitrary and let (xn) be a sequence in X0

1 converging to x. Then,by continuity of φ, we may conclude that

φ(x) = limn→∞

φ(xn) = limn→∞

φ(xn) = φ(x),

and so φ = φ. 2

Let (X, dX) be a metric space and let Xs be the set of Cauchy sequences from X, that is,

Xs := {(xn) in X| (xn) is a Cauchy sequence}.

On Xs, define the relation ∼ as follows

(xn) ∼ (x′n) iff dX(xn, x′n) → 0 as n→ ∞ .

Exercise: Show that ∼ is an equivalence relation on Xs (Exercise Sheet 7, Question 1).

We say that sequences (xn) and (x′n) in Xs are equivalent if (xn) ∼ (x′n). Let X be the space ofequivalence classes of elements of Xs (the equivalence class of (xn) ∈ Xs is {(x′n)| (x′n) ∼ (xn)} =x ∈ X; any such sequence (x′n) is a representative of x).

Let x, y ∈ X be arbitrary. Let (xn) and (yn) be any representatives of x and y, respectively.

23

Lemma 2.8. limn→∞ dX(xn, yn) exists.

Proof. Exercise Sheet 7, Question 2.

Lemma 2.9. Let (xn) and (x′n) (respectively, (yn) and (y′n)) be any two representatives of x ∈ X(respectively, y ∈ Y ). Then

limn→∞

dX(xn, yn) = limn→∞

dX(x′n, y′n).

Proof. Exercise Sheet 7, Question 3.

In view of Lemmas 2.8 and 2.9, we see that

dX(x, y) := limn→∞

dX(xn, yn), where (xn) and (yn) are any representatives of x and y,

defines a map X × X → R.

Exercise: Show that dX satisfies the axioms of a metric. (Exercise Sheet 7, Question 4.)

We are now in a position to state and prove the main result of this section which asserts that everymetric space has a completion, unique up to isometries.

Theorem 2.14.(a) Any metric space has a completion.(b) Any two completions are isometric.

Proof of Theorem 2.14

(a) Let X = (X, dX) be a metric space. Let X = (X, dX) be the metric space of equivalence

classes of Cauchy sequences from X. Let X0 be the subset of X of equivalence classes which havea representative of the form (x, x, ...), x ∈ X and define the metric dX0 := dX |X0 . Consider themap

φ : (X0, dX0) → (X, dX), x 7→ x, where (x, x, ...) is a representative of x.

It is straightforward to verify that φ is a bijection. Moreover, for every x, y ∈ X0 (with represen-tatives (x, x, ...) and (y, y, ...)), we have

dX0(x, y) = dX(x, y) = dX(φ(x), φ(y))

and so X0 and X are isometric. We proceed to show that X0 is dense in X.

Let x ∈ X and ε > 0 be arbitrary. Let (xn) be a representative of x and choose N ∈ N be suchthat dX(xn, xm) < ε/2 for all n,m > N . Let y ∈ X0 be the equivalence class with representative(xN+1, xN+1, ...). Then

dX(x, y) = limn→∞

dX(xn, yn) = limn→∞

dX(xn, xN+1) ≤ ε/2 < ε ,

and so we may infer that X0 is dense in X.

It remains to show that (X, dX) is complete. Let (x(n)) be a Cauchy sequence in X. Since X0 is

dense in X, we may choose, for each n ∈ N, and element xn ∈ X such that dX(x(n), xn) < 1/n,

where xn ∈ X0 is the equivalence class with representative (xn, xn, ...). Let ε > 0 be arbitrary.Since (x(n)) is Cauchy, there exists N ∈ N such that

dX(x(n), x(m)) < ε/3 ∀ n,m > N.

Therefore, we have

dX(xn, xm) = dX(xn, xm) ≤ dX(xn, x(n)) + dX(x(n), x(m)) + dX(x(m), xm)

<1

n+ε

3+

1

m≤ ε ∀ n,m > max

{N ,

3

ε

},

24

and so (xn) is a Cauchy sequence in X. Now, let x be the element of X with representative(xn). Let ε > 0 be arbitrary. Since (xn) is Cauchy, there exists N ∈ N such that dX(xn, xm) =dX(xn, xm) < ε/2 for all n,m > N . For each m > N , we have

dX(x, xm) = limn→∞

dX(xn, xm) ≤ ε

2.

Therefore,

dX(x, x(m)) ≤ dX(x, xm) + dX(xm, x(m)) <

ε

2+

1

m≤ ε ∀ m > max

{N,

2

ε

}.

We may now conclude that dX(x, x(m)) → 0 as m → ∞ and so the Cauchy sequence (x(n)) in X

is convergent with limit x ∈ X.

(b) Let (X1, dX1) and (X2, dX2

) be two completions of (X, dX) with associated dense subsets X01 ,

X02 and bijective isometries φ1 : X

01 → X, φ2 : X

02 → X. Define ψ0 : X0

1 → X02 by ψ0 := φ−1

2 ◦ φ1.Clearly, ψ0 is bijective; moreover,

dX02(ψ0(x), ψ0(y)) = dX0

2(φ−1

2 (φ1(x)), φ−12 (φ1(y))) = dX(φ1(x), φ1(y)) = dX0

1(x, y)

and so ψ0 is an isometry. By Lemma 2.7, ψ0 extends uniquely to an isometry ψ : X1 → X2 and(ψ0)−1 : X0

2 → X01 extends uniquely to an isometry ψ : X2 → X1. Clearly,

(ψ ◦ψ)(x) =((ψ0)−1 ◦ψ0

)(x) = x ∀ x ∈ Xo

1 and (ψ ◦ ψ)(x) =(ψ0 ◦ (ψ0)−1

)(x) = x ∀ x ∈ X0

2 .

Therefore,

ψ ◦ ψ∣∣∣X0

1

= idX01

and ψ ◦ ψ∣∣∣X0

2

= idX02.

Let x ∈ X1 be arbitrary and let (xn) be a sequence in X01 with xn → x as n→ ∞. By continuity

of ψ and ψ, we have(ψ ◦ ψ)(x) = lim

n→∞(ψ ◦ ψ)(xn) = lim

n→∞xn = x.

Similarly, (ψ ◦ ψ)(x) = x for all x ∈ X2. We have now shown that

ψ ◦ ψ = idX1, ψ ◦ ψ = idX2

,

which implies that ψ a bijection, with inverse ψ−1 = ψ. Therefore, the completions (X1, dX1) and

(X2, dX2) are isometric. 2

25

Chapter 3

Hilbert spaces

Throughout, F = R or F = C.

Definition 3.1. An inner product (prehilbert) space is a vector space X over F equipped with afunction ⟨ · , · ⟩ : X ×X → F satisfying the following axioms:

• ⟨αx+ βy, z⟩ = α⟨x, z⟩+ β⟨y, z⟩ ∀ x, y, z ∈ X ∀ α, β ∈ F;

• ⟨x, y⟩ = ⟨y, x⟩ ∀ x, y ∈ X;

• ⟨x, x⟩ ≥ 0 ∀ x ∈ X, and ⟨x, x⟩ = 0 iff x = 0.

Here, z denotes the complex conjugate of z ∈ C. If F = R, then the complex conjugate in thesecond of the above axioms can be ignored. Also, if F = R then the above axioms imply that ⟨·, ·⟩is bilinear, that is, we also have

⟨x, αy + βz⟩ = α⟨x, y⟩+ β⟨x, z⟩ ∀x, y, z ∈ Z ∀α, β ∈ R.

If F = C, then

⟨x, αy + βz⟩ = ⟨αy + βz, x⟩ = α⟨y, x⟩+ β⟨z, x⟩ = α⟨x, y⟩+ β⟨x, z⟩ ∀x, y, z ∈ X ∀α, β ∈ C.

Theorem 3.1. Let X be an inner product space.

(a) |⟨x, y⟩|2 ≤ ⟨x, x⟩⟨y, y⟩ ∀ x, y ∈ X (the Cauchy–Schwarz inequality).

(b) ∥x∥ :=√⟨x, x⟩ defines a norm on X.

(c) ∥x+ y∥2 + ∥x− y∥2 = 2∥x∥2 + 2∥y∥2 ∀ x, y ∈ X (the parallelogram law).

(d) ⟨ · , · ⟩ : X ×X → R is continuous.

Proof of Theorem 3.1

(a) For y = 0 the assertion is true, since

⟨x, 0⟩ = ⟨x, z − z⟩ = ⟨x, z⟩ − ⟨x, z⟩ = 0 = ⟨x, x⟩⟨0, 0⟩ .

Assume y = 0. Let α ∈ F be a parameter. We have

0 ≤ ⟨x+ αy, x+ αy⟩ = ⟨x, x+ αy⟩+ α⟨y, x+ αy⟩ = ⟨x, x⟩+ α⟨x, y⟩+ α⟨y, x⟩+ |α|2⟨y, y⟩

Choose

α = −⟨x, y⟩⟨y, y⟩

= −⟨y, x⟩⟨y, y⟩

26

in which case, we have

0 ≤ ⟨x, x⟩ − |⟨x, y⟩|2

⟨y, y⟩− |⟨x, y⟩|2

⟨y, y⟩+

|⟨x, y⟩|2

⟨y, y⟩= ⟨x, x⟩ − |⟨x, y⟩|2

⟨y, y⟩.

whence the result.

(b) Clearly, ∥x∥ ≥ 0 for all x ∈ X and ∥x∥ = 0 if, and only if, x = 0. Also, ∥αx∥ = |α|∥x∥ for allx ∈ X and all α ∈ F. Furthermore,

∥x+ y∥2 = ⟨x+ y, x+ y⟩ = ⟨x, x, ⟩+ ⟨x, y⟩+ ⟨x, y⟩+ ⟨y, y⟩ = ∥x∥2 + 2Re⟨x, y⟩+ ∥y∥2

≤ ∥x∥2 + 2|⟨x, y⟩|+ ∥y∥2 ≤ ∥x∥2 + 2∥x∥ ∥y∥+ ∥y∥2 = (∥x∥+ ∥y∥)2 ∀ x, y ∈ X,

whence the triangle inequality.

(c) For all x, y ∈ X, we have

∥x+ y∥2 + ∥x− y∥2 = ∥x∥2 + 2Re⟨x, y⟩+ ∥y∥2 + ∥x∥2 − 2Re⟨x, y⟩+ ∥y∥2 = 2∥x∥2 + 2∥y∥2.

(d) Let (xn) and (yn) be convergent sequences in (X, ∥ · ∥) with respective limits x, y ∈ (X, ∥ · ∥).Then we have

⟨xn, yn⟩ − ⟨x, y⟩ = ⟨xn, yn⟩ − ⟨xn, y⟩+ ⟨xn, y⟩ − ⟨x, y⟩ = ⟨xn, yn − y⟩+ ⟨xn − x, y⟩ ,

whence|⟨xn, yn⟩ − ⟨x, y⟩| ≤ ∥xn∥ ∥yn − y∥+ ∥xn − x∥ ∥y∥ → 0 as n→ ∞.

2

Part (b) of Theorem 3.1 implies that an inner product space is a special case of a normed space.

3.1 Complete inner product spaces – Hilbert spaces

Complete inner product spaces are called Hilbert spaces.

David Hilbert (1862-1943)

Example 3.1 (Finite-dimensional unitary space).The space Cn is the vector space of n-tuples of complex numbers equipped with the inner product

⟨x, y⟩ :=n∑

j=1

xj yj , x = (x1, . . . , xn), y = (y1, . . . , yn).

and induced norm

∥x∥ =√⟨x, x⟩ =

√√√√ n∑j=1

|xj |2 .

27

Let (xm) =((x

(m)1 , ..., x

(m)n )

)be a Cauchy sequence in Cn. Let ε > 0 be arbitrary. Then there

exists N ∈ N such that

∥xm − xk∥2 =n∑

j=1

|x(m)j − x

(k)j |2 < ε2 ∀ m, k > N ,

whence,

|x(m)j − x

(k)j | < ε ∀ m, k > N, j = 1, ..., n,

and so

|Rex(m)j − Rex

(k)j | < ε & |Imx

(m)j − Imx

(k)j | < ε ∀ m, k > N, j = 1, ..., n,

Therefore, the sequences(Rex

(m)j

)and

(Imx

(m)j

), j = 1, ..., n, are Cauchy sequences in R and so,

by completeness of R, are convergent with limits aj , bj ∈ R, j = 1, ..., n. Writing

x =(a1 + ib1, . . . , an + ibn

)∈ Cn,

It follows that the sequence (xm) in Cn is convergent with limit x ∈ Cn. Therefore, Cn is completeand so is a Hilbert space.

Example 3.2 (Square-summable real sequences).Denote, by l2, the space of square-summable real sequences, that is, the space of sequences x = (xn)with the property that

∑∞n=1 x

2n <∞. With the obvious notions of addition, x+y = (xn)+(yn) :=

(xn + yn), and scalar multiplication, αx = α(xn) := (αxn), l2 is a real vector space. For every

x = (xn) ∈ l2 and y = (yn) ∈ l2, we have 2|xnyn| ≤ x2n + y2n for all n ∈ N and so

N∑n=1

|xnyn| ≤1

2

∞∑n=1

x2n +1

2

∞∑n=1

y2n ∀ N ∈ N

and so the series∑∞

n=1 xnyn is absolutely convergent. Therefore, for every x, y ∈ l2, the followingis well defined

⟨x, y⟩ :=∞∑

n=1

xnyn, x = (xn), y = (yn)

and the inner product axioms hold. We equip l2 with this inner product, which induces the normgiven by

∥x∥ =√⟨x, x⟩ =

√√√√ ∞∑n=1

x2n ,

and proceed to show that it is a complete inner product space.

Theorem 3.2. l2 is a Hilbert space.

Proof of Theorem 3.2 Let (x(n)) =((x

(n)k ))be a Cauchy sequence in l2. For every k ∈ N we

have |x(n)k − x(m)k | ≤ ∥x(n) − x(m)∥ and so (x

(n)k ) is Cauchy sequence in R, hence, convergent to

some xk ∈ R. Set x := (xk). We will prove that x ∈ l2 and that ∥x(n) − x∥ → 0 as n→ ∞.

Let ε > 0 be arbitrary. Since (x(n)) is a Cauchy sequence, there exists N ∈ N such that ∥x(n) −x(m)∥ < ε/2 for all n,m > N . Therefore, we have√√√√ k∑

j=1

(x(n)j − x

(m)j

)2<ε

2, ∀ k ∈ N, ∀ n,m > N.

Passing to the limit as m→ ∞, we obtain√√√√ k∑j=1

(x(n)j − xj

)2 ≤ ε

2, ∀ k ∈ N, ∀ n > N .

28

Passing to the limit as k → ∞, yields

∥x(n) − x∥ ≤ ε

2< ε, ∀ n > N.

Therefore, ∥x(n) − x∥ → 0 as n→ ∞. Moreover, since

∞ > ∥x(N+1)∥+ ∥x(N+1) − x∥ ≥ ∥x(N+1) + x− x(N+1)∥ = ∥x∥,

it follows that x ∈ l2 2

The space l2 is the simplest infinite-dimensional example of a Hilbert space.

3.2 Convexity and the projection lemma

Definition 3.2. A subset K of a vector space is said to be convex if

x, y ∈ K =⇒ (1− t)x+ ty ∈ K, ∀ t ∈ [0, 1].

In words: K contains the line segment joining any two of its points.

b

b

xy

K

Definition 3.3. In a metric space (with metric d), the distance from a point p to a set K is

dist(p,K) := infq∈K

d(p, q)

The following is the Projection Lemma.

Lemma 3.1. Let K be a closed, convex, non-empty subset of a Hilbert space H. Then, for eachp ∈ H, there exists unique q ∈ K such that ∥p− q∥ = dist(p,K).

bbq p

K

Proof of Lemma 3.1

Existence. For brevity, write ∆ := dist(p,K). There exists a sequence (qn) in K such that∥qn − p∥ → ∆. Now invoke the parallelogram law (Theorem 3.1(c), with x = qn − p, y = qm − p )to obtain

∥qn + qm − 2p∥2 + ∥qn − qm∥2 = 2∥qn − p∥2 + 2∥qm − p∥2 ∀n,m ∈ N.

By convexity of K, (qn + qm)/2 is in K and so, by the definition of ∆, we have

∥qn + qm − 2p∥2 = 4

∥∥∥∥qn + qm2

− p

∥∥∥∥2 ≥ 4∆2 ∀n,m ∈ N .

We may now infer that

∥qn − qm∥2 ≤ 2∥qn − p∥2 + 2∥qm − p∥2 − 4∆2 ∀n,m ∈ N

29

and so, for each ε > 0, there exists N ∈ N such that

∥qn − qm∥ < ε ∀n,m > N.

Therefore, (qn) is a Cauchy sequence in K ⊂ H and so converges to some q ∈ H. By closedness ofK, we have q ∈ K and, by continuity of the norm, ∥q − p∥ = ∆.

Uniqueness. Let q, q′ ∈ K be such that ∥p−q∥ = ∥p−q′∥ = ∆. By the parallelogram law (Theorem3.1(c), with x = q − p, y = q′ − p ), we have

∥q + q′ − 2p∥2 + ∥q − q′∥2 = 2∥q − p∥2 + 2∥q′ − p∥2 = 4∆2.

By convexity of K and the definition of ∆,

∥q + q′ − 2p∥2 = 4

∥∥∥∥q + q′

2− p

∥∥∥∥2 ≥ 4∆2

Therefore,∥q − q′∥2 ≤ 4∆2 − 4∆2 = 0

which implies q = q′. 2

3.3 Orthogonality and the projection theorem

Definition 3.4.

(i) Elements x and y of an inner product space are said to be perpendicular or orthogonal if⟨x, y⟩ = 0 (in which case, we write x ⊥ y).

(ii) Let Y , Z be subspaces of a vector space X. We say that X is a direct sum of Y and Z andwrite X = Y ⊕ Z if every x ∈ X can be written in a unique way as

x = y + z, y ∈ Y, z ∈ Z.

(iii) For a subspace Y of an inner product space X the orthogonal complement Y ⊥ is the subspacegiven by

Y ⊥ := {x ∈ X| x ⊥ y, ∀y ∈ Y }.The following is the Projection Theorem.

Theorem 3.3. Let Y be a closed vector subspace of a Hilbert space H. Then H = Y ⊕ Y ⊥.

Proof of Theorem 3.3 Let x ∈ H be arbitrary. Noting that Y is closed, convex, and non-empty(0 ∈ Y ), we may invoke the Projection Lemma 3.1 to conclude the existence of a unique pointy ∈ Y nearest to x. Define z := x − y. We will show that z ∈ Y ⊥. Suppose that z ∈ Y ⊥. Thenthere exists v ∈ Y such that ⟨z, v⟩ = 0. For each α ∈ F, we have y + αv ∈ Y and so

∥z∥2 = (dist(x, Y ))2 ≤ ∥x− (y + αv)∥2 = ∥z − αv∥2 = ∥z∥2 − α⟨v, z⟩ − α⟨z, v⟩+ |α|2∥v∥2

= ∥z∥2 + ∥v∥2∣∣α− (⟨z, v⟩/∥v∥2)

∣∣2 − (|⟨z, v⟩|/∥v∥)2.Setting α = ⟨z, v⟩/∥v∥2, yields the contradiction

0 ≤ −|⟨z, v⟩|2

∥v∥2< 0.

Therefore, there exist y ∈ Y and z ∈ Y ⊥ such that x = y + z.

Let y′ ∈ Y and z′ ∈ Y ⊥ be such that x = y′ + z′. Then

y − y′ = z′ − z, y − y′ ∈ Y, z′ − z ∈ Y ⊥ .

Therefore,∥y − y′∥2 = ⟨y − y′, y − y′⟩ = ⟨y − y′, z′ − z⟩ = 0 ,

and so y = y′ which, in turn, implies z = z′. 2

30

3.4 Orthonormal basis

Let {xα}α∈I be an indexed set in an inner product space X. The elements of this set are assumedto be indexed without repetition, that is, xα = xβ whenever α = β. Here, the set I is the index set.Possible index sets are: I = {1, . . . , n} (finite), or I = N (countable), or I may be uncountable.

Definition 3.5.A set {xα}α∈I is said to be linearly independent if, for every finite subset {xα1 , . . . , xαn},

n∑j=1

cjxαj = 0 =⇒ c1 = · · · = cn = 0.

Definition 3.6. The space of all finite linear combinations of elements from {xα}α∈I is called thespan of this set and denoted span{xα}α∈I .

Definition 3.7. A set {xα}α∈I is said to be complete in X if span{xα}α∈I is dense in X.

Definition 3.8. A set {xα}α∈I is said to be orthogonal in X if

xα ⊥ xβ , ∀ α, β ∈ I, α = β.

Definition 3.9. A set {xα}α∈I is said to be orthonormal in X if it is orthogonal in X and

⟨xα, xα⟩ = 1, ∀ α ∈ I.

Let {xα}α∈I be orthonormal in X and let {xα1 , ..., xαn) be any finite subset. Assume c1xα1 + · · ·+cnxαn = 0 for some scalars c1, ..., cn. Taking inner product with xαj yields cj = 0, j = 1, ..., n.This proves the following:

Lemma 3.2. An orthonormal set in an inner product space is linearly independent.

Given a countable linearly independent set {x1, x2, ...} in X, there is a standard inductive pro-cedure, called the Gram-Schmidt process, for generating an orthonormal set {y1, y2, ...} with theproperty that span{x1, ..., xn} = span{y1, ..., yn} for all n ∈ N. The initial step in the processis to set y1 := x1/∥x1∥. The inductive step is as follows: assume given an orthonormal set{y1, ..., yn} with span{x1, ..., xn} = span{y1, ..., yn}, then define vn+1 := xn+1 −

∑nj=1⟨xn+1, yj⟩yj

and, observing that vn+1 = 0 because xn+1 is not in span{x1, ..., xn} = span{y1, ..., yn}, we setyn+1 := vn+1/∥vn+1∥; noting that ⟨vn+1, yj⟩ = 0 for j = 1, ...n, we see that {y1, ..., yn+1} is anorthonormal set and it is clear that span{y1, ..., yn+1} = span{x1, ..., xn+1}; this completes theinductive step.

We summarize the above in the following theorem.

Theorem 3.4. Let {x1, x2, . . .} be a countable linearly independent set in an inner product space.Then there exists an orthonormal set {y1, y2, . . .} such that, for all n ∈ N,

span{x1, ..., xn} = span{y1, ..., yn}.

In particular, the sets {x1, x2, . . .} and {y1, y2, . . .} have the same spans, and so one set is completeif, and only if, the other set is complete.

Definition 3.10. A complete orthonormal set in an inner product space X is called an orthonormalbasis for X.

3.5 Approximation in Hilbert spaces

In the remainder of this chapter, we will use the Projection Lemma 3.1 and the Projection Theorem3.3 in studying the approximation problem in Hilbert spaces.

31

Let {x1, . . . , xn} be a finite orthonormal set in the inner product space X. Consider the Gaussapproximation problem: given x ∈ X, find c1, . . . , cn ∈ F such that∥∥∥∥∥x−

n∑k=1

ckxk

∥∥∥∥∥is minimal.

Johann Carl Friedrich Gauss (1777-1855)

Theorem 3.5. In a Hilbert space, the Gauss approximation problem has the unique solution ck =⟨x, xk⟩, k = 1, . . . , n. The element z := x−

∑nk=1 ckxk is orthogonal to span{x1, . . . , xn}. Moreover,

∥z∥2 = ∥x∥2 −n∑

k=1

|ck|2 (Bessel’s equality) (3.1)

andn∑

k=1

|ck|2 ≤ ∥x∥2 (Bessel’s inequality). (3.2)

Proof of Theorem 3.5 Let H be a Hilbert space and let {x1, . . . , xn} be a finite orthonormal setin H. Let x ∈ H be arbitrary. The subspace Y := span{x1, . . . , xn} is finite dimensional and sois complete. Let (ym) be a convergent sequence in Y with limit y ∈ H. Then (ym) is Cauchy inH and so is Cauchy in Y . By completeness of Y , it follows that y ∈ Y and so Y is closed. By theProjection Theorem 3.3, there is a unique y ∈ Y and a unique z ∈ Y ⊥ such that x = y + z and,furthermore, ∥x−y∥ is minimal. Since y ∈ Y , we have y = c1x1+ · · ·+ cnxn for some c1, ..., cn ∈ Fand, since z ∈ Y ⊥, it follows that

⟨x, xk⟩ = ⟨y + z, xk⟩ = ⟨y, xk⟩+ ⟨z, xk⟩ = ⟨y, xk⟩ = ck, k = 1, ..., n.

Moreover,

∥x∥2 = ∥y + z∥2 = ∥y∥2 + ∥z∥2 = ∥z∥2 +n∑

k=1

|ck|2 ,

whence Bessel’s equality and Bessel’s inequality. 2

32