Mathematics for Control Theory

IntroductionLogic and Sets

Relations and FunctionsMathematical Induction

Hanz Richter

Mechanical Engineering Department

Cleveland State University

Course rationale and aims

2 / 40

The course is primarily directed at PhD students specializing in control andrelated areas (dynamic systems, mechatronics, optimization). The decision tooffer the course was based on:

1. Even for students focusing more on control applications (as opposed todevelopment of new algorithms and analysis methods), the controlsliterature stands out from other engineering fields for its widespread

reliance on advanced mathematics (topics not normally covered in thetypical graduate curriculum). PhD students in controls have much morepressure to catch up with the language and basic techniques used byleading researchers in the field.

Course rationale and aims...

3 / 40

2 The course prepares you to read: Frequently, the formality of themathematical language used in the literature obstructs understanding ofthe key concepts presented in the paper. Familiarity with these concepts,even at a basic level, can make a big difference.

3 The course prepares you to write: Suppose you found an interestingpattern or result in your research, mostly by simulations. Formalizing theresult in mathematical terms and offering a proof or a rational conjectureis key to publishing your work. The course will give you the opportunity topractice writing mathematical content for an engineering audience.

Course rationale and aims...

4 / 40

4 The topics themselves have been carefully chosen to bridge key topics insystems and control and to give you a good foundation for futureindependent study.

5 These slides are only the rythm of the piece. The accompanying bookchapters assigned to you for reading are the rest of the music. If you onlylisten to the drum section, you will miss most of the piece. You must readalong and do recommended exercises and homework. Maybe 1 hour a dayof exclusive dedication to reading will be sufficient, plus time forhomework. You will be asked to explain your solutions or concepts to theclass as one way to earn points towards your grade.

Course organization

5 / 40

The course is roughly divided in 3:

1. Foundations: We cover essentials such as set theory, functions, metricspaces, convergence.

2. Analysis: Includes: Normed spaces and inner product spaces, Lp spaces.

3. Geometric methods: Includes: Differential equations on manifolds, Liegroups, applications to nonlinear control and dynamics.

Several important papers in controls and/or dynamics will be used as partialprojects. A paper will be chosen by students for the final project, which couldbe a draft paper written by the student, with significant mathematical content(instructor will approve).

Grading will be through progressive point accumulation, up to 80. The finalproject will be worth 20 points.

Reading materials

6 / 40

We will use multiple book chapters, sometimes with overlapping material.Book chapters and papers will be made available at the beginning of eachsection or subsection.

For this section (introduction, logic, sets, relations, functions, mathematicalinduction), we will use:

■ Hofstadter, Douglas R. (1999) [1979], Gödel, Escher, Bach: An Eternal Golden

Braid, Basic Books, ISBN 0-465-02656-7 (provided material from chapter 1).

■ Landin, Joseph, (1989) [1969], An Introduction to Algebraic Structures, DoverPublications, ISBN 0-486-65940-2 (provided material from chapter 1, pp. 1-28)

■ Anthony N. Michel and Charles J. Herget (2007) [1981], Algebra and Analysis for

Engineers and Scientists, Birkhäuser, e-ISBN-13: 978-0-8176-4707-0 (chapter 1)

■ Kenneth R. Davidson and Allan P. Donsig [2002], Real Analysis with Real

Applications, Prentice-Hall, ISBN 0-13-041647-9 (chapter 1, except 1.3 LinearAlgebra and 1.4 Calculus).

Recommended: use LATEX

7 / 40

Incorporating mathematical notation into a document shouldn’t be anightmare. The same goes for scientific documents containing numberedbibliographic references, figures, and tables.

1. LATEXsources for sample documents will be made available.

2. LATEX- oriented document sharing online tools are available for free. Youmay submit assignments simply by sharing them with the instructor usingOverleaf.

3. Online resources for learning LATEXand resolving issues are plentiful.

4. Most students gain working knowledge of LATEXin a couple of weeks.

5. LATEXis highly recommended, not mandatory.

Logic: the rules of the game in mathematics

8 / 40

The ideas presented below are inspired by reading the book “Gödel, Escher,Bach: An Eternal Golden Braid”, by computer scientist Douglas Hofstadter.We will refer to this book as GEB.

■ Mathematics advances by generating new “truths” on the basis ofpreviously known ones and following rules of deduction: logic.

■ This process is not unlike a game, where each new position of the pieces(a “truth”) is generated from others by followig the rules.

■ The process starts with some accepted truths (the initial layout of pieces isalways the same and universally accepted). These are called axioms orpostulates. One axiom of Euclidean geometry (one of Euclid’s fivepostulates) is “given a straight line and a point not belonging to it, there’sexactly one line passing through the point which does not intersect thefirst line”.

■ Axioms are not proven. Several axioms may be used to obtain new truthsby a process of proof. Proof may be reduced to a series of symbolpermutations per the rules of the game (see GEB). In other words, thegeneration of truth may be mechanized.

Hierarchy used in math writing

9 / 40

These types of “truth” require a proof. Labeling results as proposition, lemmaor theorem is a personal decision, but you will observe the following in theliterature:

■ Truths are labeled “proposition”, “lemma”, “theorem” according to theirgenerality and importance.

■ A proposition is a minor result, an accessory to build an upcoming moregeneral result.

■ A lemma is more far-reaching and significant than a proposition, but stillan accessory to build upcoming theorems. Sometimes a lemma ends uphaving the same reach as theorems: the Kalman-Yakubovich lemma ofpassivity theory in controls. The matrix inversion lemma.

■ A theorem is the type of truth at the top of the hierarchy in a theory. TheCayley-Hamilton Theorem has large importance in Linear Algebra.

■ A corollary is an immediate consequence of a theorem. It’s proven simplyby invoking the theorem (applying it to a special situation, for example).

Relative truth, incomplete truth, vacuous truth

10 / 40

Observe the following:

1. Axioms can be overriden, to create a new theory. A good example ishyperbolic geometry (due to Lobachevsky). Euclid’s parallel postulate isreplaced with: “given a line and a point outside it, there are at least twodistinct lines passing through the point which do not intersect the firstline”. This results in a new theory, essential to Einstein’s relativity.

2. A consistent system with axioms and rules of deduction meant to derivetruths about the natural numbers will never reach them all. Kurt Gödelproved this for arithmetic in 1931: there are truths that can never bereached by using the axioms and the rules, no matter the procedure.These are called undecidable propositions. Gödel’s result is known asIncompleteness Theorem.

3. Reading: Pages 33-41 in GEB, try the “MU puzzle” provided there.

The following proposition is vacuously true: If x ∈ R is such that x2 = −1,then x = 3.5.This kind of proposition establishes that the members of the empty set satisfyany arbitrary property. Vacuous propositions may arise in the proof process.

Logical Propositions

11 / 40

A proposition in logics is a statement that can be evaluated as true or false:

All cats have at most 4 legs

There are exactly 24 primes between 1 and 100

Statements whose truth cannot be evaluated due to subjective factors orambiguity are not propositions:

Let’s eat snow

This statement is false

Conditional truth: Mathematical results (propositions, lemmas, theorems)are often implications, one-sided or two-sided:

Suppose A. Then B. (A⇒ B)

Suppose A. Then B if and only if C. (A⇒ (B ⇐⇒ C))

Truth of Implication

12 / 40

The validity of a statement like A⇒ B is independent of whether A is true ornot. What must be done is to establish the truth of the implication. Informally:

If a tire becomes deflated, contact between it and the surface occurs at more

than one point.

The implication is true, we don’t have to think about the specific condition ofthe tire. In a statement like (A⇒ (B ⇐⇒ C)), we must establish that if A istrue, then B and C are equivalent (B will occur when and only when Coccurs).

Finally note that one-sided implications are not reversible. If contact with thesurface occurs at more than one point, we can’t conclude that the tire isdeflated. It could have been placed on a V-shaped surface.

Modus Ponens and Truth Tables

13 / 40

Sometimes we do want to go farther than just the truth of the implication. Wecan establish the truth of the actual propositions. Modus ponens is one suchscheme:A⇒ BA is trueThen B must be true.

A truth table maps out all outcomes of assumed truths in a proposition. Thetruth table for A⇒ B is:

A B A ⇒ B

T T T

T F F

F T T

F F T

Clearly, the implication is false (invalid) when B fails to occur when A hashappened. Think about the interpretation of this table for the tire-surfaceexample.

Logic Operators

14 / 40

The following are used to construct statements:

■ AND, symbol ∧

■ OR, symbol ∨

■ NOT, symbol ∼

■ XOR (exclusive OR), symbol ⊻

As an exercise, review the truth tables of all these.

Logic Quantifiers

15 / 40

Quantifiers are used to make statements that apply to certain elements in sets.We use:

■ (There) exists (a): ∃ (this means that at least one element exists). This isthe existential quantifier.

■ There is no : ∄

■ There exists only one: ∃!

■ For all: ∀. This is the universal quantifier

■ For some (no symbol): It’s used to say there is some element (orelements) for which a property holds. It’s pretty much the same as ∃.

Logical equivalence We will say that two propositions are equivalent if theyyield the same truth table. For instance p⇒ q ≡∼ p ∨ q

Examples

16 / 40

1. A tautology is a proposition which always evaluates to true regardless ofthe truth values of its variables. Determine if the following is a tautology (byinspection, then verify manually or with an online truth table generator:http://mrieppel.net/prog/truthtable.html

(((p ∨ q) ⇒ r) ∧ q) ⇒ r

2. Is the following proposition true? (Q denotes the rationals)

∀x ∈ R, ∃! y ∈ R s.t. x 6= 0 ⇒ y/x ∈ Q

3.Prove the following DeMorgan’s laws by showing truth table equality:

∼ (p ∧ q) ≡∼ p∨ ∼ q

∼ (p ∨ q) ≡∼ p∧ ∼ q

4. What is ∼ ∀?

Sets and Notation

17 / 40

We define a set as an unambiguous collection of objects, called elements ofthe set. A set needs to satisfy:

Given an arbitrary element, exactly one of two possibilities must hold:

1. The object belongs to the set

2. The object does not belong to the set

Can we unambiguously define the set of people who will buy a new car thisyear?

Explicit and Implicit Set Descriptions

18 / 40

An explicit description lists the elements; an implicit description summarizesthe rules for belonging to the set.For example, these descriptions correspond to the same set:

A = {0, 1, 1, 2, 3, 5, 8}

A = {x : x ≤ 5 and x is a term of the Fibonacci series}

Give implicit descriptions of the following sets:

A =

{

1,1

2,1

4,1

8....

}

A = {2, 3, 5, 7, 11}

Set Inclusion and Equality

19 / 40

■ Set A is contained in B (notation: A ⊂ B) if the following implication istrue:

x ∈ A⇒ x ∈ B

■ Set A is the same as B (notation: A = B) if the following doubleimplication is true:

x ∈ A⇒ x ∈ B and y ∈ B ⇒ y ∈ A

■ Frequently, we must show that two sets are the same. This can be done byseparately proving each inclusion.

■ We can also use known relationships of set algebra.

Exercise: Show by double inclusion that the following sets are equal:

A = {n : n = 4k + 1 for some k ∈ Z}

B = {n : n = 4k − 3 for some k ∈ Z}

Empty Set and Complement

20 / 40

■ The empty set, denoted ∅, is defined as the set without elements.

■ For any set A, ∅ ⊂ A.

■ Let X be a nonempty set such that A ⊂ X. We use X in context todefine complement.

■ The complement of A is denoted A′:

A′ = {x ∈ X : x /∈ A}

Show by double inclusion:

■ (A′)′ = A

■ A = B if and only if A′ = B′.

Practice rigorous and formal writing with these very simple proofs.

Binary Operations with Sets

21 / 40

Let A and B be subsets of X

■ Union: A ∪B = {x ∈ X : x ∈ A or x ∈ B}

■ Intersection: A ∩B = {x ∈ X : x ∈ A and x ∈ B}

■ Difference: A−B = {x ∈ X : x ∈ A and x /∈ B}

More definitions:

■ A set with a natural number of elements is finite. A set with one elementis a singleton.

■ Is A a singleton or an empty set? : A = {∅}

■ Let A be any set. The power set of A is: P(A) = {B : B ⊂ A}

If A is finite with n elements, how many elements does P(A) have?

Some Set Identities

22 / 40

Let A and B be subsets of X. Then the following holds:

■ A ∩B = B ∩A

■ A ∪B = B ∪A

■ A ∩∅ = ∅

■ A ∪∅ = A

■ A ∩A = ∅

■ A ∪A = A

■ A ∪A′ = X

■ A ∩A′ = ∅

■ A ∩B ⊂ A

■ A ∩B = A iff A ⊂ B

■ A ⊂ A ∪B

■ A = A ∪B iff B ⊂ A

■ A ∩ (B ∩ C) = (A ∩B) ∩ C

■ A ∪ (B ∪ C) = (A ∪B) ∪ C

■ A∩(B∪C) = (A∩B)∪(A∩C)

■ (A∩B)∪C = (A∪B)∩(B∪C)

Indexed Intersections and Unions

23 / 40

Let A1, A2, ....Ai, ... be a collection of sets indexed by i, where i belongs to asubset I of the natural numbers.Then:

⋃

i∈I

Ai = {x ∈ X : x ∈ Ai for some i ∈ I}

⋂

i∈I

Ai = {x ∈ X : x ∈ Ai ∀ i ∈ I}

DeMorgan’s Laws:

[

⋃

i∈I

Ai

]′

=⋂

i∈I

A′i

[

⋂

i∈I

Ai

]′

=⋃

i∈I

A′i

Homework: Write your own proof to one of the DeMorgan’s identities (0.5 pt)above and prepare yourself to explain it to the class (1 pt)

Ordered Pairs and Cartesian Product

24 / 40

Let A and B be two sets. An ordered pair (a, b) is formed with an elementa ∈ A and an element b ∈ B. The order is important, so:

(a, b) = (c, d) iff a = c and b = d

The Cartesian product A×B is defined as the set of all ordered pairs formedwith elements from A and B:

A×B = {(a, b) : a ∈ A, b ∈ B}

Ordered n-tuples and products can also be considered:

(x1, x2, ..xn)

andX1 ×X2 × ...Xn

Note that, in general, A×B 6= B ×A.

Functions (Mappings)

25 / 40

Given to nonempty sets X and Y , a function f : X → Y is a subset of X × Ysuch that (x, y1) = (x, y2) ⇒ y1 = y2. We say that f maps X into Y . X isthe domain of f , while the set of y s.t. (x, y) ∈ f is the range, R(f).Functions can be:

■ Surjective: R(f) = Y (also called onto Y )

■ If y1 = y2 ⇒ (x1, y1) = (x2, y2) for all (x1, y1), (x2, y2) ∈ f , the functionis injective or one-to-one

■ If f is one-to-one and onto, it is bijective

You should make a clear distinction between a function and its value at a givenelement of the domain. sin is a function, sin(x) for some real number x is areal number.

Inverse Function

26 / 40

Theorem: (see proof in Michel and Herget): Let f : X → Y . Consider thefollowing subset of R(f)×X:

g = {(y, x) ∈ R(f)×X : (x, y) ∈ f}

Then g is a function from R(f) into X if and only if f is injective.

In other words, f must be injective for an inverse (f−1 = g) to exist.

Composition of Functions

27 / 40

Let X, Y and Z be nonempty sets and let f : X → Y and g : Y → Z befunctions. The set

{(x, z) ∈ X × Z : z = g(f(x))}

is a function from X into Z, denoted g ◦ f . It can be shown that g ◦ f is ontoif g and f are themselves onto. Moreover, g ◦ f is 1-1 if g and f themselvesare 1-1. In other words, onto and 1-1 are preserved by composition.

Further, if f and g are bijective, then:

(g ◦ f)−1 = (f−1 ◦ g−1)

Homework (0.5 pt): The above formula would fail without the “onto”requirement. Give a counter-example using injective functions which are notonto.

Equivalence Relations

28 / 40

A relation is more general than a function. Given sets X and Y , a relationfrom X to Y is simply a subset of X × Y . A relation from X to X is called arelation on X

Let ρ be a relation on X. If (x, y) ∈ ρ we write xρy (“x is related to y”). Anequivalence relation on X satisfies:

1. xρx ∀x ∈ X (reflexive)

2. xρy ⇒ yρx (symmetric)

3. xρy and yρz ⇒ xρz (transitive)

Define ρ on the set of all humans that exist now or ever existed as ”having thesame mother”. Is ρ an equivalence relation?

Equivalence Relations

29 / 40

Let A 6= ∅. A partition P of set A is a collection of subsets S ⊂ A satisfying:

1. If S ∈ P , then S 6= ∅

2. If S ∈ P and T ∈ P , then either S = T or S ∩ T = ∅

3.⋃

S∈P S = A

In rough words, a partition is made up of non-overlapping sets which cover A.(Warning: A separate mathematical definition exists for “covering”).

The subsets S are called equivalence classes of the partition. There’s animportant connection between partitions and equivalence classes: Elements ofA can be related by “belonging to the same S”. This is formalized next.

Equivalence Relations and Partitions

30 / 40

First, we establish that a partition induces an equivalence relation:

Theorem (Landin Th. 8) Let P be a partition of A. Define a relation ρ on A asxρy if ∃S ∈ P such that x ∈ S and y ∈ S. Then ρ is an equivalence relation.

Now we establish the opposite:

Theorem (Landin Th. 9) Let ρ be an equivalence relation on the set A. Foreach x ∈ A let

Sx = {y : y ∈ A and yρx}

LetP (ρ) = {S : S = Sx for some x ∈ A}

Then P (ρ) is a partition of A.

Following both proofs in Landin to full understanding is highly suggested.

More on Equivalence Relations and Partitions

31 / 40

■ Start with a nonempty set A, define some partition P . An equivalencerelation arises, ρ(P ). In turn, ρ induces a partition P ′(ρ(P )). We end upwith the initial partition: P ′ = P . The same happens if we start with anequivalence relation, induce a partition and then again an equivalencerelation.

■ If ρ1 and ρ2 are equivalence relations on A, ρ1 = ρ2 ⇐⇒ P (ρ1) = P (ρ2).

■ If P1 and P2 are partitions of A, P1 = P2 iff the equivalence relations theyinduce are equal.

Because of these results, there’s a unique P to a given ρ on A, called “thepartition determined by ρ. Viceversa, there’s a unique ρ to a given P of A,called “the equivalence relation determined by P .

Three proof techniques

32 / 40

Many proofs are conducted by one or more of the following methods:

1. Proof by exhaustion: All cases are covered, and the proof is direct.

2. Proof by contradiction: In a statement of the type A⇒ B, we assume Aand then we incorrectly assume ∼ B. We then try to uncover acontradiction. This proves that ∼ B is impossible whenever A occurs.Then the implication must be true and the proof is concluded. The sameidea can be used for double implications and for statements involvingquantifiers. For example, if we say A holds ∀x, the negative assumption is∃x s.t ∼ A holds.

3. Proof by induction: applies to properties stated for elements of countablesets (more next).

Mathematical Induction Principle

33 / 40

There is a formal basis for this principle (hereditary sets). See Landin if youwish to see details.

A countable set is one where a bijection exists between its elements and theset of natural numbers.

The induction principle can be used to show that certain property holds for allelements of the countable set. Let xn denote an element of the set, wheren ∈ N.

1. Prove that the property holds for n = 0 (index shifting may be necessaryin some cases)

2. Prove the following implication:If the property holds for n, then if holds for n+ 1.

The left-hand side of the implication is called inductive hypothesis.

Example: Simple proof by induction

34 / 40

Proposition: All integers whose last digit is 0 or 5 are divisible by 5.

Proof: Without loss of generality (w.l.o.g.) assume that the integer isnon-negative. If the integer is zero, there’s nothing to prove. A positiveinteger ending in a0 has the form

n∑

i=0

10iai

for some constants a0, a1..an ∈ N which are not all zero. Proceed by induction.For n = 0, a0 is either 0 or 5, which is clearly divisible by 5. Now assume that

n∑

i=0

10iai

is divisible by 5.

Example...

35 / 40

We must show thatn+1∑

i=0

10iai

is divisible by 5. But:

n+1∑

i=0

10iai =n∑

i=0

10iai + 10n+1an+1

The first term is divisible by 5 (inductive hypothesis). Further, the second termis divisible by 5 because an+1 ∈ N and 10n+1an+1/5 = 2× 10nan+1 ∈ N. �.

Note that because of the bijection of the set 0, 5, 10, 15, 20... with N, we arejust proving a property of the natural numbers. This is the case in any proofby induction!

Equivalence between logic and sets

36 / 40

Set operations (union, intersection, complement..etc.) and logic operations(OR, AND, NOT..., etc) are equivalent in the following sense:

Suppose ϕ(x) means “property ϕ applies to x, and similarly for ψ(x). Define:

A = {x | ϕ(x)} and B = {x | ψ(x)}

Then

A ∪B = {x | x ∈ A ∨ x ∈ B} = {x | ϕ(x) ∨ ψ(x)} = {x |(

ϕ ∨ ψ)

(x)}

Likewise:A ∩B = {x |

(

ϕ ∧ ψ)

(x)}

A′ = {x |∼ ϕ(x)}

A ⊂ B = {x |(

ϕ⇒ ψ)

(x)}

The last formula is interpreted as: “elements for which property ψ impliesproperty ϕ”, or “property ϕ holds whenever ψ holds”, which is clearly aninclusion.

Ordered sets, sup and inf

37 / 40

The familiar relations >, = and < defined for the real numbers fit thedefinition of “relation” given earlier (are they equivalence relations?).

The real numbers are totally ordered, that is, any two reals satisfy exactly oneof the above relations.

■ A subset S of R has an upper-bound if ∃b s.t. x ≤ b ∀x ∈ S.

■ A subset S of R has an lower-bound if ∃l s.t. x ≥ l ∀x ∈ S.

■ b is the supremum of S if no number smaller than b is an upper bound forS (b is the lowest upper bound). Notation: b = sup S.

■ l is the infimum of S if no number greater than l is a lower bound for S (bis the greatest lower bound). Notation: l = inf S.

■ The completeness axiom states that every bounded nonempty subset of Rmust have a supremum.

Examples

38 / 40

Let x(t) be the unique solution to the differential equation

x+ x = 1

with initial condition x(0) = 0. Let X = {x(t), t ≥ 0}. What is sup X? Is supX an element of X?

The infinity norm of a transfer matrix such that Y (s) = G(s)U(s) is given by:

||G||∞ = supw∈R

σ[G(jw)]

where σ denotes singular value.

For the SISO case ||G||∞ is simply the peak frequency response.

sup and inf for real sets

39 / 40

Given a set A ∈ R, sup A = a if and only if:

1. a is an upper bound of A

2. For all ǫ > 0, ∃s ∈ A s.t. s > a− ǫ.

Similarly, inf A = b if and only if:

1. b is a lower bound of A

2. For all ǫ > 0, ∃s ∈ A s.t. s < b+ ǫ.

When a ∈ A or b ∈ A we speak of maximum and minimum, respectively.Proofs involving sup and inf may require direct use of the definitions or maytake advantage of useful identities (next).

Properties of sup and inf for real sets

40 / 40

Let A and B be nonempty sets of real numbers and let c be a real constant.Define:

A+B = {a+ b : a ∈ A, b ∈ B}

cA = {ca : a ∈ A}

Then:

1. A ⊂ B =⇒ (sup A ≤ sup B and inf A ≥ inf B)

2. A 6= ∅ =⇒ inf A ≤ sup A

3. If A 6= ∅ and B 6= ∅ then:

inf (A+B) = inf A+ inf B

sup (A+B) = sup A+ sup B

4. inf (cA) = c inf A if c ≥ 0 and inf (cA) = c sup A if c < 0

5. sup (cA) = c inf A if c < 0 and sup (cA) = c sup A if c ≥ 0