MATH-1030Matrix Theory & Linear Algebra I
Lin Jiu
Dalhousie University
May 7th, 2019
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 7th, 2019 1 / 14
Syllabus
1 1st assignment of today: read the syllabus posted on Brightspace.Lin Jiu, Chase 307, [email protected] Office hours: Mondays and Tuesdays 12:00--1:30 PM or by appointmentOffice hours THIS WEEK: Wednesday (May 8th) and Thursday (9th) 12:00–1:30 PMLearning Centre: Chase 119. Tutors are available Monday–Friday 1–5pmHomework (10%), Classwork (10%), Midterm I (20%), Midterm II (20%), Final Exam (40%)
You need to pass the final (50% or higher) to pass the course.Midterm I: May 21, Midterm II: June 6th, 6:00-7:30 PM, Dunn 117Final Exam: June 20, 6:00-9:00 PM, Dunn 117Homework on BrightSpace (WeBWork) (five problem sets with 2 points for each)Classwork: on each lecture days (not including today and days with exams, i.e., 10 times), there will be one problem in the middle of thelecture. You can form a group of 2 or 3 to turn your classwork in. Each problem is worth 1 point
2 2nd assignment of the today: find one or two partners.3 For midterms and final, you will expect questions of
True of False without explanations; multiple choice; ←−NO PARTIAL CREDITSTu-re of False WITH explanations, shorted or detailed answers: ←−PARTIAL CREDITS
4 A missed midterm cannot be written at another time. If you miss the midterm without prior permission, then it will count as a0. Exceptions are made in two cases: (1) if you obtain the instructor’s prior permission to miss a midterm, or (2) if you havean officially valid excuse such as a medical doctor’s note. In these cases, the weight of the missed midterm will be shifted tothe final exam (e.g., the final exam will then count 60% instead of 40%). There is no make-up option for the final examexcept in cases of an officially valid excuse such as a medical doctor’s note.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 7th, 2019 2 / 14
Linear Algebra
1 LINEAR:Example of a linear equation:
2x = 8
Example of a non-linear equation:2x2 = 8
Example of another linear equation:2x + 3y + 10z − 5w = 13 (∗)
2 SOLUTION(S) of (∗)Solution 1: x = 2, y = 3, z = w = 0.
2 · 2+ 3 · 3 = 4+ 9 = 13.
Solution 2: x = 0, y = 1, z = 1, w = 0.3 · 1+ 10 · 1 = 13.
Solution 3: x = 132 , y = z = w = 0.
2 · 132
= 13.
Fact: There are infinitely many solutions of (∗).
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 7th, 2019 3 / 14
Tuples
2x + 3y + 10z − 5w = 13 (∗)
NotationWe shall use the tuple notation for the solutions.
(x , y , z ,w) = (2, 3, 0, 0)
Tuple:
(x , y) -pair 2-tuple(x , y , z) -triple 3-tuple
· · · etc.
RemarkNote that tuples are ordered, and therefore different from sets. For example,
(2, 3) 6= (3, 2) while {2, 3} = {3, 2}.
The set of solutions of (∗) can be written(/listed) as{(2, 3, 0, 0), (0, 1, 1, 0),
(132, 0, 0, 0
), . . .
}
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 7th, 2019 4 / 14
Systems of linear equations
Example
2x + 3y + 10z − 5w = 13 (∗)x + 2y + 3z + 4w = 5
y − z + 2w = 0
DefinitionA solution of a system of equations is an assigned of numbers to the variables, making all the equation true.
Example
(2, 3, 0, 0) is not a solution, since 1 · 2+ 2 · 3 = 8 6= 5.
(0, 1, 1, 0) is a solution:2+ 3 = 51− 1 = 0
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 7th, 2019 5 / 14
Linear equation
DefinitionA linear equation is an equation of the form
a1x1 + a2x2 + . . .+ anxn = b. (1)
Here,
x1, . . . , xn − variables/unknownsa1, . . . , an − coefficients
b − constant term/right-hand side (RHS)
If an equation is written in the form of (1), it is in standard form.
Example
2x1 + 3x2 + 0x3 = 5
is a linear equation in standard form.
Example
2x − 3y + 5 = 7z + 2y
It is a linear equation (x1, x2, x3) = (x , y , z), but not in standard form. We can rewrite it to standard form by algebra:
2x + (−5)y + (−7)z = −5,
or2x − 5y − 7z = −5.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 7th, 2019 6 / 14
System of linear equations
DefinitionA system of linear equations is a list of equations
a11x1 + a12x2 + . . .+ a1nxn = b1a21x1 + a22x2 + . . .+ a2nxn = b2
...am1x1 + am2x2 + . . .+ amnxn = bm
(2)
Similarly, this is the standard form, and
x1, . . . , xn − variables/unknowns; a11, . . . , a1n, a21, . . . , amn − coefficients; b1, . . . , , bm − constant term
A system of equations is inconsistent if it has no solution. It is called consistent if it has one or more solutions.
Example2x − 7w = 52y + 3x = 15
x + w = y + 5a system of linear equations, but not in standard form⇒
2x + 0y − 7w = 53x + 2y + 0w = 15
x − y + w = 5
Solution
(x , y ,w) =
(18539
,513
,2539
). consistent
ExampleAre the following systems consistent or inconsistent?
(1)
{x + y = 52x + 2y = 11
(2)
{x = 0x = 1
(3)0x = 1 (4)
x + y = 7x − y = 3x + 2y = 1
All are inconsistent.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 7th, 2019 7 / 14
Geometric interpretation
When there are 2 or 3 variables, we can visualize the system.
Example {x + y = 7x − y = 3
Each equation represents a line.
This system has a unique solution(x , y) = (5, 2).
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 7th, 2019 8 / 14
Geometric interpretation
Example {x + y = 52x + 2y = 10
This system has infinity many solutions{(5, 0), (4, 1), (0, 5), . . .}
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 7th, 2019 9 / 14
Geometric interpretation
Example x + y = 7x − y = 3x + 2y = 1
This system has no solution (inconsistent).
Example x + y + z = 12x + y − z = 5x − y + 2z = 7
It has a unique solution (SageMath). This particular example involves three planes intersection in a point. There are many otherpossibilities for how 3 planes intersect (e.g., two or all three planes could be parallel).
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 7th, 2019 10 / 14
Solving systems of equations algebraically
Elementary operations:Interchange (swap) two equations.Multiply an equation by a non-zero number. For example,
x + 2y = 5 ∼ 2x + 4y = 10.
Add a multiple of one equation to another. For example,
E1 :2x + y = 5E2 :x − 2y = 0
E1←E1−2E2∼ E1 :0x + 5y = 5E2 :x − 2y = 0
More precisely,
LHS : (2x + y)− 2(x − 2y) = 5yRHS : 5− 2 · 0 = 5.
From this, we can easily see y = 1 and then x = 2.Performing an elementary operation does NOT change the set of solutions of the system of equations, but (hopefully) it will makethe system simpler.
RemarkThe symbol “∼” means that tow systems of equations are equivalent, i.e., they have the SAME solutions.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 7th, 2019 11 / 14
Example
Example
E1 :x + 2y + z = 8E2 :2x + y + z = 7E3 :x − y − z = −4
E2←E2−2E1∼E1 :x + 2y + z = 8E2 : − 3y − z = −9E3 :x − y − z = −4
E3←E3−E1∼E1 :x + 2y + z = 8E2 : − 3y − z = −9E3 : −3y − 2z = −12
E3←E3−E2∼E1 :x + 2y + z = 8E2 : − 3y − z = −9E3 : − z = −3
E3←−E3∼E1 :x + 2y + z = 8E2 : − 3y − z = −9E3 : z = 3
Back Substitution:z = 3 E2
=⇒ −3y − 3 = −9 =⇒ −3y = −6⇒ y = 2
(y , z) = (2, 3) E1=⇒ x + 4+ 3 = 8 =⇒ x = 1
This system has the unique solution(x , y , z) = (1, 2, 3)
It is consistent.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 7th, 2019 12 / 14
Matrix Notation
DefinitionThe augmented matrix of the system of linear equations
a11x1 + a12x2 + . . .+ a1nxn = b1a21x1 + a22x2 + . . .+ a2nxn = b2
...am1x1 + am2x2 + . . .+ amnxn = bm
is a11 · · · a1n b1...
......
am1 · · · amn bm
.
ExampleE1 : x + 2y + z = 8E2 : 2x + y + z = 7E3 : x − y − z = −4
⇒R1R2R3
1 2 1 82 1 1 71 −1 −1 −4
NotationThe following shorter notation of elementary operations will apply:
Swap rows Ri and Rj : Ri ←→ Rj .Multiply Ri by a nonzero number a: Ri ←− aRi .Add a times Ri to Rj : Rj ←− Rj + aRi .
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 7th, 2019 13 / 14
Example
Example
E1 : x + 2y + z = 8E2 : 2x + y + z = 7E3 : x − y − z = −4
=⇒R1R2R3
1 2 1 82 1 1 71 −1 −1 −4
R2 ← R2 − 2R1
∼R1R2R3
1 2 1 80 −3 −1 −91 −1 −1 −4
R3 ← R3 − R1
∼R1R2R3
1 2 1 80 −3 −1 −90 −3 −2 −12
R3 ← R3 − R2
∼R1R2R3
1 2 1 80 −3 −1 −90 0 −1 −3
R3 ← −R3
∼R1R2R3
1 2 1 80 −3 −1 −90 0 1 3
Substitution as before. Or, we can continue as
R1R2R3
1 2 1 80 −3 −1 −90 0 1 3
R2 ← R2 + R3
∼R1R2R3
1 2 1 80 −3 0 −60 0 1 3
R2 ← −R2
3
∼R1R2R3
1 2 1 80 1 0 20 0 1 3
R1 ← R1 − 2R2 − R3 ∼R1R2R3
1 0 0 10 1 0 20 0 1 3
⇒ (x , y , z) = (1, 2, 3).
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 7th, 2019 14 / 14
MATH-1030Matrix Theory & Linear Algebra I
Lin Jiu
Dalhousie University
May 9th, 2019
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 9th, 2019 1 / 18
Matrix Notation
DefinitionThe augmented matrix of the system of linear equations
a11x1 + a12x2 + . . .+ a1nxn = b1a21x1 + a22x2 + . . .+ a2nxn = b2
...am1x1 + am2x2 + . . .+ amnxn = bm
is a11 · · · a1n b1...
......
am1 · · · amn bm
.
NotationThe following shorter notation of elementary operations will apply:
Swap rows Ri and Rj : Ri ←→ Rj .Multiply Ri by a nonzero number a: Ri ←− aRi .Add a times Ri to Rj : Rj ←− Rj + aRi .
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 9th, 2019 2 / 18
ExampleE1 : x + 2y + z = 8E2 : 2x + y + z = 7E3 : x − y − z = −4
⇒R1R2R3
1 2 1 82 1 1 71 −1 −1 −4
R2 ← R2 − 2R1
2 1 1 7−2 · ( 1 2 1 8 )
2 1 1 7− 2 4 2 16
0 −3 −1 −9
E1 : x + 2y + z = 8E2 : 2x + y + z = 7E3 : x − y − z = −4
⇒R1R2R3
1 2 1 82 1 1 71 −1 −1 −4
R2←R2−2R1∼R1R2R3
1 2 1 80 −3 −1 −91 −1 −1 −4
R1R2R3
1 2 1 80 −3 −1 −91 −1 −1 −4
R3 ← R3 − R1
1 −1 −1 −4− 1 2 1 8
0 −3 −2 −12E1 : x + 2y + z = 8E2 : 2x + y + z = 7E3 : x − y − z = −4
⇒R1R2R3
1 2 1 82 1 1 71 −1 −1 −4
R2←R2−2R1∼R1R2R3
1 2 1 80 −3 −1 −91 −1 −1 −4
R3←R3−R1∼R1R2R3
1 2 1 80 −3 −1 −90 −3 −2 −12
R1R2R3
1 2 1 80 −3 −1 −90 −3 −2 −12
R3 ← R3 − R2
0 −3 −1 −9− 0 −3 −2 −12
0 0 1 3
R3←R3−R2∼R1R2R3
1 2 1 80 −3 −1 −90 0 1 3
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 9th, 2019 3 / 18
ExampleE1 : x + 2y + z = 8E2 : 2x + y + z = 7E3 : x − y − z = −4
⇒R1R2R3
1 2 1 82 1 1 71 −1 −1 −4
∼∼∼∼ R1R2R3
1 2 1 80 −3 −1 −90 0 1 3
⇒ E1 : x + 2y + z = 8E2 : −3y − z = −9E3 : z = 3
Back Substitution:
z = 3 E2=⇒ −3y − 3 = −9 =⇒ y = 2
(y , z) = (2, 3) E1=⇒ x + 2 · 2+ 3 = 8 =⇒ x = 1
Or, we can continue as follows
R1R2R3
1 2 1 80 −3 −1 −90 0 1 3
R2 ← R2 + R3
0 −3 −1 −9+ 0 0 1 3
0 −3 0 −6
R2←R2+R3∼R1R2R3
1 2 1 80 −3 0 −60 0 1 3
R1R2R3
1 2 1 80 −3 0 −60 0 1 3
R2←− R23∼
R1R2R3
1 2 1 80 1 0 20 0 1 3
R2 ← R1 − R3
1 2 1 8
− 0 0 1 3
1 2 0 5
∼R1R2R3
1 2 0 50 1 0 20 0 1 3
R1R2R3
1 2 0 50 1 0 20 0 1 3
R1 ← R1 − 2R2
1 2 0 5
−2 · ( 0 1 0 2)
1 0 0 1
∼R1R2R3
1 0 0 10 1 0 20 0 1 3
E1 : x + 2y + z = 8E2 : 2x + y + z = 7E3 : x − y − z = −4
⇒R1R2R3
1 2 1 82 1 1 71 −1 −1 −4
∼∼∼∼ R1R2R3
1 0 0 10 1 0 20 0 1 3
⇒ E1 : x = 1E2 : y = 2E3 : z = 3
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 9th, 2019 4 / 18
Example
E1 : x + 2y + z = 7E2 : 2x + 2y − 3z = 0E3 : 3x + 4y − 2z = 6
=⇒R1R2R3
1 2 1 72 2 −3 03 4 −2 6
R2 ← R2 − 2R1
∼R1R2R3
1 2 1 70 −2 −5 −143 4 −2 6
R3 ← R3 − 3R1
∼R1R2R3
1 2 1 70 −2 −5 −140 −2 −5 −15
R3 ← R3 − R2
∼R1R2R3
1 2 1 70 −2 −5 −140 0 0 −1
This an inconsistent system. The last indicates an equation
0x + 0y + 0z = 0 = −1.
If the augmented matrix contains a row of all 0’s on the LHS (coefficients) and a non-zero value on the RHS (constant term), as[0 0 · · · 0 b( 6= 0)
],
then the system is inconsistent.[Question] Any other possible cases? (unique solution, no solution, more solutions (infinitely many))
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 9th, 2019 5 / 18
ExampleExample
E1 : x + 2y + z = 7E2 : 2x + 2y − 3z = 0E3 : 3x + 4y − 2z = 7
=⇒R1R2R3
1 2 1 72 2 −3 03 4 −2 7
R2 ← R2 − 2R1
∼R1R2R3
1 2 1 70 −2 −5 −143 4 −2 7
R3 ← R3 − 3R1
∼R1R2R3
1 2 1 70 −2 −5 −140 −2 −5 −14
R3 ← R3 − R2 ∼R1R2R3
1 2 1 70 −2 −5 −140 0 0 0
We were expecting an equation involving only z . However, there is no such equation, which means z can be any number. We let
z = s ,
where s is called a parameter(free variable), i.e., s is any number. We substitute back:
R2 : −2y − 5z = −14⇒ y = 7− 52z = 7− 5
2s
R1 : x + 2y + z = 7⇒ x = 7− 2y − z = 7− 2(7− 5
2s
)− z = −7+ 4s.
General solution (parametrized solution):
(x , y , z) =
(4s − 7, 7− 5
2s, s
)for any number s
Special solutions:s 0 1 2 3
(x , y , z) (−7, 7, 0)(−3, 9
2 , 1)
(1, 2, 2)(5,− 1
2 , 3)
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 9th, 2019 6 / 18
Example
Example
E1 : x + 2y + 3z = 6E2 : 2x + 4y + 6z = 12
E3 : −x − 2y − 3z = −6(E2 = 2E1, E3 = −E1) =⇒
R1R2R3
1 2 3 62 4 6 12−1 −2 −3 −6
R2 ← R2 − 2R1
∼R1R2R3
1 2 3 60 0 0 0−1 −2 −3 −6
R3 ← R3 + R1
∼R1R2R3
1 2 3 60 0 0 00 0 0 0
The last two rows are trivial, which indicates that both y and z can be any number. Let
z = s, y = t,
for parameters s and t.R1 : x + 2y + 3z = 6⇒ x = 6− 2y − 3z = 6− 2t − 3s.
The (2-parameter) general solution is(x , y , z) = (6− 2t − 3s, t, s) .
Special solutions aret\s 0 1 2 30 (6, 0, 0) (3, 0, 1) (0, 0, 2) (−3, 0, 3)1 (4, 1, 0) (1, 1, 1) (−2, 1, 2) (−5, 1, 3)2 (2, 2, 0) (−1, 2, 1) (−4, 2, 2) (−7, 2, 3)3 (0, 3, 0) (−3, 3, 1) (−6, 3, 2) (−9, 3, 3)
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 9th, 2019 7 / 18
Echelon Form
DefinitionAn entry of an augmented matrix is called a leading entry or pivot entry if it is the leftmost non-zero entry of a row. An augmentedmatrix is in echelon form (also called row echelon form) if
1 All rows of zeros are below all non-zero rows.2 Each leading entry of a row is in a column to the right of the leading entry of any row above it.
A column containing a pivot entry is also called a pivot column.
Example ∗ ∗ ∗ ∗ ∗ ∗ ∗0 ∗ ∗ ∗ ∗ ∗ ∗0 0 0 ∗ ∗ ∗ ∗0 0 0 0 0 0 00 0 0 0 0 0 0
{∗− nonzero entry;∗− leading/pivot entry.
Echelon forms: 0 5 2 1 3 50 0 0 0 1 60 0 0 0 0 00 0 0 0 0 0
,
1 4 0 0 5 00 0 1 0 2 00 0 0 1 4 00 0 0 0 0 1
,
3 0 6 20 1 4 00 0 2 10 0 0 0
Not in echelon form
0 0 0 01 2 3 30 1 0 20 0 0 10 0 0 0
,
1 2 32 4 −64 0 7
,
0 2 3 31 5 0 20 0 1 00 0 0 0
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 9th, 2019 8 / 18
Gaussian Elimination
Goal: Use elementary row operations to transform the system of equations to echelon form.Input: Augmented matrix
Output: Augmented matrix in echelon form row equivalent to the input.
Algorithm: Gaussian EliminationThis algorithm provides a method for using row operations to take a matrix to its echelon form. We begin with the matrix in itsoriginal form.
1 Starting from the left, find the first non-zero column. This is the first pivot column, and the position at the top of this columnwill be the position of the first pivot entry. Switch rows if necessary to place a non-zero number in the first pivot position.
2 Use row operations to make the entries below the first pivot entry (in the first pivot column) equal to zero.3 Ignoring the row containing the first pivot entry, repeat steps 1 and 2 with the remaining rows. Repeat the process until there
are no more non-zero rows left.
Example
0 0 0 2 3 50 2 1 0 1 40 0 0 1 2 30 1 2 3 4 5
=
0 0 0 2 3 50 2 1 0 1 40 0 0 1 2 30 1 2 3 4 5
R1 ↔ R2 ∼
0 2 1 0 1 40 0 0 2 3 50 0 0 1 2 30 1 2 3 4 5
R4 ← R4 −12
∼
0 2 1 0 1 40 0 0 2 3 50 0 0 1 2 30 0 3
2 3 72 3
=
0 2 1 0 1 40 0 0 2 3 50 0 0 1 2 30 0 3
2 3 72 3
R2 ↔ R4
0 2 1 0 1 40 0 3
2 3 72 3
0 0 0 1 2 30 0 0 2 3 5
R4 ← R4 − 2R3
∼
x y z w v0 2 1 0 1 40 0 3
2 3 72 3
0 0 0 1 2 30 0 0 0 −1 −1
E1 2y + z + v = 4E2 3
2z + 3w + 72v = 3
E3 w + 2v = 3E4 −v = −1
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 9th, 2019 9 / 18
Terminology1 Any column containing a pivot entry is called a pivot column.2 The corresponding variables are called pivot variables.3 Any variable that is not a pivot variable is called a free variable.
In the previous example, x is a free variable and y , z ,w , v are pivot variables.
Back substitutionEvery free variable becomes a parameter.Every pivot/leading variable has an equation, which we can use to find its value.
Examplex y z w v0 1 2 0 3 50 0 0 1 −2 40 0 0 0 0 00 0 0 0 0 0
We have
v : free variable v = t
w : from R2, pivot variable w − 2v = 4 =⇒ w = 2v + 4 = 2t + 4z : free variable z = s
y : from R1, pivot variable y + 2z + 3v = 5 =⇒ y = 5− 2z − 3v = 5− 2s − 3tx : free variable x = r
General solution(x , y , z ,w , v) = (r , 5− 2s − 3t, s, 2t + 4, t).
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 9th, 2019 10 / 18
Rank
Remark1 If a system is inconsistent, then the RHS is a pivot column. 1 2 0 3
0 0 1 50 0 0 7
−→ inconsistent, no solution.[0 0 · · · 0 b( 6= 0)
]2 If the system is consistent, the number of free variables determines the number of solutions:
no free variables=⇒no parameters=⇒unique solution;at least one free variable =⇒ infinitely many solution.
DefinitionThe rank of a matrix is the number of pivot entries of its echelon form.
Example
A =
0 1 2 0 3 50 0 0 1 −2 40 0 0 0 0 00 0 0 0 0 0
B =
1 0 0 10 1 0 20 0 1 3
rank(A) = 2 rank(B) = 3
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 9th, 2019 11 / 18
Rank
RemarkIf a system of m equations in n variables has rank r and is consistent, then there are n − r free variables, i.e., n − r parameters inthe general solution.
n = r ⇒ n − r = 0⇒unique solutionn > r ⇒ n − r > 0⇒infinitely many solutions.
Examplex y z w v0 1 2 0 3 50 0 0 1 −2 40 0 0 0 0 00 0 0 0 0 0
A system of 4 equations in 5 variables. rank= 2. m = 4, n = 5 and r = 2.
5− 2 = 3 free variables: (x , z , v) = (r , s, t)
v : = t
w : = 2t + 4z : = s
y : = 5− 2s − 3tx : = r
General solution(x , y , z ,w , v) = (t, 5− 2s − 3t, s, 2t + 4, t).
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 9th, 2019 12 / 18
Classwork
Consider system
x + 2y + z = 5x + 3y + 2z = 9
2x + 5y + 3z = 14
and its corresponding matrix form 1 2 1 51 3 2 92 5 3 14
Find its rank.
Lecture resumes at 7:45.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 9th, 2019 13 / 18
Example
Solve the linear system
x + 2y + z = 5x + 3y + 2z = 92x + 5y + 3z = 14
1 2 1 51 3 2 92 5 3 14
R2 ← R2 − R1 ∼
1 2 1 50 1 1 42 5 3 14
R3 ← R3 − 2R1
∼
1 2 1 50 1 1 40 1 1 4
R3 − R2 ∼
1 2 1 50 1 1 40 0 0 0
x + 2y + z = 5y + z = 4
pivot variables x and y ; free variable z
z = tR2=⇒ y = 4− z = 4− tR1=⇒ x = 5− 2y − z = 5− 2 (4− t))− t = t − 3
General solution:(x , y , z) = (−3+ t, 4− t, t).
It is also written as column form: xyz
=
−3+ t4− tt
=
−340
+ t
1−11
.
We shall come back to this form in Chapter 2.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 9th, 2019 14 / 18
Gauss-Jordan elimination
It is similar to Gaussian elimination, but withmore elementary row operators;and less back substitution.
DefinitionA matrix B is in reduced echelon form if it is in echelon form and additionally
all pivot entries are equal to 1;all entries above pivot entries are equal to 0.
Examplex + 2y + z = 5x + 3y + 2z = 9
2x + 5y + 3z = 14−→
1 2 1 51 3 2 92 5 3 14
∼ 1 2 1 5
0 1 1 40 0 0 0
︸ ︷︷ ︸
Echelon Form
R1 ← R1 − 2R2 ∼
1 0 −1 −30 1 1 40 0 0 0
︸ ︷︷ ︸
Reduced Echelon Form
pivot variables x and y ; free variable z
z = t Equations:
{x − z = −3y + z = 4
=⇒
{y = 4− z = 4− t
x = −3+ z = −3+ t=⇒ (x , y , z) = (−3+ t, 4− t, t)
xyz
=
−340
+ t
1−11
=
−340
+
t−tt
=
−3+ t4− tt
.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 9th, 2019 15 / 18
Example
2x + y + w = 8x + y + z = 6
3x + 2y + z + v = −2→
2 1 0 1 0 81 1 1 0 0 63 2 1 0 1 −2
R1 ↔ R2 ∼
1 1 1 0 0 62 1 0 1 0 83 2 1 0 1 −2
R2 ← R2 − 2R1
R3 ← R3 − 3R1
∼
1 1 1 0 0 60 −1 −2 1 0 −40 −1 −2 0 1 −20
R2 ← −R2 ∼
1 1 1 0 0 60 1 2 −1 0 40 −1 −2 0 1 −20
R1 ← R1 − R2
R3 ← R3 + R2
∼
1 0 −1 1 0 20 1 2 −1 0 40 0 0 −1 1 −16
R3 ← −R3 ∼
1 0 −1 1 0 20 1 2 −1 0 40 0 0 1 −1 16
R1 ← R1 − R3
R2 ← R2 + R3
∼
1 0 −1 0 1 −140 1 2 0 −1 200 0 0 1 −1 16
free variables z = s and v = t
R1 :x = −14+ z − v = −14+ s − t
R2 :y = 20− 2z + v = 20− 2s + t
R3 :w = 16+ v = 16+ t
General solution xyzwv
=
−14200160
+ t
−11011
+ s
1−2100
=
−14200160
+
−tt0tt
+
s−2ss00
=
−14− t + s20+ t − 2s
s16+ t
t
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 9th, 2019 16 / 18
Uniqueness
TheoremReduced echelon forms are unique.
DefinitionTwo matrices are called row equivalent, if one can be obtained from the other by elementary row operations.
ExampleLet
A =
1 2 1 30 3 3 31 1 0 2
, B =
2 4 2 62 5 3 71 6 5 7
BR1← 1
2R1∼ 1 2 1 3
2 5 3 71 6 5 7
R2←R2−2R1∼
1 2 1 30 1 1 11 6 5 7
R3←R3−5R2∼
1 2 1 30 1 1 11 1 0 2
R2←3R2∼
1 2 1 30 3 3 31 1 0 2
= A
A and B are row equivalent.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 9th, 2019 17 / 18
Example
TheoremTwo matrices are row equivalent if and only if (iff) they have the same reduced echelon form.
Example
A =
1 2 1 30 3 3 31 1 0 2
, B =
2 4 2 62 5 3 71 6 5 7
AR3←R3−R1∼
1 2 1 30 3 3 30 −1 −1 −1
R2← 1
3R2∼ 1 2 1 3
0 1 1 10 −1 −1 −1
R3←R3+R2∼
1 2 1 30 1 1 10 0 0 0
R1←R1−2R2∼
1 0 −1 10 1 1 10 0 0 0
BR1← 1
2R1∼ 1 2 1 3
2 5 3 71 6 5 7
R2←R2−2R1∼
1 2 1 30 1 1 11 6 5 7
R3←R3−R1∼
1 2 1 30 1 1 10 4 4 4
R3←R3−4R2∼
1 2 1 30 1 1 10 0 0 0
R1←R1−2R2∼
1 0 −1 10 1 1 10 0 0 0
Thus, we confirm that A and B are row equivalent.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 9th, 2019 18 / 18
MATH-1030Matrix Theory & Linear Algebra I
Lin Jiu
Dalhousie University
May 14th, 2019
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 1 / 33
Homogeneous systems of equations
DefinitionA system of equations is called homogeneous if each of the constantterms is equal to 0. A homogeneous system therefore has the form
a11x1 + a12x2 + . . .+ a1nxn = 0a21x1 + a22x2 + . . .+ a2nxn = 0
...am1x1 + am2x2 + . . .+ amnxn = 0
aij—coefficients
xj—variables
Example2x + 3y + 4z = 7
x − 2y = 0x + y + z = 3
Non-homogenous
2x + 3y + 4z = 0x − 2y = 0
x + y + z = 0
HomogenousLin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 2 / 33
Example
DefinitionA system of equations is called homogeneous if each of the constantterms is equal to 0. A homogeneous system therefore has the form
a11x1 + a12x2 + . . .+ a1nxn = 0a21x1 + a22x2 + . . .+ a2nxn = 0
...am1x1 + am2x2 + . . .+ amnxn = 0
RemarkHomogeneous systems are always consistent. They always have thefollowing (trivial) solution (x1, x2, . . . , xn) = (0, 0, . . . , 0).Therefore, we want to know, for a given homogeneous system, doesit have non-trivial solutions? Infinitely many solutions.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 3 / 33
Example
2x + 3y + 4z = 0x − 2y = 0
x + y + z = 0→
2 3 4 01 −2 0 01 1 1 0
R1↔R2∼ 1 −2 0 0
2 3 4 01 1 1 0
R2←R2−2R1R3←R3−R1∼
1 −2 0 00 7 4 00 3 1 0
R2←R2−2R3∼ 1 −2 0 0
0 1 2 00 3 1 0
R1←R1+2R2R3←R3−3R2∼
1 0 4 00 1 2 00 0 −5 0
R3←−R3
5∼ 1 0 4 0
0 1 2 00 0 1 0
R1←R1−4R3R2←R2−2R3∼
1 0 0 00 1 0 00 0 1 0
=⇒ (x , y , z) = (0, 0, 0)
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 4 / 33
Example2x + 3y + 4z = 0
x − 2y = 0x + y + z = 0
−→
2 3 4 01 −2 0 01 1 1 0
∼∼∼∼ 1 0 0 0
0 1 0 00 0 1 0
Thus, it only has the trivial solution
(x , y , z) = (0, 0, 0)[Question] What is the rank of the augmented matrix (or its reducedechelon form)? (Elementary row operations do not change the rank)
rank = 3 = # of unknowns.
FactIf a homogeneous system of n variables has rank n, then it only hasthe trivial solution.
# of free variables/parameters = # of unknowns− rank
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 5 / 33
Example
x + 3y − 2z = 02x + 4y − 2z = 0
y − z = 0→
1 3 −2 02 4 −2 00 1 −1 0
R2←R2−2R1∼ 1 3 −2 0
0 −2 2 00 1 −1 0
R2↔R3∼
1 3 −2 00 1 −1 00 −2 2 0
R1←R1−3R2R3←R3+2R2∼
1 0 1 00 1 −1 00 0 0 0
3 variables with rank 2:
x y z 1 0 1 00 1 −1 00 0 0 0
=⇒
{pivot variables x , y
free variable z = t
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 6 / 33
Example
x + 3y − 2z = 02x + 4y − 2z = 0
y − z = 0→
1 3 −2 02 4 −2 00 1 −1 0
∼∼x y z 1 0 1 00 1 −1 00 0 0 0
z = t ⇒
{R1 : x + z = 0R2 : y − z = 0
⇒
{x = −ty = t
(x , y , z) = (−t, t, t)
General solution:xyz
=
−ttt
= t ·
−111
basic solution(t = 1)−
−111
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 7 / 33
TheoremTheoremGiven a homogeneous system with n variables and of rank r , thesystem is always consistent:
1 if r = n, it only has the unique, trivial solution; (all zeros)2 if r < n, there are non-trivial solutions. In this case, the number
of parameters is n − r and there are n − r basic solutions.
Example
x + 4y + 3z = 03x + 12y + 9z = 0
→[1 4 3 03 12 9 0
]R2←R2−3R1∼
[1 4 3 00 0 0 0
]parameters: z = s, y = t =⇒ x = 0− 4y − 3z = −4t − 3sxy
z
=
−4t − 3sts
= t
−410
+s
−301
basicsolutions:
−410
and
−301
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 8 / 33
TheoremTheoremIf the general solution has the formx1
...xn
= s1
•...•
+ · · ·+ sr
•...•
basicsolutions:
•...•
, . . . ,
•...•
DefinitionLet A be a system of linear equations. The associated homogeneoussystem is the system B , which has the same left-hand side as A, butall 0’s on the right-hand side.Example
x + 4y + 3z = 23x + 12y + 9z = 6
associated homogeneous system−−−−−−−−−−−−−−−−−−−−−→
x + 4y + 3z = 03x + 12y + 9z = 0
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 9 / 33
TheoremTheoremIf the general solution has the formx1
...xn
= s1
•...•
+ · · ·+ sr
•...•
basicsolutions:
•...•
, . . . ,
•...•
DefinitionLet A be a system of linear equations. The associated homogeneoussystem is the system B , which has the same left-hand side as A, butall 0’s on the right-hand side.Example
x + 4y + 3z = 23x + 12y + 9z = 6
associated homogeneous system−−−−−−−−−−−−−−−−−−−−−→
x + 4y + 3z = 03x + 12y + 9z = 0
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 9 / 33
Comparison
x + 4y + 3z = 23x + 12y + 9z = 6
(A)→[1 4 3 23 12 9 6
]R2←R2−3R1∼
[1 4 3 20 0 0 0
]free variables: z = s, y = t =⇒ x = 2− 4y − 3z = 2− 4t − 3sxy
z
=
2− 4t − 3sts
=
200
+ t
−410
+ s
−301
x + 4y + 3z = 0
3x + 12y + 9z = 0(B)→
[1 4 3 03 12 9 0
]R2←R2−3R1∼
[1 4 3 00 0 0 0
]xyz
=
−4t − 3sts
= t
−410
+ s
−301
(x , y , z) = (2, 0, 0)is a solution to A
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 10 / 33
ComparisonTheoremLet A be a system of equations, and let B be the associatedhomogeneous system. Then
the general solution of A(GA) =a particular solution of A(PA)+ the general solution of B(GB).
Example
x + 4y + 3z = 23x + 12y + 9z = 6
(A) −→x + 4y + 3z = 0
3x + 12y + 9z = 0(B) ∼
[1 4 3 00 0 0 0
]xyz
︸︷︷︸GA
=
200
︸︷︷︸PA
+ t
−410
+ s
−301
︸ ︷︷ ︸
GB
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 11 / 33
Summary of Chpt. 1
Given a system (or just one equation), tell whether it is linear:
2x + 4y = 0linear
{3x + 5y = 32xy = 4
NOT linear
Given a linear system, write down the augmented matrix:
x + 2y + z = 0x + 3y + 2z = 32x + 5y + 3z = 3
=⇒
1 2 1 01 3 2 32 5 3 3
and vice versa.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 12 / 33
Summary of Chpt. 1
The most important eliminations:Gaussian elimination and Gauss-Jordan elimination
Gaussian Elimination↓
echelon form
Gauss-Jordan Elimination↓
redued echelon form
↓ ∗ ∗ ∗ ∗0 0 ∗ ∗0 0 0 0
↓ 1 ∗ 0 ∗
0 0 1 ∗0 0 0 0
through THREE elementary row operations
Ri ←→ Rj Rj ←− aRj Rj ←− Rj + aRi (i 6= j , a 6= 0)
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 13 / 33
Summary of Chpt. 1
(reduced) echelon form: 1 2 1 01 3 2 32 5 3 3
∼ 1 2 1 0
0 1 1 30 0 0 0
⇒ {x + 2y + z = 0y + z = 3
1 Consistent or inconsistent? For the example, it is consistent.
Inconsistent example:
1 2 0 30 0 4 50 0 0 7
2 Pivot variable(s): x and y ;3 Free variable(s): z = t4 rank=# of pivot entries= 2
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 14 / 33
Summary of Chpt. 1
(reduced) echelon form: 1 2 1 01 3 2 32 5 3 3
∼ 1 2 1 0
0 1 1 30 0 0 0
⇒ {x + 2y + z = 0y + z = 3
5 Solution z = t free, x , y pivotechelon form =⇒ back substitution y = 3− z = 3− tx = −z − 2y = −6 + treduced echelon form 1 2 1 0
0 1 1 30 0 0 0
∼ 1 0 −1 −6
0 1 1 30 0 0 0
xyz
=
•••
+t
•••
=
••0
+t
••1
=
−630
+t
1−11
=
−6+ t3− tt
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 15 / 33
Summary of Chpt. 1
6 Determine whether two systems have the same solutions.Compare the reduced echelon forms of the two systems
x + 2y + z = 33y + 3z = 3x + y = 2
2x + 4y + 2z = 62x + 5y + 3z = 7x + 6y + 5z = 7
A =
1 2 1 30 3 3 31 1 0 2
, B =
2 4 2 62 5 3 71 6 5 7
1 2 1 3
0 3 3 31 1 0 2
∼ 1 0 −1 1
0 1 1 10 0 0 0
∼ 2 4 2 6
2 5 3 71 6 5 7
So they have the same solutions. (Two matrices are rowequivalent.)
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 16 / 33
Summary of Chpt. 1
Some important facts:1 A system of linear equations can have 0, 1 and infinitely many
solutions.0 solution—inconsistent 1 or infinitely many—consistent
2 A consistent system of m linear equations, n variables and rankr . # of free variables/parameters = n − r
n = r unique solutionn > r infinitely many
Hard & Tricky Problem: What happens if r > n? For example, asystem of 3 equations with 2 variables and rank 3.The answer is that the system if inconsistent.
# of free variables‖
2− 3 = −1
• • •• • •• • •
∼ ∗ ∗ ∗0 ∗ ∗
0 0 ∗
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 17 / 33
Classwork
Solve the homogeneous system
x + 2y + z = 0x + 3y + 2z = 0
2x + 5y + 3z = 0
Lecture resumes at 7:45.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 18 / 33
Solution
x + 2y + z = 0x + 3y + 2z = 02x + 5y + 3z = 0
−→
1 2 1 01 3 2 02 5 3 0
∼
1 2 1 00 1 1 00 0 0 0
∼
1 0 −1 00 1 1 00 0 0 0
z is free: z = t. xy
z
= t ·
1−11
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 19 / 33
Coordinate systemOn the plane (R2):
The coordinate system allows us to describe every point in the planeby a pair of real numbers, called the coordinates of the point.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 20 / 33
VectorsUnlike a point, which describes a location in a coordinate system, avector describes an offset or a distance and direction.
v =
[21
]2 and 1 of v are called components. w =
[3−1
]Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 21 / 33
Space
Zero vector
0 =
[00
].
More generally,a point in an n-dimensional space is given by n coordinates andwritten as
P = (x1, . . . , xn)
a vector in an n-dimensional space is given by n components andwritten as
v =
x1...xn
0 =
0...0
.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 22 / 33
Space
DefinitionThe set of n-dimensional column vectors is denoted by
Rn :=
x1...xn
x1, . . . , xn ∈ R
and is called n-dimensional Euclidean space.
ExampleR2 : Euclidean planeR3 : vectors in 3-dimensional Euclidean spaceRn : vectors in n-dimensional Euclidean space
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 23 / 33
Points VS Vectors1 Given two points P and Q, we can define a vector v =
−→PQ
2 In n-dimensional space, the coordinate vector of a pointP = (x1, . . . , xn) is the vector
−→OP =
x1...xn
.
O = (0, 0, . . . , 0) is the origin.
Example (n = 2)
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 24 / 33
Addition of vectors
If
v =
x1
x2...xn
, w =
y1
y2...yn
=⇒ v + w =
x1 + y1
x2 + y2...
xn + yn
.
Slide w so that the tail of w is on the tip of v. Then draw the arrowwhich goes from the tail of v to the tip of w . This arrow representsthe vector v + w.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 25 / 33
Addition of vectors
Example
[23
]+
[−14
]=
[17
] 123−45
+
110−11
=
233−56
Negation
v =
x1
x2...xn
=⇒ −v =
−x1
−x2...−xn
Rotate v by 180◦
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 26 / 33
Properties of AdditionProperties
(A1) u + v = v + u(A2) (u + v) + w = u + (v + w)
(A3) u + 0 = u(A4) u + (−u) = 0
Exampleu + v + w + (−u) + w + (−w)
(A2) = u + v + w + (−u) + (w + (−w))
(A4) = u + v + w + (−u) + 0(A3) = u + v + w + (−u)(A1) = u + (−u) + v + w(A4) = 0 + v + w(A3) = v + w.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 27 / 33
Properties of AdditionProperties
(A1) u + v = v + u(A2) (u + v) + w = v + (u + w)
(A3) u + 0 = u(A4) u + (−u) = 0
Remark(A1)
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 28 / 33
Scalar Multiplication
a ·
x1
x2...xn
=
ax1
ax2...
axn
Example
3 ·
12−1
=
36−3
‘
Properties(SM1) k (u + v) = ku + kv(SM2) (k + `)u = ku + `u(SM3) k (`u) = (k`)u(SM4) 1 · u = u
Example
u + 2v − 3u− 5v + 6u(A1) = (u− 3u + 6u) + (2v − 5v)
(SM2) = (1− 3 + 6)u + (2− 5) v= 4u− 3v
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 29 / 33
Example
2x + y + w = 8x + y + z = 6
3x + 2y + z + v = −2→
2 1 0 1 0 81 1 1 0 0 63 2 1 0 1 −2
∼ 1 0 −1 0 1 −14
0 1 2 0 −1 200 0 0 1 −1 16
free variables z = s and v = tGeneral solution:xyzwv
=
−14200160
+t
−11011
+s
1−2100
=
−14200160
+−tt0tt
+
s−2ss00
=
−14− t + s20 + t − 2s
s16 + t
t
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 30 / 33
ExampleConsider four points A(2, 3), B(−3, 4), C (−8,−1) and D (−3,−2)
−→AB =
[−3− 24− 3
]=
[−51
]−→DC =
[−8− (−3)−1− (−2)
]=
[−51
]=−→AB
−→AD =
[−3− 2−2− 3
]=
[−5−5
]−→CB =
[−3− (−8)4− (−1)
]=
[55
]= −−→AD =
−→DA
A,B ,C ,D form a parallelogram
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 31 / 33
A(2, 3), B(−3, 4), C (−8,−1) D (−3,−2)
−→AB +
−→BC =
−→AB +
−→BC =
−→AC
−→AB+
−→BC =
[−51
]+
[−5−5
]=
[−10−4
]−→AC =
[−8− 2−1− 3
]=
[−10−4
]−→AB−
−→BC =? in terms of
−→XY ,
for X and Y from A, B , C andD.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 32 / 33
A(2, 3), B(−3, 4), C (−8,−1) D (−3,−2)
−→AB−
−→BC =
−→AB+
−→CB =
−→DC+
−→CB =
−→DB
−→AB −
−→BC =
[−51
]−[−5−5
]=
[−51
]+
[55
]=
[06
]−→DB =
[−3− (−3)4− (−2)
]=
[06
]
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 33 / 33
MATH-1030Matrix Theory & Linear Algebra I
Lin Jiu
Dalhousie University
May 16th, 2019
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 1 / 30
Recall
v =
x1x2...xn
, w =
y1y2...yn
=⇒ v + w =
x1 + y1x2 + y2
...xn + yn
. a ·
x1x2...xn
=
ax1ax2...
axn
Example 123−45
+
110−11
=
233−56
3 ·
12−1
=
36−3
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 2 / 30
Linear Combination
DefinitionA linear combination of vectors v1, . . . , vn is an expression of the form
w = a1v1 + a2v2 + · · ·+ anvn
(=
n∑k=1
akvk
).
Example
Is the vector w =
231
a linear combination of v1 =
121
and v2 =
352
? By
definition, we need to determine whether there exist scalars a1 and a2 such that
w = a1v1 + a2v2231
= a1
121
+ a2
352
⇐⇒231
=
a1 + 3a22a1 + 5a2a1 + 2a2
⇐⇒a1 + 3a2 = 22a1 + 5a2 = 3a1 + 2a2 = 1
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 3 / 30
Linear Combination
w =
231
, v1 =
121
, v2 =
352
w = a1v1 + a2v2 ⇐⇒
a1 + 3a2 = 22a1 + 5a2 = 3a1 + 2a2 = 1
A system of linear equations. Of course E3− E1 gives a2 but we would like toreview the Gauss-Jordan elimination. 1 3 2
2 5 31 2 1
R2←R2−2R1R3←R3−R1∼
1 3 20 −1 −10 −1 −1
R3←R3−R2R2←−R2∼
1 3 20 1 10 0 0
R1←R1−3R2∼
1 0 −10 1 10 0 0
⇒ [a1a2
]=
[−11
]⇒ w = −v1 + v2
So the answer is yes, w is a linear combination of v1 and v2.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 4 / 30
Example
Is the vector u =
232
a linear combination of v1 =
121
and v2 =
352
?
u = a1v1 + a2v2 ⇔
232
= a1
121
+ a2
352
=
a1 + 3a22a1 + 5a2a1 + 2a2
⇔a1 + 3a2 = 22a1 + 5a2 = 3a1 + 2a2 = 2 1 3 2
2 5 31 2 2
R2←R2−2R1R3←R3−R1∼
1 3 20 −1 −10 −1 0
R3←R3−R2∼ 1 3 2
0 1 10 0 1
The system is inconsistent. Such a1 and a2 do not exist. u is NOT a linearcombination of v1 and v2.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 5 / 30
Geometrically....
The set of all linear combinations of u and v is the set of (position vectors of)points in the plane spanned by u and v. Here, u and v are not parallel, i.e.,v 6= ku for some k . (Otherwise au + bv = au + bku = (a+ bk)u —a line)
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 6 / 30
Length/Norm
Definition
The norm of a vector u =
x1...xn
is given by
‖u‖ :=√x21 + · · ·+ x2
n
Example
u =
[34
]By definition, ‖u‖ =
√32 + 42 =
√9+ 16 =
√25 = 5
v =
123−2
=⇒ ‖v‖ =√
12 + 22 + 32 + (−2)2 =√1+ 4+ 9+ 4 =
√18 = 3
√2
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 7 / 30
Geometric interpretation
By Pythagorean’s Theorem, let the length of u bec , then
c2 = 32 + 42 ⇒ c =√
32 + 42 = 5.
FactGeometrically, ‖u‖ is the length of the vector u.
RemarkWe do not distinguish the norm of a vector andthe length of a vector.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 8 / 30
Unit vector
DefinitionA vector is called a unit vector if it has length 1.
ExampleThe following vectors are unit vectors:
100
,010
,001
,−1000
,[
1√2
1√2
]
√(1√2
)2
+
(1√2
)2
=
√12+
12= 1
u =
[34
]is NOT a unit vector. ‖u‖ = 5 6= 1.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 9 / 30
Unit vector
DefinitionIf u is any non-zero vector, then 1
‖u‖u is a unit vector, which is called the result ofnormalizing the vector u.
Remark1 If u 6= 0 (non-zero), ‖u‖ is a positive number (length) , so is 1
‖u‖ .
1‖u‖
u is the scalar multiplication of1‖u‖
and u
2 If u = 0 =
0...0
, ‖0‖ =√02 + · · ·+ 02 = 0 , so that 1
‖0‖ does not exist.
3 Sometimes we write u‖u‖
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 10 / 30
Unit vector
Example
Normalize the vector u =
[34
]. Since ‖u‖ =
√32 + 42 = 5,
1‖u‖
u =15
[34
]=
15 · 3
15 · 4
=
35
45
.Let v = 1
‖u‖u: ‖v‖ =√( 3
5
)2+( 4
5
)2=√
925 + 16
25 = 1 v is a unit vector.
Laws of norm/length‖u‖ ≥ 0. ‖u‖ = 0 if and only if (iff) u = 0‖ku‖ = |k| ‖u‖Triangle inequality ‖v + w‖ ≤ ‖v‖+ ‖w‖
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 11 / 30
Unit vector
Laws of norm/length1 ‖u‖ ≥ 0. ‖u‖ = 0 if and only if (iff) u = 02 ‖ku‖ = |k| ‖u‖3 Triangle inequality ‖v + w‖ ≤ ‖v‖+ ‖w‖
Example
Let u =
23−6
‖u‖ =√
22 + 32 + (−6)2 =√4+ 9+ 36 = 7
−2u =
−4−612
=⇒ ‖2u‖ =√(−4)2 + (−6)2 + 122 = 14 = |−2| ‖u‖
Remark
If u is non-zero∥∥∥∥ 1‖u‖u
∥∥∥∥ =∣∣∣ 1‖u‖
∣∣∣ · ‖u‖ 1= 1‖u‖ · ‖u‖ = 1⇒ 1
‖u‖u is a unit vector
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 12 / 30
Dot Product
DefinitionLet
u =
u1...un
, and v =
v1...vn
The dot product of u and v is defined by
u · v :=
u1...un
•v1...vn
= u1v1 + u2v2 + · · ·+ unvn
(=
n∑k=1
ukvk
).
Example 1−10
•200
= 1 · 2+ (−1) · 0+ 0 · 0 = 2+ 0+ 0 = 2.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 13 / 30
Examples
Let u =
[11
], v =
[−12
]and w =
[1−1
].
u · v = 1 · (−1) + 1 · 2 = −1+ 2 = 1v ·w = (−1) · 1+ 2 · (−1) = −3u ·w = 1 · 1+ 1 · (−1) = 0
u ·w = 0, u · v > 0, v ·w < 0
We can observe from the picture that:u and w is perpendicular (= 90◦);the (smaller) angle between u and v is acute (< 90◦);the (smaller) angle between v and w is obtuse (> 90◦);
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 14 / 30
Geometric interpretation
We only consider the smaller angle (the one≤ 180◦) between two vectors.
u · v = ‖u‖ · ‖v‖ cos θ
Recall that if both u and v are non-zero,
‖u‖ > 0, ‖v‖ > 0 and cos θ
> 0, θ < 90◦
= 0, θ = 90◦
< 0, θ > 90◦
If the angle θ is acute (θ < 90◦), then u · v > 0;If the angle θ is obtuse (θ > 90◦), then u · v < 0;If two vectors are perpendicular (θ = 90◦), thenu · v = 0;Vice versa: u · v gives certain information for theangle between u and v
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 15 / 30
Example
Find the angle between
u =
1−10
and v =
200
Directly, 1
−10
•200
= 1 · 2+ (−1) · 0+ 0 · 0 = 2.
In addition,
‖u‖ =√
12 + (−1)2 + 02 =√2 ‖v‖ =
√22 + 02 + 02 = 2.
Thus, let θ be the angle between u and v.
cos θ =u · v‖u‖‖v‖
=2√2 · 2
=1√2⇒ θ = 45◦.
How about sin θ? sin 45◦ = 1√2sin2 θ + cos2 θ = 1 =⇒ sin2 θ = 1− cos2 θ = 1
2
0◦ ≤ θ ≤ 180◦ =⇒ sin θ ≥ 0 =⇒ sin θ =
√12=
1√2
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 16 / 30
Laws
u · v = v · u (Be careful, it is not associative! (u · v)w 6= u(v ·w))The dot product gives a scalar(number). We can NOT compute u · v ·w! Inaddition,
u =
[10
], v =
[11
], w =
[01
]=⇒
{u · vv ·w
= 1= 1
=⇒
{(u · v)w(v ·w)u
= w= u
u · u = ‖u‖2 ≥ 0, u · u = 0 if and only if (iff) u = 0
u · u = ‖u‖‖u‖ cos 0◦ = ‖u‖2
(u + v) ·w = u ·w + v ·w, u · (v + w) = u · v + u ·w(ku) · v = k(u · v) = u · (kv)|u · v| ≤ ‖u‖‖v‖
|u · v| = |‖u‖ · ‖v‖ cos θ| = ‖u‖ · ‖v‖ |cos θ| ≤ ‖u‖ · ‖v‖
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 17 / 30
Example
1 u · v = v · u2 u · u = ‖u‖2 ≥ 0, u · u = 0 if and only if (iff) u = 03 (u + v) ·w = u ·w + v ·w, u · (v + w) = u · v + u ·w4 (ku) · v = k(u · v) = u · (kv)5 |u · v| ≤ ‖u‖‖v‖
Problem. Using the properties of the dot product, prove the parallelogramidentity:
‖a + b‖2 + ‖a− b‖2 = 2‖a‖2 + 2‖b‖2
Proof.
‖a + b‖2 + ‖a− b‖22= (a + b) · (a + b) + (a− b) · (a− b)3= a · a + a · b + b · a + b · b + a · a + (a) · (−b) + (−b) · a + (−b) · (−b)4= a · a + a · b + b · a + b · b + a · a− a · b− b · a + (−1)2 b · b1= a · a + a · b + a · b + b · b + a · a− a · b− a · b + b · b= 2a · a + 2b · b2= 2‖a‖2 + 2‖b‖2
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 18 / 30
Example
‖a + b‖2 + ‖a− b‖2 = 2‖a‖2 + 2‖b‖2
Given a parallelogram ABCD, if wedenote the length of the line segmentAC by |AC | and similarly for othersegments, we have|AC |2 + |BD|2 = |AB|2 + |CD|2 +
|BC |2 + |AD|2 = 2 |AB|2 + 2 |BC |2
−→AC =
−→AB +
−→BC = a + b−→
DB =−→DA+
−→AB = −b + a = a− b
‖−→AB‖ = ‖
−→DC‖ = ‖a‖
‖−→AD‖ = ‖
−→BC‖ = ‖b‖
Definition ((This will not appear in the tests))Read section 2.6.5 for
compv(u) :=u · v‖v‖
(a scalar) and projv(u) =u · v‖v‖2
v (a vector)
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 19 / 30
Classwork
Let θ be the angle between vectors
u =
102
and v =
1−20
.Find cos θ and sin θ
Lecture resumes at 7:45.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 20 / 30
Solution
u · v =
102
· 1−20
= 1 · 1+ 0 · (−2) + 2 · 0 = 1
Meanwhile
‖u‖ =√
12 + 02 + 22 =√5, ‖v‖ =
√12 + (−2)2 + 02 =
√5
Thus,
cos θ =u · v‖u‖‖v‖
=1√
5 ·√5=
15
From cos2 θ + sin2 θ = 1, we have
sin2 θ = 1− cos2 θ = 1− 125
=2425
Notice that θ ≤ 180◦, which indicates that sin θ ≥ 0.
sin θ =
√2425
=2√6
5.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 21 / 30
Cross Product ONLY in R3
DefinitionGiven
u =
u1u2u3
and v =
v1v2v3
,the cross product of u and v is defined by
u× v =
u1u2u3
×v1v2v3
=
u2v3 − u3v2u3v1 − u1v3u1v2 − u2v1
.Example
u =
341
v =
201
=⇒ u× v =
4 · 1− 1 · 01 · 2− 3 · 13 · 0− 4 · 2
=
4−1−8
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 22 / 30
Geometric Meaning
u× v is orthogonal(perpendicular) to both uand v.‖u× v‖ = ‖u‖ · ‖v‖ sin θ is the area of theparallelogram spanned by u and v.u, v, and u× v form a right-hand system.
Use your right handthumb uindex finger vthen the middle finger givesyour the direction of u× v.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 23 / 30
Right-hand coordinate system
i =
100
, j =
010
, k =
001
Any vector in R3 can be uniquely written as a linear combination of i, j, andk: xy
z
= x i + y j + zk = x
100
+ y
010
+ z
001
Table
i× i = 0 i× j = k i× k = −jj× i = −k j× j = 0 j× k = ik× i = j k× j = −i k× k = 0
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 24 / 30
Laws
i× i = 0 i× j = k i× k = −jj× i = −k j× j = 0 j× k = ik× i = j k× j = −i k× k = 0
1 u× v = −v × u2 u× u = 0 (sin 0◦ = 0)3 u× (v + w) = u× v + u×w,4 (u + v)×w = u×w + v ×w,5 (ku)× v = k(u× v) = u× (kv)
Example
u =
341
v =
201
u× v = (3i + 4j + k)× (2i + k)3,4,5= 6i× i + 3i× k + 8j× i + 4j× k + 2k× i + k× k
Table = 60− 3j− 8k + 4i + 2j + 0= 4i− j− 8k
=
4−1−8
=
4 · 1− 1 · 01 · 2− 3 · 13 · 0− 4 · 2
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 25 / 30
Area
1. Find the area of the parallelogram spanned by
u =
341
v =
201
.Answer is
‖u× v‖ =∥∥∥∥ 4−1−8
∥∥∥∥ =√
42 + (−1)2 + (−8)2 =√16+ 1+ 64 =
√81 = 9.
2. Find the area of the triangle that has
u =
341
v =
201
as two of its sides.Answer is
12‖u‖‖v‖ sin θ = 1
2‖u× v‖ = 92
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 26 / 30
Box product
Addition— u + v vector+vector=vector;Scalar Multiplication— ku scalar·vector=vector;Dot product— u · v vector·vector=scalar/number;Cross product— u× v vector×vector=vector;Norm/Length— ‖u‖ ‖vector‖ =scalar/number;Normalization— 1
‖u‖u =unit vector;
DefinitionThe box product of three vectors u, v, and w is (u× v) ·w. It is equal to thesigned volume of the parallelepiped spanned by u, v, and w. The sign is
positive, if u, v, and w form a right-handed system;negative, if u, v, and w form a left-handed system;
The actual, unsigned volume is
volume = |(u× v) ·w| .
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 27 / 30
Box product
Volume = Base Area × HeightBase Area = ‖u× v‖Height = ‖w‖ cos θVolume = ‖u× v‖‖w‖ cos θ = (u× v) ·w
Fact(u× v) ·w = (v ×w) · u = (w × u) · v See Proposition 2.51 of the textbook.
ExampleFind the volume of the parallelepiped determined by the vectors 1
−7−5
, 1−2−6
, and −32
3
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 28 / 30
Box product
1−7−5
, 1−2−6
, and −32
3
1−7−5
× 1−2−6
=
(−7) (−6)− (−5) (−2)(−5) 1− 1 (−6)1 (−2)− (−7) 1
=
3215
3215
· −32
3
= 32 · (−3) + 1 · 2+ 5 · 3 = −79
Volume = |−79| = 79.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 29 / 30
Summary of Chpt. 2
1 Points and Vectors2 Operations:
Addition—u + v
Scalar Multiplication—ku
Dot product—u · vCross product—u× v
Norm/Length—‖u‖;Normorlization— 1
‖u‖u
Box Product (u× v) · w
Definition (Calculations)
Laws/Properties
Geometric Interpretations
3 Linear combinations ⇐⇒ Linear systems
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 16th, 2019 30 / 30
MATH-1030Matrix Theory & Linear Algebra I
Lin Jiu
Dalhousie University
May 21st, 2019
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 21st, 2019 1 / 17
A system of linear equations
Standard form3x + 2y + 4z = 7
x − y + z = 22y + 3z = 5
Vector form x
310
+ y
2−12
+ z
413
=
725
=
3x + 2y + 4zx − y + z
0x + 2y + 3z
Goal of the chapter: Matrix form3 2 4
1 −1 10 2 3
·xyz
=
725
←− GOAL
Augmented matrix: 3 2 4 71 −1 1 20 2 3 5
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 21st, 2019 2 / 17
DefinitionDefinitionAn m × n-matrix is an array of scalars with m rows and n columns.
m rows
a11 a12 a13 · · · a1na21 a22 a23 · · · a2n...
......
. . ....
am1 am2 am3 · · · amn
︸ ︷︷ ︸
n columns
The dimension of the matrix is m × n.The scalars in the matrix are called entries. An entry at the ith row and jthcolumn is an (i , j)-entry.
Example
If A =
[2 3 10 0 2
], 2 is the (1, 1)-entry; both (2, 1) and (2, 2) entries are 0. The
dimension of A is 2× 3.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 21st, 2019 3 / 17
Equality, Addition
Two matrices are equal if they have the same dimension (i.e., the same number ofrows and columns) and the same entries.
Example [2 3 10 1 1
]=
[2 3 10 1 1
] [0 0 00 0 0
]6=
0 00 00 0
DefinitionWe can add two m × n matrices (same dimension) and get an m × n matrix.a11 · · · a1n
.... . .
...am1 · · · amn
+
b11 · · · b1n...
. . ....
bm1 · · · bmn
=
c11 · · · c1n...
. . ....
cm1 · · · cmn
where
cij = aij + bij , 1 ≤ i ≤ m, 1 ≤ j ≤ n
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 21st, 2019 4 / 17
Example
1 2 3−1 05 0
+
2 21 10 −1
=
2+ 2 3+ 2−1+ 1 0+ 15+ 0 0+ (−1)
=
4 50 15 −1
2 2 3
−1 05 0
+
[2 1 02 1 −1
]not defined
RemarkRecall the addition of vectors:u1
...un
+
v1...vn
=
u1 + v1...
un + vn
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 21st, 2019 5 / 17
Scalar multiplication
Definition
k ·
a11 · · · a1n...
. . ....
am1 · · · amn
=
ka11 · · · ka1n...
. . ....
kam1 · · · kamn
Example
2 ·[1 10 3
]=
[2 20 6
]
RemarkRecall the scalar multiplication of vectors:
a
u1...un
=
au1...
aun
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 21st, 2019 6 / 17
Negation, Zero Matrix
Definition
1 If A =
a11 · · · a1n...
. . ....
am1 · · · amn
, then −A := (−1) · A =
−a11 · · · −a1n...
. . ....
−am1 · · · −amn
−
2 3−1 05 0
=
−2 −31 0−5 0
2 The zero matrix of dimension m × n is the m × n matrix with all entries 0:
0 =
0 0 · · · 00 0 · · · 0...
.... . .
...0 0 · · · 0
︸ ︷︷ ︸
n columns
m rows
RemarkIt depends on the context that whether 0is the scalar or the zero matrix of acertain dimension.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 21st, 2019 7 / 17
Laws
Let A, B, C , and 0 have the same dimen-sion.
Addition1 (A+ B) + C = A+ (B + C)2 A+ B = B + A3 A+ 0 = A = 0 + A4 A+ (−A) = 0
Scalar Multiplication1 (k + `)A = kA+ `A2 k(A+ B) = kA+ kB3 k(`A) = (k`)A4 1A = A, (−1)A = −A, 0A = 0
For vectors:(A1) u + v = v + u(A2) (u + v) + w =
u + (v + w)
(A3) u + 0 = u(A4) u + (−u) = 0
(SM1) k (u + v) = ku + kv(SM2) (k + `)u = ku + `u(SM3) k (`u) = (k`)u(SM4) 1 · u = u
FactA column vector in Rn is a special n × 1 matrix.
For example, v =
[12
]is a 2× 1 matrix.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 21st, 2019 8 / 17
MultiplicationFact (Row and Column Vectors)
1 A column vector
x1...xn
is an n × 1 matrix.
2 A row vector[x1, · · · , xn
]is a 1× n matrix.
3 A scalar x is, sometimes, usefully, considered as a 1× 1 matrix[x]. If the
matrix contains only 1 entry, the square brackets “[]” can be considered asredundant.
Definition (Matrix Multiplication)We can multiply an m × n matrix by a k × ` matrix ONLY if n = k . In this case,the product is an m × ` matrix. In generala11 · · · a1n
.... . .
...am1 · · · amn
b11 · · · b1`
.... . .
...bn1 · · · bn`
=
c11 · · · c1`...
. . ....
cm1 · · · cm`
,
where cij = ai1b1j + ai2b2j + · · ·+ ainbnjLin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 21st, 2019 9 / 17
Multiplication
Remember this: ci j is determined by the i-th row of the first matrix and the j-thcolumn of the second matrix. More precisely,
the i-th row of the first matrix is the row vector:
[ai1, ai2, . . . , ain
] column vector=⇒
ai1ai2...ain
∈ Rn
the j-th column of the second matrix is
b1jb2j...bnj
∈ Rn
compute the dot productai1...ain
·b1j
...bnj
= ai1b1j + ai2b2j + · · ·+ ainbnj = cij .
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 21st, 2019 10 / 17
Example
Example
[1 3 52 4 6
]·
1 23 42 42 6
=? dimensions: 2×3 and 4×2 3 6= 4 =⇒ not defined.
RemarkThis is why we have dot and cross product for vectors but not regular“multiplication” .
u =
u1...un
v =
v1...vn
Both have dimension n × 1.
# of colums in u = 1 6= n = # of rows in v
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 21st, 2019 11 / 17
Example
[2 1 1 −10 2 1 4
]1 0 10 1 21 0 01 1 0
=?
First of all, the dimensions are 2× 4 and 4× 3. The result is a matrix ofdimension 2× 3. Now, we write down the two rows of the first matrix:
R1 :[2 1 1 −1
]R2 :
[0 2 1 4
] column vectors=⇒ v1 =
211−1
v2 =
0214
and the three column vectors from the second matrix
w1 =
1011
, w2 =
0101
, w3 =
1200
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 21st, 2019 12 / 17
Example
v1 =
211−1
v2 =
0214
w1 =
1011
, w2 =
0101
, w3 =
1200
[2 1 1 −10 2 1 4
]1 0 10 1 21 0 01 1 0
=
[a11 a12 a13
a21 a22 a23
]
a11 = v1 · w1 = 2 · 1 + 1 · 0 + 1 · 1 + (−1) · 1 = 2
a12 = v1 · w2 = 2 · 0 + 1 · 1 + 1 · 0 + (−1) · 1 = 0
a13 = v1 · w3 = 2 · 1 + 1 · 2 + 1 · 0 + (−1) · 0 = 4
a21 = v2 · w1 = 0 · 1 + 2 · 0 + 1 · 1 + 4 · 1 = 5
a22 = v2 · w2 = 0 · 0 + 2 · 1 + 1 · 0 + 4 · 1 = 6
a23 = v2 · w3 = 0 · 1 + 2 · 2 + 1 · 0 + 4 · 0 = 4
[2 1 1 −10 2 1 4
]1 0 10 1 21 0 01 1 0
‖[
v1 · w1 v1 · w2 v1 · w3
v2 · w1 v2 · w2 v2 · w3
]‖[
2 0 45 6 4
]Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 21st, 2019 13 / 17
Alternative calculation
column-method: Let A be an m × n matrix and v ∈ Rn is a vector, which can betreated as an n × 1 matrix. Av is an m × 1 matrix as well as a vector in Rm.row-method: Let A be an m × n matrix and u a 1×m matrix. Namelyu =
[u1 · · · um
]is a row vector. uA is a 1× n matrix as well as a row vector
in Rn.
Example
[2 1 1 −10 2 1 4
]1011
=
[2 1 1 −10 2 1 4
]1011
= 1
[20
]+ 0
[12
]+ 1
[11
]+ 1
[−14
]=
[20
]+
[00
]+
[11
]+
[−14
]=
[25
]Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 21st, 2019 14 / 17
Alternative calculation
[2 1 1 −10 2 1 4
]1011
= 1[20
]+ 0
[12
]+ 1
[11
]+ 1
[−14
]=
[25
]
[2 1 1 −10 2 1 4
]0101
= 0[20
]+ 1
[12
]+ 0
[11
]+ 1
[−14
]=
[12
]+
[−14
]=
[06
]
[2 1 1 −10 2 1 4
]1200
= 1[20
]+ 2
[12
]+ 0
[11
]+ 0
[−14
]=
[20
]+
[24
]=
[44
]
[2 1 1 −10 2 1 4
]︸ ︷︷ ︸
A
1 0 10 1 21 0 01 1 0
↑ ↑ ↑w1 w2 w3
=[Aw1 Aw2 Aw3
]=
[2 0 45 6 4
]
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 21st, 2019 15 / 17
Row and column methods
Row-method and column-method: Read Proposition 4.23 and 4.29. Let Am×n andBn×` be matrices such that
A =
a1...
am
and B =[b1 · · · b`
], for
{row vectors a1, . . . , ancolumn vectors b1, . . . ,b`
in Rn
AB =
colume method︷ ︸︸ ︷[Ab1 · · · Ab`
]=
row method︷ ︸︸ ︷a1B...
amB
=
a1b1 · · · a1b`
.... . .
...amb1 · · · amb`
Example (Column method)
[2 1 1 −10 2 1 4
]A
1 0 10 1 21 0 01 1 0
w1 w2 w3
= [Aw1 Aw2 Aw3] Aw1 = 1[20
]+ 0
[12
]+ 1
[11
]+ 1
[−14
]=
[25
]
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 21st, 2019 16 / 17
Example (row method)
[2 1 1 −10 2 1 4
]1 0 10 1 21 0 01 1 0
=
[2 0 45 6 4
]
[2 1 1 −1
] 1 0 10 1 21 0 01 1 0
= 2[1 0 1
]+ 1
[0 1 2
]+ 1
[1 0 0
]− 1
[1 1 0
]=
[2 0 2
]+[0 1 2
]+[1 0 0
]−[1 1 0
]=
[2 0 4
][0 2 1 4
] 1 0 10 1 21 0 01 1 0
= 0[1 0 1
]+ 2
[0 1 2
]+ 1
[1 0 0
]+ 4
[1 1 0
]=
[0 0 0
]+[0 2 4
]+[1 0 0
]+[4 4 0
]=
[5 6 4
]Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 21st, 2019 17 / 17
MATH-1030Matrix Theory & Linear Algebra I
Lin Jiu
Dalhousie University
May 23rd, 2019
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 1 / 33
Multiplication
A =
a11 · · · a1n...
. . ....
am1 · · · amn
m×n
, B =
b11 · · · b1`...
. . ....
bn1 · · · bn`
k×`
AB exists ONLY if n = k .
AB =
c11 · · · c1`...
. . ....
cm1 · · · cm`
m×`
, cij = ai1b1j + · · ·+ ainbnj
formula, row method, column method
A =
a1...
am
B =
[b1 · · · b`
]⇒ AB =
a1B...
amB
=[Ab1 · · · Ab`
]=
a1b1 · · · a1b`
.... . .
...amb1 · · · amb`
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 2 / 33
Example[1 23 4
] [1 01 1
]=?
First of all the dimensions are 2× 2 and 2× 2. ⇒ Result: a 2× 2-matrix.
A =
[1 23 4
]⇒{
a1 = [1 2]a2 = [3 4]
B =
[1 01 1
]⇒ b1 =
[11
], b2 =
[01
](I) Formula:
a1b1 =
[12
]·[11
]= 1 · 1+ 2 · 1 = 3
a1b2 =
[12
]·[01
]= 1 · 0+ 2 · 1 = 2
a2b1 =
[34
]·[11
]= 3 · 1+ 4 · 1 = 7
a2b2 =
[34
]·[01
]= 3 · 0+ 4 · 1 = 4
Thus,
AB =
[a1b1 a1b2a2b1 a2b2
]=
[3 27 4
]Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 3 / 33
Example
A =
[1 23 4
]⇒{
a1 = [1 2]a2 = [3 4]
B =
[1 01 1
]⇒ b1 =
[11
], b2 =
[01
](II) Column method: AB = A [b1 b2] = [Ab1 Ab2]
Ab1 =
[1 23 4
] [11
]= 1
[13
]+ 1
[24
]=
[37
]Ab2 =
[1 23 4
] [01
]= 0
[13
]+ 1
[24
]=
[24
] ⇒ AB =
[3 27 4
]
(III) Row method: AB =
[a1a2
]B =
[a1Ba2B
]a1B = [1 2]
[1 01 1
]= 1 [1 0] + 2 [1 1] = [1 0] + [2 2] = [3 2]
a2B = [3 4][1 01 1
]= 3 [1 0] + 4 [1 1] = [3 0] + [4 4] = [7 4]
⇒ AB =
[3 27 4
]Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 4 / 33
Remarks
1 Let
v =
a1...an
, w =
b1...bn
∈ Rn =⇒ v ·w = a1b1 + · · ·+ anbn
[a1 · · · an
]︸ ︷︷ ︸1×n
b1...bn
︸ ︷︷ ︸n×1
= [v ·w]
We identify a 1× 1 matrix and a scalar.2 Recall our goal:
3x + 2y + 4z = 7x − y + z = 2
0x + 2y + 3z = 5⇔
3 2 41 −1 10 2 3
·xyz
=
725
3 2 41 −1 10 2 3
·
xyz
= x
310
+ y
2−12
+ z
413
(columnmethod
)=
3x + 2y + 4zx − y + z
0x + 2y + 3z
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 5 / 33
Identity matrix
DefinitionThe n-th identity matrix is a square matrix of dimension n × n, given by
In =
1 0 0 · · · 00 1 0 · · · 00 0 1 · · · 0...
......
. . ....
0 0 0 · · · 1
n×n
Example
I2 =
[1 00 1
]I3 =
1 0 00 1 00 0 1
I4 =
1 0 0 00 1 0 00 0 1 00 0 0 1
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 6 / 33
Laws
1 (AB)C = A(BC )
2 Most of the times,AB 6= BA
3 A(B + C ) = AB + AC
4 (A+ B)C = AC + BC
5 k(AB) = (kA)B = A(kB)
6 If A has dimension m × n,AIn = A = ImA.
ALWAYS remember that sincematrix multiplication is not com-mutative, we need to specify mul-tiplying a matrix to the LEFT orto the RIGHT.
Example
1.[0 10 0
][1 23 4
]=
[3 40 0
][1 23 4
] [0 10 0
]=
[0 10 3
]2.[a b
] [cd
]=[ac + bd
][cd
] [a b
]=
[ac bcad bd
]3.[0 10 0
] [13
]=
[30
][13
] [0 10 0
]Not defined
4.[
1 00 1
] [1 2 32 4 1
]=
[1 2 32 4 1
][1 2 32 4 1
] 1 0 00 1 00 0 1
=
[1 2 32 4 1
]Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 7 / 33
Remark
Please REMEMBER: matrix multiplication is totally different from themultiplication of numbers/scalars.
3 · 4 = 12 = 4 · 3 but most of the times AB 6= BA.x2 = 0⇒ x = 0, but A2 = AA = 0 does not mean A = 0.
A =
[0 10 0
]⇒ A2 =
[0 · 0+ 1 · 0 0 · 1+ 1 · 00 · 0+ 0 · 0 0 · 1+ 0 · 0
]=
[0 00 0
]This actually
shows we can have A 6= 0 6= B, but AB = 0Recall that, for numbers, if xy = 0 then either x = 0 or y = 0.ab = ac ⇒ b = c if a 6= 0 but AB = AC does not imply that B = C .AB = AC ⇒ AB − AC = 0 ⇒ A (B − C ) = 0
Let A =
[0 10 0
], B =
[0 30 0
], C =
[0 20 0
]⇒ B − C = A
and A(B − C ) = AA = A2 = 0.Exercise 4.4.12–Exercise 4.4.18
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 8 / 33
Simplification
(AB)C = A(BC ) A(B + C ) = AB + AC k(AB) = (kA)B = A(kB)AB 6= BA (A+ B)C = AC + BC AIn = A = ImA
Example
Let X be 2× 2, satisfying 3X[2 13 4
]+
[23
] [1 4
]= X
[1 23 4
] [−1 53 −1
]X =?
3X[2 13 4
]= X
(3[2 13 4
])= X
[6 39 12
][23
] [1 4
]=
[2 · 1 2 · 43 · 1 3 · 4
]=
[2 83 12
][1 23 4
] [−1 53 −1
]=
[1 · (−1) + 2 · 3 1 · 5+ 2 · (−1)3 · (−1) + 4 · 3 3 · 5+ 4 · (−1)
]=
[5 39 11
]X
[6 39 12
]+
[2 83 12
]= X
[5 39 11
]⇒ X
[6 39 12
]− X
[5 39 11
]= −
[2 83 12
]⇒[−2 −8−3 −12
]= X
([6 39 12
]−[5 39 11
])= X
[1 00 1
]= X .
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 9 / 33
Further question
3X[2 13 4
]+
[23
] [1 4
]= X
[1 23 4
] [−1 53 −1
]⇒ X = −
[2 83 12
]Problem
How about 3X[2 03 4
]+
[23
] [1 4
]= X
[1 23 4
] [−1 53 −1
]?
3X[2 03 4
]− X
[1 23 4
] [−1 53 −1
]= X
[6 09 12
]− X
[5 39 11
]= X
[1 −30 1
]Namely, it holds that X
[1 −30 1
]= −
[23
] [1 4
]= −
[2 83 12
]Of course, we can set X =
[a bc d
]and observe that
−[2 83 12
]=
[a bc d
] [1 −30 1
]=
[a −3a+ bc −3c + d
]Use the FOUR equations to solve for a, b, c , and d . Or, we can introduce theinverse matrix.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 10 / 33
Matrix Inverses
Definition1. Let A be a matrix. We say that A is left invertible if there exists a matrix Bsuch that
BA = I (Here we do not specify the dimension)
Then B is called a left inverse of A. Am×n ⇒ Bn×m2. A is right invertible if there exists a matrix B such that
AB = I .
Then B is called a right inverse of A.3. A is invertible if it has both left and right inverses. In this case, A is a squarematrix (Theorem 4.48) and both left and right inverses coincide. Namely, thereexist a (square) matrix B (with the same dimension as A) such that
AB = BA = I . We write B = A−1.
FactNot every matrix is invertible. Not every square matrix is invertible.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 11 / 33
Examples
1 Let A =
[1 0 00 1 0
]and B =
1 00 10 0
. We can compute that
AB =
[1 0 00 1 0
] 1 00 10 0
=
[1 00 1
]= I
and
BA =
1 00 10 0
[ 1 0 00 1 0
]=
1 0 00 1 00 0 0
6= I
B is a right inverse of A, but not a left inverse.
2 Let A =
[1 −30 1
]and B =
[1 30 1
].
AB =
[1 −30 1
] [1 30 1
]=
[1 · 1− 3 · 0 1 · 3− 3 · 10 · 1+ 1 · 0 0 · 3+ 1 · 1
]=
[1 00 1
]= I = BA.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 12 / 33
Example
[1 −30 1
] [1 30 1
]=
[1 30 1
] [1 −30 1
]= I ⇒
[1 30 1
]=
[1 −30 1
]−1
3X[2 03 4
]+
[23
] [1 4
]= X
[1 23 4
] [−1 53 −1
]⇒ X
[1 −30 1
]= −
[2 83 12
]Now, multiplying both sides by
[1 30 1
], to the right
X
[1 −30 1
] [1 30 1
]= −
[2 83 12
] [1 30 1
]The left-hand side is X
[1 −30 1
] [1 30 1
]= XI = X
The right-hand side is
−[2 83 12
] [1 30 1
]= −
[2 · 1+ 8 · 0 2 · 3+ 8 · 13 · 1+ 12 · 0 3 · 3+ 12 · 1
]= −
[2 143 21
]X =
[−2 −14−3 −21
]Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 13 / 33
Find the inverses
Given A =
1 2 10 1 12 4 3
, check if A is invertible. If so, find the inverse.
Method: Make an augmented system[A | I
]. Attempt row reductions to get[
I | B]. If this succeeds, then A is invertible and B = A−1.
[A | I
]=
1 2 1 1 0 00 1 1 0 1 02 4 3 0 0 1
R3←R3−2R1∼ 1 2 1 1 0 0
0 1 1 0 1 00 0 1 −2 0 1
R1←R1−2R2∼
1 0 −1 1 −2 00 1 1 0 1 00 0 1 −2 0 1
R1←R1+R3R2←R2−R3∼
1 0 0 −1 −2 10 1 0 2 1 −10 0 1 −2 0 1
A is invertible with A−1 =
−1 −2 12 1 −1−2 0 1
Exercise: check AA−1 = A−1A = I
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 14 / 33
Find the inverses
Given A =
[1 −30 1
], check if A is invertible. If so, find the inverse.
[A | I
]=
[1 −3 1 00 1 0 1
]R1←R1+3R2∼
[1 0 1 30 1 0 1
]⇒ A−1 =
[1 30 1
]
RemarkAfter Midterm II, we will study an alternative method to compute the inverse,which sometimes is simpler for 2× 2 matrices.
Example
Let A =
1 −2 21 0 22 −2 4
. Find A−1 if it exists.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 15 / 33
Example
[A | I
]=
1 −2 2 1 0 01 0 2 0 1 02 −2 4 0 0 1
R2←R2−R1R3←R3−2R1∼
1 −2 2 1 0 00 2 0 −1 1 00 2 0 −2 0 1
R3←R3−R2∼
1 −2 2 1 0 00 2 0 −1 1 00 0 0 −1 −1 1
From here, we see rank(A) = 2 < 3. So the (unique) reduced echelon form of A is1 0 0
0 1 00 0 0
6=1 0 00 1 00 0 1
.
There is no way to obtain I on the left-hand side of this augmented matrix.Hence, there is no way to complete the algorithm, and the inverse of A does notexist. Namely, A is NOT invertible.Remark (A being n × n )If rank(A) = n,
[A | I
]∼[I | B
]succeeds to get B = A−1. (full rank)
If rank(A) < n, then A is not invertible.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 16 / 33
Laws & Properties
1 Inverse is unique. If both B and C are inverses of A, then B = C .
Proof.C = IC = (BA)C = B(AC ) = BI = B.
2 I−1 = I .3 If both A and B are invertible, (AB)−1 = B−1A−1. (B−1A−1AB = I .)
(1/(xy) = (1/x) (1/y))4 If A is invertible, so is A−1. Moreover, (A−1)−1 = A.5 If A is invertible and k 6= 0. (kA)−1 = k−1A−1 = 1
kA−1.
Theorem1 If A is invertible, then A is a square matrix.2 If A is a square matrix, then the following are equivalent:
A is invertible;A has a left inverse;A has a right inverse;
3 Let A be an m × n matrix.If A is left invertible, then m ≥ n.If A is right invertible, then m ≤ n.If A is invertible, then m = n.
rank(A) ≤ m, rank(A) ≤ nleft invertible iff rank(A) = nright invertible iff rank(A) = m
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 17 / 33
Classwork
Let
A =
1 0 11 −1 11 1 −1
Determine whether A is invertible. If A isinvertible, find A−1.
Lecture resumes at 7:50
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 18 / 33
Solution
[A | I
]=
1 0 1 1 0 01 −1 1 0 1 01 1 −1 0 0 1
R2←R2−R1R3←R3−R1∼
1 0 1 1 0 00 −1 0 −1 1 00 1 −2 −1 0 1
R2←−R2∼
1 0 1 1 0 00 1 0 1 −1 00 1 −2 −1 0 1
R3←R3−R2∼
1 0 1 1 0 00 1 0 1 −1 00 0 −2 −2 1 1
R3←− 1
2R3∼ 1 0 1 1 0 0
0 1 0 1 −1 00 0 1 1 − 1
2 − 12
R1←R1−R3∼
1 0 0 0 12
12
0 1 0 1 −1 00 0 1 1 − 1
2 − 12
A−1 =
0 12
12
1 −1 01 − 1
2 − 12
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 19 / 33
Solve system of equations
Consider the systemx + z = 1
x − y + z = 3x + y − z = 2
. We can write it as Aa = b where
A =
1 0 11 −1 11 1 −1
, a =
xyz
, b =
132
.
From the previous slide, we have
A−1 =
0 12
12
1 −1 01 − 1
2 − 12
Multiplying both sides of Aa = b by A−1 to the LEFT:
A−1Aa = A−1b⇐⇒ a = A−1b
a =
xyz
= A−1b =
0 12
12
1 −1 01 − 1
2 − 12
132
=
0 · 1+ 12 · 3+
12 · 2
1 · 1− 1 · 3+ 0 · 21 · 1− 1
2 · 3−12 · 2
=
52−2− 3
2
.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 20 / 33
Solve system of equations
How about another systemx + z = 0
x − y + z = 1x + y − z = 3
?
013
xyz
= A−1
013
=
0 12
12
1 −1 01 − 1
2 − 12
013
=
2−1−2
This illustrates that for a system Ax = b where A−1 exists, it is easy to find thesolution when the vector b is changed.
x + z = 0x − y + z = 0x + y − z = 0
(homogeneous)
⇒
xyz
= A−1
000
=
0 12
12
1 −1 01 − 1
2 − 12
000
=
0 · 0+ 12 · 0+
12 · 0
1 · 0− 1 · 0+ 0 · 01 · 0− 1
2 · 0−12 · 0
=
000
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 21 / 33
Application (Solve matrix equation)
Find a matrix X such that [1 11 2
]X =
[1 23 4
]
Let A =
[1 11 2
]and C =
[1 23 4
].
AX = C =⇒ A−1AX = A−1C = IX = X .[1 1 1 01 2 0 1
]R2←R2−R1∼
[1 1 1 00 1 −1 1
]R1←R1−R2∼
[1 0 2 −10 1 −1 1
]Thus,
X = A−1C =
[2 −1−1 1
] [1 23 4
]=
[2 · 1− 1 · 3 2 · 2− 1 · 4−1 · 1+ 1 · 3 −1 · 2+ 1 · 4
]=
[−1 02 2
].
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 22 / 33
Application (Solve matrix equation)
Find a matrix X such that[1 11 2
]X =
[1 23 4
]A =
[1 11 2
]
Let X =
[a bc d
]. Recall the column method:
[1 11 2
] [ac
]=
[13
] [1 11 2
] [bd
]=
[24
][13
]=
[1 11 2
] [ac
]= a
[11
]+ c
[12
]=
[a+ ca+ 2c
]⇒ c = 2, a = −1
[24
]=
[1 11 2
] [bd
]= b
[11
]+ d
[12
]=
[b + db + 2d
]⇒ d = 2, b = 0
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 23 / 33
Brief Explanation
Consider the example of A =
1 2 10 1 12 4 3
. Let B =
a b cd e fg h i
such that
AB = A
a b cd e fg h i
= I =
1 0 00 1 00 0 1
.
By the column method,
A
adg
=
100
A
beh
=
010
A
cfi
=
001
it is equivalent to three systems (with the same LHS A): Recall the reducedechelon form 1 2 1 1
0 1 1 02 4 3 0
Row operations∼∼∼∼∼∼
1 0 0 a0 1 0 d0 0 1 g
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 24 / 33
Brief Explanation
Similarly 1 2 1 00 1 1 12 4 3 0
Row operations∼∼∼∼∼∼
1 0 0 b0 1 0 e0 0 1 h
1 2 1 0
0 1 1 02 4 3 1
Row operations∼∼∼∼∼∼
1 0 0 c0 1 0 f0 0 1 i
KEY: The row operations for the three systems are the same, completelydetermined by A (the coefficients) 1 2 1 1 0 0
0 1 1 0 1 02 4 3 0 0 1
Row operations∼∼∼∼∼∼
1 0 0 a b c0 1 0 d e f0 0 1 g h i
A−1 =
a b cd e fg h i
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 25 / 33
Symbolic matrix equation
Assume all matrices A, B, C and X are n × n and also assume all matrices areinvertible as needed. If X satisfies AX (X + C )−1 = B, solve for X .Solution. Multiplying both sides by X + C to the right
AX (X + C )−1 (X + C ) = B (X + C ) (∗)
AX = BX + BC =⇒ AX − BX = BC =⇒ (A− B)X = BC
Multiply both sides by (A− B)−1 to the left
(A− B)−1 (A− B)X = (A− B)−1BC = X . (∗∗)
X = (A− B)−1BC
If you multiply matrices to the wrong side, you will have either in (∗)
(X + C )AX (X + C )−1 = (X + C )B
or in (∗∗)(A− B)X (A− B)−1 = BC (A− B)−1
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 26 / 33
Transpose
DefinitionThe transpose of an m × n matrix A, denoted by AT , is an n ×m matrix, whose(i , j)-entry is just the (j , i)-entry of A. Namely, it is obtained from A by switchingrows and columns, , i.e., the i-th column of AT is the i-th row of A. Moreprecisely,
A =
a11 · · · a1na21 · · · a2n...
. . ....
am1 · · · amn
=⇒ AT =
a11 a21 · · · am1...
.... . .
...a1n a2n · · · amn
A =
1 2 34 5 67 8 9
=⇒ AT =
1 4 72 5 83 6 9
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 27 / 33
Transpose
Example
B =
[1 0 3−2 1 0
]=⇒ BT =
1 −20 13 0
v =
1024
=⇒ vT =[1 0 2 4
] [1 0 2 4
]T=
1024
= v
RemarkNow we see that the transpose of a row ( resp. column) vector is a column(resp. row) vector.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 28 / 33
Dot product
Recall that for v =
v1...vn
and w =
w1...wn
v ·w = v1w1 + · · ·+ vnwn =[v1 · · · vn
] w1...wn
= vTw.
RemarkTo save space, we sometimes use the transpose of a row vector to express thecorresponding column vector. Namely,12
3
=[1 2 3
]T.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 29 / 33
Laws of transposes
(A+ B)T = AT + BT (rA)T = rAT
(AT )T = A (AB)T = BTAT
0T = 0 IT = I (A−1)T = (AT )−1
ExampleLet
A =
1 30 1−1 0
B =
[1 23 −1
]=⇒ C = AB =
10 −13 −1−1 −2
BT =
[1 32 −1
]AT =
[1 0 −13 1 0
]=⇒ BTAT =
[10 3 −1−1 −1 −2
]= CT
Namely, Let v =[0 1
]Tand w =[1 3
]T .v ·w = 3 = w · v ⇔ vTw = 3 = wTv.
v ·w = vTw = 3 = [3] = [3]T =(vTw
)T= wT
(vT)T
= wTv = w · v
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 30 / 33
(Anti-)Symmetric matrices
DefinitionA matrix A is called symmetric if AT = A. It is said to be antisymmetric(sometimes also called skew symmetric) if AT = −A.
Example
A =
1 2 32 4 103 10 15
I4 =
1 0 0 00 1 0 00 0 1 00 0 0 1
0 =
[0 00 0
]are symmetric.
B =
0 2 3−2 0 10−3 −10 0
C =
0 1 3−1 0 2−3 −2 0
0 =
[0 00 0
]are antisymmetric.
[1 20 1
]is neither.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 31 / 33
Trace (*)
DefinitionLet A be an n× n-matrix (square matrix). The trace, denoted by tr(A) is the sumof its diagonal entries.
A =
a11 · · · a1n...
. . ....
an1 · · · ann
=⇒ tr(A) = a11 + a22 + · · ·+ ann
Example
A =
1 2 32 4 103 10 15
=⇒ tr(A) = 1+ 4+ 15 = 20.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 32 / 33
Summary of Chpt. 3
The only object is matrix.1 dimension (check before calculation), entry, row vectors & column vectors.2 addition & scalar multiplication similar to those for vectors3 multiplication:
requirement for dimensionscalculation formula, row & column methods
4 inversedefinition and laws/propertiesfind the inverseapplications: solving linear systems matrix equations (including symbolicequations )
5 transpose
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 33 / 33
MATH-1030Matrix Theory & Linear Algebra I
Lin Jiu
Dalhousie University
May 28th, 2019
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 1 / 34
Span
DefinitionGiven vectors v1, . . . , vk in Rn, the span of v1, . . . , vk is the set of all their linearcombinations. In symbols,
span {v1, . . . , vk} = {a1v1 + a2v2 + · · ·+ akvk | a1, . . . , ak ∈ R} .
Example
Which of the following vectors are in the span of v1 =
122
and v2 =
202
?(a) u =
524
(b)w =
323
(c) t =
1−20
What do we want?
Determine whether there exist a1 and a2 such that
u = a1v1 + a2v2 w = a1v1 + a2v2 t = a1v1 + a2v2
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 2 / 34
Solution
v1 = [1 2 2]T , v2 = [2 0 2]T , u = [5 2 4]T , w = [3 2 3]T , t = [1 − 2 0]T .
a1v1 + a2v2 = a1
122
+ a2
202
=
1 22 02 2
[a1a2
]
(a)
1 2 52 0 22 2 4
(b)
1 2 32 0 22 2 3
(c)
1 2 12 0 −22 2 0
1 2 5 3 1
2 0 2 2 −22 2 4 3 0
R2←R2−2R1R3←R3−2R1∼
1 2 5 3 10 −4 −8 −4 −40 −2 −6 −3 −2
R2←− 1
4 R2R3←− 1
2 R3∼ 1 2 5 3 1
0 1 2 1 10 1 3 3
2 1
R3←R3−R2∼
1 2 5 3 10 1 2 1 10 0 1 1
2 0
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 3 / 34
Solution
v1 = [1 2 2]T , v2 = [2 0 2]T , u = [5 2 4]T , w = [3 2 3]T , t = [1 − 2 0]T . 1 2 5 3 12 0 2 2 −22 2 4 3 0
∼∼∼ 1 2 5 3 1
0 1 2 1 10 0 1 1
2 0
Only the system of (c) is consistent. Therefore, only t is the in the span of v1 andv2. t ∈ span {v1, v2}What is the solution? (a1, a2) = (−1, 1)
v2−v1 = [2 0 2]T−[1 2 2]T =([1 2 2]− [2 0 2]
)T= [1 −2 0]T = t
Is this solution important?No, the key is to know whether the system is consistent.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 4 / 34
Geometric Interpretation of Span
Let v1 and v2 be non-parallel vectors. Then, span {v1, v2}, the set of all linearcombinations, is a plane through the origin.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 5 / 34
Geometric Interpretation of Span
Remark
The span of 1 vector is usually a line. (span{0} is just the origin.)The span of 2 vectors is usually a plane. (It could be a line if both areparallel) ( If v = ku for some k (parallel), then au + bv = au + bku= (a+ bk)u — a line)The span of 3 vectors is usually a 3-dimensional space. (It could be a planeor a line)Higher-dimensional spaces:
Example (in R6)
v1 = [ 1 0 1 2 3 4 ]T , v2 = [ 1 1 1 2 2 2 ]T ,
v3 = [ 0 0 1 3 7 4 ]T , v4 = [ 1 0 1 0 3 0 ]T .
span {v1, v2, v3, v4}Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 6 / 34
Redundancy
Example
v1 =
110
, v2 =
012
, v3 =
122
.
They are not mutually parallel. However, it is not hard to see the fact:
v1 + v2 =
110
+
012
=
122
= v3
Thus, v3 is “redundant”, i.e., span {v1, v2, v3} = span {v1, v2}.span {v1, v2} ⊆ span {v1, v2, v3}: a1v1 + a2v2 = a1v1 + a2v2 + 0v3
span {v1, v2, v3} ⊆ span {v1, v2}:a1v1 + a2v2 + a3v3 = a1v1 + a2v2 + a3 (v1 + v2) = (a1 + a3) v1 +(a2 + a3) v2
RemarkThis is an example that a span of 3 vectors is a plane.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 7 / 34
Redundancy
DefinitionLet v1, . . . , vk be vectors and consider span {v1, . . . , vk}. We say that the vectorvj is redundant if vj is a linear combination of the vectors preceding it, i.e., vj isa linear combination of v1, . . . , vj−1. In this case, vj contributes nothing to thespan, i.e., span {v1, . . . , vk} = span {v1, . . . , vj−1, vj+1, . . . vk} .
ExampleFind the redundant vectors in the sequence
v1 =
110
, v2 =
220
, v3 =
101
, v4 =
−1−21
, v5 =
111
v1 is not redundant. There is no v0. In fact, v1 is never redundant unlessv1 = 0. span∅ = {0}v2 = 2v1, so it is redundant.v3 is not a linear combination of v1 and v2, so it is not redundant. why?
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 8 / 34
Example
v1 =
110
, v2 =
220
, v3 =
101
, v4 =
−1−21
, v5 =
111
v1 is not redundant. v2 is redundant. v3 is not redundant. 3rd component
v4? 1 2 1 −11 2 0 −20 0 1 1
R2←R2−R1∼ 1 2 1 −1
0 0 −1 −10 0 1 1
R3←R3+R2∼ 1 2 1 −1
0 0 −1 −10 0 0 0
consistent=⇒ redundant.v5? 1 2 1 −1 1
1 2 0 −2 10 0 1 1 1
R2←R2−R1∼ 1 2 1 −1 1
0 0 −1 −1 00 0 1 1 1
R3←R3+R2∼ 1 2 1 −1 1
0 0 −1 −1 00 0 0 0 1
inconsistent=⇒not redundant.
span {v1, v2, v3, v4, v5} = span {v1, v3, v5}
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 9 / 34
Casting-out algorithm
Input: A finite sequence of vectors v1, . . . , vk in Rn
Output: Redundant vectorsSteps: (1) Form an n × k matrix whose columns are the k given vectors.
(2) Reduce it to an echelon form.(3) Non-pivot columns correspond to redundant vectors.
ExampleFind the redundant vectors in the sequence
v1 =
110
, v2 =
220
, v3 =
101
, v4 =
−1−21
, v5 =
111
1 2 1 −1 1
1 2 0 −2 10 0 1 1 1
R2←R2−R1∼1 2 1 −1 1
0 0 −1 −1 00 0 1 1 1
R3←R3+R2∼1 2 1 −1 1
0 0 −1 −1 00 0 0 0 1
Non-pivot vectors are v2 and v4 , which are redundant.span {v1, v2, v3, v4, v5} = span {v1, v3, v5}
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 10 / 34
Example
Find the redundant vectors in the sequence
v1 =
0000
, v2 =
0110
, v3 =
0220
, v4 =
1111
, v5 =
1331
, v6 =
1234
0 0 0 1 1 10 1 2 1 3 20 1 2 1 3 30 0 0 1 1 4
R1↔R2∼
0 1 2 1 3 20 0 0 1 1 10 1 2 1 3 30 0 0 1 1 4
R3←R3−R1∼
0 1 2 2 3 20 0 0 1 1 10 0 0 0 0 10 0 0 1 1 4
R4←R4−R2∼
0 1 2 2 3 20 0 0 1 1 10 0 0 0 0 10 0 0 0 0 3
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 11 / 34
Example
Find the redundant vectors in the sequence
v1 =
0000
, v2 =
0110
, v3 =
0220
, v4 =
1111
, v5 =
1331
, v6 =
1234
0 0 0 1 1 10 1 2 1 3 20 1 2 1 3 30 0 0 1 1 4
∼∼∼0 1 2 2 3 20 0 0 1 1 10 0 0 0 0 10 0 0 0 0 3
R4←R4−3R3∼0 1 2 2 3 20 0 0 1 1 10 0 0 0 0 10 0 0 0 0 0
Non-pivot vectors are v1, v3 and v5 , which are redundant.
span {v1, v2, v3, v4, v5, v6} = span {v2, v4, v6}
Remarkspan {v1, v2, v3} = {a1v1 + a2v2 + a3v3 | a1, a2, a3 ∈ R}
span {v1, v2} = {a1v1 + a2v2 | a1, a2 ∈ R}span {v1} = {a1v1 | a1 ∈ R}
span∅ = {0} the set containing only the zero vector.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 12 / 34
Linear (in)dependence
DefinitionA set of vectors {v1, . . . , vk} is called
linearly independent if there are no redundant vectors;linearly dependent if it is not linearly independent, i.e., if there are redundantvectors.
ExampleAre the following vectors linearly independent?
v1 =
110
, v2 =
022
, v3 =
10−1
1 0 11 2 00 2 −1
R2←R2−R1∼ 1 0 1
0 2 −10 2 −1
R3←R3−R2∼ 1 0 1
0 2 −10 0 0
Since v3 is redundant, we see they are linearly dependent.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 13 / 34
Linear (in)dependence
DefinitionA set of vectors {v1, . . . , vk} is called
linearly independent if there are no redundant vectors;linearly dependent if it is not linearly independent, i.e., if there are redundantvectors.
ExampleAre the following vectors linearly independent?
v1 =
110
, v2 =
022
, v3 =
10−1
1 0 1
1 2 00 2 −1
R2←R2−R1∼ 1 0 1
0 2 −10 2 −1
R3←R3−R2∼ 1 0 1
0 2 −10 0 0
Since v3 is redundant, we see they are linearly dependent.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 13 / 34
Extended casting-out algorithm
The ordinary casting-out algorithm.Except we go to the reduced echelon form.The entries of the non-pivot columns determine the coefficients for writingeach redundant vector as a linear combination of preceding non-redundantvectors.
ExampleConsider the vectors
v1 =
1000
, v2 =
1111
, v3 =
0222
, v4 =
3111
(a) Is the set {v1, v2, v3, v4} linearly independent? v4 = 2v1 + v2(b) If not, show how each redundant vector can be written as a linear combinationof preceding vectors.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 14 / 34
Example
v1 =
1000
, v2 =
1111
, v3 =
0222
, v4 =
3111
Solution(a) By the casting-out algorithm,
1 1 0 30 1 2 10 1 2 10 1 2 1
R3←R3−R2R4←R4−R2∼
1 1 0 30 1 2 10 0 0 00 0 0 0
⇒ v3 and v4
are redundant.
(b) Proceed to the reduced echelon form:1 1 0 30 1 2 10 0 0 00 0 0 0
R1←R1−R2∼1 0 −2 20 1 2 10 0 0 00 0 0 0
⇒{
v3 = (−2)v1 + 2v2
v4 = 2v1 + v2
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 15 / 34
Remark
You might (or might not) get confused, by comparing with an early example
2x + y + w = 8x + y + z = 6
3x + 2y + z + v = −2→
2 1 0 1 0 81 1 1 0 0 63 2 1 0 1 −2
∼ 1 0 −1 0 1 −14
0 1 2 0 −1 200 0 0 1 −1 16
z = s and v = t =⇒
xyzwv
=
−14200160
+ t
−11011
+ s
1−2100
We have to switch the signs for the entries, except for those on the right-handside.
1 1 0 30 1 2 10 1 2 10 1 2 1
∼1 0 −2 20 1 2 10 0 0 00 0 0 0
⇒{
v3 = (−2)v1 + 2v2
v4 = 2v1 + v2
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 16 / 34
Remark
The reason is: when expressing v3 and v4 as linear combinations of v1 and v2,they ARE on the right-hand side.
v1 =
1000
, v2 =
1111
, v3 =
0222
, v4 =
3111
v3 = a1v1 + a2v2 = a1
1000
+ a2
1111
⇔1 10 10 10 1
[a1a2
]=
0222
→
1 1 00 1 20 1 20 1 2
1 1 00 1 20 1 20 1 2
R3←R3−R2R4←R4−R2∼
1 1 00 1 20 0 00 0 0
R1←R1−R2∼
1 0 −20 1 20 0 00 0 0
⇒v3‖
(−2)v1+2v2
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 17 / 34
Classwork
v1 =
213
, v2 =
112
, v3 =
011
, v4 =
100
, v5 =
001
(a) Is the set {v1, v2, v3, v4, v5} linearly independent?
(b) If not, show how each redundant vector can be written
as a linear combination of preceding vectors.
Lecture resumes at 7:45.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 18 / 34
Solution
v1 =
213
, v2 =
112
, v3 =
011
, v4 =
100
, v5 =
001
2 1 0 1 0
1 1 1 0 03 2 1 0 1
R1↔R2∼ 1 1 1 0 0
2 1 0 1 03 2 1 0 1
R2←R2−2R1R3←R3−3R1∼
1 1 1 0 00 −1 −2 1 00 −1 −2 0 1
R2←−R2∼
1 1 1 0 00 1 2 −1 00 −1 −2 0 1
R3←R3+R2R1←R1−R2∼
1 0 −1 1 00 1 2 −1 00 0 0 −1 1
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 19 / 34
Solution
v1 =
213
, v2 =
112
, v3 =
011
, v4 =
100
, v5 =
001
2 1 0 1 0
1 1 1 0 03 2 1 0 1
∼ 1 0 −1 1 0
0 1 2 −1 00 0 0 −1 1
R3←−R3∼
1 0 −1 1 00 1 2 −1 00 0 0 1 −1
R2←R2+R3R1←R1−R3∼
1 0 −1 0 10 1 2 0 −10 0 0 1 −1
v3 = −v1 + 2v2 = −
213
+ 2
112
=
011
, v5 = v1 − v2 − v4
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 20 / 34
Alternative characterization
TheoremThe vectors v1, . . . , vk are linearly independent iff (if and only if) the vectorequation
a1v1 + a2v2 + · · ·+ akvk = 0
has only the trivial solution. Namely it only has the solution a1 = · · · = ak = 0.
Example
v1 =
110
v2 =
123
v3 =
−3−23
The equation above (k = 3) becomes
a1v1 + a2v2 + a3v3 = 0⇐⇒
1 1 −31 2 −20 3 3
a1a2a3
=
000
Homogeneous
Check the rank of the coefficient matrix. Find its (reduced) echelon form:Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 21 / 34
Example
v1 =
110
v2 =
123
v3 =
−3−23
1 1 −31 2 −20 3 3
R2←R2−R1∼1 1 −30 1 10 3 3
R3←R3−3R1∼1 1 −30 1 10 0 0
R1←R1−R2∼1 0 −40 1 10 0 0
The rank is 2 < 3, so the vector equation has non-trivial solutions
The three vectors, v1, v2 and v3 are linearly dependent.In particular,
v3 = −4v1 + v2 ⇐⇒ 4v1 − v2 + v3 = 0⇐⇒ (a1, a2, a3) = (4,−1, 1)
This is the basic solution. General solution: t (4,−1, 1)
t (4v1 − v2 + v3) = 4tv1 − tv2 + tv3 = t0 = 0
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 22 / 34
ProofTheoremThe vectors v1, . . . , vk are linearly independent iff the vector equation
a1v1 + a2v2 + · · ·+ akvk = 0
has only the trivial solution.
Proof.We need to prove two directions.
“if part”
If a1v1 + a2v2 + · · ·+ akvk = 0 has only the trivial solution, thenv1, . . . , vk are linearly independent.
By contradiction. Suppose the statement is false, i.e., v1, . . . , vk are linearlydependent. WLOG (=without loss of generality), we let vj be redundant.This means, there are a1, . . . , aj−1 such thata1v1 + a2v2 + · · ·+ aj−1vj−1 = vj =⇒ a1v1 + a2v2 + · · ·+ aj−1vj−1 − vj = 0.
(a1, . . . , aj−1,−1, 0, . . . , 0) is a non-trivial solution.
This is a contradiction. Therefore, the statement is true, i.e., v1, . . . , vk arelinearly independent.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 23 / 34
ProofTheoremThe vectors v1, . . . , vk are linearly independent iff the vector equation
a1v1 + a2v2 + · · ·+ akvk = 0
has only the trivial solution.
Proof (Cont’d).
“only if part”
If v1, . . . , vk are linearly independent, then a1v1 + a2v2 +· · ·+ akvk = 0 only has the trivial solution.
Again, to prove it by contradiction, we assumea1v1 + a2v2 + · · ·+ akvk = 0
has a non-trivial solution, say (a1, . . . , ak) 6= (0, . . . , 0). Let j be the largestindex such that aj 6= 0 , i.e.,
(a1, . . . , ak) = (a1, . . . , aj , 0, . . . , 0) =⇒ a1v1 + a2v2 + · · ·+ ajvj = 0
Thus, vj = −a1
ajv1 −
a2
ajv2 − · · · −
a1
ajvj−1.
This contradicts that v1, . . . , vk are linearly independent.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 24 / 34
Properties of linear independence
1. ReorderingIf v1, . . . , vk are linearly independent, then any reordering of this sequence ofvectors is also linearly independent.
ExampleIf v1, v2, v3 are linearly independent, then v3, v1, v2 are linearlyindependent.
2.SubsetIf v1, . . . , vk are linearly independent, so are v1, . . . , vj for any j < k .
RemarkThis seems to work only for the first j(< k) vector. Combiningwith the “Reordering” property, we see any j vectors chosen fromv1, . . . , vk are linearly independent.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 25 / 34
Properties of linear independence
3. DimensionLet v1, . . . , vk be linearly independent in Rn. Then, k ≤ n.
Proof.The casting-out algorithm involves an n × k matrix. If k > n,[
v1 · · · vn vn+1 · · · vk]
i.e., there are more columns than rows, there must be at least onenon-pivot column. This implies v1, . . . , vk must be linearlydependent if k > n.
RemarkThere will be a general version of this property for subspaces.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 26 / 34
Example
Assume v1, v2, and v3 are linearly independent in Rn.[Question] Determine whether the following vectors are linearly independent:
w1 = v1 + v2, w2 = v2 + 2v3, w3 = 2v1 + v2 − v3
We use the alternative characterization of linear independence.Consider the vector equation
a1w1 + a2w2 + a3w3 = 0
and ask whether this equation has a non-trivial solution.If “yes”, then w1, w2 and w3 are linearly dependent.If “no”, then w1, w2 and w3 are linearly independent.a1w1 + a2w2 + a3w3 = a1 (v1 + v2) + a2 (v2 + 2v3) + a3 (2v1 + v2 − v3)
Simplify(a1 + 2a3) v1 + (a1 + a2 + a3) v2 + (2a2 − a3) v3 = 0
Because v1, v2 and v3 are linearly independent, we know that
b1v1 + b2v2 + b3v3 = 0 =⇒ (b1, b2, b3) = (0, 0, 0) =⇒a1 + 2a3 = 0
a1 + a2 + a3 = 02a2 − a3 = 0
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 27 / 34
Example
a1w1 + a2w2 + a3w3 = 0⇔a1 + 2a3 = 0
a1 + a2 + a3 = 02a2 − a3 = 0 1 0 2
1 1 10 2 −1
∼ 1 0 2
0 1 −10 2 −1
∼ 1 0 2
0 1 −10 0 1
This system only has the trivial solution (a1, a2, a3) = (0, 0, 0).w1,w2 and w3 are linearly independent.
Remark (Linear transformation)
[w1 w2 w3
]=
1 0 21 1 10 2 −1
[v1 v2 v3]
After Midterm II, we shall learn an alternative method.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 28 / 34
Uniqueness of linear combinations
TheoremAssume v1, . . . , vk are linearly independent and w ∈ span {v1, . . . , vk}. Then, wcan be written as a linear combination of v1, . . . , vk in a unique way.
Proof.Assume there are two ways:
w = b1v1 + b2v2 + · · ·+ bkvkw = c1v1 + c2v2 + · · ·+ ckvk
Subtraction yields
0 = (b1 − c1) v1 + (b2 − c2) v2 + · · ·+ (bk − ck) vk
Since v1, . . . , vk are linearly independent, the only solution to the equation is
b1 − c1 = b2 − c2 = · · · = bk − ck = 0⇐⇒ b1 = c1, b2 = c2, . . . , bk = ck .
They are the same linear combination.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 29 / 34
Subspaces in Rn
DefinitionA subset V of Rn is called a subspace of Rn if the following 3 conditions hold:
0 ∈ V ;“V is closed under addition” : Whenever u,w ∈ V , then u + w ∈ V .“V is closed under scalar multiplication” : Whenever u ∈ V and k ∈ R, thenku ∈ V .
ExampleLet V be a plane through the origin in 3-dimensional space. Then, V is asubspace of R3.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 30 / 34
Examples
Let W be a line through the origin. Then, it is a subspace.
Let A be the closed unit disc. A ={[
x y]T ∈ R2 | x2 + y2 ≤ 1
}.
This is not a subspace:0 ∈ A
u =
[10
],w =
[01
]∈ A but u + w =
[11
]/∈ A
(12 + 12 = 2 > 1
);
2u =
[20
]/∈ A
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 31 / 34
Solutions to a homogeneous system
ExampleThe solution space of a system of homogeneous equations. Let A be an m × nmatrix and V be the set of solutions of the homogeneous system
Au = 0 i.e., V = {u ∈ Rn | Au = 0} .
Then, V is a subspace of Rn.1 0 ∈ V , ⇐= A0 = 0 the trivial solution;2 for any u,w ∈ V , =⇒ Au = 0 = Aw =⇒ A (u + w) = Au+Aw = 0+ 0 = 0
=⇒ u + w ∈ V ;3 for any u ∈ V , k ∈ R, =⇒ Au = 0 =⇒ A (ku) = k (Au) = k0 = 0
=⇒ ku ∈ V
RemarkThe solution space of a homogeneous system = span {basic solutions}
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 32 / 34
Span
ExampleLet v1, . . . , vk be vectors in Rn. Then, span {v1, . . . , vk} is a subspace.
1 0 ∈ span {v1, . . . , vk} ⇐= 0 = 0v1 + · · ·+ 0vk ;2 Addition: Assume u,w ∈ span {v1, . . . , vk}
u = a1v1 + · · ·+ akvkw = b1v1 + · · ·+ bkvk
=⇒ u + w = (a1 + b1) v1 + · · ·+ (ak + bk) vk
=⇒ u + w ∈ span {v1, . . . , vk}3 Scalar multiplication:
u ∈ span {v1, . . . , vk} =⇒ u = a1v1 + · · ·+ akvk
=⇒ ru = (ra1) v1 + · · ·+ (rak) vk ∈ span {v1, . . . , vk}
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 33 / 34
SpanTheoremEvery subspace V of Rn is equal to a span. Moreover, we can writeV = span {u1, . . . ,uk}, where u1, . . . ,uk are linearly independent.
Proof.If V = {0}, then V = span∅ and we are done.If V 6= {0}, then V contains some non-zero vector. Pick any non-zero vectoru1 ∈ V . If V = span{u1}, we are done.Otherwise, pick u2 in V but not in span{u1}. If V = span{u1,u2}, we aredone.Otherwise, we pick u3 ∈ V \span{u1,u2} (A\B = {x ∈ A | x /∈ B}) andcheck V
?= span{u1,u2,u3} etc.
Note that:1 After j steps, vectors u1, . . . ,uj are linearly independent.2 Since there are at most n linearly independent vectors in Rn , this process
must stop after finitely many steps.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 34 / 34
MATH-1030Matrix Theory & Linear Algebra I
Lin Jiu
Dalhousie University
May 30th, 2019
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 1 / 34
Recall
1 Linearly (in)dependent(extended) casting-out algorithmhomogeneous system
2 Span of a set of vectorsRedundant vectors can be eliminated.For example, a span of three vectors, may not be a space.
3 Subspaces of Rn
lines, planes containing the originthe solution space of homogeneous systemTheorem. Every subspace V of Rn is equal to a span. Moreover, we can writeV = span {u1, . . . , uk}, where u1, . . . , uk are linearly independent.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 2 / 34
Bases
DefinitionLet V be a subspace of Rn. {u1, . . . ,uk} is a basis of V if the following twoconditions hold:
1 V = span {u1, . . . ,uk}2 u1, . . . ,uk are linearly independent.
RemarkThe space V is “exactly” the span of the basis. None of the vectors in a basis isredundant.
Example1. A basis of R3:
e1 =
100
e2 =
010
e3 =
001
is called the standard basis of R3. Check the right-hand system: i, j, k in Chpt. 2.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 3 / 34
Examples
R3 = span {e1, e2, e3} ⇐= every vector v =
xyz
∈ R3: v = xe1 + ye2 + ze3
It is obvious that they are linearly independent. I3 has rank 3.
PropositionLet ei be the vector in Rn whose i-th component is 1 and all of whose othercomponents are 0. In other words, ei is the i-th column of the identity matrix.
e1 =
100...0
, e2 =
010...0
, e3 =
001...0
, . . . , en =
000...1
Then, {e1, . . . , en} is a basis for Rn. It is called the standard basis of Rn.
rank(In) = n.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 4 / 34
Examples
For the standard basis of Rn:
e1 =
100...0
, e2 =
010...0
, e3 =
001...0
, . . . , en =
000...1
This is also an orthonormal basis (orthogonal+normal)
Orthogonal: For i 6= j , ei · ej = 0Normal: ‖ei‖ = 1
You will study more in MATH-2040 (§11)
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 5 / 34
Examples
2. A non-standard basis of R3: u1 =
111
u2 =
110
u3 =
100
xyz
= a1u1 + a2u2 + a3u3 =
1 1 11 1 01 0 0
a1a2a3
→ 1 1 1 x
1 1 0 y1 0 0 z
rank = 3⇒ consistent & unique solution
1 1 1 x1 1 0 y1 0 0 z
R2←R2−R1R3←R3−R1∼
1 1 1 x0 0 −1 y − x0 −1 −1 z − x
R2↔R3∼ 1 1 1 x
0 −1 −1 z − x0 0 −1 y − x
=⇒ R3 = span {u1,u2,u3}u1,u2,u3 are linearly independent : x = y = z = 0 1 1 1 0
1 1 0 01 0 0 0
R2←R2−R1R3←R3−R1∼
1 1 1 00 0 −1 00 −1 0 0
R2↔R3∼ 1 1 1 0
0 −1 0 00 0 −1 0
rank= 3 =⇒unique trivial solution =⇒linearly independent.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 6 / 34
Examples
RemarkRecall Theorem 1.26 Consider a system of m equations in n variables, and assumethat the coefficient matrix has rank r . Assume further that the system isconsistent.
If r = n , then the system has a unique solution.If r < n , then the system has infinitely many solutions, with n − rparameters.
Now, suppose we do not assume the system is consistent, but let m = n = r .The system is consistent, since there are no zero rows for the coefficient matrix inits echelon form (m = n = r). Moreover, this system has a unique solution.
# of equations = # of variables = rank =⇒ unique solution
Example3. The columns of an invertible matrix. Let A be an invertible n × n matrix and{u1, . . . ,un} be the columns of A.
A =[u1 · · · un
]⇒[A I
]∼[I A−1 ] =⇒ rank(A) = n
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 7 / 34
Examples
Ax = v⇐⇒[
u1 · · · un v]has a unique solution.
v =[x1 · · · xn
]T ∈ Rn: v = a1u1 + · · ·+ anun =⇒ Rn = span {u1, . . . ,un}v = 0 =⇒ u1, . . . ,un are linearly independent.
The columns of an n × n invertible matrix form a basis of Rn.4. Find a basis for the space
V = span
120
,
240
,
101
,
442
,
341
.
It is obvious that the second vector is redundant. u2 = 2u1It suffices to cast out the redundant vectors.
The result will only contain linearly independent vectors.And this does not change the span (still V ).
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 8 / 34
Examples
4. Find a basis for the space V = span
120
,
240
,
101
,
442
,
341
.
Solution. Use the casting-out algorithm:1 2 1 4 32 4 0 4 40 0 1 2 1
R2←R2−2R1∼1 2 1 4 30 0 −2 −4 −20 0 1 2 1
R2←− 1
2R2∼1 2 1 4 30 0 1 2 10 0 1 2 1
R3←R3−R2∼
1 2 1 4 30 0 1 2 10 0 0 0 0
This means u2, u4 and u5 are redundant. Therefore,
V = span
120
,
240
,
101
,
442
,
341
= V = span
basis︷ ︸︸ ︷120
,
101
.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 9 / 34
Examples
5. Find a basis of the solution space of the system of homogeneous equations
1 2 1 4 32 4 0 4 40 0 1 2 1
xyzwv
=
000
⇔ 1 2 1 4 3 0
2 4 0 4 4 00 0 1 2 1 0
Solution. 1 2 1 4 3 02 4 0 4 4 00 0 1 2 1 0
R2←R2−2R1∼ R2←− 12R2∼
1 2 1 4 3 00 0 1 2 1 00 0 1 2 1 0
R3←R3−R2∼ R1←R1−R2∼
1 2 0 2 2 00 0 1 2 1 00 0 0 0 0 0
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 10 / 34
Examples
1 2 1 4 32 4 0 4 40 0 1 2 1
xyzwv
=
000
∼ 1 2 0 2 2 0
0 0 1 2 1 00 0 0 0 0 0
x and z are pivot while y , w and v are free.
y = t
w = s
v = r
⇒x =
z =
−2y − 2w − 2v−2w − v
= −2t − 2s − 2r= −2s − r
=⇒
xyzwv
=
−2t − 2s − 2r
t−2s − r
sr
xyzwv
=
−2t − 2s − 2r
t−2s − r
sr
= t
−21000
+ s
−20−210
+ r
−20−101
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 11 / 34
Examples
The solution space is span
−21000
,
−20−210
,
−20−101
. The vectors are linearly
independent due to the 1’s in red. Basis:{[−2 1 0 0 0]T , [−2 0 − 2 1 0]T , [−2 0 − 1 0 1]T
}FactThe solution space of a homogeneous system is the span of basic solution(s).Basic solutions are linearly independent. (Parameters)
6. Consider the subspace V of R3, consisting of all vectors[x y z
]T withx = z . In symbols,
V =
xyz
∈ R3 x = z
.
Find a basis for V .Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 12 / 34
Examples
Solution. Solve the homogeneous system x − z = 0. V is the solution space ofthe equation x = z , the homogeneous system x − z = 0
[1 0 −1 0
]⇒
z = t
y = s
⇒ x = z = t⇒
xyz
= s
010
+ t
101
Basis:
010
,
101
V =
xyz
x = z
= span
010
,
101
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 13 / 34
Coordinate
DefinitionLet B = {u1, . . . ,uk} be a basis of some subspace V . Then, every vector w ∈ Vcan be uniquely written as a linear combination of u1, . . . ,uk :
w = a1u1 + · · ·+ akuk .
We say a1, . . . , ak are the coordinates of w with respect to the basis B. We shalluse the notation
[w]B =[a1 a2 · · · ak
]T.
Example1. Let V = R3.
With respect to the standard basis e1 =
100
, e2 =
010
, e3 =
001
every vector v =
xyz
= xe1 + ye2 + ze3 ⇒
xyz
are the coordinates.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 14 / 34
Examples
Recall a non-standard basis of R3: B =
u1 =
111
,u2 =
110
,u3 =
100
1 1 1 x1 1 0 y1 0 0 z
∼ 1 1 1 x
0 −1 −1 z − x0 0 −1 y − x
∼ 1 0 0 z
0 1 0 y − z0 0 1 x − y
Every vector v =
xyz
= zu1 + (y − z)u2 + (x − y)u3
⇒ [v]B =
zy − zx − y
= A
xyz
=⇒ A =
0 0 10 1 −11 −1 0
Exercise:
A−1 =
1 1 11 1 01 0 0
=[u1 u2 u3
]xyz
= xe1 + ye2 + ze3 = v = zu1 +(y − z)u2 +(x − y)u3 =
1 1 11 1 01 0 0
zy − zx − y
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 15 / 34
Example
2. Let B = {u1,u2,u3} =
1010
,
1120
,
0011
be a basis for a certain
3-dimensional subspace of R4. Let w =[3 2 6 1
]T . Find the coordinates ofw with respect to (w. r. t. ) B, i.e, [w]B .Solution. We need to solve a1u1 + a2u2 + a2u3 = w.
1 1 0 30 1 0 21 2 1 60 0 1 1
R3←R3−R1∼
1 1 0 30 1 0 20 1 1 30 0 1 1
R3←R3−R2∼
1 1 0 30 1 0 20 0 1 10 0 1 1
R4←R4−R3∼
1 1 0 30 1 0 20 0 1 10 0 0 0
R1←R1−R2∼
1 0 0 10 1 0 20 0 1 10 0 0 0
[w]B =
a1a2a3
=
121
⇒ w = u1 + 2u2 + u3.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 16 / 34
Example
3. Let B = {u1,u2,u3} =
1010
,
1120
,
0011
.
Find the vector v whose coordinates w. r. t. B are[2 −1 1
]T .Solution.
v = 2u1+(−1)u2+1u3 = 2
1010
+(−1)
1120
+1
0011
=
2020
+−1−1−20
+0011
=
1−111
Factvector⇔coordinates One direction is easier than the other, which is due to that“vector⇒coordinates” needs to calculate the (left) inverse.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 17 / 34
Classwork
Let
S =
xyzw
x + y = z + w
x + z = y + w
.
(a) Find a basis B for S .
(b) Obviously, v = [3 3 3 3]T ∈ S . Find [v]B
Lecture resumes at 7:55.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 18 / 34
Solution
S =
xyzw
x + y = z + w
x + z = y + w
⇒x + y − z − w = 0
x − y + z − w = 0⇒[1 1 −1 −11 −1 1 −1
]xyzw
=
[00
]
(a) [1 1 −1 −1 01 −1 1 −1 0
]R2←R2−R1∼
[1 1 −1 −1 00 −2 2 0 0
]R2←− 1
2R2∼[
1 1 −1 −1 00 1 −1 0 0
]R1←R1−R2∼
[1 0 0 −1 00 1 −1 0 0
]
z = t
w = s⇒
xyzw
= t
0110
+ s
1001
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 19 / 34
Solution
S =
xyzw
x + y = z + w
x + z = y + w
= span
0110
,
1001
⇒ B =
0110
,
1001
(b) Easily by observation, [v]B =
[33
]. Or, we can try to solve
3333
= v = a
0110
+ b
1001
=
0 11 01 00 1
[ab]
0 1 31 0 31 0 30 1 3
∼
1 0 30 1 31 0 30 1 3
∼
1 0 30 1 30 0 00 0 0
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 20 / 34
Properties of Bases
TheoremSuppose u1, . . . ,ur are linearly independent elements of span {v1, . . . , vs}. Then,r ≤ s.
Proof.For j = 1, . . . , r , since uj ∈ span {v1, . . . , vs}, uj = a1jv1 + a2jv2 + · · ·+ asjvs .Let
A =
a11 · · · a1r...
. . ....
as1 · · · asr
an s × r matrix.
Assume r > s. The system Ax = 0 has a non-trivial solution x 6= 0 for the systema11 · · · a1r...
. . ....
as1 · · · asr
x1...xr
=
0...0
for all j = 1, . . . , s
x1aj1 + · · ·+ xrajr = 0Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 21 / 34
Properties of Basesx1u1 + · · ·+ xrur = x1 (a11v1 + a21v2 + · · ·+ as1vs) + · · ·
+ xr (a1rv1 + a2rv2 + · · ·+ asrvs)= (x1a11 + · · ·+ xra1r ) v1 + · · ·
+ (x1ar1 + · · ·+ xrarr ) vr= 0v1 + · · ·+ 0vr = 0
We have a non-trivial linear combination of {u1, . . .ur} giving the zero vector 0.This violates that u1, . . . ,ur are linearlly independent ⇒ r ≤ s.
TheoremLet V be a subspace of Rn. Then, any two bases of V have the same numberof elements.
Proof.Let {u1, . . . ,us} and {v1, . . . , vr} be two bases of V .
V = span {u1, . . . ,us} and {v1, . . . , vr} are linearly independent. =⇒ r ≤ s.V = span {v1, . . . , vr} and {u1, . . . ,us} are linearly independent. =⇒ s ≤ r .
Therefore, r = s.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 22 / 34
DimensionDefinitionLet V be a subspace of Rn with basis v1, . . . , vk . Then we say V has dimensionk . dimV = k .
Properties1 Every subspace V of Rn has a basis and therefore a dimension.2 Every linearly independent set of vectors u1, . . . ,ur ∈ V can be extended to
a basis.3 Every spanning set v1, . . . , vr of V can be shrunk to a basis of V .4 If V and W are subspaces of Rn with V ⊆W (V is a subset of W ), then
dimV ≤dimW .5 Let V be a k-dimensional space. Then,
every linearly independent set has ≤ k vectors;every spanning set has ≥ k vectors.
6 Let u1, . . . ,uk be vectors in a k-dimensional space V .
V = span {u1, . . . ,uk} if and only if u1, . . . ,uk are linearly independent.
Namely, u1, . . . ,uk form a basis iff they are linearly independent.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 23 / 34
Proofs1 Every subspace V of Rn has a basis and therefore a dimension. (See the
slides on Tuesday that V = span {u1, . . . ,uk})2 Every linearly independent set of vectors u1, . . . ,ur ∈ V can be extended to
a basis.Proof. By (1), V has a basis v1, . . . , vs . Apply the casting-out algorithm tou1, . . . ,ur , v1, . . . , vs .
The output form a basis forspan {u1, . . . , ur , v1, . . . , vs} = span {v1, . . . , vs} = V ;u1, . . . , ur are linearly independent, so they “survived”.
3 Every spanning set v1, . . . , vr of V can be shrunk to a basis of V .Proof. Casting-out algorithm removes redundant vectors.
4 If V and W are subspaces of Rn with V ⊆W then dimV ≤dimW .Proof. Let v1, . . . , vk be a basis of V . (2) allows us to extend it to a basis ofW . Thus, dimV ≤dimW .
5 Let V be a k-dimensional space. Then,every linearly independent set has ≤ k vectors;every spanning set has ≥ k vectors.Proof. Apply the first property of bases: Suppose u1, . . . , ur are linearly
independent elements of span {v1, . . . , vk}. Then, r ≤ k.
If V = span{u1, . . . , ur} with r < k, then dimV ≤ r < k, which is acontradiction.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 24 / 34
Proofs
6. Let u1, . . . ,uk be vectors in a k-dimensional space V .
V = span {u1, . . . ,uk} if and only if u1, . . . ,uk are linearly independent.
Namely, u1, . . . ,uk form a basis iff they are linearly independent.
Proof.We show both directions.
If u1, . . . ,uk are linearly independent, by (2) we extend it to a basis of V .dimV = k =# of elements in the basis, {u1, . . . ,uk} is A basis.V = span {u1, . . . ,uk}.If V = span {u1, . . . ,uk}, (3) allows us to shrunk to a basis of V , containingk elements. . No one is redundant. u1, . . . ,uk are linearly independent.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 25 / 34
Example
Extend {u1,u2} to a basis of R4, where u1 =
11−12
and u2 =
12−24
.Solution. Recall the standard basis of R4 that {e1, e2, e3, e4}. We apply thecasing-out algorithm to {u1,u2, e1, e2, e3, e4}
1 1 1 0 0 01 2 0 1 0 0−1 −2 0 0 1 02 4 0 0 0 1
R2←R2−R1R3←R3+R1R4←R4−2R1∼
1 1 1 0 0 00 1 −1 1 0 00 −1 1 0 1 00 2 −2 0 0 1
R3←R3+R2R4←R4−2R2∼
1 1 1 0 0 00 1 −1 1 0 00 0 0 1 1 00 0 0 −2 0 1
R4←R4+2R3∼
1 1 1 0 0 00 1 −1 1 0 00 0 0 1 1 00 0 0 0 2 1
Basis: {u1,u2, e2, e3}
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 26 / 34
Example
Let V =
xyzw
x + 2y + z − w = 0
. Note u1 =
1−110
,u2 =
−2111
∈ V .
Extend {u1,u2} to a basis of V .Solution. First of all, solve for V . [1 2 1 − 1 | 0] y = t, z = s, w = rx = −2t − s + r
xyzw
= t
−2100
︸ ︷︷ ︸
v1
+ s
−1010
︸ ︷︷ ︸
v2
+ r
1001
︸︷︷︸
v3
Casting-out algorithm is applied to {u1,u2, v1, v2, v3}1 −2 −2 −1 1−1 1 1 0 01 1 0 1 00 1 0 0 1
∼
1 0 0 1 −10 1 0 0 10 0 1 1 −20 0 0 0 0
Basis: {u1,u2, v1}
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 27 / 34
Column, row and null spaces of a matrix
Definition
Let A =
a11 · · · a1n...
. . ....
am1 · · · amn
be an m × n-matrix.
The column space of A is the subspace of Rm spanned by the columns of A.
col(A) = span
a11
...am1
,
a12...
am2
, . . . ,
a1n...
amn
.
The row space of A is the subspace of Rn spanned by the rows of A.
row(A) = span{[
a11 · · · a1n]T
,[a21 · · · a2n
]T, · · · ,
[am1 · · · amn
]T}.
The null space of A is the subspace of Rn given by the solutions to thehomogeneous system Av = 0. null(A) = {v ∈ Rn | Av = 0} .
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 28 / 34
Column, row and null spaces of a matrix
TheoremIf A is an m × n matrix of rank r , then, dim(col(A)) = dim(row(A)) = r , anddim(null(A)) = n − r .
DefinitionThe dimension of the null space is also called the nullity of A, i.e.,nullity(A) = dim(null(A)).
Theorem (Rank-Nullity Theorem)For an m × n matrix A: rank(A) + nullity(A) = n.
ExampleFind a basis and the dimension of each of (a) the column space, (b) the row space(c) and the null space of the matrix
A =
1 2 3 2 33 2 5 4 75 0 5 5 10
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 29 / 34
ExampleFind the reduced echelon form:1 2 3 2 3
3 2 5 4 75 0 5 5 10
R2←R2−2R1R3←R3−5R1∼
1 2 3 2 30 −4 −4 −2 −20 −10 −10 −5 −5
R2←− 1
4 R2R3←− 1
10 R3∼1 2 3 2 30 1 1 1
212
0 1 1 12
12
R3←R3−R2R1←R1−2R2∼
1 0 1 1 20 1 1 1
212
0 0 0 0 0
1 Column space: pivot vectors are the first two columns.
Basis :
135
,
220
dim(col(A)) = rank(A) = 2.
2 Row space: pivot vectors are the first two rows.Basis :
{[1 2 3 2 3
]T,[3 2 5 4 7
]T}dim(row(A)) = 2.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 30 / 34
Example
3 Null space: Av = 0 1 2 3 2 3 03 2 5 4 7 05 0 5 5 10 0
∼ 1 0 1 1 2 0
0 1 1 12
12 0
0 0 0 0 0 0
x3 = s
x4 = t
x5 = r
=⇒x1 = −x3 − x4 − 2x5
x2 = −x3 −12x4 −
12x5
= −s − t − 2r
= −s − 12t − 1
2r
x1x2x3x4x5
= s
−1−1100
+ t
−1− 1
2010
+ r
−2− 1
2001
Basis:
−1−1100
,
−1− 1
2010
,
−2− 1
2001
dim(null(A)) = 3.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 31 / 34
RemarkBe careful when interchanging rows.
A =
1 1 11 1 01 1 01 0 0
∼1 1 10 0 −10 0 −10 −1 −1
∼1 1 10 −1 −10 0 −10 0 −1
∼1 1 10 −1 −10 0 −10 0 0
Basis for the row space: {R1,R3,R4} .
Midterm IIThere will be 20 problems.The True/False problems will NOT require explanations.True/False, Multiple choice, Short answer
RemarkPlease be aware of that: in the final exam, short answer problems will be replacedby detailed answer problems. Namely, to obtain full credits, you need to show allthe steps, i.e., only providing the final answer is NOT sufficient.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 32 / 34
Summary of Chpt. 5
span linearly (in)dependent vectors basis
1 span: set of linear combinations span {u1, . . . ,u`}Natural question: Do we need all the u1, . . . ,u` ?To answer: casting-out algorithm By casting out the redundant vectors, theremaining vectors are linearly independent.Every subspace is a span of a set of vectors.
2 linearly (in-)dependent vectors: Besides (extended) casting-out algorithm, wehave another criterion to determine whether a set of vectors. Thecorresponding homogeneous system:
a1u1 + · · ·+ a`u` = 0.
check whether it ONLY has the trivial solution.3 basis: a set of linearly independent vectors, whose span gives the subspace
The number of vectors in a basis is fixed. dimension of a subspaceShrink and extend to obtain a basis.Bases for the row, column and null spaces of a matrix
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 33 / 34
Summary of Chpt. 5
There are lots of examples in this chapter. The key is the elementary rowoperation.
Read the slides/textbook carefully, by focusing on the examples.Make sure, for different type of problems, you understand the steps &reasons.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 34 / 34
MATH-1030Matrix Theory & Linear Algebra I
Lin Jiu
Dalhousie University
June 4th, 2019
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 1 / 35
Vector functions
Example (Function y = f (x))
f (x) = x2 =⇒ f (0) = 0, f (1) = 1, f (2) = 4, . . .
Example (Vector function w = f (v), with both input and outputvectors)
f
xyz
=
[x + zy2 − xz
]=⇒ f
100
=
[1+ 0
02 − 1 · 0
]=
[10
]
f
010
=
[0+ 0
12 − 0 · 0
]=
[01
]
f
111
=
[1+ 1
12 − 1 · 1
]=
[20
]Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 2 / 35
Examples
RemarkIf f inputs an n-dimensional vector and outputs an m-dimensional vector, we write
f : Rn −→ Rm.
Example(a) The example above is f : R3 −→ R2.
f
xyz
=
[x + zy2 − xz
]
(b)g
([xy
])=
x + yx − y2y
is an example g : R2 −→ R3
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 3 / 35
Linear Transformation
Recall that at the very beginning of this class, we quickly introduced linearequations and non-linear equations. We shall also classify the vector functions.
DefinitionA vector function T : Rn −→ Rm is called linear or a linear transformation if
1 T (v + w) = T (v) + T (w);2 T (kv) = kT (v) .
Namely, T is linear iff T preserves the addition and scalar multiplication of thevectors. Note that the addition and scalar multiplication on both sides of the twoidentities are different, since the vectors on both sides usually do not have thesame dimension. v,w ∈ Rn T (v) ,T (w) ∈ Rm
ExampleWhich of the following vector functions are linear?
(a) T1
xyz
=
[x + 2zy − z
], (b) T2
xyz
=
[2
x − z
], (c) T3
xyz
=
[x2
yz
]Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 4 / 35
Solution
(c) T3
xyz
=
[x2
yz
]is obviously not linear since it involves x2 and yz .
T3
100
=
[12
0 · 0
]=
[10
]; T3
200
=
[40
].
200
= 2
100
However, 2T3
100
= 2[10
]=
[20
]6=[40
]= T3
2
100
.
It violates the second condition. Namely, the scalar multiplication is notpreserved.
(a) T1
xyz
=
[x + 2zy − z
]is linear. Let v =
xyz
,w =
x ′y ′z ′
∈ R3 and k ∈ R.
We need to checkT1 (v + w)
???= T1 (v) + T1 (w)
T1 (kv) ???= kT1 (v)
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 5 / 35
Example
(a) T1
xyz
=
[x + 2zy − z
], v =
xyz
, w =
x ′y ′z ′
Check T1 (v + w)
???= T1 (v) + T1 (w);
T1 (v) = T1
xyz
=
[x + 2zy − z
]T1 (w) =
[x ′ + 2z ′
y ′ − z ′
]
=⇒ T1 (v) + T1 (w) =
[x + x ′ + 2z + 2z ′
y ′ + y ′ − z − z ′
]On the other hand,
v+w =
x + x ′
y + y ′
z + z ′
⇒ T1 (v + w) =
[x + x ′ + 2 (z + z ′)y + y ′ − (z + z ′)
]=
[x + x ′ + 2z + 2z ′
y ′ + y ′ − z − z ′
]
=⇒ T1 (v) + T1 (w) = T1 (v) + T1 (w)
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 6 / 35
Example
(a) T1
xyz
=
[x + 2zy − z
], v =
xyz
, w =
x ′y ′z ′
Check T1 (kv)
???= kT1 (v);
T1 (v) =[x + 2zy − z
]kT1 (v) =
[k (x + 2z)k (y − z)
]=
[kx + 2kzky − kz
]Meanwhile,
kv =
kxkykz
=⇒ T1 (kv) =[kx + 2kzky − kz
]= kT1 (v)
RemarkIt is not surprising that both components of T1 (x + 2z , y − z) are linear in x , y ,and z . If a vector function is linear, all its components are linear functions.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 7 / 35
Example
(b) T2
xyz
=
[2
x − z
]Apparently, both components of T2 are linear functions. However, T2 is notlinear, due to the constant 2.
T2
100
=
[2
1− 0
]=
[21
], T2
200
=
[22
]
⇒ 2T
100
6= T2
2
100
In general, due to this 2, T2 (v) =
[2∗
]and T2 (kv) =
[2∗∗
]so that
T2 (v) 6= T2 (kv).
RemarkFor a vector function, even if all the components are linear functions, this doesguarantee the function to be linear. (KEY: Without constant terms)
Proposition (Alternative characterization, i.e., T is linear iff)T (av+ bw) = aT (v) + bT (w). (a = b = 1 addition, b = 0 scalar multiplication )
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 8 / 35
Linear Transformation
LemmaEvery linear transformation T satisfies T (0) = 0.
Example
Recall T2
xyz
=
[2
x − z
]. T2 (0) = T2
000
=
[20
]6= 0.
Proof.Let v be any vector in Rn. Then, T (v) ∈ Rm
0 = 0 · T (v)(2)= T (0 · v) = T (0)
Remark
This is only necessary,but not sufficient.
Recall T3
xyz
=
[x2
yz
]⇒ T3
000
=
[00
] butNOTlinear.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 9 / 35
Matrix transformation
TheoremLet A be an m × n matrix. The function
T : Rn → Rm
v 7→ Av (T (v) = Av)
is always linear. Such a function is called a matrix transformation.
Proof.Just check the two conditions.
1 T (v + w) = A (v + w) = Av + Aw = T (v) + T (w)
2 T (kv) = A (kv) = kAv = kT (v)
RemarkThe dimensions are compatible.
v ∈ Rn — an n × 1 matrix;Am×n —Av is well defined and has dimension m × 1 — a vector in Rm
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 10 / 35
ExampleRecall the previous example
T1
xyz
=
[x + 2zy − z
]
It is actually a matrix transformation. T1 : R3 −→ R2
T1
xyz
=
[x + 2zy − z
]2×1
=
[]2×3
xyz
3×1
=
[1 0 20 1 −1
]xyz
PropositionEvery linear transformation is a matrix transformation. (Proof comes later)
RemarkGiven a vector function, if all the components are linear without the constantterm, we can write it as a matrix transformation. Therefore, it is linear.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 11 / 35
Example
Let T
xyzw
=
x − 2w + zx − y2y + z
. Is T linear? If yes, write it as a matrix
transformation.
Solution. All the three components are linear without the constant term. So, Tis linear.
Since T : R4 −→ R3, we should find a 3× 4 matrix A such that T (v) = Av.x − 2w + zx − y2y + z
=
xx0
+
0−y2y
+
z0z
+
−2w00
= x
110
+ y
0−12
+ z
101
+w
−200
=
1 0 1 −21 −1 0 00 2 1 0
xyzw
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 12 / 35
Linear combination
RemarkIf T is a linear transformation, then T preserves not only addition and scalarmultiplication, but all linear combinations:
T (a1v1 + a2v2 + · · ·+ akvk)(1)= T (a1v1) + T (a2v2) + · · ·+ T (akvk)(2)= a1T (v1) + a2T (v2) + · · ·+ akT (vk)
Example
Assume T (u1) =
[10
], T (u2) =
[02
], and T (u3) =
[11
]. Find T (4u1 + 2u2 − u3).
Solution.T (4u1 + 2u2 − u3) = 4T (u1) + 2T (u2)− T (u3)
= 4[10
]+ 2
[02
]−[11
]=
[40
]+
[04
]−[11
]=
[33
]Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 13 / 35
Example
Let T : R3 −→ R2 be linear. If
T
121
=
[01
], T
133
=
[20
], T
411
=
[22
],
find T
314
.
Solution. For simplicity, we let u1 =
121
, u2 =
133
, u3 =
411
, andv =
314
. We first need to express v as a linear combination of u1, u2, and u3.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 14 / 35
Example
u1 =
121
, u2 =
133
, u3 =
411
, v =
314
1 1 4 3
2 3 1 11 3 1 4
R2←R2−2R1∼
1 1 4 30 1 −7 −51 3 1 4
R3←R3−R1∼
1 1 4 30 1 −7 −50 2 −3 1
R3←R3−2R2∼
1 1 4 30 1 −7 −50 0 11 11
R3← 111R3∼
1 1 4 30 1 −7 −50 0 1 1
R2←R2+7R3∼
1 1 4 30 1 0 20 0 1 1
R1←R1−4R3∼
1 1 0 −10 1 0 20 0 1 1
R1←R1−R2∼
1 0 0 −30 1 0 20 0 1 1
⇒ v = −3u1 + 2u2 + u3
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 15 / 35
Example
v = −3u1 + 2u2 + u3
T (u1) =
[01
], T (u2) =
[20
], T (u3) =
[22
]
T (v) = T (−3u1 + 2u2 + u3)
= −3T (u1) + 2T (u2) + T (u3)
= −3[01
]+ 2
[20
]+
[22
]=
[0−3
]+
[40
]+
[22
]=
[6−1
]What will happen if v is NOT a linear combination of u1, u2, and u3?[Answer]: Then, we do not know T (v).
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 16 / 35
Classwork
Let
v1 =
220
, v2 =
101
, v3 =
133
,u =
317
and a linear transformation T satisfy
T (v1) =
[11
], T (v2) =
[02
], T (v3) =
[22
].
Find T (u).
Lecture resumes at 7:45.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 17 / 35
Solution
v1 =
220
, v2 =
101
, v3 =
133
, u =
317
,T (v1) =
[11
],T (v2) =
[02
],T (v3) =
[22
]
First of all, we try to write u as a linear combination of v1, v2, v3. Namely, finda1, a2, a3 such that u = a1v1 + a2v2 + a3v3.31
7
= a1
220
+ a2
101
+ a3
133
=
2 1 12 0 30 1 3
a1a2a3
2 1 1 3
2 0 3 10 1 3 7
R2←R2−R1∼
2 1 1 30 −1 2 −20 1 3 7
R3←R3+R2∼
2 1 1 30 −1 2 −20 0 5 5
R3← 1
5R3∼
2 1 1 30 −1 2 −20 0 1 1
R2←R2−2R3∼
2 1 1 30 −1 0 −40 0 1 1
R2←−R2∼
2 1 1 30 1 0 40 0 1 1
R1←R1−R2∼
2 0 1 −10 1 0 40 0 1 1
R1←R1−R3∼
2 0 0 −20 1 0 40 0 1 1
R1← 12R1∼
1 0 0 −10 1 0 40 0 1 1
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 18 / 35
Solution
v1 =
220
, v2 =
101
, v3 =
133
, u =
317
,T (v1) =
[11
],T (v2) =
[02
],T (v3) =
[22
]
u = a1v1 + a2v2 + a3v3 →
2 1 1 32 0 3 10 1 3 7
∼ 1 0 0 −1
0 1 0 40 0 1 1
⇒a1
a2
a3
=
−141
u = −v1 + 4v2 + v3 ⇒ T (u) = T (−v1 + 4v2 + v3)
= −T (v1) + 4T (v2) + T (v3)
= −[11
]+ 4
[02
]+
[22
]=
[19
]Next, we study several properties.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 19 / 35
Basis
TheoremA linear transformation T is completely determined by its action on a basis.
ExampleAn unknown linear transformation T : R3 −→ R2 satisfies
T
100
=
[23
], T
010
=
[12
], T
001
=
[−11
].
Find a formula for T . (formula=matrix transformation form)Solution.
T
xyz
= T
x
100
+ y
010
+ z
001
= xT
100
+ yT
010
+ zT
001
= x
[23
]+ y
[12
]+ z
[−11
]=
[2 1 −13 2 1
] xyz
Let A =
[2 1 −13 2 1
]. Then, T (v) = Av.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 20 / 35
Matrix transformationTheoremEvery linear transformation is a matrix transformation.
Proof.Suppose T : Rn −→ Rm is a linear transformation. Recall the standard basis ofRn , i.e, {e1, . . . , en}. Now, denote
w1 = T (e1) , w2 = T (e2) , . . . ,wn = T (en) .
Form A =[w1 w2 · · · wn
], i.e., A is the matrix with w1, . . . ,wn being its
columns and dimension m × n (wk ∈ Rm). We shall show that T (v) = Av.
For all v ∈ Rn, v =
x1...xn
⇒ T (v) = T (x1e1 + · · ·+ xnen) = x1w1 + · · ·+ xnwn
This implies
T (v) =[w1 w2 · · · wn
] x1...xn
= Av.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 21 / 35
ExampleRevisit the class work
v1 =
220
, v2 =
101
, v3 =
133
, u =
317
,T (v1) =
[11
],T (v2) =
[02
],T (v3) =
[22
]
1 Is {v1, v2, v3} a basis for R3? Check whether they are linearly independent: 2 1 12 0 30 1 3
∼∼∼ 1 0 0
0 1 00 0 1
2 1 1 32 0 3 10 1 3 7
∼
1 0 0 −10 1 0 40 0 1 1
They are linearly independent and form a basis for R3. dimR3 = 3.
2 Can we find a formula for T? To apply the same method, we need to knowT (e1) ,T (e2) ,T (e3). Since {v1, v2, v3} is a basis, we can express e1, e2, e3as linear combinations of v1, v2, v3. Find a, b, c such that
e1 = av1 + bv2 + cv3 ⇔
2 1 12 0 30 1 3
abc
=
100
⇔ 2 1 1 1
2 0 3 00 1 3 0
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 22 / 35
ExampleSimilarly, find d , e, f such that
e2 = dv1 + ev2 + f v3 ⇔
2 1 12 0 30 1 3
def
=
010
⇔ 2 1 1 0
2 0 3 10 1 3 0
and g , h, i such that
e3 = gv1 + hv2 + iv3 ⇔
2 1 12 0 30 1 3
ghi
=
001
⇔ 2 1 1 0
2 0 3 00 1 3 1
Observe that (or, trying to solve three systems together) 2 1 1
2 0 30 1 3
a d gb e hc f i
=[e1 e2 e3
]= I3
Find 2 1 12 0 30 1 3
−1
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 23 / 35
Example 2 1 1 1 0 02 0 3 0 1 00 1 3 0 0 1
R2←R2−R1∼
2 1 1 1 0 00 −1 2 −1 1 00 1 3 0 0 1
R3←R3+R2∼
2 1 1 1 0 00 −1 2 −1 1 00 0 5 −1 1 1
R3←
R35∼
2 1 1 1 0 00 −1 2 −1 1 00 0 1 − 1
515
15
R2←R2−2R3R1←R1−R3∼
2 1 1 1 0 00 −1 0 − 3
535 − 2
50 0 1 − 1
515
15
R1←R1+R2−R3∼
2 0 0 35
25 − 3
50 −1 0 − 3
535 − 2
50 0 1 − 1
515
15
R2←−R2R1← 1
2 R1∼
1 0 0 310
15 − 3
100 1 0 3
5 − 35
25
0 0 1 − 15
15
15
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 24 / 35
Example 2 1 1
2 0 3
0 1 3
−1
=
310
15 − 3
1035 − 3
525
− 15
15
15
e1 =
310
v1 +35v2 −
15v3
e2 =15v1 −
35v2 +
15v3
e3 = − 310
v1 +25v2 +
15v3
=⇒
T (e1) =310
T (v1) +35T (v2)−
15T (v3)
T (e2) =15T (v1)−
35T (v2) +
15T (v3)
T (e3) = −310
T (v1) +25T (v2) +
15T (v3)
=⇒
T (e1) =310
[11
]+
35
[02
]− 1
5
[22
]T (e2) =
15
[11
]− 3
5
[02
]+
15
[22
]T (e3) = −
310
[11
]+
25
[02
]+
15
[22
]=
[− 1
101110
]=
[ 35− 3
5
]=
[ 110910
]
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 25 / 35
Example
v1 =
220
, v2 =
101
, v3 =
133
,T (v1) =
[11
],T (v2) =
[02
],T (v3) =
[22
]
T (e1) =
[− 1
101110
], T (e2) =
[ 35− 3
5
], T (e3) =
[ 110910
]
T
xyz
= T (xe1 + ye2 + ze3) = x
[− 1
101110
]+ y
[ 35− 3
5
]+ z
[ 110910
]
S =
[− 1
1035
110
1110 − 3
5910
]⇒ T (v) = Sv
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 26 / 35
Example
v1 =
220
, v2 =
101
, v3 =
133
,T (v1) =
[11
],T (v2) =
[02
],T (v3) =
[22
]
T (e1) =
[− 1
101110
], T (e2) =
[ 35− 3
5
], T (e3) =
[110910
], S =
[− 1
1035
110
1110 − 3
5910
]
Recall the steps A =
2 1 1
2 0 3
0 1 3
⇒ A−1 =
310
15 − 3
1035 − 3
525
− 15
15
15
T (e1) =
310
[11
]+
35
[02
]− 1
5
[22
]T (e2) =
15
[11
]− 3
5
[02
]+
15
[22
]T (e3) = −
310
[11
]+
25
[02
]+
15
[22
] ⇒S =
[1 0 21 2 2
]A−1
by the column method.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 27 / 35
Theorem
Theorem (Exercise 6.2.2)Assume that T : Rn → Rm is a linear transformation, and that u1, . . . ,un is abasis of Rn. For all i = 1, . . . , n, let vi = T (ui ). Let A be the matrix that hasu1, . . . ,un as its columns, and let B be the matrix that has v1, . . . , vn as itscolumns. Show that A is invertible and the matrix of T is BA−1.
Example
u1 =
220
,u2 =
101
,u3 =
133
,T (u1) =
[11
],T (u2) =
[02
],T (u3) =
[22
]
A =
2 1 12 0 30 1 3
is invertible sinceu1,u2,u3 is a basis
⇒ linearly independent ⇒ rank(A) = 3
A−1 =
310
15 − 3
1035 − 3
525
− 15
15
15
, B =
[1 0 21 2 2
], S = BA−1 =
[− 1
1035
110
1110 − 3
5910
]
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 28 / 35
Composition
For functions, we can compose two functions: f (x) = x2, g (x) = x + 1
g ◦ f (x) = g (f (x)) = g(x2) = x2 + 1
Similarly, consider two linear transformations:T : Rm → Rn S : Rk → Rm
v 7→ Av w 7→ Bw
We can compose T and S and get that
T ◦ S : Rk → Rn
w 7→ T (S (w)) = T (Bw) = A (Bw) = (AB)w
So the matrix multiplication originally was to solve the question:How can we compose two linear transformations?
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 29 / 35
Inverse
DefinitionLet T ,S : Rn → Rn be linear transformations. Suppose that for each v ∈ Rn ,
(S ◦ T )(v) = v
and(T ◦ S)(v) = v.
Then S is called the inverse of T , and we write S = T−1.
Theorem (Theorem 6.22: Matrix of the inverse transformation)Let T : Rn → Rn be a linear transformation and let A be the correspondingn × n-matrix. Then T has an inverse if and only if the matrix A is invertible. Inthis case, the matrix of T−1 is A−1.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 30 / 35
Example
v1 =
220
, v2 =
101
, v3 =
133
,T (v1) =
[11
],T (v2) =
[02
],T (v3) =
[22
]
S =
[− 1
1035
110
1110 − 3
5910
]= BA−1 =
[1 0 21 2 2
]2 1 12 0 30 1 3
−1
We define
T1 : R3 −→ R3, T2 : R3 −→ R2
e1 7→ v1 e1 7→[11
]e2 7→ v2 e2 7→
[02
]e3 7→ v3 e3 7→
[22
]
T1 (u) = Au T2 (w) = Bw.
v1, v2, v3 is another basis forR3, A is invertible (i.e., T1has an inverse map T−1
1with matrix transformation ofA−1)
T = T2 ◦ T−11 ⇔ S = BA−1
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 31 / 35
Geometric interpretation (Read Section 6.3 *)
1. A =
[0 −11 0
]rotation by 90◦.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 32 / 35
Geometric interpretation (Read Section 6.3 *)
2. A =
[−1 00 1
]reflection about the y -axis
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 33 / 35
Geometric interpretation (Read Section 6.3 *)
3. A =
[cos θ − sin θsin θ cos θ
]counterclockwise rotation by θ.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 34 / 35
Summary of Chpt. 6
Linear Transformations:1 definition: Trick: All components are linear without the constant term.2 property: T (0) = 0 preserves addition and scalar multiplication ⇔ preserves
linear combination3 linear transformations are all matrix transformations: T (v) = Av
Given certain values, e.g., T (v1), T (v2), T (v3), find T (u) foru ∈ span {v1, v2, v3}Given T (v1), T (v2), T (v3) on a basis, find the matrix of the transformation.
4 composition↔matrix multiplication5 inverse
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 35 / 35
MATH-1030Matrix Theory & Linear Algebra I
Lin Jiu
Dalhousie University
June 6th, 2019
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 6th, 2019 1 / 13
Determinant
GoalTo an n × n-matrix A, we will define a scalar det(A), called the determinant of A.In particular, we will use the determinant to determine some properties of a squarematrix.
invertible?rank?
Recall: An n × n matrix is invertible iff rank(A) = n.
ClaimA is invertible iff det(A) 6= 0.
We shall begin with the case n = 2.
Definition
For a 2× 2 matrix A =
[a bc d
], the determinant is defined by det(A) := ad − bc
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 6th, 2019 2 / 13
ExamplesExample
A =
[2 10 3
]=⇒ det(A) = 2 · 3− 1 · 0 = 6
B =
[−3 70 3
]=⇒ det(B) = (−3) · 3− 7 · 0 = −9
C =
[3 64 8
]=⇒ det(C ) = 3 · 8− 6 · 4 = 24− 24 = 0.
By the elementary row operations, we see CR1← 1
3R1∼[1 24 8
]R2←R2−4R1∼
[1 20 0
]
RemarkThese three examples above shows that the determinant can be positive, negativeand zero. Also, both A and B are in echelon forms, so rank(A) = rank(B) = 2,while rank(C ) = 1 < 2. Please recall for dot product: u · v = 0⇔ u and v areorthogonal/perpendicular. Later on we shall see det(A) = 0⇔ rank(A) < n for ann × n matrix A.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 6th, 2019 3 / 13
Examples
RemarkThe determinant is also often denoted by enclosing the matrix with two verticallines. Thus
det
[a bc d
]=
∣∣∣∣ a bc d
∣∣∣∣ = ad − bc.
Example
A =
[2 10 3
]=⇒ det(A) =
∣∣∣∣ 2 10 3
∣∣∣∣ = 2 · 3− 1 · 0 = 6
B =
[−3 70 3
]=⇒ det(B) =
∣∣∣∣ −3 70 3
∣∣∣∣ = (−3) · 3− 7 · 0 = −9
C =
[3 64 8
]=⇒ det(C ) =
∣∣∣∣ 3 64 8
∣∣∣∣ = 3 · 8− 6 · 4 = 24− 24 = 0.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 6th, 2019 4 / 13
3× 3
Definition
For a 3× 3 matrix A =
a11 a12 a13a21 a22 a23a31 a32 a33
, the determinant is defined by
det(A) := a11a22a33 + a12a23a31 + a13a21a32−(a31a22a13 + a32a23a11 + a33a21a12)
Of course, it is almost impossible to memorize.
RemarkHere is a picture for help.
1 Copy the first two columns next to the matrix (right-hand side)2 The blue lines correspond to the positive terms of the determinant:
a11a22a33, a12a23a31, and a13a21a32.3 The red lines correspond to the negative terms: a31a22a13, a32a23a11, and
a33a21a12Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 6th, 2019 5 / 13
Examples
A =
0 1 23 1 01 1 −1
0 1 2
3 1 01 1 −1
0 13 11 1
det (A) =
∣∣∣∣∣∣0 1 23 1 01 1 −1
∣∣∣∣∣∣= 0 · 1 · (−1) + 2 · 0 · 1+ 2 · 3 · 1− (2 · 1 · 1+ 0 · 0 · 1+ 1 · 3 · (−1))= 6− (−1)= 7
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 6th, 2019 6 / 13
Cross product
u =
u1u2u3
, v =
v1v2v3
⇒ u× v =
u1u2u3
×v1v2v3
=
u2v3 − u3v2u3v1 − u1v3u1v2 − u2v1
.
Recall the right-hand side system i =
100
, j =
010
, k =
001
.Theorem
u× v =
∣∣∣∣∣∣u1 v1 iu2 v2 ju3 v3 k
∣∣∣∣∣∣ .Namely, u× v = det (A) for A =
u1 v1 iu2 v2 ju3 v3 k
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 6th, 2019 7 / 13
Cross product
u1u2u3
×v1v2v3
=
u2v3 − u3v2u3v1 − u1v3u1v2 − u2v1
A =
u1 v1 iu2 v2 ju3 v3 k
∣∣∣∣∣∣u1 v1 iu2 v2 ju3 v3 k
∣∣∣∣∣∣u1 v1u2 v2u3 v3
det(A) = u1v2k + v1ju3 + iu2v3 − iv2u3 − u1jv3 − v1u2k= (u2v3 − v2u3) i + (v1u3 − u1v3) j + (u1v2 − v1u2) k
=
u2v3 − u3v2u3v1 − u1v3u1v2 − u2v1
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 6th, 2019 8 / 13
Triangular matrices
Definition
A square matrix A =
a11 · · · a1n...
. . ....
an1 · · · ann
is upper triangular if aij = 0 whenever
i > j .In other words, a matrix is upper triangular if entries below the main diagonal are0.Similarly, a square matrix is lower triangular if all entries above the main diagonalare 0.Let ∗ refers to any non-zero numbers:
∗ ∗ · · · ∗ ∗0 ∗ · · · ∗ ∗...
.... . .
......
0 0 · · · ∗ ∗0 0 · · · 0 ∗
︸ ︷︷ ︸
upper triangular
∗ 0 · · · 0 0∗ ∗ · · · ∗ 0...
.... . .
......
∗ ∗ · · · ∗ 0∗ ∗ · · · ∗ ∗
︸ ︷︷ ︸
lower triangular
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 6th, 2019 9 / 13
Triangular matrices
TheoremFor an n × n triangular matrix A, the determinant is equal to the product of theentries on the main diagonal. Namely,
det(A) = a11a22 · · · ann.
Example
A =
1 2 3 160 2 6 −70 0 3 330 0 0 −1
=⇒ det(A) =
∣∣∣∣∣∣∣∣1 2 3 160 2 6 −70 0 3 330 0 0 −1
∣∣∣∣∣∣∣∣ = 1 · 2 · 3 · (−1) = −6.
∣∣∣∣∣∣∣∣−1 0 0 06 4 0 04 1 −2 0
10000000 8000 9 2
∣∣∣∣∣∣∣∣ = (−1) · 4 · (−2) · 2 = 16
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 6th, 2019 10 / 13
Row operations
RemarkBe careful that matrices of the following forms are not triangular matrices:
[0 13 4
] 2 4 03 0 05 0 0
Recall the three elementary row operations:
Ri ←→ Rj , Rj ←− aRj , Rj ←− Rj + aRi (i 6= j , a 6= 0)
TheoremLet A be an n × n-matrix.
1 If B is obtained from A by switching two rows, then det (B) = − det(A);2 If B is obtained from A by multiplying one row by a non-zero scalar k , then
det (B) = k det(A);3 If B is obtained from A by adding a multiple of one row to another row, then
det (B) = det(A);
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 6th, 2019 11 / 13
Example A
Ri←→Rj∼ B ⇒ det (B) = − det(A)
ARj←−kRj∼ B ⇒ det (B) = k det(A)
ARj←−Rj+aRi∼ B ⇒ det (B) = det(A)
Let A =
2 4 −81 8 11 4 −2
, find det (A) .
1 We see 2 4 −81 8 11 4 −2
R1↔R2∼
1 8 12 4 −81 4 −2
R2←R2−2R1∼
1 8 10 −12 −101 4 −2
R3←R3−R1∼
1 8 10 −12 −100 −4 −3
R3←R3− 13R2∼
1 8 10 −12 −100 0 1
3
det (A) = −
∣∣∣∣∣∣1 8 12 4 −81 4 −2
∣∣∣∣∣∣ = −∣∣∣∣∣∣
1 8 10 −12 −101 4 −2
∣∣∣∣∣∣ = −∣∣∣∣∣∣
1 8 10 −12 −100 −4 −3
∣∣∣∣∣∣ = −∣∣∣∣∣∣
1 8 10 −12 −100 0 1
3
∣∣∣∣∣∣Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 6th, 2019 12 / 13
ExampleA
Ri←→Rj∼ B ⇒ det (B) = − det(A)
ARj←−kRj∼ B ⇒ det (B) = k det(A)
ARj←−Rj+aRi∼ B ⇒ det (B) = det(A)
A =
2 4 −81 8 11 4 −2
det (A) = −
∣∣∣∣∣∣1 8 10 −12 −100 0 1
3
∣∣∣∣∣∣ = −1 · (−12) · 13 = 4
2. Alternatively, 2 4 −81 8 11 4 −2
R2← 12R2∼
1 2 −41 8 11 4 −2
R2←R2−R1∼
1 2 −40 6 51 4 −2
R3←R3−R1∼
1 2 −40 6 50 2 2
R3←R3− 13R2∼
1 2 −40 6 50 0 − 1
3
∣∣∣∣∣∣
1 2 −41 8 11 4 −2
∣∣∣∣∣∣ = 12 det(A). ⇒ det(A) = 2
∣∣∣∣∣∣1 2 −41 8 11 4 −2
∣∣∣∣∣∣ = · · · = 2
∣∣∣∣∣∣1 2 −40 6 50 0 − 1
3
∣∣∣∣∣∣ = 4
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 6th, 2019 13 / 13
MATH-1030Matrix Theory & Linear Algebra I
Lin Jiu
Dalhousie University
June 11th, 2019
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 1 / 28
Recall
The determinant can be considered as a function: det : {square matrices} −→ R.
det
([a bc d
])=
∣∣∣∣a bc d
∣∣∣∣ = ad − bc
det
a11 a12 a13a21 a22 a23a31 a32 a33
= a11a22a33 + a12a23a31 + a13a21a32
−(a31a22a13 + a32a23a11 + a33a21a12)
Triangular: If A is upper/lower triangular, det(A) = a11a22 · · · ann.
RemarkBe careful that matrices of the following forms are not triangular matrices:
[0 13 4
] 2 4 03 0 05 0 0
MAIN DIAGONAL
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 2 / 28
Row operations
Recall the three elementary row operations:
Ri ←→ Rj , Rj ←− aRj , Rj ←− Rj + aRi (i 6= j , a 6= 0)
TheoremLet A be an n × n-matrix.
1 If B is obtained from A by switching two rows, then det (B) = − det(A);2 If B is obtained from A by multiplying one row by a non-zero scalar k , then
det (B) = k det(A);3 If B is obtained from A by adding a multiple of one row to another row, then
det (B) = det(A);
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 3 / 28
Example
A
Ri←→Rj∼ B ⇒ det (B) = − det(A)
ARj←−kRj∼ B ⇒ det (B) = k det(A)
ARj←−Rj+aRi∼ B ⇒ det (B) = det(A)
Let A =
2 4 −81 8 11 4 −2
, find det (A) .
1 We see 2 4 −81 8 11 4 −2
A
R1↔R2∼
1 8 12 4 −81 4 −2
A1
R2←R2−2R1∼
1 8 10 −12 −101 4 −2
A2
R3←R3−R1∼
1 8 10 −12 −100 −4 −3
A3
R3←R3− 13R2∼
1 8 10 −12 −100 0 1
3
A4
det (A4) = 1 (−12) 13 = −4 By the properties:
det (A4) = det (A3) = det (A2) = det (A1) = − det (A) =⇒ det (A) = 4
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 4 / 28
ExampleA
Ri←→Rj∼ B ⇒ det (B) = − det(A)
ARj←−kRj∼ B ⇒ det (B) = k det(A)
ARj←−Rj+aRi∼ B ⇒ det (B) = det(A)
A =
2 4 −81 8 11 4 −2
det (A) = −
∣∣∣∣∣∣1 8 10 −12 −100 0 1
3
∣∣∣∣∣∣ = −1 · (−12) · 13 = 4
2. Alternatively, 2 4 −81 8 11 4 −2
R2← 12R2∼
1 2 −41 8 11 4 −2
R2←R2−R1∼
1 2 −40 6 51 4 −2
R3←R3−R1∼
1 2 −40 6 50 2 2
R3←R3− 13R2∼
1 2 −40 6 50 0 1
3
∣∣∣∣∣∣
1 2 −41 8 11 4 −2
∣∣∣∣∣∣ = 12 det(A). ⇒ det(A) = 2
∣∣∣∣∣∣1 2 −41 8 11 4 −2
∣∣∣∣∣∣ = · · · = 2
∣∣∣∣∣∣1 2 −40 6 50 0 1
3
∣∣∣∣∣∣ = 4
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 5 / 28
Recall
det
a11 a12 a13a21 a22 a23a31 a32 a33
= a11a22a33 + a12a23a31 + a13a21a32
−(a31a22a13 + a32a23a11 + a33a21a12) a11 a12 a13a21 a22 a23a31 a32 a33
R2←R2− a21a11
R1∼
a11 a12 a130 a22 − a12a21
a11a23 − a13a21
a11a31 a32 a33
R3←R3− a31
a11R1
∼
a11 a12 a130 a22 − a12a21
a11a23 − a13a21
a110 a32 − a12a31
a11a33 − a12a31
a11
R3←R3− a11a32−a12a31
a11a22−a12a21R2
∼
a11 a12 a130 a22 − a12a21
a11a23 − a13a21
a110 0 b
,
where b = a33 − a12a31a11− a11a32−a12a31
a11a22−a12a21
(a23 − a13a21
a11
)Eventually, by simplification,∣∣∣∣∣∣
a11 a12 a13a21 a22 a23a31 a32 a33
∣∣∣∣∣∣ =∣∣∣∣∣∣a11 a12 a130 a22 − a12a21
a11a23 − a13a21
a110 0 b
∣∣∣∣∣∣ = b (a11a22 − a12a21)
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 6 / 28
Minors
GoalWe shall compute the determinant of an n× n , through the cofactor expansion.Basically, this expansion express an n × n determinant as a linear combination of(n − 1)× (n − 1) determinants.
DefinitionLet A be an n × n matrix. The ijth minor of A is the determinant of the(n − 1)× (n − 1) matrix obtained from A , by deleting the ith row and jthcolumn. We write Mij for the ijth minor.
Example
A =
1 −1 2−3 5 10 2 3
M11 =
∣∣∣∣∣∣1 −1 2−3 5 10 2 3
∣∣∣∣∣∣ =∣∣∣∣ 5 12 3
∣∣∣∣ = 5 · 3− 1 · 2 = 13
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 7 / 28
Example
A =
1 −1 2−3 5 10 2 3
=⇒ M11 =
∣∣∣∣5 12 3
∣∣∣∣ = 13
M21 =
∣∣∣∣∣∣1 −1 2−3 5 10 2 3
∣∣∣∣∣∣ =∣∣∣∣ −1 2
2 3
∣∣∣∣ = (−1) · 3− 2 · 2 = −7
M31 =
∣∣∣∣∣∣1 −1 2−3 5 10 2 3
∣∣∣∣∣∣ =∣∣∣∣ −1 2
5 1
∣∣∣∣ = (−1) · 1− 2 · 5 = −11
M12 ==
∣∣∣∣ −3 10 3
∣∣∣∣ = −9M13 =
∣∣∣∣ −3 50 2
∣∣∣∣ = −6 M22 =
∣∣∣∣ 1 20 3
∣∣∣∣ = 3 M23 =
∣∣∣∣ 1 −10 2
∣∣∣∣ = 2
M32 =
∣∣∣∣ 1 2−3 1
∣∣∣∣ = 7 M33 =
∣∣∣∣ 1 −1−3 5
∣∣∣∣ = 2
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 8 / 28
Cofactor
Definition
The ijth cofactor of A is Cij = (−1)i+jMij .
∣∣∣∣∣∣∣∣∣+ − + − · · ·− + − + · · ·+ − + − · · ·...
......
.... . .
∣∣∣∣∣∣∣∣∣Example
A =
1 −1 2−3 5 10 2 3
=⇒M11 = 13 M12 = −9 M13 = −6M21 = −7 M22 = 3 M23 = 2M31 = −11 M32 = 7 M33 = 2∣∣∣∣∣∣
+ − +− + −+ − +
∣∣∣∣∣∣ =⇒C11 = 13 C12 = 9 C13 = −6C21 = 7 C22 = 3 C23 = −2
C31 = −11 C32 = −7 C33 = 2
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 9 / 28
Determinant expansion
TheoremLet A be an n × n matrix. Then det(A) is calculated by picking a row (orcolumn) and taking the product of each entry in that row (column) with itscofactor and adding these products together. This process is known as expandingalong the ith row (or column).
det (A) =
∣∣∣∣∣∣∣a11 · · · a1n...
. . ....
an1 · · · ann
∣∣∣∣∣∣∣along the 1st row: det(A) = a11C11 +a12C12 + · · ·+ a1nC1n
along the last column: det(A) = a1nC1n +a2nC2n + · · ·+ annCnn
along the ith row: det(A) = ai1Ci1 + ai2Ci2 + · · ·+ ainCin
along the jth column: det(A) = a1jC1j + a2jC2j + · · ·+ anjCnj
RemarkRemember that: we only pick ONE row or column.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 10 / 28
Example
A =
1 −1 2−3 5 10 2 3
=⇒C11 = 13 C12 = 9 C13 = −6C21 = 7 C22 = 3 C23 = −2
C31 = −11 C32 = −7 C33 = 2
along the 1st row:
det(A) = 1 · 13 + (−1) · 9 + 2 · (−6) = 13− 9− 12 = −8
along the 2nd row:
det(A) = (−3) · 7 + 5 · 3 + 1 · (−2) = −21 + 15− 2 = −8
along the 3rd row:
det(A) = 0 · (−11) + 2 · (−7) + 3 · 2 = 0− 14 + 6 = −8
along the 1st column:
det(A) = 1 · 13 + (−3) · 7 + 0 · (−11) = 13− 21 + 0 = −8
along the 2nd column:
det(A) = (−1) · 9 + 5 · 3 + 2 · (−7) = −9 + 15− 14 = −8
along the 3rd column:
det(A) = 2 · (−6) + 1 · (−2) + 3 · 2 = −12− 2 + 6 = −8
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 11 / 28
Example
A =
1 −1 2−3 5 10 2 3
=⇒C11 = 13 C12 = 9 C13 = −6C21 = 7 C22 = 3 C23 = −2
C31 = −11 C32 = −7 C33 = 2
∣∣∣∣∣∣+ − +− + −+ − +
∣∣∣∣∣∣
∣∣∣∣∣∣1 −1 2−3 5 10 2 3
∣∣∣∣∣∣ = 0 ·∣∣∣∣ −1 2
5 1
∣∣∣∣−2 · ∣∣∣∣ 1 2−3 1
∣∣∣∣+ 3 ·∣∣∣∣ 1 −1−3 5
∣∣∣∣= −2 (1 · 1− 2 · (−3)) + 3 (1 · 5− (−1) · (−3)) = −8
For a triangular matrix∣∣∣∣∣∣∣∣∣∣∣
a11 a12 a13 · · · a1n0 a22 a23 · · · a2n0 0 a33 · · · a3n...
......
. . ....
0 0 0 · · · ann
∣∣∣∣∣∣∣∣∣∣∣= a11 · (−1)1+1
∣∣∣∣∣∣∣∣∣a22 a23 · · · a2n0 a33 · · · a3n...
.... . .
...0 0 · · · ann
∣∣∣∣∣∣∣∣∣= a11 · a22
∣∣∣∣∣∣∣a33 · · · a3n...
. . ....
0 · · · ann
∣∣∣∣∣∣∣= · · · = a11a22 · · · ann.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 12 / 28
Example
u =
u1u2u3
, v =
v1v2v3
⇒ u×v =
u1u2u3
×v1v2v3
=
u2v3 − u3v2u3v1 − u1v3u1v2 − u2v1
=
∣∣∣∣∣∣u1 v1 iu2 v2 ju3 v3 k
∣∣∣∣∣∣ .Expand along the last column:∣∣∣∣∣∣
u1 v1 iu2 v2 ju3 v3 k
∣∣∣∣∣∣ = i∣∣∣∣ u2 v2u3 v3
∣∣∣∣−j∣∣∣∣ u1 v1u3 v3
∣∣∣∣+ k∣∣∣∣ u1 v1u2 v2
∣∣∣∣= i
∣∣∣∣ u2 v2u3 v3
∣∣∣∣+j∣∣∣∣ u3 v3u1 v1
∣∣∣∣+ k∣∣∣∣ u1 v1u2 v2
∣∣∣∣= (u2v3 − u3v2) i + (u3v1 − u1v3) j + (u1v2 − u2v1) k
=
u2v3 − u3v2u3v1 − u1v3u1v2 − u2v1
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 13 / 28
Example
Compute
∣∣∣∣∣∣∣∣1 2 3 00 1 −1 21 −3 5 10 0 2 3
∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣+ − + −− + − ++ − + −− + − +
∣∣∣∣∣∣∣∣
Solution. The key is to pick a row/column containing as many 0 as possible.last row:∣∣∣∣∣∣∣∣
1 2 3 00 1 −1 21 −3 5 10 0 2 3
∣∣∣∣∣∣∣∣ = −0 · ||+ 0 · || − 2
∣∣∣∣∣∣1 2 00 1 21 −3 1
∣∣∣∣∣∣+ 3
∣∣∣∣∣∣1 2 30 1 −11 −3 5
∣∣∣∣∣∣∣∣∣∣∣∣+ − +− + −+ − +
∣∣∣∣∣∣ = −2
(1∣∣∣∣ 1 2−3 1
∣∣∣∣− 2∣∣∣∣ 0 2
1 1
∣∣∣∣)︸ ︷︷ ︸1st row
+ 3(
1∣∣∣∣ 1 −1−3 5
∣∣∣∣+ 1∣∣∣∣ 2 3
1 −1
∣∣∣∣)︸ ︷︷ ︸1st column
= −2 ((1 + 6)− 2 (−2)) + 3 ((5− 3) + (−5))
= −2 (7 + 4) + 3 (2− 5) = −22− 9 = −31
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 14 / 28
Example
∣∣∣∣∣∣∣∣∣∣1 0 1 2 30 −1 0 3 01 2 1 0 00 0 0 4 01 0 0 5 2
∣∣∣∣∣∣∣∣∣∣=
∣∣∣∣∣∣∣∣∣∣1 0 1 2 30 −1 0 3 01 2 1 0 00 0 0 4 01 0 0 5 2
∣∣∣∣∣∣∣∣∣∣= 4 · (−1)4+4
∣∣∣∣∣∣∣∣1 0 1 30 −1 0 01 2 1 01 0 0 2
∣∣∣∣∣∣∣∣= 4
(−1) · (−1)2+2
∣∣∣∣∣∣1 1 31 1 01 0 2
∣∣∣∣∣∣ = −4
∣∣∣∣∣∣1 1 31 1 01 0 2
∣∣∣∣∣∣= −4
(1 · (−1)3+1
∣∣∣∣ 1 31 0
∣∣∣∣+ 2 · (−1)3+3∣∣∣∣ 1 11 1
∣∣∣∣)= −4 ((−3) + 2 · (1− 1)) = 12
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 15 / 28
Classwork
Evaluate the determinant:∣∣∣∣∣∣∣∣∣1 1 0 −12 3 1 24 4 3 −23 4 1 3
∣∣∣∣∣∣∣∣∣Lecture resumes at 7:45.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 16 / 28
Solution
Expand along the first row seems to be (a little) easier:∣∣∣∣∣∣∣∣1 1 0 −12 3 1 24 4 3 −23 4 1 3
∣∣∣∣∣∣∣∣ = 1
∣∣∣∣∣∣3 1 24 3 −24 1 3
∣∣∣∣∣∣− 1
∣∣∣∣∣∣2 1 24 3 −23 1 3
∣∣∣∣∣∣+ 0
∣∣∣∣∣∣2 3 24 4 −23 4 3
∣∣∣∣∣∣− (−1)
∣∣∣∣∣∣2 3 14 4 33 4 1
∣∣∣∣∣∣=
∣∣∣∣∣∣3 1 24 3 −24 1 3
∣∣∣∣∣∣−∣∣∣∣∣∣2 1 24 3 −23 1 3
∣∣∣∣∣∣+∣∣∣∣∣∣2 3 14 4 33 4 1
∣∣∣∣∣∣∣∣∣∣∣∣3 1 24 3 −24 1 3
∣∣∣∣∣∣ = 3∣∣∣∣3 −21 3
∣∣∣∣− ∣∣∣∣4 −24 3
∣∣∣∣+ 2∣∣∣∣4 34 1
∣∣∣∣ = 3 (9 + 2)− (12 + 8) + 2 (4− 12) = −3
∣∣∣∣∣∣2 1 24 3 −23 1 3
∣∣∣∣∣∣ = 2∣∣∣∣3 −21 3
∣∣∣∣− ∣∣∣∣4 −23 3
∣∣∣∣+ 2∣∣∣∣4 33 1
∣∣∣∣ = 2 (9 + 2)− (12 + 6) + 2 (4− 9) = −6
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 17 / 28
Solution
∣∣∣∣∣∣∣∣1 1 0 −12 3 1 24 4 3 −23 4 1 3
∣∣∣∣∣∣∣∣ = 1
∣∣∣∣∣∣3 1 24 3 −24 1 3
∣∣∣∣∣∣− 1
∣∣∣∣∣∣2 1 24 3 −23 1 3
∣∣∣∣∣∣+ 0
∣∣∣∣∣∣2 3 24 4 −23 4 3
∣∣∣∣∣∣− (−1)
∣∣∣∣∣∣2 3 14 4 33 4 1
∣∣∣∣∣∣=
∣∣∣∣∣∣3 1 24 3 −24 1 3
∣∣∣∣∣∣︸ ︷︷ ︸−3
−
∣∣∣∣∣∣2 1 24 3 −23 1 3
∣∣∣∣∣∣︸ ︷︷ ︸−6
+
∣∣∣∣∣∣2 3 14 4 33 4 1
∣∣∣∣∣∣∣∣∣∣∣∣2 3 14 4 33 4 1
∣∣∣∣∣∣ = 2∣∣∣∣4 34 1
∣∣∣∣− 3∣∣∣∣4 33 1
∣∣∣∣+ ∣∣∣∣4 43 4
∣∣∣∣ = 2 (4− 12)− 3 (4− 9) + (16− 12) = 3
Thus, ∣∣∣∣∣∣∣∣1 1 0 −12 3 1 24 4 3 −23 4 1 3
∣∣∣∣∣∣∣∣ = (−3)− (−6) + 3 = 6.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 18 / 28
Solution
Alternatively, we can also use the row operations:
1 1 0 −12 3 1 24 4 3 −23 4 1 3
A
R2 ← R2 − 2R1
R3 ← R3 − 4R1
R4 ← R4 − 3R1∼
1 1 0 −10 1 1 40 0 3 20 1 1 6
A1
R4←R4−R2∼
1 1 0 −10 1 1 40 0 3 20 0 0 2
A2∣∣∣∣∣∣∣∣
1 1 0 −12 3 1 24 4 3 −23 4 1 3
∣∣∣∣∣∣∣∣ = det (A) = det (A1) = det (A2) = 1 · 1 · 3 · 2 = 6.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 19 / 28
Properties of determinants
det(I ) = 1det (AB) = det(A) det(B) This is importantA is invertible iff det(A) 6= 0 , and in this case det
(A−1
)= 1
det(A)
1 = det(I ) = det(AA−1) = det(A) det
(A−1)
det(AT)= det(A)
det(kA) = kn det(A) (Recall that ARj←−kRj∼ B ⇒ det (B) = k det(A))
If A has a row of zeros or a column of zeros, then det(A) = 0.If A has two equal rows or columns, then det(A) = 0. (rank(A) < n)
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 20 / 28
Full rank
For an n × n square matrix A, it is said to have full rank if rank(A) = n. Let A bea full rank n × n-matrix.
For x = [x1 x2 . . . xn]T and b = [b1 b2 . . . bn]
T ∈ Rn, the system Ax = b isconsistent and has a unique solution.Write the column vectors of A as A = [v1 v2 . . . vn]. The casting-outalgorithm will result in an echelon form of rank n, so that every column ispivot. v1, v2, . . . , vn are linearly independent ⇒ they form a basis of Rn.Alternatively, you can let b = 0 in the system above. The unique solution isthe trivial solution, indicating linear independence.The reduced echelon form of A is In. det (A) = C det(In) for some constantC 6= 0. Since det (In) = 1, det(A) = C 6= 0.[A|I ] ∼
[I |A−1
]so that A is invertible.
rank(AT ) = n
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 21 / 28
Example
1. How many solutions:
{5x + 2y = −97x + 3y = −13
? (Midterm I, Problem 11)
Solution.∣∣∣∣ 5 27 3
∣∣∣∣ = 5 · 3− 2 · 7 = 1 6= 0
2. Which of the following is a basis of R3 ? (Midterm II, Problem 14)
(A)
1
00
,
300
,
121
(B)
1
10
,
02−1
,
211
(C)
1
23
,
78−1
(D)
1
10
,
111
,
012
,
103
Solution. A basis of R3 should contain 3 vectors.
For (A),
∣∣∣∣∣∣1 3 10 0 20 0 1
∣∣∣∣∣∣ = 1 · 0 · 1 = 0 —not linearly independent.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 22 / 28
Example
3. Which two vectors would extend the set
2−6−20
,
1−2−10
to a basis of R4?
(Midterm II
Problem 16
)
(A)
0100
,
0001
(B)
1000
,
1010
(C)
1000
,
0001
(D)
1000
,
0100
Solution. We can compute determinants one by one:
(A)
∣∣∣∣∣∣∣∣2 1 0 0−6 −2 1 0−2 −1 0 00 0 0 1
∣∣∣∣∣∣∣∣ = 1
∣∣∣∣∣∣2 1 0−6 −2 1−2 −1 0
∣∣∣∣∣∣ = 1 · (−1)∣∣∣∣ 2 1−2 −1
∣∣∣∣ = 0
Similarly, (B)
∣∣∣∣∣∣∣∣2 1 1 1−6 −2 0 0−2 −1 0 10 0 0 0
∣∣∣∣∣∣∣∣ = 0, (D)
∣∣∣∣∣∣∣∣2 1 1 0−6 −2 0 1−2 −1 0 00 0 0 0
∣∣∣∣∣∣∣∣ = 0
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 23 / 28
Inverse
DefinitionLet A be an n × n matrix.
The cofactor matrix of A is cof (A) =
C11 C12 · · · C1nC21 C22 · · · C2n...
.... . .
...Cn1 Cn2 · · · Cnn
.
The adjugate of A is adj (A) = (cof (A))T =
C11 C21 · · · Cn1C12 C22 · · · Cn2...
.... . .
...C1n C2n · · · Cnn
.
Theorem
A−1 =1
det(A)adj (A)
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 24 / 28
Example:(A−1 = 1
det(A)adj (A))
Recall an early example:
A =
1 −1 2−3 5 10 2 3
=⇒C11 = 13 C12 = 9 C13 = −6C21 = 7 C22 = 3 C23 = −2
C31 = −11 C32 = −7 C33 = 2=⇒ det(A) = −8
cof (A) =
13 9 −67 3 −2−11 −7 2
adj(A) =
13 7 −119 3 −7−6 −2 2
=⇒ A−1 =1−8
13 7 −119 3 −7−6 −2 2
=
− 138 − 7
8118
− 98 − 3
878
34
14 − 1
4
Check AA−1 = I as an exercise
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 25 / 28
Example 2× 2
Let A =
[1 2−3 4
]=⇒ det (A) = 1 · 4− 2 (−3) = 10 6= 0.
M11 = 4 M12 = −3M21 = 2 M22 = 1 =⇒ C11 = 4 C12 = 3
C21 = −2 C22 = 1
cof (A) =
[4 3−2 1
]=⇒ adj(A) = cof (A)T =
[4 −23 1
]Thus,
A−1 =1
det(A)adj(A) =
110
[4 −23 1
]=
[ 25 − 1
5310
110
]Theorem [
a bc d
]−1
=1
ad − bc
[d −b−c a
]Proof. [
a bc d
] [d −b−c a
]=
[ad − bc 0
0 ad − bc
]Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 26 / 28
Cramer’s rule (*)
TheoremSuppose A is an invertible n × n-matrix and we wish to solve the system Ax = b ,where x = [x1 x2 . . . xn]
T and b = [b1 b2 . . . bn]T ∈ Rn.
Then xi can be computed by the rule xi = det(Ai )/ det(A) where Ai is the matrixobtained by replacing the ith column of A with b.
Example{5x + 2y = −97x + 3y = −13
⇒ A =
[5 27 3
]⇒ det(A) = 1 6= 0. x =
[xy
]and b =
[−9−13
]
x =det (A1)
det (A)=
∣∣∣∣ −9 2−13 3
∣∣∣∣1
= (−9) · 3− 2 (−13) = −1
y =det (A2)
det (A)=
∣∣∣∣5 −97 −13
∣∣∣∣1
= 5 · (−13)− (−9) · 7 = −2
5 (−1) + 2 (−2) = −5− 4 = −9
7 (−1) + 3 (−2) = −7− 6 = −13
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 27 / 28
Summary of Chpt. 7
Determinant: ONLY square matrices have determinants.1 Calculation
2× 2 and triangular;
Row operations;
A
Ri←→Rj∼ B ⇒ det (B) = − det(A)
ARj←−kRj∼ B ⇒ det (B) = k det(A)
ARj←−Rj+aRi∼ B ⇒ det (B) = det(A)
minor, cofactor, expansionYou should be able to compute a 4× 4 or 5× 5 determinant.
2 Property:det (A) 6= 0rank(A) = nrow/column vectors of A are linearly independent ⇒ form a basis of Rn
A is invertible with det(A−1) 6= 0 A−1 = 1det(A)
adj (A)
the system Ax = b has a unique solution ⇒if b = 0, the homogeneous systemhas only the trivial solution
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 11th, 2019 28 / 28
MATH-1030Matrix Theory & Linear Algebra I
Lin Jiu
Dalhousie University
June 13th, 2019
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 1 / 25
Eigenvalues and eigenvectors
The red arrow changes direction butthe blue arrow does not. The blue ar-row is an eigenvector of this mappingbecause it does not change direction,and since its length is unchanged, itseigenvalue is 1.
DefinitionLet A be an n × n matrix, v be a non-zero vector, and λ be a scalar such thatAv = λv. Then,
v is called an eigenvector of A,and λ is called the corresponding eigenvalue.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 2 / 25
Example
1. A =
[2 00 1
]. Which of the following are eigenvectors of A?
u1 =
[10
]u2 =
[01
]u3 =
[11
]
Au1 =
[2 00 1
] [10
]=
[2 · 1+ 0 · 00 · 1+ 1 · 0
]=
[20
]= 2u1 YES eigenvalue 2
Au2 =
[2 00 1
] [01
]= 0
[20
]+ 1
[01
]=
[01
]= u2 YES eigenvalue 1
Au3 =
[2 00 1
] [11
]=
[2 00 1
] [10
]+
[2 00 1
] [01
]=
[21
]6= ku3 NO
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 3 / 25
Example
2. Let A =
[4 −36 −5
]and u1 =
[13
], u2 =
[12
], u3 =
[11
], and u4 =
[00
]. Which
are the eigenvalues of A? Find the corresponding eigenvalues too.Solution.
Au1 =
[4 −36 −5
] [13
]=
[4 · 1+ (−3) · 36 · 1+ (−5) · 3
]=
[−5−9
]6= λu1 NO
Au2 =
[4 −36 −5
] [12
]= 1
[46
]+ 2
[−3−5
]=
[−2−4
]= −2u2 YES
Eigenvalueλ = −2
Au3 =
[4 −36 −5
] [11
]= 1
[46
]+ 1
[−3−5
]=
[11
]= u3 YES
Eigenvalueλ = 1
Au4 =
[4 −36 −5
] [00
]=
[00
]= u4 NO An eigenvector must be non-zero.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 4 / 25
Example
3. Let A =
−3 0 0−6 −1 23 −6 6
. The eigenvalues are 2, 3 and −3. Find the
corresponding eigenvector.Solution. (1) We need to solve v1 from the equation: Av1 = 2v1 ⇐⇒ Av1 = 2Iv1
⇐⇒ 0 = Av1 − 2Iv1 = (A− 2I ) v1 (A− 2I ) v1 = 0 –Homogeneous system
A− 2I =
−3 0 0−6 −1 23 −6 6
− 2
1 0 00 1 00 0 1
=
−5 0 0−6 −3 23 −6 4
−5 0 0 0−6 −3 2 03 −6 4 0
R1←− 15R1∼
1 0 0 0−6 −3 2 03 −6 4 0
R2←R2+6R1R3←R3−3R1∼ 1 0 0 0
0 −3 2 00 −6 4 0
R3←R3−2R2∼
1 0 0 00 −3 2 00 0 0 0
R2←− 13R2∼
1 0 0 00 1 − 2
3 00 0 0 0
xyz
=
023 tt
t=3=⇒ v1 =
023
(to avoid fractions) Exercise check: Av1 = 2v1
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 5 / 25
Example
(2) Av2 = 3v2 ⇐⇒ Av2 = 3Iv2 ⇐⇒ (A− 3I ) v2 = 0 — (A− 3I ) v2 = 0
A− 3I =
−3 0 0−6 −1 23 −6 6
− 3
1 0 00 1 00 0 1
=
−6 0 0−6 −4 23 −6 3
R1←− 16R1∼
1 0 0−6 −4 23 −6 3
R2←R2+6R1R3←R3−3R1∼
1 0 00 −4 20 −6 3
R2←−R24∼
1 0 00 1 − 1
20 −6 3
R3←R3+6R2∼
1 0 00 1 − 1
20 0 0
1 0 0 0
0 1 − 12 0
0 0 0 0
⇒xyz
=
0t2t
t=2=⇒ v2 =
012
Double-check:
Av2 =
−3 0 0−6 −1 23 −6 6
012
=
−3 · 0+ 0 · 1+ 0 · 2−6 · 0− 1 · 1+ 2 · 23 · 0− 6 · 1+ 6 · 2
=
036
= 3v2
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 6 / 25
Example(3) Av3 = −3v3 ⇐⇒ (A+ 3I ) v3 = 0 — (A+ 3I ) v3 = 0
A+ 3I =
−3 0 0−6 −1 23 −6 6
+ 3
1 0 00 1 00 0 1
=
0 0 0−6 2 23 −6 9
R1↔R2R2↔R3∼
−6 2 23 −6 90 0 0
R1←−
R66∼
1 − 13 − 1
33 −6 90 0 0
R2←R2−3R1∼
1 − 13 − 1
30 −5 100 0 0
R2←−R2
5∼
1 − 13 − 1
30 1 −20 0 0
R1←R1+
R23∼
1 0 −10 1 −20 0 0
1 0 −1 0
0 1 −2 00 0 0 0
=⇒
xyz
=
t2tt
t=1=⇒ v3 =
121
Double-check:
Av3 =
−3 0 0−6 −1 23 −6 6
121
= 1
−3−63
+ 2
0−1−6
+ 1
026
=
−3−6−3
= −3v3
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 7 / 25
Remark
From the two examples above, we see, for an square matrix A:1 Given an eigenvector v, we can compute Av = λv to find the corresponding
eigenvalue.2 Given an eigenvalue λ, since Av = λv⇐⇒ (A− λI ) v = 0, we can proceed
to find the solution space of the homogeneous system.3 It is obvious that for an eigenvalue λ, there are more than one eigenvectors:
If Av = λv, for any non-zero scalar k: A (kv) = k (Av) = kλv = λ (kv);If both v1 and v2 are eigenvectors of A with respect to the SAME eigenvalueλ: A (v1 + v2) = Av1 + Av2 = λv1 + λv2 = λ (v1 + v2)This is natural since for an eigenvalue λ, the solution space of thehomogeneous system: (A− λI ) v = 0 contains all the eigenvectors and thezero vector, which is a subspace. Naturally, it is closed under addition andscalar multiplication.
4 Just given the matrix A, can we find the eigenvalues and eigenvectors?
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 8 / 25
Finding eigenvalues
Without given any information, we can first find the eigenvalues and then thecorresponding eigenvectors.
a squre matrix −→ eigenvalue(s) −→ eigenvectors.
In general, λ is an eigenvalue if and only if there exists a non-zero vector v suchthat
Av = λv ⇐⇒ Av = λIv ⇐⇒ Av − λIv = 0 ⇐⇒ (A− λI ) v = 0⇐⇒ The homogeneous system (A− λI ) v = 0 has non-trivial solutions.⇐⇒ det (A− λI ) = 0 ←− This only involves λ not v.Therefore, we can take the following two steps:
1 Solve the equation det (A− λI ) = 0. Solutions λ1, . . . , λn are eigenvalues.2 For each λi , to find the corresponding eigenvectors, we solve the
homogeneous system (A− λi I )v = 0. In particular, we choose the basicsolutions as the representatives of the eigenvectors.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 9 / 25
Example
Find the eigenvalues and eigenvectors of A =
[1 1−2 4
].
Solution. 1. Solve λ from the equation det (A− λI ) = 0.
A− λI =[1 1−2 4
]− λ
[1 00 1
]=
[1 1−2 4
]−[λ 00 λ
]=
[1−λ 1−2 4−λ
]A− λI is obtained by adding −λ to all the main diagonal entries.
det (A− λI ) = (1− λ) (4− λ)− 1 (−2) = 4 −λ −4λ +λ2 +2 = λ2 − 5λ+ 6
det (A− λI ) = λ2 − 5λ+ 6 = (λ− 3) (λ− 2) ax2 + bx + c =
a
(x − −b+
√b2−4ac
2a
)(x − −b−
√b2−4ac
2a
)
=⇒ det (A− λI ) = 0 has two solutions: λ1 = 2 and λ2 = 3, which are theeigenvalues of A.2. For each eigenvalue (λ1, λ2 here), find the corresponding eigenvector, bysolving the homogeneous system
(A− λI ) v = 0
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 10 / 25
Example
A =
[1 1−2 4
]=⇒ λ1 = 2, λ3 = 3
λ1 = 2: A− 2I =[1−2 1−2 4−2
]=
[−1 1−2 2
]Row operations:[
−1 1−2 2
]R1←−R1∼
[1 −1−2 2
]R2←R2+2R1∼
[1 −10 0
]⇒[
1 −1 00 0 0
]Suppose the two unknows are x and y . Then, y = t is free and x = y = t.
General solution:[tt
]and we pick the basic solution v1 =
[11
], as the
representative of the eigenvectors with respect to λ1 = 2.
Check:
Av1 =
[1 1−2 4
] [11
]=
[1 · 1+ 1 · 1
(−2) · 1+ 4 · 1
]=
[22
]= 2
[11
]= λ1v1.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 11 / 25
Example
A =
[1 1−2 4
]=⇒ λ1 = 2, λ3 = 3
λ2 = 3: A− 3I =[1−3 1−2 4−3
]=
[−2 1−2 1
]Row operations:[
−2 1−2 1
]R2←R2−R1∼
[−2 10 0
]R1←− 1
2R1∼[1 − 1
20 0
]⇒[
1 − 12 0
0 0 0
]Suppose the two unknows are x and y . Then, y = t is free andx = y/2 = t/2.
General solution:[t/2t
]and we pick the basic solution v2 =
[1/21
], as the
representative of the eigenvectors with respect to λ2 = 3.
Check:Av2 =
[1 1−2 4
] [1/21
]= 1
2
[1−2
]+ 1
[14
]=
[ 323
]= 3
[1/21
]= λ2v2.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 12 / 25
Example
Therefore, for matrix
A =
[1 1−2 4
]It has two eigenvalues: λ1 = 2, λ3 = 3
For λ1 = 2, the basis of corresponding eigenvectors is{[11
]}For λ2 = 3, the basis of corresponding eigenvectors is{[
1/21
]}How about a 3× 3 matrix?
Example
Find the eigenvalues and eigenvectors of A =
−2 −3 60 1 0−3 −3 7
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 13 / 25
Example
A =
−2 −3 60 1 0−3 −3 7
Solve the equation
det (A− λI ) = 0: A− λI =
−2− λ −3 60 1− λ 0−3 −3 7− λ
det (A− λI ) =
∣∣∣∣∣∣−2− λ −3 6
0 1− λ 0−3 −3 7− λ
∣∣∣∣∣∣ = (1− λ)∣∣∣∣−2− λ 6−3 7− λ
∣∣∣∣= (1− λ) [(−2− λ) (7− λ)− 6 (−3)]= (1− λ)
[−14+ 2λ− 7λ+ λ2 + 18
]= (1− λ)
(λ2 − 5λ+ 4
)= (1− λ) (λ− 1) (λ− 4)= − (λ− 1)2 (λ− 4)
The roots of det (A− λI ) = − (λ− 1)2 (λ− 4) = 0 are the eigenvalues of A:λ1 = 1 and λ2 = 4
For λ1 = 1: A− I =
−3 −3 60 0 0−3 −3 6
∼1 1 −20 0 00 0 0
y = t
z = s⇒ x = −t + 2s
General solution:
xyz
= t
−110
+ s
201
basicsolutions
:
−110
,201
.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 14 / 25
Example
A =
−2 −3 60 1 0−3 −3 7
⇒ A− 4I =
−6 −3 60 −3 0−3 −3 3
R1←R1/(−6)
R2←R2/(−3)R3←R3/(−3)∼
1 1/2 −10 1 01 1 −1
R3←R3−R1∼
1 1/2 −10 1 00 1/2 0
R1←R1−R2/2R3←R3−R2/2∼
1 0 −10 1 00 0 0
y = 0, z = t =⇒ x = z = t
General solution:
xyz
= t
101
=⇒ Basic solution:
101
.A has two eigenvalues: λ1 = 1, λ2 = 4
Eigenvalue λ = 1: Multiplicity 2 ; Basis of eigenvectors :
−110
,201
Eigenvalue λ = 4: Multiplicity 1 ; Basis of eigenvectors :
101
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 15 / 25
Classwork
A =
1 3 30 4 30 0 7
Find the eigenvalues and eigenvectors of A
Lecture resumes at 7:55.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 16 / 25
Solution
A =
1 3 30 4 30 0 7
=⇒ A− λI =
1− λ 3 30 4− λ 30 0 7− λ
=⇒ det (A− λI ) = (1− λ) (4− λ) (7− λ)=⇒ det (A− λI ) = 0 has three roots: λ1 = 1, λ2 = 4, λ3 = 7
λ1 = 1: A− I =
0 3 30 3 30 0 6
∼0 1 10 1 10 0 1
∼0 1 10 0 10 0 0
∼0 1 00 0 10 0 0
xyz
=
t00
= t
100
. Basic solution: v1 =
100
.λ2 = 4: A− 4I =
−3 3 30 0 30 0 3
∼1 −1 −10 0 10 0 1
∼1 −1 00 0 10 0 0
xyz
=
tt0
= t
110
. Basic solution: v2 =
110
.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 17 / 25
Solution
A =
1 3 30 4 30 0 7
=⇒ det (A− λI ) = (1− λ) (4− λ) (7− λ)
λ3 = 7: A− 7I =
−6 3 30 −3 30 0 0
∼1 − 1
2 − 12
0 1 −10 0 0
∼1 0 −10 1 −10 0 0
xyz
=
ttt
= t
111
. Basic solution: v3 =
111
.A has three eigenvalues:
Eigenvalue λ = 1: Basis of eigenvectors :
100
Eigenvalue λ = 4: Basis of eigenvectors :
110
Eigenvalue λ = 7: Basis of eigenvectors :
111
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 18 / 25
Characteristic polynomial
DefinitionIf A is a square matrix , the expression p (λ) = det (A− λI ) is always apolynomial in λ. It is called the characteristic polynomial of A. Moreover, theeigenvalues of A are the roots of the characteristic polynomial.
Example
1. A =
1 3 30 4 30 0 7
The characteristic polynomial is :
det (A− λI ) =
∣∣∣∣∣∣1− λ 3 30 4− λ 30 0 7− λ
∣∣∣∣∣∣ = (1− λ) (4− λ) (7− λ) .
2. A =
[1 1−2 4
]The characteristic polynomial is
det (A− λI ) =∣∣∣∣1− λ 1−2 4− λ
∣∣∣∣ = (1− λ) (4− λ)− 1 (−2) = (λ− 3) (λ− 2)Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 19 / 25
Properties
A =
1 3 30 4 30 0 7
=⇒ p1 (λ) =
∣∣∣∣∣∣1− λ 3 30 4− λ 30 0 7− λ
∣∣∣∣∣∣ = (1− λ) (4− λ) (7− λ).
If A is an n × n matrix, the degree of p (λ) is n. There are n eigenvaluescounting multiplicity.
B =
−2 −3 60 1 0−3 −3 7
=⇒ p2 (λ) = (4− λ) (λ− 1)2 ⇒
{λ1 = λ2 = 1λ3 = 4
The constant term of the characteristic polynomial = p (0) = det (A)p1 (0) = 1 · 4 · 7 = det (A) Exercise: check, p2 (0) = 4 = det (B)
(*) The n − 1 term
p (λ) = (−1)nλn + (−1)n−1 tr(A)λn−1 + · · ·+ det(A).
For triangluar matrics: p(λ) = (a11 − λ) (a22 − λ) · · · (ann − λ) Eigenvaluesare the entries on the main diagonal.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 20 / 25
Multiplicity
Recall the example B =
−2 −3 60 1 0−3 −3 7
=⇒ p2 (λ) = (4− λ) (λ− 1)2
B has two eigenvalues: λ1 = 1, λ2 = 4
Eigenvalue λ1 = 1: Multiplicity 2 ; Basis of eigenvectors :
−110
,201
Eigenvalue λ2 = 4: Multiplicity 1 ; Basis of eigenvectors :
101
Focus on λ1 = 1, the solution space of the homogeneous system (A− I ) v = 0.
has dimension 2, since the basis
−11
0
,201
has two vectors.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 21 / 25
Multiplicity
Consider another example A =
2 1 00 2 00 0 1
1. Find the characteristic polynomial: det (A− λI ) =
∣∣∣∣∣∣2− λ 1 00 2− λ 00 0 1− λ
∣∣∣∣∣∣= (2− λ)2 (1− λ)
2. Find the roots (eigenvalues):λ1 = 1 with multiplicity 1and λ2 = 2 with multiplicity 2.
3. Find the eigenvectors:
λ1 = 1: A− I =
1 1 00 1 00 0 0
∼ 1 0 0
0 1 00 0 0
=⇒ v1 =
00t
= t
001
λ2 = 2: A− 2I =
0 1 00 0 00 0 −1
∼ 0 1 0
0 0 10 0 0
=⇒ v2 =
t00
= t
100
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 22 / 25
Multiplicity
We see for λ1 = 2, it has multiplicity 2 in the equation p (λ) = det (A− λI ) = 0and the dimension of the solution space of (A− 2I ) v = 0 is 1.
DefinitionLet A be a square matrix.
1 If λ is a root of the characteristic polynomial of multiplicity k , we say that λis an eigenvalue of algebraic multiplicity k .
2 For an eigenvalue λ, the solution space {v|(A− λI )v = 0} = {v | Av = λv},denoted by Eλ, is called an eigenspace of A, with respect to λ.
3 If λ has an m-dimensional eigenspace , i.e., dimEλ = m, we say λ hasgeometric multiplicity m.
Theorem
1 ≤ geometric multiplicity ≤ algebraic multiplicity.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 23 / 25
Examples
As we computed before:
Matrix
−2 −3 60 1 0−3 −3 7
has two eigenvalues:
Eigenvalue λ1 = 1: algebraic multiplicity = geometric multiplicity = 2Eigenvalue λ2 = 4: algebraic multiplicity = geometric multiplicity = 1
Matrix
2 1 00 2 00 0 1
has two eigenvalues:
Eigenvalue λ1 = 1: algebraic multiplicity = geometric multiplicity = 1Eigenvalue λ2 = 2: algebraic multiplicity = 2, while geometric multiplicity = 1
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 24 / 25
Preview
Next time (the last lecture) we shall see that:An n × n-matrix is diagonalizable iff for all its eigenvalues, the algebraicmultiplicity= geometric multiplicity.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 25 / 25
MATH-1030Matrix Theory & Linear Algebra I
Lin Jiu
Dalhousie University
June 18th, 2019
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 18th, 2019 1 / 24
Recall
DefinitionLet A be an n × n matrix, v be a non-zero vector, and λ be a scalar such thatAv = λv. Then,
v is called an eigenvector of A,and λ is called the corresponding eigenvalue.
RemarkAn eigenvector cannot be a zero vector, but an eigenvalue can be 0.3 2 1
0 2 10 0 0
003
=
000
.To find eigenvalues and eigenvectors, we need to :
1 Find the characteristic polynomial p (λ) = det (A− λI )2 Find the eigenvalues, which are the roots of p(λ) = 0.3 For each λi , solve the homogeneous system (A− λI ) v = 0, by providing the
basic solutions.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 18th, 2019 2 / 24
Recall
DefinitionLet A be a square matrix.
1 If λ is a root of the characteristic polynomial of multiplicity k , we say that λis an eigenvalue of algebraic multiplicity k .
2 For an eigenvalue λ, the solution space {v|(A− λI )v = 0} = {v | Av = λv},denoted by Eλ, is called an eigenspace of A, with respect to λ.
3 If λ has an m-dimensional eigenspace, i.e., dimEλ = m, we say λ hasgeometric multiplicity m.
Theorem
1 ≤ geometric multiplicity ≤ algebraic multiplicity.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 18th, 2019 3 / 24
Diagonal matrices (n × n)
Definition
A diagonal matrix is of the form
d1 0 0 · · · 00 d2 0 · · · 00 0 d3 · · · 0...
......
. . ....
0 0 0 · · · dn
.Namely, only the entries on the main diagonal can be non-zero.
Example
I2 =
[1 00 1
] 1 0 00 1 00 0 0
RemarkIt does not require d1, . . . , dn to be nonzero.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 18th, 2019 4 / 24
Diagonal matricesAdding, subtracting, multiplying diagonal matrices are much easier and resultsare also diagonal matrices.[
a 00 b
]+
[c 00 d
]=
[a+ c 00 b + d
][a 00 b
] [c 00 d
]=
[a · c + 0 · 0 a · 0+ 0 · c0 · c + b · 0 0 · 0+ b · d
]=
[ac 00 bd
]In particular,
[a 00 b
] [a 00 b
]=
[a2 00 b2
]=⇒
([a 00 b
])100
=
[a100 00 b100
].
In general: all operations are computed componentwise on the diagonal.c1 0 · · · 00 c2 · · · 0...
.... . .
...0 0 · · · cn
+
d1 0 · · · 00 d2 · · · 0...
.... . .
...0 0 · · · dn
=
c1 + d1 0 · · · 0
0 c2 + d2 · · · 0...
.... . .
...0 0 · · · cn + dn
c1 0 · · · 00 c2 · · · 0...
.... . .
...0 0 · · · cn
d1 0 · · · 00 d2 · · · 0...
.... . .
...0 0 · · · dn
=
c1d1 0 · · · 00 c2d2 · · · 0...
.... . .
...0 0 · · · cndn
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 18th, 2019 5 / 24
Example
Let A =
1 0 00 4 00 0 9
. Find a matrix B such that B2 = A.
Solution. From the computation above, we see B =
1 0 00 2 00 0 3
satisfies
B2 =
1 0 00 2 00 0 3
1 0 00 2 00 0 3
=
12 0 00 22 00 0 32
=
1 0 00 4 00 0 9
= A
Of course there are other solutions−1 0 00 2 00 0 3
,1 0 00 −2 00 0 3
,−1 0 0
0 −2 00 0 3
,1 0 00 2 00 0 −3
−1 0 0
0 2 00 0 −3
,1 0 00 −2 00 0 −3
,−1 0 0
0 −2 00 0 −3
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 18th, 2019 6 / 24
An old example
Recall an early example A =
−3 0 0−6 −1 23 −6 6
p (A) = − (λ− 1)2 (λ− 4)
λ = 1 basis
−11
0
,201
λ = 4 basis
101
=⇒
A
−110
=
−110
A
201
=
201
A
101
= 4
101
=
404
Let P =
−1 2 11 0 00 1 1
=⇒AP =
A−11
0
A
201
A
101
=
−1 2 41 0 00 1 4
P
1 0 00 1 00 0 4
︸ ︷︷ ︸
D
=
P100
P
010
P
004
=
−1 21 00 1
P
004
=
−1 2 41 0 00 1 4
.
We see AP = PD ⇐⇒ A = PDP−1.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 18th, 2019 7 / 24
Diagonalization
TheoremLet A be an n × n matrix, with n linearly independent eigenvectors v1, . . . , vnand corresponding eigenvalues λ1, . . . , λn. Avk = λkvk k = 1, . . . , n ( allowingrepetitions of λ1, . . . , λn) Then, there exists an invertible matrix P and a diagonalmatrix D such that
P−1AP = D ⇐⇒ A = PDP−1
In the case, we say that A is diagonalizable. In particular,P is the matrix whose columns are v1, . . . , vn.D is the matrix whose main diagonal entries are λ1, . . . , λn.
RemarkThe key is that A has n linearly independent eigenvectors.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 18th, 2019 8 / 24
Example
Diagonalize the matrix A =
[2 −21 5
]Solution. 1. Characteristic polynomial
det (A− λI ) =∣∣∣∣ 2− λ −2
1 5− λ
∣∣∣∣ = (2− λ) (5− λ)− (−2) 1 = 10− 7λ+ λ2 + 2
= λ2 − 7λ+ 12 = (λ− 3) (λ− 4)
2. Eigenvalues: det (A− λI ) = 0 =⇒ λ1 = 3 λ2 = 43. Eigenvectors
A− 3I =[−1 −21 2
]∼[1 20 0
]=⇒ x = −2y =⇒ v1 =
[−21
]A− 4I =
[−2 −21 1
]∼[1 10 0
]=⇒ x = −y =⇒ v2 =
[−11
]4. Diagonalize: D =
[3 00 4
], P =
[−2 −11 1
]=⇒ P−1 = adj(P)
det(P)
M11= 1, M12 = 1,M21 = −1, M22 = −2
⇒ cof (P) =
[1 −1
−(−1) −2
]⇒ adj(P) =
[1 1−1 −2
]Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 18th, 2019 9 / 24
Example
A =
[2 −21 5
]⇒ λ1 = 3, v1 =
[−21
]λ2 = 4, v2 =
[−11
]D =
[3 00 4
], P =
[−2 −11 1
]=⇒ P−1 = adj(P)
det(P) = 1−1
[1 1−1 −2
]=
[−1 −11 2
]P−1AP = D ⇐⇒ A = PDP−1
5. Check:P−1AP =
[−1 −11 2
] [2 −21 5
] [−2 −11 1
]=
[(−1) · 2+ (−1) · 1 (−1) · (−2) + (−1) · 5
1 · 2+ 2 · 1 1 · (−2) + 2 · 5
] [−2 −11 1
]=
[−3 −34 8
] [−2 −11 1
]=
[(−3) · (−2) + (−3) · 1 (−3) · (−1) + (−3) · 1
4 · (−2) + 8 · 1 4 · (−1) + 8 · 1
]=
[3 00 4
]= D
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 18th, 2019 10 / 24
Example
Diagonalize the matrix A =
2 0 01 4 −1−2 −4 4
Solution. 1. Characteristic polynomial
det (A− λI ) =
∣∣∣∣∣∣2− λ 0 01 4− λ −1−2 −4 4− λ
∣∣∣∣∣∣ = (2− λ)∣∣∣∣4− λ −1−4 4− λ
∣∣∣∣= (2− λ) [(4− λ) (4− λ)− (−4) (−1)]= (2− λ)
[λ2 − 8λ+ 16− 4
]= (2− λ)
(λ2 − 8λ+ 12
)= (2− λ) (λ− 2) (λ− 6) = − (λ− 2)2 (λ− 6)
2. Eigenvalues: det (A− λI ) = 0 =⇒ λ1 = λ2 = 2, λ3 = 63. Eigenvectors
For λ = 2, A− 2I =
0 0 01 2 −1−2 −4 2
∼ 1 2 −1
0 0 00 0 0
⇒ x = −2y + zxyz
=
−2t + sts
= t
−210
+ s
101
=⇒ v1 =
−210
, v2 =
101
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 18th, 2019 11 / 24
Example
A =
2 0 01 4 −1−2 −4 4
=⇒λ1 = λ2 = 2
λ3 = 6, v1 =
−210
, v2 =
101
For λ = 6, A− 6I =
−4 0 01 −2 −1−2 −4 −2
∼ 1 0 0
1 −2 −11 2 1
∼
1 0 00 −2 −10 2 1
∼ 1 0 0
0 −2 −10 0 0
∼ 1 0 0
0 1 1/20 0 0
⇒
{x = 0y = − z
2
⇒
xyz
=
0−r/2r
=
0−1/21
=⇒ v3 =
0−1/21
D =
2 0 00 2 00 0 6
P =
−2 1 01 0 −1/20 1 1
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 18th, 2019 12 / 24
Example
A =
2 0 01 4 −1−2 −4 4
=⇒ D =
2 0 00 2 00 0 6
P =
−2 1 01 0 −1/20 1 1
P−1 =
− 14
12
14
12 1 1
2− 1
2 −1 12
=⇒ PDP−1 = A
RemarkThe order is not unique. Since both v1 and v2 are eigenvectors with respectto λ = 2, we can let
D =
2 0 00 2 00 0 6
P1 =
1 −2 00 1 −1/21 0 1
Or, we can consider
D2 =
2 0 00 6 00 0 2
P2 =
1 0 −20 −1/2 11 1 0
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 18th, 2019 13 / 24
Remark
The key is to make sure that eigenvalues and eigenvectors are located at the samecolumns.
We do not really require the basic solution(s)
A =
2 0 01 4 −1−2 −4 4
=⇒ D =
2 0 00 2 00 0 6
P =
−2 1 01 0 −1/20 1 1
•
D =
2 0 00 2 00 0 6
P3 =
−2 1 01 0 −10 1 2
•
D =
2 0 00 2 00 0 6
P4 =
−2 1 01 0 10 1 −2
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 18th, 2019 14 / 24
Break
During the break, please consider this question.Diagonalize
A =
8 0 10−6 −3 −6−5 0 −7
.
Namely, find a diagonal matrix D and an invertiblematrix P such that
A = PDP−1
Lecture resumes at 7:30.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 18th, 2019 15 / 24
Solution
A =
8 0 10−6 −3 −6−5 0 −7
1. Characteristic polynomial: det (A− λI ) =
∣∣∣∣∣∣8− λ 0 10−6 −3− λ −6−5 0 −7− λ
∣∣∣∣∣∣= − (λ+ 3)
∣∣∣∣8− λ 10−5 −7− λ
∣∣∣∣= − (λ+ 3) [(8− λ) (−7− λ)− (−5) · 10] = − (λ+ 3)
(λ2 − λ− 6
)= − (λ+ 3) (λ+ 2) (λ− 3)
2. Eigenvalues: det (A− λI ) = 0 =⇒ λ1 = −3, λ2 = −2, λ3 = 33. Eigenvectors
For λ1 = −3, A+ 3I =
11 0 10−6 0 −6−5 0 −4
∼ 1 0 10
111 0 11 0 4
5
∼
1 0 11 0 10
111 0 4
5
∼ 1 0 1
0 0 − 111
0 0 − 15
∼ 1 0 0
0 0 10 0 0
=⇒
xyz
= t
010
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 18th, 2019 16 / 24
Solution
A =
8 0 10−6 −3 −6−5 0 −7
⇒ λ1 = −3, λ2 = −2, λ3 = 3, v1 =
010
For λ2 = −2, A+ 2I =
10 0 10−6 −1 −6−5 0 −5
∼ 1 0 1
6 1 60 0 0
∼
1 0 10 1 00 0 0
=⇒
xyz
= t
−101
For λ3 = 3, A− 3I =
5 0 10−6 −6 −6−5 0 −10
∼ 1 0 2
1 1 10 0 0
∼
1 0 20 1 −10 0 0
=⇒
xyz
= t
−211
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 18th, 2019 17 / 24
Solution
A =
8 0 10−6 −3 −6−5 0 −7
λ1 = −3, v1 =
010
λ2 = −2, v2 =
−101
λ3 = 3, v3 =
−211
Thus,
P =
0 −1 −21 0 10 1 1
D =
−3 0 00 −2 00 0 3
such that
A = PDP−1
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 18th, 2019 18 / 24
Application
Example
Let A =
−3 0 0−6 −1 23 −6 6
. Find A100
Solution
P =
−1 2 11 0 00 1 1
=⇒ P−1 =
0 1 01 1 −1−1 −1 2
and D =
1 0 00 1 00 0 4
=⇒
P−1AP = D ⇔ A = PDP−1.
A100 =(PDP−1) (PDP−1) · · · (PDP−1) (PDP−1)︸ ︷︷ ︸
100 copies
= PD���P−1PD���P−1PD · · ·���P−1PDP−1 = PD100P−1. Thus,
A100 =
−1 2 11 0 00 1 1
1 0 00 1 00 0 4100
0 1 01 1 −1−1 −1 2
=
−b + 2 −b + 1 2b − 20 1 0
−b + 1 −b + 1 2b − 1
b=4100
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 18th, 2019 19 / 24
Application
Example
Let A =
[2 −21 5
]. Find a square root of A. Namely, find B such that B2 = A.
Solution
P =
[−2 −11 1
]=⇒ P−1 =
[−1 −11 2
]and D =
[3 00 4
]=⇒ P−1AP = D ⇔ A = PDP−1. We just need to find a square root of D. Let
E =
[√3 00 2
]=⇒ E 2 =
[√3 00 2
] [√3 00 2
]=
[√3 ·√3 0
0 2 · 2
]=
[3 00 4
]= D
Now, consider B := PEP−1.
B2 =(PEP−1) (PEP−1) = PEP−1PEP−1 = PE 2P−1 = PDP−1 = A.
B := PEP−1 =
[−2+ 2
√3 −4+ 2
√3
2−√3 4−
√3
]Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 18th, 2019 20 / 24
Remark
1 Since A = PDP−1,
An = PDnP−1 and A1n = PD
1n P−1
where (for m = n or m = 1/n).d1 0 0 · · · 00 d2 0 · · · 00 0 d3 · · · 0...
......
. . ....
0 0 0 · · · dn
m
=
dm
1 0 0 · · · 00 dm
2 0 · · · 00 0 dm
3 · · · 0...
......
. . ....
0 0 0 · · · dmn
However, to get the square root, we need d1, . . . , dn ≥ 0.
2 Recall another example A =
2 1 00 2 00 0 1
which has:
eigenvalue λ = 1, multiplicity 1, Basis of eigenvectors
001
;
eigenvalue λ = 2 multiplicity 2, Basis of eigenvectors
100
;
In this case,A is NOTdiagonalizable.(n linearlyindependenteigenvectors.)
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 18th, 2019 21 / 24
Remark
TheoremEigenvectors with respect to distinct eigenvalues are linearly independent. Let Abe an n × n matrix and λ1, . . . , λk be its distinct eigenvalues. For each λj , pickANY eigenvector vj with respect to λj . Then, {v1, . . . , vk} are linearlyindependent.
Proof. (* You do not need to remember the proof).Suppose vj = a1v1 + · · ·+ aj−1vj−1, for a1, . . . , aj−1 are not all zero.λjvj = Avj = A (a1v1 + · · ·+ aj−1vj−1) = a1λ1v1 + · · ·+ aj−1λj−1vj−1=⇒ 0 = a1 (λ1 − λj) v1 + · · ·+ aj−1 (λj−1 − λj) vj−1.From here, one can prove that {v1, . . . , vk} are linearly independent by inductionon j and contradiction.
RemarkAs we know, the solution space of a homogeneous system= span {basic solutions}. Namely, the basic solutions of the eigenspace for a giveneigenvalue are linearly independent.
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 18th, 2019 22 / 24
Remark
TheoremLet A be an n × n matrix and λ1, . . . , λk (k ≤ n) be its distinct eigenvalues. A isdiagonalizable iff for each λj , j = 1, . . . , k , its algebraic multiplicity equals itsgeometric multiplicity. In particular, if A has n distinct eigenvalues i.e., k = n, Ais diagonalizable.
Example
A =
2 1 00 2 00 0 1
. A is of ”the closest” form to a diagonal matrix.
Jordan canonical form (*)
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 18th, 2019 23 / 24
Summary of Chpt. 8
Eigenvalues and eigenvectors for square matrix A
For nonzero vector v: Av = λvFind the eigenvalues and eigenvectors:
1 Find the characteristic polynomials p (λ) = det (A− λI )2 Find the roots of p (λ), i.e., eigenvalues p (λ) = 0 λ1, . . . , λn
3 For each λj , find the eigenvectors by solving the homogeneous system(A− λj I ) v = 0.
Diagonalize a matrix A is diagonalizable iff A has n linearly independenteigenvectors v1, . . . , vn with respect to eigenvalues λ1, . . . , λn (allowingrepetitions). Then,
P = [v1, . . . , vn] , D = diag {λ1, . . . , λn}
such thatAP = PD ⇐⇒ A = PDP−1
Compute An
Find B such that Bm = A
Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 18th, 2019 24 / 24