ENGG2013 Unit 12Everything is related

Feb, 2011.

Yesterday

• Row-rank of an mn matrix.– The row rank of a matrix is the maximal number

of linearly independent rows.– Row-rank is an integer between 0 to m.

• How to use row operations and calculate row-rank by RREF.– Row operations do not change the row-rank– We can see the row-rank of a RREF easily, just

count the number of non-zero rows.

kshum ENGG2013 2

Calculating row-rank from definition

• Let A be a 67 matrix.• Test the linear independence for1.Each of the 6 rows. (6 cases)2.all pairs of rows. (15 cases)3.All triples of rows. (20 cases)4.All combinations of four rows (15 cases)5.All combinations of five rows (6 cases)6.All six rows together. (1 case)kshum ENGG2013 3

Calculating row-rank from definition

• Example: If – any combination of 5 or more rows are linearly

dependent– there is a combination of 4 rows which are linearly

independent,then we say that the row rank is 4.• Example: If

– any combination of 2 or more rows are linearly dependent

– there is a non-zero row,then we say that the row rank is 1.

kshum ENGG2013 4

Linear independence

• Definition: Given a set of k vectors v1, v2, …, vk, each of them has n components. We call them linear independent when

c1v1+c2 v2+ … +c2 vk = 0

is possible only if all coefficients in the linear combination are equal to zero, c1=c2= … =c2=0

Otherwise, they are linear dependent.

kshum ENGG2013 5

The zero vector

Linear equations

• We need to solve a system of linear equations and show that they there is only one solution, namely, the all-zero solution.

• For example, if we want to test whether

are linearly independent, we write down an vector equation c1v1+c2 v2+ c3vk=0, and see is c1=c2=c3=0 is the only choice for the coefficients. If yes, they are linearly independent. If there is some non-zero choice for c1,c2, and c3, then they are linearly dependent.

kshum ENGG2013 6

RREF

• We can use reduced row-echelon form to see if there is any non-trivial solution(“non-trivial” here means non-zero).

kshum ENGG2013 7

1

11

1

If there are free variables,then there are infinitelymany non-zero solutions

Matrix inverse

• If we write the system of linear equations in matrix from Ax=b,

we can solve it by multiplying both sides (from the left) by the inverse of A, provided that the inverse exists.

kshum ENGG2013 8

A missing link from Unit 7

• Definition: Given a square matrix A, if B is an matrix such that BA = I and AB = I, (Here “I” stands for the identity matrix, then we say tht B is the inverse of A, and write A-1=B.

• To check that a matrix B is the inverse of A, we said in Unit 7 that t is sufficient to check either 1. BA = I, or2. AB = I.

kshum ENGG2013 9

Example

• For example, if we want to check that the

inverse of is

we only need to verify that

kshum ENGG2013 10

Is there any justification for this laziness?

Left and right inverse

• To streamline the presentation, we distinguish left-inverse and right-inverse of a matrix.

• The notions of left- and right-inverse apply to rectangular matrix in general.

• Let A be an mn matrix.• Definition:

– A matrix X is called a left-inverse of A if XA = In.

– A matrix Y is called a right-inverse of A if AY = Im.

kshum ENGG2013 11

n n indentity matrix

m m indentity matrix

Left- and Right-inverse forrectangular matrix are complicated• Consider the matrix

• We can check that

for any a and b. Therefore, A has infinitely many right-inverse.

• Also, A does not have any left-inverse.kshum ENGG2013 12

Row- and column-rank

• As an mirror image of row-rank, we define the column-rank of a matrix M by the maximum number of linearly independent columns in M.

(Here, M may be rectangular matrix, not necessarily square matrix.)

• Similarly to the arguments yesterday, we can see that column operations do not change the column-rank.

kshum ENGG2013 13

The transpose operator• The transpose operator interchange rows and columns

of a matrix.• It provides a useful tool for transporting any proof and

calculation of rows into proof and calculation of columns, and vice versa.

• For example,

– The results in p.10 are translated to

The matrix has no right-inverse,

but has infinitely many left-inverses.

kshum ENGG2013 14

Determinant

• From Unit 9, we also know that we can solve system of linear equations by Cramer’s rule.

• We have formulae for computing the inverse using cofactors and adjoint:

kshum ENGG2013 15

• What are the relationship between them– Determinant– Rank– Left-inverse and right-inverse– RREF– Solution(s) to system of

linear equations– Linear independence

kshum ENGG2013 16

A unifying theorem

• Given an nn matrix A, the followings are logically equivalent:

1. A has a left-inverse. (X, s.t. XA=I.)2. The only solution to Ax=0 is x=0.3. A has column-rank n. (Columns of A are linearly independent)

4. A has a right-inverse. (Y, s.t. AY=I.)5. A has row-rank n. (Rows of A are linearly independent)

6. The RREF of A is the nn identity matrix 7. A is a product of elementary matrices.8. The determinant of A is non-zero.

kshum ENGG2013 17

kshum ENGG2013 18

1 2

1. A has a left-inverse. (X, s.t. XA=I.)2. The only solution to Ax=0 is x=0.

Suppose that A has a left-inverse X, such that XA=I.

Suppose that x = [x1 x2 … xn]T is a solution to Ax = 0. Just multiply both side by X from the left.

kshum ENGG2013 19

2 3

2. The only solution to Ax=0 is x=0.3. A has column-rank n. (Columns of A are linearly

independent)

Suppose that Ax= 0 implies x=0.Let the columns of A be c1, c2,…, cn.

Suppose that x1c1+x2c2+…+xn cn = 0.

But it’s equivalent to writing Ax=0.Hence, x1 =x2 =… =xn =0.

Columns of A are linearly independent.kshum ENGG2013 20

4 5

4. A has a right-inverse. (Y, s.t. AY=I.)5. A has row-rank n. (Rows of A are linearly independent)

The reason is exactly the same as 1 2.

kshum ENGG2013 21

5 65. A has row-rank n. (Rows of A are linearly

independent)6. The RREF of A is the nn identity matrixSuppose that the n rows of A are linearly independent.From Unit 11, we know that row operations does not

affect row-rank. If the RREF of A cannot contain n-1 or less non-zero rows,

then the rank of the RREF is n-1 or less, implying that A has rank n-1 or less.

Therefore all rows of the RREF are not zero. The only choice is the identity matrix.

kshum ENGG2013 22

6 7

6. The RREF of A is the nn identity matrix 7. A is a product of elementary matrices.Definition: An elementary matrix is what you get after applying

an elementary row operation to an identity matrix.

kshum ENGG2013 23

Exchange row 1and row 2

Add 3 times row 1to row 3

Compare with the fact that every positive integer can be factorized as a product of primes.

The elementary matrices are the building blocks of invertible matrices in the same sense

Exchange row 1and row 2

Elementary matrices

• Definition: An elementary matrix is what you get after applying an elementary row operation to an identity matrix.

kshum ENGG2013 24

Add 3 times row 1to row 3

Elementary matrices

Row operations are “undo-able”

• We can undo any row operation by another row operation, without loss of information.– First kind: To undo exchanging row i and row j, we

exchange row i and row j again.– Second kind: The reverse process of multiplying row

i by a non-zero factor k, is to multiply the same row by 1/k.

– Third kind: The reverse process of adding c times row j to row i, is to subtract c times row j from row i.

kshum ENGG2013 25

The same ideas in terms of matrix

• Given any elementary matrix, say E, we can find another elementary matrix, say E’, such that E’ E = In, the nn identity matrix.

• As we have shown in Unit 7, elementary row operation is the same as multiplying a suitable elementary matrix from the left.

kshum ENGG2013 26

From Unit 7

• If we can row reduce matrix A to the identity matrix, then we can multiply a series of elementary matrices, say E1, E2, E3, …, Ep, from the left and obtain the identify matrix.

• For each E1, E2, E3, …, Ep, we can find elementary matrix Ei’ such that Ei’ E= I. (i=1,2,…,p)

• Therefore A can be written as a product of elementary matrices

kshum ENGG2013 27

7 1

7. A is a product of elementary matrices.8. A has a left-inverse. (X, s.t. XA=I.)Suppose that A can be written as a product of

elementary matrices A = E1 E2 E3 Ek-1 Ek .

For each elementary Ei, we can find Ei’ such that

Ej’ Ej = I, (for j = 1,2,…,k)

We can take X as Ek’ Ek-1

’ E2’ E1

’

kshum ENGG2013 28

7 4

7. A is a product of elementary matrices.8. A has a right-inverse. (Y, s.t. AY = I.)Suppose that A can be written as a product of

elementary matrices A = E1 E2 E3 Ek-1 Ek .

For each elementary Ei, we can find Ei’’

such that

Ej Ej’’

= I, (for j = 1,2,…,k)

We can take Y as Ek’’ Ek-1

’’ E2’’ E1

’’

kshum ENGG2013 29

The Logical Road map so far

• 4 5 6 7 1 2 3

kshum ENGG2013 30

3 7

3. A has column-rank n. (Columns of A are linearly independent)

7. A is a product of elementary matrices.Apply the arguments in as in 5 6 7, to the

transpose of A.5’ AT has row-rank n. (Transpose interchange rows and

column)

6’ The RREF of AT is the nn identity matrix 7’ AT is a product of elementary matrices.kshum ENGG2013 31

3 7

3. A has column-rank n. (Columns of A are linearly independent)

7. A is a product of elementary matrices.• Suppose that AT is the product of some elementary

matrices, say E1, E2, … , Ek, AT = E1 E2 E3 Ek.Take the transpose of both sides, and use the fact that

– the transpose of an elementary matrix is also elementary, and

– (E F)T = FT ET

A = EkT Ek-1

T Ek-2T

E1T.

kshum ENGG2013 32

Logical road map so far

• 4 5 6 7 1 2 3

kshum ENGG2013 33

7 8

7. A is a product of elementary matrices.8. The determinant of A is non-zero.We need the following observation:• Determinant of elementary matrix is nonzero• Let E be an elementary matrix and B is any

matrix of the same size as E, thendet(EB) = det(E) det(B)

kshum ENGG2013 34

Elementary matrix (first kind)

• First kind: Row exchange. determinant of a matrix E corresponding to row exchange is -1.

• Suppose we apply a row exchange to a matrix B, and the resulting matrix is say B’.

Then, det(B) = det(B’). det(EB) = det(B’)= - det(B) = det(E)det(B)

kshum ENGG2013 35

Elementary matrix (second kind)

• Second kind: Multiply a row by a non-zero constant k. The determinant of matrix E corresponding this row multiplication is k.

• Suppose we apply multiply the i-th row of matrix C to obtain matrix C’. Then, det(C’) = k det(C).

det(EC) = det(C’)= k det(B) = det(E)det(B)

kshum ENGG2013 36

Elementary matrix (third kind)

• Third kind: Add c times the i-th row to the j-th row. The determinant of matrix E corresponding to this action is 1.

• Suppose we add c times i-th row of matrix D to the j-th row. Let the resulting matrix be D’. Then, det(D’) = det(D).

det(ED) = det(D’)= det(D) = det(E) det(B)

kshum ENGG2013 37

7 8

7. A is a product of elementary matrices.8. The determinant of A is non-zero.If A can be factor as a product of elementary

matrices, i.e., A= E1 E2 Ep

Then det(A) = det(E1 E2 Ep)= det(E1 (E2 E3 Ep)) = det(E1)det(E2 E3 Ep)= det(E1) det(E2) det(E3 Ep)= … = det(E1) det(E2) det(Ep) This is non-zero because each factor is non-zero.kshum ENGG2013 38

Logical road map so far

• 4 5 6 7 1 2 3

• Need to prove 8 any one from 1 to 7• Let’s prove the implication 8 5

kshum ENGG2013 39

8

8 5

• Logically, it is equivalent to proving the negation of (5) implies the negation of (8).

~(5): A has row-rank < n. (Rows of A are linearly dependent)

~(8): The determinant of A is zero.Suppose that the rows of A are linearly

dependent. From Unit 11, we know that one row is a linear combination of the other rows.

kshum ENGG2013 40

8 5

• Suppose that the 1-st row is a linear combination of the other rows

• r1 = c2r2 + c3r3 + c4r4 + … + cnrn

where c2 to cn are some constants.

kshum ENGG2013 41

By row reduction

QED

• 4 5 6 7 1 2 3

• Implications include:– Left-inverse is automatically the same as the right-

inverse.– Row-rank = n if and only column-rank = n– And much more…

kshum ENGG2013 42

8

A question from tutorial 2

• Can we say that if the determinant is zero, then the row vectors are linearly dependent?

• The answer is yes. This is a corollary of the unifying theorem in p.17.

kshum ENGG2013 43

TheoremThree vectors are co-planar if and only if the determinant obtained by writing the three vectors together is zero.

Proof: Let the three vectors be

Assume that they lie on the same plane.Let this plane ax + by + cz = 0.

(a, b, and c are constants, not all zero)

kshum ENGG2013 44

Proof ()•

• The system of linear equations (a,b, and c are the variables)

has non zero solution.• because condition 2 implies the condition 8 in

theorem in p.17, we can conclude that

kshum ENGG2013 45

a,b,c are constantsnot all zero

Proof ()

• For the reverse direction, suppose that

• From the unifying theorem, we know that condition 8 implies condition 2, i.e., if the above determinant is zero, then

has a solution (a,b,c) which is not all zero. • Therefore, it is guaranteed that we can find three real

numbers, a, b, c, not all zero, such that all three points lie on the plane ax+by+cz =0.

kshum ENGG2013 46

QED