2E1 LA Notes 1

8/12/2019 2E1 LA Notes 1

1/23

2E1: Linear Algebra|Lecture Notes 1 Matrices and matrix algebra

1 Matrices and determinants

1.1 Matrices

Definition:An mn matrix is a rectangular array of numbers

(m rows and n columns) enclosed in brackets. The numbers

are called theelements of the matrix.

Examples:

(i)A23matrix has 2 rows and 3 columns:

A=

1 2 3

5 6 7

(ii)Heres a33squarematrix:

A= 1 2 35 6 7

8 9 1 0

(iii) Column vectorsare matrices with only one column:

b=

1

5

8

(iv) Row vectorsare matrices which only have one row:

b= (1 2 3) .

Unless specifically stated otherwise, we will assume that vec-

tors are column vectors.

1.1

8/12/2019 2E1 LA Notes 1

2/23


A general real matrix, A Rmn withm nelements is of

the form

A=

a11 a12 a13 . . . a1na21 a22 a23 . . . a2na31 a32 a33 . . . a2n

... ...

... . . . ...

am1 am2 am3 . . . amn

(1)

We refer to the elements via double indices as follows

(i)The first index represents the row.

(ii)The second index represents the column.

Example

a32is the element in row 3, column 2 of the matrix A.

Notation/Conventions:

Use lowercase boldface (or underlined) letters for vectors

a b c (or a, b, c)

Use uppercase boldface (or underlined) letters for matrices

A B C (or A, B, C )

Refer to the respective elements by lowercase letters with the

appropriate number of indices e.g.

bi is a vector element

aij is a matrix element

1.2

8/12/2019 2E1 LA Notes 1

3/23


1.2 Special matrices

The unit matrix, I, is a square matrix whose only non-zeroelements are on the diagonal and are equal to one, e.g.

I=

1 0 0

0 1 0

0 0 1

, I=

1 0 0 . . . 0 0

0 1 0 . . . 0 0

0 0 1 . . . 0 0...

... ... . . .

... ...

0 0 0 . . . 1 00 0 0 . . . 0 1

All elements of thezero matrix, 0, are equal to zero, e.g.

0= 0 0 0

0 0 0

0 0 0 , 0=

0 0 0 . . . 0 0

0 0 0 . . . 0 0

0 0 0 . . . 0 0

... ... ... . . . ... ...0 0 0 . . . 0 0

0 0 0 . . . 0 0

A diagonal matrix only has non-zero elements on the main

diagonal. These non-zero elements can have any value, e.g.

D =

d11 0 00 d22 0

0 0 d33

, D =

d11 0 . . . 0 0

0 d22 . . . 0 0...

... . . . ...

...

0 0 . . . dn1,n1 0

0 0 . . . 0 dnn

are square diagonal matrices.

1.3

8/12/2019 2E1 LA Notes 1

4/23


1.3 Matrix algebra

1.3.1 Matrix equality

Two matrices areequalif they have the same size and if their

corresponding elements are identical , i.e.

A= B

if and only if

aij =bij fori = 1,...,m; j = 1,...,n

1.3.2 Matrix addition

Two matrices can only be added if they have the same size.

The result is another matrix of the same size.

We add matrices by adding their corresponding elements, i.e.

A= B + C

is obtained (element-wise) via

aij =bij+cij fori = 1,...,m; j = 1,...,n

Example

A= 1 2 3

5 6 7

B=

10 1 235 16 3

A + B=

1 + 10 2 + 1 3 + 23

5 + 5 6 + 16 7 + 3

=

11 3 26

10 22 10

1.4

8/12/2019 2E1 LA Notes 1

5/23


1.3.3 Multiplication of a matrix by a scalar

A matrix is multiplied by a scalar (= a number) by multiplyingeach element of the matrix by that scalar.

The result is a matrix of the same size.

Hence

A= B

is given (element-wise) by

aij = bij fori = 1,...,m; j = 1,...,n

Example

A=

1 2 3

5 6 7

3A

= 31 32 33

35 36 37 = 3 6 9

15 18 21 1.3.4 Matrix Vector multiplication

Before defining the product of a matrix and a vector, let us

recall the notion of thedot product(or,scalar product) of two

vectors.

Let a Rn and b Rn be two column vectors with n real

elements each:

a=

a1a2a3

...

an

, b=

b1b2b3

...

bn

.

1.5

8/12/2019 2E1 LA Notes 1

6/23


Then thedot product(or,scalar product) ofaandbis defined

as

a b= a1b1+a2b2+a3b3+ +anbn.

Note that the row vector corresponding to the (column) vector

acan be defined by thevector transposeoperation:

a

T

=

a1a2

a3...

an

T

= a1 a2 a3 . . . an .

Then the product of the row vector aT and the column vector

bcan be defined as follows:

aT b def= a b= a1b1+a2b2+a3b3+ +anbn (2)

Example

a =

2

1

3

, b =

5

2

4

; aT b =

2 1 3

5

2

4

=

21

3

52

4

= 10212 =4.

1.6

8/12/2019 2E1 LA Notes 1

7/23


Now we can define a matrix vector multiplication.

The product of an mn matrix Aand a column vector xoflengthn produces a column vector bof lengthm:

Amn

xn1

= bm1

.

This implies that thenumber of columns of the matrixmust

be equal to thenumber of rows in the column vector!

The elements of the vectorb are calculated by taking

dot products of the rows of the matrix with the column vector:

if rows ofA Rmn are vectors aT1 , aT2 ,. . ., a

Tm R

n then

A=

aT1

aT2...

aTm

A x=

aT1 x

aT2 x...

aTm x

=b (3)

whereaTi xare dot products defined in equation (2).

Element wise this isn

j=1

aijxj =bi fori = 1,...,m.

1.7

8/12/2019 2E1 LA Notes 1

8/23

8/12/2019 2E1 LA Notes 1

9/23


1.3.5 Matrix Vector multiplication using Falks scheme

For practical computations useFalks schemeand evaluate

Ax= b

in tabular form as follows:

x

A b

Example

A=

1 2 3

5 6 7

, x=

32

4

Evaluate Ax as follows:

32

4

1 2 3

5 6 7

Now place the elements of the resulting vector at the inter-

section of the column vector and the rows of the matrix:3

2

4

1 2 3

5 6 7

13 + 22 + 34

53 + 62 + 74 =

19

55

1.9

8/12/2019 2E1 LA Notes 1

10/23


1.3.6 Matrix Matrix multiplication

The product of an mn matrix A and an np matrix Bproduces anmpmatrix C, i.e.

Amn

Bnp

= Cmp

.

This implies that thenumber of columns of the first matrix

must be equal to thenumber of rows in the second matrix.

Here are some examples:

A=

1 2 3

5 6 7

B=

1 2 35 6 7

8 9 1 0

23 33

C=

1 2 3 25 6 7 2

8 9 1 0 7

D=

1 2

5 6

8 9

3 7

34 42

We can form

AB the result is a23matrix

AC the result is a24matrix

CD the result is a32matrix

It isnotpossible to form

BA or AD

1.10

8/12/2019 2E1 LA Notes 1

11/23


The product matrix of two matrices is obtained by taking dot

products of the rows of the left matrix with the columns of the

right matrix.

If the rows ofA Rmn are the vectors aT1 , aT2 ,. . ., a

Tm R

n

and the columns ofB Rnp are b1, b2,. . ., bp Rn. Let

A=

aT1

a

T

2...

aTm

and B= b1 b2 . . . bp

then the matrix matrix product is

AB=

aT1 b1 aT1 b2 . . . a

T1 bp

aT2 b1 aT2 b2 . . . aT2 bp...

... . . . ...

aTm b1 aTm b2 . . . a

Tm bp

Rmp (4)

Element-wise this is

nj=1

aijbjk =cik fori = 1,...,m; k= 1,...,p.

1.11

8/12/2019 2E1 LA Notes 1

12/23


Example

A= 1 23 4

B= 4 3 2

2 1 1

The respective row and column matrices are

aT1 = (1 2)

aT2 = (3 4)b1=

4

2

, b2=

3

1

, b3=

2

1

,

the matrix-matrix product is then

AB=

(1 2)

4

2

(1 2)

3

1

(1 2)

2

1

(3 4)

4

2

(3 4)

3

1

(3 4)

2

1

which can be evaluated by a series of dot products. That is

AB=

14 + 22 13 + 21 12 + 21

34 + 42 33 + 41 32 + 41

Giving the final result

C= 8 5 4

20 13 10

1.12

8/12/2019 2E1 LA Notes 1

13/23


1.3.7 Matrix Matrix multiplication using Falks scheme

For practical computation we again use Falks scheme and

evaluate AB= C in tabular form as follows:

B

A C

Example

A= 1 2

3 4

, B=

4 3 22 1 1

Evaluate AB as follows

4 3 2

2 1 1

1 2 (14 + 22) (13 + 21) (12 + 21)

3 4 (34 + 42) (33 + 41) (32 + 41)

Place the elements at the intersection of the rows of the left

matrix and the columns of the right matrix

The result

4 3 2

2 1 1

1 2 ( 14 + 22) (13 + 21) (12 + 21)

3 4 ( 34 + 42) (33 + 41) (32 + 41)

therefore gives

AB=

8 5 4

20 13 10

1.13

8/12/2019 2E1 LA Notes 1

14/23


1.3.8 Differences from multiplication with numbers

(i)Matrix multiplication isnot commutative

AB= BA

Example

1 2

4 1

2 2

3 1

=

12 + 23 12 + 21

42 + 13 42 + 11

=

8 4

11 9

2 2

3 1

1 2

4 1

=

21 + 24 22 + 21

31 + 14 32 + 11

= 10 67 7

We must be careful how we multiply out!

(ii) AB= 0 does not imply A= 0, B= 0 or BA= 0

Example 1 1

2 2

1 1

1 1

=

0 0

0 0

1 1

1 1

1 1

2 2

=

1 1

1 1

1.14

8/12/2019 2E1 LA Notes 1

15/23


(iii) AC= ADdoes not necessarily imply C= D

Example 1 1

2 2

2 1

2 2

=

4 3

8 6

1 1

2 2

3 0

1 3

=

4 3

8 6

(iv) BUT other properties are similar to numbers

A(B + C) = AB + AC distributive law

A(BC) = (AB)C associative law

1.15

8/12/2019 2E1 LA Notes 1

16/23


1.4 Transpose of a matrix

The transpose of a matrix is obtained by interchanging itsrows and columns

aTij =aji fori = 1,...,m; j = 1,...,n

The transpose is denoted by a superscript T and the general

matrix given in equation (1) becomes

AT =

a11 a21 a31 . . . am1a12 a22 a32 . . . am2a13 a23 a33 . . . am3

... ...

... . . . ...

a1n a2n a3n . . . amn

Example

A= 1 2 3

5 6 7

A

T = 1 52 6

3 7

IfA= AT then A is asymmetric matrix, e.g.

A=

3 2 1

2 7 01 0 8

The matrix transpose also satisfies the followingrules:

i)(AT)T = A for any matrix A;

ii)(A + B)T = AT + BT and (AB)T = BTAT,

provided that matrices A and B have compatible dimensions.

1.16

8/12/2019 2E1 LA Notes 1

17/23


< Proof of the identity (AB)T = BTAT

If the rows ofA Rmn are the vectors aT1 , aT2 ,. . ., aTm Rnand the columns ofB Rnp are b1, b2,. . ., bp R

n.

Taking the transpose of equation (4)

(AB)T =

aT1 b1 a

T2 b1 . . . a

Tm b1

aT1 b2 aT2 b2 . . . a

Tm bp

.

.. .

.. .

. . .

..aT1 bp a

T2 bp . . . a

Tm bp

For column vectors ai and bj vector multiplication a

Ti bj is

defined as the dot product betweenaiand bj. The dot product

is commutative so

aTi bj =ai bj =bj aj =bT

j ai

which implies that

(AB)T =

bT1 a1 bT1 a2 . . . b

T1 am

bT2 a1 bT2 a2 . . . b

T2 am

... ... . . .

...

bTp a1 bT

p a2 . . . bT

p am

= BTAT

since

BT =

bT1bT2

...

bTp

and AT =

a1 a2 . . . am

. >

Here and further in these notes, the material between the

markers < and > is for advanced reading.1.17

8/12/2019 2E1 LA Notes 1

18/23


1.5 Determinant of a matrix

The determinant of a22matrix

A=

a11 a12a21 a22

is writtendet A or|A|or

a11 a12a21 a22

=

a11a22a12a21

Example

A=

1 2

4 7

, det A= 78 =15.

The determinant of a33matrix is written as

|A|=

a11 a12 a13a21 a22 a23a31 a32 a33

=a11

a22 a23a32 a33a12

a21 a23a31 a33 +a13

a21 a22a31 a32

=a11(a22a33a32a23)a12(a21a33a31a23)+ a13(a21a32a31a22)

That is the33 determinant is defined in terms of determi-

nants of2 2sub-matrices ofA. These are called theminors

ofA.

1.18

8/12/2019 2E1 LA Notes 1

19/23


Example

m12=

a11 a12 a13a21 a22 a23a31 a32 a33

= a21 a23a31 a33

is obtained by suppressing the elements in row 1 and column

2 of matrix A.

Cofactors

Thecofactorcij is defined as the coefficient ofaij in the de-

terminantA. If is given by the formula

cij = (1)i+jmij

where theminoris the determinant of order(n 1) (n 1)

formed by deleting the column and row containing aij.Examples

c11= (1)1+1m11= +1

a11 a12 a13a21 a22 a23a31 a32 a33

=a22a33a32a23

c23= (1)2+3m23=1

a11 a12 a13a21 a22 a23a31 a32 a33

=a11a32+ a31a12

1.19

8/12/2019 2E1 LA Notes 1

20/23


General determinant

The value of annndeterminant equals the sum of the prod-ucts of the elements in any row (or column) and their cofac-

tors, i.e.

|A|=n

j=1

aijcij, for i= 1, . . . , n1, orn

or

|A|=

ni=1

aijcij, for j = 1, . . . , n1, orn

Example

For a33matrix

det A=a11c11+a12c12+a13c13 (1st row)

or

det A= a12c12+a22c22+a32c32 (2nd column)

Points to note:

the determinantdet A is equal to zero if

(i)rows or columns ofA are multiples of each other,

(ii)rows or columns are linear combinations of each other,

(iii)entire rows or columns are zero;

ifdet A= 0the matrix A is called a singular matrix;

for any square matrices A and B there holds

det A= det(AT), det(AB) = det(A)det(B).

for the unit matrix I one hasdet I= 1.

1.20

8/12/2019 2E1 LA Notes 1

21/23


1.6 The matrix inverse

The inversea1

of a scalar (=a number)a is defined by

a a1 = 1.

For square matrices we use a similar definition: the inverse

A1 of annmatrix A fulfils the relation

AA1 = I

where Iis thenn unit matrix defined earlier.

Note: ifA1 exists then

det(A)det(A1) = det(AA1) = det I= 1.

Hence,det(A1) = (det A)1.

Example

The inverse of

A= 3 27 5 is given by

B= A1 =

5 2

7 3

since

AB=

5 2

7 33 2 ( 3527) (32 + 23)

7 5 ( 7557) (72 + 53)

which gives

AB=

1 0

0 1

= I

as required.

1.21

8/12/2019 2E1 LA Notes 1

22/23


The matrix inverse can be computed as follows

1.Find the determinantdet A

2.Find the cofactors of all elements in A and form a new

matrix Cof cofactors, where each element is replaced by

its cofactor.

3.The inverse ofA is now given as

A1

= C

T

det A

Note: the inverse A1 exists if (and only if)det A= 0.

ExampleFind the inverse of

A=

1 1 23 1 23 2 1

.

det A= 1

1 2

2 1

(1)

3 2

3 1

+ 2

3 1

3 2

= 13 + 1(3) + 23

= 6

Since the determinant is nonzero an inverse exists.

1.22

8/12/2019 2E1 LA Notes 1

23/23


Calculate the matrix of minors

M=

1 22 1 3 23 1 3 13 21 22 11 23 1

1 13 2

1 2

1 2

1 2

3 2

1 1

3 1

= 3 3 35 7 14 8 2

Modify the signs according to whetheri+j is even or odd to

calculate the matrix of cofactors

C=

3 3 3

5 7 14 8 2 .

It follows that

A1 =

1

6C

T =1

6

3 5 43 7 8

3 1 2

.

To check that we have made no mistake we can compute

A1

A=1

6

3 5 43 7 8

3 1 2

1 1 23 1 2

3 2 1

=

1 0 00 1 0

0 0 1

.

This way of computing the inverse is only useful for hand

calculations in the cases of22or33matrices.

1.23

Documents

2E1 LA Notes 1