[email protected] PDL, RD, IS (CoE) WS 2016 1/ 52 · Engineering Analysis 1 : Linear Algebra Dr. Paul D. Ledger, Dr Rob Daniels, Dr Igor Sazonov [email protected] College

Applications Simultaneous Equations Gauss Elimination Matrices Determinates Eigenvalue Problems

Engineering Analysis 1 : Linear Algebra

Dr. Paul D. Ledger, Dr Rob Daniels, Dr Igor Sazonov

[email protected]

College of Engineering, Swansea University, UK

PDL, RD, IS (CoE) WS 2016 1/ 52


Outline

1 Applications

2 Simultaneous Equations

3 Gauss Elimination

4 Matrices

5 Determinates

6 Eigenvalue Problems

PDL, RD, IS (CoE) WS 2016 2/ 52


Applications

Being able to work with and solve systems of linear equations is a key skill required forengineers and appears in all areas of engineering and beyond, e.g.

Traffic Modeling Truss Analysis Electrical Circuits

Pipe Networks Flow Simulation Internet search engines

PDL, RD, IS (CoE) WS 2016 3/ 52


Outline

1 Applications


3 Gauss Elimination

4 Matrices

5 Determinates


PDL, RD, IS (CoE) WS 2016 4/ 52


What is a linear equation?

The simplest linear equation is that between two variables that gives a straight linewhen plotted on a graph

y = kx + c

Here k = 3 is the gradient and theintercept on the y-axis is c = 2. x

y

y = 3x+ 2

2

−

2

3

If k 6= 0 then x = (y � c)/k is the “solution” to y = kx + c given y.

PDL, RD, IS (CoE) WS 2016 5/ 52


Simultaneous Equations

Given the equations of two lines with gradients k and ` and intercepts c and d:

y = kx + c

y = `x + d

or y � kx = c

y � `x = d

one might to find the point (x, y) (if it exists) where the lines intersect.

This problem is an example of solving a set of 2 linear (simultaneous) equations and2 unknowns.

Importantly, our unknowns need not be x and y, but can be other letters, symbols orwith indices appropriate to the practical problem at hand.

The number of equations and unknowns in a set of simultaneous equations can bemore than 2 and need not be equal in general!

PDL, RD, IS (CoE) WS 2016 6/ 52


Examples

Example

Solve

x + 2y = 5

2x + 3y = 8

Solution

Two equations and two unknowns. Solution is x = 1 and y = 2.Note that in general a linear equation system may have m equations and n unknowns.Example

Solve

x + y = 4

2x + 2y = 5

Solution

Here m = n = 2 No solution since 2(x + y) = 8 and 2x + 2y = 5

PDL, RD, IS (CoE) WS 2016 7/ 52


Examples

Example

Solve

x + 2y = 5

2x + 3y = 8

Solution


Solve

x + y = 4

2x + 2y = 5

Solution


PDL, RD, IS (CoE) WS 2016 7/ 52


Examples

Example

Solve

x + 2y = 5

2x + 3y = 8

Solution


Solve

x + y = 4

2x + 2y = 5

Solution


PDL, RD, IS (CoE) WS 2016 7/ 52


Examples

Example

Solve

x + 2y = 5

2x + 3y = 8

Solution


Solve

x + y = 4

2x + 2y = 5

Solution


PDL, RD, IS (CoE) WS 2016 7/ 52


Examples

Example

Solve

u � v + w = 2

2u + v � w = 4

Solution

A possible solution is u = 2, v = 0 and w = 0 another is u = 2, v = 1 and w = 1. Ingeneral there are infinitely many solutions, namely u = 2, v = ↵ and w = ↵ where ↵ isany real number.Example

Solve

x

1

+ x

2

= 2

x

1

� x

2

= 1

x

1

= 4

Solution

Here m = 3 and n = 2. No solution since x

1

= 3/2 and x

1

= 4.

PDL, RD, IS (CoE) WS 2016 8/ 52


Examples

Example

Solve

u � v + w = 2

2u + v � w = 4

Solution


Solve

x

1

+ x

2

= 2

x

1

� x

2

= 1

x

1

= 4

Solution


1

= 3/2 and x

1

= 4.

PDL, RD, IS (CoE) WS 2016 8/ 52


Examples

Example

Solve

u � v + w = 2

2u + v � w = 4

Solution


Solve

x

1

+ x

2

= 2

x

1

� x

2

= 1

x

1

= 4

Solution


1

= 3/2 and x

1

= 4.

PDL, RD, IS (CoE) WS 2016 8/ 52


Examples

Example

Solve

u � v + w = 2

2u + v � w = 4

Solution


Solve

x

1

+ x

2

= 2

x

1

� x

2

= 1

x

1

= 4

Solution


1

= 3/2 and x

1

= 4.

PDL, RD, IS (CoE) WS 2016 8/ 52


Outline

1 Applications


3 Gauss Elimination

4 Matrices

5 Determinates


PDL, RD, IS (CoE) WS 2016 9/ 52


Equivalent linear systems

From now one we will label the unknowns as x

1

, x

2

, · · · x

n

for reasons that will shortlybecome clear. Note the following operations:

Exchanging equations

x

1

+ 2x

2

= 5

2x

1

+ 3x

2

= 8

is equivalent to 2x

1

+ 3x

2

= 8

x

1

+ 2x

2

= 5

Both equation systems posses the same solution set.Addition of factored equation to another equation

x

1

+ 2x

2

= 5

2x

1

+ 3x

2

= 8

is equivalent to 1 x

1

+ 2x

2

= 5

�x

2

= �2

To obtain the revised second equation, we must multiplied the first equation by 2 andthen subtracted it from the second equation in the original system. We call the rightupper triangular form, which makes it easier to solve. Both equation systems possesthe same solution set.cyan indicates a row swap, blue indicates a pivot row with pivot highlighted, redindicates a row after elimination.

PDL, RD, IS (CoE) WS 2016 10/ 52


Upper triangular form and back substitution

Example

Solve

3x

1

+ 2x

2

+ x

3

= 1

x

2

� x

3

= 2

2x

3

= 4

Here m = n = 3.Solution

From the third equation we have x

3

= 2. Substituting this in to the second equationgives x

2

= 4 and finally using both values in the first equation gives x

1

= �3.Example

2x

2

+ 2x

3

= 1

2x

1

+ 4x

2

+ 5x

3

= 9

x

1

� x

2

+ 2x

3

= 3

Here m = n = 3.

PDL, RD, IS (CoE) WS 2016 11/ 52



Example

Solve

3x

1

+ 2x

2

+ x

3

= 1

x

2

� x

3

= 2

2x

3

= 4



3


2


1

= �3.Example

2x

2

+ 2x

3

= 1

2x

1

+ 4x

2

+ 5x

3

= 9

x

1

� x

2

+ 2x

3

= 3

Here m = n = 3.

PDL, RD, IS (CoE) WS 2016 11/ 52



Example

Solve

3x

1

+ 2x

2

+ x

3

= 1

x

2

� x

3

= 2

2x

3

= 4



3


2


1

= �3.Example

2x

2

+ 2x

3

= 1

2x

1

+ 4x

2

+ 5x

3

= 9

x

1

� x

2

+ 2x

3

= 3

Here m = n = 3.

PDL, RD, IS (CoE) WS 2016 11/ 52


Upper triangular form and back substitutionSolution

As the coefficient of x

1

is zero in the first equations (it is not a valid pivot) so weexchange the first and second equations

2x

1

+ 4x

2

+ 5x

3

= 9

2x

2

+ 2x

3

= 1

x

1

� x

2

+ 2x

3

= 3

Next we multiply the first equation by 1

2

and subtract it from the third equation

2 x

1

+ 4x

2

+ 5x

3

= 9

2x

2

+ 2x

3

= 1

�3x

2

�1

2

x

3

= �3

2

Multiply the second equation by 3

2

and add it to the third equation, giving

2x

1

+ 4x

2

+ 5x

3

= 9

2 x

2

+ 2x

3

= 1

5

2

x

3

= 0

By back substitution we find the solution x

1

= 7

2

, x

2

= 1

2

and x

3

= 0. It follows that thisis the only solution to the linear system.

PDL, RD, IS (CoE) WS 2016 12/ 52


Gauss elimination

Consider a system where m = n = 3

a

11

x

1

+ a

12

x

2

+ a

13

x

3

= b

1

a

21

x

1

+ a

22

x

2

+ a

23

x

3

= b

2

a

31

x

1

+ a

32

x

2

+ a

13

x

3

= b

3

The values a

ij

are the real coefficients of the unknown x

j

in the ith equation of thesystem. The number b

i

is the right hand side of the ith equation.we express the system in the form

x

1

x

2

x

3

1

a

11

a

12

a

13

b

1

a

21

a

22

a

23

b

2

a

31

a

32

a

33

b

3

The Gauss elimination procedure comprises of two stages: elimination and backsubstitution.

PDL, RD, IS (CoE) WS 2016 13/ 52


ExampleExample

x

1

+ 2x

2

+ 3x

3

+ x

4

= 5

2x

1

+ x

2

+ x

3

+ x

4

= 3

x

1

+ 2x

2

+ x

3

= 4

x

2

+ x

3

+ 2x

4

= 0

Solution

Write the equation in schematic representation and apply Gauss algorithmx

1

x

2

x

3

x

4

1

1 2 3 1 5

2 1 1 1 31 2 1 0 40 1 1 2 0

x

1

x

2

x

3

x

4

1

1 2 3 1 50 -3 -5 -1 -70 0 -2 -1 -10 1 1 2 0

x

1

x

2

x

3

x

4

1

1 2 3 1 5

0 -3 -5 -1 -70 0 -2 -1 -10 0 � 2

3

5

3

� 7

3

x

1

x

2

x

3

x

4

1

1 2 3 1 5

0 -3 -5 -1 -70 0 -2 -1 -10 0 0

6

3

� 6

3

We then determine the solution through back substitution giving x

4

= �1, x

3

= 1,x

2

= 1 and x

1

= 1

PDL, RD, IS (CoE) WS 2016 14/ 52


ExampleExample

x

1

+ 2x

2

+ 3x

3

+ x

4

= 5

2x

1

+ x

2

+ x

3

+ x

4

= 3

x

1

+ 2x

2

+ x

3

= 4

x

2

+ x

3

+ 2x

4

= 0

Solution

Write the equation in schematic representation and apply Gauss algorithmx

1

x

2

x

3

x

4

1

1 2 3 1 5

2 1 1 1 31 2 1 0 40 1 1 2 0

x

1

x

2

x

3

x

4

1

1 2 3 1 50 -3 -5 -1 -70 0 -2 -1 -10 1 1 2 0

x

1

x

2

x

3

x

4

1

1 2 3 1 5

0 -3 -5 -1 -70 0 -2 -1 -10 0 � 2

3

5

3

� 7

3

x

1

x

2

x

3

x

4

1

1 2 3 1 5

0 -3 -5 -1 -70 0 -2 -1 -10 0 0

6

3

� 6

3

We then determine the solution through back substitution giving x

4

= �1, x

3

= 1,x

2

= 1 and x

1

= 1

PDL, RD, IS (CoE) WS 2016 14/ 52


Example

Example

x

1

+ x

2

= 4

2x

1

+ 2x

2

= 5

Solution

First we write the equation in schematic representationx

1

x

2

1

1 1 4

2 2 5Then we proceed with the Gauss elimination algorithm, giving

x

1

x

2

1

1 1 4

0 0 -3Clearly 0x

1

+ 0x

2

6= �3 so the linear system has no solution.

PDL, RD, IS (CoE) WS 2016 15/ 52


Example

Example

x

1

+ x

2

= 4

2x

1

+ 2x

2

= 5

Solution


1

x

2

1

1 1 4

2 2 5Then we proceed with the Gauss elimination algorithm, giving

x

1

x

2

1

1 1 4

0 0 -3Clearly 0x

1

+ 0x

2

6= �3 so the linear system has no solution.

PDL, RD, IS (CoE) WS 2016 15/ 52


Example

Example

Deduce the solution set(s) and how they depend on s for

2x

1

� x

2

+ 3x

3

� x

4

+ x

5

= �2

2x

1

� x

2

+ 3x

3

� x

5

= �3

�4x

1

+ 2x

2

� 4x

3

+ 5x

4

� 5x

5

= 3

�2x

3

+ 2x

4

� 7x

5

= �5 + s

�2x

1

+ x

2

� x

3

+ 4x

5

= 5

Solution


1

x

2

x

3

x

4

x

5

1

2 �1 3 �1 1 -22 -1 3 0 -1 -3-4 2 -4 5 -5 30 0 -2 2 -7 -5+s-2 1 -1 0 4 5

then we proceed with the Gauss elimination algorithm:

PDL, RD, IS (CoE) WS 2016 16/ 52


Example

Example

Deduce the solution set(s) and how they depend on s for

2x

1

� x

2

+ 3x

3

� x

4

+ x

5

= �2

2x

1

� x

2

+ 3x

3

� x

5

= �3

�4x

1

+ 2x

2

� 4x

3

+ 5x

4

� 5x

5

= 3

�2x

3

+ 2x

4

� 7x

5

= �5 + s

�2x

1

+ x

2

� x

3

+ 4x

5

= 5

Solution


1

x

2

x

3

x

4

x

5

1

2 �1 3 �1 1 -22 -1 3 0 -1 -3-4 2 -4 5 -5 30 0 -2 2 -7 -5+s-2 1 -1 0 4 5

then we proceed with the Gauss elimination algorithm:

PDL, RD, IS (CoE) WS 2016 16/ 52


Example continuedx

1

x

2

x

3

x

4

x

5

1

2 �1 3 �1 1 -20 0 0 1 -2 -10 0 2 3 -3 -10 0 -2 2 -7 -5+s0 0 2 -1 5 3

x

1

x

2

x

3

x

4

x

5

1

2 �1 3 �1 1 -20 0 2 3 -3 -10 0 0 1 -2 -10 0 -2 2 -7 -5+s0 0 2 -1 5 3

x

1

x

2

x

3

x

4

x

5

1

2 �1 3 �1 1 -20 0 2 3 -3 -10 0 0 1 -2 -10 0 0 5 -10 -6+s0 0 0 -4 8 4

x

1

x

2

x

3

x

4

x

5

1

2 �1 3 �1 1 -20 0 2 3 -3 -10 0 0 1 -2 -10 0 0 0 0 -1+s0 0 0 0 0 0

For s 6= 1 we have no solution. For s = 1 we find

x

4

= �1 + 2x

5

x

3

=1

2

(�1 + 3x

5

� 3x

4

) = 1 �3

2

x

5

x

1

=1

2

(�2 � x

5

+ x

4

� 3x

3

+ x

2

) = �3 +11

4

x

5

+1

2

x

2

The solution set for s = 1 consists of two free parameters � and ↵ and has infinitelymany solutions x

1

= �3 + 11

4

� + 1

2

↵, x

2

= ↵, x

3

= 1 � 3

2

�, x

4

= �1 + 2�, x

5

= �.PDL, RD, IS (CoE) WS 2016 17/ 52


Linear systems of equations with multiple right hand sides

Example

2x

2

+ 2x

3

= b

1

2x

1

+ 4x

2

+ 5x

3

= b

2

x

1

� x

2

+ 2x

3

= b

3

a)b

1

= 1

b

2

= 9

b

3

= 3

b)b

1

= 2

b

2

= 13

b

3

= 1

c)b

1

= 5

b

2

= �4

b

3

= 2

Solution

x

1

x

2

x

3

1

a

1

b

1

c

0 2 2 1 2 52 4 5 9 13 -41 -1 2 3 1 2

x

1

x

2

x

3

1

a

1

b

1

c

1 -1 2 3 1 22 4 5 9 13 -40 2 2 1 2 5

x

1

x

2

x

3

1

a

1

b

1

c

1 -1 2 3 1 20 6 1 3 11 -80 2 2 1 2 5

x

1

x

2

x

3

1

a

1

b

1

c

1 -1 2 3 1 20 6 1 3 11 -80 0

5

3

0 � 5

3

23

3

By back substitution we obtain the solutions a) x

1

= 7

2

, x

2

= 1

2

, x

3

= 0, b) x

1

= 5,x

2

= 2, x

3

= �1 and c) x

1

= � 279

30

, x

2

= � 63

30

, x

3

= 23

5

.

PDL, RD, IS (CoE) WS 2016 18/ 52


Linear systems of equations with multiple right hand sides

Example

2x

2

+ 2x

3

= b

1

2x

1

+ 4x

2

+ 5x

3

= b

2

x

1

� x

2

+ 2x

3

= b

3

a)b

1

= 1

b

2

= 9

b

3

= 3

b)b

1

= 2

b

2

= 13

b

3

= 1

c)b

1

= 5

b

2

= �4

b

3

= 2

Solution

x

1

x

2

x

3

1

a

1

b

1

c

0 2 2 1 2 52 4 5 9 13 -41 -1 2 3 1 2

x

1

x

2

x

3

1

a

1

b

1

c

1 -1 2 3 1 22 4 5 9 13 -40 2 2 1 2 5

x

1

x

2

x

3

1

a

1

b

1

c

1 -1 2 3 1 20 6 1 3 11 -80 2 2 1 2 5

x

1

x

2

x

3

1

a

1

b

1

c

1 -1 2 3 1 20 6 1 3 11 -80 0

5

3

0 � 5

3

23

3

By back substitution we obtain the solutions a) x

1

= 7

2

, x

2

= 1

2

, x

3

= 0, b) x

1

= 5,x

2

= 2, x

3

= �1 and c) x

1

= � 279

30

, x

2

= � 63

30

, x

3

= 23

5

.

PDL, RD, IS (CoE) WS 2016 18/ 52


Rank of a linear system

The rank of a linear equation system is defined as the number, r, of non–zero rowsafter performing the Gauss elimination algorithm.

Using the rank, the solution set of the linear system of equations can be described: Alinear equation system with m equations and n unknowns has at least one solution if

r = m, orr < m and c

i

= 0, i = r + 1, · · · ,m where c is the right hand side vector afterGauss elimination.

PDL, RD, IS (CoE) WS 2016 19/ 52


Example

Example

x

1

� x

2

+ x

3

= 2

2x

1

+ x

2

� x

3

= 4

Solution

Write the equation in schematic representation and proceed with the Gauss eliminationx

1

x

2

x

3

1

1 �1 1 2

2 1 -1 4

x

1

x

2

x

3

1

1 �1 1 2

0 3 -3 0The rank of this system is r = 2 and we have at least one solution, explicitly

x

2

= x

3

x

1

= 2 + x

2

� x

3

= 2

which means that x

3

is a free parameter. Thus we have infinitely many solutions,x

1

= 2, x

2

= x

3

= ↵ where ↵ is any real number.

PDL, RD, IS (CoE) WS 2016 20/ 52


Example

Example

x

1

� x

2

+ x

3

= 2

2x

1

+ x

2

� x

3

= 4

Solution

Write the equation in schematic representation and proceed with the Gauss eliminationx

1

x

2

x

3

1

1 �1 1 2

2 1 -1 4

x

1

x

2

x

3

1

1 �1 1 2

0 3 -3 0The rank of this system is r = 2 and we have at least one solution, explicitly

x

2

= x

3

x

1

= 2 + x

2

� x

3

= 2

which means that x

3

is a free parameter. Thus we have infinitely many solutions,x

1

= 2, x

2

= x

3

= ↵ where ↵ is any real number.

PDL, RD, IS (CoE) WS 2016 20/ 52


Example

Example

x

1

+ x

2

= 2

x

1

� x

2

= 1

x

1

= 4

Solution


1

x

2

1

1 1 21 -1 11 0 4

Then we proceed with the Gauss elimination algorithm, givingx

1

x

2

1

1 1 20 -2 -10 -1 2

x

1

x

2

1

1 1 20 -2 -10 0 5

2

The rank of this system is r = 2 < m = 3 and c

3

6= 0 so that the system has no solution.

PDL, RD, IS (CoE) WS 2016 21/ 52


Example

Example

x

1

+ x

2

= 2

x

1

� x

2

= 1

x

1

= 4

Solution


1

x

2

1

1 1 21 -1 11 0 4

Then we proceed with the Gauss elimination algorithm, givingx

1

x

2

1

1 1 20 -2 -10 -1 2

x

1

x

2

1

1 1 20 -2 -10 0 5

2

The rank of this system is r = 2 < m = 3 and c

3

6= 0 so that the system has no solution.

PDL, RD, IS (CoE) WS 2016 21/ 52


Outline

1 Applications


3 Gauss Elimination

4 Matrices

5 Determinates


PDL, RD, IS (CoE) WS 2016 22/ 52


Matrix Definitions

Matrix notation is a compact way of expressing systems of linear equations.A m ⇥ n matrix has m rows and n columns. The entries of a matrix are called elements.The element of a matrix A which lies on the ith row and jth column is denoted by a

ij

or(A)

ij

. We write a matrix as follows

A =

0

BBB@

a

11

a

12

· · · a

1n

a

21

a

22

· · · a

2n

......

a

m1

a

m2

· · · a

mn

1

CCCA

For example

A =

✓2 3 1

5 1 2

◆

is a 2 ⇥ 3 matrix. In the first row is the second element (A)12

= a

12

= 3.A n ⇥ n matrix has an equal number of rows and columns and is called a square

matrix.Two matrices are equal if they are the same size and their elements are the same.

PDL, RD, IS (CoE) WS 2016 23/ 52


Some Common Matrices

A n ⇥ n matrix is called a diagonal matrix if (D)ij

= 0 for i 6= j. The elements(D)

ii

= d

ii

are called the diagonal elements. We write0

@5 0 0

0 2 0

0 0 3

1

A = diag(5, 2, 3)

The n ⇥ n matrix I

n

= diag(1, 1, · · · , 1) is called the identity matrix. For example

I

3

=

0

@1 0 0

0 1 0

0 0 1

1

A

The 1-column or n ⇥ 1 matrices are commonly known as column vectors and wewrite them with lower case letters. The elements of column vectors are calledcomponents. Components are only identified with a single index. For example,the 4 ⇥ 1 matrix

b =

0

BB@

b

1

b

2

b

3

b

4

1

CCA =

0

BB@

2

�4

7

0

1

CCA

is a column vector with b

1

= 2, b

2

= �4, b

3

= 7 and b

4

= 0.

PDL, RD, IS (CoE) WS 2016 24/ 52


Connection with Vectors in Geometry/ Physics (similar but not thesame!)

You’ve probably seen vectors before? e.g. ~a =

0

@a

1

a

2

a

3

1

A, here we usually talk about

components a

1

, a

2

, a

3

being with respect to a coordinate system e.g. x, y, z

For such vectors we can perform addition

a+b

x

y

za b

~a +~b =

0

@a

1

a

2

a

3

1

A+

0

@b

1

b

2

b

3

1

A =

0

@a

1

+ b

1

a

2

+ b

2

a

3

+ b

3

1

A

scalar multiplication

2a

x

y

za

↵~a = ↵

0

@a

1

a

2

a

3

1

A =

0

@↵a

1

↵a

2

↵a

3

1

A

and the dot product if |~a| =q

a

2

1

+ a

2

2

+ a

2

3

is the vector’s length or magnitude

θ

a

b

~a ·~b =

0

@a

1

a

2

a

3

1

A ·

0

@b

1

b

2

b

3

1

A = a

1

b

1

+ a

2

b

2

+ a

3

b

3

= |~a||~b| cos ✓

Same operations apply to column vectors but without reference to a coordinate system.PDL, RD, IS (CoE) WS 2016 25/ 52


Addition of matrices

Consider two m ⇥ n matrices A and B. To add the matrices A and B together, we addthe respective elements of A and B together. Written more precisely: the m ⇥ n matrixA + B with (A)

ij

+ (B)ij

is called the sum of matrices A and B

Example

A =

✓3 1 0

2 �2 1

◆B =

✓1 2 0

0 1 1

◆

Find A + B.Solution

A + B =

✓3 1 0

2 �2 1

◆+

✓1 2 0

0 1 1

◆=

✓4 3 0

2 �1 2

◆

PDL, RD, IS (CoE) WS 2016 26/ 52


Addition of matrices

Consider two m ⇥ n matrices A and B. To add the matrices A and B together, we addthe respective elements of A and B together. Written more precisely: the m ⇥ n matrixA + B with (A)

ij

+ (B)ij

is called the sum of matrices A and B

Example

A =

✓3 1 0

2 �2 1

◆B =

✓1 2 0

0 1 1

◆

Find A + B.Solution

A + B =

✓3 1 0

2 �2 1

◆+

✓1 2 0

0 1 1

◆=

✓4 3 0

2 �1 2

◆

PDL, RD, IS (CoE) WS 2016 26/ 52


Matrix Multiplication

Multiplication by a scalar If a m ⇥ n matrix is multiplied by scaler number ↵, thismeans that every element of the matrix is multiplied by ↵.Multiplication of two matrices Let A be an m ⇥ n matrix and B a n ⇥ p matrix. Them ⇥ p matrix AB, with (AB)

ij

=P

n

k=1

(A)ik

(B)kj

is called the matrix product of matrices A

and B.The matrix product AB is only possible if the number of columns of matrix A is exactlythe same as the number of rows of matrix B (due to connection with vector dot

product).

i

m

n

n

p

p

m

B ABA

x =

(AB)i’th row

j’th column

i’th row

j’th column

j

The ith row of matrix A is multiplied by jth of matrix B to obtain the element (AB)ij

ofmatrix AB:

(AB)ij

= a

i1

b

1j

+ a

i2

b

2j

+ · · ·+ a

in

b

nj

PDL, RD, IS (CoE) WS 2016 27/ 52


Example

Example

A =

✓3 1 0

2 �2 1

◆is a 2 ⇥ 3 matrix and B =

0

@1 1 0 0

1 2 2 1

2 �1 �1 2

1

A is a 3 ⇥ 4 matrix

Find AB, if possibleSolution

AB =

✓4 5 2 1

2 �3 �5 0

◆. The two elements in the first column were computed as

follows

(AB)11

=�

3 1 0

�0

@1

1

2

1

A = 3 · 1 + 1 · 1 + 0 · 2 = 4

(AB)21

=�

2 �2 1

�0

@1

1

2

1

A = 2 · 1 + (�2) · 1 + 1 · 2 = 2

PDL, RD, IS (CoE) WS 2016 28/ 52


Example

Example

A =

✓3 1 0

2 �2 1

◆is a 2 ⇥ 3 matrix and B =

0

@1 1 0 0

1 2 2 1

2 �1 �1 2

1

A is a 3 ⇥ 4 matrix

Find AB, if possibleSolution

AB =

✓4 5 2 1

2 �3 �5 0

◆. The two elements in the first column were computed as

follows

(AB)11

=�

3 1 0

�0

@1

1

2

1

A = 3 · 1 + 1 · 1 + 0 · 2 = 4

(AB)21

=�

2 �2 1

�0

@1

1

2

1

A = 2 · 1 + (�2) · 1 + 1 · 2 = 2

PDL, RD, IS (CoE) WS 2016 28/ 52


Matrix Multiplication Properties

For m ⇥ n matrices A and B, the commutative law of addition holds

A + B = B + A

For m ⇥ n matrices A, B and C the associative law of addition holds

(A + B) + C = A + (B + C)

For every m ⇥ n matrix A, n ⇥ p matrix B and p ⇥ q matrix C, the associative law ofmultiplication holds

(AB)C = A(BC)

For m ⇥ n matrices A and B and n ⇥ p matrices C and D, the distributive law ofmultiplication holds

(A + B)C = AC + BC

A(C + D) = AC + AD

Note, however that the commutative law of multiplication does NOT hold formatrices. That is to say that in general for two matrices A and B

AB 6= BA

For every m ⇥ n matrix A, it holds that I

m

A = AI

n

= A. Thus giving the name for theidentity matrix I

m

and I

n

.

PDL, RD, IS (CoE) WS 2016 29/ 52


Example

Example

A =

✓2 6

1 3

◆B =

✓1 4

5 2

◆

Find AB and BA

Solution

AB =

✓32 20

16 10

◆6= BA =

✓6 18

12 36

◆

PDL, RD, IS (CoE) WS 2016 30/ 52


Example

Example

A =

✓2 6

1 3

◆B =

✓1 4

5 2

◆

Find AB and BA

Solution

AB =

✓32 20

16 10

◆6= BA =

✓6 18

12 36

◆

PDL, RD, IS (CoE) WS 2016 30/ 52


Transpose

Let A be a m ⇥ n matrix. Then the n ⇥ m matrix A

T with (AT)ij

= (A)ji

is called thetranspose of A. A matrix is called symmetric if A

T = A holds.The matrix transpose obeys the following rules

For general m ⇥ n matrices A and B, (A + B)T = A

T + B

T holdsFor every m ⇥ n matrix A and every n ⇥ p matrix B, (AB)T = B

T

A

T holds.Example

Determine the transpose of the following matrices

A =

✓1 2 3 4

5 6 7 8

◆B =

0

@2 3 �5

3 �1 2

�5 2 7

1

A

Solution

A

T =

0

BB@

1 5

2 6

3 7

4 8

1

CCA 6= A is NOT symmetric B

T =

0

@2 3 �5

3 �1 2

�5 2 7

1

A = B is symmetric

PDL, RD, IS (CoE) WS 2016 31/ 52


Transpose

Let A be a m ⇥ n matrix. Then the n ⇥ m matrix A

T with (AT)ij

= (A)ji

is called thetranspose of A. A matrix is called symmetric if A

T = A holds.The matrix transpose obeys the following rules

For general m ⇥ n matrices A and B, (A + B)T = A

T + B

T holdsFor every m ⇥ n matrix A and every n ⇥ p matrix B, (AB)T = B

T

A

T holds.Example

Determine the transpose of the following matrices

A =

✓1 2 3 4

5 6 7 8

◆B =

0

@2 3 �5

3 �1 2

�5 2 7

1

A

Solution

A

T =

0

BB@

1 5

2 6

3 7

4 8

1

CCA 6= A is NOT symmetric B

T =

0

@2 3 �5

3 �1 2

�5 2 7

1

A = B is symmetric

PDL, RD, IS (CoE) WS 2016 31/ 52


Matrix notation for linear systems

Considera

11

x

1

+ a

12

x

2

+ · · ·+ a

1n

x

n

= b

1

......

a

m1

x

1

+ a

m2

x

2

+ · · ·+ a

mn

x

n

= b

m

Define the matrix

A =

0

B@

a

11

· · · a

1n

......

a

m1

· · · a

mn

1

CA

and the column vectors

x =

0

B@

x

1

...x

n

1

CA b =

0

B@

b

1

...b

m

1

CA

The matrix A is called the coefficient matrix and b is called the right hand side of thelinear equation system. The equation system is equivalent to

Ax = b

This can be solved using the Gauss elimination algorithmPDL, RD, IS (CoE) WS 2016 32/ 52


Matrix inverse

The matrix inverse only makes sense for square matrices.

The n ⇥ n matrix X is called the inverse of matrix A if AX = I

n

.

If the matrix A has an inverse, the matrix A is called invertible or regular, if the matrixhas no inverse it is called singular.

For a regular n ⇥ n matrix A, we denote its inverse by A

�1.Let A and B be invertible n ⇥ n matrices, then

AA

�1 = A

�1

A = I

n

A

�1 is invertible and (A�1)�1 = A

AB is invertible and (AB)�1 = B

�1

A

�1

A

T is invertible and (AT)�1 = (A�1)T

The solution to a n ⇥ n linear equation system Ax = b can be computed as x = A

�1

b,since A

�1

Ax = I

n

x = A

�1

b .

But it is much faster to use Gaues elimination!

PDL, RD, IS (CoE) WS 2016 33/ 52


Calculation of the Matrix Inverse

For a regular n ⇥ n matrix A : Denote the matrix inverse by X and note that

AX =�

a

(1) · · · a

(n)� �

x

(1) · · · x

(n)�= I

n

=�

b

(1) · · · b

(n)�

where a

(1), · · · , a

(n) are the columns of matrix A, x

(1), · · · , x

(n) are the columns of X

and b

(1), · · · , b

(n) are the columns of I

n

To determine x

(1), · · · , x

(n), we can solve linear systems Ax

(1) = b

(1), · · · ,Ax

(n) = b

(n)

for x

(1), · · · , x

(n). Then, the inverse of A is given A

�1 = X whose columns arex

(1), · · · , x

(n).

Special case if n = 2 then A

�1 = 1

a

11

a

22

�a

12

a

21

✓a

22

�a

12

�a

21

a

11

◆if

A =

✓a

11

a

12

a

21

a

22

◆.

PDL, RD, IS (CoE) WS 2016 34/ 52


ExampleExample

Determine the inverse of the following matrix

A =

0

@0 3 �2

4 �2 1

2 �1 1

1

A

Solution

We follow a similar procedure to that undertaken when solving linear equations withmultiple right hand sides.

x

1

x

2

x

3

1

1

1

2

1

3

0 3 -2 1 0 04 -2 1 0 1 02 -1 1 0 0 1

x

1

x

2

x

3

1

1

1

2

1

3

2 -1 1 0 0 14 -2 1 0 1 00 3 -2 1 0 0

x

1

x

2

x

3

1

1

1

2

1

3

2 -1 1 0 0 10 0 -1 0 1 -20 3 -2 1 0 0

x

1

x

2

x

3

1

1

1

2

1

3

2 -1 1 0 0 10 3 -2 1 0 00 0 -1 0 1 -2

x

(1) =

0

@1

6

1

3

0

1

A, x

(2) =

0

@1

6

� 2

3

�1

1

A, x

(3) =

0

@1

6

4

3

2

1

A, A

�1 =

0

@1

6

1

6

1

6

1

3

� 2

3

4

3

0 �1 2

1

A. Note

that A

�1

A = I

3

PDL, RD, IS (CoE) WS 2016 35/ 52


Rank and linear independence

The rank of a matrix A is the same as the rank of the linear equation system Ax = 0. Itis denoted by r = rank A .

Linear independence of a set of column vectors means that each of the vectors cannot be obtained from multiples of the other vectors.

Consider a sequence of n column vectors a

(1), a

(2), · · · , a

(n) each of length m. We canconstruct a m ⇥ n matrix whose columns are these vectors

A =�

a

(1)a

(2) · · · a

(n)�

We then compute the rank r of the m ⇥ n matrix A :

If r = n the column vectors are linearly independent.If r < n the column vectors are linearly dependent.If r = m the column vectors are called generating.If r = n = m the column vectors are generating and linearly independent and forma basis.

PDL, RD, IS (CoE) WS 2016 36/ 52


Example

Example

Determine whether the following vectors are linearly dependent or not0

@1

1

1

1

A and

0

@0

0

0

1

A

Solution

We form the matrix whose columns are the two vectors

A =

0

@1 0

1 0

1 0

1

A

Next, we perform Gauss elimination on the system Ax = 0

x

1

x

2

1

1

1 0 01 0 01 0 0

x

1

x

2

1

1

1 0 00 0 00 0 0

We observe that r = 1, m = 3 and n = 2. This means that r < n so that the system islinearly dependent and not generating.

PDL, RD, IS (CoE) WS 2016 37/ 52


Example

Example

Determine whether the following vectors are linearly dependent or not0

@1

1

1

1

A and

0

@0

0

0

1

A

Solution

We form the matrix whose columns are the two vectors

A =

0

@1 0

1 0

1 0

1

A

Next, we perform Gauss elimination on the system Ax = 0

x

1

x

2

1

1

1 0 01 0 01 0 0

x

1

x

2

1

1

1 0 00 0 00 0 0

We observe that r = 1, m = 3 and n = 2. This means that r < n so that the system islinearly dependent and not generating.

PDL, RD, IS (CoE) WS 2016 37/ 52


Outline

1 Applications


3 Gauss Elimination

4 Matrices

5 Determinates


PDL, RD, IS (CoE) WS 2016 38/ 52


Definition and Properties

Determinates of square matrices can be used to characterise whether a matrix isregular or singular. It can can also be used to compute certain products of vectors (seeEG190) and calculate volumes.A determinate is a number which can be computed from each square matrix A.It iswritten as det A or |A|:

��a

11

a

12

a

21

a

22

�� = a

11

a

22

� a

12

a

21

��

a

11

a

12

a

13

a

21

a

22

a

23

a

31

a

32

a

33

��= a

11

��a

22

a

23

a

32

a

33

�� a

12

��a

21

a

23

a

31

a

33

��+ a

13

��a

21

a

22

a

31

a

32

��

PDL, RD, IS (CoE) WS 2016 39/ 52


Example

Example

Determine the determinants of the following matrices

A =

✓3 2

1 2

◆B =

0

@1 2 1

2 3 2

4 1 2

1

A

Solution

det A =

��3 2

1 2

�� = 3 · 2 � 2 · 1 = 4

det B =

��

1 2 1

2 3 2

4 1 2

��= 1

��3 2

1 2

�� 2

��2 2

4 2

��+ 1

��2 3

4 1

��= 1 · 4 � 2 · (�4) + 1 · (�10) = 2

PDL, RD, IS (CoE) WS 2016 40/ 52


Example

Example

Determine the determinants of the following matrices

A =

✓3 2

1 2

◆B =

0

@1 2 1

2 3 2

4 1 2

1

A

Solution

det A =

��3 2

1 2

�� = 3 · 2 � 2 · 1 = 4

det B =

��

1 2 1

2 3 2

4 1 2

��= 1

��3 2

1 2

�� 2

��2 2

4 2

��+ 1

��2 3

4 1

��= 1 · 4 � 2 · (�4) + 1 · (�10) = 2

PDL, RD, IS (CoE) WS 2016 40/ 52


Determinates of larger matrices

We first note the propertiesThe determinate of a triangular matrix is equal to the product of the diagonalterms.For every n ⇥ n matrix A, it holds that det A = det A

T

If the n ⇥ n matrix A is invertible then det A 6= 0 and det A

�1 = 1

det A

Gauss elimination can be used to reduce a matrix to to upper triangular form. Thedeterminate is then just the product of the diagonal terms!

But, if we swap rows, we must multiply the final result by �1 for each row swap.

PDL, RD, IS (CoE) WS 2016 41/ 52


Example

Example

Determine the determinant of the following matrix using Gauss elimination

A =

0

@0 3 �2

4 �2 1

2 �1 1

1

A

Solution

det A =

��

0 3 �2

4 �2 1

2 �1 1

��= �

��

2 �1 1

4 �2 1

0 3 �2

��

= �

��

2 �1 1

0 0 �1

0 3 �2

��=

��

2 �1 1

0 3 �2

0 0 �1

��

The determinate is then the product of the diagonal terms det A = 2 · 3 · (�1) = �6

PDL, RD, IS (CoE) WS 2016 42/ 52


Example

Example

Determine the determinant of the following matrix using Gauss elimination

A =

0

@0 3 �2

4 �2 1

2 �1 1

1

A

Solution

det A =

��

0 3 �2

4 �2 1

2 �1 1

��= �

��

2 �1 1

4 �2 1

0 3 �2

��

= �

��

2 �1 1

0 0 �1

0 3 �2

��=

��

2 �1 1

0 3 �2

0 0 �1

��

The determinate is then the product of the diagonal terms det A = 2 · 3 · (�1) = �6

PDL, RD, IS (CoE) WS 2016 42/ 52


Determinates and linear systems

The determinate of a matrix A allow us to say alot about a n ⇥ n linear system. Wesummarise key results below

If det A 6= 0 the homogeneous linear equation system Ax = 0 has only the trivialsolution.If det A = 0 the homogeneous linear equation system Ax = 0 has infinitely manysolutions.If det A 6= 0 the linear equation system Ax = b has for a general right hand sidevector exactly one solution.If det A = 0 the linear equation system Ax = b has no solution or infinitely manysolutions, depending on the right hand side vector.

PDL, RD, IS (CoE) WS 2016 43/ 52


Outline

1 Applications


3 Gauss Elimination

4 Matrices

5 Determinates


PDL, RD, IS (CoE) WS 2016 44/ 52


Eigenvalues and eigenvectors

Consider a n ⇥ n matrix A.The number � is called an eigenvalue of matrix A, if there exists a vector(non-trivial) x such that Ax = �x holds.If � is an eigenvalue of the matrix A, then the vector x, for which Ax = �x holds, iscalled the eigenvector of matrix A corresponding to eigenvalue �.

Noting that � is an eigenvalue of A when there is a vector x 6= 0 such that Ax � �x = 0

holds then

Ax � �I

n

x = 0 or (A � �I

n

)x = 0.

� is therefore the eigenvalue of the matrix A when the homogeneous equation system(A � �I

n

)x = 0 has a non–trivial solution.

It follows that this is exactly the case when det (A � �I

n

) = 0.

PDL, RD, IS (CoE) WS 2016 45/ 52


Example

Example

Determine the eigenvalues of the following matrix

A =

0

@�2 1 0

1 �2 1

0 1 �2

1

A

Solution

A � �I

n

=

0

@�2 � � 1 0

1 �2 � � 1

0 1 �2 � �

1

A

Next, we compute the determinate of this matrix

det (A � �I) = (�2 � �)h(�2 � �)2 � 1

i� 1 [(�2 � �)� 0]

= (�2 � �)h(�2 � �)2 � 2

i= �(2 + �)(�2 + 4�+ 2)

The cubic equation det (A � �I) = 0 has the following roots, �1

= �2, �2

= �2 +p

2

and �3

= �2 �p

2 which are also in turn the eigenvalues of matrix A.

PDL, RD, IS (CoE) WS 2016 46/ 52


Example

Example


A =

0

@�2 1 0

1 �2 1

0 1 �2

1

A

Solution

A � �I

n

=

0

@�2 � � 1 0

1 �2 � � 1

0 1 �2 � �

1

A

Next, we compute the determinate of this matrix

det (A � �I) = (�2 � �)h(�2 � �)2 � 1

i� 1 [(�2 � �)� 0]

= (�2 � �)h(�2 � �)2 � 2

i= �(2 + �)(�2 + 4�+ 2)

The cubic equation det (A � �I) = 0 has the following roots, �1

= �2, �2

= �2 +p

2

and �3

= �2 �p

2 which are also in turn the eigenvalues of matrix A.

PDL, RD, IS (CoE) WS 2016 46/ 52


Algebraic multiplicity

In general for a n ⇥ n matrix A we observe that det (A � �I

n

) is a polynomial of nthdegree in �.

We call the polynomial det (A � �I

n

) the characteristic polynomial of matrix A anddenote it by P

A

(�).

If the polynomial P

A

(�) has a root �⇤ which is repeated k times, we call k the algebraic

multiplicity of eigenvalue �⇤.

PDL, RD, IS (CoE) WS 2016 47/ 52


Example

Example


A =

0

@2 1 1

1 2 1

1 1 2

1

A

Solution

This time we use Gauss elimination to compute det (A � �I

n

)

det (A � �I

n

) =

��

2 � � 1 1

1 2 � � 1

1 1 2 � �

��= �

��

1 1 2 � �1 2 � � 1

2 � � 1 1

��

= �

��

1 1 2 � �0 1 � � �1 + �0 �1 + � 1 � (2 � �)2

��= �

��

1 1 2 � �

0 1 � � �1 + �

0 0 �4 + 5�� 2

��

(1)

Therefore we have P

A

(�) = �(�� 1)(4 � 5�+ �2) = �(�� 1)2(�� 4). The twoeigenvalues are 1 and 4.The eigenvalue � = 1 has algebraic multiplicity 2.The eigenvalue � = 4 has algebraic multiplicity 1.

PDL, RD, IS (CoE) WS 2016 48/ 52


Example

Example


A =

0

@2 1 1

1 2 1

1 1 2

1

A

Solution

This time we use Gauss elimination to compute det (A � �I

n

)

det (A � �I

n

) =

��

2 � � 1 1

1 2 � � 1

1 1 2 � �

��= �

��

1 1 2 � �1 2 � � 1

2 � � 1 1

��

= �

��

1 1 2 � �0 1 � � �1 + �0 �1 + � 1 � (2 � �)2

��= �

��

1 1 2 � �

0 1 � � �1 + �

0 0 �4 + 5�� 2

��

(1)

Therefore we have P

A

(�) = �(�� 1)(4 � 5�+ �2) = �(�� 1)2(�� 4). The twoeigenvalues are 1 and 4.The eigenvalue � = 1 has algebraic multiplicity 2.The eigenvalue � = 4 has algebraic multiplicity 1.

PDL, RD, IS (CoE) WS 2016 48/ 52


Eigenvectors

For every eigenvalue, �, we now wish to compute the non–trivial solution x such that

(A � �I

n

)x = 0

We call this set of nontrivial solutions the eigenspace of A corresponding to

eigenvalue � and is given the symbol E�. The dimension of E� is called thegeometric multiplicity of the eigenvalue �. The geometric multiplicity is alwaysgreater or equal to 1.The span is the set of all linear combinations of a set of vectors which make up theeigenspace.

PDL, RD, IS (CoE) WS 2016 49/ 52


Example

Example

Given the following matrix

A =

0

@�2 1 0

1 �2 1

0 1 �2

1

A

For which we have already found that its eigenvalues are �1

= �2, �2

= �2 +p

2 and�

3

= �2 �p

2, now compute the corresponding eigenspacesSolution

Eigenspace for � = �2. The coefficient matrix in (A � �I

3

)x = 0 is

A + 2I

3

=

0

@0 1 0

1 0 1

0 1 0

1

A

With Gauss elimination we can find the eigenspace E�2

x

1

x

2

x

3

1

0 1 0 01 0 1 00 1 0 0

x

1

x

2

x

3

1

1 0 1 00 1 0 00 1 0 0

x

1

x

2

x

3

1

1 0 1 00 1 0 00 0 0 0

The solution set is {x

3

= ↵, x

2

= 0, x

1

= �↵|↵ 2 R} or

PDL, RD, IS (CoE) WS 2016 50/ 52


Example Continued

E�2

=

8<

:↵

0

@�1

0

1

1

A | ↵ 2 R

9=

; = span

8<

:

0

@�1

0

1

1

A

9=

;

Eigenspace to � = �2 +p

2. Proceeding with Gauss elimination we havex

1

x

2

x

3

1

�p

2 1 0 01 �

p2 1 0

0 1 �p

2 0

x

1

x

2

x

3

1

1 �p

2 1 0�p

2 1 0 00 1 �

p2 0

x

1

x

2

x

3

1

1 �p

2 1 00 �1

p2 0

0 1 �p

2 0

x

1

x

2

x

3

1

1 �p

2 1 00 �1

p2 0

0 0 0 0

The solution has the form x

3

= ↵, x

2

=p

2↵, x

1

= ↵, ↵ 2 R. Thus

E�2+p

2

=

8<

:↵

0

@1p2

1

1

A | ↵ 2 R

9=

; = span

8<

:

0

@1p2

1

1

A

9=

;

PDL, RD, IS (CoE) WS 2016 51/ 52


Example Continued

Eigenspace to � = �2 �p

2. In a similar fashion to the above, we get

E�2�p

2

=

8<

:↵

0

@1

�p

2

1

1

A | ↵ 2 R

9=

; = span

8<

:

0

@1

�p

2

1

1

A

9=

;

PDL, RD, IS (CoE) WS 2016 52/ 52

Documents

[email protected] PDL, RD, IS (CoE) WS 2016 1/ 52 · Engineering Analysis 1 : Linear Algebra Dr. Paul D. Ledger, Dr Rob Daniels, Dr Igor Sazonov [email protected] College