Solving Linear Equations · Reflection Questions Solving a system of algebraic equations is not new to any of us. ... (VTK) such as OpenGL, Catia, computational geometry is used to

Solving Linear Equations

MA295; Module 1: Matrix Algebra Page 1 Unit 2: Solving Systems of Equations Using Matrices This section is partially adapted from the algebra course module of Dr. Elliott Jacobs

Introduction

Applications

Goal and

Objectives

Reflection

Questions

Solving a system of algebraic equations is not new to any of us. In the first

lesson we learned how write a set of system equations to a matrix form.

What is new is how to solve them most efficiently by using matrix

manipulations.

We apply algebraic equations almost everywhere. We will apply system

algebraic equations to find the intersections of two lines, or two to three

planes. The examples demonstrate the connection of algebraic equations

to geometric figures, which are the fundamental ideas of analytic

geometry based on the Cartesian Coordinates.

This lesson is to demonstrate how use Matrix and Gauss Jordon Algorithm

to solve linear system equations either manually or by MATLAB. The

objectives are to learn

1. How to solve a set of system equations by using inverse matrix if

unique solution exists

2. How to follow the Gauss-Jordon Algorithm and use matrix manipulation

to solve general system equations

3. How to interpret to a set of system 2 or 3 equations in geometry

context

4. How to use MATLAB to solve system equations

In all visual software toolkits (VTK) such as OpenGL, Catia, computational

geometry is used to render correct visualization. For example, when a

straight line intersects a plane and the plane blocks the visibility of the

line, the line behind the plane has to be clipped right after the intersecting

point. If you are hired as the programmer to render the visualization, you

may need to answer the following questions.

1. How do you represent the equations of lines and planes (planar

regions) without explicitly naming all the variables?

2. How do you know if a line will intersect the planar region, if it will, how

do you find the solutions of the intersection point?

3. What is OpenGL? (please find it from Google/wiki) Can you find the

right MATLAB commands to answer the two questions above?



Uniquely determined system equations and solution by inverse matrix

System

equations in

matrix form

A well-posed general system equations typically have the same number of

variables and equations in the following form

a11 x1 + a12 x2 + a13 x3 … + a1n xn = b1

a21 x1 + a22 x2 + a23 x3 … + a2n xn = b1

an1 x1 + an2 x2 + an3 x3 … + ann xn = bn

Where aij’s and bi’s are given constants is called a linear system of

algebraic equations in the unknown’s x1, x2, … xn.

In the first lesson, you learned that a system of equations can be

expressed into matrix and vector form concisely as follows

[ ] [ ]

The matrix A is called a coefficient matrix of the system, is called the

variable vector, and the vector is called constant vector of the system.

If matrix exists, left-multiply it on both sides, we have

can write the solutions:

[

] [

]

Find the

original

vector when

the

transformed

vector is

known.

Assume the transformation in 3D space has its transformation

matrix

⌊

⌋

If we know that the transformed vector , use MATLAB to

find the original vector

>> A= [2 6 8; 4 15 19; 2 0 3];

>> b = [16, 38, 6]';

>> x =inv(A) * b

x = 0.0000 0.0000

2.0000

Use Gauss-

Jordon

We will use Gauss-Jordan elimination algorithm to solve the above

system. The basic idea for this formulation is to use the first equation to



Algorithm to

solve system eliminate 1x in all other equations; then use the second equation to

eliminate x2 in all the others; and so on. If all goes well, the resulting

system will be “uncoupled” and the values of the unknowns’ 1 2 nx , x , ... x

will be apparent.

Solve the system

1 2 3

1 2 3

1 3

2x + 6x + 8x = 16

4x + 15x + 19x = 38

2x + + 3x = 6

Step 1: write the augmented matrix for the system

2 6 8 16

4 15 19 38

2 0 3 6

Step 2

(a) Obtain a leading 1 in the first row, first column. (If there is a 0 in this

position, interchange the first row with a row below it so that a non-zero

entry appears there).

Divide the first row by 2

1 3 4 8

4 15 19 38

2 0 3 6

(b) Obtain zeros in the other positions in the first column by adding

appropriate multiples of the first row to the other rows.

Replace the second row of the sum of itself and –4 times the first.

Replace the third row with the sum of itself and –2 times the first.

1 3 4 8

0 3 3 6

0 -6 -5 -10

Step 3

(a): Obtain a leading 1 in the second row, second column (If there is a

zero in this position, interchange the second row and the row below it so

that a nonzero appears there. If this is not possible, go to the next

column).

Multiply the second row by 1/3.



1 3 4 8

0 1 1 2

0 -6 -5 -10

(b): Obtain zeros in the second column by adding appropriate multiples of

the second row to the other rows.

Replace the third row with the sum of itself and 6 times the second.

[

]

Step 4:

Use reverse substitution to solve all three variables one by one.

The third equation shows , substitute to the second equation

,

We have , substitute and to the first equation , we find that .

Hence the solution is that

[

] [ ]

Self-Check

Exercises

1. Solve the following system by Gauss-Jordan method.

x + 2y + 3z = 5

2x + 5y + 3z = 3 x + 8z = 17

2. Use MATLAB to solve the same problem above

Gauss-Jordon Algorithm to solve system equations in matrix form

Example 3

A set of

inconsistent

system

equations

Consider the system equation

We can try to use the MATLAB first,

>> B= [2 1 -1;1 2 1; -1 1 2];

>> b = [3 0 0]'; >> x = inv(B) * b

Warning: Matrix is singular to working precision.

x = NaN



NaN NaN

The MATLAB gives warning message that the matrix B is singular matrix.

Therefore, the inverse does not exist. We may recall that we run into

that problem in last lesson when we discuss singular matrix. If we add

the first equation to the third, we have The equation (2) and (4) contradict each other. Hence, we call the set of

system equations as inconsistent. No solution exists for the system.

Example 4

A set of

under-

determined

system

equations

The only difference between example 3 and 4 is on the constants of the

second equation. Since the matrix

[

]

is singular as we discussed in example 3, the inverse of A does not exist.

Hence, we cannot find unique solution.

If we add the first equation to the third, we have The equation (2) and (4) are identical, which means that the second

equation is redundant to the combination of the first and second.

In such a case, the system actually is short of one equation for obtaining

unique solution. In geometry, if we map each equation to a plane, the

first and second planes intersect to a straight line, which lies right in the

second plane. Hence the system has infinite solutions.

Gauss-Jordon

Algorithm to

solve the

system

equation

Let us use the Gauss-Jordon Algorithm to solve the system equation

Step 1,

Write the system in matrix form

[

]

Step 2:

(a) Swap the first and second rows

[

]

(b) Add the first row to the third row and (-2) multiples of the first row

to the second row.



[

]

(c) multiple (-1/3) to the second row and (1/3) to the third row,

[

]

Step 3

Add (-1) multiple of the second row to the third row

[

]

The third equation is 0 = 0. So the system only has two independent

equations, i.e. one variable is free to take any number t. Since all

elements under the major diagonal are zeros, we can the matrix in such

a form as upper triangular matrix.

Step 4

Let by the second equation , so we have Substitute the first equation , we have

Since t is arbitrary real number, we have infinite many solutions as

follows:

[

] [

] [

] [

]

We can interpret the vector solution as a straight line that passes the

point

(1, 1, 0) and is parallel to the vector [1, -1, 1]’.

Example 5

A set of over-

determined

System

equations

Solve the system

1 2 3

1 2 3

1 3

2x + 6x + 8x = 16

4x + 15x + 19x = 38

2x + + 3x = 6

From example 2, we know that the first three equations yield a unique

solution [0, 0, 2]’.

The fourth equation will either contradict to the first three, or is

redundant. In this case, if we substitute the solution [0, 0, 2]’ to the

fourth equation, it yields 2 = 2.1, which is false. We have more

independent equations than the number of variables. We call such a

system an over determined system.

If we consider the inevitable errors of measurement and mathematical

models in real world problems, an over determined system is most likely



helpful to reduce the errors and gives good approximate solutions. We

will discuss such a system in our next module.

General equations of planes in 3D and interpretation of algebraic equations

Point Normal

Equations

Figure 1

A plane is a two-dimensional surface. It is spanned by two linearly

independent vectors . That is any vector on a plane can be

expressed as .

We call the right hand side of the equation a linear combination of the

vector .

For example, any vector on XY plane are spanned by the two vectors:

[ ] [

] [

]

Normal

Vectors

A normal vector of a plane is a vector that is perpendicular to two linear

independent (not parallel) vectors on the plane.

A normal vector that is also perpendicular to any vector on the plane.

Proof: Since vector on a plane can be expressed as a linear

combination of the two linearly independent vectors , there exist

two constants such that .

Let be a normal vector of the plane,

by definition .

Therefore .



The equation of a plane with nonzero normal vector through

the point is =0, where is an

arbitrary point on the plane (see Figure 1).

The point normal equation of a plane is

If we rewrite the equation to m we get a general equation of the plane as

, where

The plane with general equation above is perpendicular to the normal

vector and lies at a distance

from the origin.

The plane whose general equation is passes the origin.

In geometry, each algebraic equation represents a plane in 3D.

So, we may consider the problem in example 2 the same as finding the

unique point that three planes intersect each other.

It is obvious that not all three planes will intersect to a unique point.

Three planes can be parallel each other as illustrated in example 3. So no

solution exists. Three planes can intersect to a common straight line.

Then we may find infinite many solutions as demonstrated in example 4.

Self-Check

Exercises

3. Find the distance of the plane 3x+4y + 12 z= 26 from the origin.

4. Find the distance between two parallel planes 3x+4y + 12 z=

26 and 3x+4y + 12 z= 39.

5. Prove the disttance formula above by projecting any arbitary vector

on the plane to the normal vector.

Hint: the length of the projected vector at normal direction is the

distance.

Homogeneous systems

Definition

Homogeneous

Systems

If the constant vector of the system is a zero vector, (i.e. all elements of

the vector are zeros, denote as ), we call the system as a homogeneous

system.

If we substitute all variables with 0 to each equation, we get 0 = 0.

Hence, all homogenous systems have the zero vectors as their solutions.

In geometry, since all planes pass the origin, the origin is a solution to

the system – a trivial solution.

http://mathworld.wolfram.com/Nonzero.html

http://mathworld.wolfram.com/NormalVector.html

http://mathworld.wolfram.com/Point-PlaneDistance.html

http://mathworld.wolfram.com/Origin.html



Example 6

Nonsingular

Coefficient

Matrix

Example 7

Find the null

space of the

projection

Our interests with homogeneous systems are their nontrivial solutions.

The equations

⌊

⌋

Since we know that the inverse of A exists, we then know the system has

unique solution, , .

The conclusion is that a homogeneous system has a nonsingular

coefficient matrix, there is only a trivial solution . No further

discussion is required.

We learning in lesson 1.2.1, that the projection matrix to project a 3D

vector to the plane ,

[

] .

The points (vectors ) that transform to the zero vector in coordinate is

called the null space of the transformation. Find the null space of the

projection in example 6 is to solve the system equations , multiple

3 on both sides, we only need to solve

We have the same coefficient matrix as example 4, Follow the same

previous two steps of example 4 except changing the right most column

to all zeros Step 3

Add (-1) multiple of the second row to the third row

[

]

The third equation is 0 = 0. So it is clear that the system only has two

independent equations, i.e. one variable is free to take any number t.



Example 8

Step 4 Let by the second equation , so we have Substitute the first equation , we have Since t is arbitrary real number, we have infinite many solutions as

follows:

[

] [

]

We can interpret the vector solution as a straight line that passes the

origin (0, 0, 0) and is parallel to the vector . See figure 2, the

vector is exactly the normal vector of the plane , which can be

rewrite as . Of course, such a straight line is projected to the origin

since it is parallel to the normal vector.

Comparing the equations in example 4 and example 7, we notice that

each pair of corresponding planes shares the same normal vector.

Therefore, each pair of corresponding equations represents a pair of

parallel planes. It explains why the solutions of the two systems are a

pair of parallel straight lines that share the same directional vector [1, -1,

1]’. The only difference is that the line as the solution to example 4

passes point (1, 1, 0) and the line as the solution of example 7 passes

origin.

Solve the homogenous system equation

Solution,

It is clear that that the second equation is twice of the first equation and the third equation is (-3) times of the first equations. Therefore, this system has only one independent equation and the other two are redundant. In Geometry the solution is a plane .

As we described above, a plane is a 2d surfaces that is spanned by two free vectors. We have two free variables of our choices. Rewrite the equation to , let , then , the

solution is:

[ ] [

] ⌊

⌋ [

]

As we described above, a plane is a 2D surface (two free variables) that is spanned by two independent vectors. The solution of this example is a plane that is spanned by two vectors [1, 0, 2]’ and [0, 1,

1]’. Because the expression of free variables is our free choices, the representation of the vectors is not unique. However, once the two free variables are selected, the third variable must satisfy the

constraint given by the only equation.



Example 9

A system of 4

variables

Conclusion: All homogeneous systems have solution. Depending on

the number of free variable(s) which is defined as the difference between the number of variables and independent equations, the solutions of homogeneous system with3 variables can have unique

solution, infinite solutions in a straight line or infinite solutions of points on a plane.

The concept of the number of free variables can be generalized to a system of any number of variables. If we apply to Gauss-Jordon algorithm to a system, the number of nonzero rows of the upper

triangular matrix is called the rank of the matrix, the number of free variables equals to the difference between the number of variables and the rank. For example, the rank of the example 6 is 3, number of

free variable is 0. So we have unique solution. The rank of the example 7 is 2, the number of free variable is 1. Then, the solution is a straight line (1D). The rank of example 8 is 1, the number of free

variable is 2. Therefore, the solution is a plane that is a 2D surface.

Let us use the Gauss-Jordon Algorithm to solve the system equation Step 1, write the system into matrix form

[

]

Step 2:

(a) The second row minus twice of the first row and the fourth row add

twice of the third row,

[

]

(b) Rearrange all the rows in different order,

[

]

(c) The second row minus twice of the first row

[

]

Step 4



It is clear that the rank of the system is 2 and the number of free variables is 2. We can choose , and , then the second equation

give us

The first equation gives

Hence the solution is

[

] [

] [

], to make it look better, we may let

, then we have

[

] [

] [

],

In geometry, we call it as a 2D hyper-plane, or an 2D manifold.

MATLAB command to solve linear system equations

Example 10

Solve an

system with

non-singular

Coefficient

Matrix

By command

A\b

We have shown how to use inv(A) to solve systems that have unique

solution.

>> A= [2 6 8; 4 15 19; 2 0 3];

>> b = [16, 38, 6]';

>> det(A) % use the determinant to check if the matrix is non-singular

ans =

6

>> x = A\b

x =

0

0

2

We can use MATLAB to find all solutions of under-determined systems

Step 1, solve the homogeneous system by finding the spanning vectors of

the null space

Step 2, Find a particular solution to the non-homogeneous system

equations and check your answer because the system may have no

solution

Step 3, Compose the solution in vector form with parameters for free



Example 11

Solve

Under-

determined

System

equations

by MATLAB

command

pinv(A)

Pseudo-

inverse

operator

And

Null(A, ‘r’)

variables.

It is easy to see that the second equation is twice of the first one and the

fourth equation is (-2) of the third one. We may select the two

independent equations in example 9 to solve the system by MATLAB

Step 1: Solve the Null vectors

>> A = [1 -3 1 -1; 2 1 -1 1]

A =

1 -3 1 -1

2 1 -1 1

>> format rat

>> z = null(A, 'r')

z =

2/7 -2/7

3/7 -3/7

1 0

0 1

This first step solves the problem 9, we may rewrite the equation with

parameters t and s so the solution is expressed as

[

] [

] [

],

Step 2, Find a particular solution and check it

>>b = [7; 14]

>> p = A\b

p =

7

1/3560408122994089

0

0

>> A * p % check AP = b to find if the p above truly solves the problem



Self-Check

Exercise

ans =

7

14

Step three, compose the solution

[

] [

] [

] [

]

6. Find the vectors in the null space to the projection transformation T T from to defined as

[

] [

] [

]

We will not address the over-determined systems until next module.

Homework

Exercises

Use Gauss-Jordon algorithm to solve the following system equations and

then use MATLAB to check your answers.

1, Solve the homogeneous system

Use MATLAB to solve the following under-determined systems, please

write the solution in vector form with parameters for free variable.

2,

3.



Answers to Self-Check Exercises

1. [

] [

] [

] [

]

Solution is x = 1, y = -1 and z = 2.

2. MATLAB Solution

A = [1 2 3; 2 5 3; 1 0 8]

b = [5 3 17]’

solu = inv(A) * b

[1.0000 -1.0000 2.0000]’

Or use A\b you get the same answer.

3. Answer: d =

√

4. Answer: d =

√

5. Proof:

Let n = [a, b , c] be the (nonzero) normal vector, V1 = [x1, y1, z1] be an

arbitrary vector in the plane ax + by + cz =d1, V2 = [x2, y2, z2] be an

arbitrary vector in the plane ax + by + cz =d2.

Plug V1 and V2 to the plane equation above, respectively, we have

a x1 + b y1 + c z1 = d1, and a x2 + b y2 + c z2 = d2,

V = v2 – v1 is a vector that initiates at a point on first plane and terminates at

a point on the second plane. The distance between the two planes is the

projection of V onto the normal vector n.

√

6. Solution, [

]

The second row subtract 3 times of the first row, and the third row add 2 times of the first

row, we have

[

]

Hence, x1 + x2 + x3 = 0, The rank of the matrix is 1, there are two freedoms, let x2 = t,

x3 =s, then x1 = -x2 – x3 = -t –s

Solution is [

] [

] [

] [

], the null space is the plane spanning by the two

linearly independent vectors [-1, 1, 0]’ and [-1, 0, 1]’.

Documents

Solving Linear Equations · Reflection Questions Solving a system of algebraic equations is not new to any of us. ... (VTK) such as OpenGL, Catia, computational geometry is used to