8/22/15

1

Introduction to systems of linear equations

Larson 1.1

Copyright 2015

Linear equation?

•  A linear equation is one in which the terms are either constants or a variable multiplied by a constant. The variables are to the first power only. Examples:

3 2 84 5 25

2 3 8 0

xx y

x y z

+ = −− =

− + =

Copyright 2015

More generally... •  We can express these linear equations

generally as:

•  More generally still, a linear equation is of the form:

ax bax by c

ax by cz d

=+ =

+ + =

1 1 2 2 3 3 ... n na x a x a x a x b+ + + + =

Copyright 2015

Systems of linear equations

•  If we have a finite set of these linear equations, we call it a system.

4 3 122 2 system:

2 6

2 3 73 3 system: 2 2 4

3 3 11

x yx y

x y zx y zx y z

− =⎧× ⎨ + =⎩

+ + =⎧⎪× − − =⎨⎪ + + =⎩

Copyright 2015

Systems, continued

•  A general m by n system of linear equations; solutions are ordered n-tuples, (x1, x2, x3,..., xn):

a11x1 + a12x2 + a13x3 + ...+ a1nxn = b1

a21x1 + a22x2 + a23x3 + ...+ a2nxn = b2

a31x1 + a32x2 + a33x3 + ...+ a3nxn = b3

!

am1x1 + am2x2 + am3x3 + ...+ amnxn = bmCopyright 2015

Types of linear systems •  When classified according to the number

of solutions they posses, there are three kinds of linear systems:

1)  Consistent, independent (one solution); 2)  Consistent, dependent (infinitely many

solutions); 3)  Inconsistent (no solution).

•  The visual representation for these systems depends on where we are (i.e., R2, R3, and so forth).

8/22/15

2

Possible 2x2 systems

Copyright 2015

Possible 3x3 systems

Copyright 2015

Types of linear systems

Linear system

Consistent

Independent (one solution)

Dependent (infinitely many

solutions) Inconsistent (no solution)

Copyright 2015

Solving linear systems •  One approach uses elementary row

operations. Each operation produces an equivalent system (a system with the same solution set): –  Interchange/swap two equations; – Multiply an equation by a nonzero constant; – Add a multiple of one equation to (a multiple

of) another equation. •  One way to solve is to reduce the system

until back-substitution can be used. Copyright 2015

Reporting solutions •  Inconsistent systems have no solution. •  Consistent systems have at least one

solution. –  If the system is composed of n independent

equations, the solution is an n-tuple, which is what you would report:

–  If the system is dependent, the solution is also reported as an n-tuple, but typically we expect it to be expressed in terms of parameterized free variables.

Copyright 2015

x1, x2, x3,..., xn( )

Gaussian Elimination and Gauss-Jordan Elimination

Larson 1.2

8/22/15

3

Copyright 2015

Matrix representation

•  Systems can be conveniently represented in matrix form:

•  The matrix above is 2 rows by 3 columns. It’s called an augmented matrix because the rightmost column contains the constants from the RHS of the system.

4 3 12 4 3 122 6 1 2 6

x yx y− = −⎫ ⎡ ⎤

⇒⎬ ⎢ ⎥+ = ⎭ ⎣ ⎦

Copyright 2015

What is convenient about matrices? •  When a system is expressed in matrix

form, it is more convenient to solve the system using elementary row operations:

–  Interchange/swap two rows (equations); – Multiply a row (equation) by a nonzero

constant; – Add a multiple of one row (equation) to a

another row (equation). •  It is advisable to learn the calculator

syntax for these operations.

Copyright 2015

Down the road... •  You’ll see that expressing a linear system

in matrix format offers many advantages over other expressions.

•  It can be shown that applying elementary operations to a system of linear equations does not change the solution set of the system. The system you started with will have the same solution set as the row-equivalent system you produce via elementary row operations.

TI-83/84 row operation syntax •  All of these commands are accessed via

the MATRIXàMath submenu. •  To swap row Ri with Rj ( ):

–  rowswap(matrix name, Ri, Rj) •  To multiply row Ri by a number ( ):

– *row(value, matrix name, Ri) •  To multiply row Ri by a number, and add

the result to Rj ( ): – *row+(value, matrix name, Ri, Rj)

Copyright 2015

Ri ↔ Rj

cRi

cRi + Rj

Notation for elementary row operations •  Larson

–  Row swap:

–  Multiply a row by a constant:

–  Multiply a row by a constant and add the result to another row:

•  MKL –  Row swap:

–  Multiply a row by a constant:

–  Multiply a row by a constant and add the result to another row:

Copyright 2015

Ri ↔ Rj

cRi → Ri

Rj + cRi → Rj

Ri ↔ Rj

cRi

cRi + Rj

Copyright 2015

Reduced forms of a matrix

•  Applying elementary row operations to a matrix eventually yields a different form of the matrix: – Row echelon form; – Reduced row echelon form.

•  Given a matrix, you need to know these forms when you see them, and also be able to produce either of these forms on demand.

8/22/15

4

Copyright 2015

Row echelon form •  When a matrix is transformed into row

echelon form via elementary row operations, it will have: – Rows that are all zeros will be at the bottom; – Nonzero rows will have leading entries of one

(makes life easy), and the leading entries will be to the left of the leading entries which appear below them.

– Note that some authors refer to leading entries as “pivots.”

Copyright 2015

Examples: row echelon form •  The row operations are omitted in the

example below. 2 3 11 1 2 3 11

2 3 2 3 2 3 2 33 4 5 13 3 4 5 13

RowOps

x y zx y zx y z

− + = −⎡ ⎤⎢ ⎥+ + = − ⇒ − ⎯⎯⎯→⎢ ⎥⎢ ⎥− − = − −⎣ ⎦

4 5 133 3 3 1 4 3 5 3 13 3

16 35 0 1 16 17 35 1717 17

0 0 1 11

x y z

y z

z

− − =− −⎡ ⎤

⎢ ⎥+ = − ⇒ −⎢ ⎥⎢ ⎥⎣ ⎦=

Copyright 2015

Row echelon form •  Row echelon form is not unique; the matrix

used in the preceding example has three different row echelon forms. The other two: 2 3 11 1 2 3 11

2 3 2 3 2 3 2 33 4 5 13 3 4 5 13

RowOps

x y zx y zx y z

− + = −⎡ ⎤⎢ ⎥+ + = − ⇒ − ⎯⎯⎯→⎢ ⎥⎢ ⎥− − = − −⎣ ⎦

1 2 3 11 1 3 2 1 3 20 1 7 10 and 0 1 4 7 25 70 0 1 1 0 0 1 1

− −⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥− − − −⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦

Copyright 2015

Row echelon form, cont.

•  You can identify the type of system you have (i.e., independent, inconsistent, or dependent) when it is in row echelon form.

•  When the leading entries are called pivots, the process of obtaining the row echelon form is frequently called “pivoting.”

Copyright 2015

Row echelon forms 1 2 3 11 1 2 3 11

Independent 2 3 2 3 0 1 7 10

system:3 4 5 13 0 0 1 1

RowOps

− −⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥− ⎯⎯⎯→ − −⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥− −⎣ ⎦ ⎣ ⎦

2 1 1 1 1 1 2 1 2 1 2Inconsistent

1 2 1 1 0 1 1 1system:

1 1 2 1 0 0 0 1

RowOps

− − − − − −⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥− ⎯⎯⎯→ − −⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥−⎣ ⎦ ⎣ ⎦

3 1 1 4 1 1 3 1 3 4 3Dependent

2 3 2 7 0 1 8 7 29 7system:

1 2 3 11 0 0 0 0

RowOps

− − − −⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎯⎯⎯→⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥− − −⎣ ⎦ ⎣ ⎦

Copyright 2015

General matrix in row echelon form •  Notice:

•  Leading entries are all ones; •  What will turn out to be coefficients of free

variables in the solution are marked with fv.

•  How can you identify the free variables?

1 * fv * … * fv *0 1 fv * … * fv *0 0 0 1 … * fv *0 0 0 0 … 1 fv *0 0 0 0 … 0 0 00 0 0 0 … 0 0 0

⎡

⎣

⎢⎢⎢⎢⎢⎢⎢

⎤

⎦

⎥⎥⎥⎥⎥⎥⎥

8/22/15

5

Copyright 2015

Reduced row echelon form

•  When a matrix is transformed into reduced row echelon form via elementary row operations, it is basically row echelon form plus: – The rest of the entries in a column containing

a leading one will be zero.

Copyright 2015

Examples: reduced row echelon form •  The row operations are omitted in the

example below. 2 3 11 1 2 3 11

2 3 2 3 2 3 2 33 4 5 13 3 4 5 13

RowOps

x y zx y zx y z

− + = −⎡ ⎤⎢ ⎥+ + = − ⇒ − ⎯⎯⎯→⎢ ⎥⎢ ⎥− − = − −⎣ ⎦

2 1 0 0 23 0 1 0 31 0 0 1 1

xyz

= ⎡ ⎤⎢ ⎥= − ⇒ −⎢ ⎥⎢ ⎥= ⎣ ⎦

Copyright 2015

Reduced row echelon form

•  Reduced row echelon form is unique; the matrix used in the preceding example has one and only one reduced row echelon form (as do all matrices).

•  You can also identify the type of system you have when it is in reduced row echelon form.

Copyright 2015

Reduced row echelon forms 1 2 3 11 1 0 0 2

Independent 2 3 2 3 0 1 0 3

system:3 4 5 13 0 0 1 1

RowOps

−⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥− ⎯⎯⎯→ −⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥− −⎣ ⎦ ⎣ ⎦

2 1 1 1 1 0 1 0Inconsistent

1 2 1 1 0 1 1 0system:

1 1 2 1 0 0 0 1

RowOps

− − − −⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥− ⎯⎯⎯→ −⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥−⎣ ⎦ ⎣ ⎦

3 1 1 4 1 0 5 7 19 7Dependent

2 3 2 7 0 1 8 7 29 7system:

1 2 3 11 0 0 0 0

RowOps

− − − −⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎯⎯⎯→⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥− − −⎣ ⎦ ⎣ ⎦

Copyright 2015

General matrix in reduced row echelon form •  Notice:

•  Leading entries are all ones, and other entries in

a column with a leading one are zero; •  What will turn out to be coefficients of free

variables in the solution are marked with fv.

1 0 fv 0 … 0 fv *0 1 fv 0 … 0 fv *0 0 0 1 … 0 fv *0 0 0 0 … 1 fv *0 0 0 0 … 0 0 00 0 0 0 … 0 0 0

⎡

⎣

⎢⎢⎢⎢⎢⎢⎢

⎤

⎦

⎥⎥⎥⎥⎥⎥⎥

Copyright 2015

Solving a system

•  Two methods, using elementary row operations: – Gaussian elimination (put the matrix in row

echelon form, solve using back-substitution); – Gauss-Jordan elimination (put the matrix in

reduced row echelon form). •  Of the two methods, Gauss-Jordan is

computationally more expensive.

8/22/15

6

Gaussian Elimination vs. Gauss-Jordan

Copyright 2015

System in row echelon form

Gaussian Elimination:

solving via back-substitution

Gauss-Jordan: obtaining reduced-row echelon form

via row ops Copyright 2015

Counting operations

•  Solving linear systems efficiently is an important practical concern.

•  Numerical Analysis is the study of how to do arithmetic efficiently using computers. – Starting with concrete examples, and

progressing to general examples with nxn systems, it can be shown how many operations are required to solve such a system.

Copyright 2015

Gaussian elimination •  It can be shown that for an nxn system,

this method requires: 3

2Multiplication/division ops.:3 3n nn+ −

3 2 5Addition/subtraction ops.:3 2 6n n n+ −

3 232 3 7 2Total ops.: for large

3 2 6 3n n n n n+ − ≈

Copyright 2015

Gauss-Jordan elimination •  It can be shown that for an nxn system,

this method requires: 3

2Multiplication/division ops.:2 2n nn+ −

3

Addition/subtraction ops.:2 2n n−

3 2 3Total ops.: for large n n n n n+ − ≈

Copyright 2015

Gaussian Elimination vs. Gauss-Jordan

n GE GJ 2 9 10 3 28 33 4 62 76

10 805 1,090 20 5,910 8,380

100 681,550 1,009,900 1000 668,165,500 1,000,999,000

Copyright 2015

Homogeneous systems •  A homogeneous system is a special kind

of linear system in which the constant terms are all zero.

4 3 02 2 homogeneous system:

2 0

2 3 03 3 homogeneous system: 2 2 0

3 3 0

x yx y

x y zx y zx y z

− =⎧× ⎨ + =⎩

+ + =⎧⎪× − − =⎨⎪ + + =⎩

8/22/15

7

Copyright 2015

General homogeneous system

•  In general, a homogeneous system is of the form:

a11x1 + a12x2 + a13x3 + ...+ a1nxn = 0a21x1 + a22x2 + a23x3 + ...+ a2nxn = 0a31x1 + a32x2 + a33x3 + ...+ a3nxn = 0

!

am1x1 + am2x2 + am3x3 + ...+ amnxn = 0

Copyright 2015

Solutions of a homogeneous system •  A homogeneous system can’t be

inconsistent. They can only be: – Consistent (in which case they have only the

trivial solution, where x1 = 0, x2 = 0,..., xn = 0). – Dependent (in which case we have the trivial

solution plus infinitely many other solutions).

1 2

1 2

4 3 0Dependent homogeneous system: 8 6 0

x xx x− =⎧

⎨− + =⎩

Copyright 2015

Solutions, cont. •  One way to guarantee a homogeneous

system is dependent is to have more unknowns than equations.

•  Theorem: if an mxn homogeneous system has a row-reduced echelon form with r nonzero rows, the solution will have n – r free variables.

1 2 3 4

1 2 3 4

1 2 3 4

3 2 4 0Dependent 3 4

2 3 0homogeneous system:

4 2 2 0

x x x xx x x xx x x x

+ − + =⎧× ⎪ + + + =⎨

⎪ − − + =⎩

Copyright 2015

Example •  If we put the system on the previous slide

into a matrix and row-reduce it, we obtain:

•  The solution contains three leading variables, and one free variable, x4. It is customary to express such a solution set in parametric form (see examples 5 & 6).

3 2 4 1 0 1 0 0 1 01 2 1 3 0 0 1 0 3 5 04 2 1 2 0 0 0 1 4 5 0

RowOps

−⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎯⎯⎯→⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥− −⎣ ⎦ ⎣ ⎦

Graphical perspective

•  A homogeneous system can’t be inconsistent. –  Inconsistency requires no common

intersection (see slides 7, 8); – A homogeneous system must contain the

origin (i.e., common intersection). •  Hence, homogeneous systems can only

be consistent (with the trivial solution) or dependent (with infinitely many solutions).

Copyright 2015