Linear Systems Numerical Methods. Linear equations N unknowns, M equations N unknowns, M equations...

Linear SystemsLinear Systems

Numerical MethodsNumerical Methods

Linear equationsLinear equations• NN unknowns, unknowns, MM equations equations

wherewhere

coefficient coefficient matrixmatrix

Determinants and Cramer’s RuleDeterminants and Cramer’s Rule

[A] : [A] : coefficient matrixcoefficient matrix bxA

333231

232221

131211

333231

232221

131211

x 33323

x 33331

x 33231

D : Determinant of A matrix

Solving methodsSolving methods

• Direct methodsDirect methods– Gauss eliminationGauss elimination

– Gauss-Jordan eliminationGauss-Jordan elimination

– LU decompositionLU decomposition

– Singular value decompositionSingular value decomposition

– ……

• Iterative methodsIterative methods– Jacobi iterationJacobi iteration

– Gauss-Seidel iterationGauss-Seidel iteration

– ……

Linear SystemsLinear Systems

• Solve Ax=b, where A is an Solve Ax=b, where A is an nnnn matrix and matrix andb is an b is an nn1 column vector1 column vector

• Can also talk about non-square systems Can also talk about non-square systems wherewhereA is A is mmnn, b is , b is mm1, and x is 1, and x is nn11– OverdeterminedOverdetermined if if mm>>nn::

“more equations than unknowns”“more equations than unknowns”

– UnderdeterminedUnderdetermined if if nn>>mm::“more unknowns than equations”“more unknowns than equations”Can look for best solution using least squaresCan look for best solution using least squares

Gauss EliminationGauss Elimination

• Solve Solve AxAx = = bb

• Consists of two phases:Consists of two phases:–Forward eliminationForward elimination–Back substitutionBack substitution

• Forward EliminationForward Eliminationreduces reduces AxAx = = bb to an upper to an upper triangular system triangular system TxTx = = b’b’

• Back substitutionBack substitution can then can then solve solve TxTx = = b’b’ for for xx

1131211

3333231

2232221

1131211

ForwardElimination

BackSubstitution

21231311

xaxabx

Gauss EliminationGauss Elimination

• Fundamental operations:Fundamental operations:1.1. Replace one equation with linear combinationReplace one equation with linear combination

of other equationsof other equations

2.2. Interchange two equationsInterchange two equations

3.3. Re-label two variablesRe-label two variables

• Combine to reduce to trivial systemCombine to reduce to trivial system

• Simplest variant only uses #1 operations,Simplest variant only uses #1 operations,but get better stability by addingbut get better stability by adding#2 (partial pivoting) or #2 and #3 (full #2 (partial pivoting) or #2 and #3 (full pivoting)pivoting)

Gaussian EliminationGaussian EliminationForward Elimination

x1 - x2 + x3 = 6 3x1 + 4x2 + 2x3 = 9 2x1 + x2 + x3 = 7

x1 - x2 + x3 = 6 0 +7x2 - x3 = -9 0 + 3x2 - x3 = -5

x1 - x2 + x3 = 6 0 7x2 - x3 = -9 0 0 -(4/7)x3=-(8/7)

-(3/1)

Solve using BACK SUBSTITUTION: x3 = 2 x2=-1 x1 =3

-(2/1) -(3/7)

Back SubstitutionBack Substitution

1x0 +1x1 –1x2 +4x3 8=

– 2x1 –3x2 +1x3 5=

2x2 – 3x3 0=

2x3 4=x3 = 2

1x0 +1x1 –1x2 0=

– 2x1 –3x2 3=

2x2 6=

x2 = 3

1x0 +1x1 3=

– 2x1 12=

x1 = –6

1x0 9=

x0 = 9

for i n down to 1 do

/* calculate xi */

x [ i ] b [ i ] / a [ i, i ]

/* substitute in the equations above */for j 1 to i-1 do

b [ j ] b [ j ] x [ i ] × a [ j, i ]endfor

endfor

Back Substitution(* Pseudocode *)

Time Complexity? O(n2)

Forward EliminationForward Elimination

4x0 +6x1 +2x2 – 2x3 = 8

2x0 +5x2 – 2x3 = 4

–4x0 – 3x1 – 5x2 +4x3 = 1

8x0 +18x1 – 2x2 +3x3 = 40

-(2/4)

MULTIPLIERS

-(-4/4)

-(8/4)

4x0 +6x1 +2x2 – 2x3 = 8

+4x2 – 1x3 = 0

+3x1 – 3x2 +2x3 = 9

+6x1 – 6x2 +7x3 = 24

– 3x1

-(3/-3)

MULTIPLIERS

Forward Elimination

-(6/-3)

4x0 +6x1 +2x2 – 2x3 = 8

+4x2 – 1x3 = 0

1x2 +1x3 = 9

2x2 +5x3 = 24

– 3x1

MULTIPLIER

Forward Elimination

4x0 +6x1 +2x2 – 2x3 = 8

+4x2 – 1x3 = 0

1x2 +1x3 = 9

3x3 = 6

– 3x1

Forward EliminationForward Elimination

Gauss-Jordan EliminationGauss-Jordan Elimination

b660 0 0 x99

Gauss-Jordan Elimination: Gauss-Jordan Elimination: ExampleExample

:Matrix Augmented

1 1 2 | 8

0 1 5 | 9

0 4 2| 14

R2 R2 - (-1)R1

R3 R3 - ( 3)R1

Scaling R2:

R2 R2/(-1)

R1 R1 - (1)R2

R3 R3-(4)R2

1 1 2 | 8

0 1 5| 9

0 4 2| 14

Scaling R3:

R3 R3/(18)

R1 R1 - (7)R3

R2 R2-(-5)R3

RESULT:

x1=8.45, x2=-2.89, x3=1.23

Time Complexity? O(n3)

• Solve:Solve:

• Only care about numbers – form Only care about numbers – form “tableau” or “augmented matrix”:“tableau” or “augmented matrix”:

• Given:Given:

• Goal: reduce this to trivial systemGoal: reduce this to trivial system

and read off answer from right columnand read off answer from right column

• Basic operation 1: replace any row byBasic operation 1: replace any row bylinear combination with any other rowlinear combination with any other row

• Here, replace row1 with Here, replace row1 with 11//22 * row1 + 0 * * row1 + 0 *

row2row2

• Replace row2 with row2 – 4 * row1Replace row2 with row2 – 4 * row1

• Negate row2Negate row2

• Replace row1 with row1 – Replace row1 with row1 – 33//22 * row2 * row2

• Read off solution: xRead off solution: x1 1 = 2, x= 2, x2 2 = 1= 1

Augmented MatricesAugmented Matrices

• Matrices are Matrices are rectangular arrays of numbersrectangular arrays of numbers that can aid us by that can aid us by eliminating the need to eliminating the need to write the variableswrite the variables at each step of the at each step of the reduction.reduction.

• For example, the For example, the systemsystem

may be represented by the may be represented by the augmentedaugmented matrixmatrix

CoefficiCoefficient ent

MatrixMatrix

2 4 6 22

3 8 5 27

2 4 6 22

3 8 5 27

2 4 6 22

3 8 5 27

1 1 2 2

2 4 6 22

3 8 5 27

1 1 2 2

Matrices and Gauss-JordanMatrices and Gauss-Jordan

• Every stepEvery step in the in the Gauss-Jordan elimination Gauss-Jordan elimination methodmethod can be expressed with can be expressed with matricesmatrices, rather , rather than systems of equations, thus simplifying the than systems of equations, thus simplifying the whole process:whole process:

• Steps expressed as Steps expressed as systems of equationssystems of equations::

• Steps expressed as Steps expressed as augmented matricesaugmented matrices::

2 4 6 22

3 8 5 27

2 4 6 22

3 8 5 27

2 4 6 22

3 8 5 27

1 1 2 2

2 4 6 22

3 8 5 27

1 1 2 2

Toggle slides Toggle slides back and back and forth to forth to compare compare

before and before and changeschanges

Matrices and Gauss-JordanMatrices and Gauss-Jordan

• Every stepEvery step in the in the Gauss-Jordan elimination Gauss-Jordan elimination methodmethod can be expressed with can be expressed with matricesmatrices, , rather than systems of equations, thus rather than systems of equations, thus simplifying the whole process:simplifying the whole process:

1 2 3 11

0 2 4 6

1 1 2 2

1 2 3 11

0 2 4 6

1 1 2 2

2 3 11

Matrices and Gauss-JordanMatrices and Gauss-Jordan• Every stepEvery step in the in the Gauss-Jordan elimination Gauss-Jordan elimination

methodmethod can be expressed with can be expressed with matricesmatrices, , rather than systems of equations, thus rather than systems of equations, thus simplifying the whole process:simplifying the whole process:

1 2 3 11

0 2 4 6

0 3 5 13

1 2 3 11

0 2 4 6

0 3 5 13

2 3 11

3 5 13

2 3 11

3 5 13

1 2 3 11

0 1 2 3

0 3 5 13

1 2 3 11

0 1 2 3

0 3 5 13

2 3 11

3 5 13

2 3 11

3 5 13

1 0 7 17

0 1 2 3

0 3 5 13

1 0 7 17

0 1 2 3

0 3 5 13

3 5 13

Toggle slides Toggle slides

back and back and forth to forth to compare compare

1 0 7 17

0 1 2 3

0 0 11 22

1 0 7 17

0 1 2 3

0 0 11 22

1 0 7 17

0 1 2 3

0 0 1 2

1 0 7 17

0 1 2 3

0 0 1 2

1 0 0 3

0 1 2 3

0 0 1 2

1 0 0 3

0 1 2 3

0 0 1 2

1 0 0 3

0 1 0 1

0 0 1 2

1 0 0 3

0 1 0 1

0 0 1 2

Row Row Reduced Reduced

Form of the Form of the MatrixMatrix

Linear Systems Numerical Methods. Linear equations N unknowns, M equations N unknowns, M equations...

Documents

Euclidean m-Space & Linear Equations Systems of Linear Equations

Solving Systems of Equations. Rule of Thumb: More equations than unknowns system is unlikely to have a solution. Same number of equations as unknowns

2 Equations with 2 Unknowns

Linear Equations in Two Variables Graphing Linear Equations

NumericalAnalysisLectureNotesolver/num_/lnq.pdf · 13.1. Least Squares. Linear systems with more equations than unknowns typically do not have solutions. In such situations, the least

Gauss Elimination. A System of Linear Equations Two Equations, Two Unknowns: Lines in a Plane

2.2 – Linear Equations. Linear equation 2.2 – Linear Equations Linear equation – equation with only addition,

Tamilnadu Board Class 10 Maths Chapter 3 - BYJU'S...Algebra 69 Definition Remarks 3.2 System of linear equations in two unknowns n class IX, we have studied the linear equation I ax

Lab 3: Parallel Methods of Solving the Linear Equation Systems · The Gaussian elimination consists in sequential elimination of the unknowns in the equations of the linear equation

SYLLAABBUUSS FFOORR .B.Sc. MMMAATTHHEEMAATTIICCSS …uni-mysore.ac.in/sites/default/files/content/... · System of m linear equations in n unknowns – Matrices associated with linear

Solving Linear Equations - · PDF fileSolving Linear Equations - WordPress.com

Linear equations lesson 4 graphing linear equations

Linear relations and equations - John Wiley & Sons Linear relations and equations 1.1 Kick off with CAS 1.2 Linear relations 1.3 Solving linear equations 1.4 Developing linear equations

Chapter 8williams/Classes/300W2012/PDFs/PPTs/India.pdf · • Solution of linear equations with as many as five unknowns. • The solution of a quadratic equation • Arithmetic and

8.3 Simultaneous First-Order Differential-equations in Two Unknowns (1)

Linear Equations In 2 Unknowns Meaning Of Linear Equations

2.4 – Writing Linear Equations. 2.4 – Writing Linear Equations Forms:

Chapter 5. Linear Algebra - math.uh.eduetgen/Ch5-slides-Part1-Sp20notes.pdf · Linear equations in two unknowns x, y: ax + by = α. A solution of the equation is an or-dered pair

UIVERSITY OF DELHI DEPARTMET OF MATHEMATICS B.A. …maths.du.ac.in/Courses/BABSc/B.A.(Prog)(2019).pdf · iii) Solving simultaneous linear equations with at most four unknowns. iv)

Last lecture summary Fundamental system in linear algebra : system of linear equations Ax = b. nice case – n equations, n unknowns –matrix notation –row