SSCE2393NUMERICALMETHODS

CHAPTER2LINEAREQUATIONSYSTEMS

FarhanaJohar,DepartmentofMathematicalSciences,FacultyofScience,UTM.farhanajohar@utm.my

OverviewChapter2

LinearEquationSystems

GAUSSELIMINATION JACOBI GAUSSELIMINATIONWITH GAUSS-SEIDEL

PARTIALPIVOTING

• DOOLITTLE• CROUT• THOMASMETHOD• THOMASALGORITHM• CHOLESKY

ELIMINATIONMETHOD

LUFACTORIZATION

ITERATION METHOD

mnmnmm

nn

nn

bxaxaxa

bxaxaxabxaxaxa

=+++

=+++

=+++

…!!!!!

…

…

2211

22222121

11212111

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

nnmnmm

n

n

b

bb

x

xx

aaa

aaaaaa

!!…

!!!!……

2

1

2

1

21

22221

11211

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

nmnmm

n

n

b

bb

aaa

aaaaaa

!…

!!!!……

2

1

21

22221

11211

2.1INTRODUCTIONOFLINEAREQUATIONSYSTEMSLinearequations: bax =

Inmatrixform( bxA =×nm )Inaugmentedmatrixform

Wewillonlyconsider nn×A matrix(squarematrix).

EliminationmethodSolutions LU Iteration

Backward subst.

⎥ ⎥ ⎥

⎦

⎤

⎢ ⎢ ⎢

⎣

⎡

3

2

1

33 32 31

23 22 21

13 12 11

b

b

b

a a a

a a a

a a a

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

3

2

1

33

2322

131211

000

fff

uuuuuu

3331321211

2233322

3333

/)(/)(

uxuxudxuxufx

ufx

−−=

−=

=

⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛

⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛

⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛=

3

2

1

3

2

1

33

2322

131211

000

fff

x

x

x

uuuuuu

2.1EliminationMethod

• GaussElimination• Gausseliminationwithpartialpivoting

2.1.1GaussEliminationGiven: Writteninaugmentedmatrixformas:

fUx =

bAx =

3333232131

2323222121

1313212111

bxaxaxabxaxaxabxaxaxa

=++

=++

=++

Elementary row operations

Elementary row operations

212 59.0 BBB +−→

Example1:SolvethefollowinglinearsystemusingGausseliminationmethod.

84.148.104.368.275.030.193.048.156.553.448.151.2

321

321

321

−=−+

−=−+

=++

xxxxxxxxx

Solution

⎥⎥⎥

⎦

⎤

−

−

⎢⎢⎢

⎣

⎡

−

−

84.175.056.5

48.104.368.230.193.048.153.448.151.2

2.51 1.48 4.530 0.06 −3.972.68 3.04 −1.48

⎡

⎣

⎢⎢⎢

5.56−4.03−1.84

⎤

⎦

⎥⎥⎥

multiplier, 59.0112121 == aam multiplier, 07.1113131 == aam

313 07.1 BBB +−→

2.51 1.48 4.530 0.06 −3.970 1.46 −6.33

⎡

⎣

⎢⎢⎢

5.56−4.03−7.79

⎤

⎦

⎥⎥⎥B3→−24.33B2 +B3

multiplier,m32 = a32 a22 = 24.33

2.51 1.48 4.530 0.06 −3.970 0 90.26

⎡

⎣

⎢⎢⎢

5.56−4.0390.26

⎤

⎦

⎥⎥⎥

∴Usingbackwardsubstitution,weget:

x3 =1x2 = −1x1 =1

Exercise:SolvethefollowinglinearsystemsusingGausseliminationmethod:1.Use4decimalplaces.

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−

=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡ −

11210

511042115

3

2

1

xxx

2.Use3decimalplaces.

8x1 + 2x2 − x3 + 2x4 = −2x1 −8x2 + x3 − 2x4 = 42x1 − x2 + 7x3 − x4 = −1

−2x1 + x2 −3x3 −8x4 = 3

Ans: 5555.21,7222.12,0555.13 ==−= xxx 083.01,458.02,231.03,325.04 −=−=−=−= xxxx

2.1.2GaussEliminationwithPartialPivotingwhy????• tosolveproblemthatinvolvedivisionbyzero(incase 11a =0or 22a =0)

• reducetheround-offerrors.ThealgorithmfollowstheGausseliminationmethodexcept:• Interchangerowswhenneededatthek-thstepsothatthe

absolutevalueofpivotelement kka isthelargestelementcomparetotheotherelementsunderneaththepivot.

Then,dotheeliminationprocess.Example2:UseGausseliminationmethodwithpartialpivotingtosolve:

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

9868

0015491001924300

4

3

2

1

xxxx

Solution

0 0 3 42 9 1 00 1 9 45 1 0 0

8689

⎛

⎝

⎜⎜⎜⎜

⎞

⎠

⎟⎟⎟⎟

B1↔ B4

5 1 0 02 9 1 00 1 9 40 0 3 4

9688

⎛

⎝

⎜⎜⎜⎜

⎞

⎠

⎟⎟⎟⎟

m21=25

B2 :−m21B1 +B2⎯ →⎯⎯⎯⎯⎯⎯

5 1 0 00 8.6 1 00 1 9 40 0 3 4

92.488

⎛

⎝

⎜⎜⎜⎜

⎞

⎠

⎟⎟⎟⎟

m32 =18.6

B3 :−m32B2 +B3⎯ →⎯⎯⎯⎯⎯⎯

5 1 0 00 8.6 1 00 0 8.8837 40 0 3 4

92.4

7.72098

⎛

⎝

⎜⎜⎜⎜

⎞

⎠

⎟⎟⎟⎟

m43=3

8.8837B4 :−m43B3 +B4⎯ →⎯⎯⎯⎯⎯⎯

5 1 0 00 8.6 1 00 0 8.8837 40 0 0 2.6492

92.4

7.72095.3927

⎛

⎝

⎜⎜⎜⎜

⎞

⎠

⎟⎟⎟⎟

Thenwehave:5x1 + x2 = 9

8.6 x2 + x3 = 2.48.8837 x3+ 4 x4 = 7.7209

2.6492 x4 = 5.3927

Usebackwardsubst.togetx4 = 2.0356x3 = −0.0474 or x = x̂ = (1.743, 0.285, − 0.048, 2.036)x2 = 0.2846x1 =1.7431

Exercise:SolvethefollowingsystemsusingGausseliminationmethodwithpartialpivoting.Use3decimalplaces.1.

2.

4 1 −15 1 26 1 1

⎛

⎝

⎜⎜⎜

⎞

⎠

⎟⎟⎟

x1x2x3

⎛

⎝

⎜⎜⎜⎜

⎞

⎠

⎟⎟⎟⎟=

−246

⎛

⎝

⎜⎜⎜

⎞

⎠

⎟⎟⎟

Ans:1. 2.

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

jihgfed

cba

000000

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

tsrqpunmlk

00000

0

2.2LUFACTORIZATIONMETHODForalinearsystem

Ax=bUsesubstitutionofA=LU,whereLisalowertriangularmatrix,andUisuppertriangularmatrix.

LUx=bLet

Ux=YYields

LY=bL=U=

Procedure:Step1:FromA=LU,solveforLandU.

Step2:FromLY=b,solveforYbyforwardsubstitution.

Step3:FromUx=Y,solveforxbybackwardsubstitution.

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

1010010001

ihged

b

2.2.1DoolittleMethod

A=LU

Diagonalelementformatrix 1=L

Ax=b

objectiveàtogetthevalueof x Step:1. LUA = ,findthematrixfor L andU 2. bLy = ,solvefor y usebackwardandforward substitution3. yUx = ,solvefor x

13425146228923

321

321

321

−=−+

=+−

=++

xxxxxxxxx

08.0226.65.03

5.45.132

2134

32

21

1

=−−++

−=+−

=+

=

xxxxxx

xxx

Example3:SolvethisequationsystemusingDoolittlemethod.Dothecalculationin4decimalplaces.Example4:SolvethisequationsystemusingDoolittlemethod.

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

1000100

101

sqpnml

bAx =

2.2.2CroutMethod

LUA =

Diagonalelementformatrix 1=U objectiveàtofindthevalueof x Step:1. LUA = ,determine L andU

2. bLy = ,solvefor y useforwardandbackward

3. yUx = ,solvefor x substitutions

13425146228923

321

321

321

−=−+

=+−

=++

xxxxxxxxx

Example5:SolvethislinearsystemusingCroutmethod.

bAx =

bAx =

2.2.3ThomasMethod Checkthisfirst!MakesurethatmatrixAmustbeTridiagonalMatrix

d1 e1 0 ! 0c2 d2 e2 ! "

0 ! ! ! 0" ! cn−1 dn−1 en−10 0 0 cn dn

⎡

⎣

⎢⎢⎢⎢⎢⎢⎢

⎤

⎦

⎥⎥⎥⎥⎥⎥⎥

Ifnotàrearrangethematrix

L =

α1 0 0 0 0c2 α2 0 0 00 c3 α3 0 00 0 ! ! 00 0 0 cn αn

⎡

⎣

⎢⎢⎢⎢⎢⎢⎢

⎤

⎦

⎥⎥⎥⎥⎥⎥⎥

U =

1 β1 0 0 00 1 β2 0 00 0 1 β3 00 0 ! ! βn−10 0 0 0 1

⎡

⎣

⎢⎢⎢⎢⎢⎢⎢

⎤

⎦

⎥⎥⎥⎥⎥⎥⎥

objectiveàtofindthevalueof x

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

−

−

=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

−−

112011

3200514001210031

4

3

2

1

xxxx

Step:1. LUA = ,determinethematrixfor L andU 2. bLw= ,solvefor y useforwardandbackward substitutions3. wUx = ,solvefor x Example6:SolvethislinearsystemequationusingThomasmethod.

2.2.4ThomasAlgorithmGeneralizationofThomasmethod

Ø suitabletosolvelargesystemØ Easyforprogrammingcoding

Remember!AlwayscheckwhetherMatrixAistridiagonalmatrixornot…Checkthisfirst!

d1 e1 0 0 0c2 d2 e2 0 00 c3 d3 e3 00 0 ! ! en−10 0 0 cn dn

⎡

⎣

⎢⎢⎢⎢⎢⎢⎢

⎤

⎦

⎥⎥⎥⎥⎥⎥⎥

Ifnotàrearrangethematrix

d1 e1 0 0 0c2 d2 e2 0 00 c3 d3 e3 00 0 ! ! en−10 0 0 cn dn

⎡

⎣

⎢⎢⎢⎢⎢⎢⎢

⎤

⎦

⎥⎥⎥⎥⎥⎥⎥

x1x2x3"xn

⎡

⎣

⎢⎢⎢⎢⎢⎢⎢

⎤

⎦

⎥⎥⎥⎥⎥⎥⎥

=

b1b2b3"bn

⎡

⎣

⎢⎢⎢⎢⎢⎢⎢

⎤

⎦

⎥⎥⎥⎥⎥⎥⎥

ei

id ic

Formula:Thomas’sAlgorithm:

11 d=α

1−−= iiii cd βα , ni ...,,3,2=

i

iieα

β = , 1...,,3,2,1 −= ni

111 /αby = yi = (bi − ciyi−1) /αi , ni ...,,3,2=

xn = yn xi = yi −βi xi+1 , 1...,,2,1 −−= nni

Table:

i 1 2 … n

id

ie

ic

ib

iα

iβ

iy

ix

92849692

843

21

432

321

43

=+

=++

=++

=+

xxxxxxxx

xx

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

−

−

=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

−−

112011

3200514001210031

4

3

2

1

xxxx

Example7:SolvethislinearsystemusingThomasmethod.Example8:Given(i) Whatconditionthatneedstobefulfilledbeforeusing

Thomas’sAlgorithm?(ii) Ifthesystemabovesatisfiedthecondition,solvethe

equationsystem.

bAx =

bAx =

/

2.2.5CholeskyMethodMatrixAmustbesymmetricpositive-definite

Definition Rules(theorem)

0,0 ≠∀> xAxxT 1. 0≠A

2. niaii ,,2,1,0 …=∀>

3. iikj aani

njnk

maxmax1

11 ≤≤

≤≤≤≤

≤

4. njiji

jjaiiaija ,,2,1,,)( 2 …=∀≠

<

LUA =

where TLU =

Targetàtofindthevalueof x Step:1. LUA = ,determine L andU 2. bLy = ,solvefor y useforwardandbackward substitutions3. yUx = ,solvefor x

153637

74

321

321

321

=++

=++−

=+−

xxxxxx

xxx

Example9:ShowthatmatrixAforthefollowinglinearsystemissymmetricpositive-definitebyusingdefinition.Then,solvethesystemoflinearequationsbyCholeskyMethod.

2.3ITERATIVEMETHOD

IsAStrictlyDiagonallyDominantMatrix?

∑≠=

>n

ijj

ijii aa1

ifnotàrearrangerows.e.g.

Showthat⎥⎥⎦

⎤

⎢⎢⎣

⎡−

721152023

isaSDDmatrix.

Solution:

127:3

125:2

023:1

+>

+>−

+>

B

B

B

Mustsatisfyconvergencecriterioni.e.Amustbestrictlydiagonallydominantmatrix

2.3.1JacobiMethodFormula:

33

)(232

)(1313)1(

3

22

)(323

)(1212)1(

2

11

)(313

)(2121)1(

1

axaxabx

axaxabx

axaxabx

kkk

kkk

kkk

−−=

−−=

−−=

+

+

+

Initialguess:

Tx )000()0( …= Stoptheiterationwhen:

{ } ε<−−≤≤

)1()(1

kix

kix

nimax

foragivenvalueofε andtake)(kxx ≈

Example10:SolvethefollowinglinearsystemusingJacobimethod.Set 0x =)0( .Take 05.0=ε .

192343112

3271

321

3231

=+

=−+

=+

+

−

xxxxxxxxx

SolutionIsmatrixASDD?Rearrangerows:

311219234

3231

3271

321

=+

=+

=−+

−

+

xxxxxxxxx

Theiterationformula:

1231

72194

3

)(2

)(1)1(

3

)(3

)(1)1(

2

)(3

)(2)1(

1

kkk

kkk

kkk

xxx

xxx

xxx

+−=

−−=

+−=

+

+

+

k )(

1kx )(

2kx )(

3kx )1(

1)(

1−− kk xx )1(

2)(

2−− kk xx )1(

3)(

3−− kk xx

0 0 0 0 1 2 3 4 5

{ }ε<==

−=−≤≤

01.0}01.0,01.0,01.0{max

)4()5(

31

)4()5( max iii

xxxx

)00.3,00.2,01.1()5( =≈∴ xx

2.3.2GaussSeidelMethodFormula:

33

)1(232

)1(1313)1(

3

22

)(323

)1(1212)1(

2

11

)(313

)(2121)1(

1

axaxabx

axaxabx

axaxabx

kkk

kkk

kkk

+++

++

+

−−=

−−=

−−=

Initialguess:Tx )000()0( …=

Stoptheiterationwhen:

{ } ε<−+

≤≤

)()1(1

kix

kix

nimax

foragivenvalueofε andtake)(kxx ≈

Example11:SolvethefollowinglinearsystemusingGauss-Seidelmethod.Set 0x =)0( .Take 05.0=ε .

192343112

3271

321

3231

=+

=−+

=+

+

−

xxxxxxxxx

SolutionIsmatrixASDD?Rearrangerows:

311219234

3231

3271

321

=+

=+

=−+

−

+

xxxxxxxxx

Theiterationformula:

1231

72194

3

)1(2

)1(1)1(

3

)(3

)1(1)1(

2

)(3

)(2)1(

1

+++

++

+

+−=

−−=

+−=

kkk

kkk

kkk

xxx

xxx

xxx

k )(

1kx )(

2kx )(

3kx )1(

1)(

1−− kk xx )1(

2)(

2−− kk xx )1(

3)(

3−− kk xx

0 0 0 0 1 2 3 4

{ }ε<==

−=−≤≤

00.0}00.0,00.0,00.0{max

)3()4(

31

)3()4( max iii

xxxx

)00.3,00.2,00.1()5( =≈∴ xx

Exercise:1.WritetheGauss-Seidelformulafor

2171782115

321

321

321

=++−

=−+

=+−

xxxxxxxxx

Thenfindthevalueofx1,x2,andx3.Dothecalculationin3decimalplaces.2.SolvethefollowingsystemusingGauss–Seidelmethodandstoptheiterationwhen 0005.0)1()( <−

∞

−kk xx .

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

−=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

−

−−

−−

−

1511256

8130110123111102110

4

3

2

1

xxxx