Chap. 9 Gauss Elimination

Chapter 9Chapter 9

Gauss EliminationGauss Elimination


9.1 Solving small numbers of equations 9.2 Naive Gauss Elimination 9.3 Pivoting 9.4 Tridiagonal Systems

MATLAB M-files GaussNaive, GaussPivot, Tridiag

Small MatricesSmall Matrices

For small numbers of equations, can be solved by hand

Graphical Cramer's rule Elimination

nnnn22n11n

2nn2222121

1nn1212111

bxaxaxa

bxaxaxab xaxaxa

Graphical MethodGraphical Method

12

12

21

21

x3x3x2x rearrange 3xx

3xx2

2x1 – x2 = 3x1 + x2 = 3

One solution


2x1 – x2 = 3

2x1 – x2 = – 1

No solution


6x1 – 3x2 = 92x1 – x2 = 3

Infinite many solution


2x1 – x2 = 3

2.1x1 – x2 = 3

Ill conditioned

Cramer’s RuleCramer’s RuleCompute the determinant D2 x 2 matrix

3 x 3 matrix 21122211

2221

1211 aaaaaaaa

D

3231

222113

3331

232112

3332

232211

333231

232221

131211

aaaa

aaaaa

aaaaa

a

aaaaaaaaa

D

Cramer’s RuleCramer’s RuleTo find xk for the following system

Replace kth column of as with bs (i.e., aik bi ) nnnn22n11n

2nn2222121

1nn1212111

bxa...xaxa

bxa...xaxabxa...xaxa

))(

ijk D(a

matrix newDx

ExampleExample3 x 3 matrix

333231

232221

131211

aaaaaaaaa

D

33231

22221

112113

3

33331

23221

131112

2

33323

23222

131211

1

baabaabaa

D1

DDx

abaabaaba

D1

DDx

aabaabaab

D1

DDx

Ill-Conditioned SystemIll-Conditioned System

What happen if the determinant D is very small or zero?

Divided by zero (linearly dependent system)Divided by a small number: Round-off errorLoss of significant digits

0AdetD

Eliminate x2

Subtract to get

2222121

1212111

bxaxabxaxa

2122221212112

1222122211122

baxaaxaabaxaaxaa

aaaababax

aaaababax

babaxaaxaa

21122211

1212112

21121122

2121221

2121221211211122

Elimination MethodElimination Method

Not very practical for large number (> 4) of equations

MATLAB’s MethodsMATLAB’s Methods

Forward slash ( / )Back-slash ( \ )Multiplication by the inverse of the

quantity under the slash

b*AinvxbAxbAx

bAx1

)(\

Gauss EliminationGauss EliminationManipulate equations to eliminate one of the unknownsDevelop algorithm to do this repeatedlyThe goal is to set up upper triangular matrix

Back substitution to find solution (root)

nn

n333

n22322

n1131211

a

aaaaaaaaa

U

Basic Gauss EliminationBasic Gauss Elimination

Direct Method (no iteration required)Forward eliminationColumn-by-column elimination of the

below-diagonal elementsReduce to upper triangular matrixBack-substitution

Naive Gauss EliminationNaive Gauss EliminationBegin with

Multiply the first equation by a21 / a11 and subtract from second equation

nnnn22n11n

2nn2222121

1nn1212111

bxa...xaxa

bxa...xaxabxa...xaxa

nnnn22n11n

111

212nn1

11

21n2212

11

2122111

11

2121

1nn212111

bxa...xaxa

baabxa

aaa...xa

aaaxa

aaa

bxa...xaxa

Gauss EliminationGauss EliminationReduce to

Repeat the forward elimination to get

nnnn22n11n

2nn2222

1nn1212111

bxa...xaxa

bxa...xa bxa...xaxa

nnnn22n

2nn2222

1nn1212111

bxa...xa

bxa...xa bxa...xaxa


First equation is pivot equationa11 is pivot elementNow multiply second equation by a'32 /a'22

and subtract from third equation

222

323nn2

22

32n3323

22

3233

2nn2323222

1nn1313212111

baabxa

aaaxa

aaa

bxaxaxabxaxaxaxa

Gauss EliminationGauss EliminationRepeat the elimination of ai2 and get

Continue and get nnnn33n

3nn3333

2nn2323222

1nn1313212111

bxaxa

bxaxabxaxaxabxaxaxaxa

)()( 1nnn

1nnn

3nn3333

2nn2323222

1nn1313212111

bxa

bxaxabxaxaxabxaxaxaxa

Now we can perform back substitution to get {x}By simple division

Substitute this into (n-1)th equation

Solve for xn-1

Repeat the process to solve for xn-2 , xn-3 , …. x2, x1

)()(,

)(,

2n1nn

2nn1n1n

2n1n1n bxaxa

Back SubstitutionBack Substitution

)(

)(

1nnn

1nn

n abx

Back substitution: starting with xn Solve for xn1 , xn2 , … , 3, 2, 1

Back SubstitutionBack Substitution

a

xabx

abx

1iii

n

1ijj

1iij

1ii

i

1nnn

1nn

n

)(

)()(

)(

)(

for i = n1, n2, …, 1

0a 1iii )( Naive Gauss EliminationNaive Gauss Elimination

Elimination of first columnElimination of first column

)()()()()()(

1f41f31f2

baaa0baaa0baaa0baaaa

41

31

21

4444342

3343332

2242322

114131211

114141

113131

112121

444434241

334333231

224232221

114131211

aafaafaaf

baaaabaaaabaaaabaaaa

///

Elimination of second columnElimination of second column

)()()()(

//

2f42f3

baa00baa00baaa0baaaa

aafaaf

baaa0baaa0baaa0baaaa

42

32

44443

33433

2242322

114131211

224242

223232

4444342

3343332

2242322

114131211

Elimination of third columnElimination of third column

Upper triangular matrix

)()(

/

3f4

ba000baa00baaa0baaaa

aaf

aaa00aaa00baaa0baaaa

43444

33433

2242322

114131211

33434344443

33433

2242322

114131211

1141431321211

2242432322

3343433

4444

a/)xaxaxab(xa/)xaxab(x

a/)xab(xa/bx

Back-SubstitutionBack-Substitution

Upper triangular matrix

0a,a,a,a 44332211

ba000baa00baaa0baaaa

444

33433

2242322

114131211

ExampleExample

41

31

21

41

31

21

f14f13f12

5141020241100042013201

6f0f

1f

14226241101322113201

)()()()()()(

Forward EliminationForward Elimination

42

32

42

32

f24f23

5141400241000042013201

1f1/2f

5141020241100042013201

)()()()(

Upper Triangular MatrixUpper Triangular Matrix

43

43

f34

3370000241000042013201

14f

5141420241100042013201

)()(


13/70x3x21x

8/35x2x

4/352x4x

33/707033/x

431

32

43

4

33/704/358/35

13/70

x

3370000241000042013201

MATLAB Script File: MATLAB Script File: GaussNaiveGaussNaive

Print all factor and Aug(do not suppress output)

Eliminate first column

Eliminate second column

Eliminate third column Back-substitution

Aug = [A, b]

>> format short>> x = GaussNaive(A,b)m = 4n = 4Aug = 1 0 2 3 1 -1 2 2 -3 -1 0 1 1 4 2 6 2 2 4 1factor = -1Aug = 1 0 2 3 1 0 2 4 0 0 0 1 1 4 2 6 2 2 4 1factor = 0Aug = 1 0 2 3 1 0 2 4 0 0 0 1 1 4 2 6 2 2 4 1factor = 6Aug = 1 0 2 3 1 0 2 4 0 0 0 1 1 4 2 0 2 -10 -14 -5

factor = 0.5000Aug = 1 0 2 3 1 0 2 4 0 0 0 0 -1 4 2 0 2 -10 -14 -5factor = 1Aug = 1 0 2 3 1 0 2 4 0 0 0 0 -1 4 2 0 0 -14 -14 -5

factor = 14Aug = 1 0 2 3 1 0 2 4 0 0 0 0 -1 4 2 0 0 0 -70 -33

x4

x3

x2

x1

x = 0 0 0 0.4714x = 0 0 -0.1143 0.4714x = 0 0.2286 -0.1143 0.4714x = -0.1857 0.2286 -0.1143 0.4714

Algorithm for Gauss eliminationAlgorithm for Gauss elimination1. Forward elimination for each equation j, j = 1 to n-1

for all equations k greater than j (a) multiply equation j by akj /ajj

(b) subtract the result from equation kThis leads to an upper triangular matrix2. Back-Substitution

(a) determine xn from (b) put xn into (n-1)th equation, solve for xn-1

(c) repeat from (b), moving back to n-2, n-3, etc. until all equations are solved

)1n(nn

)1n(nn a/bx

Operation CountOperation Count

Important as matrix gets large

For Gauss elimination

Elimination routine uses on the order of O(n3/3) operations

Back-substitution uses O(n2/2)

Operation CountOperation Count

))(())((,

))(())((,

))(())((,))(())((,

3121nn1n

2knkn1knknn1kk

n2n1n2nn321n1nn1nn21

flopsiontion/DivisMultiplica

flopsubtractionAddition/S

iLoop Inner

kLoop Outer

Total operation counts for elimination stage = 2n3/3 + O(n2)Total operation counts for back substitution stage = n2 + O(n)

Operation CountOperation Count Number of flops (floating-point operations) for

Naive Gauss elimination

%....%.%.

859910676106861000000106761000539866666768155010000671550100588766780510070510

nEliminatio toDue Percentage

32n

FlopsTotal

onSbustitutiBack

nElimination

868

3

Computation time increase rapidly with n Most of the effort is incurred in the elimination step Improve efficiency by reducing the elimination effort

Partial PivotingPartial Pivoting

Problems with Gauss elimination division by zero round off errors ill conditioned systems

Use “Pivoting” to avoid this Find the row with largest absolute coefficient below the

pivot element Switch rows (“partial pivoting”) complete pivoting switch columns also (rarely used)

Round-off ErrorsRound-off ErrorsA lot of chopping with more than n3/3 operations

More important - error is propagated

For large systems (more than 100 equations), round-off error can be very important (machine dependent)

Ill conditioned systems - small changes in coefficients lead to large changes in solution

Round-off errors are especially important for ill-conditioned systems

Ill-conditioned SystemIll-conditioned System

2x1 – x2 = 3

2.1x1 – x2 = 3

Consider

Since slopes are almost equal

22

21

22

212

12

11

12

112

2222121

1212111

abx

aax

abx

aax

bxaxabxaxa

22

21

12

11

aa

aa

Ill-Conditioned SystemIll-Conditioned System

0aaaa

D2221

1211 Divided by small number

)1n(nn

n333

n22322

n1131211

a

aaaaaaaaa

U

)(detdet 1nnn332211 aaaaUA

DeterminantDeterminant

Calculate determinant using Gauss elimination

Gauss Elimination with Partial PivotingGauss Elimination with Partial Pivoting

Forward elimination

for each equation j, j = 1 to n-1

first scale each equation k greater than j then pivot (switch rows) Now perform the elimination

(a) multiply equation j by akj /ajj

(b) subtract the result from equation

Partial (Row) PivotingPartial (Row) Pivoting

1x4x2x2x62x4xx1x3x2x2x1 x3x2x

4321

432

4321

431

14226241101322113201

bA

Forward EliminationForward EliminationInterchange rows 1 & 4

41

31

21

41

31

21

f14f13f12

5/67/35/31/3024110

5/67/37/37/3014226

1/6f0f

1/6f

13201241101322114226

)()()()()()(

Forward EliminationForward Elimination

No interchange required

42

32

42

32

f24f23

5/7220033/145000

5/67/37/37/3014226

1/7f3/7f

5/67/35/31/3024110

5/67/37/37/3014226

)()()()(


Interchange rows 3 & 4

13/70/6x 2x 2x 41x8/357/3/x 7/3x 7/35/6x

4/35/2x 25/7x33/70/533/14x

2341

342

43

4

)()()(

)()(

33/704/358/35

13/70

x

0f

33/1450005/722005/67/37/37/30

14226

43

MATLAB M-File: MATLAB M-File: GaussPivotGaussPivot

Partial Pivoting (switch rows)

[big,i] = max(x)

largest element in {x}

index of the largest element

Partial Pivoting

>> format short>> x=GaussPivot0(A,b)Aug = 1 0 2 3 1 -1 2 2 -3 -1 0 1 1 4 2 6 2 2 4 1big = 6i = 4ipr = 4Aug = 6 2 2 4 1 -1 2 2 -3 -1 0 1 1 4 2 1 0 2 3 1factor = -0.1667Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 1.0000 1.0000 4.0000 2.0000 1.0000 0 2.0000 3.0000 1.0000factor = 0Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 1.0000 1.0000 4.0000 2.0000 1.0000 0 2.0000 3.0000 1.0000factor = 0.1667Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 1.0000 1.0000 4.0000 2.0000 0 -0.3333 1.6667 2.3333 0.8333

Aug = [A b]

Interchange rows 1 and 4

Find the first pivot element and its index

Eliminate first columnNo need to interchange

big = 2.3333i = 1ipr = 2factor = 0.4286Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 0 0 5.0000 2.3571 0 -0.3333 1.6667 2.3333 0.8333factor = -0.1429Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 0 0 5.0000 2.3571 0 0 2.0000 2.0000 0.7143big = 2i = 2ipr = 4Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 0 2.0000 2.0000 0.7143 0 0 0 5.0000 2.3571factor = 0Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 0 2.0000 2.0000 0.7143 0 0 0 5.0000 2.3571

x = 0 0 0 0.4714x = 0 0 -0.1143 0.4714x = 0 0.2286 -0.1143 0.4714x =

-0.1857 0.2286 -0.1143 0.4714

Back substitutionSecond pivot element and index

Third pivot element and index

Eliminate second column

Eliminate third column

Interchange rows 3 and 4

No need to interchange

Save factors fij for

LU Decomposition

1

4

2 3

5

F14

F23F12

F24

F45

H1

F35

F25

V3V1

TRUSSTRUSS

W = 100 kg

Statics: Force BalanceStatics: Force Balance

0FFFF

0FFF

0FFFF

0FFF

0FFF

0FVF

0FFFFF

100FFF

0FFHF

0FVF

4535255x

35255y

4524144x

24144y

35233x

3533y

252423122x

25242y

141211x

1411y

coscos

sinsin

coscos

sinsin

cos

sin

coscos

sinsin

sin

sin

,

,

,

,

,

,

,

,

,

,

Node 1

Node 2

Node 3

Node 4

Node 5

Example: Forces in a Simple TrussExample: Forces in a Simple Truss

0000000

10000

FFFFFFFVHV

1coscos00000000sinsin0000000100cos0cos0000000sin0sin00000cos001000000sin0000010000coscos10100000sinsin00000000000cos101000000sin0001

45

35

25

24

23

14

12

3

1

1

function [A,b]=Truss(alpha,beta,gamma,delta)

A=zeros(10,10);A(1,1)=1; A(1,5)=sin(alpha);A(2,2)=1; A(2,4)=1; A(2,5)=cos(alpha);A(3,7)=sin(beta); A(3,8)=sin(gamma);A(4,4)=-1; A(4,6)=1; A(4,7)=-cos(beta); A(4,8)=cos(gamma);A(5,3)=1; A(5,9)=sin(gamma);A(6,6)=-1; A(6,9)=-cos(delta);A(7,5)=-sin(alpha); A(7,7)=-sin(beta);A(8,5)=-cos(alpha); A(8,7)=cos(beta); A(8,10)=1;A(9,8)=-sin(gamma); A(9,9)=-sin(delta);A(10,8)=-cos(gamma); A(10,9)=cos(delta); A(10,10)=-1;

b=zeros(10,1); b(3,1)=100;

Define Matrices A and b in script fileDefine Matrices A and b in script file

>> alpha=pi/6; beta=pi/3; gamma=pi/4; delta=pi/3;>> [A,b] = Truss(alpha,beta,gamma,delta)A = 1.0000 0 0 0 0.5000 0 0 0 0 0 0 1.0000 0 1.0000 0.8660 0 0 0 0 0 0 0 0 0 0 0 0.8660 0.7071 0 0 0 0 0 -1.0000 0 1.0000 -0.5000 0.7071 0 0 0 0 1.0000 0 0 0 0 0 0.7071 0 0 0 0 0 0 -1.0000 0 0 -0.5000 0 0 0 0 0 -0.5000 0 -0.8660 0 0 0 0 0 0 0 -0.8660 0 0.5000 0 0 1.0000 0 0 0 0 0 0 0 -0.7071 -0.8660 0 0 0 0 0 0 0 0 -0.7071 0.5000 -1.0000b = 0 0 100 0 0 0 0 0 0 0>> x = GaussPivot(A,b)x = 40.5827 0 48.5140 70.2914 -81.1655 34.3046 46.8609 84.0287 -68.6091 -93.7218

Gauss Elimination with Partial PivotingGauss Elimination with Partial Pivoting

Simple truss

Banded MatrixBanded Matrix

HBW: Half Band Width

•Banded Matrix

xwvut

srqpzk

nmlkji

hgfedc

ba

0000000000000

0000000000000000000000000000000000000000000

ai,j= 0if j > i + HBor j< i - HB

HB: Half BandwidthB: Bandwidth

B = 2*HB + 1

In this exampleHB = 1 & B = 3

Tridiagonal MatrixTridiagonal Matrix

Only three nonzero elements in each equation (3n instead of n2 elements)

Subdiagonal, diagonal, superdiagonal

Row Scaling (not implemented in textbook) -- scale the diagonal element to aii = 1Solve by Gauss elimination

n

1n

i

3

2

1

n

1n

i

3

2

1

nn

1n1n1n

iii

333

222

11

rr

r

rrr

xx

x

xxx

fegfe

gfe

gfegfe

gf

Tridiagonal MatrixTridiagonal Matrix

Special case of banded matrix with bandwidth = 3 Save storage, 3 n instead of n n

Tridiagonal MatrixTridiagonal Matrix Forward elimination

Back substitution

n32k r

ferr

gfeff

1k1k

kkk

1k1k

kkk

,,,

1232n1nk f

xgrx

frx

k

1kkkk

n

nn

,,,,,

Use factor = ek / fk1

to eliminate subdiagonal element

Apply the same matrix operations to right hand side

Hand Calculations: Tridiagonal MatrixHand Calculations: Tridiagonal Matrix

53253

xxxx

25150005021001520021

4

3

2

1

....

411

5053rferr

1501

50251gfeff

11112r

ferr

11112g

feff

13125r

ferr

12125g

feff

33

444

33

444

11

223

22

333

11

222

11

222

)(..

).(..

)(

)(

)(

)(

11

223f

xgrx

21

311f

xgrx

31

4501f

xgrx

414

frx

1

2111

2

3222

3

4333

4

44

))((

))((

))(.(

(a) Forward elimination (b) Back substitution

MATLAB M-file:MATLAB M-file: Tridiag Tridiag

» [e,f,g,r] = example

e = 0 -2.0000 4.0000 -0.5000 1.5000 -3.0000f = 1.0000 6.0000 9.0000 3.2500 1.7500 13.0000g = -2.0000 4.0000 -0.5000 1.5000 -3.0000 0r = -3.0000 22.0000 35.5000 -7.7500 4.0000 -33.0000

» x = Tridiag (e, f, g, r)x = 1 2 3 -1 -2 -3

Example: Tridiagonal matrixExample: Tridiagonal matrixfunction [e,f,g,r] = example

e=[ 0 -2 4 -0.5 1.5 -3];f=[ 1 6 9 3.25 1.75 13];g=[-2 4 -0.5 1.5 -3 0];r=[-3 22 35.5 -7.75 4 -33];

334

75.75.35

223

xxxxxx

133375.15.1

5.125.35.05.094

46221

6

5

4

3

2

1

Note: e(1) = 0 and g(n) = 0

CVEN 302-501CVEN 302-501Homework No. 6Homework No. 6

Chapter 8Problem 8.2 (20), 8.3(10), 8.7(15)Chapter 9Problem 9.5 (15), 9.8(20), 9.11(20).

Due Monday 10/06/08 at the beginning Due Monday 10/06/08 at the beginning of the periodof the period

Documents

Chap. 9 Gauss Elimination