70
1 Gauss Elimination Gauss Elimination

1 Gauss Elimination Small Matrices For small numbers of equations, solvable by hand Graphical Cramer's rule Elimination 2

Embed Size (px)

Citation preview

Page 1: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

1

Gauss EliminationGauss Elimination

Page 2: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Small Matrices

For small numbers of equations, solvable by hand Graphical Cramer's rule Elimination

2

nnnn22n11n

2nn2222121

1nn1212111

bxaxaxa

bxaxaxa

b xaxaxa

Page 3: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

3

12

12

21

21

x3x3x2x

rearrange 3xx

3xx2

2x1 – x2 = 3x1 + x2 = 3

One solution

Graphical Method

Page 4: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

2x1 – x2 = 3

2x1 – x2 = – 1

No solution

Graphical Method

Page 5: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

6x1 – 3x2 = 92x1 – x2 = 3

Infinite many solutions

Graphical Method

Page 6: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

2x1 – x2 = 3

2.1x1 – x2 = 3

ill conditioned

Graphical Method 方程式斜率非常接近,視覺上無法判定交點

Page 7: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Cramer’s Rule: Determinant

Compute the determinant D ( 行列式 ) 2 x 2 matrix

3 x 3 matrix

7

211222112221

1211 aaaaaa

aaD

3231

222113

3331

232112

3332

232211

333231

232221

131211

aa

aaa

aa

aaa

aa

aaa

aaa

aaa

aaa

D

Page 8: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Cramer’s Rule

To find xk for the following system

Replace kth column of as with bs (i.e., aik bi )

8

nnnn22n11n

2nn2222121

1nn1212111

bxa...xaxa

bxa...xaxabxa...xaxa

)

)(

ijk D(a

matrix newDx

Page 9: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Cramer’s Rule :例

3x3 matrix

9

333231

232221

131211

aaa

aaa

aaa

D

33231

22221

11211

33

33331

23221

13111

22

33323

23222

13121

11

baa

baa

baa

D

1

D

Dx

aba

aba

aba

D

1

D

Dx

aab

aab

aab

D

1

D

Dx

Page 10: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

ILL-Conditioned System

What happen if the determinant D is very small or zero?

Divided by zero (linearly dependent system) Divided by a small number: Round-off error Loss of significant digits

10

0AdetD

Page 11: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Eliminate x2

Subtract to get

2222121

1212111

bxaxa

bxaxa

2122221212112

1222122211122

baxaaxaa

baxaaxaa

aaaa

babax

aaaa

babax

babaxaaxaa

21122211

1212112

21121122

2121221

2121221211211122

Not practical for large number (> 4) of equations

Elimination Method

Page 12: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

MATLAB’s Methods

Forward slash ( / ) Back-slash ( \ ) Multiplication by the inverse of the quantity under

the slash

12

b*Ainvx

bAxbAx

bAx1

)(

\

Page 13: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Gauss Elimination

Manipulate equations to remove one of the unknowns Develop algorithm to do this repeatedly The goal is to set up upper triangular matrix

Back substitution to find solution (root)13

nn

n333

n22322

n1131211

a

aa

aaa

aaaa

U

Page 14: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Basic Gauss Elimination

Direct method (no iteration required) Forward elimination Column-by-column elimination of the below-

diagonal elements Reduce to upper triangular matrix Back-substitution

14

Page 15: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Naive Gauss Elimination ( 單純高斯消去法 )

Begin with

Multiply the first equation by a21 / a11 and subtract from second equation

15

nnnn22n11n

2nn2222121

1nn1212111

bxa...xaxa

bxa...xaxa

bxa...xaxa

nnnn22n11n

111

212nn1

11

21n2212

11

2122111

11

2121

1nn212111

bxa...xaxa

ba

abxa

a

aa...xa

a

aaxa

a

aa

bxa...xaxa

Page 16: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Forward Elimination

Reduce to

Repeat the forward elimination to get

16

nnnn22n11n

2nn2222

1nn1212111

bxa...xaxa

bxa...xa

bxa...xaxa

nnnn22n

2nn2222

1nn1212111

bxa...xa

bxa...xa

bxa...xaxa

Page 17: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Forward Elimination

First equation is pivot equation ( 軸元方程式 ) a11 is pivot element ( 軸元元素 ) Now multiply second equation by a'32 /a'22 and

subtract from third equation

17

222

323nn2

22

32n3323

22

3233

2nn2323222

1nn1313212111

ba

abxa

a

aaxa

a

aa

bxaxaxa

bxaxaxaxa

Page 18: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Forward Elimination

Repeat the elimination of ai2 and get

Continue and get

18

nnnn33n

3nn3333

2nn2323222

1nn1313212111

bxaxa

bxaxa

bxaxaxa

bxaxaxaxa

)()( 1nnn

1nnn

3nn3333

2nn2323222

1nn1313212111

bxa

bxaxa

bxaxaxa

bxaxaxaxa

Page 19: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Back Substitution ( 向後代換 )

Now we can perform back substitution to get {x} By simple division

Substitute this into (n-1)th equation

Solve for xn-1

Repeat the process to solve for xn-2 , xn-3 , …. x2, x1

19

)()(,

)(,

2n1nn

2nn1n1n

2n1n1n bxaxa

)(

)(

1nnn

1nn

n a

bx

Page 20: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Back Substitution

Back substitution: starting with xn

Solve for xn1 , xn2 , … , x3, x2, x1

20

a

xab

x

a

bx

1iii

n

1ijj

1iij

1ii

i

1nnn

1nn

n

)(

)()(

)(

)(

for i = n1, n2, …, 1

0a 1iii )(

Naive Gauss EliminationNaive Gauss Elimination

Page 21: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

)()(

)()(

)()(

1f4

1f3

1f2

baaa0

baaa0

baaa0

baaaa

41

31

21

4444342

3343332

2242322

114131211

114141

113131

112121

444434241

334333231

224232221

114131211

aaf

aaf

aaf

baaaa

baaaa

baaaa

baaaa

/

/

/

Elimination of First Column

Page 22: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

)()(

)()(

/

/

2f4

2f3

baa00

baa00

baaa0

baaaa

aaf

aaf

baaa0

baaa0

baaa0

baaaa

42

32

44443

33433

2242322

114131211

224242

223232

4444342

3343332

2242322

114131211

Elimination of Second Column

Page 23: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Upper triangular matrix

)()(

/

3f4

ba000

baa00

baaa0

baaaa

aaf

aaa00

aaa00

baaa0

baaaa

43444

33433

2242322

114131211

33434344443

33433

2242322

114131211

Elimination of Third Column

Page 24: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

1141431321211

2242432322

3343433

4444

a/)xaxaxab(x

a/)xaxab(x

a/)xab(x

a/bx

Upper triangular matrix

0a,a,a,a 44332211

ba000

baa00

baaa0

baaaa

444

33433

2242322

114131211

Back Substitution

Page 25: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

41

31

21

41

31

21

f14

f13

f12

5141020

24110

00420

13201

6f

0f

1f

14226

24110

13221

13201

)()(

)()(

)()(

Example

Page 26: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

42

32

42

32

f24

f23

5141400

24100

00420

13201

1f

1/2f

5141020

24110

00420

13201

)()(

)()(

Forward Elimination

Page 27: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

43

43

f34

3370000

24100

00420

13201

14f

5141420

24110

00420

13201

)()(

Upper Triangular Matrix

Page 28: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

13/70x3x21x

8/35x2x

4/352x4x

33/707033/x

431

32

43

4

33/70

4/35

8/35

13/70

x

3370000

24100

00420

13201

Back Substitution

Page 29: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

29

function x = GaussNaive(A,b)% x = GaussNaive(A,b):% Gauss elimination without pivoting.% input:% A = coefficient matrix% b = right hand side vector% output:% x = solution vector[m,n] = size(A);if m~=n, error('Matrix A must be square'); endnb = n+1;Aug = [A b];% forward eliminationfor k = 1:n-1 for i = k+1:n factor = Aug(i,k)/Aug(k,k); Aug(i,k:nb) = Aug(i,k:nb)-factor*Aug(k,k:nb); end disp(Aug);end% back substitutionx = zeros(n,1);x(n) = Aug(n,nb)/Aug(n,n);for i = n-1:-1:1 x(i) = (Aug(i,nb)-Aug(i,i+1:n)*x(i+1:n))/Aug(i,i);end

Page 30: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Print all factor and Aug

(do not suppress output)Eliminate first column

Eliminate second column

Eliminate third column Back-substitution

Aug = [A, b]

>> format short>> x = GaussNaive(A,b)m = 4n = 4Aug = 1 0 2 3 1 -1 2 2 -3 -1 0 1 1 4 2 6 2 2 4 1factor = -1Aug = 1 0 2 3 1 0 2 4 0 0 0 1 1 4 2 6 2 2 4 1factor = 0Aug = 1 0 2 3 1 0 2 4 0 0 0 1 1 4 2 6 2 2 4 1factor = 6Aug = 1 0 2 3 1 0 2 4 0 0 0 1 1 4 2 0 2 -10 -14 -5

factor = 0.5000Aug = 1 0 2 3 1 0 2 4 0 0 0 0 -1 4 2 0 2 -10 -14 -5factor = 1Aug = 1 0 2 3 1 0 2 4 0 0 0 0 -1 4 2 0 0 -14 -14 -5

factor = 14Aug = 1 0 2 3 1 0 2 4 0 0 0 0 -1 4 2 0 0 0 -70 -33

x4

x3

x2

x1

x = 0 0 0 0.4714x = 0 0 -0.1143 0.4714x = 0 0.2286 -0.1143 0.4714x = -0.1857 0.2286 -0.1143 0.4714

Page 31: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Gauss Elimination: Algorithm

Forward elimination for each equation j, j = 1 to n-1

for all equations k greater than j

(a) multiply equation j by akj /ajj

(b) subtract the result from equation k This leads to an upper triangular matrix

Back substitution (a) determine xn from

(b) put xn into (n-1)th equation, solve for xn-1

(c) repeat from (b), moving back to n-2, n-3, etc. until all equations are solved

31

)1n(nn

)1n(nn a/bx

Page 32: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Operation Count For Gauss Elimination

Elimination routine on the order of O(n3/3) operations

Total operation counts for elimination stage = 2n3/3 + O(n2)

Back-substitution uses O(n2/2)

Total operation counts for back substitution stage = n2 + O(n)

))(())((,

))(())((,

))(())((,

))(())((,

3121nn1n

2knkn1knknn1kk

n2n1n2nn32

1n1nn1nn21

flops

iontion/DivisMultiplica

flops

ubtractionAddition/S

i

Loop Inner

k

Loop Outer

Page 33: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Operation Count For Gauss Elimination

#flops (floating-point operations) for Naive Gauss elimination

Computation time increases rapidly with n Most effort incurs in the elimination step Improve efficiency by reducing the elimination effort

33

%....

%.

%.

859910676106861000000106761000

539866666768155010000671550100

588766780510070510

nEliminatio to

Due Percentage

3

2nFlops

Total

onSbustituti

Back

nElimination

868

3

Page 34: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Partial Pivoting ( 部分軸元法 )

Problems with Gauss elimination division by zero round off errors ill conditioned systems

Use “pivoting” to avoid this Find the row with largest absolute coefficient below the

pivot element Switch rows (“partial pivoting”) Complete pivoting switch columns (rarely used)

34

Page 35: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Round-off Errors ( 捨入誤差 )

A lot of chopping with more than n3/3 operations More important - error is propagated For large systems (more than 100 equations),

round-off error is important (machine dependent) Ill conditioned systems - small changes in

coefficients lead to large changes in solution Round-off errors are especially important for ill-

conditioned systems

35

Page 36: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

2x1 – x2 = 3

2.1x1 – x2 = 3

Ill-conditioned System

Page 37: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Ill-conditioned System

Consider

Since slopes are almost equal

22

21

22

212

12

11

12

112

2222121

1212111

a

bx

a

ax

a

bx

a

ax

bxaxa

bxaxa

22

21

12

11

a

a

a

a

0aa

aaD

2221

1211 Divided by small number

Page 38: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Determinant

Calculate determinant using Gauss elimination

38

)1n(nn

n333

n22322

n1131211

a

aa

aaa

aaaa

U

)(detdet 1nnn332211 aaaaUA

Page 39: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Gauss Elimination with Partial Pivoting

Forward elimination for each equation j, j = 1 to n-1

• first scale each equation k greater than j• then pivot (switch rows)• Now perform the elimination

(a) multiply equation j by akj /ajj

(b) subtract the result from equation

39

Page 40: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

1x4x2x2x6

2x4xx

1x3x2x2x

1 x3x2x

4321

432

4321

431

14226

24110

13221

13201

bA

Partial (Row) Pivoting

Page 41: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Interchange rows 1 & 4

41

31

21

41

31

21

f14

f13

f12

5/67/35/31/30

24110

5/67/37/37/30

14226

1/6f

0f

1/6f

13201

24110

13221

14226

)()(

)()(

)()(

Forward Elimination

Page 42: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

No interchange required

42

32

42

32

f24

f23

5/72200

33/145000

5/67/37/37/30

14226

1/7f

3/7f

5/67/35/31/30

24110

5/67/37/37/30

14226

)()(

)()(

Forward Elimination

Page 43: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Interchange rows 3 & 4

13/70/6x 2x 2x 41x

8/357/3/x 7/3x 7/35/6x

4/35/2x 25/7x

33/70/533/14x

2341

342

43

4

)(

)()(

)(

)(

33/70

4/35

8/35

13/70

x

0f

33/145000

5/72200

5/67/37/37/30

14226

43

Back-Substitution

Page 44: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

44

function x = GaussPivot(A,b)% x = GaussPivot(A,b):% Gauss elimination without pivoting.

[m,n]=size(A);if m~=n, error('Matrix A must be square'); endnb=n+1;Aug=[A b];% forward eliminationfor k = 1:n-1 % partial pivoting [big,i]=max(abs(Aug(k:n,k))); ipr=i+k-1; if ipr~=k Aug([k,ipr],:)=Aug([ipr,k],:); end for i = k+1:n factor=Aug(i,k)/Aug(k,k); Aug(i,k:nb)=Aug(i,k:nb)-factor*Aug(k,k:nb); end disp(Aug);end% back substitutionx=zeros(n,1);x(n)=Aug(n,nb)/Aug(n,n);for i = n-1:-1:1 x(i)=(Aug(i,nb)-Aug(i,i+1:n)*x(i+1:n))/Aug(i,i);end

Partial pivoting (switch rows)

[big,i] = max(x)

largest element in {x}

index of the largest element

Page 45: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

>> format short>> x=GaussPivot0(A,b)Aug = 1 0 2 3 1 -1 2 2 -3 -1 0 1 1 4 2 6 2 2 4 1big = 6i = 4ipr = 4Aug = 6 2 2 4 1 -1 2 2 -3 -1 0 1 1 4 2 1 0 2 3 1factor = -0.1667Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 1.0000 1.0000 4.0000 2.0000 1.0000 0 2.0000 3.0000 1.0000factor = 0Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 1.0000 1.0000 4.0000 2.0000 1.0000 0 2.0000 3.0000 1.0000factor = 0.1667Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 1.0000 1.0000 4.0000 2.0000 0 -0.3333 1.6667 2.3333 0.8333

Aug = [A b]

Interchange rows 1 and 4

Find the first pivot element and its index

Eliminate first columnNo need to interchange

Page 46: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

big = 2.3333i = 1ipr = 2factor = 0.4286Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 0 0 5.0000 2.3571 0 -0.3333 1.6667 2.3333 0.8333factor = -0.1429Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 0 0 5.0000 2.3571 0 0 2.0000 2.0000 0.7143big = 2i = 2ipr = 4Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 0 2.0000 2.0000 0.7143 0 0 0 5.0000 2.3571factor = 0Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 0 2.0000 2.0000 0.7143 0 0 0 5.0000 2.3571

x = 0 0 0 0.4714x = 0 0 -0.1143 0.4714x = 0 0.2286 -0.1143 0.4714x =

-0.1857 0.2286 -0.1143 0.4714

Back substitutionSecond pivot element and index

Third pivot element and index

Eliminate second column

Eliminate third column

Interchange rows 3 and 4

No need to interchange

Save factors fij for

LU decomposition

Page 47: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Banded Matrix

47

HBW: Half band width

Page 48: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Banded Matrix

48

xw

vut

srq

pzk

nml

kji

hgf

edc

ba

0000000

000000

000000

000000

000000

000000

000000

000000

0000000ai,j= 0if j > i + HBor j < i - HB

HB: Half bandwidthB: Bandwidth

B = 2*HB + 1

In this exampleHB = 1 & B = 3

Page 49: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Tridiagonal Matrix ( 三對角線系統 )

Only three nonzero elements in each equation (3n instead of n2 elements) Subdiagonal, diagonal, superdiagonal

Solve by Gauss elimination

49

Page 50: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Tridiagonal Matrix :例

Special case of banded matrix with bandwidth = 3 Save storage, 3 n instead of n n

50

n

1n

i

3

2

1

n

1n

i

3

2

1

nn

1n1n1n

iii

333

222

11

r

r

r

r

r

r

x

x

x

x

x

x

fe

gfe

gfe

gfe

gfe

gf

Page 51: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Tridiagonal Matrix

Forward elimination

Back substitution

51

n32k

rf

err

gf

eff

1k1k

kkk

1k1k

kkk

,,,

1232n1nk f

xgrx

f

rx

k

1kkkk

n

nn

,,,,,

Use factor = ek / fk1

to eliminate subdiagonal element

Apply the same matrix operations to right hand side

Page 52: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

Tridiagonal Matrix: Hand Calculations

52

53

2

5

3

x

x

x

x

2515000

50210

0152

0021

4

3

2

1

...

.

411

5053r

f

err

1501

50251g

f

eff

111

12r

f

err

111

12g

f

eff

131

25r

f

err

121

25g

f

eff

33

444

33

444

11

223

22

333

11

222

11

222

)(.

.

).(.

.

)(

)(

)(

)(

11

223

f

xgrx

21

311

f

xgrx

31

4501

f

xgrx

41

4

f

rx

1

2111

2

3222

3

4333

4

44

))((

))((

))(.(

(a) Forward elimination (b) Back substitution

Page 53: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

53

function x = Tridiag(e,f,g,r)% x = Tridiag(e,f,g,r):% Tridiagonal system solver.% input:% e = subdiagonal vector% f = diagonal vector% g = superdiagonal vector% r = right hand side vector% output:% x = solution vectorn=length(f);% forward eliminationfor k = 2:n factor = e(k)/f(k-1); f(k) = f(k) - factor*g(k-1); r(k) = r(k) - factor*r(k-1);end% back substitutionx(n) = r(n)/f(n);for k =n-1:-1:1 x(k) = (r(k)-g(k)*x(k+1))/f(k);end

Page 54: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

» [e,f,g,r] = example

e = 0 -2.0000 4.0000 -0.5000 1.5000 -3.0000f = 1.0000 6.0000 9.0000 3.2500 1.7500 13.0000g = -2.0000 4.0000 -0.5000 1.5000 -3.0000 0r = -3.0000 22.0000 35.5000 -7.7500 4.0000 -33.0000

» x = Tridiag (e, f, g, r)x = 1 2 3 -1 -2 -3

function [e,f,g,r] = example

e=[ 0 -2 4 -0.5 1.5 -3];

f=[ 1 6 9 3.25 1.75 13];

g=[-2 4 -0.5 1.5 -3 0];

r=[-3 22 35.5 -7.75 4 -33];

33

4

75.7

5.35

22

3

x

x

x

x

x

x

133

375.15.1

5.125.35.0

5.094

462

21

6

5

4

3

2

1

Note: e(1) = 0 and g(n) = 0

Tridiagonal Matrix: 運算例

Page 55: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

55

補充: Big-O

Page 56: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

56

Concept of Order of Growth

We say fA(n)=30n+8 is (at most) order n, or O(n)

It is, at most, roughly proportional to n fB(n)=n2+1 is order n2, or O(n2)

It is (at most) roughly proportional to n2

Any function whose exact (tightest) order is O(n2) is faster-growing than any O(n) function Later we will introduce Θ for expressing exact order

Page 57: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

57

Definition: O(g) (Asymptotic Upper Bound)

“f is at most order g”, or “f is O(g)”, or “f = O(g)” all just mean that fO(g)

Let g be any function RR. Define “at most order g”, written O(g), to be:

{f:RR | c,k: x>k: f(x) cg(x)} “Beyond some point k, function f is at most a constant c

times g (i.e., proportional to g)”

Page 58: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

58

About the Definition O(g)

Page 59: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

59

Big-O 範例 (1/3)

Page 60: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

60

Big-O 範例 (2/3)

Page 61: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

61

Big-O 範例 (3/3)

Show that 30n+8 is O(n) To show c,k: n > k: 30n+8 cn

• Let c = 31, k = 8. Assume n > k = 8. Thencn = 31n = 30n + n > 30n+8, so 30n+8 < cn

Show that n2+1 is O(n2) To show c,k: n > k: n2+1 cn2

• Let c = 2, k = 1. Assume n > 1. Then cn2 = 2n2 = n2+n2 > n2+1, or n2+1< cn2

Page 62: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

62

Note 30n+8 isn’t less than n anywhere (n>0) It isn’t even less than 31n everywhere But it is less than 31n everywhere to the right of n=8

n>k=8

Big-O Example

Increasing n

Val

ue o

f fu

ncti

on

n

30n+8cn =31n

30n+8 O(n)

Page 63: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

63

Page 64: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

64

應知的定理

Page 65: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

65

Page 66: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

66

定理應用範例

Page 67: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

67

Summary

For any g:RR, “at most order g”,O(g) {f:RR | c,k x > k |f(x)| |cg(x)| } Often, we deal only with positive functions and can

ignore absolute value symbols

“fO(g)” often written “f is O(g)” or “f = O(g)” The latter form is an instance of a more general

convention...

Page 68: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

68

(g) {f:RR | c,k x > k, |f(x)| > |cg(x)|}

Remarks O(.) gives worst-case guarantees (good news), while

gives a lower bound (bad news)

的定義 (Asymptotic Lower Bound)

Page 69: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

69

Definition: (g), Exactly Order g (Asymptotic Tight Bound)

If fO(g) and gO(f ), then we say “g and f are of the same order” or “f is (exactly) order g” and write f(g)

Another equivalent definition:(g) {f:RR | c1c2k > 0 x > k: |c1g(x)| |f(x)| |c2g(x)| } “Everywhere beyond some point k, f(x) lies in between

two multiples of g(x)”

Page 70: 1 Gauss Elimination Small Matrices  For small numbers of equations, solvable by hand  Graphical  Cramer's rule  Elimination 2

70