6.2 Pivoting Strategies 1/17 Chapter 6 Direct Methods for Solving Linear Systems -- Pivoting Strategies Example: Solve the linear system using 4-digit

6.2 Pivoting Strategies

1/17

Chapter 6 Direct Methods for Solving Linear Systems -- Pivoting Strategies

Example: Solve the linear system

using 4-digit arithmetic with rounding.

78.46130.6291.5

17.5914.59003000.0

21

21

xx

xx

Solution: The exact solutions are .000.1 and ,00.10 21 xx

1764003000.0/291.5/ 112121 aam

10430014.59130.6 2122 ma

10440017.5978.46 212 mb

Apply Gaussian elimination:

1044001043000

17.5914.59003000.0

001.12 x 00.10003000.0

001.114.5917.591

x

Trouble maker

Small pivot element may cause trouble.


Partial Pivoting (or maximal column pivoting)

-- Determine the smallest p k such that ||max|| )()( kik

nik

kpk aa

and interchange the pth and the kth rows.

78.46130.6291.5

17.5914.59003000.0

17.5914.59003000.0

78.46130.6291.5

14.5914.590

78.46130.6291.500.10;000.1 12 xx

Example: Solve the linear system

using 4-digit arithmetic with rounding.

78.46130.6291.5

59170059140000.30

21

21

xx

xx

Small relative to the entries in its row.

2/17


Scaled Partial Pivoting (or scaled-column pivoting)

-- Place the element in the pivot position that is largest relative to the entries in its row.

||max1

ijnj

i as

Step 1: Define a scale factor si for each row as

Step 2: Determine the smallest p k such that i

kik

nikp

kpk

s

a

s

a ||max

|| )()(

and interchange the pth and the kth rows.

Note: Note:

The scaled factors The scaled factors ssii must be computed must be computed only onceonly once, otherwise , otherwise

this method would be too slow.this method would be too slow.

3/17


Complete Pivoting (or maximal pivoting)

-- Search all the entries aij for i, j = k, …, n, to find the entry with the largest magnitude. Both row and column interchanges are performed to bring this entry to the pivot position.

Result of solving 3 by 3 linear systems with direct Gaussian elimination

Result of solving 3 by 3 linear systems with complete pivoting

4/17


Amount of Computation

Partial Pivoting: Requires about O(n2) additional comparisons.

Scaled Partial Pivoting: Requires about O(n2) additional comparisons and O(n2) divisions.

Complete Pivoting: Requires about O(n3/3) additional comparisons.

Note: Note:

If the new scaled factors were determined each time a row If the new scaled factors were determined each time a row

interchange decision was to be made, then the scaled partial interchange decision was to be made, then the scaled partial

pivoting would add pivoting would add O(O(nn33/3) comparisons/3) comparisons in addition to the in addition to the O(O(nn22) )

divisions.divisions.

5/17

Chapter 6 Direct Methods for Solving Linear Systems -- Matrix Factorization

6.5 Matrix Factorization

Matrix Form of Gaussian Elimination

Step 1: )0(/ 111111 aaam ii

Let L1 =

1...

1

1

1

21

nm

m

, then ][ )1()1(1 bAL

)1(1

)1(1

)1(11 ... baa n

)2(A )2(b

Step n 1:

)(

)2(2

)1(1

)(

)2(2

)2(22

)1(1

)1(12

)1(11

121

..

.......

...

...

...

nn

nnn

n

n

nn

b

b

b

a

aa

aaa

bALLL

where

Lk =

1...

1

1

,

,1

kn

kk

m

m

6/17


1kL

1...

1

1

,

,1

kn

kk

m

m

1

11

21

1 ... nLLL

1

1

1

jim ,L

unitary lower-triangular matrix

Let U =

)(

)2(2

)2(22

)1(1

)1(12

)1(11

..

....

...

...

nnn

n

n

a

aa

aaa

LUA

LU factorization of A

Hey hasn’t GE given me enough headache? Why do I have to know its matri

x form??!

When you have to solve the system for

different with a fixed

A.bCould you be more specific, please?

Factorize A first, then for every you only have to

solve two simple triangular systems and

.

b

byL

yxU

7/17


Theorem: If Gaussian elimination can be performed on the linear system Ax = b without row interchanges, then the matrix A can be factored into the product of a lower-triangular matrix L and an upper-triangular matrix U.

If L has to be unitary, then the factorization is unique.

Proof (for uniqueness): If the factorization is NOT unique, then there exist L1, U1, L2 and U2 such that A = L1U1 = L2U2 .

121 UU 2

11

1222

11 LLUULL

Upper-triangularLower-triangular

with diagonal entries 1

I

Note: The factorization with Note: The factorization with UU being unitary being unitary is called the is called the Crout’Crout’s factorizations factorization. Crout’s factorization can be obtained by the . Crout’s factorization can be obtained by the LULU factorization of factorization of AAtt. That is, find . That is, find AAtt = = LULU, then , then AA = = UUtt L Ltt is the Cro is the Crout’s factorization of ut’s factorization of AA..

8/17


Doolittle Factorization – a compact form of LU factorization

Repeated computations.What a waste!

nn

n

nnnn

n

u

uu

l

l

aa

aa

..

.

..

....

...

1...

......

1

1

......

..

....

..

.... 111

1

21

1

111

),min(

1

ji

kjkki ul jia

9/17


),min(

1

ji

kjkkiji ula

Fix i :For j = i, i+1, …, n we have ijkj

i

kikij uula

1

1

lii = 1

kj

i

kikijij ulau

1

1 a

Interchange i and j . For j = i, i+1, …, n we have iijiki

i

kjkji ulula

1

1

ii

i

kkijkjiji uulal /)(

1

1

b

Algorithm: Doolittle FactorizationStep 1: u1j = a1j; lj1 = aj1 / u11; ( j = 1, …, n )

Step 2: compute and for i = 2, …, n1;

Step 3:

a b

1

1

n

kknnknnnn ulau

HW: p.397 #7

10/17

Chapter 6 Direct Methods for Solving Linear Systems -- Special Types of Matrices

6.6 Special Types of Matrices

Strictly Diagonally Dominant Matrix

for each i = 1, …, n.

n

ijj

ijii aa,1

||||

Theorem: A strictly diagonally dominant matrix A is nonsingular. Moreover, Gaussian elimination can be performed without row or column interchanges, and the computations will be stable with respect to the growth of roundoff errors.

Proof: A is nonsingular – proof by contradiction.

Gaussian elimination can be performed without row or column interchanges – proof by induction: each of the matrices A(2), A(3), …, A(n) generated by the Gaussian elimination is strictly diagonally dominant.

Omitted.

11/17


Choleski’s Method for Positive Definite Matrix

Review: A is positive definite

Definition: A matrix A is positive definite if it is symmetric and if

x tA x > 0 for every n-dimensional vector x 0.

A1 is positive definite as well, and aii > 0.

max | aij | max | akk |; ( aij )2 < aii ajj for each i j.

Each of A’s leading principal submatrices Ak has a positive determinant.

HW: Read the proofs onp. 401-402

12/17


Consider the LU factorization of a positive definite A:

U =uij

=

u11 uij / uii

11

1

u22

unn

UD~

A is symmetric tUL~ tLDLA

Let D1/2 =

11u

22u

nnu

2/1~LDL

is still a lower-triangular matrix

Why is uii > 0?Since det(Ak) > 0

tLLA~~

A is positive definiteA can be factored in the form LDLt, where L is a unitary lower-triangular matrix and D is a diagonal matrix with positive diagonal entries.

A can be factored in the form LLt, where L is lower-triangular with nonzero diagonal entries.

13/17


Algorithm: Choleski’s MethodTo factor the symmetric positive definite nn matrix A into LLt, where L is lower-triangular.

Input: the dimension n; entries aij for 1 i, j n of A.

Output: the entries lij for 1 j i and 1 i n of L.

Step 1 Set ;

Step 2 For j = 2, …, n, set ;

Step 3 For i = 2, …, n1, do steps 4 and 5

Step 4 Set ;

Step 5 For j = i+1, …, n, set ;

Step 6 Set ;

Step 7 Output ( lij for j = 1, …, i and i = 1, …, n );

STOP.

1111 al

1111 / lal jj

1

1

2i

k ikiiii lal

ii

i

k ikjkjiji lllal

1

1

1

1

2n

k nknnnn lal

LDLt is faster, but must be modified to

solve Ax = b.

14/17


Crout Reduction for Tridiagonal Linear System

nnnn

nnn

f

f

f

x

x

x

ba

cba

cba

cb

2

1

2

1

111

222

11

Step 1: Find the Crout factorization of A

1

1

1

1

2

1

n

nn

A

Step 2: Solve fyL ,

1

11

fy ),...,2(

)( 1 niyrf

yi

iiii

Step 3: Solve yxU )1,...,1(, 1 nixyxyx iiiinn

The process cannot continue if i = 0. Hence not all the

tridiagonal linear system can be solvedby this method.

15/17


Theorem: If A is tridiagonal and diagonally dominant. Moreover, if 0,0,0||||,0|||| 11 iinn caabcb

Then A is nonsingular, and the linear system can be solved.

Note: Note:

If If AA is strictly diagonally dominant, then it is not necessary to is strictly diagonally dominant, then it is not necessary to

have all the entries have all the entries aaii, , bbii, and , and ccii being nonzero. being nonzero.

The method is stable in a sense that all the values obtained duThe method is stable in a sense that all the values obtained du

ring the process will be bounded by the values of the original entring the process will be bounded by the values of the original ent

ries.ries.

The amount of computation is O(The amount of computation is O(nn).).

HW: p.412 #17

16/17


Lab 03. There is No Free Lunch

Time Limit: 1 second; Points: 4

One day, CYJJ found an interesting piece of commercial from newspaper: the Cyber-restaurant was offering a kind of "Lunch Special" which was said that one could "buy one get two for free". That is, if you buy one of the dishes on their menu, denoted by di

with price pi , you may get the two neighboring dishes di1 and di+1 for

free! If you pick up d1, then you may get d2 and the last one dn for

free, and if you choose the last one dn, you may get dn1 and d1 for

free.

However, after investigation CYJJ realized that there was no free lunch at all. The price pi of the i-th dish was actually calculated

by adding up twice the cost ci of the dish and half of the costs of the

two "free" dishes. Now given all the prices on the menu, you are asked to help CYJJ find the cost of each of the dishes.

17/17

Documents

6.2 Pivoting Strategies 1/17 Chapter 6 Direct Methods for Solving Linear Systems -- Pivoting Strategies Example: Solve the linear system using 4-digit